[PDB Data Ownership-TF] IXP assignment IP address (netixlan) ownership
Chris Caputo
ccaputo at alt.net
Fri Jan 24 10:47:48 PST 2020
> Are the users with issues synching a copy of peeringDB locally for use
> in their provisioning process?
I don't think so, but I could be wrong. Fredrik may chime in.
My sense of of what happened is as follows:
- 1. IXP gave Network an IP assignment prior to the assignment being in
the IXP's IX-F JSON export.
- 2. Network added IP assignment to their PeeringDB page.
- 3. IX-F importer run at 0000 UTC removed the IP assignment added by
the Network since the IXPs IX-F JSON export did not have a
corresponding record.
- 4. Network noticed the IP assignment had gone missing and re-added the
IP assignment, since surely IXP will update their IX-F JSON export
soon.
- 5. Next day at 0000 UTC, IX-F importer ran again, and data was removed
again.
- 6. Process repeats and frustration ensues, with the Network having no
idea when it won't be a waste of effort to re-add the IP
assignment, unless they scrutinize the IXPs IX-F JSON export data.
Not scalable and a pain in the neck.
What #627 does is change the above to be:
- 1. IXP gives Network an IP assignment prior to the assignment being in
the IXPs IX-F JSON export.
- 2. Network adds IP assignment to their PeeringDB page.
- 3. PeeringDB considers the Network's IP addition in the context of
whether the IXP regularly provides IX-F JSON exports. This is
where the big if-then-else statement at
https://github.com/peeringdb/peeringdb/issues/627#issuecomment-572844724
is evaluated, the gist being:
- 3a. If IXP does export JSON and the data is not in conflict, the
data is presented as normal.
- 3b. If IXP does export JSON and the data is in conflict, the data
is flagged as being in conflict. No presentation via REST API or
/ix/ page, while data is presented on Network's /net/ page with a
conflict indicator.
- 3c. If IXP does not export JSON, simply present the Network
provided data.
- 4. As time passes, if/when the IX-F JSON export data has a matching IP
assignment, any conflict is automatically resolved and the data is
presented as normal.
Thus #627 avoids the frustration of a network needing to add and re-add
data until the IXP providing an IX-F JSON export contains matching data.
The network still owns their data - it hasn't been removed from their web
page - and the IX-F JSON exporting IXP still owns their subnet,
controlling whether provisioning kicks off. (This also prevents
unnecessary broadcasts/multicasts on peering fabrics, due to premature
provisioning.)
> I don't see how it helps operators who rely on API calls ( of
> potentially conflicted data ) for automation in an attempt to jump start
> slow internal provisioning intervals, but I can live with it as the
> taskforce is here to define data ownership and in the process prevent
> data quality features like IX-F import from breaking some folks.
On the upside, in cases where an IX-F JSON export exists, bad data is
prevented from entering provisioning systems.
Thanks,
Chris
On Fri, 24 Jan 2020, William Marantz wrote:
> Hi Chris,
> Thanks for all the input! I now think I am being very thick here.
>
> Are the users with issues synching a copy of peeringDB locally for use
> in their provisioning process? If so, I understand now how this proposal
> helps them ( as the data they rely on for their own IP interface configs
> still exists in the DB ) and I totally missed a very important detail
> due to my API/web focus.
>
> I don't see how it helps operators who rely on API calls ( of
> potentially conflicted data ) for automation in an attempt to jump start
> slow internal provisioning intervals, but I can live with it as the
> taskforce is here to define data ownership and in the process prevent
> data quality features like IX-F import from breaking some folks.
>
> Thanks
>
> -Bill
>
> On Fri, Jan 24, 2020 at 12:25 PM Chris Caputo <ccaputo at alt.net> wrote:
> Hi Bill,
>
> On Fri, 24 Jan 2020, William Marantz wrote:
> > I was engaged in a side conversation with Chris on this proposal to
> > clarify some things and would like to bring the discussion to the list.
> > I'm going to wear two different hats when looking at this proposal.
> >
> > 1) From a data ownership perspective I like this proposal as it won't
> > lead to full deletion of end user network specified IP data but will
> > limit its visibility to others and indicate the IX-F conflict to the
> > network.
> >
> > 2) From a poor coder and automated peering operator perspective I have
> > some concerns with it depending on how the folks with reported issues
> > use the API.
> >
> > a) does anyone know the access method and tables/objects the users who
> > had data deletion issues used?
>
> I'm not sure, but it is important the solution be useful regardless of how
> data is submitted. fkorsback raised the flag about the original issue,
> resulting in the IX-F JSON importer being disabled, and he has expressed
> support for #627.
>
> > If they are getting netixlan via the API the conflicted IP data won't be
> > there in this proposal. If they call a net object will the data be
> > viewable in the netixlan_set data element in your proposal Chris? I'd
> > assume yes if it will be available via the net web page. If the user is
> > using netixlan calls as part of their provisioning process, I don't see
> > how this proposal solves the problem. I'd suggest to clearly document
> > the potentially conflicted view ( net ) and conflict free view (
> > netixlan ). Users can then code to whatever view meets the needs of
> > their network.
>
> No, it won't be visible in the /net/ result or other results as long as it
> is conflicted. The idea here is to prevent PeeringDB from presenting (via
> web or API) any data that could be interpreted as valid when it is not
> valid. Conflicted data is not valid. While that conflict can be denoted
> in the web pages, so a network has a clue that an issue has arisen, for
> API queries conflicted data would be withheld to prevent accidental
> automation using conflicted data.
>
> For API developers the lack of consistency between a GET after a PUT/POST
> is the way a conflict is indicated.
>
> I've added the above to the issue on GitHub.
>
> > b) Some networks are very slow at peer provisioning and IX port
> > provisioning, so peers may have a strong desire to get provisioning
> > started as soon as the IXP assigns the IPs but that may not yet be
> > reflected in IX-F. This means a provisioning system would rely on the
> > conflicting data if that system is built to only obtain IPs from
> > peeringDB. As long as the caveats are documented, I'd like to support
> > such functionality as I feel it would aid many operators.
>
> The idea with the proposal is that if an IXP is regularly (ex. within last
> 30 days) providing IX-F JSON data, the IXP is asserting authority over
> their address space, which is their right to do. Thus conflicted data
> will not be allowed to be presented. I've recommended in the proposal
> that PeeringDB pull the IX-F JSON updates hourly to minimize the conflict
> period for soon-to-be-valid data. Further, it is expected that IX-F JSON
> data be kept current by the IXP such that any assignments go into their
> IX-F JSON export when the IXP port for the network is in production (or
> earlier, depending on their local policy).
>
> > The provisioning code I wrote makes API calls to get netixlan but I'm
> > happy to change it to make net calls if this proposal goes through.
>
> I don't think you will need to change any code for this proposal, except
> possibly code which performs PUT/POST updates. But even if PUT/POST code
> is unmodified and continues to periodically PUT/POST conflicted data,
> since it is not showing up in a subsequent GET, that is okay.
>
> Thanks,
> Chris
>
> > Best Regards
> >
> > -Bill
> >
> >
> > On Fri, Jan 24, 2020 at 9:25 AM Filiz Yilmaz <filiz at peeringdb.com> wrote:
> >
> > Dear all,
> >
> > Bringing this to your attention again:
> >
> > https://github.com/peeringdb/peeringdb/issues/627
> >
> > There seems to be support from Product Com as well as some of the TF members for it.
> >
> > Thoughts?
> >
> > Filiz
> >
> >
> >
> > ---------- Forwarded message ----------
> > Date: Thu, 9 Jan 2020 19:18:13 +0000 (UTC)
> > From: Chris Caputo <ccaputo at alt.net>
> > Reply-To: Chris Caputo <ccaputo-dated-1588965493.0ca23d at alt.net>
> > To: DTF PeeringDB <dataownership-tf at lists.peeringdb.com>
> > Subject: Re: [PDB Data Ownership-TF] IXP assignment IP address (netixlan)
> > ownership (was: conditions for being listed in a facility)
> >
> > On Thu, 9 Jan 2020, Arnold Nipper wrote:
> > On 09.01.2020 19:02, Chris Caputo wrote:
> > I believe the importer needs to perform an additional task:
> >
> > - populate/update a new database table, potentially known as
> > netixlan_ixp, which represents the IXP viewpoint.
> >
> > and once that is implemented, it can cease removing conflicted assignments
> > unless a network has opted for IX-F JSON automatic updates.
> >
> >
> > There is no reason to do so. See #585 [0] which is ready for implementation.
> >
> > [...]
> > [0] https://github.com/peeringdb/peeringdb/issues/585
> >
> >
> > Arnold, #585 has the word "Interim" in its very title. I am proposing a
> > longer term solution that does not put the repeated burden of resolution
> > of ownership issues on the Admin Committee. I shared my proposal with the
> > PC and got positive feedback, so I have now created an issue for it:
> >
> > https://github.com/peeringdb/peeringdb/issues/627
> >
> > Chris
> > --
> > DataOwnership-TF mailing list
> > DataOwnership-TF at lists.peeringdb.com
> > https://lists.peeringdb.com/cgi-bin/mailman/listinfo/dataownership-tf
> >
> >
> > --
> > DataOwnership-TF mailing list
> > DataOwnership-TF at lists.peeringdb.com
> > https://lists.peeringdb.com/cgi-bin/mailman/listinfo/dataownership-tf
> >
> >
> >
> > --
> > Bill Marantz
> > Principal Network Engineer
> > Backbone Engineering
> > Mobile: 848-404-4613email: wmarantz at salesforce.com
> >
> >
> >
>
>
>
> --
> Bill Marantz
> Principal Network Engineer
> Backbone Engineering
> Mobile: 848-404-4613email: wmarantz at salesforce.com
>
>
>
More information about the DataOwnership-TF
mailing list