[PDB Tech] IX-F JSON to PeeringDB importer (temporarily) disabled

[Mullally, Ronan]

On 30 Aug 2019, at 20:50, Job Snijders <job at ntt.net> wrote:
> 
> I'll give it another few days before we take any further action to allow
> some time to collect more feedback from the community.

As the intent is to improve the accuracy / reliabilty of (asn,ix) pairs, a new attribute - along the same lines as Arnold has suggested - which indicates whether both the peer and the IX agree on a given pair might be a reasonably simple way forward.

The website could display entries differently when they don’t.  API queries could specify whether they wanted to see results that matched (or not), and - if the attribute was more than just a boolean - where the peer asserts a presence but the IX does not (for example).  The latter would have been useful in the instances where I’ve seen problems arise.


-Ronan

> On Fri, Aug 30, 2019 at 08:12:36PM +0200, Lukas Tribus wrote:
>> Hello Job,
>> 
>> 
>> [ resending email from gmail, because "Client host rejected: 554 SPAM
>> not tolerated here (Mailgun)" ]
>> 
>> On Fri, 30 Aug 2019 at 17:30, Job Snijders <job at instituut.net> wrote:
>>> A significant challenge here is that sometimes records in PeeringDB
>>> are out of date (which hinders new IXP participants), and sometimes
>>> IX-F data doesn't (yet) contain information or this information is
>>> out-of-date (either case hinders a different group). I think the path
>>> forward is to express the relations and automated actions between the
>>> information sources as a state machine which (unlike the current
>>> version) does account for the full lifecycle from provisioning to
>>> decommissioning of circuits to IXPs.
>> 
>> A state machine handling the full life-cycle seems overly complex,
>> with lots of moving parts. I fear that would also result in a number
>> of disputes that is similarly unsatisfactory. I mean, you don't really
>> know how the IX constructs this data - a ASN may be removed from that
>> list for a short period of time to due to a temporary payment issue.
>> In fact it looks like the disputes are mainly caused by
>> bad/imprecise/outdated information from IXes, so building more
>> complexity around it seems like a bad idea to me.
>> 
>> I would opt for simpler, more predictable fixes.
>> 
>> Why not limit the IX-F overwrite to cases where there is an actual ASN
>> mismatch between PDB and IXF? Or in other words: don't delete
>> addresses in PDB when the data is simply missing in the IXF data?
>> Sure, obsolete IP addresses won't get cleaned up immediately. But they
>> would get cleaned up when it matters (when the IP is reassigned),
>> without the false-positives we have today.
>> 
>> We can still notify ASN's about the fact that an IP address is no
>> longer in IXF (once a month or so), to spur the cleanup of obsolete
>> addresses.
>> 
>> I didn't read through all the github issues, so maybe what I am saying
>> doesn't tick all the boxes, but my strong opinion about this is: KISS.
>> 
>> If we really NEED peeringdb to be always uptodate without obsolete IP
>> addresses for IXF enabled IXP's, there is only 1 choice: an IXP
>> providing IXF data needs to be 100% authoritative of the data, in 100%
>> of the cases. An operator should not even be able to add addresses for
>> IXF-enabled IXP's. I'd vote against this approach, but it I think this
>> would still be better than a state machine handling billing disputes.
>> 
>> 
>> Complex processes (and changes thereof) at PDB make it harder for
>> network operators (and IX'es) to comprehend what happens. Even though
>> I'd like operators to read pdb-announce and understand PDB processes,
>> the reality is that PDB needs to be self-explanatory and as
>> predictable as possible for a large number of operators and IX'es,
>> small, big and huge.
>> 
>> I can also see that there is confusion about the "Allow IXP updates"
>> flag. I can understand that, it's easy to interpret this as "nobody is
>> gonna change my data unless flagged", so people are surprised when
>> their data changes. We can make this clearer. "Allow manual IXP
>> updates (automation may remove IP addresses assigned to different
>> ASNs)" or something like this (if true, of course).
>> 
>> 
>> 
>> KISSes,
>> 
>> Lukas
>> _______________________________________________
>> Pdb-tech mailing list
>> Pdb-tech at lists.peeringdb.com
>> https://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech
> _______________________________________________
> Pdb-tech mailing list
> Pdb-tech at lists.peeringdb.com
> https://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech