[PDB Announce] IX-F JSON to PeeringDB importer (temporarily) disabled

[Job Snijders]

Dear all,

Summary:

It has come to our attention that specific aspects of an integration
between PeeringDB and IX-F (in its current form) is causing
considerable issues for some PeeringDB users. We have temporarily
disabled importing data from IX-F into PeeringDB to alleviate some
headache.

Background:

Every day PeeringDB ingests JSON formatted information from IX-F
("IX-F data"). The intent of this ingestion process is to help align
information submitted to PeeringDB by network operators and
information submitted to IX-F by IXP operators.

The expectation was that this would help streamline scenarios such as
where an ASN disconnects from an IXP but didn't update their PeeringDB
record reflecting disconnection, and if the IXP subsequently would
re-assign the IP address to a newly participating ASN, often times the
new ASN wasn't able to add the new IXP connection to their PeeringDB
record without intervention from the PeeringDB Admin Committee.

However there also are scenarios where an ASN is in the process of
connecting a new IXP circuit, and wants to update their PeeringDB
record and add this new connection ahead of the completion of the full
provisioning process to streamline their peering discussions. In such
cases the lack of information about this new circuit in the IX-F data
could cause a deletion of this user submitted information.

A significant challenge here is that sometimes records in PeeringDB
are out of date (which hinders new IXP participants), and sometimes
IX-F data doesn't (yet) contain information or this information is
out-of-date (either case hinders a different group). I think the path
forward is to express the relations and automated actions between the
information sources as a state machine which (unlike the current
version) does account for the full lifecycle from provisioning to
decommissioning of circuits to IXPs. I hope you as community will
continue chime in on what we should do next.

On github there are various issues open related to this problem space:

https://github.com/peeringdb/peeringdb/issues/505
https://github.com/peeringdb/peeringdb/issues/513
https://github.com/peeringdb/peeringdb/issues/518
https://github.com/peeringdb/peeringdb/issues/540

We want PeeringDB to be a useful, reliable, and consistent tool for
everyone. However the current implementation of the process to augment
PeeringDB's data with IX-F data appears to be unable to account for
all possible scenarios in a way that is without exception helpful to
all PeeringDB's users; hence the decision to disable the importer for
now.

Kind regards,

Job