[PDB Tech] IX-F JSON to PeeringDB importer (temporarily) disabled

[Asbjørn Sloth Tønnesen]

Hi,

On Fri, 30 Aug 2019 at 17:30, Job Snijders <job at instituut.net> wrote:
 > A significant challenge here is that sometimes records in PeeringDB
 > are out of date (which hinders new IXP participants), and sometimes
 > IX-F data doesn't (yet) contain information or this information is
 > out-of-date (either case hinders a different group). I think the path
 > forward is to express the relations and automated actions between the
 > information sources as a state machine which (unlike the current
 > version) does account for the full lifecycle from provisioning to
 > decommissioning of circuits to IXPs.

Since the thing that's unwanted is stale incorrect netixlan entries,
it should be sufficient to just limit auto-deletion to netixlan's which
hasn't been created/updated in X time. X time should be long enough
to not cause problems with provisioning, a good value might be
3 months.

A problem there might be that "updated", could have changed based
on IX-F or a PDB admin change. Would it make sense to have a separate
"updated-by-owner" timestamp, to indicate if and when the object
owner last touched the object? Could also be "updated-by-net" and
"updated-by-ix" columns.

On Fri, 30 Aug 2019 at 20:12, Lukas Tribus <tribuslukas58 at gmail.com> wrote:
> A state machine handling the full life-cycle seems overly complex,
> with lots of moving parts. I fear that would also result in a number
> of disputes that is similarly unsatisfactory. I mean, you don't really
> know how the IX constructs this data - a ASN may be removed from that
> list for a short period of time to due to a temporary payment issue.
> In fact it looks like the disputes are mainly caused by
> bad/imprecise/outdated information from IXes, so building more
> complexity around it seems like a bad idea to me.

The temporary missing from the IX-F feed cases, could be handled by a
"marked_for_deletion" column with current timestamp.

In the new/updated netixlan case covered above the flag, will first be
put on the object after the X time since created/updated has expired
and it is still not in the IX-F feed.

A daily job deletes objects which was "marked_for_deletion" more
than eg. 2 weeks ago. If "updated-by-owner" is newer than
"marked_for_deletion" the flag is removed.

If it reappears in IX-F data the flag is also removed.

As an alternative for the "updated-by-owner" column, the website
code could automatically remove the "marked_for_deletion" flag,
if the object is updated by the network.

This boils down to long leash if the network is active, shorter if not.

The "updated-by-owner" could also just be on an org/net level,
rather than individual netixlan.

To sum-up reformulated as simple rules:

An netixlan SHALL NOT be automatically deleted if:
- the owner has touched it less than X time ago
- if it has been missing from IX-F feed for less than Y time
where Y time < X time

-- 
Best regards
Asbjørn Sloth Tønnesen