[PDB Tech] IX-F JSON to PeeringDB importer (temporarily) disabled

[LOOS Eric (BCS/PSD)]

Dears,

I also believe this ties in with the discussion on data ownership. Probably that discussion also will have to accommodate when information can be considered "new". In other words, absence of data might or might not mean something, whereas a reissued IP address has a much higher likelihood of meaning deletion.
The discussion below goes a bit too much in detail until we address the broader issue IMHO.

Kind regards,

Eric


> -----Original Message-----
> From: Pdb-tech <pdb-tech-bounces at lists.peeringdb.com> On Behalf Of
> Asbjørn Sloth Tønnesen
> Sent: 07 September 2019 23:59
> To: pdb-tech at lists.peeringdb.com
> Subject: Re: [PDB Tech] IX-F JSON to PeeringDB importer (temporarily)
> disabled
>
> Hi,
>
> On Fri, 30 Aug 2019 at 17:30, Job Snijders <job at instituut.net> wrote:
>  > A significant challenge here is that sometimes records in PeeringDB  > are
> out of date (which hinders new IXP participants), and sometimes  > IX-F data
> doesn't (yet) contain information or this information is  > out-of-date (either
> case hinders a different group). I think the path  > forward is to express the
> relations and automated actions between the  > information sources as a
> state machine which (unlike the current  > version) does account for the full
> lifecycle from provisioning to  > decommissioning of circuits to IXPs.
>
> Since the thing that's unwanted is stale incorrect netixlan entries, it should be
> sufficient to just limit auto-deletion to netixlan's which hasn't been
> created/updated in X time. X time should be long enough to not cause
> problems with provisioning, a good value might be
> 3 months.
>
> A problem there might be that "updated", could have changed based on IX-F
> or a PDB admin change. Would it make sense to have a separate "updated-
> by-owner" timestamp, to indicate if and when the object owner last touched
> the object? Could also be "updated-by-net" and "updated-by-ix" columns.
>
> On Fri, 30 Aug 2019 at 20:12, Lukas Tribus <tribuslukas58 at gmail.com> wrote:
> > A state machine handling the full life-cycle seems overly complex,
> > with lots of moving parts. I fear that would also result in a number
> > of disputes that is similarly unsatisfactory. I mean, you don't really
> > know how the IX constructs this data - a ASN may be removed from that
> > list for a short period of time to due to a temporary payment issue.
> > In fact it looks like the disputes are mainly caused by
> > bad/imprecise/outdated information from IXes, so building more
> > complexity around it seems like a bad idea to me.
>
> The temporary missing from the IX-F feed cases, could be handled by a
> "marked_for_deletion" column with current timestamp.
>
> In the new/updated netixlan case covered above the flag, will first be put on
> the object after the X time since created/updated has expired and it is still
> not in the IX-F feed.
>
> A daily job deletes objects which was "marked_for_deletion" more than eg.
> 2 weeks ago. If "updated-by-owner" is newer than "marked_for_deletion"
> the flag is removed.
>
> If it reappears in IX-F data the flag is also removed.
>
> As an alternative for the "updated-by-owner" column, the website code
> could automatically remove the "marked_for_deletion" flag, if the object is
> updated by the network.
>
> This boils down to long leash if the network is active, shorter if not.
>
> The "updated-by-owner" could also just be on an org/net level, rather than
> individual netixlan.
>
> To sum-up reformulated as simple rules:
>
> An netixlan SHALL NOT be automatically deleted if:
> - the owner has touched it less than X time ago
> - if it has been missing from IX-F feed for less than Y time where Y time < X
> time
>
> --
> Best regards
> Asbjørn Sloth Tønnesen
> _______________________________________________
> Pdb-tech mailing list
> Pdb-tech at lists.peeringdb.com
> https://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech
This e-mail cannot be used for other purposes than BICS business use. See more on https://bics.com/mail-disclaimer