[PDB Tech] Peeringdb sync problem

[Jan.Boogman at swisscom.com]

Hi

every now and then I stumble over syncing problems like this, which seem to be caused by bad data in the db:

pdb at pdb2:~$ peeringdb sync
Syncing to https://www.peeringdb.com/api
Updating resources: org fac net ix ixfac ixlan ixpfx netfac netixlan poc
Fetching & updating all: org
Updates to be processed: 4
Fetching & updating all: fac
Updates to be processed: 1
Fetching & updating all: net
Updates to be processed: 18903
Traceback (most recent call last):
  File "/usr/local/bin/peeringdb", line 11, in <module>
    sys.exit(main())
  File "/home/pdb/.local/lib/python2.7/site-packages/peeringdb/cli.py", line 62, in main
    return handler(config=cfg, **vars(options))
  File "/home/pdb/.local/lib/python2.7/site-packages/peeringdb/commands.py", line 20, in _wrapped
    r = func(*a, **k)
  File "/home/pdb/.local/lib/python2.7/site-packages/peeringdb/commands.py", line 219, in handle
    client.update_all(rs)
  File "/home/pdb/.local/lib/python2.7/site-packages/peeringdb/_update.py", line 66, in update_all
    self._atomic_update(lambda: ctx.sync_resource(r, since=since))
  File "/home/pdb/.local/lib/python2.7/site-packages/peeringdb/_update.py", line 78, in _atomic_update
    sync_func()
  File "/home/pdb/.local/lib/python2.7/site-packages/peeringdb/_update.py", line 66, in <lambda>
    self._atomic_update(lambda: ctx.sync_resource(r, since=since))
  File "/home/pdb/.local/lib/python2.7/site-packages/peeringdb/_tasks_sequential.py", line 77, in _wrapped
    return _consume_task(gen)
  File "/home/pdb/.local/lib/python2.7/site-packages/peeringdb/_tasks_sequential.py", line 55, in _consume_task
    item = gen.send(r)
  File "/home/pdb/.local/lib/python2.7/site-packages/peeringdb/_tasks_sequential.py", line 38, in gather
    for r in job:
  File "/home/pdb/.local/lib/python2.7/site-packages/peeringdb/_update.py", line 287, in sync_row
    B.clean(obj)
  File "/home/pdb/.local/lib/python2.7/site-packages/django_peeringdb/client_adaptor/backend.py", line 145, in clean
    obj.full_clean()
  File "/home/pdb/.local/lib/python2.7/site-packages/django/db/models/base.py", line 1250, in full_clean
    raise ValidationError(errors)
django.core.exceptions.ValidationError: {'route_server': [u'Enter a valid URL.']}
pdb at pdb2:~$

Normally it fixes itself after some time (the bad data has been fixed in the db?)

Is there any possibility skip this bad data?

Best regards
Jan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.peeringdb.com/pipermail/pdb-tech/attachments/20200522/29dacc78/attachment.htm>