[PDB Tech] Sync issue - fac_id 3 missing?

[Stefan Pratter]

Hi,

There have been bugs (some now fixed as of django-peeringdb 0.4.0 i 
believe) that would cause missing objects in the local db. Probably 
easiest and reasonable to reset the db and sync fresh, if that's an 
option for you.

Furthermore the client currently has a few short comings when it comes 
to how it handles syncs for deleted objects so it sometimes can't 
correct itself from errors like the one described above.

In this case it's cascading down a line of objects required by other 
objects (eg fac as parent of a netfac) and its trying to obtain it from 
the api, but since it's deleted and the endpoint /api/fac/3 returns a 
404 and bails.

This could be fixed by changing the query the client does to request 
missing objects to a ?since query, as that can retrieve soft-deleted 
objects.

Also, as a side note, this issue is more likely to happen the longer you 
sit on a database without syncing.

Not sure this was much help, but hope this gives some clarity anyhow.

Stefan


On 14/04/2017 18.04, Ruairi Carroll wrote:
> Hey,
>
> I'm trying to run a sync of the DB locally, and after some initial
> struggles, I'm finally able to start to sync the data.
>
> However, my sync bombs out when when trying to sync fac_id 3, as per my
> logs below (Full log is on PasteBin here <https://pastebin.com/r55K4r8Y>).
>
> {'fac': *[u'facility instance with id 3 does not exist.']*} : errors:
> {'fac': [u'facility instance with id 3 does not exist.']}
> fac: Missing Object, dict: {'_validators': [], 'auto_created': False,
> 'serialize': True, '_unique': False, 'unique_for_year': None, 'blank':
> False, 'help_text': u'', 'null': False, 'to_fields': ['id'],
> 'db_tablespace': '', 'db_index': True, 'is_relation': True,
> 'unique_for_month': None, 'unique_for_date': None, 'primary_key': False,
> 'concrete': True, 'swappable': True, 'max_length': None, 'rel':
> <ManyToOneRel: django_peeringdb.networkfacility>, 'from_fields':
> [u'self'], 'verbose_name': u'fac', '_choices': [], 'creation_counter':
> 129, 'validators': [], 'editable': True, 'related_model': <class
> 'django_peeringdb.models.concrete.Facility'>, 'error_messages':
> {u'unique': <django.utils.functional.__proxy__ object at 0x80a4d6a90>,
> u'invalid': <django.utils.functional.__proxy__ object at 0x80a55c550>,
> u'invalid_choice': <django.utils.functional.__proxy__ object at
> 0x809ac0990>, u'blank': <django.utils.functional.__proxy__ object at
> 0x80a4d6a50>, u'null': <django.utils.functional.__proxy__ object at
> 0x80a4d6a10>, u'unique_for_date': <django.utils.functional.__proxy__
> object at 0x80a4d6ad0>}, '_related_fields':
> [(<django.db.models.fields.related.ForeignKey: fac>,
> <django.db.models.fields.AutoField: id>)], '_error_messages': None,
> 'db_constraint': True, '_verbose_name': None, 'name': 'fac',
> 'db_column': None, 'default': 0, 'attname': u'fac_id', 'column':
> u'fac_id', 'model': <class
> 'django_peeringdb.models.concrete.NetworkFacility'>, 'opts': <Options
> for NetworkFacility>}
> *fac.3 not found locally, trying to fetch object... *
> Starting new HTTPS connection (1): www.peeringdb.com
> <http://www.peeringdb.com>
> *"GET /api/fac/3?depth=0 HTTP/1.1" 404 None*
>
>
> When I look deeper here, I see that this is the core of the issue:
>
> Starting new HTTPS connection (1): www.peeringdb.com
> <http://www.peeringdb.com>
> "*GET /api/netfac?since=1484585605&limit=0* HTTP/1.1" 200 None
> netfac last update 1484585605 1808 changed
> data to be processed 1808
>
> Which, if we look at the data this returns, contains "fac_id": 3.
>
> So, suggestions on how to proceed here?
>
> /Ruairi
>
>
> _______________________________________________
> Pdb-tech mailing list
> Pdb-tech at lists.peeringdb.com
> http://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech
>