[PDB Tech] incomplete local sync

Greg Dendy gregdendy at gmail.com
Fri Mar 24 13:42:39 PDT 2017 

Thanks again.  That cleared up the upgrade errors, but revealed issues running peeringdb functions. 

$ peeringdb drop_tables
Traceback (most recent call last):
  File "/bin/peeringdb", line 11, in <module>
    sys.exit(cli())
  File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/peeringdb/cli.py", line 176, in drop_tables
    from peeringdb.localdb import LocalDB
  File "/usr/lib/python2.7/site-packages/peeringdb/localdb.py", line 7, in <module>
    from django.db.models import get_app, get_models
ImportError: cannot import name get_app

At this point, active versions were:
django 1.10.6      <— UH-OH!
django-peeringdb 0.3.1
peeringdb 0.5.0
pip 9.0.1
setuptools 34.3.2

Referencing Chris’ thread earlier this month, I downrev’d django to 1.8.17 and now the spice can flow.  This appears to be a good set of versions to be on: 

django 1.8.17
django-peeringdb 0.3.1
peeringdb 0.5.0
pip 9.0.1
setuptools 34.3.2


Never was there a tale of more woe, than this of me and my poor django.

Thanks Stefan.

Greg


> On Mar 24, 2017, at 12:01 PM, Stefan Pratter <stefan at 20c.com> wrote:
> 
> Install error are caused by old pip version, can upgrade by running
> 
> pip install pip --upgrade
> pip install setuptools --upgrade
> 
> install should work after that
> 
> Stefan
> On 24/03/2017 20.55, Greg Dendy wrote:
>> Good one, that describes my experience syncing, multiple runs needed to push through errors. My versions:
>> 
>> django-peeringdb: 0.2.2
>> peeringdb: 0.5.0
>> 
>> Here’s how it fails on a fresh sync:
>> 
>> https://www.peeringdb.com:443 "GET /api/net?since=0 HTTP/1.1" 200 None
>> net last update 0 8933 changed
>> data to be processed 8933
>> Traceback (most recent call last):
>>  File "/bin/peeringdb", line 11, in <module>
>>    sys.exit(cli())
>>  File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in __call__
>>    return self.main(*args, **kwargs)
>>  File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in main
>>    rv = self.invoke(ctx)
>>  File "/usr/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
>>    return _process_result(sub_ctx.command.invoke(sub_ctx))
>>  File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in invoke
>>    return ctx.invoke(self.callback, **ctx.params)
>>  File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
>>    return callback(*args, **kwargs)
>>  File "/usr/lib/python2.7/site-packages/peeringdb/cli.py", line 167, in sync
>>    db.sync()
>>  File "/usr/lib/python2.7/site-packages/peeringdb/localdb.py", line 141, in sync
>>    call_command('pdb_sync', interactive=False)
>>  File "/usr/lib64/python2.7/site-packages/django/core/management/__init__.py", line 120, in call_command
>>    return command.execute(*args, **defaults)
>>  File "/usr/lib64/python2.7/site-packages/django/core/management/base.py", line 445, in execute
>>    output = self.handle(*args, **options)
>>  File "/usr/lib/python2.7/site-packages/django_peeringdb/management/commands/pdb_sync.py", line 85, in handle
>>    self.sync(tables)
>>  File "/usr/lib/python2.7/site-packages/django_peeringdb/management/commands/pdb_sync.py", line 92, in sync
>>    self.update_db(cls, self.get_objs(cls))
>>  File "/usr/lib/python2.7/site-packages/django_peeringdb/management/commands/pdb_sync.py", line 127, in update_db
>>    sync.sync_obj(cls, row)
>>  File "/usr/lib/python2.7/site-packages/django_peeringdb/sync.py", line 25, in sync_obj
>>    obj.full_clean()
>>  File "/usr/lib64/python2.7/site-packages/django/db/models/base.py", line 1171, in full_clean
>>    raise ValidationError(errors)
>> django.core.exceptions.ValidationError: {'info_traffic': [u"Value u'10 Tbps+' is not a valid
>> choice.”]}
>> 
>> 
>> So, next try to upgrade django-peeringdb. Here’s how that fails:
>> 
>> $ pip install django-peeringdb --upgrade
>> Collecting django-peeringdb
>>  Using cached django-peeringdb-0.3.1.tar.gz
>> Collecting django_countries>=0.1.0 (from django-peeringdb)
>>  Using cached django_countries-4.2-py2.py3-none-any.whl
>> Collecting django_handleref<0.2.0,>=0.1.4 (from django-peeringdb)
>>  Using cached django-handleref-0.1.5.tar.gz
>> Collecting django_inet<0.4,>=0.3.2 (from django-peeringdb)
>>  Using cached django-inet-0.3.2.tar.gz
>>    Complete output from command python setup.py egg_info:
>>    /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_requires'
>>      warnings.warn(msg)
>>    error in django-inet setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers
>> 
>>    ----------------------------------------
>> Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-9Dm5Ax/django-inet/
>> 
>> 
>> 
>> Greg
>> 
>> 
>>> On Mar 23, 2017, at 9:49 PM, Stefan Pratter <stefan at 20c.com> wrote:
>>> 
>>> The one situation that comes to mind that would cause what you are describing is if you are running an older version of django-peeringdb and it's failing validation on some of the enum fields that had values added to them - e.g. they are valid on the server, but not valid locally - which would affect a lot of the entities that are missing for you.
>>> 
>>> Furthermore it would notify you of that failure initially, but consecutive runs of sync on that same database may appear to complete successfully depending on which objects they touch.
>>> 
>>> To be sure, i just ran a local sync on a fresh database and on an existing one and it behaves normally for me in both cases.
>>> 
>>> Tested with these versions:
>>> 
>>> django-peeringdb==0.3.1
>>> peeringdb==0.5.0
>>> 
>>> What is the output when you run peeringdb sync on a fresh database?
>>> 
>>> Stefan
>>> 
>>> On 23/03/2017 23.58, Greg Dendy wrote:
>>>> Hola PDB-Tech
>>>> 
>>>> I’ve noticed that my peeringdb sql syncs over the last week or so are
>>>> considerably smaller than normal, by about 75%.  The sync process
>>>> appears to complete successfully, but network, facility, and IX records
>>>> appear to be missing.
>>>> 
>>>> Anyone else seeing this behavior?
>>>> 
>>>> Are there any known issues with the sync function that could cause this
>>>> behavior?
>>>> 
>>>> Thanks,
>>>> 
>>>> Greg
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Pdb-tech mailing list
>>>> Pdb-tech at lists.peeringdb.com
>>>> http://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech
>>>> 
>>> _______________________________________________
>>> Pdb-tech mailing list
>>> Pdb-tech at lists.peeringdb.com
>>> http://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech
>>

More information about the Pdb-tech mailing list