Django migrations issues with foreign dependency - django

I am trying to deal with an "it works on my machine" type of problem. I have a django project using jenkins which fails on migrations. The step is triggered with this command: python manage.py test -v 2 --noinput and there goes traceback:
Applying gloodny.0020_auto_20180314_1338... OK
Applying openinghours.0001_initial... OK
Applying payu_payment.0002_auto_20171222_1407... OK
Applying sessions.0001_initial... OK
Traceback (most recent call last):
File "manage.py", line 22, in <module>
execute_from_command_line(sys.argv)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
utility.execute()
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/core/management/commands/test.py", line 30, in run_from_argv
super(Command, self).run_from_argv(argv)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/core/management/commands/test.py", line 74, in execute
super(Command, self).execute(*args, **options)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/core/management/commands/test.py", line 90, in handle
failures = test_runner.run_tests(test_labels)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/test/runner.py", line 532, in run_tests
old_config = self.setup_databases()
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/test/runner.py", line 482, in setup_databases
self.parallel, **kwargs
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/test/runner.py", line 726, in setup_databases
serialize=connection.settings_dict.get("TEST", {}).get("SERIALIZE", True),
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/db/backends/base/creation.py", line 78, in create_test_db
self.connection._test_serialized_contents = self.serialize_db_to_string()
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/db/backends/base/creation.py", line 122, in serialize_db_to_string
serializers.serialize("json", get_objects(), indent=None, stream=out)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/core/serializers/__init__.py", line 129, in serialize
s.serialize(queryset, **options)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/core/serializers/base.py", line 79, in serialize
for count, obj in enumerate(queryset, start=1):
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/db/backends/base/creation.py", line 118, in get_objects
for obj in queryset.iterator():
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/db/models/query.py", line 52, in __iter__
results = compiler.execute_sql()
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 848, in execute_sql
cursor.execute(sql, params)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/db/utils.py", line 95, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/var/lib/jenkins/workspace/papuportal/env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "openinghours_company" does not exist
LINE 1: ...mpany"."slug", "openinghours_company"."logo" FROM "openingho...
This is caused by openinghours django library.
Problem I am facing is more complicated since locally those migrations run without error:
Applying gloodny.0020_auto_20180314_1338... OK
Applying gloodny.0021_auto_20180320_1426... OK
Applying gloodny.0022_auto_20180320_1550... OK
Applying openinghours.0001_initial... OK
Applying openinghours.0002_auto_20180320_1426... OK
Applying payu_payment.0002_auto_20171222_1407... OK
Applying sessions.0001_initial... OK
Applying thumbnail.0001_initial... OK
I have set OPENINGHOURS_PREMISES_MODEL settings variable to a model I actually have, I have also tried to add a dependency:
dependencies = [
('payu_payment', '0001_initial'),
('openinghours', '0002_auto_20180320_1426'),
]
but then the build failed with this error:
Migration payu_payment.0002_auto_20171222_1407 dependencies reference nonexistent parent node (u'openinghours', u'0002_auto_20180320_1426')
How can I fix those migrations?
EDIT I fixed a typo pointed out in comments, but now I can run tests with command mentioned above locally and on jenkins I get the same error.

Related

Django's remove_stale_contenttypes command fails with DoesNotExist error

I recently tried to run remove_stale_contenttypes in a Django 3.1 app, after deleting some custom permissions. The first time it run it warned me that some entries would be deleted, and I went ahead and entered 'yes'. The command failed with a DoesNotExist error for one of the models, and after that I can't even see the warning, it fails right away, with the following output:
Traceback (most recent call last):
File "C:\Users\Yamil\Google Drive\Proyectos\monely\manage.py", line 22, in <module>
main()
File "C:\Users\Yamil\Google Drive\Proyectos\monely\manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\django\core\management\__init__.py", line 401, in execute_from_command_line
utility.execute()
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\django\core\management\__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\django\core\management\base.py", line 330, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\django\core\management\base.py", line 371, in execute
output = self.handle(*args, **options)
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\django\contrib\contenttypes\management\commands\remove_stale_contenttypes.py", line 55, in handle
collector.collect([ct])
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\django\db\models\deletion.py", line 295, in collect
if sub_objs:
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\django\db\models\query.py", line 291, in __bool__
self._fetch_all()
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\django\db\models\query.py", line 1303, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\polymorphic\query.py", line 64, in _polymorphic_iterator
real_results = self.queryset._get_real_instances(base_result_objects)
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\polymorphic\query.py", line 387, in _get_real_instances
if base_object.polymorphic_ctype_id == self_model_class_id:
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\django\db\models\query_utils.py", line 149, in __get__
instance.refresh_from_db(fields=[field_name])
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\django\db\models\base.py", line 635, in refresh_from_db
db_instance = db_instance_qs.get()
File "C:\Users\Yamil\anaconda3\envs\monely\lib\site-packages\django\db\models\query.py", line 429, in get
raise self.model.DoesNotExist(
productos.models.DoesNotExist: Producto matching query does not exist.
Now, I noticed that the model that failed is a PolymorphicModel (django-polymorphic), but I have no idea what could be going on. The application seems to be working fine, it passes the tests. I also briefly checked the database and couldn't really find any unused contenttypes, but the command keeps failing. How can I find the root of this problem?

Unable to restore DB after Database change to Postgres (Django)

I have migrated from SQLite to Postgres, but now I am unable to restore my data on Postgres, i am using Django db-backup package, it is giving me this error on dbrestore command
Traceback (most recent call last):
File "C:\Users\atifs\Documents\food_deliveryapp\fooddelivery\manage.py", line 22, in <module>
main()
File "C:\Users\atifs\Documents\food_deliveryapp\fooddelivery\manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "C:\Users\atifs\Documents\food_deliveryapp\virtual-env\lib\site-packages\django\core\management\__init__.py", line 419, in execute_from_command_line
utility.execute()
File "C:\Users\atifs\Documents\food_deliveryapp\virtual-env\lib\site-packages\django\core\management\__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Users\atifs\Documents\food_deliveryapp\virtual-env\lib\site-packages\django\core\management\base.py", line 354, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\Users\atifs\Documents\food_deliveryapp\virtual-env\lib\site-packages\django\core\management\base.py", line 398, in execute
output = self.handle(*args, **options)
File "C:\Users\atifs\Documents\food_deliveryapp\virtual-env\lib\site-packages\dbbackup\management\commands\dbrestore.py", line 53, in handle
self._restore_backup()
File "C:\Users\atifs\Documents\food_deliveryapp\virtual-env\lib\site-packages\dbbackup\management\commands\dbrestore.py", line 94, in _restore_backup
self.connector.restore_dump(input_file)
File "C:\Users\atifs\Documents\food_deliveryapp\virtual-env\lib\site-packages\dbbackup\db\base.py", line 92, in restore_dump
result = self._restore_dump(dump)
File "C:\Users\atifs\Documents\food_deliveryapp\virtual-env\lib\site-packages\dbbackup\db\postgresql.py", line 56, in _restore_dump
stdout, stderr = self.run_command(cmd, stdin=dump, env=self.restore_env)
File "C:\Users\atifs\Documents\food_deliveryapp\virtual-env\lib\site-packages\dbbackup\db\postgresql.py", line 21, in run_command
return super(PgDumpConnector, self).run_command(*args, **kwargs)
File "C:\Users\atifs\Documents\food_deliveryapp\virtual-env\lib\site-packages\dbbackup\db\base.py", line 150, in run_command
raise exceptions.CommandConnectorError(
dbbackup.db.exceptions.CommandConnectorError: Error running: psql --host=localhost --port=5432 --username=postgres --no-password --set ON_ERROR_STOP=on --single-transaction postgres
ERROR: syntax error at or near "AUTOINCREMENT"
LINE 1: ...S "auth_group" ("id" integer NOT NULL PRIMARY KEY AUTOINCREM...

django-elasticsearch-dsl with AWS elasticsearch

I'm trying to use AWS ElasticBeanStalk to deploy a Django app using django-elasticsearch-dsl and I need to run ./manage.py search_index --rebuild the first time I load it. So within my container commands I've got:
03_elasticsearch:
command: "source /opt/python/run/venv/bin/activate && ./src/manage.py search_index --rebuild"
The problem is it is waiting for a n/Y to the question Are you sure you want to delete the 'hjsm-local' indexes? [n/Y]:
How do I rebuild my indecies?
UPDATE
When I use
03_elasticsearch:
command: "source /opt/python/run/venv/bin/activate && ./src/manage.py search_index --rebuild -f"
I get this error
Traceback (most recent call last):
File "./src/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/opt/python/run/venv/local/lib64/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/opt/python/run/venv/local/lib64/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/python/run/venv/local/lib64/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
self.execute(*args, **cmd_options)
File "/opt/python/run/venv/local/lib64/python3.6/site-packages/django/core/management/base.py", line 353, in execute
output = self.handle(*args, **options)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 134, in handle
self._rebuild(models, options)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 114, in _rebuild
self._create(models, options)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 84, in _create
index.create()
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch_dsl/index.py", line 102, in create
self.connection.indices.create(index=self._name, body=self.to_dict(), **kwargs)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch/client/indices.py", line 110, in create
params=params, body=body)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch/transport.py", line 327, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 110, in perform_request
self._raise_error(response.status, raw_data)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch/connection/base.py", line 114, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.TransportError: <exception str() failed>
(ElasticBeanstalk::ExternalInvocationError)
As per the source code, include the -f parameter in your command to force the acceptance of the prompt.

Using django-elasticsearch-dsl with AWS ElasticSearch - raise HTTP_EXCEPTIONS.get(status_code, TransportError)

I'm trying to convert up a Django application using django-elasticsearch-dsl and a elasticsearch server to using AWS ElasticSearch which I am deploying using ElsaticBeanStalk
When I use
03_elasticsearch:
command: "source /opt/python/run/venv/bin/activate && ./src/manage.py search_index --rebuild -f"
I get this error
Creating index 'hjsm-local'
Traceback (most recent call last):
File "./src/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/opt/python/run/venv/local/lib64/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/opt/python/run/venv/local/lib64/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/python/run/venv/local/lib64/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
self.execute(*args, **cmd_options)
File "/opt/python/run/venv/local/lib64/python3.6/site-packages/django/core/management/base.py", line 353, in execute
output = self.handle(*args, **options)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 134, in handle
self._rebuild(models, options)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 114, in _rebuild
self._create(models, options)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 84, in _create
index.create()
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch_dsl/index.py", line 102, in create
self.connection.indices.create(index=self._name, body=self.to_dict(), **kwargs)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch/client/indices.py", line 110, in create
params=params, body=body)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch/transport.py", line 327, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 110, in perform_request
self._raise_error(response.status, raw_data)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/elasticsearch/connection/base.py", line 114, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.TransportError: <exception str() failed>
(ElasticBeanstalk::ExternalInvocationError)
I do not understand what this is telling me - can anyone advise?
UPDATE
I think I am getting an HTTP 406 error, but I do not understand why. If I run locally using localhost:9200 rather than my aws eleasticsearch server, it works fine.
The issue was it needed ElasticSearch 2.4. I have to setup a server with this running and when I connected through it worked!

Django 2.0 haystack whoosh update index, rebuild index throw error

I am using django 2.0 with haystack+whoosh as a search. I configured as it is said in the documentation. Occurred problem is when i run ./manage.py rebuild_index it shows this error:
Traceback (most recent call last):
File "./manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
utility.execute()
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/django/core/management/__init__.py", line 365, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/django/core/management/base.py", line 335, in execute
output = self.handle(*args, **options)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/haystack/management/commands/rebuild_index.py", line 36, in handle
call_command('clear_index', **options)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/django/core/management/__init__.py", line 133, in call_command
', '.join(sorted(valid_options)),
TypeError: Unknown option(s) for clear_index command: batchsize, workers. Valid options are: commit, help, interactive, no_color, nocommit, noinput, pythonpath, settings, skip_checks, stderr, stdout, traceback, using, verbosity, version.
after that i tried update_index it shows me this error:
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/haystack/management/commands/update_index.py", line 214, in handle self.update_backend(label, using)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/haystack/management/commands/update_index.py", line 257, in update_backend
commit=self.commit, max_retries=self.max_retries)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/haystack/management/commands/update_index.py", line 84, in do_update
backend.update(index, current_qs, commit=commit)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/haystack/backends/whoosh_backend.py", line 185, in update
doc = index.full_prepare(obj)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/haystack/indexes.py", line 208, in full_prepare
self.prepared_data = self.prepare(obj)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/haystack/indexes.py", line 199, in prepare
self.prepared_data[field.index_fieldname] = field.prepare(obj)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/haystack/fields.py", line 205, in prepare
return self.convert(super(CharField, self).prepare(obj))
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/haystack/fields.py", line 88, in prepare
values = self.resolve_attributes_lookup(current_objects, attrs)
File "/home/zorig/.virtualenvs/ftm/lib/python3.5/site-packages/haystack/fields.py", line 131, in resolve_attributes_lookup
values.append(current_object())
TypeError: __call__() missing 1 required keyword-only argument: 'manager'
So has anyone faced this error? Is it related to django 2.0 because i managed it work well with django 1.11.
Your installed version of Haystack does not support Django 2.0. Support for Django 2.0 was added in Haystack 2.8, which was released in March 2018.