Related
So I get a ModelDoesNotExist error when I run manage.py runserver because in a file in my directory, I make a query to a table which has not been populated yet.
def __init__(self):
self.spotify_object = SocialToken.objects.get(account__provider="spotify")
The above is a class instantiated to perform some sort of authentication and the SocialToken table gets populated only after I login. Now, I was wondering if there was a way to escape the error by triggering this part of the code only after I login? I only use the class in an endpoint, and during that period, the table would have been populated but the fact that it is not populated before running the server is causing a DoesNotExist error. Is there a solution to this?
Traceback
File "C:\Users\Kwaku Biney\Desktop\sparison-1\project\Sparison\views.py", line 4, in <module>
from .authentication import SparisonCacheHandler
File "C:\Users\Kwaku Biney\Desktop\sparison-1\project\Sparison\authentication.py", line 43, in
<module>
cache_handler = SparisonCacheHandler() ,
File "C:\Users\Kwaku Biney\Desktop\sparison-1\project\Sparison\authentication.py", line 25,
in __init__
self.spotify_object = SocialToken.objects.get(account__provider="spotify")
File "C:\Users\Kwaku Biney\Desktop\sparison-1\project\venv\lib\site-
packages\django\db\models\manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "C:\Users\Kwaku Biney\Desktop\sparison-1\project\venv\lib\site-
packages\django\db\models\query.py", line 429, in get
raise self.model.DoesNotExist(
allauth.socialaccount.models.DoesNotExist: SocialToken matching query does not exist.
In my views.py, I import the class which has the query and the error comes up.
There are two ways to avoid the DoesNotExist Error.
A) Use .filter() instead of .get()
Filtering leads to an empty queryset when the search comes up empty.
def __init__(self):
self.spotify_object = SocialToken.objects.filter(account__provider="spotify")
B) Use get_object_or_404() instead of .get()
This is a built-in function by django:
Calls get() on a given model manager, but it raises Http404 instead of the model’s DoesNotExist exception.
Django Documentation
def __init__(self):
self.spotify_object = SocialToken.objects.get_object_or_404(account__provider="spotify")
Hope I could help you. Have a nice day.
I have created a flask migrate script ,however, when I running the upgrade function, I get the following error:
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> 6378428b838a, empty message
Traceback (most recent call last):
File "migrate.py", line 22, in <module>
manager.run()
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/flask_script/__init__.py", line 417, in run
result = self.handle(argv[0], argv[1:])
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/flask_script/__init__.py", line 386, in handle
res = handle(*args, **config)
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/flask_script/commands.py", line 216, in __call__
return self.run(*args, **kwargs)
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/flask_migrate/__init__.py", line 95, in wrapped
f(*args, **kwargs)
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/flask_migrate/__init__.py", line 280, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/alembic/command.py", line 298, in upgrade
script.run_env()
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/alembic/script/base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/alembic/util/compat.py", line 173, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "migrations/env.py", line 96, in <module>
run_migrations_online()
File "migrations/env.py", line 90, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
self.get_context().run_migrations(**kw)
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/alembic/runtime/migration.py", line 518, in run_migrations
step.migration_fn(**kw)
File "/Users/slatifi/git/StaffTrainingLog/migrations/versions/6378428b838a_.py", line 23, in upgrade
batch_op.create_foreign_key(None, 'organisation', ['organisation'], ['id'])
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/contextlib.py", line 119, in __exit__
next(self.gen)
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/alembic/operations/base.py", line 325, in batch_alter_table
impl.flush()
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/alembic/operations/batch.py", line 106, in flush
fn(*arg, **kw)
File "/Users/slatifi/git/StaffTrainingLog/venv/lib/python3.7/site-packages/alembic/operations/batch.py", line 390, in add_constraint
raise ValueError("Constraint must have a name")
ValueError: Constraint must have a name
I have seen other people with the same error and they simply added render_as_batch to the env.py file. I did this but I still get the same error. Any thoughts?
Note: This is the modification I made in the env.py file:
with connectable.connect() as connection:
context.configure(
connection=connection,
target_metadata=target_metadata,
process_revision_directives=process_revision_directives,
**current_app.extensions['migrate'].configure_args,
render_as_batch=True
)
This is the upgrade script created by the migration
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = '2838e3e96536'
down_revision = None
branch_labels = None
depends_on = None
def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table('user', schema=None) as batch_op:
batch_op.add_column(sa.Column('organisation', sa.String(length=5), nullable=False))
batch_op.create_foreign_key(None, 'organisation', ['organisation'], ['id'])
# ### end Alembic commands ###
def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table('user', schema=None) as batch_op:
batch_op.drop_constraint(None, type_='foreignkey')
batch_op.drop_column('organisation')
# ### en
d Alembic commands ###
This is normal because SQLite3 doesn't support ALTER tables.
You can pass the render_as_batch=True during the instantiation of Flask-Migrate like this :
migrate = Migrate(app,db,render_as_batch=True)
Don't need to modify the env file of Flask-Migrate.
In addition, to totally avoid the problem for your futures migrations, you can create a constraint naming templates for all types of constraints to the SQLAlchemy metadata, and then I think you will get consistent names.
See how to do this in the Flask-SQLAlchemy documentation. I'm copying the code example from the docs below for your convenience:
from sqlalchemy import MetaData
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
convention = {
"ix": 'ix_%(column_0_label)s',
"uq": "uq_%(table_name)s_%(column_0_name)s",
"ck": "ck_%(table_name)s_%(constraint_name)s",
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
"pk": "pk_%(table_name)s"
}
metadata = MetaData(naming_convention=convention)
db = SQLAlchemy(app, metadata=metadata)
The other answers show how to configure Flask to use named constraints in the future, but that doesn't solve the problem of dropping existing unnamed constraints in Alembic migrations. To handle any existing unnamed constraints, you need to also define the naming convention in the migration script, as explained in the Alembic docs.
For example, suppose you're trying to do a migration that adds ON DELETE CASCADE to an existing foreign key in your test table. If you've followed one of the other answers (e.g. https://stackoverflow.com/a/62651160/470844) and added the naming_convention to the Flask init script, then flask db upgrade will generate something like this:
def upgrade():
with op.batch_alter_table('test', schema=None) as batch_op:
batch_op.drop_constraint(None, type_='foreignkey')
batch_op.create_foreign_key(batch_op.f('fk_test_user_id_user'), 'user', ['user_id'], ['id'], ondelete='CASCADE')
Note that while the create_foreign_key call uses a constraint name (i.e. fk_test_user_id_user), the drop_constraint call still uses None as the constraint name, which will cause ValueError: Constraint must have a name error in this question's title.
To fix that, you need to edit the migration to use the naming_convention, and replace the None with the generated constraint name. For example, you'd change the above upgrade to:
naming_convention = {
"ix": 'ix_%(column_0_label)s',
"uq": "uq_%(table_name)s_%(column_0_name)s",
"ck": "ck_%(table_name)s_%(column_0_name)s",
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
"pk": "pk_%(table_name)s"
}
def upgrade():
with op.batch_alter_table('test', schema=None, naming_convention=naming_convention) as batch_op:
batch_op.drop_constraint('fk_test_user_id_user', type_='foreignkey')
batch_op.create_foreign_key(batch_op.f('fk_test_user_id_user'), 'user', ['user_id'], ['id'], ondelete='CASCADE')
(In this example, you'd also need to disable SQLite foreign key support while running the migration script, e.g. using the technique here, to avoid the batch migration itself triggering a cacading delete. The problem is documented here.)
Installing Flask-Migrate helps to avoid code references to declarative_base or modifying the env.py file.
For apps with app factory, split the metadata assignment and the db initiation between the extensions.py and __init__.py files.
extensions.py:
from sqlalchemy import MetaData
from flask_sqlalchemy import SQLAlchemy
metadata = MetaData(
naming_convention={
"ix": 'ix_%(column_0_label)s',
"uq": "uq_%(table_name)s_%(column_0_name)s",
"ck": "ck_%(table_name)s_%(constraint_name)s",
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
"pk": "pk_%(table_name)s"
}
)
db=SQLAlchemy(metadata=metadata)
__init__.py:
from flask import Flask, Blueprint
from config import ...
from flask_migrate import Migrate
from extensions import db
...
def create_app():
app = Flask(__name__,
template_folder='../app/templates')
app.config.from_object(config[...])
config[...].init_app(app)
db.init_app(app)
with app.app_context():
db.create_all()
migrate = Migrate(app, db)
...
from .templates.main import main
app.register_blueprint(main)
return app
I'm using Django + Haystack + Elasticsearch.
When I send a request to this view
from haystack.views import FacetedSearchView
from .models import Object
class ObjectView(FacetedSearchView):
def extra_context(self):
extra = super(ObjectView, self).extra_context()
if not self.results:
extra['objects'] = Object.objects.all()
else:
searchqueryset = self.form.search()
results = [ result.pk for result in searchqueryset ]
extra['facets'] = self.results.facet_counts()
extra['objects'] = Object.objects.filter(pk__in=results)
extra['results'] = self.results
return extra
this error is raised:
File "/home/deploy/.virtualenvs/deploy/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 115, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/home/deploy/.virtualenvs/deploy/local/lib/python2.7/site-packages/haystack/views.py", line 49, in __call__
return self.create_response()
File "/home/deploy/.virtualenvs/deploy/local/lib/python2.7/site-packages/haystack/views.py", line 129, in create_response
(paginator, page) = self.build_page()
File "/home/deploy/.virtualenvs/deploy/local/lib/python2.7/site-packages/haystack/views.py", line 106, in build_page
self.results[start_offset:start_offset + self.results_per_page]
File "/home/deploy/.virtualenvs/deploy/local/lib/python2.7/site-packages/haystack/query.py", line 266, in __getitem__
self._fill_cache(start, bound)
File "/home/deploy/.virtualenvs/deploy/local/lib/python2.7/site-packages/haystack/query.py", line 164, in _fill_cache
results = self.query.get_results(**kwargs)
File "/home/deploy/.virtualenvs/deploy/local/lib/python2.7/site-packages/haystack/backends/__init__.py", line 485, in get_results
self.run(**kwargs)
File "/home/deploy/.virtualenvs/deploy/local/lib/python2.7/site-packages/haystack/backends/elasticsearch_backend.py", line 942, in run
results = self.backend.search(final_query, **search_kwargs)
File "/home/deploy/.virtualenvs/deploy/local/lib/python2.7/site-packages/haystack/backends/__init__.py", line 26, in wrapper
return func(obj, query_string, *args, **kwargs)
File "/home/deploy/.virtualenvs/deploy/local/lib/python2.7/site-packages/haystack/backends/elasticsearch_backend.py", line 521, in search
distance_point=kwargs.get('distance_point'), geo_sort=geo_sort)
File "/home/deploy/.virtualenvs/deploy/local/lib/python2.7/site-packages/haystack/backends/elasticsearch_backend.py", line 571, in _process_results
raw_suggest = raw_results['suggest']['suggest']
KeyError: 'suggest'
A curious fact: the problem occurs only when the project is under production settings, even when I haven't changed a single thing involving Haystack or Elasticsearch in the settings_production module(except for the URL key).
project/settings_production.py
'URL': 'http://0.0.0.0:9200/'
In production, I'm using nothing more than a simple FastCGI.
And here's what really bothers me: sometimes I get no errors on this view, and everything works just fine...
Please, someone has an idea of what's going on?
Thanks a lot!
UPDATE:
SO, I setup my whole project in another computer. After some tests I verified:
this problem is not related to my production settings like I
described above;
the error is not raised when the elasticsearch service is stopped;
if the service is running:
when the method Object.objects.all() returns some QuerySet results, I got no errors;
when the method Object.objects.all() returns an empty QuerySet, the problem persists;
I guess this is some kind of bug in the Haystack's elasticsearch_backend module.
Still, i'm not sure.
Yup, it's a bug in haystack. I've put in a pull request, but in the meantime, options to get running are:
Set INCLUDE_SPELLING in your haystack settings to False, or
Use our fork: https://github.com/greenkahuna/django-haystack
So I was developing an app in Django and needed a function from the 1.4 version so I decided to update.
But then a weird error appeared when I wanted to do syncdb
I am using the new manage.py and as You can see it makes some of the tables but then fails :
./manage.py syncdb
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table django_site
Traceback (most recent call last):
File "./manage.py", line 9, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/__init__.py", line 443, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/__init__.py", line 382, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/base.py", line 196, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/base.py", line 232, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/base.py", line 371, in handle
return self.handle_noargs(**options)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/commands/syncdb.py", line 91, in handle_noargs
sql, references = connection.creation.sql_create_model(model, self.style, seen_models)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/db/backends/creation.py", line 44, in sql_create_model
col_type = f.db_type(connection=self.connection)
TypeError: db_type() got an unexpected keyword argument 'connection'
I had the same issue, the definition for my custom field was missing the connection parameter.
from django.db import models
class BigIntegerField(models.IntegerField):
def db_type(self, connection):
return "bigint"
Although already old, answered and accepted question but I am adding my understanding I have added it because I am not using customized type and it is a Django Evolution error (but not syncdb)evolve --hint --execute. I think it may be helpful for someone in future. .
I am average in Python and new to Django. I also encounter same issue when I added some new features to my existing project. To add new feature I had to add some new fields of models.CharField() type,as follows.
included_domains = models.CharField(
"set of comma(,) seprated list of domains in target emails",
default="",
max_length=it_len.EMAIL_LEN*5)
excluded_domains = models.CharField(
"set of comma(,) seprated list of domains NOT in target emails",
default="",
max_length=it_len.EMAIL_LEN*5)
The Django version I am using is 1.3.1:
$ python -c "import django; print django.get_version()"
1.3.1 <--------# version
$python manage.py syncdb
Project signature has changed - an evolution is required
Django Evolution: Django Evolution is an extension to Django that allows you to track changes in your models over time, and to update the database to reflect those changes.
$ python manage.py evolve --hint
#----- Evolution for messagingframework
from django_evolution.mutations import AddField
from django.db import models
MUTATIONS = [
AddField('MessageConfiguration', 'excluded_domains', models.CharField, initial=u'', max_length=300),
AddField('MessageConfiguration', 'included_domains', models.CharField, initial=u'', max_length=300)
]
#----------------------
Trial evolution successful.
Run './manage.py evolve --hint --execute' to apply evolution.
The trial was susses and when I tried to apply changes in DB
$ python manage.py evolve --hint --execute
Traceback (most recent call last):
File "manage.py", line 25, in <module>
execute_manager(settings)
File "/var/www/sites/www.taxspanner.com/django/core/management/__init__.py", line 362, in execute_manager
utility.execute()
File "/var/www/sites/www.taxspanner.com/django/core/management/__init__.py", line 303, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/var/www/sites/www.taxspanner.com/django/core/management/base.py", line 195, in run_from_argv
self.execute(*args, **options.__dict__)
File "/var/www/sites/www.taxspanner.com/django/core/management/base.py", line 222, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/django_evolution-0.6.9.dev_r225-py2.7.egg/django_evolution/management/commands/evolve.py", line 60, in handle
self.evolve(*app_labels, **options)
File "/usr/local/lib/python2.7/dist-packages/django_evolution-0.6.9.dev_r225-py2.7.egg/django_evolution/management/commands/evolve.py", line 140, in evolve
database))
File "/usr/local/lib/python2.7/dist-packages/django_evolution-0.6.9.dev_r225-py2.7.egg/django_evolution/mutations.py", line 426, in mutate
return self.add_column(app_label, proj_sig, database)
File "/usr/local/lib/python2.7/dist-packages/django_evolution-0.6.9.dev_r225-py2.7.egg/django_evolution/mutations.py", line 438, in add_column
sql_statements = evolver.add_column(model, field, self.initial)
File "/usr/local/lib/python2.7/dist-packages/django_evolution-0.6.9.dev_r225-py2.7.egg/django_evolution/db/common.py", line 142, in add_column
f.db_type(connection=self.connection), # <=== here f is field class object
TypeError: db_type() got an unexpected keyword argument 'connection'
To understand this exception I check that this exception is something similar to:
>>> def f(a):
... print a
...
>>> f('b', b='a')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() got an unexpected keyword argument 'b'
>>>
So the function signature has been changed.
Because I have not added any new customized or enum fields but only two similar fields that was already in model and char type field is supported by most of database (I am ussing PostgreSQL) even I was getting this error!
Then I read from #: Russell Keith-Magee-4 Reply.
What you've hit here is the end of the deprecation cycle for code that
doesn't support multiple databases.
In Django 1.2, we introduced multiple database support; in order to
support this, the prototype for get_db_preb_lookup() and
get_db_prep_value() was changed.
For backwards compatibility, we added a shim that would transparently
'fix' these methods if they hadn't already been fixed by the
developer.
In Django 1.2, the usage of these shims raised a
PendingDeprecationWarning. In Django 1.3, they raised a
DeprecationWarning.
Under Django 1.4, the shim code was been removed -- so any code that
wasn't updated will now raise errors like the one you describe.
But I am not getting any DeprecationWarning warning assuming because of newer version of Django Evolution.
But from above quote I could understand that to support multiple databases function signature is added and an extra argument connection is needed. I also check the db_type() signature in my installation of Django as follows:
/django$ grep --exclude-dir=".svn" -n 'def db_type(' * -R
contrib/localflavor/us/models.py:8: def db_type(self):
contrib/localflavor/us/models.py:24: def db_type(self):
:
:
Ialso refer of Django documentation
Field.db_type(self, connection):
Returns the database column data type for the Field, taking into account the connection
object, and the settings associated with it.
And Then I could understand that to resolve this issue I have to inherited models.filed class and overwrite def db_type() function. And because I am using PostgreSQL in which to create 300 chars type field I need to return 'char(300)'. In my models.py I added:
class CharMaxlengthN(models.Field):
def db_type(self, connection):
return 'char(%d)' % self.max_length # because I am using postgresql
If you encounter similar problem please check your underline DB's manual that which type of column you need to create and return a string.
And changed the definition of new fields (that I need to add) read comments:
included_domains = CharMaxlengthN( # <--Notice change
"set of comma(,) seprated list of domains in target emails",
default="",
max_length=it_len.EMAIL_LEN*5)
excluded_domains = CharMaxlengthN( # <-- Notice change
"set of comma(,) seprated list of domains NOT in target emails",
default="",
max_length=it_len.EMAIL_LEN*5)
Then I executed same command that was failing previously:
t$ python manage.py evolve --hint --execute
You have requested a database evolution. This will alter tables
and data currently in the None database, and may result in
IRREVERSABLE DATA LOSS. Evolutions should be *thoroughly* reviewed
prior to execution.
Are you sure you want to execute the evolutions?
Type 'yes' to continue, or 'no' to cancel: yes
Evolution successful.
I also check my DB and tested my new added features It is now working perfectly, and no DB problem.
If you wants to create ENUM field read Specifying a mySQL ENUM in a Django model.
Edit: I realized instead of sub classing models.Field I should have inherit more specific subclass that is models.CharField.
Similarly I need to create Decimal DB fields so I added following class in model:
class DecimalField(models.DecimalField):
def db_type(self, connection):
d = {
'max_digits': self.max_digits,
'decimal_places': self.decimal_places,
}
return 'numeric(%(max_digits)s, %(decimal_places)s)' % d
I recently updgraded to django 1.3. After the upgrade, I get the following error whenever I used request.POST:
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/django/core/handlers/base.py", line 86, in get_response
response = None
File "/public/gdp/trunk/src/ukl/lis/process/utils/error_handler.py", line 15, in __call__
return self.function(*args, **kwargs)
File "/usr/lib/python2.4/site-packages/django/views/decorators/cache.py", line 30, in _cache_controlled
# and this:
File "/public/gdp/trunk/src/ukl/lis/process/authentication/views.py", line 438, in login
form = loginForm(request.POST)
File "/usr/lib/python2.4/site-packages/django/core/handlers/modpython.py", line 101, in _get_post
self._load_post_and_files()
AttributeError: 'ModPythonRequest' object has no attribute '_load_post_and_files'
Once I reverted back to django 1.0 the error is fixed.
Why is django 1.3 alone throwing this error? How to correct it?
Stepping from Django 1.0 to Django 1.3 is a big jump, a lot of items might have been deprecated or no longer used, I recommend you to just check some of the documentation for the middleware_classes
Django Middleware documentation
I tried re-installing my mod-python and now the error is fixed. Now thinking of migrating to mod_wsgi.