Django - can't run tests while the application works normally - django

Problem:
The problem is that I have an application which works ok, but when I try to run tests with the command:
coverage run manage.py test --settings=crm.settings.test
there occurs an error in the very beggining:
Creating test database for alias 'default'...
IntegrityError: profiles_usermodel.current_project_id may not be NULL
Previously I ran tests and everything worked nicely. Then I significantly changed the models and the application and tried to run tests -- as a result I got the mentioned above problem. What I did wrong?
My settings:
I have separate settings for tests:
My_application
|___My_application
|___urls.py
|___ __init__.py
|___wsgi.py
|___settings
|___base.py
|___test.py
|___local.py
|___profiles
|___models.py
|___views.py
|___tests
|___ __init__.py
|___models.py
The test settings are as follows:
"""Local test settings and globals which allows us to run our
test suite locally."""
from .base import *
########## TEST SETTINGS
TEST_RUNNER = 'discover_runner.DiscoverRunner'
TEST_DISCOVER_TOP_LEVEL = PROJECT_ROOT
TEST_DISCOVER_ROOT = PROJECT_ROOT
TEST_DISCOVER_PATTERN = "*"
########## IN-MEMORY TEST DATABASE
DATABASES = {
"default": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": ":memory:",
"USER": "",
"PASSWORD": "",
"HOST": "",
"PORT": "",
},
}
INSTALLED_APPS += ('coverage',)
And there is the models from My_application/profiles/models.py (ommited several fields and methods):
from django.db import models
from django.contrib.auth.models import (
BaseUserManager, AbstractBaseUser, PermissionsMixin )
from django.contrib.auth.models import Group
class Project(models.Model):
name = models.CharField(max_length=255)
class UserModel(AbstractBaseUser, PermissionsMixin):
email = models.EmailField(max_length=255, unique=True, db_index=True,)
current_project = models.ForeignKey(Group, related_name='usermodel_current')
USERNAME_FIELD = 'email'
Also I use South for db migrations and django 1.5c2.

The error is being pretty explicit.
The test runner is attempting to create the database, and insert a record into 'profiles_usermodel'. This insertion is failing because the field 'current_project' cannot be null, but the creation process isn't setting anything into that field.
We'd have to see your tests to narrow down exactly where this is occurring.
If you don't care about getting it working "right" and just want it to work, update your user model, and add null=True to the kwargs of UserModel.current_project.

Related

django: adding custom permissions stopped working

I have a django project with multiple apps. In one of the apps when I add custom permissions to any model and run makemigration, the migration-file to add the permission is created. When I apply the migration I get no error messages but the permission isn't added to the auth_permission table.
class Meta:
app_label = 'my_app'
permissions = (
('assign_work_type', 'Assign work type'),
)
The migration completes without errors
I have tried doing the same in other apps and that works. I have also tried adding a column to the current app and that works as well. Anyone got any idea what it could be? I am running django 1.11.26
UPDATE
Here is the content of the migration file
# -*- coding: utf-8 -*-
# Generated by Django 1.11.26 on 2019-11-25 11:13
from __future__ import unicode_literals
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('timereport', '0143_auto_20191122_1754'),
]
operations = [
migrations.AlterModelOptions(
name='worktype',
options={'permissions': (('assign_work_type', 'Assign work type'),)},
),
]
After quite some investigation I found that the affected app was missing the models_module, i.e. the "models.py" file. I have all my models in a /model/ directory and a while back I deleted the models.py file thinking it was of no use.
Adding the models.py file back solved the issue

Django test database requires Postgres Extension enabled

I have a Django app which requires Postgres' fuzzystrmatch extension to be enabled on the database.
Django's unittest framework creates and destroys a new database. I need this new database to have the extension turned on for testing.
I can use './manage.py test --keepdb' to keep the database and then manually turn on the extension, but this is an ineloquent solution.
Any idea how I can enable this extension programmatically?
Create an empty migration and use the CreateExtension operation:
from django.contrib.postgres.operations import CreateExtension
class Migration(migrations.Migration):
...
operations = [
CreateExtension(name='fuzzystrmatch'),
...
]
Relevant docs
Edit:
If that doesn't work, reviewing how Django actually handles those classes internally would be my next suggestion.
In case you've disabled the migrations in the test settings, the migrations will not be run, so you can't use django CreateExtension.
Example:
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": "test-database",
"USER": "username",
"PASSWORD": "password",
"HOST": "postgresql-host",
"PORT": "5432",
"TEST": {
"MIGRATE": False,
},
},
}
In this case you can manually call SQL code to create the extension.
If you use unitest, you can for example create a custom TestCase class inherited by all your tests like so:
from django.test import TestCase from django.db import connection
class PostgresExtensionTestCase(TestCase):
#classmethod
def setUpClass(cls):
with connection.cursor() as cursor:
cursor.execute("CREATE EXTENSION IF NOT EXISTS unaccent;")
super().setUpClass()
If you use pytest, you can create a fixture in your conftest.py like so (for unaccent in the example):
from django.db import connection
#pytest.fixture(autouse=True)
def activate_postgresql_unaccent():
with connection.cursor() as cursor:
cursor.execute("CREATE EXTENSION IF NOT EXISTS unaccent;")

right way to create a django data migration that creates a group?

I would like to create data migrations that create Permissions and Groups, so that my other developers can just run the migrations and get everything set up. I was able to create the migrations and run them just fine, but now I'm getting an error when running my tests.
But if I do this:
from django.contrib.auth.models import Group
def add_operations_group(apps, schema_editor):
Group.objects.get_or_create(name='operations')
I get:
django.db.utils.OperationalError: no such table: auth_group
If I do this:
def add_operations_group(apps, schema_editor):
Group = apps.get_model("django.contrib.auth", "group")
Group.objects.get_or_create(name='operations')
I get:
LookupError: No installed app with label 'django.contrib.auth'
Is there a way to do this? Or is there a "Django Way" to make sure things like permissions and groups are created?
This is how I do it:
from django.db import models, migrations
def apply_migration(apps, schema_editor):
Group = apps.get_model('auth', 'Group')
Group.objects.bulk_create([
Group(name=u'group1'),
Group(name=u'group2'),
Group(name=u'group3'),
])
def revert_migration(apps, schema_editor):
Group = apps.get_model('auth', 'Group')
Group.objects.filter(
name__in=[
u'group1',
u'group2',
u'group3',
]
).delete()
class Migration(migrations.Migration):
dependencies = [
('someapp', 'XXXX_some_migration'),
]
operations = [
migrations.RunPython(apply_migration, revert_migration)
]
Although, there must be a more Djangonic way.
Answer from César is correct. To make it more Django create the migration file automatically by going to your django app root folder and entering:
python manage.py makemigrations <yourappname> --empty
Note: You may need python3 instead of python depending on your system configuration.
This creates an empty migration file in a sub directory of your app called 0001_initial.py
You can then alter it as per César instructions. Which worked correctly with Django 2.2

django: data migrate permissions

I have a bunch of new permissions which I need to migrate. I tried doing it through data migration but complains about ContentType not being available.
Doing quick research I found out that ContentType table is populated after all the migrations applied.
I even tried using update_all_contenttypes() from from django.contrib.contenttypes.management import update_all_contenttypes
which causes migration to load data which is not consistent to the fixture.
What is the best way to migrate permission data in Django?
Here is a quick and dirty way to ensure all permissions for all apps have been created:
def add_all_permissions(apps=None, schema_editor=None):
from django.contrib.auth.management import create_permissions
if apps is None:
from django.apps import apps
for app_config in apps.get_app_configs():
app_config.models_module = True
create_permissions(app_config, verbosity=0)
app_config.models_module = None
class Migration(migrations.Migration):
dependencies = [('myapp', '0123_do_the_thing')]
operations = [
migrations.RunPython(add_all_permissions,
reverse_code=migrations.RunPython.noop)
# ...
]
NOTE: edited to include ruohola's excellent suggestion
There are 2 ways to solve this:
1) The ugly way:
Run manage.py migrate auth before your wanted migration
2) Recommended way:
from django.contrib.auth.management import create_permissions
def add_permissions(apps, schema_editor):
apps.models_module = True
create_permissions(apps, verbosity=0)
apps.models_module = None
# rest of code here....
Here are steps for adding custom permissions to the User model:
First create a migration file, for example under your authentication application,
Here i named it 0002_permission_fixtures.py:
account (your authentication application)
|_migrations
|__ 0001_initial.py
|__ 0002_permission_fixtures.py
|__ __init__.py
Then adding your permission objects, as follow:
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations
def forwards_func(apps, schema_editor):
# Get models that we needs them
user = apps.get_model("auth", "User")
permission = apps.get_model("auth", "Permission")
content_type = apps.get_model("contenttypes", "ContentType")
# Get user content type object
uct = content_type.objects.get_for_model(user)
db_alias = schema_editor.connection.alias
# Adding your custom permissions to User model:
permission.objects.using(db_alias).bulk_create([
permission(codename='add_sample', name='Can add sample', content_type=uct),
permission(codename='change_sample', name='Can change sample', content_type=uct),
permission(codename='delete_sample', name='Can delete sample', content_type=uct),
])
class Migration(migrations.Migration):
dependencies = [
('contenttypes', '__latest__'),
]
operations = [
migrations.RunPython(
forwards_func,
),
]
To run this migration, first migrate contenttype model, and then migrate your application (here is account).
$ python manage.py migrate contenttypes
$ python manage.py migrate account

Django OperationalError: missing table; migration does not recognize missing table

I'm having trouble in Django 1.7, I am trying to save a user to a table, but I'm getting an error that the table does not exist.
Here is the code I'm executing:
from django.conf import settings
from django.contrib.auth import BACKEND_SESSION_KEY, SESSION_KEY, get_user_model
User = get_user_model()
from django.contrib.sessions.backends.db import SessionStore
from django.core.management.base import BaseCommand
class Command(BaseCommand):
def handle(self, email, *_, **__):
session_key = create_pre_authenticated_session(email)
self.stdout.write(session_key)
def create_pre_authenticated_session(email):
user = User.objects.create(email=email)
session = SessionStore()
session[SESSION_KEY] = user.pk
session[BACKEND_SESSION_KEY] = settings.AUTHENTICATION_BACKENDS[0]
session.save()
return session.session_key
However, at
user = User.objects.create(email=email)
I get an Error message :
django.db.utils.OperationalError: no such table: accounts_user
Here is the user model at accounts/models.py that I'm trying to use to build the table:
from django.db import models
from django.utils import timezone
class User(models.Model):
email = models.EmailField(primary_key=True)
last_login = models.DateTimeField(default=timezone.now)
REQUIRED_FIELDS = ()
USERNAME_FIELD = 'email'
def is_authenticated(self):
return True
I've run sqlmigrate against this migration with 'manage.py accounts 0001.initial' and I have gotten the correct create table SQL back, but running 'manage.py migrate' gives me the following :
Operations to perform:
Apply all migrations: sessions, admin, lists, contenttypes, accounts, auth
Running migrations:
No migrations to apply.
The migration is just the result of running 'makemigration' from the shell, no custom code. I do see accounts listed in the included applications, but the migration isn't being ran, so my site is in an odd spot where Django says the table is missing when I try to use it, but Django says it exists when I try to run the migration to create it. Why does Django erroneously think that the table already exists when I can look at the database and see that it doesn't?
#user856358 Your comment about the other sqlite file seems like the root cause. I encountered the same error, and it was resolved by removing that file and running another migration. In my case, the file was located as specified in settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, '../database/db.sqlite3'),
}
}
By removing the .sqlite3 file there, I was able to successfully run the migration and resolve the no-such-table error...
django.db.utils.OperationalError: no such table: accounts_user
$ rm ../database/db.sqlite3
$ python3 manage.py migrate