I'm creating fixtures for permissions in Django. I'm able to get them loaded the way it's needed. However, my question is..say I want to load a fixture for the table auth_group_permissions, I need to specify a group_id and a permission_id, unfortunately fixtures aren't the best way to handle this. Is there an easier way to do this programmatically? So that I can get the id for particular values and have them filled in? How is this normally done?
As of at least Django >=1.7, it is now possible to store permissions in fixtures due to the introduction of "natural keys" as a serialization option.
You can read more about natural keys in the Django serialization documentation
The documentation explicitly mentions the use case for natural keys being when..
...objects are automatically created by Django during the database synchronization process, the primary key of a given relationship isn’t easy to predict; it will depend on how and when migrate was executed. This is true for all models which automatically generate objects, notably including Permission, Group, and User.
So for your specific question, regarding auth_group_permissions, you would dump your fixture using the following syntax:
python manage.py dumpdata auth --natural-foreign --natural-primary -e auth.Permission
The auth_permissions table must be explicitly excluded with the -e flag as that table is populated by the migrate command and will already have data prior to loading fixtures.
This fixture would then be loaded in the same way as any other fixtures
The proper solution is to create the permissions in the same manner the framework itself does.
You should connect to the built-in post_migrate signal either in the module management.py or management/__init__.py and create the permissions there. The documentation does say that any work performed in response to the post_migrate signal should not perform any database schema alterations, but you should also note that the framework itself creates the permissions in response to this signal.
So I'd suggest that you take a look at the management module of the django.contrib.auth application to see how it's supposed to be done.
Just to add to #jonpa's comment, if you are using multitenant app and you want to directly save the fixtures to a file you can do:
python manage.py tenant_command dumpdata --schema=<schema_name> --natural-foreign --natural-primary -e auth.Permission --indent 4 > /path/to/fixtures/fixtures.json
Related
I am just starting with Django, and I would like to make an app that uses my existing sqlite db.
I read the docs and I found that you can create models from a db, using inspectdb; altho I can't find an example of how you use that command, on an existing db.
I copied the db file inside the directory of my project, ran the command and I see that a sqlite3 file is created in my directory project.
Altho the file has nothing to do with the database that I made. I tried to pass the db name to the inspectdb command but it says that it doesn't accept parameters.
So how can I actually tell the command to use my db to create the model for my app?
There must be some obvious step that I am missing...this is what I did:
-created the project
-created the app
-copied my db inside the project folder
-ran inspectdb
But I see the model empty, and a new db called db.sqlite3 created
Found the answer: there is a variable that has to be set, to define which one is the db that the application will use. the default is set to "db.sqlite3", which explain why I am getting this behavior.
Once you modify the name with the database that I already made, the command run without issues.
Not sure if it is just me getting stomped, but this info about the name that has to be changed was not mentioned anywhere...
Thanks
My django app works with tables that are not managed and have the following defined in my model like so:
class Meta:
managed = False
db_table = 'mytable'
When I run a simple test that imports the person, I get the following:
(person)bob#sh ~/person/dapi $ > python manage.py test
Creating test database for alias 'default'...
DatabaseError: (1060, "Duplicate column name 'db_Om_no'")
The tests.py is pretty simple like so:
import person.management.commands.dorecall
from person.models import Person
from django.test import TestCase
import pdb
class EmailSendTests(TestCase):
def test_send_email(self):
person = Person.objects.all()[0]
Command.send_email()
I did read in django docs where it says "For tests involving models with managed=False, it’s up to you to ensure the correct tables are created as part of the test setup.". So I understand that my problem is that I did not create the appropriate tables. So am I supposed to create a copy of the tables in the test_person db that the test framework created?
Everytime I run the tests, the test_person db gets destroyed (I think) and re-setup, so how am I supposed to create a copy of the tables in test_person. Am I thinking about this right?
Update:
I saw this question on SO and added the ManagedModelTestRunner() in utils.py. Though ManagedModelTestRunner() does get run (confirmed through inserting pbd.set_trace()), I still get the Duplicate column name error. I do not get errors when I do python manage.py syncdb (though this may not mean much since the tables are already created - will try removing the table and rerunning syncdb to see if I can get any clues).
I had the same issue, where I had an unmanaged legacy database that also had a custom database name set in the models meta property.
Running tests with a managed model test runner, as you linked to, solved half my problem, but I still had the problem of Django not knowing about the custom_db name:
django.db.utils.ProgrammingError: relation "custom_db" does not exist
The issue was that ./manage.py makemigrations still creates definitions of all models, managed or not, and includes your custom db names in the definition, which seems to blow up tests. By installing:
pip install django-test-without-migrations==0.2
and running tests like this:
./manage.py test --nomigrations
I was able to write tests against my unmanaged model without getting any errors.
I'm not 100% sure I'm doing this right, but I think I've found an issue where auth.Permission objects aren't being created soon enough for migrations to use them when you initialize a DB from scratch.
The important details:
I'm trying to initialize a Django DB from scratch using ./manage.py syncdb --migrate --noinput
I have 11 migrations in my chain
The 1st migration creates a new model called myapp.CompanyAccount
The 9th migration tries to fetch the permission myapp.change_companyaccount with:
p = orm[ "auth.Permission" ].objects.get( codename = "change_companyaccount" )
At that point, an exception is raised:
django.contrib.auth.models.DoesNotExist: Permission matching query does not exist
I had assumed that the default permissions that are defined for every object (as per http://docs.djangoproject.com/en/dev/topics/auth/#default-permissions) would have been created by the time the 1st migration finished, but it doesn't appear that they are. If I re-run the migration after the exception, it works the second time because apparently the permission now exists and the 9th migration can execute without error.
Is there anything that can be done to "flush" everything sometime before the 9th migration runs so that the whole thing can run in a single pass without bailing out?
Thanks for any help / advice.
EDIT: In response to John's comment below, I found out that the following command-line sequence will work:
./manage.py syncdb (this initializes the default Django tables)
./manage.py migrate myapp 0001 (this causes the CompanyAccount table to be created)
./manage.py migrate myapp (this migrates all the way to the end without error)
Unfortunately, skipping step #2 above means that you get the same exception in the 0009 migration, which tells me that my original suspicion was correct that default permissions on new models are not created by South immediately, and are somehow only pushed into the database when the entire migration chain finishes.
This is better than where I was (I'm at least avoiding exceptions now) but I still need to manually segment the migration around the creation of new models that latter migrations might need to touch the permissions of, so this isn't a complete solution.
As it turns out, the answer is to manually call db.send_pending_create_signals() at some point before you try to access the default permission since South only does this "flushing" step quite late in the process. Thanks to Andrew Godwin from South for replying to this on the South mailing list here:
http://groups.google.com/group/south-users/browse_thread/thread/1de2219fe4f35959
Don't you have to run the default "syncdb" on a virgin database in order to create the South migration table; before you can use south. Are you doing that? It typically creates the permissions table at that time since you have django.contrib.auth in your INSTALLED_APPS.
http://south.aeracode.org/docs/installation.html#configuring-your-django-installation
I'm testing a django app not written by myself, which uses two fixtures: initial_data.json and testing.json. Both fixtures files contain conflicting data (throwing an integrity error).
For testing, I've specified TestCase.fixtures = ['testing.json'], but initial_data.json is loaded too.
How can I avoid loading initial_data.json (not renaming it) in the testcase?
Quoting from Django Website:
If you create a fixture named
initial_data.[xml/yaml/json], that
fixture will be loaded every time you
run syncdb. This is extremely
convenient, but be careful: remember
that the data will be refreshed every
time you run syncdb. So don't use
initial_data for data you'll want to
edit.
So I guess there's no way to say "okay, don't load initial data just this once". Perhaps you could write a short bash script that would rename the file. Otherwise you'd have to dig into the Django code.
More info here: http://docs.djangoproject.com/en/dev/howto/initial-data/#automatically-loading-initial-data-fixtures
You might want to think about whether initial_data.json is something your app actually needs. It's not hard to "manually" load your production data with ./manage.py loaddata production.json after running a syncdb (how often do you run syncdb in production, anyway?), and it would make loading your testing fixture much easier.
If you want to have tables with no initial data, this code will help you:
edit tests.py:
from django.core import management
class FooTest(TestCase):
#classmethod
def setUpClass(cls):
management.call_command('flush', interactive=False, load_initial_data=False)
this will remove your data and syncdb again without loading initial data.
Every now and then, you have the need to rename a model in Django (or, in one recent case I encountered, split one model into two, with new/different names). (Yes, proper planning helps to avoid this situation).
After renaming corresponding tables in the db and fixing affected code, one problem remains: Any permissions granted to Users or Groups to operate on those models still references the old model names. Is there any automated or semi-automated way to fix this, or is it just a matter of manual db surgery? (in development you can drop the auth_permissions table and syncdb to recreate it, but production isn't so simple).
Here's a snippet that fills in missing contenttypes and permissions. I wonder if it could be extended to at least do some of the donkey work for cleaning up auth_permissions.
If you happened to have used a South schema migration to rename the table, the following line in the forward migration would have done this automatically:
db.send_create_signal('appname', ['modelname'])
I got about half-way through a long answer that detailed the plan of attack I would take in this situation, but as I was writing I realized there probably isn't any way around having to do a maintenance downtime in this situation.
You can minimize the downtime by having a prepared loaddata script of course, although care needs to be taken to make sure the auth_perms primary keys are in sync.
Also see short answer: no automated way to do this of which I'm aware.
I recently had this issue and wrote a function to solve it. You'll typically have a discrepancy with both the ContentType and Permission tables if you rename a model/table. Django has built-in helper functions to resolve the issue and you can use them as follow:
from django.contrib.auth.management import create_permissions
from django.contrib.contenttypes.management import update_all_contenttypes
from django.db.models import get_apps
def update_all_content_types_and_permissions():
for app in get_apps():
create_permissions(app, None, 2)
update_all_contenttypes()
I changed verbose names in my application, and in Django 2.2.7 this is the only way I found to fix permissions:
from django.core.management.base import BaseCommand, CommandError
from django.contrib.auth.models import Permission
class Command(BaseCommand):
help = 'Fixes permissions names'
def handle(self, *args, **options):
for p in Permission.objects.filter(content_type__app_label="your_app_label_here"):
p.name = "Can %s %s"%(p.codename.split('_')[0], p.content_type.model_class()._meta.verbose_name)
p.save()