I ran into a difficult problem related to Django migrations and searching for answer
I created few database models and inserted data on those models by using migrations file
Example Migrations 0001_users.py
Example Model :
migrations.CreateModel(
name='User',
fields=[
('user_id', models.CharField(max_length=256)),
('name', models.CharField(max_length=256)),
],
)
Example Migrations 0002_users_data.py
Example Data:
user = User(user_id="DEV", name="dev")
user.save()
user = User(user_id="STAGE", name="stage")
user.save()
Now i add an ForeignKey in above table say address with some default value in Migrations file 0003_updated_column.py and its working fine locally,as i have already created a table, inserted data and now modifying old data with new column
But, If any new user want to pull as a fresh repo, he is getting issues in running migrations as my first,second migrations file will have hard-coded users table data with out newly added ForeignKey on it but models file will have newly added ForeignKey as well
Please help me on this
Related
In Django, I am trying to move a custom User model from one app to another. When I follow instructions such as found here, and try to apply the migration, I get errors with ValueError: Related model 'newapp.user' cannot be resolved that originate from django.contrib.admin.models.LogEntry.user, which is defined as models.ForeignKey(settings.AUTH_USER_MODEL, ...), so that is a model from the Django admin (which I also use) that has a foreign key to the User model. How can I do this migration?
For swappable models (models that can be swapped out through a value in settings) such a move is non-trivial. The root cause of the problem here is that LogEntry has a foreign key to settings.AUTH_USER_MODEL, but the history of settings.AUTH_USER_MODEL itself is not managed or known to Django migrations. When you change AUTH_USER_MODEL to point to the new User model, that retroactively changes the migration history as Django sees it. To Django it now looks like LogEntry's foreign key has always referenced the new User model in destapp. When you run the migration that creates the table for LogEntry (e.g. when re-initializing the database or running tests), Django cannot resolve the model and fails.
See also this issue and the comments there.
To work around this problem, AUTH_USER_MODEL needs to point to a model that exists in the initial migration for the new app. There are a few approaches to get this working. I'll assume the User model is moved from sourceapp to destapp.
Move the migration definition for the new User model from the last migration (where Django makemigrations would automatically put it) to the initial migration of destapp. Leave it wrapped in a SeparateDatabaseAndState state operation, because the database table is already created by the migrations in the sourceapp. Then, you'll need to add a dependency from the initial migration of destapp to the last migration of sourceapp. The problem is that if you try to apply the migrations as they are now, it will fail because destapp's initial migration has already been applied while it's dependency (sourceapp's last migration) has not been. So you will need to apply the migrations in sourceapp before adding the above migrations in destapp. In the gap between applying sourceapp and destapp's migrations the User model won't exist so your application will be temporarily broken.
Apart from temporarily breaking the application, this has the other problem that now destapp will depend on sourceapp's migrations. If you can do that, that's fine, but if there already exists a dependency from a sourceapp migration to a destapp migration this won't work and you've now created a circular dependency. If that's the case, look at the next options.
Forget about the User migration history. Just define the User class in destapp's initial migration, without the SeparateDatabaseAndState wrapper. Make sure you have CreateModel(..., options={'db_table': 'sourceapp_user'}, ...), so the database table will be created the same as it would when User lived in sourceapp. Then edit sourceapp's migration(s) where User is defined, and remove those definitions. After that, you can create a regular migration where you remove User's db_table setting so the database table gets renamed to what it should be for destapp.
This only works if there are no or minimal migrations in the migration history of sourceapp.User. Django now thinks User always lived in destapp, but its table was named as sourceapp_user. Django cannot track any database-level changes to sourceapp_user anymore since that information was removed.
If this works for you, you can either forego any dependencies between sourceapp and destapp, if sourceapp's migrations don't need User to be there, or have sourceapp's initial migration depend on destapp's initial migration so that the table for User is created before sourceapp's migrations are run.
If both don't work in your situation, another option is to add the definition for User to sourceapp's initial migration (without SeparateDatabaseAndState wrapper), but have it use a dummy table name (options={'db_table': 'destapp_dummy_user'}). Then, in the newest migration where you actually want to move User from sourceapp to destapp, do
migrations.SeparateDatabaseAndState(database_operations=[
migrations.DeleteModel(
name='User',
),
], state_operations=[
migrations.AlterModelTable('User', 'destapp_user'),
])
This will delete the dummy table in the database, and point the User model to the new table. The new migration in sourceapp should then contain
migrations.SeparateDatabaseAndState(state_operations=[
migrations.DeleteModel(
name='User',
),
], database_operations=[
migrations.AlterModelTable('User', 'destapp_user'),
])
so it is effectively the mirror image of the operation in the last destapp migration. Now only the last migration in destapp needs to depend on the last migration in sourceapp.
This approach appears to work, but it has one big disadvantage. The deletion of destapp.User's dummy database table also deletes all foreign key constraints to that table (at least on Postgres). So LogEntry now no longer has a foreign key constraint to User. The new table for User doesn't recreate those. You will have to add the missing constraints back in again manually. Either by manually updating the database or by writing a raw sql migration.
Update content type
After applying one of the three above options there's still one loose end. Django registers every model in the django_content_type table. That table contains a line for sourceapp.User. Without intervention that line will stay there as a stale row. That isn't a big problem as Django will automatically register the new destapp.User model. But it can be cleaned up by adding the following migration to rename the existing content type registration to destapp:
from django.db import migrations
# If User previously existed in sourceapp, we want to switch the content type object. If not, this will do nothing.
def change_user_type(apps, schema_editor):
ContentType = apps.get_model("contenttypes", "ContentType")
ContentType.objects.filter(app_label="sourceapp", model="user").update(
app_label="destapp"
)
class Migration(migrations.Migration):
dependencies = [
("destapp", "00xx_previous_migration_here"),
]
operations = [
# No need to do anything on reversal
migrations.RunPython(change_user_type, reverse_code=lambda a, s: None),
]
This function only works if there is no entry in django_content_type for destapp.User yet. If there is, you'll need a smarter function:
from django.db import migrations, IntegrityError
from django.db.transaction import atomic
def change_user_type(apps, schema_editor):
ContentType = apps.get_model("contenttypes", "ContentType")
ct = ContentType.objects.get(app_label="sourceapp", model="user")
with atomic():
try:
ct.app_label="destapp"
ct.save()
return
except IntegrityError:
pass
ct.delete()
How can i show model historical changes and show them and can modify?
for example
class MM(models.Model):
chnge1=model.ForeignKey('self')
changeText = models.TextField(max_length=200)
If you are using Django to manage your models you can always look through the migration files to see what has changed. The migration files are designed to keep track of changes to your models so that Django can apply those changes to the database.
See more here: Django Migrations
A typical migration file might look like this:
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [('migrations', '0001_initial')]
operations = [
migrations.DeleteModel('Tribble'),
migrations.AddField('Author', 'rating', models.IntegerField(default=0)),
]
You can see the 'Tribble' model was delete and a 'rating' field was added to the 'Author' model.
If you have many migrations for a single app and don't want to go through each migration file to see the changes try "squashing" your migration files into (usually) a single file and you can go through all the changes from there.
Squashing Migrations: Squashing Migrations
I am migrating to django-oscar2.1
I have forked catalogue app, it has some existing migrations. Newly added field is_public is in migration #10. I have a custom migration #5 which is using create_from_breadcrumbs method. So when I tried to run migrate on fresh db migration#5 trigger error is_public field not exists in db.
create_from_breadcrumbs
Oscar loads a model with latest state which contains is_public in models but not in db.
In migration #5 I am loading model like this
Category = apps.get_model("catalogue", "Category")
You cannot use create_from_breadcrumbs in a migration like that. The reason for this is explained in Django's documentation on migrations - that the version of the model used in external code may be different from the one that this migration expects - which is why it fails.
If you want to create model objects as part of your migration you need to use the historic version of the model in the migration. This means defining your own version of create_from_breadcrumbs, and using the historic version of the model as suggested by the Django documentation:
def create_from_breadcrumbs_for_migration(apps, schema_editor):
Category = apps.get_model('catalogue', 'Category')
# Perform logic to create categories here.
class Migration(migrations.Migration):
dependencies = [
('catalogue', '0004_xxxx'),
]
operations = [
migrations.RunPython(create_from_breadcrumbs_for_migration),
]
That said, I would argue that migrations are not meant for this sort of data population in the first place. You would in my view be better off writing a separate management command for this, which you run after the migrations have been applied.
I added a field to one of my models, but in the 'models' folder I have two other python files which have only View models from which I query views in my database. When I run the makemigrations command, the new migrations file that is created includes also adding these view models to my database as tables (which I don't want to). How can I ignore these changes, and only commit the one addition of a field to an actual table on the database.
I think I maybe have to delete the migrations.CreateModel... in the new migrations file and only keep the migrations.addField... , then run the 'migrate' command. I didn't proceed with this because I'm not sure and maybe it will mess up my database in some way.
Thanks in advance to anyone who can help.
when you make a model for database view you must add meta class managed = false and db_table like this:
class MyViewModel(models.Model):
field: models.CharField(max_length=100)
class Meta:
managed = False
db_table = 'database_view_name'
when you write this and run makemigrations a migration generated contains this model but when you run migrate this migration doesnt change anything on database.
you also can create view using migrations in python. see migrations.RunPython for more details
I have an existing django-cms ver. 3.1.3 project where I would like to replace the default django User model with the one found in django-custom-user (to have email as user name for my users). I have added the custom_user app to my INSTALLED_APPS and set AUTH_USER_MODEL = 'custom_user.EmailUser'. Finally I applied the migrations.
Everything seems to work fine in my custom models, but the django-cms models that have a reference to the auth_user table (GlobalPagePermission, PagePermission, PageUser and UserSettings) are not updated to have a foreign-key reference to the new custom user table.
The django-cms documentation says that is generally advisable to add custom user models in the beginning of a project, but here I am, in the middle of a project, and would very much like to avoid having to delete my cms models (with data) to make it work.
If I were in the beginning of a project, and had added the custom user model before migrating the django-cms models, would the django-cms models actually get a reference to the custom user model table instead of the default one?
Is there any way for me to make migrations for the django-cms models, so they use the new custom user model instead of the default one?
UPDATE
Im trying to implement what yakky suggested:
from __future__ import unicode_literals
from django.conf import settings
from django.db import models, migrations
import django.db.models.deletion
from django.utils.translation import ugettext_lazy as _
def up(apps, schema_editor):
PageUser = apps.get_model("cms", "PageUser")
db_alias = schema_editor.connection.alias
for pu in PageUser.objects.all():
pu['emailuser_ptr_id'] = pu['user_ptr_id']
pu.save()
def down(apps, schema_editor):
PageUser = apps.get_model("cms", "PageUser")
db_alias = schema_editor.connection.alias
for pu in PageUser.objects.all():
pu['user_ptr_id'] = pu['emailuser_ptr_id']
pu.save()
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
(‘myapp’, ‘previous-migration’),
]
operations = [
migrations.RenameField(
model_name='PageUser',
old_name='user_ptr_id',
new_name='user_ptr_id_old',
),
migrations.AddField(
model_name='PageUser',
name='emailuser_ptr_id',
field=models.ForeignKey(null=True, to=settings.AUTH_USER_MODEL, verbose_name=_('user'), blank=True),
preserve_default=True,
),
migrations.RunPython(up, down),
migrations.RemoveField(
model_name='PageUser',
name='user_ptr_id_old',
),
]
It fails with KeyError: ('myapp', u'pageuser') suggesting that it looks for the PageUser model in the my custom app and not in the cms app. How do I apply these migrations to the cms models?
Being said that replacing the Django User model it's not advisable in the middle of a project lifetime (see https://docs.djangoproject.com/en/1.9/topics/auth/customizing/#substituting-a-custom-user-model) as Django does not provide any way to modify the relations after their are being created, to replace the custom user model you can write a custom migration that alter the foreignkey in the django CMS models to point to theand create a new one to the new user model.
Something like:
...
migrations.RenameField(
model_name='globalpagepermission',
old_name='user',
new_name='user_old',
),
migrations.AddField(
model_name='globalpagepermission',
name='user',
field=models.ForeignKey(null=True, to=settings.AUTH_USER_MODEL, verbose_name=_('user'), blank=True),
preserve_default=True,
),
migrations.RunPython(copy_data),
migrations.RemoveField(
model_name='globalpagepermission',
name='user_old',
),
...
Repeat for any relevant model / field.
copy_data method will copy the value for the foreign keys from user_old to user
Code is completely untested but the general idea should work.
Alternatively you can write a RunSQL migration to get the same result.
Users with the same primary keys must exist also for the new model for this to work.
Thanks to https://code.djangoproject.com/ticket/25313 I managed to make it work. This is what I did:
Backed up my database!
Created a new app email_username to hold my user model (named User as recommended in the link). A new app was necessary because my main app (myapp) is dependent on django-cms, and if I put my new user model in myapp, django-cms would become dependent on my app - this would create a circular reference.
class User(AbstractEmailUser):
# Needed to add username, first_name and last_name here, because cms.PageUser model depends on these fields
username = models.CharField(max_length=100, verbose_name=_('username'))
first_name = models.CharField(max_length=100, verbose_name=_('first_name'))
last_name = models.CharField(max_length=100, verbose_name=_('last_name'))
class Meta:
db_table = 'auth_user' # the model will use the existing auth_user table (at first)
Set AUTH_USER_MODEL = ‘email_username.User’ in settings.py
Deleted all migrations of myapp and truncated the django_migrations table
Ran manage.py makemigrations and got
Cannot resolve bases for [] This can
happen if you are inheriting models from an app with migrations (e.g.
contrib.auth) in an app with no migrations; see
https://docs.djangoproject.com/en/1.8/topics/migrations/#dependencies
for more
... so I removed the cms app from settings.py, ran makemigrations again which now suceeded, then re-added and ran makemigrations again.
Ran manage.py migrate --fake, but got
Cannot resolve bases for [ModelState: 'djangocms_file.File', ModelState: 'djangocms_video.Video', ModelState: 'djangocms_link.Link', ModelState: 'djangocms_googlemap.GoogleMap', ModelState: 'djangocms_picture.Picture', ModelState: 'djangocms_teaser.Teaser']
... so I removed these apps as well from settings.py, ran migrate --fake, re-added the apps and migrate --fake again.
Removed the db_table setting from the new User model, ran makemigrations followed by migrate (without fake!)
Everything now seems to be in order. All foreign keys that previously referenced the standard auth_user table is now referincing the table of my new user model. Existing users have even been transferred because of the db_table trick.