I am writing a migration that requires me to fill a field with existing data from another field (with same type and constraints). Is it possible in Django to copy the data in a single operation? The destination column already exists when I need to copy the data.
In SQL, I would have written something like that:
UPDATE my_table SET column_b = column_a;
Edit
The current answer proposes to loop over the model instances, but that is what I want to avoid. Can it be done without a loop?
As the comment mentioned, you can simply write a migration for this. I think the below should work though I haven't tested it. It uses the queryset update API and F to avoid looping
from __future__ import unicode_literals
from django.apps import apps
from django.db import migrations, models
from django.db.models import F
def copy_field(apps, schema):
MyModel = apps.get_model('<your app>', 'MyModel')
MyModel.objects.all().update(column_a=F('column_b'))
class Migration(migrations.Migration):
dependencies = [
('<your app>', '<previous migration>'),
]
operations = [
migrations.RunPython(code=copy_field),
]
Related
My app that currently depends on Postgres and Django's Postgres-only JSONField. The field works well and I've no interest in another project, but I have prospective-users who want to use my app, but can't while it relies on Postgres.
Django 3.1 has a cross-platform version of this field —which will work for my needs— but I don't want to force everybody up to Django 3.1; I would like to offer people a choice between Postgres or Django 3.1+. On paper, this is simple enough with a conditional import...
try:
from django.db.models import JSONField
except ImportError:
from django.contrib.postgres.fields import JSONField
And if I installed Django 3.1 and generated a migration, it could take me from django.contrib.postgres.fields.JSONField to django.db.models.JSONField. But...
New users will still execute the initial migration. I will still have a dependency on Postgres.
Sub-Django 3.1 users won't be able to execute the new migration. I now have a dependency on Django 3.1.
This is worse than when I started. How do I do this sort of field migration in a way that will work for everybody?
Migrations are just code. Just because they're auto-generated doesn't mean you shouldn't change them. You're encouraged to, at least to check they're generated correctly, but also there's no harm in writing them yourself.
This works for me:
Model:
from django.db import models
try:
from django.db.models import JSONField
except ImportError:
from django.contrib.postgres.fields import JSONField
class MyModel(models.Model):
stuff = JSONField()
Migration:
from django.db import migrations, models
try:
from django.db.models import JSONField
except ImportError:
from django.contrib.postgres.fields import JSONField
class Migration(migrations.Migration):
dependencies = [('testapp', '0001_initial')]
operations = [
migrations.CreateModel(
name='MyModel',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('stuff', JSONField()),
],
),
]
Keep in mind that if you need to change this field in the future, you will need to go through this process again.
I have got this from Django source code
from django.db.models import JSONField as BuiltinJSONField
class JSONField(BuiltinJSONField):
system_check_deprecated_details = {
'msg': (
'django.contrib.postgres.fields.JSONField is deprecated. Support '
'for it (except in historical migrations) will be removed in '
'Django 4.0.'
),
'hint': 'Use django.db.models.JSONField instead.',
'id': 'fields.W904',
}
This indicates that, django.contrib.postgres.fields.JSONField is going to be deprecated. Also, Django uses the django.db.models.JSONField as postgres special JSONField.
Apart from that, I have generated the SQL command using sqlmigrate command and it was like,
BEGIN;
--
-- Create model MyModel
--
CREATE TABLE "myapp_mymodel" ("id" serial NOT NULL PRIMARY KEY, "stuff" jsonb NOT NULL);
COMMIT;
Surprisingly, I have got same SQL command using Django==3.0 and Django==3.1 and in the database, the field is a jsonb type
These pieces of information conclude that you don't have to worry about this new JSONField upgrade.
What changes need to be done?
You don't need to generate a new migration file, but edit existing migration files which have django.contrib.postgres.fields.JSONField reference with the try...except block.
That's it!!!
All new migrations will be correct automatically if this solution with a deconstruct() method will be used.
You can create a compatible custom JSONField that encapsulates both variants.
Create a small file fields.py in your application:
try:
from django.db.models import JSONField as OrigJSONField
except ImportError:
from django.contrib.postgres.fields import JSONField as OrigJSONField
class JSONField(OrigJSONField):
def deconstruct(self):
# the original path was 'django.db.models.JSONField' or 'django.contrib.postgres.fields....'
name, path, args, kwargs = super().deconstruct()
# Substitute 'my_app' by your application name everywhere.
path = 'my_app.fields.JSONField'
return name, path, args, kwargs
Change a line in your models.py:
from my_app.fields import JSONField
Edit all existing migrations that use a JSONField:
import my_app.fields
...
('stuff', my_app.fields.JSONField()),
(or equivalently)
from my_app.fields import JSONField
...
('stuff', JSONField()).
No migration is created after that because no difference is found by makemigrations.
All future makemigrations after a changed model will be created automatically with my_app.fields.JSONField(). Compatibility is a benefit of this solution.
Reflection about Django Release Notes 3.1 and future:
They describe a plain upgrade that requires to create a new migration that only upgrades the import path, but it generates no SQL e.g. by sqlmigrate. It is easier than to edit old migrations.
Maybe you you also upgrade in two years to Django >= 3.1 only and you will have two alternatives:
A) Only the import in models.py will be edited and you create a new nearly empty formal migration similarly to release notes. You will be not able to remove the import path my_app.fields.JSONField because it must be importable from old migrations similarly that Django can never remove the path django.contrib.postgres.fields.JSONField. The file my_app/fields.py could be simplified to one line from django.db.models import JSONField as JSONField after an unconditional upgrade of requirements.
B) Editing old migrations again to only the new JSONField is possible, but unreasonable.
(It is every user's responsibility that an edit in a migrations file doesn't require a changed database state and all migrations remain consistent each other. That is why not much about it can be found, except such clear cases.)
I gave point to Tom Carrick for his answer but I think it should also include the usage of Meta.required_db_vendor = 'postgres' on Django < 3.1
# models.py
from django.db import models
try:
from django.db.models import JSONField
postgres_only = False
except ImportError:
from django.contrib.postgres.fields import JSONField
postgres_only = True
class MyModel(models.Model):
stuff = JSONField()
class Meta:
if postgres_only:
required_db_vendor = 'postgres'
# migrations/0001_initial.py
from django.db import migrations, models
try:
from django.db.models import JSONField
postgres_only = False
except ImportError:
from django.contrib.postgres.fields import JSONField
postgres_only = True
class Migration(migrations.Migration):
dependencies = [('testapp', '0001_initial')]
operations = [
migrations.CreateModel(
name='MyModel',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('stuff', JSONField()),
],
options={'required_db_vendor': 'postgres'} if postgres_only else None,
),
]
This will ensure that users of your reusable app cannot use it on a different database than PostgreSQL on Django < 3.1 and seamlessly enable support on 3.1 without requiring a migration on PostgreSQL users using your library and updating to Django 3.1.
I want my application to have default data such as user types.
What's the most efficient way to manage default data after migrations?
It needs to handle situations such as, after I add a new table, it adds the default data for it.
You need to create an empty migration file and Do your stuff in operations block, as explained in docs.
Data Migrations
As well as changing the database schema, you can also use migrations to change the data in the database itself, in conjunction with the schema if you want.
Now, all you need to do is create a new function and have RunPython use it
Docs explains this with an example to show ,how to communicate with your models.
From Docs
To create an empty migration file,
python manage.py makemigrations --empty yourappname
And this is the example how to update a newly added field.
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations, models
def combine_names(apps, schema_editor):
# We can't import the Person model directly as it may be a newer
# version than this migration expects. We use the historical version.
Person = apps.get_model("yourappname", "Person")
for person in Person.objects.all():
person.name = "%s %s" % (person.first_name, person.last_name)
person.save()
class Migration(migrations.Migration):
initial = True
dependencies = [
('yourappname', '0001_initial'),
]
operations = [
migrations.RunPython(combine_names),
]
The accepted answer is fine. But, since OP asked the question in the context of adding new rows and not updating existing entries. Here is the code snippet for adding new entries :
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('loginmodule', '0002_login_avatar'),
]
def insertData(apps, schema_editor):
Login = apps.get_model('loginmodule', 'Login')
user = Login(name = "admin", login_id = "admin", password = "password", email = "admin#pychat.com", type = "Admin", avatar="admin.jpg")
user.save()
operations = [
migrations.RunPython(insertData),
]
Update:
most users are looking for data migration as suggested by #durdenk in https://stackoverflow.com/a/39742847/3627387. But what OP was asking is about a way to add data after migrations, that is why this is accepted answer.
Original answer:
I think what you are looking for is fixtures https://docs.djangoproject.com/en/1.10/howto/initial-data/
From docs
It’s sometimes useful to pre-populate your database with hard-coded data when you’re first setting up an app. You can provide initial data via fixtures.
Also read this https://code.djangoproject.com/wiki/Fixtures
Answer is given above just to show how to insert new rows to the table.
from django.db import migrations, models
from yourapp.models import <yourmodel>
def combine_names(apps, schema_editor):
obj = <yourmodel>(arrib=value)
obj.save()
For example let's say you have model Person
person = Person(first_name='raj', last_name='shah')
person.save()
I'm trying to rename a model and I would like to write the migration in the way that it doesn't depend on the old name still present while it being applied. Can I somehow get data from a database table that no longer has a model in my migration code?
Details:
I have a Region model that I want to move into a more generic GeoObject model and remove from the models.py. If I write my migration code that creates GeoObjects from existing Regions with from models import Region I'll have to keep Region model until my main database will migrate. But I'd like to write a migration so that it doesn't depend on Region model being present, just check that the database table exists and use it. Is it possible to do it using Django instruments, without depending on a specific database type if possible?
Yes, you can.
But first of all, you really shouldn't import any model inside migration.
Take look at RunPython operation, that will allow you to run any python code inside your migration. RunPython will pass to your function 2 parameters: apps and schema_editor. First parameter contains structure of your models at stage of applying that migration, so if actual removing of model is later on that migration, you can still access that model using apps passed into function.
Let's say your model looked like this:
class SomeModel(models.Model):
some_field = models.CharField(max_length=32)
Now you're deleting that model, automatically created migration will contain:
class Migration(migrations.Migration):
dependencies = [
('yourapp', '0001_initial'), # or any other dependencies
]
operations = [
migrations.DeleteModel(
name='Main',
),
]
You can modify that migration by injecting RunPython just above DeleteModel operation:
operations = [
migrations.RunPython(
move_data_to_other_model,
move_data_back, # for backwards migration - if you won't ever want to undo this migration, just don't pass that function at all
),
migrations.DeleteModel(
name='SomeModel',
),
]
and creating 2 functions before Migration class:
def move_data_to_other_model(apps, schema_editor):
SomeModel = apps.get_model('yourapp', 'SomeModel')
for something in SomeModel.objects.all():
# do your data migration here
o = OtherModel.objects.get(condition=True)
o.other_field = something.some_field
def move_data_back(apps, schema_editor):
SomeModel = apps.get_model('yourapp', 'SomeModel')
for something in OtherModel.objects.all():
# move back your data here
SomeModel(
some_field=something.other_field,
).save()
It doesn't matter that your model is no longer defined in models.py, django can rebuild that model based on migration history. But remember: save method from your models (and other customized methods) won't be called in migrations. Also any pre_save or post_save signals won't be triggered.
IMPORTANT: This question is no longer relevant.
In a Django 1.7 migration I try to create Comment entries programatically with the following code:
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
class Migration(migrations.Migration):
def create_genericcomment_from_bookingcomment(apps, schema_editor):
BookingComment = apps.get_model('booking', 'BookingComment')
Comment = apps.get_model('django_comments', 'Comment')
for comment in BookingComment.objects.all():
new = Comment(content_object=comment.booking)
new.save()
dependencies = [
('comments', '0001_initial'),
('django_comments', '__first__'),
]
operations = [
migrations.RunPython(create_genericcomment_from_bookingcomment),
]
And it produces an error:
TypeError: 'content_object' is an invalid keyword argument for this function
However, the same code (i.e. Comment(content_object=comment.booking)) works when executed in the shell.
I tried to create a blank model with new = Comment() and then set all the necessary fields manually but even though I set content_type and object_pk fields accordingly, they content_type was not actually saved and I received django.db.utils.IntegrityError: null value in column "content_type_id" violates not-null constraint
Any idea how to properly create a model with a generic foreign key in a migration? Or any workaround?
This is an issue of migrations' model loader. You load your models using default
Comment = apps.get_model('django_comments', 'Comment')
It loads the Comment model in some special way, so some features like generic relations don't work.
There is a bit hacky solution: load your models as usual:
from django_comments import Comment
I want to create groups in django programmatically, but not in a view, but rather in something like model (for example using migrations). How to do it? There's no information about it in google and docs (at least not here: https://docs.djangoproject.com/en/1.7/topics/auth/default/#groups)
Okay, it seems you're using Django 1.7's new migrations system. This is similar to but not exactly like South.
A migration that involves altering the data in the tables is a data migration, and you typically need to write Python code to do the migration.
From the Django docs, there's this example:
# -*- coding: utf-8 -*-
from django.db import models, migrations
def combine_names(apps, schema_editor):
# We can't import the Person model directly as it may be a newer
# version than this migration expects. We use the historical version.
Person = apps.get_model("yourappname", "Person")
for person in Person.objects.all():
person.name = "%s %s" % (person.first_name, person.last_name)
person.save()
class Migration(migrations.Migration):
dependencies = [
('yourappname', '0001_initial'),
]
operations = [
migrations.RunPython(combine_names),
]
Note that the code to run during the migration is in the combine_names function, which is called by the migrations.RunPython(combine_names) entry in the operations list of the migration. Your migration should do its group creation in a function like that, along with whatever other data migration is needed.
You should probably use a line like
Group = apps.get_model("auth", "Group")
my_group, created = Group.objects.get_or_create(name='group1')
to create your groups, in case there is already a group of that name in the table.
Don't put code to run during a migration into the root level of the Python file; if you do so, it will be run every time that migration is imported, for example, every time you run ./manage.py runserver.
P.S. You need to put your migrations.RunPython entry at the right point in the operations list; it won't work if you put it after an operation that deletes a table it needs, for example.
Groups are just like any other Django model. You can create them as you would anything else.
my_group = Group.objects.create(name='group1')