I have created a model in Django.
class MyModel(models.Model):
features = TextField(blank=True, default='')
There are several possible ways to store the data in the feature field. Some examples below.
feature1;feature2
feature1, feature2
feature1,feature2
And so on. I need to create a GIN index for that field. I would probably do it in postgreSQL in the following way
CREATE INDEX features_search_idx ON "mymodel" USING gin (regexp_split_to_array("mymodel"."features", E'[,;\\s]+'));
Would it be possible to do the same thing by a migration?
Yes.
Create an empty migration: python manage.py makemigration yourapp --empty -n pour_gin
Add a migrations.RunSQL() operation in the migration file.
class Migration(migrations.Migration):
dependencies = [
# ...
]
operations = [
migrations.RunSQL(
sql="""CREATE INDEX features_search_idx ON "mymodel" USING gin (regexp_split_to_array("mymodel"."features", E'[,;\\s]+'));""",
reverse_sql=migrations.RunSQL.noop, # TODO: replace me with DROP INDEX
),
]
Related
I have this code and I want it to just create the groups every time the program runs so that if the database is deleted it will still be a sufficient program itself and someone won't have to create groups again, do you know an easy way to do this?
system_administrator = Group.objects.get_or_create(name='system_administrator')
manager = Group.objects.get_or_create(name='manager')
travel_advisor = Group.objects.get_or_create(name='travel_advisor')
If you lose your DB, you'd have to rerun migrations on a fresh db before the program could run again. So I think data migrations might be a good solution for this? A data migration, is a migration that runs python code to alter the data in the DB, not the schema as a normal migration does.
You could do something like this:
In a new migration file (you can run python manage.py makemigrations --empty yourappname to create an empty migration file for an app)
def generate_groups(apps, schema_editor):
Group = apps.get_model('yourappname', 'Group')
Group.objects.get_or_create(name="system_administrator")
Group.objects.get_or_create(name="manager")
Group.objects.get_or_create(name="travel_advisor")
class Migration(migrations.Migration):
dependencies = [
('yourappname', 'previous migration'),
]
operations = [
migrations.RunPython(generate_groups),
]
Worth reading the docs on this https://docs.djangoproject.com/en/3.0/topics/migrations/#data-migrations
You can do it in the ready method of one of your apps.
class YourApp(Appconfig):
def ready(self):
# important do the import inside the method
from something import Group
Group.objects.get_or_create(name='system_administrator')
Group.objects.get_or_create(name='manager')
Group.objects.get_or_create(name='travel_advisor')
The problem with the data migrations approach is that it is useful for populate the database the first time. But if the groups are deleted once the data migration has run, you will need to populate them again.
Also remember that get_or_create return a tuple.
group, created = Group.objects.get_or_create(name='manager')
# group if an instance of Group
# created is a boolean
I have a data migration as below in which I want to use create_user method of CustomUser, get an instance of the created user, and use this instance to create instance of Partner model.
It is worth mentioning that I have a Partner model that has a one-to-one relationship with CustomUser.
I have two options:
# Option One:
def populate_database_create_partner(apps, schema_editor):
Partner = apps.get_model('partners', 'Partner')
CustomUser.objects.create_user(
id=33,
email='test_email#email.com',
password='password',
first_name='test_first_name',
last_name="test_last_name",
is_partner=True,
)
u = CustomUser.objects.get(id=33)
partner = Partner.objects.create(user=u, )
class Migration(migrations.Migration):
dependencies = [
('accounts', '0006_populate_database_createsuperuser'),
]
operations = [
migrations.RunPython(populate_database_create_partner),
]
In option one, I see this error:
ValueError: Cannot assign "<CustomUser: test_email#email.com>": "Partner.user" must be a "CustomUser" instance.
I then test this:
# Option Two:
def populate_database_create_partner(apps, schema_editor):
Partner = apps.get_model('partners', 'Partner')
CustomUser = apps.get_model('accounts', 'CustomUser')
CustomUser.objects.create_user(
id=33,
email='test_email#email.com',
password='password',
first_name='test_first_name',
last_name="test_last_name",
is_partner=True,
)
u = CustomUser.objects.get(id=33)
partner = Partner.objects.create(user=u, )
class Migration(migrations.Migration):
dependencies = [
('accounts', '0006_populate_database_createsuperuser'),
]
operations = [
migrations.RunPython(populate_database_create_partner),
]
I the see this error:
CustomUser.objects.create_user(
AttributeError: 'Manager' object has no attribute 'create_user'
The create_user method does not work.
If I do not use the create_user method and simply use CustomUser.objects.create(...), I will not be able to set password in here.
Django only keeps limited historical information about each version of your models. One of the things it doesn't keep track of, as documented here, is custom model managers.
The good news is that there's a way to force the migrations system to use your custom manager:
You can optionally serialize managers into migrations and have them available in RunPython operations. This is done by defining a use_in_migrations attribute on the manager class.
As noted, this just allows your migration to use the version of the manager that exists when the migration is run; so, if you later make changes to it, you could break the migration. A safer alternative is to just copy the relevant create_user code into the migration itself.
I am adding a new field to an existing db table. it is to be auto-generated with strings.
Here is my code:
from django.utils.crypto import get_random_string
...
Model:
verification_token = models.CharField(max_length=60, null=False, blank=False, default=get_random_string)
I generate my migration file with ./manage.py makemigrations and a file is generated.
I verify the new file has default set to field=models.CharField(default=django.utils.crypto.get_random_string, max_length=60)
so all seems fine.
Proceed with ./manage.py migrate it goes with no error from terminal.
However when i check my table i see all the token fields are filled with identical values.
Is this something i am doing wrong?
How can i fix this within migrations?
When a new column is added to a table, and the column is NOT NULL, each entry in the column must be filled with a valid value during the creation of the column. Django does this by adding a DEFAULT clause to the column definition. Since this is a single default value for the whole column, your function will only be called once.
You can populate the column with unique values using a data migration. The procedure for a slightly different use-case is described in the documentation, but the basics of the data migrations are as follows:
from django.db import migrations, models
from django.utils.crypto import get_random_string
def generate_verification_token(apps, schema_editor):
MyModel = apps.get_model('myapp', 'MyModel')
for row in MyModel.objects.all():
row.verification_token = get_random_string()
row.save()
class Migration(migrations.Migration):
dependencies = [
('myapp', '0004_add_verification_token_field'),
]
operations = [
# omit reverse_code=... if you don't want the migration to be reversible.
migrations.RunPython(generate_verification_token, reverse_code=migrations.RunPython.noop),
]
Just add this in a new migration file, change the apps.get_model() call and change the dependencies to point to the previous migration in the app.
It maybe the token string to sort, so django will save some duplicates values. But, i'm not sure it is your main problem.
Anyway, I suggest you to handle duplicates values using while, then filter your model by generated token, makesure that token isn't used yet. I'll give you exampe such as below..
from django.utils.crypto import get_random_string
def generate_token():
token = get_random_string()
number = 2
while YourModel.objects.filter(verification_token=token).exists():
token = '%s-%d' % (token, number)
number += 1
return token
in your field of verification_token;
verification_token = models.CharField(max_length=60, unique=True, default=generate_token)
I also suggest you to using unique=True to handle duplicated values.
I'm looking to add a multi-column index to a postgres database. I have a non blocking SQL command to do this which looks like this:
CREATE INDEX CONCURRENTLY shop_product_fields_index ON shop_product (id, ...);
When I add db_index to my model and run the migration, will it also run concurrently or will it block writes? Is a concurrent migration possible in django?
There are AddIndexConcurrently and RemoveIndexConcurrently in Django 3.0:
https://docs.djangoproject.com/en/dev/ref/contrib/postgres/operations/#django.contrib.postgres.operations.AddIndexConcurrently
Create a migration and then change migrations.AddIndex to AddIndexConcurrently. Import it from django.contrib.postgres.operations.
With Django 1.10 migrations you can create a concurrent index by using RunSQL and disabling the wrapping transaction by making the migration non-atomic by setting atomic = False as a data attribute on the migration:
class Migration(migrations.Migration):
atomic = False # disable transaction
dependencies = []
operations = [
migrations.RunSQL('CREATE INDEX CONCURRENTLY ...')
]
RunSQL: https://docs.djangoproject.com/en/stable/ref/migration-operations/#runsql
Non-atomic Migrations: https://docs.djangoproject.com/en/stable/howto/writing-migrations/#non-atomic-migrations
You could use the SeparateDatabaseAndState migration operation to provide a custom SQL command for creating the index. The operation accepts two lists of operations:
state_operations are operations to apply on the Django model state.
They do not affect the database.
database_operations are operations to apply to the database.
An example migration may look like this:
from django.db import migrations, models
class Migration(migrations.Migration):
atomic = False
dependencies = [
('myapp', '0001_initial'),
]
operations = [
migrations.SeparateDatabaseAndState(
state_operations=[
# operation generated by `makemigrations` to create an ordinary index
migrations.AlterField(
# ...
),
],
database_operations=[
# operation to run custom SQL command (check the output of `sqlmigrate`
# to see the auto-generated SQL, edit as needed)
migrations.RunSQL(sql='CREATE INDEX CONCURRENTLY ...',
reverse_sql='DROP INDEX ...'),
],
),
]
Do what tgroshon says for new django 1.10 +
for lesser versions of django i have had success with a more verbose subclassing method:
from django.db import migrations, models
class RunNonAtomicSQL(migrations.RunSQL):
def _run_sql(self, schema_editor, sqls):
if schema_editor.connection.in_atomic_block:
schema_editor.atomic.__exit__(None, None, None)
super(RunNonAtomicSQL, self)._run_sql(schema_editor, sqls)
class Migration(migrations.Migration):
dependencies = [
]
operations = [
RunNonAtomicSQL(
"CREATE INDEX CONCURRENTLY",
)
]
You can do something like
import django.contrib.postgres.indexes
from django.db import migrations, models
from django.contrib.postgres.operations import AddIndexConcurrently
class Migration(migrations.Migration):
atomic = False
dependencies = [
("app_name", "parent_migration"),
]
operations = [
AddIndexConcurrently(
model_name="mymodel",
index=django.contrib.postgres.indexes.GinIndex(
fields=["field1"],
name="field1_idx",
),
),
AddIndexConcurrently(
model_name="mymodel",
index=models.Index(
fields=["field2"], name="field2_idx"
),
),
]
Ref: https://docs.djangoproject.com/en/dev/ref/contrib/postgres/operations/#django.contrib.postgres.operations.AddIndexConcurrently
There is no support for PostgreSQL concurent index creation in django.
Here is the ticket requesting this feature - https://code.djangoproject.com/ticket/21039
But instead, you can manually specify any custom RunSQL operation in the migration -
https://docs.djangoproject.com/en/stable/ref/migration-operations/#runsql
Consider this structure:
some_table(id: small int)
and I want change it to this:
some_table(id: string)
Now I do this with three migrations:
Create a new column _id with type string
(datamigration) Copy data from id to _id with string conversion
Remove id and rename _id to id
Is there a way to do this with only one migration?
You can directly change the type of a column from int to string. Note that, unless strict sql mode is enabled, integers will be truncated to the maximum string length and data is possibly lost, so always make a backup and choose a max_length that's high enough. Also, the migration can't easily be reversed (sql doesn't directly support changing a string column to an int column), so a backup is really important in this one.
Django pre 1.7 / South
You can use db.alter_column. First, create a migration, but don't apply it yet, or you'll lose the data:
>>> python manage.py schemamigration my_app --auto
Then, change the forwards method into this:
class Migration(SchemaMigration):
def forwards(self, orm):
db.alter_column('some_table', 'id', models.CharField(max_length=255))
def backwards(self, orm):
raise RuntimeError('Cannot reverse this migration.')
This will alter the column to match the new CharField field. Now apply the migration and you're done.
Django 1.7
You can use the AlterField operation to change the column. First, create an empty migration:
>>> python manage.py makemigrations --empty my_app
Then, add the following operation:
class Migration(migrations.Migration):
operations = [
migrations.AlterField('some_model', 'id', models.CharField(max_length=255))
]
Now run the migration, and Django will alter the field to match the new CharField.