I am looking for a decent solution for any migrations in database itself.
We have custom SQL script which adds pg_trgm extention to PostgreSQL database, which is simply
CREATE EXTENSION pg_trgm;
and
FROM postgres:14.5-alpine
COPY init.sql /docker-entrypoint-initdb.d/init.sql
in database`s Dockerfile
After that I can easily add specialized GIN indexes to models, i.e.
class MyModel(models.Model):
"""
My model comment
"""
objects = MyManager()
field = models.CharField(max_length=100)
class Meta:
indexes = (
models.Index(Upper("field"), name="field_idx"),
GinIndex(
fields=["field"],
opclasses=["gin_trgm_ops"],
name="field_gin_idx",
),
GinIndex(
OpClass(Upper("field"), name="gin_trgm_ops"),
name="field_gin_idx",
),
So I want to reflect that "custom script" in Django pytest tests.
P.S.: I know there is a trigram extention for that purpose, but it was a TL decision to put custom database migration in separate place
Do you have any idea on how to prepare such fixture?
I was looking for related pytest fixtures but haven't found any solution
Related
I am migrating to django-oscar2.1
I have forked catalogue app, it has some existing migrations. Newly added field is_public is in migration #10. I have a custom migration #5 which is using create_from_breadcrumbs method. So when I tried to run migrate on fresh db migration#5 trigger error is_public field not exists in db.
create_from_breadcrumbs
Oscar loads a model with latest state which contains is_public in models but not in db.
In migration #5 I am loading model like this
Category = apps.get_model("catalogue", "Category")
You cannot use create_from_breadcrumbs in a migration like that. The reason for this is explained in Django's documentation on migrations - that the version of the model used in external code may be different from the one that this migration expects - which is why it fails.
If you want to create model objects as part of your migration you need to use the historic version of the model in the migration. This means defining your own version of create_from_breadcrumbs, and using the historic version of the model as suggested by the Django documentation:
def create_from_breadcrumbs_for_migration(apps, schema_editor):
Category = apps.get_model('catalogue', 'Category')
# Perform logic to create categories here.
class Migration(migrations.Migration):
dependencies = [
('catalogue', '0004_xxxx'),
]
operations = [
migrations.RunPython(create_from_breadcrumbs_for_migration),
]
That said, I would argue that migrations are not meant for this sort of data population in the first place. You would in my view be better off writing a separate management command for this, which you run after the migrations have been applied.
I added a field to one of my models, but in the 'models' folder I have two other python files which have only View models from which I query views in my database. When I run the makemigrations command, the new migrations file that is created includes also adding these view models to my database as tables (which I don't want to). How can I ignore these changes, and only commit the one addition of a field to an actual table on the database.
I think I maybe have to delete the migrations.CreateModel... in the new migrations file and only keep the migrations.addField... , then run the 'migrate' command. I didn't proceed with this because I'm not sure and maybe it will mess up my database in some way.
Thanks in advance to anyone who can help.
when you make a model for database view you must add meta class managed = false and db_table like this:
class MyViewModel(models.Model):
field: models.CharField(max_length=100)
class Meta:
managed = False
db_table = 'database_view_name'
when you write this and run makemigrations a migration generated contains this model but when you run migrate this migration doesnt change anything on database.
you also can create view using migrations in python. see migrations.RunPython for more details
I need to do some unit tests in my Django project. The problem is that almost every use case depends on prepopulated database objects.
For example, I want to create a product and test, if there were all pre_save signals successful.
from django.contrib.auth.models import User
from django.test import TestCase
from .models import Product
class ProductTestCase(TestCase):
def setUp(self):
self.user = User.objects.create(username='test_user')
self.product = Product.objects.create(name='Test product',user=self.user)
def test_product_exists(self):
self.assertIsNotNone(self.product)
def product_is_active_by_default(self):
...
I can't do that because product has to have User object related. But I can't create a User object because User has to have related plan object. There are multiple plans in my production database from which one is default but there are no plans inside test database.
So to be able to do unit tests I need to prepopulate test database with multiple objects from multiple apps.
How can I do that?
you can simply use django fixtures for that :-)
first populate a sample db with data then export data with python manage.py dumpdata
then in one of your apps create a directory named fixtures and put exported json file there (named tests.json or something else)
in your test class load fixtures like this
class ProductTestCase(TestCase):
fixtures = ['tests.json', ]
checkout django docs
PS: checkout factory boy too (#Gabriel Muj) answer
I don't recommend using fixture since you will need to maintain them each time you make changes to the model. Here is a better approach on creating objects for tests by using this library https://factoryboy.readthedocs.io/en/latest/ which is more flexible.
I'm trying to rename a model and I would like to write the migration in the way that it doesn't depend on the old name still present while it being applied. Can I somehow get data from a database table that no longer has a model in my migration code?
Details:
I have a Region model that I want to move into a more generic GeoObject model and remove from the models.py. If I write my migration code that creates GeoObjects from existing Regions with from models import Region I'll have to keep Region model until my main database will migrate. But I'd like to write a migration so that it doesn't depend on Region model being present, just check that the database table exists and use it. Is it possible to do it using Django instruments, without depending on a specific database type if possible?
Yes, you can.
But first of all, you really shouldn't import any model inside migration.
Take look at RunPython operation, that will allow you to run any python code inside your migration. RunPython will pass to your function 2 parameters: apps and schema_editor. First parameter contains structure of your models at stage of applying that migration, so if actual removing of model is later on that migration, you can still access that model using apps passed into function.
Let's say your model looked like this:
class SomeModel(models.Model):
some_field = models.CharField(max_length=32)
Now you're deleting that model, automatically created migration will contain:
class Migration(migrations.Migration):
dependencies = [
('yourapp', '0001_initial'), # or any other dependencies
]
operations = [
migrations.DeleteModel(
name='Main',
),
]
You can modify that migration by injecting RunPython just above DeleteModel operation:
operations = [
migrations.RunPython(
move_data_to_other_model,
move_data_back, # for backwards migration - if you won't ever want to undo this migration, just don't pass that function at all
),
migrations.DeleteModel(
name='SomeModel',
),
]
and creating 2 functions before Migration class:
def move_data_to_other_model(apps, schema_editor):
SomeModel = apps.get_model('yourapp', 'SomeModel')
for something in SomeModel.objects.all():
# do your data migration here
o = OtherModel.objects.get(condition=True)
o.other_field = something.some_field
def move_data_back(apps, schema_editor):
SomeModel = apps.get_model('yourapp', 'SomeModel')
for something in OtherModel.objects.all():
# move back your data here
SomeModel(
some_field=something.other_field,
).save()
It doesn't matter that your model is no longer defined in models.py, django can rebuild that model based on migration history. But remember: save method from your models (and other customized methods) won't be called in migrations. Also any pre_save or post_save signals won't be triggered.
In django models we have option named managed which can be set True or False
According to documentation the only difference this option makes is whether table will be managed by django or not. Is management by django or by us makes any difference?
Is there any pros and cons of using one option rather than other?
I mean why would we opt for managed=False? Will it give some extra control or power which affects my code?
The main reason for using managed=False is if your model is backed by something like a database view, instead of a table - so you don't want Django to issue CREATE TABLE commands when you run syncdb.
Right from Django docs:
managed=False is useful if the model represents an existing table or a database view that has been created by some other means. This is the only difference when managed=False. All other aspects of model handling are exactly the same as normal
When ever we create the django model, the managed=True implicitly is
true by default. As we know that when we run python manage.py makemigrations the migration file(which we can say a db view) is
created in migration folder of the app and to apply that migration i.e
creates the table in db or we can say schema.
So by managed=False, we restrict Django to create table(scheme, update
the schema of the table) of that model or its fields specified in
migration file.
Why we use its?
case1: Sometime we use two db for the project for
example we have db1(default) and db2, so we don't want particular
model to be create the schema or table in db1 so we can this or we can
customize the db view.
case2. In django ORM, the db table is tied to django ORM model, It
help tie a database view to bind with a django ORM model.
Can also go through the link:
We can add our raw sql for db view in migration file.
The raw sql in migration look like: In 0001_initial.py
from future import unicode_literals
from django.db import migrations, models
class Migration(migrations.Migration):
initial = True
dependencies = [
]
operations = [
migrations.RunSQL(
CREATE OR REPLACE VIEW app_test AS
SELECT row_number() OVER () as id,
ci.user_id,
ci.company_id,
),
]
Above code is just for overview of the looking of the migration file, can go through above link for brief.