My django app is containerized along side postgresql. The problem is that migrations do not seem to be persisting in the directory. Whenever I run docker exec -it <container_id> python manage.py makemigrations forum, the same migrations are detected. If I spin down the stack and spin it back up the and run makemigrations again, I see the same migrations detected. Changes to the fields, adding models, deleting models, none ever get detected. These migrations that do appear seem to be getting written to the database, as when I try to migrate, I get an error that there are existing fields already. But if I look at my migrations folder, only the init.py folder is present. All the migrate commands add no changes to the migrations folder.
I also tried unregistered the post model from the admin and spinning up the stack, yet I still see it present in the admin. Same things with changes to the templates. No change I make sticks from inside docker seems to stick.
*Note this problem started after I switched to wsl 2 and enabled it in docker desktop (windows)
**Update migrations can be made from bash of docker container
I found out what the problem was. My docker-stack.yml file was pointed to a directory that did not exist in the dockerfile.
Related
I have a django REST project and a PostgreSQL database deployed to DigitalOcean. When I develop locally, I have a separate dockerized REST server and a separate PostgreSQL database to test backend features without touching production data.
My question arises when I'm adding/modifying model fields that require me to make migrations using python [manage.py](https://manage.py) makemigrations and python [manage.py](https://manage.py) migrate command. Here is my current situation so far:
What I was supposed to do
IN LOCAL ENV, to create the migration files,
python manage.py makemigrations
python manage.py migrate
Now commit these newly created files, something like below.
git add app/migrations/...
git commit -m 'add migration files' app/migrations/...
IN PRODUCTION ENV, run only the below command.
python manage.py migrate
What I did so far
IN LOCAL ENV, created the migration files,
python manage.py makemigrations
python manage.py migrate
I committed & pushed the changes to production WITHOUT the created migration file
IN PRODUCTION ENV, ran BOTH commands.
python manage.py makemigrations
python manage.py migrate
The production server successfully added the isActive field to the database and is working fine, but I still have a 0011_user_isActive.py migration file in my local changes that hasn't been staged/committed/pushed to github repo.
And because I ran makemigrations command in production env, it probably created the same migration file that I haven't pushed from local env.
My questions are:
What happens if I push the local migration file to production? Wouldn't it create a conflict when I run migration command on digitalocean console in the future?
How should I fix this situation?
I am just scared I'm going to corrupt/conflict my production database as I'm very inexperienced in databases and have too much to risk at the moment. Would appreciate any tips on best practices when dealing with such situations!
As docs says:
The migration files for each app live in a “migrations” directory inside of that app, and are designed to be committed to, and distributed as part of, its codebase. You should be making them once on your development machine and then running the same migrations on your colleagues’ machines, your staging machines, and eventually your production machines.
So it's best practice is to push your migration files and make your local and production migration files sync.
And if you got conflict when pushing migraions files and pulling them, the makemigrations --merge command is for solving that.
Also docs says:
Because migrations are stored in version control, you’ll occasionally come across situations where you and another developer have both committed a migration to the same app at the same time, resulting in two migrations with the same number.
Don’t worry - the numbers are just there for developers’ reference, Django just cares that each migration has a different name. Migrations specify which other migrations they depend on - including earlier migrations in the same app - in the file, so it’s possible to detect when there’s two new migrations for the same app that aren’t ordered.
When this happens, Django will prompt you and give you some options. If it thinks it’s safe enough, it will offer to automatically linearize the two migrations for you. If not, you’ll have to go in and modify the migrations yourself - don’t worry, this isn’t difficult, and is explained more in Migration files below.
Also be aware that in case of updating existed data in production, you can use RunPython in migration file. Read about it here.
I have a wagtail app that deployed to docker, but then I get a very strange error, below are the steps:
Step 1:
docker-compose run app /venv/bin/python manage.py makemigrations`
Migrations for 'locations':
bakerydemo/locations/migrations/0008_alter_locationoperatinghours_day.py
- Alter field day on locationoperatinghours`
Step 2:
docker-compose run app /venv/bin/python manage.py migrate locations
Operations to perform:
Apply all migrations: locations
Running migrations:
===> No migrations to apply. <=== HUH?
Your models in app(s): 'locations' have changes that are not yet reflected in a migration, and so won't be applied.
===> Run 'manage.py makemigrations' to make new migrations,
and then re-run 'manage.py migrate' to apply them. <=== DOUBLE HUH?
Any wagtail or django or docker afficianados that can tell me what might be going on here? A similar question regarding Heroku mentioned running running the migrations before Heroku-izing, that is something I tried here, but it created an error in my locations app after dockerizing the container. The solution is from https://github.com/wagtail/bakerydemo and I added a few customizations to the locations app.
You need to create your migration files locally and re-build your image with the new migration files included. Your migration files are just like any other code in the application and should be included in the image as part of the image build process.
When you run makemigrations in the first command, you are creating a new container that creates the new migration files in that container... but when you run the second command, you create yet another separate container. Because the container created by the second command is separate, it won't contain the files created from the fist command -- hence, why you see the message No migrations to apply because the new migration files are not present.
I have a Postgres database full of data. And I made several changes to my Django app models.
mange.py makemigrations worked fine and created the migration files. But manage.py migrate execute only one file. And when I launch it again it doesn’t execute the rest as if they are already applied.
I deleted the migration files that were not applied and did another makemigration but it says no changes detected.
Any ideas how to reflect the models changes on the database without losing the data ?
Thanks
Django keeps track of which migrations it has applied already, so when you run the migrate command it will execute only the migrations that Django thinks that are missing.
I deleted the migration files that were not applied and did another makemigration but it says no changes detected.
This was a bad idea, it will make your migrations inconsistent.
If you want to go back in time, instead of deleting migrations, the proper way to do this is by reverting migrations. You can use the same migrate command and specify to which migration point you want your database model to be.
Check this answer for further information about reverting migrations; django revert last migration
I have deployed app from github repository to the heroku account of my customer as collaborator but this time I had to add some new models.
However I realized that when I deploy my changes from github heroku does not run makemigrations and migrate.
I I read some answers on stackoverflow and understood this is how it is supposed to be.
However my question is what should I do ? What is the best practise to deploy change models to heroku app. (I assume it is not deleting and recreating my app again since customer already has data there.)
(I am able to run makemigrations and migrate from the bash manually but when I have 30+ deployments it's a pain)
Check out the new feature on Heroku called "Release Phase": https://devcenter.heroku.com/articles/release-phase It will allow you to run migrations during the deployment. Just add whatever command you want to your Procfile, like this:
web: your_web_command
release: python manage.py migrate
The release command will run after your app is done building, and before it's launched.
I deployed a new django app on Heroku which worked out fine. However, today I changed my model a little bit (added a new field) and afterwards deleted my development server file db-sqlite3 and the migrations folder in order to reflect the changes in the development database. This worked out fine on the local server but when I pushed this to heroku I get a 500 error when trying to access the model in the django-admin section.
I tried to run some migrations via heroku but I get the following error message:
! These migrations are in the database but not on disk:
<joins: 0002_auto__add_field_join_ip_address>
<joins: 0003_auto__add_unique_join_email>
<joins: 0004_auto__add_field_join_ref_id>
<joins: 0005_auto__add_unique_join_email_ref_id>
<joins: 0006_auto__add_unique_join_ref_id>
<joins: 0007_auto__del_unique_join_ref_id>
<joins: 0008_auto__del_unique_join_email__add_unique_join_ref_id>
<joins: 0009_auto__add_field_join_friend>
! I'm not trusting myself; either fix this yourself by fiddling
! with the south_migrationhistory table, or pass --delete-ghost-migrations
! to South to have it delete ALL of these records (this may not be good).
(lwc) Daniels-MacBook-Pro:src danielrichter$ heroku run python manage.py migrate --delete-ghost-migrations
I can see that in my local migrations folder I only have the 0001_initial migration and somehow missing the other but I have no idea how to resolve the issue.
I have seen that others ran into the same error message, but I did not understand the proposed answers, since I am quite new to Django and coding in general. So if there is someone who could give me a hint how to resolve this I would be very thankful!
many many thanks!
Apparently the database thinks you have applied the migrations mentioned in the message, but it can't find the files on disk. Which you confirm. Maybe there was something wrong with your version management and you lost these files? I'd try to see if you can recover the files, then the problem would be over.
If not, it is a bit more difficult. The migrations mentioned have probably already been executed but the files are lost. Also you have made new changes which have not been applied. You should try to get your code state back to the state where the last missing migration was executed. You can then make a new migration file (python manage.py schemamigration --auto your_app_name) which can replace the missing migration files. This migration will be called 0002_something (0001 being present and 0002 being the next). After this replacement migration you can have new migrations (0003 and further).
On the server, before updating your version of the code (so you don't already have 0002) execute python manage.py migrate --delete-ghost-migrations. This will delete the references to the missing migration. Afterwards you can update your version and get the new 0002 etc migrations.
Call python manage.py migrate --fake your_app_name 0002. This will tell the database that the migration was applied, without actually applying anything. This is good because the changes were already applied by the lost migration files.
After this, you can run normal migrations: python manage.py migrate and it should be good.
Hope this helps.
and afterwards deleted my development server file db-sqlite3 and the
migrations folder in order to reflect the changes in the development
database
This is your issue, you shouldn't delete any migration files, if you make a change in your database, django (or south) will create a new migration file for those changes, then you run the migration command to apply those changes to your database, you have to commit those new migration files and send them to heroku, so the changes will be applied to the remote database too.
The workflow is like this:
You have some migration files or the initial one
Edit your models by adding/removing fields
Create migration file(s), in django (without South):
python manage.py makemigrations
Apply those changes to your local database by runnig:
heroku run python manage.py migrate
Add those model changes and migration files to your git index
Push the changes to heroku and run the migration command:
heroku run python manage.py migrate