I am struggling with Django migrations when deploying into several environments. I have the following ones:
Dev locally on my laptop with sqlite3 database
Test in AWS (deployed via Beanstalk) connected to AWS RDS Production Database
Prod in AWS (deployed via Beanstalk) connected to AWS RDS Production Database.
My workflow is once I have done my dev locally, I deploy to the Test instance, which should make the backward compatible changes to the prod database, where I run my tests, before deploying to the Prod instance.
In my .gitignore, I have excluded all the migrations folder as when I used to apply the same migrations I created in dev, it lead to some inconsistencies and errors during deployment. I thought it would be cleaner to recreate the migrations on the test or prod servers when deploying.
In my .ebextensions I then have the following config that are executed when deploying the application:
01_makemigration:
command: "source /opt/python/run/venv/bin/activate && python manage.py makemigrations --noinput"
leader_only: true
02_migrate:
command: "source /opt/python/run/venv/bin/activate && python manage.py migrate --noinput"
leader_only: true
However when I deploy to the Test platform after having made changes to my models, the migrations don't seem to happen. I have tried to run those commands manually but it tells me that everything is up to date and no migrations are to be applied.
I have also tried to "rebase" the migrations using the following sequence:
On the Database:
> delete from django_migrations;
On the App:
> rm -rf calc/migrations/
> source /opt/python/run/venv/bin/activate
> python manage.py migrate --fake
> python manage.py makemigrations calc
> python manage.py migrate --fake-initial
But it does not seem to work either.
Would someone be able to advise what is the right way of applying migrations in that type of scenario? I would prefer to avoid committing dev migrations and create new and clean ones on the test environment (and the prod database) but I dont seem to find the right way to do it.
Thank you
Regards
Yann
You must not exclude your migrations from git. They are a part of your code base and need to be deployed with your app. You shouldn't be running makemigrations in prod, only migrate.
Related
Go to aws's setup guide for deploying a django app to elastic beanstalk here and go to the "Add a database migration configuration file" section.
You'll see that in the db-migrate.config file, they migrate the django app when it is deployed. I am wondering why they do not run makemigrations as well?
Thanks!
I'm no expert on django, but from what I can understand (e.g. here), makemigrations should be run when you change your existing models.
In the tutorial there are no changes made to existing models, thus its not executed.
If you want you can add makemigrations command to the db-migrate.config and check if it will have effect.
Possible new version of db-migrate.config:
container_commands:
10_makemigrate:
command: "python manage.py makemigrations"
leader_only: true
20_migrate:
command: "django-admin.py migrate"
leader_only: true
option_settings:
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: ebdjango.settings
The reason this is not done is because migrate and makemigrations are two very different (though related) operations. While Marcin is right that there are no changes in the tutorial to warrant running makemigrations, that is not why this isn't added to the config file.
makemigrations just generates the migration files from any new or altered models, while migrate actually runs these migrations on the database. You ideally should not be generating migration files after deploying your code. These files should be generated before deploying, and then they will be deployed to the remote instances (just like any other code updates), which will then allow you to run migrate with them, and update the remote database.
I am trying to deploy my Django server to Amazon through Beanstalk, and so far it's been ok except I've made a few changes to my models and when I deployed the instance on Aws is not updating accordingly.
I have followed the guide from amazon and created a file named db-migrate.config with the content
container_commands:
01_migrate:
command: "django-admin.py migrate"
leader_only: true
option_settings:
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: myAppName.settings
but obviously it doesn't seem to be working. I tried to access my django instance on Aws with
eb ssh myAppEnv
but when I enter I saw nothing and I couldn't find the code for my django server anywhere, thus i am unable to debug and manually run makemigrations also.
Anyone can help me with this?
Only way i was able to fix this was by specifying the exact applications i wanted to makemigrate and migrate.
02_migrateapps:
command: "source /opt/python/run/venv/bin/activate && python3 manage.py makemigrations organisations shows media exhibitors && python3 manage.py migrate --noinput"
leader_only: true
It's a real pain but each time i make a new app i'll need to add it to the list of makemigrations.
Hope this helps.
I'm deploying Django app to the AWS EB using CLI and noticed that EB doesn't see new migrations files for the first time. So, when I have new migrations I need to deploy twice. I looked at logs and indeed migrations were not found for the first time and found for the second time.
Here is my code for migrations:
container_commands:
01_migrate:
command: "django-admin.py migrate"
leader_only: true
02_collectstatic:
command: "python ras-server/manage.py collectstatic --noinput"
Am I need to change commands order? Also, I think that issue could be with Jenkins as I deploy from Jenkins. Any suggestions?
The issue was with Jenkins: for some reason when I deployed using execute shell migrations where not found for the first time.
The solution is to use Elastic Beanstalk Deployment plugin. Also, it takes less time to deploy with the plugin.
Same Error for me. In my case. I forgot to include App name in the migration. Try including App name exams
01_migrate:
command: "python manage.py makemigrations exams --noinput"
command: "python manage.py migrate exams --noinput"
leader_only: true
I'm getting this error. I know you usually get this error because the databases wasn't properly migrated.
When I run heroku local web, the website works fine when I go to localhost:5000.
However after I deploy the app to heroku with git push heroku master, the error comes up.
In other words, it works in my local environment. But it does not work after deploying to heroku.
I have Heroku-Postgres installed as an add-on in heroku.
What could be causing this?
excute migrations and makemigrations in bash heroku. open the terminal in the local project folder and give the following commands:
heroku run bash
~$ ./manage.py makemigrations
~$ ./manage.py migrate
~$ exit
The following steps did it for me
Make all the necessary migrations (python manage.py makemigrations) and migrate (python manage.py migrate) locally,
push to heroku master
and finally running heroku run python manage.py migrate
Solved the issue
I experienced the same error after making a change to a model and then deploying this change to heroku.
The only way I managed to fix this problem was to do the following:
Reset the database in heroku
Delete the migrations files from the migrations folder for the broken app locally (but keep the directory and the __init__.py file)
Run python manage.py makemigrations and python manage.py migrate. This will repopulate the migrations folder with clean migration files.
Push changes to master (ensure that you do not have migrations directories in .gitignore files.
Deploy changes to heroku
Run the heroku shell heroku run bash
Run python manage.py migrate
I was able to do this because my tables did not have much data in, but I would try to avoid resetting the database if I had more data in my tables.
In development when I was running django on a local server, I first added South in my installed apps and then did
python manage.py syncdb
After that, whenever I made a change to the database, I'd do
python manage.py scheemamigration
python manage.py migrate appName
I now am using AWS elastic beanstalk and do
git add .
git commit "change made"
git aws.push
to update the aws server. However, I cannot run
python manage.py syncdb
because it says
Unknown command 'syncdb'
so I cannot syncdb and do scheemamigrations. What is the best way for me syncdb and do scheema migrations using South now that I am using AWS servers.
You need to make a container command, heres a snippet from the aws docs...
On your local computer, update your configuration file (e.g., myapp.config) in the > .ebextensions directory.
container_commands:
01_syncdb:
command: "django-admin.py syncdb --migrate --noinput"
leader_only: true
See http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_Python_django.html (Step 6, point number 2) Sorry no anchors in aws docs..
EDIT: Added in migrate flag to syncdb, and changed aws doc reference to a more pertinent one