I am considering to add the command release: python manage.py migrate --no-input to my deployment, so Heroku is automatically migrating, once I push my repository. I now wonder if --no-input is a 'good' idea?
One more specific question: If migrate normally asks me 'Did you rename this field'. With --no-input, will it answer automatically yes? I couldn't find much detailed information in the official Django documentation.
We don't use the interactive flag (--no-input) in our pipeline and to the best of my knowledge it doesn't make much difference. If you inspected Django source code, you would find out that the interactive flag only falls through to the pre_migrate and post_migrate signals that are emitted prior to and post migration.
As far as I know, no internal Django app uses the interactive argument inside these signals. Some external packages perhaps could use that, but I personally have never encountered that.
As for your question:
One more specific question: If migrate normally asks me 'Did you rename this field'. With --no-input, will it answer automatically yes? I couldn't find much detailed information in the official Django documentation.
This happens in the makemigrations management command, not in migrate. You usually do (and should do) the former locally, so no need to include that in your deployment pipeline.
Related
On our staging server we observed a runtime error stating a field is missing from a database,
column our_table.our_field does not exist
LINE 1: ...d"."type", "our_table"...
The field was added during a recent update with a complicated migration squashing process. It's possible that some errors were made during this process, but "manage.py showmigrations" command shows that the migration has been applied and "manage.py makemigrations" does not create any new migration. As we do not run tests on our staging or production databases, we are trying to figure out the most effective method for identifying such errors.
In short, how can we identify mismatches between the database and Django models caused by an incorrect migration like the following?
python manage.py migrate our_app --fake
I suppose I am looking for something like
python manage.py check_database
Edit: Many thanks for the suggestions. However, this is more of a deployment than a development question because the problem likely occurred when our devops tried to apply the squashed migrations while retaining data on the staging server (which will be case on production). It was scary to learn that such inconsistency can occur when makemigrations and showmigrations do not show any problem and can therefore also happen on production.
The bottom line is that we need some way to ensure our database matches our models after deployment.
Django migrations has excellent behavior in terms of single migrations, where, assuming you leave atomic=True, a migration will be all-or-nothing: it will either run to completion or undo everything.
Is there a way to get this all-or-nothing behavior for multiple migrations? That is to say, is there a way to either run multiple migrations inside an enclosing transaction (which admittedly could cause other problems) or to rollback all succeeded migrations on failure?
For context, I'm looking for a single command or setting to do this so that I can include it in a deploy script. Currently, the only part of my deploy that isn't rolled back in the event of a failure are database changes. I know that this can be manually done by running python manage.py migrate APP_NAME MIGRATION_NUMBER in the event of a failure, but this requires knowledge of the last run migration on each app.
This isn't a feature in Django (at least as of 4.1). To do this yourself, your code needs to do the following:
Get a list of current apps that have migrations. This can be done by getting the intersection of apps from settings.INSTALLED_APPS and the apps in the MigrationRecorder.
Get the latest migration for each app.
Run migrations.
If migrations fail, run the rollback command for the latest migration for each app.
Here's an implementation of this as a custom management command: https://github.com/zagaran/django-migrate-or-rollback. This is done as a child-class of the migrate management command so that it exposes all of the command line options of migrate. To use this library, pip install django-migrate-or-rollback and add "django_migrate_or_rollback" to your INSTALLED_APPS in settings.py. Then you can use python manage.py migrate_or_rollback in place of the traditional migrate command.
Disclosure: I am the author of the library referenced (though I made the library based on this StackOverflow answer).
I have accidently deleted one of migrations folders and and have no backup for it.
What are my options?
DB is postgres. Right now everything is OK.(I have moved instead migration folder I have on my DEV server with SQL lite) So I am just getting red message on server that not all migrations have been applied.
But next time if i run migration i will be in trouble.
What is my way out?
Migrations are mainly for backward compatibility, and tracking/versioning of the changes to models/database. If you do not really care about historical changes, etc.. then you can just delete the migrations directory and do:
python manage.py makemigrations <app_name>
This creates the initial migrations (like starting from a clean slate) - and moving forward you can track the historical migrations moving forward. More on this can be read here
Now, when you run the migrations, you can do
python manage.py migrate <app_name> --fake-initial
to fake the initial migration.
Now might be a good time to have a version control on your application
Use version control.
You are not the first developer to delete important files, but often recovery takes less than a second - thanks to version control systems (also called revision control systems). Please stop everything else and install and use one of Git, Mercury or Subversion.
Don't use FTP
It's totally. I mean totally insecure. Always use SFTP
Don't use sqlite3 for local with other db for production
sqlite doesn't enforce strict type checking. Postgresql on the other hand is very particular about it. Additionally sqlite only has a subset of the functionality that's found on postgresql. Last but not least different RDBMS have different intricacies. If you use one locally and another in production, there is always a chance that your code will break when you deploy to live
Managing without the migration files
This is not a big loss as long as your database is in sync with your models.
If you database is not in sync with your models, you can use
./manage.py inspectdb
to recreate local models that represent the actual structure in the db. Then you do makemigrations and migrate (as explained by karthik) on the generated models.
Then replace that with your live models and do the step again.
I'm wondering if there's a way to write migrations that won't ask me for input at the terminal. For example, while writing a migration that deletes a model, Django asks me to confirm this at the command line with a yes/no. I want to avoid these prompts because the same code will be used for automatic deployments (can't type in prompts during automatic deployment).
I'm aware there's the noinput option. I'm more interested in learning how to write migrations that don't require keyboard input, if possible. I've also noted that sometimes even when using this option, I get prompted anyway (one example that comes to mind was a migration that asked me to input default data for new columns being added to the model; noinput didn't stop Django from asking for my input anyway).
There are two situations here: makemigrations and migrate.
migrate is 100% covered by the --noinput option: unless you install a custom, third-party pre/post migrate signal hook that ignores the option, you'll get no user prompts with this option.
makemigrations is not 100% covered by the --noinput option, and while it's a nice-to-have, it's absolutely not necessary. Migrations should always be created and tested in your development environment. I've yet to see a development environment where you can't provide manual input.
1.9 will in time introduce better support for the --noinput in combination with makemigrations (at the moment, it only supports --noinput in combination with --merge).
I have a django app running on heroku. I would like to run my South migrations before any code that depends on them goes live. After a quick review of current recommended practices I have found two suggested migration procedures.
Recommendation 1
Commit and push all changes
Run heroku run python manage.py migrate <APP_NAME> for each app
This suffers from having a period in between steps 1 and 2 where my code is assuming the latest schema is in place, but the db hasn't yet been updated.
Recommendation 2
commit and push all database changes.
Migrate.
Push all code changes.
This solves the previous problem, but adds a lot more complexity to the deployment process, and some day I will mess this up.
Potential Solution?
It seems that I can avoid the problem in Recommendation 1 and keep my deployment to a single step by utilising a custom post_compile script
that calls python $MANAGE_FILE migrate <APP_NAME> for each of my apps (in dependency order).
I have not seen this recommended anywhere, so my question is twofold. Can you see any potential problem with this approach, and do you have a better method?
If your application can afford some downtime, the easiest way seems to me to
Pause your app using $ heroku maintenance:on
Migrate all the apps at once with heroku run python manage.py migrate
Restart your app: $ heroku maintenance:off
Is it enough or do you have more complex needs ?