I have read: https://symfony.com/doc/master/bundles/DoctrineMigrationsBundle/index.html
I'm curious about: doctrine:migrations:rollup. I know what it DOES (deletes my migrations, creates a single one with the whole dump of the current db).
But I don't understand how to use it in production:
When I rollup in local dev, push this into my CD-Pipeline, it would fail on production right? Because the dump would be executed above the already made migrations.
Do I have to delete all migration versions manually and just mark the "rolled up"-dump-migration as done in production?
I found this:
https://github.com/doctrine/migrations/issues/662
The whole process is several steps:
delete all your migrations in dev
create a new migration with dump-schema in dev
(I was misled, that the rollup command does this, but it does not)
deploy to production, but without automatic migrations
then use rollup command in PRODUCTION manually. It does delete all versions from the db and inserts only the rollup version (the one with the dumped schema in it)
then commit and push to your continuous delivery tool. The migration will not be executed, because the manually run rollup command has already inserted the dumped schema version.
Related
In Django, when working on multiple new features simultaneously, what is the best way to handle database migrations with git?
(as in python manage.py makemigrations , and python manage.py migrate which are used to update the database after the models.py file has been changed)
I’m in middle of working on feature1, on its own branch.
Now I need to create feature2 and push it to production.
Should I:
fork new feature2 branch off of production, create feature2, merge back to prod, migrate.
fork new feature2 branch off of production, create feature2, migrate, merge back to prod.
Or some other way?
And when I go back to feature1, what do i do to insure everything will be up to date?
If you use a database migration tool by command line in conjunction with git you can take advantage of up and downs SQL scripts.
Let's say you want to add some changes to your DB, through the migration tool you can define a namescript1-up.sql and namescript1-down.sql and you can checkout to a specific database version from the command line (after a git checkout).
For example, I use golang-migrate for my go apps (I think it can be used for any language when used by command line).
Anyway, I would say this is an improper use of a migration tool, which is more suitable to apply the scripts regarding only the schema variations and not regarding ordinary INSERT or UPDATE sql statements.
I would suggest working on a containerized version of your DB, so you can destroy and recreate anything on flight.
I just cloned a large django project and if I run the migrations I get errors. I wanted to know if it's necessary to go into each migration file and fix it, or to somehow skip them. I know in rails that in a situation like this one is advised to load the schema instead of running all of the past migrations, and wondered if it was the same for django. I know that one option is to delete all of the migrations, but since I am on a team that already has their local databases set up, I cant commit a deletion of all the migrations for my whole team, just so I can migrate my local database. How do I migrate my new local database given there are tons of migrations with tons of bugs and I mustn't delete the migration files for my git repo?
I have accidently deleted one of migrations folders and and have no backup for it.
What are my options?
DB is postgres. Right now everything is OK.(I have moved instead migration folder I have on my DEV server with SQL lite) So I am just getting red message on server that not all migrations have been applied.
But next time if i run migration i will be in trouble.
What is my way out?
Migrations are mainly for backward compatibility, and tracking/versioning of the changes to models/database. If you do not really care about historical changes, etc.. then you can just delete the migrations directory and do:
python manage.py makemigrations <app_name>
This creates the initial migrations (like starting from a clean slate) - and moving forward you can track the historical migrations moving forward. More on this can be read here
Now, when you run the migrations, you can do
python manage.py migrate <app_name> --fake-initial
to fake the initial migration.
Now might be a good time to have a version control on your application
Use version control.
You are not the first developer to delete important files, but often recovery takes less than a second - thanks to version control systems (also called revision control systems). Please stop everything else and install and use one of Git, Mercury or Subversion.
Don't use FTP
It's totally. I mean totally insecure. Always use SFTP
Don't use sqlite3 for local with other db for production
sqlite doesn't enforce strict type checking. Postgresql on the other hand is very particular about it. Additionally sqlite only has a subset of the functionality that's found on postgresql. Last but not least different RDBMS have different intricacies. If you use one locally and another in production, there is always a chance that your code will break when you deploy to live
Managing without the migration files
This is not a big loss as long as your database is in sync with your models.
If you database is not in sync with your models, you can use
./manage.py inspectdb
to recreate local models that represent the actual structure in the db. Then you do makemigrations and migrate (as explained by karthik) on the generated models.
Then replace that with your live models and do the step again.
Migrations allow transforming from one database schema to another while maintaining current data in the database. Django allows creating migrations using the command python manage.py makemigrations
Each time makemigrations is run a new migration file 000n.. is added based on the changes detected in the models.py file.
Sometimes after making small changes to models.py, I want to run makemigrations but do not want a new migration to be created because the previous migrations haven't been used yet which allows them to be merged together, primarily because running each migration in production can take a lot of time when there is a lot of data in the database so merging migrations before hand may be preferable.
Is there a way to allow a new migration 000n.. to be merged with an existing migration 000(n-1).. ?
The command you are looking for is squashmigrations. It will merge all the unapplied migrations of a given app into a single file.
I want to run makemigrations but do not want a new migration to be
created because the previous migrations haven't been used yet
This is not a problem, Django runs migrations from top to bottom, meaning: your latest migration file will wait until other previous migration files are ran.
because running each migration in production can take a lot of time
when there is a lot of data in the database
How much data do you have in the database? If really much, then you must already have replications and redundant db servers. In this case, switch the reads and writes to one, say slave server, run the migrations in the master. and then switch the traffic back to master, and before that make sure that the lag between them is 0 and new schema is replicated properly among them
Actually we are group of 3 people working on the same project, and each one individually make changes in django database. After running migrations in individual machine, it creates migration file for each migration. When someone pushes updated code in remote git repository, it creates conflict with others' migrations of same name.
Because of this reason, I lost my whole data once. Kindly give suggestions what should I do with this migration thing?
here's what i do: whenever I want to fetch from remote i check if a duplicate migration will be fetched. (we have a script that checks for all migration directories if there are filenames which have the same starting number.) if that is the case, I 'merge' the migrations, usually like this:
Find the last migration before the duplicate, let's say it's migration 000X
Make sure you are on your local source version, before the duplicate is added.
migrate back to after migration n:
python manage migrate app 000X
pull the new version including the duplicates.
remove your duplicate migrations
run schemamigration
python manage schemamigration --auto
Now you should get a new migration adding your model changes on top of the changes that were made in the migration you pulled.