There are already similar questions for South, but I have started my project with Django 1.7 and am not using South.
During development a lot of migrations have been created, however the software is not yet delievered and there exists no database that must be migrated. Therefore I would like to reset the migrations as if my current model was the original one and recreate all databases.
What is the recommended way to do that?
EDIT: As of Django 1.8 there is a new command named squashmigrations which more or less solves the problem described here.
I got this. I just figured this out and it is good.
First, to clear migrations table:
./manage.py migrate --fake <app-name> zero
Remove app-name/migrations/ folder or contents.
Make the migrations:
./manage.py makemigrations <app-name>
Finally tidy up your migrations without making other database changes:
./manage.py migrate --fake <app-name>
In the Django 1.7 version of migrations the reset functionality that used to be in South has been dropped in favor of new functionality for 'squashing' your migrations. This is supposed to be a good way to keep the number of migrations in check.
https://docs.djangoproject.com/en/dev/topics/migrations/#squashing-migrations
If you still want to really start from scratch i assume you still could by emptying the migrations table and removing the migrations after which you would run makemigrations again.
I just had the same problem.
Here's my workaround.
#!/bin/sh
echo "Starting ..."
echo ">> Deleting old migrations"
find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc" -delete
# Optional
echo ">> Deleting database"
find . -name "db.sqlite3" -delete
echo ">> Running manage.py makemigrations"
python manage.py makemigrations
echo ">> Running manage.py migrate"
python manage.py migrate
echo ">> Done"
The find command: http://unixhelp.ed.ac.uk/CGI/man-cgi?find
Assuming this is your project structure,
project_root/
app1/
migrations/
app2/
migrations/
...
manage.py
remove_migrations.py
you can run the script remove_migrations.py from the the place indicated above to delete all migrations files.
#remove_migrations.py
"""
Run this file from a Django =1.7 project root.
Removes all migration files from all apps in a project.
"""
from unipath import Path
this_file = Path(__file__).absolute()
current_dir = this_file.parent
dir_list = current_dir.listdir()
for paths in dir_list:
migration_folder = paths.child('migrations')
if migration_folder.exists():
list_files = migration_folder.listdir()
for files in list_files:
split = files.components()
if split[-1] != Path('__init__.py'):
files.remove()
Manually deleting can be tiring if you have an elaborate project. This saved me a lot of time. Deleting migration files is safe. I have done this an umpteenth number of times without facing any problems...yet.
However when I deleted the migrations folder, makemigrations or migrate did not create the folder back for me. The script makes sure that the migration folder with its __init__.py stays put, only deleting the migration files.
Delete files:
delete_migrations.py (in root of prj):
import os
for root, dirs, files in os.walk(".", topdown=False):
for name in files:
if '/migrations' in root and name != '__init__.py':
os.remove(os.path.join(root, name))
DELETE FROM django_migrations Where app in ('app1', 'app2');
./manage.py makemigrations
./manage.py migrate --fake
OR, you can write migration from this all
I try different commands and some of the answers help me. Only this sequence in my case fixed both broken dependencies in migrations in MYAPP and clean all past migrations starting from scratch.
Before doing this ensure that database is already synced (e.g. do not add a new Model field here or change Meta options).
rm -Rf MYAPP/migrations/*
python manage.py makemigrations --empty MYAPP
python manage.py makemigrations
python manage.py migrate --fake MYAPP 0002
Where 0002 is the migration number returned by the last makemigrations command.
Now you can run makemigrations / migrate again normally because migration 0002 is stored but not reflected in the already-synced database.
If you don't care about previous migrations, what about just removing all migrations in the migrations/ directory? you will start the migration sequence from scratch, taking your current model as reference as if you had written the whole model now.
If you don't trust me enought to remove, then try to move them away instead.
A simple way is
Go to every app and delete the migration files.
Then go to the django-migrtaions table in the database and truncate it(delete all entries).
After that you can create migrations once again.
cd to src directory
cd /path/to/src
delete migration directories
rm -rf your_app/migrations/
note that this should be done for each app separately
migrate
python3.3 manage.py migrate
if you wish to start again
python3.3 manage.py makemigrations your_app
If you're in development mode and you just want to reset everything (database, migrations, etc), I use this script based on Abdelhamid Ba's answer. This will wipe the tables of the database (Postgres), delete all migration files, re-run the migrations and load my initial fixtures:
#!/usr/bin/env bash
echo "This will wipe out the database, delete migration files, make and apply migrations and load the intial fixtures."
while true; do
read -p "Do you wish to continue?" yn
case $yn in
[Yy]* ) make install; break;;
[Nn]* ) exit;;
* ) echo "Please answer yes or no.";;
esac
done
echo ">> Deleting old migrations"
find ../../src -path "*/migrations/*.py" -not -name "__init__.py" -delete
# Optional
echo ">> Deleting database"
psql -U db_user -d db_name -a -f ./reset-db.sql
echo ">> Running manage.py makemigrations and migrate"
./migrations.sh
echo ">> Loading initial fixtures"
./load_initial_fixtures.sh
echo ">> Done"
reset-db.sql file:
DO $$ DECLARE
r RECORD;
BEGIN
-- if the schema you operate on is not "current", you will want to
-- replace current_schema() in query with 'schematodeletetablesfrom'
-- *and* update the generate 'DROP...' accordingly.
FOR r IN (SELECT tablename FROM pg_tables WHERE schemaname = current_schema()) LOOP
EXECUTE 'DROP TABLE IF EXISTS ' || quote_ident(r.tablename) || ' CASCADE';
END LOOP;
END $$;
migration.sh file:
#!/usr/bin/env bash
cd ../../src
./manage.py makemigrations
./manage.py migrate
load_initial_fixtures.sh file:
#!/usr/bin/env bash
cd ../../src
./manage.py loaddata ~/path-to-fixture/fixture.json
Just be sure to change the paths to corresponds to your app. I personally have these scripts in a folder called project_root/script/local, and django's sources are in project_root/src.
After deleting each "migrations" folder in my app (manually), I ran:
./manage.py dbshell
delete from django_migrations;
Then I thought I could just do ./manage.py makemigrations to regenerate them all. However, no changes were detected. I then tried specifying one app at a time: ./manage.py makemigrations foo, ./manage.py makemigrations bar. However, this resulted in circular dependencies that could not be resolved.
Finally, I ran a single makemigrations command that specified ALL of my apps (in no particular order):
./manage.py makemigrations foo bar bike orange banana etc
This time, it worked - circular dependencies were automatically resolved (it created additional migrations files where necessary).
Then I was able to run ./manage.py migrate --fake and was back in business.
Related
I made a spelling error in my model and now one of my columns is misspelled. I'd like to drop all tables in the database, fix the error in the model.py, and recreate the database with the correct spelling in model.
I've tried to use the suggestions in the this article but the table still exists after I follow the commands outlined there.
Anyone have a quick way to do this?
Delete the sqlite database file (often db.sqlite3) in your django project folder (or wherever you placed it)
Delete everything except __init__.py file from migration folder in all django apps (eg: rm */migrations/0*.py)
Make changes in your models (models.py).
Run the command python manage.py makemigrations or python3 manage.py makemigrations
Then run the command python manage.py migrate.
That's all.
If your changes to the models are not detected by makemigrations command, please check this answer
rm -f tmp.db db.sqlite3
rm -r my-app/migrations
python manage.py makemigrations
python manage.py migrate
Removes the database.
Removes the migrations from your app.
Re-runs the migrations. Note: you could also do: python manage.py makemigrations my-app
Migrate changes.
You can just delete your sqlite file.
Regarding your question, you should use Django migration system to do database changes for you project using makemigrations and migrate commands
migration docs
To do everything in one go, run all below commands in a shell script (Linux flavors) or in a batch file (Windows). and run that script where your manage.py exists:
find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc" -delete
find . -path "*.sqlite3" -delete
python manage.py makemigrations
python manage.py migrate
python manage.py migrate
python manage.py shell -c "from django.contrib.auth.models import User; User.objects.create_superuser('admin', 'admin#example.com', 'adminpass')"
I prefer them running by copy-pasting directly in one go. make sure to change superuser credentials.
Note- This will DELETE all databases and their corresponding migrations
Django 1.9.7, db.sqlite3 as DB
I have a Django project with several apps. For app "A", I had migrations, but I deleted them by accident and pushed to the remote git. Also, a lot of new stuff for other apps was pushed to the git during the day. Other apps don't depend on the "A" app models.
Everything worked until I decided to add a new field to the model of the "A" app. I got OperationalError: no such column: error. I tried to make initial migrations for the app "A" python manage.py migrate --fake-initial. I got new migrations but I still have the OperationalError: no such column:.
How to fix "A" app migrations without affecting other apps migrations?
From git point of view, you can do revert to previous commit.
git revert sha #commit sha of the last commit
OR
git reset --hard HEAD~n #n how many commits to remove.
git push --force
Fixing through django(possible if you didn't add any migrations later.),
python manage.py makemigrations APP_A --empty
python manage.py makemigrations APP_A
python manage.py migrate --fake
Unfortunately git revert didn't help me. In the end, I solved the problem by executing the following steps:
1.Manually delete all tables related to the "A" app in db.sqlite3.
2.Create new migrations and db.sqlite3 tables from existing schema:
python manage.py makemigrations A --empty
python manage.py makemigrations A
python manage.py migrate
3.Dump the tables data back into db.sqlite3 from a backup:
sqlite3 ~/Backup/A/db.sqlite3 ".dump table_name" | grep -v "CREATE" | sqlite3 db.sqlite3
OK, this is driving me nuts. I'm trying to remove and reset all my south migrations in my DEV environment for two apps, and start again with a clean slate, as my migrations files list was getting very long and I don't need them, as the models are constantly changing, and it's just DEV. Whatever I try, South constantly complains that the DB tables are already there.
Going on this answer on SO, I first remove my migrations dir entirely from each app:
rm -r <my-app>/migrations
No problem. Then reset the south tables:
python manage.py reset south
Again, no worries.
Now tell south that I want those apps managed:
python manage.py convert_to_south <appname>
All seems fine, it even creates the initial migration and runs the --fake:
Created 0001_initial.py. You can now apply this migration with: ./manage.py migrate CC
- Soft matched migration 0001 to 0001_initial.
Running migrations for CC:
- Migrating forwards to 0001_initial.
> CC:0001_initial
(faked)
OK, so according to my (incorrect) understanding, I now have my existing tables managed by south. South knows that the tables ALREADY exist, as we ran initial as --fake.
Now I add a new field to a model, run schemamigration then migrate on that new schema, and guess what, South complains that the tables already exist.
django.db.utils.DatabaseError: table "_south_new_CC_coveredcall" already exists
What the hell? What on earth am I doing wrong?
Caveat: it's late here and I'm tired, but it will be something along these lines:
Rather than telling South that you want to migrate an existing app (which implies a schema exists), you can fake-zero the apps, delete the migrations and make a new intitial one for each app and fake apply it. That basically gets South to replace multiple migrations with one per app
$ ./manage.py dumpdata myapp1 myapp2 > dumped.json # just in case!
$ ./manage.py migrate myapp1 zero --fake
$ ./manage.py migrate myapp2 zero --fake
$ rm /path/to/myapp1/migrations/.py*
$ rm /path/to/myapp2/migrations/.py*
$ ./manage.py schemamigration myapp1 --initial
$ ./manage.py schemamigration myapp2 --initial
$ ./manage.py migrate myapp1 --fake
$ ./manage.py migrate myapp2 --fake
The new 0001 migrations for myapp1 and myapp2 will have the same result as the multiple ones that actually created the existing schema, so all will fit just fine (as long as there have been no custom SQL migrations etc)
Try this code:
rm <app>/migrations/*
python manage.py schemamigration <app> --initial
python manage.py migrate <app> 0001 --fake --delete-ghost-migrations
Thanks to all who contributed - it turns out that as I'm using SQlite in dev, It's this that is causing all the issues. See another question on SO for clarification. Answering my own question here as it may be useful for others - I'm switching to mySQL, as the issue I stated above doesnt exist in my PRD env, which uses mySQL. Migration all works seamlessly.
I'm deploying a project on a new development environment.
As I'm using South I did:
$ python manage.py syncdb --all
$ python manage.py migrate --fake
I used syncdb --all to apply actual state of models.
Then migrate --fake to mark all models as migrated.
But after that, my model is not on the last version (missing fields)
What am I doing wrong ?
I assume all my modifications have migrations.
If I do
$ python manage.py syncdb
It seems to create the first state since when I used South (that is expected)
But then
$ python manage.py migrate
Some tables appears as already created
Actually this, should have been fine for my case
$ python manage.py syncdb --all
$ python manage.py migrate --fake
Having to redeploy my app recently, I faced the same issue.
I just realized that I had a double initial migration on the model that were causing the problem
0001_initial.py
0002_initial.py
0003_auto__add_field_mytable_myfield.py
I simply deleted & renamed
0001_initial.py
0002_auto__add_field_mytable_myfield.py
Then redone the whole database deployment (obviously not forgetting to update the already applied migrations on my other hosts)
--fake option does not avoid errors while trying to create new migrations. It records that migrations have applied without actually applying them.
Also, you need --ignore-ghost-migrations or --delete-ghost-migrations to achieve what you are looking for.
To convert an existing project to south, you first need to convert the app
Now, if you have already run --fake, to recover, you can do this:
Go to ./manage.py dbshell
DELETE FROM south_migrationhistory WHERE id > 0; //Note that this would delete everything in the table.
If you want to remove migrations of a specific app,
DELETE FROM south_migrationhistory WHERE app_name = 'blah'
When I make a change to my models (only to add a column during development!), Django won't issue any ALTER TABLE statements to update the database. Is there a way to get this implemented or worked around? - other then adding the columns manually?
Note I'm not really looking for a full-migration solution, just something that lets me keep coding as I add columns on the way.
Use python manage.py sqlclear YOURAPP in conjunction with dumpdata and loaddata (simplified version of answer by fish2000, which also uses a specific app):
DB=/path/to/db.sqlite3
APP=YOURAPPNAME
tmpdate=$(date "+%Y%m%d-%H%M%S")
echo "+ Dumping django project database to fixture DUMP-${tmpdate}.json ..." &&\
python manage.py dumpdata $APP --format='json' --indent=4 --verbosity=1 > datadumps/DUMP-${tmpdate}.json &&\
echo '+ Backing up sqlite binary store...' &&\
cp $DB $DB.bk &&\
echo '+ Rebuilding database structure from model defs...' &&\
python manage.py sqlclear $APP &&\
echo "+ Reloading project data from fixture dump DUMP-${tmpdate}.json ..." &&\
python manage.py loaddata datadumps/DUMP-${tmpdate}.json
Use a migration tool such as South.
Sorry, this is a little late but I thought I would post it here anyways in case anyone else is having this problem. If you are still in development and all of your data is dummy data (meaning you don't want to keep any of it), then all you have to do is delete the database and run syncdb again.
Check out evolution http://code.google.com/p/django-evolution/
If you don't want to set up migrations – you may be able to use a trick like this:
export JANGY_PROJECT='/Users/fish/Dropbox/ost2/ost2'
export BPYTHON_INIT_SCRIPT='${JANGY_PROJECT}/utils/django_shell_imports.py'
export PYTHONPATH="${JANGY_PROJECT}:${PYTHONPATH}"
alias jangy="python manage.py"
alias bp="cd $JANGY_PROJECT && bpython --interactive $BPYTHON_INIT_SCRIPT"
function jangyfresh () {
tmpdate=$(date "+%Y%m%d-%H%M%S") &&\
cd $JANGY_PROJECT &&\
echo "+ Dumping django project database to fixture DUMP-${tmpdate}.json ..." &&\
python manage.py dumpdata --format='json' --indent=4 --verbosity=1 > datadumps/DUMP-${tmpdate}.json &&\
echo '+ Backing up sqlite binary store and taking database offline...' &&\
mv sqlite/data.db sqlite/data.db.bk &&\
echo '+ Rebuilding database structure from model defs...' &&\
python manage.py syncdb &&\
echo '+ Graceful-restarting Apache...' &&\
sudo apachectl graceful &&\
echo '+ Enabling write access on new sqlite file...' &&\
chmod a+rw sqlite/data.db &&\
echo "+ Reloading project data from fixture dump DUMP-${tmpdate}.json ..." &&\
python manage.py loaddata datadumps/DUMP-${tmpdate}.json &&\
echo '+ Rebuilding project database structure...'
}
... which what that bash function does is:
Dumps the database out to a fixture,
named with a date/time stamp
Backs up and deletes
the binary database file (SQLite in this
case, which it's comparatively easy
to delete the file in question)
Resyncs the the DB from models (regenerating the binary DB file)
(optional) Fixes the DB files' perms (which dropbox can screw with, in my case here)
Repopulates the new DB from the last fixture
(optional) restarts apache
I use this during development to back things up and start from scratch – sometimes it works if you add a column, sometimes it'll complain about the newly defined model field not having a database column.
In these cases I run the command, edit the models.py file, delete the sqlite file and reload the last fixture.
Obviously, I don't do this on a production install, nor I would not recommend that.