I am trying to move my django project into a production environment and in doing so I switched from using sqlite to postgres. In my development environment, whenever I made changes to my models or anything that would significantly change how the database was setup, I would literally just drag my sqlite file to the trash and just run syncdb to create a new empty one (probably bad practice). Now that I am using postgres, I am wanting to do the same thing without actually deleting the database. Basically I was wondering if there was a way to completely empty it or clear it out and then just run syncdb and start over?
I also welcome any alternative suggestions that might lead me down the right path, I'm very new to this.
You can use flush. Just run this command:
python manage.py flush
First if you have initial data in your database you can use dumbpdata command:
python manage.py dumpdata > initial_data.json
For specific app run:
python manage.py dumpdata <app_name> > initial_data.json
Second run the flush command to clean your database:
python manage.py flush
Third and last, run loaddata command to load the initial data into your database and create superuser by running createsuperuser command
python manage.py loaddata initial_data.json
python manage.py createsuperuser
In case flush does not work, you can drop the whole database.
Go to windows command line.
If server is 'postgres' and db name is 'mydb', run:
C:\> psql -U postgres
You will see a postgres-# prompt. Next is to close connections running the following:
SELECT * FROM pg_stat_activity WHERE pg_stat_activity.datname='mydb';
SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = 'mydb';
Drop database once for all: DROP DATABASE mydb;
Related
I am migrating a project from Django 1.1 to Django 3.0 I am done with the project. When I am dumping the production dump to my local in the newly converted project I get "Table already exists".
Here's what I am doing.
mysql> create database xyx;
docker exec -i <container-hash> mysql -u<user> -p<password> xyx < dbdump.sql
then I run the migrate, as I have had to do some changes to the previously given models.
./manage.py migrate --fake-initial
this is the output I get
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (1050, "Table 'city' already exists")
So, what to do ?
Alright boys and girls, here's the approach I followed to solve this problem.
I dumped the entire database.
docker exec -i <container-hash> mysql -u<username> -p<password> <dbname> < dump.sql
Now I listed all the migrations I made using
./manage.py showmigrations <app-name>
This will give me the list of all the migrations I have applied, now from inspecting the migrations, I realized that from the 7th migration to the 30th migration I had done my changes.
Here's the tedious part which any sys admin can write a script to do in less than 4 lines of bash script. You can generate the raw SQL of any migration with this command.
./manage.py sqlmigrate <app-name> <migration-name> > changes-i-made.sql
Now that I have created my changes-i-made.sql file I'll need to run this script 22 more times but with >> otherwise everytime you run the command with a single > it will keep overwriting your changes file.
Now once all of your migration changes are recorded inside a file, open up your sql shell connect to the database and start pasting the changes or do some sql magic to pick all the changes directly from the file.
Once you're done go ahead and fake all the migrations, cause you don't need Django to do them you already did.
./manage.py migrate --fake
and then login to your production instance and get ready to fuck with your senior team lead who said you couldn't do it.
I just checked to see if this approach is working and the future migrations will be working, so I created one and everything works like a breeze.
I created many test accounts in my heroku server. Now they all have empty columns, and therefore do not work. And now I want to remove all data about existing users/accounts, including superuser, then recreate. How I can do this?
You can go to your terminal, where you have executed the command 'python manage.py runserver' and execute the command - 'python manange.py shell', then your shell will start.
Then execute the command:-
step-1: from django.contrib.auth.models import User
step-2: User.objects.all().delete() # this will delete all the user from your table
step-3: exit() # exit from the shell
step-4: python manage.py createsuperuser # for creating the superuser
This is totally in server side.
Go to your heroku database and reset database from the setting.
after that.
$ heroku run python manage.py makemigrations
$ heroku run python manage.py migrate
Create New Admin ( can be any or the same your existing)
All done, 100% that works for me.
I have screwed up my database so I tried to redo it. I did:
1) deleted all tables via sql
2) deleted the migrations folder
So when I now do manage.py makemigrations myapp it is creating the migration folder and the initial.py file, which looks fine. Also the __init__.py is there. However, if i do manage.py makemigrations myapp I always get the message "No migrations to apply."
I looked up the database in the shell and the tables are not there.
I am using Django 1.8.
Django keeps track of all the applied migrations in django_migrations table.
So just delete all the rows in the django_migrations table that are related to you app like:
DELETE FROM django_migrations WHERE app='your-app-name';
and then do:
python manage.py makemigrations
python manage.py migrate
Django keeps track of all the applied migrations in django_migrations table.
So, you can follow this method:
Delete the related rows from django_migrations.
run python manage.py migrate.
I usually ran into that issue myself. Here is the workaround I found:
Replace your database sqlite3 with this
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': '<your_username>$<your_database_name>',
'USER': '<your_username>',
'PASSWORD': '<your_mysql_password>',
'HOST': '<your_mysql_hostname>',
}
}`
The fill in for can be found in the databases tab on your Pythonanywhere dashboard.
Push it to github from your Terminal and pull it down from Pythonanywhere Bash again. You may have to add this on Bash Console: pip install mysqlclient
As #Anush mentioned, Django does keep track of all of the migrations in the django_migrations table. As mentioned, you can use raw SQL to remove the migration row to reset Django's migration history for the concerned app.
DELETE FROM django_migrations WHERE app='my-app';
If you are uncomfortable deleting rows like this, then you can use Django's management command replacing my-app for the name of your app.
python manage.py migrate --fake my-app zero
That command will remove the migration history rows from the database leaving you to start your migration again from scratch. You will then need to:
Delete all of the migration files within your app's 'migrations' directory but ensure that the __init__.py file remains.
Ensure that your app's data models are set up as you need them
Make the migrations again with the command python manage.py makemigrations {your_app_name}. This will create the 0001_initial.py file within your app's migrations directory.
You will not be able to actually migrate the initial database migration because the database table already exists, so you will need to fake the migration with the command, python manage.py migrate my_app --fake-initial
Instead of run:
python manage.py migrate
just run:
python manage.py migrate --run-syncdb
If you have many problems and require a solution
Create a database backup (more than one if necessary)
My solution was to run the following query in the SQL shell
drop database database_name with (force);
This completely removes the database, be careful
It solved the error for me
Greetings
After running into this problem on Django 3.0 I spent an entire day troubleshooting this. Sharing this so that others would not need to. Most of the times running
python manage.py makemigrations
python manage.py migrate
Should solve the problem. I followed the above step after deleting the files in migrations folder (except init.py), however I was not able to solve the problem. I found that every time you run makemigrations, a python file called 000x_xxx.py is created in the migrations folder. I looked inside the file and found that all my required changes were there but it also had other create table operations that I did not need (i.e those tables had already been created). If you run migrate with --fake or --fake-initial, the whole execution will be faked (i.e executed without actually making any change in the database).
To solve the problem I opened the generated 0000x_xxx.py file and deleted the changes I wanted to make in the database and ran "python manage.py makemigrations" again. This created a new migration file with the changes I wanted. After that I ran the following command.
python manage.py migrate --fake-initial appname
This solved the problem. Not sure if this should be this complicated.
Using django 1.7.7 I want to use django's migration to add or remove a field.
so I modified model.py and ran
python manage.py makemigrations myproj
Migrations for 'myproj':
0001_initial.py:
- Create model Interp
- Create model InterpVersion
python manage.py migrate myproj
Operations to perform:
Apply all migrations: myproj
Running migrations:
Applying myproj.0001_initial... FAKED
python manage.py runserver
Then checked the admin page, it is not updated.
Then I tried removing the migration folder and tried again; the migrate command says there are no migrations to apply.
How can I do the migration?
Note: I want to use the new technique using django's migration not the old south approach.
Make sure that the migrations/ folder contains a __init__.py file
Lost half an hour over that.
Deleting the migration directory is never a good idea, because Django then loses track of which migration has been applied and which not (and once an app is deployed somewhere it can become quite difficult to get things back in sync).
Disclaimer: whenever things like that occur it is best to backup the database if it contains anything valuable. If in early development it is not necessary, but once things on the backend get out of sync there is a chance of things getting worse. :-)
To recover, you could try resetting your models to match exactly what they were before you have added/removed the fields. Then you can run
$ python manage.py makemigrations myproj
which will lead to an initial migration (0001_initial...). Then you can tell Django to fake that migration, which means to tell it to set its internal counter to this 0001_initial:
With Django 1.7:
$ python manage.py migrate myproj
With Django >= 1.8:
$ python manage.py migrate myproj --fake-initial
Now, try to change your model and run makemigrations again. It should now create a 0002_foobar migration that you could run as expected.
In my case, the migrations were not being reflected in mysql database. I manually removed the row of 'myapp'(in your case 'myproj') from the table 'django_migrations' in mysql database and ran the same commands again for migration.
Most of the above solutions would help in the issue, however, I wanted to point out another possible (albeit rare) possibility that the allow_migrate method of database router may be returning False when it should have returned None.
Django has a setting DATABASE_ROUTERS which will be used to determine which database to use when performing a database query.
From the docs:
if you want to implement more interesting database allocation behaviors, you can define and install your own database routers.
A database router class implements up to four methods:
db_for_read(model, **hints)
db_for_write(model, **hints)
allow_relation(obj1, obj2, **hints)
allow_migrate(db, app_label, model_name=None, **hints)
From the documentation:
allow_migrate(db, app_label, model_name=None, **hints)
Determine if the migration operation is allowed to run on the database with alias db. Return True if the operation should run, False if it shouldn’t run, or None if the router has no opinion.
It is possible that one of the database routers in sequence is returning False for the migration that you're trying to run, in which case the particular operation will not be run.
I find Django migrations a bit of a mystery, and tend to prefer external tools (liquibase, for example).
However, I just ran into this "No migrations to apply" problem as well. I also tried removing the migrations folder, which doesn't help.
If you've already removed the migrations folder, here's an approach that worked for me.
First, generate the new "clean" migrations:
$ python manage.py makemigrations foo
Migrations for 'foo':
dashboard/foo/migrations/0001_initial.py
- Create model Foo
- Create model Bar
Then look at the SQL and see if it looks reasonable:
$ python manage.py sqlmigrate foo 0001
BEGIN;
--
-- Create model Foo
--
CREATE TABLE "foo" ("id" serial NOT NULL PRIMARY KEY, ... "created_at" timestamp with time zone NOT NULL, "updated_at" timestamp with time zone NOT NULL);
CREATE INDEX "..." ON "foo" (...);
COMMIT;
Then apply execute that same SQL on your database.
I'm using Postgres but it will be similar for other engines.
One way is to write the content to a file:
$ python manage.py sqlmigrate foo 0001 > foo.sql
$ psql dbname username < foo.sql
BEGIN
CREATE TABLE
CREATE INDEX
COMMIT
Another is pipe the SQL directly:
$ python manage.py sqlmigrate foo 0001 | psql dbname username
Or copy and paste it, etc.
pip install django-extensions
and add this in the install app of settings.py
INSTALLED_APPS = [
'django_extensions'
]
Then run
python ./manage.py reset_db
Then run migrations again
python manage.py makemigrations
python manage.py migrate
Now, run migrations for your installed apps
python manage.py makemigrations your_app_name
python manage.py migrtate your_app_name
Done! See Your Database...
In addition to the other answers, make sure that in models.py, you have managed = True in each table's meta
You can remove your db
python manage.py makemigrations
python manage.py migrate
python manage.py migrate --run-syncdb
and see your data base this is working :)
Similar to Andrew E above but with a few changes especially where you haven't deleted the migrations folder in your quest to resolve the issue
1 - In your intact migration folder just examine the 000*.py files counting from the highest down to initial.py till you find the one where your Model is defined, say 0002_entry.py
2 - python manage.py sqlmigrate app-name 0002 > 0002_sql.txt to capture the SQL commands
3 - Edit this file to ensure there are no hard CR/LFs and the ALTER, CREATE INDEX commands are each on own single line
4 - Log into your DB (I have Postgres) and run these commands
In Database delete row myproj from the table django_migrations.
Delete all migration files in the migrations folder.
Then run python manage.py makemigrations and python manage.py migrate commands.
Are there django commands that
A. Delete all tables
B. delete all data in all tables
C. Create all tables as defined in the model?
I cannot find these right now!
And by commands i mean those little things that are like
runserver
etc
A. Delete all tables
manage.py sqlclear will print the sql statement to drop all tables
B. delete all data in all tables
manage.py flush returns the database to the state it was in immediately after syncdb was executed
C. Create all tables as defined in the model?
manage.py syncdb Creates the database tables for all apps in INSTALLED_APPS whose tables have not already been created.
See this page for a reference of all commands: https://docs.djangoproject.com/en/dev/ref/django-admin/
But you should definitely look into using south as someone already mentioned. It's the best way to manage your database.
N.B: syncdb is deprecated in favour of migrate, since Django 1.7.
If you have the client libraries installed for your database, you can run this:
python manage.py sqlflush | python manage.py dbshell
This doesn't drop the tables, but truncates them.
There isn't a command that does the it all in one go, but this "one liner" will drop all the tables and then re-create them. It would only work if you were running on a system that provides these utilities at the shell.
echo 'from django.conf import settings; print settings.INSTALLED_APPS; quit();' | python manage.py shell --plain 2>&1 | tail -n1 | sed -r "s|^.*\((.*)\).*$|\1|; s|[',]| |g; s|django\.contrib\.||g" | xargs python manage.py sqlclear | python manage.py dbshell && python manage.py syncdb
Neither manage.py sqlclear nor manage.py reset is capable of dropping all tables at once, both require an appname parameter.
You should take a look at Django Extensions, it gives you access to manage.py reset_db as well as many other useful management commands.
I recommend using django-south. It allows you to sync your models to your database not just when you add a field, but also when you delete a field/model. I really find it to be an essential component of building a Django site. Once installed, you could run a command like:
./manage.py migrate app zero
You can learn more here: http://south.aeracode.org/docs/about.html
And a simpler oneliner to drop all the tables for django 1.5+:
python2 manage.py sqlflush | sed 's/TRUNCATE/DROP TABLE/g'| python2 manage.py dbshell
I was able todrop my tables. Run
python manage.py sqlclear appname
Take note of the commands given. Then run
python manage.py dbshell
Add the commands given in the first step. line by line. When you are done, type '.exit' (for SQlite3). Resync your DB and you should be good to go. To be sure, check the tables with:
python manage.py shell
>>> from yourapp import yourclasses
>>> yourviews.objects.all()
it should return a []. I hope this helps.
A implies B, right?
For A, see How to drop all tables from the database with manage.py CLI in Django?
For C,
python manage.py syncdb
Ofcourse for smart data migration I go with what #bento mentioned: django-south