How to get my database to reflect changes made to models.py? - django

I made a change to my models, and I cannot for the life of me figure out how to get my database to reflect my models. So far I have tried the following:
python manage.py shelldb
SELECT * FROM sqlite_master WHERE type='table';
DROP TABLE appname_modelname;
When I tried that I got: "unknown command: shelldb".
I also tried:
python manage.py dbshell
DROP TABLE accounts_userreview;
No error, but my db still doesn't reflect my models.
Finally, I just altogether deleted my database by dragging it into the trash and then doing syncdb, it then made me create a new superuser, but STILL the database that was created does not reflect the changes to my models.
I'm at a loss here, What else can I do here? Also, I'm new to learning Django, is there some kind of layer in between the models and my database? I would assume there is since deleting and then rebuilding the database didn't work.
Would appreciate any advice here.

if you are on Django < 1.7 , you have to use some migration tool i.e. - South . Django 1.7 has inbuilt migration.
For more info migrations

Run these commands in terminal to make migrations for Django 1.7
python manage.py makemigrations
python manage.py syncdb
for Django < 1.7
use South

Related

Django Programming error column does not exist even after running migrations

I run python manage.py makemigrations and I get:
No changes detected
Then, python manage.py migrate and I get:
No migrations to apply.
Then, I try to push the changes to production:
git push heroku master
Everything up-to-date
Then, in production, I repeat the command:
heroku run python manage.py migrate
No migrations to apply.
Just in case, I run makemigrations in production:
heroku run python manage.py makemigrations
No changes detected
WHY then I get a
ProgrammingError at ....
column .... does not exist
"No changes detected" means the database is coherent with the code.
How can I debug this?¡?
I got the same problem (column not exist) but when I try to run migrate not with makemigrations (it is the same issue I believe)
Cause: I removed the migration files and replaced them with single pretending intial migration file 0001 before running the migration for the last change
Solution:
Drop tables involved in that migration of that app (consider a backup workaround if any)
Delete the rows responsible of the migration of that app from the table django_migrations in which migrations are recorded, This is how Django knows which migrations have been applied and which still need to be applied.
And here is how solve this problem:
log in as postgres user (my user is called posgres):
sudo -i -u postgres
Open an sql terminal and connect to your database:
psql -d database_name
List your table and spot the tables related to that app:
\dt
Drop them (consider drop order with relations):
DROP TABLE tablename ;
List migration record, you will see migrations applied classified like so:
id | app | name | applied
--+------+--------+---------+
SELECT * FROM django_migrations;
Delete rows of migrations of that app (you can delete by id or by app, with app don't forget 'quotes'):
DELETE FROM django_migrations WHERE app='yourapp';
log out and run your migrations merely (maybe run makemigrations in your case):
python manage.py migrate --settings=your.settings.module_if_any
Note: it is possible that in your case will not have to drop all the tables of that app and not all the migrations, just the ones of the models causing the problem.
I wish this can help.
Django migrations are recorded in your database under the 'django_migrations' table. This is how Django knows which migrations have been applied and which still need to be applied.
Have a look at django_migrations table in your DB. It may be that something went wrong when your migration was applied. So, delete the row in the table which has the migration file name that is related to that column that 'does not exist'. Then, try to re-run a migration.
Here's what i tried and it worked:
Go and add manually the column to your table
run python manage.py makemigrations
go back drop that column you added
run python manage.py migrate
I had a similar issue - the error message appeared when I clicked on the model on the django-admin site. I solved it by commenting out the field in models.py, then running migrations. Following this I uncommented the field and re ran the migrations. After that the error message disappeared.
My case might be a bit obscure, but if it helps someone, it is worth documenting here.
I was calling a function in one of my migrations, which imported a Model of said migration regularly, i.e.
from myApp.models import ModelX
The only way models should be imported in migrations would be using e.g. RunPython:
def myFunc(apps, schema_editor):
MyModel = apps.get_model('myApp 'MyModel')
and then calling that function like so:
class Migration(migrations.Migration):
operations = [
migrations.RunPython(initialize_mhs, reverse_code=migrations.RunPython.noop),
]
Additionally the original import worked until I modified the model in a later migration, making this error harder to locate.
So, I always run into this sort of problem, so today I decided to try and work it out at the database level. Thing is, I changed a model field name which Django didn't bother reflecting in a migration file. I only found out later when I ran into problems. I later looked at the migration files and discovered there was no migration for that change. But I didn't notice because I made other changes as well, so once I saw a migration file I was happy.
My advice. Create migration for each change one at a time. That way you get to see if it happened or not.
So here's my working through it in MySQL.
open mysql console.
show databases; # see all my dbs. I deleted a few
drop database <db-name>; # if needed
use <db-name>; # the database name for your django project
show tables; # see all tables in the database
DESCRIBE <table-name>; # shows columns in the database
SHOW COLUMNS FROM <db-name>; # same thing as above
ALTER TABLE <table-name> CHANGE <old-column-name> <new-column-name> <col-type>; # now I manually updated my column name
If you're using postgresql, just google the corresponding commands.
The issue was in the Models for me, for some reason Django was adding '_id' to the end of my Foreign Key column. I had to explicitly set the related named to the Foreign Key. Here 'Cards' is the parent table and 'Prices' is the child table.
class Cards(models.Model):
unique_id = models.CharField(primary_key=True, max_length=45)
name = models.CharField(max_length=225)
class Prices(models.Model):
unique_id = models.ForeignKey(Cards, models.DO_NOTHING)
Works after changing to:
class Cards(models.Model):
unique_id = models.CharField(primary_key=True, max_length=45)
name = models.CharField(max_length=225)
class Prices(models.Model):
unique_id = models.ForeignKey(Cards, models.DO_NOTHING, db_column='unique_id')
When I get this error, my extreme way to solve it is to reset my database:
Reset your database
For Postgresql on Heroku:
Heroku > your_app > Resources > database > add-ons > click on your database and open it
For postgresql
settings > Reset database
Delete all files in your_app > migrations > __pycache__ except __init.py__
Delete all files in your_app > migrations except __pycache__ folder and __init.py__
Then run:
python manage.py makemigrations
python manage.py migrate
python manage.py createsuperuser
type in to create your superuser, then run:
python manage.py makemigrations
python manage.py migrate
python manage.py
If you are able to inspect your models from your admin section, then it should be all okay now.
Just remove corresponding row migrations for that model in 'django_migrations' model in database.
And re run python manage.py migrate app_name
I tried all these answers with not much luck! What I did to have this problem solved with no harm was to go back through the migration files and find where the actual model was being created for the first time then manually add the field (in the column not being existed error message). Until if you run makemigrations --dry-run you get/see "No changes detected" and that worked. Basically, in my case, I had to carefully take my desired db changes back in time in proper migration file, rather creating a new migration now at the end of migration dependency chain.
Open the latest py file created after running the makemigrations command inside migrations folder of that particular app.
Inside class Migration there is a list attribute called 'operations'.
Remove the particular elements migrations.RemoveField(...).
Save and run python manage.py migrate.
A easier solution to the problem is to make your models exactly like it is in the migration first. and run python manage.py migrate.
Then revert those changes
Run
python manage.py makemigrations
python manage.py migrate
To check for which migrations are applied and which are not, use -:
python manage.py showmigrations
I solved a similar problem by deleting all migrations files (Don't forget to make a backup) and python manage.py makemigrations all of them into one clean file in development and pulling new files on the server. Before then I had dropped existing tables on the PostgreSQL.
I was getting this error for some reason when Django was looking for a column of type ForeignKey named category_id when the actual name in the database was category. I tried every Django solution I could imagine (renaming field, explicitly setting column name, etc.). I didn't want to drop tables or rows as this was a production database. The solution was simply to rename the column manually using SQL. In my case:
ALTER TABLE table_name
RENAME COLUMN category TO category_id;
Make sure you backup your database, ensure this won't break any other applications consuming that particular table, and consider having a fallback column if necessary.
What helped me in the end was simply dropping the database and creating it again as well as deleting all migrations files (including cache). (only removing migrations file didn't work for me at all)
sudo su - postgres
psql
DROP DATABASE 'yourdatabase';
CREATE DATABASE 'yourdatabase';
GRANT ALL PRIVILEGES ON DATABASE 'yourdatabase' to 'yourdjangouser';
then just
python manage.py makemigrations
python manage.py migrate
python manage.py runserver
If you're in development and you make some examples of data that's not important, this step is beneficial for me: just flush your data, make migrations, and migrate:
python manage.py flush
python manage.py makemigrations
python manage.py migrate
After that, you may create a new database from scratch, I hope this information was helpful.
Solved this issue by running
python manage.py migrate
in Heroku Bash shell

django - schema migration - how to add a field

I have a django 1.8 app working with a db.
I'm trying to change the schema of a table using the built-in migration.
Here are the steps I did:
In my dev invironment, I grabbed the app source and ran
python manage.py sycdb
then I ran
python manage.py loaddata ~/my_data.json
then I modified modes.py. Added a field and renamed a field...all from the same table 'TABLE1' which had no data.
then
python manage.py makemigrations myapp
python manage.py migrate
Error: django.db.utils.OperationalError: table "myapp_someother_table" already exists
then ran
python manage.py migrate --fake-initial
worked!
but when I browsed to the admin page for TABLE1, I get this error:
OperationalError: no such column: myapp_table1.my_new_field_id
I checked the db and yes, there is no such column.
How can I procceed from here? I prefer to fix this via django.
If I fix it straight in the db, then the migration goes out of sync.
Migrations do not automagically see that you have made changes. Migrations detect changes by comparing the current model with the historical model saved in the migration files.
In this case, you didn't have any historical models, since you didn't have any migrations. Django was not able to detect any changes in your models, even though they were different from your database.
The correct way to make changes to your model is to first run manage.py makemigration <my_app>, and then make the changes to your model, followed by another manage.py makemigrations.
You might not be able to do it via pure django and keep your data. I don't have personal experience with south but there are a lot of mentions if this tool. Just in case if nothing else works for you...
Here is what I did to make things work, but there must be a better way so please add more answers/comments...
I deleted the sqlite db and the migration folder
I made the desired changes to model.py
ran syncdb
ran loaddata to load the json data dump that I had saved previously.
just started the dev server

Django 1.7 on Heroku: how do I get makemigrations to rescan the database?

I have Django 1.7 running on Heroku. I've made a change to the models.py file (added a column to a table) but Django doesn't seem to be able to detect this. When I run
python manage.py makemigrations appname
it responds No changes detected in app.
I've tried deleting the appname/migrations folder, but that doesn't help.
Is there a way to get Django to rescan the database and check for differences? This was easy with South.
https://docs.djangoproject.com/en/1.7/topics/migrations/#the-commands
Have you tried
python manage.py migrate
It seems the migrate is "responsible for applying migrations, as well as unapplying and listing their status."

newbie difficulty using south with pycharm - DatabaseError: no such table: south_migrationhistory

I'm using sqlite3 and pycharm to learn more about django, and googled to find that south is recommended to make it easier to modify models after they have been created.
I'm trying to follow the advice on http://south.aeracode.org/docs/tutorial/part1.html#starting-off.
The most success I've had so far is to create a simple model and run syncdb before adding south to installed_apps. That way the intial tables are created and I get a chance to create a super user. (Django admin seems to fret if there are no users).
Then I add south to installed_apps, and run django_manage.py schemamigration bookmarks --initial
It seems to work fine. A new directory is created called migrations with a couple of files in it in my app folder and an encouraging message.
"Created 0001_initial.py. You can now apply this migration with: ./manage.py migrate bookmarks"
The next step - django_manage.py" migrate bookmarks generates the following error message
django.db.utils.DatabaseError: no such table: south_migrationhistory.
I thought that table would be created in the first schememigration step. What am I missing? Can anyone help?
Marg
South uses a table if its own to keep track of which migrations have been applied. Before you can apply any migrations, this must have been created, using python ./manage.py syncdb.
As well as for setting up south, you will find syncdb sometimes necessary for non-south apps in your project, such as the very common django.contrib.auth.
Note that as a convenience, you can run both in one go like this
python ./manage.py syncdb --migrate
My latest (unsuccessful) effort was the following
Create application – synch db – superuser created
Test run –admin screen shows basic tables
Add south, and syncdb from command line with manage.py syncdb – south_migrationhistory table created. Add basic vanilla model
Tried various combinations of manage.py syncdb –manage, and
schemamigration from Pycharm (if run from within pycharm a
migrations directory is created within the app
– if run from the command line the directory does not seem to be
created.)
Django admin screen shows table – but if I try to edit
the table it says that it doesn’t exist
Check database structure
using SQLite browser - table for newly created model doesn’t exist
I’m starting to think that the whole thing is not worth the time wasting hassle – maybe I’m better off just modifying the tables in SQLite browser
Answer in the similar question:
Run syncdb to add the Django and South tables to the database.

Adding fields to an already existing database for Django (version < 1.7)

I'm using Django ( < v1.7), with SQLite3 as the database engine.
I am trying to add a new field to an already existing model's class. This class already has data associated to it.
manage.py syncdb is not adding the new fields to the db.
Is there was a way to:
Add the new Field to the already existing class, and make sure a db entry is allocated to it?
Populate this new field with an empty string for all the already existing db entries?
This answer is still getting visibility but is outdated. Since 1.7 Django ships with a built-in migration system, written by the same author as South, and has deprecated syncdb though it will still work.
You will simply need to run a few commands to automatically add new columns:
python manage.py makemigrations
python manage.py migrate
It will be useful to understand what's happening under the hood, but those are the basics. Please ask new questions to get answers on 1.7 and migrations if you are still reading this old post.
For django < 1.7
syncdb will not add any new columns. See
http://docs.djangoproject.com/en/dev/ref/django-admin/#syncdb
You will have to add them manually. For example,. replace <> with relevant info:
python manage.py dbshell
ALTER TABLE <appname_modelname> ADD COLUMN <column_type> DEFAULT '';
You can see what Django might have done to generate columns on a fresh syncdb by using:
python manage.py sqlall app_name
and copying and pasting ALTER TABLE statements from there.
Otherwise, you can look into third-party apps like Django-South which are database migration tools.
Install south in your django and you can easily handle the existing tables. check this
If you really want to use Django-South, install it on your django, after adding your new fields to your existing model run
python manage.py schemamigration --initial
It will create a file in your project app. then,
python manage.py migrate
thats it your table is altered.
There are other options of migration apps (although South is the most used).
I have used django-evolution for my projects and it was very easy to install and start using.
South seems to be more complete, but for simpler tasks, django-evolution may be suitable.