I'm pretty new to Django and it's deployment on Heroku.
I've got a Postgres database up and running on the app server. My app requirements need me to add a new column to my existing database which has a sizable amount of data in it, which I can't lose.
Looking around, I found a solution described by Mike Ball here.
I have the following queries though:
What exactly is South? (I read the docs but didn't get a clear idea)
Will it help me save and move my existing data from my current database?
As a complete newbie, is the above link an easy way to move the data?
Also, in general, if you could hook me up with a good guide for general DBMS concepts, I'd be very grateful.
Thanks!
What exactly is South? (I read the docs but didn't get a clear idea)
south migrates your database schema as it changes in time. the schema has to start with a django models.py file. If you use 'manage.py syncdb ...' to create your database then you can probably use south.
Will it help me save and move my existing data from my current database?
As long as you used syncdb to create your database using a models.py file in django, then south can change that database and add the new column. basically, south records the changes you make to the models.py file in migration files, then you can apply those migration files to your database which update it non-destructively.
As a complete newbie, is the above link an easy way to move the data?
south doesn't move your data. it allows you to add the new columns to your existing database without destroying the database. to move your data you will need to backup the data to a file, then copy the file to another machine, then restore the backup. That's not what south does.
Related
currently I am using PostgreSQL database in my project but I also want to use SQLite for localhost, so I want to run migrate command but there are errors because in SQLite array field is not used so I want to convert array field to JSONfield and makemigrations but in migrations old migrations also present. S I want to write custom logic in migrations. So, it use old migrations when database is PostgreSQL and new migrations when it is sqlite3.
I don't want create new migrations and migration table every time I switch databases.
SQLite is more of a flat file system. I think the original idea is that you can store a small amount of data on a device and update the main database, or fetch info from a database, when the device is 'idle' as a background process. I know there may be some people putting this comment down but essentially SQLite is 'Light' and a flat file. Those considerations should be taken into account. btw I see that there is MYSQL for Andriod but I have not tried it out.
I migrated a database that was manipulated via SQL to use Django's migration module: https://docs.djangoproject.com/en/3.0/topics/migrations/.
Initially I thought of using migrations only for changes in the model such as changes in columns or new tables, performing deletes, inserts and updates through SQL, as they are constant and writing a migration for each case would be a little impractical.
Could using the migration module without using data migrations cause me some kind of problem or inconsistency in the future?
You can think about data migration if you made a change that required also a manual fix on the data.
Maybe you decided to normalize something in the database, e.g. to split a name column to the first name and the last name. If you have only one instance of the application with one database and you are the only developer then you will also not write a data migration, but if you want to change it on a live production site with 24 x 7 hours traffic or you cooperate with other developers then you probably prepare a data migration for their databases or you will thoroughly test the migration on a copy of live data that the update will work on the production site correctly without issues and with minimal shutdown. If you don't write a data migration and you had no problem immediately then it is OK and will be not worse then a possible ill-conceived data migration.
Situation: a project I'm working in had a file corruption or something. Models are the latest version, but the SQLite DB had to be rolled back before some columns were added/removed/modified. Migration files are all gone. Trying to create migrations anew results in the new columns being present in the initial migration file, so I can neither migrate for real (due to the table existing) nor fake it (since the columns are missing in the DB).
Given those circumstances, how can I make an initial migration matching the columns currently present in the DB so I can fake it and then make a real second migration to bring the tables in line with the models? The only thing that comes to mind is manually tweaking the models to match the DB schema, making the initial migration, faking it and then restoring the new version of the models, but I'd much prefer having this done automatically.
django inpsectdb to the rescue.
But first, learn how to use git if you had used proper version control, you would not be facing this difficulty now.
First step, add the code to version control.
Delete the existing models files
Use inspectdb to generate a models.py from the tables in the database. This is not perfect, you will have to edit the file manually and you may have to spread it out between different models files manually.
Now delete the contents of the migrations table
do a ./manage.py makemigrations (yourapp)
Do the fake migration you mentioned
replace the generated models.py with your current models.py (a git checkout of that file will do the trick nicely)
do a makemigrations and migrate again.
good luck.
All I did was add a field to a model, and now I get an error that says this column does not exist.
In an attempt to rebuild the database I used -flush (i dont care about losing the data), thinking this would rebuild the database, but I still get the same error.
I was told by someone else to use South because I'm running Django 1.6.
I followed the tutorial and literally the first time I ran syncdb, I got the following (probably unrelated) error:
dist-packages/easy-thumbnails/
raise Improperly_Configured(SOUTH_ERROR_MESSAGE)
django.core.exceptions.ImproperlyConfigured:
For South Support, customize the SOUTH_MIGRATION_MODULES setting like so:
South_Migration_Module = {
'easy_thumbnails': 'easy_thumbnails.south_migrations',
}
Ultimately all I want to do is have my db reflect my models. Back when I was working on this project in my dev environment I would literally just drag my sqlite file to the trash and then run syncdb, but I cannot do that now because I'm using postgres.
So my question is how can I accomplish this seemingly simple task? Whether that means addressing the South error, or just not using South altogether (which I would prefer), I would appreciated any help.
You can manually open up a db shell (>>> python manage.py dbshell) and drop the table with DROP TABLE <table_name>;. In order for syncdb to recreate a table with the new field, it is not enough to empty the table.
To solve the error, just do exactly what it says: add the SOUTH_MIGRATION_MODULES setting to your settings.
With that said, I'd definitely advice you to use south. It eases making changes to your models, and it allows you to preserve test data in your development environment that you'd otherwise have to recreate. That's all nice, but when your project goes live, it is absolutely mandatory that your data is preserved when making a change to your models. South is the best tool for that.
South has been such an integral part of pretty much any Django project, that Django has worked together with the developers of South to include it as a core feature from Django 1.7 onwards.
I just tested it myself. I had Django models, and there have already been instances of the models in the database.
Then I added a dummy integer field to a model and ran manage.py syncdb. Checked the database, and nothing happened to the table. I don't see the extra field added in.
Is this the expected behavior? What's the proper way of modifying the model, and how will that alter the data that's already in the database?
Django will not alter already existing tables, they even say so in the documentation. The reason for this is that django can not guarantee that there will be no information lost.
You have two options if you want to change existing tables. Either drop them and run syncdb again, but you will need to store your data somehow if you want to keep it. The other options is to use a migrations tool to do this for you. Django can show you the SQL for the new database schema and you can diff that to the current version of the database to create the update script.
You could even update your database mannually if it is a small change and you don't want to bother with migrations tools, though I would recommend to use one.
Please use south for any kind of changes to get reflected to your database tables,
here goes the link for using south
Link for South documentation