I have some existing SQL statements that I'd like to use as a custom hook after the CREATE TABLE command, in [myapp]/sql/[model-name].sql.
My app is in INSTALLED_APPS. I see listed if I run ./manage.py sql.
My custom hook is found; I see the SQL statements output if I run any of the following:
./manage.py sqlall <myapp>
./manage.py sqlcustom <myapp>
./manage.py sql <myapp>
I'm using postgres 9.x on my mac.
If I psql to that same database (with no user) and copy them from the .sql file and paste them into the psql command input, they all work... so I believe they're valid SQL understood by postgres. These are all pretty simple INSERT statements (fixtures addressed below).
However, if I run ./manage.py syncdb those statements are either not run, or they are ignored or silent errors happen; all I know is that the new rows do not appear in the database. I am tailing the postgres log file and nothing is logged when I run syncdb, so I don't know if it's not finding my .sql file, or parsing it and finding some error before it gets to the database.
I have created a .json file, for fixtures, with the equivalent of those statements, and ./manage.py loaddata <path-to-json-file> works correctly: my site now shows those values in the database. This makes me believe that my settings file is correct and the database I'm writing to inside postgres is set correctly, and I have write permissions when I run ./manage.py.
I saw in some other post that the django documentation is wrong and I should put the custom hook in the 'models' directory, but I don't know if that's right; if sqlall and sqlcustom find my hook, shouldn't syncdb find it? Also I don't (yet) have a models directory and may not need it.
For various reasons I'd rather not use JSON format, but if I have to I will... however I've invested so much time in the .sql format I really want to know what's going on (and I've seen enough existing related questions that this might help others).
I believe I found it, although it's based on behaviors not any real research. I simply changed 'tile' to 'tilexx' everywhere and it worked. This django-project post indicates that if there is some sort of python class name conflict the custom SQL won't be executed... and 'tile' is a pretty common thing.
So the answer is to change the name of my class to something a bit more unique.
I've been searching for an answer to a similar problem, trying to initialize an sqlite database with data I dumped from a Flask application I'm porting to Django. Like the OP, all of the following did the right thing:
./manage.py sqlall <myapp> ./manage.py sqlcustom <myapp> ./manage.py sql <myapp>
However, the insert statements in myapp/sql/myapp.sql were not being run. A careful reading of the documentation revealed this clue:
Django provides a hook for passing the database arbitrary SQL that’s executed just after the CREATE TABLE statements when you run syncdb.
(emphasis added)
The issue is that I had already registered my models and run syncdb, so the table in question already existed in the database, although it held no data yet. I deduce that because of this, the CREATE TABLE statement was not being run on subsequent executions of syncdb, and therefore, my custom sql could not be run after that statement. The solution was to DROP table table_name and then run syncdb again, at which point my custom sql was run.
Related
I am running a GIS application while getting this error. I have attached error snapshot . Can Someone guide where is error ? If need to see code . Let me know which file you need.
Especially the first line of your exception value pretty much says it all.
Column users.parent_id does not exist
The application is trying to access parent.id from the users table from your database, which obviously does not exist. With other words, your database is not sync with the model structure in the source code. Probably all you have to do, is to run the migrations to add all the missing structures or changes to your database.
If you've developed some of the database structures yourself, you have to run the makemigrations command to build the new migration set for your database.
./manage.py makemigrations
If you have created new migration files or if you have installed and integrated modules into your app, you have to apply the migrations to your database.
./manage.py migrations
I have an app that I recently did a migration for that included adding a new table. I can see that the migration worked, the table is populated, I can access it, etc. Even when I run
python manage.py migrate my_app -l
I can see the migration as having been successfully run.
However when I run my unit tests I get:
OperationalError: no such table: my_app_my_table
The only thing I can thing of is that the in memory database is not running all the migrations? Or it's not doing it correctly? Any help would be greatly appreciated.
It could be the case that your migrations aren't actually generating the SQL code you intended, and it's probably worth running sqlmigrate on the specific migrations to make sure it's generating code to make the table.
If the SQL code looks right, something weird could have happened and django thinks you ran the migration but it didn't actually change the database. In that case, you could try restoring an older version of the database and running the migrations on that?
This is a very basic question, but the other SO questions I read do not seam to answer it.
I checked out a project from a repository which contains some models some of them having migrations files.
I did not create any local database yet
What is the correct way of creating all the tables for the project?
I thought I start by executing
./manage.py syncdb
this creates some tables but for the once using south it tells me to run migrate. So I execute
./manage.py migrate
but then I get the error that
DatabaseError: table "mytable" already exists
I usually use ./manage.py syncdb --migrate and it does not throw that kind of error.
Are you sure that none of you other apps use a table with the same name?
You can checkout which tables your app wants to create by opening a django shell and doing sth like:
./manage.py sql YourAppName
I'm using sqlite3 and pycharm to learn more about django, and googled to find that south is recommended to make it easier to modify models after they have been created.
I'm trying to follow the advice on http://south.aeracode.org/docs/tutorial/part1.html#starting-off.
The most success I've had so far is to create a simple model and run syncdb before adding south to installed_apps. That way the intial tables are created and I get a chance to create a super user. (Django admin seems to fret if there are no users).
Then I add south to installed_apps, and run django_manage.py schemamigration bookmarks --initial
It seems to work fine. A new directory is created called migrations with a couple of files in it in my app folder and an encouraging message.
"Created 0001_initial.py. You can now apply this migration with: ./manage.py migrate bookmarks"
The next step - django_manage.py" migrate bookmarks generates the following error message
django.db.utils.DatabaseError: no such table: south_migrationhistory.
I thought that table would be created in the first schememigration step. What am I missing? Can anyone help?
Marg
South uses a table if its own to keep track of which migrations have been applied. Before you can apply any migrations, this must have been created, using python ./manage.py syncdb.
As well as for setting up south, you will find syncdb sometimes necessary for non-south apps in your project, such as the very common django.contrib.auth.
Note that as a convenience, you can run both in one go like this
python ./manage.py syncdb --migrate
My latest (unsuccessful) effort was the following
Create application – synch db – superuser created
Test run –admin screen shows basic tables
Add south, and syncdb from command line with manage.py syncdb – south_migrationhistory table created. Add basic vanilla model
Tried various combinations of manage.py syncdb –manage, and
schemamigration from Pycharm (if run from within pycharm a
migrations directory is created within the app
– if run from the command line the directory does not seem to be
created.)
Django admin screen shows table – but if I try to edit
the table it says that it doesn’t exist
Check database structure
using SQLite browser - table for newly created model doesn’t exist
I’m starting to think that the whole thing is not worth the time wasting hassle – maybe I’m better off just modifying the tables in SQLite browser
Answer in the similar question:
Run syncdb to add the Django and South tables to the database.
After making some changes in my models (eg. new field in a model and a new model) what is the best way of reflecting these changes to my populated database?
PS: I wanted to see many solutions in one place rated. Apparently more solutions are already listed here.
Another technique is to use the dumpdata and loaddata arguments to manage.py, killing your database in-between:
python manage.py dumpdata > dump.json
With an external tool, drop any affected tables, or kill the whole db
python manage.py loaddata dump.json
See manage.py docs for more.
I've asked a similar question here and got quite a few answers.
There are quite a lot of ways of doing it, like manually doing the dumping and reloading with SQL, using fixtures or using one of the "emerging" schema-evolution packages for Django:
Django Evolution
South
dmigrations
(there's a DjangoCon video of a panel on schema-evolution in Django where these 3 solutions are discussed)
Depends on the scope of the changes. If it's beyond an ALTER, you're doing major surgery. Make backups of model as well as database so you can go back.
My preference is to put your new (revised, corrected, expanded) model in as a NEW application. It won't have URL's or anything, just a model.
Creating the new model as a new application. Create tests, just to be sure it works.
syncdb to build this interim implementation of the new model.
Write a little one-time utility to query your old model, and load your new model. You might want to try this in pure SQL. I prefer to write a simple query, build and save loop.
After the new model is loaded, you can dump this to a JSON file.
Once you've pulled the data out of your old model, you can rebuild your DB in the preferred new format.
Move the new model into your existing application.
Drop the old versions of the application's tables.
syncdb to build the new tables.
Load the JSON file with the data.
Django now has its own built-in migrations, documented at:
https://docs.djangoproject.com/en/dev/topics/migrations/
Look with manage.py sqlall what the parameters are for the new columns and manually add them in your database with Alter table statements. This way you don't have to redo your database; It requires some SQL knowledge though...
Take a look here (Scroll down to "Making Changes to a Database Schema")
Perform these steps in order may help you:
For more details,
clickhere: http://south.readthedocs.org/en/latest/
1) python manage.py schemamigration apps.appname --initial
Above step creates migration folder as default.
2) python manage.py migrate apps.appname --fake
generates a fake migration.
3) python manage.py schemamigration apps.appname --auto
Then you can add fields as you wish and perform the above command.
4) python manage.py migrate apps.appname
Then migrate the files to the database.