Background:
I'm dealing with quite complex django application and I'm looking for a way to make my life a little bit easier.
One of the models (tables) serves as source of "options" for most "select lists" in the system (simple field <-> dictionary construct). It should always be populated with the default values and/or extenden when the there are new entries provided with a new version of the system.
I manage it all now manually so:
I run migrations for the main app
Unload the dictionary tables from the dev enviroment
Insert them manually into the enviroment I'm currently upgrading
I migrate the rest of the apps which may contain refferences to the dictionary tables
All of this because of default values provided in ForeignKeys.
Actual Question:
Is it possible to add table entries (table content) to the makemigrations process for particular tables??
Yes, you can do anything you like in a migration via the RunPython operation. This is called a data migration and is fully documented.
Related
I am currently working on a Django project with around 30 models and there are lots of relations(For example, foreign key relations) between the models.
My doubt is "After 6 months, let's say I want to add a new field(s) in one of the model/table(s) in models.py, and make migrations, the new migration files will get created without affecting the initial migration files which were created 6 months ago."
Will the relations be maintained after adding new columns in different tables? (or) do I have to go to pgadmin console and tweak them accordingly?
One way is to erase all the migrations and start fresh, but this is not recommended often especially if there is production data (or) there are frequent changes in the database schema.
#Mahesh, You may use --fake-initial to avoid the existing tables error at the time of new migrations(When you want to add new column).
Relation will be maintained since you already declared it and unless you change it in a new migration.
How to add a new field to a model with new Django migrations?
And in Docs: https://docs.djangoproject.com/en/3.0/ref/django-admin/#cmdoption-migrate-fake-initial
If you don't change Django version, adding new fields on models will not create any problem, even after many years. But, there are some situations this might create problems. For example, if Django is updated and you have installed the latest version.
I'm working in a development environment on a flask-app with a Postgres 10 database that has ~80 tables. There are lots of relationships and ForeignKeyConstraints networking it all together.
It was working fine with Flask-Migrate. I'd bootstrapped and migrated up to this point with ~80 tables. But, I wanted to test out some new scripts to seed the database tables, and thought it would be quickest to just drop the database and bring it back up again using Flask-Migrate.
In this process, the migration folder was deleted, so I just started over fresh with a db init. Then ran db migrate. I manually fixed a few imports in the migrate script. Finally, I ran db upgrade.
However, now with all these 80 create_table commands in my migrate script, when I run db_upgrade, I receive an error:
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) relation "items" does not exist
I receive this error for every Child table that has a ForeignKeyConstraint if the Child table is not in an order which is below the Parent table in the migration file.
But, the autogenerated script from db migrate has the tables sorted alphabetically, ordered by table name.
Referring to documentation, I don't see this importance of sort order mentioned.
Bottom line is, it seems I'm either forced to write a script to sort all these tables in an order where the Parent table is above the Child table. Or else, just cut and paste like a jigsaw puzzle until all the tables are in the required order.
What am I missing? Is there an easier way to do this with Flask-Migrate or Alembic?
After researching this, it seems flask-migrate and/or Alembic does not have any built-in methods to resolve this sort order issue. I fixed it by cutting and pasting the tables in an order which ensured the Parent table was above the child tables in the migration file.
I've just encountered this myself, and could not find a better and/or official answer.
My approach was to separate the table creation from the creation of foreign key constraints:
Edit Alembic's auto-generated migration script: In each table create operation, remove all lines creating foreign key constraints
Run Alembic's upgrade command (tables are created, minus the FK constraints, of course)
Run Alembic's migrate command (additional migration script created, that adds all FK constraints)
Run Alembic's upgrade command (FK constraints added to tables)
I've faced some problems when using flask-sqlalchemy and flask-migrate, I solved it using python interactive shell.
>>> from yourapp import db, create_app
>>> db.create_all(app=create_app())
Check this link to get more information.
Happy coding...
I am doing a poc in Django and i was trying to create the admin console module for inserting,updating and deleting records through django admin console through models and it was doing fine
I have 2 questions.
1.I need to have model objects for existing tables which needs to be present in a particular schema.say schema1.table1
Here as of now i was doing poc for public schema.
So can it be done in a fixed defined schema and if yes how.Any reference would be very helpful
2.Also i wanted to update few columns in the table through console and the rest of the columns will be done automatically like currentimestamp and created date etc.Is it possible through default django console and if yes kindly share any reference
Steps for 1
What i have done as of now is created a class in model.py with attributes as author,title,body,timeofpost
Then i used sqlmigrate after makemigrations app to create the table and after migrating have been using the admin console for django to insert and update the records for the table created.But this is for POC only.
Now i need to do the same but for existing tables with whom i can interact and insert or update record for those existing tables through admin console.
Also the tables are getting created in public schema by default.But i am using postgres and the existing tables are present in different schemas and i wanted to insert,update and delete for this existing tables.
I am stuck up here as i dont know how to configure model with existing database schema tables through which we can interact through django console and also for different schemas and not in public schema
Steps for 2:
Also i wanted the user to give input for few columns like suppose in this case time of creation is not required to be given as input by user .Rather it should be taken care when the database is updating or creating
Thanks
In order for Django to "interact" with an existing database you need to create a model for it which can be done automatically as shown here. This assumes that your "external" database isn't going to be changed often because you'll have to keep your models in sync which is tricky - there are other approaches if you need that.
As for working with multiple database schemas - is there a reason you can't put your POC table in the same database as the others? Django supports multiple databases, but it will be harder to setup. See here.
Finally, it sounds like you are interested in setting the Django default field attribute. For an example of current time see here.
I am new to Django. I have completed the tutorial and am reading the documentation for more learning. As I try to add to my understanding, say, in new Managers or ModelForms I am curious as to what needs South (or even just scrapping it and rewriting the app).
update django database to reflect changes in existing models
The link above says basically that any column change it is necessary, while the link below is more of what I am asking. Can someone generalize when it is not needed (eg: Adding a new Form/ModelForm based on an existing Model? Adding a Manager?) If no changes are made to the columns of the database South is not then not necessary?
Does changing a django models related_name attribute require a south migration?
There are two types of south migrations: schema and data.
Data migrations are used to change data in the DB and not the schema of the DB.
The schema migrations are the ones you are interested in. They are used to keep track of changes to the DB schema and one should accompany any changes to your models that result in DB schema change (create table, drop table, drop column, change null constrain e.g.)
Some great insights might be found if you read two consecutive migrations for a django app.
In each of them you can find code that applies the migreation, code that reverts the migrations and a snapshot of the DB schema.
P.S. It is quite easy to check if south migration is needed for a particular change in your models. Just run a schemamigration for the modified django app and delete the newly created migration if such was created. As creating a south migration is different that running it, this is a great way to test and learn.
Keep in mind that south is a piece of software like any other and it does 'support' bugs.
Related_name attribute changes only affect your project and django uses it to make queries.
Changes like blank = True/False, null = True/False, symmetrical = True/False require database changes although symmetrical = True/False does not trigger update by south, but the setting definitely makes difference at field creation.
Column changes, like the link in your post shows, require updates in database and that's what south does very good.
I just tested it myself. I had Django models, and there have already been instances of the models in the database.
Then I added a dummy integer field to a model and ran manage.py syncdb. Checked the database, and nothing happened to the table. I don't see the extra field added in.
Is this the expected behavior? What's the proper way of modifying the model, and how will that alter the data that's already in the database?
Django will not alter already existing tables, they even say so in the documentation. The reason for this is that django can not guarantee that there will be no information lost.
You have two options if you want to change existing tables. Either drop them and run syncdb again, but you will need to store your data somehow if you want to keep it. The other options is to use a migrations tool to do this for you. Django can show you the SQL for the new database schema and you can diff that to the current version of the database to create the update script.
You could even update your database mannually if it is a small change and you don't want to bother with migrations tools, though I would recommend to use one.
Please use south for any kind of changes to get reflected to your database tables,
here goes the link for using south
Link for South documentation