I am trying to write a migration which grants readonly permissions to each schema in my multi-tenant postgres DB.
The migrations run once per schema, so what I would like to do would be capture the name of the schema for which it is running, and then use that schema_name in my SQL statement to grant permissions for that schema.
In django, I can create a migration operation called 'RunPython', and from within that python code I can determine for which schema the migrations are currently running (schema_editor.connection.connection_name).
What I want to do is pass that information to the next migration operation, namely "RunSQL", so that the SQL I run can be:
"GRANT SELECT ON ALL TABLES IN SCHEMA {schema_name_from_python_code} TO readaccess;"
If anyone can shed any light on this issue it would be greatly appreciated. Cheers!
I was able to figure this out by getting rid of the migrations.runSQL. I just have migrations.RunPython. From within that python forward_func I am able to access the DB and write sql there (with the necessary string interpolation)
:)
Related
We have noticed that if a table is empty in SQL Server, the empty table does not come via DMS. Only after inserting a record it starts to show up.
Just checking, is there a way to get the schema only from DMS?
Thanks
You can use Schema conversion tool for moving DB objects and Schema. Its a free tool by AWS and can be installed on On-Prem server or on EC2. It gives a good report before you can actually migrate the DB schema and other DB objects. It shows how many Tables, SP's Funcs etc can be directly migrated and shows possible solutions too.
We currently use the following mechanism to create a BigQuery table with a pre-defined schema and we created the infrastructure.
https://www.terraform.io/docs/providers/google/r/bigquery_table.html
The dev team decided to modify the schema by adding another column, so we are planning to modify the schema changes in the above terraform script to enable this.
What would be the best way to manage such schema migrations in production environments?
Since in a production environment, we would be expected to retain the table data while the schema migration is performed
It seems you cannot modify the schema of the table and retain data using Terraform. Instead you can use bq command-line for the same. https://cloud.google.com/bigquery/docs/managing-table-schemas#bq.
Looks like there was a fix for it -
https://github.com/hashicorp/terraform-provider-google/issues/8503
I'm working in a development environment on a flask-app with a Postgres 10 database that has ~80 tables. There are lots of relationships and ForeignKeyConstraints networking it all together.
It was working fine with Flask-Migrate. I'd bootstrapped and migrated up to this point with ~80 tables. But, I wanted to test out some new scripts to seed the database tables, and thought it would be quickest to just drop the database and bring it back up again using Flask-Migrate.
In this process, the migration folder was deleted, so I just started over fresh with a db init. Then ran db migrate. I manually fixed a few imports in the migrate script. Finally, I ran db upgrade.
However, now with all these 80 create_table commands in my migrate script, when I run db_upgrade, I receive an error:
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) relation "items" does not exist
I receive this error for every Child table that has a ForeignKeyConstraint if the Child table is not in an order which is below the Parent table in the migration file.
But, the autogenerated script from db migrate has the tables sorted alphabetically, ordered by table name.
Referring to documentation, I don't see this importance of sort order mentioned.
Bottom line is, it seems I'm either forced to write a script to sort all these tables in an order where the Parent table is above the Child table. Or else, just cut and paste like a jigsaw puzzle until all the tables are in the required order.
What am I missing? Is there an easier way to do this with Flask-Migrate or Alembic?
After researching this, it seems flask-migrate and/or Alembic does not have any built-in methods to resolve this sort order issue. I fixed it by cutting and pasting the tables in an order which ensured the Parent table was above the child tables in the migration file.
I've just encountered this myself, and could not find a better and/or official answer.
My approach was to separate the table creation from the creation of foreign key constraints:
Edit Alembic's auto-generated migration script: In each table create operation, remove all lines creating foreign key constraints
Run Alembic's upgrade command (tables are created, minus the FK constraints, of course)
Run Alembic's migrate command (additional migration script created, that adds all FK constraints)
Run Alembic's upgrade command (FK constraints added to tables)
I've faced some problems when using flask-sqlalchemy and flask-migrate, I solved it using python interactive shell.
>>> from yourapp import db, create_app
>>> db.create_all(app=create_app())
Check this link to get more information.
Happy coding...
I am using Google Cloud Datastore(not NDB) for my project.
python2.7 and Django.
I want to create a new model, lets say Tag model.
class Tag(db.Model):
name = ndb.StringProperty()
feature = ndb.StringProperty(default='')
I have added property to a model many times, but not yet created new model.
My question is when I have changed model schema in Django for my another project using mySQL, I always executed manage.py migrate.
Do I have to execute the migration command for Datastore as well?
Or just defining the model is all I have to do?
Thanks in advance!
Unlike SQL databases like MySQL, Cloud Datastore doesn't require you to create kinds (similar to tables) in advance. Other than defining it in your code, no admin steps are required to create the kind.
When you write the first entity of that kind, it's created implicitly for you.
You can even query for kinds that don't exist yet without an error, you'll just get no entities back:
Of course you have to migrate, except if you are using the same database from the another project. Anyway if you type migrate it will create the tables from your models but if you are working with a existing database nothing is going to happen
I am doing a poc in Django and i was trying to create the admin console module for inserting,updating and deleting records through django admin console through models and it was doing fine
I have 2 questions.
1.I need to have model objects for existing tables which needs to be present in a particular schema.say schema1.table1
Here as of now i was doing poc for public schema.
So can it be done in a fixed defined schema and if yes how.Any reference would be very helpful
2.Also i wanted to update few columns in the table through console and the rest of the columns will be done automatically like currentimestamp and created date etc.Is it possible through default django console and if yes kindly share any reference
Steps for 1
What i have done as of now is created a class in model.py with attributes as author,title,body,timeofpost
Then i used sqlmigrate after makemigrations app to create the table and after migrating have been using the admin console for django to insert and update the records for the table created.But this is for POC only.
Now i need to do the same but for existing tables with whom i can interact and insert or update record for those existing tables through admin console.
Also the tables are getting created in public schema by default.But i am using postgres and the existing tables are present in different schemas and i wanted to insert,update and delete for this existing tables.
I am stuck up here as i dont know how to configure model with existing database schema tables through which we can interact through django console and also for different schemas and not in public schema
Steps for 2:
Also i wanted the user to give input for few columns like suppose in this case time of creation is not required to be given as input by user .Rather it should be taken care when the database is updating or creating
Thanks
In order for Django to "interact" with an existing database you need to create a model for it which can be done automatically as shown here. This assumes that your "external" database isn't going to be changed often because you'll have to keep your models in sync which is tricky - there are other approaches if you need that.
As for working with multiple database schemas - is there a reason you can't put your POC table in the same database as the others? Django supports multiple databases, but it will be harder to setup. See here.
Finally, it sounds like you are interested in setting the Django default field attribute. For an example of current time see here.