I was wondering this could produce any problem if I directly add rows or remove some from a model table. I thought maybe Django records the number of rows in all tables? or this could mess up the auto-generated id's?
I don't think it matters but I'm using MySql.
No, it's not a problem because Django does the same that you do "directly" to the database, it execute SQL statements, and the auto generated id is handled by the database server (MySql server in this case), no matter where that SQL queries comes from, whatever it is Mysql Client or Django.
Since you can have Django work on a pre-existing database (one that wasn't created by Django), I don't think you will have problems if you access/write the tables of your own app (you might want to avoid modifying Django's internal tables like auth, permission, content_type etc until you are familiar with them)
When you create a model through Django, Django doesn't store the count or anything (unless your app does), so it's okay if you create the model with Django on the SQL database, and then have another app write/read from that same SQL table
If you use Django signals, those will not be triggered by modifying the SQL table directly through the DB, so you might want to pay attention to side effects like that.
Your RDBMS handles it's own auto generated IDs and referential integrity, counts etc, so you don't have to worry about messing it up.
Related
i'm looking for a "best-practice" guide/solution to the following situation.
I have a Django project with a MySql DB which i created and manage. I have to import data, every 5 minutes, from a second (external, not managed by me) db in order to do some actions. I have read rights for the external db and all the necessary information.
I have read the django docs regarding the usage of multiple database: register the db in settings.py, migrate using the --database flag, query/access data by routing to the db (short version) and multiple question on this matter on stackoverflow.
So my plan is:
Register the second database in settings.py, use inspectdb to add to the model, migrate, define a method which reads data from the external db and add it to the internal (own) db.
However I do have some questions:
Do i have to register the external db if i don't manage it?
(Most probably yes in order to use ORM or the cursors to access the data)
How can i migrate the model if I don't manage the DB and don't have write permissions? I also don't need all the tables (around 250, but only 5 needed).
(is fake migration an option worth considering? I would use inspectdb and migrate only the necessary tables.)
Because I only need to retrieve data from the external db and not to write back, would it suffice to have a method that constantly gets the latest data like the second solution suggested in this answer
Any thoughts/ideas/suggestions are welcomed!
I would not use Django's ORM for it, but rather just access the DB with psycopg2 and SQL, get the columns you care about into dicts, and work with those. Otherwise any minor change to that external DB's tables may break your Django app, because the models don't match anymore. That could create more headaches than an ORM is worth.
I am connecting to a legacy mysql database in cloud using my django project.
i need to fetch data and insert data if required in DB table.
do i need to write model for the tables which are already present in the db?
if so there are 90 plus tables. so what happens if i create model for single table?
how do i talk to database other than creating models and migrating? is there any better way in django? or model is the better way?
when i create model what happens in the backend? does it create those tables again in the same database?
There are several ways to connect to a legacy database; the two I use are either by creating a model for the data you need from the legacy database, or using raw SQL.
For example, if I'm going to be connecting to the legacy database for the foreseeable future, and performing both reads and writes, I'll create a model containing only the fields from the foreign table I need as a secondary database. That method is well documented and a bit more time consuming.
However, if I'm only reading data from a legacy database which will be retired, I'll create a read-only user on the legacy database, and use raw SQL to grab what I need like so:
from django.db import connections
cursor = connections["my_secondary_db"].cursor()
cursor.execute("SELECT * FROM my_table")
for row in cursor.fetchall():
insert_data_into_my_new_system_model(row)
I'm doing this right now with a legacy SQL Server database from our old website as we migrate our user and product data to Django/PostgreSQL. This has served me well over the years and saved a lot of time. I've used this to create sync routines all within a single app as Django management commands, and then when the legacy database is done being migrated to Django, I've completely deleted the single app containing all of the sync routines, for a clean break. Good luck!
For the app I'm building I need to be able to create a new data model in models.py as fast as possible automatically.
I created a way to do this by making a seperate python program that opens models.py, edits it, closes it, and does server migrations automatically but there must be a better way.
edit: my method works on my local server but not on pythonanywhere
In the Django documentation, I found SchemaEditor, which is exactly what you want. Using the SchemaEditor, you can create Models, delete Models, add fields, delete fields etc..
Here's an excerpt:
Django’s migration system is split into two parts; the logic for
calculating and storing what operations should be run
(django.db.migrations), and the database abstraction layer that turns
things like “create a model” or “delete a field” into SQL - which is
the job of the SchemaEditor.
Don't rewrite your models.py file automatically, that is not how it's meant to work. When you need more flexibility in the way you store data, you should do the following:
think hard about what kind of data you want to store and make your data model more abstract to fit more cases, if needed.
Use JSON fields to store arbitrary JSON data with your model (e.g. for the Postgres database)
if it's not a fit, don't use Django's ORM and use a different store (e.g. Redis for key-value or MongoDB for JSON documents)
What are the cons of allowing a user to add a column to a table in the database at runtime in a production environment. Is there a correct way to do it?
Normally when using a relational DB we never extend the DB at runtime.
At high performance, this is basically impossible (requires modifying the entire dataset, so the request will hang for the user)
Besides that, we do not really want to give users the power to grow our dataset (adding a new column means requiring an extra field for every row in the DB).
However, some relational DBs like Postgres support unstructured data like JSON. This might serve your purpose.
My django application need to collect user data(name age country etc) based on his email domain( 'gmail' as in xyz#gmail.com).I wist to create a new table every time i encounter a new email domain.
Can this be done in django ?
This is a bad idea. Your tables would all have the same structure. All of your data should be stored in a single table, with a domain column to keep the data separate. Why would you want a different table for each domain? Whatever reason you have, there's a better way to do it.
This idea goes against everything in the design of the relational database, and the Django ORM on top of it.