I need to manually add records to django_session table. With my own SQL.
Can I leave session_data column empty? What purpose does it serve?
session_data contains the actual data for the session. With Django sessions, the cookie that tracks the session actually just contains an ID, which points to a record in the database that contains the actual session data. session_data should not be blank; you can store a blank string, but it should be serialized the same way the Django database backend does it. Unfortunately, this involves pickling the data (with Python's pickle module) and then converting that into a base-64 representation, so it may be non-trivial to do this outside of Python.
Why must you use your own SQL? Why not write a script (preferably a management command) that taps into Django's sessions framework to create your objects?
Related
I am connecting to a legacy mysql database in cloud using my django project.
i need to fetch data and insert data if required in DB table.
do i need to write model for the tables which are already present in the db?
if so there are 90 plus tables. so what happens if i create model for single table?
how do i talk to database other than creating models and migrating? is there any better way in django? or model is the better way?
when i create model what happens in the backend? does it create those tables again in the same database?
There are several ways to connect to a legacy database; the two I use are either by creating a model for the data you need from the legacy database, or using raw SQL.
For example, if I'm going to be connecting to the legacy database for the foreseeable future, and performing both reads and writes, I'll create a model containing only the fields from the foreign table I need as a secondary database. That method is well documented and a bit more time consuming.
However, if I'm only reading data from a legacy database which will be retired, I'll create a read-only user on the legacy database, and use raw SQL to grab what I need like so:
from django.db import connections
cursor = connections["my_secondary_db"].cursor()
cursor.execute("SELECT * FROM my_table")
for row in cursor.fetchall():
insert_data_into_my_new_system_model(row)
I'm doing this right now with a legacy SQL Server database from our old website as we migrate our user and product data to Django/PostgreSQL. This has served me well over the years and saved a lot of time. I've used this to create sync routines all within a single app as Django management commands, and then when the legacy database is done being migrated to Django, I've completely deleted the single app containing all of the sync routines, for a clean break. Good luck!
For the app I'm building I need to be able to create a new data model in models.py as fast as possible automatically.
I created a way to do this by making a seperate python program that opens models.py, edits it, closes it, and does server migrations automatically but there must be a better way.
edit: my method works on my local server but not on pythonanywhere
In the Django documentation, I found SchemaEditor, which is exactly what you want. Using the SchemaEditor, you can create Models, delete Models, add fields, delete fields etc..
Here's an excerpt:
Django’s migration system is split into two parts; the logic for
calculating and storing what operations should be run
(django.db.migrations), and the database abstraction layer that turns
things like “create a model” or “delete a field” into SQL - which is
the job of the SchemaEditor.
Don't rewrite your models.py file automatically, that is not how it's meant to work. When you need more flexibility in the way you store data, you should do the following:
think hard about what kind of data you want to store and make your data model more abstract to fit more cases, if needed.
Use JSON fields to store arbitrary JSON data with your model (e.g. for the Postgres database)
if it's not a fit, don't use Django's ORM and use a different store (e.g. Redis for key-value or MongoDB for JSON documents)
I'm trying to use a regular python script to add data to a table that was created via the standard django process (start project/app/create model etc).
In the same DB, I set up another table with the same columns as the django DB to test on, and wrote a script that successfully parsed data and inserted it into that DB.
When I changed the table name so that the data would be written to the standard Django table, nothing was inserted and no error was thrown.
Is there something that prevents access to the Django tables that I'm unaware of?
I was wondering this could produce any problem if I directly add rows or remove some from a model table. I thought maybe Django records the number of rows in all tables? or this could mess up the auto-generated id's?
I don't think it matters but I'm using MySql.
No, it's not a problem because Django does the same that you do "directly" to the database, it execute SQL statements, and the auto generated id is handled by the database server (MySql server in this case), no matter where that SQL queries comes from, whatever it is Mysql Client or Django.
Since you can have Django work on a pre-existing database (one that wasn't created by Django), I don't think you will have problems if you access/write the tables of your own app (you might want to avoid modifying Django's internal tables like auth, permission, content_type etc until you are familiar with them)
When you create a model through Django, Django doesn't store the count or anything (unless your app does), so it's okay if you create the model with Django on the SQL database, and then have another app write/read from that same SQL table
If you use Django signals, those will not be triggered by modifying the SQL table directly through the DB, so you might want to pay attention to side effects like that.
Your RDBMS handles it's own auto generated IDs and referential integrity, counts etc, so you don't have to worry about messing it up.
How can I update an already populated Django database with records from a data fixture (or other serialized records)?
I know I can use Django data fixtures to provide initial data. Can I use the same already written features to update the database from data fixtures (or similar serialized data like a JSON document)?
The “insert or update from serialised data” operation should be idempotent:
If a record doesn't exist (by its key) in the database, it should be inserted.
If a record already exists (by its key) in the database, it should be updated to match the data fixture.
The end state should be that all data from the data fixture should be updated in the database, no matter if the records already existed.
Specifically, can I update existing rows by specifying pk=null and using natural keys?
How can I use the existing Django “load data” features (whether loaddata or something else similar in Django), to read serialised data, insert records if they don't exist and update them if they already exist?
Yes, you can with loaddata:
python3 -m django loaddata path/to/fixture/some_new_data.json
BUT:
don't call your new data initial_data.json else it will be reloaded every time you syncdb or run a South migraton and will wipe out any changes made since the fixture was loaded. (As mentioned mid-way through this page)
related to the above, loading data via fixtures can be a brittle way to do things unless you're sure that you new data won't arrive while you're writing your fixture. If it does, you risk a clash and either a failure or 'incorrect' data in your DB.
Depending on the nature of your data, it may be better to use a data migration in South. A data migration might be good when you're setting just a couple of values for lots of records (eg, changing the default of a field, or filling in an empty field with data). A data migration also lets you apply some checks/logic to things, via Python, so that you can target which records you update, or update different records in different ways, rather than writing lots of fixtures. If, however, you want to load in a ton of new records, a fixture would make more sense.