I have a Django application that does quite expensive computation and then populates a database. I would like to serve the results using the Django Rest Framework. I would like to keep the application and the API on separate server instances so that I can ensure that the outgoing API is never deprived of resources. This means that both projects will be using the same tables in a shared database. Is there a way to achieve this without at least one of the projects having to perform raw SQL queries? I.e. Can I share both the database and the models (with querysets etc) between two projects on separate servers? Or is that not possible?
Note: I've looked at a lot of similar questions but the answers were not useful.
Thanks!
Related
I m designing a website where security of data is an issue.
I ve read this book : https://books.agiliq.com/projects/django-multi-tenant/en/latest/index.html
I'm still thinking about the right database structure for users.
And im hesitating between shared database with isolated schemas, isolated databases in a shared app, or completely isolated tenants using docker.
As security of data is an issue, i would like to avoid to put all the users in the same table in the database or in different schemas in the same database. However i dont understand well if i should put each user in a separate database (create a database per user, sqlite for example but i dont know if it would communicate well with postgres). What is the best practice for this in terms of security?
Im wondering how these options affect database speed compared to a shared database with a shared schema, which was the basic configuration of the course i attended on django.
I dont have good knowledge on databases so your help on the performance issue would be very appreciated!
Also, if I want to do some stats and use tenants data, how difficult is it to query completely isolated tenants using docker or isolated databases, in particular if each user is a separate docker or database?
I know some ways like scrapy-djangoitem but as it has mentioned:
DjangoItem is a rather convenient way to integrate Scrapy projects with Django models, but bear in mind that Django ORM may not scale well if you scrape a lot of items (ie. millions) with Scrapy. This is because a relational backend is often not a good choice for a write intensive applications (such as a web crawler), specially if the database is highly normalized and with many indices.
So what is the best way to use scraped items in db and django models?
It is not about Django ORM but rather about the database you choose as backend. What it says is that if you are expecting to write millions of items to your tables, relational database systems might not be your best choice here (MySQL, Postgres ...) and it can be even worse in terms of performance if you add many indicies since your application is write-heavy (Database must update B-Trees or other structures for keeping index on every write).
I would suggest sticking with Postgres or MySQL for now and look for another solution if you start to have performance issues on database level.
We've lots of intranet projects written in Django. As projects grows auth is always painful and duplicated.
E.g. user needs to login to three internal systems:
http://192.168.x.x/proj1/
http://192.168.x.x/proj2/
http://192.168.x.x/proj3/
Basically he has to create three accounts on three systems. So I figured if the MySQL table auth_user and django_session could be shared (using MySQL federated table engine) across three django projects, login session and info could be shared. So create account once and login one and use all three systems.
Is it safe? Anyone done similar in practice?
Sharing auth and session tables is quite an acceptable solution as long as maintainers of each project are aware of this architecture.
Some issues to be aware of:
Each project needs to be aware of the data stored in session by the other projects in order not to overwrite it inadvertently. This is not an issue if you only use the session for authentication.
The messages framework will not work as expected. Again, this might not be an issue in your projects.
It's probably a good idea to use the same Django version across all projects. For example, Django 1.6 changed the way it stores sessions by default. The User model might also change between versions.
If you want to use a Custom User Model, you need to use the same model on all projects. This might be a good option if you need to share extra profile data across projects though.
Since all three projects will have access to the DB, you'd better consider those as a single project/entity from a security point of view. Some data may leak from one project to the other through the session and be exposed where it shouldn't.
Also, some security issues may arise: if for example two projects use inadvertently the same variable name to store conflicting data, considered safe by one project but unsafe by another, a user could inject malicious data which will be considered safe.
But these things could happen if you have a single project maintained by multiple programmers too. So, as long as you make sure these are not issues for your projects you should be fine.
I have to create a django project which has two dictinct parts. The producer provides some UI to the user and eventually writes to some models. The consumer uses this data and does some processing and shows some pretty graphs etc.
Technically, these are completely isolated apps used by completely distinct user set. I would love to make them separate django projects altogether but unfortunately they share the db structure. The database is effectively a pipe that connects the two.
How should I structure the project? From what I read, I should ideally have isolated models per app. Unfortunately that is not possible.
Any recommendations?
If you define a "app" and use it inside of your "Django Project" I assume you use INSTALLED_APP's in settings.py to make it known within your environment.
If you look it from this point - its the same as if you use "django-social-auth" in two different projects/services and you share the same DB. I can't judge if its common or uncommon to use the same DB, its a design decision you have to make and you need to be happy with.
If you just like to have users and webusers seperated in your admin, please have a look at
separating-staff-and-user-accounts
I have two different Django projects that are meant to run in parallel and do pretty different things.
However they need to share a common database table, the Client table..
Both projects contains multiple apps that needs to contain foreign keys mapped to that Client model..
I'm not sure what would be the best approach..
Assuming both projects are working on the same db, just import the model you want to reference to.
from first_project.some_app.models import Client, OtherSharedModel
class SomeModelInSecondProject(models.Model):
client = models.ForeignKey(Client)
Unfortunately, Django's support for multiple databases does not support cross-database relations. You could fake this on one of the systems (ie. have the table referenced, but handle the key refs yourself), but you would need to be very careful to document what you are doing to make sure you maintain referential integrity in the app that is 'faking' it.
I haven't tested it but another alternative, if you're sharing the same db and having both projects in the same server, is to just merge them into one project, organize their apps in different directories and if you must you can use two different setting files. Please see this question related to that: How to keep all my django applications in specific folder. It's just a different approach that doesn't require you to reference a different project (I'm not sure how recommendable that is).