I'm trying to optimise my app by keeping the number of queries to a minimum... I've noticed I'm getting a lot of extra queries when doing something like this:
class Category(models.Model):
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=127, blank=False)
class Project(models.Model):
categories = models.ManyToMany(Category)
Then later, if I want to retrieve a project and all related categories, I have to do something like this :
{% for category in project.categories.all() %}
Whilst this does what I want it does so in two queries. I was wondering if there was a way of joining the M2M field so I could get the results I need with just one query? I tried this:
def category_list(self):
return self.join(list(self.category))
But it's not working.
Thanks!
Which, whilst does what I want, adds an extra query.
What do you mean by this? Do you want to pick up a Project and its categories using one query?
If you did mean this, then unfortunately there is no mechanism at present to do this without resorting to a custom SQL query. The select_related() mechanism used for foreign keys won't work here either. There is (was?) a Django ticket open for this but it has been closed as "wontfix" by the Django developers.
What you want is not seem to possible because,
In DBMS level, ManyToMany relatin is not possible, so an intermediate table is needed to join tables with ManyToMany relation.
On Django level, for your model definition, django creates an ectra table to create a ManyToMany connection, table is named using your two tables, in this example it will be something like *[app_name]_product_category*, and contains foreignkeys for your two database table.
So, you can not even acces to a field on the table with a manytomany connection via django with a such categories__name relation in your Model filter or get functions.
Related
I need to create a Django PostgreSQL Database with a field in multiple tables to use it like a filter for the users, but I don't know what technique to use for performance.
I can create a table with the field and make a foreign key for every table.
class Tablefilter(models.Model):
filter_field = models.Field()
class Tablefilted(models.Model):
table_filter = models.ForeignKey(Tablefilter)
Or in my models just create a extend for that field in every model.
class Tablefilter(models.Model):
filter_field = models.Field()
class Tablefilted(Tablefilter):
field = models.Field()
Your question is missing some elaboration.
However, filtering is better to be done through the views, not via the models. So basically have some parameters on the frontend supplied to our views, and use Django's filter by inbuilt function.
I'm using python and psycopg2 to scrape some data and insert them into my database.
I've already created the movies and actors tables inside my Django models.py and inside my movies table, there is a many to many relationship between movies and actors table.
below is my models.py:
class Movie(models.Model):
name = models.CharField(max_length=55)
summary = models.TextField(max_length=1024)
actor = models.ManyToManyField(Actor, blank=True)
when I create a movie from Django-admin I select which actors are included in the movie and everything works fine and all the related actors for that movie will show up on my website.
But the problem is that when I try to insert scraped data into my database outside of my Django project folder, the related actors won't be shown on my website because obviously, I have not set the many to many relationship between them.
I have tried creating a junction table using SQL commands which gets the movie id and the actor's id and links them together but I don't know how I should tell Django to use that table and show the related actors for each movie.
This is the SQL code I use to insert into my db:
INSERT INTO movies(name, summary)
VALUES ('movie name', 'sth')
and the code to insert to actors table:
INSERT INTO actors(name, bio)
VALUES ('actorname', 'sth')
Both actors and movies table have auto generated id and I insert them insto the junction table using the code below:
INSERT INTO movie_actors (actor_id, movie_id)
VALUES (
(SELECT actor_id from actors where name='actor name'),
(SELECT movie_id from movie where name='movie name')
)
Am I doing it right?
I would really appreciate it if someone could help me with this.
Django automatically creates a table for many2many relationships. From docs:
ManyToManyField.through
Django will automatically generate a table to manage many-to-many relationships. However, if you want to manually specify the intermediary table, you can use the through option to specify the Django model that represents the intermediate table that you want to use.
The most common use for this option is when you want to associate extra data with a many-to-many relationship.
So you must find the name of the table that django had already created.
Secondly, I suggest that you use django's ORM instead of raw queries so you don't have these kind of problems anymore.
Django automatically creates a through table for M2M relations, if you need you can specify custom through table. In your case I think there is no need of custom through table.
I using Django ORM instead of writing raw query.
INSERT INTO movies(name, summary) VALUES ('movie name', 'sth')
instead of tis raw query you can use the following ORM query:
movie = Movie.objects.create(name="movie name", summary="movie sammuary")
This will create a movie entry in the Movie table.
Next to create user entry you can use the following query:
actor = Actor.objects.create(name="actor name", bio="actor bio")
Now you created the entries in both the table, next you can establish the realtion, for that you have to use the following query:
movie.actor.add(actor)
Incase if you want to add multiple actors at the same time, you create multiple actors object and use following query:
movie.actor.add(actor1, actor2, actor2)
For more details you can check django's offical documentation
I am used in creating orm and leaving django responsible for creating the tables.But in a project I am involved I have to create a simple CRUD application a frontend for an existing database. The database was created by creating the tables manually. So I have two tables Table1 and Table2 which have a many to many relationship through Tables12. Tables12 looks like the table that django would normaly create using a ManyToManyField thus it has two fields the id's of the two models. So after using django's inspectdb, django successfully created the models according to the SQLite database. The many to many tables like Tables12 was created like the following(as stated above):
class Tables12(models.Model):
table1 = models.ForeignKey(Table1)
table2 = models.ForeignKey(Table2)
class Meta:
managed = False
db_table = "Tables12"
unique_together = (("table1_id", "table2_id"),)
Trying the following gives me an error:
>> table2 = Table2.objects.get(pk=1)
>>tables12 = Tables12.objects.filter(table2=table2)
>>tables12
OperationalError: no such column: Tables12.id
I am guessing Django's orm is expecting an id field in every models created. How can I bypass this behavior? Is there a way to edit the tables so as they look more like django's orm but behave as the existing db's tables? Like:
class Table1(models.Model):
#pre exsiting fields
table2 = models.ManyToManyField(Table2)
or
class Table2(models.Model):
#pre existing fields
table1 = models.ManyToManyField(Table1)
but without destroying database records and without creating tables from start.
You can remove the Tables12 model, and specify the db_table argument to a ManyToManyField:
class Table1(models.Model):
tables2 = models.ManyToManyField(Table2, db_table='Tables12')
You would still not be able to query the Tables12 model directly (it still exists and has an id field), but this would allow you to use the JOINs Django generates for a ManyToManyField:
table1 = Table1.objects.get(pk=1)
tables2 = table1.tables2.all()
This would still not allow you to use Django to write to the join table, but it allows you to use the data that's in it. If at all possible, I'd strongly recommend adding a primary key.
Consider the following many-to-one relationship. Users can own many widgets:
class Widget(models.Model):
owner = models.ForeignKey('User')
class User(models.Model):
pass
I have a fairly complicated query that returns me a queryset of Widgets. From that I want to list the distinct values of User.
I know I could loop my Widgets and pull out users with .values('owner') or {% regroup ... %} but the dataset is huge and I'd like to do this at the database so it actually returns Users not Widgets first time around.
My best idea to date is pulling out a .values_list('owner', flat=True) and then doing a User.objects.filter(pk__in=...) using that. That pulls it back to two queries but that still seems like more than it should need to be.
Any ideas?
Use backward relationship:
User.objects.distinct().filter(widget__in=your_widget_queryset)
I have a simple case with 2 models: Item and Category with ManyToMany between them. I want to show a page listing all categories and for each category list of items. I have hundreds of categories so django hits db hundreds of times (when iterating thru categories and calling items.all() for each one). I need to select data from the intermediate table manually and use select_related() to pull item and category for each record - one query instead of hundreds.
I know that introducing 'through' would solve the problem but I don't want to do it now because it may break existing code (using through makes you can't use add, create, or assignment to create relationships - which I want to avoid for now).
So, is it possible at all without creating a model for intermediate table?
You could make a model for your existing table, and just not use it as the through field for the m2m, and make it unmanaged. eg:
class ItemCategory(models.Model):
item = models.ForeignKey('Item')
category = models.ForeignKey('Category')
class Meta:
db_table = 'the_name_of_the_existing_m2m_table'
managed = False
Something like that, anyway.