Using a GenericRelation to map Records to Persons, I have the following code, which works correctly, but there is a performance problem I am trying to address:
models.py
class RecordX(Record): # Child class
....
class Record(models.Model): # Parent class
people = GenericRelation(PersonRecordMapping)
def people_all(self):
# Use select_related here to minimize the # of queries later
return self.people.select_related('person').all()
class Meta:
abstract = True
class PersonRecordMapping(models.Model):
person = models.ForeignKey(Person)
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey('content_type', 'object_id')
class Person(models.Model):
...
In summary I have:
RecordX ===GenericRelation===> PersonRecordMapping ===ForeignKey===> Person
The display logic in my application follows these relationships to display all the Persons mapped to each RecordX:
views.py
#rendered_with('template.html')
def show_all_recordXs(request):
recordXs = RecordX.objects.all()
return { 'recordXs': recordXs }
The problem comes in the template:
template.html
{% for recordX in recordXs %}
{# Display RecordX's other fields here #}
<ul>
{% for map in record.people_all %}{# <=== This generates a query every time! #}
<li>{{ map.person.name }}</li>
{% endfor %}
</ul>
{% endfor %}
As indicated, every time I request the Persons related to the RecordX, it generates a new query. I cannot seem to figure out how prefetch these initially to avoid the redundant queries.
If I try selected_related, I get an error that no selected_related fields are possible here (error message: Invalid field name(s) given in select_related: 'xxx'. Choices are: (none)). Not surprising, I now see -- there aren't any FKs on this model.
If I try prefetch_related('people'), no error is thrown, but I still get the same repeated queries on every loop in the template as before. Likewise for prefetch_related('people__person'). If I try prefetch_related('personrecordmapping'), I get an error.
Along the lines of this answer, I considered doing trying to simulate a select_related via something like:
PersonRecordMapping.objects.filter(object_id__in=RecordX.objects.values_list('id', flat=True))
but I don't understand the _content_object_cache well enough to adapt this answer to my situation (if even the right approach).
So -- how can I pre-fetch all the Persons to the RecordXs to avoid n queries when the page will display n RecordX objects?
Thanks for your help!
Ack, apparently I needed to call both -- prefetch_related('people', 'people__person'). SIGH.
Related
I have multiple related models and on a single page I need to access data from different models on a variety of conditions. Most of this data is accessed without a form as only a few fields need to be manipulated by the user.
To give you an idea of how interconnected my Models are:
class User(models.Model):
first = models.ManyToManyField(First)
second= models.ManyToManyField(Second)
class First(models.Model):
[...]
class Second(models.Model):
first = models.ManyToManyField(First)
class Third(models.Model):
first = models.ForeignKey(First)
class Fourth(models.Model):
user = models.ForeignKey(User)
second = models.ForeignKey(Second)
third = models.ForeignKey(Third)
class Fifth(models.Model):
fourth = models.ForeignKey(Fourth)
class Sixth(models.Model):
fifth = models.ForeignKey(Fifth)
I'm having a lot of difficulty passing this data to my template and displaying it in efficiently. I originally displayed it to the user in a horrendous way as I just wanted to see if my data was accessible/displaying properly. This is an example of some of the dodgy code I used to test that my data was being passed properly:
{% for second in user.second.all %}
{% for first in second.first.all %}
[...]
{% for fourth in user.fourth.all %}
[...]
{% if fourth.first_id == first.id and fourth.second_id == second.id %}
{% if fourth.checked == True %}
[...]
{% else %}
[...]
{% endif %}
{% endif %}
{% endfor %}
{% endfor %}
{% endfor %}
[...]
Isn't it an abomination, and it goes deeper. Now I am trying to reformat this. I know that my processing should be done within my view rather than my template. The thing is I can't just filter out a lot of the data in my views as I need a lot of it in the template at different times (eg I might have 20 pieces of data in a model and I need to access all 20, but some when the id match, some when the id and type match etc. It's a lot of template side logic - I think that's bad).
This is the kind of thing I have been trying to do so far but am having no luck:
second = Second.objects.filter(user__id=user.id) .select_related('First')
Any help with how to join/pass all these models to my view and access the data without the nested nested loops, or even just pointers on how to approach this would be appreciated. I's also unsure if I should be aiming to join as many models as possible then pass into my view or pass many separate models.
Thank you.
you are serializer the data use a drf serializer you will need to use a nested serializer
example for nested serializer
class HospitalsSerializer(serializers.ModelSerializer):
class Meta:
model = Hospitals
fields = '__all__'
class HealthProductSerializers(serializers.ModelSerializer):
hospital_list = HospitalsSerializer(many=True)
class Meta:
model = HealthProduct
exclude = ("created_at", "updated_at")
My code looks like this:
models.py
class Tag(models.Model):
name = models.CharField(max_length=42)
class Post(models.Model):
user = models.ForeignKey(User, related_name='post')
#...various fields...
tags = models.ManyToManyField(Tag, null=True)
views.py
posts = Post.objects.all().values('id', 'user', 'title')
tags_dict = {}
for post in posts: # Iteration? Why?
p = Post.objects.get(pk=[post['id']]) # one extra query? Why?
tags_dict[post['id']] = p.tags.all()
How am I supposed to create a dictionary with tags for each Post object with minimum set of queries? Is it possible to avoid iterating, too?
Yes you will need a loop. But you can save one extra query in each iteration, you don't need to get post object to get all its tags. You can directly query on Tag model to get tags related to post id:
for post in posts:
tags_dict[post['id']] = Tag.objects.filter(post__id=post['id'])
Or use Dict Comprehension for efficiency:
tags_dict = {post['id']: Tag.objects.filter(post__id=post['id']) for post in posts}
If you have Django version >= 1.4 and don't really need a dictionary, but need to cut down the count of queries, you can use this method like this:
posts = Post.objects.all().only('id', 'user', 'title').prefetch_related('tags')
It seems to execute only 2 queries (one for Post and another for Tag with INNER JOIN).
And then you can access post.tags.all without extra queries, because tags was already prefetched.
{% for post in posts %}
{% for tag in post.tags.all %}
{{ tag.name }}
{% endfor %}
{% endfor %}
I read many times this and this tutorials, but I cant understand, how to do the following:
models:
class Car(models.Model):
field1
field2
field3
class CarOptions(models.Model):
car = models.OneToOneField(Car, primary_key=True)
field4
field5
class CarPictures(models.Model):
car = models.ForeignKey(Car)
field6
field7
So, I need get all information about the car in one sql-query. How it wrote in docs:
car = get_object_or_404(Car, pk=car_id)
But here is a strange (it discribes like "other side" of a ForeignKey relation) poll.choice_set.all, that doesnt works with my code. Some copy-past code, sorry, there is no links in docs:
# Give the Poll a couple of Choices. The create call constructs a new
# choice object, does the INSERT statement, adds the choice to the set
# of available choices and returns the new Choice object. Django creates
# a set to hold the "other side" of a ForeignKey relation
# (e.g. a poll's choices) which can be accessed via the API.
>>> p = Poll.objects.get(pk=1)
# Display any choices from the related object set -- none so far.
>>> p.choice_set.all()
[]
# Create three choices.
>>> p.choice_set.create(choice='Not much', votes=0)
<Choice: Not much>
>>> p.choice_set.create(choice='The sky', votes=0)
<Choice: The sky>
>>> c = p.choice_set.create(choice='Just hacking again', votes=0)
# Choice objects have API access to their related Poll objects.
>>> c.poll
<Poll: What's up?>
# And vice versa: Poll objects get access to Choice objects.
>>> p.choice_set.all()
[<Choice: Not much>, <Choice: The sky>, <Choice: Just hacking again>]
I havent got choice_set.all(). I add all information from the admin interface. With foreign keys all works well, but I need do a few sql-querys, not one. And in the docs they discribed it, like a one sql-query, but they have choice_set.all(). How its possible to do with my model? I need all information in the template (html), can you give me some example, how it works?
Thanks.
Related managers' names are automatically generated from model names. You have car.carpictures_set and car.caroptions (this isn't a "set" because it's a one-to-one relationship).
You can define your own related names:
class Car(models.Model):
...
class CarOptions(models.Model):
car = models.OneToOneField(Car, primary_key=True, related_name='options')
class CarPictures(models.Model):
car = models.ForeignKey(Car, related_name='pictures')
Then you'll have car.options and car.pictures.
Related objects reference
Let's say this is your view
cars = Car.objects.filter()
car_options = CarOptions.objects.filter()
car_pictures = CarPictures.objects.filter()
Here how it relates in html
{% for car in cars %}
{% for option in car.car_options %}
{% endfor %}
{% for picture in car.car_pictures %}
{% endfor %}
{% endfor %}
I recently started evaluating Django for migrating our archaic web application written 10 years ago. I have been reading up Django documentation for the last few days, but haven't been able to figure out the best way to achieve a multi table database join in my case:
Model:
class Product(models.Model):
productid = models.IntegerField(primary_key=True, db_column='ProductId')
productname = models.CharField(max_length=120, db_column='ProductName')
class Testcases(models.Model):
testcaseid = models.IntegerField(primary_key=True, db_column='TestCaseId')
testcasename = models.CharField(max_length=240, db_column='TestCaseName')
class Testmatrix(models.Model):
testmatrixid = models.IntegerField(primary_key=True, db_column='TestMatrixId')
productid = models.ForeignKey(Product, db_column='ProductId')
testcaseid = models.ForeignKey(Testcases, db_column='TestCaseId')
class Status(models.Model):
testmatrixid = models.ForeignKey(Testmatrix, db_column='TestMatrixId')
title = models.CharField(max_length=240, db_column='Title', blank=True)
(Note that model was generated by inspectdb and I'd prefer not to modify it at this point in time)
View:
from django.shortcuts import render_to_response
from mysite.testmatrix.models import Product, Testcases, Testmatrix, Status
def get_products(request):
tm = list(Testmatrix.objects.filter(productid='abc'))
return render_to_response('products.html', {'tm': tm})
template is designed to be minimal at this point to help focus on the real issue in (views/model).
Template: (products.html)
{% extends "main.html" %}
{% block body %}
<table>
{% for tm in tm %}
<tr>
<td>{{ tm.testmatrixid }}</td>
<td>{{ tm.testcaseid.testcasename }}</td>
</tr>
{% endfor %}
</table>
{% endblock %}
Problem:
Although I'm able to join Testmatrix and Testcase models, I am unable to generate an equivalent queryset by joining all of TestMatrix, TestCase, Status records on say productid='abc'
I tried the following:
1) Use select_related between Testmatrix and Testcases and Product tables and was able to access attributes across all three models (testmatrixid, productid, productname, testcaseid, testcasename). However I'm not sure how to extend this auto foreign key referencing to Status model. This would have been easier if all Foreign Keys were defined within Testmatrix itself. But Status has a Foreign Key to TestMatrix.
2) I tried using something like: entries = Status.objects.filter(testmatrixid__productid=pid). This again gave me a queryset as a result of joining Testmatrix and Status, but not Testcases.
Pardon any blaring mistakes or bloopers. This is my very first post!
So you need to access a related_object. It is very simple.
First, add related_name here:
class Status(models.Model):
testmatrixid = models.ForeignKey(Testmatrix, db_column='TestMatrixId', related_name='statuses')
Now you can get all the statuses for desired Testmatrix like
test_matrix.statuses.all()
If you don't want to hit DB, when you access statuses, don't forget to use select_related.
Without any specific error messages, it is hard to diagnose the cause of said errors. However, in your example, in views.get_products: tm = list(Testmatrix.objects.filter(productid='abc')) will not work, because 'abc' is a string and your productid is actually a Product object (not just an integer, even though the field is an integer foreign key to your reference table's pk); you can do tm = list(Testmatrix.objects.filter(productid=Product.objects.get(product_name='abc')), assuming that 'abc' is the product name of the product record. When you set a field to models.ForeignKey(...), you address that reference record as an object, not an id.
Other than that, nothing blaring, your template looks solid and your models look fine to me. I would suggest creating some test cases to see where the errors lie: Django Testing; Also, this is also a great tutorial to understand TDD and unit testing with Django. Using unittests, you can verify every step of your application and make future updates with assurance.
I have a multi model with different OneToOne relationships from various models to a single parent. Consider this example:
class Place(models.Model):
name = models.CharField(max_length=100)
address = models.CharField(max_length=100)
...
class Restaurant(models.Model):
place = models.OneToOneField(Place)
...
class Shop(models.Model):
place = models.OneToOneField(Place)
...
Regardless if the above model makes sense in real life scenario, how can someone identify if an object of Place has a relationship to any of the other models, in django view?
In a template, you can say
{% if place.restaurant %}
<!-- stuff goes here -->
{% endif %}
The reason for this is that OneToOneField entries actually create an attribute in the model they are referencing. In normal Python code, saying place.restaurant where no restaurant is defined will throw an exception, but templates will swallow such exceptions.
If you do need to do something like this in Python code, the simplest-to-understand way is to wrap it in a try/except block:
place = Place.objects.all()[0]
try:
restaurant = place.restaurant
# do something with restaurant
except ObjectDoesNotExist:
# do something without restaurant
EDIT: As I say in my comment, if you only want a Place to be either a Restaurant or a Shop but never both, then you shouldn't be using OneToOneField and should instead use model inheritance.
Assuming that a Place could have two or more other possibilities, I recommend doing something like this:
class Place(Model):
# define your fields here
# you could generate this automatically with trickery
OTHER_MODELS = ["restaurant", "shop"]
#property
def relationships(self):
if not hasattr(self, "_relationships"):
self._relationships = {}
for attr in OTHER_MODELS:
try:
self._relationshops[attr] = getattr(self, attr)
except ObjectDoesNotExist:
pass
return self._relationships
The above would let you say place.relationships and get back a dictionary that looked like
{"restaurant": <Restaurant Object>, "shop": <Shop Object>}
although one or both of those might be missing, depending on whether they exist. This would be easier to work with than having to catch a potential exception every time you reverse the OneToOneField relationship.
Because Restaurant has a foreign key pointing into Place, it leaves a related name field on the class, so that the pointed-to class (Place) can find its contents:
# python
import yourproject.settings
from django.db.models.base import ObjectDoesNotExist
try:
r = place.restaurant_set.get()
do_something( r.restaurant_field )
except ObjectDoesNotExist:
print "place has no restaurant"
And from a template, assuming you have access to placeobject from your context somehow:
{% with placeobject.restaurant_set.get as r %}
{% if r %}
{{ r.restaurant_field }}
{% else %}
No restaurant
{% endif %}
{% endwith %}