How to reduce DB queries? - django

Models:
class Technology(models.Model):
name = models.CharField(max_length=100, unique=True)
slug = models.SlugField(max_length=100, unique=True)
class Site(models.Model):
name = models.CharField(max_length=100, unique=True)
slug = models.SlugField(max_length=100, unique=True)
technology = models.ManyToManyField(Technology, blank=True, null=True)
Views:
def portfolio(request, page=1):
sites_list = Site.objects.select_related('technology').only('technology__name', 'name', 'slug',)
return render_to_response('portfolio.html', {'sites':sites_list,}, context_instance=RequestContext(request))
Template:
{% for site in sites %}
<div>
{{ site.name }},
{% for tech in site.technology.all %}
{{ tech.name }}
{% endfor %}
</div>
{% endfor %}
But in that example each site makes 1 additional query to get technology list. Is there any way to make it in 1 query somehow?

What you are looking for is an efficient way to do reverse foreign-key lookups. A generic approach is:
qs = MyRelatedObject.objects.all()
obj_dict = dict([(obj.id, obj) for obj in qs])
objects = MyObject.objects.filter(myrelatedobj__in=qs)
relation_dict = {}
for obj in objects:
relation_dict.setdefault(obj.myobject_id, []).append(obj)
for id, related_items in relation_dict.items():
obj_dict[id].related_items = related_items
I wrote a blogpost about this a while ago, you can find more info here: http://bit.ly/ge59D2

How about:
Using Django's session framework; load list request.session['lstTechnology'] = listOfTechnology on startup. And use session in rest of the app.

Related

How to access 'related_name' of a model via intermediate model's Foreign key inside template tag?

models.py
class Book(models.Model):
title = models.CharField(max_length=100)
author = models.ForeignKey(User, on_delete=models.CASCADE)
active = models.BooleanField(default=True)
...
class Review(models.Model):
paper = models.ForeignKey(Book, null=True, on_delete=models.CASCADE, related_name='book_class_related_name')
user = models.ForeignKey(User, on_delete=models.CASCADE)
comment = RichTextField()
status = models.CharField(max_length=10, choices=options, default='draft')
...
class TrackReviewRequests(models.Model):
paperid = models.ForeignKey(Book, null=True, on_delete=models.CASCADE, related_name='book_track')
numberOfTimesReviewRequestSent = models.PositiveSmallIntegerField(default=0)
...
views.py
reviews_in_draft = Review.objects.filter(paper__active=True).filter(status='draft')
return render(request,
'accounts/profile.html',
{
'reviews_in_draft': reviews_in_draft,
})
profile.html
Here I tried accessing the 'numberOfTimesReviewRequestSent' using the following code:
{% for review in reviews_in_draft %}
{{ review.paper.book_track.numberOfTimesReviewRequestSent }}
{% endfor %}
But I am getting empty string.
Then I wrote a method inside the Book model
def get_TrackReviewRequests_numberOfTimesReviewRequestSent(self):
return self.book_track.numberOfTimesReviewRequestSent
and tried accessing the numberOfTimesReviewRequestSent in the profile.html using the following code:
{{ review.paper.get_TrackReviewRequests_numberOfTimesReviewRequestSent }}
But this time I got the error stating
'RelatedManager' object has no attribute 'numberOfTimesReviewRequestSent'
Ultimately, I want to access the numberOfTimesReviewRequestSent in the template using the context variable.
A Bookcan have multiple TrackReviewRequests (Since TrackReviewRequests has a foreign key to Book, if in reality there can only be one then you should use a OneToOneField [Django docs] instead) hence review.paper.book_track is not an instance of TrackReviewRequests but as the error says a RelatedManager.
Hence when you write review.paper.book_track.numberOfTimesReviewRequestSent it doesn't make much sense. You can instead loop over the related instances if you want in the template like so:
{% for review in reviews_in_draft %}
{% for review_request in review.paper.book_track.all %}
{{ review_request.numberOfTimesReviewRequestSent }}
{% endfor %}
{% endfor %}

Optimise django query when fetching information from multiple models

I am new to Django and using Django 3.0.6.
With the following code, I have been able to achieve the desired results and display detailed book information onto the template. However, on average, ORM makes 8 to 9 database queries to get detailed information about the book. I am looking for expert help to optimize my database queries so that I could fetch book-related information with fewer queries.
I tried using select_related() and prefetch_related() but without any luck, maybe I did it improperly. Is there a scope of using Q object or union(), just my thought? How can I achieve the same results with minimum queries to the database?
Please help me with detailed code, if possible.
models.py
class Publisher(models.Model):
publisher_name = models.CharField(max_length=50)
class Author(models.Model):
author_name = models.CharField(max_length=50)
class Booktype(models.Model):
book_type = models.CharField(max_length=20) # Hard Cover, Soft Cover, Kindle Edition, Digital PDF etc.
class Book(models.Model):
book_title = models.TextField()
slug = models.SlugField(max_length=50, unique=False)
published_date = models.DateField(auto_now=False, auto_now_add=False)
publisher = models.ForeignKey(Publisher, on_delete=models.CASCADE)
author = models.ForeignKey(Author, on_delete=models.CASCADE)
book_type = models.ManyToManyField(Booktype, through='BookPrice', through_fields=('book', 'book_type'))
# I created this separate model due to havy content and to keep Book model light
class BookDetail(models.Model):
a = models.TextField(null=True, blank=True)
b = models.TextField(null=True, blank=True)
c = models.TextField(null=True, blank=True)
book = models.OneToOneField(Book, on_delete=models.CASCADE)
class BookPrice(models.Model):
book_type = models.ForeignKey(Booktype, on_delete=models.CASCADE)
book = models.ForeignKey(Book, on_delete=models.CASCADE)
price = models.DecimalField(max_digits=7, decimal_places=2)
view.py
def get_book_details(request, book_id, slug):
book = Book.objects.get(id=book_id, slug=slug)
context = {'book': book}
return render(request, 'products/book_detail.html', context)
book_detail.html Template
# 1st databse query
{{ book.book_title }}
{{ book.id }}
{{ book.published_date }}
# 2nd databse query
{{ book.publisher.publisher_name }}
# 3rd databse query
{{ book.author.author_name }}
# 4th databse query
{{ book.bookdetail.a }}
{{ book.bookdetail.b }}
{{ book.bookdetail.c }}
# 5th to 9th databse query depending upon avaialble Book Types
{% for x in book.bookprice_set.all %}
{{ x.book_type }} {{ x.price|floatformat }}
{% endfor %}
You can use .select_related(…) [Django-doc] to fetch the publisher, author and bookdetail. We can use prefetch_related
def get_book_details(request, book_id, slug):
book = Book.objects.select_related(
'publisher', 'author', 'bookdetail'
).prefetch_related(
'bookprice_set', 'bookprice_set__book_type'
).get(id=book_id, slug=slug)
context = {'book': book}
return render(request, 'products/book_detail.html', context)

Every field in Django modelform shows "Select a valid choice. That choice is not one of the available choices."

I've already read many other threads complaining about this error message but I still can't figure this out. I try removing the fields that give the error, and the error message just moves to another field the next time I try to submit. They are CharField, Foreign Key, and other types.
forms.py
class TemporaryresponseForm(forms.ModelForm):
gender_custom = forms.CharField(
required=False,
label="",
)
ethnicity = forms.ModelChoiceField(
queryset=Ethnicity.objects.all(),
widget=forms.RadioSelect(),
empty_label=None,
required=True,
label="Which of the following best describes your ethnicity?"
)
...
class Meta:
model = Temporaryresponse
fields = [...'gender_custom', 'ethnicity',...]
views.py
def tr(request):
if request.method == "POST":
form = TemporaryresponseForm(request.POST)
if form.is_valid():
tempresponse = form.save(commit=False)
tempresponse.ip = "123456"
tempresponse.save()
return redirect('politicalpollingapp/index.html')
else:
form = TemporaryresponseForm()
return render(request, 'politicalexperimentpollapp/tr.html', {'form': form})
def nr(request, pk):
return render(request, 'politicalexperimentpollapp/nr.html', {'tempresponse': tempresponse})
tr.html template
{% extends 'politicalexperimentpollapp/base.html' %}
{% block extrahead %}
{% load crispy_forms_tags %}
{{ form.media }}
{% endblock extrahead%}
...
<form method="POST">
{% csrf_token %}
{{ form.as_p }}
<div><br></div>
<div class="text-center"><button type="submit" class="save btn btn-primary">CONTINUE</button></div>
</form>
..
models.py
class Ethnicity(models.Model):
ethnicity = models.CharField(max_length=200)
def __str__(self):
return '%s' % (self.ethnicity)
...
class Temporaryresponse(models.Model):
birth_year = models.PositiveIntegerField()
voting_registration = models.ForeignKey(Voting_registration, models.SET_NULL, null=True)
party_identification = models.ForeignKey(Party_identification, models.SET_NULL, null=True)
gender = models.ForeignKey(Gender, models.SET_NULL, null=True)
gender_custom = models.CharField(max_length=200, blank=True)
ethnicity = models.ForeignKey(Ethnicity, models.SET_NULL, null=True)
race = models.ManyToManyField(Race)
zip_code = models.IntegerField()
ip = models.CharField(max_length=200, blank=True)
policy_areas_of_importance = models.ManyToManyField(Policy_category, blank=True)
likelihood_of_voting = models.PositiveIntegerField(models.SET_NULL, null=True, blank=True)
Oddly no error shows up in my Chrome console - it's only because I am showing errors on the actual page. I'm not sure if that's normal. Thanks in advance for any help, I'm ripping my hair out at this point.
I discovered that I was using the wrong language for the "race" form field. I had used ModelChoiceField but it should be ModelMultipleChoiceField as follows:
race = forms.ModelMultipleChoiceField(queryset=Race.objects.all(), widget=forms.CheckboxSelectMultiple, label="5. Which of the following best describes your race? Please select all that apply.")

django: How can i get multiple instances of model Userinfo when each instance of Userinfo has multiple instances in Education model

I have two models that I am working with. First one is a education model in which one user can enter multiple educational qualifications instances:
class Education(models.Model):
user = models.ForeignKey(User,on_delete=models.CASCADE)
degree_name = models.CharField(max_length=150,null=True,blank=True)
institute_name = models.CharField(max_length=150, null=True, blank=True)
date_start = models.CharField(null=True,blank=True,max_length=25)
date_end = models.CharField(null=True,blank=True,max_length=25)
description = models.TextField(null=True,blank=True,max_length=1000)
Second Model is the 'User info' model in which one user can have maximum one instance:
class Userinfo(models.Model):
user = models.ForeignKey(User,on_delete=models.CASCADE)
user_info = models.ForeignKey(User_info,related_name='user_info',on_delete=models.CASCADE,null=True)
profile_pic = models.FileField(null=True,blank=True)
dob = models.CharField(max_length=25,null=True,blank=True)
nationality = models.CharField(max_length=100, null=True, blank=True)
headline = models.CharField(max_length=160, null=True,blank=True)
summary = models.TextField(max_length=1000, null=True, blank=True)
current_salary = models.FloatField(null=True,blank=True)
japanese_level = models.CharField(max_length=50, null=True, blank=True)
english_level = models.CharField(max_length=50, null=True, blank=True)
career_level = models.CharField(max_length=50,null=True,blank=True)
availability = models.CharField(max_length=50, null=True, blank=True)
expected_salary = models.FloatField(null=True, blank=True)
job_role = models.CharField(max_length=50,null=True)
When I use any query to get any instance of 'User info' like:
Userinfo.objects.filter(user=request.user)
How can i related both models so that when looping through Userinfo, I should be able to get multiple instances of it in Education model. How should I change my models and query them ?
I see that you already have a foreign key to the User model inside your Education model. There is no need for a foreign key in the UserInfo Model. You can fetch all the Education instances for a given user just by making an extra call:
Education.objects.filter(user=request.user)
or you can change request.user to the actual user that you need to get.
EDIT:
without making any changes to your code, you can get the multiple instances in the following way:
example views.py
def myView(request):
user_info = Userinfo.objects.get(user=request.user) #using get since only 1 instance always
educations = Education.objects.filter(user=request.user) #fetching all the instances for the education
context_dict = {"user_info": user_info}
educations_list = []
for e in educations:
educations_list.append(e)
# do whatever you need with the educations
# you can access user_info fields just by `user_info.field_name`
# and you can access the current education fields by `e.field_name`
context_dict["educations"] = educations_list
return render(request, "template.html", context_dict)
example usage in template.html
{% if user_info %}
<p>{{ user_info.field_name }}</p>
{% if educations %}
{% for e in educations %}
<div>{{ e.field_name }}</div>
{% endfor %}
{% endif %}
{% endif %}
EDIT 2 (including multiple userinfo instances)
views.py
def myView(request):
user_infos = Userinfo.objects.filter() # fetch all instances
context_dict = {}
result = []
for u in user_infos:
temp = []
educations_list = []
educations = Education.objects.filter(user=u.user) # fetch educations for the currently iterated user from user_infos
for e in educations:
educations_list.append(e)
temp.append(u) # append the current user_info
temp.append(educations_list) # append the corresponding educations
result.append(temp)
context_dict["result"] = result
return render(request, "template.html", context)
template.html
{% if result %}
{% for r in result %}
<div>{{ r.0 }}</div> <!-- r.0 is your currently iterated user_info can be used like: r.0.profile_pic for example -->
{% if r.1 %}
{% for e in r.1 %}
<div>e.degree_name</div> <!-- e is the current education on the current user_info -->
{% endfor %}
{% endif %}
{% endfor %}
{% endif %}
the code in the views.py is not perfect and might be worth to refactor a bit (how to build the final dictionary), but i believe this will give you an idea of how to do it.
Hope this helps!
ui = Userinfo.objects.filter(user=request.user)
this query will give you all the instances of Userinfo for request.user. you can access the value of Education attributes with looping like this:
for u in ui:
ui.education.degree_name
# and so on for other fields.
I think maybe your UserInfo model can have a OneToOne relationsship with user and then do something like
UserInfo.objects.filter(user=request.user).education_set.all()
Hope this helps.
Good luck!

django - can't filter object and transfer to template

I've got a model which I'm trying to filter according to an argument passed in the url, then display the filtered object via a template, but I don't know what I'm doing wrong.
Here's the urls.py:
url(r'^courses/(?P<course_code>\w+)/$', views.course, name="course"),
Here's the view:
from website.models import Course
def course(request, course_code):
current_course = Course.objects.filter(short_title='course_code')
template = loader.get_template('website/course.html')
context = Context({
'current_course': current_course,
})
return HttpResponse(template.render(context))
Here's the model:
class Course(models.Model):
title = models.CharField(max_length=200)
short_title = models.CharField(max_length=5)
course_type = models.CharField(max_length=100)
start_date = models.DateTimeField()
end_date = models.DateTimeField()
fee = models.IntegerField()
places = models.IntegerField()
venue = models.CharField(max_length=200)
description = models.TextField()
short_description = models.TextField()
age_low = models.IntegerField()
age_high = models.IntegerField()
And here's the template:
{% if current_course %}
{% for course in current_course %}
{{ current_course.title }}
{% endfor %}
{% else %}
<p>Sorry, that course doesn't exist.</p>
{% endif %}
And when I load the page /courses/CR1 (the course with short_title="CR1" definitely exists because it renders fine on another template where I'm not filtering but just displaying all the courses), it gives me "Sorry, that course doesn't exist."
Can anyone see what I'm doing wrong?
In this line:
current_course = Course.objects.filter(short_title='course_code')
You're checking for course titles with the exact text 'course_code'. You mean to use the value of the variable course_code:
current_course = Course.objects.filter(short_title=course_code)