Navigating many2many relationships in both directions - django

I'm trying to understand how Django returns columns from foreign keys, particularly the m2m situation, easy in SQL but I'm trying to get into Django.
In this example I have 3 models, Sample which has a m2m with Container
and Location which has a 1-to-many with Container.
Scenario 1a: From the Sample table get the Containers that sample is in(return sample_number and container_name).
Scenario 1b: From the Container get the related Samples (return container_number and sample_number).
Scenario 2a: From the Location model get the containers (location_name and container_names).
Scenario 2b: From the Container model get the location (Container_name and location_name).
Hopefully this will serve as a good overall reference for others.
# models.py
class Location(models.Model):
location_id = models.AutoField(primary_key=True)
location_name = models.CharField(max_length=100, blank=True, null=True)
class Sample(models.Model):
sample_id = models.AutoField(primary_key=True)
sample_number = models.IntegerField()
class Container(models.Model): #like a friend
container_id = models.AutoField(primary_key=True)
container_name = models.CharField(max_length=50, blank=True, null=True)
location_id = models.ForeignKey(Location, db_column='location_id', on_delete = models.PROTECT, related_name = 'location')
samples = models.ManyToManyField('Sample', through='ContainerSamples', related_name='containers')
# views.py - Implements a filter
def detailcontainer(request, container_id):
container = get_object_or_404(Container, pk=container_id)
samples = container.samples.all()
container_contents = container.samples.all()
unassigned_samples = Sample.objects.all()
qs = Sample.objects.all()
context = {
'queryset': qs,
'container':container,
'container_contents': container_contents,
'unassigned_samples': unassigned_samples,
}
return render(request, 'container/detailcontainer.html', context)
# templates
{% for unassigned in unassigned_samples %}
# 1a [solved]
{% for unassigned in unassigned_samples %}
{{ unassigned.sample_number }}
{% for container in unassigned.containers.all %}
{{ container.location_id }}.{{ container.container_name }}
{% endfor %}
{% endfor %}
# 1b
{{ unassigned.____________ }} # the container_name
{{ unassigned.____________ }} # the related samples (sample_number)
# 2a
{{ unassigned.____________ }} # the location_name
{{ unassigned.____________ }} # the related container names (container_name)
# 2b
{{ unassigned.____________ }} # the container_name
{{ unassigned.____________ }} # the location_name
{% endfor %}

Scenario 1a: From the Sample table get the Containers that sample is in(return sample_number and container_name).
container_set = sample.containers.all()
for container in container_set:
print([container.container_name, sample.sample_name])
Scenario 1b: From the Container get the related Samples (return container_number and sample_number).
sample_set = container.samples.all()
for sample in sample_set:
print([sample.sample_number, container.container_number])
Scenario 2a: From the Location model get the containers (location_name and container_names).
container_set = location.location.all() # You have Container.location_id.related_name = 'location', I don't know why.
for container in container_set:
print([location.location_name, container.container_name])
Scenario 2b: From the Container model get the location (Container_name and location_name).
print([container.container_name, container.location_id.location_name])
Also your models should be written as follows:
# models.py
class Location(models.Model):
name = models.CharField(max_length=100, blank=True, null=True)
class Sample(models.Model):
number = models.IntegerField()
class Container(models.Model): #like a friend
container = models.AutoField(primary_key=True)
name = models.CharField(max_length=50, blank=True, null=True)
location = models.ForeignKey(Location, on_delete = models.PROTECT, related_name='containers')
samples = models.ManyToManyField(Sample, through='ContainerSamples', related_name='containers')
If you're using the same definition for the autofield on the models, there' no reason to add it. When you're defining properties on a model such as a name, you shouldn't prefix it with the model name. It should be understood that it's the model's name. And finally related_name is the field on the referenced model for the relationship to traverse back to the current model.

Related

Optimise django query when fetching information from multiple models

I am new to Django and using Django 3.0.6.
With the following code, I have been able to achieve the desired results and display detailed book information onto the template. However, on average, ORM makes 8 to 9 database queries to get detailed information about the book. I am looking for expert help to optimize my database queries so that I could fetch book-related information with fewer queries.
I tried using select_related() and prefetch_related() but without any luck, maybe I did it improperly. Is there a scope of using Q object or union(), just my thought? How can I achieve the same results with minimum queries to the database?
Please help me with detailed code, if possible.
models.py
class Publisher(models.Model):
publisher_name = models.CharField(max_length=50)
class Author(models.Model):
author_name = models.CharField(max_length=50)
class Booktype(models.Model):
book_type = models.CharField(max_length=20) # Hard Cover, Soft Cover, Kindle Edition, Digital PDF etc.
class Book(models.Model):
book_title = models.TextField()
slug = models.SlugField(max_length=50, unique=False)
published_date = models.DateField(auto_now=False, auto_now_add=False)
publisher = models.ForeignKey(Publisher, on_delete=models.CASCADE)
author = models.ForeignKey(Author, on_delete=models.CASCADE)
book_type = models.ManyToManyField(Booktype, through='BookPrice', through_fields=('book', 'book_type'))
# I created this separate model due to havy content and to keep Book model light
class BookDetail(models.Model):
a = models.TextField(null=True, blank=True)
b = models.TextField(null=True, blank=True)
c = models.TextField(null=True, blank=True)
book = models.OneToOneField(Book, on_delete=models.CASCADE)
class BookPrice(models.Model):
book_type = models.ForeignKey(Booktype, on_delete=models.CASCADE)
book = models.ForeignKey(Book, on_delete=models.CASCADE)
price = models.DecimalField(max_digits=7, decimal_places=2)
view.py
def get_book_details(request, book_id, slug):
book = Book.objects.get(id=book_id, slug=slug)
context = {'book': book}
return render(request, 'products/book_detail.html', context)
book_detail.html Template
# 1st databse query
{{ book.book_title }}
{{ book.id }}
{{ book.published_date }}
# 2nd databse query
{{ book.publisher.publisher_name }}
# 3rd databse query
{{ book.author.author_name }}
# 4th databse query
{{ book.bookdetail.a }}
{{ book.bookdetail.b }}
{{ book.bookdetail.c }}
# 5th to 9th databse query depending upon avaialble Book Types
{% for x in book.bookprice_set.all %}
{{ x.book_type }} {{ x.price|floatformat }}
{% endfor %}
You can use .select_related(…) [Django-doc] to fetch the publisher, author and bookdetail. We can use prefetch_related
def get_book_details(request, book_id, slug):
book = Book.objects.select_related(
'publisher', 'author', 'bookdetail'
).prefetch_related(
'bookprice_set', 'bookprice_set__book_type'
).get(id=book_id, slug=slug)
context = {'book': book}
return render(request, 'products/book_detail.html', context)

return objects pointing to other objects in django

I have a model UserSelect
class UserSelect(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)
bowler = models.ForeignKey(Bowlers, on_delete=models.CASCADE, related_name='bowlers', null=True, blank=True)
batsman = models.ForeignKey(Batsmen, on_delete=models.CASCADE, related_name='batsman', null=True, blank=True)
team = models.IntegerField(blank=True, default='1')
which is pointing to my two models bowler and batsman using foreign key
I am calling this model values in my view as
def team(request, pk):
user_sel = UserSelect.objects.filter(user=request.user, team=pk)
context = {
'user_sel': user_sel,
}
return render(request, 'team_analysis.html', context)
here I am getting all objects by giving pk but how do I access data of my other two models bowler and batsman through it in views?
You can access them in views like this:
for user in user_sel:
print(user.bowler) # bowler model object
print(user.batsman) # batsman model object
Also you can access them in templates as well:
{% for user in user_sel %}
{{ user.bowler }}
{{ user.bowler.pk }}
{{ user.batsman }}
{{ user.batsman.pk }}
{% endfor %}

django: How can i get multiple instances of model Userinfo when each instance of Userinfo has multiple instances in Education model

I have two models that I am working with. First one is a education model in which one user can enter multiple educational qualifications instances:
class Education(models.Model):
user = models.ForeignKey(User,on_delete=models.CASCADE)
degree_name = models.CharField(max_length=150,null=True,blank=True)
institute_name = models.CharField(max_length=150, null=True, blank=True)
date_start = models.CharField(null=True,blank=True,max_length=25)
date_end = models.CharField(null=True,blank=True,max_length=25)
description = models.TextField(null=True,blank=True,max_length=1000)
Second Model is the 'User info' model in which one user can have maximum one instance:
class Userinfo(models.Model):
user = models.ForeignKey(User,on_delete=models.CASCADE)
user_info = models.ForeignKey(User_info,related_name='user_info',on_delete=models.CASCADE,null=True)
profile_pic = models.FileField(null=True,blank=True)
dob = models.CharField(max_length=25,null=True,blank=True)
nationality = models.CharField(max_length=100, null=True, blank=True)
headline = models.CharField(max_length=160, null=True,blank=True)
summary = models.TextField(max_length=1000, null=True, blank=True)
current_salary = models.FloatField(null=True,blank=True)
japanese_level = models.CharField(max_length=50, null=True, blank=True)
english_level = models.CharField(max_length=50, null=True, blank=True)
career_level = models.CharField(max_length=50,null=True,blank=True)
availability = models.CharField(max_length=50, null=True, blank=True)
expected_salary = models.FloatField(null=True, blank=True)
job_role = models.CharField(max_length=50,null=True)
When I use any query to get any instance of 'User info' like:
Userinfo.objects.filter(user=request.user)
How can i related both models so that when looping through Userinfo, I should be able to get multiple instances of it in Education model. How should I change my models and query them ?
I see that you already have a foreign key to the User model inside your Education model. There is no need for a foreign key in the UserInfo Model. You can fetch all the Education instances for a given user just by making an extra call:
Education.objects.filter(user=request.user)
or you can change request.user to the actual user that you need to get.
EDIT:
without making any changes to your code, you can get the multiple instances in the following way:
example views.py
def myView(request):
user_info = Userinfo.objects.get(user=request.user) #using get since only 1 instance always
educations = Education.objects.filter(user=request.user) #fetching all the instances for the education
context_dict = {"user_info": user_info}
educations_list = []
for e in educations:
educations_list.append(e)
# do whatever you need with the educations
# you can access user_info fields just by `user_info.field_name`
# and you can access the current education fields by `e.field_name`
context_dict["educations"] = educations_list
return render(request, "template.html", context_dict)
example usage in template.html
{% if user_info %}
<p>{{ user_info.field_name }}</p>
{% if educations %}
{% for e in educations %}
<div>{{ e.field_name }}</div>
{% endfor %}
{% endif %}
{% endif %}
EDIT 2 (including multiple userinfo instances)
views.py
def myView(request):
user_infos = Userinfo.objects.filter() # fetch all instances
context_dict = {}
result = []
for u in user_infos:
temp = []
educations_list = []
educations = Education.objects.filter(user=u.user) # fetch educations for the currently iterated user from user_infos
for e in educations:
educations_list.append(e)
temp.append(u) # append the current user_info
temp.append(educations_list) # append the corresponding educations
result.append(temp)
context_dict["result"] = result
return render(request, "template.html", context)
template.html
{% if result %}
{% for r in result %}
<div>{{ r.0 }}</div> <!-- r.0 is your currently iterated user_info can be used like: r.0.profile_pic for example -->
{% if r.1 %}
{% for e in r.1 %}
<div>e.degree_name</div> <!-- e is the current education on the current user_info -->
{% endfor %}
{% endif %}
{% endfor %}
{% endif %}
the code in the views.py is not perfect and might be worth to refactor a bit (how to build the final dictionary), but i believe this will give you an idea of how to do it.
Hope this helps!
ui = Userinfo.objects.filter(user=request.user)
this query will give you all the instances of Userinfo for request.user. you can access the value of Education attributes with looping like this:
for u in ui:
ui.education.degree_name
# and so on for other fields.
I think maybe your UserInfo model can have a OneToOne relationsship with user and then do something like
UserInfo.objects.filter(user=request.user).education_set.all()
Hope this helps.
Good luck!

How can I increase page load speed?

I need to have multilingual site. For this purpose I wrote django module, which collects lots of info about countries, cities and their translations to almost all languages.
Below is the short version of models of this module:
class LanguagesGroups(models.Model):
class Meta:
verbose_name = 'Language Group'
class Languages(models.Model):
iso_code = models.CharField("ISO Code", max_length=14, db_index=True)
group = models.ForeignKey(LanguagesGroups, on_delete=models.CASCADE, verbose_name='Group of ISO',
related_name='group', db_index=True)
class Cities(models.Model):
population = models.IntegerField(null=True)
territory_km2 = models.IntegerField(null=True)
class CitiesTranslations(models.Model):
common_name = models.CharField(max_length=188, db_index=True)
city = models.ForeignKey(Cities, on_delete=models.CASCADE, verbose_name='Details of City')
lang_group = models.ForeignKey(LanguagesGroups, on_delete=models.CASCADE, verbose_name='Language of city',
null=True)
class Meta:
index_together = (['common_name', 'city'],
['city', 'lang_group'])
I want to show to users some data about places which user requested with translated versions of cities (depending on user settings):
class Profile(models.Model):
title = models.CharField(_('title'), max_length=120)
info = models.TextField(_('information'), max_length=1500, blank=True)
city = models.ForeignKey(Cities, verbose_name=_('city'), null=True, blank=True)
def get_city(self):
user_lang = get_language() # en
lang_group = Languages.objects.get(iso_code=user_lang).group # 1823
return CitiesTranslations.objects.get(city=self.city, lang_group=lang_group).common_name
template.html
{% for item in object_list %}
{{ item.title }}
{{ item.get_city }}
{{ item.info }}
{% endfor %}
When I add {{ item.get_city }}, in case of pagination and just 25 items per page, the page load speed goes down up to 18 times and amount of queries (according to django-debug-tool) goes up from 2 to 102. django-debug-tool tells me about 25 duplications.
How can I fix this slowness?
EDIT:
My view
class ProfileListView(ListView):
model = Profile
template_name = 'profiles/profiles_list.html'
context_object_name = 'places_list'
paginate_by = 25
First of all, if you want speed - you should try caching.
You can also optimize your query.
def get_city(self):
user_lang = get_language() # en
return CitiesTranslations.objects.get(
city=self.city_id, lang_group__group__iso_code=user_lang
).common_name
What you probably also want is to get all your stuff in batches, not with individual method calls. Assuming we have your object_list:
city_ids = [x.city_id for x in object_list]
city_translations = CitiesTranslations.objects.filter(
city__in=city_ids, lang_group__group__iso_code=user_lang
).values_list('city_id', 'common_name')
city_translations = dict(city_translations)
for obj in object_list:
obj.city_name = city_translations[obj.city_id]
You can put this code somewhere in your view. You will also have to change {{ item.get_city }} to {{ item.city_name }} in the templates.

How to reduce DB queries?

Models:
class Technology(models.Model):
name = models.CharField(max_length=100, unique=True)
slug = models.SlugField(max_length=100, unique=True)
class Site(models.Model):
name = models.CharField(max_length=100, unique=True)
slug = models.SlugField(max_length=100, unique=True)
technology = models.ManyToManyField(Technology, blank=True, null=True)
Views:
def portfolio(request, page=1):
sites_list = Site.objects.select_related('technology').only('technology__name', 'name', 'slug',)
return render_to_response('portfolio.html', {'sites':sites_list,}, context_instance=RequestContext(request))
Template:
{% for site in sites %}
<div>
{{ site.name }},
{% for tech in site.technology.all %}
{{ tech.name }}
{% endfor %}
</div>
{% endfor %}
But in that example each site makes 1 additional query to get technology list. Is there any way to make it in 1 query somehow?
What you are looking for is an efficient way to do reverse foreign-key lookups. A generic approach is:
qs = MyRelatedObject.objects.all()
obj_dict = dict([(obj.id, obj) for obj in qs])
objects = MyObject.objects.filter(myrelatedobj__in=qs)
relation_dict = {}
for obj in objects:
relation_dict.setdefault(obj.myobject_id, []).append(obj)
for id, related_items in relation_dict.items():
obj_dict[id].related_items = related_items
I wrote a blogpost about this a while ago, you can find more info here: http://bit.ly/ge59D2
How about:
Using Django's session framework; load list request.session['lstTechnology'] = listOfTechnology on startup. And use session in rest of the app.