RelatedManager not returning excepted results - django

I am currently working on a django app where I have two models:
class Job(models.Model):
def __unicode__(self):
return self.job_number
area = models.ForeignKey(Area)
customer = models.ForeignKey(Customer)
job_address = models.ForeignKey(Address, null=True)
job_status = models.ForeignKey(JobStatus)
surveyor = models.ForeignKey(Employee, null=True)
job_dates = models.OneToOneField(JobDates, null=True)
job_title = models.CharField(max_length=50, null=False)
job_number = models.CharField(max_length=15, null=False, unique=True)
deposit_amount = models.DecimalField(null=True, decimal_places=2, max_digits=10)
is_deposit_paid = models.BooleanField(default=False)
is_deposit_required = models.BooleanField(default=False)
particular = models.SmallIntegerField(null=True)
description = models.TextField(null=True)
surveyor_note = models.TextField(null=True)
contractor_note = models.TextField(null=True)
class Picture(models.Model):
def __unicode__(self):
return self.file_name
job = models.ForeignKey(Job, related_name='picture_job_set')
url = models.CharField(max_length=450, null=False, unique=True)
file_name = models.CharField(max_length=50, null=False)
is_main_pic = models.BooleanField(default=False)
title = models.CharField(max_length=50, null=True)
description = models.TextField(null=True)
date_uploaded = models.DateTimeField(null=False)
uploaded_by = models.ForeignKey(Employee, null=False)
So it is a one to many relationship where each job can have many pictures associated with it. Now in the picture model, I have the "is_main_pic" column which is used to display a picture in a listview and when they click on it, it goes to details and shows the rest of the pictures. Now my problem is whenever I try to filter to just get the main picture within each job in the list view, it returns all the pictures for each job in the list view if the condition is met for any of the pictures. My query looks like this:
Jobs.objects.filter(picture_job_set__is_main_pic=True).prefetch_related('picture_job_set')
Now when I go to access the picture_job_set, it contains all pictures associated with a job, even if their "is_main_pic" column is false.
I would like to be able to in the template view just to call first on the picture_job_set as there will always only be one main picture for each job. As of right now I am able to get around this by simply having a for loop in the template with an if statement to determine which is the main pic but this seems messy and inefficient.
Am I misunderstanding how RelatedManager works? Do I have to call two queries to achieve this? Coming from a ASP MVC background, I could achieve this with something like:
List<JobIndexViewModel> viewModel = await db.Jobs
.Select(x => new JobIndexViewModel
{
JobID = x.JobID,
JobTitle = x.JobTitle,
JobNumber = x.JobNumber,
BidExpireDate = x.JobDates.BidExpireDate,
JobPostedDate = x.JobDates.JobPostedDate,
City = x.Address.City,
PictureUrl = x.Pictures.Where(t => t.IsMainPic == true).Select(t => t.Url).FirstOrDefault()
}).ToListAsync();

I, playing django now, interprete your query as:
Give me all Jobs that has a picture from picture_job_sets, inasmuch as
one of the pictures is marked as main picture.
So like you said, it returns all the pictures for each job in the list view if the condition is met for any of the pictures.
Quoting you: there will always only be one main picture for each job. If there will always be a main picture, you can easily start the query from the pictures:
Picture.objects.filter(is_main_pic=True).select_related('Job')
Then you can iterate over each picture and do .job on each to get the related job.
One last thing though, I suspect you have this problem because of your design. Won't you have a simpler design if you move a main picture as a field to your Job model?

Related

How to relationship such a connection in django?

I need to implement a "end-to-end" connection between products.
like this:
I automatically put down two-way links between two products. But when I link product A to product B and product B to product C, there is no connection between A and C. It is necessary that they communicate themselves when putting down the two previous links.
Models.py
from django.db import models
class Part(models.Model):
brand = models.CharField('Производитель', max_length=100, blank=True)
number = models.CharField('Артикул', max_length=100, unique=True)
name = models.CharField('Название', max_length=100, blank=True)
description = models.TextField('Комментарий', blank=True, max_length=5000)
analog = models.ManyToManyField('self',blank=True, related_name='AnalogParts')
images = models.FileField('Главное изображение', upload_to = 'parts/', blank=True)
images0 = models.FileField('Дополнительное фото', upload_to = 'parts/', blank=True)
images1 = models.FileField('Дополнительное фото', upload_to = 'parts/', blank=True)
images2 = models.FileField('Дополнительное фото', upload_to = 'parts/', blank=True)
def __str__(self):
return str(self.number)
return self.name
class Meta:
verbose_name = 'Запчасть'
verbose_name_plural = 'Запчасти'
Your diagram looks like Parts are connected linear. The easiest way to accomplish that is to add OneToOneFields to the model it self like so:
next = models.OneToOneField('self', related_name='previous', null=True)
Then you can do something like
part = Part.object.get(name='wheel')
while part.next:
part = part.next
# At this point Part is the last part in the row
print(part)
This is the most easy way. Depending whether you have linear connections between parts only you have to adjust your fields. There is no solution to all of the graph problems unless you realize some graph structure/database which will solve common problems (I think GraphQL can do such stuff, but iam not sure). Also keep in mind that this will most probably execute one SQL query per loop iteration.

Django Many to Many Data Duplication?

Background
I'm storing data about researchers. eg, researcher profiles, metrics for each researcher, journals they published in, papers they have, etc.
The Problem
My current database design is this:
Each Researcher has many journals (they published in). The journals have information about it.
Likewise for Subject Areas
But currently, this leads to massive data duplication. Eg, the same journal can appear many times in the Journal table, just linked to a different researcher, etc.
Is there any better way to tackle this problem? Like right now, I have over 5000 rows in the journal column but only about 1000 journals.
Thank you!
EDIT: This is likely due to the way im saving the models for new data (mentioned below). Could anyone provide the proper way to loop and save hashes to models?
Model - Researcher
class Researcher(models.Model):
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
scopus_id = models.BigIntegerField(db_index=True) # Index to make searches quicker
academic_rank = models.CharField(max_length=100)
title = models.CharField(max_length=200,default=None, blank=True, null=True)
salutation = models.CharField(max_length=200,default=None, blank=True, null=True)
scopus_first_name = models.CharField(max_length=100)
scopus_last_name = models.CharField(max_length=100)
affiliation = models.CharField(default=None, blank=True, null=True,max_length = 255)
department = models.CharField(default=None, blank=True, null=True,max_length = 255)
email = models.EmailField(default=None, blank=True, null=True)
properties = JSONField(default=dict)
def __str__(self):
return "{} {}, Scopus ID {}".format(self.scopus_first_name,self.scopus_last_name,self.scopus_id)
Model - Journal
class Journal(models.Model):
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
researchers = models.ManyToManyField(Researcher)
title = models.TextField()
journal_type = models.CharField(max_length=40,default=None,blank=True, null=True)
abbreviation = models.TextField(default=None, blank=True, null=True)
issn = models.CharField(max_length=50, default=None, blank=True, null=True)
journal_rank = models.IntegerField(default=None, blank=True, null=True)
properties = JSONField(default=dict)
def __str__(self):
return self.title
How I'm currently saving them:
db_model_fields = {'abbreviation': 'Front. Artif. Intell. Appl.',
'issn': '09226389',
'journal_type': 'k',
'researchers': <Researcher: x, Scopus ID f>,
'title': 'Frontiers in Artificial Intelligence and Applications'}
# remove researchers or else create will fail (some id need to exist error)
researcher = db_model_fields["researchers"]
del db_model_fields["researchers"]
model_obj = Journal(**db_model_fields)
model_obj.save()
model_obj.researchers.add(researcher)
model_obj.save()
Here is how it works :
class Journal(models.Model):
# some fields
class Researcher(models.Model):
# some fields
journal = models.ManyToManyField(Journal)
Django gonna create a relation table :
Behind the scenes, Django creates an intermediary join table to represent the many-to-many relationship
So you'll have many rows in this table, which is how it works, but journal instance and researcher instance in THEIR table will be unique.
Your error is maybe coming from how you save. Instead of :
model_obj = Journal(**db_model_fields)
model_obj.save()
Try to just do this:
model_obj = Journal.objects.get_or_create(journal_id)
This way you'll get it if it already exists. As none of your fields are unique, you're creating new journal but there's no problem cause django is generating unique ID each time you add a new journal.

Excluding objects from Django queryset based on recency

I have a reddit-like Django app where users can post interesting urls (links) and then publicly comment under them. The two data models to represent this are:
class Link(models.Model):
description = models.TextField(validators=[MaxLengthValidator(500)])
submitter = models.ForeignKey(User)
submitted_on = models.DateTimeField(auto_now_add=True)
class Publicreply(models.Model):
submitted_by = models.ForeignKey(User)
answer_to = models.ForeignKey(Link)
submitted_on = models.DateTimeField(auto_now_add=True)
description = models.TextField(validators=[MaxLengthValidator(250)])
How do I query for all Links which have at least 1 or more publicreply, and secondly where the latest publicreply is not by self.request.user? I sense something like the following:
Link.objects.filter(publicreply__isnull=False).exclude(**something here**)
Please advise! Performance is key too, hence the simpler the better!
For performance and simplicity you could cache both the number of replies and the latest reply:
class Link(models.Model):
...
number_of_replies = models.PositiveIntegerField(default=0)
latest_reply = models.ForeignKey('myapp.Publicreply', related_name='+', blank=True, null=True, on_delete=models.SET_NULL)
When a reply is entered, update the corresponding link.number_of_replies and link.latest_reply.
The query would then be:
Link.objects.filter(number_of_replies__gte=1)\
.exclude(latest_reply__user=request.user)

Django, o2o/one-to-one modeling with multiple models

I have a model Job.
class Job(models.Model):
job_number = models.AutoField(primary_key=True)
date_opened = models.DateField()
staff_opened = models.ForeignKey(User, related_name="jobs_opened")
date_closed = models.DateField(blank=True, null=True)
staff_closed = models.ForeignKey(User, related_name="jobs_closed", db_index=True, blank=True, null=True)
date_promised = models.DateField(blank=True, null=True)
date_estimate = models.DateField(blank=True, null=True)
customer = models.CharField("Customer", max_length=50, db_index=True)
slug = models.SlugField(max_length=60, unique=True)
And I also have a number of different type of jobs that hold more information, depending on what it is:
class WorkshopJob(Job):
job = models.OneToOneField(Job, parent_link=True)
invoice_number = models.CharField("Invoice Number", max_length=30, blank=True)
part = models.ForeignKey(PartNumber)
serial_number = models.CharField(max_length=20, blank=True)
and
class EngineeringJob(Job):
job = models.OneToOneField(Job, parent_link=True)
work_order = models.CharField(max_length=30)
reported_fault = models.TextField()
findings = models.TextField(blank=True)
work_performed = models.TextField(blank=True)
Any particular Job can only have one Engineering Job, one Workshop Job - but it can have one of each, too.
I never instantiate a Job on it's own - there is no AddJob view or page - only the subclasses.
The part I'm struggling with is the link - if I am viewing the detail of one subclass, how can I "add" another type of Job to the same Job?
IE if the I had an engineering job with job_id=1, how do I "pass" the job_id=1 to the new Workshop Job in the view?
I've tried adding get_initial(self) to the views, but it isn't working for me.
I think I'm going to rejig my set up.
Previously I had urls like this:
url(r'^workshop/add/$', views.WorkshopJobAdd.as_view(), name='wsjob_add'),
url(r'^workshop/(?P<slug>[-\w]+)/$', views.WorkshopJobDetail.as_view(), name='wsjob_detail'),
url(r'^workshop/(?P<slug>[-\w]+)/edit/$', views.WorkshopJobEdit.as_view(), name='wsjob_edit'),
url(r'^workshop/(?P<slug>[-\w]+)/invoice/$', views.WSJInvoice.as_view(), name='wsj_invoice'),
url(r'^engineering/add/$', views.EngineeringJobAdd.as_view(), name='wsjob_add'),
url(r'^engineering/(?P<slug>[-\w]+)/$', views.EngineeringJobDetail.as_view(), name='wsjob_detail'),
url(r'^engineering/(?P<slug>[-\w]+)/edit/$', views.EngineeringJobEdit.as_view(), name='wsjob_edit'),
url(r'^engineering/(?P<slug>[-\w]+)/invoice/$', views.EngineeringInvoice.as_view(), name='wsj_invoice'),
And was trying to pass the job_id between views.
But I think the easiest/sensible solution to my problem is to change the urls to something more like:
url(r'^job/(?P<slug>[-\w]+)/workshop/add/$', views.WorkshopJobAdd.as_view(), name='wsjob_add'),
url(r'^job/(?P<slug>[-\w]+)/workshop/$', views.WorkshopJobDetail.as_view(), name='wsjob_detail'),
url(r'^job/(?P<slug>[-\w]+)/workshop/edit/$', views.WorkshopJobEdit.as_view(), name='wsjob_edit'),
url(r'^job/(?P<slug>[-\w]+)/workshop/invoice/$', views.WSJInvoice.as_view(), name='wsj_invoice'),
And then adding a new subclassed model to an existing Job would just be a matter of using the slug. I was making things far to hard for myself.
I think this is another case of "CBVs aren't always the best solution".
Which is not the end of the world - I think I do wish for an easy way to tell when it's the case though :\

Filtering on foreign key relationship - Django

I want to find the number of articles for which a specific user has created articlehistory records.
The models for that look like this:
class Article(models.Model):
"""The basic entity of this app.)"""
documentID = models.CharField(blank=True, max_length=1000)
cowcode = models.IntegerField(blank=True, null=True)
pubdate = models.DateField(default=datetime.datetime.today)
headline = models.CharField(blank=True, max_length=1500)
source = models.CharField(blank=True, max_length=5000)
text = models.TextField(blank=True, max_length=1000000)
assignments = models.ManyToManyField(Assignment)
class Meta:
ordering = ['pubdate']
def __unicode__(self):
return self.headline
class ArticleHistory(models.Model):
"""(Modelname description)"""
article = models.ForeignKey(Article, related_name='Article History')
coder = models.ForeignKey(User, related_name='Article History')
last_updated = models.DateTimeField(default=datetime.datetime.now)
def __unicode__(self):
return self.last_updated
The way I'm trying to do this at the moment is like this:
assignment.finished_articles = Article.objects.filter(cowcode=country).filter(pubdate__range=(start_date,end_date), articlehistory__coder=request.user.id).count()
This doesn't work, however and exhibits another weird behaviour:
I try to do this:
for assignment in assignments:
country = assignment.country.cowcode
start_date = assignment.start_date
end_date = assignment.end_date
articles = Article.objects.filter(cowcode=country).filter(pubdate__range=(start_date,end_date)).select_related()
assignment.article_num = articles.count()
#assignment.finished_articles = Article.objects.filter(cowcode=country).filter(pubdate__range=(start_date,end_date), articlehistory__coder=request.user.id).count()
This works fine, unless I try to include finished_articles, then article_num gets shortened to one result.
It would be really great if anyone has a pointer to who to solve this.
Make use of reverse relation of ForeignKey created by parameter related_name:
Rename attribute related name to "article_history_set".
Now, it gives you easy pointer: user.article_history_set is a set of Article History objects where coder is set to this user.
Then you can find which article it is related to by doing article_history.article.
At the end, you have to get rid of repetition and get length of that list.
Here you have more about related_name attribute: https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.ForeignKey.related_name