Django: Get distinct values from a foreign key model - django

Django newbie, so if this is super straightfoward I apologize.
I am attempting to get a listing of distinct "Name" values from a listing of "Activity"s for a given "Person".
Models setup as below
class Activity(models.Model):
Visit = models.ForeignKey(Visit)
Person = models.ForeignKey(Person)
Provider = models.ForeignKey(Provider)
ActivityType = models.ForeignKey(ActivityType)
Time_Spent = models.IntegerField(blank=True, null=True)
Repetitions = models.CharField(max_length=20, blank=True, null=True)
Weight_Resistance = models.CharField(max_length=50, blank=True, null=True)
Notes = models.CharField(max_length=500, blank=True, null=True)
class ActivityType(models.Model):
Name = models.CharField(max_length=100)
Activity_Category = models.CharField(max_length=40, choices=Activity_Category_Choices)
Location_Category = models.CharField(max_length=30, blank=True, null=True, choices=Location_Category_Choices)
I can get a listing of all activities done with a given Person
person = Person.objects.get(id=person_id)
activity_list = person.activity_set.all()
I get a list of all activities for that person, no problem.
What I can't sort out is how to generate a list of distinct/unique Activity_Types found in person.activity_set.all()
person.activity_set.values('ActivityType').distinct()
only returns a dictionary with
{'ActivityType':<activitytype.id>}
I can't sort out how to get straight to the name attribute on ActivityType
This is pretty straightforward in plain ol' SQL, so I know my lack of groking the ORM is to blame.
Thanks.
Update: I have this working, sort of, but this CAN'T be the right way(tm) to do this..
distinct_activities = person.activity_set.values('ActivityType').distinct()
uniquelist = []
for x in distinct_activities:
valuetofind = x['ActivityType']
activitytype = ActivityType.objects.get(id=valuetofind)
name = activitytype.Name
uniquelist.append((valuetofind, name))
And then iterate over that uniquelist...
This has to be wrong...

unique_names = ActivityType.objects.filter(
id__in=Activity.objects.filter(person=your_person).values_list('ActivityType__id', flat=True).distinct().values_list('Name', flat=True).distinct()
This should do the trick. There will be not a lot of db hits also.
Writing that down from my phone, so care for typos.

Related

Group By Django queryset by a foreignkey related field

I have a model Allotment
class Kit(models.Model):
kit_types = (('FLC', 'FLC'), ('FSC', 'FSC'), ('Crate', 'Crate'), ('PP Box', 'PP Box'))
kit_name = models.CharField(max_length=500, default=0)
kit_type = models.CharField(max_length=50, default=0, choices=kit_types, blank=True, null=True)
class AllotmentFlow(models.Model):
flow = models.ForeignKey(Flow, on_delete=models.CASCADE)
kit = models.ForeignKey(Kit, on_delete=models.CASCADE)
asked_quantity = models.IntegerField(default=0)
alloted_quantity = models.IntegerField(default=0)
class Allotment(models.Model):
transaction_no = models.IntegerField(default=0)
dispatch_date = models.DateTimeField(default=datetime.now)
send_from_warehouse = models.ForeignKey(Warehouse, on_delete=models.CASCADE)
flows = models.ManyToManyField(AllotmentFlow)
For a stacked graph I am trying to get the data of different kit_type alloted in different months.
For that I have tried annotate but it isn't getting the desired results
dataset = Allotment.objects.all().annotate(
month=TruncMonth('dispatch_date')).values(
'month').annotate(dcount=Count('flows__kit__kit_type')).values('month', 'dcount')
Expected Output:
[{'month':xyz, 'kit_type':foo, count:123},...]
I am getting the month and count of kit type from above but how do I segregate it by kit_type?
having a field that represents your choice field names in this query is difficult
instead how about use the Count filter argument and annotate to get what you want
dataset = Allotment.objects.all().annotate(month=TruncMonth('dispatch_date')).values('month').annotate(
FLC_count=Count('flows__kit__kit_type', filter=Q(flows__kit__kit_type="FLC")),
FSC_count=Count('flows__kit__kit_type', filter=Q(flows__kit__kit_type="FSC")),
Crate_count=Count('flows__kit__kit_type', filter=Q(flows__kit__kit_type="Crate")),
PP_Box_count=Count('flows__kit__kit_type', filter=Q(flows__kit__kit_type="PP_Box")),
).values('month', 'FLC_count', 'FSC_count', 'Crate_count', 'PP_Box_count')

PostgreSQL Programming error in Django Query

I have a query as below which returns the grade of all students of a specific class in a specific term(semester) in a specific session(academic year):
grades = Grade.objects.filter(term='First', student__in_class=1,session=1).order_by('-total')
then another query that annotate through the grades in order to get the sum of the 'total' field.
grades_ordered = grades.values('student')\
.annotate(total_mark=Sum('total')) \
.order_by('-total_mark')
At first everything works fine until when i migrated from using SQLite to postgreSQL then the following error begins to show up.
ERROR:
function sum(character varying) does not exist
LINE 1: SELECT "sms_grade"."student_id", SUM("sms_grade"."total") AS...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
EDIT:
here is my model
class Grade(models.Model):
session = models.ForeignKey(Session, on_delete=models.CASCADE)
term = models.CharField(choices=TERM, max_length=7)
student = models.ForeignKey(Student, on_delete=models.CASCADE)
subject = models.ForeignKey(Subject, on_delete=models.CASCADE)
fca = models.CharField(max_length=10)
sca = models.CharField(max_length=10)
exam = models.CharField(max_length=10)
total = models.CharField(max_length=3, blank=True, null=True)
grade = models.CharField(choices=GRADE, max_length=1, blank=True, null=True)
remark = models.CharField(max_length=50, blank=True, null=True)
any help you can provide would be appreciated.
thanks
Store numbers in integer or decimal not in text/varchar field
total = models.Integer(max_length=3, blank=True, null=True)
see this link
also read this

Django ORM simple Join

I want to perform simple join operation like this.
raw SQL : select * from risks r join sku_details s on r.sku_id = s.sku_id;
model Details:
class SkuDetails(models.Model):
sku_id = models.DecimalField(primary_key=True, max_digits=65535, decimal_places=65535)
sku_desc = models.TextField(blank=True, null=True)
category = models.TextField(blank=True, null=True)
class Risks(models.Model):
risk_id = models.DecimalField(primary_key=True, max_digits=65535, decimal_places=65535)
risk_group_short_desc = models.TextField(blank=True, null=True)
risk_group_desc = models.TextField(blank=True, null=True)
var = models.DecimalField(max_digits=65535, decimal_places=65535, blank=True, null=True)
sku = models.ForeignKey(SkuDetails, models.DO_NOTHING, blank=True, null=True)
After joining I want all the column of both the table in flat structure through Django ORM...
In raw SQL I will get all the column ... But not getting from ORM
Please Help !!!
Getting all values in a list of dictionaries is quite easy with values():
Risks.objects.values(
'risk_id',
'risk_group_short_desc`,
# ... fields you need from Risks
'sku__sku_id',
# ... fields you need from SkuDetails
)
You can check out values_list() as well.
You can try this withselect_related. Relevant helping material As both model with foreign-key relation.

Django Many to Many Data Duplication?

Background
I'm storing data about researchers. eg, researcher profiles, metrics for each researcher, journals they published in, papers they have, etc.
The Problem
My current database design is this:
Each Researcher has many journals (they published in). The journals have information about it.
Likewise for Subject Areas
But currently, this leads to massive data duplication. Eg, the same journal can appear many times in the Journal table, just linked to a different researcher, etc.
Is there any better way to tackle this problem? Like right now, I have over 5000 rows in the journal column but only about 1000 journals.
Thank you!
EDIT: This is likely due to the way im saving the models for new data (mentioned below). Could anyone provide the proper way to loop and save hashes to models?
Model - Researcher
class Researcher(models.Model):
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
scopus_id = models.BigIntegerField(db_index=True) # Index to make searches quicker
academic_rank = models.CharField(max_length=100)
title = models.CharField(max_length=200,default=None, blank=True, null=True)
salutation = models.CharField(max_length=200,default=None, blank=True, null=True)
scopus_first_name = models.CharField(max_length=100)
scopus_last_name = models.CharField(max_length=100)
affiliation = models.CharField(default=None, blank=True, null=True,max_length = 255)
department = models.CharField(default=None, blank=True, null=True,max_length = 255)
email = models.EmailField(default=None, blank=True, null=True)
properties = JSONField(default=dict)
def __str__(self):
return "{} {}, Scopus ID {}".format(self.scopus_first_name,self.scopus_last_name,self.scopus_id)
Model - Journal
class Journal(models.Model):
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
researchers = models.ManyToManyField(Researcher)
title = models.TextField()
journal_type = models.CharField(max_length=40,default=None,blank=True, null=True)
abbreviation = models.TextField(default=None, blank=True, null=True)
issn = models.CharField(max_length=50, default=None, blank=True, null=True)
journal_rank = models.IntegerField(default=None, blank=True, null=True)
properties = JSONField(default=dict)
def __str__(self):
return self.title
How I'm currently saving them:
db_model_fields = {'abbreviation': 'Front. Artif. Intell. Appl.',
'issn': '09226389',
'journal_type': 'k',
'researchers': <Researcher: x, Scopus ID f>,
'title': 'Frontiers in Artificial Intelligence and Applications'}
# remove researchers or else create will fail (some id need to exist error)
researcher = db_model_fields["researchers"]
del db_model_fields["researchers"]
model_obj = Journal(**db_model_fields)
model_obj.save()
model_obj.researchers.add(researcher)
model_obj.save()
Here is how it works :
class Journal(models.Model):
# some fields
class Researcher(models.Model):
# some fields
journal = models.ManyToManyField(Journal)
Django gonna create a relation table :
Behind the scenes, Django creates an intermediary join table to represent the many-to-many relationship
So you'll have many rows in this table, which is how it works, but journal instance and researcher instance in THEIR table will be unique.
Your error is maybe coming from how you save. Instead of :
model_obj = Journal(**db_model_fields)
model_obj.save()
Try to just do this:
model_obj = Journal.objects.get_or_create(journal_id)
This way you'll get it if it already exists. As none of your fields are unique, you're creating new journal but there's no problem cause django is generating unique ID each time you add a new journal.

Haystack searching multiple fields

I am currently building a page in django, where there are 4 form fields, 2 text, 2 select fields, and when submitted it takes those fields and searches several models for matchinng items.
the model looks like this:
class Person(models.Model):
user = models.ForeignKey(User, blank=True, null=True, verbose_name="the user associated with this profile")
first_name = models.CharField(max_length=255)
last_name = models.CharField(max_length=255)
about = models.TextField(max_length=255, blank=True, null=True)
birthdate = models.DateField(blank=True, null=True, verbose_name="Birthdate (yyyy-mm-dd)")
GENDER_CHOICES = (
(u'M', u'Male'),
(u'F', u'Female'),
)
gender = models.CharField(max_length=1, choices = GENDER_CHOICES, default = 'M')
picture = models.ImageField(upload_to='profile', blank=True, null=True)
nationality = CountryField(blank=True, null=True)
location = models.CharField(max_length=255, blank=True, null=True)
command_cert = models.BooleanField(verbose_name="COMMAND certification")
experience = models.ManyToManyField('userProfile.MartialArt', blank=True, null=True)
and I am trying to search the first_name field, the last_name field, the nationality field, and the experience field, but say if the first_name field is blank, I need to pass an empty value so it returns all rows, then filter from there with last name the same way, for some reason it is not working at all for me. this is my sqs:
results = SearchQuerySet().models(Person).filter(first_name=sname, last_name=slastname, nationality=scountry, experience__pk=sexperience)
any ideas?
Without seeing specific errors or a stack trace, it's hard to determine what "is not working at all".
Edit: Looking at your provided view code, I would remove the filter and return all of the objects for your Fighter, Referee, Insider, and Judge models. This is to ensure that the issue here lies in the filter, and not something else.
Then, once I'd verified that objects are being placed into results, I'd put in the filters one at a time to determine what the problematic filter is. Give this a try and reply back with your results.