Django group by add non existing choices - django

I have a model field that contains choices:
db_redirection_choices = (('A', 'first'), ('B', 'second'))
redirection_type = models.CharField(max_length=256, choices=db_redirection_choices, blank=True, null=True)
At some point I'm performing a group by on that column, counting all existing choices:
results = stats.values('redirection_type').annotate(amount=Count('redirection_type')).order_by('redirection_type')
However, this will give me only results for existings choices. I'd like to add the ones that are not even present with 0 to the results
e.g. If the table contains only the entry
Id | redirection_type
--------------------------
1 | 'A'
then the annotate will return only
'A': 1
of course that's normal, but I'd still like to get all non-existing choices in the results:
{'A': 1, 'B': 0}
What's the easiest way of accomplishing this?

I don't think there is an easy way to do it with the ORM, except maybe using a conditional expression, but that would make your query a lot more complicated, I think.
Why not do a simple post-processing in Python?
db_redirection_choices = (('A', 'first'), ('B', 'second'))
# I think your queryset will have a similar shape
results = [{'redirection_type': 'A', 'amount': 1}]
results_map = {
**{choice: 0 for choice, _display in db_redirection_choices},
**{res['redirection_type']: res['amount'] for res in results}
}
assert results_map == {'A': 1, 'B': 0}
If you don't need further processing in the ORM, that seems like the easiest.

Related

Django - update dictionary with missing date values, set to 0

So to display a small bargraph using Django and Chart.js I constructed the following query on my model.
views.py
class BookingsView(TemplateView):
template_name = 'orders/bookings.html'
def get_context_data(self, **kwargs):
today = datetime.date.today()
seven_days = today + datetime.timedelta(days=7)
bookings = dict(Booking.objects.filter(start_date__range = [today, seven_days]) \
.order_by('start_date') \
.values_list('start_date') \
.annotate(Count('id')))
# Edit set default for missing dictonairy values
for dt in range(7):
bookings.setdefault(today+datetime.timedelta(dt), 0)
# Edit reorder the dictionary before using it in a template
context['bookings'] = OrderedDict(sorted(bookings.items()))
This led me to the following result;
# Edit; after setting the default on the dictionary and the reorder
{
datetime.date(2019, 8, 6): 12,
datetime.date(2019, 8, 7): 12,
datetime.date(2019, 8, 8): 0,
datetime.date(2019, 8, 9): 4,
datetime.date(2019, 8, 10): 7,
datetime.date(2019, 8, 11): 0,
datetime.date(2019, 8, 12): 7
}
To use the data in a chart I would like to add the missing start_dates into the dictionary but I'm not entirely sure how to do this.
So I want to update the dictionary with a value "0" for the 8th and 11th of August.
I tried to add the for statement but I got the error;
"'datetime.date' object is not iterable"
Like the error says, you can not iterate over a date object, so for start_date in seven_days will not work.
You can however use a for loop here like:
for dt in range(7):
bookings.setdefault(today+datetime.timedelta(dt), 0)
A dictionary has a .setdefault(..) function that allows you to set a value, given the key does not yet exists in the dicionary. This is thus shorter and more efficient than first checking if the key exists yourself since Python does not have to perform two lookups.
EDIT: Since python-3.7 dictionaries are ordered in insertion order (in the CPython version of python-3.6 that was already the case, but seen as an "implementation detail"). Since python-3.7, you can thus sort the dictionaries with:
bookings = dict(sorted(bookings.items()))
Prior to python-3.7, you can use an OrderedDict [Python-doc]:
from collections import OrderedDict
bookings = OrderedDict(sorted(bookings.items()))

Django dynamic verbose name

I'm struggling to think about how to achieve this. What I want to do is have a series of questions (to represent a Likert table) in a CharField object like so:
for a in range(1, 11):
locals()['ATL' + str(a)] = models.PositiveIntegerField(
choices=[
[1, 'Disagree Completely'],
[2, 'Disagree Strongly'],
[3, 'Disagree'],
[4, 'Neutral'],
[5, 'Agree'],
[5, 'Agree Strongly'],
[7, 'Agree Completely'],
],
widget=widgets.RadioSelectHorizontal(),
verbose_name = Constants.ATL_qu_list[a-1])
del a
And then change the verbose name for the question depending on the question number (again, I know I'm not supposed to be using locals() to store variables). Is there an easier way of achieving a dynamic label though? Thanks!
Okay, here's my answer (as well as a clarification for what I am looking for). Basically I had a series of Likert questions to put to participants which I wanted to represent as CharFields. Because each Likert question uses the same seven choice scale, it seems like inefficient coding to repeat the same functionality and only change the verbose name between each declaration.
Accordingly, I've instead used this method to achieve what I want:
# Reads in the list of survey questions
with open('survey/survey_questions.csv') as csvfile:
data_read = list(csv.reader(csvfile))
...
for a in range(1, 11):
locals()['ATL' + str(a)] = models.PositiveIntegerField(
choices=[
[1, 'Disagree Completely'],
[2, 'Disagree Strongly'],
[3, 'Disagree'],
[4, 'Neutral'],
[5, 'Agree'],
[6, 'Agree Strongly'],
[7, 'Agree Completely'],
],
widget=widgets.RadioSelectHorizontal(),
verbose_name = data_read[a-1][0])
del a

Weird behavior in Django queryset union of values

I want to join the sum of related values from users with the users that do not have those values.
Here's a simplified version of my model structure:
class Answer(models.Model):
person = models.ForeignKey(Person)
points = models.PositiveIntegerField(default=100)
correct = models.BooleanField(default=False)
class Person(models.Model):
# irrelevant model fields
Sample dataset:
Person | Answer.Points
------ | ------
3 | 50
3 | 100
2 | 100
2 | 90
Person 4 has no answers and therefore, points
With the query below, I can achieve the sum of points for each person:
people_with_points = Person.objects.\
filter(answer__correct=True).\
annotate(points=Sum('answer__points')).\
values('pk', 'points')
<QuerySet [{'pk': 2, 'points': 190}, {'pk': 3, 'points': 150}]>
But, since some people might not have any related Answer entries, they will have 0 points and with the query below I use Coalesce to "fake" their points, like so:
people_without_points = Person.objects.\
exclude(pk__in=people_with_points.values_list('pk')).\
annotate(points=Coalesce(Sum('answer__points'), 0)).\
values('pk', 'points')
<QuerySet [{'pk': 4, 'points': 0}]>
Both of these work as intended but I want to have them in the same queryset so I use the union operator | to join them:
everyone = people_with_points | people_without_points
Now, for the problem:
After this, the people without points have their points value turned into None instead of 0.
<QuerySet [{'pk': 2, 'points': 190}, {'pk': 3, 'points': 150}, {'pk': 4, 'points': None}]>
Anyone has any idea of why this happens?
Thanks!
I should mention that I can fix that by annotating the queryset again and coalescing the null values to 0, like this:
everyone.\
annotate(real_points=Concat(Coalesce(F('points'), 0), Value(''))).\
values('pk', 'real_points')
<QuerySet [{'pk': 2, 'real_points': 190}, {'pk': 3, 'real_points': 150}, {'pk': 4, 'real_points': 0}]>
But I wish to understand why the union does not work as I expected in my original question.
EDIT:
I think I got it. A friend instructed me to use django-debug-toolbar to check my SQL queries to investigate further on this situation and I found out the following:
Since it's a union of two queries, the second query annotation is somehow not considered and the COALESCE to 0 is not used. By moving that to the first query it is propagated to the second query and I could achieve the expected result.
Basically, I changed the following:
# Moved the "Coalesce" to the initial query
people_with_points = Person.objects.\
filter(answer__correct=True).\
annotate(points=Coalesce(Sum('answer__points'), 0)).\
values('pk', 'points')
# Second query does not have it anymore
people_without_points = Person.objects.\
exclude(pk__in=people_with_points.values_list('pk')).\
values('pk', 'points')
# We will have the values with 0 here!
everyone = people_with_points | people_without_points

Django model group as a list

I have a model with test data as below
id days
1, 30
1, 40
2, 10
2, 20
1, 90
I want output as
1, [30,40,90]
2, [10,20]
How can I get this in Django?
It's not much Django, it's pure python. To get the result as a mapping on 'id' as key:
result = {}
for obj in Mymodel.objects.all():
if result.has_key(obj.id):
result[obj.id].append(obj.days)
else:
result[obj.id] = [obj.days]
print result
>>> {1: [30, 40, 90], 2: [10, 20]}
The order of the elements in each list is not defined. If you require these to be ordered, best would be to append .order_by('days') on the Queryset.
A final remark: Your 'id' is not unique. I would consider a non-pk-column named 'id' a bad practice, since 'id' is Django's default name for the automatically created pk-field.

Using the "extra fields " from django many-to-many relationships with extra fields

Django documents give this example of associating extra data with a M2M relationship. Although that is straight forward, now that I am trying to make use of the extra data in my views it is feeling very clumsy (which typically means "I'm doing it wrong").
For example, using the models defined in the linked document above I can do the following:
# Some people
ringo = Person.objects.create(name="Ringo Starr")
paul = Person.objects.create(name="Paul McCartney")
me = Person.objects.create(name="Me the rock Star")
# Some bands
beatles = Group.objects.create(name="The Beatles")
my_band = Group.objects.create(name="My Imaginary band")
# The Beatles form
m1 = Membership.objects.create(person=ringo, group=beatles,
date_joined=date(1962, 8, 16),
invite_reason= "Needed a new drummer.")
m2 = Membership.objects.create(person=paul, group=beatles,
date_joined=date(1960, 8, 1),
invite_reason= "Wanted to form a band.")
# My Imaginary band forms
m3 = Membership.objects.create(person=me, group=my_band,
date_joined=date(1980, 10, 5),
invite_reason= "Want to be a star.")
m4 = Membership.objects.create(person=paul, group=my_band,
date_joined=date(1980, 10, 5),
invite_reason= "Wanted to form a better band.")
Now if I want to print a simple table that for each person gives the date that they joined each band, at the moment I am doing this:
bands = Group.objects.all().order_by('name')
for person in Person.objects.all():
print person.name,
for band in bands:
print band.name,
try:
m = person.membership_set.get(group=band.pk)
print m.date_joined,
except:
print 'NA',
print ""
Which feels very ugly, especially the "m = person.membership_set.get(group=band.pk)" bit. Am I going about this whole thing wrong?
Now say I wanted to order the people by the date that they joined a particular band (say the beatles) is there any order_by clause I can put on Person.objects.all() that would let me do that?
Any advice would be greatly appreciated.
You should query the Membership model instead:
members = Membership.objects.select_related('person', 'group').all().order_by('date_joined')
for m in members:
print m.band.name, m.person.name, m.date_joined
Using select_related here we avoid the 1 + n queries problem, as it tells the ORM to do the join and selects everything in one single query.