Break django values down into count of each value - django

I have a model defined similar to below
class MyModel(models.Model):
num_attempts = models.IntegerField()
num_generated = models.IntegerField()
num_deleted = models.IntegerField()
Assuming my data looked something like this:
|id|num_attempts|num_generated|num_deleted
1 1 2 0
2 2 0 1
3 3 2 1
4 3 1 2
I want to get a count of the instances at each possible value for each possible field.
For example, a return sample could look like this.
{
'num_attempts_at_1': 1,
'num_attempts_at_2': 1,
'num_attempts_at_3': 2,
'num_generated_at_0': 1,
'num_generated_at_1': 1,
'num_generated_at_2': 2,
'num_deleted_at_0': 1,
'num_deleted_at_1': 2,
'num_deleted_at_2': 1
}
This above example assumes a lot, like naming of the variables after and that it would be serialized. None of that matters but rather just how do I get it broken down like that from the database. It would be best to have this done in one query if possible.
We are using Postgres as the database.
Here is sorta close, but not quite.
qs.values_list('num_attempts','num_generated','num_deleted').annotate(Count('id'))
Gives this (not the same data as the example above)
[{'num_attempts': 4, 'id__count': 3, 'num_deleted': 3, 'num_generated': 6}, {'num_attempts': 3, 'id__count': 12, 'num_deleted': 2, 'num_generated': 2}, {'num_attempts': 2, 'id__count': 5, 'num_deleted': 0, 'num_generated': 6}]
Now with some custom python I was able to do this, but really want a database solution if possible.
def get(self, request, *args, **kwargs):
qs = self.get_queryset()
return_data = {}
for obj in qs:
count = obj.pop('id__count')
for k, v in obj.items():
key = "{}_at_{}".format(k, v)
value = return_data.get(key, 0) + count
return_data[key] = value
return Response(return_data)

Related

How to get top 5 records in django dict data

I m having two tables 1) Visit 2) disease. visit table having a column for disease. I m trying to get top 5 disease from visit table.
dis=disease.objects.all()
for d in dis:
v=visits.objects.filter(disease=d.disease_name).count()
data={
d.disease_name : v
}
print (data)
This print all disease with respective count. as below:
{'Headache': 2}
{'Cold': 1}
{'Cough': 4}
{'Dog Bite': 0}
{'Fever': 2}
{'Piles': 3}
{'Thyroid': 4}
{'Others': 9}
I want to get top 5 from this list based on count. How to do it?
Thank you all for your reply, I got an other simple solution for it.
from django.db.models import Count
x = visits.objects.values('disease').annotate(disease_count=Count('disease')).order_by('-disease_count')[:5]
print(x)
it returns as below:
<QuerySet [{'disease': 'Others', 'disease_count': 9}, {'disease': 'Thyroid', 'disease_count': 4}, {'disease': 'Cough', 'disease_count': 4}, {'disease': 'Piles', 'disease_count': 3}, {'disease': 'Headache', 'disease_count': 2}]>
I think this is simplest solutions. It working for me...
Add data in a list and sort list based on what you want:
dis=disease.objects.all()
l = list()
for d in dis:
v=visits.objects.filter(disease=d.disease_name).count()
data={
d.disease_name : v
}
l.append(data)
l.sort(reverse=True, key=lambda x:list(x.values())[0])
for i in range(min(len(l), 5)):
print(l[i])
You can sort these values by writing code like that:
diseases = list(Disease.objects.values_list('disease_name', flat=True))
visits = list(
Visits.objects.filter(disease__in=diseases).values_list('disease', flat=True))
data = {}
for name in diseases:
count = visits.count(name)
data[name] = count
sorted_data = sorted(data.items(), key=operator.itemgetter(1), reverse=True)
new_data = {}
for idx in range(min(len(sorted_data), 5)):
item = sorted_data[idx]
new_data[item[0]] = item[1]
print(new_data)
It's little messy, but it does the job:
I also optimised your queries, so the code should also run bit faster (when you do logic like that, use list and .values_list(...) because it caches data in memory - and using native python functions on list instead of QuerySet like .count() should also be faster than hitting database).

How do swap the indexes of a Posts?

I have model with order indices, for example - 1, 2, 3, 5, 8, 10... (empty indices were allegedly removed - 4, 6, 7, 9...)
I need to get the index and swap it with the nearest index next to it. (for Up and Down move).
What I have:
def set_order_index(item: Posts, distinction: int):
items_to_swap = Posts.objects.filter(
order_index=item.order_index + distinction)
if len(items_to_swap) > 0:
item_to_swap = items_to_swap[0]
item_to_swap.order_index = item.order_index
item_to_swap.save()
item.order_index += distinction
item.save()
And for Up views '+1':
def up_item(request, id):
item = Posts.objects.get(id=id)
set_order_index(item, +1)
return redirect('index')
And for Down '-1'...
But my set_order_index only does +1 and -1 always, and if it gets the same value, then swaps places. That is if index 3, the set_order_index will only make 4 (+1), but I need to swap 3 with 5. Because there is no index 4.
And if I use Post with index 10 to Down - I need to swap with 8, but my set_order_index only do -1, and I get only 9. But I need 10 to swap with 8 immediately.
In my head, I understand how to do this through QuerySet, getting the indices of all that is > and < and then select the first or last - it is necessary, the neighbor. But in the code, I can't do it for it to work.
def swap_up(item):
nextItem = Posts.objects.filter(order_index__gt=item.order_index).order_by('order_index').first()
if nextItem:
item_index = item.order_index
item.order_index = nextItem.order_index
nextItem.order_index = item_index
item.save()
nextItem.save()
def swap_down(item):
prevItem = Posts.objects.filter(order_index__lt=item.order_index).order_by('order_index').last()
if prevItem:
item_index = item.order_index
item.order_index = prevItem.order_index
prevItem.order_index = item_index
item.save()
prevItem.save()

Combine two Django Querysets based on common field

I have two Querysets (actually, list of dicts) like:
q1 = M1.objects.filter(id=pk).values('p_id', 'q1_quantity')
# q1: <Queryset[{'p_id': 2, 'q1_quantity': 4}, {'p_id': 3, 'q1_quantity': 5}]>
q2 = M2.objects.filter(p_id__in=[q1[x]['p_id'] for x in range(len(q1))]).values('p_id', 'q2_quantity')
# q2: <Queryset[{'p_id': 2, 'q2_quantity': 2}, {'p_id': 2, 'q2_quantity': 5}, {'p_id': 3, 'q2_quantity': 1}, {'p_id': 3, 'q2_quantity': 7}]>
q1 has distinct key:value pairs, while q2 has repeated keys.
1) I want to sum all the values of q2 by common p_id, such that q2 becomes:
# q2: <Queryset[{'p_id': 2, 'q2_quantity': 7}, {'p_id': 3, 'q2_quantity': 8}]>
2) Then, merge q1 and q2 into q3, based on common p_id, like:
q3 = ?
# q3: <Queryset[{'p_id': 2, 'q1_quantity': 4, 'q2_quantity': 7}, {'p_id': 3, 'q1_quantity': 5, 'q2_quantity': 8}]>
I have looked into union(). But don't know how to go about summing the queryset (q2) and then merging it with q1.
Can someone please help me?
The problem is that you're implementing inefficient models, having 2 separate models with repeated fields will force you to make 2 queries. You may want to consider having them all in one model, or the M2 model extends M1.
models.py
class M(models.Model):
p_id = #Your Field...
q1_quantity = #Your Field...
q2_quantity = #Your Field...
then on your views.py
q = M.objects.filter(id=pk).values('p_id', 'q1_quantity', 'q2_quantity')
Potential Issue: In the code you posted, the commented section shows a queryset of more than 1 object and pk as primary key should be unique and therefore should return a unique object queryset.
1) I want to sum all the values of q2 by common p_id, such that q2 becomes:
# q2: <Queryset[{'p_id': 2, 'q2_quantity': 7}, {'p_id': 3, 'q2_quantity': 8}]>
Used itertools.combinations:
from itertools import combinations
compare = []
for a, b in combinations(q2, 2):
if a['p_id'] == b ['p_id']:
a['q2_quantity'] += b['q2_quantity']
if len(compare) <= 0:
compare.append(a)
else:
[compare[d]['q2_quantity'] for d in range(len(compare)) if a['p_id'] == compare[d]['p_id']]
else:
if len(compare) <= 0:
compare.append(a)
compare.append(b)
else:
if any([a['p_id'] == compare[d]['p_id'] for d in range(len(compare))]):
pass
else:
compare.append(a)
if any([b['p_id'] == compare[d]['p_id'] for d in range(len(compare))]):
pass
else:
compare.append(b)
2) Then, merge q1 and q2 into q3, based on common p_id, like:
q3 = ?
# q3: <Queryset[{'p_id': 2, 'q1_quantity': 4, 'q2_quantity': 7}, {'p_id': 3, 'q1_quantity': 5, 'q2_quantity': 8}]>
As per this SO post:
from collections import defaultdict
from itertools import chain
collector = defaultdict(dict)
for collectible in chain(cp, compare):
collector[collectible['p_id']].update(collectible.items())
products = list(collector.values())

Sum values of each key in list of dictionaries python

I have a list of dictionaries which is coming from Django query set.
Like this:
email_sent_count = [
{
'second_follow_count': 1,
'first_follow_count': 1,
'initial_count': 1,
'third_follow_count': 0
},
{
'second_follow_count': 1,
'first_follow_count': 0,
'initial_count': 1,
'third_follow_count': 1
},
{
'second_follow_count': 1,
'first_follow_count': 1,
'initial_count': 1,
'third_follow_count': 1
}
]
Now, I want the sum of each key separately.
like this:
inital_contact = 3
first_followup = 2
second_followup = 3
third_followup = 2
I am trying the following solution:
initial_contact = sum(map(lambda x: x['initial_count'], email_sent_count))
first_followup = sum(map(lambda x: x['first_follow_count'], email_sent_count))
second_followup = sum(map(lambda x: x['second_follow_count'], email_sent_count))
third_followup = sum(map(lambda x: x['third_follow_count'], email_sent_count))
But now, I'm getting 11 keys in all dictionaries and I'm implementing 11 lambda function so is there any good way to solve this issue rather than calling 11 times lambda function
By following ORM I getting above email_sent_count
i = Q(inital_contact__range=(from_date, to_date))
f1 = Q(first_followup__range=(from_date, to_date))
f2 = Q(second_followup__range=(from_date, to_date))
f3 = Q(third_followup__range=(from_date, to_date))
email_count = campaign_contact.filter(i | f1 | f2 | f3).annotate(initial_count=Count('inital_contact'),
first_follow_count=Count('first_followup'),
second_follow_count=Count('second_followup'),
third_follow_count=Count('third_followup'),
).values('initial_count', 'first_follow_count',
'second_follow_count', 'third_follow_count'
So, is there is a solution which is directly working with ORM ?
if you don't mind getting a dictionary as result you could use collections.defaultdict like this:
from collections import defaultdict
sums = defaultdict(int)
for item in email_sent_count:
for key, value in item.items():
sums[key] += value
which results in
defaultdict(<class 'int'>,
{'second_follow_count': 3, 'initial_count': 3,
'first_follow_count': 2, 'third_follow_count': 2})
and you can access the individual sums just like a dictionary: sums['second_follow_count'].
...or maybe even better with collections.Counter:
from collections import Counter
sums = Counter()
for item in email_sent_count:
for key, value in item.items():
sums[key] += value
# Counter({'second_follow_count': 3, 'initial_count': 3,
# 'first_follow_count': 2, 'third_follow_count': 2})
Or if you prefer do do it yourself without using Counter or DefaultDict:
from pprint import pprint
email_sent_count = [
{
'second_follow_count': 1,
'first_follow_count': 1,
'initial_count': 1,
'third_follow_count': 0
},
{
'second_follow_count': 1,
'first_follow_count': 0,
'initial_count': 1,
'third_follow_count': 1
},
{
'second_follow_count': 1,
'first_follow_count': 1,
'initial_count': 1,
'third_follow_count': 1
}
]
# create empty dict with all keys from any inner dict and initialized to 0
alls = dict( (x,0) for y in email_sent_count for x in y)
pprint(alls)
for d in email_sent_count:
for k in d:
alls[k] += d[k]
pprint(alls)
Output:
{'first_follow_count': 0,
'initial_count': 0,
'second_follow_count': 0,
'third_follow_count': 0}
{'first_follow_count': 2,
'initial_count': 3,
'second_follow_count': 3,
'third_follow_count': 2}
Finally, I changed the ORM query that gets me the exact result:
ORM looks like this:
email_sent_count = campaign_contact.filter(i | f1 | f2 | f3
).annotate(initial_count=Count('inital_contact'),
first_follow_count=Count('first_followup'),
second_follow_count=Count('second_followup'),
third_follow_count=Count('third_followup')
).aggregate(initial_sum=Sum('initial_count'),
first_follow_sum=Sum('first_follow_count'),
second_follow_sum=Sum('second_follow_count'),
third_follow_sum=Sum('third_follow_count'))
output...
{'third_follow_sum': 2, 'second_follow_sum': 3, 'first_follow_sum': 2, 'initial_sum': 3}
So, there is not for loops, no lambda... I think performance wise it will work.
And Thanks to all to provide me the solutions, by looking other solution I'm able to solve this. :)

Can't query the sum of values using aggregators

I want to sum the values of all existing rows grouping by another field.
Here's my model structure:
class Answer(models.Model):
person = models.ForeignKey(Person)
points = models.PositiveIntegerField(default=100)
correct = models.BooleanField(default=False)
class Person(models.Model):
# irrelevant model fields
Sample dataset:
Person | Points
------ | ------
4 | 90
3 | 50
3 | 100
2 | 100
2 | 90
Here's my query:
Answer.objects.values('person').filter(correct=True).annotate(points_person=Sum('points'))
And the result (you can see that all the person values are separated):
[{'person': 4, 'points_person': 90}, {'person': 3, 'points_person': 50}, {'person': 3, 'points_person': 100}, {'person': 2, 'points_person': 100}, {'person': 2, 'points_person': 90}]
But what I want (sum of points by each person):
[{'person': 4, 'points_person': 90}, {'person': 3, 'points_person': 150}, {'person': 2, 'points_person': 190}]
Is there any way to achieve this using only queryset filtering?
Thanks!
Turns out I had to do the inverse filtering, by the Person's and not the Answers, like so:
Person.objects.filter(answer__correct=True).annotate(points=Sum('answer__points'))
Now I get the total summed points for each person correctly.