How to get data for a given search query - django

I have a table for ManyToMany relationship. Where each Tutor need to input multiple days that he wants to tutor student. Like:
Availability Tutor:
user available_day time
t1 Sun 6,7,8
t2 mon 3,4
t1 mon 1,2
I would like to get all tutor where availablity is based on search day.
Model:
class TutorProfile(models.Model):
user = models.ForeignKey(
settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
tutor_availablility = models.ManyToManyField(
Day)
queryset:
def get_queryset(self):
subject_param = self.request.GET.get('subject')
bookday = self.request.GET.get('book_day')
grade = self.request.GET.get('grade')
li = list(bookday.split(","))
print("Book Info:", li)
for i in range(len(li)):
print(li[i])
_day_name = datetime.strptime(li[i], "%d-%m-%Y").strftime("%A")
print("Day Name:", _day_name)
day_num = Day.objects.get(day_name=_day_name)
print("Day_Num",day_num)
result = TutorProfile.objects.filter(
tutor_availablility=day_num).all()
return result
When I run this query it only loop over one time for one day but not for multiple days.
Now how may I run this query so that it will not stop after one loop.
After One loop it says: An exception occurred
An exception occurred
I tried to apply try catch block here. But getting no clue. Why stop loop after one cycle?

After one loop, you are returning the results. Instead, use the Q object to store the conditions and then filter the queryset outside of the for loop:
from django.db.models import Q
...
query = Q()
for i in range(len(li)):
_day_name = datetime.strptime(li[i], "%d-%m-%Y").strftime("%A")
day_num = Day.objects.get(day_name=_day_name)
query2 = query | Q(
tutor_availablility=day_num)
return TutorProfile.objects.filter(query)
Update
Although the following code should not change the outcome, but try like this:
li = list(bookday.split(","))
query = list(map(lambda x: datetime.strptime(x, "%d-%m-%Y").strftime("%A"), li)
return TutorProfile.objects.filter(tutor_availability__day_name__in = query)

Related

Aggregation on 'one to many' for matrix view in Django

I have two tables like below.
These are in 'one(History.testinfoid) to many(Result.testinfoid)' relationship.
(Result table is external database)
class History(models.Model): # default database
idx = models.AutoField(primary_key=True)
scenario_id = models.ForeignKey(Scenario)
executor = models.CharField(max_length=255)
createdate = models.DateTimeField()
testinfoid = models.IntegerField(unique=True)
class Result(models.Model): # external (Result.objects.using('external'))
idx = models.AutoField(primary_key=True)
testinfoid = models.ForeignKey(History, to_field='testinfoid', related_name='result')
testresult = models.CharField(max_length=10)
class Meta:
unique_together = (('idx', 'testinfoid'),)
So, I want to express the Count by 'testresult' field in Result table.
It has some condition such as 'Pass' or 'Fail'.
I want to express a count query set for each condition. like this.
[{'idx': 1, 'pass_count': 10, 'fail_count': 5, 'executor': 'someone', ...} ...
...
{'idx': 10, 'pass_count': 1, 'fail_count': 10, 'executor': 'someone', ...}]
Is it possible?
It is a two level aggregation where the second level should be displayed as table columns - "matrix view".
A) Solution with Python loop to create columns with annotations by the second level ("testresult").
from django.db.models import Count
from collections import OrderedDict
qs = (History.objects
.values('pk', 'executor', 'testinfoid',... 'result__testresult')
.annotate(result_count=Count('pk'))
)
qs = qs.filter(...).order_by(...)
data = OrderedDict()
count_columns = ('pass_count', 'fail_count', 'error_count',
'expected_failure_count', 'unexpected_success_count')
for row in qs:
data.setdefault(row.pk, dict.fromkeys(count_columns, 0)).update(
{(k if k != result_count else row['result__testresult'] + '_count'): v
for k, v in row_items()
if k != 'result__testresult'
}
)
out = list(data.values())
The class OrderedDict is used to preserve order_by().
B) Solution with Subquery in Django 1.11+ (if the result should be a queryset. e.g. to be sorted or filtered finally in an Admin view by clicking, and if a more complicated query is acceptable and number of columns *_count is very low.). I can write a solution with subquery, but I'm not sure if the query will be fast enough with different database backends. Maybe someone other answers.

Building up subqueries of derived django fields

I have a few transformations I need to perform on my table before I aggregate.
I need to multiply transaction_type (which is either 1 or -1) by amount to yield a signed_amount. Then I need to sum all signed_amounts by primary_category (which is a foreign key to secondary category which is a foreign key of my table).
DEBIT = -1
CREDIT = 1
TRANSACTION_TYPE_CHOICES = (
(DEBIT, 'debit'),
(CREDIT, 'credit'),
)
class Transaction(models.Model):
amount = models.DecimalField(max_digits=7, decimal_places=2)
transaction_type = models.IntegerField(choices=TRANSACTION_TYPE_CHOICES)
secondary_category = models.ForeignKey(Secondary_Category)
class Primary_Category(models.Model):
name = models.CharField("Category Name", max_length=30)
category = models.ForeignKey(Primary_Category_Bucket)
class Secondary_Category(models.Model):
name = models.CharField("Category Name", max_length=30)
primary_category = models.ForeignKey(Primary_Category)
I'm stuck on the first bit though.
from django.db.models import Sum, Count, F
original_transactions = Transaction.objects.all()
original_transactions.signed_amount = F('transaction_type') * F('amount')
for transaction in original_transactions:
print transaction.signed_amount
When I try to sanity check that signed_amount is being calculated, I get an error that 'Transaction' object has no attribute 'signed_amount'. I don't want to save signed_amount to the database. I just want to generate it as derived field so I can calculate my totals.
How do I calculate this derived field and subsequently aggregate by primary_category.name?
User python decorator property on a method for class Transaction:
class Transaction(models.Model):
amount = models.DecimalField(max_digits=7, decimal_places=2)
transaction_type = models.IntegerField(choices=TRANSACTION_TYPE_CHOICES)
secondary_category = models.ForeignKey(Secondary_Category)
#property
def signed_amount(self):
return self.amount * self.transaction_type
Then for each Transaction object you can do transaction.signed_amount.
I'm not sure if the aggregation part could be done using queries, but if you don't have that many PrimaryCategory, then python would be good enough to achieve it.
Or you can do this.
all_transactions = Transaction.objects.all().order_by('secondary_category__primary_category_id')
sum = 0
if all_transactions:
primary_category_id = all_transactions[0].secondary_category.primary_category_id
for transaction in all_transactions:
if primary_category_id == transaction.secondary_category.primary_category_id:
sum += (transaction.amount * transaction_type)
else:
sum = (transaction.amount * transaction_type)
print sum

Filtering over a list of date ranges in one ORM call

I am having the following structure:
class Player(models.Model):
name = models.CharField(max_length=50)
dob = models.DateField(max_length=80)
class Game(models.Model):
name = models.CharField(max_length=50)
class Attempts(models.Model):
player = models.ForeignKey("Player")
game = models.ForeignKey("Game")
won = models.BooleanField()
The class Attempts logs the attempts of each player and whether he has won or lost a game.Now I want to get number of people attempting a particular game in agr groups.For example people aged 0-18 have attempted a game 5 times and so on.I have consulted the following this post in django users and made the following code:
ranges = [(0,18),(18,25),(25,50)]
now = datetime.now()
groups = {}
for (a,b) in ranges:
# print str(a) + " " +str(b)
newer_date = (now - relativedelta(years=a)).date()
older_date = (now - relativedelta(years=b)).date()
print str(newer_date)
print str(older_date)
groups[str(a)+","+str(b)] = Attempts.objects.filter(game__pk = "101",player__dob__range=(older_date,newer_date)).count()
This also gives result correctly as:
{"0,18": 2, "25,50": 1, "18,25": 1}
How ever the queries made by this code are equal to number of categories in the ranges.I want to some how get this data in one single ORM query.Is it possible to do so ?

Find object in datetime range

Let's assume that I have modeL;
class MyModel(...):
start = models.DateTimeField()
stop = models.DateTimeField(null=True, blank=True)
And I have also two records:
start=2012-01-01 7:00:00 stop=2012-01-01 14:00:00
start=2012-01-01 7:00:03 stop=2012-01-01 23:59:59
Now I want to find the second query, so start datetime should be between start and stop, and stop should have hour 23:59:59. How to bould such query?
Some more info:
I think this requires F object. I want to find all records where start -> time is between another start -> time and stop -> time, and stop -> time is 23:59:59, and date is the same like in start
YOu can use range and extra:
from django.db.models import Q
q1 = Q( start__range=(start_date_1, end_date_1) )
q1 = Q( start__range=(start_date_2, end_date_2) )
query = (''' EXTRACT(hour from end_date) = %i
and EXTRACT(minute from end_date) = %i
and EXTRACT(second from end_date) = %i''' %
(23, 59,59)
)
MyModel.objects.filter( q1 | q2).extra(where=[query])
Notice: Posted before hard answer requirement changed 'time is 23:59:59, and date is the same like in start'
To perform the query: "start datetime should be between start and stop"
MyModel.objects.filter(start__gte=obj1.start, start__lte=obj1.stop)
I don't quite understand your second condition, though. Do you want it to match only objects with hour 23:59:59, but for any day?
dt = '2012-01-01 8:00:00'
stop_hour = '23'
stop_minute = '59'
stop_sec = '59'
where = 'HOUR(stop) = %(hour)s AND MINUTE(stop) = %(minute)s AND SECOND(stop) = %(second)s' \
% {'hour': stop_hour, 'minute': stop_minute, 'seconds': stop_ec}
objects = MyModel.objects.filter(start__gte=dt, stop__lte=dt) \
.extra(where=[where])

Django ORM equivalent for this SQL..calculated field derived from related table

I have the following model structure below:
class Master(models.Model):
name = models.CharField(max_length=50)
mounting_height = models.DecimalField(max_digits=10,decimal_places=2)
class MLog(models.Model):
date = models.DateField(db_index=True)
time = models.TimeField(db_index=True)
sensor_reading = models.IntegerField()
m_master = models.ForeignKey(Master)
The goal is to produce a queryset that returns all the fields from MLog plus a calculated field (item_height) based on the related data in Master
using Django's raw sql:
querySet = MLog.objects.raw('''
SELECT a.id,
date,
time,
sensor_reading,
mounting_height,
(sensor_reading - mounting_height) as item_height
FROM db_mlog a JOIN db_master b
ON a.m_master_id = b.id
''')
How do I code this using Django's ORM?
I can think of two ways to go about this without relying on raw(). The first is pretty much the same as what #tylerl suggested. Something like this:
class Master(models.Model):
name = models.CharField(max_length=50)
mounting_height = models.DecimalField(max_digits=10,decimal_places=2)
class MLog(models.Model):
date = models.DateField(db_index=True)
time = models.TimeField(db_index=True)
sensor_reading = models.IntegerField()
m_master = models.ForeignKey(Master)
def _get_item_height(self):
return self.sensor_reading - self.m_master.mounting_height
item_height = property(_get_item_height)
In this case I am defining a custom (derived) property for MLog called item_height. This property is calculated as the difference of the sensor_reading of an instance and the mounting_height of its related master instance. More on property here.
You can then do something like this:
In [4]: q = MLog.objects.all()
In [5]: q[0]
Out[5]: <MLog: 2010-09-11 8>
In [6]: q[0].item_height
Out[6]: Decimal('-2.00')
The second way to do this is to use the extra() method and have the database do the calculation for you.
In [14]: q = MLog.objects.select_related().extra(select =
{'item_height': 'sensor_reading - mounting_height'})
In [16]: q[0]
Out[16]: <MLog: 2010-09-11 8>
In [17]: q[0].item_height
Out[17]: Decimal('-2.00')
You'll note the use of select_related(). Without this the Master table will not be joined with the query and you will get an error.
I always do the calculations in the app rather than in the DB.
class Thing(models.Model):
foo = models.IntegerField()
bar = models.IntegerField()
#Property
def diff():
def fget(self):
return self.foo - self.bar
def fset(self,value):
self.bar = self.foo - value
Then you can manipulate it just as you would any other field, and it does whatever you defined with the underlying data. For example:
obj = Thing.objects.all()[0]
print(obj.diff) # prints .foo - .bar
obj.diff = 4 # sets .bar to .foo - 4
Property, by the way, is just a standard property decorator, in this case coded as follows (I don't remember where it came from):
def Property(function):
keys = 'fget', 'fset', 'fdel'
func_locals = {'doc':function.__doc__}
def probeFunc(frame, event, arg):
if event == 'return':
locals = frame.f_locals
func_locals.update(dict((k,locals.get(k)) for k in keys))
sys.settrace(None)
return probeFunc
sys.settrace(probeFunc)
function()
return property(**func_locals)