I am trying to find a better way to loop over orders for the next seven days including today, what I have already:
unfilled_orders_0 = Model.objects.filter(delivery_on__date=timezone.now() + timezone.timedelta(0))
context['todays_orders'] = unfilld_orders_0.aggregate(field_1_sum=Sum('field_1'), field_2_sum=Sum('field_2'),field_3_sum=Sum('field_3'), field_4_sum=Sum('field_4'),field_5_sum=Sum('field_5'))
I'm wondering if I can somehow avoid having to do this seven times--one for each day. I assume there is a more efficient way to do this.
You can do this with a single ORM / db query, by providing Sum with an extra filter argument:
days_ahead = 7
fields = ["field_1", "field_2", ...]
aggregate_kwargs = {
f"s_{field}_{day}": Sum(field, filter=Q(delivery_on__date=now+timedelta(days=day)))
for field in fields
for day in range(days_ahead)
}
unfilled_orders = Model.objects.filter(delivery_on__date__lt=now+timedelta(days=days_ahead)
context.update(unfilled_orders.aggregate(**aggregate_kwargs))
You can approach it with a for loop and store the data in the context like so
from django.utils import timezone
from django.db.models import Sum
context = {}
for i in range(7):
qs = Model.objects.filter(delivery_on__date=(timezone.now() + timezone.timedelta(i)).date())
context = {}
context[f'orders_{i}'] = qs.aggregate(
field_1_sum=Sum('field_1'),
field_2_sum=Sum('field_2'),
field_3_sum=Sum('field_3'),
field_4_sum=Sum('field_4'),
field_5_sum=Sum('field_5'))
This query will hit 7 times the database, otherwise you can use another approach which will hit the db only once
context = {}
qs = Model.objects.filter(delivery_on__date__lte=timezone.now()-timezone.timedelta(days=7)).order_by('delivery_on')
dates = qs.values('delivery_on__date', flat=True).distinct()
for i in dates:
_qs = qs.filter(create_ts__date=i)
context[f'orders_{i}'] = _qs.aggregate(
field_1_sum=Sum('field_1'),
field_2_sum=Sum('field_2'),
field_3_sum=Sum('field_3'),
field_4_sum=Sum('field_4'),
field_5_sum=Sum('field_5'))
You define how many days backwards the qs will be including all orders then distinct the dates and filter the already filtered qs for the dates.
Related
this is my model and i am using postgresql:
class TripRequest(models.Model):
passenger = models.ForeignKey('user.Passenger', on_delete=models.DO_NOTHING)
beginning_point = models.PointField()
destination_point = models.PointField()
trip_beginning_time = models.DateTimeField()
...
i'm not very familiar with advance orm queries
i want to group trip requests that have nearest (for example in a specific hour) beginning time and after that filter it furthermore like nearest beginning point or nearest driver.
now there are answers on SO that group the query and return the count or other information of all the group items but i want to just group the instances by hour and have a collection of instances for further filtering how can i do that?
You can use django's Trunc database function:
from django.db.models.functions import Trunc
def get_dt_stats(model):
current_date = timezone.now().replace(hour=0, minute=0, second=0, microsecond=0)
proxy = model.objects.filter(trip_beginning_time__gte=current_date) # Or any filter you need
proxy = proxy.values(x=Trunc('trip_beginning_time', 'hour', tzinfo=tzlocal.get_localzone()))
proxy = proxy.annotate(amount=Count('id'))
return {k['x'].strftime('%Y-%m-%d %H:00'): k['amount'] for k in list(proxy)}
That function returns a dict with counts of model objects grouped by hours of trip_beginning_time
I have two models:
Model A is an AbstractUserModel and Model B
class ModelB:
user = ForeignKey(User, related_name='modelsb')
timestamp = DateTimeField(auto_now_add=True)
What I want to find is how many users have at least one ModelB object created at least in 3 of the 7 past days.
So far, I have found a way to do it but I know for sure there is a better one and that is why I am posting this question.
I basically split the query into 2 parts.
Part1:
I added a foo method inside the User Model that checks if a user meets the above conditions
def foo(self):
past_limit = starting_date - timedelta(days=7)
return self.modelsb.filter(timestamp__gte=past_limit).order_by('timestamp__day').distinct('timestamp__day').count() > 2
Part 2:
In the Custom User Manager, I find the users that have more than 2 modelsb objects in the last 7 days and iterate through them applying the foo method for each one of them.
By doing this I narrow down the iterations of the required for loop. (basically its a filter function but you get the point)
def boo(self):
past_limit = timezone.now() - timedelta(days=7)
candidates = super().get_queryset().annotate(rc=Count('modelsb', filter=Q(modelsb__timestamp__gte=past_limit))).filter(rc__gt=2)
return list(filter(lambda x: x.foo(), candidates))
However, I want to know if there is a more efficient way to do this, that is without the for loop.
You can use conditional annotation.
I haven't been able to test this query, but something like this should work:
from django.db.models import Q, Count
past_limit = starting_date - timedelta(days=7)
users = User.objects.annotate(
modelsb_in_last_seven_days=Count('modelsb__timestap__day',
filter=Q(modelsb__timestamp__gte=past_limit),
distinct=True))
.filter(modelsb_in_last_seven_days__gte = 3)
EDIT:
This solution did not work, because the distinct option does specify what field makes an entry distinct.
I did some experimenting on my own Django instance, and found a way to make this work using SubQuery. The way this works is that we generate a subquery where we make the distinction ourself.
counted_modelb = ModelB.objects
.filter(user=OuterRef('pk'), timestamp__gte=past_limit)
.values('timestamp__day')
.distinct()
.annotate(count=Count('timestamp__day'))
.values('count')
query = User.objects
.annotate(modelsb_in_last_seven_days=Subquery(counted_modelb, output_field=IntegerField()))
.filter(modelsb_in_last_seven_days__gt = 2)
This annotates each row in the queryset with the count of all distinct days in modelb for the user, with a date greater than the selected day.
In the subquery I use values('timestamp__day') to make sure I can do distinct() (Because a combination of distinct('timestamp__day') and annotate() is unsupported.)
I have the following model in django:
class task(models.Model):
admin = models.BooleanField()
date = modesl.DateField()
I am trying to achieve a filter which provides me with a query_set that prioritize if admin = True
So assume I have these 3 rows :
admin = True , date = 01-01-2019
admin = False , date = 01-01-2019
admin = False , date = 02-02-2019
The output of the query set will be :
admin = True , date = 01-01-2019
admin = False , date = 02-02-2019
it should filter out the row with 01-01-2019 which admin=False because there is already an admin=True row which should take prioritization.
I could do it by fetching all and removing it from the query_set myself, but want to make sure there is no other way of doing it before.
Rather than looping through the QuerySet and removing them yourself, one thing you could do is:
Fetch all the dates where admin is True
Fetch all the objects where either:
i. admin is True
ii. The date is not in part 1 (e.g. admin is False)
This can be achieved with the following:
from django.db.models import Q
true_dates = task.objects.filter(admin=True).values_list("date", flat=True)
results = task.objects.filter(Q(admin=True)|~Q(date__in=true_dates))
This will most likely be more efficient than looping through your results yourself.
Note that since querysets are 'lazy' (this means only evaluated when they absolutely have to be) this will result in just 1 db hit
Tim's answer is close, but incomplete, because he doesn't use Subquery().
This answer provides the same results, without having an additional query hit the database:
from django.db.models import Subquery, Q
dates = Task.objects.filter(admin=True)
tasks = Task.objects.filter(Q(admin=True) | ~Q(date__in=Subquery(dates.values('date')))
On o postgresql db based Django, how can I filter by time for a datetimefield as below?
class Foo(models.Model):
start_date = models.DateTimeField()
end_date = models.DateTimeField()
IE: I want to filter Foo objects with "16:30" start_date and "19:00" end_date.
Thanks.
What about adding in a TimeField?
http://docs.djangoproject.com/en/dev/ref/models/fields/#timefield
Otherwise you would need to write a custom query to take advantage of the database´s time capabilites since DateTimeFields don't have that capability natively in Django.
You could consider writing a function to denormalize hours and minutes from start_date to a new start_time field and then query the start_time field.
Solution to my own q:
def query_by_times(start_hour, start_min, end_hour, end_min):
query = 'EXTRACT(hour from start_date) = %i and EXTRACT(minute from start_date) = %i and EXTRACT(hour from end_date) = %i and EXTRACT(minute from end_date) = %i' % (start_hour, start_min, end_hour, end_min)
return Foo.objects.extra(where=[query])
In your situation, database normalization looks like thebest solution, since execution time and system load will rise as your related database table keeps more records...
Another solution to do this without using database filer functions is using filter alongside lambda:
from datetime import time
records = Foo.objects.all()
filtered = filter(lambda x: (time(16,30)==x.start_date.time() and time(19,0)==x.end_date.time()), records)
But using this on a large dataset will need too much time and system resource.
Try
Foo.objects.filter(start_date = MyDate1, end_date = MyDate2)
where MyDate1 and MyDate2 are your defined datetime objects. It should work, right?
I think which you need uses the range[1].
[1]http://docs.djangoproject.com/en/1.2/ref/models/querysets/#range
I need to find data within a certain set of parameters
I am building a small booking system, that lets user see what vehicles are available for booking for their little safari trip.
The system has bookings that have been entered previously or made previously by a client.
If a booking's pickup_date = 2011-03-01 and dropoff_date = 2011-03-15 and I run a query with pickup=2011-03-09 and dropoff=2011-03-14 in my views as below, it doesn't return any results to see if a booking within that timeframe has been made.
views.py
def dates(request, template_name='groups/_dates.html'):
pickup=request.GET.get('pickup','None');
dropoff=request.GET.get('dropoff','None');
order = Order.objects.filter(pick_up__lte=pickup).filter(drop_off__gte=dropoff)
context = {'order':order,}
return render_to_response(template_name,context,
context_instance=RequestContext(request))
Any suggestions on how to do this?
Or should I be looking at an alternate way of running this query?
Could it be posible that as your passing the raw string to the queryset is not on the correct format, try converting the strings to datetime objects.
Later you can try using the range lookup is more efficient on some DB engines and more simple to read and code.
from django.db.models import Q
start_date = datetime.date(2005, 1, 1)
end_date = datetime.date(2005, 3, 31)
orders = Order.objects.filter(drop_off__gte=start_date, pick_up__lte=end_date)
# Or maybe better
orders = Order.objects.filter(Q(drop_off__gte=start_date), Q(pick_up__lte=end_date))
Can you try this :
order = Order.objects.filter(pick_up**__date__**lte=pickup).filter(drop_off**__date__**gte=dropoff)
https://docs.djangoproject.com/fr/2.0/ref/models/querysets/#date