I am relatively new to Django. I'm having problem when filtering data. I have two models, given below:
Account(models.Model):
name = models.CharField(max_length=60)
hotel = models.ForeignKey(Hotel)
account_type = models.CharField(choices=ACCOUNT_TYPE, max_length=30)
Transaction(models.Model):
account = models.ForeignKey(Account, related_name='transaction')
transaction_type = models.CharField(choices=TRANSACTION_TYPE, max_length=15)
description = models.CharField(max_length=100, blank=True, null=True)
date_created = models.DateTimeField(default=timezone.now)
ACCOUT_TYPE is:
ACCOUNT_TYPE = (
(0, 'Asset'),
(1, 'Liabilities'),
(2, 'Equity'),
(3, 'Income'),
(4, 'Expense')
)
I want to filter all the transactions where the account type is Income and Expense within a given date range. How can I combine those filters in Django?
I have tried like this:
income_account = Account.objects.filter(account_type=3)
expense_account = Account.objects.filter(account_type=4)
transactions = Transaction.objects.filter(Q(
account=income_account,
date_created__gte=request.data['start_date'],
date_created__lte=request.data['end_date']
) & Q(
account=expense_account,
date_created__gte=request.data['start_date'],
date_created__lte=request.data['end_date'])).order_by('date_created')
But it's not working. It raises the following error:
ProgrammingError: more than one row returned by a subquery used as an expression
income_account and expense_account is not single object, it is a list of objects. So instead of this account=income_account and this account=expense_account try to use in: account__in=income_account and account__in=expense_account.
Also you probably could simplify queryset like this:
accounts = Account.objects.filter(Q(account_type=3) | Q(account_type=4))
transactions = Transaction.objects.filter(
account__in=accounts,
date_created__gte=request.data['start_date'],
date_created__lte=request.data['end_date']
).order_by('date_created')
Instead of having multiple querysets, you can have only one, as Q allows ORing of filters. You could do:
Transaction.objects.filter(
(Q(account__account_type=3) | Q(account__account_type=4)) &
Q(date_created__range=[start_date, end_date])
)
The __range can be used to get dates between the specified start_date and end_date.
You can always use in to lookup records by multiple values. So, if you want Transaction where ACCOUNT_TYPE are Income, Expenseyou can use it like this.
Transaction.objects.filter(Q(account__in=[3,4]) & Q(date_created__gte=request.data['start_date']) & Q(date_created__lte=request.data['end_date'])).order_by('date_created')
This will work for you:-
result = Account.objects.filter((account_type__in['Income','Expense'])
OR
result = Account.objects.filter((account_type__in['0','4'])
I have put 0 and 4 as string because you have mention account_type as CharField.
Related
class IncomeStream(models.Model):
product = models.ForeignKey(Product, related_name="income_streams")
from_date = models.DateTimeField(blank=True, null=True)
to_date = models.DateTimeField(blank=True, null=True)
value = MoneyField(max_digits=14, decimal_places=2, default_currency='USD')
class Product(models.Model):
...
class Sale(models.Model):
product = models.ForeignKey(Product, related_name="sales")
created_at = models.DateTimeField(auto_now_add=True)
...
With the above model, suppose I want to add a value to some Sales using .annotate.
This value is called cpa (cost per action): cpa is the value of the IncomeStream whose from_date and to_date include the Sale created_at in their range.
Furthermore, from_date and to_date are both nullable, in which case we assume they mean infinity.
For example:
<IncomeStream: from 2021-10-10 to NULL, value 10$, product TEST>
<IncomeStream: from NULL to 2021-10-09, value 5$, product TEST>
<Sale: product TEST, created_at 2019-01-01, [cpa should be 5$]>
<Sale: product TEST, created_at 2021-11-01, [cpa should be 10$]>
My question is: is it possible to write all these conditions using only the Django ORM and annotate? If yes, how?
I know F objects can traverse relationships like this:
Sale.objects.annotate(cpa=F('product__income_streams__value'))
But then where exactly can I write all the logic to determine which specific income_stream it should pick the value from?
Please suppose no income stream have overlapping dates for the same product, so the above mentioned specs never result in conflicts.
Something like this should get you started
subquery = (
IncomeStream
.objects
.values('product') # group by product primary key i.e. product_id
.filter(product=OuterRef('product'))
.filter(from_date__gte=OuterRef('created_at'))
.filter(to_date__lte=OuterRef('created_at'))
.annotate(total_value=Sum('value'))
)
Then with the subquery
Sale
.objects
.annotate(
cpa=Subquery(
subquery.values('total_value')
) # subquery should return only one row so
# so just need the total_value column
)
Without the opportunity to play around with this in the shell myself I not 100%. It should be close though anyway.
Question is regarding filtering X most recent entries in each category of queryset.
Goal is like this:
I have a incoming queryset based on the following model.
class UserStatusChoices(models.TextChoices):
CREATOR = 'CREATOR'
SLAVE = 'SLAVE'
MASTER = 'MASTER'
FRIEND = 'FRIEND'
ADMIN = 'ADMIN'
LEGACY = 'LEGACY'
class OperationTypeChoices(models.TextChoices):
CREATE = 'CREATE'
UPDATE = 'UPDATE'
DELETE = 'DELETE'
class EntriesChangeLog(models.Model):
content_type = models.ForeignKey(
ContentType,
on_delete=models.CASCADE,
)
object_id = models.PositiveIntegerField(
)
content_object = GenericForeignKey(
'content_type',
'object_id',
)
user = models.ForeignKey(
get_user_model(),
verbose_name='user',
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name='access_logs',
)
access_time = models.DateTimeField(
verbose_name='access_time',
auto_now_add=True,
)
as_who = models.CharField(
verbose_name='Status of the accessed user.',
choices=UserStatusChoices.choices,
max_length=7,
)
operation_type = models.CharField(
verbose_name='Type of the access operation.',
choices=OperationTypeChoices.choices,
max_length=6,
)
And I need to filter this incoming queryset in a such way to keep only 4 most recent objects (defined by access_time field) in each category. Categories are defined by ‘content_type_id’ field and there are 3 possible options.
Lets call it ‘option1’, ‘option2’ and ‘option3’
This incoming queryset might contain different amount of objects of 1,2 or all 3 categories. This is can’t be predicted beforehand.
DISTINCT is not possible to use as after filtering operation this queryset might be ordered.
I managed to get 1 most recent object in a following way:
# get one most recent operation in each category
last_operation_time = Subquery(
EntriesChangeLog.objects.filter(user=OuterRef('user')).values('content_type_id').
annotate(last_access_time=Max(‘access_time’)).values_list('last_access_time', flat=True)
)
queryset.filter(access_time__in=last_operation_time)
But I have a hard time to figure out how to get last 4 most recent objects instead of last one.
This is needed for Django-Filter and need to be done in one query.
DB-Postgres 12
Do you have any ideas how to do such filtration?
Thanks...
pk_to_rank = queryset.annotate(rank=Window(
expression=DenseRank(),
partition_by=('content_type_id',),
order_by=F('access_time').desc(),
)).values_list('pk', 'rank', named=True)
pks_list = sorted(log.pk for log in pk_to_rank if log.rank <= value)
return queryset.filter(pk__in=pks_list)
Managed to do it only this way by spliting queryset in 2 parts. Option with 3 unions is also possible but what if we have 800 options instead 3 - make 800 unions()??? ges not...
I'm having problem filtering in django-models.
I want to return all records of a particular animal but excluding the last item based on the latest created_at value and sorted in a descending order.
I have this model.
class Heat(models.Model):
# Fields
performer = models.CharField(max_length=25)
is_bred = models.BooleanField(default=False)
note = models.TextField(max_length=250, blank=True, null=True)
result = models.BooleanField(default=False)
# Relationship Fields
animal = models.ForeignKey(Animal, related_name='heats', on_delete=models.CASCADE)
created_at = models.DateTimeField(auto_now_add=True, editable=False)
last_updated = models.DateTimeField(auto_now=True, editable=False)
I was able to achieved the desired result by this raw sql script. But I want a django approach.
SELECT
*
FROM
heat
WHERE
heat.created_at != (SELECT MAX((heat.created_at)) FROM heat)
AND heat.animal_id = '2' ORDER BY heat.created_at DESC;
Please help.
It will be
Heat.objects.order_by("-created_at")[1:]
For a particular animal it will then be:
Heat.objects.filter(animal_id=2).order_by("-created_at")[1:]
where [1:] on a queryset has a regular python slice syntax and generates the correct SQL code. (In this case simply removes the first / most recently created element)
Upd: as #schwobaseggl mentioned, in the comments, slices with negative index don't work on django querysets. Therefore the objects are reverse ordered first.
I just converted your SQL query to Django ORM code.
First, fetch the max created_at value using aggregation and do an exclude.
from django.db.models import Max
heat_objects = Heat.objects.filter(
animal_id=2
).exclude(
created_at=Heat.objects.all().aggregate(Max('created_at'))['created_at__max']
)
Get last record:
obj= Heat.objects.all().order_by('-id')[0]
Make query:
query = Heat.objects.filter(animal_id=2).exclude(id=obj['id']).all()
The query would be :
Heat.objects.all().order_by('id')[1:]
You could also put any filter you require by replacing all()
I have a History model like below
class History(models.Model):
class Meta:
app_label = 'subscription'
ordering = ['-start_datetime']
subscription = models.ForeignKey(Subscription, related_name='history')
FREE = 'free'
Premium = 'premium'
SUBSCRIPTION_TYPE_CHOICES = ((FREE, 'Free'), (Premium, 'Premium'),)
name = models.CharField(max_length=32, choices=SUBSCRIPTION_TYPE_CHOICES, default=FREE)
start_datetime = models.DateTimeField(db_index=True)
end_datetime = models.DateTimeField(db_index=True, blank=True, null=True)
cancelled_datetime = models.DateTimeField(blank=True, null=True)
Now i have a queryset filtering like below
users = get_user_model().objects.all()
queryset = users.exclude(subscription__history__end_datetime__lt=timezone.now())
The issue is that in the exclude above it is checking end_datetime for all the rows for a particular history object. But i only want to compare it with first row of history object.
Below is how a particular history object looks like. So i want to write a queryset filter which can do datetime comparison on first row only.
You could use a Model Manager method for this. The documentation isn't all that descriptive, but you could do something along the lines of:
class SubscriptionManager(models.Manager):
def my_filter(self):
# You'd want to make this a smaller query most likely
subscriptions = Subscription.objects.all()
results = []
for subscription in subscriptions:
sub_history = subscription.history_set.first()
if sub_history.end_datetime > timezone.now:
results.append(subscription)
return results
class History(models.Model):
subscription = models.ForeignKey(Subscription)
end_datetime = models.DateTimeField(db_index=True, blank=True, null=True)
objects = SubscriptionManager()
Then: queryset = Subscription.objects().my_filter()
Not a copy-pastable answer, but shows the use of Managers. Given the specificity of what you're looking for, I don't think there's a way to get it just via the plain filter() and exclude().
Without knowing what your end goal here is, it's hard to say whether this is feasible, but have you considered adding a property to the subscription model that indicates whatever you're looking for? For example, if you're trying to get everyone who has a subscription that's ending:
class Subscription(models.Model):
#property
def ending(self):
if self.end_datetime > timezone.now:
return True
else:
return False
Then in your code: queryset = users.filter(subscription_ending=True)
I have tried django's all king of expressions(aggregate, query, conditional) but was unable to solve the problem so i went with RawSQL and it solved the problem.
I have used the below SQL to select the first row and then compare the end_datetime
SELECT (end_datetime > %s OR end_datetime IS NULL) AS result
FROM subscription_history
ORDER BY start_datetime DESC
LIMIT 1;
I will select my answer as accepted if not found a solution with queryset filter chaining in next 2 days.
I have 2 tables as
class ItemFollowers(models.Model):
item = models.ForeignKey(Items, models.DO_NOTHING, db_column='item')
user = models.ForeignKey(AuthUser, models.DO_NOTHING, db_column='user')
And the other one is
class UsrPosts(models.Model):
item = models.ForeignKey('Items', models.DO_NOTHING, db_column='item')
# Some other fields
How can I select the UsrPosts related to the items followed by some user? i.e. I can have records in ItemFollowers like (item0, user0), (item1, user0), (item5, user0). I need to filter UsrPosts based upon the user (aka. request.user.id)
Here is a inefficient non-working way to get UsrPostts
itms = ItemFollowers.objects.filter(user_id=request.user.id)
qry = Q(item_id=itms[0].item.id) | ..... | Q(item_id=itms[N].item.id)
posts = UsrPosts.objects.filter(qry)
Is there some filter magic to get it in one transaction?
itms = ItemFollowers.objects.filter(user_id=request.user.id).values_list('item')
posts = UsrPosts.objects.filter(item__in = itms)