I currently have two different models.
class Journal(models.Model):
date = models.DateField()
from_account = models.ForeignKey(Account,related_name='transferred_from')
to_account = models.ForeignKey(Account,related_name='transferred_to')
amount = models.DecimalField(max_digits=8, decimal_places=2)
memo = models.CharField(max_length=100,null=True,blank=True)
class Ledger(models.Model):
date = models.DateField()
bank_account = models.ForeignKey(EquityAccount,related_name='paid_from')
account = models.ForeignKey(Account)
amount = models.DecimalField(max_digits=8, decimal_places=2)
name = models.ForeignKey(Party)
memo = models.CharField(max_length=100,null=True,blank=True)
I am creating a report in a view and get the following error:
Merging 'ValuesQuerySet' classes must involve the same values in each case.
What I'm trying to do is only pull out the fields that are common so I can concatenate both of them e.g.
def report(request):
ledger = GeneralLedger.objects.values('account').annotate(total=Sum('amount'))
journal = Journal.objects.values('from_account').annotate(total=Sum('amount'))
report = ledger & journal
...
If I try to make them exactly the same to test e.g.
def report(request):
ledger = GeneralLedger.objects.values('memo').annotate(total=Sum('amount'))
journal = Journal.objects.values('memo').annotate(total=Sum('amount'))
report = ledger & journal
...
I get this error:
Cannot combine queries on two different base models.
Anyone know how this can be accomplished?
from itertools import chain
report = chain(ledger, journal)
Itertools for the win!
If you want to do an Union, you should convert these querysets into python set objects.
If it is possible to filter the queryset itself rightly, you should really do that!
Use itertools.chain:
from itertools import chain
report = list(chain(ledger, journal))
Note: you need to turn the resulting object into a list for Django to be able to process it.
I had the same issue. I solved it using the union method combined_queryset = qs1.union(qs2)
Using your example: report = ledger.union(journal)
Related
I would simply like to transform a prefetch_related query into a Panda dataframe with all the information from the two models below. This should be very simple but somehow nothing works. I get a 'Capture_set is not defined' with the code below.
Any idea ?
class Capture(models.Model):
species_name = models.CharField(max_length=50)
total_capture = models.IntegerField()
class Species(models.Model):
species_name = models.ForeignKey(Capture, on_delete=models.DO_NOTHING)
length = models.IntegerField()
weight = models.IntegerField()
data = pd.DataFrame(list(Species.objects.all().prefetch_related(Capture_set)))
I know is not the way you were looking for, but you can achieve this with values(). Let's suppose your related name is "captures"
import pandas
query = Species.objects.all().values('captures__pk', 'captures__species_name',
'captures__total_capture', 'pk', 'length',
'weight')
data = pandas.DataFrame.from_records(query)
I'm trying to sort (order) by statistical data stored in a ManyToOne relationship. Suppose I have the following code:
class Product(models.Model):
info = ...
data = models.IntegerField(default=0.0)
class Customer(models.Model):
info = ...
purchases = models.ManyToManyField(Product, related_name='customers', blank=True)
class ProductStats(models.Model):
ALL = 0
YOUNG = 1
OLD = 2
TYPE = ((ALL, 'All'), (YOUNG, 'Young'), (OLD, 'Old'),)
stats_type = models.SmallIntegerField(choices=TYPE)
product = models.ForeignKey(Product, related_name='stats', on_delete=models.CASCADE)
data = models.FloatField(default=0.0)
Then I would like to sort the products by their stats for the ALL demographic (assume every product has a stats connected to it for ALL). This might look something like the following:
products = Product.objects.all().order_by('stats__data for stats__stats_type=0')
Currently the only solution I can think of is either to create a new stats class just for all and use a OneToOneField for Product. Or, add a OneToOneField for Product pointing to the ALL stats in ProductStats.
Thank you for your help.
How about like this using multiple fields in order_by:
Product.objects.all().order_by('stats__data', 'stats__stats_type')
# it will order products from stats 0, then 1 then 2
Or if you want to get data for only stats_type 0:
Product.objects.filter(stats__stats_type=0).order_by('stats__data')
You can annotate the value of the relevant demographic and order by that:
from django.db.models import F
Product.objects.all().filter(stats__stats_type=0).annotate(data_for_all=F('stats__data').order_by('data_for_all')
I have a History model like below
class History(models.Model):
class Meta:
app_label = 'subscription'
ordering = ['-start_datetime']
subscription = models.ForeignKey(Subscription, related_name='history')
FREE = 'free'
Premium = 'premium'
SUBSCRIPTION_TYPE_CHOICES = ((FREE, 'Free'), (Premium, 'Premium'),)
name = models.CharField(max_length=32, choices=SUBSCRIPTION_TYPE_CHOICES, default=FREE)
start_datetime = models.DateTimeField(db_index=True)
end_datetime = models.DateTimeField(db_index=True, blank=True, null=True)
cancelled_datetime = models.DateTimeField(blank=True, null=True)
Now i have a queryset filtering like below
users = get_user_model().objects.all()
queryset = users.exclude(subscription__history__end_datetime__lt=timezone.now())
The issue is that in the exclude above it is checking end_datetime for all the rows for a particular history object. But i only want to compare it with first row of history object.
Below is how a particular history object looks like. So i want to write a queryset filter which can do datetime comparison on first row only.
You could use a Model Manager method for this. The documentation isn't all that descriptive, but you could do something along the lines of:
class SubscriptionManager(models.Manager):
def my_filter(self):
# You'd want to make this a smaller query most likely
subscriptions = Subscription.objects.all()
results = []
for subscription in subscriptions:
sub_history = subscription.history_set.first()
if sub_history.end_datetime > timezone.now:
results.append(subscription)
return results
class History(models.Model):
subscription = models.ForeignKey(Subscription)
end_datetime = models.DateTimeField(db_index=True, blank=True, null=True)
objects = SubscriptionManager()
Then: queryset = Subscription.objects().my_filter()
Not a copy-pastable answer, but shows the use of Managers. Given the specificity of what you're looking for, I don't think there's a way to get it just via the plain filter() and exclude().
Without knowing what your end goal here is, it's hard to say whether this is feasible, but have you considered adding a property to the subscription model that indicates whatever you're looking for? For example, if you're trying to get everyone who has a subscription that's ending:
class Subscription(models.Model):
#property
def ending(self):
if self.end_datetime > timezone.now:
return True
else:
return False
Then in your code: queryset = users.filter(subscription_ending=True)
I have tried django's all king of expressions(aggregate, query, conditional) but was unable to solve the problem so i went with RawSQL and it solved the problem.
I have used the below SQL to select the first row and then compare the end_datetime
SELECT (end_datetime > %s OR end_datetime IS NULL) AS result
FROM subscription_history
ORDER BY start_datetime DESC
LIMIT 1;
I will select my answer as accepted if not found a solution with queryset filter chaining in next 2 days.
I have been using Django for a couple of years now but I am struggling today with adding a HAVING constraint to a GROUP BY.
My queryset is the following:
crm_models.Contact.objects\
.filter(dealercontact__dealer__pk__in=(265,),
dealercontact__activity='gardening',
date_data_collected__gte=datetime.date(2012,10,1),
date_data_collected__lt=datetime.date(2013,10,1))\
.annotate(nb_rels=Count('dealercontact'))
which gives me the following MySQL query:
SELECT *
FROM `contact`
LEFT OUTER JOIN `dealer_contact` ON (`contact`.`id_contact` = `dealer_contact`.`id_contact`)
WHERE (`dealer_contact`.`active` = True
AND `dealer_contact`.`activity` = 'gardening'
AND `contact`.`date_data_collected` >= '2012-10-01'
AND `contact`.`date_data_collected` < '2013-10-01'
AND `dealer_contact`.`id_dealer` IN (265))
GROUP BY `contact`.`id_contact`
ORDER BY NULL;
I would get exactly what I need with this HAVING constraint:
HAVING SUM(IF(`dealer_contact`.`type`='customer', 1, 0)) = 0
How can I get this fixed with a Django Queryset? I need a queryset in this instance.
Here I am using annotate only in order to get the GROUP BY on contact.id_contact.
Edit: My goal is to get the Contacts who have no "customer" relation in dealercontact but have "ref" relation(s) (according to the WHERE clause of course).
Models
class Contact(models.Model):
id_contact = models.AutoField(primary_key=True)
title = models.CharField(max_length=255L, blank=True, choices=choices_custom_sort(TITLE_CHOICES))
last_name = models.CharField(max_length=255L, blank=True)
first_name = models.CharField(max_length=255L, blank=True)
[...]
date_data_collected = models.DateField(null=True, db_index=True)
class Dealer(models.Model):
id_dealer = models.AutoField(primary_key=True)
address1 = models.CharField(max_length=45L, blank=True)
[...]
class DealerContact(Auditable):
id_dealer_contact = models.AutoField(primary_key=True)
contact = models.ForeignKey(Contact, db_column='id_contact')
dealer = models.ForeignKey(Dealer, db_column='id_dealer')
activity = models.CharField(max_length=32, choices=choices_custom_sort(ACTIVITIES), db_index=True)
type = models.CharField(max_length=32, choices=choices_custom_sort(DEALER_CONTACT_TYPE), db_index=True)
I figured this out by adding two binary fields in DealerContact: is_ref and is_customer.
If type='ref' then is_ref=1 and is_customer=0.
Else if type='customer' then is_ref=0 and is_customer=1.
Thus, I am now able to use annotate(nb_customers=Sum('is_customer')) and then use filter(nb_customers=0).
The final queryset consists in:
Contact.objects.filter(dealercontact__dealer__pk__in=(265,),
dealercontact__activity='gardening',
date_data_collected__gte=datetime.date(2012,10,1),
date_data_collected__lt=datetime.date(2013,10,1))\
.annotate(nb_customers=Sum('dealercontact__is_customer'))\
.filter(nb_customers=0)
Actually there is a way you can add your own custom HAVING and GROUP BY clauses if you need.
Just use my example with caution - if Django ORM code/paths will change in future Django versions, you will have to update your code too.
Image you have Book and Edition models, where for each book there can be multiple editions and you want to select first US edition date within Book queryset.
Adding custom HAVING and GROUP BY clauses in Django 1.5+:
from django.db.models import Min
from django.db.models.sql.where import ExtraWhere, AND
qs = Book.objects.all()
# Standard annotate
qs = qs.annotate(first_edition_date=Min("edition__date"))
# Custom HAVING clause, to limit annotation by US country only
qs.query.having.add(ExtraWhere(['"app_edition"."country"=%s'], ["US"]), AND)
# Custom GROUP BY clause will be needed too
qs.query.group_by.append(("app_edition", "country"))
ExtraWhere can contain not just fields, but any raw sql conditions and functions too.
Are you not using raw query just because you want orm object? Using Contact.objects.raw() generate instances similar filter. Refer to https://docs.djangoproject.com/en/dev/topics/db/sql/ for more help.
My goal is to get the Contacts who have no "customer" relation in
dealercontact but have "ref" relation(s) (according to the WHERE
clause of course).
This simple query fulfills this requirement:
Contact.objects.filter(dealercontact__type="ref").exclude(dealercontact__type="customer")
Is this enough, or do you need it to do something more?
UPDATE: if your requirement is
Contacts that have a "ref" relations, but do not have "customer"
relations with the same dealer
you can do this:
from django.db.models import Q
Contact.objects.filter(Q(dealercontact__type="ref") & ~Q(dealercontact__type="customer"))
class Nutrient(models.Model):
tagname = models.CharField(max_length=10)
class FoodNutrientAmount(models.Model):
nutrient = models.ForeignKey(Nutrient)
food = models.ForeignKey(Food)
amount = models.FloatField()
class Food(models.Model):
nutrients = models.ManyToManyField(
Nutrient,
through=FoodNutrientAmount,
)
So, I can get the Foods ordered by the amount of tagname=FOL Nutrient with a list comprehension:
ordered_fnas = FoodNutrientAmount.objects.filter(
nutrient__tagname="FOL"
).order_by('-amount')
ordered_foods_by_most_fol = [fna.food for fna in ordered_fnas]
Can I get such an iterable as a queryset without taking the whole thing into memory?
Maybe there is a different approach using Food.objects.annotate or extra? I can't think of a great way to do it at the moment.
I can get close with values_list; but, I get the ordered list of pks and not the queryset of Food objects that I want.
FoodNutrientAmount.objects.filter(
nutrient__tagname='FOL'
).order_by('-amount').values_list('food', flat=True)
Edit:
This is a Many-to-many relationship. So you can probably leverage that. How about adding default ordering to FoodNutrientAmount and then you can just do normal manytomany queries.
class FoodNutrientAmount(models.Model):
nutrient = models.ForeignKey(Nutrient)
food = models.ForeignKey(Food)
amount = models.FloatField()
class Meta:
ordering = ('-amount',)
Then you can just call -
nutritious_foods = Food.objects.filter(nutrients__tagname='FOL').order_by('foodnutrientamount')