I've got a problem for the following data:
Class Car:
name = ...
year_price_data = ManyToMany(YearPriceData)
Class YearPriceData:
year = Integer
min_price = Integer
max_price Integer
So you can have this car for example:
Ford Focus:
---2014: 25,000$ - 40,000$
---2012: 7000$ - 23,000$
Now, I want to query all cars that match a certain year-price values:
for example: all cars that are sold in 2014 in the price range of 18,000-20,000$
(Should returns nothing in our example, yes?)
so this is the query ive made:
Car.objects.filter(year_price_data__year=2014)
.filter(Q(year_price_data__min_price__lte=20000,year_price_data__min_price__gte=18000) | Q(year_price_data__max_price__lte = 20000,year_price_data__max_price__gte=18000))
The problem that because of the OR it returns the row of 2012, when it supposed to return nothing!
So my question is: how to make an OR query just for every manytomany row, so the query will be exact?
Firstly, I don't see why you want a ManyToMany relationship here. Although a car can have many yearpricedata instances, each yearpricedata only relates to one car. So the proper relationship is a ForeignKey from YearPriceData to Car.
Either way, you should read the documentation on spanning multi-valued relationships. As explained there, in order for the two conditions to be both applied on the same entities, you need to have them in the same filter call. This should work:
Car.objects.filter(year_price_data__year=2014, Q(year_price_data__min_price__lte=20000,year_price_data__min_price__gte=18000) | Q(year_price_data__max_price__lte = 20000,year_price_data__max_price__gte=18000))
Related
Been searching the web for a couple hours now looking for a solution but nothing quite fits what I am looking for.
I have one model (simplified):
class SimpleModel(Model):
name = CharField('Name', unique=True)
date = DateField()
amount = FloatField()
I have two dates; date_one and date_two.
I would like a single queryset with a row for each name in the Model, with each row showing:
{'name': name, 'date_one': date_one, 'date_two': date_two, 'amount_one': amount_one, 'amount_two': amount_two, 'change': amount_two - amount_one}
Reason being I would like to be able to find the rank of amount_one, amount_two, and change, using sort or filters on that single queryset.
I know I could create a list of dictionaries from two separate querysets then sort on that and get the ranks from the index values ...
but perhaps nievely I feel like there should be a DB solution using one queryset that would be faster.
union seemed promising but you cannot perform some simple operations like filter after that
I think I could perhaps split name into its own Model and generate queryset with related fields, but I'd prefer not to change the schema at this stage. Also, I only have access to sqlite.
appreciate any help!
Your current model forces you to have ONE name associated with ONE date and ONE amount. Because name is unique=True, you literally cannot have two dates associated with the same name
So if you want to be able to have several dates/amounts associated with a name, there are several ways to proceed
Idea 1: If there will only be 2 dates and 2 amounts, simply add a second date field and a second amount field
Idea 2: If there can be an infinite number of days and amounts, you'll have to change your model to reflect it, by having :
A model for your names
A model for your days and amounts, with a foreign key to your names
Idea 3: You could keep the same model and simply remove the unique constraint, but that's a recipe for mistakes
Based on your choice, you'll then have several ways of querying what you need. It depends on your final model structure. The best way to go would be to create custom model methods that query the 2 dates/amount, format an array and return it
Lets consider the following scenario - books can be shared by multiple students.
class Student(models.Model):
books = models.ManyToManyField(Book, related_name='students')
I would like to find, for a given student, all the books shared/owned only by this student. I did the following:
s = Student.objects.get(id=1)
s.books.annotate(student_count=Count('students').filter(student_count=1)
However, it returns all the books of student s (equivalent to s.books.all()). Later, I got the desired result with following query:
Book.objects.annotate(student_count=Count('students')).\
filter(students=s, student_count=1)
Why do I not get the expected result with the first query?
This queryset
s.books.annotate(student_count=Count('students').filter(student_count=1)
generates the same SQL query as
Book.objects.filter(students=s).annotate(student_count=Count('students')).filter(student_count=1)
and because order of annotate() and filter() matters, you do not get the expected result (every book will have student_count equal to 1).
My model consists of a Portfolio, a Holding, and a Company. Each Portfolio has many Holdings, and each Holding is of a single Company (a Company may be connected to many Holdings).
Portfolio -< Holding >- Company
I'd like the Portfolio query to return the sum of the product of the number of Holdings in the Portfolio, and the value of the Company.
Simplified model:
class Portfolio(model):
some fields
class Company(model):
closing = models.DecimalField(max_digits=10, decimal_places=2)
class Holding(model):
portfolio = models.ForeignKey(Portfolio)
company = models.ForeignKey(Company)
num_shares = models.IntegerField(default=0)
I'd like to be able to query:
Portfolio.objects.some_function()
and have each row annotated with the value of the Portfolio, where the value is equal to the sum of the product of the related Company.closing, and Holding.num_shares. ie something like:
annotate(value=Sum('holding__num_shares * company__closing'))
I'd also like to obtain a summary row, which contains the sum of the values of all of a user's Portfolios, and a count of the number of holdings. ie something like:
aggregate(Sum('holding__num_shares * company__closing'), Count('holding__num_shares'))
I would like to do have a similar summary row for a single Portfolio, which would be the sum of the values of each holding, and a count of the total number of holdings in the portfolio.
I managed to get part of the way there using extra:
return self.extra(
select={
'value': 'select sum(h.num_shares * c.closing) from portfolio_holding h '
'inner join portfolio_company as c on h.company_id = c.id '
'where h.portfolio_id = portfolio_portfolio.id'
}).annotate(Count('holding'))
but this is pretty ugly, and extra seems to be frowned upon, for obvious reasons.
My question is: is there a more Djangoistic way to summarise and annotate queries based on multiple fields, and across related tables?
These two options seem to move in the right direction:
Portfolio.objects.annotate(Sum('holding__company__closing'))
(ie this demonstrates annotation/aggregation over a field in a related table)
Holding.objects.annotate(Sum('id', field='num_shares * id'))
(this demonstrates annotation/aggregation over the product of two fields)
but if I attempt to combine them: eg
Portfolio.objects.annotate(Sum('id', field='holding__company__closing * holding__num_shares'))
I get an error: "No such column 'holding__company__closing'.
So far I've looked at the following related questions, but none of them seem to capture this precise problem:
Annotating django QuerySet with values from related table
Product of two fields annotation
Do I just need to bite the bullet and use raw / extra? I'm hoping that Django ORM will prove the exception to the rule that ORMs really only work as designed for simple queries / models, and anything beyond the most basic ones require either seriously gnarly tap-dancing, or stepping out of the abstraction, which somewhat defeats the purpose...
Thanks in advance!
I'm having trouble reducing the number of queries for a particular view. It's a fairly heavy one but I'm sure it can be reduced:
Profile:
name = CharField()
Officers:
club= ManyToManyField(Club, related_name='officers')
title= CharField()
Club:
name = CharField()
members = ManyToManyField(Profile)
Election:
club = ForeignKey(Club)
elected = ForeignKey(Profile)
title= CharField()
when = DateTimeField()
Clubs have members and officers (president, tournament director). People can be members of multiple clubs etc...
Officers are elected at elections, the results of which are stored.
Given a player how can I find out the most recently elected officer at each of the players clubs?
At the moment I have
clubs = Club.objects.filter(members=me).prefetch_related('officers')
for c in clubs:
officers = c.officers.all()
most_recent = Elections.objects.filter(club=c).filter(elected__in=officers).order_by('-when')[:1].get()
print(c.name + ' elected ' + most_recent.name + ' most recently')
Problem is the looped query, it's nice and fast if you're a member of 1 club but if you join fifty my database crawls.
Edit:
The answer from Nil does what I want but doesn't get the object. I don't really need the object but I do need another field as well as the datetime. If it's helpful the query:
Club.objects.annotate(last_election=Max('election__when'))
produces the raw SQL
SELECT "organisation_club"."id", "organisation_club"."name", MAX("organisation_election"."when") AS "last_election"
FROM "organisation_club"
LEFT OUTER JOIN "organisation_election" ON ( "organisation_club"."id" = "organisation_election"."club_id" )
GROUP BY "organisation_club"."id", "organisation_club"."name"
I'd really like an ORM answer if at all possible (or a 'mostly' ORM answer).
I believe this is what you're looking for:
from django.db.models import Max, F
Election.objects.filter(club__members=me) \
.annotate(max_date=Max('club__election_set__when')) \
.filter(when=F('max_date')).select_related('elected')
Relations can be followed forwards and backwards again in a single statement, allowing you to annotate the max_date for any election related to the club of the current election. The F class allows you to filter a queryset based on selected fields in SQL, including any extra fields added through annotation, aggregation, joins etc.
What you want is defined here in SQL term: query the Election table, group them by Club and keep only the last election of each club.
Now, how can we translate that in Django ORM? Looking at the documentation, we learn that we can do it with an annotation. The trick is that you need to think in reverse. You want to annotate (add a new data) each club with its last election. This gives us:
Club.objects.annotate(last_election=Max('election__when'))
# Use it in a for loop like that
for club in Club.objects.annotate(last_election=Max('election__when')):
print(club, club.last_election)
Sadly, this only adds the date, which doesn't answer your question! You want the name or the complete Club object. I searched and I still don't know how to do it properly. If everything fails though, you can still use a raw SQL query in Django using a query like in the first link.
The simplest way I can think of is filtering partially at the application level
If you do
e = Election.objects.filter(club__members=me).select_related('elected')
or
e = me.club_set.election_set.select_related('elected')
This is a single query and it should get back all the elections that happened for the all the clubs that the member me is in. Then you can use python to just get the most recent date. Of course, if you have many elections per club, you end up fetching much more data than will be used.
Another way which should do it in two queries:
# Get all member's clubs & most recent election
clubs = Club.objects.filter(members=me).annotate(last_election=Max('election__when'))
# Create filters for election based on the club id and the latest election time
election_Q = [Q(club__id=c.id) & Q(when=c.last_election) for c in clubs]
# Combine filters with an OR
election_filter = reduce(lambda f1, f2: f1 | f2, election_Q)
# Get elections restricting by specific clubs & election date
elections = Election.objects.filter(election_filter).select_related('elected')
for e in elections:
print '%s elected %s most recently at %s' % (e.club.name, e.elected, e.when)
This builds upon #Nil's method and uses its result to build a query in python, then feeds it into the second query. However, there is a limit with the size of a SQL statement and if there are a lot of clubs that a member is in, then you may hit the limit. The limit is fairly high though and I've only ever reached it when importing large datasets in a single INSERT statement so I think it should be fine for your purpose.
Sorry I cannot think of a way that the Django ORM can link them together using a single SQL query. The Django ORM is actually quite limited for complex queries so if you really need the efficiency I think it's probably best to write the raw SQL query.
I have a table which has a list of invoices and their details:
class Invoice(models.Model):
invoiceNum = models.CharField(etc...)
invoiceDate = models.DateField(etc...)
customerID = models.ForeignKey(etc...)
isPaid = models.CharField(etc...)
The Invoice records do not hold the actual invoice total. Instead, an invoice's total is made up of several Invoice_Line_Item records, held in another table:
class Invoice_Line_Item(models.Model):
invNum = models.ForeignKey(Invoice)
itemName = models.CharField(etc...)
itemPrice = models.DecimalField(etc...)
I have a webapp that shows all the invoices in a big HTML table, along with all the details of that invoice on the table's tr row. Details such as, Invoice Date, Invoice Number, Customer ID, all come from that Invoice table. There are hundreds of invoices to display in this HTML table.
What I would like to do is also show each invoice's total value - which is the sum of all the line items. However, I can't think of a simple way to acomplish this since the invoice details and the line items that make up the invoice's total are in two different tables.
One way I thought is to pass the entire Invoice_Line_Item querySet into the HTML template, then for each invoice displayed in a table tr, I could iterate over the entire Invoice_Line_Item querySet, adding up all the line items that match the current invoice. This, however, seems hugely inefficient.
Any better ideas on how to do this?
Thanks!
One word: Aggregation
Invoice_Line_Item.objects.filter(invNum=invoice).aggregate(Sum('itemPrice'))
https://docs.djangoproject.com/en/dev/topics/db/aggregation/
Another way is to store the total in Invoice and update it whenever you change a related Invoice_Line_Item
One more word: annotate.
from django.models import Sum
Invoice.objects.filter( .. ).annotate( InvTotal=Sum( 'invoice_line_number__itemPrice' ) )
InvTolal becomes a new attribute of Invoice object, you can use it in template the same way as invoiceNum or invoiceDate.
With this approach you do not have to pass any additional data structures to your template, only a QuerySet of Invoices.
Please note:
Argument of Sum is a string, which is a concatenation of the name of related model converted to lowercase, than double '_', and than the name of a field in related model.