I currently have a line in my Django app like this:
db.execute("SELECT SUM(price * qty) FROM inventory_orderline WHERE order_id = %s", [self.id])
I'd rather perform this through the models interface provided my Django, but can't find any references.
I'm pretty sure this can be done with an annotation, but the examples only cover a few of them and I can't find a list in the documentation.
I'd like to do something like this:
self.line_items.annotate(lineprice=Product('orderline__price', 'orderline__qty')).aggregate(Sum('lineprice'))
Can anyone suggest an annotation class to use to perform the multiplication? Even better, a link to the API listing all these annotation/aggregation classes?
It's not clearly documented, but this can be done with the F() function:
from django.db.models import F
self.line_items.annotate(lineprice=F('orderline__price') * F('orderline__qty')).aggregate(Sum('lineprice'))
Unfortunately, there aren't enough inbuilt Aggregate functions and specifically, there is not one for Product.
But that doesn't limit you in any way other than having to write a "non-concise" ORM query. Specifically for your case, you should be able to do:
self.line_items.extra(select=("lineprice": "orderline__price*orderline__qty")).aggregate(Sum('lineprice'))
Related
You can query Django's JSONField, either by direct lookup, or by using annotations. Now I realize if you annotate a field, you can all sorts of complex queries, but for the very basic query, which one is actually the preferred method?
Example: Lets say I have model like so
class Document(models.Model):
data = JSONField()
And then I store an object using the following command:
>>> Document.objects.create(data={'name': 'Foo', 'age': 24})
Now, the query I want is the most basic: Find all documents where data__name is 'Foo'. I can do this 2 ways, one using annotation, and one without, like so:
>>> from django.db.models.expressions import RawSQL
>>> Document.objects.filter(data__name='Foo')
>>> Document.objects.annotate(name = RawSQL("(data->>'name')::text", [])).filter(name='Foo')
So what exactly is the difference? And if I can make basic queries, why do I need to annotate? Provided of course I am not going to make complex queries.
There is no reason whatsoever to use raw SQL for queries where you can use ORM syntax. For someone who is conversant in SQL but less experienced with Django's ORM, RawSQL might provide an easier path to a certain result than the ORM, which has its own learning curve.
There might be more complex queries where the ORM runs into problems or where it might not give you the exact SQL query that you need. It is in these cases that RawSQL comes in handy – although the ORM is getting more feature-complete with every iteration, with
Cast (since 1.10),
Window functions (since 2.0),
a constantly growing array of wrappers for database functions
the ability to define custom wrappers for database functions with Func expressions (since 1.8) etc.
They are interchangable so it's matter of taste. I think Document.objects.filter(data__name='Foo') is better because:
It's easier to read
In the future, MariaDB or MySql can support JSON fields and your code will be able to run on both PostgreSQL and MariaDB.
Don't use RawSQL as a general rule. You can create security holes in your app.
If I perform a prefetch_related('toppings') for a queryset, and I want to later filter(spicy=True) by fields in the related table, Django ignores the cached info and does a database query. I found that this is documented (under the Note box) and seems to happen for all forms of caching (select_related(), already evaluated querysets, etc.) when another filter() is performed.
However, is there some sort of super secret hidden time-saving shortcut to filter locally (using the cache and not hitting the database) without having to write the python code to loop the queryset (using list/dict comprehension, etc.)? Maybe something like a filter_locally(spicy=True)?
EDIT:
One of the reasons why a list/comprehension doesn't work well for me is because a list/dict does not have the queryset methods. In my case, the first level M2M field, toppings, isn't the end goal for me and I need to check a 2nd related M2M field (which I have already pre-fetched as well). While this is also possible using list comprehension, it's just much simpler to have something such as filter_locally(spicy=True, origin__country='Spain') because:
it allows accessing many levels of related fields with minimal effort
it allows chaining other queryset methods
it's easier to read because it's consistent with the familiar filter()
it's easier to modify existing code using filter() without prefetch to add this optimization in without much changes.
But from the responses, Django has no such support :(
You have to write the python code to loop through the queryset (a list/dict comprehension is ideal). All the filter() code knows how to do is add filtering language to the SQL sent to the database. Filtering locally is a totally different problem than filtering remotely, so the solutions to those two separate problems won't be able to share any logic.
A list comprehension one-liner would be pretty straightforward, though; the syntax might not be much more complex than with filter().
If you're filtering on a boolean doing the list comprehension is pretty easy. You can also swap out the topping.spicy==True for a string comparison or whatever.
I would do something like:
qs = Pizza.objects.all().prefetch_related('toppings')
res = list(qs)
def get_spicy(qs):
res = list(qs)
return [pizza for pizza in res if any(topping.spicy==True for
topping in pizza.toppings.all())]
That is if you want to return the pizza object if any of its toppings is spicy. You can also replace the any() with all() to check for all, and do a lot of pretty powerful queries with this syntax. I'm somewhat surprised that there is no easy way to do this in django. It seems like a lot of these simple queries should be easy to implement in a generic manner.
The above code assumes a many2many. It should be easy to modify to work with a simple FK relationship such as a one2one or one2many.
Hope this was helpful.
I am working on a test project using django-nonrel.
After enabling the admin interface and adding some entities to the database, I added a search_field to the ModelAdmin class. As I tried to search I got the following error:
DatabaseError: Lookup type 'icontains' isn't supported
In order to fix this, I added an index like this:
from models import Empresa
from dbindexer.api import register_index
register_index(Empresa, {'nombre': 'icontains'})
But now I am getting the following error:
First ordering property must be the same as inequality filter property, if specified for this query; received key, expected idxf_nombre_l_icontains
Am I trying to do something that is not supported by django-nonrel and dbindex yet?
Thanks in advance for any help
I have the same problem (on another case), know the cause of it, but currently have no solution.
It is because of GAE's database limitation in which if a query contain an inequality comparison, that is ' < , > , >= ' or something like that, any ordering of any member of the entities (other than the member that use the inequality comparison) must be preceded by an ordering of the member with inequality comparison first.
If we are directly using GAE's database, this limitation can easily be overcome by first set the order by the member that use the inequality first, than sort with whatever you want to sort.
Unfortunately, the django-nonrel and djangoappengine's database wrapper seems to be unable to do that (I've tried the order by first technique using django model, still error, maybe it's just me), not to mention the use of dbindexer as the wrapper of djangoappengine.db which itself is a wrapper of GAE's database......
Bottomline, debugging can be a hell for this mess. You may want to use GAE datastore directly just for this case, or wait for djangoappengine team to come up with better alternative.
I kind of fixed it by changing the ordering property in the ModelAdmin subclass:
class EmpresaAdmin(admin.ModelAdmin):
search_fields = ('nombre',)
#order by the atribute autogenerated by dbindex
ordering = ('idxf_nombre_l_icontains',)
Does anyone know a better way to fix this?
i'm triyng to make a full text search with postgresql and django So I've created a function search_client(text) which returns a list of clients. To call it from the DB i use something like this:
SELECT * FROM search_client('something')
and i'm not really sure how to call it from django. i know i could do something like
cursor = connection.cursor()
cursor.execute("SELECT * FROM search_client('something')")
result = cursor.fetchall()
but that will only return a list of values, and i'd like to have a list of objects, like when i use the "filter()" method.
Any ideas?? thanks for your time!
If your goal is a full-featured search engine, have a look at django-haystack. It rocks.
As for your question, the new (Django 1.2) raw method might work:
qs = MyModel.objects.raw("SELECT * FROM search_client('something')")
If you're using Django 1.2, you can use the raw() ORM method to execute custom SQL but get back Django models. If you're not, you can still execute the SQL via the extra() method on default QuerySet, and pump it into a custom method to either then go pull the real ORM records, or make new, temporary, objects
First, you probably don't want to do this. Do you have proof that your database function is actually faster?
Implement this in Python first. When you can prove that your Python implementation really is the slowest part of your transaction, then you can try a stored procedure.
Second, you have the extra method available in Django.
http://docs.djangoproject.com/en/1.2/ref/models/querysets/#django.db.models.QuerySet.extra
Note that compute-intensive database procedures are often slow.
Say my_instance is of model MyModel.
I'm looking for a good way to do:
my_model = get_model_for_instance(my_instance)
I have not found any really direct way to do this.
So far I have come up with this:
from django.db.models import get_model
my_model = get_model(my_instance._meta.app_label, my_instance.__class__.__name__)
Is this acceptable? Is it even a sure-fire, best practice way to do it?
There is also _meta.object_name which seems to deliver the same as __class__.__name__. Does it? Is better or worse? If so, why?
Also, how do I know I'm getting the correct model if the app label occurs multiple times within the scope of the project, e.g. 'auth' from 'django.contrib.auth' and let there also be 'myproject.auth'?
Would such a case make get_model unreliable?
Thanks for any hints/pointers and sharing of experience!
my_model = type(my_instance)
To prove it, you can create another instance:
my_new_instance = type(my_instance)()
This is why there's no direct way of doing it, because python objects already have this feature.
updated...
I liked marcinn's response that uses type(x). This is identical to what the original answer used (x.__class__), but I prefer using functions over accessing magic attribtues. In this manner, I prefer using vars(x) to x.__dict__, len(x) to x.__len__ and so on.
updated 2...
For deferred instances (mentioned by #Cerin in comments) you can access the original class via instance._meta.proxy_for_model.
my_new_instance = type(my_instance)()
At least for Django 1.11, this should work (also for deferred instances):
def get_model_for_instance(instance):
return instance._meta.model
Source here.