everybody.
I work with Django 1.3 and Postgres 9.0. I have very complex sql query which extends simple model table lookup with some extra fields. And it wrapped in table function since it is parameterized.
A month before I managed to make it work with the help of raw query but RawQuerySet lacks a lot of features which I really need (filters, count() and clone() methods, chainability).
The idea looks simple. QuerySet lets me to perform this query:
SELECT "table"."field1", ... ,"table"."fieldN" FROM "table"
whereas I need to do this:
SELECT "table"."field1", ... ,"table"."fieldN" FROM proxy(param1, param2)
So the question is: How can I do this? I've already started to create custom manager but can't substitute model.db_table with custom string (because it's being quoted and database stops to recognize the function call).
If there's no way to substitute table with function I would like to know if I can create QuerySet from RawQuerySet (Not the cleanest solution but simple RawQuerySet brings so much pain in... one body part).
If your requirement is that you want to use Raw SQL for efficiency purposes and still have access to model methods, what you really need is a library to map what fields of the SQL are the columns of what models.
And that library is, Unjoinify
Related
I'm thinking of using a raw query to quickly get around limitations with either my brain or the Django ORM, but I don't want to redevelop the infrastructure required to support the existing ORM code such as filters. Right now I'm stuck with two dead ends:
Writing an inner raw query and reusing that like any other query set. Even though my raw query selects the correct columns, I can't filter on it:
AttributeError: 'RawQuerySet' object has no attribute 'filter'
This is corroborated by another answer, but I'm still hoping that that information is out of date.
Getting the SQL and parameters from the query set and wrapping that in a raw query. It seems the raw SQL should be retrievable using queryset.query.get_compiler(DEFAULT_DB_ALIAS).as_sql() - how would I get the parameters as well (obviously without actually running the query)?
One option for dealing with complex queries is to write a VIEW that encapsulates the query, and then stick a model in front of that. You will still be able to filter (and depending upon your view, you may even get push-down of parameters to improve query performance).
All you need to do to get a model that is backed by a view is have it as "unmanaged", and then have the view created by a migration operation.
It's better to try to write a QuerySet if you can, but at times it is not possible (because you are using something that cannot be expressed using the ORM, for instance, or you need to to something like a LATERAL JOIN).
I have some MySQL code that looks like this:
SELECT
visitor AS team,
COUNT(*) AS rg,
SUM(vscore>hscore) AS rw,
SUM(vscore<hscore) AS rl
FROM `gamelog` WHERE status='Final'
AND date(start_et) BETWEEN %s AND %s GROUP BY visitor
I'm trying to translate this into a Django version of that query, without making multiple queries. Is this possible? I read up on how to do Sum(), and Count(), but it doesn't seem to work when I want to compare two fields like I'm doing.
Here's the best I could come up with so far, but it didn't work...
vrecord = GameLog.objects.filter(start_et__range=[start,end],visitor=i['id']
).aggregate(
Sum('vscore'>'hscore'),
Count('vscore'>'hscore'))
I also tried using 'vscore>hscore' in there, but that didn't work either. Any ideas? I need to use as few queries as possible.
Aggregation only works on single fields in the Django ORM. I looked at the code for the various aggregation functions, and noticed that the single-field restriction is hardwired. Basically, when you use, say, Sum(field), it just records that for later, then it passes it to the database-specific backend for conversion to SQL and execution. Apparently, aggregation and annotation are not standardized in SQL.
Anyway, you probably need to use a raw SQL query.
Can Django support Oracle nested tables or varrays or collections in some manner? Asking just for completeness as our project is reworking the data model, attempting to move away from EAV organization, but I don't like creating a bucket load of dependent supporting tables for each main entity.
e.g.
(not the proper Oracle syntax, but gets the idea across)
Events
eventid
report_id
result_tuple (result_type_id, result_value)
anomaly_tuple(anomaly_type_id, anomaly_value)
contributing_factors_tuple(cf_type_id, cf_value)
etc,
where the can be multiple rows of the tuples for one eventid
each of these tuples can, of course exist as separate tables, but this seems to be more concise. If it 's something Django can't do, or I can't modify the model classes to do easily, then perhaps just having django create the extra tables is the way to go.
--edit--
I note that django-hstore is doing something very similar to what I want to do, but using postgresql's hstore capability. Maybe I can branch off of that for an Oracle nested table implementation. I dunno...I'm pretty new to python and django, so my reach may exceed my grasp in this case.
Querying a nested table gives you a cursor to traverse the tuples, one member of which is yet another cursor, so you can get the rows from the nested table.
I have a model called "Story" that has two integer fields called "views" and "votes". When I retrieve all the Story objects I would like to annotate the returned QuerySet with a "ranking" field that is simply "views"/"votes". Then I would like to sort the QuerySet by "ranking". Something along the lines of...
Story.objects.annotate( ranking=CalcRanking('views','votes') ).sort_by(ranking)
How can I do this in Django? Or should it be done after the QuerySet is retrieved in Python (like creating a list that contains the ranking for each object in the QuerySet)?
Thanks!
PS: In my actual program, the ranking calculation isn't as simple as above and depends on other filters to the initial QuerySet, so I can't store it as another field in the Story model.
In Django, the things you can pass to annotate (and aggregate) must be subclasses of django.db.models.aggregates.Aggregate. You can't just pass arbitrary Python objects to it, since the aggregation/annotation actually happens inside the database (that's the whole point of aggregate and annotate). Note that writing custom aggregations is not supported in Django (there is no documentation for it). All information available on it is this minimal source code: https://code.djangoproject.com/browser/django/trunk/django/db/models/aggregates.py
This means you either have to store the calculations in the database somehow, figure out how the aggregation API works or use raw sql (raw method on the Manager) to do what you do.
My initial question was here and was related to the postgres backend. Postgres subquery, ordering by subquery
Now my problem has moved onwards to the Django ORM layer. I essentially want to order a query by a postgres function ('idx', taken from the above stackoverflow work)
I've gone through trying to use model.objects.extra(order_by ) or simply order_by but I believe both of these need the order_by parameter to be an attribute or a field known to Django.
I'm trying to think how to solve this without having to revert to using an entirely raw SQL query through a model manager.
You can use extra to add the function result to your query and then order by it. Something like:
MyModel.objects.extra(select={'idx': 'idx(foo, bar)'}, order_by=['idx'])