I am making a mutli-series line-graph and I want to have a series for the data overall. At first, I thought I would need two queries to retrieve the data: one query for overall, and one query for the data sorted by groups. But then I learned MySQL has this functionality built in through an operator called ROLLUP that you can add to your GROUP BY clause. Unfortunately, after lots of googling, I couldn't find any working examples. Does this functionality exist in ActiveRecord 4.2? If so, how can I use it?
Note that my query averages the data before the DB returns it so I cannot simply run one query and have my app run more computations on the result to get the overall value.
After further investigation, it seems ActiveRecord simply does not support the ROLLUP operator. Does RAILS have GROUP BY...WITH ROLLUP query? offered a solution but it didn't work for me nor my coworkers.
Related
Sql way means using using annotation, aggregate, subquery, when, case, window etc. i.e get all the extra calculation columns from sql
Till now if I want to get some addition information from the data of each row like calculate something etc, I used to get the table in queryset and loop over each object and then store the desired result in a list of dicts and pass it to the template. Ofcourse i was using prefetch so that i can avoid N+1 queries.
But since django 1.11 we can do the same thing in more expressive way using sql (i.e using annotation, aggregate, subquery, when, case, window etc) and no python.
Disadvantage:
One disadvantage i found using sql way rather than python way is debugging.
I have to do complex calculations on the data of each queryset object. So that i can check step by step.
If i do it in the sql way i can only see the final result but not be able to track the steps.
Advantage:
I didnt tried, but heard from The Dramatic Benefits of Django Subqueries and Annotations that its quite fast.
Presently most of my sql takes less than 100 ms and most of the time goes in dom loading. So by using sql way will it help me any way.
I appreciate that Django created more functions which help to write the sql expressively.
I am writing a simple app in django that searches for records in database.
Users inputs a name in the search field and that query is used to filter records using a particular field like -
Result = Users.objects.filter(name__icontains=query_from_searchbox)
E.g. -
Database consists of names- Shiv, Shivam, Shivendra, Kashiva, Varun... etc.
A search query 'shiv' returns records in following order-
Kahiva, Shivam, Shiv and Shivendra
Ordered by primary key.
My question is how can i achieve the order -
Shiv, Shivam, Shivendra and Kashiva.
I mean the most relevant first then lesser relevant result.
It's not possible to do that with standard Django as that type of thing is outside the scope & specific to a search app.
When you're interacting with the ORM consider what you're actually doing with the database - it's all just SQL queries.
If you wanted to rearrange the results you'd have to manipulate the queryset, check exact matches, then use regular expressions to check for partial matches.
Search isn't really the kind of thing that is best suited to the ORM however, so you may which to consider looking at specific search applications. They will usually maintain an index, which avoids database hits and may also offer a percentage match ordering like you're looking for.
A good place to start may be with Haystack
I have some MySQL code that looks like this:
SELECT
visitor AS team,
COUNT(*) AS rg,
SUM(vscore>hscore) AS rw,
SUM(vscore<hscore) AS rl
FROM `gamelog` WHERE status='Final'
AND date(start_et) BETWEEN %s AND %s GROUP BY visitor
I'm trying to translate this into a Django version of that query, without making multiple queries. Is this possible? I read up on how to do Sum(), and Count(), but it doesn't seem to work when I want to compare two fields like I'm doing.
Here's the best I could come up with so far, but it didn't work...
vrecord = GameLog.objects.filter(start_et__range=[start,end],visitor=i['id']
).aggregate(
Sum('vscore'>'hscore'),
Count('vscore'>'hscore'))
I also tried using 'vscore>hscore' in there, but that didn't work either. Any ideas? I need to use as few queries as possible.
Aggregation only works on single fields in the Django ORM. I looked at the code for the various aggregation functions, and noticed that the single-field restriction is hardwired. Basically, when you use, say, Sum(field), it just records that for later, then it passes it to the database-specific backend for conversion to SQL and execution. Apparently, aggregation and annotation are not standardized in SQL.
Anyway, you probably need to use a raw SQL query.
Can Django support Oracle nested tables or varrays or collections in some manner? Asking just for completeness as our project is reworking the data model, attempting to move away from EAV organization, but I don't like creating a bucket load of dependent supporting tables for each main entity.
e.g.
(not the proper Oracle syntax, but gets the idea across)
Events
eventid
report_id
result_tuple (result_type_id, result_value)
anomaly_tuple(anomaly_type_id, anomaly_value)
contributing_factors_tuple(cf_type_id, cf_value)
etc,
where the can be multiple rows of the tuples for one eventid
each of these tuples can, of course exist as separate tables, but this seems to be more concise. If it 's something Django can't do, or I can't modify the model classes to do easily, then perhaps just having django create the extra tables is the way to go.
--edit--
I note that django-hstore is doing something very similar to what I want to do, but using postgresql's hstore capability. Maybe I can branch off of that for an Oracle nested table implementation. I dunno...I'm pretty new to python and django, so my reach may exceed my grasp in this case.
Querying a nested table gives you a cursor to traverse the tuples, one member of which is yet another cursor, so you can get the rows from the nested table.
I am hoping someone can help me out with a quick question I have regarding chaining Django querysets. I am noticing a slow down because I am evaluating many data points in the database to create data trends. I was wondering if there was a way to have the chained filters evaluated locally instead of hitting the database. Here is a (crude) example:
pastries = Bakery.objects.filter(productType='pastry') # <--- will obviously always hit DB, when evaluated
cannoli = pastries.filter(specificType='cannoli') # <--- can this be evaluated locally instead of hitting the DB when evaluated, as long as pastries was evaluated?
I have checked the docs and I do not see anything specifying this, so I guess it's not possible, but I wanted to check with the 'braintrust' first ;-).
BTW - I know that I can do this myself by implementing some methods to loop through these datapoints and evaluate the criteria, but there are so many datapoints that my deadline does not permit me manually implementing this.
Thanks in advance.
QuerySet methods always produce SQL that returns the desired expression. This is why you cannot e.g. call various methods after slicing; SQL does not support that syntax. The ORM does nothing more than assemble said SQL. If you want fancier processing then you will need to perform it in Python code yourself.