I am assisting a project which uses flask-sqlalchemy.
I would like a db query to count the number of records in the table.
Can I use table.query.filter_by(condition).count() directly?
Or is there anything that I need to add?
Please assist me. I am a begineer. Thanks in advance!
None of the given answers address flask-sqlalchemy specifically, where you would use exactly the example you gave:
Table.query.filter_by(condition).count()
You can perform .count() without filters:
Table.query.count()
You can also count using M2M relationships:
ParentTable.children.count()
And you can use any of these directly in your jinja templates like:
{{ Table.query.filter_by(condition).count() }}
Bonus points (for the performance minded):
.count() is a bit slow (especially with MySQL, thanks to a poor handling of subqueries), so instead, you can use a custom count that looks like this:
db.session.execute(Table.query.filter_by(condition).statement.with_only_columns([func.count()]).order_by(None)).scalar()
That's assuming db is your SQLAlchemy instance (ie, db = SQLAlchemy(app)). It's a mouthful, but it will save you a little bit of overhead on big queries.
Per this source, the following should work for you:
session.query(Class_name).filter_by(condition).count()
Class_name is the name of the mapped class for your table.
Yes, you can do that. But remember, that count() uses one more query, like this:
Select count(q.*) from (select * from table) q
However, you can make it more efficient by using only one query. You can use this:
from sqlalchemy.sql.functions import func
number = session.query(func.count(table.id).label('number').first().number
The SQL query will be like this:
Select count(table.id) as number from table
Simple is best:
accounts = db.session.execute('select count(id) as c from user').scalar()
Related
If we chain a call to filter() after a call to distinct(), the filter is applied to the query before the distinct. How do I filter the results of a query after applying distinct?
Example.objects.order_by('a','foreignkey__b').distinct('a').filter(foreignkey__b='something')
The where clause in the SQL resulting from filter() means the filter is applied to the query before the distinct. I want to filter the queryset resulting from the distinct.
This is probably pretty easy, but I just can't quite figure it out and I can't find anything on it.
Edit 1:
I need to do this in the ORM...
SELECT z.column1, z.column2, z.column3
FROM (
SELECT DISTINCT ON (b.column1, b.column2) b.column1, b.column2, c.column3
FROM table1 a
INNER JOIN table2 b ON ( a.id = b.id )
INNER JOIN table3 c ON ( b.id = c.id)
ORDER BY b.column1 ASC, b.column2 ASC, c.column4 DESC
) z
WHERE z.column3 = 'Something';
(I am using Postgres by the way.)
So I guess what I am asking is "How do you nest subqueries in the ORM? Is it possible?" I will check the documentation.
Sorry if I was not specific earlier. It wasn't clear in my head.
This is an old question, but when using Postgres you can do the following to force nested queries on your 'Distinct' rows:
foo = Example.objects.order_by('a','foreign_key__timefield').distinct('a')
bar = Example.objects.filter(pk__in=foo).filter(some_field=condition)
bar is the nested query as requested in OP without resorting to raw/extra etc. Tested working in 1.10 but docs suggest it should work back to at least 1.7.
My use case was to filter up a reverse relationship. If Example has some ForeignKey to model Toast then you can do:
Toast.objects.filter(pk__in=bar.values_list('foreign_key',flat=true))
This gives you all instances of Toast where the most recent associated example meets your filter criteria.
Big health warning about performance though, using this if bar is likely to be a huge queryset you're probably going to have a bad time.
Thanks a ton for the help guys. I tried both suggestions and could not bend either of those suggestions to work, but I think it started me in the right direction.
I ended up using
from django.db.models import Max, F
Example.objects.annotate(latest=Max('foreignkey__timefield')).filter(foreignkey__timefield=F('latest'), foreign__a='Something')
This checks what the latest foreignkey__timefield is for each Example, and if it is the latest one and a=something then keep it. If it is not the latest or a!=something for each Example then it is filtered out.
This does not nest subqueries but it gives me the output I am looking for - and it is fairly simple. If there is simpler way I would really like to know.
No you can't do this in one simple SELECT.
As you said in comments, in Django ORM filter is mapped to SQL clause WHERE, and distinct mapped to DISTINCT. And in a SQL, DISTINCT always happens after WHERE by operating on the result set, see SQLite doc for example.
But you could write sub-query to nest SELECTs, this depends on the actual target (I don't know exactly what's yours now..could you elaborate it more?)
Also, for your query, distinct('a') only keeps the first occurrence of Example having the same a, is that what you want?
I want to get all the questinos with no answers.I use this:
all_questions=[q for q in Question.objects.all() if not q.answer_set.all()]
It works. But then I need to invoke order_by method with all_questions, so I need to change it to a queryset, how?
Or, is there a standard method like Question.objects.filter(answer_count=0) ? I find hard but no results.
Solution: Change answer_count__gt=0 to answer_count=0.
all_questions=Question.objects.annotate(answer_count=Count('answer')).filter(answer_count=0)
You should be able to use an annotation much more efficiently than doing one query per question.
Question.objects.annotate(answer_count=Count('answer')).filter(answer_count=0)
That said, you could just add the order_by directly into your Questions.objects.all() query. But like I said, it's much less efficient to do a query per question.
I have some MySQL code that looks like this:
SELECT
visitor AS team,
COUNT(*) AS rg,
SUM(vscore>hscore) AS rw,
SUM(vscore<hscore) AS rl
FROM `gamelog` WHERE status='Final'
AND date(start_et) BETWEEN %s AND %s GROUP BY visitor
I'm trying to translate this into a Django version of that query, without making multiple queries. Is this possible? I read up on how to do Sum(), and Count(), but it doesn't seem to work when I want to compare two fields like I'm doing.
Here's the best I could come up with so far, but it didn't work...
vrecord = GameLog.objects.filter(start_et__range=[start,end],visitor=i['id']
).aggregate(
Sum('vscore'>'hscore'),
Count('vscore'>'hscore'))
I also tried using 'vscore>hscore' in there, but that didn't work either. Any ideas? I need to use as few queries as possible.
Aggregation only works on single fields in the Django ORM. I looked at the code for the various aggregation functions, and noticed that the single-field restriction is hardwired. Basically, when you use, say, Sum(field), it just records that for later, then it passes it to the database-specific backend for conversion to SQL and execution. Apparently, aggregation and annotation are not standardized in SQL.
Anyway, you probably need to use a raw SQL query.
How can one write an alias for the column name in django query set.
Would be useful for union-style combinations of two linked field to the same foreign model (for instance).
for example in mysql :
select m as n, b as a from xyz
how can i do this in django query set ?
models.Table.objects.all().values('m', 'b')
Any help really appreciate it.
You can annotate the fields as you want, with the F expression:
from django.db.models import F
models.Table.objects.all().values('m', 'b').annotate(n=F('m'), a=F('b'))
Although this could have been done before by using extra(select={'n':'m','a':'b'}), I agree that this really should have been a part of values() itself.
To that end, and inspired by Alex's ticket, I have just posted a patch that adds this feature. I hope you'll find it useful!
Your reason in the comment makes no sense. Each field in the model has its own column in the database, and there's never any danger of mixing them up. You can of course tell a field to use a column name that's different from the field name:
myfield = models.CharField(max_length=10, db_column='differentname')
but I don't know if that will help you, because I still don't know what your problem is.
I'm presuming this isn't possible, so I've raised a ticket for the feature to be added, I think there is some merit to being able to do this. Please see the ticket for more information.
https://code.djangoproject.com/ticket/16735
you can also use .alias() in your queryset.
.alias(alias=SomeExpression()).annotate(
SomeOtherExpression('alias'))
.alias(alias=SomeExpression()).order_by('alias')
.alias(alias=SomeExpression()).update(field=F('alias'))
for your specific case, this would be the answer
models.Table.objects.all().alias(
n=F('m'), a=F('b')).values('m', 'a')
)
I really can't understand what you are trying to do however it sounds like what you are looking for is the extra queryset method. This for most purposes acts in the same manner as AS does in sql.
everybody.
I work with Django 1.3 and Postgres 9.0. I have very complex sql query which extends simple model table lookup with some extra fields. And it wrapped in table function since it is parameterized.
A month before I managed to make it work with the help of raw query but RawQuerySet lacks a lot of features which I really need (filters, count() and clone() methods, chainability).
The idea looks simple. QuerySet lets me to perform this query:
SELECT "table"."field1", ... ,"table"."fieldN" FROM "table"
whereas I need to do this:
SELECT "table"."field1", ... ,"table"."fieldN" FROM proxy(param1, param2)
So the question is: How can I do this? I've already started to create custom manager but can't substitute model.db_table with custom string (because it's being quoted and database stops to recognize the function call).
If there's no way to substitute table with function I would like to know if I can create QuerySet from RawQuerySet (Not the cleanest solution but simple RawQuerySet brings so much pain in... one body part).
If your requirement is that you want to use Raw SQL for efficiency purposes and still have access to model methods, what you really need is a library to map what fields of the SQL are the columns of what models.
And that library is, Unjoinify