The sql subquery is:
SELECT *
FROM ( SELECT *
FROM article
ORDER BY Fid desc
LIMIT 0, 200
) as l
WHERE keyId = 1
AND typeId = 0
I tried this:
rets = Article.objects.order_by("-Fid").values('Fid')
kwargs['keyId'] = 1
kwargs['typeId'] = 0
re = Article.objects.filter(Fid__in=rets).filter(**kwargs).values()
But it's not working. Can anyone explain how I can do this?
In your case I guess you can resort to raw SQL (untested). Note that using raw SQL you have to know the real table and column names (just test the statement directly on the database first, to see if it flies).
For example:
Article.objects.raw("""SELECT * from (
SELECT * FROM yourapp_article
ORDER BY fid DESC
LIMIT 0, 200
) AS q1 WHERE key_id=1 AND type_id=0""")
[update]
wuent wrtote:
thanks for your help. But the raw SQL is not my wanted. I need keep my program orm style. – wuent
If you are used to more powerful/consistent ORMs like SQLAlchemy or even peewee, give up your hope. The Django ORM has a very crippled design and AFAIK you can't do this kind of thing using it - the first version of this answer started with a rant about this.
Looking at your query again, I got the impression that you do not need a subquery, try querying the table directly - my guess is the result will be the same.
How about this?
rets = Article.objects.order_by("-Fid").values_list('Fid', flat=True)
kwargs['keyId'] = 1
kwargs['typeId'] = 0
re = Article.objects.filter(Fid__in=rets).filter(**kwargs).values()
Related
I have this query which selects values from two different tables and used array agg over matched IDs how can I get same results using the queryset. Thank you!
select
sf.id_s2_users ,
array_agg(sp.id)
from
s2_followers sf
left join s2_post sp on
sp.id_s2_users = sf.id_s2_users1
where
sp.id_s2_post_status = 1
and sf.id_s2_user_status = 1
group by
sf.id_s2_users
You can run raw SQL queries with Django's ORM if that's what you wanted. You don't have to change your query in that case, you can check documentation here.
I am using symfony 2.8.39 and Doctrine 2.4.8 and have problems with paged results. Underlying is an Mysql5.7 server.
The documentation on doctrine paging says:
Paginating Doctrine queries is not as simple as you might think in the
beginning. If you have complex fetch-join scenarios with one-to-many
or many-to-many associations using the "default" LIMIT functionality
of database vendors is not sufficient to get the correct results.
https://www.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/pagination.html
This is exactly the situation I have. My statement in SQL translation looks like this:
SELECT sc.id, sc.name, scc.prio, sd.description
FROM sang_contents sc
JOIN sang_categories_contents scc
JOIN sang_descriptions sd
JOIN sang_languages sl
WHERE
sc.id = scc.content_id AND
scc.category_id = 20 AND
scc.is_enabled = 1 AND
sc.id = sd.content_id AND
sd.language_id = sl.id AND
sd.description != "" AND
sl.name = "DE"
ORDER BY scc.prio ASC, sc.id DESC
As ORM is at Version 3.0 and this problem exists since the beginning I don't think it will be fixed anytime by ORM.
So what to do to achieve proper results for paging?
My idea to solve this is so far to paginate over simplified data the paging should be able to handle correctly:
create a table containing the result for all categories and languages and access it with an extra entity.
The disadvantage is, that I would have to update this table every time a change is done in the for connected tables.
Would you suggest another solution to this problem?
I guess 3rd party software like
https://github.com/KnpLabs/KnpPaginatorBundle/releases
or
https://github.com/whiteoctober/WhiteOctoberPagerfantaBundle/releases
are just sitting on top of the ORM pagination and would not fix the underlying problem.
Correct?
This is my code at the moment:
$page = max(0, $request->query->getInt('page', 0));
$pageRequest = new PageRequest($itemsPerPage, $page);
$query = $this->em->createQuery(
'SELECT sc, sd
FROM NamiApiCoreBundle:Content sc
JOIN sc.categoryContents scc
JOIN sc.descriptions sd
JOIN sd.language sl
WHERE
sc.id = scc.content AND
scc.category = :id AND
scc.enabled = 1 AND
sc.id = sd.content AND
sd.language = sl.id AND
sd.description != \'\' AND
sl.iso = :lang
ORDER BY scc.priority ASC, sc.id DESC'
)
->setFirstResult($pageRequest->getOffset())
->setParameter('lang', $lang)
->setParameter('id', $categoryId)
->useResultCache(true, $this->cache_lifetime);
if ($itemsPerPage > 0) {
$query->setMaxResults($pageRequest->getSize());
}
$paginator = new Paginator($query);
I have the following sample code:
queries = []
q1 = select([columns]).where(table.c.id == #).limit(#)
queries.append(q1)
q2 = select([columns]).where(table.c.id == #).limit(#)
queries.append(q2)
final_query = union_all(*queries)
The generated SQL should be this:
(select columns from table where id = # limit #)
UNION ALL
(select columns from table where id = # limit #)
But, I'm getting
select columns from table where id = # limit #
UNION ALL
select columns from table where id = # limit #
I tried using subquery, as follows for my queries:
q1 = subquery(select([columns]).where(table.c.id == #).limit(#))
The generated query then looks like this:
SELECT UNION ALL SELECT UNION ALL
I also tried doing
q1 = select([columns]).where(table.c.id == #).limit(#)).subquery()
But, I get the error:
'Select' object has no attribute 'subquery'
Any help to get the desired output with my subqueries wrapped in parentheses?
Note: this is not a duplicate of this question, because I'm not using Session.
EDIT
Okay, this works, but I don't believe it is very efficient, and it's adding an extra select * from (my sub query), but it works.
q1 = select('*').select_from((select(columns).where(table.c.id == #).limit(#)).alias('q1'))
So, if anyone has any ideas to optimize, or let me know if this is as good as it gets. I would appreciate it.
The author of SQLAlchemy seems to be aware of this and mentions a workaround for it on the SQLAlchemy 1.1 changelog page. The general idea is to do .alias().select() on each select.
stmt1 = select([table1.c.x]).order_by(table1.c.y).limit(1).alias().select()
stmt2 = select([table2.c.x]).order_by(table2.c.y).limit(2).alias().select()
stmt = union(stmt1, stmt2)
class Log:
project = ForeignKey(Project)
msg = CharField(...)
date = DateField(...)
I want to select the four most recent Log entries where each Log entry must have a unique project foreign key. I've tries the solutions on google search but none of them works and the django documentation isn't that very good for lookup..
I tried stuff like:
Log.objects.all().distinct('project')[:4]
Log.objects.values('project').distinct()[:4]
Log.objects.values_list('project').distinct('project')[:4]
But this either return nothing or Log entries of the same project..
Any help would be appreciated!
Queries don't work like that - either in Django's ORM or in the underlying SQL. If you want to get unique IDs, you can only query for the ID. So you'll need to do two queries to get the actual Log entries. Something like:
id_list = Log.objects.order_by('-date').values_list('project_id').distinct()[:4]
entries = Log.objects.filter(id__in=id_list)
Actually, you can get the project_ids in SQL. Assuming that you want the unique project ids for the four projects with the latest log entries, the SQL would look like this:
SELECT project_id, max(log.date) as max_date
FROM logs
GROUP BY project_id
ORDER BY max_date DESC LIMIT 4;
Now, you actually want all of the log information. In PostgreSQL 8.4 and later you can use windowing functions, but that doesn't work on other versions/databases, so I'll do it the more complex way:
SELECT logs.*
FROM logs JOIN (
SELECT project_id, max(log.date) as max_date
FROM logs
GROUP BY project_id
ORDER BY max_date DESC LIMIT 4 ) as latest
ON logs.project_id = latest.project_id
AND logs.date = latest.max_date;
Now, if you have access to windowing functions, it's a bit neater (I think anyway), and certainly faster to execute:
SELECT * FROM (
SELECT logs.field1, logs.field2, logs.field3, logs.date
rank() over ( partition by project_id
order by "date" DESC ) as dateorder
FROM logs ) as logsort
WHERE dateorder = 1
ORDER BY logs.date DESC LIMIT 1;
OK, maybe it's not easier to understand, but take my word for it, it runs worlds faster on a large database.
I'm not entirely sure how that translates to object syntax, though, or even if it does. Also, if you wanted to get other project data, you'd need to join against the projects table.
I know this is an old post, but in Django 2.0, I think you could just use:
Log.objects.values('project').distinct().order_by('project')[:4]
You need two querysets. The good thing is it still results in a single trip to the database (though there is a subquery involved).
latest_ids_per_project = Log.objects.values_list(
'project').annotate(latest=Max('date')).order_by(
'-latest').values_list('project')
log_objects = Log.objects.filter(
id__in=latest_ids_per_project[:4]).order_by('-date')
This looks a bit convoluted, but it actually results in a surprisingly compact query:
SELECT "log"."id",
"log"."project_id",
"log"."msg"
"log"."date"
FROM "log"
WHERE "log"."id" IN
(SELECT U0."id"
FROM "log" U0
GROUP BY U0."project_id"
ORDER BY MAX(U0."date") DESC
LIMIT 4)
ORDER BY "log"."date" DESC
I have the following method:
def select_query(self):
sql = "SELECT * FROM {t} WHERE 1".format(t=self._meta.db_table)
for column_name in self.distinguishing_column_names():
sql = sql + " AND {c} = {v}".format(c=column_name, v=getattr(self, column_name))
return sql
This will give me a query like this:
SELECT * FROM customer WHERE 1 AND name = JOHN SMITH AND customer_number = 11423 AND social_security_number = 1234567890 AND phone = 2323523353
Obviously, that's not going to work. Is there a way to get Django to quote this for me?
Note: I'm not asking for a prepared statement. That's something different.
Do you need to return a query this way? The proper way would be to call cursor with the query and the params as argument:
Does Python support MySQL prepared statements?
The correct way to format a query seems to be:
query = query % db.literal(args)
Where db is a mysql.Connection (or presumably any connection)
Apparently the answer is "no."