I'm performing a query in a function inside the views.py as follows:
input_dict = {'id': 'id',
'table': 'my_table',
'first_col': 'my_first_col',
'first_name': 'my_first_name'}
query = '''
SELECT %(id)s
FROM %(table)s
WHERE %(first_col)s = %(first_name)s '''
qs = my_model.objects.raw(query, input_dict)
print(qs)
which prints:
qs: <RawQuerySet:
SELECT id
FROM my_table
WHERE my_first_col = my_first_name >
However, when I try to run this line:
ids = [item.id for item in qs]
it gives me an error:
psycopg2.errors.SyntaxError: syntax error at or near "'my_table'"
LINE 3: FROM 'my_table'
and also:
django.db.utils.ProgrammingError: syntax error at or near "'my_table'"
LINE 3: FROM 'my_table'
What should I do?
django are not able to excecut your query.
I believe what you trying to acheive via the raw function it is not possible.
read the docs https://docs.djangoproject.com/en/3.0/topics/db/sql/ for better understanding how the function work
your dynamic options are realted either to the table fields and the where params:
the select fields
you can map fields in the query to model fields using the translations
argument to raw()
like :
name_map = {'first': 'first_name', 'last': 'last_name', 'bd': 'birth_date', 'pk': 'id'}
Person.objects.raw('SELECT * FROM some_other_table', translations=name_map)
the where params
If you need to perform parameterized queries, you can use the params
argument to raw()
lname = 'Doe'
Person.objects.raw('SELECT * FROM myapp_person WHERE last_name = %s', [lname])
at the end of the day, this will work, although it's not recomended unless you know exactly what you are doing as it might expose your system to sql attacks
query = '''
SELECT %(id)s
FROM %(table)s
WHERE %(first_col)s = %(first_name)s '''
query = query%input_dict
qs = my_model.objects.raw(query)
I have a table which contains list of some web sites and a table with statistics of them.
class Site(models.Model):
domain_name = models.CharField(
max_length=256,
unique=True,
)
class Stats(models.Model):
date = models.DateField()
site = models.ForeignKey('Site')
google_pr = models.PositiveIntegerField()
class Meta:
unique_together = ('site', 'date')
I want to see all sites and statistics for a concrete date. If a stats record for the date doesn't exist, then the selection must contain only site.
If I use:
Site.objects.filter(stats__date=my_date)
I will not get sites which have no records for my_date in stats table. Because in this case the SQL query will be like the following:
SELECT *
FROM site
LEFT OUTER JOIN stats ON site.id = stats.site_id
WHERE stats.date = 'my_date'
The query condition will exclude records with NULL-dates and sites without stats will be not included to the selection.
In my case I need join stats table, which has already been filtered by date:
SELECT *
FROM site
LEFT OUTER JOIN
(SELECT *
FROM stats
WHERE stats.date = 'my-date') AS stats
ON site.id = stats.site_id
How can I translate this query to Django ORM?
Thanks.
In Django v2.0 use FilteredRelation
Site.objects.annotate(
t=FilteredRelation(
'stats', condition=Q(stats__date='my-date')
).filter(t__google_pr__in=[...])
I had a similar problem and wrote the following utility function for adding left outer join on a subqueryset using Django ORM.
The util is derived from a solution given to add custom left outer join to another table (not subquery) using Django ORM. Here is that solution: https://stackoverflow.com/a/37688104/2367394
Following is the util and all related code:
from django.db.models.fields.related import ForeignObject
from django.db.models.options import Options
from django.db.models.sql.where import ExtraWhere
from django.db.models.sql.datastructures import Join
class CustomJoin(Join):
def __init__(self, subquery, subquery_params, parent_alias, table_alias, join_type, join_field, nullable):
self.subquery_params = subquery_params
super(CustomJoin, self).__init__(subquery, parent_alias, table_alias, join_type, join_field, nullable)
def as_sql(self, compiler, connection):
"""
Generates the full
LEFT OUTER JOIN (somequery) alias ON alias.somecol = othertable.othercol, params
clause for this join.
"""
params = []
sql = []
alias_str = '' if self.table_alias == self.table_name else (' %s' % self.table_alias)
params.extend(self.subquery_params)
qn = compiler.quote_name_unless_alias
qn2 = connection.ops.quote_name
sql.append('%s (%s)%s ON (' % (self.join_type, self.table_name, alias_str))
for index, (lhs_col, rhs_col) in enumerate(self.join_cols):
if index != 0:
sql.append(' AND ')
sql.append('%s.%s = %s.%s' % (
qn(self.parent_alias),
qn2(lhs_col),
qn(self.table_alias),
qn2(rhs_col),
))
extra_cond = self.join_field.get_extra_restriction(
compiler.query.where_class, self.table_alias, self.parent_alias)
if extra_cond:
extra_sql, extra_params = compiler.compile(extra_cond)
extra_sql = 'AND (%s)' % extra_sql
params.extend(extra_params)
sql.append('%s' % extra_sql)
sql.append(')')
return ' '.join(sql), params
def join_to(table, subquery, table_field, subquery_field, queryset, alias):
"""
Add a join on `subquery` to `queryset` (having table `table`).
"""
# here you can set complex clause for join
def extra_join_cond(where_class, alias, related_alias):
if (alias, related_alias) == ('[sys].[columns]',
'[sys].[database_permissions]'):
where = '[sys].[columns].[column_id] = ' \
'[sys].[database_permissions].[minor_id]'
children = [ExtraWhere([where], ())]
return where_class(children)
return None
foreign_object = ForeignObject(to=subquery, from_fields=[None], to_fields=[None], rel=None)
foreign_object.opts = Options(table._meta)
foreign_object.opts.model = table
foreign_object.get_joining_columns = lambda: ((table_field, subquery_field),)
foreign_object.get_extra_restriction = extra_join_cond
subquery_sql, subquery_params = subquery.query.sql_with_params()
join = CustomJoin(
subquery_sql, subquery_params, table._meta.db_table,
alias, "LEFT JOIN", foreign_object, True)
queryset.query.join(join)
# hook for set alias
join.table_alias = alias
queryset.query.external_aliases.add(alias)
return queryset
join_to is the utility function you want to use. For your query you can use it in as follows:
sq = Stats.objects.filter(date=my_date)
q = Site.objects.filter()
q = join_to(Site, sq, 'id', 'site_id', q, 'stats')
And following statement would print a query similar to you example query (with subquery).
print q.query
Look at it this way: you want to see statistics with accompanying site data for certain date, which translates to:
Stats.objects.filter(date=my_date).select_related('site')
I'm stuck with adding a filter to a custom manager in Django. This is my current custom manager that is working:
class VoteAwareManager(models.Manager):
def _get_score_annotation(self):
model_type = ContentType.objects.get_for_model(self.model)
table_name = self.model._meta.db_table
return self.extra(select={
'active': 'select active from %s mh where mh.main_id = %s.id and mh.active = true and mh.date_begin = (select max(date_begin) from euvoudebicicletaengine_mainhistoric where main_id = mh.main_id) and mh.date_end >= now()' % (MainHistoric._meta.db_table, table_name),
'row_num': '(row_number() over(order by (SELECT COALESCE(SUM(vote / ((extract(epoch from now() - time_stamp )/3600)+2)^1.5),0) FROM %s WHERE content_type_id=%d AND object_id=%s.id) DESC))' % (Vote._meta.db_table, int(model_type.id), table_name), # To know the position(#number) on the front page
'score': 'SELECT COALESCE(SUM(vote / ((extract(epoch from now() - time_stamp )/3600)+2)^1.5),0) FROM %s WHERE content_type_id=%d AND object_id=%s.id' % (Vote._meta.db_table, int(model_type.id), table_name)
}
)
def most_loved(self,):
return self._get_score_annotation().order_by('-score')
def most_hated(self):
return self._get_score_annotation().order_by('score')
I need to add a filter to the most_loved and most_hated to active=True that will be the SQL equivalent to where active=true in the main sql expression.
Any clues on how to do it?
I think you probably need to write a SQL view (to replace your extra() function) and create a new unmanaged model for the view (including active as a field in your model).
As in this question. Or this (possibly out of date) one.
Then use the view in your _get_score_annotation and add a filter to the queryset you're getting from that function.
def _get_score_annotation(self):
return ContentTypeView.objects.filter(# any filtering you need)
def most_loved(self,):
return self._get_score_annotation().filter(active=True).order_by('-score')
I'm trying to improve the performance of this code:
orderitems = OrderItem.objects.filter(order__contact=contact)
for orderitem in orderitems:
try:
pv = ProductVariation.objects.get(product=orderitem.product)
if pv.parent_id == parent_product.id:
return True
except:
pass
Essentially I want to get rid of the 'for' loop because its slow. I would like to do it using a single queryset if possible, but I just can't get my head around the syntax. Here is the SQL that I effectively want to reproduce. It creates a list which is fairly short so I can iterate through that looking for a match:
SELECT parent_id
FROM configurable_productvariation
WHERE product_id IN (
SELECT product_id
FROM shop_orderitem
WHERE order_id
IN (
SELECT id
FROM shop_order
WHERE contact_id = 4));
The '4' is the 'contact' referred to in the first line of python.
Many thanks,
Thomas
This should generate sql like yours
product_ids = OrderItem.objects.filter(order__contact=contact).values_list('product_id', flat=True)
ProductVariation.objects.filter(product_id__in=product_ids)
How can I assign an alias name to the main table of a QuerySet in Django?
queryset = Price.objects
queryset = queryset.extra(
where = ['p1.created = (select max(p2.created) from products_price p2 where p2.product_id = p1.product_id)']
)
I would like to set the 'p1' alias name to the Price main table to use it in the subselect.
Edit: Note that there is a lastest Price for each Produtc.
You can see the sql query to do the next:
queryset = Price.objects.all()
print queryset.query
If you know first sql query. You can do the subquery better.
Although, I do the next:
price_max = Price.objects.all().order_by('-created')[0]
queryset = Price.objects.filter(created=price_max)
Or the best:
https://docs.djangoproject.com/en/1.3/topics/db/aggregation/#generating-aggregates-over-a-queryset
from django.db.models import Max
price_max = Price.objects.aggregate((Max('created'))['created__max']
queryset = Price.objects.filter(created=price_max)