With a raw query the columns for the SQL that will be executed can be accessed like this.
query_set = Model.objects.raw('select * from table')
query_set.columns
There is no columns attribute for django.db.models.query.QuerySet.
I'm not using raw ... I was debugging a normal query set using Q filters ... it was returning way too many records ... so I wrote code to see what the results would be with normal SQL. I'd like the see the columns and the actual SQL being called on a query set so I can diagnose what the issue is without guessing.
How do you get the columns or the SQL that will be executed from a django.db.models.query.QuerySet instance?
Ok, so based on your comment, if you want to see what django has created in terms of SQL there's an attribute you can use on the Queryset.
Once you've written your query, qs = MyModel.objects.all(), you can then inspect that by doing qs.query, which if you print that out will show you the SQL query itself.
You'll have to inspect this query to see what columns are being included.
The query object is a class called Query which I can't find mention of in the django docs, but it's source is here; https://github.com/django/django/blob/master/django/db/models/sql/query.py#L136
Related
I am trying load some data into datatables. I am trying to specify columns in the model.objects query by using .only() --- at first glance at the resulting QuerySet, it does in fact look like the mySQL query is only asking for those columns.
However, When I try to pass the QuerySet into Paginator, and/or a Serializer, the result has ALL columns in it.
I cannot use .values_list() because that does not return the nested objects that I need to have serialized as part of my specific column ask. I am not sure what is happening to my .only()
db_result_object = model.objects.prefetch_related().filter(qs).order_by(asc+sort_by).only(*columns_to_return)
paginated_results = Paginator(db_result_object,results_per_page)
serialized_results = serializer(paginated_results.object_list,many=True)
paginated_results.object_list = serialized_results.data
return paginated_results
This one has tripped me up too. In Django, calling only() doesn't return data equivalent to a SQL statement like this:
SELECT col_to_return_1, ... col_to_return_n
FROM appname_model
The reason it doesn't do it like this is because Django returns data to you not when you construct the QuerySet, but when you first access data from that QuerySet (see lazy QuerySets).
In the case of only() (a specific example of what is called a deferred field) you still get all of the fields like you normally would, but the difference is that it isn't completely loaded in from the database immediately. When you access the data, it will only load the fields included in the only statement. Some useful docs here.
My recommendation would be to write your Serializer so that it is only taking care of the one specific filed, likely using a SerializerMethodField with another serializer to serialize your related fields.
To optimize a lot my database I would like to make as less as possible any query.
I'm trying to get an object, increment the field "count_limit" and make an If statement after on the Customer instance.
To achieve it I've made this query who worked well.
Customer.objects.filter(user=user).update(count_limit=F('count_limit') + 1)
So after this query, count_limit has been incremented by 1 as I wanted.
When I'm trying to get the Customer instance as a result of this query, it returns "1".
Is it possible to make both, update the instance and get it as a return object ?
Thanks a lot
The update() method will return the number of updated rows. If you are using Postgres, then you can use the returning clause with the raw query.
query = 'UPDATE customer SET count_limit=(customer.count_limit + 1) WHERE customer.user_id=%s returning *'
updated_obj = Customer.objects.raw(query, [user.id])
I don't know if this can be achieved by ORM, but suggestions will be appreciated.
Make sure that the table name in raw query is correct. If you haven't definer db_table in the meta class of your model, then by default it will be myapp_model.
And to prevent SQL injection, from the Docs:
Do not use string formatting on raw queries or quote placeholders in
your SQL strings!
Follow Docs on raw()
You are looking for F functions: https://docs.djangoproject.com/en/3.0/ref/models/expressions/#f-expressions
Example from their documentation how to increase a counter
from django.db.models import F
reporter = Reporters.objects.get(name='Tintin')
reporter.stories_filed = F('stories_filed') + 1
reporter.save()
I writing a project using Django REST Framework, Django and Postgres as a database. I want to bulk delete in one query. So, it is possible to do without writing a query using pure SQL?
There is an example, but the count of executet query equal length of a list of ids (for example, if in delete_ids 2 ids, Django will execute 2 queries):
delete_ids = [...]
MyModel.objects.filter(id__in=delete_ids).delete()
Not possible using the filter and delete together using raw sql query.
https://docs.djangoproject.com/en/2.1/topics/db/sql/
MyModel.objects.raw('DELETE FROM my_model WHERE id IN (%s)', [','.join([list_of_ids])])
For fast deletes won't advice but you can also use
sql.DeleteQuery(MyModel).delete_qs(qs, using=qs.db)
jackotyne's answer is incorrect as a DELETE statement cannot be run with django raw. The idea behind django raw is that it returns a queryset, but DELETE won't do that.
Please read the reply to this answer.
You will need a database cursor as stated in the django documentation.
with connection.cursor() as cursor:
cursor.execute(
'DELETE FROM "appname_modelname" WHERE id IN (%s)' % ', '.join(delete_ids)
)
Of course it is better to filter with django and get a queryset and do a bulk delete with queryset.delete(), but that is not always possible depending on the data's logic.
I am trying to get the field value of a joined table. This is the generated sql of ORM query.
SELECTsubnets_subnetoption.id,
subnets_subnetoption.subnet_id,subnets_subnetoption.value_id,
subnets_subnet.id,subnets_subnet.parent_id,
subnets_subnet.base_address,subnets_subnet.bits,
subnets_subnet.bcast_address,subnets_subnet.is_physical,
subnets_subnet.name,subnets_subnet.responsible,
subnets_subnet.building_floor,subnets_subnet.comments,
subnets_subnet.vlan_common_name,subnets_subnet.creation_date,
subnets_subnet.modification_date,subnets_subnet.sec_level,
subnets_subnet.confid,subnets_subnet.access_type,
subnets_subnet.zone_type,options_value.id,
options_value.content,options_value.comment,
options_value.option_id,options_option.id,
options_option.name,options_option.required,
options_option.scope_id,options_scope.id,
options_scope.nameFROMsubnets_subnetoptionINNER JOIN
subnets_subnetON (subnets_subnetoption.subnet_id=
subnets_subnet.id) INNER JOINoptions_valueON
(subnets_subnetoption.value_id=options_value.id) INNER JOIN
options_optionON (options_value.option_id=
options_option.id) INNER JOINoptions_scopeON
(options_option.scope_id=options_scope.id) WHERE
subnets_subnetoption.subnet_id` = 1
SubnetOption.objects.select_related().filter(subnet_id=subnet['id']).query
I need only options_value.content and options_option.name, but query set i giving the subnetoption table values only. How can I get the joined tables values. I am new to django
SubnetOption.objects.filter(subnet_id=subnet['id']).select_related().values('options_value__content')
or
SubnetOption.objects.filter(subnet_id=subnet['id']).select_related('modelname_in_wholelowercase')
try this once
You can use raw query of django, which means you can put SQL query as it is, for reference
Raw sql queries in Django views
In one of the django apps we use two database engine A and B, both are the same database but with different schemas. We have a table called C in both schemas but using db routing it's always made to point to database B. We have formed a valuelist queryset from one of the models in A, tried to pass the same in table C using filter condition __in but it always fetches empty though there are matching records. When we convert valueslist queryset to a list and use it in table C using filter condition __in it works fine.
Not working
data = modelindbA.objects.values_list('somecolumn',flat=True)
info = C.objects.filter(somecolumn__in=data).values_list
Working
data = modelindbA.objects.values_list('somecolumn',flat=True)
data = list(data)
info = C.objects.filter(somecolumn__in=data).values_list
I have read django docs and other SO questions, couldn't find anything relative. My guess is that since both models are in different database schemas the above is not working. I need assistance on how to troubleshoot this issue.
When you use a queryset with __in, Django will construct a single SQL query that uses a subquery for the __in clause. Since the two tables are in different databases, no rows will match.
By contrast, if you convert the first queryset to a list, Django will go ahead and fetch the data from the first database. When you then pass that data to the second query, hitting the second database, it will work as expected.
See the documentation for the in field lookup for more details:
You can also use a queryset to dynamically evaluate the list of values instead of providing a list of literal values.... This queryset will be evaluated as subselect statement:
SELECT ... WHERE blog.id IN (SELECT id FROM ... WHERE NAME LIKE '%Cheddar%')
Because values_list method returns django.db.models.query.QuerySet, not a list.
When you use it with same schema the orm optimise it and should make just one query, but when schemas are different it fails.
Just use list().
I would even recommend to use it for one schema since it can decrease complexity of query and work better on big tables.