I'm using Django 1.8 with django-rest-framework v3.2.2. I have a query that involves raw SQL:
#api_view(['GET'])
def total_spending(request, format=None):
code = request.query_params.get('code', None)
query = 'SELECT * FROM vw_presentation_summary WHERE code=%s"
cursor = connection.cursor()
cursor.execute(query, tuple([code]))
cursor.close()
My question is how to take this cursor and turn it into a data object that I can pass to django-rest-framework's Response.
Right now I'm doing it manually, which works OK:
def dictfetchall(cursor):
"Returns all rows from a cursor as a dict"
desc = cursor.description
return [
dict(zip([col[0] for col in desc], row))
for row in cursor.fetchall()
]
def total_spending(request, format=None):
...
return Response(dictfetchall(cursor))
But would it be better to use a Serializer somehow? I'm not clear if Serializers do anything useful other than define the fields you want to return.
Unless you're dealing with some complicated (including nested) representation of your model objects, a serializer is overkill if you're only going to use it for serializing objects. As you've already noticed, all of your fields can be natively serialized without any extra steps.
Serializers are great for shaping your output (renaming fields, grouping them with nested serializers) and doing it consistently. This is especially true when working with Django models, because Django doesn't natively serialize model objects down to Python dictionaries, it prefers the actual model object.
The power of a serializer comes in the deserialization, where it can map fields across models and build out relations for you. It can also do validation across all of these relations, which is something that would usually take a while to do manually.
Related
Distinct not working.I am using sqlite in backend
class getPatient(generics.ListAPIView):
def list(self, request):
queryset = Patient.objects.all().distinct()
serializer = PatientSerializer(queryset, many=True)
return Response({'patient': serializer.data})
I tried :
queryset = Patient.objects.distinct("name").all()
queryset = Patient.objects.values('name').distinct()
queryset = Patient.objects.all().distinct()
Nothing worked
I think it is not possible to use order_by and distinct in sqlite database together since in documentation here it is mentioned for example that those works only in PostgreSQL.
I have to mention that backend sqlite doesn't support distinct. Consider changing for postgresql as the most supported for django backends.
First you have to order objects by the field if you want to distinct.
I have included you the documentation and example.
By default, a QuerySet will not eliminate duplicate rows. In practice, this is rarely a problem, because simple queries such as Blog.objects.all() don’t introduce the possibility of duplicate result rows.
django distinct
Try out, by putting one of your model field. ex. "name".
class getPatient(generics.ListAPIView):
def list(self, request):
queryset = Patient.objects.order_by('name_field').distinct('name_field')
serializer = PatientSerializer(queryset, many=True)
return Response({'patient': serializer.data})
When using DjangoListObjectType from graphene_django_extras, I can define a custom qs property on a SearchFilter.
The qs function has the object as its only argument, and through that I can get the request, and in turn the query string which includes the queried fields.
Before hacking together my own parser to get these fields, am I going about this the wrong way? Or is there something else out there?
The idea is to have quite a rigid approach, as we have 7 types of paginated list types with fields that result in a few unnecessary database hits, so we want to prefetch a few fields.
Graphene has a dataloader approach which kind of looks right, but more complicated than just prefetching them at the qs stage.
Re the dataloader approach, I tried something like this
class UserLoader(DataLoader):
def batch_load_fn(self, keys):
users = {user.id: user for user in User.objects.filter(id__in=keys)}
return Promise.resolve([users.get(user_id) for user_id in keys])
user_loader = UserLoader()
class BookType(DjangoObjectType):
...
author = graphene.Field(UserType)
def resolve_author(self, *args, **kwargs):
return user_loader.load(self.author.id)
Which kind of obviously feels completely wrong, but is what they do in the docs.
Using a DataLoader is the correct approach but do self.author_id instead of self.author.id.
By calling self.author.id you're fetching the author for each book (one SQL query per author) then getting the id attribute from each author.
When you add the author ForeignKey to your Book model, behind the scenes, Django adds a column to the books table—author_id. You can pass this value to your DataLoader and have it fetch all the corresponding Authors in a single SQL query using a SELECT IN.
https://docs.djangoproject.com/en/3.2/ref/models/fields/#database-representation
You'll notice that the Graphene Docs are doing the same thing with best_friend_id in their example.
I would like to create a dict or dict-like object based on a django orm query involving prefetched objects. Due to the number of objects involved, I would like to do as much of this in the database as possible.
To use the familiar example structure, if I have an Author and Books with the usual ForeignKey from Book to Author, and I also have an Edition with a ForeignKey back to Book, I would like to do something like
Author.objects.prefetch_related(Prefetch('book_set', queryset=Book.objects.filter(cycle__id=3).select_related('edition__binding'), to_attr="bindings")) # clearly this is wrong
and ultimately call .values() or something of the sort to get a dict-like result consisting of rows of authors, which should include an entry "bindings", which contains a list of bindings that each author has been published in. The stretch goal would be to have bindings be a semicolon-separated list, ie [{"Author": "Daniel Dennett", "bindings": "paper; hardback; cloth"}, {"Author": "Jemiah Jefferson", "bindings": "paper; zine"}]
So far, I have been able to get a field like "bindings" attached to the queryset using prefetch_related and select_related, as above, but this field is not included in the result of a call to .values(). This means I have to loop over the objects, which simply takes too long for my purposes (there are many objects, and the request times out)
Create custom a Concat annotation which will mimic MySQL GROUP_CONCAT function. Then you can use .values on the annotated bindings.
For Django 1.8 your Concat class can be something like this:
from django.db import models
class Concat(models.Aggregate):
# supports GROUP_CONCAT(DISTINCT field SEPARATOR str_val)
# do not support order_by
function = 'GROUP_CONCAT'
template = '%(function)s(%(distinct)s%(expressions)s SEPARATOR "%(separator)s")'
def __init__(self, expression, distinct=False, separator=None, **extra):
super(Concat, self).__init__(
expression,
distinct='DISTINCT ' if distinct else '',
separator=separator or '',
output_field=models.CharField(),
**extra)
While in Django 1.7
class Concat(models.Aggregate):
def add_to_query(self, query, alias, col, source, is_summary):
#we send source=CharField to prevent Django from casting string to int
aggregate = SQLConcat(col, source=models.CharField(), is_summary=is_summary, **self.extra)
query.aggregates[alias] = aggregate
#for mysql
class SQLConcat(models.sql.aggregates.Aggregate):
sql_function = 'group_concat'
#property
def sql_template(self):
if self.extra.get('separator'):
return '%(function)s(%(field)s SEPARATOR "%(separator)s")'
else:
return '%(function)s(%(field)s)'
Now you can do:
Author.objects.annotate(bindings=Concat('book__edition__binding')).values('name', 'bindings')
unfortunately this wont filter your books by cycle__id=3, but you can apply the filter before the annotation happens.
Author.objects.filter(book__cycle__id=3).annotate(bindings=Concat('book__edition__binding')).values('name', 'bindings')
This will strip from the result authors without book with cycle__id=3.
(Django 1.8) I have a table which has 4 many-to-many relationship to other tables.
Two of these tables have so many entries and that is causing the admin page to load very slowly because it is trying to load all the entries in the lists.
Is there a way to avoid the internal admin page query for loading all the entries of the big tables to speed up the admin page load?
I think the best way is to only list the selected values but I'm not sure how.
I'm not sure how to use limit_choices_to in here:
class Data(models.Model):
pass # stuff here
class Report(models.Model):
data= models.ManyToManyField(Data)
I also tried adding this to my admin.py but it did not help at all. It's not limiting for some reason:
def queryset(self, request):
qs = super(MyModelAdmin, self).queryset(request)
if len(qs) > 10:
qs = qs[:10]
return qs
If you still want to use limit_choices_to, then please refer to the docs. You basically just supply the filters in a dictionary object.
To speed up the admin, my suggestions include:
1. Using raw_id_fields in your ModelAdmin. This gives you a little searchbox instead of a selectbox and avoids the overhead of listing all related objects.
2. If you are handling forward ForeignKey relationships, you can use list_select_related in your ModelAdmin as well. In your case, you are handling many-to-many relationships, so you can try overriding the get_queryset method of the ModelAdmin, and use prefetch_related like in the code below.
from django.contrib import admin
class TestModelAdmin(admin.ModelAdmin):
def get_queryset(self, request):
test_model_qs = super(TestModelAdmin, self).get_queryset(request)
test_model_qs = test_model_qs.prefetch_related('many-to-many-field')
return test_model_qs
If you really like to get your hands dirty, I highly recommend you use django-debug-toolbar. It really gives you visibility on how many and what SQL statements are being run. If you can read SQL, you can deduce what you need to input to select_related and prefetch_related.
I have an instance of a Django (1.6) model (let's take User for example). I would like to get the field values for that model, like I can do for a QuerySet, by calling QuerySet().values('first_name', 'username'). Is that possible, or should I just create a dictionary with the required fields?
Edit: A bit more insight into why I need this (maybe there are other workarounds). I want to return a Django model as a JSON response (by using json.dumps, not Django's JSON serializer), and so far, I can do that by extending the default Python JSON encoder, and treating Django models specially, by converting them to dictionaries using model_to_dict. The problem is that this doesn't get me the related objects, which I need.
Here's my code, for reference:
class JsonEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, models.Model):
return model_to_dict(obj) # here I'd like to pull some related values
return json.JSONEncoder.default(self, obj)
If you want to pull all related values by default, you can do the following:
def default(self, obj):
if isinstance(obj, models.Model):
d = model_to_dict(obj) # here I'd like to pull some related values
for field in obj._meta.fields:
if field.rel: # single related object
d[field.name] = model_to_dict(getattr(obj, field.name))
return json.JSONEncoder.default(self, obj)
This will go one level deep for single related objects, but not for many-to-many relations or reverse foreign keys. Both are possible, but you'll have to find out which methods/attributes on obj._meta return the specific fields.
If you only want to retrieve specific fields, you'll have to manually specify and fetch these fields.