Today I have written a DRF view method using prefetch_related:
def post(self, request, post_uuid, format=None):
post = Post.objects.prefetch_related('postimage_set').get(uuid=post_uuid)
postimage_set = post.postimage_set.all()
for image in postimage_set:
...
return Response('', status.HTTP_200_OK)
And I fear that I am using prefetch_related wrongfully with this. Does it make sense to use prefetch_related here or will this fetch all posts as well as all postimages and then filter this set to just one instance? I'm super thankful for any help on this.
Looks kinda unnatural. Without looking at your database structure I can only guess, that what you really want to do is:
PostImage.objects.filter(post__uuid=post_uuid) (mind the usage of a dunder between post and uuid - that simple trick follow the relation attribute) which should result in a single query.
Moreover, if you are uncertain of a number of queries that will hit the database, you can write a very precise test with one of the assertions, that is available since Django 1.3: assertNumQueries
Related
When using DjangoListObjectType from graphene_django_extras, I can define a custom qs property on a SearchFilter.
The qs function has the object as its only argument, and through that I can get the request, and in turn the query string which includes the queried fields.
Before hacking together my own parser to get these fields, am I going about this the wrong way? Or is there something else out there?
The idea is to have quite a rigid approach, as we have 7 types of paginated list types with fields that result in a few unnecessary database hits, so we want to prefetch a few fields.
Graphene has a dataloader approach which kind of looks right, but more complicated than just prefetching them at the qs stage.
Re the dataloader approach, I tried something like this
class UserLoader(DataLoader):
def batch_load_fn(self, keys):
users = {user.id: user for user in User.objects.filter(id__in=keys)}
return Promise.resolve([users.get(user_id) for user_id in keys])
user_loader = UserLoader()
class BookType(DjangoObjectType):
...
author = graphene.Field(UserType)
def resolve_author(self, *args, **kwargs):
return user_loader.load(self.author.id)
Which kind of obviously feels completely wrong, but is what they do in the docs.
Using a DataLoader is the correct approach but do self.author_id instead of self.author.id.
By calling self.author.id you're fetching the author for each book (one SQL query per author) then getting the id attribute from each author.
When you add the author ForeignKey to your Book model, behind the scenes, Django adds a column to the books table—author_id. You can pass this value to your DataLoader and have it fetch all the corresponding Authors in a single SQL query using a SELECT IN.
https://docs.djangoproject.com/en/3.2/ref/models/fields/#database-representation
You'll notice that the Graphene Docs are doing the same thing with best_friend_id in their example.
There are hundreds of questions here on various django annotate/aggregate constructions and filters but I couldn't find this simple use-case asked (or answered).
I have a "Payments" model and associated ListAPIView ViewSet endpoint with nicely setup DjangoFilters so the client can filter on created__lte, created__gte, company= etc.
Now I want to add an endpoint that derives from the above but only returns the sum of some fields and count of the total filtered objects.
I know exactly how to do this if I would just write the View "from scratch" (I can just hack the DRF View into executing get_queryset().aggregate() then feeding into a serializer and returning), but I want to do it in a "Django-way" if possible.
For example, combined with a serializer that defines "total_amount" and "nbr", this (almost) works:
queryset = models.Payment.objects.values('company').annotate(total_amount=Sum('amount'),
nbr=Count('id'))
The values() call groups by "company" (a sub-field of Payment), which in combination with annotate() performs a sum by all company payments and annotates with total_amount/nbr. Adding filtering query parameters magically adjusts what goes into the annotation properly.
The problem is, what if I don't want to group (or even filter) by "company", I just want to "group by all"? Is there a way to do that?
I realize this is already a bit magical but the Django-esque way of doing grouping for annotation is this as far as I know.
I also realize I'm probably really better off just by hijacking .retrieve() to evaluate the queryset with .aggregate() tacked at the end and on-the-fly creating the response... still curious though :)
I ended up with the "hack", overriding list() in the View with an .aggregate call and the packing it for the Response through a serializer. This was the most canonical way I could figure out (I mean, re-using as many of the moving parts of Django/DRF as possible like automatic filtering of the queryset, serializing etc).
Bonus: note the Coalesce() wrap which is needed because Sum() doesn't return 0 if the set is empty.
def list(self, request, *args, **kwargs):
queryset = self.filter_queryset(self.get_queryset())
stats = queryset.aggregate(total_amount=Coalesce(Sum('amount'), 0),
total_vat=Coalesce(Sum('vat'), 0),
nbr_payments=Count('id'))
# .aggregate() returns a dict of the results, not a QuerySet. Wrap it
# into a response through the serializer.
sclass = self.get_serializer_class()
return Response(sclass(stats).data)
I am creating an app with a rest API that should return values for instances of objects based on the url given. Right now I have the API working using ModelViewSets of my objects for the API.
For example I have three objects, user, transactions, and goals.
As it stands I can go to /mysite/api/users and return a list of all users
I can also go to /mysite/api/users/1 to return just the user with the id '1'.
I can do something similar with transactions and goals.
What I'm looking to do is go to url /mysite/api/users/1/transaction/1/goal
to find the goal associated with the transaction for that user.
I've been scouring tutorials and am not sure what the right question is to ask in order to find something useful to learn how to do this. What is the correct way to go about setting up my rest api like this?
If I understand correctly, you want to create nested ressources.
If you are using Viewsets, then the ExtendedRouter class of the drf-extensions package will allow you to achieve this.
Drf-extensions documentation about this feature: https://chibisov.github.io/drf-extensions/docs/#nested-routes
There is also this module, who also offer the same features.
You can either use url params or query params to solve your issue. I will explain the URL params solution here,
serializers.py
#Write a Goal Serializer
urls.py
#change the URL according to your environment
url(r'^users/(?P<uid>[0-9]+)/transaction/(?P<tid>[0-9]+)/goal/$', GoalViewSet.as_view({'get': 'user_transaction_goal',}), name='user-transaction-goal'),
views.py
class GoalViewSet(viewsets.ModelViewSet):
permission_classes = [IsAuthenticated]
queryset = Goal.objects.all()
def user_transaction_goal(self, request, uid, tid):
#assuming user is FK in transaction and transaction is a FK in goal
#modify the filter rule according to your model design
goals = Goal.objects.filter(transaction=tid, transaction__user=uid)
serializer = GoalSerializer(goals, many=False)
return Response(serializer.data)
As #clement mentioned you can also use plugins to handle this situation.
update I have now figured that there is a reason to define get_prep_value() and that doing so improves Django's use of the field. I have also been able to get rid of the wrapper class. All this has, finally, enabled me to also eliminate the __getattribute__ implementation with the data model, which was annoying. So, apart from Django callingto_python()` super often, I'm now fine as far as I can see. /update
One morning, you wake up and find yourself using Django 1.4.2 along with DjangoRESTFramework 2.1.2 on Python 2.6.8. And hey, things could definitely be worse. This Django admin magic provides you with forms for your easily specified relational data model, making it a pleasure to maintain the editorial part of your database. Your business logic behind the RESTful URLs accesses both the editorial data and specific database tables for their needs, and even those are displayed in the Django admin, partially because it's easily done and nice to have, partially because some automatically generated records require a mini workflow.
But wait. You still haven't implemented those binary fields as BINARY. They're VARCHARS. You had put that on your ToDo list for later. And later is now.
Okay, there are those write-once-read-many-times cases with small table sizes where an optimization would not necessarily pay. But in another case, you're wasting both storage and performance due to freuquent INSERTs and DELETEs in a table which will get large.
So what would you want to have? A clear mapping between the DB and Django, where the DB stores BINARY and Django deals with hex strings of twice the length. Can't be that hard to achieve, can it?
You search the Web and find folks who want CHAR instead for VARCHAR, others who want BLOBs, and everybody seems to do it a bit differently. Finally, you end up at Writing custom model fields where the VARCHAR -> CHAR case is officially dealt with. So you decide to go with this information.
Starting with __init__(), db_type() and to_python(), you notice that to_python() gets rarely called and add __metaclass__ = models.SubfieldBase only to figure that Django now calls to_python() even if it has done so before. The other suggestions on the page suddenly start to make more sense to you, so you're going to wrap your data in a class, such that you can protect it from repeated calls to to_python(). You also follow the suggestion to Put a __str__() or __unicode__() method on the class you're wrapping up as a field and implement get_prep_value().
While the resulting code does not do what you expect, one thing you notice is that get_prep_value() never gets called so far, so you're removing it for now. What you do figure is that Django consistently appears to get a str from the DB and a unicode from the admin, which is cool, and end up with something like this (boiled down to essentials, really).
class MyHexWrappeer(object):
def __init__(self, hexstr):
self.hexstr = hexstr
def __len__(self):
return len(self.hexstr)
def __str__(self):
return self.hexstr
class MyHexField(models.CharField):
__metaclass__ = models.SubfieldBase
def __init__(self, max_length, *args, **kwargs):
assert(max_length % 2 == 0)
self.max_length = max_length
super(MyHexField, self).__init__(max_length=max_length, *args, **kwargs)
def db_type(self, connection):
return 'binary(%s)' % (self.max_length // 2)
def to_python(self, data):
if isinstance(data, MyHexWrapper): # protect object
return data
if isinstance(data, str): # binary string from DB side
return MyHexWrapper(binascii.b2a_hex(data))
if isinstance(data, unicode): # unicode hex string from admin
return MyHexWrapper(data)
And... it won't work. The reason, of course, being that while you have found a reliable way to create MyHexWrapper objects from all sources including Django itself, the path backwards is clearly missing. From the remark above, you were thinking that Django calls str() or unicode() for admin and get_prep_value() in the direction of the DB. But if you add get_prep_value() above, it will never be called, and there you are, stuck.
That can't be, right? So you're not willing to give up easily. And suddenly you get this one nasty thought, and you're making a test, and it works. And you don't know whether you should laugh or cry.
So now you try this modification, and, believe it or not, it just works.
class MyHexWrapper(object):
def __init__(self, hexstr):
self.hexstr = hexstr
def __len__(self):
return len(self.hexstr)
def __str__(self): # called on its way to the DB
return binascii.a2b_hex(self.hexstr)
def __unicode__(self): # called on its way to the admin
return self.hexstr
It just works? Well, if you use such a field in code, like for a RESTful URL, then you'll have to make sure you have the right kind of string; that's a matter of discipline.
But then, it still only works most of the time. Because when you make such a field your primary key, then Django will call quote(getattr()) and while I found a source claiming that getattr() "nowdays" will use unicode() I can't confirm. But that's not a serious obstacle once you got this far, eh?
class MyModel((models.Model):
myhex = MyHexField(max_length=32,primary_key=True,editable=False)
# other fields
def __getattribute__(self, name):
if (name == 'myhex'):
return unicode(super(MyModel, self).__getattribute__(name))
return super(MyModel, self).__getattribute__(name)
Works like a charm. However, now you lean back and look at your solution as a whole. And you can't help to figure that it's a diversion from the documentation you referred to, that it uses undocumented or internal behavioural characteristics which you did not intend to, and that it is error-prone and shows poor usability for the developer due to the somewhat distributed nature of what you have to implement and obey.
So how can the objective be achieved in a cleaner way? Is there another level with hooks and magic in Django where this mapping should be located?
Thank you for your time.
I have a problems thats been bugging me for a while. I want to use dates in django admin to view entries between certain dates.
To do this I have customized my changelist.html for this model and put a form in there. When posted I override the queryset method like this
def queryset(self, request):
qs = super(ModelAdmin, self).queryset(request)
if request.POST.has_key('date1'):
return qs.filter(startdate__gte=request.POST['date1']).filter(startdate__lte=request.POST['date2'])
return qs
This works great but its only one little problem. The parameters are forgotten if I for example choose to sort the result in any way.
If I instead of this type in the url straight into the browser so it looks like this
http//localhost/admin/some/model/?startdate__gte=2010-01-01&startdate__lte=2010-12-30
I can sort however I want to afterwards because they´ll stick just like this
http//localhost/admin/some/model/?o=5&ot=asc&startdate__lte=2010-12-30&startdate__gte=2010-01-01
Do I need to use a filterspec to solve this?
Thanks heaps!
There is a change request over at the Django project asking for this functionality.
It's waiting for someone to write tests for the proposed patch before it's committed, so you could either do that or you could download the proposed patch (near the bottom of the page) and use it.
https://code.djangoproject.com/ticket/6903