I'm super new to django and the rest API framework. I have a project that I am working on using both and vueJS for the front end. I need to serialize some data for a chart.
For one of the API end points I am trying to group the data like so:
"day_of_the_week": {
"9am":[{"job":".."}],
"10am":[{"job":"..."}],
"11am": [{"job": ".."}],
...
}
I am using a Job class, for reference this is how the jobs end point looks like: jobs-api
So instead of what i have on the picture I am creating a new endpoint where i will only show one object that contains the data for any given day. On the front end there is a chart with filters that let the user filter the jobs by the day they request. On load, when the user has given no day of the week, the end point will return the object of 'today'.
Because I am new to this, I have no idea where to do this, my initial thought was to filter on the views.py, but for now I have done it in the serializer which gives me the error "Object of type Job is not JSON serializable".
This is how the serializer looks like:
jobs-by-day-serializer
Clearly, there is something that I am not quite grasping, so any help would be appreciated.
EDIT:
this is my views.py now, I have added the filter for the queryset to filter by day, so I can filter by day now:
jobs_by_day_viewset
I think that the correct way of doing it would be to implement the model serializer and then define the views.
A good reason for implementing different serializers is to return different result for the same model. But in your case you just need an specific subset of Job so I think the view is a better choice.
I assume you already implement JobSerializer, and probably you have a JobViewSet to L/C/U/M over the model.
So, now you just need to implement a new view for these task. Something like this could work,
from jobs.models import Job
from jobs.serializers import JobSerializer
from rest_framework import generics
class JobByDayList(generics.ListCreateAPIView):
serializer_class = JobSerializer
def get_queryset(self):
# get the day from parameters
day_query = self.kwargs['day_query']
# here your logic to get desire queryset
return queryset
The answers given provides a solution for one part of the question.
I managed to solve the second part. Once I could filter my queryset by day of the week I still had to group every object by the hour in which it had been started.
This is how i solved it:
def get(self, request):
params = self.get_params()
queryset = Job.objects.filter(
dt_start__week_day=params['day'], dt_start__gte=params['dt_start'], dt_end__lte=params['dt_end'])
queryset_started = queryset.annotate(hour=ExtractHour('dt_start'))
queryset_ended = queryset.annotate(hour=ExtractHour('dt_end'))
started = []
ended = []
total_jobs =[]
for n in range(0, 23):
queryset_end = queryset_ended.filter(hour=n)
ended.append(round(queryset_end.count() / queryset.count() * 100, 2))
queryset3 = queryset_started.filter(hour=n)
started.append(round(queryset3.count() / queryset.count() * 100, 2))
for x in range(len(started)):
total_jobs.append(round(started[x]+ended[x], 2))
I used a viewset that extended from APIview instead of the modelviewsets so that i was able to return an array of integers.
Related
When using DjangoListObjectType from graphene_django_extras, I can define a custom qs property on a SearchFilter.
The qs function has the object as its only argument, and through that I can get the request, and in turn the query string which includes the queried fields.
Before hacking together my own parser to get these fields, am I going about this the wrong way? Or is there something else out there?
The idea is to have quite a rigid approach, as we have 7 types of paginated list types with fields that result in a few unnecessary database hits, so we want to prefetch a few fields.
Graphene has a dataloader approach which kind of looks right, but more complicated than just prefetching them at the qs stage.
Re the dataloader approach, I tried something like this
class UserLoader(DataLoader):
def batch_load_fn(self, keys):
users = {user.id: user for user in User.objects.filter(id__in=keys)}
return Promise.resolve([users.get(user_id) for user_id in keys])
user_loader = UserLoader()
class BookType(DjangoObjectType):
...
author = graphene.Field(UserType)
def resolve_author(self, *args, **kwargs):
return user_loader.load(self.author.id)
Which kind of obviously feels completely wrong, but is what they do in the docs.
Using a DataLoader is the correct approach but do self.author_id instead of self.author.id.
By calling self.author.id you're fetching the author for each book (one SQL query per author) then getting the id attribute from each author.
When you add the author ForeignKey to your Book model, behind the scenes, Django adds a column to the books table—author_id. You can pass this value to your DataLoader and have it fetch all the corresponding Authors in a single SQL query using a SELECT IN.
https://docs.djangoproject.com/en/3.2/ref/models/fields/#database-representation
You'll notice that the Graphene Docs are doing the same thing with best_friend_id in their example.
There are hundreds of questions here on various django annotate/aggregate constructions and filters but I couldn't find this simple use-case asked (or answered).
I have a "Payments" model and associated ListAPIView ViewSet endpoint with nicely setup DjangoFilters so the client can filter on created__lte, created__gte, company= etc.
Now I want to add an endpoint that derives from the above but only returns the sum of some fields and count of the total filtered objects.
I know exactly how to do this if I would just write the View "from scratch" (I can just hack the DRF View into executing get_queryset().aggregate() then feeding into a serializer and returning), but I want to do it in a "Django-way" if possible.
For example, combined with a serializer that defines "total_amount" and "nbr", this (almost) works:
queryset = models.Payment.objects.values('company').annotate(total_amount=Sum('amount'),
nbr=Count('id'))
The values() call groups by "company" (a sub-field of Payment), which in combination with annotate() performs a sum by all company payments and annotates with total_amount/nbr. Adding filtering query parameters magically adjusts what goes into the annotation properly.
The problem is, what if I don't want to group (or even filter) by "company", I just want to "group by all"? Is there a way to do that?
I realize this is already a bit magical but the Django-esque way of doing grouping for annotation is this as far as I know.
I also realize I'm probably really better off just by hijacking .retrieve() to evaluate the queryset with .aggregate() tacked at the end and on-the-fly creating the response... still curious though :)
I ended up with the "hack", overriding list() in the View with an .aggregate call and the packing it for the Response through a serializer. This was the most canonical way I could figure out (I mean, re-using as many of the moving parts of Django/DRF as possible like automatic filtering of the queryset, serializing etc).
Bonus: note the Coalesce() wrap which is needed because Sum() doesn't return 0 if the set is empty.
def list(self, request, *args, **kwargs):
queryset = self.filter_queryset(self.get_queryset())
stats = queryset.aggregate(total_amount=Coalesce(Sum('amount'), 0),
total_vat=Coalesce(Sum('vat'), 0),
nbr_payments=Count('id'))
# .aggregate() returns a dict of the results, not a QuerySet. Wrap it
# into a response through the serializer.
sclass = self.get_serializer_class()
return Response(sclass(stats).data)
For my project I started to use Laravel for the api than I switched to Django / django rest framework, I did this to gain more speed as I need to query large data.
Now I got the following situation:
I got a "Group" which has "Subjects" and which has a recursive relation.
Now a group can have like 2000+ subjects(including the descendent subjects) a parent subject has +/- 30 subjects.
This is my code:
serializers
class RecursiveField(serializers.Serializer):
def to_representation(self, value):
serializer = self.parent.parent.__class__(value, context=self.context)
return serializer.data
class SubjectSerializer(serializers.ModelSerializer):
parent_of = RecursiveField(many=True, read_only=True)
class Meta:
model = Subject
fields = ("id", "name", "parent_of", "parent")
class GroupSerializer(serializers.ModelSerializer):
subjects = SubjectSerializer(many=True, read_only=True)
class Meta:
model = Group
fields = ("id", "name", "subjects")
def setup_eager_loading(cls, queryset):
return queryset.prefetch_related("subjects")
views
class GroupViewSet(ModelViewSet):
class Paginator(BasePaginator):
model = Group
queryset = Group.objects.all()
serializer_class = serializers.GroupSerializer
pagination_class = Paginator
def get_queryset(self):
return self.get_serializer().setup_eager_loading(GroupViewSet.queryset)
I tested the same request with the laravel api and its much faster, still noticeable slow but its ok(5-10 secs). With django rest framework its too slow (1 minute +/-), and thats just a page with 1 group that has 2500 subjects.
I do know what takes long, the RecursiveField class, because when I remove that the query is done in less than 2 seconds. So my question is whats the main cause, because it's creates a recursive relation (I doubt)? Or is it because I don't prefetch?
And ofcourse whats the best way to do this?
Thank you
You have a few options, but I don't think any are great. Recursive queries aren't very well supported with Django.
Rework your data model to prevent needing to use recursion to fetch the subjects from the database. You could add a root ForeignKey to Subject on Subject that would identify the root subject. This would allow you to grab all subjects in a tree fairly easily. Then you'd have to arrange them in your View/Viewset to fit the ordering (if that's necessary).
Use raw() and your database's recursive functionality to fetch the models. This would require raw SQL and can be painful to maintain.
Use django_cte. I've used this in one of my projects for a few queries, but I'm not a big fan of it. It breaks some functionality with update() being called on an empty queryset. However, it will work and it won't require you to drop down to raw SQL.
The problem is not DRF, but the data structure itself.
It is very slow in django to query all ancestors/descendants recursively, your should use a more efficient data structure.
For the same reason I wrote django-treenode, it performs tree operations without query the db.
You can read the docs here: https://github.com/fabiocaccamo/django-treenode
I am creating an app with a rest API that should return values for instances of objects based on the url given. Right now I have the API working using ModelViewSets of my objects for the API.
For example I have three objects, user, transactions, and goals.
As it stands I can go to /mysite/api/users and return a list of all users
I can also go to /mysite/api/users/1 to return just the user with the id '1'.
I can do something similar with transactions and goals.
What I'm looking to do is go to url /mysite/api/users/1/transaction/1/goal
to find the goal associated with the transaction for that user.
I've been scouring tutorials and am not sure what the right question is to ask in order to find something useful to learn how to do this. What is the correct way to go about setting up my rest api like this?
If I understand correctly, you want to create nested ressources.
If you are using Viewsets, then the ExtendedRouter class of the drf-extensions package will allow you to achieve this.
Drf-extensions documentation about this feature: https://chibisov.github.io/drf-extensions/docs/#nested-routes
There is also this module, who also offer the same features.
You can either use url params or query params to solve your issue. I will explain the URL params solution here,
serializers.py
#Write a Goal Serializer
urls.py
#change the URL according to your environment
url(r'^users/(?P<uid>[0-9]+)/transaction/(?P<tid>[0-9]+)/goal/$', GoalViewSet.as_view({'get': 'user_transaction_goal',}), name='user-transaction-goal'),
views.py
class GoalViewSet(viewsets.ModelViewSet):
permission_classes = [IsAuthenticated]
queryset = Goal.objects.all()
def user_transaction_goal(self, request, uid, tid):
#assuming user is FK in transaction and transaction is a FK in goal
#modify the filter rule according to your model design
goals = Goal.objects.filter(transaction=tid, transaction__user=uid)
serializer = GoalSerializer(goals, many=False)
return Response(serializer.data)
As #clement mentioned you can also use plugins to handle this situation.
I am trying to do a survey application in django. My model is as follows:
class mymodel(models.Model):
resptype = models.ForeignKey(Response)
ques = models.ForeignKey(Question)
response = models.CharField(max_length=5, blank=True)
Here i am using rest framework to send data to my front end. Right now i have my api defined as follows:
class mymodelList(APIView):
def get(self, request, format=None):
surveydata = mymodel.objects.all()
serialized_surveydata = mymodelSerializer(surveydata, many=True)
return Response(serialized_surveydata.data)
In my app, I have a standard set of 16 questions with multiple choice responses and the choice is saved in the response column in the model.
Now what I am trying to achieve is to calculate the count of responses for each question . ie. For question 1, what is the count that a person responded with 1 or 2 or etc.
Also i would like to know how to send the calculated counts through another json field from the rest framework because I don't have any model defined separately for this data.
EDIT:
This command did the trick for my query but i still not able to figure out how to send it to the front end as a serialized object.
x = mymodel.objects.values('ques','response').order_by().annotate(number_of_responses=Count('response'))
That's not really a great structure for your model, it would probably be easier to create separate Question and Choice classes. The Django tutorial actually uses this type of application as an example... take a look at that for some guidance
Check #detail_route or #list_route from viewsets depending on if you want to show this info per question o for all questions at once.
This will allow you to define a custom endpoint to request the information you are asking for. To do so, you may also need to define a custom serializer to pass extra data or a filter if you want to filter by question, user, etc.