TL;DR: DRF drops the inner serialized object when validating the outermost serializer.
I'm using django 2.0, django-rest-framework 3.7.7, python 3.
I want to build a REST endpoint that performs a search in the database, using some parameters received in a POST (I want to avoid GET calls, which could be cached). The parameters should act like ORs (that's why I set all fields as not required), and I'm solving that using django Q queries when extracting the queryset.
I have the following django models in app/models.py:
class Town(models.Model):
name = models.CharField(max_length=200)
province = models.CharField(max_length=2, blank=True, null=True)
zip = models.CharField(max_length=5)
country = models.CharField(max_length=100)
class Person(models.Model):
name = models.CharField(max_length=100)
birth_place = models.ForeignKey(Town, on_delete=models.SET_NULL,
null=True, blank=True,
related_name="birth_place_rev")
residence = models.ForeignKey(Town, on_delete=models.SET_NULL,
null=True, blank=True,
related_name="residence_rev")
And I wrote the following serializers in app/serializers.py:
class TownSerializer(serializers.ModelSerializer):
class Meta:
model = models.Town
fields = ("id", "name", "province", "zip", "country")
def __init__(self, *args, **kwargs):
super(TownSerializer, self).__init__(*args, **kwargs)
for field in self.fields:
self.fields[field].required = False
class PersonSerializer(serializers.ModelSerializer):
birth_place = TownSerializer(read_only=True)
residence = TownSerializer(read_only=True)
class Meta:
model = models.Person
fields = ("id", "name", "birth_place", "residence")
def __init__(self, *args, **kwargs):
super(PersonSerializer, self).__init__(*args, **kwargs)
for field in self.fields:
self.fields[field].required = False
Then I wrote a view to provide the REST interface, in api/views.py:
class PersonSearchList(views.APIView):
model_class = Person
serializer_class = PersonSerializer
permission_classes = (permissions.AllowAny,)
def post(self, request, format=None):
serializer = self.serializer_class(data=request.data)
print("initial_data", serializer.initial_data) ########
if serializer.is_valid():
self.data = serializer.validated_data
print(self.data) ########
queryset = self.get_queryset()
serialized_objects = self.serializer_class(queryset, many=True)
return Response(serialized_objects.data, status=status.HTTP_201_CREATED)
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
def get_queryset(self, *args, **kwargs):
orig_queryset = self.model_class.objects.all()
query_payload = self.data
# .. perform filtering using the query_payload data.
return queryset
And when I try to perform a query using e.g. curl:
$ curl -s -X POST -H "Content-Type: application/json" --data '{"birth_place": {"name": "Berlin", "country": "Germany"}}' http://127.0.0.1:8000/persons/ |python -m json.tool
[]
even though a Person object with birth_place set accordingly has just been created.
The two print statements that I placed in the post method of the view return:
initial_data: {'birth_place': {'name': 'Berlin', 'country': 'Germany'}}
after is_valid: OrderedDict()
So it seems like DRF drops the nested relation when validates.
How should I specify to parse and validate also the nested relation? Any suggestion is appreciated.
PS: Am I forcing a wrong design by making the request with a POST? I thought that since a search is not idempotent, and it may contain sensitive data (name, surname, birth date, etc) of a person.
I need an action which is safe (a search doesn't change the data) but not idempotent (a search in two different times can be different).
Initially I started using a generics.ListAPIView, but list() works only with GET. If there's a way to make it accept POST requests it would work like a charm.
As #Jon Clements♦ mentioned in comments, this would solve your problem
class PersonSerializer(serializers.ModelSerializer):
birth_place = TownSerializer()
residence = TownSerializer()
class Meta:
model = Person
fields = ("id", "name", "birth_place", "residence")
def __init__(self, *args, **kwargs):
super(PersonSerializer, self).__init__(*args, **kwargs)
for field in self.fields:
self.fields[field].required = False
Related
My problem is that when trying to run is_valid() on a big chunk of data in a POST-request, where the model has a foreign key, it will fetch the foreign key table for each incoming item it needs to validate.
This thread describes this as well but ended up finding no answer:
Django REST Framework Serialization POST is slow
This is what the debug toolbar shows:
My question is therefore, is there any way to run some kind of select_related on the validation? I've tried turning off validation but the toolbar still tells me that queries are being made.
These are my models:
class ActiveApartment(models.Model):
adress = models.CharField(default="", max_length=2000, primary_key=True)
company = models.ForeignKey(Company, on_delete=models.CASCADE)
class Company(models.Model):
name = models.CharField(blank=True, null=True, max_length=150)
These are my serializers:
I have tried not using the explicit PrimaryKeyRelatedField as well, having validators as [] doesn't seem to stop the validation either for some reason.
class ActiveApartmentSerializer(serializers.ModelSerializer):
company = serializers.PrimaryKeyRelatedField(queryset=Company.objects.all())
class Meta:
model = ActiveApartment
list_serializer_class = ActiveApartmentListSerializer
fields = '__all__'
extra_kwargs = {
'company': {'validators': []},
}
class ActiveApartmentListSerializer(serializers.ListSerializer):
def create(self, validated_data):
data = [ActiveApartment(**item) for item in validated_data]
# Ignore conflcits is the only "original" part of this create method
return ActiveApartment.objects.bulk_create(data, ignore_conflicts=True)
def update(self, instance, validated_data):
pass
This is my view:
def post(self, request, format=None):
# Example of incoming data in request.data
dummydata = [{"company": 12, "adress": "randomdata1"}, {"company": 12, "adress": "randomdata2"}]
serializer = ActiveApartmentSerializer(data=request.data, many=True)
# This will run a query to check the validity of their foreign keys for each item in dummydata
if new_apartment_serializer.is_valid():
print("valid")
Any help would be appreciated (I would prefer not to use viewsets)
Have you tried to define company as IntegerField in the serializer, pass in the view's context the company IDs and add a validation method in field level?
class ActiveApartmentSerializer(serializers.ModelSerializer):
company = serializers.IntegerField(required=True)
...
def __init__(self, *args, **kwargs):
self.company_ids = kwargs.pop('company_ids', None)
super().__init__(*args, **kwargs)
def validate_company(self, company):
if company not in self.company_ids:
raise serializers.ValidationError('...')
return company
The way I solved it was I took the direction athanasp pointed me in and tweaked it a bit as his solution was close but not working entirely.
I:
Created a simple nested serializer
Used the init-method to make the query
Each item checks the list of companies in its own validate method
class CitySerializer(serializers.ModelSerializer):
class Meta:
model = City
fields = ('name',)
class ListingSerializer(serializers.ModelSerializer):
# city = serializers.IntegerField(required=True, source="city_id") other alternative
city = CitySerializer()
def __init__(self, *args, **kwargs):
self.cities = City.objects.all()
super().__init__(*args, **kwargs)
def validate_city(self, value):
try:
city = next(item for item in self.cities if item.name == value['name'])
except StopIteration:
raise ValidationError('No city')
return city
And this is how the data looks like that should be added:
city":{"name":"stockhsdolm"}
Note that this method more or less works using the IntegerField() suggested by athan as well, the difference is I wanted to use a string when I post the item and not the primary key (e.g 1) which is used by default.
By using something like
city = serializers.IntegerField(required=True, source="city_id") works as well, just tweak the validate method, e.g fetching all the Id's from the model using values_list('id', flat=True) and then simply checking that list for each item.
I am trying to joint two models in django-rest-framework.
My code isn't throwing any error but also it isn't showing other model fields that need to be joined.
Below is my code snippet:
Serializer:
class CompaniesSerializer(serializers.ModelSerializer):
class Meta:
model = Companies
fields = ('id', 'title', 'category')
class JobhistorySerializer(serializers.ModelSerializer):
companies = CompaniesSerializer(many=True,read_only=True)
class Meta:
model = Jobhistory
fields = ('id', 'title', 'company_id', 'companies')
View .
class UserJobs(generics.ListAPIView):
serializer_class = JobhistorySerializer()
def get_queryset(self):
user_id = self.kwargs['user_id']
data = Jobhistory.objects.filter(user_id=user_id)
return data
model:
class Companies(models.Model):
id = models.AutoField(primary_key=True)
title = models.CharField(max_length=100, blank=True, default='')
category = models.CharField(max_length=30, blank=True, default='')
created = models.DateTimeField(auto_now_add=True)
class Meta:
ordering = ('created',)
def save(self, *args, **kwargs):
title = self.title or False
category = self.category or False
super(Companies, self).save(*args, **kwargs)
class Jobhistory(models.Model):
id = models.AutoField(primary_key=True)
company_id = models.ForeignKey(Companies)
title = models.CharField(max_length=100, blank=True, default='')
created = models.DateTimeField(auto_now_add=True)
class Meta:
ordering = ('created',)
def save(self, *args, **kwargs):
company_id = self.company_id or False
title = self.title or False
super(Jobhistory, self).save(*args, **kwargs)
Thanks in advance. Any help will be appreciated.
In your views, you have
serializer_class = JobHistorySerializer()
Remove the parenthesis from this.
The reason for this is apparent in the GenericAPIView, specifically the get_serializer() and get_serializer_class() methods:
def get_serializer(self, *args, **kwargs):
"""
Return the serializer instance that should be used for validating and
deserializing input, and for serializing output.
"""
serializer_class = self.get_serializer_class()
kwargs['context'] = self.get_serializer_context()
return serializer_class(*args, **kwargs)
def get_serializer_class(self):
"""
Return the class to use for the serializer.
Defaults to using `self.serializer_class`.
You may want to override this if you need to provide different
serializations depending on the incoming request.
(Eg. admins get full serialization, others get basic serialization)
"""
assert self.serializer_class is not None, (
"'%s' should either include a `serializer_class` attribute, "
"or override the `get_serializer_class()` method."
% self.__class__.__name__
)
return self.serializer_class
As you can see in get_serializer, it initializes that serializer class with args and kwargs that aren't provided in your view code.
I'm trying to create a new object using Django REST Framework's ModelViewSet and the ModelSerializer and associate an existing object to it's ForeignKey field. I have two models with a one-to-many relationship, basically a manufacturer and a product. The manufacturer already exists in the database and I'm trying to add a product individually and assign it's manufacturer at the same time. When I do, DRF is giving me this error:
"non_field_errors": [
"Invalid data. Expected a dictionary, but got Product."
]
Here are what my models look like (I've tried to take out the unneeded code to keep it small). When a product is created the manufacturer field is an integer representing the ID of it's related object. As you can see in my view, I query for the Manufacturer object, set it and then continue to let DRF create the object. If someone can please tell me where I'm going wrong, I'd greatly appreciate it!
models.py
class Manufacturer(models.Model):
name = models.CharField(max_length=64)
state = models.CharField(max_length=2)
class Product(models.Model):
manufacturer = models.ForeignKey(Manufacturer, related_name="products")
name = models.CharField(max_length=64)
sku = models.CharField(max_length=12)
views.py
class ProductViewSet(view sets.ModelViewSet):
model = Product
permission_classes = [IsAuthenticated]
def get_queryset(self):
# get logged in user's manufacturer
return Product.objects.filter(manufacturer=manufacturer)
def create(self, request, *args, **kwargs):
if "manufacturer" in request.data:
manufacturer = Manufacturer.objects.get(pk=request.data["manufacturer"])
request.data["manufacturer"] = manufacturer
return super(ProductViewSet, self).create(request, *args, **kwargs)
serializers.py
class ManufacturerSerializer(serializers.ModelSerializer):
class Meta:
model = Manufacturer
fields = ("id", "name", "state",)
read_only_fields = ("id",)
class ProductSerializer(serializers.ModelSerializer):
manufacturer = ManufacturerSerializer()
class Meta:
model = Product
fields = ("id", "manufacturer", "name",)
read_only_fields = ("id",)
you have to set manufacturer_id key.
change your serializer to:
class ProductSerializer(serializers.ModelSerializer):
manufacturer = ManufacturerSerializer(read_only=True)
manufacturer_id = serializers.IntegerField(write_only=True)
class Meta:
model = Product
fields = ("id", "manufacturer", "name",)
read_only_fields = ("id",)
change your create method to:
def create(self, request, *args, **kwargs):
if "manufacturer" in request.data:
request.data["manufacturer_id"] = request.data['manufacturer']
return super(ProductViewSet, self).create(request, *args, **kwargs)
I have a Branch model with a foreign key to account (the owner of the branch):
class Branch(SafeDeleteModel):
_safedelete_policy = SOFT_DELETE_CASCADE
name = models.CharField(max_length=100)
account = models.ForeignKey(Account, null=True, on_delete=models.CASCADE)
location = models.TextField()
phone = models.CharField(max_length=20, blank=True,
null=True, default=None)
create_at = models.DateTimeField(auto_now_add=True, null=True)
update_at = models.DateTimeField(auto_now=True, null=True)
def __str__(self):
return self.name
class Meta:
unique_together = (('name','account'),)
...
I have a Account model with a foreign key to user (one to one field):
class Account(models.Model):
_safedelete_policy = SOFT_DELETE_CASCADE
name = models.CharField(max_length=100)
user = models.OneToOneField(User)
create_at = models.DateTimeField(auto_now_add=True)
update_at = models.DateTimeField(auto_now=True)
def __str__(self):
return self.name + ' - ' + self.create_at.strftime('%Y-%m-%d %H:%M:%S')
I've created a ModelViewSet for Branch which shows the branch owned by the logged in user:
class BranchViewSet(viewsets.ModelViewSet):
serializer_class = BranchSerializer
permission_classes = (permissions.IsAuthenticated,)
def get_queryset(self):
queryset = Branch.objects.all().filter(account=self.request.user.account)
return queryset
Now to create a new branch, I want to save account field with request.user.account, not with data sent from the rest client (for more security). for example:
def create(self, request, *args, **kwargs):
if request.user.user_type == User.ADMIN:
request.data['account'] = request.user.account
return super(BranchViewSet, self).create(request, *args, **kwargs)
def perform_create(self, serializer):
'''
Associate branch with account
'''
serializer.save(account=self.request.user.account)
In branch serializer
class BranchSerializer(serializers.ModelSerializer):
account = serializers.CharField(source='account.id', read_only=True)
class Meta:
model = Branch
fields = ('id', 'name', 'branch_alias',
'location', 'phone', 'account')
validators = [
UniqueTogetherValidator(
queryset=Branch.objects.all(),
fields=('name', 'account')
)
]
but I got this error:
This QueryDict instance is immutable. (means request.data is a immutable QueryDict and can't be changed)
Do you know any better way to add additional fields when creating an object with django rest framework?
As you can see in the Django documentation:
The QueryDicts at request.POST and request.GET will be immutable when accessed in a normal request/response cycle.
so you can use the recommendation from the same documentation:
To get a mutable version you need to use QueryDict.copy()
or ... use a little trick, for example, if you need to keep a reference to an object for some reason or leave the object the same:
# remember old state
_mutable = data._mutable
# set to mutable
data._mutable = True
# сhange the values you want
data['param_name'] = 'new value'
# set mutable flag back
data._mutable = _mutable
where data it is your QueryDicts
Do Simple:
#views.py
from rest_framework import generics
class Login(generics.CreateAPIView):
serializer_class = MySerializerClass
def create(self, request, *args, **kwargs):
request.data._mutable = True
request.data['username'] = "example#mail.com"
request.data._mutable = False
#serializes.py
from rest_framework import serializers
class MySerializerClass(serializers.Serializer):
username = serializers.CharField(required=False)
password = serializers.CharField(required=False)
class Meta:
fields = ('username', 'password')
request.data._mutable=True
Make mutable true to enable editing in querydict or the request.
I personally think it would be more elegant to write code like this.
def create(self, request, *args, **kwargs):
data = OrderedDict()
data.update(request.data)
data['account'] = request.user.account
serializer = self.get_serializer(data)
serializer.is_valid(raise_exception=True)
self.perform_create(serializer)
headers = self.get_success_headers(serializer.data)
return Response(serializer.data, status=status.HTTP_201_CREATED, headers=headers)
Do you know any better way to add additional fields when creating an object with django rest framework?
The official way to provide extra data when creating/updating an object is to pass them to the serializer.save() as shown here
https://docs.djangoproject.com/en/2.0/ref/request-response/#querydict-objects
The QueryDicts at request.POST and request.GET will be immutable when accessed in a normal request/response cycle. To get a mutable version you need to use QueryDict.copy().
You can use request=request.copy() at the first line of your function.
I'm trying to override concept queryset in my child form, to get a custom list concepts based on the area got from request.POST, here is my list of concepts, which i need to filter based on the POST request, this lists is a fk of my child form (InvoiceDetail). is it possible to have these filters?
after doing some test when I pass the initial data as the documentation says initial=['concept'=queryset_as_dict], it always returns all the concepts, but i print the same in the view and its ok the filter, but is not ok when i render in template, so I was reading that I need to use some BaseInlineFormset. so when I test I obtained different errors:
django.core.exceptions.ValidationError: ['ManagementForm data is missing or has been tampered with']
'InvoiceDetailFormFormSet' object has no attribute 'fields'
so here is my code:
models.py
class ConceptDetail(CreateUpdateMixin): # here, is custom list if area='default' only returns 10 rows.
name = models.CharField(max_length=150)
area = models.ForeignKey('procedure.Area')
class Invoice(ClusterableModel, CreateUpdateMixin): # parentForm
invoice = models.SlugField(max_length=15)
class InvoiceDetail(CreateUpdateMixin): # childForm
tax = models.FloatField()
concept = models.ForeignKey(ConceptDetail, null=True, blank=True) # fk to override using custom queryset
invoice = models.ForeignKey('Invoice', null=True, blank=True)
views.py
class CreateInvoiceProcedureView(LoginRequiredMixin, PermissionRequiredMixin, CreateView):
template_name = 'invoice/invoice_form.html'
model = Invoice
permission_required = 'invoice.can_check_invoice'
def post(self, request, *args, **kwargs):
self.object = None
form = InvoiceForm(request=request)
# initial initial=[{'tax': 16, }] removed
invoice_detail_form = InvoiceDetailFormSet(request.POST, instance=Invoice,
request=request)
return self.render_to_response(
self.get_context_data(
form=form,
invoice_detail_form=invoice_detail_form
)
)
forms.py
class BaseFormSetInvoice(BaseInlineFormSet):
def __init__(self, *args, **kwargs):
# call first to retrieve kwargs values, when the class is instantiated
self.request = kwargs.pop("request")
super(BaseFormSetInvoice, self).__init__(*args, **kwargs)
self.queryset.concept = ConceptDetail.objects.filter(
Q(area__name=self.request.POST.get('area')) | Q(area__name='default')
)
class InvoiceForm(forms.ModelForm):
class Meta:
model = Invoice
fields = ('invoice',)
class InvoiceDetailForm(forms.ModelForm):
class Meta:
model = InvoiceDetail
fields = ('concept',)
InvoiceDetailFormSet = inlineformset_factory(Invoice, InvoiceDetail,
formset=BaseFormSetInvoice,
form=InvoiceDetailForm,
extra=1)
How can i fix it?, what do i need to read to solve this problem, I tried to debug the process, i didn't find answers.
i try to do this:
def FooForm(forms.ModelForm):
def __init__(self, *args, **kwargs):
super(FooForm, self).__init__(*args, **kwargs)
self.fields['concept'].queryset = ConceptDetail.objects.filter(area__name='default')
In a inlineformset_factory how can do it?.
After a lot of tests, my solution is override the formset before to rendering, using get_context_data.
def get_context_data(self, **kwargs):
context = super(CreateInvoiceProcedureView, self).get_context_data(**kwargs)
for form in context['invoice_detail_form']:
form.fields['concept'].queryset = ConceptDetail.objects.filter(area__name=self.request.POST.get('area'))
return context