Related
I am testing a query parameter in djnago restframework to retrieve the detail of an object. The query works in the browser but not the test. I think I am not calling the response correctly with:
response = self.client.get(reverse('quote-requests:get-calculator-detail', kwargs={'calculator_code': self.calc1.calculator_code}))
the apps urls.py
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('api/v1/quote-requests/', include('quote_requests.urls')),
]
which includes the quotes.urls.py
from django.urls import path, include
from rest_framework.routers import DefaultRouter
from quote_requests import views
router = DefaultRouter()
router.register('get-calculator', views.CalculatorCodeDetailViewSet, basename='get-calculator')
app_name = 'quote-requests'
urlpatterns = [
path('', include(router.urls)),
]
The viewset is:
class CalculatorCodeDetailViewSet(viewsets.ReadOnlyModelViewSet):
serializer_class = CalculatorCodeSerializer
lookup_field = ('calculator_code')
def get_queryset(self):
return (
Calculator.objects.filter(
calculator_code = self.request.query_params.get('calculator_code',)
)
)
The CalculatorCodeSerializer is:
class CalculatorCodeSerializer(serializers.ModelSerializer):
class Meta:
model = Calculator
fields = (
'name',
'calculator_code',
)
The test is:
def test_retrieving_calculator_detail_with_calculator_code(self):
''' test retrieving detail of a calculator '''
self.calc1 = Calculator.objects.create(
name = "Calculator 1000",
calculator_code = "HH1000",
)
response = self.client.get(reverse('quote-requests:get-calculator-detail', kwargs={'calculator_code': self.calc1.calculator_code}))
serializer = CalculatorCodeSerializer(self.calc1)
self.assertEqual(response.data, serializer.data)
This yields the error:
Traceback (most recent call last):
File "/app/quote_requests/tests/test_calculators_api.py", line 149, in test_retrieving_calculator_detail_with_calculator_code
self.assertEqual(response.data, serializer.data)
AssertionError: {'detail': ErrorDetail(string='Not found.',[14 chars]nd')} != {'name': 'Calculator 1000', 'calculator_cod[368 chars].25'}
When checking the browser link:
http://localhost:8000/api/v1/quote-requests/get-calculator/?calculator_code=HH1000
This works but test fails. Any help setting up the properly would be appreciated.
I think you are using the wrong endpoint name replace this "get-calculator-detail" with this "get-calculator"
def test_retrieving_calculator_detail_with_calculator_code(self):
# test retrieving detail of a calculator
self.calc1 = Calculator.objects.create(
name = "Calculator 1000",
calculator_code = "HH1000",
)
response = self.client.get(reverse('quote-requests:get-
calculator', kwargs={'calculator_code':
self.calc1.calculator_code}))
serializer = CalculatorCodeSerializer(self.calc1)
self.assertEqual(response.data, serializer.data)
With the help of others on Fiverr I have gotten the test to pass. I hesitate to call it answered but I am putting the detail here in case those who know a better way can comment.
TLDR - the response variable was not written correctly either the reverse does not do what I want or it cannot be done with reverse. Either way the change that worked was:
response = self.client.get('/api/v1/quote-requests/get-calculator/', {'calculator_code': self.calc1.calculator_code})
This does not return an iterable object. It returns an OrderDict which looks like this:
[{"name":"Calculator 1000","calculator_code":"HH1000","id":7,}]
I used print(response.content) to check this.
so you need to use the json function to pull the data out and match it to serializer.data - the response from the serializer.
response.data = response.json()[0]
this pulls the "[]" off and formats it like this:
{'name': 'Calculator 1000', 'calculator_code': 'HH1000', 'id': 7, }
The [0] on the end gets the object (I think) and returns it in json format.
The two fiverr users that helped me with this were:
https://www.fiverr.com/tailongk
and
https://www.fiverr.com/asipita
I have created a views function in django which returns the data of all influencers. Now I want to add the functionality that a user can make multiple lists and then add those influencers to any of the list created by them. The lists created by the users are displayed in the form of a drop down menu in front of the influencers name. How should I create the API so that the list created by the users are displayed in front of the influencers.
P.S: I want to create the API without using django rest framework.
This is what I have tried till now:
def index(request):
influencers = Influencer.objects.all()
influencer_data = serializers.serialize("json",influencers)
user_list = UserList.objects.all().filter(user_id = request.user.id)
user_list = serializers.serialize("json",user_list)
context = {
'influencer_data':influencer_data,
'user_list':user_list,
}
return HttpResponse(json.dumps(context),content_type='application/json')
I am getting the result as:
{"influencer_data": "[{\"model\": \"influencer_listings.influencer\", \"pk\": 8794, \"fields\": {\"full_name\": \"F A I Z S H A I K H \\ud83c\\udf08\", \"username\": \"mr_faizzz_07\", \"photo\": \"\", \"email_id\": \"\", \"external_url\": \"\
.............................
.............................
"user_list": "[{\"model\": \"user_listings.userlist\", \"pk\": 21, \"fields\": {\"user_id\": 5, \"list_name\": \"Campaign 1\"}}, {\"model\": \"user_listings.userlist\", \"pk\": 22, \"fields\": {\"user_id\": 5, \"list_name\": \"Delhi Campaign\"}}]"}
The return statement makes the JSON object returned a string.I want the data to be returned in the JSON format.
from itertools import chain
from django.http import HttpResponse
from django.core import serializers
def index(request):
user_list = UserList.objects.all().filter(user_id=request.user.id)
influencers = Influencer.objects.all()
queryset = list(chain(user_list, influencers ))
ser_query = serializers.serialize('json', queryset)
return HttpResponse(ser_query)
Do it like this, modify the code to your needs
I have the following:
target_content_type = models.ForeignKey(ContentType, related_name='target_content_type')
target_object_id = models.PositiveIntegerField()
target = generic.GenericForeignKey('target_content_type', 'target_object_id')
I would like dumpdata --natural to emit a natural key for this relation. Is this possible? If not, is there an alternative strategy that would not tie me to target's primary key?
TL;DR - Currently there is no sane way of doing so, short of creating a custom Serializer / Deserializer pair.
The problem with models that have generic relations is that Django doesn't see target as a field at all, only target_content_type and target_object_id, and it tries to serialize and deserialize them individually.
The classes responsible for serializing and deserializing Django models are in the modules django.core.serializers.base and django.core.serializers.python. All the others (xml, json and yaml) extend either of them (and python extends base). The field serialization is done like this (irrelevant lines ommited):
for obj in queryset:
for field in concrete_model._meta.local_fields:
if field.rel is None:
self.handle_field(obj, field)
else:
self.handle_fk_field(obj, field)
Here's the first complication: the foreign key to ContentType is handled ok, with natural keys as we expected. But the PositiveIntegerField is handled by handle_field, that is implemented like this:
def handle_field(self, obj, field):
value = field._get_val_from_obj(obj)
# Protected types (i.e., primitives like None, numbers, dates,
# and Decimals) are passed through as is. All other values are
# converted to string first.
if is_protected_type(value):
self._current[field.name] = value
else:
self._current[field.name] = field.value_to_string(obj)
i.e. the only possibility for customization here (subclassing PositiveIntegerField and defining a custom value_to_string) will have no effect, since the serializer won't call it. Changing the data type of target_object_id to something else than a integer will probably break many other stuff, so it's not an option.
We could define our custom handle_field to emit natural keys in this case, but then comes the second complication: the deserialization is done like this:
for (field_name, field_value) in six.iteritems(d["fields"]):
field = Model._meta.get_field(field_name)
...
data[field.name] = field.to_python(field_value)
Even if we customized the to_python method, it acts on the field_value alone, out of the context of the object. It's not a problem when using integers, since it will be interpreted as the model's primary key no matter what model it is. But to deserialize a natural key, first we need to know which model that key belongs to, and that information isn't available unless we got a reference to the object (and the target_content_type field had already been deserialized).
As you can see, it's not an impossible task - supporting natural keys in generic relations - but to accomplish that a lot of things would need to be changed in the serialization and deserialization code. The steps necessary, then (if anyone feels up to the task) are:
Create a custom Field extending PositiveIntegerField, with methods to encode/decode an object - calling the referenced models' natural_key and get_by_natural_key;
Override the serializer's handle_field to call the encoder if present;
Implement a custom deserializer that: 1) imposes some order in the fields, ensuring the content type is deserialized before the natural key; 2) calls the decoder, passing not only the field_value but also a reference to the decoded ContentType.
I've written a custom Serializer and Deserializer which supports GenericFK's. Checked it briefly and it seems to do the job.
This is what I came up with:
import json
from django.contrib.contenttypes.generic import GenericForeignKey
from django.utils import six
from django.core.serializers.json import Serializer as JSONSerializer
from django.core.serializers.python import Deserializer as \
PythonDeserializer, _get_model
from django.core.serializers.base import DeserializationError
import sys
class Serializer(JSONSerializer):
def get_dump_object(self, obj):
dumped_object = super(CustomJSONSerializer, self).get_dump_object(obj)
if self.use_natural_keys and hasattr(obj, 'natural_key'):
dumped_object['pk'] = obj.natural_key()
# Check if there are any generic fk's in this obj
# and add a natural key to it which will be deserialized by a matching Deserializer.
for virtual_field in obj._meta.virtual_fields:
if type(virtual_field) == GenericForeignKey:
content_object = getattr(obj, virtual_field.name)
dumped_object['fields'][virtual_field.name + '_natural_key'] = content_object.natural_key()
return dumped_object
def Deserializer(stream_or_string, **options):
"""
Deserialize a stream or string of JSON data.
"""
if not isinstance(stream_or_string, (bytes, six.string_types)):
stream_or_string = stream_or_string.read()
if isinstance(stream_or_string, bytes):
stream_or_string = stream_or_string.decode('utf-8')
try:
objects = json.loads(stream_or_string)
for obj in objects:
Model = _get_model(obj['model'])
if isinstance(obj['pk'], (tuple, list)):
o = Model.objects.get_by_natural_key(*obj['pk'])
obj['pk'] = o.pk
# If has generic fk's, find the generic object by natural key, and set it's
# pk according to it.
for virtual_field in Model._meta.virtual_fields:
if type(virtual_field) == GenericForeignKey:
natural_key_field_name = virtual_field.name + '_natural_key'
if natural_key_field_name in obj['fields']:
content_type = getattr(o, virtual_field.ct_field)
content_object_by_natural_key = content_type.model_class().\
objects.get_by_natural_key(obj['fields'][natural_key_field_name][0])
obj['fields'][virtual_field.fk_field] = content_object_by_natural_key.pk
for obj in PythonDeserializer(objects, **options):
yield obj
except GeneratorExit:
raise
except Exception as e:
# Map to deserializer error
six.reraise(DeserializationError, DeserializationError(e), sys.exc_info()[2])
I updated the OmriToptix answer for Django 2.2 and above.
In Django 2.0:
The Model._meta.virtual_fields attribute is removed.
So, the new Serializer and Deserializer:
import json
from django.contrib.contenttypes.fields import GenericForeignKey
from django.utils import six
from django.core.serializers.json import Serializer as JSONSerializer
from django.core.serializers.python import Deserializer as \
PythonDeserializer, _get_model
from django.core.serializers.base import DeserializationError
import sys
class Serializer(JSONSerializer):
def get_dump_object(self, obj):
dumped_object = super(JSONSerializer, self).get_dump_object(obj)
if hasattr(obj, 'natural_key'):
dumped_object['pk'] = obj.natural_key()
for field in obj._meta.get_fields():
if type(field) == GenericForeignKey:
content_object = getattr(obj, field.name)
dumped_object['fields'][field.name + '_natural_key'] = content_object.natural_key()
return dumped_object
def Deserializer(stream_or_string, **options):
if not isinstance(stream_or_string, (bytes, six.string_types)):
stream_or_string = stream_or_string.read()
if isinstance(stream_or_string, bytes):
stream_or_string = stream_or_string.decode('utf-8')
try:
objects = json.loads(stream_or_string)
for obj in objects:
Model = _get_model(obj['model'])
if isinstance(obj['pk'], (tuple, list)):
o = Model.objects.get_by_natural_key(*obj['pk'])
obj['pk'] = o.pk
for field in Model._meta.get_fields():
if type(field) == GenericForeignKey:
natural_key_field_name = field.name + '_natural_key'
if natural_key_field_name in obj['fields']:
content_type = getattr(o, field.ct_field)
content_object_by_natural_key = content_type.model_class().\
objects.get_by_natural_key(*obj['fields'][natural_key_field_name])
obj['fields'][field.fk_field] = content_object_by_natural_key.pk
del obj['fields'][natural_key_field_name]
for obj in PythonDeserializer(objects, **options):
yield obj
except GeneratorExit:
raise
except Exception as e:
six.reraise(DeserializationError, DeserializationError(e), sys.exc_info()[2])
Then, in your settings.py, set this configuration:
SERIALIZATION_MODULES = {
"json": "path.to.serializer_file"
}
Now, you can use:
python3 manage.py dumpdata --natural-foreign --natural-primary > dump.json
Other way, if you need to dump some data (filter querysets), you can make it from code:
from path.to.serializers import Serializer, Deserializer
# Serialize
registers = YourModel.objects.filter(some_attribute=some_value)
dump = Serializer().serialize(registers, use_natural_foreign_keys=True, use_natural_primary_keys=True)
# Deserialize
for deserialized_object in Deserializer(dump, use_natural_foreign_keys=True, use_natural_primary_keys=True):
print(deserialized_object.object) # See here https://docs.djangoproject.com/en/2.2/topics/serialization/
I'm trying to serialize some form data so that I can stuff it into a hidden field until the user is ready to submit the whole form (think of a wizard).
I'm trying this:
print simplejson.dumps(vehicle_form.cleaned_data)
But I keep getting errors like this:
<VehicleMake: Honda> is not JSON serializable
Really I just need it to output the PK for "Honda".
This doesn't work either:
print serializers.serialize('json', vehicle_form.cleaned_data)
Gives:
'str' object has no attribute '_meta'
Presumably because it's iterating over the keys, which are all strings, whereas I think it expects a queryset, which I don't have.
So how do I do this?
Okay, so far I've come up with this:
from django.utils.simplejson import JSONEncoder, dumps, loads
from django.utils.functional import curry
from django.db.models import Model
from django.db.models.query import QuerySet
from django.core.serializers import serialize
class DjangoJSONEncoder(JSONEncoder):
def default(self, obj):
if isinstance(obj, Model):
return obj.pk
elif isinstance(obj, QuerySet):
return loads(serialize('json', obj, ensure_ascii=False))
return JSONEncoder.default(self, obj)
json_encode = curry(dumps, cls=DjangoJSONEncoder)
json_decode = loads
Based on the answers I found [here][1]. Now I'm trying this:
json = json_encode(vehicle_form.cleaned_data)
data = json_decode(json)
vehicle = Vehicle(**data)
The first 2 lines work perfectly, but the 3rd results in an exception:
Cannot assign "3": "Vehicle.model" must be a "VehicleModel" instance.
Getting close! Not sure how to deal with this one though...
This is a bit of a hack, but I don't know of a better way:
try:
vehicle_data = simplejson.loads(request.POST['vehicle_data'])
except ValueError:
vehicle_data = []
vehicle_data.append(vehicle_form.raw_data())
request.POST['vehicle_data'] = simplejson.dumps(vehicle_data)
It grabs the JSON data from hidden field in your form and decodes it into a Python dict. If it doesn't exist, it starts a new list. Then it appends the new raw/uncleaned data and re-encodes it and dumps it into the hidden field.
For this to work you need to either make a copy of the POST data (request.POST.copy()) so that it becomes mutable, or hack it like I did: request.POST._mutable = True
In my form template I put this:
<input type="hidden" name="vehicle_data" value="{{request.POST.vehicle_data}}" />
And lastly, to access the raw data for a form I added these methods:
from django.forms import *
def _raw_data(self):
return dict((k,self.data[self.add_prefix(k)]) for k in self.fields.iterkeys())
def _raw_value(self, key, value=None):
if value is None:
return self.data[self.add_prefix(key)]
self.data[self.add_prefix(key)] = value
Form.raw_data = _raw_data
Form.raw_value = _raw_value
ModelForm.raw_data = _raw_data
ModelForm.raw_value = _raw_value
Since .data returns too much data (in fact, it just returns the POST data you initially passed in), plus it's got the prefixes, which I didn't want.
There is a lot of documentation on how to serialize a Model QuerySet but how do you just serialize to JSON the fields of a Model Instance?
You can easily use a list to wrap the required object and that's all what django serializers need to correctly serialize it, eg.:
from django.core import serializers
# assuming obj is a model instance
serialized_obj = serializers.serialize('json', [ obj, ])
If you're dealing with a list of model instances the best you can do is using serializers.serialize(), it gonna fit your need perfectly.
However, you are to face an issue with trying to serialize a single object, not a list of objects. That way, in order to get rid of different hacks, just use Django's model_to_dict (if I'm not mistaken, serializers.serialize() relies on it, too):
from django.forms.models import model_to_dict
# assuming obj is your model instance
dict_obj = model_to_dict( obj )
You now just need one straight json.dumps call to serialize it to json:
import json
serialized = json.dumps(dict_obj)
That's it! :)
To avoid the array wrapper, remove it before you return the response:
import json
from django.core import serializers
def getObject(request, id):
obj = MyModel.objects.get(pk=id)
data = serializers.serialize('json', [obj,])
struct = json.loads(data)
data = json.dumps(struct[0])
return HttpResponse(data, mimetype='application/json')
I found this interesting post on the subject too:
http://timsaylor.com/convert-django-model-instances-to-dictionaries
It uses django.forms.models.model_to_dict, which looks like the perfect tool for the job.
There is a good answer for this and I'm surprised it hasn't been mentioned. With a few lines you can handle dates, models, and everything else.
Make a custom encoder that can handle models:
from django.forms import model_to_dict
from django.core.serializers.json import DjangoJSONEncoder
from django.db.models import Model
class ExtendedEncoder(DjangoJSONEncoder):
def default(self, o):
if isinstance(o, Model):
return model_to_dict(o)
return super().default(o)
Now use it when you use json.dumps
json.dumps(data, cls=ExtendedEncoder)
Now models, dates and everything can be serialized and it doesn't have to be in an array or serialized and unserialized. Anything you have that is custom can just be added to the default method.
You can even use Django's native JsonResponse this way:
from django.http import JsonResponse
JsonResponse(data, encoder=ExtendedEncoder)
It sounds like what you're asking about involves serializing the data structure of a Django model instance for interoperability. The other posters are correct: if you wanted the serialized form to be used with a python application that can query the database via Django's api, then you would wan to serialize a queryset with one object. If, on the other hand, what you need is a way to re-inflate the model instance somewhere else without touching the database or without using Django, then you have a little bit of work to do.
Here's what I do:
First, I use demjson for the conversion. It happened to be what I found first, but it might not be the best. My implementation depends on one of its features, but there should be similar ways with other converters.
Second, implement a json_equivalent method on all models that you might need serialized. This is a magic method for demjson, but it's probably something you're going to want to think about no matter what implementation you choose. The idea is that you return an object that is directly convertible to json (i.e. an array or dictionary). If you really want to do this automatically:
def json_equivalent(self):
dictionary = {}
for field in self._meta.get_all_field_names()
dictionary[field] = self.__getattribute__(field)
return dictionary
This will not be helpful to you unless you have a completely flat data structure (no ForeignKeys, only numbers and strings in the database, etc.). Otherwise, you should seriously think about the right way to implement this method.
Third, call demjson.JSON.encode(instance) and you have what you want.
If you want to return the single model object as a json response to a client, you can do this simple solution:
from django.forms.models import model_to_dict
from django.http import JsonResponse
movie = Movie.objects.get(pk=1)
return JsonResponse(model_to_dict(movie))
If you're asking how to serialize a single object from a model and you know you're only going to get one object in the queryset (for instance, using objects.get), then use something like:
import django.core.serializers
import django.http
import models
def jsonExample(request,poll_id):
s = django.core.serializers.serialize('json',[models.Poll.objects.get(id=poll_id)])
# s is a string with [] around it, so strip them off
o=s.strip("[]")
return django.http.HttpResponse(o, mimetype="application/json")
which would get you something of the form:
{"pk": 1, "model": "polls.poll", "fields": {"pub_date": "2013-06-27T02:29:38.284Z", "question": "What's up?"}}
.values() is what I needed to convert a model instance to JSON.
.values() documentation: https://docs.djangoproject.com/en/3.0/ref/models/querysets/#values
Example usage with a model called Project.
Note: I'm using Django Rest Framework
from django.http import JsonResponse
#csrf_exempt
#api_view(["GET"])
def get_project(request):
id = request.query_params['id']
data = Project.objects.filter(id=id).values()
if len(data) == 0:
return JsonResponse(status=404, data={'message': 'Project with id {} not found.'.format(id)})
return JsonResponse(data[0])
Result from a valid id:
{
"id": 47,
"title": "Project Name",
"description": "",
"created_at": "2020-01-21T18:13:49.693Z",
}
I solved this problem by adding a serialization method to my model:
def toJSON(self):
import simplejson
return simplejson.dumps(dict([(attr, getattr(self, attr)) for attr in [f.name for f in self._meta.fields]]))
Here's the verbose equivalent for those averse to one-liners:
def toJSON(self):
fields = []
for field in self._meta.fields:
fields.append(field.name)
d = {}
for attr in fields:
d[attr] = getattr(self, attr)
import simplejson
return simplejson.dumps(d)
_meta.fields is an ordered list of model fields which can be accessed from instances and from the model itself.
Here's my solution for this, which allows you to easily customize the JSON as well as organize related records
Firstly implement a method on the model. I call is json but you can call it whatever you like, e.g.:
class Car(Model):
...
def json(self):
return {
'manufacturer': self.manufacturer.name,
'model': self.model,
'colors': [color.json for color in self.colors.all()],
}
Then in the view I do:
data = [car.json for car in Car.objects.all()]
return HttpResponse(json.dumps(data), content_type='application/json; charset=UTF-8', status=status)
Use list, it will solve problem
Step1:
result=YOUR_MODELE_NAME.objects.values('PROP1','PROP2').all();
Step2:
result=list(result) #after getting data from model convert result to list
Step3:
return HttpResponse(json.dumps(result), content_type = "application/json")
Use Django Serializer with python format,
from django.core import serializers
qs = SomeModel.objects.all()
serialized_obj = serializers.serialize('python', qs)
What's difference between json and python format?
The json format will return the result as str whereas python will return the result in either list or OrderedDict
To serialize and deserialze, use the following:
from django.core import serializers
serial = serializers.serialize("json", [obj])
...
# .next() pulls the first object out of the generator
# .object retrieves django object the object from the DeserializedObject
obj = next(serializers.deserialize("json", serial)).object
All of these answers were a little hacky compared to what I would expect from a framework, the simplest method, I think by far, if you are using the rest framework:
rep = YourSerializerClass().to_representation(your_instance)
json.dumps(rep)
This uses the Serializer directly, respecting the fields you've defined on it, as well as any associations, etc.
It doesn't seem you can serialize an instance, you'd have to serialize a QuerySet of one object.
from django.core import serializers
from models import *
def getUser(request):
return HttpResponse(json(Users.objects.filter(id=88)))
I run out of the svn release of django, so this may not be in earlier versions.
ville = UneVille.objects.get(nom='lihlihlihlih')
....
blablablab
.......
return HttpResponse(simplejson.dumps(ville.__dict__))
I return the dict of my instance
so it return something like {'field1':value,"field2":value,....}
how about this way:
def ins2dic(obj):
SubDic = obj.__dict__
del SubDic['id']
del SubDic['_state']
return SubDic
or exclude anything you don't want.
This is a project that it can serialize(JSON base now) all data in your model and put them to a specific directory automatically and then it can deserialize it whenever you want... I've personally serialized thousand records with this script and then load all of them back to another database without any losing data.
Anyone that would be interested in opensource projects can contribute this project and add more feature to it.
serializer_deserializer_model
Let this is a serializers for CROPS, Do like below. It works for me, Definitely It will work for you also.
First import serializers
from django.core import serializers
Then you can write like this
class CropVarietySerializer(serializers.Serializer):
crop_variety_info = serializers.serialize('json', [ obj, ])
OR you can write like this
class CropVarietySerializer(serializers.Serializer):
crop_variety_info = serializers.JSONField()
Then Call this serializer inside your views.py
For more details, Please visit https://docs.djangoproject.com/en/4.1/topics/serialization/
serializers.JSONField(*args, **kwargs) and serializers.JSONField() are same. you can also visit https://www.django-rest-framework.org/api-guide/fields/ for JSONField() details.