Converting result of MySQL query in Django to JSON - django

So I query my database using a mySQL query like so:
cursor = connection.cursor()
cursor.execute("select winner,count(winner) as count from DB")
data = cursor.fetchall()
Now I want to send the table in data to my app (as a GET request) in JSON. Doing this is not sending a properly formatted JSON response and I am unable to parse it on the client side.
return HttpResponse(json.dumps(data), content_type='application/json;charset=utf8')
The json.dumps(data) returns this:
[["John Doe", 45]]
Any help in this regard would be appreciated.

The JSON is properly formatted, but you are dumping a list, you should dump a dictionary instead... something like:
myData = {'people': data}
json.dumps(myData)
The point is this: a valid json response must start and end with curly braces, so in order to serve a valid json you have to dump a Python dictionary object as a "root object"... in other words you need at least an object with a key.
From http://json.org
JSON is built on two structures:
A collection of name/value pairs. In various languages, this is realized as an object, record, struct, dictionary, hash table, keyed
list, or associative array.
An ordered list of values. In most languages, this is realized as an array, vector, list, or sequence.

from django.core import serializers
json_data = serializers.serialize('json', data)
return HttpResponse(json_data, mimetype='application/json')
However not everything can be serialized like this into JSON, some things need a custom encoder
You should use a model and the ORM instead of writing your own SQL.
You could easily convert your statement to this simple model and succinctly ORM call.
class Winner(models.Model):
name = models.CharField()
and your database call would now be Winner.objects.all() which would give all winners
and with the count
Winner.objects.annotate(wins=Count('name'))

from django.http import JsonResponse
from django.db import connections
from django.http import HttpResponse
import json
def testRawQuery(request):
cursor = connections['default'].cursor()
cursor.execute("select winner,count(winner) as count from DB")
objs = cursor.fetchall()
json_data = []
for obj in objs:
json_data.append({"winner" : obj[0], "count" : obj[1]})
return JsonResponse(json_data, safe=False)

Related

Serialize data to json formate using native serializer Django

I have this dictionary which I need to pass to another view, knowing that possible ways of doing that are either through sessions or cache, now when I am trying to pass to session it is throwing me an error that data is not JSON serializable probably because I have DateTime fields inside this dictionary
session_data = serializers.serialize('json',session_data)
error on above statement
'str' object has no attribute '_meta'
updated
date is somewhat in this format
{'city_name': 'Srinagar', 'description': 'few clouds', 'temp': 26.74, 'feels_like': 27.07, 'max_temp': 26.74, 'min_temp': 26.74, 'sunrise': datetime.time(6, 11, 10), 'sunset': datetime.time(18, 43, 59)}
Your session_data is already a dictionary. Since Django's serializer focuses on serializing an iterable of model object, that thus will not work.
You can make use of Django's DjangoJSONEncoder [Django-doc] to serialize Python objects that include date, datetime, time and/or timedelta objects.
You thus can work with:
from django.core.serializers.json import DjangoJSONEncoder
encoder = DjangoJSONEncoder()
session_data = encoder.encode(session_data)
If you plan to return a JSON blob as a HTTP response, you can simply let the JsonResponse do the encoding work:
from django.http import JsonResponse
# …
return JsonResponse(session_data)

Get data from JsonResponse in django

I wanted to know how to get data from a JsonResponse in django. I made a JsonResponse that works like this
def pfmdetail(rsid):
snpid = parseSet(rsid)
if not snpid:
return HttpResponse(status=404)
try:
data = SnpsPfm.objects.values('start', 'strand', 'type', 'scoreref', 'scorealt',
rsid=F('snpid__rsid'), pfm_name=F('pfmid__name')).filter(snpid=snpid[0])
except SnpsPfm.DoesNotExist:
return HttpResponse(status=404)
serializer = SnpsPfmSerializer(data, many=True)
return JsonResponse(serializer.data, safe=False)
and then I call directly the method like this
def pfmTable(qset,detail):
source = pfmdetail(detail)
print(source)
df = pd.read_json(source)
but it gives me an error. I know it's wrong because with the print it returns the status of the response which is 200 so I suppose that the response is fine but how can I access the data inside the response? I tried import json to do json.load but with no success. I even tried the methods of QueryDict but stil I can't acess to the content I'm interested
P.S. I know that data contains something because if i display the jsonresponse on the browser i can see the JSON
As you can see here: https://docs.djangoproject.com/en/2.2/ref/request-response/#jsonresponse-objects.
JsonResponse object holds json in its content attribute.
So to access it try this:
df = pd.read_json(source.content)
Or to see it printed do:
print(source.content)
If you aren't using pandas, then you should process the content attribute of the JSONResponse object like this:
r = json.loads(source.decode())
I got the answer here: How to parse binary string to dict ?

How to encode a django query set specific field to a Json response?

I do a filter to obtain a particular set of objects from django data model. I need to encode only a single field of that objects to a json response.
e.g.: Item has an attribute called name.
qs_available = Item.objects.filter(Type=1).values.('name').???
return HttpResponse(json.dumps(qs_available), content_type='application/json')
How do I return the list of name values as a json response?
If you want to get only names list, you can use values_list
qs_available = list(Item.objects.filter(Type=1).values_list('name', flat=True))
return HttpResponse(json.dumps(qs_available), content_type='application/json')
You could use JsonResponse from django.http,
from django.http import JsonResponse
qs_available = Item.objects.filter(Type=1).values_list('name')
return JsonResponse(list(qs_available), safe=False)

How to serialize cleaned_data if it contains models?

I'm trying to serialize some form data so that I can stuff it into a hidden field until the user is ready to submit the whole form (think of a wizard).
I'm trying this:
print simplejson.dumps(vehicle_form.cleaned_data)
But I keep getting errors like this:
<VehicleMake: Honda> is not JSON serializable
Really I just need it to output the PK for "Honda".
This doesn't work either:
print serializers.serialize('json', vehicle_form.cleaned_data)
Gives:
'str' object has no attribute '_meta'
Presumably because it's iterating over the keys, which are all strings, whereas I think it expects a queryset, which I don't have.
So how do I do this?
Okay, so far I've come up with this:
from django.utils.simplejson import JSONEncoder, dumps, loads
from django.utils.functional import curry
from django.db.models import Model
from django.db.models.query import QuerySet
from django.core.serializers import serialize
class DjangoJSONEncoder(JSONEncoder):
def default(self, obj):
if isinstance(obj, Model):
return obj.pk
elif isinstance(obj, QuerySet):
return loads(serialize('json', obj, ensure_ascii=False))
return JSONEncoder.default(self, obj)
json_encode = curry(dumps, cls=DjangoJSONEncoder)
json_decode = loads
Based on the answers I found [here][1]. Now I'm trying this:
json = json_encode(vehicle_form.cleaned_data)
data = json_decode(json)
vehicle = Vehicle(**data)
The first 2 lines work perfectly, but the 3rd results in an exception:
Cannot assign "3": "Vehicle.model" must be a "VehicleModel" instance.
Getting close! Not sure how to deal with this one though...
This is a bit of a hack, but I don't know of a better way:
try:
vehicle_data = simplejson.loads(request.POST['vehicle_data'])
except ValueError:
vehicle_data = []
vehicle_data.append(vehicle_form.raw_data())
request.POST['vehicle_data'] = simplejson.dumps(vehicle_data)
It grabs the JSON data from hidden field in your form and decodes it into a Python dict. If it doesn't exist, it starts a new list. Then it appends the new raw/uncleaned data and re-encodes it and dumps it into the hidden field.
For this to work you need to either make a copy of the POST data (request.POST.copy()) so that it becomes mutable, or hack it like I did: request.POST._mutable = True
In my form template I put this:
<input type="hidden" name="vehicle_data" value="{{request.POST.vehicle_data}}" />
And lastly, to access the raw data for a form I added these methods:
from django.forms import *
def _raw_data(self):
return dict((k,self.data[self.add_prefix(k)]) for k in self.fields.iterkeys())
def _raw_value(self, key, value=None):
if value is None:
return self.data[self.add_prefix(key)]
self.data[self.add_prefix(key)] = value
Form.raw_data = _raw_data
Form.raw_value = _raw_value
ModelForm.raw_data = _raw_data
ModelForm.raw_value = _raw_value
Since .data returns too much data (in fact, it just returns the POST data you initially passed in), plus it's got the prefixes, which I didn't want.

How do you serialize a model instance in Django?

There is a lot of documentation on how to serialize a Model QuerySet but how do you just serialize to JSON the fields of a Model Instance?
You can easily use a list to wrap the required object and that's all what django serializers need to correctly serialize it, eg.:
from django.core import serializers
# assuming obj is a model instance
serialized_obj = serializers.serialize('json', [ obj, ])
If you're dealing with a list of model instances the best you can do is using serializers.serialize(), it gonna fit your need perfectly.
However, you are to face an issue with trying to serialize a single object, not a list of objects. That way, in order to get rid of different hacks, just use Django's model_to_dict (if I'm not mistaken, serializers.serialize() relies on it, too):
from django.forms.models import model_to_dict
# assuming obj is your model instance
dict_obj = model_to_dict( obj )
You now just need one straight json.dumps call to serialize it to json:
import json
serialized = json.dumps(dict_obj)
That's it! :)
To avoid the array wrapper, remove it before you return the response:
import json
from django.core import serializers
def getObject(request, id):
obj = MyModel.objects.get(pk=id)
data = serializers.serialize('json', [obj,])
struct = json.loads(data)
data = json.dumps(struct[0])
return HttpResponse(data, mimetype='application/json')
I found this interesting post on the subject too:
http://timsaylor.com/convert-django-model-instances-to-dictionaries
It uses django.forms.models.model_to_dict, which looks like the perfect tool for the job.
There is a good answer for this and I'm surprised it hasn't been mentioned. With a few lines you can handle dates, models, and everything else.
Make a custom encoder that can handle models:
from django.forms import model_to_dict
from django.core.serializers.json import DjangoJSONEncoder
from django.db.models import Model
class ExtendedEncoder(DjangoJSONEncoder):
def default(self, o):
if isinstance(o, Model):
return model_to_dict(o)
return super().default(o)
Now use it when you use json.dumps
json.dumps(data, cls=ExtendedEncoder)
Now models, dates and everything can be serialized and it doesn't have to be in an array or serialized and unserialized. Anything you have that is custom can just be added to the default method.
You can even use Django's native JsonResponse this way:
from django.http import JsonResponse
JsonResponse(data, encoder=ExtendedEncoder)
It sounds like what you're asking about involves serializing the data structure of a Django model instance for interoperability. The other posters are correct: if you wanted the serialized form to be used with a python application that can query the database via Django's api, then you would wan to serialize a queryset with one object. If, on the other hand, what you need is a way to re-inflate the model instance somewhere else without touching the database or without using Django, then you have a little bit of work to do.
Here's what I do:
First, I use demjson for the conversion. It happened to be what I found first, but it might not be the best. My implementation depends on one of its features, but there should be similar ways with other converters.
Second, implement a json_equivalent method on all models that you might need serialized. This is a magic method for demjson, but it's probably something you're going to want to think about no matter what implementation you choose. The idea is that you return an object that is directly convertible to json (i.e. an array or dictionary). If you really want to do this automatically:
def json_equivalent(self):
dictionary = {}
for field in self._meta.get_all_field_names()
dictionary[field] = self.__getattribute__(field)
return dictionary
This will not be helpful to you unless you have a completely flat data structure (no ForeignKeys, only numbers and strings in the database, etc.). Otherwise, you should seriously think about the right way to implement this method.
Third, call demjson.JSON.encode(instance) and you have what you want.
If you want to return the single model object as a json response to a client, you can do this simple solution:
from django.forms.models import model_to_dict
from django.http import JsonResponse
movie = Movie.objects.get(pk=1)
return JsonResponse(model_to_dict(movie))
If you're asking how to serialize a single object from a model and you know you're only going to get one object in the queryset (for instance, using objects.get), then use something like:
import django.core.serializers
import django.http
import models
def jsonExample(request,poll_id):
s = django.core.serializers.serialize('json',[models.Poll.objects.get(id=poll_id)])
# s is a string with [] around it, so strip them off
o=s.strip("[]")
return django.http.HttpResponse(o, mimetype="application/json")
which would get you something of the form:
{"pk": 1, "model": "polls.poll", "fields": {"pub_date": "2013-06-27T02:29:38.284Z", "question": "What's up?"}}
.values() is what I needed to convert a model instance to JSON.
.values() documentation: https://docs.djangoproject.com/en/3.0/ref/models/querysets/#values
Example usage with a model called Project.
Note: I'm using Django Rest Framework
from django.http import JsonResponse
#csrf_exempt
#api_view(["GET"])
def get_project(request):
id = request.query_params['id']
data = Project.objects.filter(id=id).values()
if len(data) == 0:
return JsonResponse(status=404, data={'message': 'Project with id {} not found.'.format(id)})
return JsonResponse(data[0])
Result from a valid id:
{
"id": 47,
"title": "Project Name",
"description": "",
"created_at": "2020-01-21T18:13:49.693Z",
}
I solved this problem by adding a serialization method to my model:
def toJSON(self):
import simplejson
return simplejson.dumps(dict([(attr, getattr(self, attr)) for attr in [f.name for f in self._meta.fields]]))
Here's the verbose equivalent for those averse to one-liners:
def toJSON(self):
fields = []
for field in self._meta.fields:
fields.append(field.name)
d = {}
for attr in fields:
d[attr] = getattr(self, attr)
import simplejson
return simplejson.dumps(d)
_meta.fields is an ordered list of model fields which can be accessed from instances and from the model itself.
Here's my solution for this, which allows you to easily customize the JSON as well as organize related records
Firstly implement a method on the model. I call is json but you can call it whatever you like, e.g.:
class Car(Model):
...
def json(self):
return {
'manufacturer': self.manufacturer.name,
'model': self.model,
'colors': [color.json for color in self.colors.all()],
}
Then in the view I do:
data = [car.json for car in Car.objects.all()]
return HttpResponse(json.dumps(data), content_type='application/json; charset=UTF-8', status=status)
Use list, it will solve problem
Step1:
result=YOUR_MODELE_NAME.objects.values('PROP1','PROP2').all();
Step2:
result=list(result) #after getting data from model convert result to list
Step3:
return HttpResponse(json.dumps(result), content_type = "application/json")
Use Django Serializer with python format,
from django.core import serializers
qs = SomeModel.objects.all()
serialized_obj = serializers.serialize('python', qs)
What's difference between json and python format?
The json format will return the result as str whereas python will return the result in either list or OrderedDict
To serialize and deserialze, use the following:
from django.core import serializers
serial = serializers.serialize("json", [obj])
...
# .next() pulls the first object out of the generator
# .object retrieves django object the object from the DeserializedObject
obj = next(serializers.deserialize("json", serial)).object
All of these answers were a little hacky compared to what I would expect from a framework, the simplest method, I think by far, if you are using the rest framework:
rep = YourSerializerClass().to_representation(your_instance)
json.dumps(rep)
This uses the Serializer directly, respecting the fields you've defined on it, as well as any associations, etc.
It doesn't seem you can serialize an instance, you'd have to serialize a QuerySet of one object.
from django.core import serializers
from models import *
def getUser(request):
return HttpResponse(json(Users.objects.filter(id=88)))
I run out of the svn release of django, so this may not be in earlier versions.
ville = UneVille.objects.get(nom='lihlihlihlih')
....
blablablab
.......
return HttpResponse(simplejson.dumps(ville.__dict__))
I return the dict of my instance
so it return something like {'field1':value,"field2":value,....}
how about this way:
def ins2dic(obj):
SubDic = obj.__dict__
del SubDic['id']
del SubDic['_state']
return SubDic
or exclude anything you don't want.
This is a project that it can serialize(JSON base now) all data in your model and put them to a specific directory automatically and then it can deserialize it whenever you want... I've personally serialized thousand records with this script and then load all of them back to another database without any losing data.
Anyone that would be interested in opensource projects can contribute this project and add more feature to it.
serializer_deserializer_model
Let this is a serializers for CROPS, Do like below. It works for me, Definitely It will work for you also.
First import serializers
from django.core import serializers
Then you can write like this
class CropVarietySerializer(serializers.Serializer):
crop_variety_info = serializers.serialize('json', [ obj, ])
OR you can write like this
class CropVarietySerializer(serializers.Serializer):
crop_variety_info = serializers.JSONField()
Then Call this serializer inside your views.py
For more details, Please visit https://docs.djangoproject.com/en/4.1/topics/serialization/
serializers.JSONField(*args, **kwargs) and serializers.JSONField() are same. you can also visit https://www.django-rest-framework.org/api-guide/fields/ for JSONField() details.