I am trying to deserialize a datetime value:
def default(o):
if type(o) is datetime.date or type(o) is datetime.datetime:
return o.isoformat()
def get_user_join_date(user):
return json.dumps(
user.date_joined,
sort_keys=True,
indent=1,
default=default
)
in order to use it as a value in payload:
def jwt_payload_handler(user, active_site):
payload = {
"id": user.id,
"date": get_user_join_date(user),
"username": user.username,
"role": user.role
}
The problem is that my datetime ends up being double encoded (it has 2 double quotes)
date: ""2018-09-27T12:32:17.577000+00:00""
Any idea why this might be happening and how I can solve it?
What's happening here is that the date string is encoded twice.
The return value of get_user_join_date(user) is the string '"2018-09-27T12:32:17.577000+00:00"' (so the ISO formatted datetime surrounded by double quotes).
This string is put into the payload and is again dumped to JSON by whatever you're using to create the JWT token.
A reduced example of what you are seeing would be this:
import json
import datetime
def default(o):
if type(o) is datetime.date or type(o) is datetime.datetime:
return o.isoformat()
print(json.dumps({
'date': json.dumps(
datetime.datetime.now(),
default=default
)
}))
Which produces the output you are seeing:
{"date": "\"2018-10-09T12:53:04.914694\""}
There are a few solutions to this issue:
The easiest solution to this issue would be to not use json.dumps in get_user_join_date and simply return user.date_joined.isoformat() instead.
Another solution would be to return user.date_joined from get_user_join_date and somehow pass the custom default function to the JSON encoder used by whatever is processing that JWT payload.
It might also be possible to just return the date object from get_user_join_date, but only if the library you're using is smart enough to handle datetime objects (some are).
Related
I have a one to many relation between session and camp. Now I have to get the max and min dates of all camps combined for a particular session.
I am able to do it like this:
sess = Session.objects.last()
max_min_dates = sess.camp.aggregate(Min('start_date'), Max('end_date'))
But if I try to send this from HttpResponse then I am getting this error:
TypeError: Object of type 'date' is not JSON serializable
So I need to send the formatted date values in that. How can I modify the above code to get the same?
The default encoder for json.dumps() does not support date encoding (ie. can't convert date into str). You can use django encoder instead, it supports a few more data types see the link for more info.
Django Ref
import json
from django.core.serializers.json import DjangoJSONEncoder
json_str = json.dumps(max_min_dates, cls=DjangoJSONEncoder)
I'm using JsonResponse(status=200, data=<python_dict_with_data>) to return from my Django backend API. However, upon inspecting the result of this, all number values get converted to strings (these values are the value portion of the dict, not key). This creates a problem in the frontend receiving this response because now I have to parse them as integers to do formatting and lightweight calculations. Is there a way to prevent this conversion when returned from Django?
Or is there a way the response is parsed correctly in the frontend? I'm using Axios library in React in the frontend.
Is there a way to prevent this conversion when returned from Django?
The keys will indeed be transformed into strings, because ints as keys are illegal in JSON. Indeed, if you use a validator like JSONLint, you will see that {1: 1} is invalid JSON, whereas { "1": 1 } is valid JSON. The Python JSON encoder will thus fallback on converting the integers to strings, to still produce valid content.
If you have to do lightweight calculations, likely using these as keys is not a good idea. For example if you have data that looks like:
{ 1: 4, 2: 5 }
you might consider restructuring the data, for example to:
{ "data": [ {"key": 1, "value": 4}, {"key": 2, "value": 5} ] }
You can also return it as HTTP response, and do parsing at the JavaScript end, but likely that will only result in more trouble.
For Decimal numbers, it will also use a string. Django uses by default the DjangoJSONEncoder [Django-doc] which:
Decimal, Promise (django.utils.functional.lazy() objects), UUID:
A string representation of the object.
If we for example encode a Decimal, we see:
>>> djenc = DjangoJSONEncoder()
>>> djenc.encode({'a': Decimal('0.25')})
'{"a": "0.25"}'
You can subclass the encoder, and resolve the Decimal for example to a float, but note that this can result in loss of precision. This is exactly why a string is used: to ensure that no digits are lossed:
from django.core.serializers.json import DjangoJSONEncoder
from decimal import Decimal
class MyDjangoJSONEncoder(DjangoJSONEncoder):
def default(self, o):
if isinstance(o, Decimal):
return float(o)
return super().default(o)
this then produces:
>>> mydjenc = MyDjangoJSONEncoder()
>>> mydjenc.encode({'a': Decimal('0.25')})
'{"a": 0.25}'
You can then use this encoder in your JsonResponse:
from decimal import Decimal
def myview(request):
# …
JsonResponse(encoder=MyDjangoJSONEncoder, data={'a': Decimal('0.25')})
I have a JSON field in my model and by using values option, I would like to get the key value present in the JSON field.
Assume my JSON field value is:
{"key1":"value1","key2":"value2"}
MyClass.objects.values("field1","field2","JSON Key")
Let JSON Key be "Key1"
Expected O/P:
[{field1:Value1,field2:value2,Key1:value1}]
A better solution (for Django >= 1.11) would be to use KeyTextTransform, like so:
from django.contrib.postgres.fields.jsonb import KeyTextTransform
MyModel.objects\
.annotate(key1=KeyTextTransform('key1', 'myjsonfield'))\
.values('field1','field2','key1')
I have written a custom manager function in ActiveQuerySet which accepts a list of fields and give get the particular field information from the object.
I have written the script for simple json structure..but where as u can change the way of processing json according to requirement.
The ActiveQuerySet class is as below.
class ActiveQuerySet(models.QuerySet):
def custom_values(self,*args,**kwargs):
model_fields = self.model._meta.get_fields()
model = [str(i.name) for i in model_fields]
json_fields = [str(i.name) for i in model_fields if i.get_internal_type() == 'JSONField']
responses = []
for one in self:
one_value = {}
for i in args[0]:
if i in model: # search if field is in normal model
one_value[i]=str(getattr(one,i))
else:
for jf in json_fields: #get all json fields in model
try:
json_value = eval(getattr(one,jf)) #eval to json or dict format if required
except:
json_value = getattr(one,jf)
json_keys = json_value.keys() #get the keys from json data stored
if i in json_keys:#check if key is present
one_value[i] = json_value[i]
responses.append(one_value)
return responses
MyModel.objects.all().custom_values(['field1','field2','key(which is present in JOSN field)'])
Assume my json data is stored as
{"cluster": "Mulchond", "2962": "2016-12-13", "2963": "1", "2964": "4", "2965": "0", "2966": "0", "2967": "0", "2968": "0.0318", "2969": "0.0705", "2970": "", "2971": "", "2972": "", "2973": "17.256", "2974": "48.8351", "2975": "142", "2976": "783", "2977": "276", "2978": "0.05237", "2979": "70", "2980": "0.05237", "2981": "", "2982": "", "2983": "", "2984": "142", "2985": "32", "2986": "", "2987": "20.773551", "2988": "73.649422"}
from this I want to get value of key '2988', My Query be like
MyModel.objects.filter().custom_values(['id','2988'])
o/p :
[{'2987': '20.730995', 'id': '66302'},
{'2987': '20.766556', 'id': '66303'},
{'2987': '20.773551', 'id': '66304'}]
where 'id is generated by Django and '2987' is one key which is present in JSON Field
I have a table that contains values saved as a dictionary.
FIELD_NAME: extra_data
VALUE:
{"code": null, "user_id": "103713616419757182414", "access_token": "ya29.IwBloLKFALsddhsAAADlliOoDeE-PD_--yz1i_BZvujw8ixGPh4zH-teMNgkIA", "expires": 3599}
I need to retrieve the user_id value from the field "extra_data" only not the dictionnary like below.
event_list = Event.objects.filter(season_id=season_id, event_status_id=2).value('extra_data')
If you are storing a dictionary as text in the code you can easily convert it to a python dictionary using eval - although I don't know why you'd want to as it opens you to all sorts of potential malicious code injections.
event_list = eval(Event.objects.filter(season_id=season_id, event_status_id=2).value('extra_data'))
user_id = event_list['user_id']
print user_id
Would give:
"103713616419757182414"
Edit:
On deeper inspection , thats not a Python dictionary, you could import a JSON library to import this, or declare what null is like so:
null = None
event_list = eval(Event.objects.filter(season_id=season_id, event_status_id=2).value('extra_data'))
user_id = event_list['user_id']
Either way, the idea of storing any structured data in a django textfield is fraught with danger that will come back to bite you. The best solution is to rethink your data structures.
This method worked for me. However, this works with a json compliant string
import json
json_obj = json.loads(event_list)
dict1 = dict(json_obj)
print dict1['user_id']
I have a Django model containing DecimalField. The resulting json should contain only data (no keys) so I'm using values_list() to convert queryset to list of Tuples:
MyModel.objects.filter(...).values_list('my_date_field','my_decimal_field').order_by('my_date_field')
Then, I need to serialize it to json... but json.dumps does not seems to be able to process the Decimal field... A lot of SO answers about that suggest to make your own encoder to use with json.dumps but those custom encoders are not recursive and seems not to work with a list of Tuple...
What I need is returning json with this format:
[[1162512000000,78.29],
[1162771200000,79.71],
[1162857600000,80.51],
[1162944000000,82.45],
[1163030400000,83.34],
[1163116800000,83.12],
[1163376000000,84.35]]
It seems to me that this should be a simple task but can't find a simple way to do it without having to parse and process everything manually...
Any suggestions?
Thanks a lot
Etienne
This should work:
import json
from decimal import Decimal as D
class DecimalJSONEncoder(json.JSONEncoder):
def default(self, o):
if type(o) == D:
# Here You can decide if You want decimal to be converted
# to string or float.
return float(o)
return super(DecimalJSONEncoder, self).default(o)
data = [[1162512000000, D(78.29)],
[1162771200000, D(79.71)],
[1162857600000, D(80.51)],
[1162944000000, D(82.45)],
[1163030400000, D(83.34)],
[1163116800000, D(83.12)],
[1163376000000, D(84.35)]]
encoder = DecimalJSONEncoder()
encoder.encode(data)
# Result:
# '[[1162512000000, 78.29], [1162771200000, 79.71], [1162857600000, 80.51], ...'