I have a django application with tastypie. One of the models in my app has a DecimalField. When I get a response from the API in JSON format, all the decimal fields appear as strings instead of numbers:
For example I get:
objects: [
{
id: "1",
my_decimal_field: "84.54"
}
instead of
objects: [
{
id: "1"
my_decimal_field: 84.54
}
This also happens with the id field.
¿Any thoughts?
In JavaScript, JSON decodes to double-precision floating point format, which causes a loss of precision. Decimal objects are encoded to string to maintain precision.
If you want to encode to JSON number format, you can use a FloatField.
Related
I'm using JsonResponse(status=200, data=<python_dict_with_data>) to return from my Django backend API. However, upon inspecting the result of this, all number values get converted to strings (these values are the value portion of the dict, not key). This creates a problem in the frontend receiving this response because now I have to parse them as integers to do formatting and lightweight calculations. Is there a way to prevent this conversion when returned from Django?
Or is there a way the response is parsed correctly in the frontend? I'm using Axios library in React in the frontend.
Is there a way to prevent this conversion when returned from Django?
The keys will indeed be transformed into strings, because ints as keys are illegal in JSON. Indeed, if you use a validator like JSONLint, you will see that {1: 1} is invalid JSON, whereas { "1": 1 } is valid JSON. The Python JSON encoder will thus fallback on converting the integers to strings, to still produce valid content.
If you have to do lightweight calculations, likely using these as keys is not a good idea. For example if you have data that looks like:
{ 1: 4, 2: 5 }
you might consider restructuring the data, for example to:
{ "data": [ {"key": 1, "value": 4}, {"key": 2, "value": 5} ] }
You can also return it as HTTP response, and do parsing at the JavaScript end, but likely that will only result in more trouble.
For Decimal numbers, it will also use a string. Django uses by default the DjangoJSONEncoder [Django-doc] which:
Decimal, Promise (django.utils.functional.lazy() objects), UUID:
A string representation of the object.
If we for example encode a Decimal, we see:
>>> djenc = DjangoJSONEncoder()
>>> djenc.encode({'a': Decimal('0.25')})
'{"a": "0.25"}'
You can subclass the encoder, and resolve the Decimal for example to a float, but note that this can result in loss of precision. This is exactly why a string is used: to ensure that no digits are lossed:
from django.core.serializers.json import DjangoJSONEncoder
from decimal import Decimal
class MyDjangoJSONEncoder(DjangoJSONEncoder):
def default(self, o):
if isinstance(o, Decimal):
return float(o)
return super().default(o)
this then produces:
>>> mydjenc = MyDjangoJSONEncoder()
>>> mydjenc.encode({'a': Decimal('0.25')})
'{"a": 0.25}'
You can then use this encoder in your JsonResponse:
from decimal import Decimal
def myview(request):
# …
JsonResponse(encoder=MyDjangoJSONEncoder, data={'a': Decimal('0.25')})
I was following some django rest framework tutorials and found some obscure codes. This snippet is from the customised user model, the project from which uses jwt for authentication.
As I commented in the snippet, I can't notice the reason Why they first encodes data and decode it again. I thought this kind of pattern is not only specific to this tutorial, but quite a general pattern. Could anyone explain me please?
def _generate_jwt_token(self):
"""
Generates a JSON Web Token that stores this user's ID and
has an expiry date set to 60 days into the future.
"""
dt = datetime.now() + timedelta(days=60)
token = jwt.encode({ #first encode here
'id': self.pk,
'exp': int(dt.strftime('%s'))
}, settings.SECRET_KEY, algorithm='HS256')
return token.decode('utf-8') #returns decoded object
“Encoding” usually refers to converting data to its binary representation (bytes).
JWT (JSON Web Token) encoding uses a specific data structure and cryptographic signing to allow secure, authenticated exchanges.
The steps to encode data as JWT are as follows :
The payload is converted to json and encoded using base64.
A header, specifying the token type (eg. jwt) and the signature algorithm to use (eg. HS256), is encoded similarly.
A signature is derived from your private key and the two previous values.
Result is obtained by joining header, payload and signature with dots. The output is a binary string.
More informations here.
Decoding it using UTF-8 transforms this binary string into an Unicode string :
>>> encoded_bin = jwt.encode({'some': 'data'}, 'secret_sig', algorithm='HS256')
>>> type(encoded_bin)
<class 'bytes'>
>>> encoded_string = encoded_bin.decode('utf-8')
>>> type(encoded_string)
<class 'str'>
Notes:
It is not always possible to decode bytes to string. Base64-encoding your data allows you to store any bytes as a text representation, but the encoded form requires more space (+33%) than it's raw representation.
A binary string is prefixed by a b in your Python interpreter (eg. b"a binary string")
I cannot find the right answer for my case, so I posted my question here.
I'm validating the form in ASP.NET MVC and looking for the way to validate a text field to allow only numeric and decimal numbers like
1 or 1.5 or 1.65
If I have 1,65 I do not want this to be validated.
I have put a metadata on my Model's field like this: [RegularExpression(#"^(((\d{1})*))$")]
And have
#Html.ValidationMessageFor(m => m.ResolvedAmount, "", new { #class = "error" })
to validate the field.
Also, in my function I have the following to check the validation:
var validator = $("#main").kendoValidator().data('kendoValidator');
if(validator.validate()){some logic}
I validate for the required field, but cannot get my int/decimal validation working.
What do I need to have in order to validate it?
I have a Django model containing DecimalField. The resulting json should contain only data (no keys) so I'm using values_list() to convert queryset to list of Tuples:
MyModel.objects.filter(...).values_list('my_date_field','my_decimal_field').order_by('my_date_field')
Then, I need to serialize it to json... but json.dumps does not seems to be able to process the Decimal field... A lot of SO answers about that suggest to make your own encoder to use with json.dumps but those custom encoders are not recursive and seems not to work with a list of Tuple...
What I need is returning json with this format:
[[1162512000000,78.29],
[1162771200000,79.71],
[1162857600000,80.51],
[1162944000000,82.45],
[1163030400000,83.34],
[1163116800000,83.12],
[1163376000000,84.35]]
It seems to me that this should be a simple task but can't find a simple way to do it without having to parse and process everything manually...
Any suggestions?
Thanks a lot
Etienne
This should work:
import json
from decimal import Decimal as D
class DecimalJSONEncoder(json.JSONEncoder):
def default(self, o):
if type(o) == D:
# Here You can decide if You want decimal to be converted
# to string or float.
return float(o)
return super(DecimalJSONEncoder, self).default(o)
data = [[1162512000000, D(78.29)],
[1162771200000, D(79.71)],
[1162857600000, D(80.51)],
[1162944000000, D(82.45)],
[1163030400000, D(83.34)],
[1163116800000, D(83.12)],
[1163376000000, D(84.35)]]
encoder = DecimalJSONEncoder()
encoder.encode(data)
# Result:
# '[[1162512000000, 78.29], [1162771200000, 79.71], [1162857600000, 80.51], ...'
I'm saving the following simple, valid JSON object to a model in my Django app:
{
"start_date": 1311471044.24338
"post_count": 25
}
The model looks like this:
from django.db import models
from django_extensions.db.fields import json as json
class UserProfile(models.Model):
data = json.JSONField()
To read the posted data, I basically do
posted_data = request.FILES.get('posted_data').read()
json_data = simplejson.loads(posted_data)
The data then contains the expect type (float)
logging.debug( "start_date type: " + str(type(json_data.get('start_date'))))
logging.debug( "post_count type: " + str(type(json_data.get('post_count'))))
>> 2011-07-24 10:03:01,636 DEBUG start_date type: <type 'float'>
>> 2011-07-24 10:03:01,636 DEBUG post_count type: <type 'int'>
I then save the data like this:
user_profile.data = json_data
user_profile.save()
and then I read the data back, integers are fine, but floating point numbers are quoted, for example:
print user_profile.data
{
"post_count : 25
"start_date": "1311471044.24338"
}
How can I prevent floating point numbers from being turned to strings unnecessarily?
Edit:
May have found an explanation here: Rails JSON Serialization of Decimal adds Quotes
I still would be interested in other explanations though.
This answer seems to be the best explanation:
The only "safe" way to hand decimals from language A to language B is to use a String. If your json contains "rating": 98.79999999999999 it will probably be converted to 98.79999999999998 by your JavaScript runtime.