Recommended way of serializing Django RawQuerySet with non-model fields - django

Question

Having query like SELECT *, 'hello' AS world FROM myApp_myModel I'd like to serialize it to json.
Doesn't seem like a big deal, and there are plenty of similar questions on SO but none seems to give straight answer.
So far I've tried:
data = myModel.objects.raw(query)
# gives: ModelState is not serializable
json.dumps([dict(r.__dict__) for r in data])
# doesn't serialize 'world' column, only model fields:
serializers.serialize('json', data)
#dear God:
for r in data:
for k in dict(r.__dict__):
print(getattr(r,k))

Answer 1

The issue:
Builtin django core serializers are not ready to include extra fields ( from raw neither from annotation expression) It just takes model fields from _meta.local_fields.
You can see it at django django/core/serializers/base.py source code:
concrete_model = obj._meta.concrete_model #obj is an object model
...
for field in concrete_model._meta.local_fields:
if field.serialize or field is pk_parent:
if field.remote_field is None:
if (self.selected_fields is None
or field.attname in self.selected_fields):
self.handle_field(obj, field)
else:
if (self.selected_fields is None
or field.attname[:-3] in self.selected_fields):
self.handle_fk_field(obj, field)
django rest framework at rescue:
To solve your issue you can use a non builtin functionality. You can include a REST package in your project. For example django rest framework can handle extra fields:
from django.db.models import F
from aula.apps.alumnes.models import MyModel
from rest_framework.renderers import JSONRenderer
data=MyModel.objects.annotate(dummy = F('some_field') )
class MyModelSerializer(serializers.ModelSerializer):
dummy = serializers.CharField()
class Meta:
model = MyModel
fields = ('some_other_field','dummy')
read_only_fields = (
'dummy',
)
m=MyModelSerializer(data, many=True)
JSONRenderer().render(m.data)

Answer 2

You can create a DRF searializer for the task:
http://www.django-rest-framework.org/api-guide/serializers/
i.e.
class MyModelSerializer(serializers.ModelSerializer):
world = serializers.ReadOnlyField()
class Meta:
model = MyModel
fields = (world, ...)
you can also use serializer inheritance etc - see the docs.

Answer 3

There is a clean way you can do this using Django Rest Framework
First off did you know You can also execute queries containing fields that aren’t defined on the model when doing a Raw query
for example ( REF )
>>> people = Person.objects.raw('SELECT *, age(birth_date) AS age FROM myapp_person')
>>> for p in people:
... print("%s is %s." % (p.first_name, p.age))
John is 37.
Jane is 42.
That means you can use a standard serializer. You just need to tell the serializer what to do with fields that were not originally on the model consider the below. Needed to join 3 tables to a user. The user, the company they belong to and the companies membership. If your table has thousands of users and you did the standard serialiser method field, it would result in thousands of queries to get the related companies membership each time. so instead here was the solution I used
# api.py
class UserSAMAExportListApiView(ListAPIView):
serializer_class = UserExportSerializer
model = User
def get_queryset(self):
q = User.objects.raw(
"""
SELECT
[users_user].[id],
[users_user].[email],
[companies_company].[title] AS company__title,
[companies_company].[registration_number] AS company__registration_number,
[memberships_membership].number AS company__membership__number
FROM [users_user]
LEFT OUTER JOIN [dbo].[companies_company]
ON ([users_user].[company_id] = [companies_company].[id])
LEFT OUTER JOIN [memberships_membership]
ON ([companies_company].[id] = [memberships_membership].[company_id])
WHERE ([memberships_membership].[expiry_date] >= %s)
"""
, [date.today(),]
)
return q
Then just tell your standard serialiser that there are some new fields you defined
# serializers.py
class UserExportSerializer(ModelSerializer):
class Meta:
model = User
fields = (
'id',
'email',
'company__title',
'company__registration_number',
'company__membership__number',
)
def build_unknown_field(self, field_name, model_class):
"""
Return a two tuple of (cls, kwargs) to build a serializer field with. For fields that werent originally on
The model
"""
return fields.CharField, {'read_only': True}
And that's it DRF will handle the rest in a standard way and do proper serialization for you
Note you have to override the build_unknown_fields method. This is simply saying convert all the non-standard model fields to Text, if you want you can check the field name and convert to other formats here.