Maintain the order of json data when serializing - django

I have a data set like this.
{
'album_name': 'Dear John',
'artist': 'Loney Dear',
'tracks': [
'Airport Surroundings',
'Everything Turns to You',
'I Was Only Going Out',
]
}
when I serialize it, my json file does not look like the same every time. Because order of 'tracks' change every time.
I was looking at 'to_representation' but since this data doesn't have a Key i failed to implement it as i expected.
Can any one give a hint, to make sure that 'tracks' always in same order.
Edit:
This where i am so far,
class QaDetailSerializer(ModelSerializer):
"""Serializer to map the Model instance into JSON format."""
album_name = CharField(source='album_name')
artist = StringRelatedField()
tracks = TracksSerializer()
class Meta:
"""Meta class to map serializer's fields with the model fields."""
order_by = (('id',))
model = Qa
fields = (
'id',
'album_name',
'artist',
'tracks',
)
class TracksSerializer(ModelSerializer):
def to_representation(self, value):
representation = super().to_representation(value)
attributes_dict = representation['tracks']
attribute_keys_sorted = sorted(attributes_dict.keys())
sorted_attribute_dict = collections.OrderedDict()
for key in attribute_keys_sorted:
sorted_attribute_dict[key] = attributes_dict[key]
representation['paraphrases'] = sorted_attribute_dict
return representation

The problem you are experiencing is database problem not serializer problem.
You never told your database how to sort your tracks so each time database return tracks in different order.
In your Track model (not serializer) add this Meta class
class Track(models.Model):
...fields
class Meta:
ordering = ("pk",)
this will cause your tracks to be always ordered by primary key.
You can order by any other field and you can also order by multiple fields
i.e ordering = ("music_type", "name")

Related

Django filter that also grabs a reverse ForeignKey, or better way to combine associated database info?

I am working on some backend django work that requires me to grab an Employee by filtering, but I also need to grab the EmployeeAddress object that is associated to the Employee. I was wondering if this was possible within a single query. I need the employees address, and employee info to be in a combined single dictionary, to access on the front end side with JS.
I have models as such,
Class Employee(models.model):
first_name
last_name
email
Class EmployeeAddress(models.model):
employee = models.ForeignKey(Employee):
street
city
state
I have a view, that kinda does the job, but having trouble merging merging the two separate QuerySets into a single listed dictionary with all values.
I was hoping there was just a way to get the EmployeeAddress without even writing the second query, and just grabbing that associated data in the first employee_list query?
def employee_ajax_list(request):
email = request.GET.get('email', None)
employee_list = Employee.objects.filter(email=email)
employee_address = EmployeeAddress.objects.filter(employee_id__in=employee_list).values(
'street', 'city', 'state',)
# this chain kinda works, but splits them into 2 separate dictionaries?
employee_info = list(chain(employee_list.values(), employee_address))
data = {
'employee_list': employee_info
}
return JsonResponse(data)
Just looking on some advice to make this work a little smoother!
Change this line
employee = models.ForeignKey(Employee)
to
employee = models.ForeignKey(Employee, related_name="address")
That will let you access an employee's address by doing employee.address in regular code or employee__address in queries.
For example:
Employee.objects
.filter(email=email)
.exclude(address=None)
.values(
'email',
'name',
'address__street',
'address__city',
'address__state'
)
(The .exclude() clause is in case someone doesn't have an address set.)
That should output something like:
<QuerySet [{'email': 'johnsmith#example.com',
'name': 'John Smith', 'address__street':
'123 Maple St', 'address__city': 'Springfield',
'address__state': 'MO'}]>
Maybe something like this should be a bit better and in a single query:
employee_info = EmployeeAddress.objects.filter(employee__email=email).values("employee__email", "employee__<another_field>", "street", "city", "state")
data = {
'employee_list': employee_info
}
A best way of doing it should be with DRF serializers:
class EmployeeAddressSerializer(serializers.ModelSerializer):
class Meta:
fields = "__all__"
model = Employee
class EmployeeSerializer(serializers.ModelSerializer):
# you should add related_name="addresses" in the
# foreignkey if you want this works.
addresses = EmployeeAddressSerializer(many=True)
class Meta:
fields = "__all__"
model = Employee
Then to use:
employee = Employee.objects.filter(email=email).last()
return JsonResponse(EmployeeSerializer(employee).data)

Django DRF serializer Custom relational fields How to build return value of to_internal_value?

I'm trying to implement something very similar to Djang Rest Framework tutorial Custom relational fields.
For the reminder, the provided code snippet is:
import time
class TrackListingField(serializers.RelatedField):
def to_representation(self, value):
duration = time.strftime('%M:%S', time.gmtime(value.duration))
return 'Track %d: %s (%s)' % (value.order, value.name, duration)
class AlbumSerializer(serializers.ModelSerializer):
tracks = TrackListingField(many=True)
class Meta:
model = Album
fields = ['album_name', 'artist', 'tracks']
And "This custom field would then serialize to the following representation" (quoted in the tutorial):
{
'album_name': 'Sometimes I Wish We Were an Eagle',
'artist': 'Bill Callahan',
'tracks': [
'Track 1: Jim Cain (04:39)',
'Track 2: Eid Ma Clack Shaw (04:19)',
'Track 3: The Wind and the Dove (04:34)',
...
]
}
I understand that and have implemented it for my particular case.
What I don't understand is the way to implement to_internal_value(self, data) as I want to provide a read-write API.
I understand that to_internal_value(self, data) should return an AlbumTrack object, but I don't understand how to build it. In particular how to get back the Album related id?
If we post the JSON structure above, to_internal_value(self, data) will be called once per track with 'Track 1: Jim Cain (04:39)'... for data values. I don't see how we can update the tracks model from those data values.
It seems that you're trying to implement writable nested serializers. While nested serializers are read-only by default, the DRF has a section that explains how to implement writable ones: https://www.django-rest-framework.org/api-guide/relations/#writable-nested-serializers
Since you want the TrackListingField to serialize the Track model it should inherit from ModelSerializer:
class TrackSerializer(serializers.ModelSerializer):
class Meta:
model = Track
fields = ['order', 'name', 'duration']
You'll then have to override the create method for AlbumSerializer:
def create(self, validated_data):
tracks_data = validated_data.pop('tracks')
album = Album.objects.create(**validated_data)
for track_data in tracks_data:
Track.objects.create(album=album, **track_data)
return album
Please note that the above will make one database query per track. You can make use of Track.objects.bulk_create to make only one query to create all tracks.
To answer your initial question about to_internal_value, you can see what the default is by adding this print statement to the overridden to_internal_value:
class TrackSerializer(serializers.ModelSerializer):
...
def to_internal_value(self, data):
default_return_value = super(TrackSerializer, self).to_internal_value(data)
print(default_return_value)
return default_return_value
In the case of a ModelSerializer the DRF uses an OrderedDict output for to_internal_value. Your custom to_internal_value would have to extract the order, name and duration from the data string using a regex, and put them in an OrderedDict. However in this case it'd probably be easier to use a dictionary as representation for the tracks.

serializer.data is missing some of the data

Context: I am having problem accessing fields which are validated by nested serializers.
I have a very sample model as shown below.
For 2 of the fields I have their specific serializers. When I try to access the data it returns all the fields except the one validated by the specific serializers.
Models looks like this
class Sampler(models.Model):
sample_name = models.CharField(unique=True, max_length=100)
sampling_by = JSONField(max_length=100)
extractions = JSONField(max_length=100)
max_samples = models.IntegerField(default=100)
Serializers
class ExtractionsSerializer(serializers.BaseSerializer):
table_name = serializers.CharField()
extraction_type = serializers.ChoiceField(["ABC"])
dependencies = serializers.ListField(child=RecursiveField(), allow_empty=True, required=False)
class SamplingBySerializer(serializers.BaseSerializer):
"""
Validate sampling_by
"""
def to_internal_value(self, samples):
methods = ["ABC"]
sampling_not_supported = [sample for sample in samples
if sample['by'] not in methods]
if sampling_not_supported:
raise ValidationError("{} not in {}".format(sampling_not_supported, methods))
class SamplerSerializer(serializers.ModelSerializer):
extractions = ExtractionsSerializer(read_only=True)
sampling_by = SamplingBySerializer(read_only=True)
class Meta:
model = Sampler
fields = ('sample_name', 'database', 'schema', 'sampling_by', 'extractions', 'max_samples')
Now I do
data = {
"database": "postgresql://..",
"sampling_by":[{
"by":"ABC",
"value": ["l32=turn_the"]
}],
"max_samples":3,
"extractions" : [{
"table_name": "person",
"extraction_type": "ABC"
}]
}
sampler = SamplerSerializer(data=data)
sampler.is_valid() #returns True
sampler.data => does not contain data of the nested fields. Like the `sampling_by` and `extractions`. Contains all other fields
sampler.validated_data => same problem as above
Any help would be appreciated! thanks
You probably missed the fact that your nested serializers are flagged as read_only=True
class SamplerSerializer(serializers.ModelSerializer):
extractions = ExtractionsSerializer(read_only=True)
sampling_by = SamplingBySerializer(read_only=True)
Remove that part, implement the serializer's create / update and you're good to go.
On a side note, it doesn't make sense to access serializer.data when deserializing.
Edit: the authority source is validated_data.

django saving list of foreignkey objects to m2m field w/ through model with ordering

I have a html table w/ javascript that allows ordering of rows which passes the track_id list and row order list through a form on POST.
I just added the class PlaylistTrack using through for the model so I can add ordering to the tracks.m2m field. The view I have below works before I added the through model, but now I am not sure how I should save a list of tracks w/ its associated order number since I can't use add() and I must use create(). How should I use create() in my view to save a list of track_id's and associate the order number w/ list? Can I use bulk_create?
models.py:
class Playlist(models.Model):
user = models.ForeignKey(settings.AUTH_USER_MODEL, default=1)
name = models.CharField(max_length=50)
tracks = models.ManyToManyField(Track, through='PlaylistTrack')
def __str__(self):
return self.name
class PlaylistTrack(models.Model):
track = models.ForeignKey(Track)
playlist = models.ForeignKey(Playlist)
order = models.PositiveIntegerField()
class Meta:
ordering = ['order']
views.py:
def add_track(request):
active_playlist = Playlist.objects.get(id=request.POST.get('playlist_id'))
add_tracks = request.POST.getlist('track_id')
if request.POST.get('playlist_id'):
active_playlist.tracks.add(*add_tracks) # how to use create or bulk_create?
return redirect("playlists:list_playlists")
Ozgur's answer has you mostly covered. However, you do NOT need to fetch Playlist or Track instances from the database and you CAN use bulk_create:
def add_track(request):
playlist_id = request.POST.get('playlist_id')
track_ids = enumerate(request.POST.getlist('track_id'), start=1)
PlaylistTrack.objects.bulk_create([
PlaylistTrack(playlist_id=playlist_id, track_id=track_id, order=i)
for i, track_id in track_ids
])
return redirect("playlists:list_playlists")
This reduces the entire procedure to a single db operation where you had (1 + 2n) operations before (n being the number of tracks).
You need to fetch Track objects that will be added:
Track.objects.filter(pk__in=add_tracks)
--
Since order field must be populated, you can't use .add() on M2M field. You gotta create objects by yourself:
def add_track(request):
playlist = Playlist.objects.get(id=request.POST.get('playlist_id'))
for i, track_id in enumerate(request.POST.getlist('track_id'), start=1):
track = Track.objects.get(pk=track_id)
PlaylistTrack.objects.create(track=track, playlist=playlist, order=i)
return redirect("playlists:list_playlists")

How to write a django-rest-framework serializer / field to merge data from generic relations?

I have objects with a generic relation pointing to various other objects, and I need them to be merged (inlined) so the serialized objects look like one whole objects.
E.G:
class Enrollement(models.Model):
hq = models.ForeignKey(Hq)
enrollement_date = models.Datetime()
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
object = generic.GenericForeignKey('content_type', 'object_id')
class Nurse(models.Model):
hospital = models.ForeignKey(Hospital)
enrollement = GenericRelation(Enrollement)
class Pilot(models.Model):
plane = models.ForeignKey(plane)
enrollement = GenericRelation(Enrollement)
When serialized, I'd like to get something like this:
{
count: 50,
next: 'http...',
previous: null,
results: [
{
type: "nurse",
hq: 'http://url/to/hq-detail/view',
enrollement_date: '2003-01-01 01:01:01',
hospital: 'http://url/to/hospital-detail/view'
},
{
type: "pilot",
hq: 'http://url/to/hq-detail/view',
enrollement_date: '2003-01-01 01:01:01',
plante: 'http://url/to/plane-detail/view'
},
]
}
Can I do it, and if yes, how ?
I can nest a generic relation, and I could post process the serilizer.data to obtain what I want, but I'm wondering if there is a better way.
DEAR FRIENDS FROM THE FUTURE: At time of writing, the Django REST Framework team seems to be working on adding more mature support for generic relations. But it is not yet finished. Before copy-pasting this answer into your code base, check https://github.com/tomchristie/django-rest-framework/pull/755 first to see if it's been merged into the repo. There may be a more elegant solution awaiting you. — Your ancient ancestor Tyler
Given you're using Django REST Framework, if you did want to do some post-processing (even though you seem hesitant to) you can accomplish something your goal by overriding get_queryset or list in your view. Something like this:
views.py:
from rest_framework.generics import ListAPIView
from rest_framework.response import Response
from models import *
from itertools import chain
class ResultsList(ListAPIView):
def list(self, request, *args, **kwargs):
nurses = Nurse.objects.all()
pilots = Pilot.objects.all()
results = list()
entries = list(chain(nurses, pilots)) # combine the two querysets
for entry in entries:
type = entry.__class__.__name__.lower() # 'nurse', 'pilot'
if isinstance(entry, Nurse):
serializer = NurseSerializer(entry)
hospital = serializer.data['hospital']
enrollement_date = serializer.data['enrollement.date']
hq = serializer.data['enrollement.hq']
dictionary = {'type': type, 'hospital': hospital, 'hq': hq, 'enrollement_date': enrollement_date}
if isinstance(entry, Pilot):
serializer = PilotSerializer(entry)
plane = serializer.data['plane']
enrollement_date = serializer.data['enrollement.date']
hq = serializer.data['enrollement.hq']
dictionary = {'type': type, 'plane': plane, 'hq': hq, 'enrollement_date': enrollement_date}
results.append(dictionary)
return Response(results)
serializers.py
class EnrollementSerializer(serializer.ModelSerializer):
class Meta:
model = Enrollement
fields = ('hq', 'enrollement_date')
class NurseSerializer(serializer.ModelSerializer):
enrollement = EnrollementSerializer(source='enrollement.get')
class Meta:
model = Nurse
fields = ('hospital', 'enrollement')
class PilotSerializer(serializer.ModelSerializer):
enrollement = EnrollementSerializer(source='enrollement.get')
class Meta:
model = Pilot
fields = ('plane', 'enrollement')
Returned response would look like:
[
{
type: "nurse",
hq: "http://url/to/hq-detail/view",
enrollement_date: "2003-01-01 01:01:01",
hospital: "http://url/to/hospital-detail/view"
},
{
type: "pilot",
hq: "http://url/to/hq-detail/view",
enrollement_date: "2003-01-01 01:01:01",
plane: "http://url/to/plane-detail/view"
},
]
Noteworthy:
My serializers.py may be a bit off here because my memory of how to represent generic relations in serializers is a bit foggy. YMMV.
Similarly to ^^ this assumes your serializers.py is in order and has properly set up its generic relationships in line with your models.
We do the get in source=enrollement.get because otherwise a GenericRelatedObjectManager object will be returned if we don't specify a source. That's because that's what a generic relation represents. Using .get forces a query (as in QuerySet query) which accesses the model you set as the source of the generic relation (in this case, class Enrollement(models.Model).
We have to use list(chain()) instead of the | operator because the querysets come from different models. That's why we can't do entries = nurses | pilots.
for entry in entries can surely be made more dry. GLHF.