How can I prefetch nested tables? - django

Question

I'm working on an app with a DRF API. Development has been going on for some months, but only now are we running into performance issues when populating the database with actual data. I have done some profiling and found out that many endpoints query the database hundreds of times to fetch the necessary data. This is something that can be solved with select_related and prefetch_related, that much I know, but I'm having a hard time picturing it.
Models
class LegalFile(models.Model):
code = models.CharField(max_length=75)
description = models.CharField(max_length=255)
contactrole = models.ManyToManyField(LegalFileContactRole, blank=True, related_name='contactroles')
class LegalFileContactRole(models.Model):
contact = models.ForeignKey(Contact, on_delete=models.DO_NOTHING, blank=True, related_name='legal_file_contact')
subtype = models.ForeignKey(ContactSubType, on_delete=models.DO_NOTHING, related_name='legal_file_contact_role', null=True)
class Contact(models.Model):
name = models.CharField(max_length=150)
surname_1 = models.CharField(max_length=50,blank=True, null=True)
surname_2 = models.CharField(max_length=50,blank=True, null=True)
class ContactSubType(models.Model):
pass
Say I want to list all the LegalFiles in the database, and fetch the names and both surnames of the contacts associated to each LegalFile through LegalFileContactRole. Is there a DRYer way than using lowercase notation to prefetch like LegalFile.prefetch_related('contactrole__contact__name', 'contactrole__contact__surname_1', 'contactrole__contact__surname_2')? This kind of nested relationship is a recurring thing in the app.
Editing in response to Marc Compte's comment:
I have not explained myself properly, I think. In fact, this is missing a good chunk of context - I have not mentioned serializers at all, for instance.
I have taken the approach described in this post and created a method to set eager loading up:
class LegalFileReadSerializer(serializers.ModelSerializer):
contactrole = LegalFileContactRoleSerializer(many=True)
#classmethod
def setup_eager_loading(cls, queryset):
queryset = queryset.prefetch_related('contactrole__contact__name', 'contactrole__contact__surname_1', 'contactrole__contact__surname_2')
return queryset
class Meta:
model = LegalFile
fields = '__all__'
read_only_fields = ('__all__',)
class LegalFileViewSet(CustomViewSet):
model = models.LegalFile
read_serializer_class = serializers.LegalFileReadSerializer
write_serializer_class = serializers.LegalFileWriteSerializer
def get_queryset(self):
queryset = super().get_queryset()
queryset = self.get_serializer_class().setup_eager_loading(queryset)
return queryset
(Note this is still work in progress and will have to be decoupled further for reuse, in a mixin or something)
Could I, so to speak, chain serializers? Like:
class LegalFileReadSerializer(serializers.ModelSerializer):
contactrole = LegalFileContactRoleSerializer(many=True)
#classmethod
def setup_eager_loading(cls, queryset):
queryset = queryset.prefetch_related('contactrole',)
return queryset
class Meta:
model = LegalFile
fields = '__all__'
read_only_fields = ('__all__',)
class LegalFileContactRoleSerializer(serializers.ModelSerializer):
contact = ContactReadSerializer()
subtype = ContactSubTypeReadSerializer()
#classmethod
def setup_eager_loading(cls, queryset):
queryset = queryset.prefetch_related('contact',)
return queryset
class Meta:
model = models.LegalFileContactRole
fields = '__all__'
class ContactReadSerializer(serializers.ModelSerializer):
# And so on
pass
This, in my head at least, makes sense, but I don't know if this behavior is possible or even advisable. It seems that if I chain serializers like this (implying again that it's possible to do so) I'm gonna run into problems picking up too much unneeded information.