Django Queryset Prefetch Optimization for iterating over nested results - django

Question

I'm looking for a way to optimize a queryset result processing in Django by improving database access performance, taking into consideration that I need to fetch a nested relation.
Taking these models as example:
class Movie(models.Model):
name = models.CharField(max_length=50)
class Ticket(models.Model):
code = models.CharField(max_length=255, blank=True, unique=True)
movie = models.ForeignKey(Movie, related_name='tickets')
class Buyer(models.Model):
name = models.CharField(max_length=50)
class Purchase(models.Model):
tickets = models.ManyToManyField(Ticket, related_name='purchases')
buyer = models.ForeignKey(Buyer, related_name='purchases')
and this Movie QuerySet:
movies = Movie.objects.all().prefetch_related('tickets__purchases__buyer')
In case I need to retrieve all buyers from each Movie in the above QuerySet, this is one approach:
for movie in movies:
buyers = Buyer.objects.filter(purchases__tickets__in=movie.tickets.all()).distinct()
But that will hit the database once for each Movie iterated. So to this in a single transaction, I'm doing something like:
def get_movie_buyers(movie):
buyers = set()
for ticket in movie.tickets.all():
for purchase in ticket.purchases.all():
if purchase.buyer:
buyers.add(purchase.buyer)
return buyers
for movie in movies:
buyers = get_movie_buyers(movie)
This approach hits the database once due to the prefetch_related in the QuerySet, but it doesn't look optimal as I'm iterating over many nested loops, which will then increase application memory overload instead.
There might be a better approach that I couldn't figure out yet, looking for some guidance.
UPDATE
alasdair suggested to use Prefetch object, tried that:
movies = Movie.objects.prefetch_related(
Prefetch(lookup='tickets__purchases__buyer',
to_attr='buyers')
).all()
for movie in movies:
print movie.buyers
But this gives me the following error:
'Movie' object has no attribute 'buyers'

Answer 1

The reason why it seems too difficult is the ManyToMany relation between Purchase and Tickets.
This relation allows the same ticket to be present in multiple purchases. But this will not be the case in actual data as one ticket can be purchased only once.
The query can be simplified if you remove this ManyToMany field and add a ForeignKey field in Ticket to purchase
class Ticket(models.Model):
code = models.CharField(max_length=255, blank=True, unique=True)
movie = models.ForeignKey(Movie, related_name='tickets')
purchase = models.ForeignKey(Purchase, null=True, blank=True)
Then the query can be simplified as below.
movies = Movie.objects.all().prefetch_related('tickets__purchase__buyer')
for movie in movies:
print(set(ticket.purchase.buyer for ticket in movie.tickets if ticket.purchase))
Sure this will create additional complexity while creating Purchases as you need to update the ticket objects with purchase_id.
You need to make a call on where to keep complexity based on the frequency of both actions