Django: including prefetched objects in values() dict - django

Question

I would like to create a dict or dict-like object based on a django orm query involving prefetched objects. Due to the number of objects involved, I would like to do as much of this in the database as possible.
To use the familiar example structure, if I have an Author and Books with the usual ForeignKey from Book to Author, and I also have an Edition with a ForeignKey back to Book, I would like to do something like
Author.objects.prefetch_related(Prefetch('book_set', queryset=Book.objects.filter(cycle__id=3).select_related('edition__binding'), to_attr="bindings")) # clearly this is wrong
and ultimately call .values() or something of the sort to get a dict-like result consisting of rows of authors, which should include an entry "bindings", which contains a list of bindings that each author has been published in. The stretch goal would be to have bindings be a semicolon-separated list, ie [{"Author": "Daniel Dennett", "bindings": "paper; hardback; cloth"}, {"Author": "Jemiah Jefferson", "bindings": "paper; zine"}]
So far, I have been able to get a field like "bindings" attached to the queryset using prefetch_related and select_related, as above, but this field is not included in the result of a call to .values(). This means I have to loop over the objects, which simply takes too long for my purposes (there are many objects, and the request times out)

Answer 1

Create custom a Concat annotation which will mimic MySQL GROUP_CONCAT function. Then you can use .values on the annotated bindings.
For Django 1.8 your Concat class can be something like this:
from django.db import models
class Concat(models.Aggregate):
# supports GROUP_CONCAT(DISTINCT field SEPARATOR str_val)
# do not support order_by
function = 'GROUP_CONCAT'
template = '%(function)s(%(distinct)s%(expressions)s SEPARATOR "%(separator)s")'
def __init__(self, expression, distinct=False, separator=None, **extra):
super(Concat, self).__init__(
expression,
distinct='DISTINCT ' if distinct else '',
separator=separator or '',
output_field=models.CharField(),
**extra)
While in Django 1.7
class Concat(models.Aggregate):
def add_to_query(self, query, alias, col, source, is_summary):
#we send source=CharField to prevent Django from casting string to int
aggregate = SQLConcat(col, source=models.CharField(), is_summary=is_summary, **self.extra)
query.aggregates[alias] = aggregate
#for mysql
class SQLConcat(models.sql.aggregates.Aggregate):
sql_function = 'group_concat'
#property
def sql_template(self):
if self.extra.get('separator'):
return '%(function)s(%(field)s SEPARATOR "%(separator)s")'
else:
return '%(function)s(%(field)s)'
Now you can do:
Author.objects.annotate(bindings=Concat('book__edition__binding')).values('name', 'bindings')
unfortunately this wont filter your books by cycle__id=3, but you can apply the filter before the annotation happens.
Author.objects.filter(book__cycle__id=3).annotate(bindings=Concat('book__edition__binding')).values('name', 'bindings')
This will strip from the result authors without book with cycle__id=3.