django query aggregation grouping with access to all fields - django

With models defined like so:
class Athlete(models.Model):
name = models.CharField()
class Event(models.Model):
winner = models.ForeignKey(Athlete)
distance = models.FloatField()
type_choices = [('LJ', 'Long Jump'), ('HJ', 'High Jump')]
type = models.CharField(choices=type_choices)
I want to run a query picking out all the events an Athlete has won, grouped by type. I'm currently doing it like so:
athlete = Athlete.objects.get(name='dave')
events_by_type = Events.objects.values('type').annotate(Count('winner')).filter(winner=athlete)
This gives me a dictionary of event types (short versions) and the number of times the athlete has been the winner. However that's all it gives me. If I then want to dig into one of these events to find the distance or even just the verbose type name, I can't.
Am I going about this in the right way? How can I get the events grouped, but also with access to all their fields as well?

You are not getting the event instances because you are querying Event.objects with the values method. This will provide you the data for only the specified fields:
https://docs.djangoproject.com/en/dev/ref/models/querysets/#values
Performing this kind of group by with the Django ORM is not straightforward. The proposed solution often is:
q = Events.objects.filter(winner=athlete)
q.query.group_by = ['type']
q.count()
But I'd rather do it with straight python. Maybe something like
athlete = Athlete.objects.get(name='dave')
events = Events.objects.filter(winner=athlete)
won_per_type = defaultdict(list)
for e in events:
won_per_type(e.type).append(e)

Related

Django filter exact value in list

Good day again SO. I was hoping you can help me with some of the logic.
Based on this SO Answer, I can filter the search with a list which works perfectly. However, I wish to get an EXACT id instead of at least one matches.
models:
class Condition:
condition_name = models.CharField(....)
class Jobs:
jobs = models.CharField(...)
class JobsConditions:
account = models.ForeignKey(Account...)
job_item = models.ForeignKey(Jobs...)
condition = models.ForeignKey(Condition...)
So if I try to search for Jobs with Conditions, I do the following:
cond_array = [1,2,4,5] # Append to array based on request.
condition_obj = Condition.objects.filter(id__in=cond_array)
Then compare condition_obj to JobsConditions model. How to use this so that I will only get only the jobs with exact condition? No more no less.
I think you're wanting something like this:
Filter JobsConditions by condition__id and get the associated job_item__jobs as a list:
jobs_list = (JobsConditions.objects
.filter(condition__id__in=cond_array)
.values_list('job_item__jobs', flat=True))
Filter Jobs by that jobs_list:
jobs = Jobs.objects.filter(jobs__in=jobs_list)

Giving relations an order to sort by

Given the following Django models:
class Room(models.Model):
name = models.CharField(max_length=20)
class Beacon(models.Model):
room = models.ForeignKey(Room)
uuid = models.UUIDField(default=uuid.uuid4)
major = models.PostiveIntegerField(max_value=65536)
minor = models.PositiveIntegerField(max_value=65536)
The Beacon model is a bluetooth beacon relationship to the room.
I want to select all Rooms that match a given uuid, major, minor combination.
The catch is, that I want to order the rooms by the beacon that is nearest to me. Because of this, I need to be able to assign a value to each beacon dynamically, and then sort by it.
Is this possible with the Django ORM? In Django 1.8?
NOTE - I will know the ordering of the beacons beforehand, I will be using the order they are passed in the query string. So the first beacon (uuid, major, minor) passed should match the first room that is returned by the Room QuerySet
I am envisioning something like this, though I know this won't work:
beacon_order = [
beacon1 = 1,
beacon0 = 2,
beacon3 = 3,
]
queryset = Room.objects.annotate(beacon_order=beacon_order).\
order_by('beacon_order')
If you already know the order of the beacons, there's no need to sort within the QuerySet itself. Take an ordered list called beacon_list, which contains the beacons' primary keys in order, e.g. the item at index 0 is the closest beacon's primary key, the item at index 1 is the second closest beacon's PK, etc. Then use a list comprehension:
ordered_rooms = [Room.objects.get(pk=x) for x in beacon_list]
You don't have to use the PK either, you can use anything which identifies the given object in the database, e.g. the name field.
Looks like this works:
from django.db.models import Case, Q, When
beacons = request.query_params.getlist('beacon[]')
query = Q()
order = []
for pos, beacon in enumerate(beacons):
uuid, major, minor = beacon.split(':')
query |= Q(
beacon__uuid=uuid,
beacon__major=major,
beacon__minor=minor,
)
order.append(When(
beacon__uuid=uuid,
beacon__major=major,
beacon__minor=minor,
then=pos,
))
rooms = Room.objects.filter(query).order_by(Case(*order))

Django queryset - Group on basis of foreign key and get count

I have 3 tables as follows:
class Bike:
name = CharField(...)
cc_range = IntField(...)
class Item:
bike_number = CharField(...)
bike = ForeignKey(Bike)
class Booking:
start_time = DateTimeField(...)
end_time = DateTimeField(...)
item = ForeignKey(Item, related_name='bookings')
I want to get a list of all the bikes which are not booked during a period of time (say, ["2016-01-09", "2016-01-11"]) with an item count with them.
For example, say there are two bikes b1, b2 with items i11, i12 and i21, i22. If i21 is involved in a booking (say ["2016-01-10", "2016-01-12"]) then I want something like
{"b1": 2, "b2": 1}
I have got the relevant items by
Item.objects
.exclude(bookings__booking_time__range=booking_period)
.exclude(bookings__completion_time__range=booking_period)
but am not able to group them.
I also tried:
Bike.objects
.exclude(item__bookings__booking_time__range=booking_period)
.exclude(item__bookings__completion_time__range=booking_period)
.annotate(items_count=Count('item')
But it removes the whole bike if any of it's item is booked.
I seem to be totally stuck. I would prefer doing this without using a for loop. The django documentation also don't seem to help me out (which is something rare). Is there a problem with my model architecture for the type of problem I want to solve. Or am I missing something out. Any help would be appreciated.
Thanks in advance !!
from django.db.models import Q, Count, Case, When, Value, BooleanField
bikes = models.Bike.objects.annotate(
booked=Case(
When(Q(item__bookings__start_time__lte=booking_period[1]) & Q(item__bookings__end_time__gte=booking_period[0]),
then=Value(True)),
default=Value(False),
output_field=BooleanField(),
)).filter(booked=False).annotate(item_count=Count('item'))
Please read the documentation about conditional expressions.

Django complex query without using loop

I have two models such that
class Employer(models.Model):
name = models.CharField(max_length=1000,null=False,blank=False)
eminence = models.IntegerField(null=False,default=4)
class JobTitle(models.Model):
name = models.CharField(max_length=1000,null=False,blank=False)
employer= models.ForeignKey(JobTitle,unique=False,null=False)
class People(models.Model):
name = models.CharField(max_length=1000,null=False,blank=False)
jobtitle = models.ForeignKey(JobTitle,unique=False,null=False)
I would like to list random 5 employers and one job title for each employer. However, job title should be picked up from first 10 jobtitles of the employer whose number of people is maximum.
One approach could be
employers = Employer.objects.filter(isActive=True).filter(eminence__lt=4 ).order_by('?')[:5]
for emp in employers:
jobtitle = JobTitle.objects.filter(employer=emp)... and so on.
However, loop through selected employers may be ineffiecent. Is there any way to do it in one query ?
Thanks
There is! Check out: https://docs.djangoproject.com/en/dev/ref/models/querysets/#select-related
select_related() tells Django to follow all the foreign key relationships using JOINs. This will result in one large query as opposed to many small queries, which in most cases is what you want. The QuerySet you get will be pre-populated and Django won't have to lazy-load anything from the database.
I've used select_related() in the past to solve almost this exact problem.
I have written such code block and it works. Although I loop over employers because I have used select_related('jobtitle'), I consider it doesn't hit database and works faster.
employers = random.sample(Employer.objects.select_related('jobtitle').filter(eminence__lt=4,status=EmployerStatus.ACTIVE).annotate(jtt_count=Count('jobtitle')).filter(jtt_count__gt=0),3)
jtList = []
for emp in employers:
jt = random.choice(emp.jobtitle_set.filter(isActive=True).annotate(people_count=Count('people')).filter(people_count__gt=0)[:10])
jtList.append(jt)

Django ORM - select_related and order_by with foreign keys

I have a simple music schema: Artist, Release, Track, and Song. The first 3 are all logical constructs while the fourth (Song) is a specific instance of an (Artist, Release, Track) as an mp3, wav, ogg, whatever.
I am having trouble generating an ordered list of the Songs in the database. The catch is that both Track and Release have an Artist. While Song.Track.Artist is always the performer name, Song.Track.Release.Artist may either be a performer name or "Various Artists" for compilations. I want to be able to sort by one or the other, and I can't figure out the correct way to make this work.
Here's my schema:
class Artist(models.Model):
name = models.CharField(max_length=512)
class Release(models.Model):
name = models.CharField(max_length=512)
artist = models.ForeignKey(Artist)
class Track(models.Model):
name = models.CharField(max_length=512)
track_number = models.IntegerField('Position of the track on its release')
length = models.IntegerField('Length of the song in seconds')
artist = models.ForeignKey(Artist)
release = models.ForeignKey(Release)
class Song(models.Model):
bitrate = models.IntegerField('Bitrate of the song in kbps')
location = models.CharField('Permanent storage location of the file', max_length=1024)
owner = models.ForeignKey(User)
track = models.ForeignKey(Track)
My query should be fairly simple; filter for all songs owned by a specific user, and then sort them by either Song.Track.Artist.name or Song.Track.Release.Artist.name. Here's my code inside a view, which is sorting by Song.Track.Artist.name:
songs = Song.objects.filter(owner=request.user).select_related('track__artist', 'track__release', 'track__release__artist').order_by('player_artist.name')
I can't get order_by to work unless I use tblname.colname. I took a look at the underlying query object's as_sql method, which indicates that when the inner join is made to get Song.Track.Release.Artist the temporary name T6 is used for the Artist table since an inner join was already done on this same table to get Song.Track.Artist:
>>> songs = Song.objects.filter(owner=request.user).select_related('track__artist', 'track__release', 'track__release__artist').order_by('T6.name')
>>> print songs.query.as_sql()
('SELECT "player_song"."id", "player_song"."bitrate", "player_song"."location",
"player_song"."owner_id", "player_song"."track_id", "player_track"."id",
"player_track"."name", "player_track"."track_number", "player_track"."length",
"player_track"."artist_id", "player_track"."release_id", "player_artist"."id",
"player_artist"."name", "player_release"."id", "player_release"."name",
"player_release"."artist_id", T6."id", T6."name" FROM "player_song" INNER JOIN
"player_track" ON ("player_song"."track_id" = "player_track"."id") INNER JOIN
"player_artist" ON ("player_track"."artist_id" = "player_artist"."id") INNER JOIN
"player_release" ON ("player_track"."release_id" = "player_release"."id") INNER JOIN
"player_artist" T6 ON ("player_release"."artist_id" = T6."id") WHERE
"player_song"."owner_id" = %s ORDER BY T6.name ASC', (1,))
When I put this as the table name in order_by it does work (see example output above), but this seems entirely non-portable. Surely there's a better way to do this! What am I missing?
I'm afraid I really can't understand what your question is.
A couple of corrections: select_related has nothing to do with ordering (it doesn't change the queryset at all, just follows joins to get related objects and cache them); and to order by a field in a related model you use the double-underscore notation, not dotted. For example:
Song.objects.filter(owner=request.user).order_by('track__artist__name')
But in your example, you use 'player_artist', which doesn't seem to be a field anywhere in your model. And I don't understand your reference to portability.