I'm trying to get all values in current table, and also get some fields in related tables.
class school(models.Model):
school_name = models.CharField(max_length=256)
school_type = models.CharField(max_length=128)
school_address = models.CharField(max_length=256)
class hometown(models.Model):
hometown_name = models.CharField(max_length=32)
class person(models.Model):
person_name = models.CharField(max_length=128)
person_id = models.CharField(max_length=128)
person_school = models.ForeignKey(school, on_delete=models.CASCADE)
person_ht = models.ForeignKey(hometown, on_delete=models.CASCADE)
how to quick select all info i needed into a dict for rendering.
there will be many records in person, i got school_id input, and want to get all person in this school, and also want these person's hometown_name shown.
i tried like this, can get the info i wanted. And any other quick way to do it?
m=person.objects.filter(person_school_id=1)
.values('id', 'person_name', 'person_id',
school_name=F('person_school__school_name'),
school_address=F('person_school__school_address'),
hometown_name=F('person_ht__hometown_name'))
person_name, person_id, school_name, school_address, hometown_name
if the person have many fields, it will be a hard work for list all values.
what i mean, is there any queryset can join related tables' fields together, which no need to list fields in values.
Maybe like this:
m=person.objects.filter(person_school_id=1).XXXX.values()
it can show all values in school, and all values in hometown together with person's values in m, and i can
for x in m:
print(x.school_name, x.hometown_name, x.person_name)
You add a prefetch_related query on top of your queryset.
prefetch_data = Prefetch('person_set, hometown_set, school_set', queryset=m)
Where prefetch_data will prepare your DB to fetch related tables and m is your original filtered query (so add this below your Person.objects.filter(... )
Then you do the actual query to the DB:
query = query.prefetch_related(prefetch_data)
Where query will be the actual resulting query with a list of Person objects (so add that line below the prefetch_data one).
Example:
m=person.objects.filter(person_school_id=1)
.values('id', 'person_name', 'person_id',
school_name=F('person_school__school_name'),
school_address=F('person_school__school_address'),
hometown_name=F('person_ht__hometown_name'))
prefetch_data = Prefetch('person_set, hometown_set, school_set', queryset=m)
query = query.prefetch_related(prefetch_data)
In that example I've broken down the queries into more manageable pieces, but you can do the whole thing in one big line too (less manageable to read though):
m=person.objects.filter(person_school_id=1)
.values('id', 'person_name', 'person_id',
school_name=F('person_school__school_name'),
school_address=F('person_school__school_address'),
hometown_name=F('person_ht__hometown_name')).prefetch_related('person, hometown, school')
Related
I have a simple Relation model, where a user can follow a tag just like stackoverflow.
class Relation(models.Model):
user = AutoOneToOneField(User)
follows_tag = models.ManyToManyField(Tag, blank=True, null=True, through='TagRelation')
class TagRelation(models.Model):
user = models.ForeignKey(Relation, on_delete=models.CASCADE)
following_tag = models.ForeignKey(Tag, on_delete=models.CASCADE)
pub_date = models.DateTimeField(default=timezone.now)
class Meta:
unique_together = ['user', 'following_tag']
Now, to get the results of all the tags a user is following:
kakar = CustomUser.objects.get(email="kakar#gmail.com")
tags_following = kakar.relation.follows_tag.all()
This is fine.
But, to access intermediate fields I have to go through a big list of other queries. Suppose I want to display when the user started following a tag, I will have to do something like this:
kakar = CustomUser.objects.get(email="kakar#gmail.com")
kakar_relation = Relation.objects.get(user=kakar)
t1 = kakar.relation.follows_tag.all()[0]
kakar_t1_relation = TagRelation.objects.get(user=kakar_relation, following_tag=t1)
kakar_t1_relation.pub_date
As you can see, just to get the date I have to go through so much query. Is this the only way to get intermediate values, or this can be optimized? Also, I am not sure if this model design is the way to go, so if you have any recomendation or advice I would be very grateful. Thank you.
You need to use Double underscore i.e. ( __ ) for ForeignKey lookup,
Like this :
user_tags = TagRelation.objects.filter(user__user__email="kakar#gmail.com").values("following_tag__name", "pub_date")
If you need the name of the tag, you can use following_tag__name in the query and if you need id you can use following_tag__id.
And for that you need to iterate through the result of above query set, like this:
for items in user_tags:
print items['following_tag__name']
print items['pub_date']
One more thing,The key word values will return a list of dictionaries and you can iterate it through above method and if you are using values_list in the place of values, it will return a list of tuples. Read further from here .
I have a table which looks like this
class Person(User):
"""
This model represents person's personal and
professional details.
"""
attribute1 = models.CharField(max_length=100)
attribute2 = models.TextField()
attribute3 = models.TextField()
attribute4 = models.ForeignKey(Receptionist)
referred_attribute1 = models.ManyToManyField(Hobby)
class Hobby(models.Model):
name = models.CharField(max_length=100)
Use case:
A Person P1 has Hobbies h1,h2,h3(defined[as 3 separate entries] in table Hobby).
Now I want to retrieve a Person object with all the attributes and properties that it has including Hobbies. For that I'm executing the following :
Person.objects.values("attribute1",
"attribute2",
"referred_attribute1").get(attribute3="p1's attribute")
What I want is :
{'attribute1':'p1_attribute1',
'attribute2':'p1_attribute2',
'referred_attribute':['h1','h2','h3']}
and what I get is:
[{'attribute1':'p1_attribute1',
'attribute2':'p1_attribute2',
'referred_attribute':'h1'},
{'attribute1':'p1_attribute1',
'attribute2':'p1_attribute2',
'referred_attribute':'h2'},
{'attribute1':'p1_attribute1',
'attribute2':'p1_attribute2',
'referred_attribute':'h3'}]
Is there a way to get my desired result directly from a queryset? As I do not want to manually re-arrange the above result!
Do this instead:
p1 = Person.objects.get(attribute3="p1's attribute")
p1 is the instance of the person, you can then retrieve all your attributes from it. eg:
p1_attribute1 = p1.attribute1
p1_hobbies = p1.referred_attribute1.all()
Now you can rearrange that data however you like, for example, making a list of the hobbies as follows:
hobby_list = p1_hobbies.values_list('name', flat=True)
I would rename referred_attribute1 to hobbies for code readability
Something like this should work in this case:
[{'attribute1': res.attribute1, 'attribute2': res.attribute2,
'referred_attribute1': [a.name for a in res.referred_attribute1.all()]}
for res in Person.objects.prefetch_related('referred_attribute1').filter(attribute3="p1's attribute")]
Also see Django's Prefetch-related
class Book(models.Model):
name = models.CharField(max_length=127, blank=False)
class Author(models.Model):
name = models.CharField(max_length=127, blank=False)
books = models.ManyToMany(Books)
I am trying to filter the authors so I can return a result set of authors like:
[{id: 1, name: 'Grisham', books : [{name: 'The Client'},{name: 'The Street Lawyer}], ..]
Before I had the m2m relationship on author I was able to query for any number of author records and get all of the values I needed using the values method with only one db query.
But it looks like
Author.objects.all().values('name', 'books')
would return something like:
[{id: 1, name: 'Grisham', books :{name: 'The Client'}},{id: 1, name: 'Grisham', books :{name: 'The Street Lawyer'}}]
Looking at the docs it doesn't look like that is possible with the values method.
https://docs.djangoproject.com/en/dev/ref/models/querysets/
Warning Because ManyToManyField attributes and reverse relations can
have multiple related rows, including these can have a multiplier
effect on the size of your result set. This will be especially
pronounced if you include multiple such fields in your values() query,
in which case all possible combinations will be returned.
I want to try to get a result set of n size with with the least amount of database hits authorObject.books.all() would result in at least n db hits.
Is there a way to do this in django?
I think one way of doing this with the least amount of database hits would be to :
authors = Authors.objects.all().values('id')
q = Q()
for id in authors:
q = q | Q(author__id = id)
#m2m author book table.. from my understanding it is
#not accessible in the django QuerySet
author_author_books.filter(q) #grab all of the book ids and author ids with one query
Is there a built in way to query the m2m author_author_books table or am I going to have the write the sql? Is there a way to take advantage of the Q() for doing OR logic in raw sql?
Thanks in advance.
I think you want prefetch_related. Something like this:
authors = Author.objects.prefetch_related('books').all()
More on this here.
If you want to query your author_author_books table, I think you need to specify a "through" table:
class BookAuthor(models.Model):
book = models.ForeignKey(Book)
author = models.ForeignKey(Author)
class Author(models.Model):
name = models.CharField(max_length=127, blank=False)
books = models.ManyToMany(Books, through=BookAuthor)
and then you can query BookAuthor like any other model.
How can I make an select with two (or more) joined tables using Django Models?
For example:
class Album(models.Model):
artist = models.ForeignKey(Musician)
name = models.CharField(max_length=100)
release_date = models.DateField()
num_stars = models.IntegerField()
class Song(models.Model):
album = models.ForeignKey(Album)
name = models.CharField(max_length=100)
num_stars = models.IntegerField()
SELECT * from album, song where (album.id = song.album_id) and (album.artist_id = X)
Django querysets want to return lists of model instances, which doesn't quite map to the the SQL notion of returning data from multiple tables joined into a single result table.
To achieve the effect of the SQL query you provide as an example, you could do one of the following:
1) If you want to iterate over your data by Songs, you can limit the songs you query by the artist id, like so:
songs = Song.objects.filter(album__artist__id=123)
for song in songs:
print song.name
print song.album.name
... do what ever you need to do with song here ...
If you are concerned about performance, you can add select_related() to your queryset.
# the depth argument tells django to only follow foreign key relationships
# from Song one level deep
songs = Song.objects.select_related(depth=1).filter(album__artist__id=123)
for song in songs:
# django has now already fetched the album data, so accessing it
# from the song instance does not cause an additional database hit
print song.album.name
2) If you want to iterate over your data by Album, you can do things like so:
albums = Album.objects.filter(artist__id = 123)
for album in albums:
for song in album.song_set.all():
print song.name
print album.name
... etc ...
You can perform raw sql query, but that's probably not the thing you want to do. Explain how your Models look like and what you want to achieve and probably django ORM will be enough to perform the query.
Previously had a go at asking a more specific version of this question, but had trouble articulating what my question was. On reflection that made me doubt if my chosen solution was correct for the problem, so this time I will explain the problem and ask if a) I am on the right track and b) if there is a way around my current brick wall.
I am currently building a web interface to enable an existing database to be interrogated by (a small number of) users. Sticking with the analogy from the docs, I have models that look something like this:
class Musician(models.Model):
first_name = models.CharField(max_length=50)
last_name = models.CharField(max_length=50)
dob = models.DateField()
class Album(models.Model):
artist = models.ForeignKey(Musician)
name = models.CharField(max_length=100)
class Instrument(models.Model):
artist = models.ForeignKey(Musician)
name = models.CharField(max_length=100)
Where I have one central table (Musician) and several tables of associated data that are related by either ForeignKey or OneToOneFields. Users interact with the database by creating filtering criteria to select a subset of Musicians based on data the data on the main or related tables. Likewise, the users can then select what piece of data is used to rank results that are presented to them. The results are then viewed initially as a 2 dimensional table with a single row per Musician with selected data fields (or aggregates) in each column.
To give you some idea of scale, the database has ~5,000 Musicians with around 20 fields of related data.
Up to here is fine and I have a working implementation. However, it is important that I have the ability for a given user to upload there own annotation data sets (more than one) and then filter and order on these in the same way they can with the existing data.
The way I had tried to do this was to add the models:
class UserDataSets(models.Model):
user = models.ForeignKey(User)
name = models.CharField(max_length=100)
description = models.CharField(max_length=64)
results = models.ManyToManyField(Musician, through='UserData')
class UserData(models.Model):
artist = models.ForeignKey(Musician)
dataset = models.ForeignKey(UserDataSets)
score = models.IntegerField()
class Meta:
unique_together = (("artist", "dataset"),)
I have a simple upload mechanism enabling users to upload a data set file that consists of 1 to 1 relationship between a Musician and their "score". Within a given user dataset each artist will be unique, but different datasets are independent from each other and will often contain entries for the same musician.
This worked fine for displaying the data, starting from a given artist I can do something like this:
artist = Musician.objects.get(pk=1)
dataset = UserDataSets.objects.get(pk=5)
print artist.userdata_set.get(dataset=dataset.pk)
However, this approach fell over when I came to implement the filtering and ordering of query set of musicians based on the data contained in a single user data set. For example, I could easily order the query set based on all of the data in the UserData table like this:
artists = Musician.objects.all().order_by(userdata__score)
But that does not help me order by the results of a given single user dataset. Likewise I need to be able to filter the query set based on the "scores" from different user data sets (eg find all musicians with a score > 5 in dataset1 and < 2 in dataset2).
Is there a way of doing this, or am I going about the whole thing wrong?
edit: nevermind, it's wrong. I'll keep it so you can read, but then I'll delete afterward.
Hi,
If I understand correctly, you can try something like this:
artists = Musician.objects.select_related('UserDataSets').filter( Q(userdata__score_gt=5, userdata__id=1) | Q(userdata__sorce_lt=2, userdata__id=2 )
For more info on how to use Q, check this: Complex lookups with Q objects.