Consider the following condition:
class Book(models.Model):
name = models.CharField(max_length=300)
price = models.IntegerField(default=0)
def __str__(self):
return self.name
class Store(models.Model):
name = models.CharField(max_length=300)
default = models.BooleanField(default=False)
books = models.ForeignKey(Book, on_delete=models.CASCADE)
So, for this query:
Book.objects.prefetch_related(Prefetch('store_set',
queryset=Store.objects.filter(default=False)))
.values("store__name", "name", "store__default")
The SQL query is not considering queryset default=True condition
SELECT "core_store"."name",
"core_book"."name",
"core_store"."default"
FROM "core_book"
LEFT OUTER JOIN "core_store"
ON ("core_book"."id" = "core_store"."books_id")
Result:
<QuerySet [{'store__name': 'Subway Store', 'name': 'Hello', 'store__default': False},
{'store__name': 'Times Square', 'name': 'Hello', 'store__default': False},
{'store__name': 'Subway Store', 'name': 'GoodBye', 'store__default': True},
{'store__name': 'Times Square', 'name': 'GoodBye', 'store__default': False},
{'store__name': 'Subway Store', 'name': 'Greetings', 'store__default': True},
{'store__name': 'Subway Store', 'name': 'HateWords', 'store__default': False}]>
I want to have a query set condition while prefetching the query. I am not able to find any way to do it in one query or a minimum number of queries.
I was thinking it should make a where condition with the OUTER JOIN with core_store table. Here
LEFT OUTER JOIN "core_store"
ON ("core_book"."id" = "core_store"."books_id")
If you want only the values as stated in your question then you can directly use fk relationship from the Store model to Book model as below. No need to do reverse lookup.
Store.objects.filter(default=True).values('name', 'default', 'books__name')
But if u want to fetch all Books and also their corresponding Store then you can use only on prefetch_related. .values does not work with prefetch_related as stated in the docs. You can also use a for loop after assigning a to_attr in your prefetch_related query. The for loop does not do a db query. Instead it gets the value from prefetched results.
books_stores = Book.objects.prefetch_related(Prefetch('store_set',
queryset=Store.objects.filter(default=False),
to_attr='stores'))
for book_store in book_stores:
stores = book_store.stores
Related
Lets imagine we have two models (many-to-one model).
Code below shows that a reporter can have multiple articles
class Reporter(models.Model):
first_name = models.CharField(max_length=30)
last_name = models.CharField(max_length=30)
email = models.EmailField()
def __str__(self):
return "%s %s" % (self.first_name, self.last_name)
class Article(models.Model):
headline = models.CharField(max_length=100)
pub_date = models.DateField(null=True)
reporter = models.ForeignKey(Reporter, on_delete=models.CASCADE, null=True)
def __str__(self):
return self.headline
Let's see what I have in my database on this model.
# Reporter.objects.all().values()
# <QuerySet [
# {'id': 1, 'first_name': 'John', 'last_name': 'Smith', 'email': 'john#example.com'},
# {'id': 2, 'first_name': 'Paul', 'last_name': 'Jones', 'email': 'paul#example.com'}
# ]>
# Article.objects.all().values()
# <QuerySet [
# {'id': 5, 'headline': "1st headline", 'pub_date': datetime.date(2005, 7, 29),
# 'reporter_id': 1},
# {'id': 6, 'headline': "2nd headline", 'pub_date': datetime.date(2006, 1, 17),
# 'reporter_id': 2},
# {'id': 7, 'headline': '3rd headline', 'pub_date': datetime.date(2005, 7, 27),
# 'reporter_id': 1}
# ]>
The first reporter has two publications and second has the only.
I need to get the list of all articles for each reporter.
I tried this way (according to django docs):
Article.objects.filter(reporter__first_name='John')
It's okay. It works. I also tried to instantiate the first reporter as 'r1' and then do this:
r1.article_set.all()
And this piece of code works too.
But as I'm new to django, I think that instantiating the first reporter as 'r1' and then making a query is a bit slow. It is because django makes me run r1.save() and then r1.article_set.all(). It looks like django makes 2 query into database (first query - to save an instance, the second query to run r1.article_set.all)
Is my point of view correct? And how to query all the reporter's articles as fast as Article.objects.filter(reporter__first_name='John') but using the Reporter object?
Thanks
I also tried to instantiate the first reporter as 'r1' and then do this:
r1.article_set.all()
And this piece of code works too. But as I'm new to django, I think that instantiating the first reporter as 'r1' and then making a query is a bit slow.
Yes, but Django can load the related articles all with a second query in bulk. We do this with .prefetch_related(…) [Django-doc]:
reporters = Reporter.objects.prefetch_related('article_set')
for reporter in reporters:
print(reporter.first_name)
print(reporter.article_set.all())
Instead of the N+1 queries that your implementations make (one query to fetch all reporters, and one query per reporter to fetch the related articles), this will make two queries: one to fetch all the reporters, and one to fetch all the articles related to one of these reporters. Django will then do some joining such that the articles related to the first reporter r1 end up in r1.article_set.all()
Despite having looked everywhere for similar issues I still cannot make the query working using INNER JOIN with the Django ORM... Sorry if this might sound stupid, but this is my first time with Django on a project and especially the ORM.
I have an Articles table with a Users table (named Fellows in my case), the Articles table has it's foreign key on author and references the user_id in Fellows table.
class Fellow(models.Model):
id = models.AutoField(db_column='ID', primary_key=True) # ID
user_id = models.PositiveBigIntegerField(db_column='User_ID', unique=True) # Global User ID.
nickname = models.CharField(db_column='Name', max_length=64, db_collation='utf8mb4_general_ci') # Display Name
user_password = models.CharField(db_column='User_Password', max_length=256, blank=True, null=True) # Passwd
gold = models.IntegerField(db_column='Gold') # Credits
faction = models.ForeignKey('Faction', models.RESTRICT, db_column='Faction', default=1) # ID Faction
class Meta:
managed = False
db_table = 'Fellows'
def __str__(self):
return self.nickname # Test.
class Article(models.Model):
id = models.AutoField(db_column='ID', primary_key=True) # ID
author = models.ForeignKey('Fellow', models.CASCADE, db_column='ID_User', default=1) # Global User ID
title = models.CharField(db_column='Title', max_length=32) # Title
content = models.TextField(db_column='Content') # Content
posted = models.DateTimeField(db_column='Posted') # Date Posted
source = models.CharField(db_column='Source', max_length=64, blank=True, null=True) # Source picture url of the article.
class Meta:
db_table = 'Articles'
I tried to get the display name of the related author that posted the article without success.
This is my views.py:
from .models import Article
def index(request):
"""
And then Vue.JS will take care of the rest.
"""
# articles = Article.objects.order_by('-posted')[:5] # Returns everything inside Articles table but nothing inside Fellows table.
# articles = Article.objects.select_related() # No Result.
# Still can't get display_name in index.html with this one.
articles = Article.objects.raw('SELECT Fellows.Name AS Display_Name, Articles.ID, Articles.Title, Articles.Content, Articles.Posted, Articles.Source FROM Articles INNER JOIN Fellows ON Fellows.User_ID = Articles.ID_User ORDER BY Articles.ID DESC LIMIT 5;')
data = {
'articles': articles,
}
return render(request, 'home/index.html', data)
The raw request returns everything fine only with sql interpreter, so there is two options:
Django won't perform the INNER JOIN.
I didn't figured out how to read the Display_Name in the template (index.html).
This is how I retrieve the data using VueJS (even with the raw query I can't get the display_name, it's empty).
<script>
const store = new Vuex.Store({
state: {
articles: [
{% for article in articles %}
{
title: '{{ article.title }}',
content: '{{ article.content | linebreaksbr }}',
source: "{% static 'home/img/' %}" + '{{article.source}}',
display_name: '{{article.display_name}}', // Maybe this is not how to retrieve the display_name?
},
{% endfor %}
],
},
});
// Components.
ArticleList = Vue.component('article-list', {
data: function () { return { articles: store.state.articles } },
template: '#article-list-template',
});
ArticleItem = Vue.component('article-item', {
delimiters: ['[[', ']]'],
props: ['id', 'title', 'content', 'source', 'display_name'],
template: '#article-item-template',
});
...
</script>
if someone could help me with this I would appreciate immensely! TT
Problem solved,
I had to change the foreign key constraint Articles.ID_User which now leads to Fellows.ID.
Previously the constraint led to Fellows.User_ID.
I can finally use:
articles = Article.objects.select_related('author').order_by('-posted')[:5]
And indeed finally accessing it in the front by article.author, simple as that.
Yet I still don't really understand why the raw sql query (using the mysql interpreter) with the INNER JOIN worked fine tho when referencing Fellows.User_ID, which was apparently not the case in the ORM.
Although it is working, my sql relational might be wrong or not ideal, therefore I am still open to suggestions!
class Groups:
name = models.CharField(max_length=100)
class User:
firstname = models.CharField(max_length=100)
lastname = models.CharField(max_length=100)
group = models.ManyToManyField(Groups)
i need user details with groupname as list not as separate records.
User.objects.values('firstname','lastname', 'group__name')
While i'm querying like above, i'm getting like this
<QuerySet [{'firstname': 'Market', 'lastname': 'Cloud', 'group__name':
'Group 5'}, {'firstname': 'Market', 'lastname': 'Cloud',
'group__name': 'Group 4'}]>
but i want like this
<QuerySet [{'firstname': 'Market', 'lastname': 'Cloud', 'group__name':
['Group 5', 'Group 4']}]>
is there a way, i can query like this without doing separate query.
If you're using postgres you can use ARRAY_AGG function. In Django ORM like below:
from django.contrib.postgres.aggregates import ArrayAgg
User.objects \
.annotate(list_of_group_names=ArrayAgg('group__name')) \
.order_by('id').distinct() \
.values('firstname', 'lastname', 'list_of_group_names')
Note: distinct is useful, because joining tables can result in duplicates
I need to filter all Experts by past objectives.
I have a minimal runnable example at https://github.com/morenoh149/django-rest-datatables-relations-example (btw there are fixtures you can load with test data).
my models are
class Expert(models.Model):
name = models.CharField(blank=True, max_length=300)
class Meeting(models.Model):
user = models.ForeignKey(User, on_delete=models.SET_NULL, blank=True, null=True)
expert = models.ForeignKey(Expert, on_delete=models.SET_NULL, blank=True, null=True)
objective = models.TextField(null=True, blank=True)
my datatables javascript
$("#table-analyst-search").DataTable({
serverSide: true,
ajax: "/api/experts/?format=datatables",
ordering: false,
pagingType: "full_numbers",
responsive: true,
columns: [
{
data: "objectives",
name: "objectives",
visible: false,
searchable: true,
render: (objectives, type, row, meta) => {
return objectives;
}
},
],
});
My serializer
class ExpertSerializer(serializers.ModelSerializer):
id = serializers.IntegerField(read_only=True)
objectives = serializers.SerializerMethodField()
class Meta:
model = Expert
fields = (
"id",
"objectives",
)
def get_objectives(self, obj):
request = self.context["request"]
request = self.context["request"]
meetings = Meeting.objects.filter(
analyst_id=request.user.id, expert_id=obj.id
).distinct('objective')
if len(meetings) > 0:
objectives = meetings.values_list("objective", flat=True)
objectives = [x for x in objectives if x]
else:
objectives = []
return objectives
When I begin to type in the datatables.js searchbar I get an error like
FieldError at /api/experts/
Cannot resolve keyword 'objectives' into field. Choices are: bio, company, company_id, created_at, description, email, favoriteexpert, first_name, id, is_blocked, last_name, meeting, middle_name, network, network_id, position, updated_at
Request Method: GET
Request URL: http://localhost:8000/api/experts/?format=datatables&draw=3&columns%5B0%5D%5Bdata%5D=tags&columns%5B0%5D%5Bname%5D=favoriteexpert.tags.name&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=false&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D=desc&columns%5B1%5D%5Bname%5D=&columns%5B1%5D%5Bsearchable%5D=false&columns%5B1%5D%5Borderable%5D=false&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%
fwiw, in pure django orm what I want to accomplish would be something like
Expert.objects.filter(
pk__in=Meeting.objects.filter(
objective__icontains='Plastics', user=request.user
).values('expert')
)
How can I filter experts by historical meeting objectives?
The reason for the error is that django-rest-framework-datatables is trying to translate the request into a query which can be run against the Expert table.
In your JS, you're asking for a field called 'objectives' to be returned, but there is no such field on the Expert model.
You could probably achieve what you are trying to do using the django-filter integration. In this case, you could set up a filter on the FK reference to the Meeting table. The example app demonstrates how to do this.
I think the best way to understand what's going on is to get the example application running, and if possible, set breakpoints and step through.
Incidentally, if you want to get the search box to work correctly, then you need to define a global_q() method. This is also covered in the example app.
I ended up authoring a custom django-filter
class AssociatedMeetingCharFilter(filters.CharFilter):
def global_q(self):
"""
Uses the global filter to search a meeting field of meetings owned by the logged in user
"""
if not self._global_search_value:
return Q()
kw = "meeting__{}__{}".format(self.field_name, self.lookup_expr)
return Q(**{
kw: self._global_search_value,
"meeting__user_id": self.parent.request.user.id or -1,
})
class ExpertGlobalFilterSet(DatatablesFilterSet):
name = GlobalCharFilter(lookup_expr='icontains')
objectives = AssociatedMeetingCharFilter(field_name='objective', lookup_expr='icontains')
full example at https://github.com/morenoh149/django-rest-datatables-relations-example
Simple problem. What happens when you have the name "format" as an attribute of a model with TastyPie?
How do you handle the query for http://0.0.0.0:9000/api/v1/library_type/?format=json? when you have a Model which looks like this.
class LibraryType(models.Model):
"""The information about each library type."""
format = models.IntegerField(choices=LIBRARYTYPE_CHOICES)
equiv = models.IntegerField()
name = models.CharField(max_length=96)
prefix = models.CharField(max_length=96)
description = models.CharField(max_length=255, db_column='remark')
You end up with:
{
"error": "Invalid resource lookup data provided (mismatched type)."
}
Obviously this makes sense but how do you work with it? The corresponding Resource definition.
class LibraryTypeResource(ModelResource):
class Meta:
queryset = LibraryType.objects.all()
resource_name = 'library_type'
list_allowed_methods = ['get',]
detail_allowed_methods = ['get', ]
filtering = {
'id': ('exact', ),
'name': ALL,
'format': ALL,
'prefix': ALL,
'description': ALL,
'site': ALL_WITH_RELATIONS,
}
Are you using only the json format?
If so, you can use the TASTYPIE_DEFAULT_FORMATS to set it as json and never use format=json again.
If not, you can use one of tastypie's hooks to retrieve the format query param and do whatever you want with it.
I would change the name of the query param.