So I have 2 query sets that are from the same model but different records. I already made the pythonic way of getting the result. but I want to do it with aggregate and annotate to get this done.
result = [i for i in self.old_searches if round(
abs(same_keys.get(keyword =i.keyword).current_position-i.current_position)) >=1]
so both of self.old_searches and same_keys have the same field called current_position
and for example I need first recprd of the first query be compared with first record second one. but all i need is the current_position.
Related
I have a queryset like this:
predicts = Prediction.objects.select_related('match').filter(match_id=pk)
I need to annotate this with a new field is_correct. I need to compare two string fields and the result should be annotated in this new field. the fields that I want to compare are:
predict from Prediction table
result from Match table (that has been joined through select_related)
I need to know what expression should I put inside my annotate function; below I have my current code which throughs a TypeError exception:
predicts = predicts.annotate(is_correct=(F('predict') == F('result')))
all help will be greatly appreciated.
UPDATE:
I found an alternative solution that does the job for me (filtering the Prediction based on Match result using filter and exclude), but I still like to know how to address this specific case where the new annotated field is the result of the comparison between two other fields of the queryset. For those who may need it, in Django 2.2 and later the Nullif database function does a comparison between two fields.
You can use the extra function, a hook for injecting specific clauses into the SQL.
First of all, we must know the names of the apps and the models, or the name of the tables in the database.
Assuming that in your case, the two tables are called "app_prediction" and "app_match".
The sentence would be as follows:
Prediction.objects.select_related('match').extra(
select={'is_correct': "app_prediction.predict = app_match.result"}
)
This will add a field called is_correct in your result,
in the database, the fields and tables must be called in the same way.
It would be best to see the models.
class A(models.Model)
results = models.TextField()
class B(models.Model)
name = models.CharField(max_length=20)
res = models.ManyToManyField(A)
Let's suppose we have above 2 models. A model has millions of objects.
I would like to know what would be the best efficient/fastest way to get all the results objects of a particular B object.
Let's suppose we have to retrieve all results for object number 5 of B
Option 1 : A.objects.filter(b__id=5)
(OR)
Option 2 : B.objects.get(id=5).res.all()
Option 1: My Question is filtering by id on A model objects would take lot of time? since there are millions of A model objects.
Option 2: Question: does res field on B model stores the id value of A model objects?
The reason why I'm assuming the option 2 would be a faster way since it stores the reference of A model objects & directly getting those object values first and making the second query to fetch the results. whereas in the first option filtering by id or any other field would take up a lot of time
The first expression will result in one database query. Indeed, it will query with:
SELECT a.*
FROM a
INNER JOIN a_b ON a_b.a_id = a.id
WHERE a_b.b_id = 5
The second expression will result in two queries. Indeed, first Django will query to fetch that specific B object with a query like:
SELECT b.*
FROM b
WHERE b.id = 5
then it will make exactly the same query to retrieve the related A objects.
But retrieving the A object is here not necessary (unless you of course need it somewhere else). You thus make a useless database query.
My Question is filtering by id on A model objects would take lot of time? since there are millions of A model objects.
A database normally stores an index on foreign key fields. This thus means that it will filter effectively. The total number of A objects is usually not (that) relevant (since it uses a datastructure to accelerate search like a B-tree [wiki]). The wiki page has a section named An index speeds the search that explains how this works.
I'm trying to display a chat log in django. I can get my entire chatlog in the proper order with this query.
latest_chats_list = Chat.objects.order_by('timestamp')
I want the functionality of this line (last 10 elements in order), but django doesn't allow negative indexes.
latest_chats_list = Chat.objects.order_by('timestamp')[-10:]
if I try this line, I get the messages I want, but they're in the wrong order.
latest_chats_list = Chat.objects.order_by('-timestamp')[:10]
This line gives the first 10 chats instead of the most recent.
latest_chats_list = Chat.objects.order_by('-timestamp')[:10].reverse()
last_ten = Chat.objects.all().order_by('-id')[:10]
last_ten_in_ascending_order = reversed(last_ten)
Edit (from comments)
Why not use Django's queryset.reverse() ?
Because it messes with the SQL query, as does queryset.order_by(). Slicing the queryset ([:10]) also alters the SQL query, adding LIMIT and OFFSET to it. The two can combine in not-obviously-expected ways...
On the other hand, the built-in Python function reversed(iterable) only changes the way queryset gets iterated over, not effecting the SQL at all.
Here's the model:
class ModelA:
title = charfield
m2m = foreignkey, relatedname='m2ms'
This is working:
ModelA.objects.filter(Q(title__icontains='a') & Q(title__icontains='b'))
So it returns all records whose titles contains both letters 'a' and 'b'.
Then same not working for many to many:
ModelA.objects.filter(Q(m2ms__id=1) & Q(m2ms__id=2))
ModelA m2ms list:
for x in ModelA.objects.all():
print x.m2ms.all().values_list('id', Flat=True)
#Output:
1,2,3
1,2
1
1,3,5
4,6,7
1,8
So, expected output of ModelA.objects.filter(Q(m2ms__id=1) & Q(m2ms__id=2)) should be records having these m2m ids: [1,2,3], [1,2]. But its not happening. Why ?
I cant use Q(m2ms__id__in=[1,2]) because it returns same even if I do __in=[1,2,3,4, infinite numbers]
Reason for using Q instead of filter is mentioned in this question - django filter on many to many along with Q
Read this section of the docs.
Particularly this paragraph:
To handle both of these situations, Django has a consistent way of processing filter() and exclude() calls. Everything inside a single filter() call is applied simultaneously to filter out items matching all those requirements. Successive filter() calls further restrict the set of objects, but for multi-valued relations, they apply to any object linked to the primary model, not necessarily those objects that were selected by an earlier filter() call.
I believe if you do ModelA.objects.filter(Q(m2ms__id__in=[1, 2])) or ModelA.objects.filter(m2ms__id__in=[1, 2]) it will work as you expect it to.
I have a Result object that is tagged with "one" and "two". When I try to query for objects tagged "one" and "two", I get nothing back:
q = Result.objects.filter(Q(tags__name="one") & Q(tags__name="two"))
print len(q)
# prints zero, was expecting 1
Why does it not work with Q? How can I make it work?
The way django-taggit implements tagging is essentially through a ManytoMany relationship. In such cases there is a separate table in the database that holds these relations. It is usually called a "through" or intermediate model as it connects the two models. In the case of django-taggit this is called TaggedItem. So you have the Result model which is your model and you have two models Tag and TaggedItem provided by django-taggit.
When you make a query such as Result.objects.filter(Q(tags__name="one")) it translates to looking up rows in the Result table that have a corresponding row in the TaggedItem table that has a corresponding row in the Tag table that has the name="one".
Trying to match for two tag names would translate to looking up up rows in the Result table that have a corresponding row in the TaggedItem table that has a corresponding row in the Tag table that has both name="one" AND name="two". You obviously never have that as you only have one value in a row, it's either "one" or "two".
These details are hidden away from you in the django-taggit implementation, but this is what happens whenever you have a ManytoMany relationship between objects.
To resolve this you can:
Option 1
Query tag after tag evaluating the results each time, as it is suggested in the answers from others. This might be okay for two tags, but will not be good when you need to look for objects that have 10 tags set on them. Here would be one way to do this that would result in two queries and get you the result:
# get the IDs of the Result objects tagged with "one"
query_1 = Result.objects.filter(tags__name="one").values('id')
# use this in a second query to filter the ID and look for the second tag.
results = Result.objects.filter(pk__in=query_1, tags__name="two")
You could achieve this with a single query so you only have one trip from the app to the database, which would look like this:
# create django subquery - this is not evaluated, but used to construct the final query
subquery = Result.objects.filter(pk=OuterRef('pk'), tags__name="one").values('id')
# perform a combined query using a subquery against the database
results = Result.objects.filter(Exists(subquery), tags__name="two")
This would only make one trip to the database. (Note: filtering on sub-queries requires django 3.0).
But you are still limited to two tags. If you need to check for 10 tags or more, the above is not really workable...
Option 2
Query the relationship table instead directly and aggregate the results in a way that give you the object IDs.
# django-taggit uses Content Types so we need to pick up the content type from cache
result_content_type = ContentType.objects.get_for_model(Result)
tag_names = ["one", "two"]
tagged_results = (
TaggedItem.objects.filter(tag__name__in=tag_names, content_type=result_content_type)
.values('object_id')
.annotate(occurence=Count('object_id'))
.filter(occurence=len(tag_names))
.values_list('object_id', flat=True)
)
TaggedItem is the hidden table in the django-taggit implementation that contains the relationships. The above will query that table and aggregate all the rows that refer either to the "one" or "two" tags, group the results by the ID of the objects and then pick those where the object ID had the number of tags you are looking for.
This is a single query and at the end gets you the IDs of all the objects that have been tagged with both tags. It is also the exact same query regardless if you need 2 tags or 200.
Please review this and let me know if anything needs clarification.
first of all, this three are same:
Result.objects.filter(tags__name="one", tags__name="two")
Result.objects.filter(Q(tags__name="one") & Q(tags__name="two"))
Result.objects.filter(tags__name_in=["one"]).filter(tags__name_in=["two"])
i think the name field is CharField and no record could be equal to "one" and "two" at same time.
in python code the query looks like this(always false, and why you are geting no result):
from random import choice
name = choice(["abtin", "shino"])
if name == "abtin" and name == "shino":
we use Q object for implement OR or complex queries
Into the example that works you do an end on two python objects (query sets). That gets applied to any record not necessarily to the same record that has one AND two as tag.
ps: Why do you use the in filter ?
q = Result.objects.filter(tags_name_in=["one"]).filter(tags_name_in=["two"])
add .distinct() to remove duplicates if expecting more than one unique object