Why does values query set not match model attributes - django

Using django 1.7, python 3.4, and postgres 9.4.
When i query a model's attribute directly it returns the proper value.
When i query the same model using .values(), it returns improper value.
For example:
>>>bomitem = self.itemproduction.billOfMaterials()
>>>for x in bomitem:
>>> x.ratio
>>>>50
>>>>20
>>>>bomitem.values('ratio')
>>>>[{'ratio': 5}, {'ratio': 2}]
Self and itemproduction are a OneToOneField relation.
ItemProduction.billOfMaterials() overrides Product.billOfMaterials()
def billOfMaterials(self):
bom = self.inventoryItem.product.billOfMaterials()
for x in bom:
x.ratio = x.ratio * self.qty
return bom
This basically takes a Product's Bill of Materials (recipe list) and figures the expected Bill of Materials for Production of n amount of a Product.
Also, when i try to filter on bomitem, the resulting queryset has same improper values.
>>>>bomitem.get(component=x)
>>><BOM: Product One: 5>
Why this erratic behavior?

The value in the database is is what you see with .values('ratio')
If you want the value to be x.ratio = x.ratio * self.qty, you'll have to save that model instance into the database after it has been initialiazed with your Python code with the calculation
A better approach would be to use a different field, one for ratio, and one for the calculated values you need (and decide if you want to save them into the DB or calculate on the fly, unless required, it's usually better not to store any calculated fields, that way your data cannot be inconsistent in the DB)

Related

Get models in Django that have all of the values in ManyToMany field (AND-query, no reverse lookups allowed)

I have such a model in Django:
class VariantTag(models.Model):
saved_variants = models.ManyToManyField('SavedVariant')
I need to get all VariantTag models that have saved_variants ManyToMany field with exact ids, say (250, 251), no more, no less. By the nature of the code that I am dealing with there is no way I can do reverse lookup with _set. So, I am looking for a query (or several queries + additional python code filtering) that will get me there but in such a way:
query = Q(...)
tag_queryset = VariantTag.objects.filter(query)
How is it possible to achieve?
I should probably stress out: supplied saved variants (e.g. (250, 251) should be AND - ed, not OR - ed.
Use in lookup
tag_queryset = VariantTag.objects.filter(saved_variants__in=[250,251])
So far I was able to achieve AND result by the following code:
tag_ids = VariantTag.objects.filter(variant_tag_type__name=tag_data['tag'],
saved_variants__in=saved_variant_ids).values_list('id', flat=True).distinct()
for tag_id in tag_ids:
saved_variants = list(VariantTag.objects.get(id=tag_id).saved_variants.all().values_list('id', flat=True))
if all(s in saved_variant_ids for s in saved_variants) and len(saved_variants) == len(saved_variant_ids):
return VariantTag.objects.get(id=tag_id)
So, I am doing the following:
Getting the OR - result
Iterating over the resulting ids of the retrieved model and for each one of them getting all of the ids of the ManyToMany field
Checking if all of the obtained ids of the ManyToMany field are in the required ids list (saved_variant_ids)
If yes - get the model by the id: VariantTag.objects.get(id=tag_id)
In my case there will be only one such model that have the required ids in ManyToMany field. If it is not the case for you - just append the ids of the model (in my case tag_id) to a list - then make a query for all of them.
If anyone has more concise way of doing AND ManyToMany query + code, would be interesting to see.

django annotate queryset with field comparison result

I have a queryset like this:
predicts = Prediction.objects.select_related('match').filter(match_id=pk)
I need to annotate this with a new field is_correct. I need to compare two string fields and the result should be annotated in this new field. the fields that I want to compare are:
predict from Prediction table
result from Match table (that has been joined through select_related)
I need to know what expression should I put inside my annotate function; below I have my current code which throughs a TypeError exception:
predicts = predicts.annotate(is_correct=(F('predict') == F('result')))
all help will be greatly appreciated.
UPDATE:
I found an alternative solution that does the job for me (filtering the Prediction based on Match result using filter and exclude), but I still like to know how to address this specific case where the new annotated field is the result of the comparison between two other fields of the queryset. For those who may need it, in Django 2.2 and later the Nullif database function does a comparison between two fields.
You can use the extra function, a hook for injecting specific clauses into the SQL.
First of all, we must know the names of the apps and the models, or the name of the tables in the database.
Assuming that in your case, the two tables are called "app_prediction" and "app_match".
The sentence would be as follows:
Prediction.objects.select_related('match').extra(
select={'is_correct': "app_prediction.predict = app_match.result"}
)
This will add a field called is_correct in your result,
in the database, the fields and tables must be called in the same way.
It would be best to see the models.

How do I use django's Q with django taggit?

I have a Result object that is tagged with "one" and "two". When I try to query for objects tagged "one" and "two", I get nothing back:
q = Result.objects.filter(Q(tags__name="one") & Q(tags__name="two"))
print len(q)
# prints zero, was expecting 1
Why does it not work with Q? How can I make it work?
The way django-taggit implements tagging is essentially through a ManytoMany relationship. In such cases there is a separate table in the database that holds these relations. It is usually called a "through" or intermediate model as it connects the two models. In the case of django-taggit this is called TaggedItem. So you have the Result model which is your model and you have two models Tag and TaggedItem provided by django-taggit.
When you make a query such as Result.objects.filter(Q(tags__name="one")) it translates to looking up rows in the Result table that have a corresponding row in the TaggedItem table that has a corresponding row in the Tag table that has the name="one".
Trying to match for two tag names would translate to looking up up rows in the Result table that have a corresponding row in the TaggedItem table that has a corresponding row in the Tag table that has both name="one" AND name="two". You obviously never have that as you only have one value in a row, it's either "one" or "two".
These details are hidden away from you in the django-taggit implementation, but this is what happens whenever you have a ManytoMany relationship between objects.
To resolve this you can:
Option 1
Query tag after tag evaluating the results each time, as it is suggested in the answers from others. This might be okay for two tags, but will not be good when you need to look for objects that have 10 tags set on them. Here would be one way to do this that would result in two queries and get you the result:
# get the IDs of the Result objects tagged with "one"
query_1 = Result.objects.filter(tags__name="one").values('id')
# use this in a second query to filter the ID and look for the second tag.
results = Result.objects.filter(pk__in=query_1, tags__name="two")
You could achieve this with a single query so you only have one trip from the app to the database, which would look like this:
# create django subquery - this is not evaluated, but used to construct the final query
subquery = Result.objects.filter(pk=OuterRef('pk'), tags__name="one").values('id')
# perform a combined query using a subquery against the database
results = Result.objects.filter(Exists(subquery), tags__name="two")
This would only make one trip to the database. (Note: filtering on sub-queries requires django 3.0).
But you are still limited to two tags. If you need to check for 10 tags or more, the above is not really workable...
Option 2
Query the relationship table instead directly and aggregate the results in a way that give you the object IDs.
# django-taggit uses Content Types so we need to pick up the content type from cache
result_content_type = ContentType.objects.get_for_model(Result)
tag_names = ["one", "two"]
tagged_results = (
TaggedItem.objects.filter(tag__name__in=tag_names, content_type=result_content_type)
.values('object_id')
.annotate(occurence=Count('object_id'))
.filter(occurence=len(tag_names))
.values_list('object_id', flat=True)
)
TaggedItem is the hidden table in the django-taggit implementation that contains the relationships. The above will query that table and aggregate all the rows that refer either to the "one" or "two" tags, group the results by the ID of the objects and then pick those where the object ID had the number of tags you are looking for.
This is a single query and at the end gets you the IDs of all the objects that have been tagged with both tags. It is also the exact same query regardless if you need 2 tags or 200.
Please review this and let me know if anything needs clarification.
first of all, this three are same:
Result.objects.filter(tags__name="one", tags__name="two")
Result.objects.filter(Q(tags__name="one") & Q(tags__name="two"))
Result.objects.filter(tags__name_in=["one"]).filter(tags__name_in=["two"])
i think the name field is CharField and no record could be equal to "one" and "two" at same time.
in python code the query looks like this(always false, and why you are geting no result):
from random import choice
name = choice(["abtin", "shino"])
if name == "abtin" and name == "shino":
we use Q object for implement OR or complex queries
Into the example that works you do an end on two python objects (query sets). That gets applied to any record not necessarily to the same record that has one AND two as tag.
ps: Why do you use the in filter ?
q = Result.objects.filter(tags_name_in=["one"]).filter(tags_name_in=["two"])
add .distinct() to remove duplicates if expecting more than one unique object

Django: Equivalent of "select [column name] from [tablename]"

I wanted to know is there anything equivalent to:
select columnname from tablename
Like Django tutorial says:
Entry.objects.filter(condition)
fetches all the objects with the given condition. It is like:
select * from Entry where condition
But I want to make a list of only one column [which in my case is a foreign key]. Found that:
Entry.objects.values_list('column_name', flat=True).filter(condition)
does the same. But in my case the column is a foreign key, and this query loses the property of a foreign key. It's just storing the values. I am not able to make the look-up calls.
Of course, values and values_list will retrieve the raw values from the database. Django can't work its "magic" on a model which means you don't get to traverse relationships because you're stuck with the id the foreign key is pointing towards, rather than the ForeignKey field.
If you need to filters those values, you could do the following (assuming column_name is a ForeignKey pointing to MyModel):
ids = Entry.objects.values_list('column_name', flat=True).filter(...)
my_models = MyModel.objects.filter(pk__in=set(ids))
Here's a documentation for values_list()
To restrict a query set to a specific column(s) you use .values(columname)
You should also probably add distinct to the end, so your query will end being:
Entry.objects.filter(myfilter).values(columname).distinct()
See: https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.values
for more information
Depending on your answer in the comment, I'll come back and edit.
Edit:
I'm not certain if the approach is right one though. You can get all of your objects in a python list by getting a normal queryset via filter and then doing:
myobjectlist = map(lambda x: x.mycolumnname, myqueryset)
The only problem with that approach is if your queryset is large your memory use is going to be equally large.
Anyway, I'm still not certain on some of the specifics of the problem.
You have a model A with a foreign key to another model B, and you want to select the Bs which are referred to by some A. Is that right? If so, the query you want is just:
B.objects.filter(a__isnull = False)
If you have conditions on the corresponding A, then the query can be:
B.objects.filter(a__field1 = value1, a__field2 = value2, ...)
See Django's backwards relation documentation for an explanation of why this works, and the ForeignKey.related_name option if you want to change the name of the backwards relation.

django - Dynamically add computed column to queryset

I have a database table containing 4 integer columns representing version information - Major, Minor, Cust, Rev.
I'd like to represent these as a single column in my form, with relevant padding. So:
0, 1, 2, 3 = [0.01.002.0003].
I realise that there are several ways to use SQL to do this, but I'm curious as to whether it's possible to do it on the django side? So, after:
queryset = MyModel.objects.all()
...can I then extend the queryset with an additional calculated field? Later I pass the Queryset to be serialized, and I need the calculated field present at this stage, so I don't think an attribute will work.
annotate should help you:
http://docs.djangoproject.com/en/dev/ref/models/querysets/#annotate-args-kwargs