I have a list of UUIDs that represent users in a DB table. I want to fetch all the users that match this criteria so I use the __in filter:
users = User.objects.filter(user__in=uuids)
I would like to raise an exception if not all of the UUIDs appear in the table. In other words, I expect to get a result back for each uuid such that len(users) == len(uuids). Is there an easy Django way to do it? If not, is there an easy way for me to create this behavior?
You could use exclude if you like it gives you the objects that does not match the query
#this returns true if one or more and dont need to use len()
#if it looks better for you you can try it out
#if everything is fine will return None
if User.objects.filter().exclude(user__in=uiids).first():
raise your_error
Don't think too complicated. You want to know if a list of something is fully contained in another list of something, that's just plain python:
user_ids = User.objects.values_list('user_id',flat=True)
if not set(uuids).issubset(user_ids): # or use .difference() to get the uuids missing from user_ids
raise stuff
Related
I am trying load some data into datatables. I am trying to specify columns in the model.objects query by using .only() --- at first glance at the resulting QuerySet, it does in fact look like the mySQL query is only asking for those columns.
However, When I try to pass the QuerySet into Paginator, and/or a Serializer, the result has ALL columns in it.
I cannot use .values_list() because that does not return the nested objects that I need to have serialized as part of my specific column ask. I am not sure what is happening to my .only()
db_result_object = model.objects.prefetch_related().filter(qs).order_by(asc+sort_by).only(*columns_to_return)
paginated_results = Paginator(db_result_object,results_per_page)
serialized_results = serializer(paginated_results.object_list,many=True)
paginated_results.object_list = serialized_results.data
return paginated_results
This one has tripped me up too. In Django, calling only() doesn't return data equivalent to a SQL statement like this:
SELECT col_to_return_1, ... col_to_return_n
FROM appname_model
The reason it doesn't do it like this is because Django returns data to you not when you construct the QuerySet, but when you first access data from that QuerySet (see lazy QuerySets).
In the case of only() (a specific example of what is called a deferred field) you still get all of the fields like you normally would, but the difference is that it isn't completely loaded in from the database immediately. When you access the data, it will only load the fields included in the only statement. Some useful docs here.
My recommendation would be to write your Serializer so that it is only taking care of the one specific filed, likely using a SerializerMethodField with another serializer to serialize your related fields.
let's say that I have an Address model with a postcode field. I can lookup addresses with postcode starting with "123" with this line:
Address.objects.filter(postcode__startswith="123")
Now, I need to do this search the "other way around". I have an Address model with a postcode_prefix field, and I need to retrieve all the addresses for which postcode_prefix is a prefix of a given code, like "12345". So if in my db I had 2 addresses with postcode_prefix = "123" and "234", only the first one would be returned.
Something like:
Address.objects.filter("12345".startswith(postcode_prefix))
The problem is that this doesn't work.
The only solution I can come up with is to perform a filter on the first char, like:
Address.objects.filter(postcode_prefix__startswith="12345"[0])
and then, when I get the results, make a list comprehension that filters them properly, like this:
results = [r for r in results if "12345".startswith(r.postcode_prefix)]
Is there a better way to do it in django?
Edit: This does not answer the original question but how to word a query the other way around.
I think what you are trying to do with your "something like" line is properly written as this:
Address.objects.filter(postcode__startswith=postcode_prefix)
In SQL terms, what you want to achieve reads like ('12345' is the postcode you are searching for):
SELECT *
FROM address
WHERE '12345' LIKE postcode_prefix||'%'
This is not really a standard query and I do not see any possibility to achieve this in Django using only get()/filter().
However, Django offers a way to provide additional SQL clauses with extra():
postcode = '12345'
Address.objects.extra(where=["%s LIKE postcode_prefix||'%%'"], params=[postcode])
Please see the Django documentation on extra() for further reference. Also note that the extra contains pure SQL, so you need to make sure that the clause is valid for your database.
Hope this works for you.
Bit of a mouthful but you can do this by annotating your search value and then filtering against it. All happens pretty quickly in-database.
from django.db.models import Value as V, F, CharField
Address.objects.exclude(
postcode_prefix=''
).annotate(
postcode=Value('12345', output_field=CharField())
).filter(
postcode__startswith=F('postcode_prefix')
)
The exclude is only necessary if postcode_prefix can be empty. This would result in an SQL like '%', which would match every postcode.
I'm sure you could do this via a nice templated function these days too... But this is clean enough for me.
A possible alternative. (Have no idea how it compares to the accepted solution with a column as the second param to like, in execution time)
q=reduce(lambda a,b:a|b, [Q(postcode__startswith=postcode[:i+1]) for i in range(len(postcode))])
Thus, you generate all prefixes, and or them together...
The raw SQL query that would do that you need looks something like this:
select * from postal_code_table where '1234567' like postal_code||'%'
This query will select any postal_code from your table that is a substring of '1234567' and also must start from begining, ie: '123', '1234', etc.
Now to implement this in Django, the preferred method is using a custom look up:
from django.db.models.fields import Field
from django.db.models import Lookup
#Field.register_lookup
class LowerStartswithContainedBy(Lookup):
'''Postgres LIKE query statement'''
lookup_name = 'istartswithcontainedby'
def as_sql(self, compiler, connection):
lhs, lhs_params = self.process_lhs(compiler, connection)
rhs, rhs_params = self.process_rhs(compiler, connection)
params = lhs_params + rhs_params
return f"LOWER({rhs}) LIKE LOWER({lhs}) || '%%'", params
Now you can write a django query such as the following:
PostCode.objects.filter(code__istartswithcontainedby='1234567')
Similarly, if you are just looking for substring and do not require the startswith condition, simply modify the return line of as_sql method to the following:
return f"LOWER({rhs}) LIKE '%%' || LOWER({lhs}) || '%%'", params
For more detailed explanation, see my git gist Django custom lookup
A. If not the issue https://code.djangoproject.com/ticket/13363,
you could do this:
queryset.extra(select={'myconst': "'this superstring is myconst value'"}).filter(myconst__contains=F('myfield'))
Maybe, they will fix an issue and it can work.
B. If not the issue 16731 (sorry not providing full url, not enough rep, see another ticket above) you could filter by fields that added with '.annotate', with creation of custom aggreation function, like here:
http://coder.cl/2011/09/custom-aggregates-on-django/
C. Last and successful. I have managed to do this using monkeypatching of the following:
django.db.models.sql.Query.query_terms
django.db.models.fields.Field.get_prep_lookup
django.db.models.fields.Field.get_db_prep_lookup
django.db.models.sql.where.WhereNode.make_atom
Just defined custom lookup '_starts', which has reverse logic of '_startswith'
I have a queryset containing some objects. Depending on some case or the other i now want to exclude all the objects without certain tags (_tags is the name of the TagField on my model):
self.queryset=self.queryset.exclude(_tags__id__in=avoid)
But this just leaves me with an error:
Caught FieldError while rendering:
Join on field '_tags' not permitted.
Did you misspell 'id' for the lookup type?
As i'm pretty sure i did not misspell 'id', i did some searching on how to use tagging for something like this. In the docs there is a lot about custom Managers, but somehow i just can't get it how i can use them to get what i want.
edit:
corrected the code above to
self.queryset=self.queryset.exclude(_tags__in=avoid)
where avoid is a list of integers. And that leaves me with the problem that the TagField of django-tagging is just a special CharField (or TextField?). Which will, of course, not sort out anything if i just query it against a list of integers. I could try to solve this in a way like this:
for tag in avoid:
self.queryset=self.queryset.exclude(_tags__contains=tag.name)
which is not only ugly, but also leaves me with the problem of tags made of multiple words or matching parts of other tags.
I somehow have the suspicion that this could be solved in a much prettier way by someone who has understood how django-tagging works.
How are your models defined? Is _tags a ForeignKey field?
if not remove the __id part
self.queryset=self.queryset.exclude(_tags__in=avoid)
Unfortunately, no, there's no prettier way. In fact, the actual solution is even uglier, but when all the tags are stored in a single text field, there's no other way:
from django.db.models import Q
startswith_tag = Q(_tags__startswith=tag.name+' ')
contains_tag = Q(_tags__contains=' '+tag.name+' ')
endswith_tag = Q(_tags__endswith=' '+tag.name)
self.queryset=self.queryset.exclude(startswith_tag | contains_tag | endswith_tag)
The code above assumes that tags are delimited with spaces. If not, you'll have to modify the code to match how they are delimited. The idea is that you use the delimiter as part of the search to ensure that it's the actual tag and not just part of another tag.
If you don't want to do it this way, I'd suggest switching to another tag system that doesn't dump them all into a single text field, django-taggit for instance.
As described in the comment on Chris' answer, django-tagging does not deliver the tagstring when accessing model._tag. In the end i had no other solution than to do the query and sort out the loops containing a certain tag afterwards:
itemlist = list(queryset)
avoid = some_list_of_tag_ids
# search for loops that have NONE of the avoid tags
for item in itemlist:
# has tags and [ if a tag.id in avoid this list has an element]
if (item.tags) and [tag for tag in item.tags if tag.id in avoid]:
# remove the item from the list
itemlist.remove(item)
To complete that the model for this looks like this:
class Item(models.Model):
_tags = TagField(blank=True,null=True)
def _get_tags(self):
return Tag.objects.get_for_object(self)
def _set_tags(self, tags):
Tag.objects.update_tags(tags)
tags = property(_get_tags, _set_tags)
Allthough i tried for quite a while, i found no way of chaining a query against tagging tags into a query chain for an object. For this project I'm stuck with tagging, but this is a real drawback...
I was wondering if there was any handy helpers in Django which would return the results of a query into a more 'usuable' format so I don't have to iterate through them in my view.
I have a query like this:
self.filter(key__in=keys).values('key','value')
What I want to end up is an object which looks like
{'some_key':'some value', 'some_other_key':'some_other_value'}
So in my model I could do something like this:
settings = Setting.objects.get_keys(['some_setting','some_other_setting'])
print settings.some_setting # returns 'some value'
Where 'get_keys' is a manager function which runs the above filter query. Any idea how I might do this? I wouldn't be opposed to iterating through the results in the Settings Manager because I could store them for later... I couldn't quite figure our how to create a 'global' model variable though.
Any help would be greatly appreciated!
If you use values_list rather than values, it will return a set of two-tuples, which you can then pass to dict() to create a dictionary:
return dict(self.filter(key__in=keys).values_list('key','value'))
I think what you're looking for is: http://docs.djangoproject.com/en/stable/ref/models/querysets/#in-bulk
This function takes a list of primary keys and return a dictionary of the models mapped to the keys. It sounds like this is exactly what you want?
I have first_name, last_name & alias (optional) which I need to search for. So, I need a query to give me all the names that have an alias set.
Only if I could do:
Name.objects.filter(alias!="")
So, what is the equivalent to the above?
You could do this:
Name.objects.exclude(alias__isnull=True)
If you need to exclude null values and empty strings, the preferred way to do so is to chain together the conditions like so:
Name.objects.exclude(alias__isnull=True).exclude(alias__exact='')
Chaining these methods together basically checks each condition independently: in the above example, we exclude rows where alias is either null or an empty string, so you get all Name objects that have a not-null, not-empty alias field. The generated SQL would look something like:
SELECT * FROM Name WHERE alias IS NOT NULL AND alias != ""
You can also pass multiple arguments to a single call to exclude, which would ensure that only objects that meet every condition get excluded:
Name.objects.exclude(some_field=True, other_field=True)
Here, rows in which some_field and other_field are true get excluded, so we get all rows where both fields are not true. The generated SQL code would look a little like this:
SELECT * FROM Name WHERE NOT (some_field = TRUE AND other_field = TRUE)
Alternatively, if your logic is more complex than that, you could use Django's Q objects:
from django.db.models import Q
Name.objects.exclude(Q(alias__isnull=True) | Q(alias__exact=''))
For more info see this page and this page in the Django docs.
As an aside: My SQL examples are just an analogy--the actual generated SQL code will probably look different. You'll get a deeper understanding of how Django queries work by actually looking at the SQL they generate.
Name.objects.filter(alias__gt='',alias__isnull=False)
Firstly, the Django docs strongly recommend not using NULL values for string-based fields such as CharField or TextField. Read the documentation for the explanation:
https://docs.djangoproject.com/en/dev/ref/models/fields/#null
Solution:
You can also chain together methods on QuerySets, I think. Try this:
Name.objects.exclude(alias__isnull=True).exclude(alias="")
That should give you the set you're looking for.
1. When using exclude, keep the following in mind to avoid common mistakes:
Should not add multiple conditions into an exclude() block like filter(). To exclude multiple conditions, you should use multiple exclude().
Example: (NOT a AND NOT b)
Entry.objects.exclude(title='').exclude(headline='')
equal to
SELECT... WHERE NOT title = '' AND NOT headline = ''
======================================================
2. Only use multiple when you really know about it:
Example: NOT (a AND b)
Entry.objects.exclude(title='', headline='')
equal to
SELECT.. WHERE NOT (title = '' AND headline = '')
If you want to exclude null (None), empty string (""), as well as a string containing white spaces (" "), you can use the __regex along with __isnull filter option
Name.objects.filter(
alias__isnull = False,
alias__regex = r"\S+"
)
alias__isnull=False excludes all the columns null columns
aliax__regex = r"\S+" makes sure that the column value contains at least one or more non whitespace characters.
From Django 1.8,
from django.db.models.functions import Length
Name.objects.annotate(alias_length=Length('alias')).filter(alias_length__gt=0)
You can simply do this:
Name.objects.exclude(alias="").exclude(alias=None)
It's really just that simple. filter is used to match and exclude is to match everything but what it specifies. This would evaluate into SQL as NOT alias='' AND alias IS NOT NULL.
Another approach using a generic isempty lookup, that can be used with any field.
It can also be used by django rest_framework or other apps that use django lookups:
from distutils.util import strtobool
from django.db.models import Field
from django.db.models.lookups import BuiltinLookup
#Field.register_lookup
class IsEmpty(BuiltinLookup):
lookup_name = 'isempty'
prepare_rhs = False
def as_sql(self, compiler, connection):
sql, params = compiler.compile(self.lhs)
condition = self.rhs if isinstance(self.rhs, bool) else bool(strtobool(self.rhs))
if condition:
return "%s IS NULL or %s = ''" % (sql, sql), params
else:
return "%s <> ''" % sql, params
You can then use it like this:
Name.objects.filter(alias__isempty=False)
this is another simple way to do it .
Name.objects.exclude(alias=None)