Using Django reduce(or_) - How can I store an unmatched query with 0 results in a variable - django

I have a function that takes a user's input from a list of data and searches my database for any items that matches the user's input, and returns all results that are in my database:
results = results.filter(
reduce(or_, (Q(name__icontains=itm.strip()) for itm in query))
)
I would like to handle cases where the user's input is not present in my database. since results fitlers down to what exists, how can I check if the above code failed to find at least one matching result for a query, and store that query in a variable? For example, if results queried my database for the following list: ['one', 'two', 'thee'], assuming 'thee' is not in my database but the other two are, I would like to store the string "thee" in a variable to use later


You can simply evaluate results as a Boolean:
if not results:
print('No match found.')
From QuerySet's documentation:
bool(). Testing a QuerySet in a boolean context, such as using bool(),
or, and or an if statement, will cause the query to be executed. If
there is at least one result, the QuerySet is True, otherwise False.
For example:
if Entry.objects.filter(headline="Test"):
print("There is at least one Entry with the headline Test")


You can add one query for query:
For example:
query = query.append(Q(name__isnull=True)) or something that always False


Instead of trying to do it one piece of code:
results = results.filter(
reduce(or_, (Q(name__icontains=itm.strip()) for itm in query))
)
I iterated over each item in query and checked if it exists
for each in query:
r = results.filter(name__icontains=each)
if r.exists() == False:
Do something with each
Though not as efficient as I'd like, it solves the problem for now

Related

DRF Q Filter string working in Django shell_plus, but not in django-filter code

I have a DRF project where I use Q filters to filter the query set. The search value will be used to search a string field "name" and tested if it is an integer and then tested if it is eq to "port" or part of a range defined by "port" and "port_end".
I need to modify the Q filter based on some conditions, so I thought it would be easy to define the query based on conditions and use it in the filter.
I was thinking that there might be an issue that the value is a string and the field in the DB an integer. But that is not the case I also tried to have fixed values in the string as integer and string, but that was not the issue. Also a int(value) did not change anything.
If I do this in the Django shell_plus, then I get proper results:
>>> value = "22"
>>> query = Q(name__icontains=value)|Q(port__exact=value)|(Q(port__lte=value)&Q(port_end__gte=value))
>>> items = MyObject.objects.filter(query)
>>> print(items)
If I do the same in my DRF filter code then the "items" query has only matches for the name field.
query = Q(name__icontains=value)|Q(port__exact=value)|(Q(port__lte=value)&Q(port_end__gte=value))
items = queryset.filter(query)
I have remove the filter on the name filed, then the query set is empty. Even if I only have the "exact" filter in the qs is empty.
It seems to me like the way I build the Q filter "query" does not work the way I did this in the code. But it work in the shell, so I guess the parsing might be different.

Raw query with rank over subquery / params not quoted

My Goal
I need PostgreSQL's rank() window function applied to an annotated queryset from Django's ORM. Django's sql query has to be a subquery in order to apply the window function and this is what I'm doing so far:
queryset = Item.objects.annotate(…)
queryset_with_rank = Items.objects.raw("""
select rank() over (order by points), *
from (%(subquery)s)""", { 'subquery': queryset.query }
)
The problem
Unfortunately, the query returned by queryset.query does not quote the parameters used for annotation correctly although the query itself is executed perfectly fine.
Example of returned query
The query returned by queryset_with_rank.query or queryset.query returns the following
"participation"."category" = )
"participation"."category" = amateur)
which I rather expected to be
"participation"."category" = '')
"participation"."category" = 'amateur')
Question
I noticed that the Django documentation states the following about Query.__str__()
Parameter values won't necessarily be quoted correctly, since that is done by the database interface at execution time.
As long as I fix the quotation manually and pass it to Postgres myself, everything works as expected. Is there a way to receive the needed subquery with correct quotation? Or is there an alternative and better approach to applying a window function to a Django ORM queryset altoghether?

As Django core developer Aymeric Augustin said, there's no way to get the exact query that is executed by the database backend beforehand.
I still managed to build the query the way I hoped to, although a bit cumbersome:
# Obtain query and parameters separately
query, params = item_queryset.query.sql_with_params()
# Put additional quotes around string. I guess this is what
# the database adapter does as well.
params = [
'\'{}\''.format(p)
if isinstance(p, basestring) else p
for p in params
]
# Cast list of parameters to tuple because I got
# "not enough format characters" otherwise. Dunno why.
params = tuple(params)
participations = Item.objects.raw("""
select *,
rank() over (order by points DESC) as rank
from ({subquery}
""".format(subquery=query.format(params)), []
)

how to insert results of a queryset to a table in Django

Am trying to insert results of a queryset into another table in my views but I get the error:
TypeError at /trackdata/
int() argument must be a string or a number, not 'ValuesQuerySet'
The code is this:
def trackdata(request):
usbtrack=(Usb.history.values('evidence'))
recordscount=Usb.history.values('evidence').count()
update=Evidence_source(evidence=usbtrack,frequency=recordscount,datatype='tzworksusb')
update.save()

I'm not sure what history is and what you are trying to fetch because your QuerySet should be formed as Usb.objects.filter(evidence=request.POST['evidence']).values('evidence'), for example. See the docs.
Anyway, values('evidence') returns a list of dicts (e.g. [{'evidence': some_value,}]), so you'd need to change evidence=usbtrack to something like evidence=usbtrack[0]['evidence']. You should probably use try/except and other error checking code, as well as loops because QuerySets by definition return lists, even if there is only one row in the result set. Another alternative is values_list() with flat=True, which returns a list of your query results rather than dicts:
usbtrack = Usb.objects.filter(evidence=request.POST['evidence']).values_list('evidence', flat=True)
Evidence_source(evidence=usbtrack[0], frequency=usbtrac.count(), datatype='tzworksusb')
Finally, if I guessed your intentions correctly and you are passing 'evidence' in your request, you would simply just do this:
usbtrack_count = Usb.objects.filter(evidence=request.POST['evidence']).count()
Evidence_source(evidence=request.POST['evidence'], frequency=usbtrack_count, datatype='tzworksusb')

how to deal with multiple arguments in a dictionary construct when query in sqlalchemy?

In sqlalchemy, I define a function whose argument is a dictionary may contain multiple key-values.I want to query according to the key-values. Here is my program:
def get_contact_conditions(kwds):
for fieldName, fieldValue in kwds.items():
condition = fieldName + "=:fieldName"
contact = session.query(Contact).filter(condition).params(fieldName = fieldValue).all()
session.commit()
return contact
this situation above is when the dictionary has only one key-values, but maybe there are two or three in it, then how to code condition and the values in params(). In another word how to code query clause.
Thank you!

If the only comparison you need is "=" then this should work:
def get_contact_conditions(kwds):
contact = session.query(Contact).filter_by(**kwds).all()
return contact
You can read about it here:
http://docs.sqlalchemy.org/en/rel_0_7/orm/query.html#sqlalchemy.orm.query.Query.filter_by
If you need arbitrary operators(<,>,in_,like,etc.) you can pass in a list of sqlalchemy clauses as arguments to the and_ function and pass the result to filter(). These expressions are generated when you compare a model class attribute to something, like:
Contact.id < 5
So you could do something like this:
from sqlalchemy.sql import and_
def get_contacts(*exps):
return session.query(Contact).filter(and_(*exps)).all()
# All contacts with an id less than 5 and a name that starts with user.
contacts = get_contacts(Contact.id < 5, Contact.name.like('user%'))
But you should consider that at this point the function is getting closer to just calling session.query directly.
You should read the sqlalchemy docs more to learn about the operators/comparisons that are available on model columns/relations. There are some really useful ones.

How to filter empty or NULL names in a QuerySet?

I have first_name, last_name & alias (optional) which I need to search for. So, I need a query to give me all the names that have an alias set.
Only if I could do:
Name.objects.filter(alias!="")
So, what is the equivalent to the above?

You could do this:
Name.objects.exclude(alias__isnull=True)
If you need to exclude null values and empty strings, the preferred way to do so is to chain together the conditions like so:
Name.objects.exclude(alias__isnull=True).exclude(alias__exact='')
Chaining these methods together basically checks each condition independently: in the above example, we exclude rows where alias is either null or an empty string, so you get all Name objects that have a not-null, not-empty alias field. The generated SQL would look something like:
SELECT * FROM Name WHERE alias IS NOT NULL AND alias != ""
You can also pass multiple arguments to a single call to exclude, which would ensure that only objects that meet every condition get excluded:
Name.objects.exclude(some_field=True, other_field=True)
Here, rows in which some_field and other_field are true get excluded, so we get all rows where both fields are not true. The generated SQL code would look a little like this:
SELECT * FROM Name WHERE NOT (some_field = TRUE AND other_field = TRUE)
Alternatively, if your logic is more complex than that, you could use Django's Q objects:
from django.db.models import Q
Name.objects.exclude(Q(alias__isnull=True) | Q(alias__exact=''))
For more info see this page and this page in the Django docs.
As an aside: My SQL examples are just an analogy--the actual generated SQL code will probably look different. You'll get a deeper understanding of how Django queries work by actually looking at the SQL they generate.

Name.objects.filter(alias__gt='',alias__isnull=False)

Firstly, the Django docs strongly recommend not using NULL values for string-based fields such as CharField or TextField. Read the documentation for the explanation:
https://docs.djangoproject.com/en/dev/ref/models/fields/#null
Solution:
You can also chain together methods on QuerySets, I think. Try this:
Name.objects.exclude(alias__isnull=True).exclude(alias="")
That should give you the set you're looking for.

1. When using exclude, keep the following in mind to avoid common mistakes:
Should not add multiple conditions into an exclude() block like filter(). To exclude multiple conditions, you should use multiple exclude().
Example: (NOT a AND NOT b)
Entry.objects.exclude(title='').exclude(headline='')
equal to
SELECT... WHERE NOT title = '' AND NOT headline = ''
======================================================
2. Only use multiple when you really know about it:
Example: NOT (a AND b)
Entry.objects.exclude(title='', headline='')
equal to
SELECT.. WHERE NOT (title = '' AND headline = '')

If you want to exclude null (None), empty string (""), as well as a string containing white spaces (" "), you can use the __regex along with __isnull filter option
Name.objects.filter(
alias__isnull = False,
alias__regex = r"\S+"
)
alias__isnull=False excludes all the columns null columns
aliax__regex = r"\S+" makes sure that the column value contains at least one or more non whitespace characters.

From Django 1.8,
from django.db.models.functions import Length
Name.objects.annotate(alias_length=Length('alias')).filter(alias_length__gt=0)

You can simply do this:
Name.objects.exclude(alias="").exclude(alias=None)
It's really just that simple. filter is used to match and exclude is to match everything but what it specifies. This would evaluate into SQL as NOT alias='' AND alias IS NOT NULL.

Another approach using a generic isempty lookup, that can be used with any field.
It can also be used by django rest_framework or other apps that use django lookups:
from distutils.util import strtobool
from django.db.models import Field
from django.db.models.lookups import BuiltinLookup
#Field.register_lookup
class IsEmpty(BuiltinLookup):
lookup_name = 'isempty'
prepare_rhs = False
def as_sql(self, compiler, connection):
sql, params = compiler.compile(self.lhs)
condition = self.rhs if isinstance(self.rhs, bool) else bool(strtobool(self.rhs))
if condition:
return "%s IS NULL or %s = ''" % (sql, sql), params
else:
return "%s <> ''" % sql, params
You can then use it like this:
Name.objects.filter(alias__isempty=False)

this is another simple way to do it .
Name.objects.exclude(alias=None)