Django icontains behaves differently on web vs shell - django-views

Edit: I also tried iregex and found that it behaves in the same manner as icontains below.
form_search = FormTest.objects.filter(name__iregex=rf"{new_data}")
I'm running in to a problem where an identical query behaves differently if it is run in project vs when it is run in that project's shell.
I created a sample table in my project's database (FormTest). The FormTest table contains a field called "name" where I added values such as "Mike" "Mike Smith" "John" "Jonathan", etc.
I then created a simple form to search for names in the table using icontains:
if request.method == 'POST':
input_form = FormClassSearch(request.POST)
if input_form.is_valid():
new_data = input_form.cleaned_data
new_data = str(new_data['name'])
form_search = FormTest.objects.filter(name__icontains=new_data)
form_search_q =\
FormTest.objects.filter(name__icontains=new_data).query
When I use the form to search for "Mike" it returns "Mike" and "Mike Smith". However, if I search for "Mike Smith" it returns nothing. A stranger problem exists when I try to find "Jonathan". If I search for "Jo" it returns both "John" and "Jonathan". If I try to search for "Jon" "Jona" ... "Jonathan" it returns nothing.
To make sure I was writing the query correctly, I fired up the manage.py shell:
>>> c = 'mike smith'
>>> print(c)
mike smith
>>> a = FormTest.objects.filter(name__icontains=c)
>>> a.exists()
True
>>> len(a)
1
>>> for item in a:
... print(item.name)
...
Mike Smith
>>> b = FormTest.objects.filter(name__icontains=c).query
>>> print(b)
SELECT "workout_formtest"."id", "workout_formtest"."name",
"workout_formtest"."dob", "workout_formtest"."number", "workout_formtest"."number2"
FROM "workout_formtest" WHERE UPPER("workout_formtest"."name"::text)
LIKE UPPER(%mike smith%)
I confirmed that the SQL from .query matches (shell output vs application output).
I also coded the template to print the search term, to make sure the input was being passed correctly.
Anyone have any idea what I'm doing wrong or why the application is not behaving as expected? I only have one database in settings.py - a Postgresql DB. The form I created to ADD data in to the table works perfectly.

Solved:
I don't know the exact cause, but I believe it has to do with the fact that QuerySets are lazy. Originally,the queryset was not evaluated until it was iterated through on the template. My solution is to iterate through the query set in the view, and append the results to a list:
def formsearchpage(request):
...
query_list = []
if request.method == 'POST':
input_form = ClassSearchForm(request.POST)
if input_form.is_valid():
new_data = input_form.cleaned_data
new_data = str(new_data['name'])
form_search = FormTest.objects.filter(name__iregex=rf"{new_data}")
count = len(form_search)
message = "Your input " + new_data
if count == 1:
message = 'One Match Found on ' + new_data
for item in form_search:
query_list.append(item.name)
if count >= 2:
message = 'More than one match found on ' + new_data
for item in form_search:
query_list.append(item.name)
...
The key is:
for item in form_search:
query_list.append(item.name)
query_list is then passed to the template, which displays it:
{% if query_list %}
{% for i in query_list %}
<p>{{ i }}</p>
{% endfor %}
{% endif %}
Now, when I render the template, the loop that iterates through 'query_list' displays the expected data, while the old loop that iterated through the raw QuerySet (the 'form_search' object created in the view) does not.

Related

Django Postgres JSONField query

I have a class with a json field in it
class A(models.Model)
brand = JSONField()
If I post an array of JSON like [{'brand_id:1', 'name':'b1'}, {'brand_id:2', 'name':'b2'}] it is stored as an array of JSON. This works fine.
How should I query so as to check if '1' is present in the brand_id of any dictionary in that array?
Well first of all your JSON here is malformed. I presume it is meant to be:
[{'brand_id': 1, 'name': 'b1'}, {'brand_id': 2, 'name': 'b2'}]
If that's the case, to test for 1 in such a blob, something like this will tell you if 1 is to be found anywhere in the JSON as a value:
def check_for_one(json_data):
return any([1 in data.values() for data in json_data])
But you want to know specifically if 1 is a value owned by a key
brand_id anywhere in the JSON so you can also use a loop to add in some extra conditions:
def check_for_one(json_data):
match = []
for data in json_data:
for key, value in data.items():
if key == 'brand_id' and value == 1:
match.append(True)
return any(match)
You can incorporate such logic as methods on your model class like this:
class A(models.Model):
brand = JSONField()
def check_for_one_comprehension(self):
return any([1 in data.values() for data in self.brand])
def check_for_one_loop(self):
match = []
for data in self.brand:
for key, value in data.items():
if key == 'brand_id' and value == 1:
match.append(True)
return any(match)
But, if you actually want to filter instances from the database where the JSON data is an array at the top level and brand_id == 1, that needs a different approach, and this should do it:
A.objects.filter(brand__contains=[{'brand_id': 1}])
Note the additional [{}] braces! If you just call contains=['brand_id': 1] it will throw a syntax error and if you call contains={'brand_id': 1} it will not match.
This worked:
A.objects.filter(brands__contains=[{'brand_id':1}])
I didnt check it first by using the array.
Link pointed put by #Bear Brown gives sufficient info.
Its easy in django, but finding it out took time :P.

Prevent rendering the unicode character in django template

I am executing a query to a table in my view and I pass the data to the context dictionary:
conn = psycopg2.connect(constr)
cur = conn.cursor()
sqlstr = "SELECT DISTINCT adm0_name FROM wld_bnd_adm0_gaul_2015 ORDER BY adm0_name;"
cur.execute(sqlstr)
countries = cur.fetchall()
ctx['countries'] = countries
Then I want to render the data in my template as following:
{% if countries %}
{% for cntr in countries %}
<li><a href="#" id= {{ cntr }}>{{ cntr }}</a></li>
{% endfor %}
{% endif %}
The problem is that I get a unicode format like this:
[(u'Abyei',), (u'Afghanistan',), (u...
I tried to convert my data in the view to JSON as:
countries = json.dumps(countries)
But actually then when I render the data in the template I get each single character of the JSON string separately.
As mentioned in the docs fetchall returns a list of tuples, so each of the country name is inside a tuple in the list. You need to flatten the list to get list of countries.
countries = cur.fetchall()
countries = [c[0] for c in countries]
ctx['countries'] = countries
I'm assuming you're using Python2.
From the documentation, here's what you would get after calling json.dumps:
>>> json.dumps([(u'Abyei',), (u'Afghanistan',)])
'[["Abyei"], ["Afghanistan"]]'
Since you get a single JSON formatted string serialized from dictionary, when you iterate it, you'll get a character on every iteration. This explains why you get single characters when you render the data.
Your countries is already a dictionary thus can be used without serializing; if you want to convert unicode strings, you can do something like:
>>> countries = [(u'Abyei',), (u'Afghanistan',)]
>>> [(country.encode('ascii','ignore'), ) for (country, ) in countries]
[('Abyei',), ('Afghanistan',)]

Django ORM "get" translation to SQL

For a Queryset in Django, we can call its method .query to get the raw sql.
for example,
queryset = AModel.objects.all()
print queryset.query
the output could be: SELECT "id", ... FROM "amodel"
But for retrieving a object by "get", say,
item = AModel.objects.get(id = 100)
how to get the equivalent raw sql? Notice: the item might be None.
The item = AModel.objects.get(id = 100) equals to
items = AModel.objects.filter(id = 100)
if len(items) == 1:
return items[0]
else:
raise exception
Thus the executed query equals to AModel.objects.filter(id = 100)
Also, you could check the latest item of connection.queries
from django.db import connection # use connections for non-default dbs
print connection.queries[-1]
And, as FoxMaSk said, install django-debug-toolbar and enjoy it in your browser.
It's the same SQL, just with a WHERE id=100 clause tacked to the end.
However, FWIW, If a filter is specific enough to only return one result, it's the same SQL as get would produce, the only difference is on the Python side at that point, e.g.
AModel.objects.get(id=100)
is the same as:
AModel.objects.filter(id=100).get()
So, you can simply query AModel.objects.filter(id=100) and then use queryset.query with that.
if it's just for debugging purpose you can use "the django debug bar" which can be installed by
pip install django-debug-toolbar

Is it possible to replace values in a queryset before sending it to your template?

Wondering if it's possible to change a value returned from a queryset before sending it off to the template.
Say for example you have a bunch of records
Date | Time | Description
10/05/2010 | 13:30 | Testing...
etc...
However, based on the day of the week the time may change. However this is static. For example on a monday the time is ALWAYS 15:00.
Now you could add another table to configure special cases but to me it seems overkill, as this is a rule. How would you replace that value before sending it to the template?
I thought about using the new if tags (if day=1), but this is more of business logic rather then presentation.
Tested this in a custom template tag
def render(self, context):
result = self.model._default_manager.filter(from_date__lte=self.now).filter(to_date__gte=self.now)
if self.day == 4:
result = result.exclude(type__exact=2).order_by('time')
else:
result = result.order_by('type')
result[0].time = '23:23:23'
context[self.varname] = result
return ''
However it still displays the results from the DB, is this some how related to 'lazy' evaluation of templates?
Thanks!
Update Responding to comments below:
It's not stored wrong in the DB, its stored Correctly However there is a small side case where the value needs to change.
So for example I have a From Date & To date, my query checks if todays date is between those. Now with this they could setup a from date - to date for an entire year, and the special cases (like mondays as an example) is taken care off. However if you want to store in the DB you would have to capture several more records to cater for the side case. I.e you would be capturing the same information just to cater for that 1 day when the time changes. (And the time always changes on the same day, and is always the same)
Update with Solution (Based on answer by KillianDS below)
In models.py i defined a custom property:
#property
def get_corrected_time(self):
from datetime import time
day = datetime.now().weekday()
if day == 0 or day == 1:
self.time = time(12,30)
return self.time
and in the template
{{ object.get_corrected_time|time:"P" }}
The reason for returning a datetime.time object rather then a string is, django's default date & time filters will not work on a string.
Okay, without any precise example (a simple model & clear use case would help), I can't really be sure, but I guess this is what you want to do. In your model definition, add something like this:
#property
def get_corrected_time(self):
if today_is_monday:
return '13:30'
return str(self.time)
in your template you can perfectly call this as {{ object.get_corrected_time }}
Now I see in your template tag you want to do this for example only on the first element in a queryset, you can easily do this as follows in the template (you could also do it in the view method):
{% for object in list %}
{% if forloop.first %}
{{ object.get_corrected_time }}
{% else %}
{{ object.time }}
{% endif %}
{% endfor %}

Django: Return all Values even if filtering through a related model

Thanks to some fantastic help on a previous question I have managed to put together my query. Everything works swimmingly save one issue.
careers = Career.objects.filter(job__dwarf__user = 1).annotate(dwarves_in_career = Count('job__dwarf'))
In my view this does exactly what I want and when I loop it in my template like so:
{% for career in careers reversed %}
<li>{{ career.name }}: {{ career.dwarves_in_career }}</li>
{% endfor %}
I get what I expected. My issue is that the above code will only return the Careers that have dwarves. I need it to return all careers:
Unassigned: 1
Construction: 1
Crafting: 2
Gathering: 0
Farming: 0
is my expected results. Instead the last two rows are not there because I am filtering for the specific user. I know why it's happening, what I am trying to figure out is how to get the query to return all careers, even if their dwarf count for a particular user is 0.
I think you need to swap over your annotate and filter clauses. See the documentation on this - the way you have it, it filters the queryset, but you actually want to keep the entire queryset but just filter the annotations.
So, I eventually figured it out...
from django.db.models import Count, Q
def fortress(request):
careers = Career.objects.filter(Q(job__dwarf__user = 1) | Q(job__dwarf__user__isnull = True)).annotate(dwarves_in_career = Count('job__dwarf'))
return render_to_response('ragna_base/fortress.html', {'careers': careers})
Not exactly sure what Q is but it works!
More information here: http://www.djangoproject.com/documentation/models/or_lookups/
Basically what I am doing is finding all careers with dwarves (based on user_id) OR all careers that have no dwarves (based on null)