Django RAW SQL query how to pass date comparison - django

I am working with Django Raw SQL as I want to build a relatively complex query,
As part of it, I have a case statement that says 'if the date on the record is < today then 1 else 0 - this works.
overdue_phases = task.objects.filter(orgid=orgid, taskid=taskidx).raw('''
SELECT '1' as id, case when research_due < "2022-12-01" AND research_status != "done" then 1 else 0 end as count
from app_task where taskid = "''' + taskidx + '''"''')
overdue_phases = task.objects.filter(orgid=orgid, taskid=taskidx).raw('''
SELECT '1' as id, case when research_due < "2022-12-01" AND research_status != "done" then 1 else 0 end as count
from app_task where taskid = "''' + taskidx + '''"''')
However, when I want to swap the hard coded date for TODAY() I can't make it work. I have tried passing a python date variable into the script (td = datetime.date.today()), but it doesn't return the right results!
Can anyone advise me please?

There is no need to do this. You can use condition expressions [Django-doc]:
from django.db.models import Case, Q, Value, When
overdue_phases = task.objects.filter(orgid=orgid, taskid=taskidx).annotate(
count=Case(
When(
~Q(research_status='done'),
research_due__lt='2022-12-01',
then=Value(1),
),
default=Value(0),
)
)
Or if a boolean is sufficient as well:
from django.db.models import Q
overdue_phases = task.objects.filter(orgid=orgid, taskid=taskidx).annotate(
count=~Q(research_status='done') & Q(research_due__lt='2012-12-01')
)
or for older versions of Django with an ExpressionWrapper [Django-doc]:
from django.db.models import BooleanField, ExpressionWrapper, Q
overdue_phases = task.objects.filter(orgid=orgid, taskid=taskidx).annotate(
count=ExpressionWrapper(
~Q(research_status='done') & Q(research_due__lt='2012-12-01'),
output_field=BooleanField(),
)
)
You can replace '2022-12-01' with date.today().

Related

Division between two annotations

I'm creating those two annotations as it follows:
cs = Champion.objects.all()
total_games = Match.objects.all().count()
cs = cs.annotate(
picked_games=Count(
expression='participants__match__id',
filter=Q(participants__role='BOTTOM'),
distinct=True
),
total_games=Value(str(total_games), output_field=IntegerField())
)
And everthing's alright until here. I fetch both the picked_games and total_games with correct results.
>>> cs.get(name='Jhin').picked_games
27544
>>> cs.get(name='Jhin').total_games
97410
However, if i try to divide one by another:
cs = cs.annotate(
pick_rate=ExpressionWrapper(
expression=F('picked_games') / F('total_games'),
output_field=FloatField()
)
)
This results on a 0:
>>> cs.get(name='Jhin').pick_rate
0.0
I don't understand what's problem here..
I can get the result if divide them externally, so why can't i get the result on a different column for the whole queryset?
>>> cs.get(name='Jhin').picked_games / cs.get(name='Jhin').total_games
0.28319474386613286
You should cast the numerator (or denominator) to a float before making the division, otherwise the PostgreSQL database will use integer division and thus truncate towards zero. You thus can work with:
from django.db.models import Count, F, FloatField, Q
from django.db.models.functions import Cast
total_games = Match.objects.all().count()
Champion.objects.annotate(
picked_games=Count(
expression='participants__match__id',
filter=Q(participants__role='BOTTOM'),
distinct=True
)
).annotate(
pick_rate=Cast('picked_games', output_field=FloatField()) / total_games
)

How make conditional expression on Django with default value of object?

See https://docs.djangoproject.com/en/dev/ref/models/conditional-expressions/
conditional expression with Case;
o = Object.annotate(
custom_price=Case(
When(price=1, then=Value(2)),
default=0,
output_field=DecimalField(),
)
)
How use set 'default' - current value of Object?
Now it writed only as const: 0
Want something like:
if price =1:
custom_price = 2
else:
custom_price = Object.price
F is used to perform this. So, your code should look like:
from django.db.models import Case, F, Value, When, DecimalField
o = Object.objects.annotate(custom_price=Case(
When(price=1, then=Value(2)),
default=F('price'),
output_field=DecimalField(),
)
)

How do I return a web page's table cell value without ID, using column and row names (not index)

Most questions about Python and Selenium scraping a web page's table data involve a table with an ID or Class, and some index technique using a count of rows and columns. The Xpath technique is usually not explained either.
Say I have a table without an element ID or class, let's use this one for example.
I want to return the value 'Johnson', without counting row or column numbers.
Here's my attempt (edited)...
import selenium.webdriver as webdriver
import contextlib
url = 'http://www.w3schools.com/html/html_tables.asp'
with contextlib.closing(webdriver.Firefox()) as driver:
driver.get(url)
columnref = 3
rowref = 4
xpathstr = '//tr[position()=' + str(rowref) + ']//td[position()=' + str(columnref) + ']'
data = driver.find_element_by_xpath(xpathstr).text
print data
I have gotten some good help here already, but am still using an index. I need to generate 'columnref' and 'rowref' by looking up their values. 'Last Name', and '3' respectively.
Just use this css selector to reach the cell you want tbody > tr:nth-child(4) > td:nth-child(3), and you can generate css selector for any cell with the same way. See below:
>>> driver.find_element_by_css_selector("tbody > tr:nth-child(4) > td:nth-child(3)")
<selenium.webdriver.remote.webelement.WebElement object at 0x10fdd4510>
>>> driver.find_element_by_css_selector("tbody > tr:nth-child(4) > td:nth-child(3)").text
u'Johnson'
Alternatively, you can use position() tag to locate cell position. See below:
>>> driver.find_element_by_xpath("//tr[position()=4]//td[position()= 3]").text
u'Johnson'
>>> driver.find_element_by_xpath("//tr[position()=5]//td[position()= 3]").text
u'Smith'
If you want to get the text by column name and row number you can write a function that returns the value by finding the index of the column then getting the text as below:
def get_text_column_row(table_css, header, row):
table = driver.find_element_by_css_selector(table_css)
table_headers = table.find_elements_by_css_selector('tbody > tr:nth-child(1) > th')
table_rows = table.find_elements_by_css_selector("tbody > tr > td:nth-child(1)")
index_of_column = None
index_of_row = None
for i in range(len(table_headers)):
if table_headers[i].text == header:
index_of_column = i + 1
for i in range(len(table_rows)):
if table_rows[i].text == row:
index_of_row = i + 1
xpath = '//tr[position() = %d]//td[position() = %d]' %(index_of_row, index_of_column)
return driver.find_element_by_xpath(xpath).text
and use it like below:
>>> get_text_column_row('#main > table:nth-child(6)', 'Points', '3')
u'80'
>>> get_text_column_row('#main > table:nth-child(6)', 'Last Name', '3')
u'Doe'
>>> get_text_column_row('#main > table:nth-child(6)', 'Last Name', '4')
u'Johnson'
In [1]: run table_cell_value.py
Johnson
table_cell_value.py
from selenium import webdriver
browser = webdriver.Firefox()
COLUMN_NAME = 'Last Name'
ROW_NAME = '3'
JS_SCRIPT = """
var row = column = -1;
var table = arguments[0];
for (var c = 0, m = table.rows[0].cells.length; c < m; c++) {
if (table.rows[0].cells[c].innerHTML === arguments[1]) {
column = c
}
}
for (var r = 0, n = table.rows.length; r < n; r++) {
if (table.rows[r].cells[0].innerHTML === arguments[2]) {
row = r
}
}
if (row < 0 || column < 0) {
return
}
return table.rows[row].cells[column].innerHTML
"""
browser.get('http://www.w3schools.com/html/html_tables.asp')
table = browser.find_element_by_tag_name('table')
print browser.execute_script(JS_SCRIPT, table, COLUMN_NAME, ROW_NAME)
Docs: https://developer.mozilla.org/en-US/docs/Web/API/HTMLTableElement

Find object in datetime range

Let's assume that I have modeL;
class MyModel(...):
start = models.DateTimeField()
stop = models.DateTimeField(null=True, blank=True)
And I have also two records:
start=2012-01-01 7:00:00 stop=2012-01-01 14:00:00
start=2012-01-01 7:00:03 stop=2012-01-01 23:59:59
Now I want to find the second query, so start datetime should be between start and stop, and stop should have hour 23:59:59. How to bould such query?
Some more info:
I think this requires F object. I want to find all records where start -> time is between another start -> time and stop -> time, and stop -> time is 23:59:59, and date is the same like in start
YOu can use range and extra:
from django.db.models import Q
q1 = Q( start__range=(start_date_1, end_date_1) )
q1 = Q( start__range=(start_date_2, end_date_2) )
query = (''' EXTRACT(hour from end_date) = %i
and EXTRACT(minute from end_date) = %i
and EXTRACT(second from end_date) = %i''' %
(23, 59,59)
)
MyModel.objects.filter( q1 | q2).extra(where=[query])
Notice: Posted before hard answer requirement changed 'time is 23:59:59, and date is the same like in start'
To perform the query: "start datetime should be between start and stop"
MyModel.objects.filter(start__gte=obj1.start, start__lte=obj1.stop)
I don't quite understand your second condition, though. Do you want it to match only objects with hour 23:59:59, but for any day?
dt = '2012-01-01 8:00:00'
stop_hour = '23'
stop_minute = '59'
stop_sec = '59'
where = 'HOUR(stop) = %(hour)s AND MINUTE(stop) = %(minute)s AND SECOND(stop) = %(second)s' \
% {'hour': stop_hour, 'minute': stop_minute, 'seconds': stop_ec}
objects = MyModel.objects.filter(start__gte=dt, stop__lte=dt) \
.extra(where=[where])

Django raw SQL query using LIKE on PostgreSQL

I'm trying to do a Raw SELECT in Django using LIKE in the PostgreSQL database with Psycopg2 driver.
I've tested pretty much what I've found on the web, but nothing have worked.
The situation is the following. I need to perform a SELECT like this:
select distinct on (name, adm1.name, adm2.name_local)
gn.geonameid,
case when altnm.iso_language = 'pt' then altnm.alternate_name else gn.name end as name,
adm1.name as zona,
adm2.name_local as municipio
from location_geonameslocal gn
join location_geonameszone adm1 on adm1.code = gn.country || '.' || gn.admin1
join location_geonamesmunicipality adm2 on adm2.code = gn.country || '.' || gn.admin1 || '.' || gn.admin2
left join location_geonamesalternatenames altnm on altnm.geonameid = gn.geonameid
where
(gn.fclass = 'P' or gn.fclass = 'A')
and (altnm.iso_language = 'pt' or altnm.iso_language = 'link' or altnm.iso_language is null or altnm.iso_language = '')
and gn.country = 'PT'
and (gn.name like '%Lisboa%' or altnm.alternate_name like '%Lisboa%')
order by name, adm1.name, adm2.name_local;
The important/problem part of the SELECT is this one:
and (gn.name like '%Lisboa%' or altnm.alternate_name like '%Lisboa%')
I've write a simple view to test the SELECT, it looks like this:
def get_citiesjson_view(request):
word = "Lisboa"
term = "%" + word + "%"
cursor = connection.cursor()
cursor.execute("select distinct on (name, adm1.name, adm2.name_local)\
gn.geonameid,\
case when altnm.iso_language = 'pt' then altnm.alternate_name else gn.name end as name,\
adm1.name as zona,\
adm2.name_local as municipio\
from location_geonameslocal gn\
join location_geonameszone adm1 on adm1.code = gn.country || '.' || gn.admin1\
join location_geonamesmunicipality adm2 on adm2.code = gn.country || '.' || gn.admin1 || '.' || gn.admin2\
left join location_geonamesalternatenames altnm on altnm.geonameid = gn.geonameid\
where\
(gn.fclass = 'P' or gn.fclass = 'A')\
and (altnm.iso_language = 'pt' or altnm.iso_language = 'link' or altnm.iso_language is null or altnm.iso_language = '')\
and gn.country = 'PT'\
and (gn.name like %s or altnm.alternate_name like %s)\
order by name, adm1.name, adm2.name_local;", [term, term])
data = cursor.fetchone()
mimetype = 'application/json'
return HttpResponse(data, mimetype)
Unfortunately this does not work and I can't find way to make it work. Some clues?
UPDATE: This form is actually working:
cursor.execute("... and (gn.name like %s or altnm.alternate_name like %s)... ", ['%'+term+'%', '%'+term+'%'])
This form is actually working:
cursor.execute("... and (gn.name like %s or altnm.alternate_name like %s)... ", ['%'+term+'%', '%'+term+'%'])
You should not use the default Python formatting to construct SQL query with parameters, to use the raw SQL LIKE clause you could do something like this:
sql = 'SELECT id FROM table WHERE 1 = 1'
params = []
if 'descricao' in args.keys(): # your validation
# build sql query and params correctly
sql += ' AND descricao LIKE %s'
params.append('%'+args['descricao']+'%')
with connections['default'].cursor() as cursor:
cursor.execute(sql, params)
This way you will be safe agaist SQL Injection vulnerability