Django update rows base on order in queryset - django

I have a simple django model
class Item(Model):
name = CharField()
rank = PositiveIntegerField()
created_at = DateField(auto_now_add=True)
I want to update the object rank based on their order when sorted by a field (name or created_at)
e.g. when ordering by name
[("Pen", 0, "2021-05-04"), ("Ball", 0, "2021-05-04")] => [("Pen", 1, "2021-05-04"), (Ball, 0, "2021-05-04")]
I already know I can do this using bulk_update but it means I have to fetch the objects in memory
items = Items.objects.order_by("name")
for i, item in enumerate(items):
item.rank = i
Item.objects.bulk_update(items, ["rank"])
I was wondering if there is a way to do it with 1 query directly in the database, without having to fetch the data

CREATE TABLE items (
id serial PRIMARY KEY,
name VARCHAR ( 50 ) UNIQUE NOT NULL,
rank smallint
);
insert into items(id, name, rank) values (1, 'A', 0);
insert into items(id, name, rank) values (2, 'B', 0);
insert into items(id, name, rank) values (3, 'C', 0);
select * from items;
id
name
rank
1
A
0
2
B
0
3
C
0
UPDATE items
SET rank=calculated.calc_rank
FROM
(SELECT id AS calc_id,
(ROW_NUMBER() OVER (
ORDER BY name ASC)) AS calc_rank
FROM items) AS calculated
WHERE items.id = calculated.calc_id;
select * from items;
id
name
rank
1
A
1
2
B
2
3
C
3
And perform raw SQL for Django:
sql = '''
UPDATE items
SET rank=calculated.calc_rank
FROM
(SELECT id AS calc_id,
(ROW_NUMBER() OVER (
ORDER BY name ASC)) AS calc_rank
FROM items) AS calculated
WHERE items.id = calculated.calc_id
'''
with connection.cursor() as cursor:
cursor.execute(sql)

Related

Get top n records for each group with Django queryset

I have a model like the following Table,
create table `mytable`
(
`person` varchar(10),
`groupname` int,
`age` int
);
And I want to get the 2 oldest people from each group. The original SQL question and answers are here StackOverflow and One of the solutions that work is
SELECT
person,
groupname,
age
FROM
(
SELECT
person,
groupname,
age,
#rn := IF(#prev = groupname, #rn + 1, 1) AS rn,
#prev := groupname
FROM mytable
JOIN (SELECT #prev := NULL, #rn := 0) AS vars
ORDER BY groupname, age DESC, person
) AS T1
WHERE rn <= 2
You can check the SQL output here as well SQLFIDLE
I just want to know how can I implement this query in Django's views as queryset.
Another SQL with similar output would have window function that annotates each row with row number within particular group name and then you would filter row numbers lower or equal 2 in HAVING clause.
At the moment of writing django does not support filtering based on window function result so you need to calculate row in the first query and filter People in the second query.
Following code is based on similar question but it implements limiting number of rows to be returned per group_name.
from django.db.models import F, When, Window
from django.db.models.functions import RowNumber
person_ids = {
pk
for pk, row_no_in_group in Person.objects.annotate(
row_no_in_group=Window(
expression=RowNumber(),
partition_by=[F('group_name')],
order_by=['group_name', F('age').desc(), 'person']
)
).values_list('id', 'row_no_in_group')
if row_no_in_group <= 2
}
filtered_persons = Person.objects.filter(id__in=person_ids)
For following state of Person table
>>> Person.objects.order_by('group_name', '-age', 'person').values_list('group_name', 'age', 'person')
<QuerySet [(1, 19, 'Brian'), (1, 17, 'Brett'), (1, 14, 'Teresa'), (1, 13, 'Sydney'), (2, 20, 'Daniel'), (2, 18, 'Maureen'), (2, 14, 'Vincent'), (2, 12, 'Carlos'), (2, 11, 'Kathleen'), (2, 11, 'Sandra')]>
queries above return
>>> filtered_persons.order_by('group_name', '-age', 'person').values_list('group_name', 'age', 'person')
<QuerySet [(1, 19, 'Brian'), (1, 17, 'Brett'), (2, 20, 'Daniel'), (2, 18, 'Maureen')]>

Django ListView queryset for available items

So I want to display all available items for any given date, shouldn't be that hard but somehow I ran into a problem concerning related items.
Let's say we have the following models, a model to store all bookings and a model with the Item.
Then I would create ListView to retrieve all items available between any given dates. I override the queryset to retrieve the data filled in by the user.
This seems to be working but there's an issue, even though I check if the "form_start_date" or "form_end_data" are in conflict with existing bookings, when a single Item has multiple bookings it does not work.
example
Bookings [X] for item #01:
01-01-2019 to 01-03-2019
01-11-2019 to 01-18-2019
Jan 2019 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
---------- --- --- --- --- --- --- --- --- --- ---- ---- ---- ---- ---- ---- ---- ---- ----
Item #01 X X X O O O X X X X X X X X
Item #02 X X X X X
When I check for availablity [O] for 01-06-2019 to 01-08-2019, item #01 is not available, what am I missing here?
models.py
class Booking(models.Model):
item = models.ForeignKey('Item', on_delete=models.SET_NULL, null=True)
start_date = models.DateField()
end_date = models.DateField()
class Item(models.Model):
name = models.CharField(max_length=20, unique=True)
views.py
class AvailableItems(generic.ListView):
model = Item
def get_queryset(self):
start_date = datetime.strptime(self.request.GET.get('start_date'), '%Y-%m-%d')
end_date = datetime.strptime(self.request.GET.get('end_date'), '%Y-%m-%d')
# As Willem suggested in the comments, it is easier to check for available items
available_items = (
Item.objects.filter(booking__start_date__gte = start_date, booking__end_date__lte = end_date)
)
if start_date and end_date:
return available_items
else:
return Item.objects.all()
Let us first analyze when two intervals (f1, t1) and (f2, t2) overlap. A simpler problem to solve, is finding out when two intervals do not overlap. That holds for two cases:
given t1 < f2, since then the first event ends before the second; or
given t2 < f1, since then the first event ends before the second begins.
So that means that two events overlap given t1 ≥ f2 and t2 ≥ f1.
With this knowledge, we can design a filter like:
bookings = Booking.objects.filter(
end_date__gte=form_start_date,
start_date__lte=form_end_date
)
return Item.objects.exclude(
booking__in=bookings
)
This then results in a query like:
SELECT item.*
FROM item
WHERE NOT (
item.id IN (
SELECT V1.item_id
FROM booking V1
WHERE (V1.id IN (
SELECT U0.id FROM booking U0
WHERE (U0.end_date >= 2019-01-01 AND U0.start_date <= 2019-02-02)
AND V1.item_id IS NOT NULL
)
)
)
(here 2019-01-01 and 2019-02-02 are hypothetical start and end dates).
I think it is probably better to process the two dates through a Form however to do proper validation and cleaning.
For example if we populate an empty database with the data as provided in the question, we get:
>>> i1 = Item.objects.create(name='Item #01')
>>> i2 = Item.objects.create(name='Item #02')
>>> b1 = Booking.objects.create(item=i1, start_date=)
KeyboardInterrupt
>>> from datetime import date
>>> b1 = Booking.objects.create(item=i1, start_date=date(2019,1,1), end_date=date(2019, 1, 3))
>>> b2 = Booking.objects.create(item=i1, start_date=date(2019,1,11), end_date=date(2019, 1, 18))
>>> bookings = Booking.objects.filter(
... end_date__gte=date(2019, 1, 6),
... start_date__lte=date(2019, 1, 8)
... )
>>> Item.objects.exclude(
... booking__in=bookings
... )
<QuerySet [<Item: Item object (2)>, <Item: Item object (3)>]>
>>> b3 = Booking.objects.create(item=i2, start_date=date(2019,1,2), end_date=date(2019, 1, 6))
>>> bookings = Booking.objects.filter(
... end_date__gte=date(2019, 1, 6),
... start_date__lte=date(2019, 1, 8)
... )
>>> Item.objects.exclude(
... booking__in=bookings
... )
<QuerySet [<Item: Item object (2)>]>
So first I constructed two items, and made two bookings on the first item. If we then make a query, we see that both items pop up. If we then add an extra booking for Item #02, then if we perform the query again, we see that only the first item (I first made an item for test purposes that was then removed) is showing up, since for the given range, the second item is no longer available: it has been booked by booking b3.

How to use sqlite to get values from 2 tables

I have made 2 tables using sqlite in python and i think ive done it right so they are linked with a foreign key. If my code is correct does anybody know how to get the values. The code from making the tables is below as well as the code for getting the values which returns nothing.
c.execute("SELECT value AND wage FROM classes, yourplayers WHERE Firstname=? AND Secondname=?", (self.Seller_Firstname, self.Seller_Lastname))
c.execute("CREATE TABLE IF NOT EXISTS classes(class INT NOT NULL, wage INT NOT NULL, value INT NOT NULL)")
c.execute("INSERT INTO classes(class, wage, value) VALUES (1, 1000, 5000000)")
c.execute("INSERT INTO classes(class, wage, value) VALUES (2, 5000, 1000000)")
c.execute("INSERT INTO classes(class, wage, value) VALUES (3, 10000, 15000000)")
c.execute("INSERT INTO classes(class, wage, value) VALUES (4, 25000, 20000000)")
c.execute("INSERT INTO classes(class, wage, value) VALUES (5, 50000, 25000000)")
c.execute("INSERT INTO classes(class, wage, value) VALUES (6, 100000, 35000000)")
c.execute("INSERT INTO classes(class, wage, value) VALUES (7, 150000, 50000000)")
c.execute("INSERT INTO classes(class, wage, value) VALUES (8, 225000, 80000000)")
c.execute("SELECT * FROM classes")
conn.commit()
print(c.fetchall())
c.execute('DROP TABLE IF EXISTS yourplayers')
c.execute('CREATE TABLE yourplayers(PlayerId INT PRIMARY KEY NOT NULL, Firstname VARCHAR(10) NOT NULL, Secondname VARCHAR(10) NOT NULL, contract INT NOT NULL, rating INT NOT NULL, class INT NOT NULL, morale INT NOT NULL, FOREIGN KEY (class) REFERENCES classes(class))')
x = 1
for x in range(20):
FN = open("firstnames", "r")
FNames = FN.read()
listF = FNames.split('\n')
y = randint(0,55)
N1 = listF[y]
N1 = str(N1)
SN = open("lastnames", "r")
SNames = SN.read()
listS = SNames.split('\n')
z = randint(0, 9)
N2 = listS[z]
N2 = str(N2)
c.execute("INSERT INTO yourplayers(PlayerId, Firstname, Secondname, contract, rating, class, morale) VALUES (? , ?, ?, 2, 55, 1, 55)",
(x, N1, N2))
x = x+1
c.execute('SELECT * FROM yourplayers')
conn.commit()
Each paragraph is in a seperate function but ive removed irrelevant code
A foreign key constraint is just a constraint, i.e., it prevents you from inserting data that would violate the constraint.
Foreign key constraints have no effect on queries; if you want to join tables through these values, you still have to write the join:
SELECT value, wage -- no AND here
FROM classes
JOIN yourplayers USING (class)
WHERE Firstname=?
AND Secondname=?;

python: Finding min values of subsets of a list

I have a list that looks something like this
(The columns would essentially be acct, subacct, value.):
1,1,3
1,2,-4
1,3,1
2,1,1
3,1,2
3,2,4
4,1,1
4,2,-1
I want update the list to look like this:
(The columns are now acct, subacct, value, min of the value for each account)
1,1,3,-4
1,2,-4,-4
1,3,1,-4
2,1,1,1
3,1,2,2
3,2,4,2
4,1,1,-1
4,2,-1,-1
The fourth value is derived by taking the min(value) for each account. So, for account 1, the min is -4, so col4 would be -4 for the three records tied to account 1.
For account 2, there is only one value.
For account 3, the min of 2 and 4 is 2, so the value for col 4 is 2 where account = 3.
I need to preserve col3, as I will need to use the value in column 3 for other calculations later. I also need to create this additional column for output later.
I have tried the following:
with open(file_name, 'rU') as f: #opens PW file
data = zip(*csv.reader(f, delimiter = '\t'))
# data = list(list(rec) for rec in csv.reader(f, delimiter='\t'))
#reads csv into a list of lists
#print the first row
uniqAcct = []
data[0] not in used and (uniqAcct.append(data[0]) or True)
But short of looping through and matching on each unique count and then going back through and adding a new column, I am stuck. I think there must be a pythonic way of doing this, but I cannot figure it out. Any help would be greatly appreciated!
I cannot use numpy, pandas, etc as they cannot be installed on this server yet. I need to use just basic python2
So the problem here is your data structure, it's not trivial to index.
Ideally you'd change it to something readible and keep it in those containers. However if you insist on changing it back into tuples I'd go with this construction
# dummy values
data = [
(1, 1, 3),
(1, 2,-4),
(1, 3, 1),
(2, 1, 1),
(3, 1, 2),
(3, 2, 4),
(4, 1, 1),
(4, 2,-1),
]
class Account:
def __init__(self, acct):
self.acct = acct
self.subaccts = {} # maps sub account id to it's value
def as_tuples(self):
min_value = min(val for val in self.subaccts.values())
for subacct, val in self.subaccts.items():
yield (self.acct, subacct, val, min_value)
def accounts_as_tuples(accounts):
return [ summary for acct_obj in accounts.values() for summary in acct_obj.as_tuples() ]
accounts = {}
for acct, subacct, val in data:
if acct not in accounts:
accounts[acct] = Account(acct)
accounts[acct].subaccts[subacct] = val
print(accounts_as_tuples(accounts))
But ideally, I'd keep it in the Account objects and just add a method that extracts the minimal value of the account when it's needed.
Here is another way using your initial approach.
Modify the way you import your data, so you can easily handle it in python.
import csv
mylist = []
with open(file_name, 'rU') as f: #opens PW file
data = csv.reader(f, delimiter = '\t')
for row in data:
splitted = row[0].split(',')
# this is in case you need integers
splitted = [int(i) for i in splitted]
mylist += [splitted]
Then, add the fourth column
updated = []
for acc in set(zip(*mylist)[0]):
acclist = [x for x in mylist if x[0] == acc]
m = min(i for sublist in acclist for i in sublist)
[l.append(m) for l in acclist]
updated += acclist

Get key value from display value from choices in Django

I have a model field with choices
CATEGORY_CHOICES = (
(0, 'A'),
(1, 'B'),
(2, 'C'),
)
I'm inserting rows in my database table where the category values could be A, B, or C.
Instead of using
if category:
if category == "A":
category = 0
elif category == "B":
category = 1
elif category == "C":
category = 2
before inserting my rows, can I somehow use my CATEGORY_CHOICES to translate display values to key values?
Build a dict of the display_value: value pairs and get the value from it:
CATEGORIES_D = {v: k for k, v in CATEGORY_CHOICES}
category = CATEGORIES_D[category]
How about this:
choices_dict = {}
for choice, value in CATEGORY_CHOICES:
choices_dict[value] = choice
category = choices_dict[category]