Django query assigns database columns to wrong model instance properties - django

Really weird problem - when I query for a model instance the data comes back assigned to the wrong properties.
The model:
class SaleLineItem(models.Model):
sale = models.ForeignKey(Sale, on_delete=models.CASCADE, related_name="sale_line_items")
stock_unit = models.ForeignKey(StockUnit, on_delete=models.CASCADE, related_name="sale_line_items")
currency = models.CharField(max_length=3)
price_original = models.FloatField()
price_paid = models.FloatField()
tax_amount = models.FloatField(null=True, blank=True)
num_sold = models.IntegerField()
sale_line_item_id = models.CharField(max_length=30, null=True, blank=True)
status = models.CharField(max_length=20, choices=SALE_STATUS_CHOICES, null=True, blank=True)
The database row:
id | currency | price_original | price_paid | tax_amount | num_sold | sale_line_item_id | status | sale_id | stock_unit_id
-------+----------+----------------+------------+------------+----------+-------------------+-----------+---------+---------------
15726 | THB | 130 | 130 | | 1 | | delivered | 16219 | 2
And the query:
sli = SaleLineItem.objects.get(pk=15726)
print(sli.pk)
-------------------------
16219
print(sli.stock_unit_id)
-------------------------
THB
print(sli.currency)
-------------------------
130.0
The data get populated on the object but everything is "shifted" by one column.
But if I do the query this way:
SaleLineItem.objects.filter(pk=15726).values()
-------------------------
<QuerySet [{'id': 15726, 'sale_id': 16219, 'stock_unit_id': 2, 'currency': 'THB', 'price_original': 130.0, 'price_paid': 130.0, 'tax_amount': None, 'num_sold': 1, 'sale_line_item_id': None, 'status': 'delivered'}]>
. . . the result is correct.
I thought I might have un-migrated models but I ran both makemigrations and migrate to no effect.
Same result when I use lower-level QuerySet methods:
qs = SaleLineItem.objects.all()
clone = qs._chain()
clone.query.add_q(Q(pk=15726))
print(clone)
------------------------------
<QuerySet [<SaleLineItem: SaleLineItem object (16219)>]>
Note the pk on the model __str__ is incorrect.
Any ideas what's happening here?
Running:
Python 3.7.3
Django 2.2.1
Postgres 10

Turns out it's because I overrode __init__ with an extra (non-field) argument.
#classmethod
def from_db(cls, db, field_names, values):
if len(values) != len(cls._meta.concrete_fields):
values_iter = iter(values)
values = [
next(values_iter) if f.attname in field_names else DEFERRED
for f in cls._meta.concrete_fields
]
new = cls(*values)
new._state.adding = False
new._state.db = db
return new
Database values are populated onto the model using *values, and the model expects fields in a specific order. So you can't have an extra argument in __init__ or the order gets messed up.
Edit:
Had not read this part in the docs (https://docs.djangoproject.com/en/2.1/ref/models/instances/):
You may be tempted to customize the model by overriding the __init__
method. If you do so, however, take care not to change the calling
signature . . .

Related

How to get count of subjects taken by each student?

I've two models 'Students' and 'Enrollments'.
The schema for these is as below:
class Students(models.Model):
id = models.AutoField(primary_key=True, unique=True)
name = models.CharField()
class Enrollments(models.Model):
enroll_id = models.AutoField(primary_key=True, unique=True)
student_id = models.ForeignKey(Students, on_delete=models.CASCADE)
subjects = models.charField()
I'm trying to achieve the result of following SQL query in Django Rest Framework, for getting number of subjects enrolled by students (individually).
select
s.id, s.name, count(e.subjects) as count
from Students as s
left outer join Enrollments as e
on e.student_id_id = s.id
group by s.id, s.name, e.subjects
order by count asc;
This query returns result like:
---------------------------
| id | name | count |
---------------------------
| 1 | a | 1 |
| 2 | b | 0 |
| 3 | c | 2 |
---------------------------
Can anyone please help me acheive this kind of result.
Note: I need 0 count students details also.
What you can do is when you are creating a serializer, you can add a serializer method field which will get the count for you.
Add this at the top of your serializer:
count = serializers.SerializerMethodField('get_count')
Then add a function inside your serializer like this:
def get_count(self, obj):
try:
return Enrollments.objects.filter(student_id=obj.id).count()
except:
return None
Finally, add 'count' to your field list. You can then add as many fields as you want. I hope this will get you your desired result. Also don't forget to use "select_related" in the ORM inside your view to reduce the amount of queries.

Get latest payment from related model in Django Admin

I'd like to output a table using "Premises" model in Django admin. In addition I'd like include the output in this table of an additional column, say "last utility payment". It is actually a column in a related table. There may be no payment in the database so admin should able to see either an empty cell or the date of payment.
I was able to write a DB query that displays the information I need. Its significant and worked part is given below:
SELECT jp.id,
jp.number apartment,
jp.building_number building,
jp.rent,
jp.arrears,
jpm.last_payment
FROM jasmin_premises jp
LEFT JOIN (
SELECT pm.premises_id,
max(pm.paid) last_payment
FROM jasmin_payment pm
GROUP BY pm.premises_id
) jpm ON jp.id = jpm.premises_id;
And the output is similar to the following:
id | apartment | building | rent | arrears | last_payment
--------------------------------------------------------------
170 | 1X | 6-A | 297.43 | 2.57, | NULL
72 | 2 | 4 | 289.66 | -678.38 | 2021-01-31
173 | 3Z | 7 | 432.86 | 515.72 | 2021-02-04
73 | 4 | 8-B | 292.25 | 515.44 | 2021-02-04
74 | 5 | 8-B | 112.42 | 3249.34 | NULL
75 | 6A | 122 | 328.48 | 386.23 | 2021-02-04
76 | 7 | 42 | 482.06 | 964.12 | 2021-01-31
77 | 8 | 1 | 433.71 | 867.42 | 2021-01-31
78 | 9C | 12 | 322.79 | 322.79 | 2021-02-04
79 | 10 | 122 | 324.22 | 0 | 2021-02-04
80 | 12 | 12 | 322.79 | 1232.46 | NULL
81 | 14 | 5-Z | 440.82 | 978.44 | 2021-02-04
And I'm using the following models (only a significant part):
class Premises(models.Model):
number = models.CharField(
blank=False,
null=False,
max_length=10)
building_number = models.CharField(
blank=False,
null=False,
max_length=3)
rent = models.DecimalField(
blank=False,
null=False,
max_digits=12,
decimal_places=2,
default=0.0)
area = models.DecimalField(
blank=False,
null=False,
max_digits=5,
decimal_places=2)
class Payment(models.Model):
paid = models.DateField(
blank=False,
null=False)
premises = models.ForeignKey(
Premises,
blank=False,
null=False,
on_delete=models.CASCADE,
related_name='payments',
db_index=True)
Is there a way to override admin.ModelAdmin.get_queryset (for example using annotations) to get an extra column like in my example above? Is there any other way to make a LEFT JOIN on a compound DB query using Django ORM?
to make this query in django you have to add the models.Manager() to the tables like this:
models.py
class Premises(models.Model):
# existent code
objects = models.Manager()
class Payment(models.Model):
# existent code
objects = models.Manager()
In the part of app you want to access this information
from .models import Premises, Payment
premises = Premises.objects.all()
data_to_display = []
for premise in premises:
payments = Payment.objects.filter(premises=premise).order_by('-paid')
if len(payments) == 0:
last_payment = "Null"
else:
last_payment = payments[0]
object_to_list = {
"id": premise.id,
"apartment": premise.number,
"building": premise.building_number,
"rent": premise.rent,
"arreaars": premise.area,
"last_payment": last_payment.paid
}
data_to_display.append(object_to_list)
The solution is to add an explicit subquery to a QuerySet using the Subquery expression. We'll need also to use OuterRef because a queryset in a Subquery needs to refer to a field from the outer query.
So let's create a subquery:
from django.db.models import OuterRef
payments = Payment.objects.filter(
premises=OuterRef('pk')
).order_by('-paid')
The next step is to pass payments subquery to a queryset:
from django.db.models import Subquery
# 'payments' here is from example above
premises = Premises.objects.annotate(
last_payment=Subquery(payments.values('paid')[:1])
)
Finally, lets see the used SQL to query objects rows the database:
print(premises.query)
(the output is formatted, only the significant part is shown)
SELECT "jasmin_premises"."id",
"jasmin_premises"."number",
"jasmin_premises"."building_number",
"jasmin_premises"."arrears",
"jasmin_premises"."rent",
(SELECT U0."paid"
FROM "jasmin_payment" U0
WHERE U0."premises_id" = "jasmin_premises"."id"
ORDER BY U0."paid" DESC
LIMIT 1) AS "last_payment"
FROM "jasmin_premises";
Now, after performing the tests, we can use this in our ModelAdmin:
from django.contrib import admin
from django.db.models import OuterRef, Subquery
from .models import Payment, Premises
#admin.register(Premises)
class PremisesAdmin(admin.ModelAdmin):
list_display = (
'number',
'building_number',
'rent',
'arrears',
'last_payment',
)
def get_queryset(self, request):
qs = super().get_queryset(request)
payments = Payment.objects.filter(
premises=OuterRef('pk')
).order_by('-paid')
qs = qs.annotate(
last_payment=Subquery(payments.values('paid')[:1]),
)
return qs
def last_payment(self, obj):
return obj.last_payment
last_payment.short_description = 'Last payment'
last_payment.admin_order_field = 'last_payment'
Well, this doesn't use JOINs, but this approach will force Django to execute a subquery.
Probably in some cases it may be possible to write an equivalent queryset that performs the same task more clearly or efficiently however, this is the best I have achieved so far.

Django trying to add "_id" to the primary key OneToOne column

Just started Django.
I have 2 models. Radusergroup and expiration. username is primary key on Radusergroup and a OnetoOne Field in expiration with primary_key=True. Django is trying query for username_id in expiration model although the field itself is username only.
When I dont explicitly define Managed=False it also tries to change the username field in expiration table from the database to username_id as well.
What am I doing wrong here ?
class Radusergroup(models.Model):
username = models.CharField(max_length=64,primary_key=True)
groupname = models.CharField(max_length=64)
priority = models.IntegerField()
class Meta:
managed = False
class expiration(models.Model):
username = models.OneToOneField(Radusergroup,on_delete=models.CASCADE, to_field='username', primary_key=True)
expiration = models.DateTimeField()
class Meta:
managed = False
python .\manage.py shell
>>> help(expiration())
Help on expiration in module panel_app.models object:
class expiration(django.db.models.base.Model)
| expiration(*args, **kwargs)
|
| expiration(username, expiration)
|
| Method resolution order:
| expiration
| django.db.models.base.Model
| builtins.object
|
| Methods defined here:
|
| expiration = <django.db.models.query_utils.DeferredAttribute object>
| get_next_by_expiration = _method(self, *, field=<django.db.models.fields.DateTimeField: expiration>, is_next=True, **
kwargs)
|
| get_previous_by_expiration = _method(self, *, field=<django.db.models.fields.DateTimeField: expiration>, is_next=Fals
e, **kwargs)
|
| username_id = <django.db.models.query_utils.DeferredAttribute object>
| ----------------------------------------------------------------------
Thanks dirkgroten for pointing me to the right direction.
Adding db_column solved my problem
class expiration(models.Model):
username = models.OneToOneField(Radusergroup,on_delete=models.PROTECT, to_field='username',db_column="username" , primary_key=True)

Django (1.11) ORM using postgresql query is not able to compare date

Django (1.11) ORM using postgresql query is not able to compare date. I need data in between of leave_date_from and leave_date_to.
I'm expecting one record.
My model is as follows:
class EmployeeLeaveApp(models.Model):
user = models.ForeignKey(User, models.DO_NOTHING, on_delete=models.CASCADE)
leave_date_from = models.DateField(blank=True, null=True)
leave_date_to = models.DateField(blank=True, null=True)
My View query is :
inputdate = '2018-03-05'
duplicateleave = crm_models.EmployeeLeaveApp.objects.filter(
leave_date_from__gte=inputdate,
leave_date_to__lte=inputdate,
user=user)
My table data is like below:
leave_date_from | leave_date_to | user
2018-03-01 | 2018-03-10 | 1
2018-03-07 | 2018-03-22 | 1
So far, I tried many solutions but no luck.
from datetime import datetime
inputdate = datetime.strptime(inputdate, '%Y-%m-%d')
duplicateleave = crm_models.EmployeeLeaveApp.objects.filter(
leave_date_from__gte=inputdate,
leave_date_to__lte=inputdate,
user=user
)
==== OR ==== If error comes must be str, not datetime.date then use.
inputdate = datetime.strptime(str(inputdate), '%Y-%m-%d')
duplicateleave = crm_models.EmployeeLeaveApp.objects.filter(
leave_date_from__gte=inputdate,
leave_date_to__lte=inputdate,
user=user
)
django expects a dateobject in the parameter, but you are passing string to it. so use this

Django: How to do Aggregate (GroupBy ID) & Latest timestamp?

I have a table like this:
ID | Time Stamp
1 | 2012-07-28 18:57:48.160912+01
1 | 2012-07-28 20:57:43.063327+01
2 | 2012-07-28 21:17:16.016665+01
I would like to see the latest entry of each id.
If I did this, I would get only one object with the very latest entry:
open_deals = all_deals.latest('time_stamp'))
--> 2 | 2012-07-28 21:17:16.016665+01
But I would like to get
--> 1 | 2012-07-28 20:57:43.063327+01
2 | 2012-07-28 21:17:16.016665+01
I need somehow to aggregate or Groupby the ID. But there is no function for that in the documentation.
Any tips? Thank you
Update:
I have tried the solution below:
result_list = [deal.dealchangelog_set.latest('time_stamp') for deal in open_deals]
result_set = set()
for item in result_list:
result_set.add(item.pk)
return open_deals.filter(pk__in = result_set)
Unfortunatelly as you can see the list still contains three objects instead of two. :-(
Here are my models (beware I am not using the deal_id as pk) The pk is still as integers.
In my case, I need to get the latest of the deal_id, which isn't unique. (For sake of simplicity I had shown previously the uuid in here as integer)
class Deal(models.Model):
deal_id = UUIDField()
status = models.ForeignKey(DealStatus, verbose_name=_(u"Deal Status"), null=True, blank=True)
contact = models.ForeignKey(Contact)
deal_type = models.ForeignKey(DealType)
class DealChangeLog(models.Model):
deal = models.ForeignKey(Deal)
time_stamp = CreationDateTimeField()
Update 2:
def get_open_deals(call):
all_deals = Deal.objects.filter(contact=call.contact)
closed_deals = all_deals.filter(status__in=[5, 6])
closed_deal_list = []
if closed_deals:
for item in closed_deals:
closed_deal_list.append(item.deal_id)
open_deals = all_deals.exclude(deal_id__in=closed_deal_list)
result_list = [deal.dealchangelog_set.latest('time_stamp') for deal in open_deals]
result_set = set()
for item in result_list:
result_set.add(item.pk)
return open_deals.filter(pk__in = result_set)
I'm not context aware but I would probably break up the model like this:
class Deal(models.Model):
title = models.CharField()
# maybe signal changes etcetera...
class DealChangeLog(models.Model):
deal = models.ForeignKey(Deal)
date = models.DateTimeField(auto_now=True, auto_now_add=True)
Then you could achieve your result by list comprehension:
results = [deal.dealchangelog_set.latest('date') for deal in Deal.objects.all()]
To get the results as a queryset you can do:
Deal.objects.filter(id__in=[result.deal.id for result in results])
But if you can work with the results as a list I don't see much point for the extra queries