Update multiple results at once in Django - django

I want to update budget of Category model in Django.
class Category(models.Model):
user = models.ForeignKey(User)
name = models.CharField(max_length=128)
budget = models.DecimalField(default=0.0, decimal_places=2, max_digits=12, help_text="Amount in dollars ($)")
I would have a list of new budget like this which is equal to number of data in Category:
>>> new_category_budget
[Decimal('2700.000'), Decimal('750.000'), Decimal('1500.000')]
I am updating like this:
>>> for budget in new_category_budget:
... Category.objects.filter(user=2).update(budget=budget)
...
3L
3L
3L
But all of these return the same data. What's wrong with my update statement?

What you're doing is iterating over your budget values, and updating all records with user=2 to each one, overriding the previous value.
The return value of QuerySet.update is the number of updated records. Each time you call update(), you get the result that 3 records were updated.
I don't quite understand what you are actually trying to do, but it might be something like this (untested!):
for (budget, category) in zip(new_category_budget, list(Category.objects.filter(user=2)):
category.budget=budget
category.save()
Of course, this assumes that the number of filtered categories will exactly match the number of budgets in new_category_budget, and also the order of iteration over categories is not obvious. All in all, this seems weird :)

When you call update on a QuerySet, it will set all items in the QuerySet to that value. See the example here.
So through your for loop, you are updating all the Category objects with user=2 to each budget. At the end of the for loop, all the Category objects should be have budget == new_category_budget[-1] or the last budget item.
If you want different values for each Category object, you'll need to call save on them individually.

Related

Filter with order for two fields

I have Model Klass with fields like this:
date_start = models.DateField(null=True, blank=True, default=date.today)
date_finish = models.DateField(null=True, blank=True)
As you see, date_start will be usually filled but date_finish may not. If neither one is filled we should not consider this record in further filtering.
I would like to see objects (first N results) ordered by latest date regardless if that's date_start or date_finish. To be exact: considered date shall be date_finish if exists, date_start otherwise.
Please note, that I don't want N/2 finished items and N/2 only started items concatenated but recent N "touched" ones.
My first idea is to provide this extra field of considered_date that would be filled as I proposed but I don't know how to implement this. Shall it be done:
on Model level, so new DateField to be added and have it's content always filled with sth
selected for 2 seperate conditions (N elements each), then provided with temporary extra field (but without saving into db), then 2 sets joined and ordered by again for this new condition
Fun fact: I also have BooleanField that indicates if period is closed or not. I needed it for simplicity and filtering but we could use this here as well. It's obviously handled by save() function (default True, set to False if date_finish gets filled).
To be honest this "feature" is not critical in my app. It's just displaying some "latest changes" on welcome page so can be triggered quite often.
It seems that your description of considered_date is a perfect use case for COALESCE sql function, which returns first not NULL value from its arguments.
So, in plain SQL this will return value of date_finish if it is not NULL or date_start otherwise (assuming date_start is never NULL as it has the default value)
COALESCE(your_table_name.date_finish, your_table_name.date_start);
Django ORM has an API for this function. Using it with annotate queryset method, we can build the query you want without creating extra fields on the model.
Let's call your model TestModel just for convenience
from django.db.models.functions import Coalesce
TestModel.objects.annotate(
latest_touched_date=Coalesce("date_finish", "date_start")
).order_by("-latest_touched_date")
Please note, this will work only if date_finish is bigger than date_start. (which I think is true)
You can add exclude to filter any records where both date_finish and date_start are NULLs
from django.db.models.functions import Coalesce
TestModel.objects.exclude(date_start__isnull=True, date_finish__isnull=True).annotate(
latest_touched_date=Coalesce("date_finish", "date_start")
).order_by("-latest_touched_date")
Just slice the queryset to get first N results.

django paginate with non-model object

I'm working on a side project using python and Django. It's a website that tracks the price of some product from some website, then show all the historical price of products.
So, I have this class in Django:
class Product(models.Model):
price = models.FloatField()
date = models.DateTimeField(auto_now = True)
name = models.CharField()
Then, in my views.py, because I want to display products in a table, like so:
+----------+--------+--------+--------+--------+....
| Name | Date 1 | Date 2 | Date 3 |... |....
+----------+--------+--------+--------+--------+....
| Product1 | 100.0 | 120.0 | 70.0 | ... |....
+----------+--------+--------+--------+--------+....
...
I'm using the following class for rendering:
class ProductView(objects):
name = ""
price_history = {}
So that in my template, I can easily convert each product_view object into one table row. I'm also passing through context a sorted list of all available dates, for the purpose of constructing the head of the table, and getting the price of each product on that date.
Then I have logic in views that converts one or more products into this ProductView object. The logic looks something like this:
def conversion():
result_dict = {}
all_products = Product.objects.all()
for product in all_products:
if product.name in result_dict:
result_dict[product.name].append(product)
else:
result_dict[product.name] = [product]
# So result_dict will be like
# {"Product1":[product, product], "Product2":[product],...}
product_views = []
for products in result_dict.values():
# Logic that converts list of Product into ProductView, which is simple.
# Then I'm returning the product_views, sorted based on the price on the
# latest date, None if not available.
return sorted(product_views,
key = lambda x: get_latest_price(latest_date, x),
reverse = True)
As per Daniel Roseman and zymud, adding get_latest_price:
def get_latest_price(date, product_view):
if date in product_view.price_history:
return product_view.price_history[date]
else:
return None
I omitted the logic to get the latest date in conversion. I have a separate table that only records each date I run my price-collecting script that adds new data to the table. So the logic of getting latest date is essentially get the date in OpenDate table with highest ID.
So, the question is, when product grows to a huge amount, how do I paginate that product_views list? e.g. if I want to see 10 products in my web application, how to tell Django to only get those rows out of DB?
I can't (or don't know how to) use django.core.paginator.Paginator, because to create that 10 rows I want, Django needs to select all rows related to that 10 product names. But to figure out which 10 names to select, it first need to get all objects, then figure out which ones have the highest price on the latest date.
It seems to me the only solution would be to add something between Django and DB, like a cache, to store that ProductView objects. but other than that, is there a way to directly paginate produvt_views list?
I'm wondering if this makes sense:
The basic idea is, since I'll need to sort all product_views by the price on the "latest" date, I'll do that bit in DB first, and only get the list of product names to make it "paginatable". Then, I'll do a second DB query, to get all the products that have those product names, then construct that many product_views. Does it make sense?
To clear it a little bit, here comes the code:
So instead of
#def conversion():
all_products = Product.objects.all()
I'm doing this:
#def conversion():
# This would get me the latest available date
latest_date = OpenDate.objects.order_by('-id')[:1]
top_ten_priced_product_names = Product.objects
.filter(date__in = latest_date)
.order_by('-price')
.values_list('name', flat = True)[:10]
all_products_that_i_need = Product.objects
.filter(name__in = top_ten_priced_product_names)
# then I can construct that list of product_views using
# all_products_that_i_need
Then for pages after the first, I can modify that [:10] to say [10:10] or [20:10].
This makes the code pagination easier, and by pulling appropriate code into a separate function, it's also possible to do Ajax and all those fancy stuff.
But, here comes a problem: this solution needs three DB calls for every single query. Right now I'm running everything on the same box, but still I want to reduce this overhead to two(One or Opendate, the other for Product).
Is there a better solution that solves both the pagination problem and with two DB calls?

Multi field and computed value

I need a solution for this problem:
I would like to have MultiField widget for a "value" field. This field should allow to input two values (two input widgets), and user should choose which of these values is to be saved (two checkbox widgets). However, one of these values must be recalculated with respect to other field's value.
I've taken this approach:
a MultiValueField with 4 fields:
class PriceDetailField(MultiValueField):
use_net = BooleanField(required=False)
net_value = DecimalField(required=False, decimal_places=2)
use_gross = BooleanField(required=False)
gross_value = DecimalField(required=False, decimal_places=2)
a MultiWidget:
class PriceDetailWidget(MultiWidget):
use_net = CheckboxInput()
net_value_widget = TextInput()
use_gross = CheckboxInput()
gross_value_widget = TextInput()
and a custom Form...
class PriceModelForm(ModelForm):
value = PriceDetailField()
...which overrides default model form for a model:
class Price(models.Model):
value = models.DecimalField(
max_digits=19,
decimal_places=2,
default=Decimal(0),
)
However, this solution seems to be messed up. In the form, I need to input all subfields (the MultiValueField subfields), otherwise "Please enter value" error appears (even though those fields are marked as required=False). Also,
I must recalculate the mentioned value upon a save, having returned a tuple from the field with information which checkbox was checked and the corresponding text value, then replace the tuple with the decimal value in clean_value method of form (also, saving the checkboxes state in temporary fields....). I think such a design is very weak.
The form should work both on its own and as inline (this means, the value of the field which is used to calculate the returned value can or cannot change during save).
Is such a thing even possible?
And the root of the problem: I want to store prices of items as net prices, but I would like to allow users to input them as net or gross prices, and then recalculate gross price to net price with respect to product's VAT tax level (since VAT is assigned to product or service, not to the price). A product can have many prices, so a price is backlinked to the product by a foreign key.
Cheers,
Tomek
The feature you're looking for on a MultiValueField (allowing the sub-fields to be required or not individually) is logged as a feature request on Django.
You could probably fix this for now by subclassing MultiValueField with a rewritten clean method based on the original and following suggestions from the bug report. You're not going to have a lot of the original MultiValueField left by that point though, and if you do, you should submit your patch to Django and put a note on the bug.
Ignoring for a moment the requiredness thing, have you written a compress method on your PriceDetailField? What does it look like? This is where you should be doing the work to turn your four sub-fields into a single field to save.

Django both filtered and total number of a certain field

I have such model and query
class Employer(Models.model)
name = ...
class JobTitle(Models.model)
name = ...
employer = models.ForeignKey(Employer)
and query is
Employer.objects.select_related('jobtitle')
.filter(jtt__activatedate__range=[startdate,enddate])
.annotate(jtt_count=Count('jobtitle'))
.order_by('-jtt_count')[:5]
As you see it returns 5 employer list which has maximum number of jobtitles which are related to that employer and whose activation date is in some certain range.
However, I also want to get the total number of jobtitles of each employer in that query.
Of course I may loop over each employer and make such query JobTitle.objects.filter(employer = emp) and taking length of that query but it is bad solution.
How can I achive this in that query?
Although it may not be possible to get both total number and filtered number of job titles, I may get the jobttiles of each emplyoer such that len(emp.jobtitle) however it also didn't work.
Thanks
Try the extra lookup. So, in your case it may be like this:
.extra(
select={
'jobtitle_count': 'SELECT COUNT(*) FROM YOURAPP_jobtitle WHERE YOURAPP_jobtitle.employer_id = YOURAPP_employer.id'
},
)

Query for a ManytoMany Field with Through in Django

I have a models in Django that are something like this:
class Classification(models.Model):
name = models.CharField(choices=class_choices)
...
class Activity(models.Model):
name = models.CharField(max_length=300)
fee = models.ManyToManyField(Classification, through='Fee')
...
class Fee(models.Model):
activity = models.ForeignKey(Activity)
class = models.ForeignKey(Classification)
early_fee = models.IntegerField(decimal_places=2, max_digits=10)
regular_fee = models.IntegerField(decimal_places=2, max_digits=10)
The idea being that there will be a set of fees associated with each Activity and Classification pair. Classification is like Student, Staff, etc.
I know that part works right.
Then in my application, I query for a set of Activities with:
activities = Activity.objects.filter(...)
Which returns a list of activities. I need to display in my template that list of Activities with their Fees. Something like this:
Activity Name
Student Early Price - $4
Student Regular Price - $5
Staff Early Price - $6
Staff Regular Price - $8
But I don't know of an easy way to get this info without a specific get query of the Fees object for each activity/class pair.
I hoped this would work:
activity.fee.all()
But that just returns the Classification Object. Is there a way to get the Fee Object Data for the Pair via the Activities I already queried?
Or am I doing this completely wrong?
Considering michuk's tip to rename "fee" to "classification":
Default name for Fee objects on Activity model will be fee_set. So in order to get your prices, do this:
for a in Activity.objects.all():
a.fee_set.all() #gets you all fees for activity
There's one thing though, as you can see you'll end up doing 1 SELECT on each activity object for fees, there are some apps that can help with that, for example, django-batch-select does only 2 queries in this case.
First of all I think you named your field wrong. This:
fee = models.ManyToManyField(Classification, through='Fee')
should be rather that:
classifications = models.ManyToManyField(Classification, through='Fee')
as ManyToManyField refers to a list of related objects.
In general ManyToManyField, AFAIK, is only a django shortcut to enable easy fetching of all related objects (Classification in your case), with the association table being transparent to the model. What you want is the association table (Fee in your case) not being transparent.
So what I would do is to remove the ManyToManyField field from Activity and simply get all the fees related with the activity. And thenm if you need a Classification for each fee, get the Classification separately.