Suppose I have following models:
class Thing(models.Model):
name = models.CharField(max_length=100)
ratings = models.ManyToManyField('auth.User', through='Rating')
class Rating(models.Model):
user = models.ForeignKey('auth.User')
thing = models.ForeignKey('Thing')
rating = models.IntegerField()
So I have a lot of things, and every user can rate every thing. I also have a view showing a list of all things (and they are huge in numbers) with a rating that user assigned to each of them. I need a way to retreive all the data from database: Thing objects with additional field user_rating taken from at most one (because we have a fixed User) related Rating object.
Trivial solution looks like that:
things = Thing.objects.all()
for thing in things:
try:
thing.user_rating = thing.ratings.objects.get(user=request.user).rating
except Rating.DoesNotExist:
thing.user_rating = None
But the flaw of this approach is obvious: if we have 500 things, we'll do 501 requests to database. Per one page. Per user. And this is the most viewed page of the site. This task is easily solvable with SQL JOINs but in practice I have more complicated schema and I will certainly benefit from Django model framework. So the question is: is it possible to do this Django-way? It would be really strange if it isn't, considering that such tasks are very common.
As I understood, neither annotate(), nor select_related() will help me here.
I guess you should try this:
https://docs.djangoproject.com/en/1.3/ref/models/querysets/#extra
Example
result = Thing.objects.all().extra(select={'rating': 'select rating from ratings where thing_id = id'})
Your result set gets a new field 'rating' for each 'thing' object.
I use this approach in one of my recent projects. It produces one complex query instead of n+1 queries.
Hope this helps :)
Since you are planning to display everything in one page. I can think of this approach. You can give this a try:
Get all the ratings given by the current user and Get all the Things.
Now try to create a dictionary like this:
thing_dict = {}
for thing in Thing.objects.all():
thing_dict[thing] = None
for rating in Rating.objects.filter(user = request.user):
thing_dict[rating.thing] = rating
Now thing_dict contains all the entries of model Thing as keys and has its rating as its value.
May not be the best way. I am keen on seeing what others answer.
Related
few years ego I worked with Odoo framework. and Odoo has very nice feature like this:
partner_id = field.Many2one(Partner)
partner_name = fields.Char(string='Partner name', related='partner_id.name')
basically whenever you would assign different partner_id from Partner table, partner_name would be assigned automatically. Now I started to work with django (absolute newbie), and I can't seem to find a similar functionality.
My question is what could be possible solution for this problem. Maybe there are already established external libraries that has this sort of functionality?
Expected result:
product = models.ForeignKey(Product)
product_color = models.CharField(string='Partner name', related='product.color')
having in mind that product object would have color field and it would be assigned to product_color whenever product field value Product object color value changes. Also what about storing it to database? Would be nice if there was an option to chose between storing it in database or getting it on the fly.
Cheers!
Creating a getter is pretty easy, because you can simply have functions in a Python object behave as a property:
class SampleModel(models.Model):
product = models.ForeignKey(Product)
#property
def product_color(self):
return self.product.color
This does retrieve the property on the fly, which will cause a call to the database.
Duplicating data, is usually a (more severe) antipattern. Synchronizing data, even in two tables in the same database, often turns out harder than one might expect. Even if you would use Django's signal framework for example, then some Django ORM calls can circumvent that (for example .update(..) [Django-doc]). But even if you somehow would cover those cases, then another program that talks to the database could update one of the two fields.
Most databases have triggers that can help. But again, the number of cases to cover are often larger than expected. For example, if the Product that we refer to is removed, then or the foreign key now points to a different Product, then we will need to update that field.
Therefore it is often better, to fetch the name of the related product when we need it. We can do so by (a) defining a property; or (b) make an annotation, for example in the manager.
Defining a property
We can define a property that will load the related product, and fetch the related name, like:
class Order(models.Model):
product = models.ForeignKey(Product, on_delete=models.PROTECT)
#property
def product_name(self):
return self.product.name
Then we can fetch the product name with some_order.product_name. This might not be very efficient if we need to fetch it often, since the relations are, by default, loaded lazily in Django, and thus can result in an N+1 problem.
Annotate the queryset
We can make an annotation that will fetch the name of the product in the same query when we fetch the Order, for example:
from django.db.models import F
class OrderManager(models.Manager):
def get_queryset(self):
return super().get_queryset().annotate(
product_name=F('product__name')
)
class Order(models.Model):
product = models.ForeignKey(Product, on_delete=models.PROTECT)
objects = OrderManager()
Then if we fetch an order. For example with Order.objects.get(pk=1), then that Order object will have an attribute product_name with the name of the product.
I'm trying to track Foreign Keys using django-field-history, but when I add it, it does additional queries on every page using the Model
For example
from field_history.tracker import FieldHistoryTracker
class Author(models.Model):
user = models.ForeignKey('auth.user)
field_history = FieldHistoryTracker(['user'])
will always give more queries on pages using Author, like so
SELECT ••• FROM "auth_user" WHERE "auth_user"."id" = '2'
1239 similar queries. Duplicated 1235 times.
I've tried using user_id instead of user in Field History Tracker, but it will always return None. Using user.id or anything like it just returns an error.
I really need to keep that history data, but not at the cost of thousands of additional queries.
Also, would really enjoy keeping django-field-history as my whole DB is using it, but I'm aware I might have to switch package, and if so, which one would you advise ?
As far as my understanding goes, you are trying to log which user has updated, for this you should use _field_history_user as described in the documentation.
For example:
class Pizza(models.Model):
name = models.CharField(max_length=255)
updated_by = models.ForeignKey('auth.User')
field_history = FieldHistoryTracker(['name'])
#property
def _field_history_user(self):
return self.updated_by
It would always update which user has updated the row for this table.
I have two models. One is Task model and other is reward model.
class Task(models.Model):
assigned_by = models.CharField(max_length=100)
class Reward(models.Model):
task = model.ForeignKey(Task)
Now I want to return a queryset of Task along with the reward field in it. I tried this query.
search_res = Task.objects.annotate(reward='reward').
I got this error: The annotation 'reward' conflicts with a field on the model.
Please tell how to solve this. I want an field reward in each task object.
To reach your goal with the actual models I would simply use the relations along with the task.
Let's say you have a task (or a queryset of tasks):
t = Task.objects.get(pk=1)
or
for t in Task.objects.all():
you can get the reward like this:
t.reward_set.first()
Take care of exception in case there's no reward actually linked to the task.
That incurs in quite an amount of queries for large datasets, so you could optimize the requests toward the DB with select_related or prefetch_related depending on your needs. Look at the Django docs for that.
I posted this question on the django-users list, but haven't had a reply there yet.
I have models that look something like this:
class ProductGroup(models.Model):
name = models.CharField(max_length=10, primary_key=True)
def __unicode__(self): return self.name
class ProductRun(models.Model):
date = models.DateField(primary_key=True)
def __unicode__(self): return self.date.isoformat()
class CatalogItem(models.Model):
cid = models.CharField(max_length=25, primary_key=True)
group = models.ForeignKey(ProductGroup)
run = models.ForeignKey(ProductRun)
pnumber = models.IntegerField()
def __unicode__(self): return self.cid
class Meta:
unique_together = ('group', 'run', 'pnumber')
class Transaction(models.Model):
timestamp = models.DateTimeField()
user = models.ForeignKey(User)
item = models.ForeignKey(CatalogItem)
quantity = models.IntegerField()
price = models.FloatField()
Let's say there are about 10 ProductGroups and 10-20 relevant
ProductRuns at any given time. Each group has 20-200 distinct
product numbers (pnumber), so there are at least a few thousand
CatalogItems.
I am working on formsets for the Transaction model. Instead of a
single select menu with the several thousand CatalogItems for the
ForeignKey field, I want to substitute three drop-down menus, for
group, run, and pnumber, which uniquely identify the CatalogItem.
I'd also like to limit the choices in the second two drop-downs to
those runs and pnumbers which are available for the currently
selected product group (I can update them via AJAX if the user
changes the product group, but it's important that the initial page
load as described without relying on AJAX).
What's the best way to do this?
As a point of departure, here's what I've tried/considered so far:
My first approach was to exclude the item foreign key field from the
form, add the substitute dropdowns by overriding the add_fields
method of the formset, and then extract the data and populate the
fields manually on the model instances before saving them. It's
straightforward and pretty simple, but it's not very reusable and I
don't think it is the right way to do this.
My second approach was to create a new field which inherits both
MultiValueField and ModelChoiceField, and a corresponding
MultiWidget subclass. This seems like the right approach. As
Malcolm Tredinnick put it in
a django-users discussion,
"the 'smarts' of a field lie in the Field class."
The problem I'm having is when/where to fetch the lists of choices
from the db. The code I have now does it in the Field's __init__,
but that means I have to know which ProductGroup I'm dealing with
before I can even define the Form class, since I have to instantiate the
Field when I define the form. So I have a factory
function which I call at the last minute from my view--after I know
what CatalogItems I have and which product group they're in--to
create form/formset classes and instantiate them. It works, but I
wonder if there's a better way. After all, the field should be
able to determine the correct choices much later on, once it knows
its current value.
Another problem is that my implementation limits the entire formset
to transactions relating to (CatalogItems from) a single
ProductGroup.
A third possibility I'm entertaining is to put it all in the Widget
class. Once I have the related model instance, or the cid, or
whatever the widget is given, I can get the ProductGroup and
construct the drop-downs. This would solve the issues with my
second approach, but doesn't seem like the right approach.
One way of setting field choices of a form in a formset is in the form's __init__ method by overwriting the self.fields['field_name'].choices, but since a more dynamic approach is desired, here is what works in a view:
from django.forms.models import modelformset_factory
user_choices = [(1, 'something'), (2, 'something_else')] # some basic choices
PurchaserChoiceFormSet = modelformset_factory(PurchaserChoice, form=PurchaserChoiceForm, extra=5, max_num=5)
my_formset = PurchaserChoiceFormSet(self.request.POST or None, queryset=worksheet_choices)
# and now for the magical for loop
for choice_form in my_formset:
choice_form.fields['model'].choices = user_choices
I wasn't able to find the answer for this but tried it out and it works in Django 1.6.5. I figured it out since formsets and for loops seem to go so well together :)
I ended up sticking with the second approach, but I'm convinced now that it was the Short Way That Was Very Long. I had to dig around a bit in the ModelForm and FormField innards, and IMO the complexity outweighs the minimal benefits.
What I wrote in the question about the first approach, "It's straightforward and pretty simple," should have been the tip-off.
I'd like to make a user profile app in Django (I know there are some that exist, thanks) and I'm wondering how you would structure the models to allow for an arbitrary combination fields in each sub-section.
As an example, the section 'education' may have a sub-section called 'Programming Experience', and the section 'personal info' may have a sub-section called 'favourites'.
Thinking in terms of a typical side bar navigation setup each section would be a header, and each sub-section would be a link to a form where the information can be manipulated.
Education
- Schooling
- Programming Experience
Personal Info
- Location
- Favourites
- Look a-likes
What I'd like to do is be able to add items to the sub-sections on an Arbitrary basis. Whatever the feel of the site calls for.
Maybe one site would benefit from photos of the school a user attended, while another might only need a description.
I'd like to use the admin interface to add these field types to the sub-section. So adding an item would present the choice of what type of information it is (image, video, text, etc) and what sub-section it's to be applied to.
I'd like to know how you would accomplish this; and more importantly, by jumping through as few hoops as possible.
Thanks.
Edit:
To hopefully clarify the question I'll provide a sample models.py file. This is just a quick moch-up to demonstrate the problem more accurately. I have two solutions in mind, and I think solution two will work better than solution one; but I'd also like to here what the SO community thinks and if they have any other solutions of their own.
**models.py**
class Section(models.Model):
"""
The root of categorization. Acts much like a header
"""
name = models.CharField(max_length=30)
description = models.CharField(max_length=255)
class SubSection(models.Model):
"""
The contents of each section. Contains many items of varying types as needed
by the site developer.
"""
name = models.CharField(max_length=30)
description = models.CharField(max_length=255)
section = models.ForeignKey(Section)
class Item(models.Model):
"""
I would like this to store the information here and have a foreign key to the
'SubSection' table. The problem is that there are a lot of different information
types that can be stored and I'd need a row for each type. Thus for each
entry most of the columns will be blank.
I'm thinking that it may be better to use this table as a pointer to another
table that actually contains the information. This will result in a lot of
tables but will eliminate waste.
"""
name = models.CharField(max_length=30)
description = models.CharField(max_length=255)
sub_section = models.ForeignKey(SubSection)
### Solution One
# Storing the info here results in a lot of wasted space and may not be all
# that flexible
image = models.ImageField()
text = models.CharField(max_length=255)
numeric = models.IntegerField()
time = models.TimeField()
# etc ...
### Solution Two
# Storing the field info results in more tables but allows for a better match
# of information and field type.
field_type = models.CharField(max_length=255)
field_properties = models.CommaSeparatedIntegerField(max_length=None)
### Solution Two Tables
# Solution two would require a table for each field type supported here, which
# is quite a few different types.
class ImageStorage(models.Model):
item = models.ForeignKey(Item)
information = models.ImageField()
class IntegerStorage(models.Model):
item = models.ForeignKey(Item)
information = models.IntegerField()
### etc ...
Just keep in mind it's targeted at user profiles. So a weight loss site may want the users current weight in the profile (numeric information) while a travel site may want a list of places visited (text information, could even use the IPAddressField I suppose). I just have no idea what will pop up so I'm trying to make it as generic as possible.
If I'm understanding you properly, the simplest way to do this would likely be a many-to-one relationship. I'm not sure if you wanted these on a per-user or per-site basis so I'll assume you want to customize it per-site (switching that is easy).
Create a table that looks something like this:
class Section(models.Model):
section = models.CharField()
sub-section = model.CharField()
site = models.ForeignKey(Site)
For multiple subsections that belong to the same section, simply give them the same primary section name so that you can query the DB for the accompanying subsections using that name.
Another way would be to use two tables to accomplish the same thing, but given the application I think this might be more appropriate.