We have two models (simplified versions):
class Contestant(models.Model):
email = models.EmailField(max_length=255, unique=True)
# plus some other fields
#property
def total_points(self):
return self.points.aggregate(total=Sum('value'))['total'] or 0
class Points(models.Model):
contestant = models.ForeignKey(Contestant, related_name='points')
value = models.PositiveIntegerField()
# plus some other fields which determine based on what we
# awarded ``Points.value``
When we display a list of contestants along with their total_points value, it
results in an extra query for each result - i.e. the following queries are performed:
fetch list of contestants
fetch total_points value of 1st contestant
fetch total_points value of 2nd contestant
etc
I tried altering the queryset to prefetch the data as follows:
Contestant.objects.filter(...).prefetch_related('points')
..., however even though it works, the prefetched data is not utilized when
listing contestants (so each result still tries to fetch total_points
in a separate query).
Is it possible to:
somehow tell the ORM to use prefetched values for the #property field when
populating data for individual model objects (e.g. access the prefetched value inside the Contestant.total_points #property method)?
or to prefetch them in a different way (as opposed to the example above)?
or to use a completely different approach achieving the same result?
(I'm listing results in tastypie, if it matters.)
Thank you.
When your aim is to add aggregated values to each item, you should use annotate, instead of aggregate.
For example (a simple query, no additional methods required):
Contestant.objects.filter(...).annotate(total_points=Sum('points__value'))
If you really want to put this code out of your query: you can, but a model method is not a right way to do this. Methods on models are for operations on single instances. If you want to do something on a whole QuerySet use an ORM Manager instead.
With a Manager this would look like this:
class TotalPointsManager(models.Manager):
def get_queryset(self):
return super(TotalPointsManager, self).get_queryset().annotate(total_points=Sum('points__value'))
class Contestant(models.Model):
email = models.EmailField(max_length=255, unique=True)
objects = TotalPointsManager() # You are overriding the default manager!
and then you would construct your query as usual (you can drop prefetch_related):
Contestant.objects.filter(...)
...and total_points field would become "magically" available for every object.
Related
few years ego I worked with Odoo framework. and Odoo has very nice feature like this:
partner_id = field.Many2one(Partner)
partner_name = fields.Char(string='Partner name', related='partner_id.name')
basically whenever you would assign different partner_id from Partner table, partner_name would be assigned automatically. Now I started to work with django (absolute newbie), and I can't seem to find a similar functionality.
My question is what could be possible solution for this problem. Maybe there are already established external libraries that has this sort of functionality?
Expected result:
product = models.ForeignKey(Product)
product_color = models.CharField(string='Partner name', related='product.color')
having in mind that product object would have color field and it would be assigned to product_color whenever product field value Product object color value changes. Also what about storing it to database? Would be nice if there was an option to chose between storing it in database or getting it on the fly.
Cheers!
Creating a getter is pretty easy, because you can simply have functions in a Python object behave as a property:
class SampleModel(models.Model):
product = models.ForeignKey(Product)
#property
def product_color(self):
return self.product.color
This does retrieve the property on the fly, which will cause a call to the database.
Duplicating data, is usually a (more severe) antipattern. Synchronizing data, even in two tables in the same database, often turns out harder than one might expect. Even if you would use Django's signal framework for example, then some Django ORM calls can circumvent that (for example .update(..) [Django-doc]). But even if you somehow would cover those cases, then another program that talks to the database could update one of the two fields.
Most databases have triggers that can help. But again, the number of cases to cover are often larger than expected. For example, if the Product that we refer to is removed, then or the foreign key now points to a different Product, then we will need to update that field.
Therefore it is often better, to fetch the name of the related product when we need it. We can do so by (a) defining a property; or (b) make an annotation, for example in the manager.
Defining a property
We can define a property that will load the related product, and fetch the related name, like:
class Order(models.Model):
product = models.ForeignKey(Product, on_delete=models.PROTECT)
#property
def product_name(self):
return self.product.name
Then we can fetch the product name with some_order.product_name. This might not be very efficient if we need to fetch it often, since the relations are, by default, loaded lazily in Django, and thus can result in an N+1 problem.
Annotate the queryset
We can make an annotation that will fetch the name of the product in the same query when we fetch the Order, for example:
from django.db.models import F
class OrderManager(models.Manager):
def get_queryset(self):
return super().get_queryset().annotate(
product_name=F('product__name')
)
class Order(models.Model):
product = models.ForeignKey(Product, on_delete=models.PROTECT)
objects = OrderManager()
Then if we fetch an order. For example with Order.objects.get(pk=1), then that Order object will have an attribute product_name with the name of the product.
I am developing an app where students can evaluate their teachers. I have several models, but the important ones for this question are these:
class Professor(models.Model):
name = models.CharField(max_length=50,null=True)
categories = models.ManyToManyField(Category, related_name='professors')
def __str__(self):
return self.name
class Student(models.Model):
name = models.CharField(max_length=50,null=True)
professors = models.ManyToManyField(Professor, related_name='students',through='Studentprofesor' )
def __str__(self):
return self.name
class Studentprofesor(models.Model):
student = models.ForeignKey(Student, on_delete=models.CASCADE)
professor = models.ForeignKey(Professor, on_delete=models.CASCADE)
tested = models.BooleanField(default=False)
As far as I knew the main difference between get and filter was that I couldn't use get when there were several objects with the features I was looking for. But apart from that, they worked in a similar way. get for a single object, filter for several objects.
However, in this case I get different results when I run get and filter.
if I use get:
Student.objects.get(name="Mike").professors.all()
I obtain:
<QuerySet [<Professor: Tom>, <Professor: Jenn>]>
But if I use filter:
Student.objects.filter(name="Mike").professors.all()
I obtain:
AttributeError: 'QuerySet' object has no attribute 'professors'
it's as if filter is not able to follow the manytomany relationship between the objects.
Why is this happening?
There are huge differences between .get(..) and .filter(..). In short: .get(..) obtains a single model instance that satisfies the given conditions whereas .filter(..) filters the queryset and produces a queryset that conceptually contains model instances (!) that satisfy the given conditions.
Django's .get(..)
.get means that you aim to retrieve exactly one instance. So that means that if you write:
Model.objects.get(..)
the result is a Model instance (given there is such instance). As a result we can thus obtain attributes (like .professors, etc.) from that single entity. We have guarantees if the call succeeds that: (1) there are no multiple objects for which the filter criteria hold; and (2) there is at least one element for which the filter criteria hold. So the output is always a model instance, not a None or a QuerySet.
The .get(..) function is evaluated eagerly: we immediately perform a query to the database. In case the database returns no entries, or two or more, the Model.DoesNotExist and MultipleObjectsReturned exceptions are respectively risen.
Note: since .get(..) acts eagerly, adding filters, etc. at the right of .get(..) has no use (well unless you define a filter function on that instance, but that is not a good idea either). You can however use functions like .values(), values_list, prefetch_related, etc. on the left side to change the type of output (and prefetch certain parts). For example:
Student.objects.values().get(name='Mike')
will result in a dictionary containing the values of that instance.
Django's .filter(..)
Filter on the other hand filters a queryset. That means that it is possible that after we filter, the queryset no longer contains any instances (if the filter is too restrictive) or two ore more (if the filter is too weak to pin to a single entry).
Django does not evaluate such .filter(..) eagerly. This means that by default Django will not make a query to the database to retrieve entries. Only if you call for example len(..) over the resulting queryset, or you iterate over it, Django will first perform a database query and then handle the corresponding result.
Since the result of a .filter(..) is another QuerySet, we can chain operations together. For example, we can call an extra .filter(..) or .exclude(..), values_list(..) or any other function supported by the QuerySet.
Since the result is not a model instance, we can not call the attributes of a model instance. What should be the outcome of a Student.objects.filter(..).name in case no student matches the criteria? Or what if there are multiple Students that match the given constraint?
We can however obtain the list of Professors that teach one or more Students with the name 'Mike' with:
# professors with a student called Mike
Professors.objects.filter(students__name="Mike")
A filter will never raise an Model.DoesNotExist or a MultipleObjectsReturned exception, since it is perfectly allowed to work with empty QuerySets, or QuerySets with multiple items.
Bacause filter() returns queryset (multiple students). But professors is attribute of single student instance. You can use first() with filter() to get single object:
Student.objects.filter(name="Mike").first().professors.all()
As a project to figure out Django I'm trying to build a small game.
A player has a base. A base has several type of items it can harbor. (Vehicle, Defense, Building).
I have 3 static tables which contain information for the first level of each item (in the game these values are used in formulas to calculate stuff for upgrades). I've used a sequence to insert all these items in these different tables so the ID's are unique across tables.
To keep track of what items the player has per base I have a table 'Property'. I want to use a single field as a reference to the ID of an item and trying to get this done with the Django models.
Warning: my knowledge about Django models are pretty limited and I've been stuck with this a few days now.
Is this possible and if so how can it be done?
I tried using annotations on the save method to change the value of a field by overwriting the field with the id of that object before trying to query the object by id when trying to 'get' the object, however I can't get past the obvious restriction of the model when defining that field as an Integer - I hoped it wouldn't validate until I called save()
def getPropertyItemID(func):
"""
This method sets the referral ID to an item to the actual ID.
"""
def decoratedFunction(*args):
# Grab a reference to the data object we want to update.
data_object=args[0]
# Set the ID if item is not empty.
if data_object.item is not None:
data_object.item=data_object.item.id
# Execute the function we're decorating
return func(*args)
return decoratedFunction
class Property(models.Model):
"""
This class represents items that a user has per base.
"""
user=models.ForeignKey(User)
base=models.ForeignKey(Base)
item=models.IntegerField()
amount=models.IntegerField(default=0)
level=models.SmallIntegerField(default=0)
class Meta:
db_table='property'
#getPropertyItemID
def save(self):
# Now actually save the object
super(Property, self).save()
I hope you can help me here. The end result I'd like to be able to put to use would be something like:
# Adding - automatically saving the ID of item regardless of the class
# of item
item = Property(user=user, base=base, item=building)
item.save()
# Retrieving - automatically create an instance of an object based on the ID
# of item, regardless of the table this ID is found in.
building = Property.objects.all().distinct(True).get(base=base, item=Building.objects.all().distinct(True).get(name='Tower'))
# At this point building should be an instance of the Building model
If I'm completely off and I can achieve this differently I'm all ears :)
I think you are looking for a Generic Relationship:
class Property(models.Model):
user=models.ForeignKey(User)
base=models.ForeignKey(Base)
content_type = models.ForeignKey(ContentType) # Which model is `item` representing?
object_id = models.PositiveIntegerField() # What is its primary key?
item=generic.GenericForeignKey('content_type', 'object_id') # Easy way to access it.
amount=models.IntegerField(default=0)
level=models.SmallIntegerField(default=0)
This lets you create items as you mentioned, however you would probably need to look at a different way of filtering those items out.
I posted this question on the django-users list, but haven't had a reply there yet.
I have models that look something like this:
class ProductGroup(models.Model):
name = models.CharField(max_length=10, primary_key=True)
def __unicode__(self): return self.name
class ProductRun(models.Model):
date = models.DateField(primary_key=True)
def __unicode__(self): return self.date.isoformat()
class CatalogItem(models.Model):
cid = models.CharField(max_length=25, primary_key=True)
group = models.ForeignKey(ProductGroup)
run = models.ForeignKey(ProductRun)
pnumber = models.IntegerField()
def __unicode__(self): return self.cid
class Meta:
unique_together = ('group', 'run', 'pnumber')
class Transaction(models.Model):
timestamp = models.DateTimeField()
user = models.ForeignKey(User)
item = models.ForeignKey(CatalogItem)
quantity = models.IntegerField()
price = models.FloatField()
Let's say there are about 10 ProductGroups and 10-20 relevant
ProductRuns at any given time. Each group has 20-200 distinct
product numbers (pnumber), so there are at least a few thousand
CatalogItems.
I am working on formsets for the Transaction model. Instead of a
single select menu with the several thousand CatalogItems for the
ForeignKey field, I want to substitute three drop-down menus, for
group, run, and pnumber, which uniquely identify the CatalogItem.
I'd also like to limit the choices in the second two drop-downs to
those runs and pnumbers which are available for the currently
selected product group (I can update them via AJAX if the user
changes the product group, but it's important that the initial page
load as described without relying on AJAX).
What's the best way to do this?
As a point of departure, here's what I've tried/considered so far:
My first approach was to exclude the item foreign key field from the
form, add the substitute dropdowns by overriding the add_fields
method of the formset, and then extract the data and populate the
fields manually on the model instances before saving them. It's
straightforward and pretty simple, but it's not very reusable and I
don't think it is the right way to do this.
My second approach was to create a new field which inherits both
MultiValueField and ModelChoiceField, and a corresponding
MultiWidget subclass. This seems like the right approach. As
Malcolm Tredinnick put it in
a django-users discussion,
"the 'smarts' of a field lie in the Field class."
The problem I'm having is when/where to fetch the lists of choices
from the db. The code I have now does it in the Field's __init__,
but that means I have to know which ProductGroup I'm dealing with
before I can even define the Form class, since I have to instantiate the
Field when I define the form. So I have a factory
function which I call at the last minute from my view--after I know
what CatalogItems I have and which product group they're in--to
create form/formset classes and instantiate them. It works, but I
wonder if there's a better way. After all, the field should be
able to determine the correct choices much later on, once it knows
its current value.
Another problem is that my implementation limits the entire formset
to transactions relating to (CatalogItems from) a single
ProductGroup.
A third possibility I'm entertaining is to put it all in the Widget
class. Once I have the related model instance, or the cid, or
whatever the widget is given, I can get the ProductGroup and
construct the drop-downs. This would solve the issues with my
second approach, but doesn't seem like the right approach.
One way of setting field choices of a form in a formset is in the form's __init__ method by overwriting the self.fields['field_name'].choices, but since a more dynamic approach is desired, here is what works in a view:
from django.forms.models import modelformset_factory
user_choices = [(1, 'something'), (2, 'something_else')] # some basic choices
PurchaserChoiceFormSet = modelformset_factory(PurchaserChoice, form=PurchaserChoiceForm, extra=5, max_num=5)
my_formset = PurchaserChoiceFormSet(self.request.POST or None, queryset=worksheet_choices)
# and now for the magical for loop
for choice_form in my_formset:
choice_form.fields['model'].choices = user_choices
I wasn't able to find the answer for this but tried it out and it works in Django 1.6.5. I figured it out since formsets and for loops seem to go so well together :)
I ended up sticking with the second approach, but I'm convinced now that it was the Short Way That Was Very Long. I had to dig around a bit in the ModelForm and FormField innards, and IMO the complexity outweighs the minimal benefits.
What I wrote in the question about the first approach, "It's straightforward and pretty simple," should have been the tip-off.
This is a problem concerning django.
I have a model say "Automobiles". This will have some basic fields like "Color","Vehicle Owner Name", "Vehicle Cost".
I want to provide a form where the user can add extra fields depending on the automobile that he is adding. For example, if the user is adding a "Car", he will extra fields in the form, dynamically at run time, like "Car Milage", "Cal Manufacturer".
Suppose if the user wants to add a "Truck", he will add "Load that can be carried", "Permit" etc.
How do I achieve this in django?
There are two questions here:
How to provide a form where the user can add new fields at run time?
How to add the fields to the database so that it can be retrieved/queried later?
There are a few approaches:
key/value model (easy, well supported)
JSON data in a TextField (easy, flexible, can't search/index easily)
Dynamic model definition (not so easy, many hidden problems)
It sounds like you want the last one, but I'm not sure it's the best for you. Django is very easy to change/update, if system admins want extra fields, just add them for them and use south to migrate. I don't like generic key/value database schemas, the whole point of a powerful framework like Django is that you can easily write and rewrite custom schemas without resorting to generic approaches.
If you must allow site users/administrators to directly define their data, I'm sure others will show you how to do the first two approaches above. The third approach is what you were asking for, and a bit more crazy, I'll show you how to do. I don't recommend using it in almost all cases, but sometimes it's appropriate.
Dynamic models
Once you know what to do, this is relatively straightforward. You'll need:
1 or 2 models to store the names and types of the fields
(optional) An abstract model to define common functionality for your (subclassed) dynamic models
A function to build (or rebuild) the dynamic model when needed
Code to build or update the database tables when fields are added/removed/renamed
1. Storing the model definition
This is up to you. I imagine you'll have a model CustomCarModel and CustomField to let the user/admin define and store the names and types of the fields you want. You don't have to mirror Django fields directly, you can make your own types that the user may understand better.
Use a forms.ModelForm with inline formsets to let the user build their custom class.
2. Abstract model
Again, this is straightforward, just create a base model with the common fields/methods for all your dynamic models. Make this model abstract.
3. Build a dynamic model
Define a function that takes the required information (maybe an instance of your class from #1) and produces a model class. This is a basic example:
from django.db.models.loading import cache
from django.db import models
def get_custom_car_model(car_model_definition):
""" Create a custom (dynamic) model class based on the given definition.
"""
# What's the name of your app?
_app_label = 'myapp'
# you need to come up with a unique table name
_db_table = 'dynamic_car_%d' % car_model_definition.pk
# you need to come up with a unique model name (used in model caching)
_model_name = "DynamicCar%d" % car_model_definition.pk
# Remove any exist model definition from Django's cache
try:
del cache.app_models[_app_label][_model_name.lower()]
except KeyError:
pass
# We'll build the class attributes here
attrs = {}
# Store a link to the definition for convenience
attrs['car_model_definition'] = car_model_definition
# Create the relevant meta information
class Meta:
app_label = _app_label
db_table = _db_table
managed = False
verbose_name = 'Dynamic Car %s' % car_model_definition
verbose_name_plural = 'Dynamic Cars for %s' % car_model_definition
ordering = ('my_field',)
attrs['__module__'] = 'path.to.your.apps.module'
attrs['Meta'] = Meta
# All of that was just getting the class ready, here is the magic
# Build your model by adding django database Field subclasses to the attrs dict
# What this looks like depends on how you store the users's definitions
# For now, I'll just make them all CharFields
for field in car_model_definition.fields.all():
attrs[field.name] = models.CharField(max_length=50, db_index=True)
# Create the new model class
model_class = type(_model_name, (CustomCarModelBase,), attrs)
return model_class
4. Code to update the database tables
The code above will generate a dynamic model for you, but won't create the database tables. I recommend using South for table manipulation. Here are a couple of functions, which you can connect to pre/post-save signals:
import logging
from south.db import db
from django.db import connection
def create_db_table(model_class):
""" Takes a Django model class and create a database table, if necessary.
"""
table_name = model_class._meta.db_table
if (connection.introspection.table_name_converter(table_name)
not in connection.introspection.table_names()):
fields = [(f.name, f) for f in model_class._meta.fields]
db.create_table(table_name, fields)
logging.debug("Creating table '%s'" % table_name)
def add_necessary_db_columns(model_class):
""" Creates new table or relevant columns as necessary based on the model_class.
No columns or data are renamed or removed.
XXX: May need tweaking if db_column != field.name
"""
# Create table if missing
create_db_table(model_class)
# Add field columns if missing
table_name = model_class._meta.db_table
fields = [(f.column, f) for f in model_class._meta.fields]
db_column_names = [row[0] for row in connection.introspection.get_table_description(connection.cursor(), table_name)]
for column_name, field in fields:
if column_name not in db_column_names:
logging.debug("Adding field '%s' to table '%s'" % (column_name, table_name))
db.add_column(table_name, column_name, field)
And there you have it! You can call get_custom_car_model() to deliver a django model, which you can use to do normal django queries:
CarModel = get_custom_car_model(my_definition)
CarModel.objects.all()
Problems
Your models are hidden from Django until the code creating them is run. You can however run get_custom_car_model for every instance of your definitions in the class_prepared signal for your definition model.
ForeignKeys/ManyToManyFields may not work (I haven't tried)
You will want to use Django's model cache so you don't have to run queries and create the model every time you want to use this. I've left this out above for simplicity
You can get your dynamic models into the admin, but you'll need to dynamically create the admin class as well, and register/reregister/unregister appropriately using signals.
Overview
If you're fine with the added complication and problems, enjoy! One it's running, it works exactly as expected thanks to Django and Python's flexibility. You can feed your model into Django's ModelForm to let the user edit their instances, and perform queries using the database's fields directly. If there is anything you don't understand in the above, you're probably best off not taking this approach (I've intentionally not explained what some of the concepts are for beginners). Keep it Simple!
I really don't think many people need this, but I have used it myself, where we had lots of data in the tables and really, really needed to let the users customise the columns, which changed rarely.
Database
Consider your database design once more.
You should think in terms of how those objects that you want to represent relate to each other in the real world and then try to generalize those relations as much as you can, (so instead of saying each truck has a permit, you say each vehicle has an attribute which can be either a permit, load amount or whatever).
So lets try it:
If you say you have a vehicle and each vehicle can have many user specified attributes consider the following models:
class Attribute(models.Model):
type = models.CharField()
value = models.CharField()
class Vehicle(models.Model):
attribute = models.ManyToMany(Attribute)
As noted before, this is a general idea which enables you to add as much attributes to each vehicle as you want.
If you want specific set of attributes to be available to the user you can use choices in the Attribute.type field.
ATTRIBUTE_CHOICES = (
(1, 'Permit'),
(2, 'Manufacturer'),
)
class Attribute(models.Model):
type = models.CharField(max_length=1, choices=ATTRIBUTE_CHOICES)
value = models.CharField()
Now, perhaps you would want each vehicle sort to have it's own set of available attributes. This can be done by adding yet another model and set foreign key relations from both Vehicle and Attribute models to it.
class VehicleType(models.Model):
name = models.CharField()
class Attribute(models.Model):
vehicle_type = models.ForeigngKey(VehicleType)
type = models.CharField()
value = models.CharField()
class Vehicle(models.Model):
vehicle_type = models.ForeigngKey(VehicleType)
attribute = models.ManyToMany(Attribute)
This way you have a clear picture of how each attribute relates to some vehicle.
Forms
Basically, with this database design, you would require two forms for adding objects into the database. Specifically a model form for a vehicle and a model formset for attributes. You could use jQuery to dynamically add more items on the Attribute formset.
Note
You could also separate Attribute class to AttributeType and AttributeValue so you don't have redundant attribute types stored in your database or if you want to limit the attribute choices for the user but keep the ability to add more types with Django admin site.
To be totally cool, you could use autocomplete on your form to suggest existing attribute types to the user.
Hint: learn more about database normalization.
Other solutions
As suggested in the previous answer by Stuart Marsh
On the other hand you could hard code your models for each vehicle type so that each vehicle type is represented by the subclass of the base vehicle and each subclass can have its own specific attributes but that solutions is not very flexible (if you require flexibility).
You could also keep JSON representation of additional object attributes in one database field but I am not sure this would be helpfull when querying attributes.
Here is my simple test in django shell- I just typed in and it seems work fine-
In [25]: attributes = {
"__module__": "lekhoni.models",
"name": models.CharField(max_length=100),
"address": models.CharField(max_length=100),
}
In [26]: Person = type('Person', (models.Model,), attributes)
In [27]: Person
Out[27]: class 'lekhoni.models.Person'
In [28]: p1= Person()
In [29]: p1.name= 'manir'
In [30]: p1.save()
In [31]: Person.objects.a
Person.objects.aggregate Person.objects.all Person.objects.annotate
In [32]: Person.objects.all()
Out[33]: [Person: Person object]
It seems very simple- not sure why it should not be a considered an option- Reflection is very common is other languages like C# or Java- Anyway I am very new to django things-
Are you talking about in a front end interface, or in the Django admin?
You can't create real fields on the fly like that without a lot of work under the hood. Each model and field in Django has an associated table and column in the database. To add new fields usually requires either raw sql, or migrations using South.
From a front end interface, you could create pseudo fields, and store them in a json format in a single model field.
For example, create an other_data text field in the model. Then allow users to create fields, and store them like {'userfield':'userdata','mileage':54}
But I think if you're using a finite class like vehicles, you would create a base model with the basic vehicle characteristics, and then create models that inherits from the base model for each of the vehicle types.
class base_vehicle(models.Model):
color = models.CharField()
owner_name = models.CharField()
cost = models.DecimalField()
class car(base_vehicle):
mileage = models.IntegerField(default=0)
etc