I would like to add some metadata to a certain object in my Django models. Normally this would be a good situation for a OneToOneField or even Multi-table inheritance, but in my case there's an additional special requirement:
Due to the way data is synchronized in the overall system, the database storage for the actual "target" objects occasionally get re-created. That is, from time to time all the object rows get deleted and new ones are created in their place. But I need the metadata to persist.
Besides its implicit primary key, each object also has a textual slug field that uniquely identifies it — but only within a single container. This slug is stable across the data being re-created, as is the container itself. So I'd like to use a combination of an object's slug and its container to associate my metadata, rather than the database primary key for the object.
I found the to_field option on the ForeignKey model field, but if I set for example to_field='id_in_container' it complains since the id_in_container field is not unique on its own, only together with the container field.
Is there a way to do something like this?
class Container(models.Model):
name = models.CharField(max_length=255)
class Object(models.Model):
container = models.ForeignKey(Container)
id_in_container = models.SlugField()
name = models.CharField(max_length=255)
class Meta:
unique_together = ('container', 'id_in_container')
# BUG: this won't actually work, `to_field` can't take a tuple!
DESIRED_COMBO = ('container','id_in_container')
class ObjectMetadata(models.model):
object = models. OneToOneField(Object, to_field=DESIRED_COMBO)
# … additional fields …
That is, have a OneToOneField or ForeignKey that targets multiple columns of a target model, when those columns are unique_together?
I suppose I can implement some manual helpers, perhaps like this:
class ObjectMetadata(models.model):
object_container = models.ForeignKey(Container)
object_slug = models.SlugField()
# … additional fields …
#classmethod
def get_for_object(cls, obj):
return cls.objects.get(object_container=obj.container, object_slug=obj.id_in_container)
…but I'm wondering if there's a more appropriate way to model this.
Related
Working with Django REST Framework I am wondering if it's possible to limit the choices / options of a ManyToMany field on a model to a specific QuerySet?
Using the models below (scroll down to see models), I am curious if it's possible to define this limit in the definition of the model, to achieve the following:
# Having the following Employee instance
emp = Employee(...)
# Should return only the instances with value 'case' in EmployeeSubstitute.type field
emp.substitute_case.all()
# Should return only the instances with value 'phone' in EmployeeSubstitute.type field
emp.substitute_phone.all()
Models:
class Employee(models.Model):
substitute_case = models.ManyToMany(through=EmployeeSubstitute, ...)
substitute_phone = models.ManyToMany(through=EmployeeSubstitute, ...)
class EmployeeSubstitute(models.Model):
from = models.ForeignKey(Employee, ...)
to = models.ForeignKey(Employee, ...)
type = models.CharField(choices=..., ...) # choose between type 'case' and 'phone'
I see that there's the limit_choices_to parameter, but that's not what I am looking for, since that only effects the options shown when using a ModelForm or the admin.
Well, ManyToManyField returns related objects and as docs state
By default, Django uses an instance of the Model._base_manager manager
class when accessing related objects (i.e. choice.question), not the
_default_manager on the related object. This is because Django needs to be able to retrieve the related object, even if it would otherwise
be filtered out (and hence be inaccessible) by the default manager.
If the normal base manager class (django.db.models.Manager) isn’t
appropriate for your circumstances, you can tell Django which class to
use by setting Meta.base_manager_name.
Base managers aren’t used when querying on related models, or when
accessing a one-to-many or many-to-many relationship. For example, if
the Question model from the tutorial had a deleted field and a base
manager that filters out instances with deleted=True, a queryset like
Choice.objects.filter(question__name__startswith='What') would include
choices related to deleted questions.
So if I read it correctly, no, it's not possible.
When you do queries and have through in your ManyToManyField, Django complains you should run these queries on your through model, rather than the "parent". I can't find it in the docs but I remember seeing it a few times.
substitute_case and substitute_phone is something that belongs to substitute and it is it's type. So just do that instead of creating those columns in Employee.
from django.db import models
class SubstituteTypes(models.TextChoices):
case = "case", "case"
phone = "phone", "phone"
class EmployeeSubstituteQueryset(models.QuerySet):
def from_employee(self, e):
return self.filter(_from=e)
def case(self):
return self.filter(type=SubstituteTypes.case)
def phone(self):
return self.filter(type=SubstituteTypes.phone)
class Employee(models.Model):
substitute = models.ManyToManyField(through='EmployeeSubstitute', to='self')
class EmployeeSubstitute(models.Model):
_from = models.ForeignKey(Employee, on_delete=models.CASCADE, related_name='a')
to = models.ForeignKey(Employee, on_delete=models.PROTECT, related_name='b')
type = models.CharField(choices=SubstituteTypes.choices, max_length=5, db_index=True)
objects = EmployeeSubstituteQueryset.as_manager()
Then, once you get your emp object (or only its id), you can do
EmployeeSubstitute.objects.from_employee(emp).case().all()
which is designed in Django philosophy.
I have a question whether or not it is possible to use the generic UpdateView class to edit "both sides" of a many-to-many relationship.
I have the following classes defined in models.py:
class SomeCategory(models.Model):
code = models.CharField(max_length=5)
name = models.CharField(max_length=40)
class SomeClass(models.Model):
code = models.CharField(max_length=3, unique=True)
name = models.CharField(max_length=30, unique=False)
age = models.IntegerField(null=False)
allowed_categories = models.ManyToManyField(SomeCategory)
These are both dictionary type tables that store sets of configuration data for my application. To allow editing the dictionaries I use simple UpdateViews:
class SomeClassUpdate(UpdateView):
model = SomeClass
template_name = 'admin/edit_class.html'
fields = ['code', 'name', 'age', 'allowed_categories']
ordering = ['code']
This works fine, I get a nice multi-select and everything is perfect. However, I would like to have the possibility to edit the relationship from the side of the SomeCategory table, so I can choose which SomeClass elements are linked to a certain SomeCategory:
class SomeCategoryUpdate(UpdateView):
model = SomeCategory
template_name = 'admin/edit_category.html'
fields = ['code', 'name', ??????? ]
ordering = ['code']
I have tried adding the related_name attribute to the SomeCategory model, but that did not work.
Any ideas if this can be done without using a custom ModelForm?
Key library versions:
Django==1.11.8
psycopg2==2.7.4
PS: this is my very first question asked on stackoverflow, so please let me know if my post is missing any mandatory elements.
Your issue is in the models.py file. You have two classes, but only one of them mentions the other one. You would think that this should be enough since you are using ManyToManyField after all and assume that it would automatically create every connection leading both ways... Unfortunately this is not true. On the database level it does indeed create a separate intermediary table with references to objects in both original tables, but that doesn't mean that both of them will be automatically visible in Django Admin or similar.
If you would attempt to simply create another someclass = models.ManyToManyField(SomeClass) in the SomeCategory class that would fail. Django would try to create another separate intermediary table through which the connection between two main tables is established. But because the name of the intermediary table depends on where you define the ManyToManyField connection, the second table would be created with a different name and everything would just logically collapse (two tables having two separate default ways to have a ManyToMany connection makes no sense).
The solution is to add a ManyToManyField connection to SomeCategory while also referencing that intermediary/through table that was originally created in the SomeClass class.
A couple of notes about Django/python/naming/programming conventions:
Use the name of the table you are referencing to, as the name of the field that is containing the info about that connection. Meaning that SomeClass's field with a link to SomeCategory should be named somecategory instead of allowed_categories.
If the connection is one-to-many - use singular form; if the connection is many-to-many - use plural. Meaning that in this case we should use plural and use somecategories instead of somecategory.
Django can automatically pluralize names, but it does it badly - it simply adds s letter to the end. Mouse -> Mouses, Category -> Categorys. In those kind of cases you have to help it by defining the verbose_name_plural in the special Meta class.
Using references to other classes without extra 's works only if the the class was already defined previously in the code. In the case of two classes referring to each other that is true only one way. The solution is to put the name of the referred class in the quotation marks like 'SomeCategory' instead of SomeCategory. This sort of reference, called a lazy relationship, can be useful when resolving circular import dependencies between two applications. And since by default it's better to keep the style the same and to avoid unnecessary brain energy wasting of "I will decide whether or not to use quotation marks depending on the order the classes have been organized; I will have to redo this quotation marks thingie every time I decide to move some code pieces around" I recommend that you simply use quotation marks every time. Just like when learning to drive a car - it's better to learn to always use turn signals instead of first looking around and making a separate decision of whether someone would benefit from that information.
"Stringifying" (lazy loading) model/class/table name is easy - just add 's around. You would think that stringifying the "through" table reference would work the same easy way. And you would be wrong - it will give you the ValueError: Invalid model reference. String model references must be of the form 'app_label.ModelName'. error. In order to reference the stringified "through" table you need to: (a) add 's around; (b) replace all dots (.) with underscores (_); (c) delete the reference to through!.. So SomeClass.somecategories.through becomes 'SomeClass_somecategories'.
Therefore the solution is this:
class SomeCategory(models.Model):
code = models.CharField(max_length=5)
name = models.CharField(max_length=40)
someclasses = models.ManyToManyField('SomeClass', through='SomeClass_somecategories', blank=True)
class Meta:
verbose_name_plural = 'SomeCategories'
class SomeClass(models.Model):
code = models.CharField(max_length=3, unique=True)
name = models.CharField(max_length=30, unique=False)
age = models.IntegerField(null=False)
somecategories = models.ManyToManyField('SomeCategory')
After this it should be obvious what kind of final changes to make to your UpdateView classes.
You can achieve this in the view and form, without having to specify the additional ManytoMany connections in the
models, using something like the following:
In the View
class SomeClassUpdate(UpdateView):
model = SomeClass
form_class = SomeClassUpdateForm # to specify the form
template_name = 'admin/edit_class.html'
def form_valid(self, form, *args, **kwargs):
initial_somecategorys = SomeCategory.objects.filter(allowed_categories__pk=form.instance.pk)
amended_somecategorys = form.cleaned_data['allowed_categroies']
remove = [x for x in initial_somecategorys if x not in amended_somecategorys]
add = [x for x in amended_somecategorys if x not in initial_somecategorys]
for somecategory in add:
somecategory.allowed_categories.add(form.instance)
somecategory.save()
for somecategory in remove:
somecategory.allowed_categories.remove(form.instance)
somecategory.save()
return super().form_valid(form)
In the Form
The init method at the top pre-populates the form with entries saved on the model.
class SomeClassUpdateForm(forms.ModelForm):
def __init__(self, *args, **kwargs):
super(SomeClassUpdateForm, self).__init__(*args, **kwargs)
try:
obj = kwargs['instance']
self.fields["some_categories"].initial = SomeCategory.objects.filter(allowed_categories__pk=form.instance.pk)
except (AttributeError, KeyError): # to catch NoneType if new entry being created.
pass
some_categories = forms.ModelMultipleChoiceField(
required=False,
queryset=SomeCategory.objects.all(),
)
class Meta:
model = SomeClass
fields = [
'some_categories'
..etc
]
This should work. I've writen similar code in one of my projects, and it's working fine. However, I don't know if it's
structurally best to use methods like this and not alter the model relationships or whether it's preferable to
alter the model relationships as outlined in other replies. So I'd be interested to know other peoples views on what
the best approach is.
I have a Django model that is doing way too much. Here's an abbreviated example of the model. Basically, it can represent four different Entity types, and there are recursive ForeignKey and ManyToMany relationships that point to other entities.
This project is currently using Django 1.8.x and Python 2.7.x, but I can upgrade those if the solution requires it.
class Entity(models.Model):
"""
Films, People, Companies, Terms & Techniques
"""
class Meta:
ordering = ['name']
verbose_name_plural = 'entities'
# Types:
FILM = 'FILM'
PERSON = 'PERS'
COMPANY = 'COMP'
TERM = 'TERM'
TYPE_CHOICES = (
(FILM, 'Film'),
(PERSON, 'Person'),
(COMPANY, 'Company'),
(TERM, 'Term/Technique'),
)
created = models.DateTimeField(auto_now_add=True, auto_now=False)
updated = models.DateTimeField(auto_now_add=False, auto_now=True)
type = models.CharField(max_length=4, choices=TYPE_CHOICES, default=FILM)
slug = models.SlugField(blank=True, unique=True, help_text="Automatically generated")
name = models.CharField(max_length=256, blank=True)
redirect = models.ForeignKey('Entity', related_name='entity_redirect', blank=True, null=True, help_text="If this is an alias (see), set Redirect to the primary entry.")
cross_references = models.ManyToManyField('Entity', related_name='entity_cross_reference', blank=True, help_text="This is a 'see also' — 'see' should be performed with a redirect.")
[... and more fields, some of them type-specific]
I realize this is rather messy, and I'd like to remove 'type' and make an EntityBase class that abstracts out all of the common fields, and create new Film, Person, Company, and Term models that inherit from the EntityBase abstract base class.
Once I create the new models, I think I understand how to write the data migration to move all of the field data over to the new models (iterate over objects from Entity, filtered via type, create new objects in the appropriate new model)... except the ForeignKey and ManyToMany relationships. Maybe I'm thinking about this the wrong way, but how can I transfer those relationships when, during the migration, the new object that the relationship points to may not exist yet?
I suspect this may mean a multi-step migration, but I haven't quite worked out the right way to do it.
There is nothing magical about m2m and fk fields. This is the procedure that I would follow... It might be a bit blunt, but will get the job done:
Make a BACKKKUPPPPPPppp of the database!!
Make another backup!
Create the new model and migration
Write a new data migration that will manually iterate over existing models and update the new model, one-by-one. Don't be afraid of the for loop here, unless you have millions of entries in db.
Delete redundant models and/or fields, make migration for this.
Run those migrations :)
In practice, this means a lot of restoring from the "BACKKKUPPPPPPppp" until the migrations are just right.
One little thing to take care of:
M2m fields cannot get any value if model is not yet saved (because model gets its ID on first save). I would do something like, in the manual migration:
new_instance = NewModel()
new_instance.somefield = "whatever"
new_instance.meaning = 42
....
new_instance.save()
new_instance.that_m2m_field.add(some_related_obj)
Of course, make sure you read the docs in detail, especially that bit about importing the model class - you can't just import it from myapp.models import MyModel, instead do:
MyModel = apps.get_model("myapp", "MyModel")
One possible tripping stone might be the model inheritance that you plan to introduce. Generally, you will want to operate on the child model, and access the parent from there as / if needed. Parent can be accessed via the implicit ptr attribute - in your example it would be entitybase_ptr or something similar (that is just a OneToOne field). Going in the other direction, however, (from parent to unknown child) is not as straightforward, because parent doesn't a priori know what is the class of its child.
Here's a version of my models.py file. I've removed irrelevant fields and the model names are made up, for security reasons:
class FilmStudio(models.Model):
name = models.CharField(max_length=200, unique=True)
class ProductionCompany(models.Model):
name = models.CharField(max_length=200)
film_studio = models.ForeignKey(FilmStudio)
class Meta:
# Ensure that a given combination of ProductionCompany name and FilmStudio object is unique
unique_together = ('name', 'film_studio')
class Film(models.Model):
title = models.CharField(max_length=200)
production_company = models.ForeignKey(ProductionCompany)
class Actor(models.Model):
name = models.CharField(max_length=200)
films = models.ManyToManyField(Film, blank=True)
Although it is not explicitly defined, there is a many-to-many relationship between an Actor and a FilmStudio. This is evidenced by the following call to the Python API:
FilmStudio.objects.filter(productioncompany__film__actor__name='Samuel L. Jackson').distinct()
This returns all of the FilmStudio objects which Samuel L. Jackson is related to, and each one only once. What I'd like is to define extra fields on the relationship between an Actor and a FilmStudio (it doesn't work too well in this example, I know, but it makes sense for my scenario).
Following what is described in Extra fields on many-to-many relationships, I could use an intermediate model to define extra fields on the relationship between a Film and an Actor, for instance.
But this doesn't seem to help me with my problem. I don't want to define the Actor to FilmStudio relationship explicitly, since it's an existing relationship based on other relationships.
Is it possible to define fields on the relationship that I'm describing?
As far as I know, you are not able to do that.
The reason for that is that it is nowhere to store the extra fields of that relationship. If I understand you correctly, these "extra fields" are not implicit in the actor-film or productionstudio-film relationships, so even though you say they are implicit, the extra fields themselves are explicit.
You could try to emulate it by creating an explicit direct relationship whenever it is needed. Then you could simulate the extra fields using the model as an abstraction, but I am not sure if this is what you want. If you opt for this kind of solution you can use default values (in your abstraction) for filling in relationships that don't have a instance yet.
Does that explanation make sense to you?
EDIT:
(I have not double checked that the code works, so be vary)
OK, so you have the original models:
class FilmStudio(models.Model):
name = models.CharField(max_length=200, unique=True)
class ProductionCompany(models.Model):
name = models.CharField(max_length=200)
film_studio = models.ForeignKey(FilmStudio)
class Meta:
# Ensure that a given combination of ProductionCompany name and FilmStudio object is unique
unique_together = ('name', 'film_studio')
class Film(models.Model):
title = models.CharField(max_length=200)
production_company = models.ForeignKey(ProductionCompany)
class Actor(models.Model):
name = models.CharField(max_length=200)
films = models.ManyToManyField(Film, blank=True)
# The "solution" would be:
class ActorProductionComapny(models.Model):
production_company = models.ForeignKey(ProductionCompany, related_name='actors')
actor = models.ForeignKey(Actor, related_name='companies')
# your extra fields here
someproperty = models.CharField(max_length=200)
class Meta:
# let's say one per actor
unique_together = ('production_company', 'actor')
This is going to get messy really quickly
We use a F object like this:
FilmStudio.objects.filter(productioncompany__film__actor__name='Samuel L. Jackson',
productioncompany__film__actor=F('actors__actor'),
actors__someproperty="Plays poker with CEO").distinct()
The tricky part is going to be handling default values (i.e. when there is no value) This would have to be implemented using a custom Manager, but then I am out of my depth.
I will try to explain as well as I can, but it's going to be tricky.
If you want to make a filter on the relationship you may have to do something like this:
def filter_prod(pq_query, someproperty, actor_name):
if someproperty == "Default":
# Great, this means we can ignore the parallel relationship:
return pq_query.filter(productioncompany__film__actor__name=actor_name)
else:
# Here comes the hard part
FilmStudio.objects.filter(productioncompany__film__actor__name=actor_name,
productioncompany__film__actor=F('actors__actor'),
actors__someproperty=someproperty).distinct()
The thing I am trying to illustrate here is that there are two kinds of actor-productioncompany relationships, those with custom field values (non-default), and those without.
Now, you can make a custom getter that looks something like this:
class ProductionCompany(models.Model):
name = models.CharField(max_length=200)
film_studio = models.ForeignKey(FilmStudio)
def get_actors(self):
# This one is not lazy, so be aware
actors = list(self.actors)
# Get a list of actor IDs
actor_ids = [a.actor_id for a in actors]
for actor in Actor.objects.filter(films__production_company_id=self.id):
if actor.id not in actor_ids:
actors.append(ActorProductionComapny(actor=actor, production_company=self)
actor_ids.append(actor.id)
return actors
class Meta:
# Ensure that a given combination of ProductionCompany name and FilmStudio object is unique
unique_together = ('name', 'film_studio')
This should not save the relationship to the database until you call .save() on an instance. You can also add a custom save method that ignores/aports .save() calls where all the values are default. Just remember to check if it is a new instance or not, because you don't want it to cancel a "set back to default" call. You could also make it delete on a "set back to default", but check if you are allowed to do that within .save().
For even more complex queries (mix of default and non-default) you have Q-objects (further down on the page from F objects)
In short, you need to create an extra model to store this extra relational data between Actor and FilmStudio.
class Actor(models.Model):
name = models.CharField(max_length=200)
films = models.ManyToManyField(Film, blank=True)
film_studios = models.ManyToMany(FilmStudio, through='ActorFilmStudio')
class ActorFilmStudio(models.Model):
actor = models.ForeignKey(Actor)
film_studio = models.ForeignKey(FilmStudio)
# define extra data fields here
data1 = models.TextField()
data2 = models.IntegerField()
One way to think about this: the data you're trying to store belongs to an Actor-FilmStudio relation, and is not related in anyway to Film or ProductionCompany.
Your existing ability to retrieve the a set of Actors for a given FilmStudio (or vice-versa) does not necessarily imply you can store relational data belonging to these two models using the models defined in your example.
Keep in mind that each of the models you defined in your example are backed by a table in your database. In the case of Actor.films field, Django creates an extra table to store the many-to-many relationship data.
Since you're looking to store relational data between Actor and FilmStudio, you need to consider where the data will be stored in your database. Can you store the data in the Film model? or the ProductionCompany model?
I have some models with relationships like this:
class Item(model.Model):
name = models.CharField()
class Group(models.Model):
item = models.ManyToManyField(Item)
class Serie(models.Model):
name = models.CharField()
chart = models.ForeignKey(Chart)
group = models.ForeignKey(Group)
class Chart(models.Model):
name = models.CharField()
I need to create a Chart object on the fly, without saving to the DB. But I can't do it because Django tries to use the objects primary keys when assigning the relationships.
I just want Group.add(Item()) to work without having to save the objects to the DB.
Is there any simple way around this?
Reviving here for the sake of future readers:
I've gotten around this use case by defining a private attribute that represents the relationship inside the classes and a property to inspect wether the object can be retrieved from the DB or resides in memory.
Here is a simple example:
class Parent(models.Model):
_children = []
name = models.CharField(max_length=100)
#property
def children(self):
if _children:
return self._children
else:
return self.children_set.all()
def set_virtual_children(self, value): # could use a setter for children
self._children = value # Expose _children to modification
def some_on_the_fly_operation(self):
print(','.join([c.name for c in self.children]))
class Children(models.Model):
parent = models.ForeignKey(Parent)
name = models.CharField(max_length=100)
This way, I can set the "virtual children" and use all the defined methods "on the fly"
EDIT: It seems that approach described here isn't enough for django to allow adding to the ManyToMany relationship.
Have you tried to add primary_key=True and unique=True to the name attribute of the Item model. Then doing Group.add(Item("item_name_here")) should work if you have the possibility to create the name on the fly.
I didn't test it, but I think your way failed because add() wants to use the primary-key which by default is the autoincrementing id that is assigned when it is saved to the database.