I would like to discuss the case of using GenericRelation and GenericForeignKey.
I have the 2 models, Appartement and Mission. And I need to create LockCode model which will be connected to Appartment or Mission. I have added the GenericForeignKey to the LockCode and GenericRelation to the Appartement and Mission:
class LockCode(TimeStampedModel):
context_type = models.ForeignKey(ContentType, on_delete=models.CASCADE)
context_id = models.PositiveIntegerField()
context = GenericForeignKey('context_type', 'context_id')
class Mission(DirtyFieldsMixin, models.Model):
lock_codes = GenericRelation(
LockCode,
content_type_field='context_type',
object_id_field='context_id',
related_query_name='mission'
)
class Appartment(DirtyFieldsMixin, models.Model):
lock_codes = GenericRelation(
LockCode,
content_type_field='context_type',
object_id_field='context_id',
related_query_name='appartment'
)
It works ok. But added the complexity level to compare with adding 2 ForeignKeys, for appartement and for mission.
class LockCode(TimeStampedModel):
appartement = models.ForeignKey(Appartement, null=True, blank=True)
mission = models.ForeignKey(Mission, null=True, blank=True)
So should I keep the GFK or use 2 simple FK?
If you go with the second option, you will have to build in logic to make sure either appartement or mission is not null. If you were to ever add more such foreign key fields, this logic would become increasingly complex.
If you are sure you are never going to add more foreign keys and you don't mind the overhead of ensuring one of them is not null then you can go ahead and use the foreign keys, but for scalability I would stick with generic relations.
The complexity does not all happen in the field definitions. It also happens at query time: given a LockCode, how do you identify whether it belongs to an apartment or mission? With two foreign keys, you would need to check both each time and catch any exceptions.
If you never need to follow the relationship that way, then yes the GFK is unnecessary and two FKs would be better.
Related
My problem is that I have a model that can take one of two foreign keys to say what kind of model it is. I want it to take at least one but not both. Can I have this still be one model or should I split it into two types. Here is the code:
class Inspection(models.Model):
InspectionID = models.AutoField(primary_key=True, unique=True)
GroupID = models.ForeignKey('PartGroup', on_delete=models.CASCADE, null=True, unique=True)
SiteID = models.ForeignKey('Site', on_delete=models.CASCADE, null=True, unique=True)
#classmethod
def create(cls, groupid, siteid):
inspection = cls(GroupID = groupid, SiteID = siteid)
return inspection
def __str__(self):
return str(self.InspectionID)
class InspectionReport(models.Model):
ReportID = models.AutoField(primary_key=True, unique=True)
InspectionID = models.ForeignKey('Inspection', on_delete=models.CASCADE, null=True)
Date = models.DateField(auto_now=False, auto_now_add=False, null=True)
Comment = models.CharField(max_length=255, blank=True)
Signature = models.CharField(max_length=255, blank=True)
The problem is the Inspection model. This should be linked to either a group or a site, but not both. Currently with this set up it needs both.
I'd rather not have to split this up into two nearly identical models GroupInspection and SiteInspection, so any solution that keeps it as one model would be ideal.
I would suggest that you do such validation the Django way
by overriding the clean method of Django Model
class Inspection(models.Model):
...
def clean(self):
if <<<your condition>>>:
raise ValidationError({
'<<<field_name>>>': _('Reason for validation error...etc'),
})
...
...
Note, however, that like Model.full_clean(), a model’s clean() method is not invoked when you call your model’s save() method.
it needs to be called manually to validate model's data, or you can override model's save method to make it always call the clean() method before triggering the Model class save method
Another solution that might help is using GenericRelations,
in order to provide a polymorphic field that relates with more than one table, but it can be the case if these tables/objects can be used interchangeably in the system design from the first place.
As mentionned in comments, the reason that "with this set up it needs both" is just that you forgot to add the blank=True to your FK fields, so your ModelForm (either custom one or the default generated by the admin) will make the form field required. At the db schema level, you could fill both, either one or none of those FKs, it would be ok since you made those db fields nullable (with the null=True argument).
Also, (cf my other comments), your may want to check that your really want to FKs to be unique. This technically turns your one to many relationship into a one to one relationship - you're only allowed one single 'inspection' record for a given GroupID or SiteId (you can't have two or more 'inspection' for one GroupId or SiteId). If that's REALLY what you want, you may want to use an explicit OneToOneField instead (the db schema will be the same but the model will be more explicit and the related descriptor much more usable for this use case).
As a side note: in a Django Model, a ForeignKey field materializes as a related model instance, not as a raw id. IOW, given this:
class Foo(models.Model):
name = models.TextField()
class Bar(models.Model):
foo = models.ForeignKey(Foo)
foo = Foo.objects.create(name="foo")
bar = Bar.objects.create(foo=foo)
then bar.foo will resolve to foo, not to foo.id. So you certainly want to rename your InspectionID and SiteID fields to proper inspection and site. BTW, in Python, the naming convention is 'all_lower_with_underscores' for anything else than class names and pseudo-constants.
Now for your core question: there's no specific standard SQL way of enforcing a "one or the other" constraint at the database level, so it's usually done using a CHECK constraint, which is done in a Django model with the model's meta "constraints" option.
This being said, how constraints are actually supported and enforced at the db level depends on your DB vendor (MySQL < 8.0.16 just plain ignore them for example), and the kind of constraint you will need here will not be enforced at the form or model level validation, only when trying to save the model, so you also want to add validation either at the model level (preferably) or form level validation, in both cases in the (resp.) model or form's clean() method.
So to make a long story short:
first double-check that you really want this unique=True constraint, and if yes then replace your FK field with a OneToOneField.
add a blank=True arg to both your FK (or OneToOne) fields
add the proper check constraint in your model's meta - the doc is succint but still explicit enough if you know to do complex queries with the ORM (and if you don't it's time you learn ;-))
add a clean() method to your model that checks your have either one or the other field and raises a validation error else
and you should be ok, assuming your RDBMS respects check constraints of course.
Just note that, with this design, your Inspection model is a totally useless (yet costly !) indirection - you'd get the exact same features at a lower cost by moving the FKs (and constraints, validation etc) directly into InspectionReport.
Now there might be another solution - keep the Inspection model, but put the FK as a OneToOneField on the other end of the relationship (in Site and Group):
class Inspection(models.Model):
id = models.AutoField(primary_key=True) # a pk is always unique !
class InspectionReport(models.Model):
# you actually don't need to manually specify a PK field,
# Django will provide one for you if you don't
# id = models.AutoField(primary_key=True)
inspection = ForeignKey(Inspection, ...)
date = models.DateField(null=True) # you should have a default then
comment = models.CharField(max_length=255, blank=True default="")
signature = models.CharField(max_length=255, blank=True, default="")
class Group(models.Model):
inspection = models.OneToOneField(Inspection, null=True, blank=True)
class Site(models.Model):
inspection = models.OneToOneField(Inspection, null=True, blank=True)
And then you can get all the reports for a given Site or Group with yoursite.inspection.inspectionreport_set.all().
This avoids having to add any specific constraint or validation, but at the cost of an additional indirection level (SQL join clause etc).
Which of those solution would be "the best" is really context-dependent, so you have to understand the implications of both and check how you typically use your models to find out which is more appropriate for your own needs. As far as I'm concerned and without more context (or in doubt) I'd rather use the solution with the less indirection levels, but YMMV.
NB regarding generic relations: those can be handy when you really have a lot of possible related models and / or don't know beforehand which models one will want to relate to your own. This is specially useful for reusable apps (think "comments" or "tags" etc features) or extensible ones (content management frameworks etc). The downside is that it makes querying much heavier (and rather impractical when you want to do manual queries on your db). From experience, they can quickly become a PITA bot wrt/ code and perfs, so better to keep them for when there's no better solution (and/or when the maintenance and runtime overhead is not an issue).
My 2 cents.
Django has a new (since 2.2) interface for creating DB constraints: https://docs.djangoproject.com/en/3.0/ref/models/constraints/
You can use a CheckConstraint to enforce one-and-only-one is non-null. I use two for clarity:
class Inspection(models.Model):
InspectionID = models.AutoField(primary_key=True, unique=True)
GroupID = models.OneToOneField('PartGroup', on_delete=models.CASCADE, blank=True, null=True)
SiteID = models.OneToOneField('Site', on_delete=models.CASCADE, blank=True, null=True)
class Meta:
constraints = [
models.CheckConstraint(
check=~Q(SiteID=None) | ~Q(GroupId=None),
name='at_least_1_non_null'),
),
models.CheckConstraint(
check=Q(SiteID=None) | Q(GroupId=None),
name='at_least_1_null'),
),
]
This will only enforce the constraint at the DB level. You will need to validate inputs in your forms or serializers manually.
As a side note, you should probably use OneToOneField instead of ForeignKey(unique=True). You'll also want blank=True.
It might be late to answer your question, but I thought my solution might fit to some other person's case.
I would create a new model, let's call it Dependency, and apply the logic in that model.
class Dependency(models.Model):
Group = models.ForeignKey('PartGroup', on_delete=models.CASCADE, null=True, unique=True)
Site = models.ForeignKey('Site', on_delete=models.CASCADE, null=True, unique=True)
Then I would write the logic to be applicable very explicitly.
class Dependency(models.Model):
group = models.ForeignKey('PartGroup', on_delete=models.CASCADE, null=True, unique=True)
site = models.ForeignKey('Site', on_delete=models.CASCADE, null=True, unique=True)
_is_from_custom_logic = False
#classmethod
def create_dependency_object(cls, group=None, site=None):
# you can apply any conditions here and prioritize the provided args
cls._is_from_custom_logic = True
if group:
_new = cls.objects.create(group=group)
elif site:
_new = cls.objects.create(site=site)
else:
raise ValueError('')
return _new
def save(self, *args, **kwargs):
if not self._is_from_custom_logic:
raise Exception('')
return super().save(*args, **kwargs)
Now you just need to create a single ForeignKey to your Inspection model.
In your view functions, you need to create a Dependency object and then assign it to your Inspection record. Make sure that you use create_dependency_object in your view functions.
This pretty much makes your code explicit and bug proof. The enforcement can be bypassed too very easily. But the point is that it needs prior knowledge to this exact limitation to be bypassed.
I think you're talking about Generic relations, docs.
Your answer looks similar to this one.
Sometime ago I needed to use Generic relations but I read in a book and somewhere else that the usage should be avoided, I think it was Two Scoops of Django.
I ended up creating a model like this:
class GroupInspection(models.Model):
InspectionID = models.ForeignKey..
GroupID = models.ForeignKey..
class SiteInspection(models.Model):
InspectionID = models.ForeignKey..
SiteID = models.ForeignKey..
I‘m not sure if it is a good solution and as you mentioned you'd rather not use it, but this is worked in my case.
I am running into a little bit of unique problem and wanted to see which solution fit best practice or if I was missing anything in my design.
I have a model - it has a field on it that represents a metric. That metric is a foreign key to an object which can come from several database tables.
Idea one:
Multiple ForeignKey fields. I'll have the benefits of the cascade options, direct access to the foreign key model instance from MyModel, (although that's an easy property to add), and the related lookups. Pitfalls include needing to check an arbitrary number of fields on the model for a FK. Another is logic to make sure that only one FK field has a value at a given time (easy to check presave) although .update poses a problem. Then theres added space in the database from all of the columns, although that is less concerning.
class MyModel(models.Model):
source_one = models.ForeignKey(
SourceOne,
null=True,
blank=True,
on_delete=models.SET_NULL,
db_index=True
)
source_two = models.ForeignKey(
SourceTwo,
null=True,
blank=True,
on_delete=models.SET_NULL,
db_index=True
)
source_three = models.ForeignKey(
SourceThree,
null=True,
blank=True,
on_delete=models.SET_NULL,
db_index=True
)
Idea two:
Store a source_id and source on the model. Biggest concern I have with this is needing to maintain logic to set these fields to null if the source is deleted. It otherwise seems like a cleaner solution, but not sure if the overhead to make sure the data is accurate is worth it. I can probably write some logic in a delete hook on the fk models to clean MyModel up if necessary.
class MyModel(models.Model):
ONE = 1
TWO = 2
THREE = 3
SOURCES = (
(ONE, "SourceOne"),
(TWO, "SourceTwo"),
(THREE, "SourceThree")
)
source_id = models.PositiveIntegerField(null=True, blank=True)
source = models.PositiveIntegerField(null=True, blank=True, choices=SOURCES)
I would love the communities opinion.
Your second idea seems fragile as the integrity is not ensured by the database as you have pointed out yourself.
Without knowing more about the use case, it's difficult to provide an enlightened advice however if your "metric" object is refered by many other tables, I wonder if you should consider approaching this the other way round, i.e. defining the relationships from the models consuming this metric.
To exemplify, let's say that your project is a photo gallery and that your model represents a tag. Tags could be associated to photos, photo albums or users (e.g.. the tags they want to follow).
The approach would be as follow:
class Tag(models.Model):
pass
class Photo(models.Model):
tags = models.ManyToManyField(Tag)
class Album(models.Model):
tags = models.ManyToManyField(Tag)
class User(AbstractUser):
followed_tags = models.ManyToManyField(Tag)
You may even consider to factor in this relationship in an abstract model as outlined below:
class Tag(models.Model):
pass
class TaggedModel(models.Model):
tags = models.ManyToManyField(Tag)
class Meta:
abstract = True
class Photo(TaggedModel):
pass
As mentioned in the comments, you are looking for a Generic Relation:
from django.contrib.contenttypes.fields import GenericForeignKey
from django.contrib.contenttypes.models import ContentType
class SourceA(models.Model):
name = models.CharField(max_length=45)
class SourceB(models.Model):
name = models.CharField(max_length=45)
class MyModel(models.Model):
content_type = models.ForeignKey(ContentType, on_delete=models.CASCADE)
object_id = models.PositiveIntegerField()
source = GenericForeignKey('content_type', 'object_id')
There are three parts to setting up a Generic Relation:
Give your model a ForeignKey to ContentType. The usual name for this field is “content_type”.
Give your model a field that can store primary key values from the models you’ll be relating to. For most models, this means a PositiveIntegerField. The usual name for this field is “object_id”.
Give your model a GenericForeignKey, and pass it the names of the two fields described above. If these fields are named “content_type” and “object_id”, you can omit this – those are the default field names GenericForeignKey will look for.
Now you can pass any Source instance to the source field of MyModel, regardless of which model it belongs to:
source_a = SourceA.objects.first()
source_b = SourceB.objects.first()
MyModel.objects.create(source=source_a)
MyModel.objects.create(source=source_b)
In Django, is it possible to create multiple many to many relationships via multiple intermediate models between two models?
For example, I have an object user and an object stock_position, and every time the user makes a trade, I want to make an intermediate model (transaction) that has user and stock_position as foreign keys.
You mean like this?
class Transactions(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)
stock_position = models.ForeignKey(StockPosition,
on_delete=models.CASCADE)
share = models.IntegerField()
class Meta:
unique_together = (('user', 'stock_position', 'share'),)
If you use this unique index, you can insert as many user/stock_position duplicates as you want, as long as the share column is different each time. Otherwise an IntegrityError will be raised (which you should usually/always have an exception for in your code)
I have a Django model that is doing way too much. Here's an abbreviated example of the model. Basically, it can represent four different Entity types, and there are recursive ForeignKey and ManyToMany relationships that point to other entities.
This project is currently using Django 1.8.x and Python 2.7.x, but I can upgrade those if the solution requires it.
class Entity(models.Model):
"""
Films, People, Companies, Terms & Techniques
"""
class Meta:
ordering = ['name']
verbose_name_plural = 'entities'
# Types:
FILM = 'FILM'
PERSON = 'PERS'
COMPANY = 'COMP'
TERM = 'TERM'
TYPE_CHOICES = (
(FILM, 'Film'),
(PERSON, 'Person'),
(COMPANY, 'Company'),
(TERM, 'Term/Technique'),
)
created = models.DateTimeField(auto_now_add=True, auto_now=False)
updated = models.DateTimeField(auto_now_add=False, auto_now=True)
type = models.CharField(max_length=4, choices=TYPE_CHOICES, default=FILM)
slug = models.SlugField(blank=True, unique=True, help_text="Automatically generated")
name = models.CharField(max_length=256, blank=True)
redirect = models.ForeignKey('Entity', related_name='entity_redirect', blank=True, null=True, help_text="If this is an alias (see), set Redirect to the primary entry.")
cross_references = models.ManyToManyField('Entity', related_name='entity_cross_reference', blank=True, help_text="This is a 'see also' — 'see' should be performed with a redirect.")
[... and more fields, some of them type-specific]
I realize this is rather messy, and I'd like to remove 'type' and make an EntityBase class that abstracts out all of the common fields, and create new Film, Person, Company, and Term models that inherit from the EntityBase abstract base class.
Once I create the new models, I think I understand how to write the data migration to move all of the field data over to the new models (iterate over objects from Entity, filtered via type, create new objects in the appropriate new model)... except the ForeignKey and ManyToMany relationships. Maybe I'm thinking about this the wrong way, but how can I transfer those relationships when, during the migration, the new object that the relationship points to may not exist yet?
I suspect this may mean a multi-step migration, but I haven't quite worked out the right way to do it.
There is nothing magical about m2m and fk fields. This is the procedure that I would follow... It might be a bit blunt, but will get the job done:
Make a BACKKKUPPPPPPppp of the database!!
Make another backup!
Create the new model and migration
Write a new data migration that will manually iterate over existing models and update the new model, one-by-one. Don't be afraid of the for loop here, unless you have millions of entries in db.
Delete redundant models and/or fields, make migration for this.
Run those migrations :)
In practice, this means a lot of restoring from the "BACKKKUPPPPPPppp" until the migrations are just right.
One little thing to take care of:
M2m fields cannot get any value if model is not yet saved (because model gets its ID on first save). I would do something like, in the manual migration:
new_instance = NewModel()
new_instance.somefield = "whatever"
new_instance.meaning = 42
....
new_instance.save()
new_instance.that_m2m_field.add(some_related_obj)
Of course, make sure you read the docs in detail, especially that bit about importing the model class - you can't just import it from myapp.models import MyModel, instead do:
MyModel = apps.get_model("myapp", "MyModel")
One possible tripping stone might be the model inheritance that you plan to introduce. Generally, you will want to operate on the child model, and access the parent from there as / if needed. Parent can be accessed via the implicit ptr attribute - in your example it would be entitybase_ptr or something similar (that is just a OneToOne field). Going in the other direction, however, (from parent to unknown child) is not as straightforward, because parent doesn't a priori know what is the class of its child.
I'm not so great with databases so sorry if I don't describe this very well...
I have an existing Oracle database which describes an algorithim catalogue.
There are two tables algorithims and xref_alg.
Algorithims can have parents and children algorithms. Alg_Xref contains these relationships with two foreign keys - xref_alg and xref_parent.
These are the Django models I have so far from the inspectdb command
class Algorithms(models.Model):
alg_id = models.AutoField(primary_key=True)
alg_name = models.CharField(max_length=100, blank=True)
alg_description = models.CharField(max_length=1000, blank=True)
alg_tags = models.CharField(max_length=100, blank=True)
alg_status = models.CharField(max_length=1, blank=True)
...
class Meta:
db_table = u'algorithms'
class AlgXref(models.Model):
xref_alg = models.ForeignKey(Algorithms, related_name='algxref_alg' ,null=True, blank=True)
xref_parent = models.ForeignKey(Algorithms, related_name='algxref_parent', null=True, blank=True)
class Meta:
db_table = u'alg_xref'
On trying to query AlgXref I encounter this:
DatabaseError: ORA-00904: "ALG_XREF"."ID": invalid identifier
So the error seems to be that it looks for a primary key ID which isn't in the table.. I could create one but seems a bit pointless. Is there anyway to get around this? Or change my models?
EDIT: So after a bit of searching it seems that Django requires a model to have a primary key. Life is too short so have just added a primary key. Will this have any impact on performance?
This is currently a limitation of the ORM provided by Django. Each model has to have one field marked as primary_key=True, if there isn't one, the framework automatically creates an AutoField with name id.
However, this is being worked on as we speak as part of this year's Google Summer of Code and hopefully will be in Django by the end of this year. For now you can try to use the fork of Django available at https://github.com/koniiiik/django which contains an implementation (which is not yet complete but should be sufficient for your purposes).
As for whether there is any benefit or not, that depends. It certainly makes the database more reusable and causes less headaches if you just add an auto incrementing id column to each table. The performance impact shouldn't be too high, the only thing you might notice is that if you have a many-to-many table like this, containing only two ForeignKey columns, adding a third one will increase its size by one half. That should, however, be irrelevant as long as you don't store billions of rows in that table.