What are the pros and cons of using GenericForeignKey vs multitable inheritance vs OneToOneField? - django

Context
I am in the process of modeling my data using Django models.
The main model is an Article. It holds the actual content.
Then each Article must be attached to a group of articles. Those group may be a Blog, a Category a Portfolio or a Story. Every Article must be attached to one, and exactly one of those. That is, either a blog, a category or a story. Those models have very different fields and features.
I thought of three ways to reach that goal (and a bonus one that really looks wrong).
Option #1: A generic foreign key
As in django.contrib.contenttypes.fields.GenericForeignKey. It would look like this:
class Category(Model):
# some fields
class Blog(Model):
# some fields
class Article(Model):
group_type = ForeignKey(ContentType)
group_id = PositiveIntegerField()
group = GenericForeignKey('group_type', 'group_id')
# some fields
On the database side, that means no relation actually exists between the models, they are enforced by Django.
Option #2: Multitable inheritance
Make article groups all inherit from an ArticleGroup model. This would look like this:
class ArticleGroup(Model):
group_type = ForeignKey(ContentType)
class Category(ArticleGroup):
# some fields
class Blog(ArticleGroup):
# some fields
class Article(Model):
group = ForeignKey(ArticleGroup)
# some fields
On the database side, this creates an additional table for ArticleGroup, then Category and Blog have an implicit foreign key to that table as their primary key.
Sidenote: I know there is a package that automates the bookkeeping of such constructions.
Option #3: manual OneToOneFields
On the database side, it is equivalent to option #2. But in the code, all relations are made explicit:
class ArticleGroup(Model):
group_type = ForeignKey(ContentType)
class Category(Model):
id = OneToOneField(ArticleGroup, primary_key=True)
# some fields
class Blog(Model):
id = OneToOneField(ArticleGroup, primary_key=True)
# some fields
class Article(Model):
group = ForeignKey(ArticleGroup)
# some fields
I don't really see what the point of that would be, apart from making explicit what Django's inheritance magic implicitly does.
Bonus: multicolumn
It seems pretty dirty so I just add it as a bonus, but it would also be possible to define a nullable ForeignKey to each of Category, Blog, ... directly on the Article model.
So...
...I cannot really decide between those. What are the pros and cons of each approach? Are there some best practices? Did I miss a better approach?
If that matters, I'm using Django 1.8.

It seems noone had advice to share on that one.
I eventually chose the multicolumn option, despite having said it looked ugly. It all came down to 3 things:
Database-based enforceability.
The way Django ORM works with the different constructs.
My own needs (namely, collection queries on the group to get the item list, and individual queries on the items to get the group).
Option #1
Cannot be enforced at the database level.
Could be efficient on queries because the way it is constructed does not fall into usual generic foreign key pitfalls. Those happen when the items are generic, not the collections.
However, due to how the ORM handles GFK, it is impossible to use a custom manager, which I need because my articles are translated using django-hvad.
Option #2
Can be enforced at the database level.
Could be somewhat efficient, but runs into ORM limitations, which is clearly not built around this use. Unless I use extra() or custom queries alot, but at some point there is no reason to use an ORM anymore.
Option #3
Would actually be a bit better than #2, as making things explicit allows easier query optimisation while using the ORM.
Multicolumn
Turns out not being so bad. It can be enforced at the database level (FK constraints plus a manual CHECK to ensure only one of the columns is non-null).
Easy and efficient. A single intuitive query does the job: select_related('category', 'blog', ...).
Though it does have the issue of being harder to extend (any new type will require altering the Article's table as well) and limiting the possible number of types, I'm unlikely to run into those.
Hope it helps anyone with the same dilemma, and still interested in hearing other opinions.

Related

How to write relation where first model is connected to exactly four instances of another in Django?

Let's say I have two very basic model classes - for simplicity let's name them a Plan and a Task. My goal is to force every plan to have exactly 4 distinct tasks (order doesn't matter). Is there some good practice for this "many-to-many with a fixed quantity of related instances" case?
from django.db import models
class Task(models.Model):
name = models.CharField(max_length=20)
class Plan(models.Model):
name = models.CharField(max_length=20)
# four_tasks = ?
I searched through Django documentation but there's no answer there (or maybe I didn't know how to search for it). I thought of 4 separate foreign keys (which should be possible by setting related_name for those) in Plan, or maybe standard many-to-many many relations. Both solutions require additional checks to ensure that there are actually 4 different tasks and they look ugly to me.
In my opinion, from the point of view of databases, the best practice would be to have the 4 separate foreign keys (as you thought) as a primary key (composite key).
This can be achieved in Django using the uniqueConstraint which is the preferable way over unique_together option:
UniqueConstraint provides more functionality than unique_together. unique_together may be deprecated in the future.
EDIT: If you are wondering how to use it, here is an answer

Is there a Django ManyToManyField with implied ownership?

Let's imagine I'm building a Django site "CartoonWiki" which allows users to write wiki articles (represented by the WikiArticle-model) as well as posting in a forum (represented by the ForumPost-model). Over time more features will be added to the site.
A WikiArticle has a number of FileUploads which should be deleted when the WikiArticle is deleted. By "deleted" I mean Django's .delete()-method.
However, the FileUpload-model is generic -- it's not specific to WikiArticle -- and contains generic file upload logic that e.g. removes the file from S3 when it's removed from the database. Other models like ForumPost will use the FileUpload-model as well.
I don't want to use GenericForeignKey nor multi-table inheritance for the reasons Luke Plant states in the blog post Avoid Django's GenericForeignKey. (However, if you can convince me that there really is no better way than the trade-offs GenericForeignKey make, I might be swayed and accept a convincing answer of that sort.)
Now, the most trivial way to do this is to have:
class FileUpload(models.Model):
article = models.ForeignKey('WikiArticle', null=True, blank=True, on_delete=models.CASCADE)
post = models.ForeignKey('ForumPost', null=True, blank=True, on_delete=models.CASCADE)
But that will have the FileUpload-model expand indefinitely with more fields -- and similar its underlying table will gain more and more columns as new models in the system start using FileUpload. This feels suboptimal both in terms of data-modeling, but also in terms of separation-of-concerns -- the FileUpload-model and table is being changed while no actual new functionality is being added to it.
My preference would really be to go the other way around:
class WikiArticle(models.Model):
uploads = models.ManyToManyField('FileUpload')
But this doesn't solve the deletion issue: If I .delete() a WikiArticle the corresponding FileUploads won't be deleted. I've tried various setups with through-models, but none seem to solve it. What I really need is a OneToMany-field -- a sort of reverse ForeignKey to indicate the ownership in the right direction without polluting the generic/reusable model.
Should FileUpload really instead be a field? Or perhaps an abstract model? (WikiArticleFileUpload, ForumPostFileUpload, and so on...).
I realize that a true ManyToManyField with implied ownership would no longer really be a ManyToManyField since the field implies sharing. E.g. a FileUpload could technically be referenced by multiple WikiArticles, so you could be removing FileUploads from other objects rather on top of the one you're deleting. The question still stands though -- it seems I need a OneToManyField to model this in a nice way.
You probably have a couple of options to solve your problem, but it also requires on the exact requirements of your application.
Using a GenericForeignKey in this situation is probably fine, escpecially due to the fact that you do not know how many other models will use your upload model. Of course as mentioned in the linked blog post eg. doing plain SQL queries might be harder but it's on you to decide if that's a problem for your use case.
Also using inheritance might be an option, so that all the referenced models inherit the relation to the upload model from a common ancestor. This might have a small impact performance-wise because you Django would need to join the tables of the models but the impact might still be not that big. On the other hand this approach might also have some advantages if eg. your articles and posts have other stuff in common as well and you could easily do stuff like "show all new posts and articles (together)".
If you handle deletion yourself as mentioned in the previous answer you can also add ManyToMany fields yourself but also consider that this method also has some disadvantages in common with using generic foreign keys (eg. a lot of stuff to join in the database...)
Probably it's fine that you just use a GenericForeignKey, especially if the number of models that use your "generic" model gets bigger (eg. more than 3-5). All in all this sounds pretty much like a use case GenericForeignKey was made for (imagine the uploads being something like "tags" belonging to the posts).
ManyToMany fields are symmetrical, even though you define them on one model with an (explicit or implicit) related_name on the other.
I can think of two methods to clean up while, or after, WikiArticles are deleted. The first is to periodically search for and delete "orphan" FileUploads. At its simplest, (assuming a related_name of articles)
deleted = FileUpload.objects.filter( articles__isnull=True).delete()
The other is to explicitly process the related articles during deleting of the article. It's straightforward to subclass the object's delete method, but this is not the only way to delete an object (bulk_delete, for example, bypasses this). Anyway,
def delete( self, *args, **kwargs):
article_pks = self.uploads.all().values_list('pk', Flat=True)
response = super().delete( *args, **kwargs)
FileUpload.objects.filter(
pk__in = article_pks, articles__isnull=True) .delete()
return response
(or even just execute the "periodically" code above, for every article-deletion, which will also tity after any deleted though othr channels)
Please thoroughly test this if you use it. Delete operations which don't do precisely what is wanted are the scariest sorts of bug!

Django: storing model property on a field vs. on a different model

I am relatively new to Django and even database design and I have some thoughts I'd like to run by some other people. This isn't really a specific question; I just want to see how other people think about this stuff.
Let's say we have a model for an application to some service. It contains all the ordinary stuff you might imagine an application to contain:
class Application(models.Model):
first_name = CharField(max_length=255)
last_name = CharField(max_length=255)
date_of_birth = DateField()
married = BooleanField()
# ...other stuff
Okay, that's all well and good. But now, imagine the webapp you are writing has the feature that you can complete your application partially, save it, and come back to it later. One way to do this is to add another attribute to the model above:
complete = BooleanField()
It works, it is pretty simple to use, but I don't really like it because it muddies the semantics of an application; it adds information that isn't intrinsically connected to the application. Another approach would be to create another model that keeps track of complete applications:
class CompleteApplication(models.Model):
application = ForeignKey(Application)
I like this a bit better, since it keeps Application clean. However, it does have the disadvantage of messing up queries. Here are the two ways to query all complete applications in the system:
Method 1:
completed_applications = Application.objects.filter(complete=True)
Method 2:
pks = CompleteApplication.objects.all().values_list("application__pk")
complete_applications = Application.object.filter(pk__in=pks)
Method 2 is two lines of code vs. one and also two queries whereas previously one sufficed, so the database performance is going to take a hit.
There is a third way to do things: instead of creating a model that keeps track of complete applications, we could create a metadata model that stores any metadata that we might want to attach to the Application model. For our purposes, this model can contain a field that tracks completeness. However, this approach also has the benefit of allowing for an arbitrary number of metadata fields to be associated with each application without requiring a new DB table for each (as is the case with Method 2 above).
class ApplicationMeta(models.Model):
application = ForeignKey(Application)
complete = BooleanField()
And, for completeness (pun intended), to query all complete applications, we would use the following statement:
completed_applications = Application.objects.all(applicationmeta__complete=True)
Nice and simple, just like Method 1, but the query is certainly more work for the database. This method also has another drawback for certain applications. Pretend, for example, that we want to track some additional information about applications: they can be confirmed, or rejected. However, if an application is not confirmed, it does NOT necessarily mean it is rejected: it could be pending review. Additionally, let's say we want to track the date of confirmation and the date of rejection (if either is applicable, of course). Then, our metadata model becomes the following:
class ApplicationMeta(models.Model):
complete = BooleanField()
confirmed = BooleanField()
rejected = BooleanField()
date_confirmed = DateField()
date_rejected = DateField()
Okay...this works, but it is starting to be a mess. Firstly, we have now opened up our system to potential error: what if somehow an ApplicationMeta instance has both rejected and confirmed set to True? We could do some fancy footwork with our class (maybe override setattr) to throw an exception if something funny happens, so we can prevent from persisting to the DB, but this is added complication that I hope is not necessary. Further, any model will either have at most one of date_confirmed or date_rejected set. Is that a problem? Here, I am not actually certain. My guess is this is likely a waste of space, but I don't actually know that. This example is simple, what if more complicated examples present us with tons of fields that will necessarily not be filled? Seems like bad design.
I'd love to hear some thoughts on these ideas.
Thanks!
If you have a huge amount of possible metadata, the third approach might make sense for performance reasons. I wouldn't do it for a few boolean- and date columns. If you're concerned about the readability of the models themselves, you can factor out any metadata into an abstract base model. You can even reuse the abstract model for other models that require the same metadata. The information will still live in your Application model.
If you do take the second or third approach, I would use a OneToOneField rather than a ForeignKey. It ensures that there are no 2 possible ApplicationMeta models for a single Application, and has the added benefit of a UNIQUE database index.
As for the status of an application, the NullBooleanField is designed for exactly that. It start as None (NULL in the db) meaning "no value". It can then be set to True (accepted) or False (rejected).

Django: using ContentType vs multi_table_inheritance

I was having a similar problem as in
How to query abstract-class-based objects in Django?
The thread suggests using multi_table_inheritance. I personally think using content_type more conceptually comfortable (just feels more close to logic, at least to me)
Using the example in the previous link, I would just add a StelarType as
class StellarType(models.Model):
"""
Use ContentType so we have a single access to all types
"""
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = generic.GenericForeignKey('content_type', 'object_id')
Then add this to the abstract base model
class StellarObject(BaseModel):
title = models.CharField(max_length=255)
description = models.TextField()
slug = models.SlugField(blank=True, null=True)
stellartype = generic.GenericForeignKey(StellarType)
class Meta:
abstract = True
To sync between StellarObject and StellarType, we can connect post_save signal to create a StellarType instance every time a Planet or Star is created. In this way, I can query StellarObjects through StellarType.
So I'd like to know what's the PRO and CON of using this approach against using multi_table_inheritance? I think both create an additional table in the databse. But how about database performance? how about usability/flexibility? Thanks for any of your input!
To me, ContentType is the way to go when you want to relate an object to one of many models that aren't fundamentally of the same "type". Like if you want to be able to key Comments to Users, Pages, and Pictures on a social network, but there's no reasonable supertype shared by those three models. Sure you could create a "Commentable" supertype, but to me that feels more like a mixin than a fundamental type from which those three things derive. Before ContentType came out, you would have had no choice but to invent supertypes for these kind of relations, which can get really ugly really quickly if you need to do it multiple times in the same application (lets say you also have Events, Alerts, Messages, etc., each of which can apply to a different set of models).
Multi-table inheritance makes the most sense when you want to attach attributes to the base model, such that they will be shared in all concrete models that extend from it, so that you can get polymorphic behavior. Commentable doesn't really fit this mold, because all of that behavior can be put on the Comment model, less so on the Commentable objects. But if you have different classes of Users that share much of the same behavior and should be aggregable, then it makes a lot more sense.
The major pro of multi-table inheritance to me is a cleaner data model, with implicit relationships and inheritance that can be taken advantage of on the Python side (polymorphism is still a bit messy though, as seen here and here). The major pro of ContentType is that it is more general and keeps auxiliary functionality out of your models, at the cost of a bit of a slightly less pristine schema (lots of "meta" fields on your models to define these relationships). And for your example, you still have to rely on post_save, which seems unnecessarily messy/magical to me, as well.
Sorry for reviving old thread. I think it all boils down to the lookup direction. Whether you look up all subclasses for a certain FK (multitable inheritance) or define the referenced class as a content type and look it up based on the table reference and id (contenttypes) makes no big difference in performance - hint: they both suck. I think content types is a nice choice if you want your app to be easily extendible, i.e. others can add new content types to reference against. Multitable is good if you only sometimes need the extra columns defined in extra tables. Sometimes it might also be a good idea to merge all your subtypes and make only one which has a few fields left empty most of the time.

Django end-user defined fields, how to? [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Django dynamic model fields
Good Morning guys!
Scenario is the following. For some models on Django, I would like to allow the end user to define his own fields. It would be great if I could keep all Django awesome features like the ORM, so I can still do calls like field__gte to search on the model, still have field validation according to field type, etc. I've thought about two ways of doing this, and I'm more than open for new suggestions. Any feedback would be VERY appreciated.
The first approach, is the Entity-Attribute-Value ( http://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%93value_model ), which django already has an app for. See http://code.google.com/p/django-custom-field/
I think this would be an OK solution, but I lose the ability to do "mymodel.objects.filter(custom_field_x=something)". Maybe there's a way to regain the ORM, any ideas? But I've heard so many bad stories about this method that I'm little scared to use it.
The second approach would be to have a database table for each of the users (probably no more than a 1000). I've read django has something in the lines of inspectdb, which actually checks which fields are there and produces the model for you. This could be useful but I think maybe I should store the fields this particular user has created and somehow dinamically tell django, hey, we also have this fields in this model. Is this possible? I know it's generally bad to have different tables for each user, but considering this scenario, how would you guys rate this method, would it be ok to have one table for each user?
The model that requires custom fields is for example Person. They might want a custom field to store address, blood type, or any other thing.
MANY THANKS in advance! Have a nice sunday!
Very similar: How to create user defined fields in Django -- but only talks about the EAV, which I would like to avoid. I'm open for new ideas!
One approach is to use a NoSQL document-based solution such as MongoDB which allows you to store objects that have a fluid structure (no such restrictions as pre-defined columns).
Pros:
No restriction on custom field types, number of types of fields, etc.
Retains ORM functionality (django-mongodb)
Other various benefits of NoSQL - which you can read about online
Avoids EAV
Cons:
Need to setup NoSQL server
Additional knowledge required on NoSQL concepts (documents vs. tables)
You may have to maintain two databases - if you decide not to migrate your entire solution to NoSQL (multi-db)
EDIT:
After reading the comments its worth pointing out that depending on which NoSQL solution you go with, you may not need reversion support. CouchDB, for example has built in support for document versioning.
what about creating another model for storing user_defined_fields?
class UserDefinedField(models.Model):
#..................
user = models.ForeignKey(User)
field_name = models.CharField(max_length=50)
field_value = models.TextField()
Then you can do UserDefinedField.objects.filter(field_name=some_name,field_value=somevalue)