I suppose that question as old as it possibly could be, but I still have a hope that maybe someone has found some magical solution.
Everything is simple, I have a database (postgresql) with small table as an example, with already existing field with Primary key:
enter image description here
My goal is to add an ID field as a second primary key.
My Model looks like this:
class CdcConn(models.Model):
id = models.BigAutoField(primary_key=True)
conn_type = models.CharField(max_length=128, blank=True, null=True)
conn_name = models.CharField(max_length=128)
class Meta:
managed = True
db_table = 'cdc_conn'
After migration the result output is:
django.db.utils.ProgrammingError: multiple primary keys for table "cdc_conn" are not allowed
Tried many options, unique_together, constraints, some additional modules such as django-viewflow, and so on.
The issue is about 15 years old, documentation says that composite pk are not allowed, but still, I can't believe that such powerful tool as Django just can't handle that simple operation.
Any thoughts would be highly appreciated.
I am trying to import a product feed (Product) into Django. I need to maintain a set of selected products (SelectedProduct) which also hold overrides for the product descriptions. I thought the best way to represent this is as a OneToOneField linking SelectedProduct with Product.
class Product(models.Model):
sku = models.CharField(max_length=32, primary_key=True)
title = models.CharField(max_length=200)
description = models.TextField(max_length=2000, blank=True, null=True)
class SelectedProduct(models.Model):
product = models.OneToOneField(Product, db_column='product_sku', on_delete=models.DO_NOTHING)
description = models.TextField(max_length=2000, blank=True, null=True)
For simplicity, each time the product feed is read, I am intending to delete all the products and re-import the whole product feed (within a transaction, so I can rollback if required).
However, I don't want to truncate the SelectedProduct at the same time, since this contains the descriptions which have been overridden. I was hoping that models.DO_NOTHING might help, but it doesn't.
I suppose I either need to temporarily disable the referential integrity while I import the feed (and delete any entries from SelectedProduct which would break the integrity) or I need to represent the relationship differently.
Any advice appreciated please :-)
Note - the above is a simplified representation. I have variants hanging off products too and will have selected variants overriding variant prices.
For the record, I went with the suggestion from #TimNyborg to use update_or_create. The data load now takes roughly 10 times as long, but as it's a batch import of a small-ish dataset the performance isn't an issue and it feels like a better solution overall.
My problem is that I have a model that can take one of two foreign keys to say what kind of model it is. I want it to take at least one but not both. Can I have this still be one model or should I split it into two types. Here is the code:
class Inspection(models.Model):
InspectionID = models.AutoField(primary_key=True, unique=True)
GroupID = models.ForeignKey('PartGroup', on_delete=models.CASCADE, null=True, unique=True)
SiteID = models.ForeignKey('Site', on_delete=models.CASCADE, null=True, unique=True)
#classmethod
def create(cls, groupid, siteid):
inspection = cls(GroupID = groupid, SiteID = siteid)
return inspection
def __str__(self):
return str(self.InspectionID)
class InspectionReport(models.Model):
ReportID = models.AutoField(primary_key=True, unique=True)
InspectionID = models.ForeignKey('Inspection', on_delete=models.CASCADE, null=True)
Date = models.DateField(auto_now=False, auto_now_add=False, null=True)
Comment = models.CharField(max_length=255, blank=True)
Signature = models.CharField(max_length=255, blank=True)
The problem is the Inspection model. This should be linked to either a group or a site, but not both. Currently with this set up it needs both.
I'd rather not have to split this up into two nearly identical models GroupInspection and SiteInspection, so any solution that keeps it as one model would be ideal.
I would suggest that you do such validation the Django way
by overriding the clean method of Django Model
class Inspection(models.Model):
...
def clean(self):
if <<<your condition>>>:
raise ValidationError({
'<<<field_name>>>': _('Reason for validation error...etc'),
})
...
...
Note, however, that like Model.full_clean(), a model’s clean() method is not invoked when you call your model’s save() method.
it needs to be called manually to validate model's data, or you can override model's save method to make it always call the clean() method before triggering the Model class save method
Another solution that might help is using GenericRelations,
in order to provide a polymorphic field that relates with more than one table, but it can be the case if these tables/objects can be used interchangeably in the system design from the first place.
As mentionned in comments, the reason that "with this set up it needs both" is just that you forgot to add the blank=True to your FK fields, so your ModelForm (either custom one or the default generated by the admin) will make the form field required. At the db schema level, you could fill both, either one or none of those FKs, it would be ok since you made those db fields nullable (with the null=True argument).
Also, (cf my other comments), your may want to check that your really want to FKs to be unique. This technically turns your one to many relationship into a one to one relationship - you're only allowed one single 'inspection' record for a given GroupID or SiteId (you can't have two or more 'inspection' for one GroupId or SiteId). If that's REALLY what you want, you may want to use an explicit OneToOneField instead (the db schema will be the same but the model will be more explicit and the related descriptor much more usable for this use case).
As a side note: in a Django Model, a ForeignKey field materializes as a related model instance, not as a raw id. IOW, given this:
class Foo(models.Model):
name = models.TextField()
class Bar(models.Model):
foo = models.ForeignKey(Foo)
foo = Foo.objects.create(name="foo")
bar = Bar.objects.create(foo=foo)
then bar.foo will resolve to foo, not to foo.id. So you certainly want to rename your InspectionID and SiteID fields to proper inspection and site. BTW, in Python, the naming convention is 'all_lower_with_underscores' for anything else than class names and pseudo-constants.
Now for your core question: there's no specific standard SQL way of enforcing a "one or the other" constraint at the database level, so it's usually done using a CHECK constraint, which is done in a Django model with the model's meta "constraints" option.
This being said, how constraints are actually supported and enforced at the db level depends on your DB vendor (MySQL < 8.0.16 just plain ignore them for example), and the kind of constraint you will need here will not be enforced at the form or model level validation, only when trying to save the model, so you also want to add validation either at the model level (preferably) or form level validation, in both cases in the (resp.) model or form's clean() method.
So to make a long story short:
first double-check that you really want this unique=True constraint, and if yes then replace your FK field with a OneToOneField.
add a blank=True arg to both your FK (or OneToOne) fields
add the proper check constraint in your model's meta - the doc is succint but still explicit enough if you know to do complex queries with the ORM (and if you don't it's time you learn ;-))
add a clean() method to your model that checks your have either one or the other field and raises a validation error else
and you should be ok, assuming your RDBMS respects check constraints of course.
Just note that, with this design, your Inspection model is a totally useless (yet costly !) indirection - you'd get the exact same features at a lower cost by moving the FKs (and constraints, validation etc) directly into InspectionReport.
Now there might be another solution - keep the Inspection model, but put the FK as a OneToOneField on the other end of the relationship (in Site and Group):
class Inspection(models.Model):
id = models.AutoField(primary_key=True) # a pk is always unique !
class InspectionReport(models.Model):
# you actually don't need to manually specify a PK field,
# Django will provide one for you if you don't
# id = models.AutoField(primary_key=True)
inspection = ForeignKey(Inspection, ...)
date = models.DateField(null=True) # you should have a default then
comment = models.CharField(max_length=255, blank=True default="")
signature = models.CharField(max_length=255, blank=True, default="")
class Group(models.Model):
inspection = models.OneToOneField(Inspection, null=True, blank=True)
class Site(models.Model):
inspection = models.OneToOneField(Inspection, null=True, blank=True)
And then you can get all the reports for a given Site or Group with yoursite.inspection.inspectionreport_set.all().
This avoids having to add any specific constraint or validation, but at the cost of an additional indirection level (SQL join clause etc).
Which of those solution would be "the best" is really context-dependent, so you have to understand the implications of both and check how you typically use your models to find out which is more appropriate for your own needs. As far as I'm concerned and without more context (or in doubt) I'd rather use the solution with the less indirection levels, but YMMV.
NB regarding generic relations: those can be handy when you really have a lot of possible related models and / or don't know beforehand which models one will want to relate to your own. This is specially useful for reusable apps (think "comments" or "tags" etc features) or extensible ones (content management frameworks etc). The downside is that it makes querying much heavier (and rather impractical when you want to do manual queries on your db). From experience, they can quickly become a PITA bot wrt/ code and perfs, so better to keep them for when there's no better solution (and/or when the maintenance and runtime overhead is not an issue).
My 2 cents.
Django has a new (since 2.2) interface for creating DB constraints: https://docs.djangoproject.com/en/3.0/ref/models/constraints/
You can use a CheckConstraint to enforce one-and-only-one is non-null. I use two for clarity:
class Inspection(models.Model):
InspectionID = models.AutoField(primary_key=True, unique=True)
GroupID = models.OneToOneField('PartGroup', on_delete=models.CASCADE, blank=True, null=True)
SiteID = models.OneToOneField('Site', on_delete=models.CASCADE, blank=True, null=True)
class Meta:
constraints = [
models.CheckConstraint(
check=~Q(SiteID=None) | ~Q(GroupId=None),
name='at_least_1_non_null'),
),
models.CheckConstraint(
check=Q(SiteID=None) | Q(GroupId=None),
name='at_least_1_null'),
),
]
This will only enforce the constraint at the DB level. You will need to validate inputs in your forms or serializers manually.
As a side note, you should probably use OneToOneField instead of ForeignKey(unique=True). You'll also want blank=True.
It might be late to answer your question, but I thought my solution might fit to some other person's case.
I would create a new model, let's call it Dependency, and apply the logic in that model.
class Dependency(models.Model):
Group = models.ForeignKey('PartGroup', on_delete=models.CASCADE, null=True, unique=True)
Site = models.ForeignKey('Site', on_delete=models.CASCADE, null=True, unique=True)
Then I would write the logic to be applicable very explicitly.
class Dependency(models.Model):
group = models.ForeignKey('PartGroup', on_delete=models.CASCADE, null=True, unique=True)
site = models.ForeignKey('Site', on_delete=models.CASCADE, null=True, unique=True)
_is_from_custom_logic = False
#classmethod
def create_dependency_object(cls, group=None, site=None):
# you can apply any conditions here and prioritize the provided args
cls._is_from_custom_logic = True
if group:
_new = cls.objects.create(group=group)
elif site:
_new = cls.objects.create(site=site)
else:
raise ValueError('')
return _new
def save(self, *args, **kwargs):
if not self._is_from_custom_logic:
raise Exception('')
return super().save(*args, **kwargs)
Now you just need to create a single ForeignKey to your Inspection model.
In your view functions, you need to create a Dependency object and then assign it to your Inspection record. Make sure that you use create_dependency_object in your view functions.
This pretty much makes your code explicit and bug proof. The enforcement can be bypassed too very easily. But the point is that it needs prior knowledge to this exact limitation to be bypassed.
I think you're talking about Generic relations, docs.
Your answer looks similar to this one.
Sometime ago I needed to use Generic relations but I read in a book and somewhere else that the usage should be avoided, I think it was Two Scoops of Django.
I ended up creating a model like this:
class GroupInspection(models.Model):
InspectionID = models.ForeignKey..
GroupID = models.ForeignKey..
class SiteInspection(models.Model):
InspectionID = models.ForeignKey..
SiteID = models.ForeignKey..
I‘m not sure if it is a good solution and as you mentioned you'd rather not use it, but this is worked in my case.
I've searched around for a while, but can't seem to find an existing question for this (although it could be an issue of not knowing terminology).
I'm new to Django, and have been attempting to take a design which should be very expandable over time, and make it work with Django's ORM. Essentially, it's a series of many-to-many relationships using a shared junction table.
The design is a generic game crafting system, which says "if you meet [require], you can create [reward] using [cost] as materials." This allows items to be sold from any number of shops using the same system, and is generic enough to support a wide range of mechanics - I've seen it used successfully in the past.
Django doesn't support multiple M2M relationships sharing the same junction table (apparently since it has no way to work out the reverse relationship), so I seem to have these options:
Let it create its own junction tables, which ends up being six or more, or
Use foreign keys to the junction table in place of a built-in MTM relationship.
The first option is a bit of a mess, since I know I'll eventually have to add additional fields into the junction tables. The second option works pretty well. Unfortunately, because there is no foreign key from the junction table BACK to each of the other tables, I'm constantly fighting the admin system to get it to do what I want.
Here are the affected models:
class Craft(models.Model):
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey('Container', related_name="craft_cost")
reward = models.ForeignKey('Container', related_name="craft_reward")
require = models.ForeignKey('Container', related_name="craft_require")
class ShopContent(models.Model):
shopId = models.ForeignKey(Shop)
cost = models.ForeignKey('Container', related_name="shop_cost")
reward = models.ForeignKey('Container', related_name="shop_reward")
require = models.ForeignKey('Container', related_name="shop_require")
description = models.CharField(max_length=300)
class Container(models.Model):
name = models.CharField(max_length=30)
class ContainerContent(models.Model):
containerId = models.ForeignKey(Container, verbose_name="Container")
itemId = models.ForeignKey(Item, verbose_name="Item")
itemMin = models.PositiveSmallIntegerField(verbose_name=u"min amount")
itemMax = models.PositiveSmallIntegerField(verbose_name=u"max amount")
weight = models.PositiveSmallIntegerField(null=True, blank=True)
optionGroup = models.PositiveSmallIntegerField(null=True, blank=True,
verbose_name=u"option group")
Is there a simpler, likely obvious way to get this working? I'm attempting to allow inline editing of ContainerContent information from each related column on the Craft edit interface.
It sounds like you have a sort of "Transaction" that has a name, description, and type, and defines a cost, reward, and requirement. You should define that as a single model, not multiple ones (ShopContent, Craft, etc.).
class Transaction(models.Model):
TYPE_CHOICES = (('Craft', 0),
('Purchase', 1),
)
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey('Container')
reward = models.ForeignKey('Container')
require = models.ForeignKey('Container')
type = models.IntegerField(choices = TYPE_CHOICES)
Now Shop etc. can have a single ManyToManyField to Transaction.
Whether or not you use this particular model, the cost, reward and require relationships should all be in one place -- as above, or in OneToOne relationships with Craft, ShopContent etc. As you guessed, you shouldn't have a whole bunch of complex Many-To-Many through tables that are all really the same.
You mention at the bottom of your post that you're
attempting to allow inline editing of ContainerContent information from each related column on the Craft edit interface.
If you're modeling several levels of relationship, and using the admin app, you'll need to either apply some sort of nested inline patch, or use some sort of linking scheme like the one I use in my recent question, How do I add a link from the Django admin page of one object to the admin page of a related object?
I am smelling something is too complicated here, but I might be wrong. As a start,
is this any better? (ContainerContent will be figured out later)
class Cost(models.Model):
name = models.CharField(max_length=30)
class Reward(models.Model):
name = models.CharField(max_length=30)
class Require(models.Model):
name = models.CharField(max_length=30)
class Craft(models.Model):
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey(Cost)
reward = models.ForeignKey(Reward)
require = models.ForeignKey(Require)
class Shop(models.Model):
name = models.CharField(max_length=30)
crafts = models.ManyToMany(Craft, blank=True)
I'm am working on an app which will handle lots of information and am looking for the best way of creating my models. Since I have never worked with apps that deal with so many records, database optimization is not a topic I know lots of, but it seems to me that a good design is a good place to start.
Right now, I have a table for customers, a table for products and a table for product-customer (since we assign a code for each product a customer buys). Since I want to track the balances, there is also a balance table. My models look like this at the moment:
class Customer(models.Model):
first_name = models.CharField(max_length=35)
last_name = models.CharField(max_length=35)
customer_ID= models.IntegerField(primary_key=True)
phone = models.CharField(max_length=10, blank=True, null=True)
class Product(models.Model):
product_ID = models.IntegerField(primary_key=True)
product_code = models.CharField(max_length=25)
invoice_date = models.DateField()
employee = models.ForeignKey(Employee, null=True, blank=True)
product_active = models.BooleanField()
class ProductCustomer(models.Model):
prod = models.ForeignKey(Product, db_index=True)
cust = models.ForeignKey(Customer, db_index=True)
product_customer_ID = models.IntegerField(primary_key=True)
[...]
class Balance(models.Model):
product_customer = models.ForeignKey(ProductCustomer, db_index=True)
balance = models.DecimalField(max_digits=10, decimal_places=2)
batch = models.ForeignKey(Batch)
[...]
The app will return the 'history' of the customer. If the pax was overdue at some point and then he paid and then was due for a refund, etc.
I was thinking if I should insert a CharField on the Pax table which would hold a dictionary with date:status (the status could be calculated and added to the dictionary when I upload the information) or if it is more efficient to do a query on the Balance table, or if there is a better solution to be implemented.
Since there are thousands of products and even more customers, we are talking about around 400K records for the balances on a weekly basis... I am concerned about what can be done to ensure the app runs smoothly.
If I understand your question you seem to be asking about whether the join conditions will impose an unreasonable burden on your lookup query. To some extent this depends on your rdbms. My recommendation is that you go with PostgreSQL over MySQL because MySQL's innodb tables are heavily optimized for primary key lookups and this means two btrees have to be traversed in order to find the records on a join. PostgreSQL on the other hand allows for physical scans of tables meaning foreign key lookups are a bit faster usually.
In general yes, the dictionary approach is fine for an app with lots of records. The questions typically come out of how you are querying and how many records you are pulling in a given query. That is a much larger factor than how many records are stored, at least for a db like PostgreSQL.