Many foreign keys in one lookup table. Bad idea? - django

I am using Django, and my tables look like
class Product(models.Model):
category = models.CharField(max_length=50)
title = models.CharField(max_length=200)
class Value(models.Model):
name = models.CharField(max_length=200, unique=True)
class Attribute(models.Model):
name = models.CharField(max_length=200)
parent = models.ForeignKey('self', related_name='children')
values = models.ManyToManyField(Value, through='ProductAttributeRelationship', related_name='values')
class Meta:
unique_together = ('name', 'parent')
class ProductAttributeRelationship(models.Model):
product = models.ForeignKey(Product, related_name='products')
value = models.ForeignKey(Value, related_name='values')
attribute = models.ForeignKey(Attribute, related_name='attributes')
class Meta:
unique_together = ('product', 'value', 'attribute', 'price')
class Price(models.Model):
regular = models.IntegerField(blank=True, null=True)
sale = models.IntegerField(blank=True, null=True)
on_sale = models.NullBooleanField(blank=True)
created = models.DateTimeField(auto_now=True)
relation = models.ForeignKey(ProductAttributeRelationship)
class Meta:
unique_together = ('regular', 'sale', 'on_sale', 'sale_percentage')
Is it a bad idea to have the 3 ForeignKeys in ProductAttributeRelationship and the ForeignKey to that in Price since a ProductAttributeRelationship may have many prices? I don't have much knowledge in this area, and have been reading up about the 5 normalized forms, but am not sure where I should, or could, fit into the recommended 3rd form.

We declare a foreign key when a value for a subrow in one table has to appear as a value of a subrow in another table. That's what you have, so declare them.
Foreign keys have nothing to do with normalization per se. A normal form is something that a table is or isn't in. Normalization is about replacing a table by multiple tables that always join to it. A foreign key constraint holds when two tables have to agree per above. It can happen that new foreign keys holds between new tables from normalizing but if so you would just declare them. They don't affect what normal forms a table is in or normalization.
(Although ProductAttributeRelationship product, value, attribute and relationship are unique, presumably it is because product and price are unique, and product has just one price and an attribute has just one value. So you should say that product and price are unique; then all four have to be. Similarly, although Price regular, sale, on_sale and sale_percentage are unique, if regular, sale and on_sale are unique with sale_percentage a function of them then you should declare the three unique.)
(PS: 1. The main issue is integrity: If there is no constraint on the subset then invalid updates are allowed. 2. If the subset is unique then the superset is unique. So if the DBMS is enforcing subset uniqueness then it is enforcing superset uniqueness. 3. Moreover every superset of a CK is unique so there's nothing special about the particular extra columns you chose. 4. SQL DBMS UNIQUE/PK usually come with an index taking space and time to manage. For integrity and basic efficiency/optimization that's wasted on non-CK columns. But there can always be other special-case reasons for indexing. 5a. One reason to declare a non-CK superkey is that SQL forces you to do so to use it as a FK target. (You can either consider this redundancy as a helpful check or a tedious obtuseness.) 5b. Another reason is that sometimes this allows declarative (vs procedural/triggered) expression of integrity constraints via FK checking.)

Related

Django | PostgreSQL | ManyToManyField vs ForeignKey performance

I am going to a Vehicle table, which has two damage types - primary and secondary (optional). Not more, only these two. And I am wondering because I expect, that the Vehicle table will have really a lot of records in near future, about 1M+ and I am not sure if I should use ManyToMany field or two ForeignKeys with disabled related_name
class Damage(models.Model):
name = models.CharField(max_length=32, unique=True)
Should I use this solution:
class Vehicle(models.Model):
damage_primary = models.ForeignKey(Damage, on_delete=models.CASCADE, related_name='+')
damage_secondary = models.ForeignKey(Damage, on_delete=models.CASCADE, blank=True, related_name='+')
OR this one:
class Vehicle(models.Model):
damage = models.ManyToManyField(Damage, on_delete=models.CASCADE)
What is the best practice, please? Because internal table, which Django will create when I will use ManyToManyField will have really a lot of records, if, in Vehicles table, there will be 1 000 000 and more records, in future.
Thank you very much for any advice!
If you have only two damage type, you want two foreign key.
The related_name does not have any database performance influence.

How are many-to-many relationship handled in Django?

I have a shirt which can contain multiple colors, and multiple colors which can have multiple shirts. Normally I would express it the following way:
In django I have the many-to-many (https://docs.djangoproject.com/en/3.1/topics/db/examples/many_to_many/)
Example:
publications = models.ManyToManyField(Publication)
--
Can I create the table "Item_colors" consisting of 2 columns (no "ID" primary key) and design the models according to my diagram using the composite key:
class Item_colors(models.Model):
class Meta:
unique_together = (('cloth_item_id', 'color_id'),)
cloth_item_id = models.ForeignKey(Cloth_item, on_delete=models.CASCADE)
color_id = models.ForeignKey(Color, on_delete=models.CASCADE)
How is the many-to-many relation handled in a DB context, and does it yield better performance?
EDIT: https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys no avoiding primary keys in favor of composite keys saving columns :( at least for now..
How is the many-to-many relation handled in a DB context, and does it yield better performance?
With a junction table in the middle, so with an item_colors table. But the table contains a primary key, as does every model in Django.
If you do not specify a through=… parameter [Django-doc] to define the model for the junction table yourself, Django will automatically create such model. This model then has two ForeignKeys to the two models it connects as discussed in the database representation section of the documentation:
Behind the scenes, Django creates an intermediary join table to represent the many-to-many relationship. By default, this table name is generated using the name of the many-to-many field and the name of the table for the model that contains it. Since some databases don’t support table names above a certain length, these table names will be automatically truncated and a uniqueness hash will be used, e.g. author_books_9cdf. You can manually provide the name of the join table using the db_table option.
But the table thus has a primary key. This might be useful if the same object occurs a second time in the relation.
You can access the through model in the Article-Publication example for example with:
Article.publications.through
You thus can define a through model yourself, for example with:
class Color(models.Model):
color = models.CharField(max_length=128)
class ClothItem(models.Model):
item_name = models.CharField(max_length=128)
colors = models.ManyToManyField(
Color,
related_name='cloth_items'
through='ClothItemColors'
)
class ClothItemColors(models.Model):
cloth_item = models.ForeignKey(ClothItem, on_delete=models.CASCADE)
color = models.ForeignKey(Color, on_delete=models.CASCADE)
class Meta:
db_table = 'item_colors'
constraints = [
models.UniqueConstraint(
fields=('cloth_item', 'color'),
name='unique_cloth_color'
)
]
often an explicit through model is used to store extra information, for example the quantity:
class ClothItemColors(models.Model):
cloth_item = models.ForeignKey(ClothItem, on_delete=models.CASCADE)
color = models.ForeignKey(Color, on_delete=models.CASCADE)
quantity = models.IntegerField(default=0)
# …

How would I represent and model this relation?

So I have the following logic. A user creates a trade, this trade contains items a user wants and items a user has. These has and wants are represented as two separate many to many relationships.
However, here is where I am getting a little confused. These items can come from different models. For example I have a model set up for particular items, say business items which have unique characteristics and a second model for engineer items. Now these items can both be traded within the same trade.
Furthermore if I were to add more models representing other unique trade items how would I be able to add this to the many to many relationship without overhaul of the structure or data.
For illustration:
class ItemTrade(models.Model):
id = models.CharField(max_length=10, primary_key=True)
user = models.ForeignKey(get_user_model())
have_items = models.ManyToManyField()
want_items = models.ManyToManyField()
num_watch = models.IntegerField(max_length=10)
num_views = models.IntegerField(max_length=20)
class Model1(models.Model):
index = models.IntegerField(max_length=20, primary_key=True)
name = models.CharField(max_length=100)
type = models.ForeignKey('DotaItemType',null=True,blank=True)
class Model2(models.Model):
index= models.IntegerField(max_length=20, primary_key=True)
name = models.CharField(max_length=100)
type = models.ForeignKey('DotaItemType',null=True,blank=True)
desc = models.CharField(max_length=20)
The indexes will overlap for each item which means that I would also need to have secondary field to identify which model the item belongs to. What is the best possible method to represent this relation?
The only way I can think of is to have a unique many to many field for each model relation within the trade but I am not sure if this is the simplest and most effective way of accomplishing this.

What is the best way to model a heterogenous many-to-many relationship in Django?

I've searched around for a while, but can't seem to find an existing question for this (although it could be an issue of not knowing terminology).
I'm new to Django, and have been attempting to take a design which should be very expandable over time, and make it work with Django's ORM. Essentially, it's a series of many-to-many relationships using a shared junction table.
The design is a generic game crafting system, which says "if you meet [require], you can create [reward] using [cost] as materials." This allows items to be sold from any number of shops using the same system, and is generic enough to support a wide range of mechanics - I've seen it used successfully in the past.
Django doesn't support multiple M2M relationships sharing the same junction table (apparently since it has no way to work out the reverse relationship), so I seem to have these options:
Let it create its own junction tables, which ends up being six or more, or
Use foreign keys to the junction table in place of a built-in MTM relationship.
The first option is a bit of a mess, since I know I'll eventually have to add additional fields into the junction tables. The second option works pretty well. Unfortunately, because there is no foreign key from the junction table BACK to each of the other tables, I'm constantly fighting the admin system to get it to do what I want.
Here are the affected models:
class Craft(models.Model):
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey('Container', related_name="craft_cost")
reward = models.ForeignKey('Container', related_name="craft_reward")
require = models.ForeignKey('Container', related_name="craft_require")
class ShopContent(models.Model):
shopId = models.ForeignKey(Shop)
cost = models.ForeignKey('Container', related_name="shop_cost")
reward = models.ForeignKey('Container', related_name="shop_reward")
require = models.ForeignKey('Container', related_name="shop_require")
description = models.CharField(max_length=300)
class Container(models.Model):
name = models.CharField(max_length=30)
class ContainerContent(models.Model):
containerId = models.ForeignKey(Container, verbose_name="Container")
itemId = models.ForeignKey(Item, verbose_name="Item")
itemMin = models.PositiveSmallIntegerField(verbose_name=u"min amount")
itemMax = models.PositiveSmallIntegerField(verbose_name=u"max amount")
weight = models.PositiveSmallIntegerField(null=True, blank=True)
optionGroup = models.PositiveSmallIntegerField(null=True, blank=True,
verbose_name=u"option group")
Is there a simpler, likely obvious way to get this working? I'm attempting to allow inline editing of ContainerContent information from each related column on the Craft edit interface.
It sounds like you have a sort of "Transaction" that has a name, description, and type, and defines a cost, reward, and requirement. You should define that as a single model, not multiple ones (ShopContent, Craft, etc.).
class Transaction(models.Model):
TYPE_CHOICES = (('Craft', 0),
('Purchase', 1),
)
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey('Container')
reward = models.ForeignKey('Container')
require = models.ForeignKey('Container')
type = models.IntegerField(choices = TYPE_CHOICES)
Now Shop etc. can have a single ManyToManyField to Transaction.
Whether or not you use this particular model, the cost, reward and require relationships should all be in one place -- as above, or in OneToOne relationships with Craft, ShopContent etc. As you guessed, you shouldn't have a whole bunch of complex Many-To-Many through tables that are all really the same.
You mention at the bottom of your post that you're
attempting to allow inline editing of ContainerContent information from each related column on the Craft edit interface.
If you're modeling several levels of relationship, and using the admin app, you'll need to either apply some sort of nested inline patch, or use some sort of linking scheme like the one I use in my recent question, How do I add a link from the Django admin page of one object to the admin page of a related object?
I am smelling something is too complicated here, but I might be wrong. As a start,
is this any better? (ContainerContent will be figured out later)
class Cost(models.Model):
name = models.CharField(max_length=30)
class Reward(models.Model):
name = models.CharField(max_length=30)
class Require(models.Model):
name = models.CharField(max_length=30)
class Craft(models.Model):
name = models.CharField(max_length=30)
description = models.CharField(max_length=300, blank=True)
cost = models.ForeignKey(Cost)
reward = models.ForeignKey(Reward)
require = models.ForeignKey(Require)
class Shop(models.Model):
name = models.CharField(max_length=30)
crafts = models.ManyToMany(Craft, blank=True)

Django many to many recursive relationship

I'm not so great with databases so sorry if I don't describe this very well...
I have an existing Oracle database which describes an algorithim catalogue.
There are two tables algorithims and xref_alg.
Algorithims can have parents and children algorithms. Alg_Xref contains these relationships with two foreign keys - xref_alg and xref_parent.
These are the Django models I have so far from the inspectdb command
class Algorithms(models.Model):
alg_id = models.AutoField(primary_key=True)
alg_name = models.CharField(max_length=100, blank=True)
alg_description = models.CharField(max_length=1000, blank=True)
alg_tags = models.CharField(max_length=100, blank=True)
alg_status = models.CharField(max_length=1, blank=True)
...
class Meta:
db_table = u'algorithms'
class AlgXref(models.Model):
xref_alg = models.ForeignKey(Algorithms, related_name='algxref_alg' ,null=True, blank=True)
xref_parent = models.ForeignKey(Algorithms, related_name='algxref_parent', null=True, blank=True)
class Meta:
db_table = u'alg_xref'
On trying to query AlgXref I encounter this:
DatabaseError: ORA-00904: "ALG_XREF"."ID": invalid identifier
So the error seems to be that it looks for a primary key ID which isn't in the table.. I could create one but seems a bit pointless. Is there anyway to get around this? Or change my models?
EDIT: So after a bit of searching it seems that Django requires a model to have a primary key. Life is too short so have just added a primary key. Will this have any impact on performance?
This is currently a limitation of the ORM provided by Django. Each model has to have one field marked as primary_key=True, if there isn't one, the framework automatically creates an AutoField with name id.
However, this is being worked on as we speak as part of this year's Google Summer of Code and hopefully will be in Django by the end of this year. For now you can try to use the fork of Django available at https://github.com/koniiiik/django which contains an implementation (which is not yet complete but should be sufficient for your purposes).
As for whether there is any benefit or not, that depends. It certainly makes the database more reusable and causes less headaches if you just add an auto incrementing id column to each table. The performance impact shouldn't be too high, the only thing you might notice is that if you have a many-to-many table like this, containing only two ForeignKey columns, adding a third one will increase its size by one half. That should, however, be irrelevant as long as you don't store billions of rows in that table.