Does Django index Autofield / ID keys in PostgreSQL? - django

Django's docs state that id fields created with AutoField are indexed:
id is indexed by the database and is guaranteed to be unique.
Similarly it applies an index to every FK relationship.
However, in PostgreSQL whilst FKs appear to be indexed, IDs are not. Here's an example:
class TestModelBase(models.Model):
name = models.CharField(max_length=50)
fkfield = models.ForeignKey(TestModelFK, blank=True, null=True,
on_delete=models.CASCADE)
m2mfield = models.ManyToManyField(TestModelM2M, related_name='base_m2m')
This model appears to apply the fkfield index, but not the id autofield. From PGAdmin below:
Am I missing something?

PostgreSQL automatically creates indexes for primary keys. From the docs:
Adding a primary key will automatically create a unique B-tree index on the column or group of columns listed in the primary key, and will force the column(s) to be marked NOT NULL.
It appears that PGAdmin does not show those indexes. This mailing list thread is the best source I could find.

Related

unique=true doesn't create index

Email = models.EmailField(max_length = 300, unique = True)
Doesn't create an index on SQLite 3. Documentation says unique = True will create a index. But checking SQLite it is not created.
Documentation
Most database backends will automatically create an index if the field is marked as unique. Indeed, in the source code [GitHub] we see:
def _field_should_be_indexed(self, model, field):
return field.db_index and not field.unique
So if a field is marked as unique, by default it will not be indexed, since the database itself will already do that.
This is the same for foreign keys, for which some databases will also create an index automatically.

Foregin key field name is followed by _id

In a model, when a foreign key field is created then Django apparently create another field with the same field name followed by _id.
for example if I have
class Post(models.Model):
user = models.ForeignKey(User,on_delete=models.CASCADE,default=None)
dated = models.DateTimeField(auto_now=True)
...
Then I will have the following fields available:
id,user,user_id,dated
I am not sure why this field (user_id) was added?
Later I wanted to override my queryset in a class view
so I was confused which one to use (user field or user_id field)
:
def get_queryset(self):
queryset = super().get_queryset()
return queryset.filter(user_id=self.request.user.id)
Or
def get_queryset(self):
queryset = super().get_queryset()
return queryset.filter(user=self.request.user.id)
I tried both and both worked just fine
My question is:
1) What is the purpose of creating this additional field ?
2) What is the difference between the original foreign key field (user in my case) and user_id field?
3) Will both fields user and user_id available in the database? what is the point of that?
4) Is the content of user and user_id identical in each record? if so ,then what the purpose of this additional field that was created automatically by django?
Thanks a lot
Django only creates one column in the database for the foreign key.
The difference between the field and the _id attribute it generates is that accessing the field performs a query for the full set of columns from the related table in order to construct the complete related object. If you want the full object, use the field (and probably also use select_related() in the initial query to save you from doing N+1 queries).
On the other hand, if all you need is the DB-level value of the foreign key, which is usually the primary key of the related object (and often that is what you want), the _id attribute shortcut is there for you and already has the data, because that's what was actually in the foreign key column.
In other words, suppose I have models like this:
class ModelA(models.Model):
name = models.TextField()
class ModelB(models.Model):
name = models.TextField()
a_instance = models.ForeignKey(ModelA)
If you query for a ModelB, like ModelB.objects.get(pk=12), you'll get a query like this:
SELECT id, name, a_instance_id
FROM your_app.modelb
WHERE id = 12;
Notice a_instance_id is the name of the column -- it's just a foreign key, all it stores is a pointer to the primary key of a ModelA instance. If you just need that primary key, accessing the a_instance_id attribute has it already without needing to do another query. If you access the a_instance field, though, you get to do another query:
SELECT id, name
FROM your_app.modela
WHERE id = (whatever the value of that foreign key was);

Django Multi-Column Foreign Key

Is is possible to define foreign keys referencing multi columns in another model?
For example one foreign key references a two-column index in the product table, and the SQL statement:
FOREIGN KEY (product_category, product_id) REFERENCES product(category, id)
BTW I've looked into django.contrib.contenttypes and don't think that's the perfect solution for this kind of scenario.
It is not supported yet. There is a ticket and possible ways to handle it if you want to. maybe you could even run custom sql
Multi-Column Primary Key support
Relational database designs use a set of columns as the primary key for a table. When this set includes more than one column, it is known as a “composite” or “compound” primary key. (For more on the terminology, here is an ​article discussing database keys).
Currently Django models only support a single column in this set, denying many designs where the natural primary key of a table is multiple columns. Django currently can't work with these schemas; they must instead introduce a redundant single-column key (a “surrogate” key), forcing applications to make arbitrary and otherwise-unnecessary choices about which key to use for the table in any given instance.
This page discusses how to have Django support these composite primary keys. There are a lot of details to get right here, but done right, it would allow for more flexibility and potential simplicity in data modeling.
Current Status
Current state is that the issue is accepted/assigned and being worked on, and there is a partial implementation at ​http://github.com/dcramer/django-compositepks. The implementation allows having composite primary keys. However, support for composite keys is missing in ForeignKey and RelatedManager. As a consequence, it isn't possible to navigate relationships from models that have a composite primary key.
Discussions:
David Cramer's initial patch
The composite foreign key API design
Ticket
Note - SqlAlchemy allows this as described below and you can use SqlAlchemy to replace Django's ORM
Foreign keys may also be defined at the table level, using the ForeignKeyConstraint object. This object can describe a single- or multi-column foreign key. A multi-column foreign key is known as a composite foreign key, and almost always references a table that has a composite primary key. Below we define a table invoice which has a composite primary key:
invoice = Table('invoice', metadata,
Column('invoice_id', Integer, primary_key=True),
Column('ref_num', Integer, primary_key=True),
Column('description', String(60), nullable=False)
)
And then a table invoice_item with a composite foreign key referencing invoice:
invoice_item = Table('invoice_item', metadata,
Column('item_id', Integer, primary_key=True),
Column('item_name', String(60), nullable=False),
Column('invoice_id', Integer, nullable=False),
Column('ref_num', Integer, nullable=False),
ForeignKeyConstraint(['invoice_id', 'ref_num'], ['invoice.invoice_id', 'invoice.ref_num'])
)
Reference
Yes its possible but you will need to create a composite key when you use multiple column constraint i.e. foreign key or primary key.
For example:
CREATE TABLE Student (
S_num INTEGER,
S_Cate INTEGER,
S_descr CHAR(200),
PRIMARY KEY (S_num, S_Cate))
CREATE TABLE sub_Student (
Ssub_ID INTEGER PRIMARY KEY,
Sref_num INTEGER,
Sref_Cate INTEGER,
sub_descr CHAR(500),
FOREIGN KEY (Sref_num, Sref_Cate) REFERENCES Student
(S_num, S_Cate))
Anyway, you can to create a "Django fixture" like this:
CREATE INDEX product_category_id_id ON product (category_id, id);
To do this, you must to create a file named product.sql on subfolder sql where your model resides. The fixture is loaded on initial syncdb.
#pratik-mandrekar's answer is excellent, but I wanted to point out that even without proper multi-column primary keys; django is able to accommodate queries spanning multi-column foreign keys. Here's an example based on a legacy database who's schema I wasn't permitted to modify:
Given:
from django.db import models
class Account(models.Model):
# Collectively, location_no and occupant_no function as the primary key for Account.
location_no = models.IntegerField()
occupant_no = models.SmallIntegerField()
name = models.CharField(max_length=100)
class Meta:
managed = False
db_table = 'csracct'
unique_together = (('location_no', 'occupant_no'),)
class Call(models.Model):
call_id = models.IntegerField(primary_key=True)
# Collectively, location_no and occupant_no act as a foreign key to Account.
location_no = models.IntegerField()
occupant_no = models.SmallIntegerField()
notes = models.TextField()
class Meta:
managed = False
db_table = 'csrcall'
Here's how you'd use extra() to fetch the 10 most recent calls for accounts with the name 'steve':
calls = Call.extra(
tables = ['csracct'],
where = [
'csracct.location_no=csrcall.location_no',
'csracct.occupant_no=csrcall.occupant_no',
'csracct.name=%s',
],
params = ['steve'],
).order_by('-call_id')[:10]
It's not the most elegant solution, but extra() is part of django's base queryset toolkit; so it plays well with the rest of your django code. Notice how we order_by, and limit/slice the queryset using the usual django methods.

How should a many-to-many table be defined

I'm having some trouble understanding many-to-many fields in Django.
When I create a many-to-many field, ex:
class GlobalPart (Models.model):
...
category_id=models.ManyToManyField(Category, related_name = 'globalpart')
...
and
class Category (Model.model):
...
category = models.CharField(max_length=250)
...
I notice that it created a new table called appname_globalpart_category_id in addition to the appname_globalpart table for the GlobalPart model.
What I'm wondering is, how should the field types in that table be defined. I would think that
there should be at least one foreign key there to relate the fields. But instead there is the primary key for the table, and the other fields are integers (globalpart_id and category_id).
So my question is -- is that normal? Or did I somehow define the many-to-many field incorrectly? And my next question is how would I get all the category_ids associated to a particular GlobalPart?
(1) short answer: Yes this is normal.
Long answer: ManyToMany table will need a foreign key to both Category and GlobalPart tables. Strictly speaking those two foreign keys should be sufficient. The extra pk that you see in there is just for django magic. You can really get away with only those two foreign keys in that table if you manually define the many-to-many table yourself. However if you let django do it for you (by using ManyToManyField) you get this extra pk
(2) I suggest changing your model fields category_id to categories:
class GlobalPart (Models.model):
categories=models.ManyToManyField(Category, related_name = 'globalpart')
This is because, ManyToManyFields refers well to "many" items. This field does not refer to "one" category_id, it refers to all related categories. So when naming it would be natural to name it accordingly.
As for accessing all categories you can do it by accessing the "categories" property. Say if your object instance named global_part, you can access categories like this:
categories = global_part.categories.all()
Instead of all(), you can use filter() or exclude() the same way you use it when querying models.
Here is a link to related django docs
What do you think a foreign key is? It's a field containing values that equate to IDs - usually primary keys - in the "foreign" table. If the other table has integer keys, as most Django tables do, then the foreign key field will be of type integer as well.
Additionally, Django creates constraints so that the database will enforce that the IDs do actually reference valid values in the foreign table. Depending on your database, these might or might not be displayed as part of the field definition.

Django Two foreign key

I have two models: UserProfile (extended from user) and Cv. I created another model that have two foreign key that come from theses models.
class cv(models.Model):
user = models.ForeignKey(User, unique=True)
cv_d= models.TextField(max_length=1100)
...
class cvv(models.Model):
user = models.ForeignKey(User)
cv= models.ForeignKey(cv)
date = models.DateTimeField(auto_now=True)
In my view, I am trying to insert value on cvv:
...
obj = cv.objects.get(pk=id,active=True)
add=cvv(user=request.user, cv=obj)
add.save()
But, I am getting the following error:
(1452, 'Cannot add or update a child row: a foreign key constraint fails
How can I insert theses 2 foreign key on my model?
Welcome to one of the many reasons why you shouldn't use MySQL. This happens most often when you have one table that is MyISAM and one table that is InnoDB. Since myISAM doesn't support FK constraints all hell breaks loose when django creates a FK between the tables.
The fix is to either make both tables InnoDB or MyISAM and not to mix them. Or even better drop the bad RDMS for something not MySQL.