I was trying to find the answer in Django Documentation, but failed to do so.
Can anyone please explain how does Django "match" the objects of the related models?
eg. I have two models, and I am showing Django that they are related:
class Reporter(models.Model):
# ...
pass
class Article(models.Model):
reporter = models.ForeignKey(Reporter, related_name='report')
Then the magic happens and Django matches the two models, and adds _id field.
My question is:
How does Django know which objects of those two models are related?
Is it checking each and every field of those objects and sees if there is a match?
EDIT:
How does Django determine that a particular Reporter object is related to a particular Article object?
I understand that when it finds a match it adds the _id field, what I do not understand is based on what django "matches" two objects from different models.
To be more specific:
Let's say that there are two Reporter objects - r1 and r2.
There is one object in Article class - a1
How does django know that a1 is related to r1 and not to r2?
Thanks for your help!
It looks like you're not really SQL-savy, because there's really no "magic" involved and it's all basic relational model design.
Your above models translates to the canonical one to many SQL schema:
CREATE TABLE yourappname_reporter (
id int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY(id)
);
CREATE TABLE yourappname_article(
id int(11) NOT NULL AUTO_INCREMENT,
reporter_id int(11) NOT NULL,
PRIMARY KEY(id),
CONSTRAINT `reporter_id_refs_id_XXXX` FOREIGN KEY (`reporter_id`) REFERENCES `yourappname_reporter` (`id`)
);
As you can see, Django adds an 'id' primary key to your models (since you didn't explicitely defined one yourself), and the Article.reporter field translates to the reporter_id foreign key which references reporter.id. Here again Django uses the reporter.id primary key as foreign key reference as a (very sensible) default since you didn't explicitely told him to target another field.
Now when you create a new Article, you have to provide a Reporter instance, ie (assuming you have a reporter with id 1):
reporter = Reporter.objects.get(id=1)
article = Article.objects.create(reporter=reporter)
Then the ORM will issue the SQL query insert into yourappname_article (reporter_id) values (1), hence relating this new article row with this reporter row, and you can now get all articles from reporter 1 with select * from yourappname_article where reporter_id=1 (which is the query Django ORM will issue for Article.objects.filter(reporter_id=1))
To make a long story short: Django ORM is only a thin wrapper over your database, and you do have to know what a relational database is, how it works and how to properly use it (including how to properly design your db schema) if you expect to do anything good. If you don't even know what a foreign key is then by all means first learn about the relational model, proper relational design and basic SQL stuff.
Related
I'm trying to add unique columns on a pivot table created via a ManyToMany association.
I found this page of the documentation explaining how to generate a database unique constraint on some columns with this example:
/**
* #Entity
* #Table(name="ecommerce_products",uniqueConstraints={#UniqueConstraint(name="search_idx", columns={"name", "email"})})
*/
class ECommerceProduct
{
}
But this only works if I create the pivot table via a third entity and, in my case, I created the pivot table using a ManyToMany relation (in the same fashion as this code).
Is there a way to add unique columns on pivot table while still using ManyToMany or do I need to rely on a third entity?
While #Table annotation proposes a uniqueConstraints option, #JoinTable does not. Thus, if you want to add a unique constraint on your association table, you will have to actually create another entity explicitly.
That being said, the default join table should not need anything more than the default configuration set up by Doctrine. Currently, when adding a ManyToMany association, the join table is composed of two fields and a composite primary key relying on both fields is created.
If your association table only contains the two basic fields referring to both sides of your association (which is necessarily the case if you use #ManyToMany), the composite primary key should be all you need.
Here is the generated SQL for the basic example where a User has a ManyToMany association with Group (from this section of the documentation):
CREATE TABLE users_groups (
user_id INT NOT NULL,
group_id INT NOT NULL,
PRIMARY KEY(user_id, group_id)
) ENGINE = InnoDB;
ALTER TABLE users_groups ADD FOREIGN KEY (user_id) REFERENCES User(id);
ALTER TABLE users_groups ADD FOREIGN KEY (group_id) REFERENCES Group(id);
As you can see, everything is properly set up with a composite primary key which will ensure that there can't be duplicate entries for the couple (user_id, group_id).
Of course there is another alternative, Alan!
If you need a Zero to Zero relationship, the only alternative is defining the unique constraint per each pk in the agregated table, to make doctrine figuring out about zero to zero relationship.
The problem is that Doctrine's people hadn't considered zero to zero relationships, so the only alternative for this is manytomany relationship with one unique constraint per pk.
If you have doubts about final-state of your doctrine implementation of your E-R model, I strongly recommend mysql-workbench-schema-exporter. With this php tool, you can easily export your mysql workbench E-R schema to a Doctrine's working classes schema, so you would be able to easily explore all your alternatives ;-)
Hope this helps
**
Difference between creating a foreign key for consistency and for joins
**
I am fine to use Foreignkey and Queryset API with Django.
I just want to understand little bit more deeply how it works behind the scenes.
In Django manual, it says
a database index is automatically created on the ForeignKey. You can
disable this by setting db_index to False. You may want to avoid the
overhead of an index if you are creating a foreign key for consistency
rather than joins, or if you will be creating an alternative index
like a partial of multiple column index.
creating for a foreign key for consistency rather than joins
this part is confusing me.
I expected that you use Join keyword if you do query with Foreign key like below.
SELECT
*
FROM
vehicles
INNER JOIN users ON vehicles.car_owner = users.user_id
For example,
class Place(models.Model):
name = models.Charfield(max_length=50)
address = models.Charfield(max_length=50)
class Comment(models.Model):
place = models.ForeignKeyField(Place)
content = models.Charfield(max_length=50)
if you use queryset like Comment.objects.filter(place=1), i expected using Join Keyword in low level SQL command.
but, when I checked it by printing out queryset.query in console, it showed like below.
(I simplified with Model just to explains. below, it shows all attributes in my model. you can ignore attributes)
SELECT
"bfm_comment"."id", "bfm_comment"."content", "bfm_comment"."user_id", "bfm_comment"."place_id", "bfm_comment"."created_at"
FROM "bfm_comment" WHERE "bfm_comment"."place_id" = 1
creating a foreign key for consistency vs creating a foreign key for joins
simply, I thought if you use any queryset, it means using foreign key for joins. Because you can get parent's table data by c = Comment.objects.get(id=1) c.place.name easily. I thought it joins two tables behind scenes. But result of Print(queryset.query) didn't how Join Keyword but Find it by Where keyword.
The way I understood from an answer
Case 1:
Comment.objects.filter(place=1)
result
SELECT
"bfm_comment"."id", "bfm_comment"."content", "bfm_comment"."user_id", "bfm_comment"."place_id", "bfm_comment"."created_at"
FROM "bfm_comment"
WHERE "bfm_comment"."id" = 1
Case 2:
Comment.objects.filter(place__name="df")
result
SELECT "bfm_comment"."id", "bfm_comment"."content", "bfm_comment"."user_id", "bfm_comment"."place_id", "bfm_comment"."created_at"
FROM "bfm_comment" INNER JOIN "bfm_place" ON ("bfm_comment"."place_id" = "bfm_place"."id")
WHERE "bfm_place"."name" = df
Case1 is searching rows which has comment.id column is 1 in just Comment table.
But in Case 2, it needs to know Place table's attribute 'name', so It has to use JOIN keyword to check values in column of Place table. Right?
So Is it alright to think that I create a foreign key for joins if i use queryset like Case2 and that it is better to create index on the Foreign Key?
for above question, I think I can take the answer from Django Manual
Consider adding indexes to fields that you frequently query using
filter(), exclude(), order_by(), etc. as indexes may help to speed up
lookups. Note that determining the best indexes is a complex
database-dependent topic that will depend on your particular
application. The overhead of maintaining an index may outweigh any
gains in query speed
In conclusion, it really depends on how my application work with it.
If you execute the following command the mystery will be revealed
./manage.py sqlmigrate myapp 0001
Take care to replace myapp with your app name (bfm I think) and 0001 with the actual migration where the Comment model is created.
The generated sql will reveal that the actual table is created with place_id int rather than a place Place that is because the RDBMS doesn't know anything about models, the models are only in the application level. It's the job of the django orm to fetch the data from the RDBMS and convert them into model instances. That's why you always get a place member in each of your Comment instances and that place member gives you access to the members of the related Place instance in turn.
So what happens when you do?
Comment.objects.filter(place=1)
Django is smart enough to know that you are referring to a place_id because 1 is obviously not an instance of a Place. But if you used a Place instance the result would be the same. So there is no join here. The above query would definitely benefit from having an index on the place_id, but it wouldn't benefit from having a foreign key constraint!! Only the Comment table is queried.
If you want a join, try this:
Comment.objects.filter(place__name='my home')
Queries of this nature with the __ often result in joins, but sometimes it results in a sub query.
Querysets are lazy.
https://docs.djangoproject.com/en/1.10/topics/db/queries/#querysets-are-lazy
QuerySets are lazy – the act of creating a QuerySet doesn’t involve
any database activity. You can stack filters together all day long,
and Django won’t actually run the query until the QuerySet is
evaluated. Take a look at this example:
I have 3 tables say, TextObj, User, SecurityCheck. The third table has a Foreign Key attribute (textobj) referencing TextObj and there is a many-to-many field (sharedWith) from SecurityCheck to User.
class SecurityCheck(models.Model):
textobj=models.ForeignKey(TextObj)
owner=models.CharField(max_length=255)
sharedWith=models.ManyToManyField(User)
def __init__(self,owner,filename,requestingUsername):
self.owner=owner
self.textobj=TextObj.filter(filename=filename)
self.sharedWith.add(User.objects.filter(username=requestingUsername))
I need to do a query which fetches all the instances of Textobj which have a particular user in the sharedWith field and a particular filename(which is an attribute of TextObj)
You can easily do queries that span (reverse) relationship:
TextObj.objects.filter(securitycheck__sharedWith=user, filename="foo")
https://docs.djangoproject.com/en/dev/topics/db/queries/#lookups-that-span-relationships
Django offers a powerful and intuitive way to “follow” relationships in lookups, taking care of the SQL JOINs for you automatically, behind the scenes. To span a relationship, just use the field name of related fields across models, separated by double underscores, until you get to the field you want.
It works backwards, too. To refer to a “reverse” relationship, just use the lowercase name of the model.
I've got 2 existing models that I need to join that are non-relational (no foreign keys). These were written by other developers are cannot be modified by me.
Here's a quick description of them:
Model Process
Field filename
Field path
Field somethingelse
Field bar
Model Service
Field filename
Field path
Field servicename
Field foo
I need to join all instances of these two models on the filename and path columns. I've got existing filters I have to apply to each of them before this join occurs.
Example:
A = Process.objects.filter(somethingelse=231)
B = Service.objects.filter(foo='abc')
result = A.filter(filename=B.filename,path=B.path)
This sucks, but your best bet is to iterate all models of one type, and issue queries to get your joined models for the other type.
The other alternative is to run a raw SQL query to perform these joins, and retrieve the IDs for each model object, and then retrieve each joined pair based on that. More efficient at run time, but it will need to be manually maintained if your schema evolves.
Suppose you have a model Entry, with a field "author" pointing to another model Author. Suppose this field can be null.
If I run the following QuerySet:
Entry.objects.filter(author=X)
Where X is some value. Suppose in MySQL I have setup a compound index on Entry for some other column and author_id, ideally I'd like the SQL to just use "author_id" on the Entry model, so that it can use the compound index.
It turns out that Entry.objects.filter(author=5) would work, no join is done. But, if I say author=None, Django does a join with Author, then add to the Where clause Author.id IS NULL. So in this case, it can't use the compound index.
Is there a way to tell Django to just check the pk, and not follow the link?
The only way I know is to add an additional .extra(where=['author_id IS NULL']) to the QuerySet, but I was hoping some magic in .filter() would work.
Thanks.
(Sorry I was not clearer earlier about this, and thanks for the answers from lazerscience and Josh).
Does this not work as expected?
Entry.objects.filter(author=X.id)
You can either use a model or the model id in a foreign key filter. I can't check right yet if this executes a separate query, though I'd really hope it wouldn't.
If do as you described and do not use select_related() Django will not perform any join at all - no matter if you filter for the primary key of the related object or the related itself (which doesn't make any difference).
You can try:
print Entry.objects.(author=X).query
Assuming that the foreign key to Author has the name author_id, (if you didn't specify the name of the foreign key column for ForeignKey field, it should be NAME_id, if you specified the name, then check the model definition / your database schema),
Entry.objects.filter(author_id=value)
should work.
Second Attempt:
http://docs.djangoproject.com/en/dev/ref/models/querysets/#isnull
Maybe you can have a separate query, depending on whether X is null or not by having author__isnull?
Pretty late, but I just ran into this. I'm using Q objects to build up the query, so in my case this worked fine:
~Q(author_id__gt=0)
This generates sql like
NOT ("author_id" > 0 AND "author_id" IS NOT NULL)
You could probably solve the problem in this question by using
Entry.objects.exclude(author_id__gt=0)