For starters, it is NOT a duplicate of Doctrine 2 JOIN ON error. I am indeed getting Expected end of string, got 'ON' but using WITH won't solve my case.
The problem there is similar but different from mine. I don't need to add a condition to my JOIN, I need to substitute the default condition with a different one.
Let's say, we have 2 hypthetical tables: album and artist, with an artist_id FK pointing from album to artist. In my case, I don't want to join artists with their albums. I want to list artists joined with unrelated albums using some arbitrary condition. So each artist will be joined with the exact same small set of albums. Believe me, in my case it does make sense - I don't want to describe it fully because it is too complex and out of the scope of my question.
SELECT * FROM artist
LEFT JOIN album ON album.some_unrelated_property = 'foo'
The example above is raw SQL (I'm using PostgreSQL) and works perfectly fine in this form.
In my code I'm using query builder (hard to avoid it, because my query is way more complex and built step by step by a series of functions). The line that causes an error is this:
$qb->leftJoin('artist.albums', 'aa', Join::ON, 'aa.someUnrelatedProperty = "foo"');
In Doctrine I'm getting the dreaded Expected end of string, got 'ON'. When I use WITH instead of ON it works, but as expected, it appends the standard join condition by artist_id which I do not want.
What's even more confusing for me, in this post: What is the difference between JOIN ON and JOIN WITH in Doctrine2? which explains a difference between ON and WITH in DQL, somebody uses an example equivalent to mine as a correct use of ON in DQL:
Case Two
DQL
FROM Album a LEFT JOIN a.Track t ON t.status = 1
Will translate in SQL
FROM Album a LEFT JOIN Track t ON t.status = 1
What am I missing here? Is it possible to achieve what I want at all, using DQL? If not, what the hell is the reason for ON to exist in DQL when there's also WITH which works in more standard cases?
Related
The Problem
I have an Article entity in Doctrine, which has a many-to-many relationship with Tag. That is, an Article can be "tagged" with multiple tags, and they are bound together by the articles_tags table in the database.
Let us assume we wanted to find all Articles that are associated with given Tag. For the case of an example, let us say we wanted to find all Articles associated with the "cars" tag: an article about cars.
DQL as Opposed to SQL
Had this been regular SQL (or some flavor of it), I would have written a query in a similar manner to the following:
SELECT * FROM articles_tags WHERE tag_id IN (
SELECT id FROM tags WHERE name = 'cars')
This would give us all article_tags where there is a Tag that goes by the name "cars". Of course, should more than one tags be used in the query at one time, duplicate articles should also be thrown out: perhaps by using a GROUP BY. Furthermore, you could even get rid of the intermediary step of first selecting the article_tags and then going to the Articles, by writing a longer query.
From my current understanding of Doctrine, which ranges no more than a few days, you cannot directly reference intermediary tables; nor does it seem as though you can write subqueries using the DQL. As such, I am at a loss.
Any pointers as to where I should start writing the query from, or any information regarding how one in general might go about handling these types of database retrievals in an ORM such as Doctrine would be highly helpful!
Query in DQL is a bit simpler than pure SQL:
$q = "
SELECT a FROM AppBundle:Article
LEFT JOIN a.tags t
WHERE t.name = :tag";
$result = $em->createQuery($q)
->setParameter('tag', $tag)
->getResult()
;
I'm just not understand the new references() function in Rails 4
I'm reading the definition here : http://apidock.com/rails/ActiveRecord/QueryMethods/references
It's meaby a english problem, but it's still not clear for me.
What is the goal of this new feature? (code was working well without it before).
Must I always add a reference for each table of my includes() ?
Thanks
When using includes, Rails will usually load the results separately. For example, if you do
Post.includes(:comments)
it will issue a query to load the post(s) and then another to load the comments with the post_id of the post(s).
It will generate a join, if, for example, you want to do
Post.includes(:comments).where(comments: { user_id: 44})
Where one of the conditions is dependent on a related table. If, however, you were to use
Post.includes(:comments).where('comments.user_id = ?', 44)
You will get an error in Rails 4. Any time that you're referencing a relationship in a conditional using a SQL snippet, you will have to use references. To fix the above, we would do
Post.includes(:comments).where('comments.user_id = ?', 44).references(:comments)
The other option, of course, is to not use SQL snippets in your conditionals, like the second code example, but it's not always possible to avoid them.
I think this is intended for when you're eager loading an association and you're using a string condition. Because Rails doesn't want to have to parse raw SQL in where clauses to figure out what you're doing, so references is intended to make it more explicit what is going on in your query.
Group.includes(:users).where('users.first_name = ?', 'John')
You should get some kind of deprecation warning if you do this. But if you use a hash syntax you should not get the deprecation warning:
Group.includes(:users).where(users: { first_name: 'John' })
So to fix the deprecation warning on the first one, you would add references(:users) to it.
Group.includes(:users).where('users.first_name = ?', 'John').references(:users)
The query method references is used to indicate that an association is referenced by a SQL string and therefore be joined over being loaded separately. As of Rails 4.1, adding a string condition of an included reference will result in an exception being raised.
Here is an example that selects all Teams which have a member named Nishant:
>> Team.includes(:members).where('members.name = ?', 'Nishant')
SQLite3::SQLException: no such column: members.name: SELECT "teams".*
FROM "teams" WHERE (members.name = 'Nishant')
ActiveRecord::StatementInvalid: SQLite3::SQLException: no such column:
members.name: SELECT "teams".* FROM "teams" WHERE (members.name =
'Nishant')
...
To get the above example to work in Rails 4.1, we must include query method references with the name of the association to join
Team.includes(:members).where("members.name = ?", 'Nishant').references(:members)
However, if you were using the hash syntax with association conditions, it would still perform a LEFT OUTER JOIN without any exception being raised:
Team.includes(:members).where(members: { name: 'Nishant' })
Note that ordering string SQL snippets on included associations will still work the same way without the need
of references:
Team.includes(:members).order('members.name')
I hope this will help you.
I have an optional foreign key for a parameter:
class Club(models.Model):
...
locationid = models.ForeignKey(location_models.Location, null=True)
...
I want to find entries of club where this foreignkey is not set. Here's the ORM query:
print Club.objects.filter(locationid=None).only('name').query
Produces
SELECT `club_club`.`id`, `club_club`.`name` FROM `club_club`
LEFT OUTER JOIN `location_location`
ON (`club_club`.`locationid_id` = `location_location`.`id`)
WHERE `location_location`.`id` IS NULL
Same query is produced when I do filter(locationid_id__isnull=True)
What I want is to query on locationid_id without involving a JOIN. I know I can write raw SQL, but is there an ORM-al way of doing this?
This seems to be quite a persistent issue, and the patch that fixed it had other side effects, so it never got applied to a release version of Django.
A solution to this is to use the extra method. This will require raw SQL, but only a limited amount and using SQL standards, so it should be compatible with all SQL databases:
location_null = '`%s`.`%s` IS NULL' % (Club._meta.db_table, Club.locationid.field.column)
Club.objects.extra(where=[location_null])
You can add this as a manager/queryset method for a more DRY solution.
The other option is to just take the performance hit. This is what I would recommend, unless benchmarking shows that the performance hit really is unacceptable in your specific case.
If we chain a call to filter() after a call to distinct(), the filter is applied to the query before the distinct. How do I filter the results of a query after applying distinct?
Example.objects.order_by('a','foreignkey__b').distinct('a').filter(foreignkey__b='something')
The where clause in the SQL resulting from filter() means the filter is applied to the query before the distinct. I want to filter the queryset resulting from the distinct.
This is probably pretty easy, but I just can't quite figure it out and I can't find anything on it.
Edit 1:
I need to do this in the ORM...
SELECT z.column1, z.column2, z.column3
FROM (
SELECT DISTINCT ON (b.column1, b.column2) b.column1, b.column2, c.column3
FROM table1 a
INNER JOIN table2 b ON ( a.id = b.id )
INNER JOIN table3 c ON ( b.id = c.id)
ORDER BY b.column1 ASC, b.column2 ASC, c.column4 DESC
) z
WHERE z.column3 = 'Something';
(I am using Postgres by the way.)
So I guess what I am asking is "How do you nest subqueries in the ORM? Is it possible?" I will check the documentation.
Sorry if I was not specific earlier. It wasn't clear in my head.
This is an old question, but when using Postgres you can do the following to force nested queries on your 'Distinct' rows:
foo = Example.objects.order_by('a','foreign_key__timefield').distinct('a')
bar = Example.objects.filter(pk__in=foo).filter(some_field=condition)
bar is the nested query as requested in OP without resorting to raw/extra etc. Tested working in 1.10 but docs suggest it should work back to at least 1.7.
My use case was to filter up a reverse relationship. If Example has some ForeignKey to model Toast then you can do:
Toast.objects.filter(pk__in=bar.values_list('foreign_key',flat=true))
This gives you all instances of Toast where the most recent associated example meets your filter criteria.
Big health warning about performance though, using this if bar is likely to be a huge queryset you're probably going to have a bad time.
Thanks a ton for the help guys. I tried both suggestions and could not bend either of those suggestions to work, but I think it started me in the right direction.
I ended up using
from django.db.models import Max, F
Example.objects.annotate(latest=Max('foreignkey__timefield')).filter(foreignkey__timefield=F('latest'), foreign__a='Something')
This checks what the latest foreignkey__timefield is for each Example, and if it is the latest one and a=something then keep it. If it is not the latest or a!=something for each Example then it is filtered out.
This does not nest subqueries but it gives me the output I am looking for - and it is fairly simple. If there is simpler way I would really like to know.
No you can't do this in one simple SELECT.
As you said in comments, in Django ORM filter is mapped to SQL clause WHERE, and distinct mapped to DISTINCT. And in a SQL, DISTINCT always happens after WHERE by operating on the result set, see SQLite doc for example.
But you could write sub-query to nest SELECTs, this depends on the actual target (I don't know exactly what's yours now..could you elaborate it more?)
Also, for your query, distinct('a') only keeps the first occurrence of Example having the same a, is that what you want?
My DQL query returns only the FROM object, which is nice if the other object were related, but it isn't.
My Query:
$query = $this->em->createQuery('SELECT c, s FROM MyBundle:Person c, MyBundle:Spot s
JOIN s.geo_data g JOIN g.features f WHERE f.active = true AND
ST_Distance(f.location, c.location) < :distance GROUP BY c, s');
This works perfectly in SQL, giving me all the spots and all the persons within :distance of them. But in DQL, it only returns the person object, and since on the database level they are not related, I have no way to fetch the correct spot.
My database setup is correct, I'm using a PostGIS backend and spots and persons are not related in any way. They just happen to be on the same map and I'm querying for spatial relationships.
According to documentation, it's intended behaviour, from what I read, s is being hydrated, but not returned anywhere at all, good job!
How can I teach DQL to please return me what I told it in SELECT? Where's the "I mean what I say, stop being a smartass" switch?
Doctrine cannot give you both entities if they are not related because if the were related you would get the first entity c where you could get s through the relation.
What you can try is selecting all fields of both entities like
SELECT c.location, ..., s.geo_data, ...
This will give you an array for each column that contains all fields from both entities.
Maybe you can use result set mapping to get the entities if desired.
If you want to stuck with Doctrine, you HAVE TO define a OneToMany relation between places and people. In this way, you could set up the PeopleRepository and set up a method like getPeopleByLocationAndMaxDistance(Location $location, $distance)
SELECT p
FROM People AS p
LEFT JOIN Places AS pl
WHERE ST_Distance(p.location, pl.location) < :distance