I have a trouble with creating JPA 2 query (actually I'm using Spring Data JPA). I have a 2 tables:
- Users (id,company_id,name,surname,status)
- UsersGroup (id,user_id,group_id).
Users can(and often are) connected to more than one group.
What I'm trying to do is to get distinct list of users according to filtering options from search form (name,surname,status) and attributes taken from logged user (company_id, group_id), because I want to show this user only users within company and group he also belongs to.
I'm using Specification/Predicate/Criteria from JPA 2 and Spring Data to do this, and I wrote this code:
public static Specification<User> filteredList(final ListFilterBean searchFilter, final LoggedUserBean loggedUser) {
return new Specification<User>() {
#Override
public Predicate toPredicate(Root<User> root,
CriteriaQuery<?> query, CriteriaBuilder cb) {
List<Predicate> predicates = new ArrayList<Predicate>();
root.join("userGroups");
predicates.add(cb.equal(root.<Long>get(User_.companyId), loggedUser.getCompanyId()));
predicates.add(cb.like(cb.upper(root.<String>get(User_.name)), getLikePattern(searchFilter.getName())));
predicates.add(cb.like(cb.upper(root.<String>get(User_.surname)), getLikePattern(searchFilter.getSurname())));
predicates.add(cb.equal(cb.upper(root.<String>get(User_.status)), searchFilter.getStatusId().charAt(0)));
query.distinct(true);
return cb.and(predicates.toArray(new Predicate[0]));
}
};
}
Which produces SQL below:
select
*
from
( select
distinct user0_.id as id5_,
user0_.name as name5_,
user0_.surname as surname5_
from
users user0_
inner join
user_group usergroup1_
on user0_.id=usergroup1_.user_id
where
user0_.company_id=1
and (
user0_.id in (select distinct user_id from user_group where user_group.group_id in (1, 2, 3))
)
and (
upper(user0_.name) like '%'
)
and (
upper(user0_.surname) like '%'
)
and upper(user0_.status)='A' )
where
rownum <= 15
Bolded part of code is the part doesn't actually exist and this is my main question. How to write expression (Predicate) to incorporate this part of SQL into query?
Thanks in advance :)
Related
I have some One-To-Many, unidirectional with Join Table relationships in a Symfony App which I need to query and I can't figure out how to do that in DQL or Query Builder.
The Like entity doesn't have a comments property itself because it can be owned by a lot of different types of entities.
Basically I would need to translate something like this:
SELECT likes
FROM AppBundle:Standard\Like likes
INNER JOIN comment_like ON comment_like.like_id = likes.id
INNER JOIN comments ON comment_like.comment_id = comments.id
WHERE likes.created_by = :user_id
AND likes.active = 1
AND comments.id = :comment_id
I've already tried this but the join output is incorrect, it selects any active Like regardless of its association with the given comment
$this->createQueryBuilder('l')
->select('l')
->innerJoin('AppBundle:Standard\Comment', 'c')
->where('l.owner = :user')
->andWhere('c = :comment')
->andWhere('l.active = 1')
->setParameter('user', $user)
->setParameter('comment', $comment)
I see 2 options to resolve this:
Make relation bi-directional
Use SQL (native query) + ResultSetMapping.
For the last option, here is example of repository method (just checked that it works):
public function getLikes(Comment $comment, $user)
{
$sql = '
SELECT l.id, l.active, l.owner
FROM `like` l
INNER JOIN comment_like ON l.id = comment_like.like_id
WHERE comment_like.comment_id = :comment_id
AND l.active = 1
AND l.owner = :user_id
';
$rsm = new \Doctrine\ORM\Query\ResultSetMappingBuilder($this->_em);
$rsm->addRootEntityFromClassMetadata(Like::class, 'l');
return $this->_em->createNativeQuery($sql, $rsm)
->setParameter('comment_id', $comment->getId())
->setParameter('user_id', $user)
->getResult();
}
PS: In case of Mysql, 'like' is reserved word. So, if one wants to have table with name 'like' - just surround name with backticks on definition:
* #ORM\Table(name="`like`")
I find the Symfony documentation very poor about unidirectional queries.
Anyway I got it working by using DQL and sub-select on the owning entity, which is certainly not as fast. Any suggestion on how to improve that is more than welcomed!
$em = $this->getEntityManager();
$query = $em->createQuery('
SELECT l
FROM AppBundle:Standard\Like l
WHERE l.id IN (
SELECT l2.id
FROM AppBundle:Standard\Comment c
JOIN c.likes l2
WHERE c = :comment
AND l2.owner = :user
AND l2.active = 1
)'
)
->setParameter('user', $user)
->setParameter('comment', $comment)
;
I want to get a list of the latest purchase of each customer, sorted by the date.
The following query does what I want except for the date:
(Purchase.objects
.all()
.distinct('customer')
.order_by('customer', '-date'))
It produces a query like:
SELECT DISTINCT ON
"shop_purchase.customer_id"
"shop_purchase.id"
"shop_purchase.date"
FROM "shop_purchase"
ORDER BY "shop_purchase.customer_id" ASC,
"shop_purchase.date" DESC;
I am forced to use customer_id as the first ORDER BY expression because of DISTINCT ON.
I want to sort by the date, so what the query I really need should look like this:
SELECT * FROM (
SELECT DISTINCT ON
"shop_purchase.customer_id"
"shop_purchase.id"
"shop_purchase.date"
FROM "shop_purchase"
ORDER BY "shop_purchase.customer_id" ASC,
"shop_purchase.date" DESC;
)
AS result
ORDER BY date DESC;
I don't want to sort using python because I still got to page limit the query. There can be tens of thousands of rows in the database.
In fact it is currently sorted by in python now and is causing very long page load times, so that's why I'm trying to fix this.
Basically I want something like this https://stackoverflow.com/a/9796104/242969. Is it possible to express it with django querysets instead of writing raw SQL?
The actual models and methods are several pages long, but here is the set of models required for the queryset above.
class Customer(models.Model):
user = models.OneToOneField(User)
class Purchase(models.Model):
customer = models.ForeignKey(Customer)
date = models.DateField(auto_now_add=True)
item = models.CharField(max_length=255)
If I have data like:
Customer A -
Purchase(item=Chair, date=January),
Purchase(item=Table, date=February)
Customer B -
Purchase(item=Speakers, date=January),
Purchase(item=Monitor, date=May)
Customer C -
Purchase(item=Laptop, date=March),
Purchase(item=Printer, date=April)
I want to be able to extract the following:
Purchase(item=Monitor, date=May)
Purchase(item=Printer, date=April)
Purchase(item=Table, date=February)
There is at most one purchase in the list per customer. The purchase is each customer's latest. It is sorted by latest date.
This query will be able to extract that:
SELECT * FROM (
SELECT DISTINCT ON
"shop_purchase.customer_id"
"shop_purchase.id"
"shop_purchase.date"
FROM "shop_purchase"
ORDER BY "shop_purchase.customer_id" ASC,
"shop_purchase.date" DESC;
)
AS result
ORDER BY date DESC;
I'm trying to find a way not to have to use raw SQL to achieve this result.
This may not be exactly what you're looking for, but it might get you closer. Take a look at Django's annotate.
Here is an example of something that may help:
from django.db.models import Max
Customer.objects.all().annotate(most_recent_purchase=Max('purchase__date'))
This will give you a list of your customer models each one of which will have a new attribute called "most_recent_purchase" and will contain the date on which they made their last purchase. The sql produced looks like this:
SELECT "demo_customer"."id",
"demo_customer"."user_id",
MAX("demo_purchase"."date") AS "most_recent_purchase"
FROM "demo_customer"
LEFT OUTER JOIN "demo_purchase" ON ("demo_customer"."id" = "demo_purchase"."customer_id")
GROUP BY "demo_customer"."id",
"demo_customer"."user_id"
Another option, would be adding a property to your customer model that would look something like this:
#property
def latest_purchase(self):
return self.purchase_set.order_by('-date')[0]
You would obviously need to handle the case where there aren't any purchases in this property, and this would potentially not perform very well (since you would be running one query for each customer to get their latest purchase).
I've used both of these techniques in the past and they've both worked fine in different situations. I hope this helps. Best of luck!
Whenever there is a difficult query to write using Django ORM, I first try the query in psql(or whatever client you use). The SQL that you want is not this:
SELECT * FROM (
SELECT DISTINCT ON
"shop_purchase.customer_id" "shop_purchase.id" "shop_purchase.date"
FROM "shop_purchase"
ORDER BY "shop_purchase.customer_id" ASC, "shop_purchase.date" DESC;
) AS result
ORDER BY date DESC;
In the above SQL, the inner SQL is looking for distinct on a combination of (customer_id, id, and date) and since id will be unique for all, you will get all records from the table. I am assuming id is the primary key as per convention.
If you need to find the last purchase of every customer, you need to do something like:
SELECT "shop_purchase.customer_id", max("shop_purchase.date")
FROM shop_purchase
GROUP BY 1
But the problem with the above query is that it will give you only the customer name and date. Using that will not help you in finding the records when you use these results in a subquery.
To use IN you need a list of unique parameters to identify a record, e.g., id
If in your records id is a serial key, then you can leverage the fact that the latest date will be the maximum id as well. So your SQL becomes:
SELECT max("shop_purchase.id")
FROM shop_purchase
GROUP BY "shop_purchase.customer_id";
Note that I kept only one field (id) in the selected clause to use it in a subquery using IN.
The complete SQL will now be:
SELECT *
FROM shop_customer
WHERE "shop_customer.id" IN
(SELECT max("shop_purchase.id")
FROM shop_purchase
GROUP BY "shop_purchase.customer_id");
and using the Django ORM it looks like:
(Purchase.objects.filter(
id__in=Purchase.objects
.values('customer_id')
.annotate(latest=Max('id'))
.values_list('latest', flat=True)))
Hope it helps!
I have a similar situation and this is how I'm planning to go about it:
query = Purchase.objects.distinct('customer').order_by('customer').query
query = 'SELECT * FROM ({}) AS result ORDER BY sent DESC'.format(query)
return Purchase.objects.raw(query)
Upside it gives me the query I want. Downside is that it is raw query and I can't append any other queryset filters.
This is my approach if I need some subset of data (N items) along with the Django query. This is example using PostgreSQL and handy json_build_object() function (Postgres 9.4+), but same way you can use other aggregate function in other database system. For older PostgreSQL versions you can use combination of array_agg() and array_to_string() functions.
Imagine you have Article and Comment models and along with every article in the list you want to select 3 recent comments (change LIMIT 3 to adjust size of subset or ORDER BY c.id DESC to change sorting of subset).
qs = Article.objects.all()
qs = qs.extra(select = {
'recent_comments': """
SELECT
json_build_object('comments',
array_agg(
json_build_object('id', id, 'user_id', user_id, 'body', body)
)
)
FROM (
SELECT
c.id,
c.user_id,
c.body
FROM app_comment c
WHERE c.article_id = app_article.id
ORDER BY c.id DESC
LIMIT 3
) sub
"""
})
for article in qs:
print(article.recent_comments)
# Output:
# {u'comments': [{u'user_id': 1, u'id': 3, u'body': u'foo'}, {u'user_id': 1, u'id': 2, u'body': u'bar'}, {u'user_id': 1, u'id': 1, u'body': u'joe'}]}
# ....
I want to use DQL to create a query which looks like this in SQL:
select
e.*
from
e
inner join (
select
uuid, max(locale) as locale
from
e
where
locale = 'nl_NL' or
locale = 'nl'
group by
uuid
) as e_ on e.uuid = e_.uuid and e.locale = e_.locale
I tried to use QueryBuilder to generate the query and subquery. I think they do the right thing by them selves but I can't combine them in the join statement. Does anybody now if this is possible with DQL? I can't use native SQL because I want to return real objects and I don't know for which object this query is run (I only know the base class which have the uuid and locale property).
$subQueryBuilder = $this->_em->createQueryBuilder();
$subQueryBuilder
->addSelect('e.uuid, max(e.locale) as locale')
->from($this->_entityName, 'e')
->where($subQueryBuilder->expr()->in('e.locale', $localeCriteria))
->groupBy('e.uuid');
$queryBuilder = $this->_em->createQueryBuilder();
$queryBuilder
->addSelect('e')
->from($this->_entityName, 'e')
->join('('.$subQueryBuilder.') as', 'e_')
->where('e.uuid = e_.uuid')
->andWhere('e.locale = e_.locale');
You cannot put a subquery in the FROM clause of your DQL.
I will assume that your PK is {uuid, locale}, as of discussion with you on IRC. Since you also have two different columns in your query, this can become ugly.
What you can do is putting it into the WHERE clause:
select
e
from
MyEntity e
WHERE
e.uuid IN (
select
e2.uuid
from
MyEntity e2
where
e2.locale IN (:selectedLocales)
group by
e2.uuid
)
AND e.locale IN (
select
max(e3.locale) as locale
from
MyEntity e3
where
e3.locale IN (:selectedLocales)
group by
e3.uuid
)
Please note that I used a comparison against a (non empty) array of locales that you bind to to the :selectedLocales. This is to avoid destroying the query cache if you want to match against additional locales.
I also wouldn't suggest building this with the query builder if there's no real advantage in doing so since it will just make it simpler to break the query cache if you add conditionals dynamically (also, it's 3 query builders involved!)
I have several database tables with 2 primary keys, id and date. I do not update the records but instead insert a new record with the updated information. This new record has the same id and the date field is NOW(). I will use a product table to explain my question.
I want to be able to request the product details at a specific date. I therefore use the following subquery in DQL, which works fine:
WHERE p.date = (
SELECT MAX(pp.date)
FROM Entity\Product pp
WHERE pp.id = p.id
AND pp.date < :date
)
This product table has some referenced tables, like category. This category table has the same id and date primary key combination. I want to be able to request the product details and the category details at a specific date. I therefore expanded the DQL as shown above to the following, which also works fine:
JOIN p.category c
WHERE p.date = (
SELECT MAX(pp.date)
FROM Entity\Product pp
WHERE pp.id = p.id
AND pp.date < :date
)
AND c.date = (
SELECT MAX(cc.date)
FROM Entity\ProductCategory cc
WHERE cc.id = c.id
AND cc.date < :date
)
However, as you can see, if I have multiple referenced tables I will have to copy the same piece of DQL. I want to somehow add these subqueries to the entities so that every time an entity is called it adds this subquery.
I have thought of adding this in a __construct($date) or some kind of setUp($date) method, but I'm kind of stuck here. Also, would it help to add #Id to Entity\Product::date?
I hope someone can help me. I do not expect a complete solution, one step in a good direction would be very much appreciated.
I think I've found my solution. The trick was (first, to update to Doctrine 2.2 and) using a filter:
namespace Filter;
use Doctrine\ORM\Mapping\ClassMetaData,
Doctrine\ORM\Query\Filter\SQLFilter;
class VersionFilter extends SQLFilter {
public function addFilterConstraint(ClassMetadata $targetEntity, $targetTableAlias) {
$return = $targetTableAlias . '.date = (
SELECT MAX(sub.date)
FROM ' . $targetEntity->table['name'] . ' sub
WHERE sub.id = ' . $targetTableAlias . '.id
AND sub.date < ' . $this->getParameter('date') . '
)';
return $return;
}
}
Add the filter to the configuration:
$configuration->addFilter("version", Filter\VersionFilter");
And enable it in my repository:
$this->_em->getFilters()->enable("version")->setParameter('date', $date);
I have an SQL query that I have written using CTE. Now, I am moving the repository to use Entity Framework 5.
I am at a loss as to how to integrate (or rewrite) the CTE-based query using Entity Framework 5.
I am using POCO entities with the EF5 and have a bunch of Map classes. There is no EDMX file etc.
I feel like a total noob right now and would appreciate any help pointing me in the right direction.
The CTE query is as following
WITH CDE AS
(
SELECT * FROM collaboration.Workspace AS W WHERE W.Id = #WorkspaceId
UNION ALL
SELECT W.* FROM collaboration.Workspace AS W INNER JOIN CDE ON W.ParentId = CDE.Id AND W.ParentId <> '00000000-0000-0000-0000-000000000000'
)
SELECT
W.Id AS Id,
W.Name AS Name,
W.Description AS Description,
MAX(WH.ActionedTimeUtc) AS LastUpdatedTimeUtc,
WH.ActorId AS LastUpdateUserId
FROM
collaboration.Workspace AS W
INNER JOIN
collaboration.WorkspaceHistory AS WH ON W.Id = WH.WorkspaceId
INNER JOIN
(
SELECT TOP 10
CDE.Id
FROM
CDE
INNER JOIN
collaboration.WorkspaceHistory AS WH ON WH.WorkspaceId = CDE.Id
WHERE
CDE.Id <> #WorkspaceId
GROUP BY
CDE.Id,
CDE.ParentId,
WH.ActorId,
WH.Action
HAVING
WH.ActorId = #UserId
AND
WH.Action <> 4
ORDER BY
COUNT(*) DESC
) AS Q ON Q.Id = WH.WorkspaceId
GROUP BY
W.Id,
W.Name,
W.Description,
WH.ActorId
HAVING
WH.ActorId = #UserId
You must create stored procedure for your SQL query (or use that query directly) and execute it through dbContext.Database.SqlQuery. You are using code-first approach where you don't have any other options. In EDMX you could use mapped table valued function but code-first doesn't have such option yet.
I have built a stored procedure which takes array of ids as input parameter and return data table using recursive cte query Take a look at the code here it's using EF and code first approach