I'm new to Rails and I've been trying to do ActiveRecord query, but simply can't seem to find how to phrase it.
Here's the SQL equivalent of what I'm trying to do
SELECT c.name, COUNT(p.*) FROM Categories AS c
JOIN products AS p ON c.id = p.category_id
GROUP BY c.name, c.id
I tried variations of
Product.select(:category).group(:category).count
and
Category.joins(:products).group(:category_id).count
But haven't found the right syntax, yet.
Is it possible to do? The project is based on Rails 4.2.
Related
I have several database tables with 2 primary keys, id and date. I do not update the records but instead insert a new record with the updated information. This new record has the same id and the date field is NOW(). I will use a product table to explain my question.
I want to be able to request the product details at a specific date. I therefore use the following subquery in DQL, which works fine:
WHERE p.date = (
SELECT MAX(pp.date)
FROM Entity\Product pp
WHERE pp.id = p.id
AND pp.date < :date
)
This product table has some referenced tables, like category. This category table has the same id and date primary key combination. I want to be able to request the product details and the category details at a specific date. I therefore expanded the DQL as shown above to the following, which also works fine:
JOIN p.category c
WHERE p.date = (
SELECT MAX(pp.date)
FROM Entity\Product pp
WHERE pp.id = p.id
AND pp.date < :date
)
AND c.date = (
SELECT MAX(cc.date)
FROM Entity\ProductCategory cc
WHERE cc.id = c.id
AND cc.date < :date
)
However, as you can see, if I have multiple referenced tables I will have to copy the same piece of DQL. I want to somehow add these subqueries to the entities so that every time an entity is called it adds this subquery.
I have thought of adding this in a __construct($date) or some kind of setUp($date) method, but I'm kind of stuck here. Also, would it help to add #Id to Entity\Product::date?
I hope someone can help me. I do not expect a complete solution, one step in a good direction would be very much appreciated.
I think I've found my solution. The trick was (first, to update to Doctrine 2.2 and) using a filter:
namespace Filter;
use Doctrine\ORM\Mapping\ClassMetaData,
Doctrine\ORM\Query\Filter\SQLFilter;
class VersionFilter extends SQLFilter {
public function addFilterConstraint(ClassMetadata $targetEntity, $targetTableAlias) {
$return = $targetTableAlias . '.date = (
SELECT MAX(sub.date)
FROM ' . $targetEntity->table['name'] . ' sub
WHERE sub.id = ' . $targetTableAlias . '.id
AND sub.date < ' . $this->getParameter('date') . '
)';
return $return;
}
}
Add the filter to the configuration:
$configuration->addFilter("version", Filter\VersionFilter");
And enable it in my repository:
$this->_em->getFilters()->enable("version")->setParameter('date', $date);
I'm quite new to FLOW3 and I'm very new to Doctrine.
I'm just running some tests, and I want to learn or understand some techniques that I'll use later on. Now I'm stuck with this doctrine part where I want to generate some stats.
$results = $this->entityManager
->createQuery('SELECT version,count(version)
FROM (SELECT device, version
FROM \MyStuff\Stats\Domain\Model\Stat
WHERE
date > \'2011-10-01 00:00:00\' and
date < \'2011-10-02 00:00:00\'
GROUP BY device) GROUP BY version')
->getResult();
I asked at other places too, where they directed me to the Doctrine Docs.
Well, now there are several examples but these 2 liners are trivial and I couldn't find any example related to that kind of subselect.
So I hope someone here can help me out.
Edit :
I'd like to solve this using dql.
I tried to solve this using a querybuilder but I was told that querybuilder != dql
Edit 2:
Now I was told that doctrine2 doesn't support subselects in "FROM (SUBSELECT)" but that it does "... WHERE IN (SUBSELECT)" and that one could rewrite my query to the IN () form. Well, trying to figure that out now.
Edit 3:
I'm failing in rewriting the from-subquery into an in-subquery. So... dql doesn't do subqueries and there is no other way to do what I want with dql ?! Then dql would lack a very important feature I'd say. Or am I just not seeing sth. here ?
Edit 4:
I finally got the in-subquery but it was about 10 times slower (4 seconds instead of 0.4) and now I was told by some doctrine guys from #doctrine, that I should use the nativeQuery function instead.
Edit 5:
It works using the nativeQuery now, see my answer for that...
Using a native query it works like that...
$rsm = new \Doctrine\ORM\Query\ResultSetMapping();
$rsm->addScalarResult('version', 'version');
$rsm->addScalarResult('count', 'count');
$results = $this->entityManager
->createNativeQuery(
'SELECT version, COUNT(version) as count FROM
(
SELECT device, version
FROM mystuff_stats_domain_model_stat
WHERE
date > \'2011-10-01 00:00:00\' and
date < \'2011-10-02 00:00:00\'
GROUP BY device
)
as devices GROUP BY version',$rsm)
->execute();
echo var_dump($results,true);
I am building an app in Symfony2, using Doctrine2 with mysql. I would like to use a fulltext search. I can't find much on how to implement this - right now I'm stuck on how to set the table engine to myisam.
It seems that it's not possible to set the table type using annotations. Also, if I did it manually by running an "ALTER TABLE" query, I'm not sure if Doctrine2 will continue to work properly - does it depend on the InnoDB foreign keys?
Is there a better place to ask these questions?
INTRODUCTION
Doctrine2 uses InnoDB which supports Foreign Keys used in Doctrine associations. But as MyISAM does not support this yet, you can not use MyISAM to manage Doctrine Entities.
On the other side, MySQL v5.6, currently in development, will bring the support of InnoDB FTS and so will enable the Full-Text search in InnoDB tables.
SOLUTIONS
So there are two solutions :
Using the MySQL v5.6 at your own risks and hacking a bit Doctrine to implement a MATCH AGAINST method : link in french... (I could translate if needed but there still are bugs and I would not recommend this solution)
As described by quickshifti, creating a MyISAM table with fulltext index just to perform the search on. As Doctrine2 allows native SQL requests and as you can map this request to an entity (details here).
EXAMPLE FOR THE 2nd SOLUTION
Consider the following tables :
table 'user' : InnoDB [id, name, email]
table 'search_user : MyISAM [user_id, name -> FULLTEXT]
Then you just have to write a search request with a JOIN and mapping (in a repository) :
<?php
public function searchUser($string) {
// 1. Mapping
$rsm = new ResultSetMapping();
$rsm->addEntityResult('Acme\DefaultBundle\Entity\User', 'u');
$rsm->addFieldResult('u', 'id', 'id');
$rsm->addFieldResult('u', 'name', 'name');
$rsm->addFieldResult('u', 'email', 'email');
// 2. Native SQL
$sql = 'SELECT u.id, u.name FROM search_user AS s JOIN user AS u ON s.user_id = u.id WHERE MATCH(s.name) AGAINST($string IN BOOLEAN MODE)> 0;
// 3. Run the query
$query = $this->_em->createNativeQuery($sql, $rsm);
// 4. Get the results as Entities !
$results = $query->getResult();
return $results;
}
?>
But the FULLTEXT index needs to stay up-to-date. Instead of using a cron task, you can add triggers (INSERT, UPDATE and DELETE) like this :
CREATE TRIGGER trigger_insert_search_user
AFTER INSERT ON user
FOR EACH ROW
INSERT INTO search_user SET user_id=NEW.id, name=NEW.name;
CREATE TRIGGER trigger_update_search_user
AFTER UPDATE ON user
FOR EACH ROW
UPDATE search_user SET name=name WHERE user_id=OLD.id;
CREATE TRIGGER trigger_delete_search_user
AFTER DELETE ON user
FOR EACH ROW
DELETE FROM search_user WHERE user_id=OLD.id;
So that your search_user table will always get the last changes.
Of course, this is just an example, I wanted to keep it simple, and I know this query could be done with a LIKE.
Doctrine ditched the fulltext Searchable feature from v1 on the move to Doctrine2. You will likely have to roll your own support for a fulltext search in Doctrine2.
I'm considering using migrations to generate the tables themselves, running the search queries w/ the native SQL query option to get sets of ids that refer to tables managed by Doctrine, then using said sets of ids to hydrate records normally through Doctrine.
Will probly cron something periodic to update the fulltext tables.
I have a select field that fetch from an entity
and I would like to customize completely my select by choosing the table the id is picked from
(here I would like to select t.id instead of tl.id as the select value)
return $er->createQueryBuilder('tl')
->addSelect('l')
->addSelect('t')
->leftJoin('tl.lang', 'l')
->leftJoin('tl.type', 't')
->where('l.isDefault = 1')
->orderBy('tl.name', 'ASC');
Due to my tables, I can't simply fetch the table t, I have to use tl
Your query is not according to the syntax defined in Doctrine 2 QueryBuilder: http://www.doctrine-project.org/docs/orm/2.0/en/reference/query-builder.html
Your query might work in Doctrine 1.2 but in Doctrine 2 you should build your query according to the syntax defined in the link I posted above.
For example ->addSelect('l') is not being used in Doctrine 2 anymore. It has become ->add('select', 'l').
You don't have to set different alias for your column. It'll be hydrated as column of the related entity.
class Log:
project = ForeignKey(Project)
msg = CharField(...)
date = DateField(...)
I want to select the four most recent Log entries where each Log entry must have a unique project foreign key. I've tries the solutions on google search but none of them works and the django documentation isn't that very good for lookup..
I tried stuff like:
Log.objects.all().distinct('project')[:4]
Log.objects.values('project').distinct()[:4]
Log.objects.values_list('project').distinct('project')[:4]
But this either return nothing or Log entries of the same project..
Any help would be appreciated!
Queries don't work like that - either in Django's ORM or in the underlying SQL. If you want to get unique IDs, you can only query for the ID. So you'll need to do two queries to get the actual Log entries. Something like:
id_list = Log.objects.order_by('-date').values_list('project_id').distinct()[:4]
entries = Log.objects.filter(id__in=id_list)
Actually, you can get the project_ids in SQL. Assuming that you want the unique project ids for the four projects with the latest log entries, the SQL would look like this:
SELECT project_id, max(log.date) as max_date
FROM logs
GROUP BY project_id
ORDER BY max_date DESC LIMIT 4;
Now, you actually want all of the log information. In PostgreSQL 8.4 and later you can use windowing functions, but that doesn't work on other versions/databases, so I'll do it the more complex way:
SELECT logs.*
FROM logs JOIN (
SELECT project_id, max(log.date) as max_date
FROM logs
GROUP BY project_id
ORDER BY max_date DESC LIMIT 4 ) as latest
ON logs.project_id = latest.project_id
AND logs.date = latest.max_date;
Now, if you have access to windowing functions, it's a bit neater (I think anyway), and certainly faster to execute:
SELECT * FROM (
SELECT logs.field1, logs.field2, logs.field3, logs.date
rank() over ( partition by project_id
order by "date" DESC ) as dateorder
FROM logs ) as logsort
WHERE dateorder = 1
ORDER BY logs.date DESC LIMIT 1;
OK, maybe it's not easier to understand, but take my word for it, it runs worlds faster on a large database.
I'm not entirely sure how that translates to object syntax, though, or even if it does. Also, if you wanted to get other project data, you'd need to join against the projects table.
I know this is an old post, but in Django 2.0, I think you could just use:
Log.objects.values('project').distinct().order_by('project')[:4]
You need two querysets. The good thing is it still results in a single trip to the database (though there is a subquery involved).
latest_ids_per_project = Log.objects.values_list(
'project').annotate(latest=Max('date')).order_by(
'-latest').values_list('project')
log_objects = Log.objects.filter(
id__in=latest_ids_per_project[:4]).order_by('-date')
This looks a bit convoluted, but it actually results in a surprisingly compact query:
SELECT "log"."id",
"log"."project_id",
"log"."msg"
"log"."date"
FROM "log"
WHERE "log"."id" IN
(SELECT U0."id"
FROM "log" U0
GROUP BY U0."project_id"
ORDER BY MAX(U0."date") DESC
LIMIT 4)
ORDER BY "log"."date" DESC