I am using symfony2 and Query builder. I get mixed results as a result of my query that follows. I know this is due to using aggregate functions in SELECT, but I DO NOT find any other way and this is exactly what I need. Any direction would be gratefully appreciated:
public function Inventory() {
$em = $this->getEntityManager();
$qb = $em->createQueryBuilder();
$qb
->add('select', 'i.id, i.name, SUM(pod.quantityorder) as quantityordered, SUM(it.quantity) as quantityreceived')
->add('from', 'AutokeenPurchasingBundle:Items i')
->leftJoin('i.purchase_order_details', 'pod')
->leftJoin('i.inventory_transactions', 'it', 'WITH', 'it.inventory_transaction_type = 1')
->add('groupBy', 'i.id')
->addGroupBy('it.item')
->addGroupBy('pod.item')
->add('orderBy', 'i.id');
$query = $qb->getQuery();
$query->useResultCache('my_cache_id');
return $result = $query->getScalarResult();
}
Related
I'm using Symfony 3.4 and Doctrine.
I need to update large amount of entities (300k+) using Doctrine.
I've read batch article form Doctrine docs and I've read topics from stack, but problem is despite size of the batch (20, 100, 200, 500) I'm getting 'out of memory' error anyway when I'm approaching 20k proccessed entities.
Here is my function.
Can someone, please, give me a hint/suggestion how to avoid this?
protected function execute(InputInterface $input, OutputInterface $output): void
{
$io = new SymfonyStyle($input, $output);
$em = $this->getContainer()->get('doctrine.orm.entity_manager');
$em->getConfiguration()->setSQLLogger(null);
$repository = $em->getRepository('AppBundle:Order');
$qb = $repository->createQueryBuilder('o');
$totalCount = (int) $qb->select($qb->expr()->count('o'))
->where($qb->expr()->eq('o.amountOut', 0))
->getQuery()
->getSingleScalarResult();
$progressBar = $io->createProgressBar($totalCount);
$query = $qb->select('o')
->where($qb->expr()->eq('o.amountOut', 0))
->getQuery();
$iterableResult = $query->iterate();
$batchSize = 100;
$i = 0;
foreach ($iterableResult as $row) {
/** #var Order $order */
$order = $row[0];
$commissionsArr = $this->calcCommissionInOutFromOrder($order);
$amountOut = $order->getTransferAmount();
$order->setAmountOut($amountOut);
$order->setCommissionIn($commissionsArr['commission_in']);
$order->setCommissionOut($commissionsArr['commission_out']);
$em->persist($order);
$progressBar->advance();
if (0 === ($i % $batchSize)) {
$em->flush();
$em->clear();
}
++$i;
}
$em->flush();
$io->success('Suckess');
}
Found actual answer in Memory leak when executing Doctrine query in loop.
Quoting: "I resolved this by adding --no-debug to my command. It turns out that in debug mode, the profiler was storing information about every single query in memory."
It actually worked. Using memory_get_usage() I've checked it.
A previous question I asked was to do with hydrating a result set when using Doctrine and query builder. My issue was how to return an array and their sub-sets:
This was for a single result set and the answer was quite simple:
$qb = $this->stoneRepository->createQueryBuilder('S');
$query = $qb->addSelect('A','P','I','C')
->leftJoin('S.attribute', 'A')
->leftJoin('A.category', 'C')
->innerJoin('S.product' , 'P')
->innerJoin('S.image' , 'I')
->where('S.id = :sid')
->setParameter('sid', (int) $stone_id)
->getQuery();
$resultArray = $query->getOneOrNullResult(\Doctrine\ORM\Query::HYDRATE_ARRAY);
return $resultArray;
My next question is how to do this exact same thing for a collection? This is what I have tried:
public function fetchAll()
{
$qb = $this->stoneRepository->createQueryBuilder('S');
$qb->addSelect('A','P','I','C')
->leftJoin('S.attribute', 'A')
->leftJoin('A.category', 'C')
->innerJoin('S.product' , 'P')
->innerJoin('S.image' , 'I')
->where('S.state=:state')
->setParameter('state' , 1 );
$adapter = new DoctrineAdapter( new ORMPaginator( $qb ) );
$collection = new StoneCollection($adapter);
return $collection;
}
The problem I am facing with this solution is that the join tables are not being populated and I am ending up with a collection of empty results.
The StoneCollection class simply extends paginator:
<?php
namespace Api\V1\Rest\Stone;
use Zend\Paginator\Paginator;
class StoneCollection extends Paginator
{
}
I am thinking that perhaps the best mehod is to get an array and to page the array?
EDIT::
I have this working although I am not keen on it as I hit the DB twice. The first time to build the array (Which is the entire result set which could be very big for some applications) and then the second time to page the results which is then returned to HAL in ApiGility for processing...
Ideally this should be done in one go however I am not sure how to hydrate the results in a single instance...
public function fetchAll( $page = 1 )
{
$qb = $this->stoneRepository->createQueryBuilder('S');
$qb->addSelect('A','P','I','C')
->leftJoin('S.attribute', 'A')
->leftJoin('A.category', 'C')
->innerJoin('S.product' , 'P')
->innerJoin('S.image' , 'I')
->where('S.state=:state')
->setParameter('state' , 1 );
$resultArray = $qb->getQuery()->getResult(\Doctrine\ORM\Query::HYDRATE_ARRAY);
$paginator = new \Zend\Paginator\Paginator(new \Zend\Paginator\Adapter\ArrayAdapter($resultArray));
$paginator->setCurrentPageNumber($page);
return $paginator;
}
The Answer to this is as I have above:
I have this working although I am not keen on it as I hit the DB twice. The first time to build the array (Which is the entire result set which could be very big for some applications) and then the second time to page the results which is then returned to HAL in ApiGility for processing...
Ideally this should be done in one go however I am not sure how to hydrate the results in a single instance...
public function fetchAll( $page = 1 )
{
$qb = $this->stoneRepository->createQueryBuilder('S');
$qb->addSelect('A','P','I','C')
->leftJoin('S.attribute', 'A')
->leftJoin('A.category', 'C')
->innerJoin('S.product' , 'P')
->innerJoin('S.image' , 'I')
->where('S.state=:state')
->setParameter('state' , 1 );
$resultArray = $qb->getQuery()->getResult(\Doctrine\ORM\Query::HYDRATE_ARRAY);
$paginator = new \Zend\Paginator\Paginator(new \Zend\Paginator\Adapter\ArrayAdapter($resultArray));
$paginator->setCurrentPageNumber($page);
return $paginator;
}
On the Doctrine documentation for Pagination they state to use $fetchJoinCollection = true, which I believe is the same as the HYDRATE you are trying to use.
Doctrine Pagination
On my pagination code for my QueryBuilder I use it like the following:
public function getAllPaginated($page, $limit){
$query = $this->createQueryBuilder('o')
->select('o')
->getQuery();
$paginator = new Paginator($query, $fetchJoinCollection = true);
$paginator->getQuery()
->setFirstResult($limit * ($page - 1)) // Offset
->setMaxResults($limit);
return $paginator;
}
I dont get how to make it work.
I have:
a table partner with fields id and name
a table partner_address with two fields: id_partner and id_address
a table address with fields id and external key id_town which references town(id)
a table town with fields id, a name, and postal_code
I want to select all partners that are in towns with specific postal_code
This query works:
SELECT p.nom, v.nom
FROM partner p
JOIN partner_address pa
ON pa.id_partner=p.id
JOIN address a
ON pa.id_address = a.id
JOIN town t
ON a.id_town=t.id
WHERE t.postal_code='13480';
Now I want to "translate" it into Doctrine 2 full syntax, following the documentation.
So I've made a custom repository:
src/Society/Bundle/MyProjectBundle/Repository/PartnerRepository.php
In this repository, I'm trying to create the corresponding function:
<?php
namespace HQF\Bundle\PizzasBundle\Repository;
use Doctrine\ORM\EntityRepository;
class PartenaireRepository extends EntityRepository
{
/**
* Get all active partners from a given postal code.
*/
public function findAllActiveByCp($cp)
{
return $this->createQueryBuilder('p')
->where('p.dateVFin IS NULL')
->andWhere('p.cp=:cp')
->addOrderBy('p.cp', 'DESC')
->setParameter('cp', $cp);
}
}
Nota: the query in the code is not the right one but this code works in another custom repository I've made, so I'm trying to start from this code.
I'm trying something like this but it doesn't work:
public function findAllActiveByCp($cp)
{
$qb = $this->createQueryBuilder('p');
return $qb
->leftJoin('partner_address pa ON pa.id_partner=p.id')
->leftJoin('address a ON pa.id_address = a.id')
->leftJoin('town t ON a.id_ville=t.id')
->where('p.dateVFin IS NULL')
->andWhere('t.cp=:cp')
->addOrderBy('t.cp', 'DESC')
->setParameter('cp', $cp);
}
I get this error:
Warning: Missing argument 2 for Doctrine\ORM\QueryBuilder::leftJoin(),
called in
/blabla/Repository/PartenaireRepository.php
on line 18 and defined in
/blabla/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/QueryBuilder.php
line 767
You have to join only properties, that the selected entity have.
In first parameter of join() or leftJoin() or xxxJoin() you pass the attribute name related to selected object, and in the second - alias for joined entity.
Try similar to this:
$q = $this->em()->createQueryBuilder();
$q->select(['item', 'itemContact'])
->from('ModuleAdmin\Entity\CustomerEntity', 'item')
->leftJoin('item.contacts', 'itemContact')
->andWhere($q->expr()->like('item.name', ':customerNameStart'));
Of course, the CustomerEntity contains OneToMany relation in field contacts.
Remember, that in select statement you have to select the root entity (in my example CustomerEntity aliased as item).
Edit by Olivier Pons to add how I found out the solution, and to mark this answer as valid, because it put me on the right track, thank you Adam!
In the file PartenaireRepository.php I've used the createQueryBuilder('p') properly. Here's how to make two joins in a row, using createQueryBuilder():
class PartenaireRepository extends EntityRepository
{
/**
* Retrieval of all partners given for a given postal code.
*/
public function findAllActiveByCp($cp)
{
return $this->createQueryBuilder('p')
->leftJoin('p.adresses', 'a')
->leftJoin('a.ville', 'v')
->where('v.cp=:cp')
->setParameter('cp', $cp);
... blabla
}
}
I believe for what you're doing, you will need to provide four arguments to the leftJoin method.
->leftJoin('partner_address', 'pa', 'ON', 'pa.id_partner = p.id')
So your query builder chain should look like this
public function findAllActiveByCp($cp)
{
$qb = $this->createQueryBuilder('p');
return $qb
->leftJoin('partner_address', 'pa', 'ON', 'pa.id_partner = p.id')
->leftJoin('address', 'a', 'ON', 'pa.id_address = a.id')
->leftJoin('town', 't', 'ON', 'a.id_ville = t.id')
->where('p.dateVFin IS NULL')
->andWhere('t.cp=:cp')
->addOrderBy('t.cp', 'DESC')
->setParameter('cp', $cp)
;
}
Summary: which is quicker: updating / flushing a list of entities, or running a query builder update on each?
We have the following situation in Doctrine ORM (version 2.3).
We have a table that looks like this
cow
wolf
elephant
koala
and we would like to use this table to sort a report of a fictional farm. The problem is that the user wishes to have a customer ordering of the animals (e.g. Koala, Elephant, Wolf, Cow). Now there exist possibilities using CONCAT, or CASE to add a weight to the DQL (example 0002wolf, 0001elephant). In my experience this is either tricky to build and when I got it working the result set was an array and not a collection.
So, to solve this we added a "weight" field to each record and, before running the select, we assign each one with a weight:
$animals = $em->getRepository('AcmeDemoBundle:Animal')->findAll();
foreach ($animals as $animal) {
if ($animal->getName() == 'koala') {
$animal->setWeight(1);
} else if ($animal->getName() == 'elephant') {
$animal->setWeight(2);
}
// etc
$em->persist($animal);
}
$em->flush();
$query = $em->createQuery(
'SELECT c FROM AcmeDemoBundle:Animal c ORDER BY c.weight'
);
This works perfectly. To avoid race conditions we added this inside a transaction block:
$em->getConnection()->beginTransaction();
// code from above
$em->getConnection()->rollback();
This is a lot more robust as it handles multiple users generating the same report. Alternatively the entities can be weighted like this:
$em->getConnection()->beginTransaction();
$qb = $em->createQueryBuilder();
$q = $qb->update('AcmeDemoBundle:Animal', 'c')
->set('c.weight', $qb->expr()->literal(1))
->where('c.name = ?1')
->setParameter(1, 'koala')
->getQuery();
$p = $q->execute();
$qb = $em->createQueryBuilder();
$q = $qb->update('AcmeDemoBundle:Animal', 'c')
->set('c.weight', $qb->expr()->literal(2))
->where('c.name = ?1')
->setParameter(1, 'elephant')
->getQuery();
$p = $q->execute();
// etc
$query = $em->createQuery(
'SELECT c FROM AcmeDemoBundle:Animal c ORDER BY c.weight'
);
$em->getConnection()->rollback();
Questions:
1) which of the two examples would have better performance?
2) Is there a third or better way to do this bearing in mind we need a collection as a result?
Please remember that this is just an example - sorting the result set in memory is not an option, it must be done on the database level - the real statement is a 10 table join with 5 orderbys.
Initially you could make use of a Doctrine implementation named Logging (\Doctrine\DBAL\LoggingProfiler). I know that it is not the better answer, but at least you can implement it in order to get best result for each example that you have.
namespace Doctrine\DBAL\Logging;
class Profiler implements SQLLogger
{
public $start = null;
public function __construct()
{
}
/**
* {#inheritdoc}
*/
public function startQuery($sql, array $params = null, array $types = null)
{
$this->start = microtime(true);
}
/**
* {#inheritdoc}
*/
public function stopQuery()
{
echo "execution time: " . microtime(true) - $this->start;
}
}
In you main Doctrine configuration you can enable as:
$logger = new \Doctrine\DBAL\Logging\Profiler;
$config->setSQLLogger($logger);
what can i do if the parameter has no value?
my query:
$query = $this->_em->createQueryBuilder()
->select('u')
->from('Users', 'u')
->where('u.id = ?1')
->andWhere('u.status= ?2')
->setParameter(1, $userid)
->setParameter(2, $status)
->getQuery();
return $query->getResult();
if theres no $status, then it doesnt display anything.
i tried putting a condition before the query to check if its null but what value can i set $status iif theres no status set
The query builder is exactly there for building conditional queries. You could do:
$qb = $this->_em->createQueryBuilder();
$query = $qb->select('u')
->from('Users', 'u')
->where('u.id = ?1')
->setParameter(1, $userid);
if ($status) {
$qb->andWhere('u.status = ?2')
->setParameter(2, $status);
}
return $qb->getQuery()->getResult();
On a side note, it is best practice to use named placeholders e. g. like this:
$qb->andWhere('u.status = :status')
->setParameter('status', $status);
You could write:
->andWhere('(u.status= ?2 or ?2 is null)')