How to change column charset and collation with Doctrine? - doctrine-orm

I'm working on a Symfony 3.4 project and I want for some columns of a table to use charset utf8mb4. The aim behind this is to allow emojis.
Here is an abstract of my Doctrine entity:
/**
* #ORM\Table(name="categories")
*/
class Category {
// ...
/**
* #var string
* #ORM\Column(name="description", type="string", length=255)
*/
private $description;
}
I first updated doctrine configuration:
doctrine:
charset: utf8mb4
Then, I updated the Doctrine configuration of the field category.description:
- * #ORM\Column(name="description", type="string", length=255)
+ * #ORM\Column(name="description", type="string", length=191, options={"charset"="utf8mb4"})
Note that I changed the length from 255 to 191 as utf8mb4 uses 4 bytes instead of 3 for utf8.
Finally, I ran update command:
bin/console doctrine:schema:update --dump-sql
What returned:
ALTER TABLE categories CHANGE description description VARCHAR(191) NOT NULL;
As you can see, there is no update about charset. Moreover, we can see a double whitespace between keywords VARCHAR(191) and NOT, letting suppose there should be something here.
The query I expect would be like this:
ALTER TABLE categories MODIFY `description` VARCHAR(191) COLLATE utf8mb4_unicode_ci NOT NULL;
I then ran the update command:
bin/console doctrine:schema:update --force
But when I re-run the --dump-sql command it returns the same query (with the double whitespaces) over and over.
Even when I manually set the charset of the column to utf8mb4 the query is still the same.
I use v2.6.3 of doctrine/orm.
Did I miss something in my configuration?
Does Doctrine handle column charset?
This does not answer my question as it's about changing collation of all tables. In my case I want to change collation of a single column of a specific table.

Short answer
Doctrine won't do this for you.
Alternative solution
Using DoctrineMigrationBundle allows to create migrations in plain SQL. Thus, you can edit auto-generated migrations to add charset/collation on desired columns.
Generate a new blank migration:
bin/console doctrine:migrations:generate
Then, add the ALTER statement:
use Doctrine\DBAL\Schema\Schema;
use Doctrine\Migrations\AbstractMigration;
final class Version20200113131732 extends AbstractMigration {
public function up(Schema $schema) : void {
$this->addSql('ALTER TABLE categories MODIFY `description` VARCHAR(191) COLLATE utf8mb4_unicode_ci NOT NULL;');
}
public function down(Schema $schema) : void {
$this->addSql('ALTER TABLE categories MODIFY `description` VARCHAR(255) COLLATE utf8_unicode_ci NOT NULL;');
}
}
Finally, just run the migration:
bin/console doctrine:migrations:migrate

It looks like doctrine partialy supports the change of collation on columns (only supported by Drizzle, Mysql, PostgreSQL>=9.1, Sqlite and SQLServer).
See doctrine annotations reference
/**
* #ORM\Table(name="categories")
*/
class Category {
// ...
/**
* #var string
* #ORM\Column(name="description", type="string", length=255, options={"collation"="ascii_general_ci"})
*/
private $description;
}

Related

Doctrine/MySQL mediumtext + unncessary ALTER TABLEs

Running on mysql 5.7 + Doctrine 2.7/DBAL 2.9, with:
/**
* #ORM\Entity()
*/
class Entity {
/**
* #ORM\Id()
* #ORM\GeneratedValue()
* #ORM\Column(type="integer")
*/
private $id;
/**
* #ORM\Column(type="string", length=100000, nullable=true)
*/
private $bigText;
}
orm:schema-tool:create creates the table Entity:
CREATE TABLE `Entity` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`bigText` mediumtext COLLATE utf8_unicode_ci,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
However, the schema-tool considers the table in need of update, running orm:schema-tool:update --dump-sql gives:
ALTER TABLE Entity CHANGE bigText bigText MEDIUMTEXT DEFAULT NULL;
Executing the update works fine, but doesn't help: it will still think the table needs updating.
I've tracked the issue to Doctrine\DBAL\Schema\Comparator's diffColumn method where the PlatformOptions of the current and the to-be column definition are compared. The to-be column has no PlatformOptions (which makes sense, as it's not on the platform yet, I suppose), while the current column does have them, namely charset and collation, in my test case here the defaults: ['charset' => 'utf8', 'collation' => 'utf8_unicode_ci'], which are set on the Column object by Doctrine\DBAL\Schema\MySqlSchemaManager->_getPortableTableColumnDefinition().
Poking around a bit in the source, I've found I can use EventListeners: because Doctrine\DBAL\Schema\AbstractSchemaManager->_getPortableTableColumnList() will emit an onSchemaColumnDefinition event and if I set $event->preventDefault() in there and create my own Column object and use $event->setColumn($column), I can circumvent MySqlSchemaManager->_getPortableTableColumnDefinition(). Or, I can go and fiddle with the other side and change the Schema created from the Classes / annotations and use postGenerateSchemaTable (which is much easier and feels less weird than messing with onSchemaColumnDefinition).
While I've learned a tiny bit more about internal flows, this feels way too complicated to handle mediumtext in MySQL without creating unnecessary ALTER TABLEs all the time.
Of course, I can sidestep the issue entirely by using type="text" instead of type="string" + a fixed length (which I've realized while browing issues here before asking this), but out of curiosity: am I missing something fundamental for dealing with fixed length fields in MySQL that fall into mediumtext length?

Doctrine2 - Annotations: eager load (join) vs. nullable column (ManyToOne)

I have this:
/**
* #ManyToOne(targetEntity="TblCity",fetch="EAGER",cascade={"persist"})
* #JoinColumn(name="tblCity",referencedColumnName="Id")
*/
and it creates the correct JOIN for table tblCity in sql and TblCity entity is plugged in my parent entity - aka "Eager-Load"
Pseudo result:
PersonEntity: {
Id: 1
...
CityEntity: {
Id: 1
...
}
}
but, this column NEEDS to be nullable
(if it runs into a "missing" foreign id it complains about missing proxy files for TblCity).
So it has to look like this:
/**
* #Column(nullable=true)
* #ManyToOne(targetEntity="TblCity",fetch="EAGER",cascade={"persist"})
* #JoinColumn(name="tblCity",referencedColumnName="Id")
*/
and poff there goes the "Eager-Load"
The generated sql is missing the JOIN of table tblCity and the column contains only the id and not the entity for TblCity
Pseudo result:
PersonEntity: {
Id: 1
...
CityEntity: 1 (as integer)
}
What am I doing wrong?
PS: I CAN'T use createQuery or such things, so please no solutions involving that
The doctrine #JoinColumn annotation has an optiobal attribute nullable which defaults to true. Read more on this here in the documentation: 21.2.15. #JoinColumn
So the proper way to declare nullable for the join column is:
#JoinColumn(name="tblCity",referencedColumnName="Id", nullable=true)
But nullable is true by default so you don't really need it...
My guess would be that your #Column annotation is overruling the whole #ManyToOne annotation in your case. That is why you get only an id and no TblCity entity.

Doctrine2 migrations rename index for foreign keys

I have problem with renaming indexes in doctrine:migrations:diff. Everytime when i ran this command, doctrine created sql queries to rename existing index in database. Is there any way to prevent this behaviour?
Full description:
we have big(about 2 hundres tables) old database on Oracle. There are naming conventions for constraint and indexes, for example:
we have table RANDOM_KEY and USER, foreign key (many to one) from RANDOM_KEY to USER, so coinstraint has name FK_RANDOM_KEY_USER and relevant index is FK_RANDOM_KEY_USER_I.
Entity declaration:
/**
* #ORM\Table(name="RANDOM_KEY")
* #ORM\Entity
*/
class RandomKey {
...
Relations declaration:
/**
* #var \User
*
* #ORM\ManyToOne(targetEntity="\User")
* #ORM\JoinColumns({
* #ORM\JoinColumn(name="USER_ID", referencedColumnName="USER_ID")
* })
*/
private $user;
Result of migrations:diff commad is:
$this->addSql("ALTER INDEX FK_RANDOM_KEY_USER_I RENAME TO IDX_C54F7889A0666B6F");
I looked into depth of doctrine, tried some debugging. Between indexes are no other differences, only name. Also I tried add indexes definition to #ORM/Table. Is there way to prevent index renaming?
Relevant composer data:
"symfony/symfony": "2.4.*",
"doctrine/orm": "2.5.*",
"doctrine/dbal": "2.5.*",
"doctrine/migrations": "1.0.*#dev",
"doctrine/doctrine-bundle": "1.2.*",
"doctrine/doctrine-migrations-bundle": "2.1.*#dev",

createNativeQuery - oci8 returns empty array

on a very simple entity :
class Users {
/**
* #ORM\Id
* #ORM\GeneratedValue(strategy="AUTO")
* #ORM\Column(type="integer")
*/
protected $userid;
/** #ORM\Column(type="string") */
protected $username;
}[...]
while trying to do a native query
$rsm = new \Doctrine\ORM\Query\ResultSetMapping;
$rsm->addEntityResult('Application\Entity\Users', 'u');
$rsm->addFieldResult('u', 'test', 'username');
$rsm->addFieldResult('u', 'userid', 'userid');
$q = $objectManager->createNativeQuery('SELECT u.username as test, u.userid from users u where u.userid=17',$rsm);
$result = $result->getResult();
$result returns an empty array with oracle (oci8 driver and pdo).
With MySQL, all is ok. The databases are exactly the same between Oracle and MySQL, same tables, same columns.
The table 'users' is not empty, because when using DQL, it works. Works too when using addRootEntityFromClassMetadata() whith a native query.
It seems that the problem occurs only with oci8+addEntityResult().
Any idea ?
Thanks by advance.
Found it in the manual.
ResultSetMapping#addFieldResult();
The first parameter is the alias of the entity result to which the field result will belong. The second parameter is the name of the column in the SQL result set. Note that this name is case sensitive, i.e. if you use a native query against Oracle it must be all uppercase. The third parameter is the name of the field on the entity result identified by $alias into which the value of the column should be set.
$rsm->addFieldResult('u', 'TEST', 'username');
$rsm->addFieldResult('u', 'USERID', 'userid');

Doctrine inserting NULL values for undefined fields

I don't know if this is a bug, but I am using Doctrine 2.3.0 and I found the persist/flush behaviour quite strange. I have a basic table:
------------------
| test |
------------------
| id INT | AI |
| field1 VARCHAR |
| field2 VARCHAR |
-----------------
When I create an entry by setting only field1:
$test = new Entities\test();
$test->setField1('foo');
$em->persist($test);
$em->flush();
Doctrine\DBAL\Logging\EchoSQLLogger tells me (which is confirmed by looking at the DB) that Doctrine performs the following query:
INSERT INTO test (field1, field2) VALUES (?, ?)
array(2) {
[1]=>
string(3) "foo"
[2]=>
NULL
}
As you can see, although I haven't set field2, Doctrine does put it in the insert statement with a NULL value.
I have that behaviour for all my entities, which is problematic because when doing inserts, my default DB values for fields I don't set are overwritten by NULL.
Is this the expected default behaviour of Doctrine, is there a way to turn that off (i.e. exclude fields I don't set from the INSERT statements)?
I should probably add that my entities were generated automatically with the reverse engineering, and that the declaration for one of the fields looks like this
/**
* #var string $field2
* // removing nullalble makes no diff.
* #ORM\Column(name="field2", type="string", length=45, nullable=true)
*/
private $field2;
/**
* Set field2
*
* #param string $field2
* #return Test
*/
public function setField2($field2)
{
$this->field2 = $field2;
return $this;
}
I suppose this is default behaviour, as it could be treated logical to create a full record in one INSERT statement. But I could be mistaking. Having some kind of #ORM\Column(type="boolean", columnDefinition="TINYINT(1) NULL DEFAULT 0") could be treated as a hack, but solution. I advise you to review database record insertion logic.