Doctrine inserting NULL values for undefined fields - doctrine-orm

I don't know if this is a bug, but I am using Doctrine 2.3.0 and I found the persist/flush behaviour quite strange. I have a basic table:
------------------
| test |
------------------
| id INT | AI |
| field1 VARCHAR |
| field2 VARCHAR |
-----------------
When I create an entry by setting only field1:
$test = new Entities\test();
$test->setField1('foo');
$em->persist($test);
$em->flush();
Doctrine\DBAL\Logging\EchoSQLLogger tells me (which is confirmed by looking at the DB) that Doctrine performs the following query:
INSERT INTO test (field1, field2) VALUES (?, ?)
array(2) {
[1]=>
string(3) "foo"
[2]=>
NULL
}
As you can see, although I haven't set field2, Doctrine does put it in the insert statement with a NULL value.
I have that behaviour for all my entities, which is problematic because when doing inserts, my default DB values for fields I don't set are overwritten by NULL.
Is this the expected default behaviour of Doctrine, is there a way to turn that off (i.e. exclude fields I don't set from the INSERT statements)?
I should probably add that my entities were generated automatically with the reverse engineering, and that the declaration for one of the fields looks like this
/**
* #var string $field2
* // removing nullalble makes no diff.
* #ORM\Column(name="field2", type="string", length=45, nullable=true)
*/
private $field2;
/**
* Set field2
*
* #param string $field2
* #return Test
*/
public function setField2($field2)
{
$this->field2 = $field2;
return $this;
}

I suppose this is default behaviour, as it could be treated logical to create a full record in one INSERT statement. But I could be mistaking. Having some kind of #ORM\Column(type="boolean", columnDefinition="TINYINT(1) NULL DEFAULT 0") could be treated as a hack, but solution. I advise you to review database record insertion logic.

Related

Doctrine/MySQL mediumtext + unncessary ALTER TABLEs

Running on mysql 5.7 + Doctrine 2.7/DBAL 2.9, with:
/**
* #ORM\Entity()
*/
class Entity {
/**
* #ORM\Id()
* #ORM\GeneratedValue()
* #ORM\Column(type="integer")
*/
private $id;
/**
* #ORM\Column(type="string", length=100000, nullable=true)
*/
private $bigText;
}
orm:schema-tool:create creates the table Entity:
CREATE TABLE `Entity` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`bigText` mediumtext COLLATE utf8_unicode_ci,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
However, the schema-tool considers the table in need of update, running orm:schema-tool:update --dump-sql gives:
ALTER TABLE Entity CHANGE bigText bigText MEDIUMTEXT DEFAULT NULL;
Executing the update works fine, but doesn't help: it will still think the table needs updating.
I've tracked the issue to Doctrine\DBAL\Schema\Comparator's diffColumn method where the PlatformOptions of the current and the to-be column definition are compared. The to-be column has no PlatformOptions (which makes sense, as it's not on the platform yet, I suppose), while the current column does have them, namely charset and collation, in my test case here the defaults: ['charset' => 'utf8', 'collation' => 'utf8_unicode_ci'], which are set on the Column object by Doctrine\DBAL\Schema\MySqlSchemaManager->_getPortableTableColumnDefinition().
Poking around a bit in the source, I've found I can use EventListeners: because Doctrine\DBAL\Schema\AbstractSchemaManager->_getPortableTableColumnList() will emit an onSchemaColumnDefinition event and if I set $event->preventDefault() in there and create my own Column object and use $event->setColumn($column), I can circumvent MySqlSchemaManager->_getPortableTableColumnDefinition(). Or, I can go and fiddle with the other side and change the Schema created from the Classes / annotations and use postGenerateSchemaTable (which is much easier and feels less weird than messing with onSchemaColumnDefinition).
While I've learned a tiny bit more about internal flows, this feels way too complicated to handle mediumtext in MySQL without creating unnecessary ALTER TABLEs all the time.
Of course, I can sidestep the issue entirely by using type="text" instead of type="string" + a fixed length (which I've realized while browing issues here before asking this), but out of curiosity: am I missing something fundamental for dealing with fixed length fields in MySQL that fall into mediumtext length?

How to change column charset and collation with Doctrine?

I'm working on a Symfony 3.4 project and I want for some columns of a table to use charset utf8mb4. The aim behind this is to allow emojis.
Here is an abstract of my Doctrine entity:
/**
* #ORM\Table(name="categories")
*/
class Category {
// ...
/**
* #var string
* #ORM\Column(name="description", type="string", length=255)
*/
private $description;
}
I first updated doctrine configuration:
doctrine:
charset: utf8mb4
Then, I updated the Doctrine configuration of the field category.description:
- * #ORM\Column(name="description", type="string", length=255)
+ * #ORM\Column(name="description", type="string", length=191, options={"charset"="utf8mb4"})
Note that I changed the length from 255 to 191 as utf8mb4 uses 4 bytes instead of 3 for utf8.
Finally, I ran update command:
bin/console doctrine:schema:update --dump-sql
What returned:
ALTER TABLE categories CHANGE description description VARCHAR(191) NOT NULL;
As you can see, there is no update about charset. Moreover, we can see a double whitespace between keywords VARCHAR(191) and NOT, letting suppose there should be something here.
The query I expect would be like this:
ALTER TABLE categories MODIFY `description` VARCHAR(191) COLLATE utf8mb4_unicode_ci NOT NULL;
I then ran the update command:
bin/console doctrine:schema:update --force
But when I re-run the --dump-sql command it returns the same query (with the double whitespaces) over and over.
Even when I manually set the charset of the column to utf8mb4 the query is still the same.
I use v2.6.3 of doctrine/orm.
Did I miss something in my configuration?
Does Doctrine handle column charset?
This does not answer my question as it's about changing collation of all tables. In my case I want to change collation of a single column of a specific table.
Short answer
Doctrine won't do this for you.
Alternative solution
Using DoctrineMigrationBundle allows to create migrations in plain SQL. Thus, you can edit auto-generated migrations to add charset/collation on desired columns.
Generate a new blank migration:
bin/console doctrine:migrations:generate
Then, add the ALTER statement:
use Doctrine\DBAL\Schema\Schema;
use Doctrine\Migrations\AbstractMigration;
final class Version20200113131732 extends AbstractMigration {
public function up(Schema $schema) : void {
$this->addSql('ALTER TABLE categories MODIFY `description` VARCHAR(191) COLLATE utf8mb4_unicode_ci NOT NULL;');
}
public function down(Schema $schema) : void {
$this->addSql('ALTER TABLE categories MODIFY `description` VARCHAR(255) COLLATE utf8_unicode_ci NOT NULL;');
}
}
Finally, just run the migration:
bin/console doctrine:migrations:migrate
It looks like doctrine partialy supports the change of collation on columns (only supported by Drizzle, Mysql, PostgreSQL>=9.1, Sqlite and SQLServer).
See doctrine annotations reference
/**
* #ORM\Table(name="categories")
*/
class Category {
// ...
/**
* #var string
* #ORM\Column(name="description", type="string", length=255, options={"collation"="ascii_general_ci"})
*/
private $description;
}

Doctrine2 - Annotations: eager load (join) vs. nullable column (ManyToOne)

I have this:
/**
* #ManyToOne(targetEntity="TblCity",fetch="EAGER",cascade={"persist"})
* #JoinColumn(name="tblCity",referencedColumnName="Id")
*/
and it creates the correct JOIN for table tblCity in sql and TblCity entity is plugged in my parent entity - aka "Eager-Load"
Pseudo result:
PersonEntity: {
Id: 1
...
CityEntity: {
Id: 1
...
}
}
but, this column NEEDS to be nullable
(if it runs into a "missing" foreign id it complains about missing proxy files for TblCity).
So it has to look like this:
/**
* #Column(nullable=true)
* #ManyToOne(targetEntity="TblCity",fetch="EAGER",cascade={"persist"})
* #JoinColumn(name="tblCity",referencedColumnName="Id")
*/
and poff there goes the "Eager-Load"
The generated sql is missing the JOIN of table tblCity and the column contains only the id and not the entity for TblCity
Pseudo result:
PersonEntity: {
Id: 1
...
CityEntity: 1 (as integer)
}
What am I doing wrong?
PS: I CAN'T use createQuery or such things, so please no solutions involving that
The doctrine #JoinColumn annotation has an optiobal attribute nullable which defaults to true. Read more on this here in the documentation: 21.2.15. #JoinColumn
So the proper way to declare nullable for the join column is:
#JoinColumn(name="tblCity",referencedColumnName="Id", nullable=true)
But nullable is true by default so you don't really need it...
My guess would be that your #Column annotation is overruling the whole #ManyToOne annotation in your case. That is why you get only an id and no TblCity entity.

Doctrine 2: There is no column with name '$columnName' on table '$table'

When I do:
vendor/bin/doctrine-module orm:schema-tool:update
Doctrine 2.4 gives me this error:
[Doctrine\DBAL\Schema\SchemaException]
There is no column with name 'resource_id' on table 'role_resource'.
My actual MySQL database schema has the column and the table, as evident from running this command (no errors thrown):
mysql> select resource_id from role_resource;
Empty set (0.00 sec)
Thus, the error must be somewhere in the Doctrine's representation of the schema. I did a var_dump() of $this object, and here is what I get (partial):
object(Doctrine\DBAL\Schema\Table)#546 (10) {
["_name" :protected] => string(13) "role_resource"
["_columns":protected] => array(0) { }
Note that indeed, the _columns key does not contain any columns, which is how Doctrine checks for column names.
In my case, the partial trace dump is as follows:
SchemaException.php#L85
Table.php#L252
Table.php#L161
Reading other posts with similar problem, seem to suggest that I may have an error in the column case (upper vs lower). While it is possible I have missed something, but looking over my actual schema on the Database and the Annotations in my code seem to suggest a match (all lowercase). Similarly, Doctrine2's code does incorporate checks for such casing errors. So I am ruling out the error casing possibility.
Another post I've seen suggests that there may be an error in my Annotations, i.e. wrong naming, syntax, or id placement. I don't know, I checked it and it seems fine. Here is what I have:
class Role implements HierarchicalRoleInterface
{
/**
* #var \Doctrine\Common\Collections\Collection
* #ORM\ManyToMany(targetEntity="ModuleName\Entity\Resource")
* #ORM\JoinTable(name="role_resource",
* joinColumns={#ORM\JoinColumn(name="role_id", referencedColumnName="id")},
* inverseJoinColumns={#ORM\JoinColumn(name="resource_id", referencedColumnName="id")}
* )
*/
protected $resource;
So at the moment, I am stuck, and unable to use the ORM's schema-generation tools. This is a persistent error. I have scraped my database, generated schema anew using ORM, but still get stuck on this error whenever I try to do an update via ORM, as I describe in this post. Where perhaps should I look next?
Update: traced it to this code:
$sql before this line ==
SELECT COLUMN_NAME AS Field,
COLUMN_TYPE AS Type,
IS_NULLABLE AS `Null`,
COLUMN_KEY AS `Key`,
COLUMN_DEFAULT AS `Default`,
EXTRA AS Extra,
COLUMN_COMMENT AS Comment,
CHARACTER_SET_NAME AS CharacterSet,
COLLATION_NAME AS CollactionName,
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = 'loginauth' AND TABLE_NAME = 'role_resource'
which when I run it form MySQL prompt, returns (some columns were trimmed):
+-------------+---------+------+-----+--------------+----------------+
| Field | Type | Null | Key | CharacterSet | CollactionName |
+-------------+---------+------+-----+--------------+----------------+
| role_id | int(11) | NO | PRI | NULL | NULL |
| resource_id | int(11) | NO | PRI | NULL | NULL |
+-------------+---------+------+-----+--------------+----------------+
and the $this->executeQuery($sql, $params, $types) returns the proper(?) statement that runs fine on my prompt, but when ->fetchAll() is called, specifically this fetchAll() it breaks down and returns an empty array. Can I have someone make sense out of this?
MORE:
Essentially, from above links, $this->executeQuery($sql, $params, $types) returns:
object(Doctrine\DBAL\Driver\PDOStatement)#531 (1) {
["queryString"]=> string(332) "SELECT COLUMN_NAME AS Field, COLUMN_TYPE AS Type, IS_NULLABLE AS `Null`, COLUMN_KEY AS `Key`, COLUMN_DEFAULT AS `Default`, EXTRA AS Extra, COLUMN_COMMENT AS Comment, CHARACTER_SET_NAME AS CharacterSet, COLLATION_NAME AS CollactionName FROM information_schema.COLUMNS WHERE TABLE_SCHEMA = 'loginauth' AND TABLE_NAME = 'role_resource'"
}
but then $this->executeQuery($sql, $params, $types)->fetchAll() (adding fetchAll()), returns this:
array(0) {
}
And that is so sad my friends :( because I don't know why it returns an empty array, when the statement in queryString above is so clearly valid and fruitful.
Check that the column names used in 'index' and 'uniqueContraints' schema definitions actually exist:
For example using Annotations:
#ORM\Table(name="user_password_reset_keys", indexes={#ORM\Index(name="key_idx", columns={"key"})} )
I had renamed my column from 'key' to 'reset_key' and this column name mismatch caused the error
/**
* #var string
*
* #ORM\Column(name="reset_key", type="string", length=255, nullable=false)
*/
private $resetKey;
Turns out that my DB permissions prevented my DB user from reading that particular table. using GRANT SELECT ... fixed the issue. Also, DBAL team traced it to a DB Permissions peculiarity returning NULL in MySQL and SQL Server.
Late answer, but may be helps some one
I had the same problem. It was because I used Symphony's command line to create my entity and camel case calling some of its properties. Then, when Doctrine create the table, also via command line, it change camel case for "_ "convention.
I was using a ORM Designer software called Skipper that exports my Entities for me. My problem for just one table only out of 30. Was that I was missing the name attribute on my annotations. Example.
#ORM\Column(name="isActive", ....
I added that attribute "name=" and it worked again!

createNativeQuery - oci8 returns empty array

on a very simple entity :
class Users {
/**
* #ORM\Id
* #ORM\GeneratedValue(strategy="AUTO")
* #ORM\Column(type="integer")
*/
protected $userid;
/** #ORM\Column(type="string") */
protected $username;
}[...]
while trying to do a native query
$rsm = new \Doctrine\ORM\Query\ResultSetMapping;
$rsm->addEntityResult('Application\Entity\Users', 'u');
$rsm->addFieldResult('u', 'test', 'username');
$rsm->addFieldResult('u', 'userid', 'userid');
$q = $objectManager->createNativeQuery('SELECT u.username as test, u.userid from users u where u.userid=17',$rsm);
$result = $result->getResult();
$result returns an empty array with oracle (oci8 driver and pdo).
With MySQL, all is ok. The databases are exactly the same between Oracle and MySQL, same tables, same columns.
The table 'users' is not empty, because when using DQL, it works. Works too when using addRootEntityFromClassMetadata() whith a native query.
It seems that the problem occurs only with oci8+addEntityResult().
Any idea ?
Thanks by advance.
Found it in the manual.
ResultSetMapping#addFieldResult();
The first parameter is the alias of the entity result to which the field result will belong. The second parameter is the name of the column in the SQL result set. Note that this name is case sensitive, i.e. if you use a native query against Oracle it must be all uppercase. The third parameter is the name of the field on the entity result identified by $alias into which the value of the column should be set.
$rsm->addFieldResult('u', 'TEST', 'username');
$rsm->addFieldResult('u', 'USERID', 'userid');