I'm using the NodeJS demo code from here: https://questdb.io/docs/develop/insert-data/ to insert data into QuestDB like this:
setInterval(() => {
run();
}, 3000);
(2 Docker containers on a bridge network)
And I have a browser window open to run
select count(*) from 'trades'
However, the query only runs once and then returns the same result. If I restart the docker containers, the query returns an updated value so I assume the values are successfully getting into the database but they are not reflected in the Postgres queries. I see the same behavior when I use the pg client in Node.
Any explanation or theory that would help me root cause this?
In QDB data is not visible to queries until it's committed .
In case of ILP receiver commits don't happen after each row or even on disconnect . Instead QDB uses a number of properties to determine when to commit efficiently .
In this case the easiest way to reduce insert to commit delay would be to reduce cairo.max.uncommitted.rows to e.g. 10 in conf/server.conf (plus instance/container restart) and then insert 10+ records .
You'll find more details on ILP commits at
https://questdb.io/docs/reference/api/ilp/tcp-receiver/#commit-strategy
How can I configure doctrine to convert for example
/**
* #var UuidInterface
* #ORM\Column(type="uuid", unique=true)
*/
private $locationId;
to database column with name location_id, for some reason it creating name like locationid when I'm running migrations:diff. I know I can add name="location_id", but is's annoying. I always use doctrine with Symfony and it automatically converts camel case to snake case. But now I'm using doctrine with a custom PHP application (not a framework), probably it can be configured somehow, but I can't find any related info in doctrine documentation.
I found https://www.doctrine-project.org/projects/doctrine-orm/en/2.7/reference/namingstrategy.html it in the doctrine documentations
I'm trying to run Sitecore 7 with Oracle and I'm stuck on an error. What I've done is
Let a DBA create all Oracle users / schemas / tables
Verified with Oracle SQL Developer that the databases are indeed created, accessible and containing tables
Make sure I'm using the correct web.config
Update my connection strings
Do all the other Sitecore setup that I would normally do (IIS site, VS solution, etc.)
Install Oracle Client tools (including all possible drivers for my 64-bit system)
I've tried two variations of connection strings:
user id=my_username;password=my_password;Data
Source=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=my_host_name)(PORT=5151))(CONNECT_DATA=(SERVICE_NAME=my_service_name)))
and this one:
Data Source=my_username/my_password#//my_host_name:5151/my_service_name
Using the exact same information, I'm perfectly able to connect using the Oracle SQL Developer.
When I launch my Sitecore site, I get the following exception:
Unhandled Execution Error
Exception Details: Oracle.DataAccess.Client.OracleException:
[OracleException (0x80004005)]
Oracle.DataAccess.Client.OracleException.HandleErrorHelper(Int32
errCode, OracleConnection conn, IntPtr opsErrCtx, OpoSqlValCtx*
pOpoSqlValCtx, Object src, String procedure, Boolean bCheck, Int32
isRecoverable) +1250
Oracle.DataAccess.Client.OracleException.HandleError(Int32 errCode,
OracleConnection conn, IntPtr opsErrCtx, Object src) +91
Oracle.DataAccess.Client.OracleConnection.Open() +4698
....snipped...
Been struggling with this for a day now and would really appreciate some help.
Anyone out there running Sitecore 7 on Oracle?
Make sure that you are using the Oracle client tools version 11g, other Oracle versions are not supported with Sitecore 7.
See this link for a reference
I have the following 2 apps in my Django project. One app (player) has a ForeignKey that references the other app (team).
App 1 - players/models.py
class Player(models.Model):
name = models.CharField(max_length=100)
team = models.ForeignKey(Team)
App 2 - teams/models.py
class Team(models.Model):
name = models.CharField(max_length=100)
Problem #1 - Everything seems to work, but when I try to run manage.py reset teams I get the following error :
Error: Error: users couldn't be reset. Possible reasons:
* The database isn't running or isn't configured correctly.
* At least one of the database tables doesn't exist.
* The SQL was invalid.
Hint: Look at the output of 'django-admin.py sqlreset users'. That's the SQL this command wasn't able to run.
The full error: (1217, 'Cannot delete or update a parent row: a foreign key constraint fails')
I made sure all my tables are InnoDB. I've seen people suggest using SET FOREIGN_KEY_CHECKS=0 as a fix, but I'm not sure how to best use this with manage.py rest
Problem #2 - When I do a mysqldump, I cannot restore my MySQL db without running into errors. My basic procedure is as follows:
Dump my db.
mysqldump --user=root --password=pass --result-file=MYBACKUP.SQL --compact MYDB
DROP the database, and CREATE a new one with the same name.
Restore my db.
mysql -u root -p MYDB < MYBACKUP.SQL
And that's it. When I attempt to restore the first time, it gives me this error
ERROR 1005 (HY000) at line 12: Can't create table 'MYDB.auth_group_permissions' (errno: 150)
When I attempt to restore for the second (and subsequent) time, it makes it one line farther, but it gives me this error:
ERROR 1050 (42S01) at line 3: Table 'auth_group' already exists
I think those errors have to do with Foreign Key Constraints, but I'm not sure how to fix it.
I know this was a mouthful but any advice would be greatly appreciated. I'll post my findings here tomorrow, if any.
Thanks in advance!
Solution for Problem 2 - So my problem with dumping and then restoring my db has to do with how I use mysql dump - the --compact flag is causing the problem.
This line ...
mysqldump --user=root --password=pass --result-file=MYBACKUP.SQL --compact MYDB
... should be ...
mysqldump --user=root --password=pass --result-file=MYBACKUP.SQL MYDB
Is it possible to generate fixtures from an existing DB in Symfony2/Doctrine? How could I do that?
Example:
I have defined 15 entities and my symfony2 application is working. Now some people are able to browse to the application and by using it it had inserted about 5000 rows until now. Now I want the stuff inserted as fixtures, but I don’t want to do this by hand. How can I generate them from the DB?
There's no direct manner within Doctrine or Symfony2, but writing a code generator for it (either within or outside of sf2) would be trivial. Just pull each property and generate a line of code to set each property, then put it in your fixture loading method. Example:
<?php
$i = 0;
$entities = $em->getRepository('MyApp:Entity')->findAll();
foreach($entities as $entity)
{
$code .= "$entity_{$i} = new MyApp\Entity();\n";
$code .= "$entity_{$i}->setMyProperty('" . addslashes($entity->getMyProperty()); . "'); \n");
$code .= "$manager->persist($entity_{$i}); \n $manager->flush();";
++$i;
}
// store code somewhere with file_put_contents
As I understand your question, you have two databases: the first is already in production and filled with 5000 rows, the second one is a new database you want to use for new test and development. Is that right ?
If it is, I suggest you to create in you test environment two entity manager: the first will be the 'default' one, which will be used in your project (your controllers, etc.). The second one will be used to connect to your production database. You will find here how to deal with multiple entity manager : http://symfony.com/doc/current/cookbook/doctrine/multiple_entity_managers.html
Then, you should create a Fixture class which will have access to your container. There is an "how to" here : http://symfony.com/doc/current/bundles/DoctrineFixturesBundle/index.html#using-the-container-in-the-fixtures.
Using the container, you will have access to both entity manager. And this is the 'magic': you will have to retrieve the object from your production database, and persist them in the second entity manager, which will insert them in your test database.
I point your attention to two points:
If there are relationship between object, you will have to take care to those dependencies: owner side, inversed side, ...
If you have 5000 rows, take care on the memory your script will use. Another solution may be use native sql to retrieve all the rows from your production database and insert them in your test database. Or a SQL script...
I do not have any code to suggest to you, but I hope this idea will help you.
I assume that you want to use fixtures (and not just dump the production or staging database in the development database) because a) your schema changes and the dumps would not work if you update your code or b) you don't want to dump the hole database but only want to extend some custom fixtures. An example I can think of is: you have 206 countries in your staging database and users add cities to those countries; to keep the fixtures small you only have 5 countries in your development database, however you want to add the cities that the user added to those 5 countries in the staging database to the development database
The only solution I can think of is to use the mentioned DoctrineFixturesBundle and multiple entity managers.
First of all you should configure two database connections and two entity managers in your config.yml
doctrine:
dbal:
default_connection: default
connections:
default:
driver: %database_driver%
host: %database_host%
port: %database_port%
dbname: %database_name%
user: %database_user%
password: %database_password%
charset: UTF8
staging:
...
orm:
auto_generate_proxy_classes: %kernel.debug%
default_entity_manager: default
entity_managers:
default:
connection: default
mappings:
AcmeDemoBundle: ~
staging:
connection: staging
mappings:
AcmeDemoBundle: ~
As you can see both entity managers map the AcmeDemoBundle (in this bundle I will put the code to load the fixtures). If the second database is not on your development machine, you could just dump the SQL from the other machine to the development machine. That should be possible since we are talking about 500 rows and not about millions of rows.
What you can do next is to implement a fixture loader that uses the service container to retrieve the second entity manager and use Doctrine to query the data from the second database and save it to your development database (the default entity manager):
<?php
namespace Acme\DemoBundle\DataFixtures\ORM;
use Doctrine\Common\DataFixtures\FixtureInterface;
use Doctrine\Common\Persistence\ObjectManager;
use Symfony\Component\DependencyInjection\ContainerAwareInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Acme\DemoBundle\Entity\City;
use Acme\DemoBundle\Entity\Country;
class LoadData implements FixtureInterface, ContainerAwareInterface
{
private $container;
private $stagingManager;
public function setContainer(ContainerInterface $container = null)
{
$this->container = $container;
$this->stagingManager = $this->container->get('doctrine')->getManager('staging');
}
public function load(ObjectManager $manager)
{
$this->loadCountry($manager, 'Austria');
$this->loadCountry($manager, 'Germany');
$this->loadCountry($manager, 'France');
$this->loadCountry($manager, 'Spain');
$this->loadCountry($manager, 'Great Britain');
$manager->flush();
}
protected function loadCountry(ObjectManager $manager, $countryName)
{
$country = new Country($countryName);
$cities = $this->stagingManager->createQueryBuilder()
->select('c')
->from('AcmeDemoBundle:City', 'c')
->leftJoin('c.country', 'co')
->where('co.name = :country')
->setParameter('country', $countryName)
->getQuery()
->getResult();
foreach ($cities as $city) {
$city->setCountry($country);
$manager->persist($city);
}
$manager->persist($country);
}
}
What I did in the loadCountry method was that I load the objects from the staging entity manager, add a reference to the fixture country (the one that already exists in your current fixtures) and persist it using the default entity manager (your development database).
Sources:
DoctrineFixturesBundle
How to work with Multiple Entity Managers
you could use https://github.com/Webonaute/DoctrineFixturesGeneratorBundle
It add ability to generate fixtures for single entity using commands like
$ php bin/console doctrine:generate:fixture --entity=Blog:BlogPost --ids="12 534 124" --name="bug43" --order="1"
Or you can create full snapshot
php app/console doctrine:generate:fixture --snapshot --overwrite
The Doctrine Fixtures are useful because they allow you to create objects and insert them into the database. This is especially useful when you need to create associations or say, encode a password using one of the password encoders. If you already have the data in a database, you shouldn't really need to bring them out of that format and turn it into PHP code, only to have that PHP code insert the same data back into the database. You could probably just do an SQL dump and then re-insert them into your database again that way.
Using a fixture would make more sense if you were initiating your project but wanted to use user input to create it. If you had in your config file the default user, you could read that and insert the object.
The AliceBundle can help you doing this. Indeed it allows to load fixtures with YAML (or PHP array) files.
For instance you can define your fixtures with:
Nelmio\Entity\Group:
group1:
name: Admins
owner: '#user1->id'
Or with the same structure in a PHP array. It's WAY easier than generating working PHP code.
It also supports references:
Nelmio\Entity\User:
# ...
Nelmio\Entity\Group:
group1:
name: Admins
owner: '#user1'
In the doctrine_fixture cookbook, you can see in the last example how to get the service container in your entity.
With this service container, you can retrieve the doctrine service, then the entity manager. With the entity manager, you will be able to get all the data from your database you need.
Hope this will help you!