In my search for a way to deal with multi tenancy for play framework and postgresql schemas,i found this article
Here.
The problem is that i cannot figure out the way that i can execute plain sql statements in play(i am totally noob),can any one help please?.
In play you have the DB object that helps you getting the raw jdbc objects. If your query is a simple update without parameters you can do
DB.execute(simpleUpdate)
if your query is a simple select without parameters you can do
DB.executeQuery(simpleSelect)
if you have something more complex, with parameters, you can then use PreparedStatement by getting directly the connexion
PreparedStatement deleteStmt = DB.getConnection().prepareStatement(myStatement);
try {
deleteStmt.setString(1, myString);
deleteStmt.executeUpdate();
} finally {
deleteStmt.close();
}
We shard Postgres database using schemas. This is solution we've came up with. We add multiple DB connections in application.conf and initialize connection with custom SQL.
# application.conf
db.default.driver=org.postgresql.Driver
db.default.url="jdbc:postgresql://localhost/mytest"
db.default.user=your_user
db.default.password=passwd
db.default.initSQL="SET timezone = 'UTC';"
db.tp0.driver=org.postgresql.Driver
db.tp0.url="jdbc:postgresql://localhost/mytest"
db.tp0.user=your_user
db.tp0.password=passwd
db.tp0.initSQL="SET search_path TO tp0, public;SET timezone = 'UTC';"
Java code
EbeanServer tp0 = Ebean.getServer("tp0");
List<moneylog> res = tp0.find(moneylog.class).findList();
Related
I am currently attempting to use transactions in my C++ app, but I have a problem with the ODBC's auto commit mode.
I am using the POCO libaries to create a connection to a PostgreSQL database on the same machine. Currently, I can send data to this database as single statements, but I cannot get my head around how to use Poco's transaction libraries to be able to send this data more quickly.
As I have several thousand records to insert, and so continuing to use single insert statements is extrememly slow and inpractical - So I am trying to use Poco's transaction to speed this up a bit (a fair bit).
The error I am encountering is a theoretically a simple one - Poco is throwing the following error:
'Invalid access: Session is in auto commit mode.'
I understand, as a result of this, I should somehow set "auto commit" to false - as it only allows me to commit data to the database line by line, rather than as a single transaction.
The problem is how I set this.
Currently, I have a session created from Session.h, that looks alot like this:
session = new Poco::Data::Session(
"ODBC",
connection_data.str()
);
Where connection data is a simple stringstream with the login information, password, database, server and "Driver={PostgreSQL ANSI};" to tell ODBC to utilize PostgreSQL's driver.
I have tried just setting a property "autocommit" to false through the session's setFeature or setProperty settings, this, of course, was to no avail. (it was more of a ditch attempt at this point).
session->setFeature("AUTOCOMMIT", false);
Looking around, I saw a possible alternative method by creating a ODBC sessionImpl directly from ODBC/session/SessionImpl.h instead of using this generic method above, and then creating a new session object from this.
The benefits of this are that ODBC's sessionImpl has references to autocommit mode in the header, which would suggest it would be able to handle this:
void autoCommit(const std::string&, bool val);
/// Sets autocommit property for the session.
However, having not used sessionImpl before, I cannot garuntee if this will work or if can can get this to work with the limited documentation available.
I am using C++ 03 (Not 11), with Visual Studio 2015
Poco 1.7.5
Boost (Where needed)
Would any one know the correct way of setting this feature (above) or a alternative method to achieving this?
edit: Looking at the source of poco, at:
https://github.com/pocoproject/poco/blob/develop/Data/ODBC/src/SessionImpl.cpp#L153
The property seems be named autoCommit, and looking at
https://github.com/pocoproject/poco/blob/develop/Data/include/Poco/Data/AbstractSessionImpl.h#L120
the case of the property names seem to matter. So, does it help if you use session->setFeature("autoCommit", false);?
Cant you just call session->begin(); and session->end(); on the corresponding Session object?
What is returned by session->canTransact()?
According to the doc begin() will start a new transaction, the doc does not mention any property that needs to be set before or after.
See: https://pocoproject.org/docs/Poco.Data.Session.html
Also faced a similar issue.
First of all before begin() need:
m_ses.setFeature("autoCommit", false);
m_ses.begin();
And the second issue is that this feature stays "autoCommit" in false for all other sessions. So don't forget for the next session call
session.setFeature("autoCommit", true);
We use H2 database to execute tests. To isolate each test from another one, the database schema and basic data-setup is dropped and re-created before each test.
Is it possible to create a restore-point after the first setup of the database and restore before each test the data of this point?
SCRIPT just creates a sql-file with all tables and datas. Not a big difference to our own initialization.
Question database restore to particular state for testing is the same, just for Oracle and Postgres.
An old question, but I find it is still relevant. AFAIK there is no restore-point support.
Here is a simple, yet fast approach to backup/restore.
Create a backup prior to running the first test:
Connection conn = DriverManager.getConnection("jdbc:h2:mem:myDatabase;DB_CLOSE_DELAY=-1;LOG=0");
Statement stat = conn.createStatement();
stat.execute("SCRIPT TO 'memFS:myDatabase.sql'");
stat.close();
conn.close();
Restore after each test:
Connection conn = DriverManager.getConnection("jdbc:h2:mem:myDatabase;DB_CLOSE_DELAY=-1;LOG=0");
Statement stat = conn.createStatement();
stat.execute("DROP ALL OBJECTS");
stat.close();
conn.close();
conn = DriverManager.getConnection("jdbc:h2:mem:myDatabase;DB_CLOSE_DELAY=-1;INIT=runscript from 'memFS:myDatabase.sql';LOG=0");
conn.close();
Note that SHUTDOWN command turned out to be faster than DROP ALL OBJECTS, but it caused some issues (connection pool unable to reestablish connection).
I would not say the above approach is slow, far from it. But with a large database and thousands of tests there is still room for improvement as the method above takes some time. I managed to achieve a few times faster backup/restore (~15ms for a DB with ~350 tables) manually composing a script performing TRUNCATE TABLE, ALTER SEQUENCE and do the INSERT of all initial data (needs SET REFERENTIAL_INTEGRITY FALSE for cleanup/restore procedure to be really fast). The code is cumbersome but was worth the effort.
It seems that c++ drivers doesn't accept mongodb connection uri format.
There's no documentation on how i should create connection string, any guess?
I need to connect to a replica set with 3 servers, and set readPreference options.
Create a connection to a replica set in MongoDB C++ client
Until the problems explained in #acm's answer are resolved, I have found a workaround to the bad Connection Strings of the C++ driver. You can create a DBClientReplicaSet using a vector of hosts and ports this way:
//First create a vector of hosts
//( you can ignore port numbers if yours are default)
vector<HostAndPort> hosts;
hosts.push_back(mongo::HostAndPort("YourHost1.com:portNumber1"));
hosts.push_back(mongo::HostAndPort("YourHost2.com:portNumber2"));
hosts.push_back(mongo::HostAndPort("YourHost3.com:portNumber3"));
//Then create a Replica Set DB Client:
mongo::DBClientReplicaSet connection("YourReplicaSetName",hosts,0);
//Connect to it now:
connection.connect();
//Authenticate to the database(s) if needed
std::string errmsg;
connection.auth("DB1Name","UserForDB1","pass1",errmsg);
connection.auth("DB2Name","UserForDB2","pass2",errmsg);
Now, you can use insert, update, etc. just as you did with DBClientConnection. For a quick fix, you can replace your references to DBClientConnection with DBClientBase (which is a parent to both DBClientConnection and DBClientReplicaSet)
Last pitfall: if you are using getLastError(), you must use it with the aimed database name like this:
connection.getLastError(std::string("DBName"));
cause otherwise it will always return "command failed: must log in" as described in this JIRA ticket.
Set the read preferences for every request
You have two ways to do that:
SlaveOK option
It lets your read queries be directed to secondary servers.
It takes place in the query options, which are at the end of the parameters of DBClientReplicaSet.query(). The options are listed in Mongo's official documentation
The one you would look for is mongo::QueryOption_SlaveOk, which will allow you to have reads made on secondary instances.
This is how you should call query();
connection.query("Database.Collection",
QUERY("_id" << id),
n,
m,
BSON("SomeField" << 1),
QueryOption_SlaveOk);
where n is the number of documents to return (0 if you don't want any limit), m the number to skip (defaults to 0), the next field is your projection and the last your query option.
To use several query option, you can use bitwise or | like this :
connection.query("Database.Collection",
QUERY("_id" << id),
n,
m,
BSON("SomeField" << 1),
QueryOption_SlaveOk | QueryOption_NoCursorTimeout | QueryOption_Exhaust);
Query::readPref option
The Query object has a readPref method which sets read preferences for a special query. It should be called for each query.
You can pass different arguments for more control. They are listed here.
So here's what you should do (I did not test that one cause I can't right now but it should work just fine)
/* you should pass an array for the tags. Not sure if this is required.
Anyway, let's create an empty array using the builder. */
BSONArrayBuilder bab;
/* if any, add your tags here */
connection.query("Database.Collection",
QUERY("_id" << id).readPref(ReadPreference_SecondaryPreferred, bab.arr()),
n,
m,
BSON("SomeField" << 1),
QueryOption_NoCursorTimeout | QueryOption_Exhaust);
Note: if any readPref option is used, it should override the slaveOk option.
Hope this helped.
Please see the connection string documentation for details on the connection string format.
(code links below are to 2.2.3 files)
To use a connection string with the C++ driver, you should use the ConnectionString class. You first call the ConnectionString::parse static method with a connection string to obtain a ConnectionString object. You then call ConnectionString::connect to obtain a DBClientBase object which you can then use to send queries.
As for read preference, at the moment I do not see a way to set the read preference in the connection string for the C++ driver, which would preclude a per-connection setting.
However, the implementation of DBClientBase returned by calling ConnectionString::parse with a string that identifies a replica set will return you an instance of DBClientReplicaSet. That class honors $readPreference in queries, so you can set your read preference on a per-query basis.
Since the current C++ drivers still do not accept the standard mongodb connection URIs, I've opened a ticket:
https://jira.mongodb.org/browse/CXX-2
Please vote for it to help get this fixed.
it seems like you can set read Preference before send a read request by call "readPref" method of your Query object. I'v not found a way to set read Preference on mongo collection object yet.
How can i write unitintegration tests that talk to a database. e.g.:
public int GetAppLockCount(DbConnection connection)
{
string query :=
"SELECT"+CRLF+
" tl.resource_type AS ResourceType,"+CRLF+
" tl.resource_description AS ResourceName,"+CRLF+
" tl.request_session_id AS spid"+CRLF+
"FROM sys.dm_tran_locks tl"+CRLF+
"WHERE tl.resource_type = 'APPLICATION'"+CRLF+
"AND tl.resource_database_id = ("+CRLF+
" SELECT dbid"+CRLF+
" FROM master.dbo.sysprocesses"+CRLF+
" WHERE spid = ##spid)";
IRecordset rdr = Connection.Execute(query);
int nCount = 0;
while not rdr.EOF do
{
nCount := nCount+1;
rdr.Next;
}
return nCount;
}
In this case i am trying to exorcise this code of bugs (the IRecordset returns empty recordset).
[UnitTest]
void TestGetLockCountShouldAlwaysSucceed();
{
DbConnection conn = GetConnectionForUnit_IMean_IntegrationTest();
GetAppLockCount(conn);
CheckTrue(True, "We should reach here, whether there are app locks or not");
}
Now all i need is a way to connect to some database when running a unit integration testing.
Do people store connection strings somewhere for the test-runner to find? A .ini or .xml or .config file?
Note: Language/framework agnostic. The code intentionally contains elements from:
C#
Delphi
ADO.net
ADO
NUnit
DUnit
in order to drive that point home.
Now all i need is a way to connect to some database when running a unit integration testing.
Either use an existing database or an in-memory database. I've tried both an currently use an existing database that is splatted and rebuilt using Liquibase scripts in an ant file.
Advantages to in-memory - no dependencies on other applications.
Disadvantages - Not quite as real, can take time to start up.
Advantages to real database - Can be identical to the real world
Disadvantages - Requires access to a 3rd party machine. More work setting up a new user (i.e. create new database)
Do people store connection strings somewhere for the test-runner to find? A .ini or .xml or .config file?
Yeap. In C# I used a .config file, in java a .props file. With in-memory you can throw this into the version control as it will be the same for each person, with a real database running somewhere it will need to be different for each user.
You will also need to consider seed data. In Java I've used dbUnit in the past. Not the most readable, but works. Now I use a Ruby ActiveRecord task.
How do you start this? First can you rebuild your database? You need to be able to automate this before going to far down this road.
Next you should build up a blank local database for your tests. I go with one-per-developer, some other teams share but don't commit. In a .NET/MS SQL world I think in memory would be quite easy to do.
I have a C++ application that uses ADO to talk to an Oracle database. I'm updating the application to support an offline documents. I've decided to implement SQLite for the local side.
I've implemented a wrapper around the ADO classes that will call the appropriate code. However, ADO's way of adding/editing/deleting rows is a bit difficult to implement for SQLite.
For ADO I'd write something like:
CADODatabase db;
CADORecordset rs( &db );
db.Open( "connection string" );
rs.Open( "select * from table1 where table1key=123" );
if (!rs.IsEOF())
{
int value;
rs.GetFieldValue( "field", value );
if (value == 456)
{
rs.Edit();
rs.SetFieldValue( "field", 456 );
rs.Update();
}
}
rs.Close();
db.Close();
For this simple example I realize that I could have just issued an update, but the real code is considerable more complex.
How would I get calls between the Edit() and Update() to actually update the data? My first thought is to have the Edit() construct a separate query and the Update() actually run it, but I'm not sure what fields will be changed nor what keys from the table to limit an update query to.
" but I'm not sure what fields will be changed nor what keys from the table to limit an update query to."
How about just selecting ROWID with the rest of the fields and then building an update based on that ?