Background:
My team is dedicated to ensuring that straight from checkout, our code compiles and unit tests run successfully. To facilitate this and test some of our NHibernate mappings, we've added a SQLite DB to our repository which is a mirror of our production SQL Server 2005 database. We're using the latest versions of: MbUnit3 (part of Gallio), System.Data.SQLite and NHibernate.
Problem:
I've discovered that the following unit test does not work with SQLite, despite executing without trouble against SQL Server 2005.
[Test]
[Rollback]
public void CompleteCanPersistNode()
{
// returns a Configuration for either SQLite or SQL Server 2005 depending on how the project is configured.
Configuration config = GetDbConfig();
ISessionFactory sessionFactory = config.BuildSessionFactory();
ISession session = sessionFactory.OpenSession();
Node node = new Node();
node.Name = "Test Node";
node.PhysicalNodeType = session.Get<NodeType>(1);
// SQLite fails with the exception below after the next line called.
node.NodeLocation = session.Get<NodeLocation>(2);
session.Save(node);
session.Flush();
Assert.AreNotEqual(-1, node.NodeID);
Assert.IsNotNull(session.Get<Node>(node.NodeID));
}
The exception I'm getting (ONLY when working with SQLite) follows:
NHibernate.ADOException: cannot open connection --->
System.Data.SQLite.SQLiteException:
The database file is locked database is locked
at System.Data.SQLite.SQLite3.Step(SQLiteStatement stmt)
at System.Data.SQLite.SQLiteDataReader.NextResult()
at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteCommand cmd, CommandBehavior behave)
at System.Data.SQLite.SQLiteCommand.ExecuteReader(CommandBehavior behavior)
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery()
at System.Data.SQLite.SQLiteTransaction..ctor(SQLiteConnection connection, Boolean deferredLock)
at System.Data.SQLite.SQLiteConnection.BeginDbTransaction(IsolationLevel isolationLevel)
at System.Data.SQLite.SQLiteConnection.BeginTransaction()
at System.Data.SQLite.SQLiteEnlistment..ctor(SQLiteConnection cnn, Transaction scope)
at System.Data.SQLite.SQLiteConnection.EnlistTransaction(Transaction transaction)
at System.Data.SQLite.SQLiteConnection.Open()
at NHibernate.Connection.DriverConnectionProvider.GetConnection()
at NHibernate.Impl.SessionFactoryImpl.OpenConnection()
--- End of inner exception stack trace ---
at NHibernate.Impl.SessionFactoryImpl.OpenConnection()
at NHibernate.AdoNet.ConnectionManager.GetConnection()
at NHibernate.AdoNet.AbstractBatcher.Prepare(IDbCommand cmd)
at NHibernate.AdoNet.AbstractBatcher.ExecuteReader(IDbCommand cmd)
at NHibernate.Loader.Loader.GetResultSet(IDbCommand st, Boolean autoDiscoverTypes, Boolean callable, RowSelection selection, ISessionImplementor session)
at NHibernate.Loader.Loader.DoQuery(ISessionImplementor session, QueryParameters queryParameters, Boolean returnProxies)
at NHibernate.Loader.Loader.DoQueryAndInitializeNonLazyCollections(ISessionImplementor session, QueryParameters queryParameters, Boolean returnProxies)
at NHibernate.Loader.Loader.LoadEntity(ISessionImplementor session, Object id, IType identifierType, Object optionalObject, String optionalEntityName, Object optionalIdentifier, IEntityPersister persister)
at NHibernate.Loader.Entity.AbstractEntityLoader.Load(ISessionImplementor session, Object id, Object optionalObject, Object optionalId)
at NHibernate.Loader.Entity.AbstractEntityLoader.Load(Object id, Object optionalObject, ISessionImplementor session)
at NHibernate.Persister.Entity.AbstractEntityPersister.Load(Object id, Object optionalObject, LockMode lockMode, ISessionImplementor session)
at NHibernate.Event.Default.DefaultLoadEventListener.LoadFromDatasource(LoadEvent event, IEntityPersister persister, EntityKey keyToLoad, LoadType options)
at NHibernate.Event.Default.DefaultLoadEventListener.DoLoad(LoadEvent event, IEntityPersister persister, EntityKey keyToLoad, LoadType options)
at NHibernate.Event.Default.DefaultLoadEventListener.Load(LoadEvent event, IEntityPersister persister, EntityKey keyToLoad, LoadType options)
at NHibernate.Event.Default.DefaultLoadEventListener.ProxyOrLoad(LoadEvent event, IEntityPersister persister, EntityKey keyToLoad, LoadType options)
at NHibernate.Event.Default.DefaultLoadEventListener.OnLoad(LoadEvent event, LoadType loadType)
at NHibernate.Impl.SessionImpl.FireLoad(LoadEvent event, LoadType loadType)
at NHibernate.Impl.SessionImpl.Get(String entityName, Object id)
at NHibernate.Impl.SessionImpl.Get(Type entityClass, Object id)
at NHibernate.Impl.SessionImpl.Get[T](Object id)
D:\dev\598\Code\test\unit\DataAccess.Test\NHibernatePersistenceTests.cs
When SQLite is used and the [Rollback] attribute is NOT specified, the test also completes successfully.
Question:
Is this an issue with System.Data.SQLite's implementation of TransactionScope which MbUnit3 uses for [Rollback] or a limitation of the SQLite engine?
Is there some way to write this unit test, working against SQLite, that will rollback so as to avoid affecting the database each time the test is run?
This is not a real answer to you question, but probably a solution to solve the problem.
I use an in-memory implementation of sql lite for my integration tests. I build up the schema and fill the database before each test. The schema creation and initial data filling happens really fast (less then 0.01 seconds per test) because it's an in-memory database.
Why do you use a physical database?
Edit: response to answer about question above:
1.) Because I migrated my schema and data directly from SQL Server 2005 and I want it to persist in source control.
I recommend to store a file with the database schema in and a file or script that creates the sample data in source control. You can generate the file using sql server studion management express, you can generate it from your NHibernate mappings or you can use a tool like sql compare and you can probably find other solutions for this when you need it. Plain text files are stored easier in version control systems then complete binary database files.
2.) Does something about the in-memory SQLite engine differ such that it would resolve this difficulty?
It might solve your problems because you can recreate your database before each test. Your database under test will be in a the state you expect it to be before each test is executed. A benefit of that is there is no need to roll back your transactions, but I have run similar test with in memory sqllite and it worked as aspected.
Check if you're not missing connection.release_mode=on_close in your SQLite NHibernate configuration. (reference docs)
BTW: always dispose your ISession and ISessionFactory.
Ditch [Rollback] and use NDbUnit. I use this myself for this exact scenario and it has been working great.
Related
I am currently attempting to use transactions in my C++ app, but I have a problem with the ODBC's auto commit mode.
I am using the POCO libaries to create a connection to a PostgreSQL database on the same machine. Currently, I can send data to this database as single statements, but I cannot get my head around how to use Poco's transaction libraries to be able to send this data more quickly.
As I have several thousand records to insert, and so continuing to use single insert statements is extrememly slow and inpractical - So I am trying to use Poco's transaction to speed this up a bit (a fair bit).
The error I am encountering is a theoretically a simple one - Poco is throwing the following error:
'Invalid access: Session is in auto commit mode.'
I understand, as a result of this, I should somehow set "auto commit" to false - as it only allows me to commit data to the database line by line, rather than as a single transaction.
The problem is how I set this.
Currently, I have a session created from Session.h, that looks alot like this:
session = new Poco::Data::Session(
"ODBC",
connection_data.str()
);
Where connection data is a simple stringstream with the login information, password, database, server and "Driver={PostgreSQL ANSI};" to tell ODBC to utilize PostgreSQL's driver.
I have tried just setting a property "autocommit" to false through the session's setFeature or setProperty settings, this, of course, was to no avail. (it was more of a ditch attempt at this point).
session->setFeature("AUTOCOMMIT", false);
Looking around, I saw a possible alternative method by creating a ODBC sessionImpl directly from ODBC/session/SessionImpl.h instead of using this generic method above, and then creating a new session object from this.
The benefits of this are that ODBC's sessionImpl has references to autocommit mode in the header, which would suggest it would be able to handle this:
void autoCommit(const std::string&, bool val);
/// Sets autocommit property for the session.
However, having not used sessionImpl before, I cannot garuntee if this will work or if can can get this to work with the limited documentation available.
I am using C++ 03 (Not 11), with Visual Studio 2015
Poco 1.7.5
Boost (Where needed)
Would any one know the correct way of setting this feature (above) or a alternative method to achieving this?
edit: Looking at the source of poco, at:
https://github.com/pocoproject/poco/blob/develop/Data/ODBC/src/SessionImpl.cpp#L153
The property seems be named autoCommit, and looking at
https://github.com/pocoproject/poco/blob/develop/Data/include/Poco/Data/AbstractSessionImpl.h#L120
the case of the property names seem to matter. So, does it help if you use session->setFeature("autoCommit", false);?
Cant you just call session->begin(); and session->end(); on the corresponding Session object?
What is returned by session->canTransact()?
According to the doc begin() will start a new transaction, the doc does not mention any property that needs to be set before or after.
See: https://pocoproject.org/docs/Poco.Data.Session.html
Also faced a similar issue.
First of all before begin() need:
m_ses.setFeature("autoCommit", false);
m_ses.begin();
And the second issue is that this feature stays "autoCommit" in false for all other sessions. So don't forget for the next session call
session.setFeature("autoCommit", true);
We use H2 database to execute tests. To isolate each test from another one, the database schema and basic data-setup is dropped and re-created before each test.
Is it possible to create a restore-point after the first setup of the database and restore before each test the data of this point?
SCRIPT just creates a sql-file with all tables and datas. Not a big difference to our own initialization.
Question database restore to particular state for testing is the same, just for Oracle and Postgres.
An old question, but I find it is still relevant. AFAIK there is no restore-point support.
Here is a simple, yet fast approach to backup/restore.
Create a backup prior to running the first test:
Connection conn = DriverManager.getConnection("jdbc:h2:mem:myDatabase;DB_CLOSE_DELAY=-1;LOG=0");
Statement stat = conn.createStatement();
stat.execute("SCRIPT TO 'memFS:myDatabase.sql'");
stat.close();
conn.close();
Restore after each test:
Connection conn = DriverManager.getConnection("jdbc:h2:mem:myDatabase;DB_CLOSE_DELAY=-1;LOG=0");
Statement stat = conn.createStatement();
stat.execute("DROP ALL OBJECTS");
stat.close();
conn.close();
conn = DriverManager.getConnection("jdbc:h2:mem:myDatabase;DB_CLOSE_DELAY=-1;INIT=runscript from 'memFS:myDatabase.sql';LOG=0");
conn.close();
Note that SHUTDOWN command turned out to be faster than DROP ALL OBJECTS, but it caused some issues (connection pool unable to reestablish connection).
I would not say the above approach is slow, far from it. But with a large database and thousands of tests there is still room for improvement as the method above takes some time. I managed to achieve a few times faster backup/restore (~15ms for a DB with ~350 tables) manually composing a script performing TRUNCATE TABLE, ALTER SEQUENCE and do the INSERT of all initial data (needs SET REFERENTIAL_INTEGRITY FALSE for cleanup/restore procedure to be really fast). The code is cumbersome but was worth the effort.
I'm doing research on unit testing in PLSQL. I set up a test database with some tables and packages with functions and procedures. Currently I'm giving the test framework 'utPLSQL' a try but stumbled upon an error when testing on a ref cursor. I can run all of my tests but the result of the test on the ref cursor says "ora-01031 insufficient privileges", that's all I get. How can I find the source of this error? Or does anyone encountered the same problem? The installation of utPLSQL was successful and all the other functionality of the test framework works.
This is the procedure I want to test:
FUNCTION F_Get_Customers_RefCurs(P_LASTNAME IN VARCHAR2)
RETURN cust_refcur
IS
cust_result cust_refcur;
BEGIN
OPEN cust_result FOR
SELECT *
FROM CUSTOMERS
WHERE LASTNAME = P_LASTNAME
ORDER BY email ASC;
return(cust_result);
END F_Get_Customers_RefCurs;
I have declared cust_refcur in the spec of the package which contains my function as following:
TYPE cust_refcur IS REF CURSOR;
And this is the test:
PROCEDURE ut_F_Get_Customers_RefCurs
IS
params utplsql_util.utplsql_params;
BEGIN
utPLSQL_Util.reg_In_Param (1,
'Tester',
params);
UTASSERT.eq_refc_query ('Get customers on last name is successful (refcursor)',
'PK_ORDERS.F_GET_CUSTOMERS_REFCURS',
params,
0,
'SELECT customerid, firstname, lastname, email, password
FROM CUSTOMERS
WHERE LASTNAME = ''Tester''
ORDER BY email ASC');
END;
I tried getting your example to work, but unfortunately, I got weird errors from utPLSQL.
Since the last version of utPLSQL on Sourceforge is from 2005 and Steven Feuerstein is now working on a commercial product that essentially does the same, I'd recommend looking into other solutions for unit testing your PL/SQL code - some links:
Oracle SQL Developer has some built-in unit test-functionality, and it's free
there's also Quest code tester (this one's commercial)
maybe cause by 'RETURN cust_refcur',the type use by utplsql is store in varchar2(10) ,
try 'return refcur' ?
-zhaozb
When running this assertion (eq_refc_query), utPLSQL needs to temporarily create a table. It does this by using EXECUTE IMMEDIATE which requires that the user have the CREATE TABLE privilege granted to them directly, rather than via a role.
[Full disclosure: I am one of the administrators of the utPLSQL project]
I found a strange problem while doing some unit-testing on Magento.
I have a test function which test a value from core_config_data table. So in order to have access to that value for test, in the setUp() function I am saving the config value in database:
public function setUp()
{
parent::setUp();
$systemConfig = new Mage_Core_Model_Config();
$systemConfig->saveConfig(
'my/custom/path/config',
12
);
}
and in my test method I am getting that value from database likeso:
$productsNo = Mage::getStoreConfig(my/custom/path/config);
but its value is null, and not a string as expected.
This is strange, because if I am refreshing the database after running the test, the value it's existing in database. And if I'm running the test again, the test will work and the value it's not null anymore.
What am I doing wrong? I don't save the value correctly, or I don't fetch it in a good way?
You only save the configuration value to the database, but you don't refresh the config cache, which is used by Mage::getStoreConfig().
To achieve saving and refreshing at the same time you could use:
Mage::app()->getConfig()
->saveConfig('my/custom/path/config', 12)
->reinit();
This way the configuration value will be available in the current and subsequent requests.
In case you don't really need persistence, that is, if you only need this configuration value for the current request, than I'd rather recommend to use:
Mage:app()->getStore()
->setConfig('my/custom/path/config', 12);
How can i write unitintegration tests that talk to a database. e.g.:
public int GetAppLockCount(DbConnection connection)
{
string query :=
"SELECT"+CRLF+
" tl.resource_type AS ResourceType,"+CRLF+
" tl.resource_description AS ResourceName,"+CRLF+
" tl.request_session_id AS spid"+CRLF+
"FROM sys.dm_tran_locks tl"+CRLF+
"WHERE tl.resource_type = 'APPLICATION'"+CRLF+
"AND tl.resource_database_id = ("+CRLF+
" SELECT dbid"+CRLF+
" FROM master.dbo.sysprocesses"+CRLF+
" WHERE spid = ##spid)";
IRecordset rdr = Connection.Execute(query);
int nCount = 0;
while not rdr.EOF do
{
nCount := nCount+1;
rdr.Next;
}
return nCount;
}
In this case i am trying to exorcise this code of bugs (the IRecordset returns empty recordset).
[UnitTest]
void TestGetLockCountShouldAlwaysSucceed();
{
DbConnection conn = GetConnectionForUnit_IMean_IntegrationTest();
GetAppLockCount(conn);
CheckTrue(True, "We should reach here, whether there are app locks or not");
}
Now all i need is a way to connect to some database when running a unit integration testing.
Do people store connection strings somewhere for the test-runner to find? A .ini or .xml or .config file?
Note: Language/framework agnostic. The code intentionally contains elements from:
C#
Delphi
ADO.net
ADO
NUnit
DUnit
in order to drive that point home.
Now all i need is a way to connect to some database when running a unit integration testing.
Either use an existing database or an in-memory database. I've tried both an currently use an existing database that is splatted and rebuilt using Liquibase scripts in an ant file.
Advantages to in-memory - no dependencies on other applications.
Disadvantages - Not quite as real, can take time to start up.
Advantages to real database - Can be identical to the real world
Disadvantages - Requires access to a 3rd party machine. More work setting up a new user (i.e. create new database)
Do people store connection strings somewhere for the test-runner to find? A .ini or .xml or .config file?
Yeap. In C# I used a .config file, in java a .props file. With in-memory you can throw this into the version control as it will be the same for each person, with a real database running somewhere it will need to be different for each user.
You will also need to consider seed data. In Java I've used dbUnit in the past. Not the most readable, but works. Now I use a Ruby ActiveRecord task.
How do you start this? First can you rebuild your database? You need to be able to automate this before going to far down this road.
Next you should build up a blank local database for your tests. I go with one-per-developer, some other teams share but don't commit. In a .NET/MS SQL world I think in memory would be quite easy to do.