How to integration test an object with database queries - unit-testing

How can i write unitintegration tests that talk to a database. e.g.:
public int GetAppLockCount(DbConnection connection)
{
string query :=
"SELECT"+CRLF+
" tl.resource_type AS ResourceType,"+CRLF+
" tl.resource_description AS ResourceName,"+CRLF+
" tl.request_session_id AS spid"+CRLF+
"FROM sys.dm_tran_locks tl"+CRLF+
"WHERE tl.resource_type = 'APPLICATION'"+CRLF+
"AND tl.resource_database_id = ("+CRLF+
" SELECT dbid"+CRLF+
" FROM master.dbo.sysprocesses"+CRLF+
" WHERE spid = ##spid)";
IRecordset rdr = Connection.Execute(query);
int nCount = 0;
while not rdr.EOF do
{
nCount := nCount+1;
rdr.Next;
}
return nCount;
}
In this case i am trying to exorcise this code of bugs (the IRecordset returns empty recordset).
[UnitTest]
void TestGetLockCountShouldAlwaysSucceed();
{
DbConnection conn = GetConnectionForUnit_IMean_IntegrationTest();
GetAppLockCount(conn);
CheckTrue(True, "We should reach here, whether there are app locks or not");
}
Now all i need is a way to connect to some database when running a unit integration testing.
Do people store connection strings somewhere for the test-runner to find? A .ini or .xml or .config file?
Note: Language/framework agnostic. The code intentionally contains elements from:
C#
Delphi
ADO.net
ADO
NUnit
DUnit
in order to drive that point home.

Now all i need is a way to connect to some database when running a unit integration testing.
Either use an existing database or an in-memory database. I've tried both an currently use an existing database that is splatted and rebuilt using Liquibase scripts in an ant file.
Advantages to in-memory - no dependencies on other applications.
Disadvantages - Not quite as real, can take time to start up.
Advantages to real database - Can be identical to the real world
Disadvantages - Requires access to a 3rd party machine. More work setting up a new user (i.e. create new database)
Do people store connection strings somewhere for the test-runner to find? A .ini or .xml or .config file?
Yeap. In C# I used a .config file, in java a .props file. With in-memory you can throw this into the version control as it will be the same for each person, with a real database running somewhere it will need to be different for each user.
You will also need to consider seed data. In Java I've used dbUnit in the past. Not the most readable, but works. Now I use a Ruby ActiveRecord task.
How do you start this? First can you rebuild your database? You need to be able to automate this before going to far down this road.
Next you should build up a blank local database for your tests. I go with one-per-developer, some other teams share but don't commit. In a .NET/MS SQL world I think in memory would be quite easy to do.

Related

C++ use Stored Procedure to return results

For the past couple years, I've been maintaining a large C++ application (v100) that utilizes some form of non-ADO database connections, but it works great.
During this time, getting a resultset from the database is quite simple. I instantiate the return class, with the database object, then Open a query.
CUpdates cUpdates(GetDatabase());
CString strQuery = "SELECT * FROM Updates";
cUpdates.Open(-1, strQuery);
Just that simple, cUpdates is filled with records.
NOW however, we want to execute a stored procedure, and return the results from it. But no matter what I try, even changing 'EXEC' to 'CALL', the call fails. Is there a similar simple method for executing a stored procedure, and returning the results, without having to totally rewrite how the application handles the database connection and returning of data?
strQuery.Format("EXEC dbo.[GetUpdates_ComputerName] '%s', %d, %d", GetWorkstationName(), m_bRetainUpdates, m_bScheduleUpdate);
cUpdates.Open(-1, strQuery); //FAILS ON EXEC
(I have tested the EXEC statement in SSMS, and it works fine)
We do also use another sql command, for strictly executing statements, but I see no way of returning data with it. Maybe there is a similar command I don't know of?
GetDatabase()->ExecuteSQL(strQuery);
note: for the record, I am C# developer (since 1.0 beta). My only experience in c++ has been learning on the fly over the past 2 years, occasionally maintaining a few of our massive systems.
It would seem that CRecordset cannot handle an EXEC statement inside of it. So we converted the new stored procedure to a Tabular Function, so I can use a SELECT instead... which works properly. (though I'd rather use the stored procedure)

C++ wrapper class SQLite advice

I'm learning C++ and have a question about classes and wrappers. I'm writing an application for a raspberry pi. I have a class called SensorClass whose methods read data from various sensors attached to the board.
class SensorClass {
public:
SensorClass();
virtual ~SensorClass();
int getTemperature();
int getPressue();
;
I want to write the data to a local sqlite database when it is read. On the SQLite website there are a number of wrapper classes.
SQLite wrappers
I'm wondering if I should use one of these to for example insert data into the database when it has been read.
I'm thinking then I would be separating the code and just calling for example the SQLite insert method in the getTemperature() function. Would this be a good idea? Which wrapper should I use?
Example SQlite wrapper class
Alternatively I could hard code the database operations in the getTemperature() method like this.
int SensorClass::getTemperature(){
// read temperature
//insert into database
/* Create SQL statement */
sql = "INSERT INTO DATAPOINTS (Temperature) " \
"VALUES (15); " \
/* Execute SQL statement */
rc = sqlite3_exec(db, sql, callback, 0, &zErrMsg);
}
Thanks for your advice
It would generally be better to separate the two things. i.e. make the sensor class do the job on sensing stuff well and only that.
Then have a separate class that does the job of logging sensor data to the database well. You may find it is better to insert entire rows into the database in one go. And you may also decide that you want to only log data periodically at a fixed sampling rate.
Then in your main application loop / via an event driven timer, you can do measurements and record data as separate steps.
e.g.
void APP_tick(void)
{
SensorValues values = sensors.readValues();
logger.writeValues(values);
}
By separating responsibility, you can then change the logger class out easily - you may end up deciding that you don't want to use a database and would rather just log the data into flat files in order to use less disk space and improve performance.
If using SQLite then you might find it worthwhile using prepared statements to avoid having to compile the SQL query every time you execute it (which is expensive in CPU terms and you are running this on a fairly limited system).

Connecting to ACCDB using QT5

What I'm trying to do is find all .db (Paradox) files from a directory and save them to .mdb. The thing is, to try if this works properly I've tried first opening an ACCDB (.db and .mdb I'll be using are resources only available in my school's internal network).
I've already checked, and I've found the sql drivers in the Qt directory, I've also added QT +=sql in my pro file, I've installed the access database engine, VBA is also installed, but I can't manage to open the connection. I'm yet to try connecting to a SQLITE database to try if it works, but I'm guessing it will. Here is the code, triggered from a button:
db.addDatabase("QODBC");
explorador.setFileMode(QFileDialog::Directory);
ruta=explorador.getExistingDirectory(this,"Seleccionar directorio");
directorio.setPath(ruta);
subdir=directorio.entryList(QDir::AllDirs);
//La lista comienza en el 2ยบ elemento
for(int i=2;i<subdir.size();i++)
{
subruta=ruta+"/"+subdir.at(i);
directorio.setPath(subruta);
db.setDatabaseName("DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ="+subruta+"/Base de datos11.accdb");
bool check=db.open();
if(check)
{
dbg.setText("Prueba");
dbg.show();
}
else
{
dbg.setText("Pruebaaa");
dbg.show();
}
}
Have any ideas?? Thank you!
I eventually solved it kind of bypassing through access. If someone else is struggling with this you need:
-Paradox DB engine (Be sure the version you download matches your .db file)
-Jet engine 4.0 or later
-access 2007 (Following version have removed Paradox DB connection)
-Administrator privileges to access the database engine folders(both jet and paradox)
Simply open the database through a connection or database object. Sintax should be something like this (Via a VBA module in access):
database.open(path,*Dont rememer what goes in here,*here goes the read/write permits,"Paradox 5.x")
Path should be pointing to a folder containing one or more .db files, which will be opened as a a single database with the files being tables. It actually genereates a new database (Either .accdb or .mdb, depending how you save it)
It is not exactly like this, but I'm on my laptop. I'll edit the answer once I'm in front of my code
Sorry it is not a c++ answer, but from what I've found it is kind of a pain in the a** to work with paradox databases. Hope this helps others with the struggle.
Ricardo

H2 database restore to particular state for testing

We use H2 database to execute tests. To isolate each test from another one, the database schema and basic data-setup is dropped and re-created before each test.
Is it possible to create a restore-point after the first setup of the database and restore before each test the data of this point?
SCRIPT just creates a sql-file with all tables and datas. Not a big difference to our own initialization.
Question database restore to particular state for testing is the same, just for Oracle and Postgres.
An old question, but I find it is still relevant. AFAIK there is no restore-point support.
Here is a simple, yet fast approach to backup/restore.
Create a backup prior to running the first test:
Connection conn = DriverManager.getConnection("jdbc:h2:mem:myDatabase;DB_CLOSE_DELAY=-1;LOG=0");
Statement stat = conn.createStatement();
stat.execute("SCRIPT TO 'memFS:myDatabase.sql'");
stat.close();
conn.close();
Restore after each test:
Connection conn = DriverManager.getConnection("jdbc:h2:mem:myDatabase;DB_CLOSE_DELAY=-1;LOG=0");
Statement stat = conn.createStatement();
stat.execute("DROP ALL OBJECTS");
stat.close();
conn.close();
conn = DriverManager.getConnection("jdbc:h2:mem:myDatabase;DB_CLOSE_DELAY=-1;INIT=runscript from 'memFS:myDatabase.sql';LOG=0");
conn.close();
Note that SHUTDOWN command turned out to be faster than DROP ALL OBJECTS, but it caused some issues (connection pool unable to reestablish connection).
I would not say the above approach is slow, far from it. But with a large database and thousands of tests there is still room for improvement as the method above takes some time. I managed to achieve a few times faster backup/restore (~15ms for a DB with ~350 tables) manually composing a script performing TRUNCATE TABLE, ALTER SEQUENCE and do the INSERT of all initial data (needs SET REFERENTIAL_INTEGRITY FALSE for cleanup/restore procedure to be really fast). The code is cumbersome but was worth the effort.

Using MbUnit3's [Rollback] for unit testing NHibernate interactions with SQLite

Background:
My team is dedicated to ensuring that straight from checkout, our code compiles and unit tests run successfully. To facilitate this and test some of our NHibernate mappings, we've added a SQLite DB to our repository which is a mirror of our production SQL Server 2005 database. We're using the latest versions of: MbUnit3 (part of Gallio), System.Data.SQLite and NHibernate.
Problem:
I've discovered that the following unit test does not work with SQLite, despite executing without trouble against SQL Server 2005.
[Test]
[Rollback]
public void CompleteCanPersistNode()
{
// returns a Configuration for either SQLite or SQL Server 2005 depending on how the project is configured.
Configuration config = GetDbConfig();
ISessionFactory sessionFactory = config.BuildSessionFactory();
ISession session = sessionFactory.OpenSession();
Node node = new Node();
node.Name = "Test Node";
node.PhysicalNodeType = session.Get<NodeType>(1);
// SQLite fails with the exception below after the next line called.
node.NodeLocation = session.Get<NodeLocation>(2);
session.Save(node);
session.Flush();
Assert.AreNotEqual(-1, node.NodeID);
Assert.IsNotNull(session.Get<Node>(node.NodeID));
}
The exception I'm getting (ONLY when working with SQLite) follows:
NHibernate.ADOException: cannot open connection --->
System.Data.SQLite.SQLiteException:
The database file is locked database is locked
at System.Data.SQLite.SQLite3.Step(SQLiteStatement stmt)
at System.Data.SQLite.SQLiteDataReader.NextResult()
at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteCommand cmd, CommandBehavior behave)
at System.Data.SQLite.SQLiteCommand.ExecuteReader(CommandBehavior behavior)
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery()
at System.Data.SQLite.SQLiteTransaction..ctor(SQLiteConnection connection, Boolean deferredLock)
at System.Data.SQLite.SQLiteConnection.BeginDbTransaction(IsolationLevel isolationLevel)
at System.Data.SQLite.SQLiteConnection.BeginTransaction()
at System.Data.SQLite.SQLiteEnlistment..ctor(SQLiteConnection cnn, Transaction scope)
at System.Data.SQLite.SQLiteConnection.EnlistTransaction(Transaction transaction)
at System.Data.SQLite.SQLiteConnection.Open()
at NHibernate.Connection.DriverConnectionProvider.GetConnection()
at NHibernate.Impl.SessionFactoryImpl.OpenConnection()
--- End of inner exception stack trace ---
at NHibernate.Impl.SessionFactoryImpl.OpenConnection()
at NHibernate.AdoNet.ConnectionManager.GetConnection()
at NHibernate.AdoNet.AbstractBatcher.Prepare(IDbCommand cmd)
at NHibernate.AdoNet.AbstractBatcher.ExecuteReader(IDbCommand cmd)
at NHibernate.Loader.Loader.GetResultSet(IDbCommand st, Boolean autoDiscoverTypes, Boolean callable, RowSelection selection, ISessionImplementor session)
at NHibernate.Loader.Loader.DoQuery(ISessionImplementor session, QueryParameters queryParameters, Boolean returnProxies)
at NHibernate.Loader.Loader.DoQueryAndInitializeNonLazyCollections(ISessionImplementor session, QueryParameters queryParameters, Boolean returnProxies)
at NHibernate.Loader.Loader.LoadEntity(ISessionImplementor session, Object id, IType identifierType, Object optionalObject, String optionalEntityName, Object optionalIdentifier, IEntityPersister persister)
at NHibernate.Loader.Entity.AbstractEntityLoader.Load(ISessionImplementor session, Object id, Object optionalObject, Object optionalId)
at NHibernate.Loader.Entity.AbstractEntityLoader.Load(Object id, Object optionalObject, ISessionImplementor session)
at NHibernate.Persister.Entity.AbstractEntityPersister.Load(Object id, Object optionalObject, LockMode lockMode, ISessionImplementor session)
at NHibernate.Event.Default.DefaultLoadEventListener.LoadFromDatasource(LoadEvent event, IEntityPersister persister, EntityKey keyToLoad, LoadType options)
at NHibernate.Event.Default.DefaultLoadEventListener.DoLoad(LoadEvent event, IEntityPersister persister, EntityKey keyToLoad, LoadType options)
at NHibernate.Event.Default.DefaultLoadEventListener.Load(LoadEvent event, IEntityPersister persister, EntityKey keyToLoad, LoadType options)
at NHibernate.Event.Default.DefaultLoadEventListener.ProxyOrLoad(LoadEvent event, IEntityPersister persister, EntityKey keyToLoad, LoadType options)
at NHibernate.Event.Default.DefaultLoadEventListener.OnLoad(LoadEvent event, LoadType loadType)
at NHibernate.Impl.SessionImpl.FireLoad(LoadEvent event, LoadType loadType)
at NHibernate.Impl.SessionImpl.Get(String entityName, Object id)
at NHibernate.Impl.SessionImpl.Get(Type entityClass, Object id)
at NHibernate.Impl.SessionImpl.Get[T](Object id)
D:\dev\598\Code\test\unit\DataAccess.Test\NHibernatePersistenceTests.cs
When SQLite is used and the [Rollback] attribute is NOT specified, the test also completes successfully.
Question:
Is this an issue with System.Data.SQLite's implementation of TransactionScope which MbUnit3 uses for [Rollback] or a limitation of the SQLite engine?
Is there some way to write this unit test, working against SQLite, that will rollback so as to avoid affecting the database each time the test is run?
This is not a real answer to you question, but probably a solution to solve the problem.
I use an in-memory implementation of sql lite for my integration tests. I build up the schema and fill the database before each test. The schema creation and initial data filling happens really fast (less then 0.01 seconds per test) because it's an in-memory database.
Why do you use a physical database?
Edit: response to answer about question above:
1.) Because I migrated my schema and data directly from SQL Server 2005 and I want it to persist in source control.
I recommend to store a file with the database schema in and a file or script that creates the sample data in source control. You can generate the file using sql server studion management express, you can generate it from your NHibernate mappings or you can use a tool like sql compare and you can probably find other solutions for this when you need it. Plain text files are stored easier in version control systems then complete binary database files.
2.) Does something about the in-memory SQLite engine differ such that it would resolve this difficulty?
It might solve your problems because you can recreate your database before each test. Your database under test will be in a the state you expect it to be before each test is executed. A benefit of that is there is no need to roll back your transactions, but I have run similar test with in memory sqllite and it worked as aspected.
Check if you're not missing connection.release_mode=on_close in your SQLite NHibernate configuration. (reference docs)
BTW: always dispose your ISession and ISessionFactory.
Ditch [Rollback] and use NDbUnit. I use this myself for this exact scenario and it has been working great.