Do we have any tools to unit testing Sybase databases around?
i am not able to find one in http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks
i am looking out for a tool that can allow me to unit test Stored procedure.Something like wee do with utPLSQL.This tool will be used by Database guys to unit test.
What exactly do you want to test? The correctness of Sybase itself or some stored procedures you've written that run in Sybase? If the latter, then you could use JUnit to execute the stored procedures via JDBC (I'm assuming based on your username that Java is a language you're comfortable with).
I've been looking in to SqlUnit for xUnit level testing of stored procedures and such. It seems promising. I'm installing and fiddling with it now.
Please try to use AnyDbTest first, (www.anydbtest.com). I think it is the very tool you are finding. even through the AnyDbTest is not support sybase at this moment. We are working to support Sybase these days.
Related
By backend I mean software systems that consume data, process files or communicate using machine interfaces (REST, SOAP, CORBA, etc...). No fancy web or UI testing is necessary. I have in mind Cucumber and Robotframework but I don't know how well suited they are in the task at hand.
There isn't an easy answer to this question.
Sounds like you got your domain right... because if you were trying to test UI/Web components acceptance testing frameworks make not be the right tool for the job.
You have a few options
SpecFlow
Cucumber
StoryTeller
FitNesse
mSpec
a few others.
I'm partial to StoryTeller & mSpec.. but each of these tools have their pros/cons.
Questions you should be asking yourself (and your team are)
Who is going to be writing/maintaining the tests?
Are self documenting acceptance tests provide value for your organization?
Which technology would integrate most easily with your current build process?
I have used Cucumber to test a batch application written in perl and plsql, an informatica transformation, and am currently using it to test a telephony ivr/queueing system. Ruby provided the gems I needed to drive the system and it was very easy for the testers to learn the language/syntax.
If it comes to Robot Framework I have an experience with bad support in Intellij for this. I'm using IntelliBot plugin. In many cases you will know your mistake like wrong syntax, cannot find keyword etc. late in test runtime.
Problems with finding imported libraries, no debug, no simple refactor like rename keyword across a project.
I recently got PHPUnit working with xDebug for testing my Zend Framework applications. The fact I use ZF shouldn't make any difference to this question- just mentioned it for completeness.
Anyway, it all works fine, but now I want to set up an in-memory database using PDO SQLite. I have done this successfully and have created tables and inserted data. However this task seemed to take ages, the syntax from my Export did not match SQLites needs so I had to play around for a while.
Also SQLite does not support constraints (is that right?) which my application does use, so the whole process seems a waste of time if I cannot test my constraints.
Is using SQLite the right solution to my problem? Does anyone have any better ways of using it or any other DB solution to unit testing?
The idea of unit tests is to test smaller parts, so one way could be to work with small amounts of (static) sample data, for example as described in http://www.phpunit.de/manual/3.4/en/database.html
If you really need to test against a full database with all it's constraints, I think there is no way around just using the specific database of your application, for example MySQL.
We use NHibernate generated schema to run unit tests against a database (integration tests I guess they are). I wondered if it was feasible to compare the generated schema against our development database. This would tell us when we had misspelt column names in our mappings or other issues like that. It would also go a long way toward keeping keys and the like consistent across the two.
Is this kind of automated compare feasible? How is the best way to go about doing it?
If you are unable to find a solution using nhibernate, you could look into something like RedGate's SQL Compare tool. This tool makes it incredibly easy to perform comparisons on different databases and see the schema differences. They also have a software development kit that allows you to leverage the power of SQL Compare in your own applications (something I have not yet gotten into, but would love to if the need ever arose).
When unit testing, is it a must to use a database when testing CRUD operations?
Can sql lite help with this? Do you have to cre-create the db somehow in memory?
I am using mbunit.
No. Integrating an actual DB would be integration testing. Not unit testing.
Yes you could use any in-memory DB like SQLite or MS SQL Compact for this if you can't abstract (mock) your DAL/DAO in any other way.
With this in mind I have to point out, that unit testing is possible all the way to DAL, but not DAL itself. DAL will have to be tested with some sort of an actual DB in integration testing.
As with all complicated question, the answer is: It depends :)
In general you should hide your data access layer behind an interface so that you can test the rest of the application without using a database, but what if you would like to test the data access implementation itself?
In some cases, some people consider this redundant since they mostly use declarative data access technologies such as ORMs.
In other cases, the data access component itself may contain some logic that you may want to test. That can be an entirely relevant thing to do, but you will need the database to do that.
Some people consider this to be Integration Tests instead of Unit Tests, but in my book, it doesn't matter too much what you call it - the most important thing is that you get value out of your fully automated tests, and you can definitely use a unit testing framework to drive those tests.
A while back I wrote about how to do this on SQL Server. The most important thing to keep in mind is to avoid the temptation to create a General Fixture with some 'representative data' and attempt to reuse this across all tests. Instead, you should fill in data as part of each test and clean it up after.
When unit testing, is it a must to use a database when testing CRUD operations?
Assuming for a moment that you have extracted interfaces round said CRUD operations and have tested everything that uses said interface via mocks or stubs. You are now left with a chunk of code that is a save method containing a bit of code to bind objects and some SQL.
If so then I would declare that a "Unit" and say you do need a database, and ideally one that is at least a good representation of your database, lest you be caught out with vender specific SQL.
I'd also make light use of mocks in order to force error conditions, but I would not test the save method itself with just mocks. So while technically this may be an integration test I'd still do it as part of my unit tests.
Edit: Missed 2/3s of your question. Sorry.
Can sql lite help with this?
I have in the past used in memory databases and have been bitten as either the database I used and the live system did something different or they took quite some time to start up. I would recommend that every developer have a developer local database anyway.
Do you have to cre-create the db somehow in memory?
In the database yes. I use DbUnit to splatter data and manually keep the schema up to date with SQL scripts but you could use just SQL scripts. Having a developer local database does add some additional maintenance as you have both the schema and the datasets to keep up to data but personally I find is worth while as you can be sure that database layer is working as expected.
As others already pointed out, what you are trying to achieve isn't unit testing but integration testing.
Having that said, and even if I prefer unit testing in isolation with mocks, there is nothing really wrong with integration testing. So if you think it makes sense in your context, just include integration testing in your testing strategy.
Now, regarding your question, I'd check out DbUnit.NET. I don't know the .NET version of this tool but I can tell you that the Java version is great for tests interacting with a database. In a few words, DbUnit allows you to put the database in a known state before a test is run and to perform assert on the content of tables. Really handy. BTW, I'd recommend reading the Best Practices page, even if you decide to not use this tool.
Really, if you are writing a test that connects to a database, you are doing integration testing, not unit testing.
For unit testing such operations, consider using some typed of mock-database object. For instance, if you have a class that encapsulates your database interaction, extract an interface from it and then create an inheriting class that uses simple in-memory objects instead of actually connecting to the database.
As mentioned above, the key here is to have your test database in a known state before the tests are run. In one real-world example, I have a couple of SQL scripts that are run prior to the tests that recreate a known set of test data. From this, I can test CRUD operations and verify that the new row(s) are inserted/updated/deleted.
I wrote a utility called DBSnapshot to help integration test sqlserver databases.
If your database schema is changing frequently it will be helpful to actually test your code against a real db instance. People use SqlLite for speedy tests (because the database runs in memory), but this isn't helpful when you want to verify that your code works against an actual build of your database.
When testing your database you want to follow a pattern similar to: backup the database, setup the database for a test, exercise the code, verify results, restore database to the starting state.
The above will ensure that you can run each test in isolation. My DBSnapshot utility will simplify your code if your writing it .net. I think its easier to use than DbUnit.NET.
I am introducing automated integration testing to a mature application that until now has only been manually tested.
The app is Windows based and talks to a MySQL database.
What is the best way (including details of any tools recommended) to keep tests independent of each other in terms of the database transactions that will occur?
(Modifications to the app source for this particular purpose are not an option.)
How are you verifying the results?
If you need to query the DB (and it sounds like you probably do) for results then I agree with Kris K, except I would endeavor to rebuild the DB after every test case, not just every suite.
This helps avoid dangerous interacting tests
As for tools, I would recommend CppUnit. You aren't really doing unit tests, but it shouldn't matter as the xUnit framework should give you the set up and teardown framework you'll need to automatically set up your test fixture
Obviously this can result in slow-running tests, depending on your database size, population etc. You may be able to attach/detach databases rather than dropping/rebuilding.
If you're interested in further research, check out XUnit Test Patterns. It's a fine book and a good website for this kind of thing.
And thanks for automating :)
Nick
You can dump/restore the database for each test suite, etc. Since you are automating this, it may be something in the setup/teardown functionality.
I used to restore the database in the SetUp function of the database related unit test class. This way it was ensured that each test runs under the same conditions.
You may consider to prepare special database content for the tests, i.e. with less data than the current production version (to keep the restore times reasonable).
The best environment for such testing, I believe, is VMWare or an equivalent. Set up your database, transaction log and so on, then record the whole lot - database as well as configuration. Then to re-test, reload the image and database and kick off the tests. This still requires maintenance of the tests as the system changes, but at least the tests are repeatable, which is one of your greatest challenges in integration testing.
For test automation, many people use Perl, but we've found that Perl programs grow like Topsy and become convoluted. The use of Python as a scripting language (we run C++ tests) is worthwhile if you're trying to build a series of structured tests.
As #Kris K. says dumping and restoring the database between each test will probably be the way to go.
Since you are looking at doing testing external to the App I would look to build the testing framework in a language where you can take advantage of better testing tools.
If you built the testing framework in Java you could take advantage of JUnit and potentially even something like FitNesse.
Don't think that just because the application under test is C++ that means you are stuck using C++ for your automated testing.
Please try AnyDbTest, I think it is the very tool you are finding. (www.anydbtest.com).
Features:
1.Writing test case with Xml, not Java/C++/C#/VB code. Not need those expensive programming tools.
2.Supports all popular databases, such as Oracle/SQL Server/My SQL
3.So many kinds of assertion supported, such as StrictEqual, SetEqual, IsSupersetOf, Overlaps, and RecordCountEqual etc. Plus, most of assertions can prefix logic not operator.
4.Allows using an Excel spreadsheet/Xml as the source of the data for the tests. As you know, Excel spreadsheet is to easily create/edit and maintain the test data.
5.Supports sandbox test model, if one test will be done in sandbox, all database operations on each DB will be rolled back meaning any changes will be undone.
6.Allows performing data pump from one database/Excel into target database in testing initialization and finalization phase. This is easy way to prepare the test data for testing.
7.Unique cross-different-type-database testing, which means target and reference result set can come from two databases, even one is SQL Server, another is Oracle.
8.Set style comparison for recordset. AnyDbTest will tell you what is the intersection, or surplus or absence between the two record sets.
9.Sequential style comparison for recordset or scalar values. It means the two result set will be compared in their original sequence.
10.Allow to export result set of SQL statement into Xml/Excel file.