Im having a failed integration test because of test pollution (tests pass or fail depending on which order they are run in).
What baffles me a bit however is that it seems that a unit test where i have mocked some data with mockDomain(Media.class,[new Movie(...)]) is still present and available in other tests, even integration tests.
Is this the expected behaviour? why doesn't the test framework clean up after itself for each test?
EDIT
Really strange, the documentation states that:
Integration tests differ from unit tests in that you have full access to the Grails environment within the test. Grails will use an in-memory HSQLDB database for integration tests and clear out all the data from the database in between each test.
However in my integration test i have the following code
protected void setUp() {
super.setUp()
assertEquals("TEST POLLUTION!",0,Movie.count())
...
}
Which gives me the output:
TEST POLLUTION! expected:<0> but was:<1>
Meaning that there is data present when there shouldn't be!
Looking at the data that is present int he Movie.list() i find that the data corresponds to data set in a previous test (unit test)
protected void setUp() {
super.setUp()
//mock the superclass and subclasses as instances
mockDomain(Media.class,[
new Movie(id:1,name:'testMovie')
])
...
}
Any idea's of why im experiencing these issues?
It's also possible that the pollution is in the test database. Check the DataSources.groovy to see what is being used for the test environment. If you have it set to use a database where
dbCreate is set to something other than "create-drop", any previous contents of the database could also be showing up.
If this is the case, the pollution has come from an entirely difference source. Instead of coming from the unit tests, it has actually come from the database, but when switching to run the integration tests you get connected to a real database with all the data it contains.
We experienced this problem, in that our test enviroment was set to have dbCreate as "update". Quite why this was set for integration tests puzzled me, so I switched to use dbCreate as "create-drop" and make sure that when running test suites we started with a clean database.
Related
I don't know whether to use unit or integration tests for what I am wanting to test in my Grails 2.2.3 application. I want to run some tests against like this:
#TestFor(Student)
#Mock(Student)
class StudentTests {
void testFoundStudent() {
def s = Student.findById(myId)
assert s != null
assert s.firstName = 'Grant'
assert s.lastName = 'McConnaughey'
}
}
This is going to require the use of our test database, so would that make it an integration test? When I run this code as a unit test it fails at assert s != null. Which means it isn't using our database, because it SHOULD find a student with that ID.
In Grails unit test you can test domain class interactions using Gorm and behind the scene Grails will use in-memory database(an implementation of a ConcurrentHashMap) to mimic this behavior here. So yes you get null because that student does not exist in in-memory database and you need to insert that data first.
Student.findOrSaveWhere (firstName: 'Grant',lastName : 'McConnaughey')
In your example, if the intention is to test the existence of that data you need to use integration test and connect it to your database using datasource.groovy , which is really not a good idea unless you have a good reason to test your data.
If you are trying to test def s = Student.findById(myId) again that is not adding any value as that is Grails dynamic finder and you probably need to trust the framework you are using.
However, in general
Unit tests are typically run without the presence of physical
resources that involve I/O such databases, socket connections or
files link
I hope this helps
I have created following four tests in a Test class that tests a findCompany() method of a CompanyService.
#Test
public void findCompany_CompanyIdIsZero() {
exception.expect(IllegalArgumentException.class);
companyService.findCompany(0);
}
#Test
public void findCompany_CompanyIdIsNegative() {
exception.expect(IllegalArgumentException.class);
companyService.findCompany(-100);
}
#Test
public void findCompany_CompanyIdDoesntExistInDatabase() {
Company storedCompany = companyService.findCompany(100000);
assertNull(storedCompany1);
}
#Test
public void findCompany_CompanyIdExistsInDatabase() {
Company company = new Company("FAL", "Falahaar");
companyService.addCompany(company);
Company storedCompany1 = companyService.findCompany(company.getId());
assertNotNull(storedCompany1);
}
My understanding says that the first three of these are unit tests. They test the behavior of the findCompany() method, checking how the method will respond on different inputs.
The fourth test, though placed in the same class, actually seems to be an integration test to me. It requires a Company to be added to the database first, so that it can be found later on. This introduces external dependencies - addCompany() and database.
Am I going right? If yes, then how should I unit test finding an existing object? Just mock the service to "find" one? I think that kills the intent of the test.
I appreciate any guidance here.
I look at it this way: the "unit" you are testing here is the CompanyService. In this sense all of your tests look like unit tests to me. Underneath your service, though, there may be another service (you mention a database) that this test is also exercising? This could start to blur the lines with integration testing a bit, but you have to ask yourself if it matters. You could stub out any such underlying service, and you may want to if:
The underlying service is slow to set up or use, making your unit tests too slow.
You want to be sure the behaviour of this test is unaffected by the underlying service - i.e. this test should only fail if there is a bug in CompanyService.
In my experience, provided the underlying service is fast enough I don't worry too much about my unit test relying on it. I don't mind a bit of integration leaking into my unit tests, as it has benefits (more integration coverage) and rarely causes a problem. If it does cause problems you can always come back to it and add stubbing to improve the isolation.
[1,2,3,4] could be unit-based (mocked | not mocked) and integration-based tests. It depends what you want to test.
Why use mocking? As Jason Sankey said ...test only service tier not underlaying tier.
Why use mocking? Your bussiness logic can have most various forms. So you can write several test for one service method, eg. create person (no address - exception, no bank account - exception, person does not have filled not-null attributes - exception).
Can you imagine that each test requested database in order to test all possibility exception states (no adress, no bank account etc.)? There is too much work to fill database in order to test all exception states. Why not to use mocked objects which eg. act like 'crippled' objects which do not contains expected values. Each test construct own 'crippled' mock object.
Mocking various states === your test will be simply as possible because each test method will be test only one state. These test will be clear and easy to understand and maintance. This is one of goals which I want to reach if I write a test.
I have a unit test project which uses Ninject to mock the database repositories. I would like to use these same tests as integration tests and use Ninject to bind my real database repositories back into their respective implementations so as to test/stress the application into the DB.
Is there a way to do this with Visual Studio 2012 or is there another test framework, other than MSTest, which allows for this type of configuration?
I would really hate to rewrite/copy these unit tests into an integration test project but I suspect I could copy the files in as links and have a single test file compiled into two projects (Unit and Integration).
Thanks
Todd
Your requirements sound really odd to me. The difference between a unit test and an integration test is much bigger than just connecting to a database or not. An integration test either has a much bigger scope, or tests if components communicate correctly. When you write a unit test, the scope of such a unit is normally small (one class/component with all dependencies mocked out), which means there is no need for using a DI container.
Let me put it differently. When the tests are exactly the same, why are you interested to do the same test with and without the database. Just leave the database in and just test that. Besides these tests, you can add 'real' unit tests, that have a much smaller scope.
With Nunit you can do this with TestCase,
say you need to use the unit and unit/integration test using CustomerRepository and OrderRepository,
[TestFixture]
public class TestCustomerRepository
{
IKernel _unit;
Ikernel _integration;
[SetUp]
public void Setup()
{
//setup both kernels
}
[TestCase("Unit")]
[TestCase("Integration")]
public void DoTest(String type)
{
var custRepo = GetRepo<ICustomerRepository>(type);
var orderRepo = GetRepo<IOrderRepository>(type);
//do the test here
}
protected T GetRepo<T>(String type)
{
if (type.Equals("Unit"))
{
return _unit.Get<T>();
}
return _integration.Get<T>();
}
}
This is the basic idea.
I've been trying to write some initial NUnit unit tests for MonoRail, having got some basics working already. However, while I've managed to check whether a Flash["message"] value has been set by a controller action, the BaseControllerTest class doesn't seem to store the output for a view at all, so whether I call RenderView or the action itself, nothing gets added to the Response.OutputContent data.
I've also tried calling InPlaceRenderView to try to get it to write to a StringWriter, and the StringWriter also seems to get nothing back - the StringBuilder that returns is also empty.
I'm creating a new controller instance, then calling
PrepareController(controller,"","home","index");
So far it just seems like the BaseControllerTest is causing any output to get abandoned. Am I missing something? Should this work? I'm not 100% sure, because while I'm also running these unit tests in MonoDevelop on Linux, although MonoRails is working OK there.
While I haven't got an ideal method for testing Views, this is possibly less important when ViewComponents can be tested adequately. To test views within the site itself, I can use Selenium. While in theory that can be made part of an NUnit test suite, that didn't run successfully under MonoDevelop in my tests (failing to start the connection to Selenium RC consistently, despite the RC interactive session working fine). However, the Selenium tests can be run as a set from Firefox, which is not too bad - unit testing with NUnit, then Integration/System testing scripting using a Selenium suite, and that setup will work in a Linux/MonoDevelop setup.
As for testing the underlying elements, you can check for redirections and check the flash value set or the like, so that's all fine, and for testing ViewComponents the part-mocked rendering does return the rendered output in an accessible form, so they've proved much easier to test in NUnit (with a base test class of BaseViewComponentTest) as follows:
[Test]
public void TestMenuComponentRendersOK()
{
var mc = new MenuComponent();
PrepareViewComponent(mc);
var dict = new System.Collections.Specialized.ListDictionary();
dict.Add("data",getSampleMenuData());
dict.Add("Name","testmenu");
// other additional parameters
mc.RenderComponent(mc,dict);
Assert.IsTrue(this.Output.Contains(""),"List items should have been added");
}
How would you test the following code?
public IList<T> Find(DetachedCriteria criteria)
{
return criteria.GetExecutableCriteria(session).List<T>();
}
I would like to mock NH implementation (like setting mocks for ISession, ISessionFactory etc.) but I am having trouble with this one.
You shouldn't really test this as that would be testing NHibernate. As a matter of fact, you can see very similar unit tests in NH source code itself.
If you wanted to test some other code that uses this code, here's how you'd stub it:
Db.Stub(x => x.Find(Arg<DetachedCriteria>.Is.Anything))).Return(new List<Blah>{new Blah()});
In my experience, if you want to test your queries (e.g. the ones that build the DetachedCriteria) you are much better of with an in-memory DB like SQLite, or better yet, a real SQL Server instance (or SQL Server CE for in-memory).