I'm confused to say the least. I might be getting Mocks wrong, Test Drivers wrong or maybe both.
My understanding of Mocks is that it's a way of imitating units, to avoid depending on specific parts of your system when testing.
My understanding of Test Drivers is that it's specifically used for Bottom-up integration testing, to imitate High Level Modules, which haven't been implemented yet.
To me that makes Test Drivers sound like a type of Mock, but are Mocks specifically a unit testing thing?
What's the difference between the two?
I understand it in the following way:
Mock:
You have standard layered application, e.g. Controller->Service->Repository. You want to unit test it. So you are using mocks and injecting Repository Mock into Service, so you can isolate Service testing.
Test Driver:
You have same application, Controller->Service->Repository. You did all unit tests and now you want to test Repository Level. It doesn't make much sense to use Mocks during Repository testing because they will not show if your sql queries are correct. So it's time for integration tests. Also you don't want to test entire system ( which may consist of 100 services ), you want to isolate your application, so you are spinning up, let's say, docker container with database ( or embedded database, like h2) and running your queries on this database for testing only.
Same example is applicable for other services clients, e.g. you want to get something by HTTP, and instead of calling real service, you spin up MOCK SERVER, e.g. wiremock.
Drivers are the modules that act as temporary replacement for a calling module and give the same output as that of the actual product.
Drivers are also used when the software needs to interact with an external system and are usually complex than stubs.
To recap:
Mocks are small in memory objects, that have very simple configuration and they used during unit testing for isolation of classes.
Test drivers are more complicated, they are used during integration testing to replace components into your infrastructure for isolation of entire application.
Related
I read a lot about unit testing and integration testing, and the unit testing part I understand fairly well - isolate the object under test, and mock out dependencies using interfaces and inject those. (Or use seams to inject testable behavior).
However something that is still a mystery to me even after searching is integration testing. Every blog and link talks about testing various components together, do's and dont's, CI Servers, etc, but there's not much explanation in the way of how this is to be done.
Is an integration test automated? Or is this a manual effort? If it is automated, do I write this as code in the native language my app is in? How do I check or verify if the integration test is working as expected?
For example, if I have 4 services (a Socket Client, a Socket Server, a Database, and a Web Application) and I want to do some integration testing on how these 4 services interact with each other. How exactly how would I approach this? I know that some dummy data will be involved, but which part of my integration test is checking if the systems are working together correctly? This part is really unclear to me.
As you said about unit testing, you need to isolate the object under test, and mock out dependencies using interfaces and inject those...
Integration test is without isolating the unit... You can test two part or more (but you shouldn't have extensive integration test suits neither big test scenario). An example of integration test is testing your code with the database, you could need for that to initiate the database and clean it to have a repeatable tests.
You could have as well interface test (client execution and integration with the server for example) and so on.
Don't forget that this tests come with a downside, they are slower and harder to maintain, but they have different purpose than unit test, which is checking that units works as expected with real dependencies.
So to wrap-up if you write a test without totally isolating the unit it is an integration test, due to its nature it's better to test the logic of your code by unit test and reserve fewer integration tests when needed to test the interaction between the units.
You can also check this nice introduction https://softwareengineering.stackexchange.com/questions/48237/what-is-an-integration-test-exactly
An interesting definition from "the pragmatic programmer": integration tests show that the major parts of a system work well together
I'm trying to add unit-test and integration-test for my grails application but I have some trouble how to distinguish between both and when to use unit or integration to test my controllers actions and services.
The tutorial I found online is not very clean. I can't find complete example to follow up.
Can you please share helpful topics?
I follow the following guidelines:
Try writing as many unit tests as you can. They can be written for controllers, services, domain classes or any other groovy classes. The idea is that unit tests are friends for developers. Writing enough unit tests will
make sure that the developer makes lesser mistakes. As they execute
quickly, this means quick verification. But unit tests cannot test the following:
Criteria queries, HQL queries
Actual database Interactions (queries, transactional behaviour, updates, db constraints etc.)
Inter modular interactions
So we write the Integration tests as well
Integration tests take longer to execute. Writing Integration tests often need bootstrapping data. But they really are helpful to test functionalities end to end (excluding the actual user interactions through UI for which functional tests are written). So Integration Tests can be written for:
Testing all database interactions as unit tests actually do not test the database interactions. This also includes testing criteria, hql etc.,
Testing transactional behaviour (which is dependent on db)
Testing implementations end to end. So this will also test how two independently created modules interact with each other and make sure we have created them correctly.
One problem with integration tests is their speed. For me, integration tests take 15+ seconds to start up. In that time, certain things do slip out of mind focus.
I prefer to go with unit tests that start in no more then 2 sec and can be run several times in those 15 seconds.
One more argument for unit tests is they force you to decouple your code. Integration tests always tempt you to just rely on some other component existing and initialized.
Important links:
http://spockframework.org/spock/docs/1.0/interaction_based_testing.html
http://docs.grails.org/latest/guide/testing.html
Unfortunately it is not just a matter preference or speed. It is a huge subject, but I can give you some advice based on my experience.
If you expect to be covering your database access code (queries, transactional behaviour) by using unit tests, you are deluding yourself. You are testing how your queries comply with the in-memory implementation of GORM. Not hibernate, not your database.
I usually have two types of tests. Unit and functional tests. The functional tests will perform a full test, running against a real database, and stimulating the system like a user would (if it is a web site via Geb, if it is a REST api, via a REST client).
The functional tests will set up a startup state by executing some kind of fixture code first. This can be registering a user and logging them in, for example. Then the test will run, and then the postconditions are checked. Here, you can check the postconditions either by accessing the database through the GORM API, or by using production API calls (danger of covering a bug with another bug).
Sometimes, your system will interact with a third system. Here, if you can, you can mock the implementation of the third system, by injecting a mock implementation into the system under test.
You have also tools like Spring Cloud Contract, that allow you to create bock a mock server for your system under test, and a specification for your third-party system. See https://cloud.spring.io/spring-cloud-contract/spring-cloud-contract.html
The unit tests, I use to thoroughly test all execution paths of a given class. I will try to trigger all exception states, all secondary scenarios, to make sure that everything is covered. I don't think it is realistic to have 100% coverage by using functional or integration tests.
Testing the results of the endpoints is straight forward.
However, shouldn't I also have a look at the db layer, if (i.e.) the data I POSTed is saved correctly? I'm unclear how/if the business logic behind the REST call should be tested.
In my perfect world (someday I'd like to live there...) there are several different tests involved in fully validating a RESTful service I am providing.
There are unit tests for each of the logical layers of the application, that use dependency injection to mock the next lower layer and validate that each unit works correctly. This might be a unit that marshals/unmarshals the parameters and response, a unit that executes business logic, and a unit that manages persistence. (There could be multiple units, each with their own tests at each layer.) These tests have no dependencies outside the testing framework, with the exception of the persistence layer which has to exercise your persistence implementation.
There are also integration tests that require a running system. This is where you call the running service, and verify that you got the expected response. You may also inspect side effects of the call. On my team, we often do that by making a different service call (or calls) that relies on the result of the first call. That exercises more of the system. We find that direct inspection of the persistence layer for side effects seldom tells us much that we can't get by using a different service call.
Yes, you should test the db layer at some point but probably not in the same unit test than the one that test the result of the endpoint.
At the Unit Test level you would like to have small/quick/independant tests so you might end up with a test/suite for the REST part and another one focussing on db layer.
Then you might have some more integration/end-to-end tests that check the whole system : doing the request and checking the db is correctly updated.
My own experience, working as a QA, is that the tests involving DB are done with integration/end-to-end tests with a functional perspective by the QA using a third-party tool (we use Robot Framework for that).
One way of doing it is use the restful service itself to test that the data correctly persisted,
For example you can use the PUT message to persist an entity and use the GET operation to retrieve it, and make sure all properties are equal. This is more of an integration test which covers end to end.
Especially your application is doing more CRUD operations, and if you use this approach you could avoid creating tests for each and every layer (such as repositories)
since the dao layer methods will be dependent on data in the database,
in complex systems some operations will depend on lots of tables,
in this way unit test are not repeatable and independent,
i'm wondering how good TDD layers do this? thx.
IMHO you are on to something... If you are communicating with your Sql Server, then you are not doing unit test but integration tests. If you do TDD, then you realise this and start to put server communication into wrappers, so you can stub and mock any test data, instead of using a framework like DbUnit to control database state. I think that Your business logic should not be directly in touch with databases -- or webservices or other external resources. If it is, odds are that you will never write anything but integration tests.
A testing framework like DbUnit is exactly what you want. From their site:
DbUnit is a JUnit extension (also usable with Ant) targeted at database-driven projects that, among other things, puts your database into a known state between test runs. This is an excellent way to avoid the myriad of problems that can occur when one test case corrupts the database and causes subsequent tests to fail or exacerbate the damage.
DbUnit also supports a variety of RDBMS's but I might recommend something like HSQLDB, which can be embedded into your project/tests so your unit tests aren't dependent on being able to connect to a database somewhere in your company's basement. :) Although, on the other hand, you will be testing using a different RDBMS than you would be using in a production environment...
Just putting this one out for debate really.
I get unit testing. Sometimes feels time consuming but I'm all for the benefits.
I've an application set up that contains a repository layer and a service layer, using IoC, and I've been unit testing the methods.
Now I know the benefits of isolating my methods for unit testing so there is little or no dependency on other methods.
The question I've got is this. If I only ever access my repository methods through my service layer methods would only testing the service layers not be good enough? I'm testing against a test database.
Could it not be considered an extension of the idea that you only need to test your public methods? Maybe I'm just trying to skip some testing ;)
Yes, you should test your repository layer. Although the majority of these tests fall into a different classification of tests. I usually refer to them as integration tests to distinguish them from my unit tests. The difference being that there is an external dependency on a resource (your database) and that these tests will likely take much longer to run.
The primary reason for testing your repositories separately is that you'll be testing different things. The repository is responsible for handling translation and interaction with whatever persistence store you're using. The service layer, on the other hand, is responsible for coordinating your various respositories and other dependencies into functionality that represents business logic, which likely involves more than just a relay to a repository method and in some instances may involve multiple calls to multiple repositories.
First, to clarify the service layer testing - when testing the service layer, the repositories should be mocked so that they are isolated from what you're testing in the service layer. As you pointed out in your comment, this gives you a more granular level of testing and isolates the code under test. Your unit tests will also run much faster now because there are no database connections slowing them down.
Now, here are a few advantages of adding integration tests to your repositories...
It allows you to test out those pieces of code as you're writing them, a la TDD.
It ensures that whatever persistence language you're using (SQL, HQL, serialized objects, etc.) is formulated correctly for the operation you're attempting to perform.
If you're using an object-relational mapper, it ensures that your mappings are defined correctly.
In the future, you may find that you need to support another type of persistence. Depending on how your repository tests are structured, you may be able to reuse a large number of the tests to verify that the new database schema works correctly. For repository methods that implement database specific logic, obviously you'll have to create separate tests.
When coupled with Continuous Integration it's nice to have the repository tests separated. Integration tests, by nature take longer to run than unit tests. As such, they're usually run at less frequent intervals so that the immediate feedback available from running unit tests is not delayed.
Those are all advantages that I've seen in various projects that I've worked on. There may be more.
All that having been said, I will admit that I'm not as thorough with the repository integration tests as I am with unit tests. When it comes to testing an update on a particular object, for example, I'm usually content testing that one database column was successfully updated rather than creating a separate test for each individual column or a larger test that verifies every column in one test. For me, it depends on the complexity of the operation that the respository method is performing and whether there's any special condition that needs to be isolated.
You should test your repository layer. However if you have integration, story or system tests that cover it, then you can make a good case of not having unit tests as well.
Unit testing is great for complex stand-a-lone objects, but there is no point spending a long time writing unit tests for simple methods that are covered by “higher level” tests.
Wouldn't this depend on how how smart the repository access layer is? If your repository takes parameters to filter (Linq to SQL for example) the given result set surely this logic will need to be tested.
Unit tests: test an individual logic (a method) without worrying the dependency of that logic. Mostly falls in white box category.
Integration test: can test end to end flow or more than one layer together to ensure its correctness. Mostly falls in black box category.
In Dao most of the time there is no business logic, it just forms a query for a particular database implementation. So no need for a unit test if we already covered it in our integration test. Still, we can write unit tests for Dao if there is some logic in it.
As dao layers are so tightly coupled with database implementation, most of the time junit test for dao has become synonyms for testing of underlying databases.
The query we build can only be validated by the underlying Database engine.
I used to write unit tests (can call integration tests) for dao by using actual database or mocking a database with a compatible database(follow the same sql standard ,for example mysql engine can be replaced by sqlite or in memory H2 database) and inject this database in dao for testing the dao layer and query build in that dao layer.
I get unit testing
Next step is Test Driven Development (TDD). It will answer your question.