Note that the question is not asking unit tests of functional code/service VS integration tests of functional code/service. The question is asking: does integration tests itself need to be unit tested as well?
For example, we have a functional code/service A, and we we have unit tests B against those functional code/service A, and we also have integration tests C against those functional code/service A too, do we need to write unit tests D against our integration tests C?
Related
I am wondering whether it would be a good practice to unit tests the unit tests, otherwise, how do we know that the unit tests are without bugs?
Unit test cases are predictable, based on code logic. Which means that you should know the expected result of this test case in advance. All test cases are designed by users. There is no need to write unit test for unit test. If the test relies on some packages or helper functions, these test fixtures need to be fully tested.
I am new to unit testing so please can anybody tell me difference b/w Mockito and TestNG. And which tool will be suitable for unit testing.
Both tools are suitable for unit testing and you can use them together. TestNG is similar to JUnit but introducing some new functionalities, and Mockito allows you to add mock objects to your tests.
BDD has been touted as "TDD done right".
However TDD is widely used with unit tests, rather than end-to-end integration tests.
Which kind of test is most appropriate for BDD?
Should we write only integration tests?
Should we also write unit tests?
If yes, should there be multiple unit-tests per scenario?
And what a unit test covers multiple scenarios? Is there a way to structure these tests when using a testing framework such as MSpec?
Which kind of test (integration tests, unit tests) is most appropriate for BDD?
I would use both in two nested loops as described in Behavior-Driven Development with SpecFlow and WatiN
* writing a failing integration tests
* writing a failing unit test as part of the solution of the integration test
* making the unittest pass
* refactor
* writing the next failing unit test as part of the integration test
* unitl the integration test passes
* writing the next failing integration tests
The reason you often see acceptance/integration tests used in the BDD cycle is because many of it's practitioners put a heavy emphasis on mocking and outside-in development. As such they tend need to include both end-to-end integration/acceptance tests as well as unit tests to ensure the total behavior of the system.
When fake objects are being used as collaborators instead of real objects in a unit test, it is quite easy to verify that the isolated unit under test behaves in the proper fashion (that is that it modifies it's state properly, and sends the proper messages). However it can not verify anything more than that in isolation the individual object behaves as expected. As such, an end-to-end acceptance test is then necessary verify that all the objects in the system when used together provide the promised value to the end user.
So basically within this approach unit tests and acceptance tests play the following roles:
Unit Tests - Objects in isolation behave in the proper manner
Acceptance/Integration Tests - All Objects together provide the promised value of the system.
Although I myself am a large proponent of this style of development, it is somewhat independent from the general BDD ideas. In my opinion acceptance/integration tests are only necessary when your isolating your system under test in your unit tests by using mocks and stubs.
I'll give the answer that seems most likely to me, but I might be off-track here.
TDD was originally implemented with unit-tests, which clarify requirements for individual components of the software by calling them and expecting certain results.
BDD simply applies the same concept to the system as a whole. So for BDD, you write integration tests, which call the system from the outside-in, by simulating mouse clicks, etc, or whatever interface the end-user uses to access the system.
BDD seems to actually be a kind of black-box/UAT testing, since it's concerned with the behaviour of the system as a whole, not with the means by which the behaviour is implemented.
So it seems advisable to write unit tests and integration tests, but at the very least, integration tests, so that if we don't have time to verify each component, we can at least verify the system as a whole.
The basic idea is to write the tests before the code, so for any feature, use the test appropriate to examine the feature. If the feature you are adding is "clicking on the report bug link should send an email", then an integration test seems appropriate (presumably with some unit tests for components of that feature). If the feature you are working on is "Average usage calculation should omit the highest and lowest value", a unit test is probably more appropriate. The important thing is to accurately be able to tell when what you are building is done, and later to be sure changes didn't break it. The rest is just bookkeeping.
This question already has answers here:
What is the difference between integration and unit tests?
(21 answers)
Closed 9 years ago.
What's the difference between unit tests and integration tests?
Are there different names for these tests? Like some people calling unit tests functional tests, etc?
A unit test is a test written by the programmer to verify that a relatively small piece of code is doing what it is intended to do. They are narrow in scope, they should be easy to write and execute, and their effectiveness depends on what the programmer considers to be useful. The tests are intended for the use of the programmer, they are not directly useful to anybody else, though, if they do their job, testers and users downstream should benefit from seeing fewer bugs.
Part of being a unit test is the implication that things outside the code under test are mocked or stubbed out. Unit tests shouldn't have dependencies on outside systems. They test internal consistency as opposed to proving that they play nicely with some outside system.
An integration test is done to demonstrate that different pieces of the system work together. Integration tests can cover whole applications, and they require much more effort to put together. They usually require resources like database instances and hardware to be allocated for them. The integration tests do a more convincing job of demonstrating the system works (especially to non-programmers) than a set of unit tests can, at least to the extent the integration test environment resembles production.
Actually "integration test" gets used for a wide variety of things, from full-on system tests against an environment made to resemble production to any test that uses a resource (like a database or queue) that isn't mocked out. At the lower end of the spectrum an integration test could be a junit test where a repository is exercised against an in-memory database, toward the upper end it could be a system test verifying applications can exchange messages.
A unit test should have no dependencies on code outside the unit tested. You decide what the unit is by looking for the smallest testable part. Where there are dependencies they should be replaced by false objects. Mocks, stubs .. The tests execution thread starts and ends within the smallest testable unit.
When false objects are replaced by real objects and tests execution thread crosses into other testable units, you have an integration test
A unit test is done in (as far as possible) total isolation.
An integration test is done when the tested object or module is working like it should be, with other bits of code.
A unit test tests code that you have complete control over whereas an integration test tests how your code uses or "integrates" with some other code.
So you would write unit tests to make sure your own libraries work as intended, and then write integration tests to make sure your code plays nicely with other code you are making use of, for instance a library.
Functional tests are related to integration tests, but refer more specifically to tests that test an entire system or application with all of the code running together, almost a super integration test.
Unit test is usually done for a single functionality implemented in Software module. The scope of testing is entirely within this SW module. Unit test never fulfils the final functional requirements. It comes under whitebox testing methodology..
Whereas Integration test is done to ensure the different SW module implementations. Testing is usually carried out after module level integration is done in SW development.. This test will cover the functional requirements but not enough to ensure system validation.
Like anything else understanding the words makes it much easier to learn the language. Can anyone chime in with all the words used in unit testing with their definitions (ex. Mock, Fixture, etc. )
This looks like a great page: http://xunitpatterns.com/Glossary.html
It includes:
SUT
synchronous test
task
TDD
test automater
test case
test code
test condition
test context
test database
test debt
test driver
test driving
test error
test failure
test fixture
test fixture
test fixture
test maintainer
test package
test reader
test result
test run
test smell
test stripper
test success
test suite
test-driven bug fixing
test-driven development
test-first development
test-last development
test-specific equality
test
In relation to mocking etc this might be useful:
This table and its references might be more useful:
http://xunitpatterns.com/Mocks,%20Fakes,%20Stubs%20and%20Dummies.html
Perhaps these articles will be more helpful:
Wikipedia:
In computer programming, unit testing is a software design and development method where the programmer gains confidence that individual units of source code are fit for use. A unit is the smallest testable part of an application. In procedural programming a unit may be an individual program, function, procedure, etc., while in object-oriented programming, the smallest unit is a method, which may belong to a base/super class, abstract class or derived/child class.
Unit testing can be done by something as simple as stepping through code in a debugger; modern applications include the use of a test framework such as xUnit.
Ideally, each test case is independent from the others; Double objects like stubs, mock or fake objects1 as well as test harnesses can be used to assist testing a module in isolation. Unit testing is typically done by software developers to ensure that the code other developers have written meets software requirements and behaves as the developer intended.
MSDN:
The primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as you expect. Each unit is tested separately before integrating them into modules to test the interfaces between modules. Unit testing has proven its value in that a large percentage of defects are identified during its use.
Extreme Rules:
Unit tests enable collective code ownership. When you create unit tests you guard your functionality from being accidentally harmed. Requiring all code to pass all unit tests before it can be released ensures all functionality always works. Code ownership is not required if all classes are guarded by unit tests.
I've also found a glossary of testing terms, but it doesn't define Mock or Fixture, but there's an option to add new ones. Once the question is answered to your satisfaction maybe that could become the canonical source.
Mock,
n.
A type of turtle, used primarily in soup.
A code construct used in Unit Testing, named after #1. A mock looks like the real thing to the code being tested, however any attempts to interact with it result only in mournful songs.
v.
To construct a mock for use in testing.