I am working on some application in Java and writing JUnit tests. I have a design question about Unit Testing. I have one class that reads a file and Create object called Song by reading different lines and parsing based on some algorithm. I have written some unit test on that. Next step after parsing is to actually convert that song to a different format based on some properties of Song object. I have another class that works as a translator. There is a method translate that takes Song object as input. Now in unit test for translator. I need a Song object with all valid properties. I am confused here that should I create a new Song object by putting same functionality as in parser or should I call the parser service to do that for me. I feel it will not be isolated if I take the second option. But in first option it's like duplicate code. Can somebody guide me on this?
There's nothing wrong in using a Builder in order to create the input data for a SUT invocation when this data is complex, however I see 2 risks here.
If the builder fails your test will fail too, but it shouldn't. As you said unit tests should be isolated from external code.
If you use code coverage as a metric to evaluate how good your unit tests are (I don't mean this is right), by looking at the builder's coverage you'll be tempted to think it's tested though obviously isn't.
My opinion is there's not a best solution fitting all the scenarios. In case the input data is not very complex try to build it "manually", otherwise use the builder.
Related
Dynamics AX 2012 comes with unit testing support.
To have meaningful tests some test data needs to be provided (stored in tables in the database).
To get a reproducable outcome of the unit tests we need to have the same data stored in the tables every time the tests are run. Now the question is, how can we accomplish this?
I learned that there is the possibility of setting the isolation level for the TestSuite to SysTestSuiteCompanyIsolateClass. This will create an empty company and delete the company after the tests have been run. In the setup() method I can fill my testdata into the tables with insert statements. This works fine for small scenarios but becomes cumbersome very fast if you have a real life project.
I was wondering if there is anyone out there with a practical solution of how to use the X++ Unit Test Framework in a real world scenario. Any input is very much appreciated.
I agree that creating test data in a new and empty company only works for fairly trivial scenarios or scenarios where you implemented the whole data structure yourself. But as soon as existing data structures are needed, this approach can become very time consuming.
One approach that worked well for me in the past is to run unit tests in a existing company that already has most of the configuration data (e.g. financial setup, inventory setup, ...) needed to run the test. The test itself runs in a ttsBegin - ttsAbort block so that the unit test does not actually create any data.
Another approach is to implement data provider methods that are test agnostic, but create data that is often used in unit tests (e.g. a method that creates a product). It takes some time to create a useful set of data provider methods, but once they exist, writing unit tests becomes a lot faster. See SysTest part V.: Test execution (results, runners and listeners) on how Microsoft uses a similar approach (or at least they used to back in 2007 for AX 4.0).
Both approaches can also be combined, you would call the data provider methods inside the ttsBegin - ttsAbort block to create the needed data only for the unit test.
Another useful method is to use doInsert or doUpdate to create your test data, especially if you are only interested in a few fields and do not need to create a completely valid record.
I think that the unit test framework was an afterthought. In order to really use it, Microsoft would have needed to provide unit test classes, then when you customize their code, you also customize their unit tests.
So without that, you're essentially left coding unit tests that try and encompass base code along with your modifications, which is a huge task.
Where I think you can actually use it is around isolated customizations that perform some function, and aren't heavily built on base code. And also with customizations that are integrations with external systems.
Well, from my point of view, you will not be able to leverage more than what you pointed from the standard framework.
What you can do is more around release management. You can setup an integration environment with the targeted data and push your nightbuild model into this environmnet at the end of the build process and then run your tests.
Yes, it will need more effort to set it up and to maintain but it's the only solution I've seen untill now to have a large and consistent set of data to run unit or integration tests on.
To have meaningful tests some test data needs to be provided (stored
in tables in the database).
As someone else already indicated - I found it best to leverage an existing company for data. In my case, several existing companies.
To get a reproducable outcome of the unit tests we need to have the
same data stored in the tables every time the tests are run. Now the
question is, how can we accomplish this?
We have built test helpers, that help us "run the test", automating what a person would do - give you have architeced your application to be testable. In essence our test class uses the helpers to run the test, then provides most of the value in validating the data it created.
I learned that there is the possibility of setting the isolation level
for the TestSuite to SysTestSuiteCompanyIsolateClass. This will create
an empty company and delete the company after the tests have been run.
In the setup() method I can fill my testdata into the tables with
insert statements. This works fine for small scenarios but becomes
cumbersome very fast if you have a real life project.
I did not find this practical in our situation, so we haven't leveraged it.
I was wondering if there is anyone out there with a practical solution
of how to use the X++ Unit Test Framework in a real world scenario.
Any input is very much appreciated.
We've been using the testing framework as stated above and it has been working for us. the key is to find the correct scenarios to test, also provides a good foundation for writing testable classes.
I have developed an application with Spring MVC that deals with bulk data insert/update. For ex: There are some use cases that insert a record with 100-125 attributes.
For bulk data insert, I'm hard coding the values to be inserted in my Unit test class. I have Transfer Object to carry the data, so i'm populating these TOs in my Unit test cases and then calling the desired operation to be tested. For every DAO, at least 4 test cases are needed for unit testing CRUD operations.
I'm finding it very hard to populate the TOs by hard coding the values in my test case source file. Imaging writing 125 setters for every unit test case. I want to populate my TOs dynamically by reading the data from an XML file or any kind of media, so that I need not change the hard coded data for test cases everytime.
Setting up the data in an XML file is much easier than hard coding the values in JUNIT source files.
I could think of some innovative solutions like setting up data in XML file and then using any JAXB implementation to read the same and populate the TOs.. but i believe there much be some easy and better way to handle this kind of requirement.
Need some help on this.
Imaging writing 125 setters for every
unit test case.
You need test helper method(s) that take care of test data. Then you call appropriate method that populates a TO: it could be as little as 0 parameters (completely random/fixed, not test driven) or as many as 125 (completely controlled by the test) (and anything in between).
Either way no setters in tests anymore.
Maybe you could use this framework:
http://jtestcase.sourceforge.net
I am not sure about the Java unit testing frameworks. But in DotNet there is a library called NBuilder which helps you to create test data very easily and quckly. Here is a blog I had written to demonstrate its capabilities.
May be you can try and see if there exists an Java alternative to NBuilder or there is a port of the same available in Java. Most of the DotNet libraries I find are ported from Java world like NHibernate, Spring.Net etc. So my guess is that you might have a Java equivalent of NBuilder as well.
Looking for some strategies for how you guys are loading default data when doing unit tests.
I use a builder that contains the default values, just like this: http://elegantcode.com/2008/04/26/test-data-builders-refined/. Then the test only specifies the value it cares for:
Customer customer = new CustomerBuilder()
.WithFirstName("this test only cares about a special ' ... first name value");
After reading the other answers, I want to clear that its not for database data. Its to build the instances/data you pass to the classes you are unit testing.
Its a matter of convenience/keeping the tests simple, plenty of times you are testing very specific behavior that depends on 1-3 fields and you don't care about the rest of the fields.
For unit testing I generally don't load data in advance - each test is designed to work against a data source that may or may not already contain existing records, and so each test writes all any records that are needed to complete the test.
When choosing values to submit to the database I use GUIDs (or other random values) whenever possible as it guarantees that values in the database are unique (e.g. if you create someone named "Mr X Y", it is helpful to know that searching for "X" should return only 1 result, and that there is no chance you have chanced on someone else in the database whose last name happens to be Y)
Often when unit testing I'm testing methods that modify data alongside methods that read data, and so my unit tests use the same API (the one being tested) to write to the database. (It's nice if each unit test covers a specific area of functionality, but it's not absolutely necessary)
If the API being tested doesn't have methods to write to the database however, I write my own set of helper functions - the exact structure is going to depend on the data source, but as an example this is where I often use LINQ to SQL.
TDD is about testing a piece of code in isolation. One create an instance of a class with its dependencies (or mocks of them), call the method under test and assert to verify the outcome of the test.
Usually with TDD one starts with a simple test, without data. When data are needed, they are created in the test fixture (the isolated environment where the test is executed) by the test setUp() method and then destroyed by the tearDown() method after the test has been run. Data are not loaded from the database.
Preferred strategy is in-transaction data. Spring offers extensive support (for both JUnit 3 and 4). With this strategy your test begins brand new transaction each time and your data is rolled back at the end of test.
Of course sometimes it's not enough: either data set is too extensive and shared across tests, or multiple transactions are part of the test scope. In that case, I recommend creating shared test data bed that is created before running test suite. There are frameworks for this (dbUnit) but you can also do without them if careful and consistent.
UPD: creating in-transaction data doesn't mean you not need test data, you are likely to end up creating re-usable and shared helper classes to maintain test data in all cases.
I typically have methods like GetCustomer() that return a generic customer. If I need to make the returned customer suite my needs for a particular test, I will simply change the property after it gets returned.
Other times I may pass some configuration information into my GetCustomer() method. For example GetCustomer(string customerType).
I've read expert's opinions that says that each test should contain its own unique data to work with and not try to make the data generic. Even though this may make each test "larger" in size, over all it will make the test clearer because the setup is specific to each test and the goals of each test. I like this advice because I've ran into many cases where trying to make the setup data generic, made things very sloppy very quick.
I am a heavy advocate of proper Test Driven Design or Behavior Driven Design and I love writing tests. However, I keep coding myself into a corner where I need to use 3-5 mocks in a particular test case for a single class. No matter which way I start, top down or bottom up I end up with a design that requires at least three collaborators from the highest level of abstraction.
Can somebody give good advice on how to avoid this pitfall?
Here's a typical scenario. I design a Widget that produces a Midget from a given text value. It always starts really simple until I get into the details. My Widget must interact with several hard to test things like file systems, databases, and the network.
So, instead of designing all that into my Widget I make a Bridget collaborator. The Bridget takes care of one half of the complexity, the database and network, allowing me to focus on the other half which is multimedia presentation. So, then I make a Gidget that performs the multimedia piece. The entire thing needs to happen in the background, so now I include a Thridget to make that happen. When all is said and done I end up with a Widget that hands work to a Thridget which talks over a Bridget to give its result to a Gidget.
Because I'm working in CocoaTouch and trying to avoid mock objects I use the self-shunt pattern where abstractions over collaborators become protocols that my test adopts. With 3+ collaborators my test balloons and become too complicated. Even using something like OCMock mock objects leaves me with an order of complexity that I'd rather avoid. I tried wrapping my brain around a daisy-chain of collaborators (A delegates to B who delegates to C and so on) but I can't envision it.
Edit
Taking an example from below let's assume we have an object that must read/write from sockets and present the movie data returned.
//Assume myRequest is a String param...
InputStream aIn = aSocket.getInputStram();
OutputStream aOut = aSocket.getOutputStram();
DataProcessor aProcessor = ...;
// This gets broken into a "Network" collaborator.
for(stuff in myRequest.charArray()) aOut.write(stuff);
Object Data = aIn.read(); // Simplified read
//This is our second collaborator
aProcessor.process(Data);
Now the above obviously deals with network latency so it has to be Threaded. This introduces a Thread abstraction to get us out of the practice of threaded unit tests. We now have
AsynchronousWorker myworker = getWorker(); //here's our third collaborator
worker.doThisWork( new WorkRequest() {
//Assume myRequest is a String param...
DataProcessor aProcessor = ...;
// Use our "Network" collaborator.
NetworkHandler networkHandler = getNetworkHandler();
Object Data = networkHandler.retrieveData(); // Simplified read
//This is our multimedia collaborator
aProcessor.process(Data);
})
Forgive me for working backwards w/o tests but I'm about to take my daughter outside and I'm rushing thru the example. The idea here is that I'm orchestrating the collaboration of several collaborators from behind a simple interface that will get tied to a UI button click event. So the outter-most test reflects a Sprint task that says given a "Play Movie" button, when it is clicked, the movie will play.
Edit
Lets discuss.
Having many mock objects shows that:
1) You have too much dependencies.
Re-look at your code and try to break it further down. Especially, try to separate data transformation and processing.
Since I don't have experience in the environment you are developing in. So let me give my own experience as example.
In Java socket, you will be given a set of InputStream and OutputStream simple so that you can read data from and send data to your peer. So your program looks like this:
InputStream aIn = aSocket.getInputStram();
OutputStream aOut = aSocket.getOutputStram();
// Read data
Object Data = aIn.read(); // Simplified read
// Process
if (Data.equals('1')) {
// Do something
// Write data
aOut.write('A');
} else {
// Do something else
// Write another data
aOut.write('B');
}
If you want to test this method, you have to ends up create mock for In and Out which may require quite a complicated classes behind them for supporting.
But if you look carefully, read from aIn and write to aOut can be separated from processing it. So you can create another class which will takes the read input and return output object.
public class ProcessSocket {
public Object process(Object readObject) {
if (readObject.equals(...)) {
// Do something
// Write data
return 'A';
} else {
// Do something else
// Write another data
return 'B';
}
}
and your previous method will be:
InputStream aIn = aSocket.getInputStram();
OutputStream aOut = aSocket.getOutputStram();
ProcessSocket aProcessor = ...;
// Read data
Object Data = aIn.read(); // Simplified read
aProcessor.process(Data);
This way you can test the processing with little need for mock. you test can goes:
ProcessSocket aProcessor = ...;
assert(aProcessor.process('1').equals('A'));
Becuase the processing is now independent from input, output and even socket.
2) You are over unit testing by unit test what should be integration tested.
Some tests are not for unit testing (in the sense that it require unnecessarily more effort and may not efficiently get a good indicator). Examples of these kind of tests are those involving concurrency and user interfaces. They require different ways of testing than unit testing.
My advice would be that you further break them down (similar to the technique above) until some of them are unit-test suitable. So you have the little hard-to-test parts.
EDIT
If you believe you already broken it into very fine pieces, perhaps, that is your problem.
Software components or sub-components are related to each other in some way like characters are combined to words, words are combined to sentences, sentences to paragraphs, paragraphs to subsection, section, chapters and so on.
My example says, your should broken subsection to paragraphs and you things you already downs to words.
Look at it this way, most of the time, paragraphs are related to other paragraphs in a less loosely degree than sentences related (or depends on) other sentences. Subsection, section are even more loosely while words and characters are more dependent (as the grammatical rules kick in).
So perhaps, you are breaking it so fine that the language syntax force to those dependencies and in turn forcing you to have so much mock objects.
If that is the case, your solution is to balance the test. If a part are depended by many and it is require a complex set of mock object (or simple more effort to test it). May be you don't need to test it. For example, If A uses B,C uses B and B is so damn hard to test. So why don't you just test A+B as one and C+B as anther. In my example, if SocketProcessor is so hard to test, too hard to the point that you will spend more time testing and maintain the tests more than developing it then it is not worth it and I will just test the whole things at once.
Without seeing your code (and with the fact that I am never develop CocaoTouch) it will be hard to tell. And I may be able to provide good comment here. Sorry :D.
EDIT 2
See your example, it is pretty clear that you are dealing with integration issue. Assuming that you already test play movie and UI separatedly. It is understandable why you need so much mock objects. If this is the first time you use these kind of integration structure (this concurrent pattern), then those mock objects may actually be needed and there is nothing much you can do about it. That's all I can say :-p
Hope this helps.
My solution (not CocoaTouch) is to continue to mock the objects, but to refactory mock set up to a common test method. This reduces the complexity of the test itself while retaining the mock infrastructure to test my class in isolation.
I do some fairly complete testing, but it's automated integration testing and not unit-testing, so I have no mocks (except the user: I mock the end-user, simulating user-input events and testing/asserting whatever's output to the user): Should one test internal implementation, or only test public behaviour?
What I'm looking for is best practices using TDD.
Wikipedia describes TDD as,
a software development technique that
relies on the repetition of a very
short development cycle: First the
developer writes a failing automated
test case that defines a desired
improvement or new function, then
produces code to pass that test and
finally refactors the new code to
acceptable standards.
It then goes on to prescribe:
Add a test
Run all tests and see if the new one fails
Write some code
Run the automated tests and see them succeed
Refactor code
I do the first of these, i.e. "very short development cycle", the difference in my case being that I test after it's written.
The reason why I test after it's written is so that I don't need to "write" any tests at all, even the integration tests.
My cycle is something like:
Rerun all automated integration tests (start with a clean slate)
Implement a new feature (with refactoring of the existing code if necessary to support the new feature)
Rerun all automated integration tests (regression testing to ensure that new development hasn't broken existing functionality)
Test the new functionality:
a. End-user (me) does user input via the user interface, intended to exercise the new feature
b. End-user (me) inspects the corresponding program output, to verify whether the output is correct for the given input
When I do the testing in step 4, the test environment captures the user input and program output into a data file; the test environment can replay such a test in the future (recreate the user input, and assert whether the corresponding output is the same as the expected output captured previously). Thus, the test cases which were run/created in step 4 are added to the suite of all automated tests.
I think this gives me the benefits of TDD:
Testing is coupled with development: I test immediately after coding instead of before coding, but in any case the new code is tested before it's checked in; there's never untested code.
I have automated test suites, for regression testing
I avoid some costs/disadvantages:
Writing tests (instead I create new tests using the UI, which is quicker and easier, and closer to the original requirements)
Creating mocks (required for unit testing)
Editing tests when the internal implementation is refactored (because the tests depend only on the public API and not on the internal implementation details).
To get rid of excessive mocking you can follow the Test Pyramid which suggests having a lot of unit + component tests and a smaller number of slow & fragile system tests. It boils down to several simple rules:
Write tests at the lowest possible level that wouldn't require mocking. If you can write a unit test (e.g. parsing a string), then write it. But if you want to check whether the parsing is invoked by the upper layer, then this would require initializing more of the stuff.
Mock external systems. Your system needs to be a self-contained, independent piece. Relying on external apps (which would have their own bugs) would complicate testing a lot. Writing mocks/stubs is much easier.
After that have couple of tests checking your app with real integrations.
With this mindset you eliminate almost all mocking.
I have a simple project, mostly consisting of back-end service code. I have this fully unit-tested, including my DAL layer...
Now I have to write the front-end. I re-use what business objects I can in my front-end, and at one point I have a grid that renders some output. I have my DAL object with some function called DisplayRecords(id) which displays the records for a given ID...
All of this DAL objects are unit tested. But is it worth it to write a unit test for the DisplayRecords() function? This function is calling a stored proc, which is doing some joins. This means that my unit-test would have to set-up multiple tables, one with 15 columns, and its return value is a DataSet (this is the only function in my DAL that returns a datset - because it wasnt worth it to create an object just for this one grid)...
Is stuff like this even worth testing? What about front-end logic in general - do people tend to skip unit tests for the ASP.NET front-end, similar to how people 'skip' the logic for private functions? I know the latter is a bit different - testing behavior vs implementation and all... but, am just curious what the general rule-of-thumb is?
Thanks very much
There are a few things that weigh into whether you should write tests:
It's all about confidence. You build tests so that you have confidence to make changes. Can you confidently make changes without tests?
How important is this code to the consumers of the application? If this is critical and central to everything, test it.
How embarrassing is it if you have regressions? On my last project, my goal was no regressions-- I didn't want the client to have to report the same bug twice. So every important bug got a test to reproduce it before it was fixed.
How hard is it to write the test? There are many tools that can help ease the pain:
Selenium is well understood and straightforward to set up. Can be a little expensive to maintain a large test suite in selenium. You'll need the fixture data for this to work.
Use a mock to stub out your DAL call, assuming its tested elsewhere. That way you can save time creating all the fixture data. This is a common pattern in testing Java/Spring controllers.
Break the code down in other ways simply so that it can be tested. For example, extract out the code that formats a specific grid cell, and write unit tests around that, independent of the view code or real data.
I tend to make quick Selenium tests and just sit and watch the app do its thing - that's a fast validation method which avoids all the manual clicking.
Fully automated UI testing is tedious and should IMO only be done in more mature apps where the UI won't change much. Regarding the 'in-between' code, I would test it if it is reused and/or complicated/ introducing new logic, but if its just more or less a new sequence of DAL method calls and specific to a single view I would skip it.