Should I write a separate method for any possible parameter value - unit-testing

I am very new to unit testing concept and I stick with writing my first one.
I have a method for normalize ID value. It should return passed value for any positive number (even if it is string with number inside) and zero (0) for any other passed value.
function normalizeId($val) {
// if $val is good positive number return $val;
// else return 0;
}
I want to write a unit test for this function and have assertion to any possible arguments. For example:
5, -5, 0, "5", "-5", 3.14, "fff", new StdClass() etc.
Should I write a method in my TestCase class for any of this condition or have one method with all conditions on separate lines?
I.e.
public function testNormalizeId() {
$this->assertEquals(5, MyClass::normalizeId(5));
$this->assertEquals(0, MyClass::normalizeId(-5));
$this->assertEquals(0, MyClass::normalizeId("fff"));
}
or
public function testNormalizeId_IfPositiveInt_GetPositiveInt() {
$this->assertEquals(5, MyClass::normalizeId(5));
}
public function testNormalizeId_IfNegativeInt_GetZeroInt() {
$this->assertEquals(0, MyClass::normalizeId(-5));
}
public function testNormalizeId_IfNotIntAsString_GetZeroInt() {
$this->assertEquals(0, MyClass::normalizeId("fff"));
}
How about best practices? I hear that the second choice is good but I'm worry about very many methods for very many possible parameter values. It can be positive number, negative number, zero, string with positive number inside, string with negative number inside, string with float inside etc etc.
Edit
Or maybe the third approach with provider?
public function testNormalizeIdProvider()
{
return array(
array(5, 5),
array(-5, 0),
array(0, 0),
array(3.14, 0),
array(-3.14, 0),
array("5", 5),
array("-5", 0),
array("0", 0),
array("3.14", 0),
array("-3.14", 0),
array("fff", 0),
array("-fff", 0),
array(true, 0),
array(array(), 0),
array(new stdClass(), 0),
);
}
/**
* #dataProvider testNormalizeIdProvider
*/
public function testNormalizeId($provided, $expected)
{
$this->assertEquals($expected, MyObject::normalizeId($provided));
}

I'm not very knowledgeable about PHP nor the unit testing frameworks that you can use therein, but in the general sphere of Unit Testing I'd recommend the second approach for these reasons
Gives a specific test case fail for a particular type of input rather than having to trawl through the actual Assert failure message to figure out which one failed.
Makes it much easier to parametrize these tests if you decide that you need to perform tests on a specific type of conversion with more than one input (e.g if you decided to have a text file containing 1,000 random strings and wanted to load these up in a test driver and run the test case for converting strings for each entry by way of functional or acceptance testing later on)
Makes it easier to change out the individual test cases for when you need some special logic to setup
Makes it easier to spot when you've missed a type of conversion because the method names read off easier against a checklist :)
(Dubious) Will maybe make it easier to spot where your "god class" might be in need of internal refactoring to use separate sub-classes to perform specific types of conversions (not saying your approach is wrong but you might find the logic for one type of conversion very nasty; when you review your 20 or 30 individual test cases that could provide the impetus to bite the bullet and develop more specialized converter classes)
Hope that helps.

Use the data provider, as you discovered yourself. There is no benefit in duplicating the exact testcase in multiple methods with only having parameters and expectations change.
Personally, I do really start with the tests all in one method for such simple cases. I'd start with a simple good case, and then gradually adding more cases. I may not feel the need to change this into a data provider instantly, because it won't pay off instantly - but on the other hand things can change, and this test structure can be a short term solution that needs refactoring.
So whenever you observe yourself adding more lines of test into such a multi test case method, stop and make it using a data provider instead.

Related

What are some of the rules of thumb to split a test into multiple tests?

Okay, so I have this piece of code:
public TbMtUserDTO RecoverUser(long userId, UpdateTbMtUserDTO updatedData)
{
TbMtUser user = _usersRepository.FindUserById(userId);
if (user == null ||
(updatedData.IdRecSet == "Password" && String.IsNullOrEmpty(updatedData.DsUpdatedPassword)))
{
return null;
}
switch (updatedData.IdRecSet)
{
case "Username":
return _mapper.Map<TbMtUserDTO>(user);
case "Password":
user.DsPassword = PasswordHasher.Hash(updatedData.DsUpdatedPassword);
_usersRepository.SaveChanges();
return _mapper.Map<TbMtUserDTO>(user);
}
throw new InvalidOperationException(
String.Format(RECOVER_USER_ERROR, updatedData.IdRecSet));
}
And while I was writing the test cases for that chunk of code, when I had to write the test for one of the "Password" case, this is what I did:
[Fact]
public void UpdatesPasswordSuccessfully()
{
string oldPassword = _user.DsPassword;
UpdateTbMtUserDTO updateTbMtUserDto = new UpdateTbMtUserDTO()
{
IdRecSet = "Password",
DsUpdatedPassword = "new_password"
};
_usersRepositoryMock
.Setup(x => x.FindUserById(It.IsAny<long>()))
.Returns(_user);
_mapperMock
.Setup(x => x.Map<TbMtUserDTO>(It.IsAny<TbMtUser>()))
.Returns(new TbMtUserDTO());
TbMtUserDTO userDto = _usersService.RecoverUser(_user.CdUser, updateTbMtUserDto);
_usersRepositoryMock.Verify(x => x.SaveChanges(), Times.Once);
Assert.NotNull(userDto);
Assert.True(oldPassword != _user.DsPassword);
}
As you can see, there are three asserts at the bottom of that test. I first check if SaveChanges was called, and then I verify that the method actually returned something, hence the NotNull assertion and that it actually modifies the password (the True assertion).
But I kind of feel that's not the right way to do it. But in my head those tests are related, but I'm unsure if I should split them into three different tests. The thing is that I must arrange the same pieces together for the three cases, which to be honest, I don't think it's a good idea either.
What do you guys think? I've been implementing unit-testing for a couple of months now so what are some of your rules of thumb in scenarios like these?
Maybe if you think about split a test into multiple tests, you should split your method to multiple classes/methods and write tests for them? I do not want to go deep into architecture, but this can be a solution. Especially i would separate this:
if (user == null || (updatedData.IdRecSet == "Password"
&& String.IsNullOrEmpty(updatedData.DsUpdatedPassword)))
{
return null;
}
And this
user.DsPassword = PasswordHasher.Hash(updatedData.DsUpdatedPassword);
_usersRepository.SaveChanges();
There is a rule that each test should test one specific aspect. However, that leaves open a bit the question, what makes an aspect? For me, a rule of thumb is, that two asserts represent two different aspects if there is a plausible change in the SUT that could impact only one of the two asserts.
To give you an example: Assume in some game you always start at a certain space location (maybe a space station) with a defined 3D coordinate. To test the initialization function, you check if that initial coordinate has the expected values. These three values together form an aspect: If you decide at some point in time that the game should starts at a different place, all three coordinates will change at once (well, in theory not all need to change, but that would be a strange coincidence).
In your example, the situation is additionally complicated by the fact, that your function does more than one thing and uses the returning of null for different purposes. To be more specific, depending on the content of the argument, the function just does a lookup (user name), or additionally makes some change (password). Thus, it is not only a question of splitting the tests, but probably also of splitting the functionality.
I could imagine splitting it up into two parts: One function that does the lookup:
TbMtUser user = _usersRepository.FindUserById(userId);
if (user != null) {
return _mapper.Map<TbMtUserDTO>(user);
} else {
return null;
}
And a second one that changes the password for an already looked-up user - which in your case may not be straightforward, because the internally used type is TbMtUser whereas the returned type is TbMtUserDTO and is not clear for me how they are related...

Is there a way to specify an NUnit test as "extra credit"?

I have a few tests for an API, and I would like to be able to express certain tests that reflect "aspirational" or "extra credit" requirements - in other words, it's great if they pass, but fine if they don't. For instance:
[Test]
public void RequiredTest()
{
// our client is using positive numbers in DoThing();
int result = DoThing(1);
Assert.That( /* result is correct */ );
}
[Test]
public void OptionalTest()
{
// we do want to handle negative numbers, but our client is not yet using them
int result = DoThing(-1);
Assert.That( /* result is correct */ );
}
I know about the Ignore attribute, but I would like to be able to mark OptionalTest in such a way that it still runs on the CI server, but is fine if it does not pass - as soon as it does, I would like to take notice and perhaps make it a requirement. Is there any major unit test framework that supports this?
I would use a Warnings to achieve this. That way - your test will print a 'warning' output, but not be a failure, and not fail your CI build.
See: https://github.com/nunit/docs/wiki/Warnings
as soon as it does, I would like to take notice and perhaps make it a requirement.
This part's a slightly separate requirement! Depends a lot on how you want to 'take notice'! Consider looking at Custom Attributes - it may be possible to write an IWrapSetUpTearDown attribute, which sends an email when the relevant test passes. See the docs, here: https://github.com/nunit/docs/wiki/ICommandWrapper-Interface
The latter is a more unusual requirement - I would expect to have to do something custom to fit your needs there!

cppUnit: setUp function executed once for multiple testmethods

I've got an object Obj doing some (elaborate) computation and want to check weather the result (let's call it aComputed and bComputed) is correct or not. Therefore I want to split this task up into multiple test methods:
testA() { load aToBe; check if number aComputed = aToBe }
testB() { load bToBe; check if number bComputed = bToBe }
The problem is, that Obj is "executed" twice (which takes a lot of time) - one time per test. The question is: How can I manage that it's just "executed" once and the result is used used by both tests?
At the moment Obj is placed inside the setUp-function and saves the results to a private member of the test-class.
Thanks for helping!
There is no easy solution that allows you to split the code into two test methods. Each test method results in a new test object with an own set of local variables.
Obviously you could work around this problem through a static variable but in the long run this normally just causes issues and breaks the ideas behind the framework.
The better idea is to just write the two CPPUNIT_ASSERT in the same test method. If the results are part of the same calculation there is most likely not much value in splitting the checks into two independent test methods.

The balance of Single responsibility/unit testability and practicality

I'm still confused about unit testing. Suppose I have something as trivial as this:
class x {
zzz someMethod(some input...) {
BufferedImage image = getter.getImageFromFile(...);
// determine resize mode:
int width = image.getWidth();
int height = image.getHeight();
Scalr.Mode resizeMode = (width > height) ? Scalr.Mode.FIT_TO_WIDTH : Scalr.Mode.FIT_TO_HEIGHT;
return ScalrWrapper.resize(image, resizeMode);
}
}
Going by rules, Scalr.Mode resizeMode = should probably be a in a separate class for better unit testability of the aforementioned method, like so:
class xxx {
mode getResizeMode(int width, int height)
{
return (width > height) ? Scalr.Mode.FIT_TO_WIDTH : Scalr.Mode.FIT_TO_HEIGHT;
}
}
class x {
zzz someMethod(some input...) {
BufferedImage image = getter.getImageFromFile(...);
// determine resize mode:
int width = image.getWidth();
int height = image.getHeight();
Scalr.Mode resizeMode = xxx.getResizeMode(width, height);
return ScalrWrapper.resize(image, resizeMode);
}
}
But it looks like such an overkill... I'm not sure which one is better but I guess this way is better. Suppose I go this route, would it be even better to do it this way?
class xxx {
mode getResizeMode(Image image)
{
return (image.getWidth() > image.getHeight()) ? Scalr.Mode.FIT_TO_WIDTH : Scalr.Mode.FIT_TO_HEIGHT;
}
}
class x {
void someMethod(some input...) {
BufferedImage image = getter.getImageFromFile(...);
// determine resize mode:
Scalr.Mode resizeMode = xxx.getResizeMode(image);
return ScalrWrapper.resize(image, resizeMode);
}
}
From what I understand, the correct way is the one where getResizeMode accepts integers as it is decoupled from the type of data whose properties are width and height. However, personally to me, the use of getResizeMode(BufferedImage) actually justifies the creation of a separate class better as some more work is removed from the main method. And since I am not going to be using getResizeMode for any sort of data other than BufferedImage in my application anyway, there is no problem of reusability. Also, I don't think I should be doing getResizeMode(int, int) simply for reusability if I see no need for it due to YAGNI principle.
So my question is: would getResizeMode(BufferedImage) be a good way according to OOD in real world? I understand it's text book good OOD, but then I have been lead to believe that 100% text book OOD is impracticle in real world. So as I am trying to learn OOD, I just want to know which path I should follow.
...Or maybe I should I just leave everything in one method like in the very first code snippet?
I don't think that resize mode calculation influences testability a lot.
As to Single Responsibility:
"A class should have only one reason to change" (https://en.wikipedia.org/wiki/Single_responsibility_principle).
Do you think that resizing mode calculation is going to change?
If not then just put in the class where this mode is needed.
This won't add any reasons to change for that class.
If the calculation is likely to change (and/or may have several versions)
then move it to a separate class (make it a strategy)
Achieving the Single Responsibility Principle (SRP) is not about creating new classes every time, one extracting a method. Moreover the SRP depends on the context.
A module should concern to the SRP.
A class should concern to the SRP.
A method should concern to the SRP.
The message from Uncle Bob is: Extract till you Drop
Beyond he said:
Perhaps you think this is taking things too far. I used to think so too. But after programming for over 40+ years, I’m beginning to come to the conclusion that this level of extraction is not taking things too far at all.
When it comes to the decision to create new classes, keep the metric high cohesion in mind. Cohesion is the degree to which the elements of a module belong together. If all methods work in one specific context and on the same set of variables, they belong to one class.
Back to your case. I would extract all the methods and put them in on class. And this one class is also nicely testable.
Little bit late to the party, but here's my 2c.
To my mind, class x is not adhering to the SRP for a different reason.
It's currently responsible for
Getting an image from a file (getter.getImageFromFile)
Resizing that image
TL;DR
The TL;DR on this is that both of your approaches are fine and both do in fact stick - with varying degrees of stickiness - to the SRP. However if you want to adhere very tightly to the SRP (which tends to lead to very testable code), you could split this into three classes first:
Orchestrator
class imageResizeService
{
ImageGetter _getter;
ImageResizer _resizer;
zzz ResizeImage(imageName)
{
image=_getter.GetImage(imageName);
resizedImage=_resizer.ResizeImage(image);
return resizedImage;
}
}
This class has a single responsibility; namely, given an image name,
return a resized version of it based on some criteria.
To do so, it orchestrates two dependencies. But it only has a single reason to change which is that the process used to get and resize an image in
general , has changed.
You can easily unit test this by mocking the getter and resizer and testing that they are called in order, that the resizer is called with the data given by the getter, and that the final return value equals that returned by the resizer, and so on (i.e. "White Box" testing)
ImageGetter
class ImageGetter
{
BufferedImage GetImage(imageName)
{
image=io.LoadFromDisk(imageName) or explode;
return image;
}
}
Again, we have a single responsiblity (load an image from disk, and return it).
The only reason to change this class would be if the mechanics of loading the image were to change - e.g. you are loading from a Database, not a Disk.
An interesting note here is that this class is ripe for further generalisation - for example to be able to compose it using a BufferedImageBuilder and a RawImageDataGetter abstraction which could have multiple implementations for Disk, Database, Http, etc. But that's YAGNI right now and a conversation for another day :)
Note on testability
In terms of unit testing this, you may run into a small problem, namely that you can't quite "unit test" it - unless your framework as a mock for the file system. In that case, you can either further abstract the loading of the raw data (as per the previous paragraph) or accept it and just perform an integration test off a known good file. Both approaches are perfectly valid and you should not worry about which you choose - whatever is easier for you.
ImageResizer
class ImageResizer
{
zzz ResizeImage(image)
{
int width = image.getWidth();
int height = image.getHeight();
Scalr.Mode resizeMode = getResizeMode(width, height);
return ScalrWrapper.resize(image, resizeMode);
}
private mode getResizemode(width, height)
{
return (width > height) ? Scalr.Mode.FIT_TO_WIDTH : Scalr.Mode.FIT_TO_HEIGHT;
}
}
This class also has but a single job, to resize an image.
The question of whether or not the getResizeMode method - currently just a private method to keep the code clean - should be a separate responsiblity has to be answered in the context of whether or not that operation is somehow independent of the image resizing.
Even if it's not, then the SRP is still being followed, because it's part of the single responsibility "Resize an Image".
Test-wise this is also really easy to test, and because it doesn't even cross any boundaries (you can create and supply the sole dependency - the image - during test runtime) you probably won't even need mocks.
Personally I would extract it to a separate class, just so that I could, in isolation, verify that given a width larger than a height, I was returned a Scalr.Mode.FIT_TO_WIDTH and vice-versa; it would also mean I could adhere to the Open Closed Principle whereby new scaling modes could be introduced without having to modify the ImageResizer class.
But really
The answer here has to be that that it depends; for example if you have a simple way to verify that, given a width of 100 and a height of 99, then the resized image is indeed scaled to "Fit to Width" then you really don't need to.
That being said I suspect you'll have an easier time testing this if you do extract that to a separate method.
Just bear in mind that if you're using a decent IDE with good refactoring tools, that should really not take you more than a couple of keystrokes, so don't worry about the overhead.

How to test asynchronuous code

I've written my own access layer to a game engine. There is a GameLoop which gets called every frame which lets me process my own code. I'm able to do specific things and to check if these things happened. In a very basic way it could look like this:
void cycle()
{
//set a specific value
Engine::setText("Hello World");
//read the value
std::string text = Engine::getText();
}
I want to test if my Engine-layer is working by writing automated tests. I have some experience in using the Boost Unittest Framework for simple comparison tests like this.
The problem is, that some things I want the engine to do are just processed after the call to cycle(). So calling Engine::getText() directly after Engine::setText(...) would return an empty string. If I would wait until the next call of cycle() the right value would be returned.
I now am wondering how I should write my tests if it is not possible to process them in the same cycle. Are there any best practices? Is it possible to use the "traditional testing" approach given by Boost Unittest Framework in such an environment? Are there perhaps other frameworks aimed at such a specialised case?
I'm using C++ for everything here, but I could imagine that there are answers unrelated to the programming language.
UPDATE:
It is not possible to access the Engine outside of cycle()
In your example above, std::string text = Engine::getText(); is the code you want to remember from one cycle but execute in the next. You can save it for later execution. For example - using C++11 you could use a lambda to wrap the test into a simple function specified inline.
There are two options with you:
If the library that you have can be used synchronously or using c++11 futures like facility (which can indicate the readyness of the result) then in your test case you can do something as below
void testcycle()
{
//set a specific value
Engine::setText("Hello World");
while (!Engine::isResultReady());
//read the value
assert(Engine::getText() == "WHATEVERVALUEYOUEXPECT");
}
If you dont have the above the best you can do have a timeout (this is not a good option though because you may have spurious failures):
void testcycle()
{
//set a specific value
Engine::setText("Hello World");
while (Engine::getText() != "WHATEVERVALUEYOUEXPECT") {
wait(1 millisec);
if (total_wait_time > 1 sec) // you can put whatever max time
assert(0);
}
}