I would like to figure out how to modify the return value of a static getter for my unit tests in Flutter and Dart.
I'm unit testing a simple function:
Future<bool> exampleFunc() async {
if (Platform.isIOS) {
// Do some iOS-specific things
return false;
} else if (Platform.isAndroid) {
// Do some Android-specific things
return true; // just as an example
}
throw 'Unexpected platform';
}
And I would like to modify the return values of the static getters of a class: I would like to tweak Platform.isIOS and Platform.isAndroid return value for the different test cases.
Please note that I know of workarounds for this issue, for example, I could split the function into two functions (one for each platform), I could inject my own enum (or in this case, even a bool might work) that represents the supported platforms. In this example, I am actually going to do that, but I would like to know how to "modify" the return values of the Platform class's getters, because sometimes, you don't want to modify the signature of a function, as others might depend on it and yet, you still want to unit test the function.
You should not mock classes that you don't own. Your unit test must be platform independent. On your case you should refactor your code to get rid of this dependency.
If you really wanna continue with this dependency at least depends on abstractions:
abstract class MyPlatform {
bool isAndroid();
bool isIos();
}
class MyPlatformImp implements MyPlatform {
#override
bool isAndroid() => Platform.isAndroid;
#override
bool isIos() => Platform.isIOS;
}
then you can mock MyPlatform on your uses.
This kind of variable you would test on Integration Tests
https://flutter.dev/docs/cookbook/testing/integration/introduction
You can also create different tests for platforms using the onPlatform attribute of test()
https://api.flutter.dev/flutter/test_api/test.html
I agree with Jhionan Santos, but there is an exception on You should not mock classes that you don't own.
On this topic, we could find a package that wrap the orgininal platform for testable. In my case I also need process to be mocked, so I also find a packakge.
https://pub.dev/packages/platform
https://pub.dev/packages/process
I recommend to use these packages, since they both own by google.
Related
I would like to figure out how to modify the return value of a static getter for my unit tests in Flutter and Dart.
I'm unit testing a simple function:
Future<bool> exampleFunc() async {
if (Platform.isIOS) {
// Do some iOS-specific things
return false;
} else if (Platform.isAndroid) {
// Do some Android-specific things
return true; // just as an example
}
throw 'Unexpected platform';
}
And I would like to modify the return values of the static getters of a class: I would like to tweak Platform.isIOS and Platform.isAndroid return value for the different test cases.
Please note that I know of workarounds for this issue, for example, I could split the function into two functions (one for each platform), I could inject my own enum (or in this case, even a bool might work) that represents the supported platforms. In this example, I am actually going to do that, but I would like to know how to "modify" the return values of the Platform class's getters, because sometimes, you don't want to modify the signature of a function, as others might depend on it and yet, you still want to unit test the function.
You should not mock classes that you don't own. Your unit test must be platform independent. On your case you should refactor your code to get rid of this dependency.
If you really wanna continue with this dependency at least depends on abstractions:
abstract class MyPlatform {
bool isAndroid();
bool isIos();
}
class MyPlatformImp implements MyPlatform {
#override
bool isAndroid() => Platform.isAndroid;
#override
bool isIos() => Platform.isIOS;
}
then you can mock MyPlatform on your uses.
This kind of variable you would test on Integration Tests
https://flutter.dev/docs/cookbook/testing/integration/introduction
You can also create different tests for platforms using the onPlatform attribute of test()
https://api.flutter.dev/flutter/test_api/test.html
I agree with Jhionan Santos, but there is an exception on You should not mock classes that you don't own.
On this topic, we could find a package that wrap the orgininal platform for testable. In my case I also need process to be mocked, so I also find a packakge.
https://pub.dev/packages/platform
https://pub.dev/packages/process
I recommend to use these packages, since they both own by google.
Consider you have the following method:
public Foo ParseMe(string filepath)
{
// break up filename
// validate filename & extension
// retrieve info from file if it's a certain type
// some other general things you could do, etc
var myInfo = GetFooInfo(filename);
// create new object based on this data returned AND data in this method
}
Currently I have unit tests for GetFooInfo, but I think I also need to build unit tests for ParseMe. In a situation like this where you have a two methods that return two different properties - and a change in either of them could break something - should unit tests be created for both to determine the output is as expected?
I like to err on the side of caution and be more wary about things breaking and ensuring that maintenance later on down the road is easier, but I feel very skeptical about adding very similar tests in the test project. Would this be bad practice or is there any way to do this more efficiently?
I'm marking this as language agnostic, but just in case it matters I am using C# and NUnit - Also, I saw a post similar to this in title only, but the question is different. Sorry if this has already been asked.
ParseMe looks sufficiently non-trivial to require a unit test. To answer your precise question, if "you have a two methods that return two different properties - and a change in either of them could break something" you should absolutely unit test them.
Even if the bulk of the work is in GetFooInfo, at minimum you should test that it's actually called. I know nothing about NUnit, but I know in other frameworks (like RSpec) you can write tests like GetFooInfo.should be_called(:once).
It is not a bad practice to test a method that is calling another method. In fact, it is a good practice. If you have a method calling another method, it is probably performing additional functionality, which should be tested.
If you find yourself unit testing a method that calls a method that is also being unit tested, then you are probably experiencing code reuse, which is a good thing.
I agree with #tsm - absolutely test both methods (assuming both are public).
This may be a smell that the method or class is doing too much - violating the Single Responsibility Principle. Consider doing an Extract Class refactoring and decoupling the two classes (possibly with Dependency Injection). That way you could test both pieces of functionality independently. (That said, I'd only do that if the functionality was sufficiently complex to warrant it. It's a judgment call.)
Here's an example in C#:
public interface IFooFileInfoProvider
{
FooInfo GetFooInfo(string filename);
}
public class Parser
{
private readonly IFooFileInfoProvider _fooFileInfoProvider;
public Parser(IFooFileInfoProvider fooFileInfoProvider)
{
// Add a null check
_fooFileInfoProvider = fooFileInfoProvider;
}
public Foo ParseMe(string filepath)
{
string filename = Path.GetFileName(filepath);
var myInfo = _fooFileInfoProvider.GetFooInfo(filename);
return new Foo(myInfo);
}
}
public class FooFileInfoProvider : IFooFileInfoProvider
{
public FooInfo GetFooInfo(string filename)
{
// Do I/O
return new FooInfo(); // parameters...
}
}
Many developers, me included, take a programming by contract approach. That requires you to consider each method as a black box. If the method delegates to another method to accomplish its task does not matter, when you are testing the method. But you should also test all large or complicated parts of your program as units. So whether you need to unit test the GetFooInfo depends on how complicated that method is.
This is a simplified version of a class I'm writing a unit test for
class SomeClass {
void methodA() {
methodB();
methodC();
methodD();
}
void methodB() {
//does something
}
void methodC() {
//does something
}
void methodD() {
//does something
}
}
While writing the unit tests for this class, I've mocked out objects using EasyMock used in each method. It was easy to set up the mock objects and their expectation
In method B,C,and D. But to test method A, I have to set up A LOT more mock objects and their expectations. Also, I’m testing method A in different conditions, meaning I have to setup the mock objects many times with different expectations.
In the end, my unit test becomes hard to maintain and pretty cluttered. I was wondering if anyone has or seen a good solution to this problem.
If I understand your question correctly, I think that this is a matter of design. The nice thing about unit testing is that writing tests often forces you to make your design better. If you need to mock too many things while testing a method it often means you should split your class into two smaller classes, which will be easier to test (and write, and maintain, and bugfix, and reuse, etc.).
In your case, the method A seems to be at a higher level than methods A, B, C. You can consider removing it to a higher level class, that would wrap SomeClass:
class HigherLevelClass {
ISomeClass someClass;
public HigherLevelClass(ISomeClass someClass)
{
this.someClass = someClass;
}
void methodA() {
someClass.methodB();
someClass.methodC();
someClass.methodD();
}
}
class SomeClass : ISomeClass {
void methodB() {
//does something
}
void methodC() {
//does something
}
void methodD() {
//does something
}
}
Now when you are testing methodA all you need to mock is the small ISomeClass interface and the three method calls.
You could extract common setup code into separate (possibly parametrized) methods, then call them whenever appropriate. If the tests for methodA have a very different fixture from the tests of the other methods, there may not be much to put into the #Before method itself, so you need to call the appropriate combination of setup helper methods from the test methods themselves. It is still a bit cumbersome, but better than duplicating code all over the place.
Depending on what unit test framework you use, there may be other options too, but the above should work with any framework.
This is an example of a Fragile test because the mock setups have too intimate knowledge of the SUT.
I don't know EasyMock, but with Moq you don't need to setup void methods. However, with Moq the methods would have to be public or protected and virtual.
For each test you're writing, consider the behaviour which is valuable for that test. You'll have some contexts you're setting up which the behaviour relies on, and some outcomes as a result of the behaviour that you want to verify.
Set up relevant contexts, verify the outcomes, and use NiceMocks for everything else.
I prefer Mockito (Java) or Moq (.NET) which work this way by default. Here's Mockito's page on Mockito vs. EasyMock so you can get the idea (EasyMock didn't have NiceMock before Mockito came along):
http://code.google.com/p/mockito/wiki/MockitoVSEasyMock
You can probably use EasyMock's NiceMock in a similar way. Hopefully this will help you detangle your tests. You can always import both frameworks and use them alongside each other / incrementally switch over if it helps.
Good luck!
I’m testing method A in different conditions, meaning I have to setup the mock objects many times with different expectations.
If you care of what methodA is doing and which collaborator function has to be called then you have to setup different expectations... I don't see how you can skip this step?!
If you testLogout you would expect a call to myCollaborator.logout() otherwise if you testLogin you would expect something like myCollaborator.login().
If you have many methods with lots/different expectations maybe is the case to split your class in collaborators
When unit testing a codebase, what are the tell-tale signs that I need to utilise mock objects?
Would this be as simple as seeing a lot of calls to other objects in the codebase?
Also, how would I unit test methods which don't return values? So if I a method is returning void but prints to a file, do I just check the file's contents?
Mocking is for external dependencies, so that's literally everything, no? File system, db, network, etc...
If anything, I probably over use mocks.
Whenever a class makes a call to another, generally I mock that call out, and I verify that the call was made with the correct parameters. Else where, I'll have a unit test that checks the concrete code of the mocked out object behaves correctly.
Example:
[Test]
public void FooMoo_callsBarBaz_whenXisGreaterThan5()
{
int TEST_DATA = 6;
var bar = new Mock<Bar>();
bar.Setup(x => x.Baz(It.Is<int>(i == TEST_DATA)))
.Verifiable();
var foo = new Foo(bar.Object);
foo.moo(TEST_DATA);
bar.Verify();
}
...
[Test]
public void BarBaz_doesSomething_whenCalled()
{
// another test
}
The thing for me is, if I try to test lots of classes as one big glob, then there's usually tonnes of setup code. Not only is this quite confusing to read as you try to get your head around all the dependencies, it's very brittle when changes need to be made.
I much prefer small succinct tests. Easier to write, easier to maintain, easier to understand the intent of the test.
Mocks/stubs/fakes/test doubles/etc. are fine in unit tests, and permit testing the class/system under test in isolation. Integration tests might not use any mocks; they actually hit the database or other external dependency.
You use a mock or a stub when you have to. Generally this is because the class you're trying to test has a dependency on an interface. For TDD you want to program to interfaces, not implementations, and use dependency injection (generally speaking).
A very simple case:
public class ClassToTest
{
public ClassToTest(IDependency dependency)
{
_dependency = dependency;
}
public bool MethodToTest()
{
return _dependency.DoSomething();
}
}
IDependency is an interface, possibly one with expensive calls (database access, web service calls, etc.). A test method might contain code similar to:
// Arrange
var mock = new Mock<IDependency>();
mock.Setup(x => x.DoSomething()).Returns(true);
var systemUnderTest = new ClassToTest(mock.Object);
// Act
bool result = systemUnderTest.MethodToTest();
// Assert
Assert.That(result, Is.True);
Note that I'm doing state testing (as #Finglas suggested), and I'm only asserting against the system under test (the instance of the class I'm testing). I might check property values (state) or the return value of a method, as this case shows.
I recommend reading The Art of Unit Testing, especially if you're using .NET.
Unit tests are only for one piece of code that works autonomously within itself. This means that it doesn't depend on other objects to do its work. You should use mocks if you are doing Test-Driven programming or Test-First programming. You would create a mock (or stub as I like to call it) of the function you will be creating and set certain conditions for the test to pass. Originally the function returns false and the test fails, which is expected ... then you write the code to do the real work until it passes.
But what I think you are referring to is integration testing, not unit testing. In that case, you should use mocks if you are waiting for other programmers to finish their work and you currently don't have access to the functions or objects they are creating. If you know the interface, which hopefully you do otherwise mocking is pointless and a waste of time, then you can create a dumbed-down version of what you are hoping to get in the future.
In short, mocks are best utilized when you are waiting for others and need something there in order to finish your work.
You should try to always return a value if possible. Sometimes you run into problems where you are already returning something, but in C and C++ you can have output parameters and then use the return value for error checking.
If I was to write a mocking library, how would this work (in other words, how do "they work?)?
One of the things which I wonder is that you are always setting expectations so really you need to compare the expectation to what the method does at runtime, so I assume reflection (resolving types at runtime) is required.
Also, when using the term "mock object", is the object stubbed out or would it be an object with pre-set expectations?
When I think how I would write my own implementation of a framework/technique, like mock objects, I realise how much I really know (or don't know) and what I would trip up on: If the mock object is pre-programmed to return set expectations and you don't call the actual real object, then wouldn't the result always be the same? Eg:
[TestMethod, Isolated]
public void FakeReturnValueByMethodArgs()
{
var fake = Isolate.Fake.Instance<ClassToIsolate>();
// MethodReturnInt will return 10 when called with arguments 3, "abc"
Isolate.WhenCalled(()=> fake.MethodReturnInt(3, " abc")).WithExactArguments().WillReturn(10);
// MethodReturnInt will return 50 when called with arguments 3, "xyz"
Isolate.WhenCalled(()=> fake.MethodReturnInt(3, "xyz")).WithExactArguments().WillReturn(50);
Assert.AreEqual(10, fake.MethodReturnInt(3, "abc"));
Assert.AreEqual(50, fake.MethodReturnInt(3, "xyz"));
}
Wouldn't this always return true?
The idea with mocking frameworks is to mock out dependencies, and not the actual classes under test. For your example, your test will always return true, because really you're only testing the mocking framework and not your actual code!
A real world mock would look more like this:
[TestMethod, Isolated]
public void FakeReturnValueByMethodArgs() {
var fake = Isolate.Fake.Instance<DependencyClass>();
// MethodReturnInt will return 10 when called with arguments 3, "abc"
Isolate.WhenCalled(()=> fake.MethodReturnInt(3, "abc")).WithExactArguments().WillReturn(10);
var testClass = new TestClass(fake);
testClass.RunMethod();
// Verify that the setup methods were execute in RunMethod()
// Not familiar with TypeMock's actual method to do this...
IsolatorExtensions.VerifyInstanceWasCalled(fake);
// Or assert on values
Assert.AreEqual(10, testClass.AProperty);
}
Notice how the mock is passed into the TestClass and a method run on it.
You can read The Purpose of Mocking to get a better idea of how mocking works.
Update: Explanation why you're testing only the mocking framework:
What you've done is create a method MethodReturnInt with the mocking framework using Isolate.WhenCalled(). When you call MethodRecturnInt in the Assert, the code will run the delegate () => fake.MethodReturnInt() and return 10. The mocking framework is effectively creating a method (albeit dynamically) that would look something like this:
public void MethodReturnInt(int value, string value2) {
Assert.Equal(3, value);
Assert.Equal("abc", value2);
return 10;
}
It's a bit more complicated than that, but this is the general idea. Since you never run any code other than the creation of 2 methods and then asserts on those two methods, you're not testing your own code and therefore only testing the mocking framework.
Yes, it will always return true. Mock objects should be used when the class under test requires another class implementation that you don't want to involve in the test run. This is most useful when it's a class that uses interfaces with multiple implementations, or there are complex/expensive/external services that you don't want to set up.
In the above code, you're mocking the class that you're "testing".
Another way of thinking about it is that the mock behaviours you record are black-box (implementation) assertions, where Assert.* are white-box (api) assertions.
You have the right idea. You will often find that they have a couple of modes of operation. If you're worried about your method not getting called or it not getting called in the right order there is quite often a 'strict' mode that causes the mock framework to throw an exception if the method isn't called by the end of the test, or is called with the wrong parameters etc.
Most of the frameworks have thought of those sorts of issues so you just need to find out how to configure it for your scenario.
One way to look at how mock system work is just look at times when you need an object but you don't want to use the real class but instead want it to give you some specific kind of data that it wouldn't (or won't do so reliably). So if you see:
Assert.IsTrue(myLogic.IsNoon(time))
you can see how the assert would want the time object to always be noon. . . well you can't do that with a real object reliably. So you need a stand-in. You can make a fake class just for the test, but that's sort of heavy. Mock frameworks are a shortcut.