Checking function equality in a F# unit test - unit-testing

I have a bunch of F# functions that implement different algorithms for the same input, kind of like the Strategy pattern. To pick the right strategy, I want to pattern match on the input argument and return the function as a value :
let equalStrategy points : seq<double> =
...
let multiplyStrategy factor (points: seq<double>) =
...
let getStrategy relationship =
match relationship with
| "=" -> equalStrategy
| "*5" -> multiplyStrategy 5.0
| _ -> raise (new System.NotImplementedException(" relationship not handled"))
Now I want to write some unit tests to make sure that I return the right strategy, so I tried something like this in nUnit :
[<TestCase("=")>]
[<Test>]
member self.getEqualstrategy( relationship:string ) =
let strategy = getStrategy relationship
Assert.AreEqual( strategy, equalStrategy )
Now I think the code is correct and will do what I want, but the assertion fails because functions don't seem to have an equality operation defined on them. so my questions are :
(a) is there a way to compare 2 functions to see if they are the same, i.e. let isFoo bar = foo == bar, that I can use in an nUnit assertion?
or
(b) is there another unit testing framework that will do this assertion for me in F#?

Testing whether an F# function returned by your getStrategy is the same function as one of the funcions you defined is also essentially impossible.
To give some details - the F# compiler generates a class that inherits from FSharpFunc when you return a function as a value. More importantly, it generates a new class each time you create a function value, so you cannot compare the types of the classes.
The structure of the generated classes is something like this:
class getStrategy#7 : FSharpFunc<IEnumerable<double>, IEnumerable<double>> {
public override IEnumerable<double> Invoke(IEnumerable<double> points) {
// Calls the function that you're returning from 'getStrategy'
return Test.equalStrategy(points);
}
}
// Later - in the body of 'getStrategy':
return new getStrategy#7(); // Returns a new instance of the single-purpose class
In principle, you could use Reflection to look inside the Invoke method and find which function is called from there, but that's not going to be a reliable solution.
In practice - I think you should probably use some other simpler test to check whether the getStrategy function returned the right algorithm. If you run the returned strategy on a couple of sample inputs, that should be enough to verify that the returned algorithm is the right one and you won't be relying on implementation details (such as whether the getStrategy function just returns a named function or whether it returns a new lambda function with the same behaviour.
Alternatively, you could wrap functions in Func<_, _> delegates and use the same approach that would work in C#. However, I think that checking whether getStrategy returns a particular reference is a too detailed test that just restricts your implementation.

Functions doesn't have equality comparer:
You will have error: The type '('a -> 'a)' does not support the 'equality' constraint because it is a function type
There is a good post here

It would be very difficult for the F# compiler to prove formally that two functions always have the same output (given the same input). If that was possible, you could use F# to prove mathematical theorems quite trivially.
As the next best thing, for pure functions, you can verify that two functions have the same output for a large enough sample of different inputs. Tools like fscheck can help you automate this type of test. I have not used it, but I've used scalacheck that is based on the same idea (both are ports from Haskell's QuickCheck)

Related

Testing Behavior equality in Akka Typed

In a test I want to assert that a certain (parameterized) Behavior is returned by a kind of factory method. How can I do that?
def myBehavior(param: Int) = Behaviors.receiveMessage { ... }
When I call myBehavior twice, I get two different objects (which is what I would expect) but they are not equal.
I thought about making a case class that extends ExtensibleBehavior. However, I don't know how to delegate to a behavior defined using the Behaviors DSL. Apart from that I don't like the indirection that is introduced by the additional class.
Is there an elegant solution for this problem?

Mocking and unit testing implementation dependencies

I am trying to wrap my head around the practice of unit testing. I have read that unit tests should not be dependent on the implementations.
The way I understand this is, for example, if we have a function int addTwoNumbers (int a, int b) we should be testing whether, for example addition return correct results (i.e. addTwoNumbers (2, 2) == 4)) and we should not care whether, for example, addTwoNumbers calls operator + just once - it might as well use bit manipulation for this.
It seemed reasonable for me and - in my opinion - offers a decent decoupling between the tests and the code.
Enter the mocking frameworks. As far I can tell from reading their documentation, their functionality amounts to
(a) generating objects that implement placeholder functionality of some interfaces of base classes (stubbing) and
(b) checking whether that functionality was called according to expectations set forth by the tester.
I have no problems with (a) - I understand that we sometimes need to hard-code some functionality of external dependencies for the testing. I do not understand though, why should we check whether the tested code called mock's functionality in a way expected by the tester.
Shouldn't we be interested only in what a tested method returns or how it modifies its out arguments and not really care about its implementation details? Does the verification functionality of the mocking frameworks not introduce tight coupling between tested and testing code?
I do not understand though, why should we check whether the tested code called mock's functionality in a way expected by the tester
Because the contract of a method is not always to return something or to modify its arguments. Sometimes the contract of the method is (or includes) to have side effects. Think for example of the following method:
void notifyServerOfError(error: string) {
this.http.post('/api/errors', {
error: error,
ip: myIpAddress
});
}
This method doesn't return anything. It doesn't modify its arguments. Its sole responsibility is to send a specific object, containing specific details, to a specific URL. And unit-testing this method should thus verify that the contract is respected.
A good way to do that is to mock the http dependency and to check that, when this method is called, its post() method indeed being called with the correct URL and the correct data.

To check if List<T> returns a collection using Moq. Guidance needed on how to assert?

I am trying to use Moq for testing a manager that has many CRUD operations defined in it. One of the methods is taking an integer parameter and returning a list based on conditions. I want to assert if the list does have a count greater than 0. How to do this using Moq? This is my attempt, guidance needed.
// set up the mock method making sure integer is passed and list is returned
var mockAssessmentManager = new Mock<IAssessmentManager>();
mockAssessmentManager.Setup(x => x.GetAssessmentElementUserRowByAssessment(It.IsAny<int>()))
.Returns(new List<AssessmentElementUser>());
// Assert.IsTrue(myReturnList.Count > 0); //want to achieve some like this.
Assert.IsTrue(mockAssessmentManager);
Am I doing it the right way? How to assert my collection of List<T> to check if count > 0?
The title "To check if List<T> returns a collection" makes no sense because List<T> is a type not a method, so it doesn't return anything it is something. Read How to ask to see how important the title is.
You say you are testing a manager, and the example shows code which is mocking a manager. But if you are mocking the manager, you aren't testing it, you should be testing the thing which uses the manager. I'm going to assume that to be the case.
Given that you are setting up your mock to return a collection when the manager is called, you can use Verify to check that that happened to your mock (the syntax is very similar to Setup). But quite often you don't need to, since the return value from code under test is often driven by the value returned by the dependency.
For example, if the return value from the code under test should be the value that the manager returned to it (which is the collection you return in the Setup), asserting that the value returned is reference equals the same as your mock returned is the solution. (There is no other way that assert could pass unless the method that was setup was called.)
There certainly doesn't seem to be any value in asserting anything about a collection which you have set up as part of your test.
Bonus tip: you haven't tagged which testing framework you're using but Assert.IsTrue(myReturnList.Count > 0); is a bad way is asserting, since if the assert fails the message is always "expected true but was false". Whereas Assert.That(myReturnList, Has.Count.GreaterThan(0)); (I'm assuming NUnit) will give much better information if it fails.
Another tip, I need to offer a caution about using It.IsAny. Take a look at this question (which is about Verify, but the syntax is very similar to Setup, and the concern about It.IsAny is the same). Read the non accepted answer and all the comments, then the accepted answer.

Unit testing higher order functions in F#

Take the following F# example:
let parse mapDate mapLevel mapMessge (groups : string list) =
{
DateTime =
mapDate(
groups.[2] |> Int32.Parse,
groups.[0] |> Int32.Parse,
groups.[1] |> Int32.Parse)
Level = mapLevel groups.[3]
Message = mapMessge groups.[4]
}
I can unit test the map functions independently that's ok, but how do I unit test that this function calls the functions passed in as arguments correctly?
In C# I would use mocks and verify the calls to them. I recently watched a pluralsight video that talked about how functional languages tend to use stubs instead of mocks. Here I could pass in a function that throws if it doesn't get the expected arguments but I'm not really sold on this approach.
I was just wondering if there were any patterns in functional programming in general for unit testing higher-order functions like this?
Well, let me disagree with given answer. Actually, there is a nice way to test higher order functions without even bothering about concrete types they might take (I consider typical HOF to be totally generic, however there is no difference: approach I suggest will work with more strict HFO rightly).
Let's take something really simple, something everyone is familiar with. How about ['t] -> ['t] function? It takes a single argument - a list of whatever type and returns list of the same type. Traditional OOP approach wouldn't work here: one need's to put a restriction on 't and test somewhat specific parameters of that type; the only way to make author to feel more confident with his implementation, is to increase unit tests numbers.
There is really great stuff named "category theory" in math. It's comparatively new filed of mathematics and studies things from the outside rather from than inside. In order to be able to describe things "from the outside" you need take a thing you're interested in and force it to interact with something you already know deep enough. Thus, category theory teaches to describe things in terms of their interrelations with other things. Can't we do the same here?..
Indeed, we can. That's actually quite easy: we got a f : ['t] -> ['t] already, but is there anything else such that we could make both interact and define something common - something that holds for each and every interaction regardless of any other factors? Let's take any g: 't -> 'y. Now we able to state: g (List.head (f ...) = List.head (List.map g (f ...)). I assume a certain argument of type ['t] to substitute .... Please note: given property is universal: it would hold for any pure functions composition of specified signatures regardless of their implementation. Also note how generic yet obvious it is: there are only two distinct "objects" interacting with each other via "composition", which could also be rewritten in terms of standard F#'s (|>), (<|) operators.
Now the fact is that for any higher order (pure) function there exists such kind of universal property; mostly, there are dozens of them. Thus one able to specify their properties in terms of composition (which is regular for FP) staying at the generic level. Having such a properties in the explicit form gives one chance to autogenerate hundreds of tests, based on inputs different not only by their values (which normally done by unit tests, except the fact they are rarely autogenerated), but also by types.
Pure functions are easier because you just have to test the outputs of your parse function. You shouldn't ever need to test using side effects like you do in imperative programming.
When writing most of your unit tests, you generally use the most simple possible for your function arguments, like identity or similar. Then you'd write one test named something like "mapLevel is applied to fourth group" where instead you make mapLevel something that's easy to recognize as changed, like toUpper. This lets you make sure you didn't accidentally copy/paste mapLevel to more than one output. Then a similar test for mapMessge.

How to test function call order

Considering such code:
class ToBeTested {
public:
void doForEach() {
for (vector<Contained>::iterator it = m_contained.begin(); it != m_contained.end(); it++) {
doOnce(*it);
doTwice(*it);
doTwice(*it);
}
}
void doOnce(Contained & c) {
// do something
}
void doTwice(Contained & c) {
// do something
}
// other methods
private:
vector<Contained> m_contained;
}
I want to test that if I fill vector with 3 values my functions will be called in proper order and quantity. For example my test can look something like this:
tobeTested.AddContained(one);
tobeTested.AddContained(two);
tobeTested.AddContained(three);
BEGIN_PROC_TEST()
SHOULD_BE_CALLED(doOnce, 1)
SHOULD_BE_CALLED(doTwice, 2)
SHOULD_BE_CALLED(doOnce, 1)
SHOULD_BE_CALLED(doTwice, 2)
SHOULD_BE_CALLED(doOnce, 1)
SHOULD_BE_CALLED(doTwice, 2)
tobeTested.doForEach()
END_PROC_TEST()
How do you recommend to test this? Are there any means to do this with CppUnit or GoogleTest frameworks? Maybe some other unit test framework allow to perform such tests?
I understand that probably this is impossible without calling any debug functions from these functions, but at least can it be done automatically in some test framework. I don't like to scan trace logs and check their correctness.
UPD: I'm trying to check not only the state of an objects, but also the execution order to avoid performance issues on the earliest possible stage (and in general I want to know that my code is executed exactly as I expected).
You should be able to use any good mocking framework to verify that calls to a collaborating object are done in a specific order.
However, you don't generally test that one method makes some calls to other methods on the same class... why would you?
Generally, when you're testing a class, you only care about testing its publicly visible state. If you test
anything else, your tests will prevent you from refactoring later.
I could provide more help, but I don't think your example is consistent (Where is the implementation for the AddContained method?).
If you're interested in performance, I recommend that you write a test that measures performance.
Check the current time, run the method you're concerned about, then check the time again. Assert that the total time taken is less than some value.
The problem with check that methods are called in a certain order is that your code is going to have to change, and you don't want to have to update your tests when that happens. You should focus on testing the actual requirement instead of testing the implementation detail that meets that requirement.
That said, if you really want to test that your methods are called in a certain order, you'll need to do the following:
Move them to another class, call it Collaborator
Add an instance of this other class to the ToBeTested class
Use a mocking framework to set the instance variable on ToBeTested to be a mock of the Collborator class
Call the method under test
Use your mocking framework to assert that the methods were called on your mock in the correct order.
I'm not a native cpp speaker so I can't comment on which mocking framework you should use, but I see some other commenters have added their suggestions on this front.
You could check out mockpp.
Instead of trying to figure out how many functions were called, and in what order, find a set of inputs that can only produce an expected output if you call things in the right order.
Some mocking frameworks allow you to set up ordered expectations, which lets you say exactly which function calls you expect in a certain order. For example, RhinoMocks for C# allows this.
I am not a C++ coder so I'm not aware of what's available for C++, but that's one type of tool that might allow what you're trying to do.
http://msdn.microsoft.com/en-au/magazine/cc301356.aspx
This is a good article about Context Bound Objects. It contains some so advanced stuff, but if you are not lazy and really want to understand this kind of things it will be really helpful.
At the end you will be able to write something like:
[CallTracingAttribute()]
public class TraceMe : ContextBoundObject
{...}
You could use ACE (or similar) debug frameworks, and in your test, configure the debug object to stream to a file. Then you just need to check the file.