I have a bunch of unit tests that work on a class which executes some tasks asynchronously. Currently I set up the class, then execute the function in question. I then go into a blocking wait until the execution is completed, then test the data I need to. Is there a better way that I can do this?
So far my tests look similar to this:
vp.Reset(); //vp is my virtual machine
bool wait = true;
Dictionary<short, Command> commands = new Dictionary<short, Command>();
commands.Add(0, CommandFactory.CreateInputCommand(0, 0));
commands.Add(1, CommandFactory.CreateHaltCommand());
vp.OnHalted += () =>
{
wait = false;
};
vp.InputChannels[0] = () => { return 2; };
vp.CurrentProgram = commands;
vp.ExecuteTillHalt();//asynchronous execution of program. There's also a way to make it execute synchronously
while (wait) ;
Assert.AreEqual(vp.Registers[0], 2);
I see two tests, one to test that an asychronous action is launched (which can be done with a mock) and the second to test that the action that is run asychronously performs correctly when run and you can do the latter synchronously.
Is there a way to break this up into two tests?
You can test the asynchronous aspect via the block and then test the data as a separate testable function that doesn't rely on how it's called.
Related
Using Google Test, I want to test the behaviour of a ClientListener.AcceptRequest method:
class ClientListener {
public:
// Clients can call this method, want to test that it works
Result AcceptRequest(const Request& request) {
queue_.Add(request);
... blocks waiting for result ...
return result;
}
private:
// Executed by the background_thread_;
void ProcessRequestsInQueue() {
while (true) {
Process(queue_.PopEarliest());
}
}
MyQueue queue_;
std::thread background_thread_ = thread([this] {ProcessRequestsInQueue();});
};
The method accepts a client request, queues it, blocks waiting for a result, returns a result when available.
The result is available when the background thread processes the corresponding request from a queue.
I have a test which looks as follows:
TEST(ListenerTest, TwoRequests) {
ClientListener listener;
Result r1 = listener.AcceptClientRequest(request1);
Result r2 = listener.AcceptClientRequest(request2);
ASSERT_EQ(r1, correctResultFor1);
ASSERT_EQ(r2, correctResultFor2);
}
Since the implementation of a ClientListener class involves multiple threads, this test might pass on one attempt but fail on another. To increase the chance of capturing a bug, I run the test multiple times:
TEST_P(ListenerTest, TwoRequests) {
... same as before ...
}
INSTANTIATE_TEST_CASE_P(Instantiation, ListenerTest, Range(0, 100));
But now make test command treats each parameterised instantiation as a separate test,
and in the logs, I see 100 tests:
Test 1: Instantiation/ListenerTest.TwoRequests/1
Test 2: Instantiation/ListenerTest.TwoRequests/2
...
Test 100: Instantiation/ListenerTest.TwoRequests/100
Given that I do not use the parameter value, is there a way to rewrite the testing code such that the make test command would log a single test executed 100 times, rather than 100 tests?
Simple answer: use --gtest_repeat when executing tests would do the trick (default is 1).
Longer answer: unit tests shouldn't be used for this kind of tests. GTest is thread-safe by design (as stated in their README), but this doesn't mean it is a good tool to perform such tests. Maybe it is a good starting point to actually begin working on real integration tests, I really recommend Python's behave framework for this purpose.
In our code messages travelling on guava's AsyncEventBus are frequently used. Now, in some of the tests some message traffic needs to finish, before assert on the state of the testee (say, some loads are scheduled on the EventBus, and these need to finish, so that we can test if things were loaded correctly).
Unfortunately, at the moment this is handled by the horrible Thread.sleep(delay); approach; very fragile and making tests flaky.
What would be the proper way to test code depending on completion of operations on common AsyncEventBus?
What I do in end-to-end or integration tests (multiple threads involved) where I need to wait until an event occurred is using a CountDownLatch. I would stay far away from sleeping threads as you've already mentioned.
This requires that in your test code you can hook the CountDownLatch.countDown() method into the callback method the EventBus is going to call. I explain that with a short example:
class SomeEventReceiver {
...
#Subscribe public void doSomethingFoo(BarEvent e) {
// your logic
}
...
}
// Unit test
...
CountDownLatch readyToAssert = new CountDownLatch(1); // could be 2 or more depending on your needs
SomeEventReceiver rec = new SomeEventReceiver(...) { // create an anonymous subclass
#Subscribe
#Override
public void doSomethingFoo(BarEvent e) { // override super method
super.doSomethingFoo(e); // execute super method's logic
readyToAssert.countDown(); // signal your test method that it's ready to assert
}
}
// put your events on the event bus and do all other necessary things
...
readyToAssert.await(); // JUnit thread is blocked until event handlers where called
assertXXX(...); // assert whatever needs to be asserted
That's my firsthand approach when testing. Obviously it's easier to do if the to-be-tested classes are designed in a testfriendly manner.
Hope that helped!
I am using tasks in WinForms (.NET 4.0) to perform lengthy operations like WCF call. Application is already in product with heavy use of Tasks (almost all the methods which uses Tasks are void).
During the unit testing we have used AutoResetEvents (in actual code) to find out when the given task is completed then perform assert.
This gives me a thought that almost all the AutoResetEvent are waste of effort. They are just fulfilling unit testing needs, nothing else.
Can we create a wrapper around Tasks likewise when actual code run... they should work in background and in case of unit testing they should be synchronous.
Similar to below link for BackgroundWorker.
http://si-w.co.uk/blog/2009/09/11/unit-testing-code-that-uses-a-backgroundworker/
Why can't you simply use the continuation for tasks in your wrapper, like this:
var task = ...
task.ContinueWith(t => check task results here)
Also, unit tests can be marked as async, if they have a return type Task, so you can use an await there, and after that do your asserts:
[Test]
public async Task SynchronizeTestWithRecurringOperationViaAwait()
{
var sut = new SystemUnderTest();
// Execute code to set up timer with 1 sec delay and interval.
var firstNotification = sut.StartRecurring();
// Wait that operation has finished two times.
var secondNotification = await firstNotification.GetNext();
await secondNotification.GetNext();
// Assert outcome.
Assert.AreEqual("Init Poll Poll", sut.Message);
}
Another approach (from the same article) is to use a custom task scheduler, which will be synchronous in case of unit testing:
[Test]
public void TestCodeSynchronously()
{
var dts = new DeterministicTaskScheduler();
var sut = new SystemUnderTest(dts);
// Execute code to schedule first operation and return immediately.
sut.StartAsynchronousOperation();
// Execute all operations on the current thread.
dts.RunTasksUntilIdle();
// Assert outcome of the two operations.
Assert.AreEqual("Init Work1 Work2", sut.Message);
}
Same MSDN magazine contains nice article about best practices for async unit testing. Also async void should be used only as an event handler, all other methods should have async Task signature.
I'm having trouble testing reactive code that's consuming a Task based service. In my class under test I consume the task and use ToObservable to do reactive-y things with it.
public void Method()
{
_svc.MyTaskServiceMethod().ToObservable().Select(....) //pipe it elsewhere and do interesting things.
}
Now in a unit test I'm testing some timing (using Moq for the service)
svcMock.Setup(x => x.MyTaskServiceMethod()).Returns(() =>
Observable.Return("VALUE", testScheduler)
.Delay(TimeSpan.FromMilliseconds(100), testScheduler)
.ToTask()
);
The problem is that despite using the test scheduler in the Return/Delay calls, the task itself is still completing on a separate thread. I'm seeing this by adding a couple of Console writes of the current managed thread id to the code.
svcMock.Setup(x => x.MyServiceMethod()).Returns(() =>
{
var task = Observable.Return("VALUE", testScheduler)
.Delay(TimeSpan.FromMilliseconds(1000), testScheduler)
.Do(x => { Console.WriteLine(Thread.CurrentThread.ManagedThreadId.ToString() + " Obs"); })
.ToTask();
task.ContinueWith((_) =>
{
Console.WriteLine(Thread.CurrentThread.ManagedThreadId.ToString() + " Task");
});
return task;
});
The Do(..) executes on the primary testing thread, and happens exactly when I expect after a testSchduler.AdvanceBy(..) call.
The task continuation is still happening in a separate thread and basically doesn't execute until after the body of the unit test has finished. So in the body of my target, nothing ever really gets pushed through my task.ToObservable() observable.
Task continuations will use a task pool thread by default, so your continuation escapes the control of the test scheduler. If you specify the option TaskContinuationOptions.ExecuteSynchronously, it will use the same thread and the result will be posted to the observable as desired:
task.ContinueWith((_) =>
{
Console.WriteLine(Thread.CurrentThread.ManagedThreadId.ToString() + " Task");
}, TaskContinuationOptions.ExecuteSynchronously);
Addendum
You may find this related discussion on the Rx site quite illuminating on the subject of concurrency in TPL -> Rx transitions, and ToObservable() in particular.
Some time ago I co-authored a unit test library based on NUnit to help with precisely with Rx and TPL testing. For that we built a Test TPL scheduler to force all TPL tasks to run without concurrency. You can see the relevant code here: https://github.com/Testeroids/Testeroids/blob/master/solution/src/app/Testeroids/TplTestPlatformHelper.cs#L87
I have a Scala unit test for an Akka actor. The actor is designed to poll a remote system and update a local cache. Part of the actor's design is that it doesn't attempt to poll while it's still processing or awaiting the result of the last poll, to avoid flooding the remote system when it experiences a slowdown.
I have a test case (shown below) which uses Mockito to simulate a slow network call, and checks that when the actor is told to update, it won't make another network call until the current one is complete. It checks the actor has not made another call by verifying a lack of interactions with the remote service.
I want to eliminate the call to Thread.sleep. I want to test the functionality of the actor without relying on waiting for a hardcoded time, in every test run, which is brittle, and wastes time. The test can poll or block, waiting for a condition, with a timeout. This will be more robust, and will not waste time when the test is passing. I also have the added constraint that I want to keep the state used to prevent extra polling var allowPoll limited in scope, to the internals of the PollingActor.
is there a way force a wait until the actor is finished messaging itself? If there's a way I can wait until then before trying to assert.
is it necessary to send the internal message at all? Couldn't I maintain the internal state with a threadsafe datastructure, such as java.util.concurrent.AtomicBoolean. I have done this and the code appears to work, but I'm not knowledgeable enough about Akka to know if it's discouraged -- a colleague recommended the self message style.
is there better, out-of-the-box functionality with the same semantics? Then I would opt for an integration test instead of a unit test, though I'm not sure if it would solve this problem.
The current actor looks something like this:
class PollingActor(val remoteService: RemoteServiceThingy) extends ActWhenActiveActor {
private var allowPoll: Boolean = true
def receive = {
case PreventFurtherPolling => {
allowPoll = false
}
case AllowFurtherPolling => {
allowPoll = true
}
case UpdateLocalCache => {
if (allowPoll) {
self ! PreventFurtherPolling
remoteService.makeNetworkCall.onComplete {
result => {
self ! AllowFurtherPolling
// process result
}
}
}
}
}
}
trait RemoteServiceThingy {
def makeNetworkCall: Future[String]
}
private case object PreventFurtherPolling
private case object AllowFurtherPolling
case object UpdateLocalCache
And the unit test, in specs2, looks like this:
"when request has finished a new requests can be made" ! {
val remoteService = mock[RemoteServiceThingy]
val actor = TestActorRef(new PollingActor(remoteService))
val slowRequest = new DefaultPromise[String]()
remoteService.makeNetworkCall returns slowRequest
actor.receive(UpdateLocalCache)
actor.receive(UpdateLocalCache)
slowRequest.complete(Left(new Exception))
// Although the test calls the actor synchronously, the actor calls *itself* asynchronously, so we must wait.
Thread.sleep(1000)
actor.receive(UpdateLocalCache)
there was two(remoteService).makeNetworkCall
}
The way we have chosen to solve this for now is to inject the equivalent of an observer into the actor (piggybacking on an existing logger which wasn't included in the listing in the question). The actor can then tell the observer when it has transitioned from various states. In the test code we perform an action then wait for the relevant notification from the actor, before continuing and making assertions.
In the test we have something like this:
actor.receive(UpdateLocalCache)
observer.doActionThenWaitForEvent(
{ actor.receive(UpdateLocalCache) }, // run this action
"IgnoredUpdateLocalCache" // then wait for the actor to emit an event
}
// assert on number of calls to remote service
I don't know if there's a more idiomatic way, this seems like a reasonable suggestion to me.