In our code messages travelling on guava's AsyncEventBus are frequently used. Now, in some of the tests some message traffic needs to finish, before assert on the state of the testee (say, some loads are scheduled on the EventBus, and these need to finish, so that we can test if things were loaded correctly).
Unfortunately, at the moment this is handled by the horrible Thread.sleep(delay); approach; very fragile and making tests flaky.
What would be the proper way to test code depending on completion of operations on common AsyncEventBus?
What I do in end-to-end or integration tests (multiple threads involved) where I need to wait until an event occurred is using a CountDownLatch. I would stay far away from sleeping threads as you've already mentioned.
This requires that in your test code you can hook the CountDownLatch.countDown() method into the callback method the EventBus is going to call. I explain that with a short example:
class SomeEventReceiver {
...
#Subscribe public void doSomethingFoo(BarEvent e) {
// your logic
}
...
}
// Unit test
...
CountDownLatch readyToAssert = new CountDownLatch(1); // could be 2 or more depending on your needs
SomeEventReceiver rec = new SomeEventReceiver(...) { // create an anonymous subclass
#Subscribe
#Override
public void doSomethingFoo(BarEvent e) { // override super method
super.doSomethingFoo(e); // execute super method's logic
readyToAssert.countDown(); // signal your test method that it's ready to assert
}
}
// put your events on the event bus and do all other necessary things
...
readyToAssert.await(); // JUnit thread is blocked until event handlers where called
assertXXX(...); // assert whatever needs to be asserted
That's my firsthand approach when testing. Obviously it's easier to do if the to-be-tested classes are designed in a testfriendly manner.
Hope that helped!
Related
I have a class Communicator that tests whether it can connect the test server. Here is how I call it:
class CommunicatorTest
{
public:
CommunicatorTest() {}
bool doTest()
{
bool _success;
Parameters params;
Communicator communicator;
communicator.connect(params, [this, &_success](bool success)
{
statusCode = errorCode;
m_condition.notify_one();
});
std::unique_lock<std::mutex> uGuard(m_mutex);
m_condition.wait(uGuard);
return _success;
}
private:
std::mutex m_mutex;
std::condition_variable m_condition;
};
bool communicatorTest()
{
CommunicatorTest test;
bool success = test.doTest();
return success;
}
TEST(CommunicatorTest, test_eq)
{
EXPECT_EQ(communicatorTest(), true);
}
I tried to use condition and mutex to make this code synchronous but it fails, the logs say only that test was running and immediately finishes.
Is there a way to test success variable from the callback using google tests?
Thanks in advance.
In those cases the best solution is to create a mock that emulates the behavior of the server. You should not rely (unless extremely necessary) in external states when running your tests.
The tests may fail because the server is not connected, there is no internet connection or whatever condition.
You can use something like Google Mock, now part of Google Test suite to emulate the behavior:
class MockServer : public Server {
public:
MOCK_METHOD2(DoConnect, bool());
....
};
Then do something like this:
TEST(ServerTest, CanConnect) {
MockServer s;
EXPECT_CALL(s, DoConnect())
..WillOnce(Return(true));
EXPECT_TRUE(server.isConnected());
}
You can simulate the error handling:
TEST(ServerTest, CannotConnect) {
MockServer s;
EXPECT_CALL(s, DoConnect())
..WillOnce(Return(false));
EXPECT_FALSE(server.isConnected());
// ... Check the rest of your variables or states that may be false and
// check that the error handler is working properly
}
As someone who writes asynchronous code, I have stumbled upon this problem many times - it seems most existing C/C++ test frameworks don't have real support for testing asynchronous code. What is mainly needed is a event loop where you can schedule things to be executed (to mock timed external events, etc), and a mechanism to register responses and optionally check the order in which they occur. So, instead of trying to somehow adopt an existing framework (which would result in probably greater effort), I created my own one. I have been using it to test a javascript-like promise class that I have developed, and it's done the job good for me. If you are interested, I have just published it on GitHub:
https://github.com/alxvasilev/async-test
I am using tasks in WinForms (.NET 4.0) to perform lengthy operations like WCF call. Application is already in product with heavy use of Tasks (almost all the methods which uses Tasks are void).
During the unit testing we have used AutoResetEvents (in actual code) to find out when the given task is completed then perform assert.
This gives me a thought that almost all the AutoResetEvent are waste of effort. They are just fulfilling unit testing needs, nothing else.
Can we create a wrapper around Tasks likewise when actual code run... they should work in background and in case of unit testing they should be synchronous.
Similar to below link for BackgroundWorker.
http://si-w.co.uk/blog/2009/09/11/unit-testing-code-that-uses-a-backgroundworker/
Why can't you simply use the continuation for tasks in your wrapper, like this:
var task = ...
task.ContinueWith(t => check task results here)
Also, unit tests can be marked as async, if they have a return type Task, so you can use an await there, and after that do your asserts:
[Test]
public async Task SynchronizeTestWithRecurringOperationViaAwait()
{
var sut = new SystemUnderTest();
// Execute code to set up timer with 1 sec delay and interval.
var firstNotification = sut.StartRecurring();
// Wait that operation has finished two times.
var secondNotification = await firstNotification.GetNext();
await secondNotification.GetNext();
// Assert outcome.
Assert.AreEqual("Init Poll Poll", sut.Message);
}
Another approach (from the same article) is to use a custom task scheduler, which will be synchronous in case of unit testing:
[Test]
public void TestCodeSynchronously()
{
var dts = new DeterministicTaskScheduler();
var sut = new SystemUnderTest(dts);
// Execute code to schedule first operation and return immediately.
sut.StartAsynchronousOperation();
// Execute all operations on the current thread.
dts.RunTasksUntilIdle();
// Assert outcome of the two operations.
Assert.AreEqual("Init Work1 Work2", sut.Message);
}
Same MSDN magazine contains nice article about best practices for async unit testing. Also async void should be used only as an event handler, all other methods should have async Task signature.
I have an Actor and when it recieves a StartMessage, it should change state using Become(Started). How do I unit test whether or not the Actor's state has changed to Started() ?
MyActor class
public class MyActor : ReceiveActor
{
public MyActor()
{
Receive<StartMessage>(s => {
Become(Started); // This is what I want to unit test
});
}
private void Started()
{
Console.WriteLine("Woo hoo! I'm started!");
}
}
Unit Test
[TestMethod]
public void My_actor_changes_state_to_started()
{
// Arrange
var actor = ActorOfAsTestActorRef<MyActor>(Props.Create(() => new MyActor()));
// Act
actor.Tell(new StartMessage());
// Assert
var actorsCurrentState = actor.UnderlyingActor.STATE; // <-- This doesn't work
Assert.AreEqual(Started, actorsCurrentState);
}
UPDATE
Related to the answer from tomliversidge: My reason for writing this unit test was academic but in reality, it's not a good unit test which is why you aren't able to do it as I'd hoped. From Petabridge's Unit Testing Guide:
In reality, if one actor wants to know the internal state of another actor then it must send that actor a message. I recommend you follow the same pattern in your tests and don’t abuse the TestActorRef. Stick to the messaging model in your tests that you actually use in your application.
You would normal test this by message passing. For example, what messages do you process in the Started state? I'm presuming your example has been simplified to the Console.WriteLine action inside of Started.
If you send the StartMessage and then a second message that is processed when in the Started state you can then assert on a response to this second message.
As a simple suggestion:
private void Started()
{
Receive<StartMessage>(msg => {
Sender.Tell(new AlreadyStarted());
}
}
if StartMessage is received whilst in the Started state, you can then assert on receiving an AlreadyStarted message.
For more info check out the Petabridge article https://petabridge.com/blog/how-to-unit-test-akkadotnet-actors-akka-testkit/
I have a Scala unit test for an Akka actor. The actor is designed to poll a remote system and update a local cache. Part of the actor's design is that it doesn't attempt to poll while it's still processing or awaiting the result of the last poll, to avoid flooding the remote system when it experiences a slowdown.
I have a test case (shown below) which uses Mockito to simulate a slow network call, and checks that when the actor is told to update, it won't make another network call until the current one is complete. It checks the actor has not made another call by verifying a lack of interactions with the remote service.
I want to eliminate the call to Thread.sleep. I want to test the functionality of the actor without relying on waiting for a hardcoded time, in every test run, which is brittle, and wastes time. The test can poll or block, waiting for a condition, with a timeout. This will be more robust, and will not waste time when the test is passing. I also have the added constraint that I want to keep the state used to prevent extra polling var allowPoll limited in scope, to the internals of the PollingActor.
is there a way force a wait until the actor is finished messaging itself? If there's a way I can wait until then before trying to assert.
is it necessary to send the internal message at all? Couldn't I maintain the internal state with a threadsafe datastructure, such as java.util.concurrent.AtomicBoolean. I have done this and the code appears to work, but I'm not knowledgeable enough about Akka to know if it's discouraged -- a colleague recommended the self message style.
is there better, out-of-the-box functionality with the same semantics? Then I would opt for an integration test instead of a unit test, though I'm not sure if it would solve this problem.
The current actor looks something like this:
class PollingActor(val remoteService: RemoteServiceThingy) extends ActWhenActiveActor {
private var allowPoll: Boolean = true
def receive = {
case PreventFurtherPolling => {
allowPoll = false
}
case AllowFurtherPolling => {
allowPoll = true
}
case UpdateLocalCache => {
if (allowPoll) {
self ! PreventFurtherPolling
remoteService.makeNetworkCall.onComplete {
result => {
self ! AllowFurtherPolling
// process result
}
}
}
}
}
}
trait RemoteServiceThingy {
def makeNetworkCall: Future[String]
}
private case object PreventFurtherPolling
private case object AllowFurtherPolling
case object UpdateLocalCache
And the unit test, in specs2, looks like this:
"when request has finished a new requests can be made" ! {
val remoteService = mock[RemoteServiceThingy]
val actor = TestActorRef(new PollingActor(remoteService))
val slowRequest = new DefaultPromise[String]()
remoteService.makeNetworkCall returns slowRequest
actor.receive(UpdateLocalCache)
actor.receive(UpdateLocalCache)
slowRequest.complete(Left(new Exception))
// Although the test calls the actor synchronously, the actor calls *itself* asynchronously, so we must wait.
Thread.sleep(1000)
actor.receive(UpdateLocalCache)
there was two(remoteService).makeNetworkCall
}
The way we have chosen to solve this for now is to inject the equivalent of an observer into the actor (piggybacking on an existing logger which wasn't included in the listing in the question). The actor can then tell the observer when it has transitioned from various states. In the test code we perform an action then wait for the relevant notification from the actor, before continuing and making assertions.
In the test we have something like this:
actor.receive(UpdateLocalCache)
observer.doActionThenWaitForEvent(
{ actor.receive(UpdateLocalCache) }, // run this action
"IgnoredUpdateLocalCache" // then wait for the actor to emit an event
}
// assert on number of calls to remote service
I don't know if there's a more idiomatic way, this seems like a reasonable suggestion to me.
I have a bunch of unit tests that work on a class which executes some tasks asynchronously. Currently I set up the class, then execute the function in question. I then go into a blocking wait until the execution is completed, then test the data I need to. Is there a better way that I can do this?
So far my tests look similar to this:
vp.Reset(); //vp is my virtual machine
bool wait = true;
Dictionary<short, Command> commands = new Dictionary<short, Command>();
commands.Add(0, CommandFactory.CreateInputCommand(0, 0));
commands.Add(1, CommandFactory.CreateHaltCommand());
vp.OnHalted += () =>
{
wait = false;
};
vp.InputChannels[0] = () => { return 2; };
vp.CurrentProgram = commands;
vp.ExecuteTillHalt();//asynchronous execution of program. There's also a way to make it execute synchronously
while (wait) ;
Assert.AreEqual(vp.Registers[0], 2);
I see two tests, one to test that an asychronous action is launched (which can be done with a mock) and the second to test that the action that is run asychronously performs correctly when run and you can do the latter synchronously.
Is there a way to break this up into two tests?
You can test the asynchronous aspect via the block and then test the data as a separate testable function that doesn't rely on how it's called.