I have a class Communicator that tests whether it can connect the test server. Here is how I call it:
class CommunicatorTest
{
public:
CommunicatorTest() {}
bool doTest()
{
bool _success;
Parameters params;
Communicator communicator;
communicator.connect(params, [this, &_success](bool success)
{
statusCode = errorCode;
m_condition.notify_one();
});
std::unique_lock<std::mutex> uGuard(m_mutex);
m_condition.wait(uGuard);
return _success;
}
private:
std::mutex m_mutex;
std::condition_variable m_condition;
};
bool communicatorTest()
{
CommunicatorTest test;
bool success = test.doTest();
return success;
}
TEST(CommunicatorTest, test_eq)
{
EXPECT_EQ(communicatorTest(), true);
}
I tried to use condition and mutex to make this code synchronous but it fails, the logs say only that test was running and immediately finishes.
Is there a way to test success variable from the callback using google tests?
Thanks in advance.
In those cases the best solution is to create a mock that emulates the behavior of the server. You should not rely (unless extremely necessary) in external states when running your tests.
The tests may fail because the server is not connected, there is no internet connection or whatever condition.
You can use something like Google Mock, now part of Google Test suite to emulate the behavior:
class MockServer : public Server {
public:
MOCK_METHOD2(DoConnect, bool());
....
};
Then do something like this:
TEST(ServerTest, CanConnect) {
MockServer s;
EXPECT_CALL(s, DoConnect())
..WillOnce(Return(true));
EXPECT_TRUE(server.isConnected());
}
You can simulate the error handling:
TEST(ServerTest, CannotConnect) {
MockServer s;
EXPECT_CALL(s, DoConnect())
..WillOnce(Return(false));
EXPECT_FALSE(server.isConnected());
// ... Check the rest of your variables or states that may be false and
// check that the error handler is working properly
}
As someone who writes asynchronous code, I have stumbled upon this problem many times - it seems most existing C/C++ test frameworks don't have real support for testing asynchronous code. What is mainly needed is a event loop where you can schedule things to be executed (to mock timed external events, etc), and a mechanism to register responses and optionally check the order in which they occur. So, instead of trying to somehow adopt an existing framework (which would result in probably greater effort), I created my own one. I have been using it to test a javascript-like promise class that I have developed, and it's done the job good for me. If you are interested, I have just published it on GitHub:
https://github.com/alxvasilev/async-test
Related
Using Google Test, I want to test the behaviour of a ClientListener.AcceptRequest method:
class ClientListener {
public:
// Clients can call this method, want to test that it works
Result AcceptRequest(const Request& request) {
queue_.Add(request);
... blocks waiting for result ...
return result;
}
private:
// Executed by the background_thread_;
void ProcessRequestsInQueue() {
while (true) {
Process(queue_.PopEarliest());
}
}
MyQueue queue_;
std::thread background_thread_ = thread([this] {ProcessRequestsInQueue();});
};
The method accepts a client request, queues it, blocks waiting for a result, returns a result when available.
The result is available when the background thread processes the corresponding request from a queue.
I have a test which looks as follows:
TEST(ListenerTest, TwoRequests) {
ClientListener listener;
Result r1 = listener.AcceptClientRequest(request1);
Result r2 = listener.AcceptClientRequest(request2);
ASSERT_EQ(r1, correctResultFor1);
ASSERT_EQ(r2, correctResultFor2);
}
Since the implementation of a ClientListener class involves multiple threads, this test might pass on one attempt but fail on another. To increase the chance of capturing a bug, I run the test multiple times:
TEST_P(ListenerTest, TwoRequests) {
... same as before ...
}
INSTANTIATE_TEST_CASE_P(Instantiation, ListenerTest, Range(0, 100));
But now make test command treats each parameterised instantiation as a separate test,
and in the logs, I see 100 tests:
Test 1: Instantiation/ListenerTest.TwoRequests/1
Test 2: Instantiation/ListenerTest.TwoRequests/2
...
Test 100: Instantiation/ListenerTest.TwoRequests/100
Given that I do not use the parameter value, is there a way to rewrite the testing code such that the make test command would log a single test executed 100 times, rather than 100 tests?
Simple answer: use --gtest_repeat when executing tests would do the trick (default is 1).
Longer answer: unit tests shouldn't be used for this kind of tests. GTest is thread-safe by design (as stated in their README), but this doesn't mean it is a good tool to perform such tests. Maybe it is a good starting point to actually begin working on real integration tests, I really recommend Python's behave framework for this purpose.
In our code messages travelling on guava's AsyncEventBus are frequently used. Now, in some of the tests some message traffic needs to finish, before assert on the state of the testee (say, some loads are scheduled on the EventBus, and these need to finish, so that we can test if things were loaded correctly).
Unfortunately, at the moment this is handled by the horrible Thread.sleep(delay); approach; very fragile and making tests flaky.
What would be the proper way to test code depending on completion of operations on common AsyncEventBus?
What I do in end-to-end or integration tests (multiple threads involved) where I need to wait until an event occurred is using a CountDownLatch. I would stay far away from sleeping threads as you've already mentioned.
This requires that in your test code you can hook the CountDownLatch.countDown() method into the callback method the EventBus is going to call. I explain that with a short example:
class SomeEventReceiver {
...
#Subscribe public void doSomethingFoo(BarEvent e) {
// your logic
}
...
}
// Unit test
...
CountDownLatch readyToAssert = new CountDownLatch(1); // could be 2 or more depending on your needs
SomeEventReceiver rec = new SomeEventReceiver(...) { // create an anonymous subclass
#Subscribe
#Override
public void doSomethingFoo(BarEvent e) { // override super method
super.doSomethingFoo(e); // execute super method's logic
readyToAssert.countDown(); // signal your test method that it's ready to assert
}
}
// put your events on the event bus and do all other necessary things
...
readyToAssert.await(); // JUnit thread is blocked until event handlers where called
assertXXX(...); // assert whatever needs to be asserted
That's my firsthand approach when testing. Obviously it's easier to do if the to-be-tested classes are designed in a testfriendly manner.
Hope that helped!
I know that there have been a few posts about this already but I wanted to post one with a concrete example to focus on the gray areas you face when choosing between testing private/internal methods and refactoring into a public class.
Say I have a simple class I want to test that has some internal code refactored into a private or internal method.
Example:
public class Guy
{
public void DoSomeWork()
{
try
{
//do some work
}
catch(Exception e)
{
LogError(e.Message);
}
try
{
//do some more work
}
catch(SomeSpecificException e)
{
LogError(e.Message);
}
}
private void LogError(string message)
{
//if logging folder doesn't exist, create it
//create a log file
//log the message
}
}
Most people would advise that I should take the error logging logic out and put it in a class marked public because it's starting to be complex enough to stand on its own, but the assembly's internal error logging logic shouldn't be exposed publicly--it's not a logging assembly.
If I don't do that, then every time I have code that could call LogError(), I need to have subsequent tests that retest all of the internal logic for LogError(). That quickly becomes oppressive.
I also understand I can mark the method as internal and make it visible to a testing assembly but it's not very "pure."
What sort of guidelines can I abide by in this type of situation?
but the assembly's internal error logging logic shouldn't be exposed publicly
That's true. It can be internal and tested as such. But this
private void LogError(string message)
{
//if logging folder doesn't exist, create it
//create a log file
//log the message
}
Calls for a separate component. Create folder? Create file? Whatever DoSomeWork does, it should have nothing to do with creating log file internals. All it should require is simple "log this message"-feature. Files, folders, formats -- it's all specific to logging component and shouldn't leak to any other.
Your sample code is perfect example of single responsibility principle violation. Extracting logging to fix it solves your testing issues completely.
One possible way to handle this is instead of logging the error in your DoSomeWork method, you simply raise the error, and let it get handled upstream. Then you test that the error was raised, and for the purposes of testing the DoSomeWork method, you don't care what happens after that.
Whether or not this is a good idea depends a lot on how the rest of your program is structured, but having LogError sprinkled all over the place may indicate you need a broader error handling strategy that this approach requires.
UPDATE:
public void DoSomeWork() {
try {
WorkPartA()
}
catch(Exception e) {
LogError(e.Message)
}
try {
WorkPartB()
}
catch(SomeSpecificException e) {
LogError(e.Message)
}
}
public void WorkPartA() {
//do some work
}
public void WorkPartB() {
//do some more work
}
As a response to your comment below, the problem might be that you need to break up your code into smaller pieces. This approach allows you to test raising the exception within WorkPartA() and WorkPartB() individually, while still allowing the process to run if WorkPartA fails. As far as testing DoSomeWork() in this case, I might not even bother. There's a high probability it would be a very low-value test anyway.
I have a Scala unit test for an Akka actor. The actor is designed to poll a remote system and update a local cache. Part of the actor's design is that it doesn't attempt to poll while it's still processing or awaiting the result of the last poll, to avoid flooding the remote system when it experiences a slowdown.
I have a test case (shown below) which uses Mockito to simulate a slow network call, and checks that when the actor is told to update, it won't make another network call until the current one is complete. It checks the actor has not made another call by verifying a lack of interactions with the remote service.
I want to eliminate the call to Thread.sleep. I want to test the functionality of the actor without relying on waiting for a hardcoded time, in every test run, which is brittle, and wastes time. The test can poll or block, waiting for a condition, with a timeout. This will be more robust, and will not waste time when the test is passing. I also have the added constraint that I want to keep the state used to prevent extra polling var allowPoll limited in scope, to the internals of the PollingActor.
is there a way force a wait until the actor is finished messaging itself? If there's a way I can wait until then before trying to assert.
is it necessary to send the internal message at all? Couldn't I maintain the internal state with a threadsafe datastructure, such as java.util.concurrent.AtomicBoolean. I have done this and the code appears to work, but I'm not knowledgeable enough about Akka to know if it's discouraged -- a colleague recommended the self message style.
is there better, out-of-the-box functionality with the same semantics? Then I would opt for an integration test instead of a unit test, though I'm not sure if it would solve this problem.
The current actor looks something like this:
class PollingActor(val remoteService: RemoteServiceThingy) extends ActWhenActiveActor {
private var allowPoll: Boolean = true
def receive = {
case PreventFurtherPolling => {
allowPoll = false
}
case AllowFurtherPolling => {
allowPoll = true
}
case UpdateLocalCache => {
if (allowPoll) {
self ! PreventFurtherPolling
remoteService.makeNetworkCall.onComplete {
result => {
self ! AllowFurtherPolling
// process result
}
}
}
}
}
}
trait RemoteServiceThingy {
def makeNetworkCall: Future[String]
}
private case object PreventFurtherPolling
private case object AllowFurtherPolling
case object UpdateLocalCache
And the unit test, in specs2, looks like this:
"when request has finished a new requests can be made" ! {
val remoteService = mock[RemoteServiceThingy]
val actor = TestActorRef(new PollingActor(remoteService))
val slowRequest = new DefaultPromise[String]()
remoteService.makeNetworkCall returns slowRequest
actor.receive(UpdateLocalCache)
actor.receive(UpdateLocalCache)
slowRequest.complete(Left(new Exception))
// Although the test calls the actor synchronously, the actor calls *itself* asynchronously, so we must wait.
Thread.sleep(1000)
actor.receive(UpdateLocalCache)
there was two(remoteService).makeNetworkCall
}
The way we have chosen to solve this for now is to inject the equivalent of an observer into the actor (piggybacking on an existing logger which wasn't included in the listing in the question). The actor can then tell the observer when it has transitioned from various states. In the test code we perform an action then wait for the relevant notification from the actor, before continuing and making assertions.
In the test we have something like this:
actor.receive(UpdateLocalCache)
observer.doActionThenWaitForEvent(
{ actor.receive(UpdateLocalCache) }, // run this action
"IgnoredUpdateLocalCache" // then wait for the actor to emit an event
}
// assert on number of calls to remote service
I don't know if there's a more idiomatic way, this seems like a reasonable suggestion to me.
What is the correct way to write a unit test for a synchronous method calling async methods.
Right now my unit test are passing, but when I try to open the page, it never returns.
Why isn't my unit test failing? How can I make it fail?
I replicated my problem with this simple code:
My passing test:
[TestMethod]
public void DoSomeWork_WhenWeDoSomeWork_ShouldReturnDone()
{
var service = new SyncService();
const string expected = "Done";
var actual = service.DoSomeWork();
Assert.AreEqual(expected, actual);
}
My view that never returns:
public ActionResult Index()
{
var syncService = new SyncService();
return View((object)syncService.DoSomeWork());
}
My service that never returns to view:
public class SyncService
{
public string DoSomeWork()
{
return SomeWork().GetAwaiter().GetResult();
}
private async Task<string> SomeWork()
{
var task1 = Task.Delay(1000);
var task2 = Task.Delay(1000);
await Task.WhenAll(task1, task2);
return "Done";
}
}
I don't think I can help you with this specific example, but I think a good general strategy is to write two tests. One to test if the synchronous method passes the correct data and an other to test if the asynchronous method works properly.
I mostly work in JavaScript and that general approach works for me. Also you can check the documentation of your testing frameworks, maybe it provides some methods for this.
First, don't block on async code (link to my blog). By blocking on async code, you're actually causing a deadlock. This deadlock does not happen in your unit test because unit tests run in a thread pool context, not an ASP.NET context (link to my blog).
There are good reasons for not having synchronous wrappers for asynchronous methods. So I recommend getting rid of DoSomeWork completely, leaving only SomeWork (renamed to SomeWorkAsync).
To solve your problem, you should use asynchronous controller actions.