Reading from a pipe randomly fails - c++

Question

I'm writing an integration test driver for a command-line executable. I control both the driver and the executable, so I can make guarantees about their behaviour- for example, the executable never reads from stdin, it just takes command-line arguments, does its thing, and then writes output to a file and stdout.
I wish to capture both the exit code and the stdout of the process for verification.
Here's the code that I'm using:
#include <Windows.h>
class Pipe {
HANDLE ReadHandle;
HANDLE writehandle;
public:
Pipe() {
SECURITY_ATTRIBUTES saAttr;
saAttr.nLength = sizeof(SECURITY_ATTRIBUTES);
saAttr.bInheritHandle = TRUE;
saAttr.lpSecurityDescriptor = NULL;
CreatePipe(&ReadHandle, &writehandle, &saAttr, 0);
}
HANDLE WriteHandle() {
return writehandle;
}
std::string Contents() {
CloseHandle(writehandle);
DWORD dwRead;
CHAR chBuf[1024];
BOOL bSuccess = FALSE;
std::string result;
for (;;)
{
bSuccess = ReadFile(ReadHandle, chBuf, 1024, &dwRead, NULL);
if (!bSuccess) break;
result += std::string(chBuf, chBuf + dwRead);
if (dwRead < 1024)
break;
}
return result;
}
~Pipe() {
CloseHandle(ReadHandle);
}
};
Wide::Driver::ProcessResult Wide::Driver::StartAndWaitForProcess(std::string name, std::vector<std::string> args, Util::optional<unsigned> timeout)
{
ProcessResult result;
Pipe stdoutpipe;
PROCESS_INFORMATION info = { 0 };
STARTUPINFO startinfo = { sizeof(STARTUPINFO) };
std::string final_args = name;
for (auto arg : args)
final_args += " " + arg;
startinfo.hStdOutput = stdoutpipe.WriteHandle();
startinfo.hStdError = INVALID_HANDLE_VALUE;
startinfo.hStdInput = INVALID_HANDLE_VALUE;
startinfo.dwFlags |= STARTF_USESTDHANDLES;
auto proc = CreateProcess(
name.c_str(),
&final_args[0],
nullptr,
nullptr,
TRUE,
NORMAL_PRIORITY_CLASS | CREATE_NO_WINDOW,
nullptr,
nullptr,
&startinfo,
&info
);
if (!proc) {
DWORD dw = GetLastError();
const char* message;
FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS,
nullptr, dw, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), (LPTSTR)&message, 0, nullptr);
std::string err = message;
LocalFree((void*)message);
throw std::runtime_error(err);
}
if (timeout == 0)
timeout = INFINITE;
result.std_out = stdoutpipe.Contents();
if (WaitForSingleObject(info.hProcess, timeout ? *timeout : INFINITE) == WAIT_TIMEOUT)
TerminateProcess(info.hProcess, 1);
DWORD exit_code;
GetExitCodeProcess(info.hProcess, &exit_code);
CloseHandle(info.hProcess);
CloseHandle(info.hThread);
result.exitcode = exit_code;
if (exit_code != 0)
return result;
return result;
}
I've got 259 integration tests that I run in this way. Some take longer than others. When I run the suite, about 1-3 will fail - different ones each time. I've looked at the outcome in the debugger, and the stdout is cut off half way through. If I don't try to capture the stdout, all the tests succeed every time, so I know that it's based around the stdout capture.
The timeout is specified, but it's a very generous 60 seconds- much longer than the tests take to run ordinarily. I spawn a new process for each test.
How can I capture stdout in a more reliable way, without getting random failures?
As a final note, it takes a long time to run the suite to capture the failure in the debugger, so it may take a while to service any requests for further information.

Answer 1

I have a theory about this, but I'm not wholly certain. The key is in the loop condition for reading the stdout of the process.
std::string result;
for (;;)
{
bSuccess = ReadFile(ReadHandle, chBuf, 1024, &dwRead, NULL);
if (!bSuccess) break;
result += std::string(chBuf, chBuf + dwRead);
if (dwRead < 1024)
break;
}
return result;
There's actually an implicit assumption here. ReadFile is a blocking API, so we would assume that it keeps blocking until it has the data we asked for or input has ended. But I hypothesize that in fact, ReadFile may return before it has a block as big as we asked for, even if the pipe has not terminated. This would cause the input reading loop to terminate.
Since the parent is no longer reading stdout, the child trying to write stdout may block waiting for somebody to clear the buffer- effectively a deadlock since nobody will. Therefore, the timeout will trigger and terminate the process, recording a failure.
The MSDN docs say this:
The ReadFile function returns when one of the following conditions occur:
The number of bytes requested is read.
A write operation completes on the write end of the pipe.
An asynchronous handle is being used and the read is occurring asynchronously.
An error occurs.
It does not say that it will return when a write operation completes and the number of bytes requested is available. In fact, it makes no comment about the write operation making as many bytes as you requested available. So effectively, it behaves semi-asynchronously, even when called synchronously.
I have rewritten the loop as follows:
std::string result;
for (;;)
{
bSuccess = ReadFile(ReadHandle, chBuf, 1024, &dwRead, NULL);
if (!bSuccess || dwRead == 0) break;
result += std::string(chBuf, chBuf + dwRead);
}
return result;
So far I have been unable to reproduce the failures with this loop (and the tests complete noticably faster).