I'm an intermediate C++ programmer, but I'm new to using Windows' API functions.
I'm trying to create a console program that will sit/sleep until either
The user inputs something in the console and presses enter
Serial data is received on a serial port that's already been opened
Searching around, it sounds like the way to do this in Windows is with Events, (which sound like they're the same basic idea as interrupts?)
I found documentation on the WaitCommEvent, and I've read about reading console input buffer events. I'm guessing the function to use is WaitForMultipleObjects, but what handles specifically do I send it so it will wait for both a serial RX event or a console standard input event?
UPDATE:
Thanks for the response!
Currently I've just been using std::cin/cout to read from and write to the console. I looked over the Console API you mentioned and saw the GetStdHandle function which will give the CONIN$ handle you mentioned. Can I just send that CONIN$handle to the wait function instead of using CreateFile and manually using ReadFile/the Console API like you suggested?
For the Serial, I know how to open my serial handle as OVERLAPPED instead of as NONOVERLAPPED, but I'm not sure what you mean by
it is not usually too difficult to modify synchronous I/O code to use
an asynchronous handle
Something like this?
uint32 read(HANDLE serialHandle, uint8* pBuffer, int32 bufferLenght)
{
DWORD dwBytesRead;
if (!ReadFile(SerialHandle, pBuffer, bufferLength, &dwBytesRead, NULL))
{ /*ERROR*/ }
else
{
// Wait on some flag or variable until read is complete
// to make this call synchronous/NONOVERLAPPED ?
return static_cast<uint32>(dwBytesRead);
}
}
What/where would that flag be to wait on until the read is complete?
From Low-Level Console Input Functions on MSDN:
A thread of an application's process can perform a wait operation to wait for input to be available in an input buffer. To initiate a wait operation, specify a handle to the input buffer in a call to any of the wait functions.
So you need to use a console handle, which you can obtain by calling CreateFile on CONIN$. You will also need to use the same handle, either with ReadFile or the console API, to read the console input; using runtime library functions is likely to mess you up due to buffering.
For the serial port, I believe you will need to use asynchronous I/O. The WaitCommEvent function (when provided with an asynchronous mode handle) accepts an OVERLAPPED structure containing a handle to a manual-reset event object. You would then use the same event handle in the call to WaitForMultipleObjects.
Unfortunately this is an all-or-nothing, so you have to open the COM handle in asynchronous mode and use asynchronous I/O exclusively. (Luckily, it is not usually too difficult to modify synchronous I/O code to use an asynchronous handle, although if there are a lot of I/O calls you might want to write a wrapper function to do the repetitive work of building the OVERLAPPED structure and waiting for the operation to complete.)
Related
This code, which I have no control over, reads a file using overlapped I/O:
// Read file asynchronously
HANDLE hFile = CreateFile(..., FILE_FLAG_OVERLAPPED, ...);
BYTE buffer[10];
OVERLAPPED oRead = { 0 };
ReadFile(hFile, buffer, 10, NULL, &oRead);
// Do work while file is being read
...
// Wait for read to finish
WaitForSingleObject(hFile, INFINITE);
// ReadFile has finished
// buffer now contains data which can be used
...
In another thread (actually in an API hook of ReadFile), I need to signal the hFile to unblock the WaitForSingleObject. Normally Windows (or the device driver handling the ReadFile) does this, but I need to simulate it.
None of the APIs I found that normally do this work with hFile, including ReleaseMutex, ReleaseSemaphore, and SetEvent. They all return Error 6 (handle is invalid). Is there an API that works with a file, named pipe, or communications device?
I know it is not recommended to WaitForSingleObject(hFile), but the above code is a given, and I need to work with it. Thanks!
So far as I know, signaling the file handle takes place internally to Windows, and there is no API even when running in kernel mode. (I believe the file system driver simply tells Windows that the operation is complete and lets Windows figure out how to notify the user-mode process. I may be wrong.)
One resolution would be to issue a genuine ReadFile (a zero-byte read might be sufficient) against the handle in order to signal it.
But it would probably be more sensible to hook WaitForSingleObject, check whether it is being called on the file handle in question, and if so modify the behaviour as appropriate.
My application checks for user input in its main thread:
while (running)
{
std::string console;
if (std::getline(std::cin, console))
{
process(&console);
}
}
Before that I have setup a sigaction to detect CTRL+C in conjunction with a function handler to shutdown other threads.
Now, when a SIGINT occurs the application crashes; GDB output:
I was looking around and found other solutions such as non-blocking input reading: (pseudo-code)
while (running)
{
if (input_avail())
{
getinput
process
}
else
sleep(1);
}
But even that fails for me at the sleep function (nanosleep):
So I'm quite curious on how other people achieve this?
(Using g++ v4.8.2 Kernel 3.10)
Additional info requested:
Before the main thread loop:
struct sigaction sigIntHandler;
sigIntHandler.sa_handler = signalinfo;
sigemptyset(&sigIntHandler.sa_mask);
sigIntHandler.sa_flags = 0;
sigaction(SIGTERM, &sigIntHandler, NULL);
sigaction(SIGQUIT, &sigIntHandler, NULL);
sigaction(SIGINT, &sigIntHandler, NULL);
Signal handler:
void signalinfo(int signum)
{
// Only setting a flag so threads know to exit.
pCore->Termination(signum);
}
Not sure if it answers your question, but the documentation of the XBoard protocol explains some common strategies how engine engine handle reading from stdin.
It basically sketches how you might implement your non-blocking pseudo code.
Source: XBoard protocol (6. Hints on input/output)
... on the input side, you are likely to want to poll during your search and stop it if new input has come in. If you implement pondering, you'll need this so that pondering stops when the user makes a move. You should also poll during normal thinking on your move, so that you can implement the "?" (move now) command, and so that you can respond promptly to a "result", "force", or "quit" command if xboard wants to end the game or terminate your engine. Buffered input makes polling more complicated -- when you poll, you must stop your search if there are either characters in the buffer or characters available from the underlying file descriptor.
The most direct way to fix this problem is to use unbuffered operating system calls to read (and poll) the underlying file descriptor directly. On Unix, use read(0, ...) to read from standard input, and use select() to poll it. See the man pages read(2) and select(2). (Don't follow the example of GNU Chess 4 and use the FIONREAD ioctl to poll for input. It is not very portable; that is, it does not exist on all versions of Unix, and is broken on some that do have it.) On Win32, you can use either the Unix-like _read(0, ...) or the native Win32 ReadFile() to read. Unfortunately, under Win32, the function to use for polling is different depending on whether the input device is a pipe, a console, or something else. (More Microsoft brain damage here -- did they never hear of device independence?) For pipes, you can use PeekNamedPipe to poll (even when the pipe is unnamed). For consoles, you can use GetNumberOfConsoleInputEvents. For sockets only, you can use select(). It might be possible to use WaitForSingleObject more generally, but I have not tried it. Some code to do these things can be found in Crafty's utility.c, but I don't guarantee that it's all correct or optimal.
A second way to fix the problem might be to ask your I/O library not to buffer on input. It should then be safe to poll the underlying file descriptor as described above. With C, you can try calling setbuf(stdin, NULL). However, I have never tried this. Also, there could be problems if you use scanf(), at least with certain patterns, because scanf() sometimes needs to read one extra character and "push it back" into the buffer; hence, there is a one-character pushback buffer even if you asked for stdio to be unbuffered. With C++, you can try cin.rdbuf()->setbuf(NULL, 0), but again, I have never tried this.
A third way to fix the problem is to check whether there are characters in the buffer whenever you poll. C I/O libraries generally do not provide any portable way to do this. Under C++, you can use cin.rdbuf()->in_avail(). This method has been reported to work with EXchess. Remember that if there are no characters in the buffer, you still have to poll the underlying file descriptor too, using the method described above.
A fourth way to fix the problem is to use a separate thread to read from stdin. This way works well if you are familiar with thread programming. This thread can be blocked waiting for input to come in at all times, while the main thread of your engine does its thinking. When input arrives, you have the thread put the input into a buffer and set a flag in a global variable. Your search routine then periodically tests the global variable to see if there is input to process, and stops if there is. WinBoard and my Win32 ports of ICC timestamp and FICS timeseal use threads to handle multiple input sources.
According to MSDN:
hEvent: If an overlapped I/O operation is issued without an I/O
completion routine (the operation's lpCompletionRoutine parameter is
set to null), then this parameter should either contain a valid handle
to a WSAEVENT object or be null.
As I'm using IOCP, when I call WSASend() or WSARecv() I pass NULL to their last parameter (i.e., lpCompletionRoutine):
WSASend(pIoRequest->GetSocket(), pIoRequest->GetWsaBuffer(), 1, NULL, pIoRequest->GetFlags(), pIoRequest, NULL);
WSARecv(pIoRequest->GetSocket(), pIoRequest->GetWsaBuffer(), 1, NULL, &(pIoRequest->GetFlags()), pIoRequest, NULL);
My "per I/O data" class (pIoRequest) looks something like:
class IoRequest : public WSAOVERLAPPED
{
public:
IoRequest()
{
...
SecureZeroMemory(this, sizeof(WSAOVERLAPPED));
hEvent = WSACreateEvent(); // A
}
...
void ResetForNextIoRequest()
{
WSACloseEvent(hEvent); // B
SecureZeroMemory(this, sizeof(WSAOVERLAPPED));
hEvent = WSACreateEvent(); // C
...
}
...
DWORD& GetFlags() { return m_dwFlags; }
...
private:
...
DWORD m_dwFlags;
...
};
It doesn’t seem to make any difference to my program’s behaviour even if I comment out the lines marked A, B and C above.
So how do you decide when to call WSACreateEvent() or simply set hEvent to NULL?
If you're using IOCP, you don't need to pass event objects, because you'll be receiving completion notifications using GetQueuedCompletionStatus(). This will work assuming you have associated the socket with the completion port using CreateIoCompletionPort().
Yes, I/O on Windows is confusing. In particular, there are at least six different ways to use sockets. First the two you seem to have come across:
Overlapped I/O using IOCP (CreateIoCompletionPort, GetQueuedCompletionStatus, WSASend etc.).
This is probably the most efficient method. You can easily integrate into the event loop any kind of event which also use IOCP. For other events, you may be able to workaround using PostQueuedCompletionStatus. This is (AFAIK) the only method which scales for large numbers of sockets.
Overlapped I/O without IOCP, that is using WSASend and friends with event objects, monitoring the events using e.g. WaitForMultipleObjects, and obtaining results using WSAGetOverlappedResult. This is relatively easy to integrate with any non-socket I/O which can also be mapped to HANDLE objects. However, WaitForMultipleObjects it is limited to monitoring no more than 64 handles at a time.
And also at least four more:
Blocking calls (send, recv, and also the WSA* versions). If you do this, you're forced to use threads. This is both hard to implement correctly and likely inefficient.
Non-blocking sockets using select(). This has the benefit that you can use similar code as on unix-like systems. However, it (AFAIK) cannot be integrated with I/O other than sockets.
Non-blocking using WSAEventSelect. This is similar to the select() method, except that instead of using select() to get notifications, you map socket events to event objects, and monitor those using e.g. WaitForMultipleObjects. It is also similar to the overlapped without IOCP method, and suffers from the same limit of no more than 64 objects.
Non-blocking using WSAAsyncSelect. This delivers socket notifications as messages to a window within a program using using the Windows message loop. This is easy to integrate into an application already using the message loop, such as many GUI applications.
Correct me if I left something out or if some of this doesn't actually work :).
I was just wondering whether in C is it possible to peek in the input buffer or perform similar trickery to know whether a call to fgets would block at a later time.
Java allows to do something like that by calling BufferedReader.ready(), this way I can implement console input something like this:
while (on && in.ready()) {
line = in.readLine();
/* do something with line */
if (!in.ready())
Thread.sleep(100);
}
this allows an external thread to gracefully shutdown the input loop by setting on to false; I'd like to perform a similar implementation in C without resorting to non portable tricks, I already know I can make a "timed out fgets" under unix by resorting to signals or (better, even though requering to take care of buffering) reimplement it on top of recv/select, but I'd prefer something that would work on windows too.
TIA
Suggest to go with socket I/O routines,preferably poll() with required millisecond as timeout and eventually you can interpret timeout ( return value = -1 ) as unavailability of data in input buffer.
In my opinion,there is no non-blocking standard I/O function to achieve this functionality.
I'm not certain what are you talking about: a socket or a file handle?
For files there should be no blocking. The function returns immediately (besides of the I/O invocation itself).
For sockets - you may use the ioctlsocket function:
The following tells if there's a rcv data pending:
ULONG nSize;
ioctlsocket(sock, FIONREAD, &nSize);
The following transfers the socket into non-blocking mode:
ULONG nEnable = 1;
ioctlsocket(sock, FIONBIO, &nEnable);
When in Non-blocking mode - functions on socket never block. If they can't fulfill the request they return an error, and the error code is WSAEWOULDBLOCK
Plus, on Windows there're dozens of much more efficient methods. Those are:
Using Overlapped I/O. This is non-trivial, but gives superior performance
Associating socket with a waitable event. This transfers the socket to a non-blocking mode, plus the specified event is signaled when a network event occurs.
Associate it with the window handle. This is convenient for UI-oriented programs.
To simplify, this is a situation where a NamedPipe SERVER is waiting for a NamedPipe CLIENT to write to the pipe (using WriteFile())
The Windows API that is blocking is ReadFile()
The Server has created the synchronous pipe (no overlapped I/O) with blocking enabled
The client has connected, and now the server is waiting for some data.
In the normal flow of things, the client sends some data and the server processes it and then returns to ReadFile() to wait for the next chunk of data.
Meanwhile an event occurs (user input for example) and the NamedPipe SERVER must now execute some other code, which it cannot do while the ReadFile() is blocking.
At this point I need to mention that the NamedPipe Client is not my application, so I have no control over it. I cannot make it send a few bytes to unblock the server. It is just going to sit there and send no data. Since I do not have control of the Client implementation I cannot change anything on that end.
One solution would be to create a separate thread in which all ReadFile() operations are performed. That way when the event occurs, I can just process the code. The problem with that, is that the event also requires a separate thread, so now I have two additional threads for each instance of this server. Since this needs to be scalable, this is undesirable.
From another thread I have tried calling
DisconnectNamedPipe()
and
CloseHandle()
they both will not return (until the client writes to the pipe.)
I cannot connect to the same pipe and write a few bytes because:
"All instances of a named pipe share the same pipe name, but each instance has
its own buffers and handles, and provides a separate conduit for client/server
communication."
http://msdn.microsoft.com/en-us/library/aa365590.aspx
I need a way to fake it out, So the $64k dollar question is:
How can I break the blocking of ReadFile()?
Try this before ReadFile :
BOOL WINAPI PeekNamedPipe(
__in HANDLE hNamedPipe,
__out_opt LPVOID lpBuffer,
__in DWORD nBufferSize,
__out_opt LPDWORD lpBytesRead,
__out_opt LPDWORD lpTotalBytesAvail,
__out_opt LPDWORD lpBytesLeftThisMessage
);
if(TotalBytesAvail > 0)
ReadFile(....);
-AV-
Take a look on CancelSynchronousIo
Marks pending synchronous I/O
operations that are issued by the
specified thread as canceled.
And CancelIo/CancelIoEx:
To cancel all pending asynchronous I/O
operations, use either:
CancelIo — this function only cancels
operations issued by the calling
thread for the specified file handle.
CancelIoEx — this function cancels all
operations issued by the threads for
the specified file handle.
http://msdn.microsoft.com/en-us/library/aa363794(VS.85).aspx
http://msdn.microsoft.com/en-us/library/aa365467(VS.85).aspx
Mike,
You can't cancel synchronous ReadFile. But you can switch to asynchronous (overlapped) operations. By doing this, you can implement a pretty scalable architecture.
Possible algorithm (just an idea):
For each new client call ReadFile
WaitForMultipleObjects where the handles are overlapped.hEvent + your
custom events
Iterate over signalled events, and schedule them for execution by threads from a threads pool.
This way you can have only few threads to receive connections and read data, while the actual data processing can be done by the threads pool.
The problem with that, is that the
event also requires a separate thread,
so now I have two additional threads
for each instance of this server.
Since this needs to be scalable, this
is undesirable.
Never in my career have I found that "more threads" == "less scalable". How many of these "server" instances do you have?
Normally, an operation needs to be performed in a separate thread if that operation is going to block and the system needs to be responsive while the operation is blocked.
Asynchronous I/O operations do not have to block any thread if they use I/O Completion Ports. See: http://msdn.microsoft.com/en-us/library/aa365198(VS.85).aspx
What happening is the server outbound pipe is left open waiting for connection while your client is trying to connect to the server inbound pipe (which is no longer existent)... What you need to do is flush out your outbound pipe in order to loop back to your inbound. You can flush out on the client side by reading the file (remember to loop the connect establishment because there is a "handshake" in there, and it will never work the first time)
Just use SetNamedPipeHandleState function
https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-setnamedpipehandlestate
Use the PIPE_NOWAIT flag when calling this function.
hNamedPipe should be the handle that returned from CreateFile function.
After that, the call to ReadFile will not block the thread when no data available.