This code, which I have no control over, reads a file using overlapped I/O:
// Read file asynchronously
HANDLE hFile = CreateFile(..., FILE_FLAG_OVERLAPPED, ...);
BYTE buffer[10];
OVERLAPPED oRead = { 0 };
ReadFile(hFile, buffer, 10, NULL, &oRead);
// Do work while file is being read
...
// Wait for read to finish
WaitForSingleObject(hFile, INFINITE);
// ReadFile has finished
// buffer now contains data which can be used
...
In another thread (actually in an API hook of ReadFile), I need to signal the hFile to unblock the WaitForSingleObject. Normally Windows (or the device driver handling the ReadFile) does this, but I need to simulate it.
None of the APIs I found that normally do this work with hFile, including ReleaseMutex, ReleaseSemaphore, and SetEvent. They all return Error 6 (handle is invalid). Is there an API that works with a file, named pipe, or communications device?
I know it is not recommended to WaitForSingleObject(hFile), but the above code is a given, and I need to work with it. Thanks!
So far as I know, signaling the file handle takes place internally to Windows, and there is no API even when running in kernel mode. (I believe the file system driver simply tells Windows that the operation is complete and lets Windows figure out how to notify the user-mode process. I may be wrong.)
One resolution would be to issue a genuine ReadFile (a zero-byte read might be sufficient) against the handle in order to signal it.
But it would probably be more sensible to hook WaitForSingleObject, check whether it is being called on the file handle in question, and if so modify the behaviour as appropriate.
Related
I need to do a C++ assignment, but I'm pretty much new to Win32 IPC.
I did a little bit of research yesterday, but I cannot find the thing I'm searching for.
Basically, I need two programs, the first creates a FileMapping with paging file, waits a buffer, display the buffer, and closes it.
The second connects to the Communication Channel, writes the buffer to the first program, and then closes.
The closest thing I've come to is this resource:
IPC Communication
but the guy there uses pipes instead of communication channels using paging file.
Also, I found that I can open a FileMapping with paging file pretty much like this:
TCHAR szMapFileName[] = _T("Local\\HelloWorld");
HANDLE hMapFile = CreateFileMapping(
INVALID_HANDLE_VALUE,
NULL, // Default security
PAGE_READWRITE, // Read/write access
0, // Max. object size
BUFFER_SIZE, // Buffer size
szMapFileName // Name of mapping object);
If somebody can provide a little help that would be very valuable(maybe a skeleton of an app?). I tried to do some research yesterday, but in vain.
Thanks
CreateFileMapping(), OpenFileMapping() and MapViewOfFile() are indeed the functions you need to call to allocate a shared memory buffer.
The first app must:
Create the File Mapping.
Create a synchronization object, preferably an Event, to wait on it.
Create an additional thread, which will be waiting for the Event to be signaled. Doing this in the main (UI) thread would block the application.
When the Event is signaled, post a custom message (WM_APP+nnn) to the main thread, to display the contents of the buffer.
It's a matter of specs or design what to do after this point, eg exit the application, just not be waiting for the buffer to receive data anymore, clear the event and wait again, etc.
The second app must:
Open the File Mapping and the Event. If failed, display an error message or warning and exit.
Write the data to the shared memory buffer.
Signal the Event.
Exit.
It could be further improved, eg the second app may not write to the buffer if the event is not in the nonsignaled state.
I'm an intermediate C++ programmer, but I'm new to using Windows' API functions.
I'm trying to create a console program that will sit/sleep until either
The user inputs something in the console and presses enter
Serial data is received on a serial port that's already been opened
Searching around, it sounds like the way to do this in Windows is with Events, (which sound like they're the same basic idea as interrupts?)
I found documentation on the WaitCommEvent, and I've read about reading console input buffer events. I'm guessing the function to use is WaitForMultipleObjects, but what handles specifically do I send it so it will wait for both a serial RX event or a console standard input event?
UPDATE:
Thanks for the response!
Currently I've just been using std::cin/cout to read from and write to the console. I looked over the Console API you mentioned and saw the GetStdHandle function which will give the CONIN$ handle you mentioned. Can I just send that CONIN$handle to the wait function instead of using CreateFile and manually using ReadFile/the Console API like you suggested?
For the Serial, I know how to open my serial handle as OVERLAPPED instead of as NONOVERLAPPED, but I'm not sure what you mean by
it is not usually too difficult to modify synchronous I/O code to use
an asynchronous handle
Something like this?
uint32 read(HANDLE serialHandle, uint8* pBuffer, int32 bufferLenght)
{
DWORD dwBytesRead;
if (!ReadFile(SerialHandle, pBuffer, bufferLength, &dwBytesRead, NULL))
{ /*ERROR*/ }
else
{
// Wait on some flag or variable until read is complete
// to make this call synchronous/NONOVERLAPPED ?
return static_cast<uint32>(dwBytesRead);
}
}
What/where would that flag be to wait on until the read is complete?
From Low-Level Console Input Functions on MSDN:
A thread of an application's process can perform a wait operation to wait for input to be available in an input buffer. To initiate a wait operation, specify a handle to the input buffer in a call to any of the wait functions.
So you need to use a console handle, which you can obtain by calling CreateFile on CONIN$. You will also need to use the same handle, either with ReadFile or the console API, to read the console input; using runtime library functions is likely to mess you up due to buffering.
For the serial port, I believe you will need to use asynchronous I/O. The WaitCommEvent function (when provided with an asynchronous mode handle) accepts an OVERLAPPED structure containing a handle to a manual-reset event object. You would then use the same event handle in the call to WaitForMultipleObjects.
Unfortunately this is an all-or-nothing, so you have to open the COM handle in asynchronous mode and use asynchronous I/O exclusively. (Luckily, it is not usually too difficult to modify synchronous I/O code to use an asynchronous handle, although if there are a lot of I/O calls you might want to write a wrapper function to do the repetitive work of building the OVERLAPPED structure and waiting for the operation to complete.)
My application checks for user input in its main thread:
while (running)
{
std::string console;
if (std::getline(std::cin, console))
{
process(&console);
}
}
Before that I have setup a sigaction to detect CTRL+C in conjunction with a function handler to shutdown other threads.
Now, when a SIGINT occurs the application crashes; GDB output:
I was looking around and found other solutions such as non-blocking input reading: (pseudo-code)
while (running)
{
if (input_avail())
{
getinput
process
}
else
sleep(1);
}
But even that fails for me at the sleep function (nanosleep):
So I'm quite curious on how other people achieve this?
(Using g++ v4.8.2 Kernel 3.10)
Additional info requested:
Before the main thread loop:
struct sigaction sigIntHandler;
sigIntHandler.sa_handler = signalinfo;
sigemptyset(&sigIntHandler.sa_mask);
sigIntHandler.sa_flags = 0;
sigaction(SIGTERM, &sigIntHandler, NULL);
sigaction(SIGQUIT, &sigIntHandler, NULL);
sigaction(SIGINT, &sigIntHandler, NULL);
Signal handler:
void signalinfo(int signum)
{
// Only setting a flag so threads know to exit.
pCore->Termination(signum);
}
Not sure if it answers your question, but the documentation of the XBoard protocol explains some common strategies how engine engine handle reading from stdin.
It basically sketches how you might implement your non-blocking pseudo code.
Source: XBoard protocol (6. Hints on input/output)
... on the input side, you are likely to want to poll during your search and stop it if new input has come in. If you implement pondering, you'll need this so that pondering stops when the user makes a move. You should also poll during normal thinking on your move, so that you can implement the "?" (move now) command, and so that you can respond promptly to a "result", "force", or "quit" command if xboard wants to end the game or terminate your engine. Buffered input makes polling more complicated -- when you poll, you must stop your search if there are either characters in the buffer or characters available from the underlying file descriptor.
The most direct way to fix this problem is to use unbuffered operating system calls to read (and poll) the underlying file descriptor directly. On Unix, use read(0, ...) to read from standard input, and use select() to poll it. See the man pages read(2) and select(2). (Don't follow the example of GNU Chess 4 and use the FIONREAD ioctl to poll for input. It is not very portable; that is, it does not exist on all versions of Unix, and is broken on some that do have it.) On Win32, you can use either the Unix-like _read(0, ...) or the native Win32 ReadFile() to read. Unfortunately, under Win32, the function to use for polling is different depending on whether the input device is a pipe, a console, or something else. (More Microsoft brain damage here -- did they never hear of device independence?) For pipes, you can use PeekNamedPipe to poll (even when the pipe is unnamed). For consoles, you can use GetNumberOfConsoleInputEvents. For sockets only, you can use select(). It might be possible to use WaitForSingleObject more generally, but I have not tried it. Some code to do these things can be found in Crafty's utility.c, but I don't guarantee that it's all correct or optimal.
A second way to fix the problem might be to ask your I/O library not to buffer on input. It should then be safe to poll the underlying file descriptor as described above. With C, you can try calling setbuf(stdin, NULL). However, I have never tried this. Also, there could be problems if you use scanf(), at least with certain patterns, because scanf() sometimes needs to read one extra character and "push it back" into the buffer; hence, there is a one-character pushback buffer even if you asked for stdio to be unbuffered. With C++, you can try cin.rdbuf()->setbuf(NULL, 0), but again, I have never tried this.
A third way to fix the problem is to check whether there are characters in the buffer whenever you poll. C I/O libraries generally do not provide any portable way to do this. Under C++, you can use cin.rdbuf()->in_avail(). This method has been reported to work with EXchess. Remember that if there are no characters in the buffer, you still have to poll the underlying file descriptor too, using the method described above.
A fourth way to fix the problem is to use a separate thread to read from stdin. This way works well if you are familiar with thread programming. This thread can be blocked waiting for input to come in at all times, while the main thread of your engine does its thinking. When input arrives, you have the thread put the input into a buffer and set a flag in a global variable. Your search routine then periodically tests the global variable to see if there is input to process, and stops if there is. WinBoard and my Win32 ports of ICC timestamp and FICS timeseal use threads to handle multiple input sources.
I am developing a Windows application that has a separate thread for processing user (or 3rd party) application input via stdin.
This thread is designed such that it waits via WaitForMultipleObjects on two events:
A death signal. When this signal is raised, the interface-processing thread shuts down.
An interface signal. When this signal is raised, there is input ready to be read. The input is read and processed.
Under Windows this thread enters a main loop where it Waits for these 2 events (where bWaitAll is FALSE). Waiting on the stdin handle has the effect of signaling when there is input ready to be read, and the other event is set from elsewhere in the application.
This works exactly as I want. It waits for an event to be raised without entering in to a busy-wait, and it waits for both event simutaneously.
I wish to port this functionality to Linux, but I'm not sure how to achieve the desired result. Fundamentally, what I really want is this:
Under Linux, how do I design a thread so that it will respond
immediately to user-input on stdin, yet it can also respond
immediately to a kill-flag being raised from elsewhere in the
application?
In order to accomplish the latter, it seems to me that I cannot use cin, gets, getch or any other function that blocks until the user has entered text. Yet I do not know how to read user input in a console-based application without blocking.
I'm open to any change in architecture (if there's a more Linux-y way to do this) that include having user input processed in a separate thread that can be terminated from elsewhere in the application. I'm using GCC 4.4, and Boost 1.51.
The standard way of doing this in Linux is to use the select(2) system call. However, select is more limited than WaitForMultipleObjects, in that it can only wait on file descriptors, not other kinds of objects (such as events). So, the typical way of working around that is to create a pipe and write a dummy value to the pipe as your "signal".
Something like this:
// Error checking omitted for expository purposes
int pipefd[2];
pipe(pipefd); // Create the pipe
while(1)
{
// Create file descriptor set of stdin and the read end of the pipe
fd_set fds;
FD_ZERO(&fds);
FD_SET(STDIN_FILENO, &fds);
FD_SET(pipefd[0], &fds);
int maxfd = MAX(STDIN_FILENO, pipefd[0]);
// Wait until input becomes available on either stdin or the pipe
int num_available = select(&fds, NULL, NULL, NULL);
// Read & process stdin if possible (will not block)
if (FD_ISSET(STDIN_FILENO, &fds))
{
int n = read(STDIN_FILENO, buffer, size);
...
}
// Read & process pipe if possible (will not block)
if (FD_ISSET(pipefd[0], &fds))
{
char dummy;
read(pipefd[0], &dummy, 1);
// Handle signal (e.g. break out of loop)
}
}
Then to signal to the thread that it's done, just write a single byte to the write end of the pipe:
char dummy = 42;
write(pipefd[1], &dummy, 1);
libev (and several similar incarnations) offers a convenient abstraction around select including being able to pend on signals.
If you have the option to alter the origin of "An interface signal" then you could consider changing it to use raise instead.
When I run this program
OVERLAPPED o;
int main()
{
..
CreateIoCompletionPort(....);
for (int i = 0; i<10; i++)
{
WriteFile(..,&o);
OVERLAPPED* po;
GetQueuedCompletionStatus(..,&po);
}
}
it seems that the WriteFile didn't return until the writing job is done. At the same time , GetQueuedCompletionStatus() gets called. The behavior is like a synchronous IO operation rather than an asynch-IO operation.
Why is that?
If the file handle and volume have write caching enabled, the file operation may complete with just a memory copy to cache, to be flushed lazily later. Since there is no actual IO taking place, there's no reason to do async IO in that case.
Internally, each IO operation is represented by an IRP (IO request packet). It is created by the kernel and given to the filesystem to handle the request, where it passes down through layered drivers until the request becomes an actual disk controller command. That driver will make the request, mark the IRP as pending and return control of the thread. If the handle was opened for overlapped IO, the kernel gives control back to your program immediately. Otherwise, the kernel will wait for the IRP to complete before returning.
Not all IO operations make it all the way to the disk, however. The filesystem may determine that the write should be cached, and not written until later. There is even a special path for operations that can be satisfied entirely using the cache, called fast IO. Even if you make an asynchronous request, fast IO is always synchronous because it's just copying data into and out of cache.
Process monitor, in advanced output mode, displays the different modes and will show blank in the status field while an IRP is pending.
There is a limit to how much data is allowed to be outstanding in the write cache. Once it fills up, the write operations will not complete immediately. Try writing a lot of data at once, with may operations.
I wrote a blog posting a while back entitled "When are asynchronous file writes not asynchronous" and the answer was, unfortunately, "most of the time". See the posting here: http://www.lenholgate.com/blog/2008/02/when-are-asynchronous-file-writes-not-asynchronous.html
The gist of it is:
For security reasons Windows extends files in a synchronous manner
You can attempt to work around this by setting the end of the file to a large value before you start and then trimming the file to the correct size when you finish.
You can tell the cache manager to use your buffers and not its, by using FILE_FLAG_NO_BUFFERING
At least it's not as bad as if you're forced to use FILE_FLAG_WRITE_THROUGH
If GetQueuedCompletionStatus is being called, then the call to WriteFile is synchronous (and it has returned), but it can still modify &o even after it's returned if it is asynchronous.
from this page in MSDN:
For asynchronous write operations,
hFile can be any handle opened with
the CreateFile function using the
FILE_FLAG_OVERLAPPED flag or a socket
handle returned by the socket or
accept function.
also, from this page:
If a handle is provided, it has to
have been opened for overlapped I/O
completion. For example, you must
specify the FILE_FLAG_OVERLAPPED flag
when using the CreateFile function to
obtain the handle.