All I can find is tutorials on how to use Overlapped I/O, but I can't find why is it called like that.
Is it because for example I can read something from a socket, and then read something else before necessarily the first read returns the bytes read?
The classic meaning dates back to the 1960's (or ealier), where overlapped I/O meant that multiple I/O transfers (normally each I/O to a different device) could occur at the same time (like concurrent reads from tape and writes to disk). An alternative classic name for this was concurrent I/O. This could be accomplished via interrupts and/or hardware similar to DMA (in those days, some of the DMA hardware implementations were more like a set of small processors)
Example article for IBM mainframe:
Overlapped I/O - IBM
I think the idea was (20 years ago) that you could start an IO, perform some computation or other work and later wait for the result. This is rarely done today. I think this idea comes from a time where select and poll were considered state of the art.
A better name would be asynchronous IO. That's what every other platform seems to call it. In fact the MSDN documentation mixes the two terms.
Overlap operation in Microsoft Windows means nothing else as asynchron in everybody else OS-language.
To stick to your example, you start a read on a socket and do not wait for success but do something completely different (maybe read on a different(!) socket). Then ask if the first operation is finished.
You can also set an event-handle for that. Or give a CALLBACK function which is called on completion.
In that case, the first call "overlapped" the rest of your operations.
Look also at wikipedia.
My guess why Microsoft is(was) calling it overlapped is that it is not like starting a thread, more like to start an async task at a time there was no standard name for it. It is more like std::async than std::thread.
Related
I have a pipe handle that was created as overlapped. I need to read and write data to it in parallel. How can I achieve this?
Named Pipe Server Using Overlapped I/O documentation page shows an example how to read and write to many pipes, but "it avoids simultaneous operations on a single pipe instance".
What is the right way to do this in C++ on Windows? I can't file the right example nor some help on the topic.
The main problem I face that normal ReadFile blocks when there is no data to read and eventually I can't write with WriteFile. I haven't found any method that can tell me is there something to read that don't block. As far as I understand I need to pass OVERLAPPED structure but don't know how to use it in case of parallel read and write to one pipe (not many).
It should be possible as it is said in Synchronous and Overlapped Pipe I/O:
Overlapped operations make it possible for one pipe to read and write data simultaneously and for a single thread to perform simultaneous I/O operations on multiple pipe handles.
All you need to do is to provide a different OVERLAPPED structure to each of the simultaneous operations. In your case, all that means is that each of the two threads needs its own OVERLAPPED structure. Since the threads are presumably running different functions, this should happen automatically unless you mess it up by using a global variable.
Note that you're over-complicating things by starting from that sample, which is focused on using overlapped I/O to avoid the need for multiple threads.
Instead, pretend you're writing each of the two functions using non-overlapped I/O, but whenever you would call ReadFile or WriteFile, include a valid OVERLAPPED structure with an event handle and follow up with WaitForSingleObject. There are a few things you should know: you have to make sure that the threads each create their own event object, and you have to handle the case where the I/O operation completes immediately (i.e., returns ERROR_SUCCESS instead of ERROR_IO_PENDING). Otherwise it's all fairly straightforward.
If you can't make it work, show your code.
As the documentation for pipes says, one process writes, another process reads. If you want to READ and WRITE, you will want two pipes, one for the "write to the other process", and one for "read the data from the other process".
[This is not unique to Windows, but since you are asking about Windows pipes, I thought it best to give the Windows docs. Linux/Unix pipes are the same way - they have two ends, a read end and a write end]
Of course, as the comment says, it seems like Windows documentations is rather contradictory (and I've only ever used windows pipes in one direction at a time).
Whilst this example doesn't read and write SIMULTANEOUSLY, I think it could relatively easily be altered so that it does.
I suspect (but since the code isn't posted) the problem is either in the call to ReadFile or in setting up the pipe itself. Overlapped calls to ReadFile are asynchronous, and you'd be required to wait for the event associated with the overlapped structure with WaitForMultipleObjects, before checking the results.
Obviously, if you are reading and writing simultaneously, you need one overlapped struct for read and one for write, to indicate which side "completed".
I'm working on a project, where a primary server thread needs to dispatch events to a series of worker threads. The work that goes on in the worker threads relies on polling (ie. epoll or kqueue depending on the UNIX system in question) with timeouts on these operations needing to be handles. This means, that a normal conditional variable or semaphore structure is not viable for this dispatch, as it would make one or the other block resulting in an unwanted latency between either handling the events coming from polling or the events originating from the server thread.
So, I'm wondering what the most optimal construct for dispatching such events between threads in a pollable fashion is? Essentially, all that needs to be delivered is a pollable "signal" that tells the worker thread, that it has more events to fetch. I've looked at using UNIX pipes (unnamed ones, as it's internal to the process) which seems like a decent solution given that a single byte can be written to the pipe and read back out when the queue is cleared -- but, I'm wondering if this is the best approach available? Or the fastest?
Alternatively, there is the possibility to use signalfd(2) on Linux, but as this is not available on BSD systems, I'd rather like to avoid this construct. I'm also wondering how great the overhead in using system signals actually is?
Jan Hudec's answer is correct, although I wouldn't recommend using signals for a few reasons:
Older versions of glibc emulated pselect and ppoll in a non-atomic fashion, making them basically worthless. Even when you used the mask correctly, signals could get "lost" between the pthread_sigprocmask and select calls, meaning they don't cause EINTR.
I'm not sure signalfd is any more efficient than the pipe. (Haven't tested it, but I don't have any particular reason to believe it is.)
signals are generally a pain to get right. I've spent a lot of effort on them (see my sigsafe library) and I'd recommend avoiding them if you can.
Since you're trying to have asynchronous handling portable to several systems, I'd recommend looking at libevent. It will abstract epoll or kqueue for you, and it will even wake up workers on your behalf when you add a new event. See event.c
2058 static inline int
2059 event_add_internal(struct event *ev, const struct timeval *tv,
2060 int tv_is_absolute)
2061 {
...
2189 /* if we are not in the right thread, we need to wake up the loop */
2190 if (res != -1 && notify && EVBASE_NEED_NOTIFY(base))
2191 evthread_notify_base(base);
...
2196 }
Also,
The worker thread deals with both socket I/O and asynchronous disk I/O, which means that it is optimally always waiting for the event queuing mechanism (epoll/kqueue).
You're likely to be disappointed here. These event queueing mechanisms don't really support asynchronous disk I/O. See this recent thread for more details.
As far as performance goes, the cost of system call is comparably huge to other operations, so it's the number of system calls that matters. There are two options:
Use the pipes as you wrote. If you have any useful payload for the message, you get one system call to send, one system call to wait and one system call to receive. Try to pass any relevant data down the pipe instead of reading them from a shared structure to avoid additional overhead from locking.
The select and poll have variants, that also waits for signals (pselect, ppoll). Linux epoll can do the same using signalfd, so it remains a question whether kqueue can wait for signals, which I don't know. If it can, than you could use them (you are using different mechanism on Linux and *BSD anyway). It would save you the syscall for reading if you don't have good use for the passed data.
I would expect passing the data over socket to be more efficient if it allows you do do away with any other locking.
i have created two classes. One for input reading (through an istream object) and parsing and the other one for processing the output of the parser.
There is one instance of each of those.
I have the parser running in a loop calling istream::get() and then creating commands for the second object based upon the input. These commands are then put on a queue which the second object processes in a separate thread.
Now it is quite obvious that I eventually need to be able to send a "Quit" command. Here the problem arises though: The "Quit" command needs to end the parsing loop as well but I can't find a way to signal the parser that it should quit because it is caught within istream::get().
I would need a way to wake it from that method, but I cannot find any...
I have thought of writing some sort of "termination sequence" to the istream object (which in this case is cin) by creating an ostream object from istream::rdbuf(). But that doesn't work - The badbit is set after the attempt to write to the buffer.
In another question at StackOverflow I saw the asio class of the Boost library mentioned, but I'd rather not depend on third party libraries.
Is there a way to wake the thread from istream::get() - i.e. is there a way to write to the istream buffer (maybe assuming it actually is cin) from within the program?
Another approach would be to kill the thread which I could find acceptable as well since there is no cleanup needed in that specific place. But how can this be done? (I'm relying on a POSIX thread implementation)
You will have to depend on something other than the standard iostream classes, because they don't provide select()-style behaviour.
Also, killing the thread is impossible with POSIX (and utterly broken in Windows). You can issue a cancellation request via pthread_cancel(), but in your case, it may be stuck in an un-cancellable system call. Of particular interest to you, read() may or may not be cancellable, depending on the environment. At least one environment says that a cancellation point may occur in read(), though admittedly it is a Windows POSIX layer. Also, Mac OS X, as recently as Leopard 10.5.1, had a broken read() implementation with respect to cancellability.
Once past this hurdle, you also have to consider the uneasy relationship between C++ destructors and pthread_cancel. Not all environments guarantee that destructors will be called, so you have to be extremely cautions when using pthread_cancel in C++ code.
In short, for interruptible I/O, use low-level I/O and select(): one fd for I/O, a second fd (created by pipe()) for signalling. Or, if you're brave, use AIO, but you're probably better off using a high level interface such as Boost.Asio.
Any chance this is implemented in .NET? - if so take a look at the Reactive Framework.
It provides a very elegant way of handling streams and especially cancelling them on the fly.
On top of this, you get a very extensible library of Linq extension for all sorts of stuff, like Buffering, Memoization, Zip ect..
We use it a lot for transforming (and parsing), modelling of streamed data.
Jeff from the Reative team has a couble of nice blogs about Streaming and Reative here:
I am writing an application needs to use large audio multi-samples, usually around 50 mb in size. One file contains approximately 80 individual short sound recordings, which can get played back by my application at any time. For this reason all the audio data gets loaded into memory for quick access.
However, when loading one of these files, it can take many seconds to put into memory because I need to read a large amount of data with ifstream, meaning my program GUI is temporarily frozen. I have tried memory mapping my file but this causes huge CPU spikes and a mess of audio every time I need to jump to a different area of the file, which is not acceptable.
So this has led me to think that performing an Asynchronous file read will solve my problem, that is the data gets read in a different process and calls a function on completion. This needs to be both compatible for Mac OS X and Windows and in C++.
EDIT: Don't want to use the Boost library because I want to keep a small code base.
boost has an asio library, which I've not used before (it's not on NASA's list of approved third-party libraries).
My own approach has been to write the file reading code twice, once for Windows, once for the POSIX aio API, and then just pick the right one to link with.
For Windows, use OVERLAPPED (you have to enable it in the CreateFile call, then pass an OVERLAPPED structure when you read). You can either have it set an event on completion (ReadFile) or call a completion callback (ReadFileEx). You'll probably need to change your main event loop to use MsgWaitForMultipleObjectsEx so you can either wait for the I/O events or allow callbacks to run, in addition to receiving WM_ window messages. MSDN has documentation for these functions.
For Linux, there's either fadvise and epoll, which will use the readahead cache, or aio_read which will allow actual async read requests. You'll get a signal when the request completes, which you should use to post an XWindows message and wake up your event processing loop.
Both are a little different in the details, but the net effect is the same -- you request a read which completes in the background, then your event dispatch loop gets woken up when the I/O finishes.
Boost.Asio library has limited implementation of asynchronous file I/O operations (only Windows wrapper for HANDLE) therefore it not suitable for you. See this question also.
You could easily implement your own asynchronous reading using standard streams and Boost.Thread library (or platform specific threads support).
I'm trying to find the best solution for nonblocking IO via stdin/stdout with the following characteristics:
As long as there is enough data, read in n-sized chunks.
If there's not enough data, read in a partial chunk.
If there is no data available, block until there is some (even though it may be smaller than n).
The goal is to allow efficient transfer for large datasets while processing 'control' codes immediately (instead of having them linger in some partially-filled buffer somewhere).
I know I can achieve this by using threads and a istream::get() loop, or by writing a bunch of platform-specific code (since you can't select() on file handles in windows)... ((There is also istream::readsome() which seems promising, but the only results I can find on google were of people saying it doesn't actually work well.))
Since I haven't done much coding w/ these APIs, perhaps there is a better way.
Maybe boost::asio can be of use for you?
I used the threads and platform specific code. See my answer to another question. I was able to put the OS-specific stuff in inputAvailable() (Linux uses select, Windows just returns true). I could then use WaitForSingleObject() with a timeout on Windows to try to let the thread complete, then TerminateThread() to kill it. Very ugly, but the team didn't want to use this bit of boost.
I did something similar to jwhitlock ... I ended up with a StdinDataIO class that wraps around the appropriate OS-specific implementation(*) so that the rest of my program can select() on the file descriptor StdinDataIO provides, remaining blissfully ignorant of Windows' limitations regarding stdin. Have a look here and here if you like, the code is all open-source/BSD-licensed.
(*) the implementation is a simple pass-through for Linux/MacOSX, and in Windows it's a rather complex process of setting up a child thread to read from stdin and send the data it receives over a socket back to the main thread... not very elegant, but it works.