I've recently bumped into something called IOCP on the windows platform, to be more precise: Input/Output Control Ports. This seems to be the most efficient way to code your server software when it needs to hold thousands of users concurrently.
(Correct me if I'm wrong, but thread-per-socket, polling, and asynchronous callbacks (thread on every callback) are not efficient enough.)
I am wondering if there are any 3rd party libraries available that implement this concept. I know the win32 api supplies us with CreateIoCompletionPort, FreeBSD has kqueue and linux in general can use /dev/epoll that works somewhat similarly... But isn't there a crossplatform library (such as boost::thread) that handles this stuff for me?
Have you looked at boost::asio?
I'm not sure if it has all this functionality yet, but I believe that's one of the things it's intended for.
I know this is a late reply to this thread, but for those interested in a x-platform library supporting async IO on both Linux/UNIX and Windows, you might want to take a look at libuv.
libuv was initially constructed as the foundational underpinnings of node.js, but it has evolved to become a powerful abstraction library for many low-level, x-platform concerns including async IO, non-blocking TCP sockets & named pipes, UDP, timers, child process spawning
, high resolution time, thread pool scheduling, etc.
libevent works on a number of *nix OSes and also runs on Windows.
Related
I have legacy application code written for a real-time operating system (RTOS). Most of the code uses an OS-Specific Inter-Process-Communication (IPC) call, that looks very similar to a signal.
It has two IPC calls:
status_code signal_push(connector, event)
event signal_wait(connector)
I want to slowly port that application to Linux in order to improve testing and debugging.
I want to create a task-description file, that covers tasks their associated IPC primitives with events and handlers.
An example would look like this:
SignalHandler(connector=crypto_connector,
events = [(cyrpto_init, crypto_init_handler),
(crypto_run, crypto_run_handler),
(crypto_done, crypto_done_handler])
Task(name=crypto, priority=1, stack_size=256, connector=crypto_connector)
From this description file it should generate a thread for each task that calls the corresponding handler, that is written by an engineer. it shall also generate call stubs similar to emit_crypto_init()
It is quite clear to me what kind of code I shall generate from this description for the RTOS but I'm still uncertain, what kind of IPC I should use for Linux. For the first version probably anything that keeps behaviour is okay. For future versions, it could be plausible to use Linux on target and therefore the IPC should have little overhead.
I found the following IPC mechanisms:
Unix signals (seems as if it is not an option)
Unix sockets (do a lot more than just sending a signal)
semaphores (do less, but we could use global integers to pass the event to the other party)
Are there more suitable IPC solutions out there, that could fit this requirement?
There are several possibilities:
Boost Signals (not to be confused with Boost Interprocess) - http://www.boost.org/doc/libs/1_51_0/doc/html/signals.html, other is
cpp-events http://code.google.com/p/cpp-events/
libsigc++
Qt Signals and Slots
For some time now I have been googling a lot to get to know about the various ways to acheive asynchronous programming/behavior on nix machines and ( as known earlier to me ) got confirmed on the fact that there is still no TRULY async pattern (concurrency using single thread) for Linux as available for Windows(IOCP).
Below are the few alternatives present for linux:
select/poll/epoll :: Cannot be done using single thread as epoll is still blocking call. Also the monitored file descriptors must be opened in non-blocking mode.
libaio:: What I have come to know about is that its implementation sucks and its still notification based instead of being completion based as with windows I/O completion ports.
Boost ASIO :: It uses epoll under linux and thus not a true async pattern as it spawns thread which are completely abstracted from user code to acheive the proactor design pattern
libevent :: Any reason to go for it if I prefer ASIO?
Now Here comes the questions :)
What would be the best design pattern for writing fast scalable network server using epoll (ofcourse, will have to use threads here :( )
I had read somewhere that "only sockets can be opened in non-blocking mode" hence epoll supports only sockets and hence cannot be used for disk I/O.
How true is the above statement and why async programming cannot be done on disk I/O using epoll ?
Boost ASIO uses one big lock around epoll call. I didnt actually understand what can be its implications and how to overcome it using asio itself. Similar question
How can I modify ASIO pattern to work with disk files? Is there any recommended design pattern ?
Hope somebody will able to answer all the questions with nice explanations also. Any link to source where the implementation details of epoll and AIO design patterns are exaplained is also appreciated.
Boost ASIO :: It uses epoll under linux and thus not a true async
pattern as it spawns thread which are completely abstracted from user
code to acheive the proactor design pattern
This is not correct. The Asio library uses epoll() by default on most recent Linux kernel versions. however, threads invoking io_service::run() will invoke callback handlers as needed. There is only one place in the Asio library that a thread is used to emulate an asynchronous interface, it is well described in the documentation:
An additional thread per io_service is used to emulate asynchronous
host resolution. This thread is created on the first call to either
ip::tcp::resolver::async_resolve() or
ip::udp::resolver::async_resolve().
This does not make the library "not a true async pattern" as you claim, in fact its name would disagree with you by definition.
1) What would be the best design pattern for writing fast scalable network server using epoll (of course, will have to use threads here :(
)
I suggest using Boost Asio, it uses the proactor design pattern.
3) Boost ASIO uses one big lock around epoll call. I didnt actually
understand what can be its implications and how to overcome it using
asio itself
The epoll reactor uses a mutex to dispatch handlers, though in practice this is not a big concern for most applications. There are application specific ways to mitigate this behavior, such as an io_service per CPU to exploit data locality. See my answer to a similar question on this topic. It is also discussed on the Asio mailing list frequently.
4) How can I modify ASIO pattern to work with disk files? Is there any
recommended design pattern?
The Asio library does not natively support file I/O as you noted. There have been several attempts to add it to the library, I'd suggest discussing on the mailing list.
First of all:
got confirmed on the fact that there is still no TRULY async pattern (concurrency using single thread) for Linux as available for Windows(IOCP).
You probably has a small misconception, asynchronous can be build on top of "polling" api.
More then that "reactor" (epoll-like) API is more powerful then "proactor" API (IOCP) as
the second can be implemented in terms of the first one (but not the other way around).
Also some operations that are "truly" asynchronous for example like disk I/O, some some other tools can be with combination of signals and Linux specific signalfd can provide full coverage of some other cases.
Bottom line. epoll is truly asynchronous I/O
I'm programming a network protocol over UDP, using C/C++ in Linux. The protocol must provide reliability, so I'm going to simulate something like TCP retransmission over UDP.
This can be done using pthreads or fork, but I believe that's an overkill and consumes a lot of system resources. A better approach is to exploit a scheduler.
I probably can't use Linux internal scheduler, since I'm programming in user space. Are there standard C/C++ libraries to accomplish this? How about 3rd party libraries?
Edit: Some people asked why I'm doing this. Why not use the TCP instead?
The answer is, since I'm implementing a tunneling protocol. If someone tunnels TCP over TCP, the efficiency will drop considerably. Here's more info Why TCP Over TCP Is A Bad Idea.
The "scheduler" you're after is called "select", and it's a user-space call available in linux. Type "man 2 select" to read the help page for how to use it.
If you need a timeout, just call select() with a timeout value. The select call will return either when new data has arrived, or a timeout has expired. You can then do retransmissions if there was a timeout.
Here's a sample of how to accomplish asynchronous coroutines with Boost. Boost manages the overhead of creating a thread to run the coroutine in this case so that you don't need to. If you would like the kernel to manage your interrupts, you can use alarm & setitimer, but they're very limited in what they can do.
Any solution will include threads, forks, or some variant of them at some level, unless you synchronously manage the transmission in the main thread using something like select().
Not clear what exactly you are trying to schedule. You can use libevent for efficient and somewhat portable interface. This is basically similar to Matthew's suggestion of using select, but using the most efficient interface (which select is not) on FreeBSD, Linux and MacOS X (actually their page now claims Windows support as well but I'm not too familiar with that).
This will give ability to do non-blocking event-driven network calls. It will not solve the scheduling part. DOing it in a separate thread is not going to hurt your performance. I think running a pthread per connection is not the best approach, but having a single scheduling thread and some worker threads dealing with the network events and maybe some non-trivial processing usually works well.
I'm about to develop some sockets related stuff in C++ and would like the software to be as portable between Windows and Linux as possible right from the start (making it portable later is tricky.)
I've looked at different libraries, there is one for C++ from alhem.net and of course there is boost::asio. boost::asio looks very promising but would be a very big dependency for applications this small.
Is it even worth writing the stuff myself or should I just use a library? If I do it myself what would be the main pitfalls?
I've developed a few portable wrappers around sockets. Make sure you don't go down the crappy lane of no return that is constituted of WinSock2 events. Other than that, as I see it, the biggest differences are:
to start networking in Windows, you need to call ::WSAStartup(), to shut it down in Windows, run ::WSACleanup(); in Linux do nothing,
close() in Linux is closesocket() in Windows,
default buffer sizes differ between both drivers and operating systems, so make sure to set them using SO_RCVBUF and SO_SNDBUF,
SO_REUSEADDR steals the address on Windows, allows frequent re-opening on Linux; you'd probably only want to use this flag in Linux,
making a socket non-blocking uses ::ioctlsocket() in Windows, ::fcntl() in Linux,
the header files are different, <sys/socket.h> and friends in Linux, <WinSock.h> in Windows,
to go portable, the easiest way is probably to use ::select() to wait for data to arrive,
fd_sets are totally different on Windows/Linux; this is only relevant if you need to optimize initialization of fd_sets, such as when adding/removing arbitrary sockets,
in Windows, any thread hanging on the socket is released with an error code when the socket is closed, in Linux the thread remains waiting. If the thread is blocking the socket with for instance ::recvfrom(), you might consider using ::sendto() to release the stalling thread under Linux.
Everything else I ever needed just worked out of the låda.
Winsocks aren't very compatible with Posix sockets:
In Winsocks a socket is of type SOCKET. On Posix it's simply a file descriptor (int), on which you can perform normal read() and write() calls.
They don't return errors the same way.
They don't support some options on recv() and send().
You have to initialize and unitialize the Winsocks library with two specials functions.
I don't think you can close Windows sockets with shutdown() or close(). It's something like closesocket() instead.
There must be more differences, but that's what I can remember right now. If you want portability with Winsocks, you'll have a small library for closing a socket, printing an error message and so on.
I'd probably go with boost::asio, personnally (I've never used it, though).
Take a look at the "Adaptive Communications Environment" (ACE) library:
(ACE Home Page)
It provides some nice abstractions and a lot of flexibility all rolled up in a portable library that supports Windows, MacOS and Linux.
It has a bit of a steep learning curve, but I got very good value from it.
How much socket stuff will you be using? I've done several apps where the socket stuff was pretty high level (open, read, write), and worked perfectly from Windows to Linux. If it's more than that - go with boost.
Honestly, I'd use boost::asio as a first preference. If you really want to get down and dirty with the sockets API, you can use the standard BSD-style sockets API on both Windows and Linux - it's just that on Windows you'll have to link to (and initialize) Winsock2, whereas on Linux you won't have a separate library to link against.
Have a look at this... http://sourceforge.net/projects/cpp-sockets/
I'm working on an instant messenger client in C++ (Win32) and I'm experimenting with different asynchronous socket models. So far I've been using WSAAsyncSelect for receiving notifications via my main window. However, I've been experiencing some unexpected results with Winsock spawning additionally 5-6 threads (in addition to the initial thread created when calling WSAAsyncSelect) for one single socket.
I have plans to revamp the client to support additional protocols via DLL:s, and I'm afraid that my current solution won't be suitable based on my experiences with WSAAsyncSelect in addition to me being negative towards mixing network with UI code (in the message loop).
I'm looking for advice on what a suitable asynchronous socket model could be for a multi-protocol IM client which needs to be able to handle roughly 10-20+ connections (depending on amount of protocols and protocol design etc.), while not using an excessive amount of threads -- I am very interested in performance and keeping the resource usage down.
I've been looking on IO Completion Ports, but from what I've gathered, it seems overkill. I'd very much appreciate some input on what a suitable socket solution could be!
Thanks in advance! :-)
There are four basic ways to handle multiple concurrent sockets.
Multiplexing, that is using select() to poll the sockets.
AsyncSelect which is basically what you're doing with WSAAsyncSelect.
Worker Threads, creating a single thread for each connection.
IO Completion Ports, or IOCP. dp mentions them above, but basically they are an OS specific way to handle asynchronous I/O, which has very good performance, but it is a little more confusing.
Which you choose often depends on where you plan to go. If you plan to port the application to other platforms, you may want to choose #1 or #3, since select is not terribly different from other models used on other OS's, and most other OS's also have the concept of threads (though they may operate differently). IOCP is typically windows specific (although Linux now has some async I/O functions as well).
If your app is Windows only, then you basically want to choose the best model for what you're doing. This would likely be either #3 or #4. #4 is the most efficient, as it calls back into your application (similar, but with better peformance and fewer issues to WSAsyncSelect).
The big thing you have to deal with when using threads (either IOCP or WorkerThreads) is marshaling the data back to a thread that can update the UI, since you can't call UI functions on worker threads. Ultimately, this will involve some messaging back and forth in most cases.
If you were developing this in Managed code, i'd tell you to look at Jeffrey Richter's AysncEnumerator, but you've chose C++ which has it's pros and cons. Lots of people have written various network libraries for C++, maybe you should spend some time researching some of them.
consider to use the ASIO library you can find in boost (www.boost.org).
Just use synchronous models. Modern operating systems handle multiple threads quite well. Async IO is really needed in rare situations, mostly on servers.
In some ways IO Completion Ports (IOCP) are overkill but to be honest I find the model for asynchronous sockets easier to use than the alternatives (select, non-blocking sockets, Overlapped IO, etc.).
The IOCP API could be clearer but once you get past it it's actually easier to use I think. Back when, the biggest obstacle was platform support (it needed an NT based OS -- i.e., Windows 9x did not support IOCP). With that restriction long gone, I'd consider it.
If you do decide to use IOCP (which, IMHO, is the best option if you're writing for Windows) then I've got some free code available which takes away a lot of the work that you need to do.
Latest version of the code and links to the original articles are available from here.
And my views on how my framework compares to Boost::ASIO can be found here: http://www.lenholgate.com/blog/2008/09/how-does-the-socket-server-framework-compare-to-boostasio.html.