I'm designing event loop for asynchronous socket IO using epoll/devpoll/kqueue/poll/select (including windows-select).
I have two options of performing, IO operation:
Non-blocking mode, poll on EAGAIN
Set socket to non-blocking mode.
Read/Write to socket.
If operation succeeds, post completion notification to event loop.
If I get EAGAIN, add socket to "select list" and poll socket.
Polling mode: poll and then execute
Add socket to select list and poll it.
Wait for notification that it is readable writable
read/write
Post completion notification to event loop of sucseeds
To me it looks like first would require less system calls when using in normal mode,
especially for writing to socket (buffers are quite big).
Also it looks like that it would be possible to reduce the overhead over number of "select"
executions, especially it is nice when you do not have something that scales well
as epoll/devpoll/kqueue.
Questions:
Are there any advantages of the second approach?
Are there any portability issues with non-blocking operations on sockets/file descriptors over numerous operating systems: Linux, FreeBSD, Solaris, MacOSX, Windows.
Notes: Please do not suggest using existing event-loop/socket-api implementations
I'm not sure there's any cross-platform problem; at the most you would have to use Windows Sockets API, but with the same results.
Otherwise, you seem to be polling in either case (avoiding blocking waits), so both approaches are fine. As long as you don't put yourself in a position to block (ex. read when there's no data, write when buffer's full), it makes no difference at all.
Maybe the first approach is easier to code/understand; so, go with that.
It might be of interest to you to check out the documentation of libev and the c10k problem for interesting ideas/approaches on this topic.
The first design is the Proactor Pattern, the second is the Reactor Pattern
One advantage of the reactor pattern is that you can design your API such that you don't have to allocate read buffers until the data is actually there to be read. This reduces memory usage while you're waiting for I/O.
from my experience with low latency socket apps:
for writes - try to write directly into the socket from writing thread (you need to obtain event loop mutex for that), if write is incomplete subscribe to write readiness with event loop (select/waitformultipleobjects) and write from event loop thread when socket gets writable
for reads - be always "subscribed" for read readiness for all sockets, so you always read from within event loop thread when the socket gets readable
Related
I have a C++ console app that uses open() [O_RDWR | O_NONBLOCK], write(), select(), read() and close() to work with device file. Also ioctl() can be called to cancel current operation. At any given time only one user can work with device.
I need to come up with C++ class having libsigc++ signals that get fired when data is available from device.
The problem: when calling select() application becomes unresponsive as it waits for the data. How to make it responsive - by calling select() in worker thread? If so - how will worker thread communicate with main thread? Maybe I should look into boost::asio?
How to make it responsive - by calling select() in worker thread
you can use dup(), this will duplicated your file descriptors... thus you can move entire read operations into another thread. thus your write thread and processing thread will be responsive, even when the read [select()] thread is in sleeping.
signal emitting overhead of libsigc++ is minimal, thus i think you can embedded code inside the read thread itself. slots can exist in different thread, this is where you will receive your signals...
I think Thrift source code [entirely boost based] might be of your interest, though thrift does not use libsigc++.
It sounds as though you've misunderstood select; the purpose of select (or poll, epoll, etc) is not "wait for data" but "wait for one or more events to occur on a series of file descriptors or a timer, or a signal to be raised".
What "responsiveness" is going missing while you're in your select call? You said it's a console app so you're not talking about a GUI loop, so presumably it is IO related? If so, then you need to refactor your select so that waiting for the data you're talking about is one element; that is, if you're using select, build FD_SETs of ALL file/socket descriptors (and stdin and stdout are file descriptors) that you want to wait on input for.
Or build a loop that periodically calls "select" with a short timeout to /test/ for any pending input and only try and read it when select tells you there is something to read.
It sounds like you have a producer-consumer style problem. There are various way to implement a solution to this problem, but most folks these days tend to use condition variable based approaches (see this C++11 based example).
There are also a number of design patterns that when implemented can help alleviate your concurrency problem, such as:
Half-Sync / Half-Async
A producer-consumer style pattern that introduces a queue between an asynchronous layer that fills the queue with events, and a synchronous layer that processes those events.
Leader / Followers
Multiple threads take turns handling events
A related discussion is available here.
For every single tutorials and examples I have seen on the internet for Linux/Unix socket tutorials, the server side code always involves an infinite loop that checks for client connection every single time.
Example:
http://www.thegeekstuff.com/2011/12/c-socket-programming/
http://tldp.org/LDP/LG/issue74/tougher.html#3.2
Is there a more efficient way to structure the server side code so that it does not involve an infinite loop, or code the infinite loop in a way that it will take up less system resource?
the infinite loop in those examples is already efficient. the call to accept() is a blocking call: the function does not return until there is a client connecting to the server. code execution for the thread which called the accept() function is halted, and does not take any processing power.
think of accept() as a call to join() or like a wait on a mutex/lock/semaphore.
of course, there are many other ways to handle incoming connection, but those other ways deal with the blocking nature of accept(). this function is difficult to cancel, so there exists non-blocking alternatives which will allow the server to perform other actions while waiting for an incoming connection. one such alternative is using select(). other alternatives are less portable as they involve low-level operating system calls to signal the connection through a callback function, an event or any other asynchronous mechanism handled by the operating system...
For C++ you could look into boost.asio. You could also look into e.g. asynchronous I/O functions. There is also SIGIO.
Of course, even when using these asynchronous methods, your main program still needs to sit in a loop, or the program will exit.
The infinite loop is there to maintain the server's running state, so when a client connection is accepted, the server won't quit immediately afterwards, instead it'll go back to listening for another client connection.
The listen() call is a blocking one - that is to say, it waits until it receives data. It does this is an extremely efficient way, using zero system resources (until a connection is made, of course) by making use of the operating systems network drivers that trigger an event (or hardware interrupt) that wakes the listening thread up.
Here's a good overview of what techniques are available - The C10K problem.
When you are implementing a server that listens for possibly infinite connections, there is imo no way around some sort of infinite loops. Usually this is not a problem at all, because when your socket is not marked as non-blocking, the call to accept() will block until a new connection arrives. Due to this blocking, no system resources are wasted.
Other libraries that provide like an event-based system are ultimately implemented in the way described above.
In addition to what has already been posted, it's fairly easy to see what is going on with a debugger. You will be able to single-step through until you execute the accept() line, upon which the 'sigle-step' highlight will disappear and the app will run on - the next line is not reached. If you put a breadkpoint on the next line, it will not fire until a client connects.
We need to follow the best practice on writing client -server programing. The best guide I can recommend you at this time is The C10K Problem . There are specific stuff we need to follow in this case. We can go for using select or poll or epoll. Each have there own advantages and disadvantages.
If you are running you code using latest kernel version, then I would recommend to go for epoll. Click to see sample program to understand epoll.
If you are using select, poll, epoll then you will be blocked until you get an event / trigger so that your server will not run in to infinite loop by consuming your system time.
On my personal experience, I feel epoll is the best way to go further as I observed the threshold of my server machine on having 80k ACTIVE connection was very less on comparing it will select and poll. The load average of my server machine was just 3.2 on having 80k active connection :)
On testing with poll, I find my server load average went up to 7.8 on reaching 30k active client connection :(.
I have been struggling to try and find my answer for this on google, as I dont know the exact terms I am looking to search for.
If someone were to build an msn messenger-like program, is it possible to have always-open connections and no while(true) loop? If so, could someone point me in the direction of how this is achieved?
Using boost::asio library for socket handling, i think it is possible to define callbacks upon data reception.
The one single magic word your looking for is asynchronous I/O. This can be achieved either through using asynchronous APIs (functions such as ReadThis() that return immediately and signal on success/failure -- like but not limited by boost::asio) or by deferring blocking calls to different threads. Picking either method requires careful weighing of both the underlying implementation and the scale of your operations.
You want to use ACE. It has a Reactor pattern which will notify you when data is available to be use.
Reactor Pattern
You could have:
while(1) {
sleep(100); // 100 ms
// check if there is a message
// process message
//...
}
This is ok, but there is an overhead on servers running 10000s of threads since threads come out of sleep and check for a message, causing context-switching. Instead, operating systems provide functions like select and epoll on Linux, which allow a thread to wait on an event.
while(1) {
// wait for message
// process message
//...
}
Using wait, the thread is not "woken up" unless a message is received.
You can only hide your while loop (or some kind of loop) somewhere buried in some library or restart the waiting for next IO in an event callback, but you aren't going to be able to completely avoid it.
That's a great question. Like nj said, you want to use asynchronous I/O. Too many programs use a polling strategy. It is not uncommon to have 1000 threads running on a system. If all of them were polling, you would have a slow system. Use asynchronous I/O whenever possible.
what about udp protocol communication ? you dont have to wait in while loop for every clients
just open one connection on specified port and call receive method
The title really says it all.
The and ... means also include pselect and ppoll..
The server project I'm working on basically structured with multiple threads. Each
thread handles one or more sessions. All the threads are identical. The protocol
takes care of which thread will host the session.
I'm using an inhouse socket class that wraps things up. The point of interest is a checkread call which calls either poll (linux) or select (windows).
In summary each thread currently calls poll on a single socket. From what I can tell, using epoll would only be of benefit if this thread was looking at multiple sockets such as what you'd get in say an HTTP server. That's not what I'm doing in my case. And the class only handles a single socket at a time.
There is some brief discussion about edge and level triggering in the man pages for epoll. I'm not really sure what it means. In the socket class I see an optimization in the windows part of the code that shortcuts the select call with an ioctlsocket & FIONREAD to check if there is any data. Wondering if that would return > 0 even if a complete UDP packet hadn't arrived at the time of the call. Is this what edge triggering is in epoll?
In some rudimentary testing, I'm also seeing no noticeable difference between using select and poll.
I can see that using ppoll might be of benefit though due to greater precision in the timeout. Any thoughts?
And yes, I am trying to optimize throughput for a session that is receiving lots of data. The server is more Network & Disk bound than CPU.
The main difference between epoll vs select or poll is that epoll scales a lot better when run in a single thread. I don't know how this would compare to using a multithreaded server using select or poll.
Look at this http://monkey.org/~provos/libevent/libevent-benchmark2.jpg
The reason for this(as far as I can tell) is that when you are using select or poll you must loop through all the connected sockets to determine which ones have data to be read. When you are using epoll, it keeps a seperate array which contains references only to sockets which have data to be read. This saves you lots of loop cycles, and the difference becomes more and more noticeable the more sockets that are connected.
Another thing to look into if performance ever becomes a major issue is io completion ports(windows only) and kqueue(FreeBSD only). It's also important to remember that epoll is linux only. In most cases select or poll will work just fine.
In the case of a single file descriptor, select and poll are more efficient than epoll due to being much simpler. (epoll has some overhead which doesn't make itself useful with only a single socket)
According to the link: http://www.intelliproject.net/articles/showArticle/index/io_multiplexing.
If you use only one descriptor:
select: 201 micro seconds.
poll: 159 micro seconds.
epoll: 176 micro seconds.
Seems poll will be a better solution in such situation.
If you have only a single socket, what's the point of polling in the first place? Wouldn't the best performance then be by just using blocking read/write?
Wrt. the performance, with only a single file descriptor I don't think there is much, if any, difference between the various approaches. If you really care, I suppose you could measure, but I find it difficult that this would particularly matter for the overall performance of your program.
Level/edge triggering. Consider you're monitoring a signal, for simplicity say some voltage in a line. Edge triggering means that something triggers when the voltage goes over or under some specific limit. Level triggering means that something is considered to be in a triggered state as long as the voltage is over/under the limit. That is, edge triggering triggers when some event happens (crossing some threshold), level triggering reflects the state of some "thing" (in this case, voltage).
To get back to network programming, and edge triggered system might be one where you get some kind of signal when a packet is received. If you don't handle the event then the signal is lost. A level triggered system, OTOH, is something like asking "is there data waiting in the buffer for me?"; if you don't handle the event and ask again, the data will still be there waiting for you.
I'm writing a daemon that needs to both run in the background and take care of tasks and also receive input directly from a frontend. I've been attempting to use sockets to take care of this task, however, I can't get it to work properly since sockets pause the program while waiting for a connection. Is there anyway to get around this?
I'm using the socket wrappers provided at http://linuxgazette.net/issue74/tougher.html
Thank you for any and all help
You will need to use threads to make the socket operations asynchronous. Or use some library that has already implemented it, one of the top ones is Boost Asio.
There are a few ways to handle this problem. This most common is using an event loop and something like libevent. Then you use non-blocking sockets.
Doing this in an event driven fashion can require a big shift in your program logic. But doing it with threads has its own complexities and isn't clearly a better choice.
Usually the daemons use event loops to avoid the problem of waiting for events.
It's the smartest solution to the problem that you present (do not wait to an asynchronous event). รง
Althought, usually the entire daemon is build over the event loop and it's callback architecture, and can cause a partial rewritting, so usually the quick and dirty solution is creating a separate thread to handle those events wich usually creates more bugs than it solves. So, use an event loop:
libevent.
glib event loop.
libev.
boost::asio
...
From your description, you have already divided your application into a frontend (receiving input) and backend (socket handling and tasks). If the input from the frontend is sent over the socket (via the backend) rather receiving input from the socket then it seems like you are describing a client and not a server. Client programs are typically not implemented as daemons.
You have created a blocking socket and need to either monitor in a separate thread execution a thread or even separate process) or make a non-blocking socket and poll frequently for updates.
The link to the LinuxGazette is a basic intro to network programming. If you would like a little more depth then take a look at Beej's Guide to Network Programming where the various API calls available to you are explained in a little detail.. and will, perhaps, make you appreciate more wrapper libraries such as Boost::ASIO.
Can be worth retaining control of the event loop yourself - its no complicated and provides flexibility down the track.
"C++ pseudo-code" for an event loop.
while (!done)
{
bool workDone = false;
// Loop over each event source or internal worker
for each module
{
// If it has work to do, do some.
if (module.hasWorkDoTo())
{
// Generally, do as little work as possible; e.g. process a single event for this module.
// But tinker with this to manage priorities if need be.
// E.g. Maybe allow the GUI to flush its queue.
module.doSomeWork();
workDone = true;
}
}
if (!workDone)
{
// System idle. No Sleep for a bit so we have benign idle baheviour.
nanosleep(...);
}
}