I'm making a messaging service that needs to use both socket io and shared memory. The routine will be the same regardless of where the input comes from, with the only difference being local messages will be passed via shared memory and non-local messages over a socket. Both events will have to unblock the same pselect call.
At this point I think the best option might be to send a signal whenever a message is written to shared memory and use it to interrupt a pselect call but I'm not quite sure how this would be done or even if it's the best route.
I'm not used to using signals. what's the best way to accomplish this?
I would consider using a pipe (see pipe(2)) or an AF_UNIX local unix(7) socket(2) (as commented by caf) at least to transmit control information -for synchronization- about the shared memory (i.e. tell when it has changed, that is when a message has been sent thru shared memory, etc.). Then you can still multiplex with e.g. poll(2) (or ppoll(2) or pselect(2) etc...)
I don't think that synchronization using signals is the right approach: signals are difficult to get right (so coding is tricky) and they are not more efficient than exchanging a few bytes on some pipe.
Did you consider to use MPI?
If you only want to signal between processes rather than pass data, then an eventfd (see eventfd(2)) will allow you to use select() with less overhead than a pipe. As with a pipe solution, the processes will require a parent/child relationship.
If you want to use signals, use sigqueue to send them - you can send an integer payload with this, for example an offset into your shared memory.
Make sure to register your signal handler with sigaction and use the sa_sigaction callback: the siginfo_t->si_int member will contain that payload.
In general, I'm not sure I can recommend using this mechanism instead of a unix pipe or eventfd, because I'm not sure whether signal delivery is really tuned for speed as you hope: benchmark to be sure.
PS. performance aside, one reason signals feel a bit icky is that you lose the opportunity to have a "well-known" rendezvous like an inet or unix port, and instead have to go looking for a PID. Also, you have to be very careful about masking to make sure the signal is delivered where you want.
PPS. You raise or send a signal - you don't throw it. That's for exceptions.
I did some additional looking and came across signalfd(2). I believe this will be the best solution - very similar to Basile Starynkevitch's suggestion but without the overhead of standard pipes and done within the kernel rather than userspace.
pipe+select+queue+lock, nothing else.
Related
I've got a ROUTER/DEALER setup where both ends need to be able to receive and send data asynchronously, as soon as it's available. The model is pretty much 0MQ's async C++ server: http://zguide.zeromq.org/cpp:asyncsrv
Both the client and the server workers poll, when there's data available they call a callback. While this happens, from another thread (!) I'm putting data in a std::deque. In each poll-forever thread, I check the deque (under lock), and if there are items there, I send them out to the specified DEALER id (the id is placed in the queue).
But I can't help thinking that this is not idiomatic 0MQ. The mutex is possibly a design problem. Plus, memory consumption can probably get quite high if enough time passes between polls (and data accumulates in the deque).
The only alternative I can think of is having another DEALER thread connect to an inproc each time I want to send out data, and just have it send it and exit. However, this implies a connect per item of data sent + construction and destruction of a socket, and it's probably not ideal.
Is there an idiomatic 0MQ way to do this, and if so, what is it?
I dont fully understand your design but I do understand your concern about using locks.
In most cases you can redesign your code to remove the use of locks using zeromq PAIR sockets and inproc.
Do you really need a std::deque? If not you could just use a zerom queue as its just a queue that you can read/write from from different threads using sockets.
If you really need the deque then encapsulate it into its own thread (a class would be nice) and make its API (push etc) accessible via inproc sockets.
So like I said before I may be on the wrong track but in 99% of cases I have come across you can always remove the locks completely with some ZMQ_PAIR/inproc if you need signalling.
0mq queue has limited buffer size and it can be controlled. So memory issue will get to some point and then dropping data will occur. For that reason you may consider using conflate option leaving only most recent data in queue.
In a case of single server and communication within single machine with many threads I suggest using publish/subscribe model where with conflate option you will receive new data as soon as you read buffer and won't have to worry about memory. And it removes blocking queue problem.
As for your implementation you are quite right, it is not best design but it is quite unavoidable. I suggest checking question Access std::deque from 3 threads while it answers your problem, it may not be the best approach.
We have an app that is entirely synchronous, and will always be because it is basically a command line interpreter to send low level commands to our hardware, and you cant have two commands going to the hardware at the same time. I will only ever have 1 client socket for this configuration operating in a synchronous manner, one command to the server, it talks to hardware, and sends value back to client, but as far as i see it currently async_read is the only way to do non blocking reads.
What is the best way to get a non blocking read/write via Beast? For example in TCP and Serial in Windows you have ways to peek into the buffer to see if data is ready to be accessed, and if there is you can issue your read command knowing it wont block because data is there. Not sure if I am just missing this functionality in Beast, although i will say having such functionality if possible would be nice.
Anyways so based on this i have a question
First, can I take the Coroutine example and instead of using yield, to create and pass it a read_handler function?
I've taken the coroutine example, and built the functions into my class, and used the exact same read_handler from this thread answer.
How to pass read handler to async_read for Beast websocket?
It compiles as he says, but setting a break point never triggers when data is received.
I dont really need the full async functionality like the async example, pushing it into different threads, in fact that makes my life more difficult because the rest of the app is not async. And because we allow input from various sources(keyboard/TCP/Serial/File), we cant block waiting for data.
What is the best way to get a non blocking read/write via Beast?
Because of the way the websocket stream is implemented, it is not possible to support non-blocking socket modes.
can I take the Coroutine example and instead of using yield, to create and pass it a read_handler function?
If you want to use completion handlers, I would suggest that instead of starting with the coroutine example you start with one of the asynchronous examples, since these are already written to use completion handlers.
Coroutines have blocking semantics, while completion handlers do not. If you try to use the coroutine example and replace the yield expression with a completion handler, the call to the initiating function will not block the way it does when using coroutines. And you should not use spawn. You said that the coroutine example is much easier, probably this is because it resembles synchronous code. If you want that ease of writing and understanding, then you have to use coroutines. Code using completion handlers will exhibit the "inversion of control" typically associated with callbacks. This is inherent to how they work and not something you can change by just starting with code that uses coroutines and changing the completion token.
I'm working on a project, where a primary server thread needs to dispatch events to a series of worker threads. The work that goes on in the worker threads relies on polling (ie. epoll or kqueue depending on the UNIX system in question) with timeouts on these operations needing to be handles. This means, that a normal conditional variable or semaphore structure is not viable for this dispatch, as it would make one or the other block resulting in an unwanted latency between either handling the events coming from polling or the events originating from the server thread.
So, I'm wondering what the most optimal construct for dispatching such events between threads in a pollable fashion is? Essentially, all that needs to be delivered is a pollable "signal" that tells the worker thread, that it has more events to fetch. I've looked at using UNIX pipes (unnamed ones, as it's internal to the process) which seems like a decent solution given that a single byte can be written to the pipe and read back out when the queue is cleared -- but, I'm wondering if this is the best approach available? Or the fastest?
Alternatively, there is the possibility to use signalfd(2) on Linux, but as this is not available on BSD systems, I'd rather like to avoid this construct. I'm also wondering how great the overhead in using system signals actually is?
Jan Hudec's answer is correct, although I wouldn't recommend using signals for a few reasons:
Older versions of glibc emulated pselect and ppoll in a non-atomic fashion, making them basically worthless. Even when you used the mask correctly, signals could get "lost" between the pthread_sigprocmask and select calls, meaning they don't cause EINTR.
I'm not sure signalfd is any more efficient than the pipe. (Haven't tested it, but I don't have any particular reason to believe it is.)
signals are generally a pain to get right. I've spent a lot of effort on them (see my sigsafe library) and I'd recommend avoiding them if you can.
Since you're trying to have asynchronous handling portable to several systems, I'd recommend looking at libevent. It will abstract epoll or kqueue for you, and it will even wake up workers on your behalf when you add a new event. See event.c
2058 static inline int
2059 event_add_internal(struct event *ev, const struct timeval *tv,
2060 int tv_is_absolute)
2061 {
...
2189 /* if we are not in the right thread, we need to wake up the loop */
2190 if (res != -1 && notify && EVBASE_NEED_NOTIFY(base))
2191 evthread_notify_base(base);
...
2196 }
Also,
The worker thread deals with both socket I/O and asynchronous disk I/O, which means that it is optimally always waiting for the event queuing mechanism (epoll/kqueue).
You're likely to be disappointed here. These event queueing mechanisms don't really support asynchronous disk I/O. See this recent thread for more details.
As far as performance goes, the cost of system call is comparably huge to other operations, so it's the number of system calls that matters. There are two options:
Use the pipes as you wrote. If you have any useful payload for the message, you get one system call to send, one system call to wait and one system call to receive. Try to pass any relevant data down the pipe instead of reading them from a shared structure to avoid additional overhead from locking.
The select and poll have variants, that also waits for signals (pselect, ppoll). Linux epoll can do the same using signalfd, so it remains a question whether kqueue can wait for signals, which I don't know. If it can, than you could use them (you are using different mechanism on Linux and *BSD anyway). It would save you the syscall for reading if you don't have good use for the passed data.
I would expect passing the data over socket to be more efficient if it allows you do do away with any other locking.
REMOVED - reason: not really needed.
my questions are:
can I use a linux UDP socket from two different threads? answer was here
I have two different events I would like to wait for using just one thread. One of such events is the addition of an element to a stack and another is the availability of data on a socket.
I can use a boost::condition_variable.wait(lock) for the stack and boost::asio::io_service for the socket. But there is no mechanism (that I am aware of) that allows me to wait for both events at the same time (polling is out of the question). Or is it?
Is there any other alternative solution for this problem that I'm not aware of? - I'll figure this one out by myself.
New Answer
But there is no mechanism (that I am
aware of) that allows me to wait for
both events at the same time (polling
is out of the question). Or is it?
Not that I'm aware of, and not without polling... you'll need a thread to wait for each asynchronous event. You can use a blocking stack or like you said use boost::condition_variable which blocks until there is something on the stack. The boost::asio::io_service will be very useful for managing the udp sockets, but it doesn't actually give you any advantage when it comes to the event handling.
Old Answer
I'm REALLY not sure what you're trying to do... what you're saying doesn't make much sense. I'll do my best to guess what you're trying to do, but I would suggest clarifying the question.
Question:
Do I really need to use the main
thread to send the data over component
A socket or can I do it from the
new-thread? (I think the answer is no,
but I'm not sure about race conditions
on sockets)
Answer:
You don't have to use the main thread to send data over the given component's socket. Now depending on the socket library you're using there might be different restrictions: you may only be able to send data on the same thread that the socket was created, or you might be able to send data from any thread... it really depends on the implementation of your socket.
Question:
how to I wait for both events?
Answer:
You can't do two things at the same time in the same thread... with that said you have two options:
Constantly poll to see if either event has occurred (on the same thread).
Have two threads that are blocking until a desired event occurs (usually when you read from a socket it blocks if there is no data).
Given the description of your problem it's unclear what you would achieve by using boost::condition_variable and/or boost::asio::io_service. Perhaps you should give us a very simple example of code that we can follow.
Question:
Is there any other alternative
solution for this problem that I'm not
aware of?
Answer:
There are always alternative solutions out there, but it's really difficult to tell what the alternatives might be given the current description of the "problem." I think that you should edit the problem again and focus on providing very concrete examples, perhaps some pseudo code, etc.
Switch to Windows and use WaitForMultipleObjects, or get this function implemented in Linux. It's quite handy, and then you can do two things on the same thread.
I'm trying to find the best solution for nonblocking IO via stdin/stdout with the following characteristics:
As long as there is enough data, read in n-sized chunks.
If there's not enough data, read in a partial chunk.
If there is no data available, block until there is some (even though it may be smaller than n).
The goal is to allow efficient transfer for large datasets while processing 'control' codes immediately (instead of having them linger in some partially-filled buffer somewhere).
I know I can achieve this by using threads and a istream::get() loop, or by writing a bunch of platform-specific code (since you can't select() on file handles in windows)... ((There is also istream::readsome() which seems promising, but the only results I can find on google were of people saying it doesn't actually work well.))
Since I haven't done much coding w/ these APIs, perhaps there is a better way.
Maybe boost::asio can be of use for you?
I used the threads and platform specific code. See my answer to another question. I was able to put the OS-specific stuff in inputAvailable() (Linux uses select, Windows just returns true). I could then use WaitForSingleObject() with a timeout on Windows to try to let the thread complete, then TerminateThread() to kill it. Very ugly, but the team didn't want to use this bit of boost.
I did something similar to jwhitlock ... I ended up with a StdinDataIO class that wraps around the appropriate OS-specific implementation(*) so that the rest of my program can select() on the file descriptor StdinDataIO provides, remaining blissfully ignorant of Windows' limitations regarding stdin. Have a look here and here if you like, the code is all open-source/BSD-licensed.
(*) the implementation is a simple pass-through for Linux/MacOSX, and in Windows it's a rather complex process of setting up a child thread to read from stdin and send the data it receives over a socket back to the main thread... not very elegant, but it works.