I am trying to port my multi threaded windows application to Unix. In my application, we have user created events which signals the thread to perform specific task.I found that the conditional variable or semaphore can be used to signal threads.My requirement is to create dynamic events on request but that is not feasible with conditional variable or semaphore.Help me to use the events like signalling concepts in Unix.
Posix threads allow you to communicate between threads with mutex and conditions. man pthreads, man pthread_cond_wait and other.
Another way to make events doesn't depend on threads and made on descriptors.
See man epoll, man poll, man select
Implementing event system manually in C may take some time, so there are libraries, that are implementing event system like libev, libuv, libevent, libevent2.
If you want more C++ then C you can use boost (like boost::signals. Don't know if there are other) or Qt, which has full signal/slot mechanism and event system built in. But it requires quite heavy dependencies.
If you want full implemented event queue and many interprise features you can look at any AMQP framework like rabbitmq.
It's hard to tell you what you need without specific problems.
Related
I have a Cap'n Proto RPC server that runs some OpenGL commands in a window. I am not interested in the window's events at all, but in order to avoid getting killed on Windows I need to poll events once a second or so. How can I do this in a simple fashion?
I have read that you can make your own EventPort, but I couldn't figure out how to actually use EventPorts. It might also be overkill when I'm not actually interested in the events. I would like prioritize RPC events over polling the window if possible.
Using something else than EZ-rpc is not a downside, as I want to move to shared memory communication later on.
So, there's this critical flaw in Windows event handling: The best way to handle network I/O, especially with many connections, is via I/O Completion Ports (IOCP). However, unfortunately, Windows provides no way for a thread to wait on IOCP events and GUI events in the same thread. This seems to be a serious design flaw in the Win32 API, yet it's been this way for decades. Weirder still, the internal NT kernel APIs do in fact support an alternative (specifically, they allow I/O completion events to be delivered via APC) but Microsoft hasn't made these APIs public, so applications that use them could break in a future version of Windows.
As a result, there are essentially two ways to design a program that simultaneously does network I/O and implements a GUI:
Use a MsgWaitForMultipleObjectsEx-based event loop instead of IOCP. You will be limited to no more than 64 connections, and the event loop will be relatively inefficient.
Have separate threads for network and GUI.
For your use case, it sounds like #1 would probably be fine, but there's another problem: The KJ event loop library (used by Cap'n Proto) doesn't implement this case yet. It only implements IOCP-based networking. There's a class Win32WaitObjectThreadPool defined in kj/async-win32.h meant to handle the GUI event loop approach... but at present it is not implemented. (PRs are welcome if you'd like to contribute!)
If you truly don't care about handling GUI events in a timely fashion, then perhaps a hack would work: You could use kj::Timer to create a loop that waits for a second, then checks the Win32 GUI event queue, then waits again, and so on. This is really ugly but would probably be easy to implement. I'm not sure if kj::Timer is exposed via EZ-rpc, so you may have to go to lower-level building blocks like kj::setupAsyncIo() instead.
I have written a MultiThread C++ Codes using boost.
I have the below code in my main thread:
while (!mInputQueue.empty() && mStartProcessJobs)
mProcessJobs.wait(lock);
the second line should be executed immediately after the first line and context switching should not occur. How can I do this?
Depending on the nature of the jobs, you can use an asynchronous service provider.
Often these exist for asynchronous IO (e.g. sockets in non-blocking mode, IO completion ports on windows, libaio etc.)
Boost Asio harnesses all these interfaces (and some more, related to timers, platform specific handles or e.g. serial ports) into a service object. This enables you to run many jobs asynchronously, potentially all on a single thread. This means that there is no context switching.
Asio's io_service has several ways of posting/dispatching jobs. Depending on which you use, jobs might even execute immediately and synchronously.
I suggest you look at some of the samples, as it looks to be precisely what you need.
PS. There are other - more low-level - libraries outside of boost that have the same kind of features but I haven't used them. I think the most popular are libuv/libevent (IIRC)
I have legacy application code written for a real-time operating system (RTOS). Most of the code uses an OS-Specific Inter-Process-Communication (IPC) call, that looks very similar to a signal.
It has two IPC calls:
status_code signal_push(connector, event)
event signal_wait(connector)
I want to slowly port that application to Linux in order to improve testing and debugging.
I want to create a task-description file, that covers tasks their associated IPC primitives with events and handlers.
An example would look like this:
SignalHandler(connector=crypto_connector,
events = [(cyrpto_init, crypto_init_handler),
(crypto_run, crypto_run_handler),
(crypto_done, crypto_done_handler])
Task(name=crypto, priority=1, stack_size=256, connector=crypto_connector)
From this description file it should generate a thread for each task that calls the corresponding handler, that is written by an engineer. it shall also generate call stubs similar to emit_crypto_init()
It is quite clear to me what kind of code I shall generate from this description for the RTOS but I'm still uncertain, what kind of IPC I should use for Linux. For the first version probably anything that keeps behaviour is okay. For future versions, it could be plausible to use Linux on target and therefore the IPC should have little overhead.
I found the following IPC mechanisms:
Unix signals (seems as if it is not an option)
Unix sockets (do a lot more than just sending a signal)
semaphores (do less, but we could use global integers to pass the event to the other party)
Are there more suitable IPC solutions out there, that could fit this requirement?
There are several possibilities:
Boost Signals (not to be confused with Boost Interprocess) - http://www.boost.org/doc/libs/1_51_0/doc/html/signals.html, other is
cpp-events http://code.google.com/p/cpp-events/
libsigc++
Qt Signals and Slots
I am thinking of writing a server application - along the lines of mySQL or Apache.
The main requirements are:
Clients will communicate with the server via TCP/IP (sockets)
The server will spawn a new child process to handle requests (ala Apache)
Ideally, I would like to use the BOOST libraries rather than attempt to reinvent my own. There must be code somewhere that does most of what I am trying to do - so I can use it (or atleast part of it as my starting point) can anyone point me to a useful link?
In the (hopefully unlikely) event that there is no code I can use as a starting point, can someone point out the most appropriate BOOST libraries to use - and a general guideline on how to proceeed.
My main worry is how to know when one of the children has crashed. AFAIK, there are two ways of doing this:
Using heartbeats between the parent and children (this quickly becomes messy, and introduces more things that could go wrong)
Somehow wrap the spawning of the process with a timeout parameter - but this is a dumb approach, because if a child is carrying out time intensive work, the parent may incorrectly think that the child has died
What is the best practises of making the parent aware that a child has died?
[Edit]
BTW, I am developing/running/deploying on Linux
On what platform (Windows/Linux/both)? Processes on Windows are considered more heavy-weight than on Linux, so you may indeed consider threads.
Also, I think it is better (like Apache does) not to spawn a process for each request but to have a process pool, so you save the cost of creating a process, especially on Windows.
If you are on Linux, can waitpid() be useful for you? You can use it in the non-blocking mode to check recurrently with some interval whether one of the child processes terminated
I can say for sure that Pion is your only stable option.
I have never used it but I intend to, and the API looks very clean.
As for the Boost libraries you would need:
Boost.Asio
Boost.Threading
Boost.Spirit (or something similar to parse the HTTP protocol)
Boost.IPC
What about using threads (which are supported by Boost) rather than forking the process? This would allow you to make queries about the state of a child and, imho, threads are simpler to handle than forking.
Generally Boost.Asio is good point to begin with.
But several points to be aware of:
Boost.Asio is very good library but it is not very fork aware, so don't try to share Asio
event loop between several fork processes - this would not work (i.e. - if boost::asio::io_service was created before fork - don't use it in more then one process after it)
Also it does not allow you to release file handler from boost::asio::XX::socket
so only way is to call dup and then pass it to child process.
But to be honest? I don't think you'll find any network event loop library that is
fork aware (maybe with exception of CppCMS's booster.aio that I had written
to be fork aware by myself).
Waiting for children is quite simple you can define a signal handler with sigaction
on SIGCHLD signal that is send then child crashes or exits.
So all you need to do is handle this signal and in main loop call waitpid when such
signal received.
With asio you can use "self-pipe" trick to wake the loop from sleep from signal handler.
First, take a look at CPPCMS. It might already fit your needs.
Now, as pointed by others, boost::asio is a good starting point but is really the basics of the task.
Maybe you'll be more interested in the works being done about server-code based on boost::asio : cpp-netlib (that is made to be submitted in boost once done) The author's blog.
I've made an FOSS library for creating C++ applications in a modular way. It's hosted at
https://github.com/chilabot/chila
here's my blog: http://chilatools.blogspot.com/view/sidebar
It's specially suited for generic server creation (that was my motivation for constructing it), but I think it can be used for any kind of application.
The part that has to be deployed with the final binary is LGPL, so it can be used with commercial applications.
I don't want to reinvent the wheel, and what I'm looking for most likely already exist in the FOSS world.
I'm looking for a pthread bases utility library that implements often used primitives to do communication between threads.
My main need is some kind of blocking queue for fixed size messages and the ability to wait for data to arrive on multiple queues at the same time (what you usually do using poll and select with file-handles).
Does something like this exist?
Programming language is C++ but I'm fine with a C library. OS is Linux but anything posix will do.
EDIT
I'm not looking for a thin wrapper around pthreads (like boost::thread or so). I already have this up and running. I'm looking for higher level primitives. Basically What java.util.concurrancey offers for the java guys.
Your requirements are already baked into POSIX Message Queues.
Instead of using select() you can do it in reverse. Rather than waiting in a select() you can use mq_notify() to tell you when there is something to read. MQs give you the option of having a signal delivered or having them spawn a new thread to read the queue.
If you are really intent on using select(), Linux makes this painless since the mqd_t type is actually a file descriptor. You can simply use the mqd_t returned from mq_open() like any other FD in select().
Note that use of a mqd_t in select() is not portable. In theory you should be able to do something similar on other systems but I have never tested it. Since POSIX MQs have a path to an entry to the filesystem you should be able to do a straight open() on the path and use the returned file descriptor in the select(), mapping it to the mqd_t used in mq_open() to determine which queue to read. Again, I have never tried it.
There's always boost::thread.
You could try OpenMP, though I'm not sure whether it's based on the pthread API or not.
For what programming language / environment?
Some options:
C: c-pthread-queue, APR queue
Python: queue module