Using C++ I'm planning to have a producer process writing a data vector and then several consumer processes reading that data. There will be a shared memory segment (Boost::Interprocess) where the data vector will be stored. Issue is: I have no control over the order in which the processes will be launched by a third party application, it could be that the consumer might launch before the producer has produced any data. What mechanisms are available to coordinate the processes so that the consumer processes can be commanded to wait patiently until the producer signals the data is ready; no matter what the order in which the processes launch?
I guess named semaphore is a good choice. Your producer and consumers application should agree (hard-coded) the name of the semaphore, something like /tmp/mySem and only the producer must create and post the semaphore while the consumers should wait for semaphores existance and state.
If creating shared memory is responsibility of producer process, then you can use boost barrier for synchronizing startup.
You can create a barrier for creating shared memory, maybe some number of jobs to deploy. After reaching this barrier, consumer processes can continue to process them.
You can look to details of boost barrier at this page
Related
I want synchronize between producer and consumer processes. I use a cyclic buffer without mutexes or semaphores.
This solution allows me to avoid waiting on synchronization.
I use shared memory cyclic buffer for the data sharing.
Now I want to allow the consumer process to sleep while the producer doesn't produce new data.
Regular named semaphore doesn't make the job for me, because it has an integer value. I need some binary semaphore, that is not incrementing, but just signals the consumer whether there is a new data or not.
I.e. the produced should not increment the semaphore value, but just to set it to 1.
I write in C++ for Linux OS
I intend to implement a thread pool to manage threads in my project. The basic structure of thread pool come to my head is queue, and some threads generate tasks into this queue, and some thread managed by thread pool are waiting to handle those task. I think this is class producer and consumer problem. But when I google thread pool implementation on the web, I find those implementation seldom use this classic model, so my question is why they don't use this classic model, does this model has any drawbacks? why they don't use full semaphore and empty semaphore to sync?
If you have multiple threads waiting on a single resource (in this case the semaphores and queue) then you are creating a bottle neck. You are forcing all tasks through one queue, even though you have multiple workers. Logically this might make sense if the workers are usually idle, but the whole point of a thread pool is to deal with a heavily loaded scenario where the workers are kept busy (for maximum through-put). Using a single input queue will be particularly bad on a multi-processor system where all workers read and write the head of the queue when they are trying to get the next task. Even though the lock contention might be low, the queue head pointer will still need to be shared/communicated from one CPU cache to another each time it is updated.
Think about the ideal case: all workers are always busy. When a new task is enqueued you want it to be dispatched to the worker that will complete its current/pending task(s) first.
If, as a client, you had a contention-free oracle that could tell you which worker to enqueue a new task to, and each worker had its own queue, then you could implement each worker with its own multi-writer-single-reader queue and always dispatch new tasks to the best queue, thus eliminating worker contention on a single shared input queue. Of course you don't have such an oracle, but this mechanism still works pretty well until a worker runs out of tasks or the queues get imbalanced. "Work stealing" deals with these cases, while still reducing contention compared to the single queue case.
See also:
Is Work Stealing always the most appropriate user-level thread scheduling algorithm?
Why there's no Producer and Consumer model implementation
This model is very generic and could have lots of different explanations, one of the implementation could be a Queue:
Try Apache APR Queue:
It's documented as Thread Safe FIFO bounded queue.
http://apr.apache.org/docs/apr-util/1.3/apr__queue_8h.html
I have 6 threads running in my application continuously. The scenario is:
One thread continuously gets the messages and inserts into a message queue. Other 4 threads can be considered as workers which continuously fetch messages from queue and process them. The other final thread populates the analytics information.
Problem:
Now the sleep durations for getting messages thread is 100ms. Worker threads is 200ms. When I ran this application the messages fetch thread is taking control and inserting into the queue thus increasing the heap. The worker threads are not getting chance to process the messages and deallocate them. Finally its resulting into out of memory.
How to manage this kind of scenario so that equal opportunity is given for messages fetch thread and worker thread.
Thanks in advance :)
You need to add back-pressure to your producer thread. Usually this will done by using blocking consumer-producer queues. Producer adds items to queue, consumers dequeues items from queue and process them. If queue is empty, consumers blocks until producer adds something to queue. If queue is full producer blocks until consumers fetch items from the queue.
One system of flow-control that I use often is to create a large pool of message objects at startup and never create any more. The *objects are stored on a thread-safe, blocking 'pool queue' and circulated around, popped from the pool by producer/s, queued to consumer/s on other blocking queues and then pushed back onto the pool queue when 'consumed'.
This caps memory use, provides flow-control, (if the pool empties, the producer/s block on it until messages are returned from consumers), and eliminates continual new/delete/malloc/free. The more complex and slower bounded queues are not necessary and all queues need only to be large enough to hold the, (known), maximum number of messages.
Using 'classic' blocking queues does not require any Sleep() calls.
Your question is a little vague so I can give you these guidelines instead of a code:
Protect mutual data with Mutex. In a multi-threaded consumer producer problem usually there is a race condition on the mutual data (the message in your program). One thread is attempting to write on the mutual memory location while the other is trying to read from the same location. The message read by the reader might be corrupted because the writer has wrote over it in the middle of reading process. You can lock the mutual memory location with a Mutex. Each one of the threads should acquire this lock in order to be able to read or modify the mutual data. This way the consumer process will be absolutely sure that data has not been modified. However you should note that acquiring this lock might hold back the producer thread so you should release the lock as soon as possible.
Use condition variables to notify consumer threads. If you do not use a notification mechanisms all consumer threads should actively check for data production which will use up system resources. The consumer threads should easily go to sleep knowing that the producer thread will notify them whenever a message is ready.
The threading library in C++ 11 has everything you need to implement a consumer producer application. However if you are not able to upgrade your compiler you could use boost threading library as well.
You want to use a bounded queue which when full will block threads trying to enqueue until more space is available.
You can use concurrent_bounded_queue from tbb, or simply use a semaphore initialized to the maximum queue size, and decrement on enqueue and increment on dequeue. boost::thread doesn't provide semaphores natively, but you can implement it using locks and condition variables.
In my application in C++ I am using pthreads. I have a main thread that parcels out work to another thread to read from disk and a third thread to write to disk. This allows the main thread to perform in real-time and not get hiccups due to disk io by the read thread keeping ahead of the main threads consumption and the write thread keeping up with the main threads production. In other words, the read thread produces data for processing by the main thread and the write thread consumes data produced by the main thread.
In my application, I have shared resources between the main thread and the read thread and I have another set of shared resources between the main thread and the write thread. During periods when no synchronization between threads is needed, the main thread freely accesses the the shared resources of the read and write threads. Then, when it wishes to supply work to be read or written, it sets up the shared resources to indicate the required work and then releases a condition variable to notify the read or write thread (as appropriate) that it should gather the request from the shared area. It then blocks itself until the requested thread completes its gathering of the request and releases a separate condition variable upon which it waits. In this way, the main thread in its normal state accesses the shared resource and then blocks itself by waiting on the condition during periods when its read or write thread needs access, whereas the read and write threads are opposite in that in their normal state they do not access the shared resource, but only do so during synchronization periods when the main thread has signaled and is waiting for them to complete their access.
Here is some very rough pseudo code (do not worry about exact flow here, this is just a rough idea of the functioning code):
MainThread:
do {
While (no pending requests) {
manipulate shared resources;
}
if (need write && ready for read requests) {
setup write request in shared resources;
Release condition startWrite variable;
Wait on condition endWrite variable;
}
if (need read && ready for read requests) {
setup read request in shared resources;
Release condition startRead variable;
Wait on condition endRead variable;
}
}
ReadThread:
do {
Notify I am ready for read requests;
Wait on condition startRead;
copy request out of shared resources;
update shared resources to account for reads completed;
Release condition EndRead;
Perform Read from disk;
}
WriteThread:
do {
Notify I am ready for write requests;
Wait on condition startWrite;
copy request out of shared resources;
update shared resources to account for writes completed;
Release condition EndWrite;
Perform write to disk;
}
My question is whether this is considered just a normal producer consumer design pattern or whether there is a more specific commonly accepted design pattern that would describe what I have created?
You're getting some background activity to perform input and output, independently of the mainline processing. Reads can be scheduled ahead of time so the data is already there when needed, and writes can be farmed out to as a background activity so the mainline doesn't have to block on comitting data to storage.
This "design pattern" has a name, being known as "asynchronous IO".
Have you looked at the read/write lock pattern? The scheduler pattern would be another idea though I'm not sure how well this would work in your given situation.
You could describe this as an instance of the "Master/Worker" pattern. The "master" thread hands out work (read or write requests) to the "worker" threads (the read thread and the write thread).
My project has a queue, a server and a timer. The server receives data and puts it in the queue and the timer process the queue. When the queue is processed, external processes are open with popen, which means that popen will block the timer until a process has ended.
Correct me if I'm wrong, but as both server and timer are linked to the same io_service, if the server receives data, it will somehow block io_service from proceeding to the next event, and the vice-versa is the timer blocking if a process in the queue is being executed.
I'm thinking in a solution based on boost::thread but I'm not sure of what architecture should I use as I never used threads. My options are:
Two threads - one for the timer and one for the server, each one using its own io_service
One thread - one for the timer with its own io_service. the server remains in main process
In both ways the queue (a simple map) must be shared, so I think I'll have some trouble with mutexes and other things
If someone wants to take a look at the code, it is at https://github.com/MendelGusmao/CGI-for-LCD-Smartie
Thanks!
I don't see why you can't have your server listening for connections, processing data, and placing that data in the queue in one thread while your timer takes those items out of the queue in another thread and then spawns processes via popen() to process the queue data. Unless there is a detail here that I've missed, the socket that the server will be listening on (or pipe, FIFO, etc.), is separate from the pipe that will be internally opened by the libc runtime via popen(), so your server and timer threads won't be blocking each other. You'll simply have to make sure that you have enough space in the queue to store the data coming in from the server without overflowing memory (i.e., if this is a high-data-rate application, and data is coming in much faster than it's being processed, you'll eventually run out of memory).
Finally, while guarding a shared queue via muextes is a good thing, it's actually unnecessary for only a single producer/consumer situation like you're currently describing if you decide to use a bounded queue (i.e., a ring-buffer). If you decide on an unbounded queue, while there are some lockless algorithms out there, they're pretty complex, and so guarding an unbounded queue like std::queue<T> with a mutex is an absolute must.
I have implemented almost the exact thing you have described using windows threads. I had my consumer wait on an event HANDLE which is fired by the producer when the queue gets too long. There was a timeout on the wait as well so that if the queue was not filled fast enough the consumer would still wait and process the queue. It was a service in windows so the main thread was used for that. And yes, mutexes will be required to access the shared object.
So I used two threads (not including the main), 1 mutex, 1 shared object. I think your better option is also two threads as it keeps the logic cleaner. The main thread just starts the two threads and then waits (or can be used for signalling, control, output), and the two other threads are just doing their own jobs.