I have five consumers and one producer. The five consumers each output different data, from the one producer, for ~10ms. During those 10ms the producer prepares the parameters for the next output. When the output parameters are set, I want to set a flag instructing the consumers to begin the next output. I only want the producer to produce when the consumers are outputting the data.
I am unsure how to synchronize the five consumers and the single producer. I currently have two flags, runFlag and doneFlag. When a consumer reads the new data I want to set runFlag to true so the calculations begin, and I want to set doneFlag to false, as the calculations have not completed. However, if I set doneFlag to false in one consumer it may be false in another consumer before that consumer can check the flag.
I hope my question is specific enough. Please let me know if there's anything else I can provide. Also, I'm just looking for a general idea of how to proceed. I know there are multiple ways to do this, but I'm unsure which method will work the best.
Thanks!
You will need 2 events and an integer reference count.
When producer has produced some thing it:
initiates read_count = 0;
sets event readme.
starts to wait on event completed;
Consumers wait on event readme. After doing their work they ATOMICALLY increment read_count. If the read_count reaches the number of consumers, 5 in your case, then it sets the completed event. Thus producer can continue and the cycle repeats itself.
A few years back, I had to create a generic work dispatcher that does post-processing. Its not a producer-consumer exactly and may be overkill for your app, but it may give you some ideas.
I particularly like using a pair of in-memory shared queues, an outbound queue and an inbound queue, arranged like a two-way channel. If you create a queue class that has the proper synchronization for reading and writing, the producer and consumers can become independent. They don't need to know how to synchronize with each other.
Your data, known to the producer and consumer, referenced in a work item class. The work item class contains all of the status flags. The data should also be thread safe.
The producer enqueues work items onto the outbound queue and each consumer dequeues a single work item. When the work is completed, the status flags are updated and the work item is posted back to inbound queue for post-processing by the producer.
IIRC, the architecture only contain three classes or so.
Related
I am working on a multithreaded middleware enviornment. The framework is basically a capturing and streaming framework. So it involves a number of threads.
To give you all a brief idea of the threading architecture:
There are seprate threads for demultiplexer, receiveVideo, DecodeVideo, DisplayVideo etc. Each thread performs its functionlity, for eg:
demultiplexer extracts audio, video packets
receivevideo receives header + payload of video packet & removes payload
DecodeVideo receives payload & decodes payload packet
DisplayVideo receives decoded packets & displays the decoded packets on display
Thus each thread feeds the extracted data to the next thread. The threads share data buffers amongst them and the buffers are synchronised through use of mutexes and semaphores. Similarly, there are other threads for handling ananlogvideo and analogaudio etc.
All the threads are spawned in during initialization but they remain blocked on a semaphore and depending upon the input(analog/digitial) selective semaphores are signalled so that specifc threads get unblocked & move on to do their work. At various stages each thread calls some lower level(driver calls)to get data or write data etc. These calls are blocking and the errors resulting from these calls(driver returning corrupted data, driver stalling) should be handled but are not being handled currently.
I wanted to implement a thread monitoring mechanism where a thread will monitor these worker threads and if an error condition occurs will take some preventive actions. As I understand certain such mechanisms are commonly used like Watchdogs in UI or MMI applications. I am trying to look for something similar.
I am using pthreads and No Boost or STL(its a legacy code, pretty much procedural C++)
Any ideas about specific framework or design patterns or open source projects which do something similar and might help in with ideas for implementing my requirement?
Can you ping the threads - periodically send each one a message on its usual input queue, interleaved with all the other normal stuff, asking it to return its status? When each handler thread gets the message, it loads the message with status stuff - how many messages its processed since the last ping, length of its input/output queue, last time that its driver returned OK, that sort of stats - and queues it back to your Thread Monitoring Mechanism. Your TMM would have to time out the replies in case some thread/s is/are stuck.
You could, maybe, just post one message down the whole chain, each thread adding its own status in different fields. That would mean only one timeout, after which your TMM would have to examine the message to see how far down the chain it got.
There are other things - I like to keep an on-screen dump, on a 1s timer, of the length of queues and depth of buffer pools. If something stuffs, I can usually tell roughly where it is, (eg. a pool is emptying and some queue is growing - the queue comsumer is wasted).
Rgds,
Martin
What about using a signalling system to wake up your monitoring thread when something's gone awry in one of your worker threads. You can emulate the signalling with an ResetEvent of some type.
When an exception occurs in your worker thread, you have some data structure you fill up with the data about the exception and then you can pass that on to your monitoring thread. You wake up the monitoring thread by using the event.
Then the monitoring thread can do what you need it to do.
I'm guessing you don't wish to have your monitoring thread active unless something has gone wrong, right?
Let's say you have an entity, say, "Person" in your system and you want to process events that modify various Person entities. It is important that:
Events for the same Person are processed in FIFO order
Multiple Person event streams be processed in parallel by different threads/processes
We have an implementation that solves this using a shared database and locks. Threads compete to acquire the lock for a Person and then process events in order after acquiring the lock. We'd like to move to a message queue to avoid polling and locking, which we feel would reduce load on the DB and simplify the implementation of the consumer code.
I've done some research into ActiveMQ, RabbitMQ, and HornetQ but I don't see an obvious way to implement this.
ActiveMQ supports consumer subscription wildcards, but I don't see a way to limit the concurrency on each queue to 1. If I could do that, then the solution would be straightforward:
Somehow tell broker to allow a concurrency of 1 for all queues starting with: /queue/person.
Publisher writes event to queue using Person ID in the queue name. e.g.: /queue/person.20
Consumers subscribe to the queue using wildcards: /queue/person.>
Each consumer would receive messages for different person queues. If all person queues were in use, some consumers may sit idle, which is ok
After processing a message, the consumer sends an ACK, which tells the broker it's done with the message, and allows another message for that Person queue to be sent to another consumer (possibly the same one)
ActiveMQ came close: You can do wildcard subscriptions and enable "exclusive consumer", but that combination results in a single consumer receiving all messages sent to all matching queues, reducing your concurrency to 1 across all Persons. I feel like I'm missing something obvious.
Questions:
Is there way to implement the above approach with any major message queue implementation? We are fairly open to options. The only requirement is that it run on Linux.
Is there a different way to solve the general problem that I'm not considering?
Thanks!
It looks like JMSXGroupID is what I'm looking for. From the ActiveMQ docs:
http://activemq.apache.org/message-groups.html
Their example use case with stock prices is exactly what I'm after. My only concern is what happens if the single consumer dies. Hopefully the broker will detect that and pick another consumer to associate with that group id.
One general way to solve this problem (if I got your problem right) is to introduce some unique property for Person (say, database-level id of Person) and use hash of that property as index of FIFO queue to put that Person in.
Since hash of that property can be unwieldy big (you can't afford 2^32 queues/threads), use only N the least significant bits of that hash.
Each FIFO queue should have dedicated worker that will work upon it -- voila, your requirements are satisfied!
This approach have one drawback -- your Persons must have well-distributed ids to make all queues work with more-or-less equal load. If you can't guarantee that, consider using round-robin set of queues and track which Persons are being processed now to ensure sequential processing for same person.
If you already have a system that allows shared locks, why not have a lock for every queue, which consumers must acquire before they read from the queue?
My process reads from a single queue tasks that need to be sent to several destinations.
We need to maintain order between the tasks (ie task that arrived in the queue at 00:00 needs to be sent before the task that arrived at 00:01) therefore we cannot use thread pool. Order needs to be maintained per destination.
One solution is to create a dedicated thread per destination. The main thread reads the
task from the queue and depending on the destination finds the correct thread.
This solution has a problem: if a worker thread is busy, the master thread would remain blocked, making the system slow. What I need is a new queue per thread. The master thread
shares the resources to the queues and the worker thread reads the new queues for incoming
messages...
I would like to share my thought with the SO community, and I am searching for a C/C++ solution close to me description. Is there a library that implements such model?
The design you want is fairly straightforward; I think you can probably write the code you need and get it working in an hour or two. Looking for a 3rd party library to implement this is probably overkill (unless I am misunderstanding the problem).
In particular, for each 'worker' thread, you need a FIFO data structure (e.g. std::queue), a Mutex, and a mechanism that the 'master' thread can use to signal the thread to wake up and check the data structure for new messages (e.g. a condition variable, or a semaphore, or even a socketpair that the worker blocks on reading, and the master can send a byte on to wake the worker up).
Then to send a task to a particular worker thread, the master would do something like this (pseudocode):
struct WorkerThreadData & workerThread = _workerThreads[threadIndexIWantToSendTo];
workerThread.m_mutex.Lock();
workerThread.m_incomingTasksList.push_back(theNewTaskObject);
workerThread.m_mutex.Unlock();
workerThread.m_signalMechanism.SignalThreadToWakeUp(); // make sure the worker looks at the task list!
... and each worker thread would have an event loop like this:
struct WorkerThreadData & myData = _workerThreads[myWorkerIndex];
TaskObject * taskObject;
while(1)
{
myData.m_signalMechanism.WaitForSignal(); // block until the main thread wakes me up
myData.m_mutex.Lock();
taskObject = (myData.m_incomingTasks.length() > 0) ? myData.m_incomingTasks.pop_front() : NULL;
myData.m_mutex.Unlock();
if (taskObject)
{
taskObject->DoTheWork();
delete taskObject;
}
}
This will never block the master thread (for any significant amount of time), since the Mutex is only held very briefly by anyone. In particular, the worker threads are not holding the mutex while they are working on a task object.
The "need to maintain order" all-but-directly states that you're going to be executing the tasks serially no matter how many threads you have. That being the case, you're probably best off with just one thread servicing the requests.
You could gain something if the requirement is a bit looser than that -- for example, if all the tasks for one destination need to remain in order, but there's no ordering requirement for tasks with different destinations. If this is the case, then your solution of a master queue sending tasks to an input queue for each individual thread sounds like quite a good one.
Edit:
Specifying the number of threads/mutexes dynamically is pretty easy. For example, to take the number from the command line, you could do something on this order (leaving out error and sanity checking for the moment):
std::vector<pthread_t> threads;
int num_threads = atoi(argv[1]);
threads.resize(num_threads);
for (int i=0; i<num_threads; i++)
pthread_create(&threads[i], NULL, thread_routine, NULL);
At the moment I am using a producer consumer model for the rendering portion of a realtime graphics application. The consumer will continually look for data in our queue(infinite loop); however I am fearful that this may cause my simulation to get out of sync of the main loop. I think this is the fast producer slow consumer problem - compounded by the fact that the simulation is restrained to a certain amount of time.
Question - what is the best method to keep this all in balance and make sure the consumer has enough time to finish, but also that the simulation does not move to the next frame before we are finished rendering our current frame(or at least be able to detect this and skip rendering the next frame - or interrupt the current frame being rendered) I am currently just interrupting and joining after each consumer is finished
Second Question: if you look at the code below you will see that I am currently just calling interrupt and join after adding rendering jobs to the queue - this allows the thread all the time it needs to complete its operation, and to respond to the interrupt when finished. How can I then reuse threads in a thread pool after interrupt_all and join_all are called? (i.e. if i call drawNextFrame again)
The producer is part of the main thread of execution (I don't think this affects anything)
pseudo code:
void renderSystem::init()
create queue to hold work;
create consumer threads of type RenderConsumer set to watch our queue;
add threads to thread_pool of consumers called 'RenderThreads'
void renderSystem::drawNextFrame()
for each thread in 'RenderThreads' divy up work;
add work assignment to queue;
//RenderThreads will now successfully start pulling data from our queue
renderThreads.interupt_all();
renderThreads.join_all();
int main()
renderer = renderSystem class;
renderer.init()
while(not_gameover)
renderer.drawNextFrame();
doOtherCoolStuff();
profit(?)
return(0)
if you need to look at the consumer class see below:
pseudo code:
RenderConsumer::operator () ()
while(true)
try to dequeue from queue
//digest any packet we get
for each ( pixel in packet )
computePrettyStuff()
//we are now done with packet that we got
this_thread::interruption_point();
I tried to make this simple and quick to digest, thank you for your time
#1. I would do this by counting the amount in the queue after each render. If it too high, then either
a. Dump the queue
b. Set a boolean variable to false
That variable will be shared between the threads, and when the producer sees that it is false, it begins waiting on a condition variable. The consumer then notifies the producer when the queue is down to an acceptable level again.
#2. Probably not possible with join_all, as the postcondition to join_all is
Every thread in the group has
terminated.
according to the reference.
It might however be possible, using barriers instead of join_all, but then you would have to find a way to provide them data, which would invariably end up needing some more shared variables.
Let's say that I have a switch statement in my thread function that evaluates for triggered events. Each case is a different event. Is it better to put the call to ResetEvent at the end of the case, or at the beginning? It seems to me that it should go at the end, so that the event cannot be triggered again, until the thread has finished processing the previous event. IF it is placed at the beginning, the event could be triggered again, while being processed.
Yes. think that is the way to go. Create a manual reset event (second parameter of CreateEvent API) so that event is not automatically reset after setting it.
If you handle incoming traffic using a single Event object (implying you have no inbound queue), you will miss events. Is this really what you want?
If you want to catch all events, a full-blown producer-consumer queue wouold be a better bet. Reference implementation for Boost.Thread here.
One problem that comes up time and
again with multi-threaded code is how
to transfer data from one thread to
another. For example, one common way
to parallelize a serial algorithm is
to split it into independent chunks
and make a pipeline — each stage in
the pipeline can be run on a separate
thread, and each stage adds the data
to the input queue for the next stage
when it's done. For this to work
properly, the input queue needs to be
written so that data can safely be
added by one thread and removed by
another thread without corrupting the
data structure.