I currently have N threads that are all open udp/tcp connections. When I receive the first packet out of any of the threads, the main thread (that called the N threads) needs to pause execution in the N threads and do some work before resuming the N threads.
My initial idea was to have all N threads use a common mutex that waits on a pthread_cond_broadcast from the main thread. As I understand it, all N threads will resume execution in some sequential order as determined by the scheduler when broadcast is called, since they all depend on the same mutex. However, I need them to resume in parallel.
This is essentially what my problem looks like:
Main thread:
//create N threads. Each thread is connected to a different
//location but uses the same code
//detect that one of the threads has received a packet
//tell all child threads to pause
pauseThreads();
//do some work on the first packet
resumeThreads();
//tell all child threads to resume
Child thread code:
while(true){
recv() data
//the other N-1 threads should ideally block here,
//since I'd like to process just the
//very first packet
//hopefully pauseThreads() has been called by the main
//thread by here if the first packet has been received.
//All threads should block here until the main thread
//is done processing the first packet. Once it's done
//processing, *firstPacket will be false and the if statement
//can be skipped over
//only one thread should ever access this
if(*firstPacket /*this is a global variable*/ ){
//process first packet
*firstPacket = false;
//the thread that receives the first packet should block here
}
//process same packet data in another format
}
The reason the threads need to restart concurrently is that speed is an issue and I can't wait for each thread to get through its own data processing one by one. I have the blocking within the if statement figured out, but I cannot think of a way to efficiently block the N-1 threads.
Best
Get server to add timestamp or sequence number to udp packets.
Alternatively
In the receiver, I assume you have exactly one net work adapter, and only one route between server and you. Otherwise packets may get reordered.
So now we need to time/sequence stamp packets, before further processing. Use a single thread for this, then farm out the rest of the task to worker threads.
Related
I would like to have a worker thread wait for a command, do it, and then send the result back to the caller. This differs from the regular producer/consumer problem because of the response back.
This is what I'm thinking:
main
{
construct workers
push to shared input buffers
notify workers
wait
print results from shared output buffer
}
worker
{
wait
read from shared input buffer
do work
notify main
}
Am I on the right track? Is there a chance that the worker could respond before main starts waiting?
(I'm using C++ if that is relevant)
You are on the right track, but you can simplify things a bit and eliminate the need to signal or wait for signal
main
{
push to shared input buffers
construct workers // create and immediately run. No need for signal
while more workers
join worker // will block until worker thread exits
print results from shared output buffer
}
worker
{
read from shared input buffer
do work
}
Depending on how you are partitioning the shared buffer, or not partitioning, you may need to protect it from concurrent writes.
I have an applications with 2 threads. The first thread (main-thread) and the second thread (tcp-client-thread). The main-thread generates some messages and puts their in queue for tcp-client-thread. tcp-client-thread has to send those messages to server. But, tcp-client-thread also has to receive some messages from server.
How can I do that? recv stops current thread. Set up timeout forrecv? Then after recv timeout check queue (from main-thread) and if there is messages send their is no any messages start recv again?
You can do your I/O in one non-spinning/non-delayed thread but it's much more complex then just simply creating another thread as suggested in another answer. In short, you'll have to modify your code to handle waiting for multiple event types simultaneously, i.e. an event on the socket OR on a condition signalling data to send, for example. On Windows, you'd use something like WSAEventSelect + WaitForMultipleObjects instead of select, and on Linux you'd use something like eventfd with select. Note that when handling the socket, if it's blocking, you'd want to check for readability before issuing a recv and check for writeability before issuing a send so you don't block on one or the other. Like I said though, easier to just create a send thread...
The thing you need is non-blocking/asynchronous I/O.
You should read some theory before trying to forge any code.
This article, for example:
http://www.wangafu.net/~nickm/libevent-book/01_intro.html
If you are going to use 2 threads, you might want to extend to 3 threads. Let the send and receive functions be on separate threads.
The send thread is sleeping until the main thread gives it data. Specifically, a function in the send software unit places data into the queue, then signals the thread to wake up. The thread wakes up and sends data until the queue is empty, then it goes back to sleep.
Conversely, the receive thread sleeps until it receives data. It appends data to another queue, notifies the main thread that data was received and goes back to sleep.
Edit 1: One Thread
Per your title, if you want to perform the I/O in one thread, you will need to have a polling loop (you can have limited waiting, but not advised).
Loop:
if (data received) then place data into input queue.
if (data in input queue) process some data (use small chunks).
if (data in output queue) send some data.
end-loop.
The idea is to keep the blocks of data small to prevent missing of incoming data. The data can be processed and output when there is no data (and with multiple iterations). This will resolve most synchronization issues.
My process reads from a single queue tasks that need to be sent to several destinations.
We need to maintain order between the tasks (ie task that arrived in the queue at 00:00 needs to be sent before the task that arrived at 00:01) therefore we cannot use thread pool. Order needs to be maintained per destination.
One solution is to create a dedicated thread per destination. The main thread reads the
task from the queue and depending on the destination finds the correct thread.
This solution has a problem: if a worker thread is busy, the master thread would remain blocked, making the system slow. What I need is a new queue per thread. The master thread
shares the resources to the queues and the worker thread reads the new queues for incoming
messages...
I would like to share my thought with the SO community, and I am searching for a C/C++ solution close to me description. Is there a library that implements such model?
The design you want is fairly straightforward; I think you can probably write the code you need and get it working in an hour or two. Looking for a 3rd party library to implement this is probably overkill (unless I am misunderstanding the problem).
In particular, for each 'worker' thread, you need a FIFO data structure (e.g. std::queue), a Mutex, and a mechanism that the 'master' thread can use to signal the thread to wake up and check the data structure for new messages (e.g. a condition variable, or a semaphore, or even a socketpair that the worker blocks on reading, and the master can send a byte on to wake the worker up).
Then to send a task to a particular worker thread, the master would do something like this (pseudocode):
struct WorkerThreadData & workerThread = _workerThreads[threadIndexIWantToSendTo];
workerThread.m_mutex.Lock();
workerThread.m_incomingTasksList.push_back(theNewTaskObject);
workerThread.m_mutex.Unlock();
workerThread.m_signalMechanism.SignalThreadToWakeUp(); // make sure the worker looks at the task list!
... and each worker thread would have an event loop like this:
struct WorkerThreadData & myData = _workerThreads[myWorkerIndex];
TaskObject * taskObject;
while(1)
{
myData.m_signalMechanism.WaitForSignal(); // block until the main thread wakes me up
myData.m_mutex.Lock();
taskObject = (myData.m_incomingTasks.length() > 0) ? myData.m_incomingTasks.pop_front() : NULL;
myData.m_mutex.Unlock();
if (taskObject)
{
taskObject->DoTheWork();
delete taskObject;
}
}
This will never block the master thread (for any significant amount of time), since the Mutex is only held very briefly by anyone. In particular, the worker threads are not holding the mutex while they are working on a task object.
The "need to maintain order" all-but-directly states that you're going to be executing the tasks serially no matter how many threads you have. That being the case, you're probably best off with just one thread servicing the requests.
You could gain something if the requirement is a bit looser than that -- for example, if all the tasks for one destination need to remain in order, but there's no ordering requirement for tasks with different destinations. If this is the case, then your solution of a master queue sending tasks to an input queue for each individual thread sounds like quite a good one.
Edit:
Specifying the number of threads/mutexes dynamically is pretty easy. For example, to take the number from the command line, you could do something on this order (leaving out error and sanity checking for the moment):
std::vector<pthread_t> threads;
int num_threads = atoi(argv[1]);
threads.resize(num_threads);
for (int i=0; i<num_threads; i++)
pthread_create(&threads[i], NULL, thread_routine, NULL);
At the moment I am using a producer consumer model for the rendering portion of a realtime graphics application. The consumer will continually look for data in our queue(infinite loop); however I am fearful that this may cause my simulation to get out of sync of the main loop. I think this is the fast producer slow consumer problem - compounded by the fact that the simulation is restrained to a certain amount of time.
Question - what is the best method to keep this all in balance and make sure the consumer has enough time to finish, but also that the simulation does not move to the next frame before we are finished rendering our current frame(or at least be able to detect this and skip rendering the next frame - or interrupt the current frame being rendered) I am currently just interrupting and joining after each consumer is finished
Second Question: if you look at the code below you will see that I am currently just calling interrupt and join after adding rendering jobs to the queue - this allows the thread all the time it needs to complete its operation, and to respond to the interrupt when finished. How can I then reuse threads in a thread pool after interrupt_all and join_all are called? (i.e. if i call drawNextFrame again)
The producer is part of the main thread of execution (I don't think this affects anything)
pseudo code:
void renderSystem::init()
create queue to hold work;
create consumer threads of type RenderConsumer set to watch our queue;
add threads to thread_pool of consumers called 'RenderThreads'
void renderSystem::drawNextFrame()
for each thread in 'RenderThreads' divy up work;
add work assignment to queue;
//RenderThreads will now successfully start pulling data from our queue
renderThreads.interupt_all();
renderThreads.join_all();
int main()
renderer = renderSystem class;
renderer.init()
while(not_gameover)
renderer.drawNextFrame();
doOtherCoolStuff();
profit(?)
return(0)
if you need to look at the consumer class see below:
pseudo code:
RenderConsumer::operator () ()
while(true)
try to dequeue from queue
//digest any packet we get
for each ( pixel in packet )
computePrettyStuff()
//we are now done with packet that we got
this_thread::interruption_point();
I tried to make this simple and quick to digest, thank you for your time
#1. I would do this by counting the amount in the queue after each render. If it too high, then either
a. Dump the queue
b. Set a boolean variable to false
That variable will be shared between the threads, and when the producer sees that it is false, it begins waiting on a condition variable. The consumer then notifies the producer when the queue is down to an acceptable level again.
#2. Probably not possible with join_all, as the postcondition to join_all is
Every thread in the group has
terminated.
according to the reference.
It might however be possible, using barriers instead of join_all, but then you would have to find a way to provide them data, which would invariably end up needing some more shared variables.
I'm using libnetfilter_queue for my project. From C app queue is accessible by "queue file descriptor". I have 5 queues and 5 threads to handle them. What I want to achieve is to wake thread when there is exactly 2 packets in queue. I came up with idea to use select function and array of ints indicating how many packets were queued in each queue. After select exit with > 0 code I check which queue has received a packet and increment value in array, if it's bigger than 2 I wake up a thread. Everything would be fine, but select indicate that queue has data to read until I call recv and I can't do that because separate thread should handle these packets. Anyone has idea how to solve this issue? I know I can set SO_RCVLOWAT but it does not solve my problem, because I don't know what size will be those 2 packets.
As recommended by Tobu, epoll is a better choice and it performs better than select.
However, most of these polling functions will indicate there is an event (data available) unless someone reads.
If possible use the following model:
Use epoll/select to watch for the incoming data wake up the worker thread.
Let the worker thread decide what to do with the data (one packet, two or more) before actually doing the work.
OR:
One Reader thread-N Worker threads: Will use epoll to wait and read all the incoming data and post it to the corresponding worker thread's queue.
Once the # of packet reaches the threshold, wake up the Worker thread (using a semaphore).
You are looking for edge-triggered event notifications — notifications that are sent when the quantity of available data changes. epoll works like that when using the EPOLLET flag, and by default will rearm the notification so that you keep being notified of new packets.
Please note that you will be notified only once if several packets arrive between two epoll_wait calls.