I have a class TaskManager that holds a queue of tasks. Each time the next task is popped and executed.
class TaskManager
{
TaskQueue m_queue;
svc_tasks()
{
while (!m_queue.empty())
{
Task* task = m_queue.pop();
task->execute();
}
}
};
Inside the Task there are certain points I would like to pause for at least SLEEP_TIME_MS milliseconds. During this pause I would like to start executing the next task. When the pause ends I would like to put the task in the queue again.
class Task
{
int m_phase = -1;
execute()
{
m_phase++;
switch(m_phase)
{
case 0:
...
do_pause(SLEEP_TIME_MS);
return;
case 1:
...
break;
}
}
};
Is there a scheduler in std (C++ 17) or boost that I could use that would call a handler function when SLEEP_TIME_MS passes?
Thank you for any advice
You can use boost::asio::high_resolution_timer with its async_wait method.
Every time when you want to schedule the operation of pushing task into queue you have to:
create high_resolution_timer
call expires_after which specifies the expiry time (SLEEP_TIME_MS) i.e. when handler is called. In your case in this handler you push a task into the queue.
call async_wait with your handler
If we assume that execute method returns bool which indicates whether a task is completed (all phases were executed), it may be rewritten into sth like this:
while (!m_queue.empty()) // this condition should be changed
{
Task* task = m_queue.pop();
bool finished = task->execute();
if (!finished)
scheduler works here - start async_wait with handler
}
If I understand correctly, you want to push task into queue when SLEEP_TIME_MS is expired, so you cannot break loop when queue is empty, because you have to wait until pending tasks will be completion. You can introduce stop flag. And break loop on demand.
Below I put a snippet of code which works in the way you described (I hope):
struct Scheduler {
Scheduler(boost::asio::io_context& io)
: io(io) {}
boost::asio::io_context& io;
template<class F>
void schedule (F&& handler) {
auto timer = std::make_shared<boost::asio::high_resolution_timer>(io);
timer->expires_after(std::chrono::milliseconds(5000)); // SLEEP_TIME_MS
timer->async_wait(
[timer,handler](const boost::system::error_code& ec) {
handler();
});
}
};
struct Task {
int phase = -1;
bool execute() {
++phase;
std::cout << "phase: " << phase << std::endl;
if (phase == 0) {
return false;
}
else {
}
return true;
}
};
struct TaskManager {
Scheduler s;
std::queue<std::shared_ptr<Task>> tasks;
std::mutex tasksMtx;
std::atomic<bool> stop{false};
TaskManager(boost::asio::io_context& io) : s(io) {
for (int i = 0; i < 5; ++i)
tasks.push(std::make_shared<Task>());
}
void run() {
while (true) {
if (stop)
break;
{
std::lock_guard<std::mutex> lock{tasksMtx};
if (tasks.empty())
continue;
}
std::shared_ptr<Task> currTask = tasks.front();
tasks.pop();
bool finished = currTask->execute();
if (!finished)
s.schedule( [this, currTask](){ insertTaskToVector(std::move(currTask)); } );
}
}
template<class T>
void insertTaskToVector(T&& t) {
std::lock_guard<std::mutex> lock{tasksMtx};
tasks.push(std::forward<T>(t));
}
};
int main() {
boost::asio::io_context io;
boost::asio::io_context::work work{io};
std::thread th([&io](){ io.run();});
TaskManager tm(io);
tm.run();
Related
In the following code, I create a toy class that has a thread which writes to a queue while the other thread reads from that queue and prints it to stdout. Now, in order to cleanly shutdown the system, I setup a handler for SIGINT. I am expecting the signal handler to set up the std::atomic<bool> variable stopFlag, which will lead threadB to push a poison pill (sentinel) on to the queue encountering which threadA will halt.
class TestClass
{
public:
TestClass();
~TestClass();
void shutDown();
TestClass(const TestClass&) = delete;
TestClass& operator=(const TestClass&) = delete;
private:
void init();
void postResults();
std::string getResult();
void processResults();
std::atomic<bool> stopFlag;
std::mutex outQueueMutex;
std::condition_variable outQueueConditionVariable;
std::queue<std::string> outQueue;
std::unique_ptr<std::thread> threadA;
std::unique_ptr<std::thread> threadB;
};
void TestClass::init()
{
threadA = std::make_unique<std::thread>(&TestClass::processResults, std::ref(*this));
threadB = std::make_unique<std::thread>(&TestClass::postResults, std::ref(*this));
}
TestClass::TestClass():
stopFlag(false)
{
init();
}
TestClass::~TestClass()
{
threadB->join();
}
void TestClass::postResults()
{
while(true)
{
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
std::string name = "ABCDEF";
{
std::unique_lock<std::mutex> lock(outQueueMutex);
outQueue.push(name);
outQueueConditionVariable.notify_one();
}
if(stopFlag)
{
/*For shutting down output thread*/
auto poisonPill = std::string();
{
std::unique_lock<std::mutex> lock(outQueueMutex);
outQueue.push(poisonPill);
outQueueConditionVariable.notify_one();
}
threadA->join();
break;
}
}
}
void TestClass::shutDown()
{
stopFlag = true;
}
std::string TestClass::getResult()
{
std::string result;
{
std::unique_lock<std::mutex> lock(outQueueMutex);
while(outQueue.empty())
{
outQueueConditionVariable.wait(lock);
}
result= outQueue.front();
outQueue.pop();
}
return result;
}
void TestClass::processResults()
{
while(true)
{
const auto result = getResult();
if(result.empty())
{
break;
}
std::cout << result << std::endl;
}
}
static void sigIntHandler(std::shared_ptr<TestClass> t, int)
{
t->shutDown();
}
static std::function<void(int)> handler;
int main()
{
auto testClass = std::make_shared<TestClass>();
handler = std::bind(sigIntHandler, testClass, std::placeholders::_1);
std::signal(SIGINT, [](int n){ handler(n);});
return 0;
}
I compiled this using gcc 5.2 using the -std=c++14 flag. On hitting Ctrl-C on my CentOS 7 machine, I get the following error,
terminate called after throwing an instance of 'std::system_error'
what(): Invalid argument
Aborted (core dumped)
Please help me understand what is going on.
What happens is that your main function exits immediately destroying global handler object and then testClass. Then the main thread gets blocked in TestClass::~TestClass. The signal handler ends up accessing already destroyed objects, which leads to the undefined behaviour.
The root cause is undefined object ownership due to shared pointers - you do not know what and when ends up destroying your objects.
A more general approach is to use another thread to handle all signals and block signals in all other threads. That signal handling thread then can call any functions upon receiving a signal.
You also do not need the smart pointers and function wrappers here at all.
Example:
class TestClass
{
public:
TestClass();
~TestClass();
void shutDown();
TestClass(const TestClass&) = delete;
TestClass& operator=(const TestClass&) = delete;
private:
void postResults();
std::string getResult();
void processResults();
std::mutex outQueueMutex;
std::condition_variable outQueueConditionVariable;
std::queue<std::string> outQueue;
bool stop = false;
std::thread threadA;
std::thread threadB;
};
TestClass::TestClass()
: threadA(std::thread(&TestClass::processResults, this))
, threadB(std::thread(&TestClass::postResults, this))
{}
TestClass::~TestClass() {
threadA.join();
threadB.join();
}
void TestClass::postResults() {
while(true) {
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
std::string name = "ABCDEF";
{
std::unique_lock<std::mutex> lock(outQueueMutex);
if(stop)
return;
outQueue.push(name);
outQueueConditionVariable.notify_one();
}
}
}
void TestClass::shutDown() {
std::unique_lock<std::mutex> lock(outQueueMutex);
stop = true;
outQueueConditionVariable.notify_one();
}
std::string TestClass::getResult() {
std::string result;
{
std::unique_lock<std::mutex> lock(outQueueMutex);
while(!stop && outQueue.empty())
outQueueConditionVariable.wait(lock);
if(stop)
return result;
result= outQueue.front();
outQueue.pop();
}
return result;
}
void TestClass::processResults()
{
while(true) {
const auto result = getResult();
if(result.empty())
break;
std::cout << result << std::endl;
}
}
int main() {
// Block signals in all threads.
sigset_t sigset;
sigfillset(&sigset);
::pthread_sigmask(SIG_BLOCK, &sigset, nullptr);
TestClass testClass;
std::thread signal_thread([&testClass]() {
// Unblock signals in this thread only.
sigset_t sigset;
sigfillset(&sigset);
int signo = ::sigwaitinfo(&sigset, nullptr);
if(-1 == signo)
std::abort();
std::cout << "Received signal " << signo << '\n';
testClass.shutDown();
});
signal_thread.join();
}
On your platform this signal handler is invoked when a real SIGINT signal comes. The list of functions that can be invoked inside of this signal handler is rather limited and calling anything else leads to an undefined behavior.
I have the following code which runs as service in linux based device.
It has one mqtt call back which receives the message whenever some
one publish it on the subscribed topic.
One thread is to process the incoming messages on the queue. Once they processed the result message will be pushed to out_message queue.
One more thread is to process the outgoing message queue.
I have used the condition_variable to for resource sharing between the threads.
Problem is after some time ( on random time ) the cpu utilization of this application reaches to 100 %. Any issues in the given code to correct my process. Please help me !! Thank you very much in advance.
void pushMessage(std::string rData) {
in_mutex.lock();
in_queue.push(rData);
in_mutex.unlock();
in_cv.notify_all();
}
void pushOutGoingMessage(Json::Value data) {
out_mutex.lock();
out_queue.push(data);
out_mutex.unlock();
out_cv.notify_all();
}
void processOutGoingMessages() {
while (true) {
Json::Value data;
{
std::unique_lock<std::mutex> lock(out_mutex);
while (out_queue.empty()) {
out_cv.wait(lock);
}
data = out_queue.front();
out_queue.pop();
lock.unlock();
}
if (!data.isNull()) {
parseOutGoingMessages(data);
}
}
}
void processMessage() {
while (true) {
std::string data = "NO_DATA";
{
std::unique_lock<std::mutex> lock(in_mutex, std::try_to_lock);
if (!lock.owns_lock()) {
} else {
while (in_queue.empty()) {
in_cv.wait(lock);
}
data = in_queue.front();
in_queue.pop();
lock.unlock();
}
}
if (data.compare("NO_DATA") != 0) {
parseMessage(data);
}
}
}
void parseOutGoingMessages(Json::Value rJsonMessage) {
// mqtt client send method
mqtt_client.push_message(rJsonMessage.toStyledString(),
rJsonMessage["destination"].asString());
}
void parseMessage(std::string rMessage) {
try {
debug(rMessage);
// application logic
} catch (std::exception &e) {
debug("ERRO HANDLED IN PARSING ::" + std::string(e.what()));
}
}
void connectMQTT() {
// connection params
}
void OnConnectionLost(void *context, char *cause) {
// retry logic
connectMQTT();
}
void delivered(void *context, MQTTClient_deliveryToken dt) {
}
int OnMessageArrived(void *context, char *topicName, int topicLen,
MQTTClient_message *message) {
if (!message->retained) {
std::string msg((char *) message->payload, message->payloadlen);
pushMessage(msg);
}
MQTTClient_freeMessage(&message);
MQTTClient_free(topicName);
return 1;
}
void send(Json::Value rData,std::string rDestination) {
Json::Value jsonNotification;
jsonNotification["destination"] = rDestination;
jsonNotification["data"] = rData;
pushOutGoingMessage(jsonNotification);
}
int main(int argc, char **argv) {
connectMQTT();
std::thread procInMessage(processMessage);
std::thread procOutMessage(processOutGoingMessages);
procInMessage.join();
procOutMessage.join();
}
Thank you #MatthewFisher and #Mike Vine. I just modifiy the push methods on the for both queues.
void pushMessage(std::string rData) {
// in_mutex.lock();
// in_queue.push(rData);
// in_mutex.unlock();
// in_cv.notify_all();
std::unique_lock<std::mutex> lock(in_mutex);
in_queue.push(rData);
lock.unlock();
in_cv.notify_all();
}
void pushOutGoingMessage(Json::Value data) {
// out_mutex.lock();
// out_queue.push(data);
// out_mutex.unlock();
// out_cv.notify_all();
std::unique_lock<std::mutex> lock(out_mutex);
out_queue.push(data);
lock.unlock();
out_cv.notify_all();
}
And the problem was solved I guess. It was purley due to (!lock.owns_lock()) {} in push method for both the queues.
I am trying to implement queue to synchronize producer-consumer problem. However I cannot figure out why my solution deadlocks.
What it should do:
Queue with fixed size containing tasks for consumer threads, producers should be blocked until there is space in the queue. (m producers, n consumers)
Technical requirements: POSIX threads (no C++11)
void taskProducer(ArData &a) {
while (true) {
Work work = a.workGenerator();
if (work == NULL) {
a.queue.finished();
return;
}
WorkTask task(work);
a.queue.push(task);
}
}
void consumer(TaskQueue &queue) {
while (true) {
Task task = queue.pop();
if (task.end) return;
task.run();
}
}
#define NUM_OF_PRODUCERS 2
//Task ktery muze vlakno zpracovat
class Task {
public:
Task() : end(false){ }
//Execute task
virtual void run() { }
//Mark task as ending Task, consumer should exit thread
bool end;
};
//Synchronous queue
class TaskQueue {
private:
//Number of consumer threads
int numOfConsumers;
//Number of producers that finished producing
int doneProducers;
//Producers wait, Consumers post
sem_t producerNotificator;
//Consumers wait, Producers post
sem_t consumerNotificator;
//Mutex for entire TaskQueue
pthread_mutex_t mut;
//Data storage
std::queue<Task> queue;
public:
explicit TaskQueue(int m_numOfConsumers) : numOfConsumers(m_numOfConsumers), doneProducers(0) {
pthread_mutex_t mut;
pthread_mutex_init(&mut, NULL);
sem_init(&consumerNotificator, 0, 0);
sem_init(&producerNotificator, 0, numOfConsumers *10);
}
~TaskQueue() {
pthread_mutex_destroy(&mut);
sem_destroy(&consumerNotificator);
sem_destroy(&producerNotificator);
}
//Waits for empty slot in queue before pushing
void push(Task &task) {
//Wait for slot in queue
sem_wait(&producerNotificator);
//Lock before any manipulation
pthread_mutex_lock(&mut);
queue.push(task);
pthread_mutex_unlock(&mut);
//Notify consumer of waiting Task
sem_post(&consumerNotificator);
}
//Waits before item is in queue and pops it
Task pop() {
//Wait for Task in queue
sem_wait(&consumerNotificator);
//Lock before any manipulation
pthread_mutex_lock(&mut);
Task task = queue.front();
queue.pop();
pthread_mutex_unlock(&mut);
//Notify producer about empty slot in queue
sem_post(&producerNotificator);
return task;
}
//Handle finishing producers
void finished() {
//Lock before any manipulation
pthread_mutex_lock(&mut);
//Check if it is not last producer
if (NUM_OF_PRODUCERS > ++doneProducers) {
pthread_mutex_unlock(&mut);
return;
}
//If it was last producer end consumers by adding end tasks into queue
for (int i = 0; i < numOfConsumers; ++i) {
Task t;
t.end = true;
queue.push(t);
}
pthread_mutex_unlock(&mut);
//Notify all consumers about new Tasks
for (int i = 0; i < numOfConsumers; ++i) {
sem_post(&consumerNotificator);
}
}
};
As far as I can tell, when second producer calls finished, it deadlocks at pthread_mutex_lock(&mut);. No idea why.
I'm wanting a reasonably reliable threaded timer, so I've written a timer object that fires a std::function on a thread. I would like to give this timer the ability to stop before it gets to the next tick; something you can't do with ::sleep (at least I don't think you can).
So what I've done is put a condition variable on a mutex. If the condition times out, I fire the event. If the condition is signalled the thread is exited. So the Stop method needs to be able to get the thread to stop and/or interrupt its wait, which I think is what it's doing right now.
There are problems with this however. Sometimes the thread isn't joinable() and sometimes the condition is signalled after its timeout but before it's put into its wait state.
How can I improve this and make it robust?
The following is a full repo. The wait is 10 seconds here but the program should terminate immediately as the Foo is created and then immediately destroyed. It does sometimes but mostly it does not.
#include <atomic>
#include <thread>
#include <future>
#include <sstream>
#include <chrono>
#include <iostream>
class Timer
{
public:
Timer() {}
~Timer()
{
Stop();
}
void Start(std::chrono::milliseconds const & interval, std::function<void(void)> const & callback)
{
Stop();
thread = std::thread([=]()
{
for(;;)
{
auto locked = std::unique_lock<std::mutex>(mutex);
auto result = terminate.wait_for(locked, interval);
if (result == std::cv_status::timeout)
{
callback();
}
else
{
return;
}
}
});
}
void Stop()
{
terminate.notify_one();
if(thread.joinable())
{
thread.join();
}
}
private:
std::thread thread;
std::mutex mutex;
std::condition_variable terminate;
};
class Foo
{
public:
Foo()
{
timer = std::make_unique<Timer>();
timer->Start(std::chrono::milliseconds(10000), std::bind(&Foo::Callback, this));
}
~Foo()
{
}
void Callback()
{
static int count = 0;
std::ostringstream o;
std::cout << count++ << std::endl;
}
std::unique_ptr<Timer> timer;
};
int main(void)
{
{
Foo foo;
}
return 0;
}
See my comment. You forgot to implement the state of the thing the thread is waiting for, leaving the mutex nothing to protect and the thread nothing to wait for. Condition variables are stateless -- your code must track the state of the thing whose change you're notifying the thread about.
Here's the code fixed. Notice that the mutex protects stop, and stop is the thing the thread is waiting for.
class Timer
{
public:
Timer() {}
~Timer()
{
Stop();
}
void Start(std::chrono::milliseconds const & interval,
std::function<void(void)> const & callback)
{
Stop();
{
auto locked = std::unique_lock<std::mutex>(mutex);
stop = false;
}
thread = std::thread([=]()
{
auto locked = std::unique_lock<std::mutex>(mutex);
while (! stop) // We hold the mutex that protects stop
{
auto result = terminate.wait_for(locked, interval);
if (result == std::cv_status::timeout)
{
callback();
}
}
});
}
void Stop()
{
{
// Set the predicate
auto locked = std::unique_lock<std::mutex>(mutex);
stop = true;
}
// Tell the thread the predicate has changed
terminate.notify_one();
if(thread.joinable())
{
thread.join();
}
}
private:
bool stop; // This is the thing the thread is waiting for
std::thread thread;
std::mutex mutex;
std::condition_variable terminate;
};
I am trying to understand the below implementation of thread pool using the pthreads. When I comment out the the for loop in the main, the program stucks, upon putting the logs it seems that its getting stuck in the join function in threadpool destructor.
I am unable to understand why this is happening, is there any deadlock scenario happening ?
This may be naive but can someone help me understand why this is happening and how to correct this.
Thanks a lot !!!
#include <stdio.h>
#include <queue>
#include <unistd.h>
#include <pthread.h>
#include <malloc.h>
#include <stdlib.h>
// Base class for Tasks
// run() should be overloaded and expensive calculations done there
// showTask() is for debugging and can be deleted if not used
class Task {
public:
Task() {}
virtual ~Task() {}
virtual void run()=0;
virtual void showTask()=0;
};
// Wrapper around std::queue with some mutex protection
class WorkQueue {
public:
WorkQueue() {
// Initialize the mutex protecting the queue
pthread_mutex_init(&qmtx,0);
// wcond is a condition variable that's signaled
// when new work arrives
pthread_cond_init(&wcond, 0);
}
~WorkQueue() {
// Cleanup pthreads
pthread_mutex_destroy(&qmtx);
pthread_cond_destroy(&wcond);
}
// Retrieves the next task from the queue
Task *nextTask() {
// The return value
Task *nt = 0;
// Lock the queue mutex
pthread_mutex_lock(&qmtx);
// Check if there's work
if (finished && tasks.size() == 0) {
// If not return null (0)
nt = 0;
} else {
// Not finished, but there are no tasks, so wait for
// wcond to be signalled
if (tasks.size()==0) {
pthread_cond_wait(&wcond, &qmtx);
}
// get the next task
nt = tasks.front();
if(nt){
tasks.pop();
}
// For debugging
if (nt) nt->showTask();
}
// Unlock the mutex and return
pthread_mutex_unlock(&qmtx);
return nt;
}
// Add a task
void addTask(Task *nt) {
// Only add the task if the queue isn't marked finished
if (!finished) {
// Lock the queue
pthread_mutex_lock(&qmtx);
// Add the task
tasks.push(nt);
// signal there's new work
pthread_cond_signal(&wcond);
// Unlock the mutex
pthread_mutex_unlock(&qmtx);
}
}
// Mark the queue finished
void finish() {
pthread_mutex_lock(&qmtx);
finished = true;
// Signal the condition variable in case any threads are waiting
pthread_cond_signal(&wcond);
pthread_mutex_unlock(&qmtx);
}
// Check if there's work
bool hasWork() {
//printf("task queue size is %d\n",tasks.size());
return (tasks.size()>0);
}
private:
std::queue<Task*> tasks;
bool finished;
pthread_mutex_t qmtx;
pthread_cond_t wcond;
};
// Function that retrieves a task from a queue, runs it and deletes it
void *getWork(void* param) {
Task *mw = 0;
WorkQueue *wq = (WorkQueue*)param;
while (mw = wq->nextTask()) {
mw->run();
delete mw;
}
pthread_exit(NULL);
}
class ThreadPool {
public:
// Allocate a thread pool and set them to work trying to get tasks
ThreadPool(int n) : _numThreads(n) {
int rc;
printf("Creating a thread pool with %d threads\n", n);
threads = new pthread_t[n];
for (int i=0; i< n; ++i) {
rc = pthread_create(&(threads[i]), 0, getWork, &workQueue);
if (rc){
printf("ERROR; return code from pthread_create() is %d\n", rc);
exit(-1);
}
}
}
// Wait for the threads to finish, then delete them
~ThreadPool() {
workQueue.finish();
//waitForCompletion();
for (int i=0; i<_numThreads; ++i) {
pthread_join(threads[i], 0);
}
delete [] threads;
}
// Add a task
void addTask(Task *nt) {
workQueue.addTask(nt);
}
// Tell the tasks to finish and return
void finish() {
workQueue.finish();
}
// Checks if there is work to do
bool hasWork() {
return workQueue.hasWork();
}
private:
pthread_t * threads;
int _numThreads;
WorkQueue workQueue;
};
// stdout is a shared resource, so protected it with a mutex
static pthread_mutex_t console_mutex = PTHREAD_MUTEX_INITIALIZER;
// Debugging function
void showTask(int n) {
pthread_mutex_lock(&console_mutex);
pthread_mutex_unlock(&console_mutex);
}
// Task to compute fibonacci numbers
// It's more efficient to use an iterative algorithm, but
// the recursive algorithm takes longer and is more interesting
// than sleeping for X seconds to show parrallelism
class FibTask : public Task {
public:
FibTask(int n) : Task(), _n(n) {}
~FibTask() {
// Debug prints
pthread_mutex_lock(&console_mutex);
printf("tid(%d) - fibd(%d) being deleted\n", pthread_self(), _n);
pthread_mutex_unlock(&console_mutex);
}
virtual void run() {
// Note: it's important that this isn't contained in the console mutex lock
long long val = innerFib(_n);
// Show results
pthread_mutex_lock(&console_mutex);
printf("Fibd %d = %lld\n",_n, val);
pthread_mutex_unlock(&console_mutex);
// The following won't work in parrallel:
// pthread_mutex_lock(&console_mutex);
// printf("Fibd %d = %lld\n",_n, innerFib(_n));
// pthread_mutex_unlock(&console_mutex);
}
virtual void showTask() {
// More debug printing
pthread_mutex_lock(&console_mutex);
printf("thread %d computing fibonacci %d\n", pthread_self(), _n);
pthread_mutex_unlock(&console_mutex);
}
private:
// Slow computation of fibonacci sequence
// To make things interesting, and perhaps imporove load balancing, these
// inner computations could be added to the task queue
// Ideally set a lower limit on when that's done
// (i.e. don't create a task for fib(2)) because thread overhead makes it
// not worth it
long long innerFib(long long n) {
if (n<=1) { return 1; }
return innerFib(n-1) + innerFib(n-2);
}
long long _n;
};
int main(int argc, char *argv[])
{
// Create a thread pool
ThreadPool *tp = new ThreadPool(10);
// Create work for it
/*for (int i=0;i<100; ++i) {
int rv = rand() % 40 + 1;
showTask(rv);
tp->addTask(new FibTask(rv));
}*/
delete tp;
printf("\n\n\n\n\nDone with all work!\n");
}
The design is more or less OK-ish but implementationwise it contains several things that are a bit overcomplicated and may introduce instabilities. I guess you prog deadlocks when you comment out the for loop because you should use pthread_cond_broadcast instead of pthread_cond_signal in your WorkQueue::finish() method.
Note: I usually implemented threadpool termination by placing NUM_THREADS number of NULL items into the workqueue and I set a finished flag only to be able to check something in my addTask() method because after finish() I usually don't let adding new tasks and I return with false from addTask() or sometimes I assert.
Another note: Its best to encapsulate threads into classes, that has several benefits and makes proting to multiple platforms easier.
There may be other bugs too as I haven't executed your program, just ran through your code.
EDIT: Here is a reworked version, I issued some modifications to your code but I don't guarantee that it works. Fingers crossed... :-)
#include <stdio.h>
#include <queue>
#include <unistd.h>
#include <pthread.h>
#include <malloc.h>
#include <stdlib.h>
#include <assert.h>
// Reusable thread class
class Thread
{
public:
Thread()
{
state = EState_None;
handle = 0;
}
virtual ~Thread()
{
assert(state != EState_Started);
}
void start()
{
assert(state == EState_None);
// in case of thread create error I usually FatalExit...
if (pthread_create(&handle, NULL, threadProc, this))
abort();
state = EState_Started;
}
void join()
{
// A started thread must be joined exactly once!
// This requirement could be eliminated with an alternative implementation but it isn't needed.
assert(state == EState_Started);
pthread_join(handle, NULL);
state = EState_Joined;
}
protected:
virtual void run() = 0;
private:
static void* threadProc(void* param)
{
Thread* thread = reinterpret_cast<Thread*>(param);
thread->run();
return NULL;
}
private:
enum EState
{
EState_None,
EState_Started,
EState_Joined
};
EState state;
pthread_t handle;
};
// Base task for Tasks
// run() should be overloaded and expensive calculations done there
// showTask() is for debugging and can be deleted if not used
class Task {
public:
Task() {}
virtual ~Task() {}
virtual void run()=0;
virtual void showTask()=0;
};
// Wrapper around std::queue with some mutex protection
class WorkQueue
{
public:
WorkQueue() {
pthread_mutex_init(&qmtx,0);
// wcond is a condition variable that's signaled
// when new work arrives
pthread_cond_init(&wcond, 0);
}
~WorkQueue() {
// Cleanup pthreads
pthread_mutex_destroy(&qmtx);
pthread_cond_destroy(&wcond);
}
// Retrieves the next task from the queue
Task *nextTask() {
// The return value
Task *nt = 0;
// Lock the queue mutex
pthread_mutex_lock(&qmtx);
while (tasks.empty())
pthread_cond_wait(&wcond, &qmtx);
nt = tasks.front();
tasks.pop();
// Unlock the mutex and return
pthread_mutex_unlock(&qmtx);
return nt;
}
// Add a task
void addTask(Task *nt) {
// Lock the queue
pthread_mutex_lock(&qmtx);
// Add the task
tasks.push(nt);
// signal there's new work
pthread_cond_signal(&wcond);
// Unlock the mutex
pthread_mutex_unlock(&qmtx);
}
private:
std::queue<Task*> tasks;
pthread_mutex_t qmtx;
pthread_cond_t wcond;
};
// Thanks to the reusable thread class implementing threads is
// simple and free of pthread api usage.
class PoolWorkerThread : public Thread
{
public:
PoolWorkerThread(WorkQueue& _work_queue) : work_queue(_work_queue) {}
protected:
virtual void run()
{
while (Task* task = work_queue.nextTask())
task->run();
}
private:
WorkQueue& work_queue;
};
class ThreadPool {
public:
// Allocate a thread pool and set them to work trying to get tasks
ThreadPool(int n) {
printf("Creating a thread pool with %d threads\n", n);
for (int i=0; i<n; ++i)
{
threads.push_back(new PoolWorkerThread(workQueue));
threads.back()->start();
}
}
// Wait for the threads to finish, then delete them
~ThreadPool() {
finish();
}
// Add a task
void addTask(Task *nt) {
workQueue.addTask(nt);
}
// Asking the threads to finish, waiting for the task
// queue to be consumed and then returning.
void finish() {
for (size_t i=0,e=threads.size(); i<e; ++i)
workQueue.addTask(NULL);
for (size_t i=0,e=threads.size(); i<e; ++i)
{
threads[i]->join();
delete threads[i];
}
threads.clear();
}
private:
std::vector<PoolWorkerThread*> threads;
WorkQueue workQueue;
};
// stdout is a shared resource, so protected it with a mutex
static pthread_mutex_t console_mutex = PTHREAD_MUTEX_INITIALIZER;
// Debugging function
void showTask(int n) {
pthread_mutex_lock(&console_mutex);
pthread_mutex_unlock(&console_mutex);
}
// Task to compute fibonacci numbers
// It's more efficient to use an iterative algorithm, but
// the recursive algorithm takes longer and is more interesting
// than sleeping for X seconds to show parrallelism
class FibTask : public Task {
public:
FibTask(int n) : Task(), _n(n) {}
~FibTask() {
// Debug prints
pthread_mutex_lock(&console_mutex);
printf("tid(%d) - fibd(%d) being deleted\n", (int)pthread_self(), (int)_n);
pthread_mutex_unlock(&console_mutex);
}
virtual void run() {
// Note: it's important that this isn't contained in the console mutex lock
long long val = innerFib(_n);
// Show results
pthread_mutex_lock(&console_mutex);
printf("Fibd %d = %lld\n",(int)_n, val);
pthread_mutex_unlock(&console_mutex);
// The following won't work in parrallel:
// pthread_mutex_lock(&console_mutex);
// printf("Fibd %d = %lld\n",_n, innerFib(_n));
// pthread_mutex_unlock(&console_mutex);
// this thread pool implementation doesn't delete
// the tasks so we perform the cleanup here
delete this;
}
virtual void showTask() {
// More debug printing
pthread_mutex_lock(&console_mutex);
printf("thread %d computing fibonacci %d\n", (int)pthread_self(), (int)_n);
pthread_mutex_unlock(&console_mutex);
}
private:
// Slow computation of fibonacci sequence
// To make things interesting, and perhaps imporove load balancing, these
// inner computations could be added to the task queue
// Ideally set a lower limit on when that's done
// (i.e. don't create a task for fib(2)) because thread overhead makes it
// not worth it
long long innerFib(long long n) {
if (n<=1) { return 1; }
return innerFib(n-1) + innerFib(n-2);
}
long long _n;
};
int main(int argc, char *argv[])
{
// Create a thread pool
ThreadPool *tp = new ThreadPool(10);
// Create work for it
for (int i=0;i<100; ++i) {
int rv = rand() % 40 + 1;
showTask(rv);
tp->addTask(new FibTask(rv));
}
delete tp;
printf("\n\n\n\n\nDone with all work!\n");
}
I think you are having a race condition there...
When you remove the for loop, the pool is destructed as soon as it gets created so there is no time for the threads to start waiting on the queue. Try putting a sleep there and you'll see.
I implemented a threadpool library, which is used widely among all our services, so here come some advices:
You are using C++, so there's no need to use pthreads, just use boost, or std:thread if available
Don't signal, push empty tasks instead (pushing a task requires to signal, of course)
Use boost::function or std:function instead of a base class
Cope with spurious wake-ups (you code doesn't seem to handle them)
pthread_cond_signal wakes-up only one thread, you must use pthread_cond_broadcast if you want to notify them all, that said, I'd recommend, again, to stick to boost's conditions (#pasztorpisti got it rigth here, he's got my upvote)