So I have a main application thread in my opengl application with all rendering and stuff. Now i need some really heavy calculation which takes about 2 3 seconds so I moved it to a seperate thread here is how I manage it:
std::atomic<bool> working = false;
void work(){
if(!working)
{
working = true;
std::thread worker(do_work);
worker.detach();
}
else
{
// Some Updations
}
}
void do_work()
{
// The actual work
// working = false;
}
Now i call work every frame and the actual work gets dispatched automatically once the previous one has finished.
Now my question is what will be some ways to optimize this?
Some ideas that come to my mind are have a thread pool but I am not sure if that is worth the trouble implementing? Or is there any other way?
You could use std::launch as some people have suggested. Or you could do a google for "c++ thread pool library" and probably find something waiting for you.
But the reality is simple: writing a thread pool is close to trivial and is a good exercise. So you could write your own and learn something. As has been suggested, you can dispatch via some sort of message queue.
A work queue would be any sort of mutex & cond_var - managed FIFO, and then you can have multiple readers and multiple writers. The entries in the queue can be any sort of runnable or bound function (such as a lambda).
Fun to write and you can toss into your person library for years to come.
Related
I'm going to start by saying I have minimal experience with the C++ STL and paralleled processing. Still doing my research...
My application has a queue that tends to get large. I use asychronous future's to handle these tasks a la carte (for a lack of better terms). The maximum tasks created are based on the number of available cores to the machine.
I store the future in a class member vector to prevent the task being bound to the scope of the method in which it is called from. Except, now I have the problem of dealing with the results after the task is completed. Here is a sample of my code to provide context to my question:
if ( ALI::WorkingTasks < CPU_HW_CONCURRENCY ) {
std::string Task = TaskQueue.front();
TaskQueue.pop();
ALI::WorkingTasks++;
ALI::AsyncTasks.push_back(std::async(std::launch::async, &ALI::ProcessCodecUI, this, Task));
}
The method that is called from std::async
bool ALI::ProcessCodecUI(std::string UIPath)
{
// long inefficient process here
ALI::WorkingTasks--;
// notify condition_variable to create more tasks here
}
In my class definitions, this is how ALI::AsyncTasks is defined.
private:
std::vector<std::future<bool>> AsyncTasks;
This is my initial implementation to get the application working at the very minimum - it works. I've done some reading on threadpools and have poked at the idea of creating my own implementation of an "a la carte" threadpool.
So my question is: How do I handle the results of the ALI::AsyncTasks? Every example I have seen deals with the future directly in the method that calls it. In my scenario, the vector keeps building up and the future never gets destroyed even after the task is completed - this creates a memory leak. I don't have anyway to self-destroy the future after ProcessCodeUI() is completed.
If I am not clear, please let me know and I will revise.
Thank you
You should not use a future if you don't have a specific point in the program where you want to block while waiting for the asynchronous operation to complete.
You could achieve what you are trying to with minimal changes by using std::thread instead of std::async then detaching the thread immediately.
if ( ALI::WorkingTasks < CPU_HW_CONCURRENCY ) {
std::string Task = TaskQueue.front();
TaskQueue.pop();
ALI::WorkingTasks++;
std::thread Thread (&ALI::ProcessCodecUI, this, Task);
Thread.detach ();
}
Then, there is nothing left to clean up when the thread terminates. Though it would be more efficient to spawn a thread per core that pulls from a common queue.
I am recently working with threads in C++11. now I am thinking about how to force stop a thread. I couldn't find it on stackoverflow, and also tried these.
One variable each thread : not so reliable
return in the main thread : I have to force quit only one not all
and I have no more ideas. I have heard about WinAPI, but I want a portable solution. (that also means I wont use fork())
Can you please give me a solution of this? I really want to do it.
One of the biggest problems with force closing a thread in C++ is the RAII violation.
When a function (and subsequently, a thread), gracefully finishes, everything it held is gracefully cleaned up by the destructors of the objects the functions/threads created.
Memory gets freed,
OS resources (handles, file descriptors etc.) are closed and returned to the OS
Locks are getting unlocked so other threads can use the shared resources they protect.
other important tasks are preformed (such as updating counters, logging, etc.).
If you brutally kill a thread (aka by TerminateThread on Windows, for example), non of these actually happen, and the program is left in a very dangerous state.
A (not-so) common pattern that can be used is to register a "cancellation token" on which you can monitor and gracefully shut the thread if other thread asks so (a la TPL/PPL). something like
auto cancellationToken = std::make_shared<std::atomic_bool>();
cancellationToken->store(false);
class ThreadTerminator : public std::exception{/*...*/};
std::thread thread([cancellationToken]{
try{
//... do things
if (cancellationToken->load()){
//somone asked the thred to close
throw ThreadTerminator ();
}
//do other things...
if (cancellationToken->load()){
//somone asked the thred to close
throw ThreadTerminator ();
}
//...
}catch(ThreadTerminator){
return;
}
});
Usually, one doesn't even open a new thread for a small task, it's better to think of a multi threaded application as a collection of concurrent tasks and parallel algorithms. one opens a new thread for some long ongoing background task which is usually performed in some sort of a loop (such as, accepting incoming connections).
So, anyway, the cases for asking a small task to be cancelled are rare anyway.
tldr:
Is there a reliable way to force a thread to stop in C++?
No.
Here is my approach for most of my designs:
Think of 2 kinds of Threads:
1) primary - I call main.
2) subsequent - any thread launched by main or any subsequent thread
When I launch std::thread's in C++ (or posix threads in C++):
a) I provide all subsequent threads access to a boolean "done", initialized to false. This bool can be directly passed from main (or indirectly through other mechanisms).
b) All my threads have a regular 'heartbeat', typically with a posix semaphore or std::mutex, sometimes with just a timer, and sometimes simply during normal thread operation.
Note that a 'heartbeat' is not polling.
Also note that checking a boolean is really cheap.
Thus, whenever main wants to shut down, it merely sets done to true and 'join's with the subsequent threads.
On occasion main will also signal any semaphore (prior to join) that a subsequent thread might be waiting on.
And sometimes, a subsequent thread has to let its own subsequent thread know it is time to end.
Here is an example -
main launching a subsequent thread:
std::thread* thrd =
new std::thread(&MyClass_t::threadStart, this, id);
assert(nullptr != thrd);
Note that I pass the this pointer to this launch ... within this class instance is a boolean m_done.
Main Commanding shutdown:
In main thread, of course, all I do is
m_done = true;
In a subsequent thread (and in this design, all are using the same critical section):
void threadStart(uint id) {
std::cout << id << " " << std::flush; // thread announce
do {
doOnce(id); // the critical section is in this method
}while(!m_done); // exit when done
}
And finally, at an outer scope, main invokes the join.
Perhaps the take away is - when designing a threaded system, you should also design the system shut down, not just add it on.
I am making some multi-threaded video game code. Before I began coding I looked at an article describing vaguely Valve's solution to multi-threaded game design. A key concept I gleaned from the article is thread synchronization. I don't know if this is how Valve does it but I imagined multiple threads each executing a game loop. At the end of each iteration, the threads pause and wait for other threads to finish their current iteration, then synchronize shared data. I figure that besides the overhead is this management scheme, there would be no different to just let the threads operate completely asynchronously. The article mentioned a thread used exclusively for syncing but I am trying to get a different solution to work correctly. This is how I (try) to do it:
// at end of loop on each thread...
sig_thread_done();
while (!is_sync_done())
{
PauseExecution(1);
}
sig_thread_done and is_sync_done are function objects from another class that controls a list of all "threads". These functions look like this:
bool Core::IsFrameDone()
{
MutexLock lock(manager_mutex);
if (waiting_components == -1)
{
waiting_components = 0;
return true;
}
return false;
}
void Core::SignalFrameDone()
{
MutexLock lock(manager_mutex);
if (++waiting_components == (int)components.size()) // components == threads
{
//sync shared data...
waiting_components = -1; // -1 signifies that all threads have completed their iteration
}
}
The problem is that a fast thread can exit its waiting loop and come back around to it again before other threads have a chance to exit there's. So the other threads miss the exit through is_sync_done returning false before another thread begins waiting and the whole system gets stuck waiting forever.
I can't find an easy way to resolve this issue. I really like this approach because synchronization doesn't get stalled while some independent thread performs the sync.
I appreciate any insight or suggestions anyone has to offer.
Link to article.
I think you are trying to re-invent a Thread barrier.
For something like this you want to sync on a barrier, with something like a Win32 Event (or an array thereof), this makes sure you cannot get the situation you described (the barrier ensures that everything syncs up to the same frame) while at the same time freeing CPU time, as waiting on events is done as a kernel signal, and sleeps the thread till that signal is received. You'd also what to use wait-free algorithms in there, these work particularly well if you have a job/task based threading model, where certain things can be decoupled from the system.
Also, here is a better publication on multi-threading the source engine, its far more in depth and technical (they also specifically state that they avoid mutexes for this sort of thing).
I am not sure how to put this question in this forum any way i am asking and hopefully get some inputs.
I am writing a thread pool for my project. I have following design.
I am maintaining vector of threads std::vector<ThreadWrapper <threadFuncParam>* > m_vecThreads;
and pushing the threds in to list m_vecThreads.push_back(pThreadWrapper);
When new request comes i am taking the thread pool as below
if(!m_vecThreads.empty() )
{
ThreadWrapper <threadFuncParam>* pWrapper = m_vecThreads.back();
m_vecThreads.pop_back();
//... Awake threadd
}
When thread job is done it is pushed back in to pool of thread.
Now while gracefull shutdown i have stop the threads gracefully now with the design above i am facing problem how can i stop threads as in vector container i am poping from vector when request is serviced, so i lost the pointer till service is completed.
Is there better i can do this or handle this scenario like map or other container which is supported by standard C++?
Another question is
During shutdown i have a scenario threads are doing process here in my case reading from database which may take time so i cannot wait till it is complete
and i want to send reply to clients for pending requests which threads are processing and i am about to kill that value is bad.
Thanks!
If you still need access to what you pass out from your pool, then you should store the items in a "used" container.
However, at that moment, you are sharing your pointers, so you should use shared_ptr and pass out weak_ptr, so the threads can also be deleted and the users don't have a dangling pointer
The best cointainer for the used items would be a set, so the returned thread can be found and removed easily.
To solve your first problem, push it on to another vector, say m_vecBusyThreads, and when it's done, take it off there (note, you'll have to have some mechanism to search for the finished thread).
For your second problem, cleanest solution is to join each thread till it has "shutdown", any other approach could end up with some undesired side effects (esp. for example if it's connecting to a db etc.) Now that you have the busy container, iterate through tell each to shutdown, then iterate through each of your free containers, shutting down and joining each thread. Then go back to the busy container and attempt to join each thread. This may give a little time to the busy threads to shutdown cleanly.
boost::threads supports this concept of interrupt points, and the idea is that you can interrupt a thread at any of these points, however some calls are not interruptible (typically blocking calls), you need to find the best way to stop each type (socket read for example may be to send a dummy packet etc.)
I have done it in C, so the solution is not "C++"ish, but I was using two arrays: one containing the threads, and the other containing a representation of used / unused (~boolean).
I would be something like:
pthread_t[INITIAL_SIZE] thread_pool;
boolean[INITIAL_SIZE] threads_availability;
int first_available = 0;
pthread_t * get_thread() {
int ind = 0;
if (first_available<=INITIAL_SIZE) {
ind = first_available;
// find the next available spot
for (first_available; first_available < INITIAL_SIZE && threads_availability[first_available]; first_available++);
threads_availability[ind] = 0;
return thread_pool[ind];
}
}
void put_thread(pthread_t* thethread)
{
int i = 0;
pthread_t *it = thread_pool;
while (!pthread_equals(it, thethread)) {
it++;
i++;
}
thread_availability[i] = 1;
}
please keep in mind that this is pseudo code, and this is not optimal.
But this is an idea.
This is not a direct answer to your problem as other people already answered your original question.
I just wanted to say that you could look into boost::asio and/or boost::thread.
I would probably go for boost::asio because it has everything you need to do asynchronous operations based on timers and whatnot. You could use shared_ptr and boost::enable_shared_from_this in order to let your "jobs" go and be destroyed automatically when they finish their job.
Example:
boost::shared_ptr<async_job> aj( new async_job(
io_, boost::bind(&my_job::handle_completion, shared_from_this(), _1, _2)));
This code would execute your custom async_job on a thread pool (io_ is boost::asio::io_service). Your 'my_job' instance will be automatically destroyed when the async_job finishes and invokes handle_completion on it. Or you can let it live if you take shared_from_this() again inside handle_completion.
HTH,
Alex
I would like to do something like the below for a multi-threaded program:
// wait for variable to become true but don't hog resources
// then re-sync queues
Is something like this a good solution?
while (!ready) {
Thread.Sleep(250); // pause for 1/4 second;
};
No, this is not a good solution. First it might sleep too long. Second it's easy for threads to get into lockstep. Here's couple of links to MSDN articles on proper synchronization techniques:
Conditional variables
Events
Here's how you do it using boost:
boost::condition_variable condvar;
boost::mutex mutex;
bool finished1 = false;
bool finished2 = false;
void longComputation1()
{
{
boost::lock_guard<boost::mutex> lock(mutex);
finished1 = false;
}
// Perform long computation
{
boost::lock_guard<boost::mutex> lock(mutex);
finished1 = true;
}
condvar.notify_one();
}
void longComputation2()
{
{
boost::lock_guard<boost::mutex> lock(mutex);
finished2 = false;
}
// Perform long computation
{
boost::lock_guard<boost::mutex> lock(mutex);
finished2 = true;
}
condvar.notify_one();
}
void somefunction()
{
// Wait for long computations to finish without "spinning"
boost::lock_guard<boost::mutex> lock(mutex);
while(!finished1 && !finished2)
{
condvar.wait(lock);
}
// Computations are finished
}
For the sake of brevity, I didn't include the thread spawning code.
The boost::lock_guard uses the RAII idiom to automatically unlock the mutex when the lock object goes out of scope. Very useful for preventing deadlocks in case of exceptions.
I find condition variables less error prone than Microsoft's Event objects. If you use boost.Thread, you'll have the added benefit of cross-platform potability.
Try to use Event (kernel object) instead of simple variable and replace your loop by:
WaitForSingleObject(hEventHandle, INFINITE);
The code above will work, and maybe appropriate in some circumstances.
You could also look at a critical section or semaphore - this will make your application block and wait until the resource becomes available,
Your thread that does the work grabs the mutex, does some work, meanwhile, the main method also tries to grab the same mutex, but can't. when the worker thread(s) exit, they release the mutex and your main thread can pass the critical section and continue.
First of all, you need to declare your 'ready' variable at least 'volatile' or this could have nasty side effects. Secondly, sleeping that long vefore reevaluating the condition is only a good idea if the duration it might take is indeed very long, let's say a few minutes.
Using the WinAPI's Event functions (CreateEvent, SetEvent(), WaitForSingleEvent()) is the best way to do it. Of course it introduces some overhead, but usually it's fine.
If you want to stick with your solution, looping and rechecking the condition a few times before you sleep again could improve performance in some scenarios.
The raw Win32 API has EVENT for doing this, here's a usage example:
http://msdn.microsoft.com/en-us/library/ms686915(VS.85).aspx
However, that API is C-oriented and particular to Windows. If writing a C++ program you might consider making your code more platform independent by using something like boost::threads, which has an analogue in Conditions.
A caveat I've found is that Windows can WaitForMultipleObjects, thus waiting on several events (and other handle classes) at a time. boost has no parallel AFAIK.
On top of good answers already provided - you will waste half the sleep time, assuming a random distribution of the occurrence you wish to detect. 125ms is an eternity in computer time.
WaitForSingleObject on a Win32 Event handle allows you to detect the required signal pseudo-immediately (depending on what other threads in your process are doing), and not do redundant checks (how many needless loops do you have to execute before the signal arrives?), provided the setting thread call SetEvent once it's done with its work. The bool is then redundant, which is as it should be.
Granted this is C#, but I've found this book to be extremely helpful for doing multi-threading development.
http://www.albahari.com/threading/
Some of the info is not language specific.