I'm going to start by saying I have minimal experience with the C++ STL and paralleled processing. Still doing my research...
My application has a queue that tends to get large. I use asychronous future's to handle these tasks a la carte (for a lack of better terms). The maximum tasks created are based on the number of available cores to the machine.
I store the future in a class member vector to prevent the task being bound to the scope of the method in which it is called from. Except, now I have the problem of dealing with the results after the task is completed. Here is a sample of my code to provide context to my question:
if ( ALI::WorkingTasks < CPU_HW_CONCURRENCY ) {
std::string Task = TaskQueue.front();
TaskQueue.pop();
ALI::WorkingTasks++;
ALI::AsyncTasks.push_back(std::async(std::launch::async, &ALI::ProcessCodecUI, this, Task));
}
The method that is called from std::async
bool ALI::ProcessCodecUI(std::string UIPath)
{
// long inefficient process here
ALI::WorkingTasks--;
// notify condition_variable to create more tasks here
}
In my class definitions, this is how ALI::AsyncTasks is defined.
private:
std::vector<std::future<bool>> AsyncTasks;
This is my initial implementation to get the application working at the very minimum - it works. I've done some reading on threadpools and have poked at the idea of creating my own implementation of an "a la carte" threadpool.
So my question is: How do I handle the results of the ALI::AsyncTasks? Every example I have seen deals with the future directly in the method that calls it. In my scenario, the vector keeps building up and the future never gets destroyed even after the task is completed - this creates a memory leak. I don't have anyway to self-destroy the future after ProcessCodeUI() is completed.
If I am not clear, please let me know and I will revise.
Thank you
You should not use a future if you don't have a specific point in the program where you want to block while waiting for the asynchronous operation to complete.
You could achieve what you are trying to with minimal changes by using std::thread instead of std::async then detaching the thread immediately.
if ( ALI::WorkingTasks < CPU_HW_CONCURRENCY ) {
std::string Task = TaskQueue.front();
TaskQueue.pop();
ALI::WorkingTasks++;
std::thread Thread (&ALI::ProcessCodecUI, this, Task);
Thread.detach ();
}
Then, there is nothing left to clean up when the thread terminates. Though it would be more efficient to spawn a thread per core that pulls from a common queue.
Related
I'm trying to create a clean and efficient design for passing off events to a background thread for evaluation, then return a selected result to the game thread.
This is my initial design
//Occurrence object passed from director on game thread to background thread OccurrenceQueue
//Execute BackgroundThread::EvaluateQueues()
//If OccurrenceQueue.Dequeue()
//Score Occurrence, then return via Occurrence.GetOwner()->PurposeSelected(Occurrence)
//EvaluateQueues()
This resulted in a clean loop of selecting a chain of purposes from an event. So now I want to move this to a background thread. Here is what I've learned so far:
Thread Safety (in UE) requires absolutely no modification of UObject
data from other threads (From what I read this is due to their custom
GC)
You can lock objects and/or design so that objects in background
thread aren't touched by game thread, but there is still a risk of
unexpected behavior due to lifetime not being extended by background
thread and synchonization issues
You cannot simply execute a function on a game thread existing object
to move the callstack back to game thread
Calling Occurrence.GetOwner()->PurposeSelected(Occurrence) from a
background thread remains in the background thread
This is the main subject I'd like to get a better understanding of
This applies to delegates in UE as well
TQueues in UE can be used across threads safely
From what I've learned above, my current design doesn't appear to be logically possible.
These are my alternatives thus far:
Use two queues
One to dequeue and score on the background thread
The other to dequeue the result on game thread via tick
Use a delegate existing on the game thread, which calls OccurrenceEvaluated.Broadcast() through tick
When a result is scored, bind Occurrence.GetOwner()->PurposeSelected(Occurrence) to OccurrenceEvaluated
I've seen that c++ utilizes something called Future() (or something like that) for ASync tasks, and it appears UE has something similar with TFuture<> && TFuture.IsReady(), but I have yet to look deeper into that and how it returns data
Same with FAsyncTask
I'm hesitant to implement any design which utilizes tick to check if data has been updated/returned from background threads.
Can anyone suggest relevant design practices, or clarify the nature of returning execution to a main thread from a background thread (I've had a hard time finding the right question to research/info regarding this)?
I found a perfect solution. As these events aren't particularly time sensitive, I just use Unreal Engine's AsyncTask() to schedule an async task on the game thread from my background thread. As #Pepjin Kramer pointed out is the same as std::async.
So simple it's basically a slap in the face.
Goodmorning,
Threading can help you get things done faster/more responsive but has a lot of pitfalls. The short answer in situations like this I use these kind of constructs
#include <future>
#include <thread>
#include <vector>
int main()
{
// make data shared
auto data = std::make_shared<std::vector<int>>();
data -> push_back(42);
// future will be a future<int> deduced from return from vector
// async will run lambda in other thread
// capture data by value! It copies the shared_ptr and will
// increase its reference count.
auto future = std::async(std::launch::async, [data]
{
auto value = data[0];
return value;
// shared_ptr data goes out of scope and reference count to data is decreased
});
// synchronize with thread and get "calculated" value
auto my_value = future.get();
// now data on the main thread goes out of scope reference count is decreased. Only when both the thread are done AND this function exits then data is deleted.
return 0;
}
Most objects take up memory and during assignments that memory isn't updated in 1 clock cycle. You can protect memory with sdt::mutex and std::scoped_lock. Try looking for those.
You often need to synchronize information/processing between threads, look at std::condition_variable (but they have pitfalls https://www.modernescpp.com/index.php/c-core-guidelines-be-aware-of-the-traps-of-condition-variables)
C++ has good support classes, std::thread and std::async/std::future. Personally I like the solution with std::future because you can return values and exceptions
(!) from other threads when you call future.get()
Learn about lambdas and captures, they're vital for use with std::thread/std::async
Life cycle of objects! When sharing objects between threads you must be sure that one thread doesn't delete objects in use by other threads. Don't use raw pointers and/or new/delete when using threads. I personally often use std::make_shared/std::shared_ptr's to data/objects when sharing information between threads.
Another tricky thing is that sometimes you cannot be sure work in another thread has started after creating it. E.g. When std::async returns a thread is created but it isn't guaranteed to have really started (operating system scheduling etc...) . If you want to be really sure it has started then after the async call returns you will have to wait on a condition_variable you set at the start of the thread function.
I hope these remarks can get you started.
I am recently working with threads in C++11. now I am thinking about how to force stop a thread. I couldn't find it on stackoverflow, and also tried these.
One variable each thread : not so reliable
return in the main thread : I have to force quit only one not all
and I have no more ideas. I have heard about WinAPI, but I want a portable solution. (that also means I wont use fork())
Can you please give me a solution of this? I really want to do it.
One of the biggest problems with force closing a thread in C++ is the RAII violation.
When a function (and subsequently, a thread), gracefully finishes, everything it held is gracefully cleaned up by the destructors of the objects the functions/threads created.
Memory gets freed,
OS resources (handles, file descriptors etc.) are closed and returned to the OS
Locks are getting unlocked so other threads can use the shared resources they protect.
other important tasks are preformed (such as updating counters, logging, etc.).
If you brutally kill a thread (aka by TerminateThread on Windows, for example), non of these actually happen, and the program is left in a very dangerous state.
A (not-so) common pattern that can be used is to register a "cancellation token" on which you can monitor and gracefully shut the thread if other thread asks so (a la TPL/PPL). something like
auto cancellationToken = std::make_shared<std::atomic_bool>();
cancellationToken->store(false);
class ThreadTerminator : public std::exception{/*...*/};
std::thread thread([cancellationToken]{
try{
//... do things
if (cancellationToken->load()){
//somone asked the thred to close
throw ThreadTerminator ();
}
//do other things...
if (cancellationToken->load()){
//somone asked the thred to close
throw ThreadTerminator ();
}
//...
}catch(ThreadTerminator){
return;
}
});
Usually, one doesn't even open a new thread for a small task, it's better to think of a multi threaded application as a collection of concurrent tasks and parallel algorithms. one opens a new thread for some long ongoing background task which is usually performed in some sort of a loop (such as, accepting incoming connections).
So, anyway, the cases for asking a small task to be cancelled are rare anyway.
tldr:
Is there a reliable way to force a thread to stop in C++?
No.
Here is my approach for most of my designs:
Think of 2 kinds of Threads:
1) primary - I call main.
2) subsequent - any thread launched by main or any subsequent thread
When I launch std::thread's in C++ (or posix threads in C++):
a) I provide all subsequent threads access to a boolean "done", initialized to false. This bool can be directly passed from main (or indirectly through other mechanisms).
b) All my threads have a regular 'heartbeat', typically with a posix semaphore or std::mutex, sometimes with just a timer, and sometimes simply during normal thread operation.
Note that a 'heartbeat' is not polling.
Also note that checking a boolean is really cheap.
Thus, whenever main wants to shut down, it merely sets done to true and 'join's with the subsequent threads.
On occasion main will also signal any semaphore (prior to join) that a subsequent thread might be waiting on.
And sometimes, a subsequent thread has to let its own subsequent thread know it is time to end.
Here is an example -
main launching a subsequent thread:
std::thread* thrd =
new std::thread(&MyClass_t::threadStart, this, id);
assert(nullptr != thrd);
Note that I pass the this pointer to this launch ... within this class instance is a boolean m_done.
Main Commanding shutdown:
In main thread, of course, all I do is
m_done = true;
In a subsequent thread (and in this design, all are using the same critical section):
void threadStart(uint id) {
std::cout << id << " " << std::flush; // thread announce
do {
doOnce(id); // the critical section is in this method
}while(!m_done); // exit when done
}
And finally, at an outer scope, main invokes the join.
Perhaps the take away is - when designing a threaded system, you should also design the system shut down, not just add it on.
I'd like to run tasks (worker threads) of the same type, but not more than a certain number of tasks at a time. When a task finishes, its result is an input for a new task which, then, can be started.
Is there any good way to implement this with async/future paradigm in C++11?
At first glance, it looks straight forward, you just spawn multiple tasks with:
std::future<T> result = std::async(...);
and, then, run result.get() to get an async result of a task.
However, the problem here is that the future objects has to be stored in some sort of queue and be waited one by one. It is, though, possible to iterate over the future objects over and over again checking if any of them are ready, but it's not desired due to unnecessary CPU load.
Is it possible somehow to wait for any future from a given set to be ready and get its result?
The only option I can think of so far is an old-school approach without any async/future. Specifically, spawning multiple worker threads and at the end of each thread push its result into a mutex-protected queue notifying the waiting thread via a condition variable that the queue has been updated with more results.
Is there any other better solution with async/future possible?
Thread support in C++11 was just a first pass, and while std::future rocks, it does not support multiple waiting as yet.
You can fake it relatively inefficiently, however. You end up creating a helper thread for each std::future (ouch, very expensive), then gathering their "this future is ready" into a synchronized many-producer single-consumer message queue, then setting up a consumer task that dispatches the fact that a given std::future is ready.
The std::future in this system doesn't add much functionality, and having tasks that directly state that they are ready and sticks their result into the above queue would be more efficient. If you go this route, you could write wrapper that match the pattern of std::async or std::thread, and return a std::future like object that represents a queue message. This basically involves reimplementing a chunk of the the concurrency library.
If you want to stay with std::future, you could create shared_futures, and have each dependent task depend on the set of shared_futures: ie, do it without a central scheduler. This doesn't permit things like abort/shutdown messages, which I consider essential for a robust multi threaded task system.
Finally, you can wait for C++2x, or whenever the concurrency TS is folded into the standard, to solve the problem for you.
You could create all the futures of "generation 1", and give all those futures to your generation 2 tasks, who will then wait for their input themselves.
facebook's folly has collectAny/collectN/collectAll on futures, I haven't try it yet, but looks promising.
Given that the "Wating for multiple futures" title attracts folks with questions like "is there a wait all for a list of futures?". You can do that adequately by keeping track of the pending threads:
unsigned pending = 0;
for (size_t i = 0; i < N; ++i) {
++pending;
auto callPause =
[&pending, i, &each, &done]()->unsigned {
unsigned ret = each();
results[i] = ret;
if (!--pending)
// called in whatever thread happens to finish last
done(results);
return ret;
};
futures[i] = std::async(std::launch::async, each);
}
full example
It might be possible to use std::experimental::when_all with a spread operator
I am not sure how to put this question in this forum any way i am asking and hopefully get some inputs.
I am writing a thread pool for my project. I have following design.
I am maintaining vector of threads std::vector<ThreadWrapper <threadFuncParam>* > m_vecThreads;
and pushing the threds in to list m_vecThreads.push_back(pThreadWrapper);
When new request comes i am taking the thread pool as below
if(!m_vecThreads.empty() )
{
ThreadWrapper <threadFuncParam>* pWrapper = m_vecThreads.back();
m_vecThreads.pop_back();
//... Awake threadd
}
When thread job is done it is pushed back in to pool of thread.
Now while gracefull shutdown i have stop the threads gracefully now with the design above i am facing problem how can i stop threads as in vector container i am poping from vector when request is serviced, so i lost the pointer till service is completed.
Is there better i can do this or handle this scenario like map or other container which is supported by standard C++?
Another question is
During shutdown i have a scenario threads are doing process here in my case reading from database which may take time so i cannot wait till it is complete
and i want to send reply to clients for pending requests which threads are processing and i am about to kill that value is bad.
Thanks!
If you still need access to what you pass out from your pool, then you should store the items in a "used" container.
However, at that moment, you are sharing your pointers, so you should use shared_ptr and pass out weak_ptr, so the threads can also be deleted and the users don't have a dangling pointer
The best cointainer for the used items would be a set, so the returned thread can be found and removed easily.
To solve your first problem, push it on to another vector, say m_vecBusyThreads, and when it's done, take it off there (note, you'll have to have some mechanism to search for the finished thread).
For your second problem, cleanest solution is to join each thread till it has "shutdown", any other approach could end up with some undesired side effects (esp. for example if it's connecting to a db etc.) Now that you have the busy container, iterate through tell each to shutdown, then iterate through each of your free containers, shutting down and joining each thread. Then go back to the busy container and attempt to join each thread. This may give a little time to the busy threads to shutdown cleanly.
boost::threads supports this concept of interrupt points, and the idea is that you can interrupt a thread at any of these points, however some calls are not interruptible (typically blocking calls), you need to find the best way to stop each type (socket read for example may be to send a dummy packet etc.)
I have done it in C, so the solution is not "C++"ish, but I was using two arrays: one containing the threads, and the other containing a representation of used / unused (~boolean).
I would be something like:
pthread_t[INITIAL_SIZE] thread_pool;
boolean[INITIAL_SIZE] threads_availability;
int first_available = 0;
pthread_t * get_thread() {
int ind = 0;
if (first_available<=INITIAL_SIZE) {
ind = first_available;
// find the next available spot
for (first_available; first_available < INITIAL_SIZE && threads_availability[first_available]; first_available++);
threads_availability[ind] = 0;
return thread_pool[ind];
}
}
void put_thread(pthread_t* thethread)
{
int i = 0;
pthread_t *it = thread_pool;
while (!pthread_equals(it, thethread)) {
it++;
i++;
}
thread_availability[i] = 1;
}
please keep in mind that this is pseudo code, and this is not optimal.
But this is an idea.
This is not a direct answer to your problem as other people already answered your original question.
I just wanted to say that you could look into boost::asio and/or boost::thread.
I would probably go for boost::asio because it has everything you need to do asynchronous operations based on timers and whatnot. You could use shared_ptr and boost::enable_shared_from_this in order to let your "jobs" go and be destroyed automatically when they finish their job.
Example:
boost::shared_ptr<async_job> aj( new async_job(
io_, boost::bind(&my_job::handle_completion, shared_from_this(), _1, _2)));
This code would execute your custom async_job on a thread pool (io_ is boost::asio::io_service). Your 'my_job' instance will be automatically destroyed when the async_job finishes and invokes handle_completion on it. Or you can let it live if you take shared_from_this() again inside handle_completion.
HTH,
Alex
So I have a thread pool that allows dynamic resizing and uses task paradigm. I wonder - when people get such thing do they stop creating threads by hand at all and just use tasks all the time? So is it common to use only thread-pool\task-executor for threads creation inside of my class?
my thread pool is based on boost::asio::io_service and works with boost::packaged_task.it is header only, having boost 1.47.0 all you need for it to work are timer, my costume thread_group and the thread_pool class. It was quite fun to develop such small thing but now I stand behind a dilemma.
my task constructions look like:
boost::shared_ptr< boost::packaged_task<int> > task(new boost::packaged_task<int>( boost::bind(calculate_the_answer_to_life_the_universe_and_everything, argument_int_value )));
this is quite over head in case when I want create a function that would newer return anything, would have in it some run again timer (for example files indexer that needs to check every 5 seconds if user has created any new file in some folder)
so for example I would have:
void infinite_thread()
{
while(true)
{
timerForCaptureFame.restart();
do_stuff();
spendedTimeForCaptureFame = (int64_t)timerForCaptureFame.elapsed();
if (spendedTimeForCaptureFame < desiredTimeForCaptureFame)
boost::this_thread::sleep(boost::posix_time::milliseconds(desiredTimeForCaptureFame - spendedTimeForCaptureFame));
}
}
and I would simply create this wraper into new thread with code like
boost::thread workerThread(infinite_thread);
But now I can have tasks so it could turn into
boost::shared_ptr< boost::packaged_task<void> > task(new boost::packaged_task<void>(infinite_thread));
task_manager->post<void>(task);
My task manager after some small amount of time would get that thread does not close itself and will generally add to itself new thread for execution keeping this one working.
So I really wonder if it is common practice having a thread_pool/task_pool to use only it (for example one per class) for threads creation or people mix there tasks with "pure" threads?
There is no clear answer. There might be things that seem better suited for regular threads and don't quite fit the task paradigm, for example threads that need to last for the whole duration of the program, or that might outlive the thread pool. If it is never going to be taken back to the pool, then you might as well handle it as a separate thing.
Then again, since you already have the thread pool, you might want to just force all threads to be tasks even if they are infinitely long tasks... but beware of the law of the instrument. It might seem that every job is a task/nail to your new pool/golden hammer.