Say I have a list of callable objects like
std::list<std::shared_ptr<Callable>> tasks;
and the task is to run them all in an infinite loop, say
void run_all(const bool& abort){
while(true){
for(const auto& ptr : tasks){
if (abort) return;
(*ptr)();
}
}
}
This is fine as long as every "task" finishes after short time. Now I'd like to add the requirement that whenever a task needs more time than a specific threshold, a new thread should be created so that the other tasks do not have do wait for a specific long running task.
The simplest solution regarding code complexity I can think of at the moment would be creating a thread for each task:
void run_all(const bool& abort){
auto job = [&](std::shared_ptr<Callable> task){
while (!abort){
(*task)();
}
};
std::list<std::thread> threads;
for(auto& ptr : tasks){
threads.emplace_back(job, ptr);
}
for(auto& t : threads){
t.join();
}
}
But this might create inappropriate many threads.
What is an appropriate way to implement running the tasks and create threads dynamically depending on how long a tasks needs to be finished? Say we have got some
std::chrono::duration threshold;
and the goal is to run the first task and continue with the next afterwards if the first one takes no longer than threshold until finish, but create a new thread to run the rest of the tasks in parallel, if the first task does not finish before threashold. The generalized goal is:
If there is no task that has been finished in some thread so that another task began to run during the certain period of time threshold, then a new thread should be created so that other tasks which may potentially run in very short time do not have to wait.
If there are more than 3 threads that finish at least one task per period threshold, one of them should be joined.
There may be tasks that itself run ad infinitum. This should have no effect on the other tasks.
What could be an appropriate implementation satisfying these requirements or at least doing something related or at least a concept of an implementation?
Or is it completely fine to just create a bunch of threads? (I think about running such an application on a low performance machine like Raspberry Pi and a set of 50 to 300 tasks that should be treated.)
Related
Currently I'm trying to make it possible to remove work queued through post or dispatch to an io_context. The work is queued by a small amount of queuer groups for which the work shall be removeable all at once:
boost::asio::io_context context;
auto work = [] {
// ...
};
boost::asio::post(context, std::move(work));
// ... now I want to remove the work
Is there such a functionality provided by the asio library?
Currently the application I'm working on, is using a thread pool which invokes io_context::run() from multiple threads.
My idea was that I could create multiple io_contexts that are dispatched by the thread pool such that one io_context represents a group that could be removed through io_context::stop(). All io_contexts would be held inside a single list which is then pooled for outstanding events.
However I believe that pooling or waiting for many io_contexts could lead to performance issues.
Is there a different solution?
No, there's no mechanism for removing posted jobs from an io_context. Alternatively, you could modify your jobs to check if a 'cancel flag' is set before they run (untested):
// create a cancellation flag
const auto cancel = std::make_shared<std::atomic<bool> >();
auto work = [=] {
// check to see if the flag has been set
// if so, return without performing our task
if(*cancel)
return;
// perform some task
};
// post our job
boost::asio::post(context, std::move(work));
...
// cancel all jobs checking this flag
*cancel = true;
In JProfiler, in the Call Tree and Hot Spots views of the CPU profiler, I have found that JProfiler is showing some methods as hot spots which aren't really hot spots. These methods are skewing my profiling work, as they are dominating these CPU views and making all other CPU consumers appear insignificant.
For example one thread is performing a Thread.sleep(300_000L) (sleeping for 5 minutes), and then doing some relatively minor work -- in a while(true) loop. JProfiler is configured to update the view every 5 seconds, and I have set the thread status selector to "Runnable". Every 5 seconds when JProfiler updates the view, I would expect the total self-time for the method to remain relatively small since the thread is sleeping and not in a runnable state, however instead I see the self time increment by about 5 seconds which would indicate (incorrectly) that the entire 5-second interval, the thread was in the runnable state. My concern is that the tool will be useless for my CPU profiling purposes if I cannot filter out the sleeping (Waiting) state.
With some testing, I have found that when the Thread.sleep() call eventually terminates, the self time drops down to near-zero again, and begins climbing again with the next invocation of Thread.sleep(). So to me it seems JProfiler is counting the method stats for the current invocation of Thread.sleep() as Runnable -- until the method actually terminates and then these stats are backed out of.
Is this a bug in JProfiler? Is there a way to get JProfiler to not count Thread.sleep() towards the Runnable state, even for long-running invocations of Thread.sleep()?
I am using a licensed version of JProfiler 8.1.4. I have also tried an evaluation version of JProfiler 10.1.
Update:
Here is a simple test case which exhibits this problem for me. I discovered that if I move the Thread.sleep() call to a separate method the problem goes away (see the in-line comments). This is not a great workaround because I'm profiling a large application and don't want to update all of the places where it calls Thread.sleep().
public class TestProfileSleep {
public static void main(String... args) {
new Thread(new Runnable() {
private void sleep(long millis) throws InterruptedException {
Thread.sleep(millis);
}
public void run() {
try {
while (true) {
Thread.sleep(60_000L); // profiling this is broken
//sleep(60_000L); // profiling this works
}
}
catch (InterruptedException ie) {
}
}
}).start();
}
}
I'm having a problem with my game freezing when adding a job to the thread pool. I've been going over my code but can't find the problem.
My thread pool is mostly standard and contains a list of jobs to perform. The worker threads fetch jobs from this list and perform them. Then they signal that they finished the job (this is so I can wait for all jobs to be finished (not just started/removed from the job list) without joining the threads (I want to use them next frame too)).
void ThreadPool::Add(std::function<void()> job) {
{
std::unique_lock<std::mutex> lock(mJobMutex);
mJobs.push(job);
++mUnfinishedJobs;
}
mJobCondition.notify_one();
}
void Worker::Execute() {
std::function<void()> job;
while (true) {
{
std::unique_lock<std::mutex> lock(mThreadPool.mJobMutex);
while (!mThreadPool.mStop && mThreadPool.mJobs.empty()) {
// Wait for new job to become available.
mThreadPool.mJobCondition.wait(lock);
}
if (mThreadPool.mStop)
return;
// Get next job.
job = mThreadPool.mJobs.front();
mThreadPool.mJobs.pop();
}
// Perform the job.
job();
// Signal that we finished the job.
{
std::unique_lock<std::mutex> lock(mThreadPool.mJobMutex);
--mThreadPool.mUnfinishedJobs;
}
mThreadPool.mFinishedCondition.notify_all();
}
}
Through some logging I managed to boil it down to mJobCondition.notify_one() in ThreadPool::Add. I placed some logging before and after that statement and it always hanged there. This is very odd to me. Sure, notify_one can miss the threads waiting for it, but if it does, it should just do nothing. It seems very odd to me that it would freeze on that line.
And if the problem is that I'm locking incorrectly and the thread pool and the worker thread are accessing memory at the same time shouldn't it crash and burn rather than freeze?
I'm on Windows using MinGW.
I also have a Wait and Stop method in the thread pool (which is what the mUnfinishedJobs variable is for) but I didn't include them since I know it's freezing when doing Add.
Here's the full threading code if you need more context.
I know I could probably use some threading library that does thread pools for me, but I want to learn how it's done.
I would like the user to input a time e.g. 1400h - which will then cause a function to run at 1400h.
How can I do this?
Context: I have a client-server program that works on the same computer - and I need several nodes to send messages simultaneously (which is the function as above)
edit: I do not want to use a sleep() function, ideally, as the issue is that the clients will be started at different times and it is much neater as a solution to call something that causes the function to execute at 1400h.
You can use std::this_thread::sleep_until, e.g.
void main()
{
auto fire_time = /**/;
std::thread thread([&]
{
std::this_thread::sleep_until(fire_time);
fire();
});
thread.join();
}
You can refactor that into a helper function, which is probably what you are looking for:
template<class Func, class Clock, class Duration>
void run_at(Func&& func, const std::chrono::time_point<Clock,Duration>& sleep_time)
{
std::thread(std::bind([&](const Func& func)
{
std::this_thread::sleep_until(sleep_time);
func();
}, std::move(func)))
.detach();
}
If the program is running the entire time, use a function such as sleep to wait the amount of time between now and 1400h. You might need to do this in a separate thread to allow the program to do other things, or replace the sleep with an event loop timeout (if the program is event-loop-based).
If the program must exit, then you must use a system facility, such as at on Unix, to arrange the program to be restarted and code to be executed at the specified time.
I believe you need a some kind of task manager. That's a basic model. Breeding sleeping threads is very wrong way to do that job. A single manager will know when to run a next task. How to run a task is another question. You can make new thread per task if you want them to be interactive. Or you can serialize them and run from within the manager thread.
I am using boost:asio with multiple io_services to keep different forms of blocking I/O separate. E.g. I have one io_service for blocking file I/O, and another for long-running CPU-bound tasks (and this could be extended to a third for blocking network I/O, etc.) Generally speaking I want to ensure that one form of blocking I/O cannot starve the others.
The problem I am having is that since tasks running in one io_service can post events to other io_service (e.g. a CPU-bound task may need to start a file I/O operation, or a completed file I/O operation may invoke a CPU-bound callback), I don't know how to keep both io_services running until they are both out of events.
Normally with a single I/O service, you do something like:
shared_ptr<asio::io_service> io_service (new asio::io_service);
shared_ptr<asio::io_service::work> work (
new asio::io_service::work(*io_service));
// Create worker thread(s) that call io_service->run()
io_service->post(/* some event */);
work.reset();
// Join worker thread(s)
However if I simply do this for both io_services, the one into which I did not post an initial event finishes immediately. And even if I post initial events to both, if the initial event on io_service B finishes before the task on io_service A posts a new event to B, io_service B will finish prematurely.
How can I keep io_service B running while io_service A is still processing events (because one of the queued events in service A might post a new event to B), and vice-versa, while still ensuring that both io_services exit their run() methods if they are ever both out of events at the same time?
Figured out a way to do this, so documenting it for the record in case anyone else finds this question in a search:
Create each N cross-communicating io_services, create a work object for each of them, and then start their worker threads.
Create a "master" io_service object which will not run any worker threads.
Do not allow posting events directly to the services. Instead, create accessor functions to the io_services which will:
Create a work object on the master thread.
Wrap the callback in a function that runs the real callback, then deletes the work.
Post this wrapped callback instead.
In the main flow of execution, once all of the N io_services have started and you have posted work to at least one of them, call run() on the master io_service.
When the master io_service's run() method returns, delete all of the initial work on the N cross-communicating io_services, and join all worker threads.
Having the master io_service's thread own work on each of the other io_services ensures that they will not terminate until the master io_service runs out of work. Having each of the other io_services own work on the master io_service for every posted callback ensure that the master io_service will not run out of work until every one of the other io_services no longer has any posted callbacks left to process.
An example (could be enapsulated in a class):
shared_ptr<boost::asio::io_service> master_io_service;
void RunWorker(boost::shared_ptr<boost::asio::io_service> io_service) {
io_service->run();
}
void RunCallbackAndDeleteWork(boost::function<void()> callback,
boost::asio::io_service::work* work) {
callback();
delete work;
}
// All new posted callbacks must come through here, rather than being posted
// directly to the io_service object.
void PostToService(boost::shared_ptr<boost::asio::io_service> io_service,
boost::function<void()> callback) {
io_service->post(boost::bind(
&RunCallbackAndDeleteWork, callback,
new boost::asio::io_service::work(*master_io_service)));
}
int main() {
vector<boost::shared_ptr<boost::asio::io_service> > io_services;
vector<boost::shared_ptr<boost::asio::io_service::work> > initial_work;
boost::thread_pool worker_threads;
master_io_service.reset(new boost::asio::io_service);
const int kNumServices = X;
const int kNumWorkersPerService = Y;
for (int i = 0; i < kNumServices; ++i) {
shared_ptr<boost::asio::io_service> io_service(new boost::asio::io_service);
io_services.push_back(io_service);
initial_work.push_back(new boost::asio::io_service::work(*io_service));
for (int j = 0; j < kNumWorkersPerService; ++j) {
worker_threads.create_thread(boost::bind(&RunWorker, io_service));
}
}
// Use PostToService to start initial task(s) on at least one of the services
master_io_service->run();
// At this point, there is no real work left in the services, only the work
// objects in the initial_work vector.
initial_work.clear();
worker_threads.join_all();
return 0;
}
The HTTP server example 2 does something similar that you may find useful. It uses the concept of an io_service pool that retains vectors of shared_ptr<boost::asio::io_service> and a shared_ptr<boost::asio::io_service::work> for each io_service. It uses a thread pool to run each service.
The example uses a round-robin scheduling for doling out work to the I/O services, I don't think that will apply in your case since you have specific tasks for io_service A and io_service B.