Moving from thread-based pipelining to task-based parallelism? (C++) - c++

Question

I'm looking at how to migrate some existing C++ code from thread-based to task-based parallelism, and whether that migration is desirable. Here's my scenario:
Suppose I have some function to execute on an event. Say I have a camera and each time a frame arrives I want to do some heavy processing and save the results. Some of the processing is serial, so if I just process each frame serially in the same thread, I don't get full CPU utilization. Say the frames arrive every 33ms and the processing latency for a frame is close to 100ms.
So in my current implementation I create say 3 threads that process frames and assign each new frame to one of these worker thread in a round-robin. So thread T0 might process frames F0, F3, F6, etc. Now I get full CPU utilization and I don't have to drop frames to maintain real-time rates.
Since the processing requires various big, temporary resources, I can allocate those up-front for each worker thread. So they do not have to be re-allocated for every frame. This strategy of per-thread resources works well for granularity: if they were being allocated per-frame this would take too long, but with many more worker threads we would run out of resources.
I do not see a way to replace this thread-based parallelism with task-based parallelism using standard C++11 or Microsoft's PPL library. If there is a pattern for doing so that could be sketched below, I would be very happy to learn it.
The question is where to store the state - the allocated temporary resources (e.g. GPU memory) - which can be re-used for subsequent frames but must not conflict with the resources for a currently processing frame.
Is it even desirable to migrate to task-based parallelism in this kind of case?

Answer 1

I figured this out. Here is an example solution:
#include <iostream>
#include <ppltasks.h>
#include <thread>
#include <vector>
using PipelineState = int;
using PipelineStateArg = std::shared_ptr<PipelineState>;
using FrameState = int;
struct Pipeline
{
PipelineStateArg state;
concurrency::task<void> task;
};
std::vector<Pipeline> pipelines;
void proc(const FrameState& fs, PipelineState& ps)
{
std::cout << "Process frame " << fs << " in pipeline " << ps << std::endl;
}
void on_frame(int index)
{
FrameState frame = index;
if (index < 2)
{
// Start a new pipeline
auto state = std::make_shared<PipelineState>(index);
pipelines.push_back({state, concurrency::create_task([=]()
{
proc(frame, *state);
})});
}
else
{
// Use an existing pipeline
auto& pipeline = pipelines[index & 1];
auto state = pipeline.state;
pipeline.task = pipeline.task.then([=]()
{
proc(frame, *state);
});
}
}
void main()
{
for (int i = 0; i < 100; ++i)
{
on_frame(i);
std::this_thread::sleep_for(std::chrono::milliseconds(33));
}
for (auto& pipeline : pipelines)
pipeline.tail.wait();
}