Submitting a task to a Thread Pool Implementation Issue - c++

I'm using thread pools for the first time, working out of Anthony Williams' Concurrency In Action book. He provides a basic thread pool class that I'm trying to use but not an implementation - just the class. The issue is that I'm new to templates and thread-pools. This is probably a simple question/answer but I can't figure it out. I have this function to submit jobs to the pool:
class SimpleThreadPool
{
public:
...
// -- Submit a task to the thread pool
template <typename FunctionType>
void submit(FunctionType f);
...
};
I'm trying to call it but honestly don't know how (have tried a few different things):
boost::interprocess::scoped_lock<std::mutex> lock(m_mtx_map);
for(const auto& elem : m_logger_map)
{
m_thread_pool->submit< std::function< void > >(elem.second->operator());
}
Here the m_thread_pool is a pointer to the thread pool and the map contains (string,pointer) pairs where the pointer points to a logging object which has it's main run/processing method in operator().
I'm getting a:
error: no matching function call 'SimpleThreadPool::submit(< unresolved overloaded function type >)'
How do I correctly use this submit function?
Thank you!
UPDATE -----
Here's the implementation in the thread pool class:
// -- Submit Method
/** void submit(FunctionType f)
* #desc : Submit a job (function pointer) to the pool
*/
template <typename FunctionType> void SimpleThreadPool::submit(FunctionType f)
{
m_q_work.push(std::function<void()>(f));
}
m_q_work is a thread-safe Queue.
// -- Launch a new worker thread
/** void m_worker_thread()
* #desc : Worker thread main method for getting tasks and executing
*/
void SimpleThreadPool::m_worker_thread()
{
while(!m_done)
{
std::function<void()> task;
if(m_q_work.try_pop(task))
{
task();
}
else
{
std::this_thread::yield();
}
}
}
I'm guessing I probably need to change the implementation based on my comment to the response below.

The following compiled in g++
#include <map>
#include <string>
#include <functional>
class SimpleThreadPool {
public:
template <typename FunctionType>
void submit(FunctionType f) { }
};
class Logger {
public:
void operator()() { }
};
int main(int argc,char **argv) {
std::map<std::string,Logger*> m_logger_map;
m_logger_map["one"]=new Logger();
SimpleThreadPool *m_thread_pool=new SimpleThreadPool();
for (const auto& elem: m_logger_map) {
m_thread_pool->submit<std::function< void() > >(*elem.second);
}
}
Changes are:
1.- std::function< void() >, instead of std::function<void> (yours only had the return type, but not the parameters (note that I've defined the operator() with empty parameters).
2.- *elem.second as the argument to submit (instead of elem.second->operator()). Note that a function object (i.e. an object of class with an operator() defined) can be used as a std::function (refer to the examples in the std::function reference)
Edit: avoid the lack of copy constructor
Instead of using directly the Logger object that cannot be copy-constructed, we can use a Wrapper object. This Wrapper object contains a reference to the Logger class.
I've also changed the code to use shared_ptr, because memory ownership becomes more complex:
#include <map>
#include <string>
#include <functional>
#include <memory>
class SimpleThreadPool {
public:
template <typename FunctionType>
void submit(FunctionType f) { }
};
class Logger {
public:
Logger() { }
Logger(const Logger&) = delete; // Logger cannot be copied
void operator()() { }
};
template <typename T>
class Wrapper {
public:
Wrapper(const std::shared_ptr<T> &l):mWrapped(l) { } // Wraps a T reference
void operator()() {
(*mWrapped)(); // expected to be a function object
}
private:
std::shared_ptr<T> mWrapped;
};
int main(int argc,char **argv) {
// Need to change the definition of the map
// to contain shared_ptr.
std::map<std::string,std::shared_ptr<Logger> > m_logger_map;
m_logger_map["one"]=std::make_shared<Logger>();
SimpleThreadPool *m_thread_pool=new SimpleThreadPool();
for (const auto& elem: m_logger_map) {
// Create the Wrapper object from the reference we've
// obtained from the map.... The wrapper can be copied
// around, but the reference will not change.
std::function<void()> f=Wrapper<Logger>(elem.second);
m_thread_pool->submit<std::function< void() > >(f);
}
}

Related

Default argument with template in C++

I'm designing an interface, by which users can define a class that tells what they want to do.
The code is something like the following,
#include <stdio.h>
class Dummy{
public:
void do(){ printf("do nothing\n"); }
};
class Task{
public:
void do(){ printf("do something\n"); }
};
template <class TASK>
void func(TASK &task = Dummy()){
task.do();
}
int main(){
func(); // do nothing
Task task;
func(task); // do something
}
How to make it work?
The main issue is this func argument:
TASK &task = Dummy()
It will not work unless it is const. This happens because non-const lvalue reference to type cannot bind to a temporary.
But if you can use const there, you can easily solve your problem:
class Dummy{
public:
void doit() const { printf("do nothing\n"); }
};
class Task{
public:
void doit() const { printf("do something\n"); }
};
template <class TASK = Dummy>
void func(const TASK &task = TASK()){
task.doit();
}
int main(){
func(); // do nothing
Task task;
func(task); // do something
}
For starters, don't have an identifier (function named) named do, since do is a language keyword. Using it as an identifier is a diagnosable error. There's no option other than changing the name of the function.
Second, the argument will of func() will need to be const, since the default value being passed is a temporary (which can only be bound to a const reference). This also means your function in the classes needs to be const qualified.
Third, when calling func() it is necessary to either pass SOME information so the compiler can work out how to instantiate the template. If you want to pass no information at all at the call site (i.e. func()) then you need to have a non-templated overload.
Fourth, use C++ streams rather than C I/O.
class Dummy
{
public:
void do_it() const { std::cout << "do nothing\n"; }
};
class Task
{
public:
void do_it() const { std::cout << "do something\n"; }
};
template <class TASK>
void func(const TASK &task)
{
task.do_it();
}
void func()
{
func(Dummy());
}
int main()
{
func(); // do nothing
Task task;
func(task); // do something
}
Option 2 is to replace the two versions of func() above with
template <class TASK = Dummy>
void func(const TASK &task = TASK())
{
task.do_it();
}

Callback using lambda with closures

I'm trying to implement a callback which passes control from an Interrupt Service Routine to a member function on a c++ class. I thought lambdas and closures would be a convenient means of doing this, but I'm having trouble implementing it. Below is a simplified version of my code.
The issue I'm stuck on is how to store the "function pointer" to the "lambda".
class Gpio
{
public:
typedef void (*ExtiHandler)();
private:
ExtiHandler handler;
public:
void enable_irq(ExtiHandler handler_in)
{
// enable interrupt
// ...
// save handler so callback can be issued later
handler = handler_in;
}
};
class Button
{
private:
Gpio& pin;
public:
Button(Gpio& pin_in) : pin(pin_in)
{
};
void button_pressed()
{
// do something
}
void init()
{
pin.enable_irq([this]() { this->button_pressed(); });
}
};
Compiling fails with the following error message;
no matching function for call to 'Gpio::enable_irq(Button::init()::<lambda()>)'candidate: void Gpio::enable_irq(Gpio::ExtiHandler) no known conversion for argument 1 from 'Button::init()::<lambda()>' to 'Gpio::ExtiHandler {aka void (*)()}' Build failed
How can I modify this code to resolve the compile error?
The problem is, that enable_irq function expects a typed function pointer of type void (*ExtiHandler)() not a lambda function.
That means, here
pin.enable_irq([this]() { this->button_pressed(); });
you are trying to store a lambda function(with capturing the instance) to a typed function pointer. You could have converted the lambda to a function pointer(easily) if it would have been a capture-less lambda.
See [expr.prim.lambda.closure] (sec 7)
The closure type for a non-generic lambda-expression with no
lambda-capture whose constraints (if any) are satisfied has a
conversion function to pointer to function with C++ language linkage
having the same parameter and return types as the closure type's
function call operator.
Since lambdas are not just ordinary functions and capturing it need to preserve a state,
you can not find any simple or conventional solution to make them assign to function pointers.
Solution - 1
The simplest solution is to use std::function instead, by paying some type erasure overhead. That means, in your code, just need to change the
typedef void(*ExtiHandler)();
to
typedef std::function<void()> ExtiHandler;
// or
// using ExtiHandler = std::function<void()>;
Solution - 2
Can this be accomplished without using the STL?
Yes. After making a small research on this topic, I came up with a type traits solution to store the lambdas with closure to the equivalent typed function pointer.
#include <iostream>
template<typename Lambda> struct convert_lambda : convert_lambda<decltype(&Lambda::operator())> {};
template<typename Lambda, typename ReType, typename... Args>
struct convert_lambda<ReType(Lambda::*)(Args...) const>
{
using funPtr = ReType(*)(Args...);
static funPtr make_function_ptr(const Lambda& t)
{
static const Lambda& lmda = t;
return [](Args... args) { return lmda(args...); };
}
};
template<typename Lambda> using convert_lambda_t = typename convert_lambda<Lambda>::funPtr;
template<typename Lambda> constexpr convert_lambda_t<Lambda> make_function_ptr(const Lambda& t)
{
return convert_lambda<Lambda>::make_function_ptr(t);
}
Usage: SEE LIVE EXAMPLE
You can now simply continue with your Gpio and Button classes, without
changing anything.:
pin.enable_irq(make_function_ptr([this]() { this->button_pressed(); }));
// or
// pin.enable_irq(make_function_ptr([&]() { this->button_pressed();}));
Or with arguments. For example
int aa = 4;
auto lmda = [&aa](const int a, const float f) { std::cout << a * aa * f << std::endl; };
void(*fPtrTest)(const int, const float) = make_function_ptr(lmda);
fPtrTest(1, 2.0f);
Drawbacks: The solution - 2:
is not capable of recognizing the optional sequence of specifiers.(i.e, mutable, constexpr)
is not capable of forwarding parameter pack to the traits. i.e,
the following is not possible:
return [](Args&&... args) { return lmda(std::forward<Args>(args)...); };
Closure object can be assigned to function pointer only if capture list of lambda is empty, in your case this condition is not met - [this].
You can use std::function as wrapper to store your closures:
#include <functional>
class Gpio
{
public:
using ExtiHandler = std::function<void()>;
private:
std::function<void()> handler;
public:
void enable_irq(const ExtiHandler& handler_in)
{
handler = handler_in;
}
};
If you don't have std library then you could implement the type erasure yourself.
Something like this ...
#include <iostream>
#include <memory>
struct function
{
struct base
{
virtual void call() = 0;
virtual base* clone() = 0;
};
template <typename Fn>
struct impl : base
{
Fn fn_;
impl(Fn&& fn) : fn_(std::forward<Fn>(fn)){}
impl(Fn& fn) : fn_(fn){}
virtual void call()
{
fn_();
}
virtual base* clone() { return new impl<Fn>(fn_); }
};
base* holder_;
function() : holder_(nullptr)
{};
template <typename Fn>
function(Fn&& fn) : holder_(nullptr)
{
holder_ = new impl<Fn>(std::forward<Fn>(fn));
}
function( function&& other)
{
holder_ = other.holder_;
other.holder_ = nullptr;
}
function(const function& other)
{
holder_ = other.holder_->clone();
}
~function()
{
if (holder_) delete holder_;
}
function& operator=(function&& other)
{
if (holder_) delete holder_;
holder_ = other.holder_;
other.holder_ = nullptr;
return *this;
}
function& operator=(const function& other)
{
if (holder_) delete holder_;
holder_ = other.holder_->clone();
return *this;
}
void operator()()
{
holder_->call();
}
};
class Gpio
{
public:
using ExtiHandler = function;
//private:
ExtiHandler handler;
//public:
void enable_irq(ExtiHandler handler_in)
{
// enable interrupt
// ...
// save handler so callback can be issued later
handler = handler_in;
}
};
class Button
{
private:
Gpio& pin;
public:
Button(Gpio& pin_in) : pin(pin_in)
{
};
void button_pressed()
{
std::cout << "Button pressed" << std::endl;
}
void init()
{
pin.enable_irq([this]() { this->button_pressed(); });
}
};
int main() {
Gpio some_pin;
Button b(some_pin);
b.init();
some_pin.handler();
return 0;
}
Demo

Combine QtConcurrent call

I have three methods and all of them returns a string, I want to run all of them using QtConcurrent and get their return into a single list or something like that. QtConcurrent::mapped is ideal as it returns an iterator but I can only run one method at time.
In JavaScript there's promise.all([method_a, method_b, method_c]), it will automatically merge their return into a single result (iterator).
How to do that in Qt?
Since you have several methods to call, you can pass them as a sequence of functors as the first argument to QtConcurrent::mapped. The mapping functor would be an apply functor that takes a functor representing the method call and returns the result of invoking it.
First, let's have our class:
// https://github.com/KubaO/stackoverflown/tree/master/questions/concurrent-combine-49802153
#include <QtConcurrent>
#include <functional>
#include <initializer_list>
#include <type_traits>
class Cls {
public:
QString method1() const { return QStringLiteral("10"); }
QString method2() const { return QStringLiteral("20"); }
QString method3() const { return QStringLiteral("30"); }
};
The apply_t functor invokes the method passed to it as an argument:
template <class Method> struct apply_t {
using result_type = typename std::result_of_t<Method()>;
auto operator()(Method method) {
return method();
}
};
Let's make it convenient to make such applicators from the type of a sequence of functors to be called:
template <class Sequence, class A = apply_t<typename std::decay_t<Sequence>::value_type>>
A make_apply(Sequence &&) { return {}; }
For convenience, we'll also have a vector generator in the spirit of e.g. make_unique, etc.:
template <class T> QVector<T> make_vector(std::initializer_list<T> init) {
return {init};
}
Then, the problem becomes fairly simple. First, we make a vector of bound methods that will be called. Then we pass the methods to call, as well as the applicator to operate on them, to QtConcurrent::mapped. The results() gives a list of all results of the method calls, in sequence.
int main() {
Cls obj;
auto const methods = make_vector({
std::bind(&Cls::method1, &obj),
std::bind(&Cls::method2, &obj),
std::bind(&Cls::method3, &obj)
});
QFuture<QString> result =
QtConcurrent::mapped(methods, make_apply(methods));
Q_ASSERT((result.results() == QStringList{"10", "20", "30"}));
}
Instead of making a custom apply_t class, we could use a lambda, wrapped to provide the result_type member type that QtConcurrent::mapped expects. See this answer for details of wrapping the lambda. The rest of this answer provides examples of such wrapping.
Since there's no built in way to do that, you can work out a class on your own to keep futures together and return a collection of results when all tasks completed. The only limitation, here, is due to the strongly typed nature of c++: each future returned by QtConcurrent::run holds the called function result, the type of which is given at compile time, as the QFuture template parameter. What if the return types of the called functions differ from each other? In the example I provide, they all return the same type, but I think one could use QVariant for that mean and get away with it.
In a promise.h:
#ifndef PROMISE_H
#define PROMISE_H
#include <QtConcurrent/QtConcurrentRun>
#include <QFutureWatcher>
class PromiseInterface
{
public:
virtual ~PromiseInterface() = default;
virtual void finished(int id) = 0;
};
class Watcher : public QObject
{
Q_OBJECT
int _id;
PromiseInterface * _promise;
public slots:
void finished()
{
_promise->finished(_id);
deleteLater();
}
public:
Watcher(int id, PromiseInterface * promise)
: _id(id),
_promise(promise)
{}
};
template <typename T>
class Promise : public PromiseInterface
{
friend class Watcher;
void finished(int id) override
{
_resolved++;
_results[id] = _watchers[id]->result();
delete _watchers[id];
if(_resolved == _results.size())
{
if(_callback != nullptr)
{
_callback(_results);
}
}
}
QList<QFutureWatcher<T> *> _watchers;
QVector<T> _results;
void (*_callback)(QVector<T>);
int _resolved;
public:
Promise(QList<QFuture<T>> futures)
{
_resolved = 0;
_callback = nullptr;
_results.resize(futures.size());
int i=0;
for(auto f : futures)
{
QFutureWatcher<T> * watcher = new QFutureWatcher<T>();
watcher->setFuture(f);
QObject::connect(watcher, &QFutureWatcher<T>::finished, new Watcher(i++, this), &Watcher::finished);
_watchers.append(watcher);
}
}
void then(void (*callback)(QVector<T>)) { _callback = callback; }
};
#endif // PROMISE_H
The Promise class is a class template with a single template parameter that matches the one of QFuture. The observed futures are passed in the constructor, while the then method accepts a the completion callback as its only argument.
The Watcher class provides a slot to catch QFutureWatcher::finished' signals. Each instance knows the promise object through a pointer to itsPromiseInterfaceand will callfinished` from the slot, passing in the id of the future that completed.
When all futures finished, the callback function is called with the vector of results passed in.
In a very simple usage example, we can execute this function concurrently:
#include <unistd.h>
int f(int r) { sleep(1); return r;}
and pass this callback to the promise then:
void callback(QVector<int> results)
{
qDebug() << results;
}
Our main:
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
QList<QFuture<int>> futures = {
QtConcurrent::run(&f, 1),
QtConcurrent::run(&f, 2),
QtConcurrent::run(&f, 3)
};
Promise<int> promise(futures);
promise.then(callback);
return a.exec();
}
After about a second, this is the expected output:
QVector(1, 2, 3)
Just in case one wonders why I put in three classes, instead of making Promise extend QObject directly, and implement the finished slot itself: Qt is not letting me do that. When the Q_OBJECT macro is added to a class template, an explicit compiler error will prompt: Template classes not supported by Q_OBJECT.
The method you're looking for is QFuture::results():
QList<T> QFuture::results() const
Returns all results from the future. If the results are not
immediately available, this function will block and wait for them to
become available.
Expanding from Qt's own QtConcurrent::mapped example:
QImage scaled(const QImage &image)
{
return image.scaled(100, 100);
}
QList<QImage> images = ...;
QList<QImage> thumbnails = QtConcurrent::mapped(images, scaled).results();
I know AsyncFuture is a c++ library that converts calls into a QFuture type and uses it like a Promise object in Javascript (Combine multiple futures with different type into a single future object). unfortunately i never used it! but there are details on this reference Qt Blog Multithreaded Programming with Future & Promise

pulling out unique_ptr from unique_ptr<T,Deleter>

Can someone please advise on how to return the unique pointer from a templatised unique pointer pool with a custom deletor.
In the code snippet below i am using ObjectPool.h as my template class to get a stack of unique pointers .
I am using ObjectPool to create a sharedpool object in DBConnection.h and later in DBConnection.cpp i am simply returning the object to DBExec .
I get compile errors in DBConnection.cpp related to conversion of pointer with deleter to normal unique pointer.
> Class that will manage connection objects.
**DBConnectionPool.h**
#ifndef DBCONNECTIONPOOL_H
#define DBCONNECTIONPOOL_H
#include "DBExec.h"
#include "ObjectPool.h"
class DBConnectionPool {
static SharedPool<DBExec> pool;
static DBConnectionPool* instance;
DBConnectionPool& operator=(const DBConnectionPool&);
DBConnectionPool(const DBConnectionPool&);;
DBConnectionPool(){};
public:
...
**static std::unique_ptr<DBExec> getQueryObject();**
};
#endif /* DBCONNECTIONPOOL_H */
**DBConnection.cpp**
>implementation of getQueryObject
**std::unique_ptr<DBExec> DBConnectionPool::getQueryObject() {
return std::move(pool.acquire());
}**
/* Class that manages the unique pointer */
**ObjectPool.h**
#ifndef OBJECTPOOL_H
#define OBJECTPOOL_H
#include <memory>
#include <stack>
#include <mutex>
#include <assert.h>
template <class T>
class SharedPool {
/* Class that manages the unique pointer */
public:
using ptr_type = std::unique_ptr<T, std::function<void(T*)> >;
SharedPool() {
}
virtual ~SharedPool() {
}
void add(std::unique_ptr<T> t) {
std::lock_guard<std::mutex> lck (mt);
pool_.push(std::move(t));
}
ptr_type acquire() {
std::lock_guard<std::mutex> lck (mt);
assert(!pool_.empty());
ptr_type tmp(pool_.top().release(),
[this](T * ptr) {
this->add(std::unique_ptr<T>(ptr));
});
pool_.pop();
return std::move(tmp);
}
bool empty() const {
std::lock_guard<std::mutex> lck (mt);
return pool_.empty();
}
size_t size() const {
std::lock_guard<std::mutex> lck (mt);
return pool_.size();
}
std::stack<std::unique_ptr<T>>& getPoolStack () {
return pool_;
}
private:
> thread safe
std::mutex mt;
std::stack<std::unique_ptr<T> > pool_;
};
#endif /* OBJECTPOOL_H */
std::unique_ptr<T,D> and std::unique_ptr<T> are unrelated types (unless D=std::default_delete<T>). So you need to specify the type of the deleter in all your typedefs.
The next problem is you can't actually type out the type of your deleter, since it's a lambda type. So you may have to switch from a lambda to a named callable type.
template <class T>
class SharedPool {
public:
class Deleter {
public:
void operator()(T* ptr) const {
pool->add(std::unique_ptr<T,Deleter>(ptr));
}
private:
explicit Deleter(SharedPool* pool) : pool(pool) {}
SharedPool* pool;
friend class SharedPool<T>;
};
using ptr_type = std::unique_ptr<T, Deleter>;
You might also consider using std::shared_ptr since it type-erases its deleter type, but you don't seem to need the sharedness.
If I understand you correctly, you want to get a regular unique_ptr from one that has a custom deleter. This is not possible because returning a unique_ptr from another unique_ptr would violate the uniqueness of that pointer. Your only option would be to convert it to something, where the deleter is hidden from the type (i.e., type erasure).
It sounds like what you want is type erasure, or at least partial type erasure. A simple solution would be to just use std::shared_ptr:
std::shared_ptr<T>(new T, custom_deleter);
This would provide some overhead over std::unique_ptr, though some overhead is unavoidable.

Missing call to bound pointer to member function

I am trying to implement a generic event handling system with templates, but I get an error at compile time when I try to add the function to a list.
error C2298: missing call to bound pointer to member function
class EventManager
{
public:
template <class OBJECT>
void subscribe(std::string eventName, OBJECT *obj, void(OBJECT::*newEvent)(Event))
{
(obj->*newEvent)(Event()); // I can call the event just fine if I execute it alone
abc.push_back(obj->*newEvent); // Error c2298 here
}
static EventManager* Get();
private:
std::vector<void(*)(Event)> eventsList;
};
And this is where I call it:
EventManager::Get()->subscribe("EventName", this, &TestClass::eventTest);
void ATideManager::eventTest(Event event)
{
//Test
}
I am quite new to C++ and templates, so I'm sure the solution is easy, but I don't understand why I can call the event but not add it to the vector. Do you know how I could possibly do that?
As the documentation for that error message says:
A pointer to member-function expression must call the member function.
In other words, you cannot store a pointer to member-function including the object on which to call it, for later use.
Here is a simpler example using names similar to yours:
struct Object
{
void f() {}
};
int main()
{
Object obj;
using member_function_pointer = void (Object::*)();
member_function_pointer ptr = &Object::f;
(obj.*ptr)();
(obj.*ptr); // error
}
And this makes a lot of sense. Your abc is probably a vector of pointers to member-function (or just a vector of function pointers); it cannot magically store an object along with each pointer in it.
Generally, pointers to member functions are not exactly the nicest, cleanest or best feature of C++, and the awful syntax renders every piece of code using it completely unreadable. Fortunately, C++11 introduced std::function and lambdas, which gives you a much better alternative:
#include <functional>
#include <iostream>
#include <string>
#include <vector>
struct Event
{
int i;
};
class EventManager
{
public:
void subscribe(std::string const& eventName,
std::function<void(Event const&)> event)
{
abc.push_back(event);
}
static EventManager& get()
{
static EventManager instance;
return instance;
}
void onEvent(Event const& event)
{
for (auto&& f : abc)
{
f(event);
}
}
private:
EventManager() {}
EventManager(EventManager const&) = delete;
EventManager& operator=(EventManager const&) = delete;
std::vector<std::function<void(Event const&)>> abc;
};
struct ATideManager
{
void f()
{
EventManager::get().subscribe("EventName",
[=](Event const& event) { eventTest(event); });
}
void eventTest(Event const& event)
{
std::cout << "eventTest: " << event.i << "\n";
}
};
int main()
{
ATideManager manager;
manager.f();
Event some_event{ 123 };
EventManager::get().onEvent(some_event);
}
I've also fixed your needless use of pointers where references can be used, and some const-correctness issues.
obj->*newEvent is invalid. A member function pointer can be used in this way only to call the pointed-to member function.
You can however use std::vector<std::function<void(Event)>> and
abc.push_back(std::bind(newEvent, obj, std::placeholders::_1);
std::bind reference