[It is not necessary to follow the links to understand the question].
I combined the implementation of the singleton pattern in this answer, together with the synchronized file writing of this other answer.
Then I wanted to see if the interface of SynchronizedFile could provide a variadic templated write method, but I couldn't figure out how to properly combine this with the std::lock_guard.
Below is a non-working example. In this case it doesn't work because (I think) the two threads manage to pump stuff into the buffer i_buf in a non-synchronized way, resulting in a garbled LOGFILE.txt.
If I put the std::lock_guard inside the general template of write then the program doesn't halt.
#include <iostream>
#include <mutex>
#include <sstream>
#include <fstream>
#include <string>
#include <memory>
#include <thread>
static const int N_LOOP_LENGTH{10};
// This class manages a log file and provides write method(s)
// that allow passing a variable number of parameters of different
// types to be written to the file in a line and separated by commas.
class SynchronizedFile {
public:
static SynchronizedFile& getInstance()
{
static SynchronizedFile instance;
return instance;
}
private:
std::ostringstream i_buf;
std::ofstream i_fout;
std::mutex _writerMutex;
SynchronizedFile () {
i_fout.open("LOGFILE.txt", std::ofstream::out);
}
public:
SynchronizedFile(SynchronizedFile const&) = delete;
void operator=(SynchronizedFile const&) = delete;
template<typename First, typename... Rest>
void write(First param1, Rest...param)
{
i_buf << param1 << ", ";
write(param...);
}
void write()
{
std::lock_guard<std::mutex> lock(_writerMutex);
i_fout << i_buf.str() << std::endl;
i_buf.str("");
i_buf.clear();
}
};
// This is just some class that is using the SynchronizedFile class
// to write stuff to the log file.
class Writer {
public:
Writer (SynchronizedFile& sf, const std::string& prefix)
: syncedFile(sf), prefix(prefix) {}
void someFunctionThatWritesToFile () {
syncedFile.write(prefix, "AAAAA", 4343, "BBBBB", 0.2345435, "GGGGGG");
}
private:
SynchronizedFile& syncedFile;
std::string prefix;
};
void thread_method()
{
SynchronizedFile &my_file1 = SynchronizedFile::getInstance();
Writer writer1(my_file1, "Writer 1:");
for (int i = 0; i < N_LOOP_LENGTH; ++ i)
writer1.someFunctionThatWritesToFile();
}
int main()
{
std::thread t(thread_method);
SynchronizedFile &my_file2 = SynchronizedFile::getInstance();
Writer writer2(my_file2, "Writer 2:");
for (int i = 0; i < N_LOOP_LENGTH; ++i)
writer2.someFunctionThatWritesToFile();
t.join();
std::cout << "Done" << std::endl;
return 0;
}
How could I successfully combine these three ideas?
The program deadlocks because write calls itself recursively while still holding the lock.
Either use a std::recursive_mutex or release the lock after writing your data out but before calling write.
E: Unlocking doesn't do the job, I didn't think this through...
E: Or lock once and defer to another private method to do the write.
template<typename... Args>
void write(Args&&... args)
{
std::unique_lock<std::mutex> lock(_writerMutex);
_write(std::forward<Args>(args)...);
}
template<typename First, typename... Rest>
void _write(First&& param1, Rest&&... param) // private method
{
i_buf << std::forward<First>(param1) << ", ";
_write(std::forward<Rest>(param)...);
}
void _write()
{
i_fout << i_buf.str() << std::endl;
i_buf.clear();
}
Related
I am currently practicing the use of multiple threads in C++. The program is simplified as follow. In this case, I have a global variable Obj, and within each task, a get function is processed by thread and thread detach will be called after.
In practice, get may take a great amount of time to run. If there are many tasks, get will be called repetitively (since each task has its own get function). I wonder if I can design a program where when one task has already obtained the data using get function and the data has been wrote to obj.text, then the rest of tasks can directly access or wait for the data from obj.text.
Can I use std::shared_ptr, std::future, std::async in c++ to implement this? If so, how to design the program? Any advice is greatly appreciated.
#include <chrono>
#include <future>
#include <iostream>
#include <memory>
#include <thread>
#include <vector>
using namespace std;
class Info {
public:
Info() { Ids = 10; };
int Ids;
std::string text;
};
Info Objs;
class Module {
public:
Module() {}
virtual void check(int &id){};
virtual void get(){};
};
class task1 : public Module {
public:
task1() { std::cout << "task1" << std::endl; }
void check(int &id) override {
thread s(&task1::get, this);
s.detach();
};
// The function will first do some other work (here, I use sleep to represent
// that) then set the value of Objs.text
void get() override {
// The task may take 2 seconds , So use text instead
std::this_thread::sleep_for(std::chrono::seconds(5));
Objs.text = "AAAA";
std::cout << Objs.text << std::endl;
};
};
class task2 : public Module {
public:
task2() { std::cout << "task2" << std::endl; }
void check(int &id) override {
thread s(&task2::get, this);
s.detach();
};
// The function will first do some other work (here, I use sleep to represent
// that) then set the value of Objs.text
void get() {
std::this_thread::sleep_for(std::chrono::seconds(5));
Objs.text = "AAAA";
std::cout << Objs.text << std::endl;
};
};
int main() {
std::vector<std::unique_ptr<Module>> modules;
modules.push_back(std::make_unique<task1>());
modules.push_back(std::make_unique<task2>());
for (auto &m : modules) {
m->check(Objs.Ids);
}
std::this_thread::sleep_for(std::chrono::seconds(12));
return 0;
}
It is a plain producer-consumer problem.
You have multiple “get()” producers. And did not implemented consumers yet.
First, you should have multiple “Info” for multithread. If there is only one Info, multithread programming is useless. I recommend “concurrent_queue”.
Second, “detach()” is not a good idea. You can’t manage child threads. You’d better use “join()”
My code sample follows. I used Visual Studio 2022
#include <chrono>
#include <iostream>
#include <thread>
#include <vector>
#include <concurrent_queue.h>
using namespace std;
class Info {
public:
Info() { Ids = 10; };
int Ids;
std::string text;
};
concurrency::concurrent_queue<Info> Objs;
void producer()
{
while (true) {
Info obj;
std::this_thread::sleep_for(std::chrono::seconds(5));
obj.text = "AAAA\n";
Objs.push(obj);
}
}
void consumer()
{
while (true) {
std::this_thread::sleep_for(std::chrono::seconds(1));
Info obj;
bool got_it = Objs.try_pop(obj);
if (got_it) {
std::cout << obj.text;
}
}
}
int main() {
const int NUM_CORES = 6;
std::vector<std::thread> threads;
for (int i = 0; i < NUM_CORES / 2; ++i)
threads.emplace_back(producer);
for (int i = 0; i < NUM_CORES / 2; ++i)
threads.emplace_back(consumer);
for (auto& th : threads) th.join();
}
When trying to learn threads most examples suggests that I should put std::mutex, std::condition_variable and std::queue global when sharing data between two different threads and it works perfectly fine for simple scenario. However, in real case scenario and bigger applications this may soon get complicated as I may soon lose track of the global variables and since I am using C++ this does not seem to be an appropriate option (may be I am wrong)
My question is if I have a producer/consumer problem and I want to put both in separate classes, since they will be sharing data I would need to pass them the same mutex and queue now how do I share these two variables between them without defining it to be global and what is the best practice for creating threads?
Here is a working example of my basic code using global variables.
#include <iostream>
#include <thread>
#include <mutex>
#include <queue>
#include <condition_variable>
std::queue<int> buffer;
std::mutex mtx;
std::condition_variable cond;
const int MAX_BUFFER_SIZE = 50;
class Producer
{
public:
void run(int val)
{
while(true) {
std::unique_lock locker(mtx) ;
cond.wait(locker, []() {
return buffer.size() < MAX_BUFFER_SIZE;
});
buffer.push(val);
std::cout << "Produced " << val << std::endl;
val --;
locker.unlock();
// std::this_thread::sleep_for(std::chrono::seconds(2));
cond.notify_one();
}
}
};
class Consumer
{
public:
void run()
{
while(true) {
std::unique_lock locker(mtx);
cond.wait(locker, []() {
return buffer.size() > 0;
});
int val = buffer.front();
buffer.pop();
std::cout << "Consumed " << val << std::endl;
locker.unlock();
std::this_thread::sleep_for(std::chrono::seconds(1));
cond.notify_one();
}
}
};
int main()
{
std::thread t1(&Producer::run, Producer(), MAX_BUFFER_SIZE);
std::thread t2(&Consumer::run, Consumer());
t1.join();
t2.join();
return 0;
}
Typically, you want to have synchronisation objects packaged alongside the resource(s) they are protecting.
A simple way to do that in your case would be a class that contains the buffer, the mutex, and the condition variable. All you really need is to share a reference to one of those to both the Consumer and the Producer.
Here's one way to go about it while keeping most of your code as-is:
class Channel {
std::queue<int> buffer;
std::mutex mtx;
std::condition_variable cond;
// Since we know `Consumer` and `Producer` are the only entities
// that will ever access buffer, mtx and cond, it's better to
// not provide *any* public (direct or indirect) interface to
// them, and use `friend` to grant access.
friend class Producer;
friend class Consumer;
public:
// ...
};
class Producer {
Channel* chan_;
public:
explicit Producer(Channel* chan) : chan_(chan) {}
// ...
};
class Consumer {
Channel* chan_;
public:
explicit Consumer(Channel* chan) : chan_(chan) {}
// ...
};
int main() {
Channel channel;
std::thread t1(&Producer::run, Producer(&channel), MAX_BUFFER_SIZE);
std::thread t2(&Consumer::run, Consumer(&channel));
t1.join();
t2.join();
}
However, (Thanks for the prompt, #Ext3h) a better way to go about this would be to encapsulate access to the synchronisation objects as well, i.e. keep them hidden in the class. At that point Channel becomes what is commonly known as a Synchronised Queue
Here's what I'd subjectively consider a nicer-looking implementation of your example code, with a few misc improvements thrown in as well:
#include <cassert>
#include <iostream>
#include <thread>
#include <mutex>
#include <queue>
#include <optional>
#include <condition_variable>
template<typename T>
class Channel {
static constexpr std::size_t default_max_length = 10;
public:
using value_type = T;
explicit Channel(std::size_t max_length = default_max_length)
: max_length_(max_length) {}
std::optional<value_type> next() {
std::unique_lock locker(mtx_);
cond_.wait(locker, [this]() {
return !buffer_.empty() || closed_;
});
if (buffer_.empty()) {
assert(closed_);
return std::nullopt;
}
value_type val = buffer_.front();
buffer_.pop();
cond_.notify_one();
return val;
}
void put(value_type val) {
std::unique_lock locker(mtx_);
cond_.wait(locker, [this]() {
return buffer_.size() < max_length_;
});
buffer_.push(std::move(val));
cond_.notify_one();
}
void close() {
std::scoped_lock locker(mtx_);
closed_ = true;
cond_.notify_all();
}
private:
std::size_t max_length_;
std::queue<value_type> buffer_;
bool closed_ = false;
std::mutex mtx_;
std::condition_variable cond_;
};
void producer_main(Channel<int>& chan, int val) {
// Don't use while(true), it's Undefined Behavior
while (val >= 0) {
chan.put(val);
std::cout << "Produced " << val << std::endl;
val--;
}
}
void consumer_main(Channel<int>& chan) {
bool running = true;
while (running) {
auto val = chan.next();
if (!val) {
running = false;
continue;
}
std::cout << "Consumed " << *val << std::endl;
};
}
int main()
{
// You are responsible for ensuring the channel outlives both threads.
Channel<int> channel;
std::thread producer_thread(producer_main, std::ref(channel), 13);
std::thread consumer_thread(consumer_main, std::ref(channel));
producer_thread.join();
channel.close();
consumer_thread.join();
return 0;
}
I'm using CRTP design pattern to implement logging mechanism for my project. Base CRTP class looks like this:
#include <fstream>
#include <memory>
#include <mutex>
#include <iostream>
#include <sstream>
template <typename LogPolicy>
class Logger
{
public:
template <typename... Args>
void operator()(Args... args)
{
loggingMutex.lock();
putTime();
print_impl(args...);
}
void setMaxLogFileSize(unsigned long maxLogFileSizeArg)
{
//if (dynamic_cast<FileLogPolicy *>(policy.get()))
// policy->setMaxLogFileSize(maxLogFileSizeArg);
}
~Logger()
{
print_impl(END_OF_LOGGING);
}
protected:
std::stringstream buffer;
std::mutex loggingMutex;
std::string d_time;
private:
static constexpr auto END_OF_LOGGING = "***END OF LOGGING***";
void putTime()
{
time_t raw_time;
time(&raw_time);
std::string localTime = ctime(&raw_time);
localTime.erase(std::remove(localTime.begin(), localTime.end(), '\n'), localTime.end());
buffer << localTime;
}
template <typename First, typename... Rest>
void print_impl(First first, Rest... rest)
{
buffer << " " << first;
print_impl(rest...);
}
void print_impl()
{
static_cast<LogPolicy*>(this)->write(buffer.str());
buffer.str("");
}
};
One of the concrete logging class is logging to file, which looks like this:
#include "Logger.hpp"
class FileLogPolicy : public Logger<FileLogPolicy>
{
public:
FileLogPolicy(std::string fileName) : logFile(new std::ofstream)
{
logFile->open(fileName, std::ofstream::out | std::ofstream::binary);
if (logFile->is_open())
{
std::cout << "Opening stream with addr " << (logFile.get()) << std::endl;
}
}
void write(const std::string content)
{
std::cout << "Writing stream with addr " << (logFile.get()) << std::endl;
(*logFile) << " " << content << std::endl;
loggingMutex.unlock();
}
virtual ~FileLogPolicy()
{
}
private:
std::unique_ptr<std::ofstream> logFile; //Pointer to logging stream
static const char *const S_FILE_NAME; //File name used to store logging
size_t d_maxLogFileSize; //File max size used to store logging
};
Basically I create object of policy class and would like to log stuff, depending on the policy chosen. So for example I create logger like this:
FileLogPolicy log("log.txt");
In this case it should use Logger to save logs to file, by calling static_cast<LogPolicy*>(this)->write(buffer.str()). Apparently calling write function works fine but stream object is changing to null. How is that possible if FileLogPolicy destructor has not been called yet? When I change logFile to be normal pointer all works well. I don't get it where is the difference.
~Logger()
{
print_impl(END_OF_LOGGING);
}
this code runs after the descendend class has been destroyed.
void print_impl()
{
static_cast<LogPolicy*>(this)->write(buffer.str());
buffer.str("");
}
it then casts this to be a pointer to a class that this is no longer.
The unique ptr is gone, and even accessing the member is UB.
I've tried to implement a very basic Thread Local Singleton class in C++ - it's a template class that other classes then inherit from. The problem is that it almost always works, but every now and again (say, 1 run in 15), it will fail with an error along the lines of:
* glibc detected * ./myExe: free(): invalid next size (fast): 0x00002b61a40008c0 ***
please forgive the rather contrived example below, but it serves to demonstrate the problem.
#include <thread>
#include <atomic>
#include <iostream>
#include <memory>
#include <vector>
using namespace std;
template<class T>
class ThreadLocalSingleton
{
public:
/// Return a reference to an instance of the object
static T& instance();
typedef unique_ptr<T> UPtr;
protected:
ThreadLocalSingleton() {}
ThreadLocalSingleton(ThreadLocalSingleton const&);
void operator=(ThreadLocalSingleton const&);
};
template<class T>
T& ThreadLocalSingleton<T>::instance()
{
thread_local T m_instance;
return m_instance;
}
// Create two atomic variables to keep track of the number of times the
// TLS class is created and accessed.
atomic<size_t> creationCount(0);
atomic<size_t> accessCount(0);
// Very simple class which derives from TLS
class MyClass : public ThreadLocalSingleton<MyClass>
{
friend class ThreadLocalSingleton<MyClass>;
public:
MyClass()
{
++creationCount;
}
string getType() const
{
++accessCount;
return "MyClass";
}
};
int main(int,char**)
{
vector<thread> threads;
vector<string> results;
threads.emplace_back([&]() { results.emplace_back(MyClass::instance().getType()); MyClass::instance().getType(); });
threads.emplace_back([&]() { results.emplace_back(MyClass::instance().getType()); MyClass::instance().getType(); });
threads.emplace_back([&]() { results.emplace_back(MyClass::instance().getType()); MyClass::instance().getType(); });
threads.emplace_back([&]() { results.emplace_back(MyClass::instance().getType()); MyClass::instance().getType(); });
for (auto& t : threads)
{
t.join();
}
// Expecting 4 creations and 8 accesses.
cout << "CreationCount: " << creationCount << " AccessCount: " << accessCount << endl;
}
I can replicate this on coliru, using the build command:
g++ -std=c++11 -O2 -Wall -pedantic -pthread main.cpp && ./a.out
Many thanks!
Thanks to both molbdnilo and Damon, who quickly pointed out the obvious - vector::emplace_back isn't thread safe, so there would be no guarantees on whether or not this code would actually work. I've replaced the main() function with the following, which seems to be more reliable.
int main(int,char**)
{
vector<thread> threads;
vector<string> results;
auto addToResult = [&results](const string& val)
{
static mutex m_mutex;
unique_lock<mutex> lock(m_mutex);
results.emplace_back(val);
};
threads.emplace_back([&addToResult]() { addToResult(MyClass::instance().getType()); MyClass::instance().getType(); });
threads.emplace_back([&addToResult]() { addToResult(MyClass::instance().getType()); MyClass::instance().getType(); });
threads.emplace_back([&addToResult]() { addToResult(MyClass::instance().getType()); MyClass::instance().getType(); });
threads.emplace_back([&addToResult]() { addToResult(MyClass::instance().getType()); MyClass::instance().getType(); });
for (auto& t : threads)
{
t.join();
}
// Expecting 4 creations and 8 accesses.
cout << "CreationCount: " << creationCount << " AccessCount: " << accessCount << endl;
}
Thanks!
I'm trying to wrap std::packaged_task inside another class in order to be used together with a task scheduler.
At the moment I got it all working except std::future support. To get std::future support I figured out I need to use std::packaged_task for the get_future() function that it provides.
I've been trying whole day all sorts of ways to get this to work, but I seem to be unable to properly declare and initialise the packaged_task using the return value from a std::bind. I have tried to decipher the implementations of all the related libstdc++ functions such as std::async, std::future, std::thread etc but with no luck.
The following code is the implementation of both the not working version and the working one. To get it to work uncomment the two /* --- WORKS*/ and comment the other related line.
#include <vector>
#include <deque>
#include <memory>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <future>
#include <iostream>
#include <chrono>
#include <functional>
#include <windows.h>
class task
{
private:
struct task_implementation_base
{
virtual void execute() = 0;
};
template <class callable>
struct task_implementation : public task_implementation_base
{
task_implementation(callable&& f) : /*m_task(std::forward<callable>(f)) WORKS*/m_task(f) { }
void execute() { m_task(); }
//callable m_task; // WORKS
std::packaged_task<typename result_of<callable>::type> m_task;
};
template <class callable>
std::shared_ptr<task_implementation<callable>> make_routine(callable&& f)
{
return std::make_shared<task_implementation<callable>>(std::forward<callable>(f));
}
public:
template <class callable, class... arguments>
task(callable&& f, arguments&&... args) : m_function(make_routine(std::bind(std::forward<callable>(f), std::forward<arguments>(args)...))) {}
void operator()() { run(); }
void run() { m_function->execute(); }
private:
std::shared_ptr<task_implementation_base> m_function;
};
int testint(int i)
{
std::cout << "test6" << " :: ran from thread " << std::this_thread::get_id() << "\n";
fflush(stdout);
return i+100;
}
void test(const char* text)
{
std::cout << text << " :: ran from thread " << std::this_thread::get_id() << "\n";
fflush(stdout);
}
class testclass
{
public:
void print1() { test("test3"); }
void print2() { test("test4"); }
void print3(const char* text) { test(text); }
};
int main()
{
testclass testclass1;
testclass* testclass2 = new testclass;
task test1(test, "test1");
task test2([]() { test("test2"); });
task test3(&testclass::print1, &testclass1);
task test4(&testclass::print2, &*testclass2);
task test5(&testclass::print3, &*testclass2, "test5");
task test6(&testint, 1);
test1();
test2();
test3();
test4();
test5();
test6();
Sleep(2000);
return 0;
}
I'm thinking the problem is typename result_of<callable>::type. I'm guessing it doesn't properly evaluates to the return type of the callable function.
I'm using c++ (Built by MinGW-builds project) 4.8.0 20121225 (experimental) on a Windows 8 64bit. I'm suspecting the errors are irrelevant since I guess I'm just simply trying to get this work the wrong way but here is a pastebin for the errors anyway: errors
std::packaged_task not only takes the result type of the invoked function as a template argument but also the types of the arguments you are passing to the to be invoked function.
You can define them as follows:
// somewhere
int foo(bool, int);
// somewhere else
std::packaged_task<int(bool, int)> p(foo);
To fix your code you need to add two empty parenthesis pairs. What I explained above also applies to std::result_of.
std::packaged_task<typename std::result_of<callable()>::type()> m_task;
It is only response to main topic question. "How to implement"
Example short implementation:
template <typename Signature> /// <---- 1
class Task;
template <typename Res, typename... ArgTypes>
class Task<Res(ArgTypes...)> /// <---- 2
{
public:
template <typename Function>
explicit Task(Function&& callback)
: _task{std::forward<Function>(callback)}
{ }
void execute(ArgTypes... args) noexcept(false)
{
//...
_task(std::forward<ArgTypes>(args)...);
}
private:
std::packaged_task<Res(ArgTypes...)> _task;
};
Not sure why step 1 & 2 are required but I did the same as in lib implementation. Maybe someone could extend this response.