Here is my simple code, I want to get in the console_task, the value of the variable i in the dialer_task... without using a global variable.
#include <stdio.h>
#include <sys/types.h>
#include <signal.h>
#include <strings.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <thread>
#include "console.hpp"
using namespace std;
void console_task(){
console();
}
void dialer_task(){
int i=0;
while (1) {
printf("LOOP %d\n",i);
i++;
sleep(5);
}
}
int main()
{
thread t1(console_task);
thread t2(dialer_task);
t1.join();
t2.join();
return 0;
}
The constraint that there may not be a global variable to share the state between the threads leaves essentially 2 viable alternatives;
Allocate the shared state on the heap and pass that on to the threads
Allocate the shared state on the original thread's stack and "feed" it to the worker threads for shared use.
The catch to both solutions is to make sure that the access is appropriately guarded or atomic.
A simple solution is to use an std::atomic and share the reference between the threads.
#include <type_traits>
#include <thread>
#include <atomic>
#include <iostream>
void console_task(std::atomic_int& j) {
using namespace std;
int i = 0;
while (++i < 50) {
cout << "task " << j << endl; // uncontrolled access to the console (demo)
std::chrono::microseconds delay{50};
this_thread::sleep_for(delay);
}
}
void dialer_task(std::atomic_int& j){
using namespace std;
int i = 0;
while ( ++i < 10) {
//cout << "LOOP " << i << endl; // uncontrolled access to the console (demo)
std::chrono::microseconds delay{145};
this_thread::sleep_for(delay);
j = i;
}
}
int main()
{
std::atomic_int i { 0 };
std::thread t1( console_task, std::ref(i));
// a lambda with reference capture could also be used
// std::thread t1( [&](){console_task(i);} );
std::thread t2( dialer_task, std::ref(i));
t1.join();
t2.join();
return 0;
}
There is a catch to the shared atomic, it needs to remain valid for the duration of the threads (as it does here).
Demo code.
Further heap based alternatives can be considered; e.g. using a shared std::mutex together with a std::shared_ptr.
Related
I am trying to set the name of my threads for ease of profiling (in ps, top, etc.).
I usually use pthread_setname_np(pthread_self(), <THREAD_NAME>) and it works like a charm.
But the one place I cannot understand is how to do it when I'm using lambda threads.
Here's an example on what I'm trying to do.
#include <iostream>
#include <thread>
#include <vector>
#include <algorithm>
#include <pthread.h>
int main()
{
// vector container stores threads
std::vector<std::thread> workers;
for (int i = 0; i < 5; i++) {
workers.push_back(std::thread([]()
{
std::cout << "thread function\n";
}));
}
std::cout << "main thread\n";
std::for_each(workers.begin(), workers.end(), [](std::thread& t)
{
//I want to set the thread name for profiling
std::string s = "mythread";
auto handle = t.native_handle();
pthread_setname_np(handle, s.c_str());
t.join();
});
return 0;
}
This does not work. If I do top -H -p <pid> in Linux I cannot see those threads named.
Thanks
In my main program I am copying a string buffer into a boost ring buffer and then trying to consume that data in a created thread and writing to a file. In the main thread also I am writing the same data to a file but both input and output file is not matching.
I feel I am doing something incredibly stupid. Please help. Also, if there are any suggesting to improve the code that would really be appreciated.
#include <iostream>
#include <vector>
#include <boost/circular_buffer.hpp>
#include <numeric>
#include <assert.h>
#include <thread>
#include <mutex>
#include <chrono>
#include <time.h>
#include <cstdint>
#include <fstream>
#include <string>
using std::cin;
using std::cout;
using std::endl;
using std::fstream;
using std::string;
#define SOME_FIXED_HARDCODED_NUMBER 40980
class MyClass {
public:
std::vector<int8_t> vec;
public:
MyClass(std::vector<int8_t> v){ vec = v; }
};
boost::circular_buffer<MyClass> cb(300);
int waiting = 1;
std::mutex my_mutex;
FILE *out_file;
FILE *in_file;
void foo()
{
while (waiting) {
std::unique_lock<std::mutex> lock(my_mutex);
if (!cb.size() || waiting == 0) {
lock.unlock();
continue;
}
if (!waiting)
break;
MyClass local_buf = cb.front();
cb.pop_front();
fwrite(local_buf.vec.data(), 1, local_buf.vec.size(), out_file);
}
}
int main(int argc, char* argv[])
{
out_file = fopen("output_data.raw", "w");
in_file = fopen("input_data.raw", "w");
std::thread th1(foo);
char *buf = {"abc"};
int counter = 0;
std::vector<int8_t> mem;
mem.insert(mem.end(), buf, buf + strlen(buf));
while (counter < SOME_FIXED_HARDCODED_NUMBER)
{
{
std::unique_lock<std::mutex> lock(my_mutex);
/* if the circular buffer is full then wait for consumer to pull the data */
while (cb.full()) {
lock.unlock();
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
std::unique_lock<std::mutex> lock(my_mutex);
}
cb.push_front(MyClass(mem));
fwrite(mem.data(), 1, mem.size(), in_file);
}
counter++;
}
waiting = 0;
th1.join();
fclose(out_file);
fclose(in_file);
return 0;
}
while (cb.full()) {
lock.unlock();
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
>>> std::unique_lock<std::mutex> lock(my_mutex);
}
The marked unique_lock doesn't do anything as it will go out of scope immediately and unlock the mutex. Hence once you leave the loop the mutex is not locked and you have a racecondition. Instead, you should use lock.lock() to relock the mutex.
There is a few more bugs. You are not waiting for your foo thread to actually drain the buffer. It will stop as soon as the waiting flag is set by the main thread. Also, waiting should be an atomic.
I've implemented thread pooling following the answer of Kerrek SB in this question.
I've implemented MPMC queue for the functions and vector threads for the threads.
Everything worked perfectly, except that I don't know how to terminate the program, in the end if I just do thread.join since the thread is still waiting for more tasks to do, it will not join and the main thread will not continue.
Any idea how to end the program correctly?
For completeness, this is my code:
function_pool.h
#pragma once
#include <queue>
#include <functional>
#include <mutex>
#include <condition_variable>
class Function_pool
{
private:
std::queue<std::function<void()>> m_function_queue;
std::mutex m_lock;
std::condition_variable m_data_condition;
public:
Function_pool();
~Function_pool();
void push(std::function<void()> func);
std::function<void()> pop();
};
function_pool.cpp
#include "function_pool.h"
Function_pool::Function_pool() : m_function_queue(), m_lock(), m_data_condition()
{
}
Function_pool::~Function_pool()
{
}
void Function_pool::push(std::function<void()> func)
{
std::unique_lock<std::mutex> lock(m_lock);
m_function_queue.push(func);
// when we send the notification immediately, the consumer will try to
get the lock , so unlock asap
lock.unlock();
m_data_condition.notify_one();
}
std::function<void()> Function_pool::pop()
{
std::unique_lock<std::mutex> lock(m_lock);
m_data_condition.wait(lock, [this]() {return !m_function_queue.empty();
});
auto func = m_function_queue.front();
m_function_queue.pop();
return func;
// Lock will be released
}
main.cpp
#include "function_pool.h"
#include <string>
#include <iostream>
#include <mutex>
#include <functional>
#include <thread>
#include <vector>
Function_pool func_pool;
void example_function()
{
std::cout << "bla" << std::endl;
}
void infinite_loop_func()
{
while (true)
{
std::function<void()> func = func_pool.pop();
func();
}
}
int main()
{
std::cout << "stating operation" << std::endl;
int num_threads = std::thread::hardware_concurrency();
std::cout << "number of threads = " << num_threads << std::endl;
std::vector<std::thread> thread_pool;
for (int i = 0; i < num_threads; i++)
{
thread_pool.push_back(std::thread(infinite_loop_func));
}
//here we should send our functions
func_pool.push(example_function);
for (int i = 0; i < thread_pool.size(); i++)
{
thread_pool.at(i).join();
}
int i;
std::cin >> i;
}
Your problem is located in infinite_loop_func, which is an infinite loop and by result doesn't terminate. I've read the previous answer which suggests throwing an exception, however, I don't like it since exceptions should not be used for the regular control flow.
The best way to solve this is to explicitly deal with the stop condition. For example:
std::atomic<bool> acceptsFunctions;
Adding this to the function pool allows you to clearly have state and to assert that no new functions being added when you destruct.
std::optional<std::function<void()>> Function_pool::pop()
Returning an empty optional (or function in C++14 and before), allows you to deal with an empty queue. You have to, as condition_variable can do spurious wakeups.
With this, m_data_condition.notify_all() can be used to wake all threads.
Finally we have to fix the infinite loop as it doesn't cover overcommitment and at the same time allows you to execute all functions still in the queue:
while (func_pool.acceptsFunctions || func_pool.containsFunctions())
{
auto f = func_pool.pop();
If (!f)
{
func_pool.m_data_condition.wait_for(1s);
continue;
}
auto &function = *f;
function ();
}
I'll leave it up to you to implement containsFunctions() and clean up the code (infinite_loop_func as member function?) Note that with a counter, you could even deal with background task being spawned.
You can always use a specific exception type to signal to infinite_loop_func that it should return...
class quit_worker_exception: public std::exception {};
Then change infinite_loop_func to...
void infinite_loop_func ()
{
while (true) {
std::function<void()> func = func_pool.pop();
try {
func();
}
catch (quit_worker_exception &ex) {
return;
}
}
}
With the above changes you could then use (in main)...
/*
* Enqueue `thread_pool.size()' function objects whose sole job is
* to throw an instance of `quit_worker_exception' when invoked.
*/
for (int i = 0; i < thread_pool.size(); i++)
func_pool.push([](){ throw quit_worker_exception(); });
/*
* Now just wait for each worker to terminate having received its
* quit_worker_exception.
*/
for (int i = 0; i < thread_pool.size(); i++)
thread_pool.at(i).join();
Each instance of infinite_loop_func will dequeue one function object which, when called, throws a quit_worker_exception causing it to return.
Follwoing [JVApen](https://stackoverflow.com/posts/51382714/revisions) suggestion, I copy my code in case anyone will want a working code:
function_pool.h
#pragma once
#include <queue>
#include <functional>
#include <mutex>
#include <condition_variable>
#include <atomic>
#include <cassert>
class Function_pool
{
private:
std::queue<std::function<void()>> m_function_queue;
std::mutex m_lock;
std::condition_variable m_data_condition;
std::atomic<bool> m_accept_functions;
public:
Function_pool();
~Function_pool();
void push(std::function<void()> func);
void done();
void infinite_loop_func();
};
function_pool.cpp
#include "function_pool.h"
Function_pool::Function_pool() : m_function_queue(), m_lock(), m_data_condition(), m_accept_functions(true)
{
}
Function_pool::~Function_pool()
{
}
void Function_pool::push(std::function<void()> func)
{
std::unique_lock<std::mutex> lock(m_lock);
m_function_queue.push(func);
// when we send the notification immediately, the consumer will try to get the lock , so unlock asap
lock.unlock();
m_data_condition.notify_one();
}
void Function_pool::done()
{
std::unique_lock<std::mutex> lock(m_lock);
m_accept_functions = false;
lock.unlock();
// when we send the notification immediately, the consumer will try to get the lock , so unlock asap
m_data_condition.notify_all();
//notify all waiting threads.
}
void Function_pool::infinite_loop_func()
{
std::function<void()> func;
while (true)
{
{
std::unique_lock<std::mutex> lock(m_lock);
m_data_condition.wait(lock, [this]() {return !m_function_queue.empty() || !m_accept_functions; });
if (!m_accept_functions && m_function_queue.empty())
{
//lock will be release automatically.
//finish the thread loop and let it join in the main thread.
return;
}
func = m_function_queue.front();
m_function_queue.pop();
//release the lock
}
func();
}
}
main.cpp
#include "function_pool.h"
#include <string>
#include <iostream>
#include <mutex>
#include <functional>
#include <thread>
#include <vector>
Function_pool func_pool;
class quit_worker_exception : public std::exception {};
void example_function()
{
std::cout << "bla" << std::endl;
}
int main()
{
std::cout << "stating operation" << std::endl;
int num_threads = std::thread::hardware_concurrency();
std::cout << "number of threads = " << num_threads << std::endl;
std::vector<std::thread> thread_pool;
for (int i = 0; i < num_threads; i++)
{
thread_pool.push_back(std::thread(&Function_pool::infinite_loop_func, &func_pool));
}
//here we should send our functions
for (int i = 0; i < 50; i++)
{
func_pool.push(example_function);
}
func_pool.done();
for (unsigned int i = 0; i < thread_pool.size(); i++)
{
thread_pool.at(i).join();
}
}
I am trying mutex lock with independent threads. The requirement is, I have many threads which will run independently and access/update a common recourse. To ensure that the recourse is updated via a single task, I used mutex. However this is not working.
I have pasted code, a representation of what I am trying to do below:
#include <iostream>
#include <map>
#include <string>
#include <chrono>
#include <thread>
#include <mutex>
#include <unistd.h>
std::mutex mt;
static int iMem = 0;
int maxITr = 1000;
void renum()
{
// Ensure that only 1 task will update the variable
mt.lock();
int tmpMem = iMem;
usleep(100); // Make the system sleep/induce delay
iMem = tmpMem + 1;
mt.unlock();
printf("iMem = %d\n", iMem);
}
int main()
{
for (int i = 0; i < maxITr; i++) {
std::thread mth(renum);
mth.detach(); // Run each task in an independent thread
}
return 0;
}
but this is terminating with the below error:
terminate called after throwing an instance of 'std::system_error'
what(): Resource temporarily unavailable
I want to know if the usage of <thread>.detach() is correct above? If I use .join() it works, but I want each thread to run independently and not wait for the thread to finish.
I also want to know what is the best way to achieve the above logic.
Try this:
int main()
{
std::vector<std::thread> mths;
mths.reserve(maxITr);
for (int i = 0; i < maxITr; i++) {
mths.emplace_back(renum);
}
for (auto& mth : mths) {
mth.join();
}
}
This way, you retain control of the threads (by not calling detach()), and you can join them all at the end, so you know they have completed their tasks.
I've been doing pretty basic stuff with std::thread without any particular reason, simply in order to learn it. I thought that the simple example I created, where few threads are operating on the same data, locking each other before doing so, worked just fine, until I realized that every time I run it the returned value is different, while very close to each other, I am pretty sure they should equal each other. Some of the values I have received:
21.692524
21.699258
21.678871
21.705947
21.685744
Am I doing something wrong or maybe there is underlying reason for that behaviour?
#include <string>
#include <iostream>
#include <thread>
#include <math.h>
#include <time.h>
#include <windows.h>
#include <mutex>
using namespace std;
mutex mtx;
mutex mtx2;
int currentValue = 1;
double suma = 0;
int assignPart() {
mtx.lock();
int localValue = currentValue;
currentValue+=10000000;
mtx.unlock();
return localValue;
}
void calculatePart()
{
int value;
double sumaLokalna = 0;
while(currentValue<1500000000){
value = assignPart();
for(double i=value;i<(value+10000000);i++){
sumaLokalna = sumaLokalna + (1/(i));
}
mtx2.lock();
suma+=sumaLokalna;
mtx2.unlock();
sumaLokalna = 0;
}
}
int main()
{
clock_t startTime = clock();
// Constructs the new thread and runs it. Does not block execution.
thread watek(calculatePart);
thread watek2(calculatePart);
thread watek3(calculatePart);
thread watek4(calculatePart);
while(currentValue<1500000000){
Sleep(100);
printf("%-12d %-12lf \n",currentValue, suma);
}
watek.join();
watek2.join();
watek3.join();
watek4.join();
cout << double( clock() - startTime ) / (double)CLOCKS_PER_SEC<< " seconds." << endl;
//Makes the main thread wait for the new thread to finish execution, therefore blocks its own execution.
}
Your loop
while(currentValue<1500000000){
Sleep(100);
printf("%-12d %-12lf \n",currentValue, suma);
}
is printing intermediate results, but you're not printing the final result.
To print the final result, add the line
printf("%-12d %-12lf \n",currentValue, suma);
after joining the threads.