I am trying to make a multi threaded function it looks like:
namespace { // Anonymous namespace instead of static functions.
std::mutex log_mutex;
void Background() {
while(IsAlive){
std::queue<std::string> log_records;
{
// Exchange data for minimizing lock time.
std::unique_lock lock(log_mutex);
logs.swap(log_records);
}
if (log_records.empty()) {
Sleep(200);
continue;
}
while(!log_records.empty()){
ShowLog(log_records.front());
log_records.pop();
}
}
}
void Log(std::string log){
std::unique_lock lock(log_mutex);
logs.push(std::move(log));
}
}
I use Sleep to prevent high CPU usages due to continuously looping even if logs are empty. But this has a very visible draw back that it will print the logs in batches. I tried to get over this problem by using conditional variables but in there the problem is if there are too many logs in a short time then the cv is stopped and waked up many times leading to even more CPU usage. Now what can i do to solve this issue?
You can assume there may be many calls to log per second.
I would probably think of using a counting semaphore for this:
The semaphore would keep a count of the number of messages in the logs (initially zero).
Log clients would write a message and increment by one the number of messages by releasing the semaphore.
A log server would do an acquire on the semaphore, blocking until there was any message in the logs, and then decrementing by one the number of messages.
Notice:
Log clients get the logs queue lock, push a message, and only then do the release on the semaphore.
The log server can do the acquire before getting the logs queue lock; this would be possible even if there were more readers. For instance: 1 message in the log queue, server 1 does an acquire, server 2 does an acquire and blocks because semaphore count is 0, server 1 goes on and gets the logs queue lock...
#include <algorithm> // for_each
#include <chrono> // chrono_literasl
#include <future> // async, future
#include <iostream> // cout
#include <mutex> // mutex, unique_lock
#include <queue>
#include <semaphore> // counting_semaphore
#include <string>
#include <thread> // sleep_for
#include <vector>
std::mutex mtx{};
std::queue<std::string> logs{};
std::counting_semaphore c_semaphore{ 0 };
int main()
{
auto log = [](std::string message) {
std::unique_lock lock{ mtx };
logs.push(std::move(message));
c_semaphore.release();
};
auto log_client = [&log]() {
using namespace std::chrono_literals;
static size_t s_id{ 1 };
size_t id{ s_id++ };
for (;;)
{
log(std::to_string(id));
std::this_thread::sleep_for(id * 100ms);
}
};
auto log_server = []() {
for (;;)
{
c_semaphore.acquire();
std::unique_lock lock{ mtx };
std::cout << logs.front() << " ";
logs.pop();
}
};
std::vector<std::future<void>> log_clients(10);
std::for_each(std::begin(log_clients), std::end(log_clients),
[&log_client](auto& lc_fut) {
lc_fut = std::async(std::launch::async, log_client);
});
auto ls_fut{ std::async(std::launch::async, log_server) };
std::for_each(std::begin(log_clients), std::end(log_clients),
[](auto& lc_fut) { lc_fut.wait(); });
ls_fut.wait();
}
Related
Lets say I have one thread that continuously updates a certain object. During the update, the object must be locked for thread safety.
Now the second thread is more of an event kind of operation. If such a thread is spawned, I'd like the running update to finish it's call and then immediately perform the event operation.
What I absolutely want to avoid is a situation where the event thread needs to wait until it gets lucky to be given computation time at a specific time the update thread doesn't lock up the data it needs to access.
Is there any way I could use the threading/mutex tools in c++ to accomplish this? Or should I save the to-be-done operation in an unlocked var and perform the operation on the update thread?
//// System.h
#pragma once
#include <mutex>
#include <iostream>
#include <chrono>
#include <thread>
class System {
private:
int state = 0;
std::mutex mutex;
public:
void update();
void reset(int e);
};
//////// System.cpp
#include "System.h"
void System::update() {
std::lock_guard<std::mutex> guard(mutex);
state++;
std::cout << state << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(1));
}
void System::reset(int e) {
std::lock_guard<std::mutex> guard(mutex);
state = e;
std::cout << state << std::endl;
}
////// ThreadTest.h
#pragma once
#include <iostream>
#include "System.h"
void loop_update(System& system);
void reset_system(System& system);
int main();
////// ThreadTest.cpp
#include "ThreadTest.h"
void loop_update(System& system) {
while (true) system.update();
};
void reset_system(System& system) {
system.reset(0);
};
int main()
{
System system;
std::thread t1 = std::thread(loop_update, std::ref(system));
int reset = 0;
while (true) {
std::this_thread::sleep_for(std::chrono::seconds(10));
std::cout << "Reset" << std::endl;
reset_system(system);
}
}
Example gives following output. You can clearly see a huge delay in the actual update.
1
...
10
Reset
11
...
16
0
1
...
10
Reset
11
...
43
0
1
If I understand you correctly, you have 2 threads using the same mutex. However, you want one thread to get a higher preference than the other to get the actual lock.
As far as I know, there ain't a way to ensure preference using the native tools. You can work around it, if you don't mind the code of both threads knowing about it.
For example:
std::atomic<int> shouldPriorityThreadRun{0};
auto priorityThreadCode = [&shouldPriorityThreadRun](){
++shouldPriorityThreadRun;
auto lock = std::unique_lock{mutex};
doMyStuff();
--shouldPriorityThreadRun;
};
auto backgroundThreadCode = [&shouldPriorityThreadRun](){
while (true)
{
if (shouldPriorityThreadRun == 0)
{
auto lock = std::unique_lock{mutex};
doBackgroundStuff();
}
else
std::this_thread::yield();
}
};
If you have multiple priority threads, those can't have priority over each other.
If you don't like the yield, you could do fancier stuff with std::condition_variable, so you can inform other threads that the mutex is available. However, I believe it's good enough.
it should already work with your current approach.
The mutex is locking concurrent access to your data, so you can lock it within the first thread to update the data.
If the event routine / your second thread comes to execution, it always has to check if the mutex is unlocked. If the mutex is unlocked - and only then, you can lock the mutex and perform the tasks of the second thread.
If I understand your code correctly (i am not a c++ expert), the std::lock_guard<std::mutex> guard(mutex); seems to be locking the mutex the entire time of the update function...
And therefore other threads merely have time to access the mutex.
When the update thread finish the job, it needs to unlock the mutex before entering the sleep state, then the reset thread could have a chance to take the lock without any delay. I also tried running your codes on my machine and observe it's still waiting for the lock. I don't know when it gets lucky to take the lock. I think in this case it's an UB
2
3
4
5
6
7
8
9
10
Reset
11
12
13
14
15
16
17
18...
void System::update() {
mutex.lock();
state++;
std::cout << state << std::endl;
mutex.unlock();
std::this_thread::sleep_for(std::chrono::seconds(1));
}
void System::reset(int e) {
mutex.lock();
state = e;
std::cout << state << std::endl;
mutex.unlock();
}
So in the compilable code below, I'm sending a Query message to be handled by another thread and I want to wait for a response or timeout if it hits a certain timeout. I don't know why the wait_until is missing the signal and hitting the timeout period when it should not be doing that. It only happens if the handler is returning a response REALLY fast. How do you propose I fix the code below?
#include <mutex>
#include <memory>
#include <condition_variable>
#include <atomic>
#include <thread>
#include <iostream>
#include <queue>
#include <zconf.h>
class Question
{
};
class Answer
{
public:
bool isAnswered = false;
};
class Query
{
std::condition_variable _cv;
std::mutex _mutex;
std::atomic_bool _questionAnswered;
std::atomic_bool _questionSet;
std::shared_ptr<Question> _question;
std::shared_ptr<Answer> _answer;
public:
void setQuestion(std::shared_ptr<Question> & question)
{
if(!_questionSet)
{
_question = question;
_questionSet = true;
}
};
void setAnswer(std::shared_ptr<Answer> answer)
{
std::unique_lock<std::mutex> lock(_mutex);
if(!_questionAnswered)
{
// Set the answer and notify the getAnswerWithTimeout() to unlock if holding
_answer = answer;
_questionAnswered = true;
lock.unlock();
_cv.notify_all();
}
};
std::shared_ptr<Answer> getAnswerWithTimeout(uint64_t micros)
{
std::unique_lock<std::mutex> lock(_mutex);
if(!_questionAnswered)
{
auto now = std::chrono::system_clock::now();
// When timeout occurs, lock down this class, set the answer as null, and set error to timeout
if (!_cv.wait_until(lock, now + std::chrono::microseconds(micros), [&]() { return (bool)_questionAnswered; }) )
{
_answer = nullptr;
_questionAnswered = true;
}
}
return _answer;
};
};
void function_to_run(std::shared_ptr<Query> query)
{
// Respond to query and set the answer
auto answer = std::make_shared<Answer>();
answer->isAnswered = true;
// Set the response answer
query->setAnswer(answer);
}
std::queue<std::shared_ptr<Query>> queryHandler;
bool keepRunning = true;
std::mutex queryHandlerMutex;
std::condition_variable queryHandlerCv;
void handleQueryHandler()
{
while (true)
{
std::shared_ptr<Query> query;
{
std::unique_lock<std::mutex> lock(queryHandlerMutex);
queryHandlerCv.wait(lock, [&] { return !keepRunning || !queryHandler.empty(); });
if (!keepRunning) {
return;
}
// Pop off item from queue
query = queryHandler.front();
queryHandler.pop();
}
// Process query with function
function_to_run(query);
}
}
void insertIntoQueryHandler(std::shared_ptr<Query> & query)
{
{
std::unique_lock<std::mutex> lock(queryHandlerMutex);
// Insert into Query Handler
queryHandler.emplace(query);
}
// Notify query handler to start if locked on empty
queryHandlerCv.notify_one();
}
std::shared_ptr<Answer>
ask(std::shared_ptr<Query> query, uint64_t timeoutMicros=0)
{
std::shared_ptr<Answer> answer = nullptr;
// Send Query to be handled by external thread
insertIntoQueryHandler(query);
// Hold for the answer to be returned with timeout period
answer = query->getAnswerWithTimeout(timeoutMicros);
return answer;
}
int main()
{
// Start Up Query Handler thread to handle Queries
std::thread queryHandlerThread(handleQueryHandler);
// Create queries in infinite loop and process
for(int i = 0; i < 1000000; i++)
{
auto question = std::make_shared<Question>();
auto query = std::make_shared<Query>();
query->setQuestion(question);
auto answer = ask(query, 1000);
if(!answer)
{
std::cout << "Query Timed out after 1000us" << std::endl;
}
}
// Stop the thread
{
std::unique_lock<std::mutex> lock(queryHandlerMutex);
keepRunning = false;
}
queryHandlerCv.notify_one();
queryHandlerThread.join();
return 0;
}
As discussed in the comments, the main issue here is the timeout period you're using (1ms), in this interval:
auto now = std::chrono::system_clock::now();
.... another thread may sneak in here ....
if (!_cv.wait_until(lock, now + std::chrono::microseconds(micros), [&]() { return (bool)_questionAnswered; }) )
{
another thread can sneak in and consume a timeslice (e.g. 10ms) and the wait_until would timeout immediately. Furthermore there are reports of unexpected behaviour with wait_until as described here:
std::condition_variable wait_until surprising behaviour
Increasing the timeout to something in the order of several timeslices will fix this. You can also adjust thread priorities.
Personally I advocate polling a condition variable with wait_for which is efficient and also bails in a timely fashion (as opposed to polling a flag and sleeping).
Time slices in non-RTOS systems tend to be in the order of 10ms, so I would not expect such short timeouts to work accurately and predictably in general-purpose systems. See this for an introduction to pre-emptive multitasking:
https://www.geeksforgeeks.org/time-slicing-in-cpu-scheduling/
as well as this:
http://dev.ti.com/tirex/explore/node?node=AL.iEm6ATaD6muScZufjlQ__pTTHBmu__LATEST
As jtbandes points out, it's worth using tools such as Clang's thread sanitiser to check for potential logic races: https://clang.llvm.org/docs/ThreadSanitizer.html
I'm learning multi-thread coding using c++. What I need to do is continuously read word from keyboard, and pass it to a data thread for data processing. I used global variable word[] to pass the data. When word[0] != 0 means a new input from keyboard. And the data thread will set word[0] to 0 once it read the data. It works! But I'm not sure if it safe or not, or there are better ways to do this. Here is my code:
#include <iostream>
#include <thread>
#include <cstdio>
#include <cstring>
using namespace std;
static const int buff_len = 32;
static char* word = new char[buff_len];
static void data_thread () { // thread to handle data
while (1)
{
if (word[0]) { // have a new word
char* w = new char[buff_len];
strcpy(w, word);
cout << "Data processed!\n";
word[0] = 0; // Inform the producer that we consumed the word
}
}
};
static void read_keyboard () {
char * linebuf = new char[buff_len];
thread * worker = new thread( data_thread );
while (1) //enter "end" to terminate the loop
{
if (!std::fgets( linebuf, buff_len, stdin)) // EOF?
return;
linebuf[strcspn(linebuf, "\n")] = '\0'; //remove new line '\n' from the string
word = linebuf; // Pass the word to the worker thread
while (word[0]); // Wait for the worker thread to consume it
}
worker->join(); // Wait for the worker to terminate
}
int main ()
{
read_keyboard();
return 0;
}
The problem with this type of multi threading implementation is busy waiting. The input reader & the data consumer both are busy waiting and wasting the cpu cycles. To overcome this you need Semaphore.
Semaphore s_full(0);
Semaphore s_empty(1);
void data_processor ()
{
while (true) {
// Wait for data availability.
s_full.wait();
// Data is available to you, consume it.
process_data();
// Unblock the data producer.
s_empty.signal();
}
}
void input_reader()
{
while (true) {
// Wait for empty buffer.
s_empty.wait();
// Read data.
read_input_data();
// Unblock data com=nsumer.
s.full.signal();
}
}
In addition this solution will work only for a single data consumer thread. But for multiple data consumer threads you'll need thread safe buffer queue and proper implementation of producer - consumer problem.
See below blog links for additional information to solve this problem:
Thread safe buffer queue:
https://codeistry.wordpress.com/2018/03/08/buffer-queue-handling-in-multithreaded-environment/
Producer - consumer problem:
https://codeistry.wordpress.com/2018/03/09/unordered-producer-consumer/
There are a few problems with your approach:
This method is not scalable. What if you have more than 1 processing thread?
You would need a mutex to synchronise read-write access to the memory stored by word. At the scale of this example, not a big deal. In a "serious" application you might not have the luxury of waiting till you get the data thread stops processing. In that case, you might be tempted to remove the while(word[0]) but that is unsafe.
You fire off a "daemon" thread (not exactly but close enough) to handle your computations. Most of the time the thread is waiting for your input and cannot proceed without it. This is inefficient, and modern C++ gives you a way around it without explicitly handling raw threads using std::async paradigm.
#include <future>
#include <string>
#include <iostream>
static std::string worker(const std::string &input)
{
// assume this is a lengthy operation
return input.substr(1);
}
int main()
{
while (true)
{
std::string input;
std::getline (std::cin, input);
if (input.empty())
break;
std::future<std::string> fut= std::async(std::launch::async, &worker, input);
// Other tasks
// size_t n_stars = count_number_of_stars();
//
std::string result = fut.get(); // wait for the task to complete
printf("Output : %s\n", result.c_str());
}
return 0;
}
Something like this in my opinion is the better approach. std::async will launch a thread (if std::launch::async option is specified) and return a waitable future. The computation will continue in the background, and you can do other work in the main thread. When you need to get the result of your computation, you can get() the result of the future(btw the future can be void too).
Also there are a lot of C-isms in your C++ code. Unless there is a reason to do so, why would you not use std::string?
In modern CPP multithreading, u should be using condition_variable, mutex, and queue to handle this. the mutex prevents mutual reach to the queue and the condition variable makes the reader thread sleep until the writer writes what it write. the following is an example
static void data_thread (std::queue<char> & dataToProcess, std::mutex & mut, std::condition_variable & cv, std::atomic<bool>& finished) { // thread to handle data
std::string readData;
while (!finished)
{
{
std::unique_lock lock{mut};
cv.wait(lock, [&] { return !dataToProcess.empty() || finished; });
if (finished) {
while (!dataToProcess.empty()){
readData += dataToProcess.front();
dataToProcess.pop();
}
}
else{
readData += dataToProcess.front();
dataToProcess.pop();
}
}
std::cout << "\nData processed\n";
}
std::cout << readData;
};
static void read_keyboard () {
std::queue<char> data;
std::condition_variable cv;
std::mutex mut;
std::atomic<bool> finished = false;
std::thread worker = std::thread( data_thread, std::ref(data), std::ref(mut), std::ref(cv), std::ref(finished) );
char temp;
while (true) //enter "end" to terminate the loop
{
if (!std::cin.get(temp)) // EOF?
{
std::cin.clear();
finished = true;
cv.notify_all();
break;
}
{
std::lock_guard lock {mut};
data.push(temp);
}
cv.notify_all();
}
worker.join(); // Wait for the worker to terminate
}
int main ()
{
read_keyboard();
return 0;
}
What you are looking for is a message queue. This needs mutex and condition variable.
Here is one on github (not mine but it popped up when I searched) https://github.com/khuttun/PolyM
and another
https://www.justsoftwaresolutions.co.uk/threading/implementing-a-thread-safe-queue-using-condition-variables.html
I will get told off for posting links, but I am not going to type the entire code here and github's not going anywhere soon
I created a thread pooling to distribute 100 computations between 4 threads.
I cannot understand why the following code stucks after 4 computations. After each computation, the thread must be released and I expect that .joinable() returns false so the program will continue.
Results:
[[[01] calculated
] calculated
2] calculated
[3] calculated
Code:
#include <string>
#include <iostream>
#include <vector>
#include <thread>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <boost/thread/thread.hpp>
#include <cmath>
class AClass
{
public:
void calculation_single(std::vector<double> *s,int index)
{
(*s)[index]=sin(double(index));
std::cout<<"["<<index<<"] calculated \n";
}
void calculation()
{
const uint N_nums=100;
const uint N_threads=4;
std::vector<double> A;
A.assign(N_nums,0.0);
std::vector<std::thread> thread_pool;
for(uint i=0;i<N_threads;i++)
thread_pool.push_back(std::thread());
uint A_index=0;
while(A_index<N_nums)
{
int free_thread=-1;
for(uint i=0;i<N_threads && free_thread<0;i++)
if(!thread_pool[i].joinable())
free_thread=i;
if(free_thread>-1)
{
thread_pool[free_thread]=
std::thread(
&AClass::calculation_single,
this,
&A,
int(A_index));
A_index++;
}
else
{
boost::this_thread::sleep(boost::posix_time::milliseconds(1));
}
}
// wait for tasks to finish
for(std::thread& th : thread_pool)
th.join();
}
};
int main()
{
AClass obj;
obj.calculation();
return 0;
}
A thread is joinable if it isn't empty basically.
A thread with a completed task is not empty.
std::thread bob;
bob is not joinable.
Your threads are. Nothing you do makes them not joinable.
Also, busy waiting is a crappy thread pool.
Create a consumer producer queue, with a pool of threads consuming tasks and an abort method. Feed tasks into the queue with via a packaged task and return a std::future<T>. Don't spawn a new thread per task.
I am currently calling some methods from an external lib file. Is there a way for these methods to callback functions in my application once they are done as these methods might be running in separate threads?
The following diagram shows what I am trying to achieve
I wanted to know what is the best way of sending a message back to the calling application ? Any boost components that might help ?
Update after the edit:
It's not clear what you have. Do you control the thread entry point for the thread started by the external library (this would surprise me)?
Assuming:
the library function accepts a callback
assuming you don't control the source for the library function, not the thread function started by this library function in a background thread
you want to have the callback processed on the original thread
you could have the callback store an record in some kind of queue that you regularly check from the main thread (no busy loops, of course). Use a lock-free queue, or synchronize access to the queue using e.g. a std::mutex.
Update Here's such a queuing version Live on Coliru as well:
#include <thread>
#include <vector>
//////////////////////////////////////////////////////////
// fake external library taking a callback
extern void library_function(int, void(*cb)(int,int));
//////////////////////////////////////////////////////////
// our client code
#include <iostream>
#include <mutex>
void callback_handler(int i, int input)
{
static std::mutex mx;
std::lock_guard<std::mutex> lk(mx);
std::cout << "Callback #" << i << " from task for input " << input << "\n";
}
//////////////////////////////////////////////////////////
// callback queue
#include <deque>
#include <future>
namespace {
using pending_callback = std::packaged_task<void()>;
std::deque<pending_callback> callbacks;
std::mutex callback_mutex;
int process_pending_callbacks() {
std::lock_guard<std::mutex> lk(callback_mutex);
int processed = 0;
while (!callbacks.empty()) {
callbacks.front()();
++processed;
callbacks.pop_front();
}
return processed;
}
void enqueue(pending_callback cb) {
std::lock_guard<std::mutex> lk(callback_mutex);
callbacks.push_back(std::move(cb));
}
}
// this wrapper to "fake" a callback (instead queuing the real
// callback_handler)
void queue_callback(int i, int input)
{
enqueue(pending_callback(std::bind(callback_handler, i, input)));
}
int main()
{
// do something with delayed processing:
library_function(3, queue_callback);
library_function(5, queue_callback);
// wait for completion, periodically checking for pending callbacks
for (
int still_pending = 3 + 5;
still_pending > 0;
std::this_thread::sleep_for(std::chrono::milliseconds(10))) // no busy wait
{
still_pending -= process_pending_callbacks();
}
}
//////////////////////////////////////////////////////////
// somewhere, in another library:
void library_function(int some_input, void(*cb)(int,int))
{
std::thread([=] {
for (int i = 1; i <= some_input; ++i) {
std::this_thread::sleep_for(std::chrono::milliseconds(rand() % 5000)); // TODO abolish rand()
cb(i, some_input);
}
}).detach();
}
Typical output:
Callback #1 from task for input 5
Callback #2 from task for input 5
Callback #1 from task for input 3
Callback #3 from task for input 5
Callback #2 from task for input 3
Callback #4 from task for input 5
Callback #5 from task for input 5
Callback #3 from task for input 3
Note that
output is interspersed for both worker threads
but because the callbacks queue is FIFO, the sequence of callbacks per worker thread is preserved
This is what I thought of, before you edited the question: Live on Coliru
#include <thread>
#include <vector>
extern int library_function(bool);
static std::vector<std::thread> workers; // TODO implement a proper pool
void await_workers()
{
for(auto& th: workers)
if (th.joinable()) th.join();
}
template <typename F, typename C>
void do_with_continuation(F f, C continuation)
{
workers.emplace_back([=] () mutable {
auto result = f();
continuation(result);
});
}
#include <iostream>
#include <mutex>
void callback(int result)
{
static std::mutex mx;
std::lock_guard<std::mutex> lk(mx);
std::cout << "Resulting value from callback " << result << "\n";
}
int main()
{
// do something with delayed processing:
do_with_continuation(std::bind(library_function, false), callback);
do_with_continuation(std::bind(library_function, true), callback);
await_workers();
}
// somewhere, in another library:
#include <chrono>
int library_function(bool some_input)
{
std::this_thread::sleep_for(std::chrono::seconds(some_input? 6 : 3));
return some_input ? 42 : 0;
}
It will always print the output in the order:
Resulting value from callback 0
Resulting value from callback 42
Notes:
make sure you synchronize access to shared state from within such a callback (in this case, std::cout is protected by a lock)
you'd want to make a thread pool, instead of an ever-growing vector of (used) threads