c++ newbie - pthread providing it's state to another pthread - c++

I am a newbie in c++ working on what should be a pretty basic file read and then process data functionality and I keep getting stuck on being able to at least provide a "state" of on thread to another so that data could be consumed. This just maybe something really basic that I am overlooking - could use some insight into using pthreads in c++.
Bellow is some basic extracted code that functions OK, reads the file and provides data to be processed. Another thread that will process the data needs to know the state of this one.
What would be the best strategy? I am attempting to request the state of the thread through a function from another thread but receive incorrect response.
Reader::Reader(){
_threadId = 1;
_msg = NONE; // enum NONE, READ, STOP
active = false;
pthread_mutex_init(&_mutex, 0);
}
Reader::~Reader(){
pthread_mutex_destroy(&_mutex);
}
void Reader::read(){
_msg = READ;
active = true;
pthread_create(&_threadId, 0, work, this);
}
void * Reader::work(void *myselfreader){
Reader * reader = (Reader*)myselfreader;
reader->loop();
pthread_exit(0);
return 0;
}
void Reader::loop(){
while(active){
pthread_mutex_lock(&_mutex);
switch(_msg){
case READ:
// do the reading of the IO file - which works fine
// once done reading the file - the _msg is set to STOP
break;
case STOP:
stopThread();
break;
default:
break;
}
pthread_mutex_unlock(&_mutex);
}
return;
}
void Reader::stopThread(){
active = false;
_msg = ENC_NONE;
pthread_join(_threadId, 0);
}
/*****************/
int Reader::getReaderState(){
// another thread needs to know the state of this thread
//
return _msg // ??
return active // ??
}

It seems that you have race condition somewhere in your code.
You need to protect your _msg variable with mutex. Every time you need to update _msg variable request mutex, update variable and close the mutex. You need to do the same thing for the reading in getReaderState() function - acquire mutex, copy variable into temp, release mutex and return temp variable.
For easy and error-prone usage, you should create getter and setter function for accessing _msg field which will be guarded by the same mutex (and not the one that you are already using):
void setMsg(int msg)
{
pthread_mutex_lock(&msg_mutex);
_msg = msg;
pthread_mutex_unlock(&msg_mutex);
}
int getMsg()
{
int tmp;
pthread_mutex_lock(&msg_mutex);
tmp = _msg;
pthread_mutex_unlock(&msg_mutex);
return tmp;
}

Related

Second thread is never triggered

I've been struggling with a multithreading issue for a bit. I've written some simple code to try and isolate the issue and I'm not finding it. What's happening is that the first thread is being woken up with data being sent to it, but second one never does. They each have their own condition_variable yet it doesn't seem to matter. Ultimately, what I'm trying to do is have a few long running threads that do a single dedicated task when needed, and staying in a wait state when not needed. And running them each in their own thread is important and a requirement.
Here's the code:
#include <glib.h>
#include <string>
#include <mutex>
#include <condition_variable>
#include <unistd.h>
#define NUM_THREADS 2
bool DEBUG = true;
pthread_t threads[NUM_THREADS];
std::mutex m_0;
std::mutex m_1;
std::condition_variable cov_0;
std::condition_variable cov_1;
bool dataReady_0 = false;
bool dataReady_1 = false;
bool keepRunning[NUM_THREADS] = { true };
void date_update (guint source_id, const char *json_data) {
if (DEBUG) {
start_threads(2);
sleep(2);
DEBUG = false;
}
g_print("From source id=%d\n", source_id);
switch (source_id) {
case 0:
dataReady_0 = true;
cov_0.notify_one();
break;
case 1:
dataReady_1 = true;
cov_1.notify_one();
break;
}
}
void start_threads (int thread_count) {
int rc;
switch (thread_count) {
case 2:
rc = pthread_create(&threads[1], nullptr, custom_thread_1, nullptr);
if (rc) {
g_print("Error:unable to create thread(1), return code(%d)\n", rc);
}
case 1:
rc = pthread_create(&threads[0], nullptr, custom_thread_0, nullptr);
if (rc) {
g_print("Error:unable to create thread(0), return code(%d)\n", rc);
}
}
}
void *custom_thread_0 (void *pVoid) {
g_print("Created thread for source id=0\n");
while (keepRunning[0]) {
// Wait until date_update() sends data
std::unique_lock<std::mutex> lck(m_0);
cov_0.wait(lck, [&]{return dataReady_0;});
dataReady_0 = false;
g_print("THREAD=0, DATA RECEIVED\n");
lck.unlock();
}
pthread_exit(nullptr);
}
void *custom_thread_1 (void *pVoid) {
g_print("Created thread for source id=1\n");
while (keepRunning[1]) {
// Wait until date_update() sends data
std::unique_lock<std::mutex> lck(m_1);
cov_1.wait(lck, [&]{return dataReady_1;});
dataReady_1 = false;
g_print("THREAD=1, DATA RECEIVED\n");
lck.unlock();
}
pthread_exit(nullptr);
}
Here's the output. As you can see the data_update function gets the "data" from the calling function for both source 0 and source 1, but only thread 0 ever seems to process anything. I'm at a bit of a loss as to the source of the problem.
Sending data for source id=1
From source id=1
Sending data for source id=0
From source id=0
THREAD=0, DATA RECEIVED
Sending data for source id=1
From source id=1
Sending data for source id=0
From source id=0
THREAD=0, DATA RECEIVED
Sending data for source id=1
From source id=1
Sending data for source id=0
From source id=0
THREAD=0, DATA RECEIVED
I'm sure I'm just missing a minor detail somewhere, but I'm fully willing to accept that perhaps I do not understand C/C++ threading correctly.
The 2nd thread is exiting because the keepRunning state flag is false. It's usually a good first step in debugging threads to log the start and exit of all threads.
But you have a much less obvious problem.
It does not appear that the appropriate mutex is held when the value of the condition variable's predicate is changed in date_update().
I'll break that down a bit more.
When cov_0.wait() is called, the predicate used is [&]{return dataReady_0;} (*), and the unique_lock passed is holding the mutex m_0. This means that whenever the value of the predicate might change, the mutex m_0 must be held.
This predicate is quite simple and will change value whenever the global variable dataReady_0 changes value.
In date_update() there is code to change the value of dataReady_0 and the mutex m_0 is not held when doing this. There should be a scoped_lock or unique_lock in the block that changes the global variable's state.
It will still mostly work without this, but you have a race! It will fail eventually!
The condition variable may check and see that the predicate is false, then another thread changes the predicate's value and does a notify, and then the first thread waits on the condition variable. It misses the notify because it was not yet waiting when it was sent. The use of the mutex to prevent the predicate from changing in a way that races with the notification is a critical component of what makes this work.
(*) You don't need the capture [&] here. This lambda could be stateless.
You should initialize all elements of the built-in array:
bool keepRunning[2] = { true, true };

What's the good way to pass data to a thread in c++?

I'm learning multi-thread coding using c++. What I need to do is continuously read word from keyboard, and pass it to a data thread for data processing. I used global variable word[] to pass the data. When word[0] != 0 means a new input from keyboard. And the data thread will set word[0] to 0 once it read the data. It works! But I'm not sure if it safe or not, or there are better ways to do this. Here is my code:
#include <iostream>
#include <thread>
#include <cstdio>
#include <cstring>
using namespace std;
static const int buff_len = 32;
static char* word = new char[buff_len];
static void data_thread () { // thread to handle data
while (1)
{
if (word[0]) { // have a new word
char* w = new char[buff_len];
strcpy(w, word);
cout << "Data processed!\n";
word[0] = 0; // Inform the producer that we consumed the word
}
}
};
static void read_keyboard () {
char * linebuf = new char[buff_len];
thread * worker = new thread( data_thread );
while (1) //enter "end" to terminate the loop
{
if (!std::fgets( linebuf, buff_len, stdin)) // EOF?
return;
linebuf[strcspn(linebuf, "\n")] = '\0'; //remove new line '\n' from the string
word = linebuf; // Pass the word to the worker thread
while (word[0]); // Wait for the worker thread to consume it
}
worker->join(); // Wait for the worker to terminate
}
int main ()
{
read_keyboard();
return 0;
}
The problem with this type of multi threading implementation is busy waiting. The input reader & the data consumer both are busy waiting and wasting the cpu cycles. To overcome this you need Semaphore.
Semaphore s_full(0);
Semaphore s_empty(1);
void data_processor ()
{
while (true) {
// Wait for data availability.
s_full.wait();
// Data is available to you, consume it.
process_data();
// Unblock the data producer.
s_empty.signal();
}
}
void input_reader()
{
while (true) {
// Wait for empty buffer.
s_empty.wait();
// Read data.
read_input_data();
// Unblock data com=nsumer.
s.full.signal();
}
}
In addition this solution will work only for a single data consumer thread. But for multiple data consumer threads you'll need thread safe buffer queue and proper implementation of producer - consumer problem.
See below blog links for additional information to solve this problem:
Thread safe buffer queue:
https://codeistry.wordpress.com/2018/03/08/buffer-queue-handling-in-multithreaded-environment/
Producer - consumer problem:
https://codeistry.wordpress.com/2018/03/09/unordered-producer-consumer/
There are a few problems with your approach:
This method is not scalable. What if you have more than 1 processing thread?
You would need a mutex to synchronise read-write access to the memory stored by word. At the scale of this example, not a big deal. In a "serious" application you might not have the luxury of waiting till you get the data thread stops processing. In that case, you might be tempted to remove the while(word[0]) but that is unsafe.
You fire off a "daemon" thread (not exactly but close enough) to handle your computations. Most of the time the thread is waiting for your input and cannot proceed without it. This is inefficient, and modern C++ gives you a way around it without explicitly handling raw threads using std::async paradigm.
#include <future>
#include <string>
#include <iostream>
static std::string worker(const std::string &input)
{
// assume this is a lengthy operation
return input.substr(1);
}
int main()
{
while (true)
{
std::string input;
std::getline (std::cin, input);
if (input.empty())
break;
std::future<std::string> fut= std::async(std::launch::async, &worker, input);
// Other tasks
// size_t n_stars = count_number_of_stars();
//
std::string result = fut.get(); // wait for the task to complete
printf("Output : %s\n", result.c_str());
}
return 0;
}
Something like this in my opinion is the better approach. std::async will launch a thread (if std::launch::async option is specified) and return a waitable future. The computation will continue in the background, and you can do other work in the main thread. When you need to get the result of your computation, you can get() the result of the future(btw the future can be void too).
Also there are a lot of C-isms in your C++ code. Unless there is a reason to do so, why would you not use std::string?
In modern CPP multithreading, u should be using condition_variable, mutex, and queue to handle this. the mutex prevents mutual reach to the queue and the condition variable makes the reader thread sleep until the writer writes what it write. the following is an example
static void data_thread (std::queue<char> & dataToProcess, std::mutex & mut, std::condition_variable & cv, std::atomic<bool>& finished) { // thread to handle data
std::string readData;
while (!finished)
{
{
std::unique_lock lock{mut};
cv.wait(lock, [&] { return !dataToProcess.empty() || finished; });
if (finished) {
while (!dataToProcess.empty()){
readData += dataToProcess.front();
dataToProcess.pop();
}
}
else{
readData += dataToProcess.front();
dataToProcess.pop();
}
}
std::cout << "\nData processed\n";
}
std::cout << readData;
};
static void read_keyboard () {
std::queue<char> data;
std::condition_variable cv;
std::mutex mut;
std::atomic<bool> finished = false;
std::thread worker = std::thread( data_thread, std::ref(data), std::ref(mut), std::ref(cv), std::ref(finished) );
char temp;
while (true) //enter "end" to terminate the loop
{
if (!std::cin.get(temp)) // EOF?
{
std::cin.clear();
finished = true;
cv.notify_all();
break;
}
{
std::lock_guard lock {mut};
data.push(temp);
}
cv.notify_all();
}
worker.join(); // Wait for the worker to terminate
}
int main ()
{
read_keyboard();
return 0;
}
What you are looking for is a message queue. This needs mutex and condition variable.
Here is one on github (not mine but it popped up when I searched) https://github.com/khuttun/PolyM
and another
https://www.justsoftwaresolutions.co.uk/threading/implementing-a-thread-safe-queue-using-condition-variables.html
I will get told off for posting links, but I am not going to type the entire code here and github's not going anywhere soon

how do i lock access to a bool with a mutex?

solved!: im copying the instance of Map in the new Thread an dont use the reference.
im learning how to use multiple threads. For this im programing a little game where i want the game to run in the main thread and the next chunk of the level shall be loaded in another thread. for this i set up a mutex around a vector to tell the loading thread what to load next. inside this mutex i also have a boolean to tell the thread when tu terminate.
initialising thread in Map::Map()
pending_orders_mutex = SDL_CreateMutex();
can_process_order = SDL_CreateCond();
chunk_loader_thread = SDL_CreateThread(Map::chunk_loader,"chunk_loader_thread",(void*)this);
loading thread
int Map::chunk_loader(void * data)
{
Map map = *(Map*)data;
bool kill_this_thread = false;
Chunk_Order actual_order;
actual_order.load_graphics = false;
actual_order.x = 0;
actual_order.y = 0;
while (!kill_this_thread)
{
SDL_LockMutex(map.pending_orders_mutex); // lock mutex
printf("3-kill_chunk_loader_thread: %d\n", map.kill_chunk_loader_thread);
kill_this_thread = map.kill_chunk_loader_thread;
printf("4-kill_chunk_loader_thread: %d\n", map.kill_chunk_loader_thread);
if (!kill_this_thread)
{
if (map.pending_orders.size())
{
actual_order = map.pending_orders.back();
map.pending_orders.pop_back();
printf("in thread processing order\n");
}
else
{
printf("in thread waiting for order\n");
SDL_CondWait(map.can_process_order, map.pending_orders_mutex);
}
}
SDL_UnlockMutex(map.pending_orders_mutex); // unlock mutex
//load actual order
}
printf("thread got killed\n");
return 0;
}
killing the thread (main thread)
SDL_LockMutex(pending_orders_mutex); // lock mutex
printf("setting kill command\n");
printf("1-kill_chunk_loader_thread: %d\n", kill_chunk_loader_thread);
kill_chunk_loader_thread = true; // send kill command
printf("2-kill_chunk_loader_thread: %d\n", kill_chunk_loader_thread);
SDL_CondSignal(can_process_order); // signal that order was pushed
SDL_UnlockMutex(pending_orders_mutex); // unlock mutex
SDL_WaitThread(chunk_loader_thread, NULL);
console output
3-kill_chunk_loader_thread: 0
4-kill_chunk_loader_thread: 0
in thread waiting for order
setting kill command
1-kill_chunk_loader_thread: 0
2-kill_chunk_loader_thread: 1
3-kill_chunk_loader_thread: 0
4-kill_chunk_loader_thread: 0
in thread waiting for order
why does the mainthread not change the "kill_chunk_loader_thread" boolean in the loading thread?
First of all, you should try to upload a minimal complete program in the question.
Its look like you set kill_chunk_loader_thread = true
but you didn't set map.kill_chunk_loader_thread = true
the section of map declaration is missig from your question, but I guess you didn't use a reference to the local or global variable, or you just perform struct copy so when you change one struct the other doesn't been affected at all.
EDIT:
Map map = *(Map*)data; copies the map structure (default copy constructor, I guess) so from now on if the source map change the copy won't.
You should keep work with the pointer, like this: Map* pMap = (Map*)data;
and check the pointer like this: kill_this_thread = pMap->kill_chunk_loader_thread; so you read from the source map.

Waiting until another process locks and then unlocks a Win32 mutex

I am trying to tell when a producer process accesses a shared windows mutex. After this happens, I need to lock that same mutex and process the associated data. Is there a build in way in Windows to do this, short of a ridiculous loop?
I know the result of this is doable through creating a custom Windows event in the producer process, but I want to avoid changing this programs code as much as possible.
What I believe will work (in a ridiculously inefficient way) would be this (NOTE: this is not my real code, I know there are like 10 different things very wrong with this; I want to avoid doing anything like this):
#include <Windows.h>
int main() {
HANDLE h = CreateMutex(NULL, 0, "name");
if(!h) return -1;
int locked = 0;
while(true) {
if(locked) {
//can assume it wont be locked longer than a second, but even if it does should work fine
if(WaitForSingleObject(h, 1000) == WAIT_OBJECT_0) {
// do processing...
locked = 0;
ReleaseMutex(h);
}
// oh god this is ugly, and wastes so much CPU...
} else if(!(locked = WaitForSingleObject(h, 0) == WAIT_TIMEOUT)) {
ReleaseMutex(h);
}
}
return 0;
}
If there is an easier way with C++ for whatever reason, my code is actually that. This example was just easier to construct in C.
You will not be able to avoid changing the producer if efficient sharing is needed. Your design is fundamentally flawed for that.
A producer needs to be able to signal a consumer when data is ready to be consumed, and to make sure it does not alter the data while it is busy being consumed. You cannot do that with a single mutex alone.
The best way is to have the producer set an event when data is ready, and have the consumer reset the event when the data has been consumed. Use the mutex only to sync access to the data, not to signal the data's readiness.
#include <Windows.h>
int main()
{
HANDLE readyEvent = CreateEvent(NULL, TRUE, FALSE, "ready");
if (!readyEvent) return -1;
HANDLE mutex = CreateMutex(NULL, FALSE, "name");
if (!mutex) return -1;
while(true)
{
if (WaitForSingleObject(readyEvent, 1000) == WAIT_OBJECT_0)
{
if (WaitForSingleObject(mutex, 1000) == WAIT_OBJECT_0)
{
// process as needed...
ResetEvent(readyEvent);
ReleaseMutex(mutex);
}
}
}
return 0;
}
If you can't change the producer to use an event, then at least add a flag to the data itself. The producer can lock the mutex, update the data and flag, and unlock the mutex. Consumers will then have to periodically lock the mutex, check the flag and read the new data if the flag is set, reset the flag, and unlock the mutex.
#include <Windows.h>
int main()
{
HANDLE mutex = CreateMutex(NULL, FALSE, "name");
if (!mutex) return -1;
while(true)
{
if (WaitForSingleObject(mutex, 1000) == WAIT_OBJECT_0)
{
if (ready)
{
// process as needed...
ready = false;
}
ReleaseMutex(mutex);
}
}
return 0;
}
So either way, your logic will have to be tweaked in both the producer and consumer.
Otherwise, if you can't change the producer at all, then you have no choice but to change the consumer alone to simply check the data for changes peridiodically:
#include <Windows.h>
int main()
{
HANDLE mutex = CreateMutex(NULL, 0, "name");
if (!mutex) return -1;
while(true)
{
if (WaitForSingleObject(mutex, 1000) == WAIT_OBJECT_0)
{
// check data for changes
// process new data as needed
// cache results for next time...
ReleaseMutex(mutex);
}
}
return 0;
}
Tricky. I'm going to answer the underlying question: when is the memory written?
This can be observed via a four step solution:
Inject a DLL in the watched process
Add a vectored exception handler for STATUS_GUARD_PAGE_VIOLATION
Set the guard page bit on the 2 MB memory range (finding it could be a challenge)
From the vectored exception handler, inform your process and re-establish the guard bit (it's one-shot)
You may need only a single guard page if the image is always fully rewritten.

Sockets and multithreading

I have an interesting (to me) problem... There are two threads, one for capturing data from std input and sending it through socket to server, and another one which receives data from blocking socket. So, when there's no reply from server, recv() call waits indefenitely, right? But instead of blocking only its calling thread, it blocks the overall process! Why this thing occurs?
boost::mutex nvtMutex;
boost::mutex strMutex;
boost::mutex quitMutex;
bool quit = false;
void *processServerOutput(void *arg)
{
NVT *nvt = (NVT*)arg;
while(1)
{
// Lock the quitMutex before trying to access to quit variable
quitMutex.lock();
if(quit)
{
quitMutex.unlock();
pthread_exit(NULL);
}
else
quitMutex.unlock();
// Receive output from server
nvtMutex.lock();
nvt->receive();
cout << Util::Instance()->iconv("koi8-r", "utf-8", nvt->getOutBuffer());
nvtMutex.unlock();
// Delay
sleep(1);
}
}
void *processUserInput(void *arg)
{
NVT *nvt = (NVT*)arg;
while(1)
{
// Get user's input
//cin.getline(str, 1023);
sleep(3);
strcpy(str, "hello");
// If we type 'quit', exit from thread
if(strcmp(str, "quit") == 0)
{
// Lock quit variable before trying to modify it
quitMutex.lock();
quit = true;
quitMutex.unlock();
// Exit from thread
pthread_exit(NULL);
}
// Send the input to server
nvtMutex.lock();
nvt->writeUserCommand(Util::Instance()->iconv("utf-8", "koi8-r", str));
nvt->send();
nvtMutex.unlock();
}
}
You are holding the nvtMutex inside the call to NVT::recv. Since both threads need to lock the mutex to make it through an iteration, until NVT::recv returns the other thread can't progress.
Without knowing the details of this NVT class, it's impossible to know if you can safely unlock the mutex before calling NVT::recv or if this class does not provide the proper thread safety you need.
If your code is implemented correctly, recv blocks only the thread that invokes it.
If this isn't the case for you, show the minimal code sample that demonstrates the problem.