I'm trying to have pthreads run multiple instances of a function at once, to increase runtime speed and efficiency. My code is supposed to spawn threads and keep them open for whenever there is more items in the queue. Then those threads are supposed to do 'something'. The code is supposed to ask to "continue?" when there are no more items in the queue, and if I type "yes", then items should be added to the queue and the threads should continue doing 'something'. This is what I have so far,
# include <iostream>
# include <string>
# include <pthread.h>
# include <queue>
using namespace std;
# define NUM_THREADS 100
int main ( );
queue<int> testQueue;
void *checkEmpty(void* arg);
void *playQueue(void* arg);
void matrix_exponential_test01 ( );
void matrix_exponential_test02 ( );
pthread_mutex_t queueLock;
pthread_cond_t queue_cv;
int main()
{
pthread_t threads[NUM_THREADS+1];
pthread_mutex_init(&queueLock, NULL);
pthread_cond_init (&queue_cv, NULL);
for( int i=0; i < NUM_THREADS; i++ )
{
pthread_create(&threads[i], NULL, playQueue, (void*)NULL);
}
string cont = "yes";
do
{
cout<<"Continue? ";
getline(cin, cont);
pthread_mutex_lock (&queueLock);
for(int z=0; z<10; z++)
{
testQueue.push(1);
}
pthread_mutex_unlock (&queueLock);
}while(cont.compare("yes"));
pthread_mutex_destroy(&queueLock);
pthread_cond_destroy(&queue_cv);
pthread_exit(NULL);
return 0;
}
void* checkEmpty(void* arg)
{
while(true)
{
pthread_mutex_lock (&queueLock);
if(!testQueue.empty()){
pthread_cond_signal(&queue_cv);}
pthread_mutex_unlock (&queueLock);
}
pthread_exit(NULL);
}
void* playQueue(void* arg)
{
while(true)
{
pthread_cond_wait(&queue_cv, &queueLock);
pthread_mutex_lock (&queueLock);
if(!testQueue.empty())
{
testQueue.pop();
cout<<testQueue.size()<<endl;
}
pthread_mutex_unlock (&queueLock);
}
pthread_exit(NULL);
}
So my issue lies with the fact that the code goes into deadlock, and I cant figure out where the issue occurs. I'm no veteran with multithreading so its very easy for me to make a mistake here. I am also running this on Windows.
You have two issues :
The condition variable queue_cv is never signaled. You can signal it with pthread_cond_signal after having pushed elements in the queue : pthread_cond_signal(&queue_cv);
In playQueue, you try to acquire the lock after returning from pthread_cond_wait : since your mutex is not reentrant, this is undefined behavior (this is likely the source of your deadlock). Just remove the pthread_mutex_lock (&queueLock);
Note:
I'm not sure what is it's true purpose, but the checkEmpty() method is never called
Related
I am trying to implement the Producer-Consumer problem operating system using semaphore and pthread. But my output is totally different from expected. Here is my code:
#include<iostream>
#include<pthread.h>
#include<fstream>
#include<unistd.h>
#include<queue>
// define queue size
#define QUEUE_SIZE 5
// declare and initialize semaphore and read/write counter
static int semaphore = 1;
static int counter = 0;
// Queue for saving characters
static std::queue<char> charQueue;
// indicator for end of file
static bool endOfFile = false;
// save arrays
char consumerArray1[100];
char consumerArray2[100];
// function to wait for semaphore
void wait()
{
while(semaphore<=0);
semaphore--;
}
// function to signal the wait function
void signal()
{
semaphore++;
}
void *Producer(void *ptr)
{
int i=0;
std::ifstream input("string.txt");
char temp;
while(input>>temp)
{
wait();
charQueue.push(temp);
//std::cout<<"Producer:\nCounter: "<<counter<<" Semaphore: "<<semaphore<<std::endl;
counter++;
std::cout<<"Procuder Index: "<<i<<std::endl;
i++;
signal();
sleep(2);
}
endOfFile = true;
pthread_exit(NULL);
}
void *Consumer1(void *ptr)
{
std::cout<<"Entered consumer 1:"<<std::endl;
int i = 0;
while(counter<=0);
while(!endOfFile)
{
while(counter<=0);
wait();
//std::cout<<"Consumer1:\nCounter: "<<counter<<" Semaphore: "<<semaphore<<std::endl;
consumerArray1[i] = charQueue.front();
charQueue.pop();
i++;
counter--;
std::cout<<"Consumer1 index:"<<i<<" char: "<<consumerArray1[i]<<std::endl;
signal();
sleep(2);
}
consumerArray1[i] = '\0';
pthread_exit(NULL);
}
void *Consumer2(void *ptr)
{
std::cout<<"Entered consumer 2:"<<std::endl;
int i = 0;
while(counter<=0);
while(!endOfFile)
{
while(counter<=0);
wait();
//std::cout<<"Consumer2:\nCounter: "<<counter<<" Semaphore: "<<semaphore<<std::endl;
consumerArray2[i] = charQueue.front();
charQueue.pop();
i++;
counter--;
std::cout<<"Consumer2 index: "<<i<<" char: "<<consumerArray2[i]<<std::endl;
signal();
sleep(4);
}
consumerArray2[i] = '\0';
pthread_exit(NULL);
}
int main()
{
pthread_t thread[3];
pthread_create(&thread[0],NULL,Producer,NULL);
int rc = pthread_create(&thread[1],NULL,Consumer1,NULL);
if(rc)
{
std::cout<<"Thread not created"<<std::endl;
}
pthread_create(&thread[2],NULL,Consumer2,NULL);
pthread_join(thread[0],NULL);pthread_join(thread[1],NULL);pthread_join(thread[2],NULL);
std::cout<<"First array: "<<consumerArray1<<std::endl;
std::cout<<"Second array: "<<consumerArray2<<std::endl;
pthread_exit(NULL);
}
The problem is my code, in some runs freezes(probably in an infinite loop) after the entire file has been read. And also both of the consumer functions read the same words even though I am popping it out after reading. Also the part of printing the array element that has been read just prints blank. Why are these problems happening? I am new to threads(as in coding using threads, I know theoretical concepts of threads) so please help me with this problem.
The pthreads standard prohibits accessing an object in one thread while another thread is, or might be, modifying it. Your wait and signal functions violate this rule by modifying semaphore (in signal) while a thread calling wait might be accessing it. You do this with counter as well.
If what you were doing in signal and wait were legal, you wouldn't need signal and wait. You could just access the queue directly the same way you access semaphore directly. If the queue needs protection (as I hope you know it does) then semaphore needs protection too and for exactly the same reason.
The compiler is permitted to optimize this code:
while(semaphore<=0);
To this code:
if (semaphore<=0) { while (1); }
Why? Because it knows that no other thread can possibly modify semaphore while this thread could be accessing it since that is prohibited by the standard. Therefore, there is no reason to read more than once.
You need to use actual sempahores and/or locks.
I need feedback on my code for following statement, am I on right path?
Problem statement:
a. Implement a semaphore class that has a private int and three public methods: init, wait and signal. The wait and signal methods should behave as expected from a semaphore and must use Peterson's N process algorithm in their implementation.
b. Write a program that creates 5 threads that concurrently update the value of a shared integer and use an object of semaphore class created in part a) to ensure the correctness of the concurrent updates.
Here is my working program:
#include <iostream>
#include <pthread.h>
using namespace std;
pthread_mutex_t mid; //muted id
int shared=0; //global shared variable
class semaphore {
int counter;
public:
semaphore(){
}
void init(){
counter=1; //initialise counter 1 to get first thread access
}
void wait(){
pthread_mutex_lock(&mid); //lock the mutex here
while(1){
if(counter>0){ //check for counter value
counter--; //decrement counter
break; //break the loop
}
}
pthread_mutex_unlock(&mid); //unlock mutex here
}
void signal(){
pthread_mutex_lock(&mid); //lock the mutex here
counter++; //increment counter
pthread_mutex_unlock(&mid); //unlock mutex here
}
};
semaphore sm;
void* fun(void* id)
{
sm.wait(); //call semaphore wait
shared++; //increment shared variable
cout<<"Inside thread "<<shared<<endl;
sm.signal(); //call signal to semaphore
}
int main() {
pthread_t id[5]; //thread ids for 5 threads
sm.init();
int i;
for(i=0;i<5;i++) //create 5 threads
pthread_create(&id[i],NULL,fun,NULL);
for(i=0;i<5;i++)
pthread_join(id[i],NULL); //join 5 threads to complete their task
cout<<"Outside thread "<<shared<<endl;//final value of shared variable
return 0;
}
You need to release the mutex while spinning in the wait loop.
The test happens to work because the threads very likely run their functions start to finish before there is any context switch, and hence each one finishes before the next one even starts. So you have no contention over the semaphore. If you did, they'd get stuck with one waiter spinning with the mutex held, preventing anyone from accessing the counter and hence release the spinner.
Here's an example that works (though it may still have an initialization race that causes it to sporadically not launch correctly). It looks more complicated, mainly because it uses the gcc built-in atomic operations. These are needed whenever you have more than a single core, since each core has its own cache. Declaring the counters 'volatile' only helps with compiler optimization - for what is effectively SMP, cache consistency requires cross-processor cache invalidation, which means special processor instructions need to be used. You can try replacing them with e.g. counter++ and counter-- (and same for 'shared') - and observe how on a multi-core CPU it won't work. (For more details on the gcc atomic ops, see https://gcc.gnu.org/onlinedocs/gcc-4.8.2/gcc/_005f_005fatomic-Builtins.html)
#include <stdio.h>
#include <pthread.h>
#include <unistd.h>
#include <stdint.h>
class semaphore {
pthread_mutex_t lock;
int32_t counter;
public:
semaphore() {
init();
}
void init() {
counter = 1; //initialise counter 1 to get first access
}
void spinwait() {
while (true) {
// Spin, waiting until we see a positive counter
while (__atomic_load_n(&counter, __ATOMIC_SEQ_CST) <= 0)
;
pthread_mutex_lock(&lock);
if (__atomic_load_n(&counter, __ATOMIC_SEQ_CST) <= 0) {
// Someone else stole the count from under us or it was
// a fluke - keep trying
pthread_mutex_unlock(&lock);
continue;
}
// It's ours
__atomic_fetch_add(&counter, -1, __ATOMIC_SEQ_CST);
pthread_mutex_unlock(&lock);
return;
}
}
void signal() {
pthread_mutex_lock(&lock); //lock the mutex here
__atomic_fetch_add(&counter, 1, __ATOMIC_SEQ_CST);
pthread_mutex_unlock(&lock); //unlock mutex here
}
};
enum {
NUM_TEST_THREADS = 5,
NUM_BANGS = 1000
};
// Making semaphore sm volatile would be complicated, because the
// pthread_mutex library calls don't expect volatile arguments.
int shared = 0; // Global shared variable
semaphore sm; // Semaphore protecting shared variable
volatile int num_workers = 0; // So we can wait until we have N threads
void* fun(void* id)
{
usleep(100000); // 0.1s. Encourage context switch.
const int worker = (intptr_t)id + 1;
printf("Worker %d ready\n", worker);
// Spin, waiting for all workers to be in a runnable state. These printouts
// could be out of order.
++num_workers;
while (num_workers < NUM_TEST_THREADS)
;
// Go!
// Bang on the semaphore. Odd workers increment, even decrement.
if (worker & 1) {
for (int n = 0; n < NUM_BANGS; ++n) {
sm.spinwait();
__atomic_fetch_add(&shared, 1, __ATOMIC_SEQ_CST);
sm.signal();
}
} else {
for (int n = 0; n < NUM_BANGS; ++n) {
sm.spinwait();
__atomic_fetch_add(&shared, -1, __ATOMIC_SEQ_CST);
sm.signal();
}
}
printf("Worker %d done\n", worker);
return NULL;
}
int main() {
pthread_t id[NUM_TEST_THREADS]; //thread ids
// create test worker threads
for(int i = 0; i < NUM_TEST_THREADS; i++)
pthread_create(&id[i], NULL, fun, (void*)((intptr_t)(i)));
// join threads to complete their task
for(int i = 0; i < NUM_TEST_THREADS; i++)
pthread_join(id[i], NULL);
//final value of shared variable. For an odd number of
// workers this is the loop count, NUM_BANGS
printf("Test done. Final value: %d\n", shared);
const int expected = (NUM_TEST_THREADS & 1) ? NUM_BANGS : 0;
if (shared == expected) {
puts("PASS");
} else {
printf("Value expected was: %d\nFAIL\n", expected);
}
return 0;
}
I am new to conditional variables and get deadlock if not using pthread_cond_broadcast().
#include <iostream>
#include <pthread.h>
pthread_mutex_t m_mut = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t cv = PTHREAD_COND_INITIALIZER;
bool ready = false;
void* print_id (void *ptr )
{
pthread_mutex_lock(&m_mut);
while (!ready) pthread_cond_wait(&cv, &m_mut);
int id = *((int*) ptr);
std::cout << "thread " << id << '\n';
pthread_mutex_unlock(&m_mut);
pthread_exit(0);
return NULL;
}
condition is changed here!
void go() {
pthread_mutex_lock(&m_mut);
ready = true;
pthread_mutex_unlock(&m_mut);
pthread_cond_signal(&cv);
}
It can work if I change the last line of go() to pthread_cond_broadcast(&cv);
int main ()
{
pthread_t threads[10];
// spawn 10 threads:
for (int i=0; i<10; i++)
pthread_create(&threads[i], NULL, print_id, (void *) new int(i));
go();
for (int i=0; i<10; i++) pthread_join(threads[i], NULL);
pthread_mutex_destroy(&m_mut);
pthread_cond_destroy(&cv);
return 0;
}
The expected answer (arbitrary order) is
thread 0
....
thread 9
However, on my machine (ubuntu), it prints nothing.
Could anyone tell me the reason? Thanks.
From the manual page (with my emphasis):
pthread_cond_signal restarts one of the threads that are waiting on the condition variable cond. If no threads are waiting on cond, nothing happens. If several threads are waiting on cond, exactly one is restarted, but it is not specified which.
pthread_cond_broadcast restarts all the threads that are waiting on the condition variable cond. Nothing happens if no threads are waiting on cond.
Each of your ten threads is waiting on the same condition. You only call go() once - that's from main(). This calls pthread_cond_signal, which will only signal one of the threads (an arbitrary one). All the others will still be waiting, and hence the pthread_join hangs as they won't terminate. When you switch it to pthread_cond_broadcast, all of the threads are triggered.
I'm creating 9 threads using something like this (all threads will process infinity loop)
void printStr();
thread func_thread(printStr);
void printStr() {
while (true) {
cout << "1\n";
this_thread::sleep_for(chrono::seconds(1));
}
}
I also create 10th thread to control them. How would I stop or kill any of this 9 threads from my 10th? Or suggest another mechanism please.
You can use, for example, atomic boolean:
#include <thread>
#include <iostream>
#include <vector>
#include <atomic>
using namespace std;
std::atomic<bool> run(true);
void foo()
{
while(run.load(memory_order_relaxed))
{
cout << "foo" << endl;
this_thread::sleep_for(chrono::seconds(1));
}
}
int main()
{
vector<thread> v;
for(int i = 0; i < 9; ++i)
v.push_back(std::thread(foo));
run.store(false, memory_order_relaxed);
for(auto& th : v)
th.join();
return 0;
}
EDIT (in response of your comment): you can also use a mutual variable, protected by a mutex.
#include <thread>
#include <iostream>
#include <vector>
#include <mutex>
using namespace std;
void foo(mutex& m, bool& b)
{
while(1)
{
cout << "foo" << endl;
this_thread::sleep_for(chrono::seconds(1));
lock_guard<mutex> l(m);
if(!b)
break;
}
}
void bar(mutex& m, bool& b)
{
lock_guard<mutex> l(m);
b = false;
}
int main()
{
vector<thread> v;
bool b = true;
mutex m;
for(int i = 0; i < 9; ++i)
v.push_back(thread(foo, ref(m), ref(b)));
v.push_back(thread(bar, ref(m), ref(b)));
for(auto& th : v)
th.join();
return 0;
}
It is never appropriate to kill a thread directly, you should instead send a signal to the thread to tell it to stop by itself. This will allow it to clean up and finish properly.
The mechanism you use is up to you and depends on the situation. It can be an event or a state checked periodically from within the thread.
std::thread objects are non - interruptible. You will have to use another thread library like boost or pthreads to accomplish your task. Please do note that killing threads is dangerous operation.
To illustrate how to approach this problem in pthread using cond_wait and cond_signal,In the main section you could create another thread called monitor thread that will keep waiting on a signal from one of the 9 thread.
pthread_mutex_t monMutex;////mutex
pthread_cond_t condMon;////condition variable
Creating threads:
pthread_t *threads = (pthread_t*) malloc (9* sizeof(pthread_t));
for (int t=0; t < 9;t++)
{
argPtr[t].threadId=t;
KillAll=false;
rc = pthread_create(&threads[t], NULL, &(launchInThread), (void *)&argPtr[t]);
if (rc){
printf("ERROR; return code from pthread_create() is %d\n", rc);
exit(-1);
}
}
creating monitor thread:
monitorThreadarg.threadArray=threads;//pass reference of thread array to monitor thread
monitorThreadarg.count=9;
pthread_t monitor_thread;
rc= pthread_create(&monitor_thread,NULL,&monitorHadle,(void * )(&monitorThreadArg));
if (rc){
printf("ERROR; return code from pthread_create() is %d\n", rc);
exit(-1);
}
then wait on 9 threads and monitor thread:
for (s=0; s < 9;s++)
{
pthread_join(threads[s], &status);
}
pthread_cond_signal(&condMon);// if all threads finished successfully then signal monitor thread too
pthread_join(monitor_thread, &status);
cout << "joined with monitor thread"<<endl;
The monitor function would be something like this:
void* monitorHadle(void* threadArray)
{
pthread_t* temp =static_cast<monitorThreadArg*> (threadArray)->threadArray;
int number =static_cast<monitorThreadArg*> (threadArray)->count;
pthread_mutex_lock(&monMutex);
mFlag=1;//check so that monitor threads has initialised
pthread_cond_wait(&condMon,&monMutex);// wait for signal
pthread_mutex_unlock(&monMutex);
void * status;
if (KillAll==true)
{
printf("kill all \n");
for (int i=0;i<number;i++)
{
pthread_cancel(temp[i]);
}
}
}
the function what will be launched over 9 threads should be something like this:
void launchInThread( void *data)
{
pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
while(1)
{
try
{
throw("exception whenever your criteria is met");
}
catch (string x)
{
cout << "exception form !! "<< pthread_self() <<endl;
KillAll=true;
while(!mFlag);//wait till monitor thread has initialised
pthread_mutex_lock(&monMutex);
pthread_cond_signal(&condMon);//signail monitor thread
pthread_mutex_unlock(&monMutex);
pthread_exit((void*) 0);
}
}
}
Please note that if you dont't put :
thread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
after launching your thread then your threads wouldn't terminate on thread_cancel call.
It is necessary that you clean up up all the data before you cancel a thread.
I wrote a multithreading program like this,
#include <queue>
#include <cstdio>
#include <cstdlib>
#include <pthread.h>
#include <unistd.h>
using namespace std;
pthread_mutex_t mu = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t ec = PTHREAD_COND_INITIALIZER;
pthread_cond_t fc = PTHREAD_COND_INITIALIZER;
queue<int> qu;
const int N = 2;
void *producer(void *arg)
{
while(1) {
pthread_mutex_lock(&mu);
int tmp = rand();
qu.push(tmp);
pthread_cond_signal(&ec);
if ((int) qu.size() > N) {
pthread_cond_wait(&fc, &mu);
}
pthread_mutex_unlock(&mu);
}
}
void *consumer(void *arg)
{
while(1) {
pthread_mutex_lock(&mu);
if ((int) qu.size() < 1) {
pthread_cond_wait(&ec, &mu);
}
int tmp = qu.front();
qu.pop();
if ((int) qu.size() <= N) {
pthread_cond_signal(&fc);
}
pthread_mutex_unlock(&mu);
//sleep(1);
}
}
int main()
{
pthread_t tid;
pthread_create(&tid, NULL, producer, NULL);
for (int i = 0; i < N; i++) {
pthread_t tid;
pthread_create(&tid, NULL, consumer, NULL);
}
printf("all created\n");
sleep(600);
}
When the qu.size() is geater than N, producer should stop producing, and when it's less than N, producer resumes producing.
The weired problem is, if I remove the sleep(1); in consumer, the program will run into segmentation fault, if I keep sleep(1);, the program runs ok.
Why? Does it mean the consumer consumes too fast?
Spurious wakeup might be the reason. Your thread will proceed if the condition is true, but if your thread proceeds you can't assume the condition is true.
Spurious wakeups from the pthread_cond_timedwait() or pthread_cond_wait() functions may occur. Since the return from pthread_cond_timedwait() or pthread_cond_wait() does not imply anything about the value of this predicate, the predicate should be re-evaluated upon such return.
So for example
if (qu.size() == 0) {
pthread_cond_wait(&ec, &mu);
}
should become
while (qu.size() == 0) {
pthread_cond_wait(&ec, &mu);
}
If you keep the sleep(1) call and the whole thing does not crash you're just lucky :)
Try initializing the mutex explicitly using the pthread_mutex_init() otherwise your pthread_mutex_lock() calls seem to fail.
From the docs:
Errors
The pthread_mutex_lock() and pthread_mutex_trylock()
functions may fail if:
EINVAL: The value specified by mutex
does not refer to an initialized mutex object.