I am writing a c++ program with threading and pipes. I am implementing a parallelized algorithm and the idea is that I have a main thread that writes data to child threads. The child thread must read this data, process it, and write back the result to the main thread.
I have stripped down a minimal reproducing, compiling version of the core logic of the communication and commented out the places where I have more code. The program runs and exits without typing out complete. Usually, the last value of i that is printed is between 1 and 9 and the program just terminates without saying anything. I would expect the program to run to completion but I am not getting any errors and the program exits gracefully so I am not sure how to debug.
NOTE: Pipes and Pthreads are mandated from somewhere else and are hard requirements. Please don't suggest a solution to use std::thread or just communicate between threads within the same address space.
#include <iostream>
#include "pthread.h"
#include "unistd.h"
#include <vector>
using namespace std;
void* func (void* args)
{
std::vector<int> v = * (std::vector<int>*)(args);
auto FH = fdopen(v[0], "r");
char buffer[1024];
int buffer_len = 1024;
while (fgets(buffer, buffer_len, FH))
{
std::string x{buffer};
}
// process the result and return it to the parent
return NULL;
}
int main()
{
std::vector<std::vector<int> *> pipes{};
std::vector<pthread_t *> threads{};
for (int i=0; i<20; i++)
{
std::cout<<i<<std::endl;
int fd[2];
if (pipe(fd) < 0)
{
std::cout<<"failed"<<std::endl;
exit(0);
}
int fd2[2];
if (pipe(fd2) < 0)
{
std::cout<<"failed"<<std::endl;
exit(0);
}
std::vector<int> *pipe_info = new std::vector<int>{fd[0], fd[1], fd2[0], fd2[1]};
auto F = fdopen(fd[1], "w");
pthread_t *thread = new pthread_t;
threads.push_back(thread);
pipes.push_back(pipe_info);
pthread_create(thread, NULL, func, (void*) pipe_info);
for (int i=0; i<100; i++)
fprintf(F, "%d", 3);
}
// read the data returned from the child threads
// using fd2 (indices 2,3) in each pipe in pies.
// free all allocated memory
for (auto thread: threads)
{
pthread_join(*thread, NULL);
delete thread;
}
std::cout<<"complete"<<std::endl;
return 0;
}
I cannot reproduce the problem, and the symptoms you describe seem improbabe. On my system the program prints all the numbers, and does not terminate, but hangs.
The reason is that pipe() is not a constructor; fdopen() is not a constructor either. The are c interfaces, and they wouldn't close by virtue of leaving the scope. You have to close fds and FILEs manually. You don't do it, and the threads patiently wait in fgets for more data or EOF. and until main close the writing end of a pipe, there will be no EOF.
Related
I am trying to implement the Producer-Consumer problem operating system using semaphore and pthread. But my output is totally different from expected. Here is my code:
#include<iostream>
#include<pthread.h>
#include<fstream>
#include<unistd.h>
#include<queue>
// define queue size
#define QUEUE_SIZE 5
// declare and initialize semaphore and read/write counter
static int semaphore = 1;
static int counter = 0;
// Queue for saving characters
static std::queue<char> charQueue;
// indicator for end of file
static bool endOfFile = false;
// save arrays
char consumerArray1[100];
char consumerArray2[100];
// function to wait for semaphore
void wait()
{
while(semaphore<=0);
semaphore--;
}
// function to signal the wait function
void signal()
{
semaphore++;
}
void *Producer(void *ptr)
{
int i=0;
std::ifstream input("string.txt");
char temp;
while(input>>temp)
{
wait();
charQueue.push(temp);
//std::cout<<"Producer:\nCounter: "<<counter<<" Semaphore: "<<semaphore<<std::endl;
counter++;
std::cout<<"Procuder Index: "<<i<<std::endl;
i++;
signal();
sleep(2);
}
endOfFile = true;
pthread_exit(NULL);
}
void *Consumer1(void *ptr)
{
std::cout<<"Entered consumer 1:"<<std::endl;
int i = 0;
while(counter<=0);
while(!endOfFile)
{
while(counter<=0);
wait();
//std::cout<<"Consumer1:\nCounter: "<<counter<<" Semaphore: "<<semaphore<<std::endl;
consumerArray1[i] = charQueue.front();
charQueue.pop();
i++;
counter--;
std::cout<<"Consumer1 index:"<<i<<" char: "<<consumerArray1[i]<<std::endl;
signal();
sleep(2);
}
consumerArray1[i] = '\0';
pthread_exit(NULL);
}
void *Consumer2(void *ptr)
{
std::cout<<"Entered consumer 2:"<<std::endl;
int i = 0;
while(counter<=0);
while(!endOfFile)
{
while(counter<=0);
wait();
//std::cout<<"Consumer2:\nCounter: "<<counter<<" Semaphore: "<<semaphore<<std::endl;
consumerArray2[i] = charQueue.front();
charQueue.pop();
i++;
counter--;
std::cout<<"Consumer2 index: "<<i<<" char: "<<consumerArray2[i]<<std::endl;
signal();
sleep(4);
}
consumerArray2[i] = '\0';
pthread_exit(NULL);
}
int main()
{
pthread_t thread[3];
pthread_create(&thread[0],NULL,Producer,NULL);
int rc = pthread_create(&thread[1],NULL,Consumer1,NULL);
if(rc)
{
std::cout<<"Thread not created"<<std::endl;
}
pthread_create(&thread[2],NULL,Consumer2,NULL);
pthread_join(thread[0],NULL);pthread_join(thread[1],NULL);pthread_join(thread[2],NULL);
std::cout<<"First array: "<<consumerArray1<<std::endl;
std::cout<<"Second array: "<<consumerArray2<<std::endl;
pthread_exit(NULL);
}
The problem is my code, in some runs freezes(probably in an infinite loop) after the entire file has been read. And also both of the consumer functions read the same words even though I am popping it out after reading. Also the part of printing the array element that has been read just prints blank. Why are these problems happening? I am new to threads(as in coding using threads, I know theoretical concepts of threads) so please help me with this problem.
The pthreads standard prohibits accessing an object in one thread while another thread is, or might be, modifying it. Your wait and signal functions violate this rule by modifying semaphore (in signal) while a thread calling wait might be accessing it. You do this with counter as well.
If what you were doing in signal and wait were legal, you wouldn't need signal and wait. You could just access the queue directly the same way you access semaphore directly. If the queue needs protection (as I hope you know it does) then semaphore needs protection too and for exactly the same reason.
The compiler is permitted to optimize this code:
while(semaphore<=0);
To this code:
if (semaphore<=0) { while (1); }
Why? Because it knows that no other thread can possibly modify semaphore while this thread could be accessing it since that is prohibited by the standard. Therefore, there is no reason to read more than once.
You need to use actual sempahores and/or locks.
I am trying to read some data from stdin in a separate thread from main thread. Main thread should be able to communicate to this waiting thread by writing to stdin, but when I run the test code (included below) nothing happens except that the message ('do_some_work' in my test code) is printed on the terminal directly instead of being output from the waiting thread.
I have tried a couple of solutions listed on SO but with no success. My code mimics one of the solutions from following SO question, and it works perfectly fine by itself but when coupled with my read_stdin_thread it does not.
Is it possible to write data into own stdin in Linux
#include <unistd.h>
#include <string>
#include <iostream>
#include <sstream>
#include <thread>
bool terminate_read = true;
void readStdin() {
static const int INPUT_BUF_SIZE = 1024;
char buf[INPUT_BUF_SIZE];
while (terminate_read) {
fd_set readfds;
struct timeval tv;
int data;
FD_ZERO(&readfds);
FD_SET(STDIN_FILENO, &readfds);
tv.tv_sec=2;
tv.tv_usec=0;
int ret = select(16, &readfds, 0, 0, &tv);
if (ret == 0) {
continue;
} else if (ret == -1) {
perror("select");
continue;
}
data=FD_ISSET(STDIN_FILENO, &readfds);
if (data>0) {
int bytes = read(STDIN_FILENO,buf,INPUT_BUF_SIZE);
if (bytes == -1) {
perror("input poll: read");
continue;
}
if (bytes) {
std::cout << "Execute: " << buf << std::endl;
if (strncmp(buf, "quit", 4)==0) {
std::cout << "quitting reading from stdin." << std::endl;
break;
}
else {
continue;
}
}
}
}
}
int main() {
std::thread threadReadStdin([] () {
readStdin();
});
usleep(1000000);
std::stringstream msg;
msg << "do_some_work" << std::endl;
auto s = msg.str();
write(STDIN_FILENO, s.c_str(), s.size());
usleep(1000000);
terminate_read = false;
threadReadStdin.join();
return 0;
}
A code snippet illustrating how to write to stdin that in turn is read by threadReadStdin would be extremely helpful.
Thanks much in advance!
Edit:
One thing I forgot to mention here that code within readStdin() is a third party code and any kind of communication that takes place has to be on its terms.
Also, I am pretty easily able to redirect std::cin and std::cout to either fstream or stringstream. Problem is that when I write to redirected cin buffer nothing really appears on the reading thread.
Edit2:
This is a single process application and spawning is not an option.
If you want to use a pipe to communicate between different threads in the same program, you shouldn't try using stdin or stdout. Instead, just use the pipe function to create your own pipe. I'll walk you through doing this step-by-step!
Opening the channel
Let's create a helper function to open the channel using pipe. This function takes two ints by reference - the read end and the write end. It tries opening the pipe, and if it can't, it prints an error.
#include <unistd.h>
#include <cstdio>
#include <thread>
#include <string>
void open_channel(int& read_fd, int& write_fd) {
int vals[2];
int errc = pipe(vals);
if(errc) {
fputs("Bad pipe", stderr);
read_fd = -1;
write_fd = -1;
} else {
read_fd = vals[0];
write_fd = vals[1];
}
}
Writing a message
Next, we define a function to write the message. This function is given as a lambda, so that we can pass it directly to the thread.
auto write_message = [](int write_fd, std::string message) {
ssize_t amnt_written = write(write_fd, message.data(), message.size());
if(amnt_written != message.size()) {
fputs("Bad write", stderr);
}
close(write_fd);
};
Reading a message
We should also make a function to read the message. Reading the message will be done on a different thread. This lambda reads the message 1000 bytes at a type, and prints it to standard out.
auto read_message = [](int read_fd) {
constexpr int buffer_size = 1000;
char buffer[buffer_size + 1];
ssize_t amnt_read;
do {
amnt_read = read(read_fd, &buffer[0], buffer_size);
buffer[amnt_read] = 0;
fwrite(buffer, 1, amnt_read, stdout);
} while(amnt_read > 0);
};
Main method
Finally, we can write the main method. It opens the channel, writes the message on one thread, and reads it on the other thread.
int main() {
int read_fd;
int write_fd;
open_channel(read_fd, write_fd);
std::thread write_thread(
write_message, write_fd, "Hello, world!");
std::thread read_thread(
read_message, read_fd);
write_thread.join();
read_thread.join();
}
It seems like I have stumbled upon the answer with the help of very constructive responses from #Jorge Perez, #Remy Lebeau and #Kamil Cuk. This solution is built upon #Jorge Perez's extremely helpful code. For brevity's sake I am not including the whole code but part comes from the code I posted and a large part comes from #Jorge Perez's code.
What I have done is taken his approach using pipes and replacing STDIN_FILENO by the pipe read fd using dup. Following link was really helpful:
https://en.wikipedia.org/wiki/Dup_(system_call)
I would really appreciate your input on whether this is a hack or a good enough approach/solution given the constraints I have in production environment code.
int main() {
int read_fd;
int write_fd;
open_channel(read_fd, write_fd);
close(STDIN_FILENO);
if(dup(read_fd) == -1)
return -1;
std::thread write_thread(write_message, write_fd, "Whatsup?");
std::thread threadReadStdin([] () {
readStdin();
});
write_thread.join();
threadReadStdin.join();
return 0;
}
Lets say I have two processes (simulated in this example with two threads) in a producer-consumer set up. That is, one process writes data to a file, the other process consumes the data in the file, then clears said file.
The set up I currently have, based on bits and pieces I've thrown together from various resources online, is that I should use a lock file to ensure that only one process can access the data file at a time. The producer acquires the lock, writes to the file, then releases the lock. Meanwhile, the consumer waits for modify events with inotify at which point it acquires the lock, consumes the data, and empties the file.
This seems relatively straightforward, but the part that's tripping me up is that when I empty the file out in my consumer thread, it triggers inotify modify event again, which sets off the whole flow again, and ends with the data file being cleared again, thus repeating forever.
I've tried a few ways to work around this problem, but none of them seem quite right. I'm worried doing this wrong will introduce potential race conditions or I'll end up skipping modify events or something.
Here is my current code:
#include <fstream>
#include <iostream>
#include <string>
#include "pthread.h"
#include "sys/file.h"
#include "sys/inotify.h"
#include "sys/stat.h"
#include "unistd.h"
const char* lock_filename = "./test_lock_file";
const char* data_filename = "./test_data_file";
int AquireLock(char const* lockName) {
mode_t m = umask(0);
int fd = open(lockName, O_RDWR | O_CREAT, 0666);
umask(m);
bool success = false;
if (fd < 0 || flock(fd, LOCK_EX) < 0) {
close(fd);
return -1;
}
return fd;
}
void ReleaseLock(int fd, char const* lockName) {
if (fd < 0) return;
remove(lockName);
close(fd);
}
void* ConsumerThread(void*) {
// Set up inotify.
int file_descriptor = inotify_init();
if (file_descriptor < 0) return nullptr;
int watch_descriptor =
inotify_add_watch(file_descriptor, data_filename, IN_MODIFY);
if (watch_descriptor < 0) return nullptr;
char buf[4096] __attribute__((aligned(__alignof__(inotify_event))));
while (true) {
// Read new events.
const inotify_event* event;
ssize_t numRead = read(file_descriptor, buf, sizeof(buf));
if (numRead <= 0) return nullptr;
// For each event, do stuff.
for (int i = 0; i < numRead; i += sizeof(inotify_event) + event->len) {
event = reinterpret_cast<inotify_event*>(&buf[i]);
// Critical section!
int fd = AquireLock(lock_filename);
// Read from the file.
std::string line;
std::ifstream data_file(data_filename);
if (data_file.is_open()) {
while (getline(data_file, line)) {
std::cout << line << std::endl;
}
data_file.close();
// Clear the file by opening then closing without writing to it.
std::ofstream erase_data_file(data_filename);
erase_data_file.close();
std::cout << "file cleared." << std::endl;
}
ReleaseLock(fd, lock_filename);
// Critical section over!
}
}
return nullptr;
}
int main(int argv, char** argc) {
// Set up other thread.
pthread_t thread;
int rc = pthread_create(&thread, NULL, ConsumerThread, nullptr);
if (rc) return rc;
// Producer thread: Periodically write to a file.
while (true) {
sleep(3);
// Critical section!
int fd = AquireLock(lock_filename);
// Write some text to a file
std::ofstream data_file(data_filename);
int counter = 0;
if (data_file.is_open()) {
std::cout << "Writing to file.\n";
data_file << "This is some example data. " << counter++ << "\n";
data_file.close();
}
ReleaseLock(fd, lock_filename);
// Critical section over!
}
pthread_exit(NULL);
return 0;
}
One idea I had was to disable tracking of modify events at the start of the consumer thread's critical section with inotify_rm_watch, then re-add it right before leaving the critical section. This doesn't seem to work though. Even with the events disabled, modify events are still getting triggered and I'm not sure why.
I've also considered just using a boolean to see if there was any file contents while consuming the file, and only clearing the file if it wasn't empty. This felt kind of hacky since it's still doing a second unnecessary iteration of the loop, but if I can't find a better solution I might just go with that. Ideally there would be a way to have only the producer thread's modifications trigger events, while the consumer could have it's own file modifications somehow ignored or disabled, but I'm not sure how to achieve that effect.
I'm confusing about the performance of my code, when dealing with single thread it only using 13s, but it's will consume 80s. I don't know whether the vector can only be accessed by one thread at a time, if so it's likely I have to use a struct array to store data instead of vector, could anyone kindly help?
#include <iostream>
#include <stdio.h>
#include <stdlib.h>
#include <vector>
#include <iterator>
#include <string>
#include <ctime>
#include <bangdb/database.h>
#include "SEQ.h"
#define NUM_THREADS 16
using namespace std;
typedef struct _thread_data_t {
std::vector<FDT> *Query;
unsigned long start;
unsigned long end;
connection* conn;
int thread;
} thread_data_t;
void *thr_func(void *arg) {
thread_data_t *data = (thread_data_t *)arg;
std::vector<FDT> *Query = data->Query;
unsigned long start = data->start;
unsigned long end = data->end;
connection* conn = data->conn;
printf("thread %d started %lu -> %lu\n", data->thread, start, end);
for (unsigned long i=start;i<=end ;i++ )
{
FDT *fout = conn->get(&((*Query).at(i)));
if (fout == NULL)
{
//printf("%s\tNULL\n", s);
}
else
{
printf("Thread:%d\t%s\n", data->thread, fout->data);
}
}
pthread_exit(NULL);
}
int main(int argc, char *argv[])
{
if (argc<2)
{
printf("USAGE: ./seq <.txt>\n");
printf("/home/rd/SCRIPTs/12X18610_L5_I052.R1.clean.code.seq\n");
exit(-1);
}
printf("%s\n", argv[1]);
vector<FDT> Query;
FILE* fpin;
if((fpin=fopen(argv[1],"r"))==NULL) {
printf("Can't open Input file %s\n", argv[1]);
return -1;
}
char *key = (char *)malloc(36);
while (fscanf(fpin, "%s", key) != EOF)
{
SEQ * sequence = new SEQ(key);
FDT *fk = new FDT( (void*)sequence, sizeof(*sequence) );
Query.push_back(*fk);
}
unsigned long Querysize = (unsigned long)(Query.size());
std::cout << "myvector stores " << Querysize << " numbers.\n";
//create database, table and connection
database* db = new database((char*)"berrydb");
//get a table, a new one or existing one, walog tells if log is on or off
table* tbl = db->gettable((char*)"hg19", JUSTOPEN);
if(tbl == NULL)
{
printf("ERROR:table NULL error");
exit(-1);
}
//get a new connection
connection* conn = tbl->getconnection();
if(conn == NULL)
{
printf("ERROR:connection NULL error");
exit(-1);
}
cerr<<"begin querying...\n";
time_t begin, end;
double duration;
begin = clock();
unsigned long ThreadDealSize = Querysize/NUM_THREADS;
cerr<<"Querysize:"<<ThreadDealSize<<endl;
pthread_t thr[NUM_THREADS];
int rc;
thread_data_t thr_data[NUM_THREADS];
for (int i=0;i<NUM_THREADS ;i++ )
{
unsigned long ThreadDealStart = ThreadDealSize*i;
unsigned long ThreadDealEnd = ThreadDealSize*(i+1) - 1;
if (i == (NUM_THREADS-1) )
{
ThreadDealEnd = Querysize-1;
}
thr_data[i].conn = conn;
thr_data[i].Query = &Query;
thr_data[i].start = ThreadDealStart;
thr_data[i].end = ThreadDealEnd;
thr_data[i].thread = i;
}
for (int i=0;i<NUM_THREADS ;i++ )
{
if (rc = pthread_create(&thr[i], NULL, thr_func, &thr_data[i]))
{
fprintf(stderr, "error: pthread_create, rc: %d\n", rc);
return EXIT_FAILURE;
}
}
for (int i = 0; i < NUM_THREADS; ++i) {
pthread_join(thr[i], NULL);
}
cerr<<"done\n"<<endl;
end = clock();
duration = double(end - begin) / CLOCKS_PER_SEC;
cerr << "runtime: " << duration << "\n" << endl;
db->closedatabase(OPTIMISTIC);
delete db;
printf("Done\n");
return EXIT_SUCCESS;
}
Like all data structures in standard library, methods of vector are reentrant, but not thread-safe. That means different instances can be accessed by multiple threads independently, but each instance may only be accessed by one thread at a time and you have to ensure it. But since you have separate vector for each thread, that's not your problem.
What is probably your problem is the printf. printf is thread-safe, meaning you can call it from any number of threads at the same time, but at the cost of being wrapped in mutual exclusion internally.
Majority of work in the threaded part of your program is done inside printf. So what probably happens is that all the threads are started and quickly get to the printf, where all but the first will stop. When the printf finishes and releases the mutex, system considers scheduling the threads that were waiting for it. It probably does, so rather slow context switch happens. And repeats after every printf.
How exactly it happens depends on which actual locking primitive is being used, which depends on your operating system and standard library versions. The system should each time wake up only the next sleeper, but many implementations actually wake up all of them. So in addition to the printfs being executed in mostly round-robin fashion, incurring one context switch for each, there may be quite a few additional spurious wake-ups in which the thread just finds the lock is held and goes back to sleep.
So the lesson from this is that threads don't make things automagically faster. They only help when:
The thread spends most of it's time doing blocking system calls. In things like network servers the threads wait for data from the socket, than from data for response to come from disk and finally for network to accept the response. In such cases, having many threads helps as long as they are mostly independent.
There is just so many threads as there are CPU threads. Currently the usual number is 4 (either quad-core or dual-core with hyper-threading). More threads can't physically run in parallel, so they provide no gain and incur a bit of overhead. 16 threads is thus overkill.
And they never help when they all manipulate the same objects, so they end up spending most of the time waiting for locks anyway. In addition to any of your own objects that you lock, keep in mind that input and output file handles have to be internally locked as well.
Memory allocation also needs to internally synchronize between threads, but modern allocators have separate pools for threads to avoid much of it; if the default allocator proves to be too slow with many threads, there are some specialized ones you can use.
similar c++ code as below is written for an embedded device. the process running on the device crashes upon start. on some other version of the device, crash is not observed. can it be related to the thread argument & thread detach being called. on normal linux desktop environment it does not crash. can anyone pls give their comments.
Thanks in advance.
#include <pthread.h>
#include <iostream>
using namespace std;
#define NUM_THREADS 2
void *PrintHello(void *msg)
{
cout<<(char*)msg<<endl;
while(1)
{
printf("Hello World! It's me, thread !\n");
sleep(2);
}
}
int main (int argc, char *argv[])
{
pthread_t threads[NUM_THREADS];
int rc;
long t;
const char* ch = "hello how r u.i'm passing argument";
for(t=0; t<NUM_THREADS; t++)
{
printf("In main: creating thread %ld\n", t);
rc = pthread_create(&threads[t], NULL, PrintHello, (void *)ch);
pthread_detach(threads[t]);
if (rc)
{
printf("ERROR; return code from pthread_create() is %d\n", rc);
exit(-1);
}
}
return 0;
}
Yes.
The C++ standard library, by default, is not thread-safe... stream objects like cout in particular.
It may or may not have anything to do with the system being embedded. It's possible that the standard library implementation for the desktop system is more thread-safe, or it just happens to be implemented slightly differently, or perhaps you just got [un]lucky that you did not observe any undesired behaviour when testing on the desktop.