Resources are not available in C++ Multi-threading - c++

I am trying to understand why when I try to use threading in writing to a file. It stops writing to this file after sometime. I understand threading but not too much deep.
Here is the sample code I am using:
#include <chrono>
#include <iostream>
#include <stdlib.h>
# include <stdio.h>
#include <thread>
#include <unistd.h>
// http://www.bogotobogo.com/cplusplus/cpptut.php
using namespace std;
float counter = 0;
void *call_from_thread(void *) {
FILE *f = fopen64("results_text.txt", "a");
if (f == NULL)
{
printf("Error opening file!\n");
exit(1);
}
fprintf(f, " this is some text to be written in the file to see the max size of writting using threads \n");
fclose(f);
printf("|I am in the thread \n" );
usleep(50);
return NULL;
}
int main() {
while (true) {
std::thread t([] {call_from_thread(NULL);});
counter++;
// To let the main thread continue running and detach from the last created thread, the thread must have finished execution before recalling it once again.
// if the thread sleeps more than the main thread, then after some time the program will run out of resources and the program crash
// to test this case: try to put 5000 in the usleep inside the thread
t.detach();
// Will make the main thread wait for the thread till it finishes
//t.join();
printf("|I am alive %f \n", counter);
usleep(100);
}
return 0;
}
I am using "detach" function to conserve the resources on my computer, but still after running some while I get the following error:
terminate called after throwing an instance of 'std::system_error'
what(): Resource temporarily unavailable
Any help is greatly appreciated.

Related

terminate called without an active exception when calling pthread_exit in segmentation fault handler

How are you?
I am going to fix the segmentation fault in a worker thread on Ubuntu 18.04.
My code is the following.
#include <thread>
#include <signal.h>
#include <string.h>
#include <pthread.h>
#include <opencv2/opencv.hpp>
void sigsegv_handler(int signum, siginfo_t *info, void *data)
{
printf("The thread was crashed\n");
pthread_exit(NULL);
}
void sleep_ms(int milliseconds)
{
#ifdef WIN32
Sleep(milliseconds);
#elif _POSIX_C_SOURCE >= 199309L
struct timespec ts;
ts.tv_sec = milliseconds / 1000;
ts.tv_nsec = (milliseconds % 1000) * 1000000;
nanosleep(&ts, NULL);
#else
usleep(milliseconds * 1000);
#endif
}
void thread_func(int i)
{
if(i == 3)
{
int *p = 0;
*p = 10;
}
printf("A new thread ran successfully\n");
}
int main()
{
/* Set SIGSEGV handler. */
struct sigaction handler;
sigemptyset(&handler.sa_mask);
handler.sa_sigaction = &sigsegv_handler;
handler.sa_flags = SA_SIGINFO;
if (sigaction(SIGSEGV, &handler, NULL) == -1)
fprintf(stderr, "Cannot set SIGSEGV handler: %s.\n", strerror(errno));
int i = 0;
while(1)
{
std::thread writer_thread(thread_func, i);
writer_thread.detach();
sleep_ms(1000);
printf("%d\n", i++);
}
return 0;
}
The code works well.
The output of this code are following.
A new thread ran successfully
0
A new thread ran successfully
1
A new thread ran successfully
2
The thread was crashed
3
A new thread ran successfully
4
A new thread ran successfully
5
A new thread ran successfully
6
A new thread ran successfully
7
But if I change the function "thread_func" as the following, the program is crashed.
void thread_func(int i)
{
if(i == 3)
{
int *p = 0;
*p = 10;
}
cv::Mat img(100, 100, CV_8UC3); // newly inserted
cv::resize(img, img, cv::Size(200, 200)); //newly inserted
printf("A new thread ran successfully\n");
}
The error messages are the following.
A new thread ran successfully
0
A new thread ran successfully
1
A new thread ran successfully
2
The thread was crashed
terminate called without an active exception
Aborted (core dumped)
Of course, I am sure there is no issue in OpenCV module.
Could u help me to fix this issue?
Thanks
The simple answer is you can't do this:
void sigsegv_handler(int signum, siginfo_t *info, void *data)
{
printf("The thread was crashed\n");
pthread_exit(NULL);
}
First, per 7.1.4 Use of library functions, paragraph 4 of the C 11 standard:
The functions in the standard library are not guaranteed to be reentrant and may modify objects with static or thread storage duration.
Or, as summarized by footnote 188:
Thus, a signal handler cannot, in general, call standard library functions.
So, absent specific guarantees from your platform about what functions you can safely call from a signal handler, you can not make any function calls from within a signal handler.
But since you are calling pthread_exit(), assuming you're using a POSIX system, POSIX does provide some guarantees about what functions you can call, termed "async-signal-safe, at https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04_03. The Linux-specific list can be found at https://man7.org/linux/man-pages/man7/signal-safety.7.html
Note that neither printf() nor pthread_exit() are on either list.
Calling printf() from within a SIGSEGV signal handler is going to be dangerous - most implementations of printf() will use some form of malloc()/free(), and SIGSEGV is often a result of a malloc()/new/free()/delete operation encountering that corrupted heap. Heap operations tend to happen under a lock of some sort to protect against simultaneous modification of heap state, so calling printf() in a SIGSEGV handler of all things creates a huge deadlock risk.
And pthread_exit() will also cause huge problems - it's not only trying to change process state in the process's address space, it's trying to make changes to the process state in kernel space. From within a signal handler, that's simply not going to work.

How to stop the thread execution in C++

I created one thread in my main program, thread execution has to stop once the main program will terminate. I am using reader.join(); to terminate the thread execution. But it is not stopping the execution.
I tried with below-mentioned code, I am using thread.join(); function, but it is failed to terminate a thread. And after the main program also my thread is kept executing.
#include <algorithm>
#include <array>
#include <atomic>
#include <mutex>
#include <queue>
#include <cstdint>
#include <thread>
#include <vector>
using namespace std;
using namespace std::chrono;
typedef pair<int, Mat> pairImage;
class PairComp {
public:
bool operator()(const pairImage& n1, const pairImage& n2) const
{
if (n1.first == n2.first)
return n1.first > n2.first;
return n1.first > n2.first;
}
};
int main(int argc, char* argv[])
{
mutex mtxQueueInput;
queue<pairImage> queueInput;
int total = 0;
atomic<bool> bReading(true);
thread reader([&]() {
int idxInputImage = 0;
while (true) {
Mat img = imread("img_folder/");
mtxQueueInput.lock();
queueInput.push(make_pair(idxInputImage++, img));
if (queueInput.size() >= 100) {
mtxQueueInput.unlock();
cout << "[Warning]input queue size is " << queueInput.size();
// Sleep for a moment
sleep(2);
}
else {
mtxQueueInput.unlock();
}
}
bReading.store(false);
});
while (true) {
pair<int, Mat> pairIndexImage;
mtxQueueInput.lock();
if (queueInput.empty()) {
mtxQueueInput.unlock();
if (bReading.load())
continue;
else
break;
}
else {
// Get an image from input queue
pairIndexImage = queueInput.front();
queueInput.pop();
}
mtxQueueInput.unlock();
cv::Mat frame = pairIndexImage.second;
cv::rectangle(frame, cv::Rect{ 100, 100, 100, 100 }, 0xff);
}
cv::imshow("out_image", frame);
waitKey(1);
if (total++ == 200)
break;
if (reader.joinable()) {
reader.join();
}
return 0;
}
thread.join() does not cause the thread to terminate, it waits until the thread ends. It's the responsibility of the thread to end its execution, for example by periodically checking for a certain condition, like a flag.
You already have an atomic flag bReading, which appears to cause the thread to exit.
if (queueInput.empty()) {
mtxQueueInput.unlock();
if (bReading.load())
continue;
else
break; // thread will exit when queue is empty and bReading == false
So all you need is to set bReading = false in the outer thread before calling thread.join().
bReading = false;
reader.join();
Note that bReading.store(false); inside your thread will have no effect.
Note: you don't need to call atomic.load() and atomic.store(), you can just use them in your code, which will call load() and store() implicitly.
I'm not aware of an built in possibility to stop a thread. Since you have a endless-loop embedded in your thread, it won't stop at any time.
std::thread::join does not terminate your thread. You have to implement something to end your loop, when you demand it.
A bool variable you set false when the thread has to exit. e.g. while(run) or something like that; for simplicity you could also use a std::atomic<bool>
A signaling variable you check. std::condition_variable
What you do at the moment is, you wait in your main-thread that your thread terminates. Since std::thread::join does't terminate your thread, your main-thread will execute forever.
NOTE: When you choose to implement the bool solution. You should protect this bool with an mutex or something alike.
Thanks for the comment. As I don't want to point everyone to boost, but you mentioned it. Find information here.
The problem is not with join which (btw) is not meant to be used to stop or terminate a thread.
The function that your thread is executing contains a while(true) which will never terminate, because it can only sleep and unlock the lock, nothing else.
This means that bReading.store will never be called and as a consequence in the main thread loop you will always go though this branch of the is
if (bReading.load())
continue;
meaning that also the main will execute forever.
std::join is used to wait from a thread that another thread has completed its work. when you do thread1.join() from the main thread what happens is that main will wait until thread1 has completed its execution before executing any other instruction.

Setting timeout for c/c++ function call

Suppose my main function calls an external function veryslow()
int main(){... veryslow();..}
Now I would like to the invocation part of very_slow in main, so that veryslow terminates if it runs out of a time bound. Something like this
int main(){... call_with_timeout(veryslow, 0.1);...}
What is a simple way to achieve that? My OS is Ubuntu 16.04.
You can call this function in a new thread, and set a timeout to terminate the thread, it will end this function call.
A POSIX example would be:
#include <stdio.h>
#include <unistd.h>
#include <pthread.h>
#include <signal.h>
pthread_t tid;
// Your very slow function, it will finish running after 5 seconds, and print Exit message.
// But if we terminate the thread in 3 seconds, Exit message will not print.
void * veryslow(void *arg)
{
fprintf(stdout, "Enter veryslow...\n");
sleep(5);
fprintf(stdout, "Exit veryslow...\n");
return nullptr;
}
void alarm_handler(int a)
{
fprintf(stdout, "Enter alarm_handler...\n");
pthread_cancel(tid); // terminate thread
}
int main()
{
pthread_create(&tid, nullptr, veryslow, nullptr);
signal(SIGALRM, alarm_handler);
alarm(3); // Run alarm_handler after 3 seconds, and terminate thread in it
pthread_join(tid, nullptr); // Wait for thread finish
return 0;
}
You can use future with timeout.
std::future<int> future = std::async(std::launch::async, [](){
veryslow();
});
std::future_status status;
status = future.wait_for(std::chrono::milliseconds(100));
if (status == std::future_status::timeout) {
// verySlow() is not complete.
} else if (status == std::future_status::ready) {
// verySlow() is complete.
// Get result from future (if there's a need)
auto ret = future.get();
}
Note that there's no built-in way to cancel an async task. You will have to implement that inside verySlow itself.
See here for more:
http://en.cppreference.com/w/cpp/thread/future/wait_for
i would pass a pointer to an interface into the function and ask for one back. with this i would enable two way communication to perform all necessary tasks--including timeout and timeout notification.

Using sigwait with std::thread and pipe

Consider a parallel program that consists of a number of worker threads. These threads have a poll-loop on some file descriptors. The program is supposed to run until ctrl-c is hit / the process receives a SIGINT. The program should never wake up unnecessarily.
I have devised the following combination of sigwait, std::thread, pipe and pthread_sigmask. Note that in the actual application, there are more file descriptors, hence I am not using atomics for shutting down the threads.
#include <thread>
#include <iostream>
#include <cstdlib>
#include <csignal>
extern "C" {
#include <unistd.h>
#include <fcntl.h>
#include <pthread.h>
#include <poll.h>
}
int fds[2];
void thread_run() {
struct pollfd pfd = {fds[0], POLLIN, 0};
int ret = poll(&pfd, 1, -1);
if (ret != 1) std::terminate();
if (!pfd.revents & POLLIN) std::abort();
}
int main()
{
int ret = pipe(fds);
if (ret) std::abort();
sigset_t ss;
sigemptyset(&ss);
sigaddset(&ss, SIGINT);
ret = pthread_sigmask(SIG_BLOCK, &ss, NULL);
if (ret) std::abort();
std::thread t(thread_run);
int sig;
ret = sigwait(&ss, &sig);
if (ret) std::abort();
char b = 0;
ret = write(fds[1], &b, 1);
if (ret != 1) std::abort();
t.join();
close(fds[0]);
close(fds[1]);
}
The program appears to work without any issues.
Is this approach conforming or am I overlooking any caveats?
Are there any specific error cases that might occur in regular operation
and can be handled more gracefully?
Would the program be still correct if i swap std::thread-creation and pthread_sigmask?
This is a standard recommended approach and it works well. See examples section in pthread_sigmask.
Cannot spot any.
It would not be correct. Most signals are process-specific, which means they get delivered to any thread in the process that does not block that signal. Hence, that signal must be blocked in all threads but the one that handles the signal.
You may like to use std::abort call for unexpected situations. std::terminate is called by the C++ runtime when exception handling fails.

Port program that uses CreateEvent and WaitForMultipleObjects to Linux

I need to port a multiprocess application that uses the Windows API functions SetEvent, CreateEvent and WaitForMultipleObjects to Linux. I have found many threads concerning this issue, but none of them provided a reasonable solution for my problem.
I have an application that forks into three processes and manages thread workerpool of one process via these Events.
I had multiple solutions to this issue. One was to create FIFO special files on Linux using mkfifo on linux and use a select statement to awaken the threads. The Problem is that this solution will operate differently than WaitForMultipleObjects. For Example if 10 threads of the workerpool will wait for the event and I call SetEvent five times, exactly five workerthreads will wake up and do the work, when using the FIFO variant in Linux, it would wake every thread, that i in the select statement and waiting for data to be put in the fifo. The best way to describe this is that the Windows API kind of works like a global Semaphore with a count of one.
I also thought about using pthreads and condition variables to recreate this and share the variables via shared memory (shm_open and mmap), but I run into the same issue here!
What would be a reasonable way to recreate this behaviour on Linux? I found some solutions doing this inside of a single process, but what about doing this with between multiple processes?
Any ideas are appreciated (Note: I do not expect a full implementation, I just need some more ideas to get myself started with this problem).
You could use a semaphore (sem_init), they work on shared memory. There's also named semaphores (sem_open) if you want to initialize them from different processes. If you need to exchange messages with the workers, e.g. to pass the actual tasks to them, then one way to resolve this is to use POSIX message queues. They are named and work inter-process. Here's a short example. Note that only the first worker thread actually initializes the message queue, the others use the attributes of the existing one. Also, it (might) remain(s) persistent until explicitly removed using mq_unlink, which I skipped here for simplicity.
Receiver with worker threads:
// Link with -lrt -pthread
#include <fcntl.h>
#include <mqueue.h>
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
void *receiver_thread(void *param) {
struct mq_attr mq_attrs = { 0, 10, 254, 0 };
mqd_t mq = mq_open("/myqueue", O_RDONLY | O_CREAT, 00644, &mq_attrs);
if(mq < 0) {
perror("mq_open");
return NULL;
}
char msg_buf[255];
unsigned prio;
while(1) {
ssize_t msg_len = mq_receive(mq, msg_buf, sizeof(msg_buf), &prio);
if(msg_len < 0) {
perror("mq_receive");
break;
}
msg_buf[msg_len] = 0;
printf("[%lu] Received: %s\n", pthread_self(), msg_buf);
sleep(2);
}
}
int main() {
pthread_t workers[5];
for(int i=0; i<5; i++) {
pthread_create(&workers[i], NULL, &receiver_thread, NULL);
}
getchar();
}
Sender:
#include <fcntl.h>
#include <stdio.h>
#include <mqueue.h>
#include <unistd.h>
int main() {
mqd_t mq = mq_open("/myqueue", O_WRONLY);
if(mq < 0) {
perror("mq_open");
}
char msg_buf[255];
unsigned prio;
for(int i=0; i<255; i++) {
int msg_len = sprintf(msg_buf, "Message #%d", i);
mq_send(mq, msg_buf, msg_len, 0);
sleep(1);
}
}