Port program that uses CreateEvent and WaitForMultipleObjects to Linux - c++

I need to port a multiprocess application that uses the Windows API functions SetEvent, CreateEvent and WaitForMultipleObjects to Linux. I have found many threads concerning this issue, but none of them provided a reasonable solution for my problem.
I have an application that forks into three processes and manages thread workerpool of one process via these Events.
I had multiple solutions to this issue. One was to create FIFO special files on Linux using mkfifo on linux and use a select statement to awaken the threads. The Problem is that this solution will operate differently than WaitForMultipleObjects. For Example if 10 threads of the workerpool will wait for the event and I call SetEvent five times, exactly five workerthreads will wake up and do the work, when using the FIFO variant in Linux, it would wake every thread, that i in the select statement and waiting for data to be put in the fifo. The best way to describe this is that the Windows API kind of works like a global Semaphore with a count of one.
I also thought about using pthreads and condition variables to recreate this and share the variables via shared memory (shm_open and mmap), but I run into the same issue here!
What would be a reasonable way to recreate this behaviour on Linux? I found some solutions doing this inside of a single process, but what about doing this with between multiple processes?
Any ideas are appreciated (Note: I do not expect a full implementation, I just need some more ideas to get myself started with this problem).

You could use a semaphore (sem_init), they work on shared memory. There's also named semaphores (sem_open) if you want to initialize them from different processes. If you need to exchange messages with the workers, e.g. to pass the actual tasks to them, then one way to resolve this is to use POSIX message queues. They are named and work inter-process. Here's a short example. Note that only the first worker thread actually initializes the message queue, the others use the attributes of the existing one. Also, it (might) remain(s) persistent until explicitly removed using mq_unlink, which I skipped here for simplicity.
Receiver with worker threads:
// Link with -lrt -pthread
#include <fcntl.h>
#include <mqueue.h>
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
void *receiver_thread(void *param) {
struct mq_attr mq_attrs = { 0, 10, 254, 0 };
mqd_t mq = mq_open("/myqueue", O_RDONLY | O_CREAT, 00644, &mq_attrs);
if(mq < 0) {
perror("mq_open");
return NULL;
}
char msg_buf[255];
unsigned prio;
while(1) {
ssize_t msg_len = mq_receive(mq, msg_buf, sizeof(msg_buf), &prio);
if(msg_len < 0) {
perror("mq_receive");
break;
}
msg_buf[msg_len] = 0;
printf("[%lu] Received: %s\n", pthread_self(), msg_buf);
sleep(2);
}
}
int main() {
pthread_t workers[5];
for(int i=0; i<5; i++) {
pthread_create(&workers[i], NULL, &receiver_thread, NULL);
}
getchar();
}
Sender:
#include <fcntl.h>
#include <stdio.h>
#include <mqueue.h>
#include <unistd.h>
int main() {
mqd_t mq = mq_open("/myqueue", O_WRONLY);
if(mq < 0) {
perror("mq_open");
}
char msg_buf[255];
unsigned prio;
for(int i=0; i<255; i++) {
int msg_len = sprintf(msg_buf, "Message #%d", i);
mq_send(mq, msg_buf, msg_len, 0);
sleep(1);
}
}

Related

Event Scheduling in C++

I am building an application where in I receive socket data. I need to reply this received data after few seconds(say 8 sec after). So I want to know is there a way to schedule an event which sends the socket data after 8 seconds automatically. I don't like to sleep unnecessarily for 8 seconds in the receiving thread or any other thread. This is what I have written so far for receiving socket data which is a pthread.
long DataSock_fd=socket(AF_INET,SOCK_DGRAM,IPPROTO_UDP);
StSocketAddress.sin_family=AF_INET; //address family
StSocketAddress.sin_addr.s_addr=inet_addr("10.10.10.10"); //load ip address
StSocketAddress.sin_port=htons(1234); //load port number
//bind the above socket to the above mentioned address, if result is less than 0(error in binding)
if(bind(DataSock_fd,(struct sockaddr *)&StSocketAddress,sizeof(StSocketAddress))<0)
{
close(DataSock_fd); //close the socket
perror("error while binding\n");
exit(EXIT_FAILURE); //exit the program
}
char Buff[1024];
long lSize = recvfrom(DataSock_fd,(char *)Buff,sizeof(Buff),0,NULL,NULL);
But I am stuck at scheduling an event that sends data after 8 seconds.
Take a look at this SO answer.
You could use <async> like this to solve your problem:
auto f = std::async(std::launch::async, [] {
std::this_thread::sleep_for(std::chrono::seconds(5));
printf("(5 seconds later) Hello");
});
you can either use boost::sleep, or chrono:: sleep_for or chrono:: sleep_until,
but if you don't want to call sleep, my best suggestion for you is to use std::mutex and lock the thread that receive the information from Time.currenttime -startTime == 8.
Approach-1
Since you don't have a C++11 enabled compiler, and am assuming you are not using frameworks such as Qt/boost etc.. Please check if the following code answer your question. It is a simple async timer implementation using pthreads
Sample code:
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
#include <time.h>
#define TIME_TO_WAIT_FOR_SEND_SECS (8)
#define FAIL_STATUS_CODE (-1)
#define SUCCESS_STATUS_CODE (0)
typedef void (*TimerThreadCbk)(void *);
typedef struct tTimerThreadInitParams
{
int m_DurationSecs; /* Duration of the timer */
TimerThreadCbk m_Callback; /* Timer callback */
void * m_pAppData; /* App data */
}tTimerThreadInitParams;
void PrintCurrTime()
{
time_t timer;
char buffer[26];
struct tm* tm_info;
time(&timer);
tm_info = localtime(&timer);
strftime(buffer, 26, "%Y-%m-%d %H:%M:%S", tm_info);
puts(buffer);
}
void* TimerThreadEntry(void *a_pTimerThreadInitParams)
{
tTimerThreadInitParams *pTimerThreadInitParams = (tTimerThreadInitParams *)a_pTimerThreadInitParams;
if(NULL != pTimerThreadInitParams)
{
/*Do validattion of init params */
sleep(pTimerThreadInitParams->m_DurationSecs);
pTimerThreadInitParams->m_Callback(pTimerThreadInitParams->m_pAppData);
}
else
{
printf("pTimerThreadInitParams is (nil)\n");
}
}
TimerCallbackForSend(void *a_pAppData)
{
(void)a_pAppData;
/* Perform action on timer expiry using a_pAppData */
printf("TimerCallbackForSend trigggered at: ");
PrintCurrTime();
}
int main()
{
/* Timer thread initialization parameters */
pthread_t TimerThread;
tTimerThreadInitParams TimerInitParams = {};
TimerInitParams.m_DurationSecs = TIME_TO_WAIT_FOR_SEND_SECS;
TimerInitParams.m_Callback = (TimerThreadCbk) TimerCallbackForSend;
/* Print current time */
printf("Starting timer at:");
PrintCurrTime();
/* Create timer thread*/
if(pthread_create(&TimerThread, NULL, TimerThreadEntry, &TimerInitParams))
{
fprintf(stderr, "Error creating thread\n");
return FAIL_STATUS_CODE;
}
else
{
printf("TimerThread created\n");
}
/* wait for the second thread to finish */
if(pthread_join(TimerThread, NULL))
{
fprintf(stderr, "Error joining thread\n");
return FAIL_STATUS_CODE;
}
else
{
printf("TimerThread finished\n");
}
return SUCCESS_STATUS_CODE;
}
Sample output:
Starting timer at:2017-08-08 20:55:33
TimerThread created
TimerCallbackForSend trigggered at: 2017-08-08 20:55:41
TimerThread finished
Notes:
This is a scratch custom implementation. You can rename main as ScheduleTimer, which will be a generic API which spawns a thread and invokes the registered callback in its own context.
Just now saw that you don't want to sleep in any of the threads.
Approach-2
Refer C: SIGALRM - alarm to display message every second for SIGALRM. May be in the signal handler you can post an event to the queue which your thread will be monitoring
Sleeping, whether by a C++ wrapper or by the system's nanosleep function -- it cannot be said often enough -- is... wrong. Unless precision and reliability doesn't matter at all, do not sleep. Never.
For anything related to timing, use a timer.
If portability is not a high priority, and since the question is tagged "Linux", a timerfd would be one of the best solutions.
The timerfd can be waited upon with select/poll/epoll while waiting for something to be received, and other stuff (signals, events) at the same time. That's very elegant, and it is quite performant, too.
Admitted, since you are using UDP, there is the temptation to not wait for readiness in the first place but to just have recvfrom block. There is however nothing inherently wrong with waiting for readiness. For moderate loads, the extra syscall doesn't matter, but for ultra-high loads, you might even consider going a step further into non-portable land and use recvmmsg to receive several datagrams in one go as indicated by the number of datagrams reported by epoll (see code example on the recvmmsg man page, which combines recvmmsg with epoll_wait).
With an eventfd, you have everything in one single event loop, in one single thread, reliable and efficient. No trickery needed, no need to be extra smart, no worries about concurrency issues.

Logging with asl layout on mac OS-X multi-threaded project

I'd like to convert all my log messages in my multi-threaded project, to use Apple System Log facility (or asl).
according to the following asl manual - https://developer.apple.com/library/ios/documentation/System/Conceptual/ManPages_iPhoneOS/man3/asl_get.3.html
When logging from multiple threads, each thread must open a separate client handle using asl_open.
For that reason, I've defined asl client per thread to be used in all my log commands. However, in facing some major difficulties in binding asl client to each asl_log command.
1. what if some of my asl log commands reside in a code that is common for
more than one thread - which asl client should i decide use on such message.
2. Even on thread unique code, one should be consistent in choosing the same
asl_client on all log functions on a single thread code scope (this is
not always easy to find in complex projects.).
Is there any easier way to adopt my project logging messages to use asl ?
I'd think about something like binding asl client to thread,
thanks
Ok, so the best solution I've found out so far is by creating a global variable asl client that is thread-specific.
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <asl.h>
#define NUMTHREADS 4
pthread_key_t glob_var_key;
void print_func() //take global var and use it as the aslclient per thread
{
asl_log(*((aslclient*) pthread_getspecific(glob_var_key)),NULL,ASL_LEVEL_NOTICE, "blablabla");
}
void* thread_func(void *arg)
{
aslclient *p = malloc(sizeof(aslclient));
// added tid to message format to distinguish between messages
uint64_t tid;
pthread_threadid_np(NULL, &tid);
char tid_str[20];
sprintf(tid_str, "%llu", tid);
*p = asl_open(tid_str,"Facility",ASL_OPT_STDERR);
pthread_setspecific(glob_var_key, p);
print_func();
sleep(1); // enable ctx switch
print_func();
pthread_setspecific(glob_var_key, NULL);
free(p);
pthread_exit(NULL);
}
int main(void)
{
pthread_t threads[NUMTHREADS];
int i;
pthread_key_create(&glob_var_key,NULL);
for (i=0; i < NUMTHREADS; i++)
pthread_create(&threads[i],NULL,thread_func,NULL);
for (i=0; i < NUMTHREADS; i++)
pthread_join(threads[i], NULL);
}

using libev with multiple threads

I want to use libev with multiple threads for the handling of tcp connections. What I want to is:
The main thread listen on incoming connections, accept the
connections and forward the connection to a workerthread.
I have a pool of workerthreads. The number of threads depends on the
number of cpu's. Each worker-thread has an event loop. The worker-thread listen if I can write on the tcp socket or if
somethings available for reading.
I looked into the documentation of libev and I known this can be done with libev, but I can't find any example how I have to do that.
Does someone has an example?
I think that I have to use the ev_loop_new() api, for the worker-threads and for the main thread I have to use the ev_default_loop() ?
Regards
The following code can be extended to multiple threads
//This program is demo for using pthreads with libev.
//Try using Timeout values as large as 1.0 and as small as 0.000001
//and notice the difference in the output
//(c) 2009 debuguo
//(c) 2013 enthusiasticgeek for stack overflow
//Free to distribute and improve the code. Leave credits intact
#include <ev.h>
#include <stdio.h> // for puts
#include <stdlib.h>
#include <pthread.h>
pthread_mutex_t lock;
double timeout = 0.00001;
ev_timer timeout_watcher;
int timeout_count = 0;
ev_async async_watcher;
int async_count = 0;
struct ev_loop* loop2;
void* loop2thread(void* args)
{
printf("Inside loop 2"); // Here one could initiate another timeout watcher
ev_loop(loop2, 0); // similar to the main loop - call it say timeout_cb1
return NULL;
}
static void async_cb (EV_P_ ev_async *w, int revents)
{
//puts ("async ready");
pthread_mutex_lock(&lock); //Don't forget locking
++async_count;
printf("async = %d, timeout = %d \n", async_count, timeout_count);
pthread_mutex_unlock(&lock); //Don't forget unlocking
}
static void timeout_cb (EV_P_ ev_timer *w, int revents) // Timer callback function
{
//puts ("timeout");
if (ev_async_pending(&async_watcher)==false) { //the event has not yet been processed (or even noted) by the event loop? (i.e. Is it serviced? If yes then proceed to)
ev_async_send(loop2, &async_watcher); //Sends/signals/activates the given ev_async watcher, that is, feeds an EV_ASYNC event on the watcher into the event loop.
}
pthread_mutex_lock(&lock); //Don't forget locking
++timeout_count;
pthread_mutex_unlock(&lock); //Don't forget unlocking
w->repeat = timeout;
ev_timer_again(loop, &timeout_watcher); //Start the timer again.
}
int main (int argc, char** argv)
{
if (argc < 2) {
puts("Timeout value missing.\n./demo <timeout>");
return -1;
}
timeout = atof(argv[1]);
struct ev_loop *loop = EV_DEFAULT; //or ev_default_loop (0);
//Initialize pthread
pthread_mutex_init(&lock, NULL);
pthread_t thread;
// This loop sits in the pthread
loop2 = ev_loop_new(0);
//This block is specifically used pre-empting thread (i.e. temporary interruption and suspension of a task, without asking for its cooperation, with the intention to resume that task later.)
//This takes into account thread safety
ev_async_init(&async_watcher, async_cb);
ev_async_start(loop2, &async_watcher);
pthread_create(&thread, NULL, loop2thread, NULL);
ev_timer_init (&timeout_watcher, timeout_cb, timeout, 0.); // Non repeating timer. The timer starts repeating in the timeout callback function
ev_timer_start (loop, &timeout_watcher);
// now wait for events to arrive
ev_loop(loop, 0);
//Wait on threads for execution
pthread_join(thread, NULL);
pthread_mutex_destroy(&lock);
return 0;
}
Using libev within different threads at the same time is fine as long as each of them runs its own loop[1].
The c++ wrapper in libev (ev++.h) always uses the default loop instead of letting you specify which one you want to use. You should use the C header instead (ev.h) which allows you to specify which loop to use (e.g. ev_io_start takes a pointer to an ev_loop but the ev::io::start doesn't).
You can signal another thread's ev_loop safely through ev_async.
[1]http://doc.dvgu.ru/devel/ev.html#threads_and_coroutines

C++ Running 2 processes at a time

A C++ question on running 2 processes at a time.
I have a client-server model kind of C++ code. My server will fork for every connection from the client. This is a system that also has a reminder module. This reminder module will need to send an email when, let's say, it counts down from 1000 to 0: when it reaches 0, it will perform its code.
But my server is already running in a while(1) loop. How do I invoke this reminder thing together while not affecting the server listening to connections?
Thanks for all help and suggestions.
You are looking for what is commonly know as threads.
Here is an example using Boost.Thread:
#include <iostream>
#include <boost/thread.hpp>
#include <boost/date_time.hpp>
bool worker_running = true;
void workerFunc() {
while (worker_running) {
boost::posix_time::seconds workTime(3);
// do something
boost::this_thread::sleep(workTime);
}
}
int main(int argc, char* argv[])
{
//before your while loop:
boost::thread workerThread(workerFunc);
//while loop here
worker_running = false;
workerThread.join();
return 0;
}

Serial code execution in a multi-threaded program in C++

The question: Is it possible to guarantee code execution can only occur in one thread at a time in a multi-threaded program? (Or something which approximates this)
Specifically: I have a controller M (which is a thread) and threads A, B, C. I would like M to be able to decided who should be allowed to run. When the thread has finished (either finally or temporarily) the control transfers back to M.
Why: Ideally I want A, B and C to execute their code in their own thread while the others are not running. This would enable each thread to keep their instruction pointer and stack while they pause, starting back where they left off when the controller gives them the control back.
What I'm doing now: I've written some code which can actually do this - but I don't like it.
In pseudo-C:
//Controller M
//do some stuff
UnlockMutex(mutex);
do{}while(lockval==0);
LockMutex(mutex);
//continue with other stuff
//Thread A
//The controller currently has the mutex - will release it at UnlockMutex
LockMutex(mutex);
lockval=1;
//do stuff
UnlockMutex(mutex);
The reason why
do{}while(lockval==0);
is required is that when the mutex is unlocked, both A and M will continue. This hack ensures that A won't unlock the mutex before M can lock it again allowing A to retake the lock a second time and run again (it should only run once).
The do-while seems like overkill, but does the job. So my question is, is there a better way?
Assuming you're running on Windows, you might try looking at Fibers. (See eg http://developer.amd.com/Pages/1031200677.aspx or just google "windows fibers".)
I suspect you're really looking for coroutines.
Check for "CriticalSection" in Win32.
C++ 11 uses an other term "lock_guard".
How do I make a critical section with Boost?
http://en.cppreference.com/w/cpp/thread/lock_guard
Your code
do{}while(lockval==0);
will eat up your CPU performance.
I presume your are coding c++ under linux and using pthread API.
Here is the code, not so much robust, but a good point to start. Hope useful to you.
Using "g++ test_controller_thread.cpp -pthread -o test_controller_thread" to make the binary executive.
// 3 threads, one for controller, the other two for worker1 and worker2.
// Only one thread can proceed at any time.
// We use one pthread_mutex_t and two pthread_cond_t to guarantee this.
#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
static pthread_mutex_t g_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t g_controller_cond = PTHREAD_COND_INITIALIZER;
static pthread_cond_t g_worker_cond = PTHREAD_COND_INITIALIZER;
void* controller_func(void *arg) {
printf("entering the controller thread. \n");
// limit the max time the controller can run
int max_run_time = 5;
int run_time = 0;
pthread_mutex_lock(&g_mutex);
while (run_time++ < max_run_time) {
printf("controller is waitting.\n");
pthread_cond_wait(&g_controller_cond, &g_mutex);
printf("controller is woken up.\n");
pthread_cond_signal(&g_worker_cond);
printf("signal worker to wake up.\n");
}
pthread_mutex_unlock(&g_mutex);
}
void* worker_func(void *arg) {
int work_id = *(int*)arg;
printf("worker %d start.\n", work_id);
pthread_mutex_lock(&g_mutex);
while (1) {
printf("worker %d is waitting for controller.\n", work_id);
pthread_cond_wait(&g_worker_cond, &g_mutex);
printf("worker %d is working.\n", work_id);
pthread_cond_signal(&g_controller_cond);
printf("worker %d signal the controller.\n", work_id);
}
pthread_mutex_unlock(&g_mutex);
}
int main() {
pthread_t controller_thread, worker_thread_1, worker_thread_2;
int worker_id_1 = 1;
int worker_id_2 = 2;
pthread_create(&controller_thread, NULL, controller_func, NULL);
pthread_create(&worker_thread_1, NULL, worker_func, &worker_id_1);
pthread_create(&worker_thread_2, NULL, worker_func, &worker_id_2);
sleep(1);
printf("\nsignal the controller to start all the process.\n\n");
pthread_cond_signal(&g_controller_cond);
pthread_join(controller_thread, NULL);
pthread_cancel(worker_thread_1);
pthread_cancel(worker_thread_2);
return 0;
}