I'd like to convert all my log messages in my multi-threaded project, to use Apple System Log facility (or asl).
according to the following asl manual - https://developer.apple.com/library/ios/documentation/System/Conceptual/ManPages_iPhoneOS/man3/asl_get.3.html
When logging from multiple threads, each thread must open a separate client handle using asl_open.
For that reason, I've defined asl client per thread to be used in all my log commands. However, in facing some major difficulties in binding asl client to each asl_log command.
1. what if some of my asl log commands reside in a code that is common for
more than one thread - which asl client should i decide use on such message.
2. Even on thread unique code, one should be consistent in choosing the same
asl_client on all log functions on a single thread code scope (this is
not always easy to find in complex projects.).
Is there any easier way to adopt my project logging messages to use asl ?
I'd think about something like binding asl client to thread,
thanks
Ok, so the best solution I've found out so far is by creating a global variable asl client that is thread-specific.
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <asl.h>
#define NUMTHREADS 4
pthread_key_t glob_var_key;
void print_func() //take global var and use it as the aslclient per thread
{
asl_log(*((aslclient*) pthread_getspecific(glob_var_key)),NULL,ASL_LEVEL_NOTICE, "blablabla");
}
void* thread_func(void *arg)
{
aslclient *p = malloc(sizeof(aslclient));
// added tid to message format to distinguish between messages
uint64_t tid;
pthread_threadid_np(NULL, &tid);
char tid_str[20];
sprintf(tid_str, "%llu", tid);
*p = asl_open(tid_str,"Facility",ASL_OPT_STDERR);
pthread_setspecific(glob_var_key, p);
print_func();
sleep(1); // enable ctx switch
print_func();
pthread_setspecific(glob_var_key, NULL);
free(p);
pthread_exit(NULL);
}
int main(void)
{
pthread_t threads[NUMTHREADS];
int i;
pthread_key_create(&glob_var_key,NULL);
for (i=0; i < NUMTHREADS; i++)
pthread_create(&threads[i],NULL,thread_func,NULL);
for (i=0; i < NUMTHREADS; i++)
pthread_join(threads[i], NULL);
}
Related
I'm writing a local network scanner on Windows to find online hosts with IP Helper Functions, which is equivalent to nmap -PR but without WinPcap. I know SendARP will block and send arp request 3 times if the remote host doesn't respond, so I use std::aync to create one threads for each host, but the problem is I want to send an ARP request every 20ms so it would not be too much arp packets in a very short time.
#include <iostream>
#include <future>
#include <vector>
#include <winsock2.h>
#include <iphlpapi.h>
#pragma comment(lib, "iphlpapi.lib")
#pragma comment(lib, "ws2_32.lib")
using namespace std;
int main(int argc, char **argv)
{
ULONG MacAddr[2]; /* for 6-byte hardware addresses */
ULONG PhysAddrLen = 6; /* default to length of six bytes */
memset(&MacAddr, 0xff, sizeof (MacAddr));
PhysAddrLen = 6;
IPAddr SrcIp = 0;
IPAddr DestIp = 0;
char buf[64] = {0};
size_t start = time(NULL);
std::vector<std::future<DWORD> > vResults;
for (auto i = 1; i< 255; i++)
{
sprintf(buf, "192.168.1.%d", i);
DestIp = inet_addr(buf);
vResults.push_back(std::async(std::launch::async, std::ref(SendARP), DestIp, SrcIp, MacAddr, &PhysAddrLen));
Sleep(20);
}
for (auto it= vResults.begin(); it != vResults.end(); ++it)
{
if (it->get() == NO_ERROR)
{
std::cout<<"host up\n";
}
}
std::cout<<"time elapsed "<<(time(NULL) - start)<<std::endl;
return 0;
}
At first I can do this by calling Sleep(20) after launching a thread, but once SendARP in these threads re-send ARP requests if no replies from remote host, it's out of my control, and I see many requests in a very short time(<10ms) in Wireshark, so my question is:
Any way to make SendARP asynchronous?
if not, can I control the sent timing of SendARP in threads?
There doesn't seem to be any way to force SendARP to act in a non-blocking manner, it would appear that when a host is unreachable, it will try to re-query several times before giving up.
As for the solution, nothing you want to hear. the MSDN Docs state that there's a newer API that deprecates SendARP called ResolveIpNetEntry2 that can also do the same thing, but it also appears to behave in the same manner.
The struct it receives contains a field called ReachabilityTime.LastUnreachable which is: The time, in milliseconds, that a node assumes a neighbor is unreachable after not having received a reachability confirmation.
However, it does not appear to have any real effect.
The best way to do it, is to use WinPCap or some other driver, there doesn't seem to be a way of solving your problem in userland.
I need to port a multiprocess application that uses the Windows API functions SetEvent, CreateEvent and WaitForMultipleObjects to Linux. I have found many threads concerning this issue, but none of them provided a reasonable solution for my problem.
I have an application that forks into three processes and manages thread workerpool of one process via these Events.
I had multiple solutions to this issue. One was to create FIFO special files on Linux using mkfifo on linux and use a select statement to awaken the threads. The Problem is that this solution will operate differently than WaitForMultipleObjects. For Example if 10 threads of the workerpool will wait for the event and I call SetEvent five times, exactly five workerthreads will wake up and do the work, when using the FIFO variant in Linux, it would wake every thread, that i in the select statement and waiting for data to be put in the fifo. The best way to describe this is that the Windows API kind of works like a global Semaphore with a count of one.
I also thought about using pthreads and condition variables to recreate this and share the variables via shared memory (shm_open and mmap), but I run into the same issue here!
What would be a reasonable way to recreate this behaviour on Linux? I found some solutions doing this inside of a single process, but what about doing this with between multiple processes?
Any ideas are appreciated (Note: I do not expect a full implementation, I just need some more ideas to get myself started with this problem).
You could use a semaphore (sem_init), they work on shared memory. There's also named semaphores (sem_open) if you want to initialize them from different processes. If you need to exchange messages with the workers, e.g. to pass the actual tasks to them, then one way to resolve this is to use POSIX message queues. They are named and work inter-process. Here's a short example. Note that only the first worker thread actually initializes the message queue, the others use the attributes of the existing one. Also, it (might) remain(s) persistent until explicitly removed using mq_unlink, which I skipped here for simplicity.
Receiver with worker threads:
// Link with -lrt -pthread
#include <fcntl.h>
#include <mqueue.h>
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
void *receiver_thread(void *param) {
struct mq_attr mq_attrs = { 0, 10, 254, 0 };
mqd_t mq = mq_open("/myqueue", O_RDONLY | O_CREAT, 00644, &mq_attrs);
if(mq < 0) {
perror("mq_open");
return NULL;
}
char msg_buf[255];
unsigned prio;
while(1) {
ssize_t msg_len = mq_receive(mq, msg_buf, sizeof(msg_buf), &prio);
if(msg_len < 0) {
perror("mq_receive");
break;
}
msg_buf[msg_len] = 0;
printf("[%lu] Received: %s\n", pthread_self(), msg_buf);
sleep(2);
}
}
int main() {
pthread_t workers[5];
for(int i=0; i<5; i++) {
pthread_create(&workers[i], NULL, &receiver_thread, NULL);
}
getchar();
}
Sender:
#include <fcntl.h>
#include <stdio.h>
#include <mqueue.h>
#include <unistd.h>
int main() {
mqd_t mq = mq_open("/myqueue", O_WRONLY);
if(mq < 0) {
perror("mq_open");
}
char msg_buf[255];
unsigned prio;
for(int i=0; i<255; i++) {
int msg_len = sprintf(msg_buf, "Message #%d", i);
mq_send(mq, msg_buf, msg_len, 0);
sleep(1);
}
}
I've searched the internet for a while now and found different solutions but then all don't really work or are to complicated for my use.
I used C++ until 2 years ago so it might be a bit rusty :D
I'm currently writing a program that posts data to an URL. It only posts the data nothing else.
For posting the data I use curl, but it blocks the main thread and while the first post is still running there will be a second post that should start.
In the end there are about 5-6 post operations running at the same time.
Now I want to push the posting with curl into another thread. One thread per post.
The thread should get a string parameter with the content what to push.
I'm currently stuck on this. Tried the WINAPI for windows but that crashes on reading the parameter. (the second thread is still running in my example while the main thread ended (waiting on system("pause")).
It would be nice to have a multi plattform solution, because it will run under windows and linux!
Heres my current code:
#define CURL_STATICLIB
#include <curl/curl.h>
#include <curl/easy.h>
#include <cstdlib>
#include <iostream>
#include <stdio.h>
#include <stdlib.h>
#include <string>
#if defined(WIN32)
#include <windows.h>
#else
//#include <pthread.h>
#endif
using namespace std;
void post(string post) { // Function to post it to url
CURL *curl; // curl object
CURLcode res; // CURLcode object
curl = curl_easy_init(); // init curl
if(curl) { // is curl init
curl_easy_setopt(curl, CURLOPT_URL, "http://10.8.27.101/api.aspx"); // set url
string data = "api=" + post; // concat post data strings
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, data.c_str()); // post data
res = curl_easy_perform(curl); // execute
curl_easy_cleanup(curl); // cleanup
} else {
cerr << "Failed to create curl handle!\n";
}
}
#if defined(WIN32)
DWORD WINAPI thread(LPVOID data) { // WINAPI Thread
string pData = *((string*)data); // convert LPVOID to string [THIS FAILES]
post(pData); // post it with curl
}
#else
// Linux version
#endif
void startThread(string data) { // FUnction to start the thread
string pData = data; // some Test
#if defined(WIN32)
CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)thread, &pData, 0, NULL); // Start a Windows thread with winapi
#else
// Linux version
#endif
}
int main(int argc, char *argv[]) {
// The post data to send
string postData = "test1234567890";
startThread(postData); // Start the thread
system("PAUSE"); // Dont close the console window
return EXIT_SUCCESS;
}
Has anyone a suggestion?
Thanks for the help!
Consider using Boost.Thread or the new C++11 threading facilities (like std::thread etc.).
Some remarks to the code of the initial question:
If staying away from std::thread or boost::thread, use _beginthreadex(..) instead of CreateThread(..) because the latter one can cause resource leaks if used with certain functions of the C runtime.
When using CreateThread(..), a cast to LPTHREAD_START_ROUTINE is not required if the signature of the passed function is correct. So casting it is simply wrong.
The were already some remarks about the lifetime of stack allocated variables and what happens if the address of these are passed to a thread function.
Don't use system("PAUSE") in order to keep the code protable. Instead use the following snippet:
void wait_for_key_press()
{
std::cin.clear();
std::cin.ignore(std::cin.rdbuf()->in_avail());
std::cin.get();
}
Use std::thread for threads. It's a relatively new thing, part of the newest C++11 standard, but it will probably be the most portable way to do threads in the near future.
See how easy it is to make 5 threads that will busy-wait (unless optimized by the compiler):
#include<thread>
#include<vector>
int main()
{
std::vector<std::thread> threads;
for (int i=0; i< 5; i++)
{
threads.push_back(std::thread([] () {
for (long long j=0; j < 1000000000000LL; j++) ;
}));
}
for (auto & thread : threads)
{
thread.join();
}
}
You might want to avoid multi-threading by using libcurl 's multi-operations interface, which enables you to run several concurrent HTTP requests in the same (single) thread.
The question: Is it possible to guarantee code execution can only occur in one thread at a time in a multi-threaded program? (Or something which approximates this)
Specifically: I have a controller M (which is a thread) and threads A, B, C. I would like M to be able to decided who should be allowed to run. When the thread has finished (either finally or temporarily) the control transfers back to M.
Why: Ideally I want A, B and C to execute their code in their own thread while the others are not running. This would enable each thread to keep their instruction pointer and stack while they pause, starting back where they left off when the controller gives them the control back.
What I'm doing now: I've written some code which can actually do this - but I don't like it.
In pseudo-C:
//Controller M
//do some stuff
UnlockMutex(mutex);
do{}while(lockval==0);
LockMutex(mutex);
//continue with other stuff
//Thread A
//The controller currently has the mutex - will release it at UnlockMutex
LockMutex(mutex);
lockval=1;
//do stuff
UnlockMutex(mutex);
The reason why
do{}while(lockval==0);
is required is that when the mutex is unlocked, both A and M will continue. This hack ensures that A won't unlock the mutex before M can lock it again allowing A to retake the lock a second time and run again (it should only run once).
The do-while seems like overkill, but does the job. So my question is, is there a better way?
Assuming you're running on Windows, you might try looking at Fibers. (See eg http://developer.amd.com/Pages/1031200677.aspx or just google "windows fibers".)
I suspect you're really looking for coroutines.
Check for "CriticalSection" in Win32.
C++ 11 uses an other term "lock_guard".
How do I make a critical section with Boost?
http://en.cppreference.com/w/cpp/thread/lock_guard
Your code
do{}while(lockval==0);
will eat up your CPU performance.
I presume your are coding c++ under linux and using pthread API.
Here is the code, not so much robust, but a good point to start. Hope useful to you.
Using "g++ test_controller_thread.cpp -pthread -o test_controller_thread" to make the binary executive.
// 3 threads, one for controller, the other two for worker1 and worker2.
// Only one thread can proceed at any time.
// We use one pthread_mutex_t and two pthread_cond_t to guarantee this.
#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
static pthread_mutex_t g_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t g_controller_cond = PTHREAD_COND_INITIALIZER;
static pthread_cond_t g_worker_cond = PTHREAD_COND_INITIALIZER;
void* controller_func(void *arg) {
printf("entering the controller thread. \n");
// limit the max time the controller can run
int max_run_time = 5;
int run_time = 0;
pthread_mutex_lock(&g_mutex);
while (run_time++ < max_run_time) {
printf("controller is waitting.\n");
pthread_cond_wait(&g_controller_cond, &g_mutex);
printf("controller is woken up.\n");
pthread_cond_signal(&g_worker_cond);
printf("signal worker to wake up.\n");
}
pthread_mutex_unlock(&g_mutex);
}
void* worker_func(void *arg) {
int work_id = *(int*)arg;
printf("worker %d start.\n", work_id);
pthread_mutex_lock(&g_mutex);
while (1) {
printf("worker %d is waitting for controller.\n", work_id);
pthread_cond_wait(&g_worker_cond, &g_mutex);
printf("worker %d is working.\n", work_id);
pthread_cond_signal(&g_controller_cond);
printf("worker %d signal the controller.\n", work_id);
}
pthread_mutex_unlock(&g_mutex);
}
int main() {
pthread_t controller_thread, worker_thread_1, worker_thread_2;
int worker_id_1 = 1;
int worker_id_2 = 2;
pthread_create(&controller_thread, NULL, controller_func, NULL);
pthread_create(&worker_thread_1, NULL, worker_func, &worker_id_1);
pthread_create(&worker_thread_2, NULL, worker_func, &worker_id_2);
sleep(1);
printf("\nsignal the controller to start all the process.\n\n");
pthread_cond_signal(&g_controller_cond);
pthread_join(controller_thread, NULL);
pthread_cancel(worker_thread_1);
pthread_cancel(worker_thread_2);
return 0;
}
We have a qthreads-based workflow engine where worker threads pick up bundles of input as they are placed on a queue, then place their output on another queue for other worker threads to run the next stage; and so on until all the input has been consumed and all the output has been generated.
Typically, several threads will be running the same task and others will be running other tasks at the same time. We want to benchmark performance of these threaded tasks in order to target optimization efforts.
It's easy to get the real (elapsed) time that a given thread, running a given task, has taken. We just look at the difference between the return values of the POSIX times() function at the start and end of the thread's run() procedure. However, I cannot figure out how to get the corresponding user and system time. Getting these from the struct tms that you pass to times() doesn't work, because this structure gives total user and system times of all threads running while the thread in question is active.
Assuming this is on Linux how about getrusage() with RUSAGE_THREAD? Solaris also offers RUSAGE_LWP which is similar and I guess there's probably equivalents for other POSIX-like systems.
Crude example:
#define _GNU_SOURCE
#include <sys/time.h>
#include <sys/resource.h>
#include <stdio.h>
#include <pthread.h>
#include <assert.h>
#include <unistd.h>
struct tinfo {
pthread_t thread;
int id;
struct rusage start;
struct rusage end;
};
static void *
thread_start(void *arg)
{
struct tinfo *inf = arg;
getrusage(RUSAGE_THREAD, &inf->start);
if (inf->id) {
sleep(10);
}
else {
const time_t start = time(NULL);
while (time(NULL) - start < 10); // Waste CPU time!
}
getrusage(RUSAGE_THREAD, &inf->end);
return 0;
}
int main() {
static const int nrthr = 2;
struct tinfo status[nrthr];
for (int i = 0; i < nrthr; ++i) {
status[i].id = i;
const int s = pthread_create(&status[i].thread,
NULL, &thread_start,
&status[i]);
assert(!s);
}
for (int i = 0; i < nrthr; ++i) {
const int s = pthread_join(status[i].thread, NULL);
assert(!s);
// Sub-second timing is available too
printf("Thread %d done: %ld (s) user, %ld (s) system\n", status[i].id,
status[i].end.ru_utime.tv_sec - status[i].start.ru_utime.tv_sec,
status[i].end.ru_stime.tv_sec - status[i].start.ru_stime.tv_sec);
}
}
I think something similar is possible on windows using GetProcessTimes()