All, my multithreaded program keeps getting interrupted and "Alarm clock" is being written to console.
After some googling, many people mention this is related to "sleep," but none of my threads are calling sleep. The current process is
Pass a function and parameters to a starter function
Starter function starts n threads and passes function from (1) and proper parameters to each thread
Each thread runs function with the parameters it was given, each function is independent only relying on parameters and local memory.
Starter function joins each created thread pthread_join(cur_thread[i], 0)
I've been trying to use strace and gdb to figure out whats throwing the signal. gdb just says "No stack" and strace doesn't stop at all... I'm thinking I'm about to over write the SIGALRM signal handler...
I'm out of ideas, could anyone offer any suggestions?
New information: This seems to be related to libcurl. I use it to get mjpegs from the network. If I do not use libcurl, the SIGALRM does not fire. My libcurl code is as follows:
static void * capture_function(void * cam)
{
MyData * camera = (MyData *)cam;
//Create the curl
camera->m_curl = curl_easy_init();
//Connect to the camera
curl_easy_setopt(camera->m_curl, CURLOPT_URL, camera->m_ip);
//Set up our callback functions
curl_easy_setopt(camera->m_curl, CURLOPT_WRITEFUNCTION, CurlWriteCallback);
curl_easy_setopt(camera->m_curl, CURLOPT_WRITEDATA, (void *)camera);
curl_easy_perform(camera->m_curl);
camera->m_state = camera->m_state & ~CAPTURING; //Remove the capturing flag;
curl_easy_cleanup(camera->m_curl);
return NULL;
}
This was related to libcurl. I had to use
setopt(handle, CURLOPT_NOSIGNAL, 1);
Related
I am creating a C++ dll and I need to make a single non-blocking http request. What I mean is that at a certain point in the dll, I need to kick off a download of a file in the background, but resume the program after the single download is kicked off.
I am currently using libcurl as my http client. I read that the multi-interface (https://curl.haxx.se/libcurl/c/libcurl-multi.html) will give me non-blocking functionality, but all of the examples I see are for making multiple http requests simultaneously, and waiting for all of them to finish w/a while loop for example. Am I able to use this interface to make a single http request, and resume my program while it downloads? And if so, can it fire a callback upon completion? Or do I have to continually check the handle to see if the download is complete?
Example of single request with libcurl multi interface:
int main() {
curl_global_init(CURL_GLOBAL_DEFAULT);
// init easy curl session
auto curl_easy = curl_easy_init();
// set url
auto url = "https://[foo].exe";
curl_easy_setopt(curl_easy, CURLOPT_URL, url);
// set curl to write file to directory
auto path = "C:\\foo\\app.exe";
auto filePtr = std::fopen(path, "wb");
curl_easy_setopt(curl_easy, CURLOPT_WRITEFUNCTION, writeFile);
curl_easy_setopt(curl_easy, CURLOPT_WRITEDATA, filePtr);
// init multi curl session
auto multi_handle = curl_multi_init();
// add easy handle to multi handle
curl_multi_add_handle(multi_handle, curl_easy);
auto res = curl_multi_perform(multi_handle, &running);
auto running = 0;
while (running) {
res = curl_multi_perform(multi_handle, &running);
}
I'd like to not do the while (running) loop and go do other things in main while the file downloads. I may be misunderstanding the "non-blocking" nature of curl_multi_perform(). Is it non-blocking only in the sense that many requests can happen simultaneously? If so, I don't think using it to make a single request gets me anything I wouldn't have with curl_easy_perform. Apologies if this is veering outside of stack overflow, I don't want a libcurl tutorial. But should I instead be using something like a blocking libcurl call (curl_easy_perform) inside of std::async()?
I've been working with easy_perform till now and it worked as expected. But due to timeouts and single threaded application, there's latency in running multiple operations. I'm looking at optimizing these calls by converting them into asynchronous with multi_perform interface, Though I am having hard time understanding correct way to make use of it.
From my understanding, Flow looks something like following :
Create a easy_handle for call
Add this standard easy handle to the multi stack using curl_multi_add_handle
curl_multi_perform : This is where it gets tricky.
As I understand it, This call is happening in a loop.
My application is calling this API to read/write whatever there is to read or write right now etc.
If running_handles is changed from the previous call, there is data to read which we should retrieve using curl_multi_info_read
Clean up when easy handle is processed
curl_multi_remove_handle
curl_easy_cleanup
curl_multi_cleanup
Q:
Does that mean, My application needs to do periodic polling to check if there's data to read?
Is there a way to handle this with callbacks? and the callback method should trigger action in my application in asynchronous way.
Refs I've already reviewed :
Looking at http://www.godpatterns.com/2011/09/asynchronous-non-blocking-curl-multi.html , It says the same thing. Correct me if I'm wrong.
stackoverflow thread and other related : How to do curl_multi_perform() asynchronously in C++?
The prerequisite knowledge needed to understand curl_multi API is Async Sockets.
curl_multi_perform is not a blocking API. As explained in documentation:
When an application has found out there's data available for the multi_handle or a timeout has elapsed, the application should call this function to read/write whatever there is to read or write right now etc. curl_multi_perform returns as soon as the reads/writes are done.
It just needs to be called periodically.
Does that mean, My application needs to do periodic polling to check if there's data to read?
Yes. curl_multi_fdset conveniently extracts the related file descriptors so that you can select on them (select = wait), but you're free to add other descriptors to the same select call, thus interleaving curl work with your own work. Here's an example of how to do it.
Is there a way to handle this with callbacks?
Yes. The transferred data is passed during the curl_multi_perform call into a CURLOPT_WRITEFUNCTION callback. Note: curl_multi_info_read is not for reading data, it's for reading information about the transfer.
for (/* each transfer */) {
curl_easy_setopt(eh, CURLOPT_WRITEFUNCTION, write_cb);
curl_easy_setopt(eh, CURLOPT_WRITEDATA, /* pass userp value */);
curl_multi_add_handle(multi_handle, easy_handle);
}
int still_running;
do {
if (curl_multi_perform(cm, &still_running)) { // will call write_cb() when data is read
/* handle error */ break;
}
if (curl_multi_wait(cm, NULL, 0, 1000, NULL)) {
/* handle error */ break;
}
} while(still_running);
Here's a complete example of using a data callback with multi-transfer: 10-at-a-time.
Note: curl_multi_wait used in this example is a convenience wrapper around a select call.
I am trying to create a concurrent c++ TCP server using threads. In particular I was wondering if I could use std::async to accept connections and serve each one in its own thread.
So far I have created a rough mockup but can't really tell if I am on the correct path.
void networking::TCP_Server::acceptConnection() {
std::string stringToSend{"This is a test string to be replied to"};
int new_fd = accept(listeningFD, nullptr, nullptr);
send(new_fd, stringToSend.c_str(), stringToSend.size(), 0);
sleep(3);
std::cout << ("End of thread");
}
///LISTEN FOR CONNECTIONS ON listeningFD
///CREATE A LIST OF FILE DESCRIPTORS FOR POLL fds[]
(fds[i].fd == listeningFD) {
do {
std::cout << ("New incoming connection - %d\n", new_fd);
std::async(std::launch::async, acceptConnection)
} while (new_fd != -1);
} /* End of existing connection is readable */
} /* End of loop through pollable descriptors */
I am connecting at the same time to the server with two clients and would expect for the loop to run through both new connections and create a thread for each one. As of now it is as it runs in deferred mode, one gets accepted, the other waits until the first finishes.
Any ideas?
(Pardon any mistakes in the code)
std::async returns a std::future which the code doesn't save into a variable, hence its destructor is called immediately. std::future::~future() blocks the calling thread until the future becomes ready.
You may like to use (detached) std::thread instead of std::async.
There are more scalable strategies to handle many clients. I highly recommend reading old but instructive The C10K problem.
You may also like to get familar with Asio C++ Library.
I think it is essential to send a message to the camera per some minutes ,so I initialize my camera and send message to camera every once in a while in my main thread , and in the other thread I open the liveview to process my other jobs, but in the liveview thread , it wait time out when sending this message :
EdsSetPropertyData(theCamera, kEdsPropID_Evf_Mode, 0, sizeof(evfMode), &evfMode)
I never met this when all jobs handled in just one thread， I don't know why it happends like this , could someone help me ? Here is some of my code.
my main thread : (I don't send keepalive message in the thread,but also timeout when starting liveview !)
CanonCameraWrapper& wrapper = param->wrapper;
bool setup_ok = wrapper.setup(0);
if (!setup_ok)
{
wrapper.destroy();
}
wrapper.setDownloadPath("img");
pthread_t camera_thread;
pthread_create(&camera_thread, NULL, camera_thread_func, (void *)(param));
pthread_join(camera_thread, NULL);
the other thread
void * camera_thread_func(void * arg)
{
global_param* param = (global_param*)arg;
CanonCameraWrapper& wrapper = param->wrapper;
wrapper.beginLiveView();//**//it wait time out here!!!**
...
}
I believe there are two things you have to be aware of. One is this notice in the Canon SDK documentation:
When creating applications that run under Windows, a COM initialization is required for each thread in order to
access a camera from a thread other than the main thread.
To create a user thread and access the camera from that thread, be sure to execute CoInitializeEx( NULL,
COINIT_APARTMENTTHREADED ) at the start of the thread and CoUnInitialize() at the end.
Sample code is shown below. This is the same when controlling EdsVolumeRef or EdsDirectoryItemRef objects
from another thread, not just with EdsCameraRef
The other thing is: you cannot access (most of) the SDK at the same time. So if you are using multiple threads you have to be careful about this
I have come to use curl synchronously doing a http request. My question is how can I do it asynchronously?
I did some searches which lead me to the documentation of curl_multi_* interface from this question and this example but it didn't solve anything at all.
My simplified code:
CURLM *curlm;
int handle_count = 0;
curlm = curl_multi_init();
CURL *curl = NULL;
curl = curl_easy_init();
if(curl)
{
curl_easy_setopt(curl, CURLOPT_URL, "https://stackoverflow.com/");
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeCallback);
curl_multi_add_handle(curlm, curl);
curl_multi_perform(curlm, &handle_count);
}
curl_global_cleanup();
The callback method writeCallback doesn't get called and nothing happens.
Please advise me.
EDIT:
According to #Remy's below answer, I got this but seems that it's not quite what I really needed. Cause using a loop is still a blocking one. Please tell me if I'm doing wrong or misunderstanding something. I'm actually pretty new to C++.
Here's my code again:
int main(int argc, const char * argv[])
{
using namespace std;
CURLM *curlm;
int handle_count;
curlm = curl_multi_init();
CURL *curl1 = NULL;
curl1 = curl_easy_init();
CURL *curl2 = NULL;
curl2 = curl_easy_init();
if(curl1 && curl2)
{
curl_easy_setopt(curl1, CURLOPT_URL, "https://stackoverflow.com/");
curl_easy_setopt(curl1, CURLOPT_WRITEFUNCTION, writeCallback);
curl_multi_add_handle(curlm, curl1);
curl_easy_setopt(curl2, CURLOPT_URL, "http://google.com/");
curl_easy_setopt(curl2, CURLOPT_WRITEFUNCTION, writeCallback);
curl_multi_add_handle(curlm, curl2);
CURLMcode code;
while(1)
{
code = curl_multi_perform(curlm, &handle_count);
if(handle_count == 0)
{
break;
}
}
}
curl_global_cleanup();
cout << "Hello, World!\n";
return 0;
}
I can now do 2 http requests simultaneously. Callbacks are called but still need to finish before executing following lines. Will I have to think of thread?
Read the documentation again more carefully, particularly these portions:
http://curl.haxx.se/libcurl/c/libcurl-multi.html
Your application can acquire knowledge from libcurl when it would like to get invoked to transfer data, so that you don't have to busy-loop and call that curl_multi_perform(3) like crazy. curl_multi_fdset(3) offers an interface using which you can extract fd_sets from libcurl to use in select() or poll() calls in order to get to know when the transfers in the multi stack might need attention. This also makes it very easy for your program to wait for input on your own private file descriptors at the same time or perhaps timeout every now and then, should you want that.
http://curl.haxx.se/libcurl/c/curl_multi_perform.html
When an application has found out there's data available for the multi_handle or a timeout has elapsed, the application should call this function to read/write whatever there is to read or write right now etc. curl_multi_perform() returns as soon as the reads/writes are done. This function does not require that there actually is any data available for reading or that data can be written, it can be called just in case. It will write the number of handles that still transfer data in the second argument's integer-pointer.
If the amount of running_handles is changed from the previous call (or is less than the amount of easy handles you've added to the multi handle), you know that there is one or more transfers less "running". You can then call curl_multi_info_read(3) to get information about each individual completed transfer, and that returned info includes CURLcode and more. If an added handle fails very quickly, it may never be counted as a running_handle.
When running_handles is set to zero (0) on the return of this function, there is no longer any transfers in progress.
In other words, you need to run a loop that polls libcurl for its status, calling curl_multi_perform() whenever there is data waiting to be transferred, repeating as needed until there is nothing left to transfer.
The blog article you linked to mentions this looping:
The code can be used like
Http http;
http:AddRequest("http://www.google.com");
// In some update loop called each frame
http:Update();
You are not doing any looping in your code, that is why your callback is not being called. New data has not been received yet when you call curl_multi_perform() one time.