After a lookup both on SO and other places, I've noticed there is a lot of conflicting information about the cURL options CONNECTTIMEOUT vs TIMEOUT.
CONNECTTIMEOUT is definitely the timeout just for the connection phase,
TIMEOUT is stated as being timeout for the entire cURL process (including CONNECTTIMEOUT) or the timeout after the connection phase has finished, depending on who you ask.
Furthermore, the official libcurl docs explain CONNECTTIMEOUT as
set maximum time the request is allowed to take
which is quite ambiguous language as it could be referring to e.g a HTTP request or speaking about the entire process as a request
CONNECTTIMEOUT is the time curl waits for during the connection. after that curl abandons the effort to connect. on the other hand, TIMEOUT is the total duration of receiving a response for a given request for which curl will wait, including the time it takes to connect and the time that the server takes to reply. here is the official link for both:
https://curl.haxx.se/libcurl/c/CURLOPT_CONNECTTIMEOUT.html
https://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT.html
Related
I'm using AWS SQS and making a GetQueueURL request at startup. The problem is that if there is no network connection, the request blocks and doesn't time out until more than 400 seconds have passed (using default ClientConfiguration parameters).
The only timeout parameters I can see in ClientConfiguration are:
httpRequestTimeoutMs
requestTimeoutMs
connectTimeoutMs
However, changing these seem to have no effect in the case I mentioned above. I tried setting all three to 1000ms and, without a network connection, the GetQueueUrl request still blocks for several minutes until it times out.
If it helps, when the timeout does occur, the error I get on the request is: curlCode: 28, Timeout was reached
What am I missing here? Thanks!
When I invoke my lambda function it takes between 1 and 15 seconds to execute. If I invoke the function via the C++ SKD, I get timeouts. These timeouts seem to occur after a few seconds (this is human-judgment only, I did not actually time it).
Question: How do I tell the SDK to wait for slow lambdas to return and not to timeout?
Things that did not work:
In the JS SDK you can change this in the HTTP settings. This is is no such option in the C++ SDK HTTPOptions.
It does not help to give the lambda client a config with a larger connectionTimeoutMS (socket timeout). Also, the httpRequestTimeoutMs of the client is set to 0 by default, meaning it will wait forever.
I am using synchronous requests, which do not seem to have an extra option for timeouts.
Additional information:
I am using a single client to run multiple requests in parallel.
Error also happens if I am using async requests.
Related:
How do I troubleshoot retry and timeout issues when invoking a Lambda function using an AWS SDK?
Same thing gave me hard time once. You may have got the solution but for others, here is what I did. There is client configuration which can edit default connection time. Default connection time for sending request is 1 sec and for receiving it is 3 sec, if you get the request done in this time period than it is good otherwise a retry according to lambda setting will be invoked. The behavior of these two is well explained in their respective header file.
you can also play with memory size of lambda higher the memory of lambda lower the response time for the same.
Aws::Client::ClientConfiguration m_ClientConfig;
m_ClientConfig.requestTimeoutMs = 300000; // i.e. 300 seconds
m_ClientConfig.connectTimeoutMs = 300000;
/**
* Socket read timeouts for HTTP clients on Windows. Default 3000 ms. This should be more than adequate for most services. However, if you are transfering large amounts of data
* or are worried about higher latencies, you should set to something that makes more sense for your use case.
* For Curl, it's the low speed time, which contains the time in number milliseconds that transfer speed should be below "lowSpeedLimit" for the library to consider it too slow and abort.
*/
long requestTimeoutMs;
/**
* Socket connect timeout. Default 1000 ms. Unless you are very far away from your the data center you are talking to. 1000ms is more than sufficient.
*/
long connectTimeoutMs;
I have initialized curlHandle using curl_easy_init() and have set some options eg. url, type of request, timeout, ssl verification etc.
I read that re-using same curlHandle increases performance and did little sample code writting as well, it happened to be true.
So I initialized handle once in the constructor of my class and then resuing the same handle every time just changing header or request, but if I do not use that curlHandle for 10 minutes and then if I try to reuse then it throws an exception that could not resolve host.
Is there the time limit for which you can use curlHandle?
Because I did not see any such mention in any SO question or curl documentation.
Or is it like after certain timeout you have to do curl_easy_init() again?
[EDIT]
I am using wolfSSL for SSL communication with cURL.
By default session-id caching is disabled in wolfSSL and enabled in curl
because of CURLOPT_SSL_SESSIONID_CACHE defaulting to 1.
Session-id times out after inactivity of 500 seconds( Approx. 8 minutes) whereas cURL tries to reuse the same session-id.
This is causing the failure in SSL_set_session of wolfSSL and this causes curl to fail after 8-10 minutes of inactivity.
Curl version used 7.49.1
No, there's no time limit for a curl handle and no, it is not supposed to cause any particular errors after having been idle for ten or more minutes. If you can reproduce this with a recent version of libcurl, it might be a bug...
I have a while loop, and inside that loop I send a PUT request into google firebase REST api. It works very well, but if I want to fasten things up (the while loop waits for the curl response every round of the loop which is very slow sometimes, over 200ms), I'm trying to add the CURLOPT_TIMEOUT_MS and set it to a low 1 millisecond.
TLDR;
after adding line
curl_easy_setopt(curl, CURLOPT_TIMEOUT_MS, 1L);
My curl does not send anything to the server anymore. Or does the server somehow force the client to receive the returning value from the request?
You tell curl to fail the operation if it isn't completed within 1 millisecond. Not many requests are completed that quickly, especially not if you're using DNS or just use connections over the Internet.
So yes, most transfers will then just return CURLE_OPERATION_TIMEDOUT (28) with no content.
This is a bug of CURL.
If your timeout setting is less than 1s, it will directly return an error.
Solution is:
curl_easy_setopt(conn, CURLOPT_NOSIGNAL, 1);
conn is the pointer of CURL, e.g.:
CURL *conn = NULL;
curl_easy_setopt(conn, CURLOPT_NOSIGNAL, 1);
I'm using libCURL to perform an HTTP GET request toward a device that responds with a continuous flow of data in a multipart HTTP response.
I'd like to handle the unfortunate but possible case where the device is disconnected/shutdown or is not reachable anymore on the network.
By default libCURL does not have a few seconds timeout as I need, so I tried:
setting the CURLOPT_CONNECTTIMEOUT options,
but this only works at connection stage, not while already receiving data.
setting the CURLOPT_TIMEOUT option,
but this seems to always force a timeout even when data is still received.
My question is: how can I properly handle a timeout with libCURL, in the case described above?
For your scenario instead of
curl_easy_setopt(curl, CURLOPT_TIMEOUT, <your timeout in seconds>);
use
curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1);
curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, <your timeout in seconds>);
The above two lines make sure that if the average speed drops below 1 byte per second, in a time frame of X seconds, then the operation is aborted (timeout).
See reference here.