I have a while loop, and inside that loop I send a PUT request into google firebase REST api. It works very well, but if I want to fasten things up (the while loop waits for the curl response every round of the loop which is very slow sometimes, over 200ms), I'm trying to add the CURLOPT_TIMEOUT_MS and set it to a low 1 millisecond.
TLDR;
after adding line
curl_easy_setopt(curl, CURLOPT_TIMEOUT_MS, 1L);
My curl does not send anything to the server anymore. Or does the server somehow force the client to receive the returning value from the request?
You tell curl to fail the operation if it isn't completed within 1 millisecond. Not many requests are completed that quickly, especially not if you're using DNS or just use connections over the Internet.
So yes, most transfers will then just return CURLE_OPERATION_TIMEDOUT (28) with no content.
This is a bug of CURL.
If your timeout setting is less than 1s, it will directly return an error.
Solution is:
curl_easy_setopt(conn, CURLOPT_NOSIGNAL, 1);
conn is the pointer of CURL, e.g.:
CURL *conn = NULL;
curl_easy_setopt(conn, CURLOPT_NOSIGNAL, 1);
Related
After a lookup both on SO and other places, I've noticed there is a lot of conflicting information about the cURL options CONNECTTIMEOUT vs TIMEOUT.
CONNECTTIMEOUT is definitely the timeout just for the connection phase,
TIMEOUT is stated as being timeout for the entire cURL process (including CONNECTTIMEOUT) or the timeout after the connection phase has finished, depending on who you ask.
Furthermore, the official libcurl docs explain CONNECTTIMEOUT as
set maximum time the request is allowed to take
which is quite ambiguous language as it could be referring to e.g a HTTP request or speaking about the entire process as a request
CONNECTTIMEOUT is the time curl waits for during the connection. after that curl abandons the effort to connect. on the other hand, TIMEOUT is the total duration of receiving a response for a given request for which curl will wait, including the time it takes to connect and the time that the server takes to reply. here is the official link for both:
https://curl.haxx.se/libcurl/c/CURLOPT_CONNECTTIMEOUT.html
https://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT.html
I have a web proxy that starts a TCP listener socket that accepts connections from clients. The listener accepts connections via:
clientConnection, clientAddress = listenerSocket.accept()
and then a new thread handles the client connection from there.
To mock a client connection, I am using telnet to connect to the proxy and issue commands. The proxy needs to receive data from telnet and I need to make sure that I receive all of it. To achieve this, I am doing the following:
while True:
requestBytes = clientConnection.recv(1024)
if not requestBytes:
break
requestBuffer += requestBytes
The proxy then decodes the bytes and does some things with them that takes a little bit of time, and then has to send a response back to the same client. However, when using the above code the connection with clientConnection gets closed long before I can process the bytes and respond.
Here's what I don't understand, when I use the following instead:
while True:
requestBytes = clientConnection.recv(1024)
requestBuffer += requestBytes
break
It works just fine and the clientConnection remains intact. This obviously has a problem if I receive more than 1024 bytes, but the clientConnection does not get closed.
More specifically, the error occurs after I have a response to send to the client and call:
clientConnection.sendall(response)
clientConnection.shutdown(1)
clientConnection.close()
The line clientConnection.shutdown(1) throws the error:
[Errno 107] Transport endpoint is not connected
which is confusing because somehow it was able to still call sendall on the previous line. Note that I did not actually receive anything on the client side.
I am sure that the connection is not getting closed elsewhere in the code. What exactly is happening here and what is the best way to do something like recvall and keep the clientConnection open?
I have initialized curlHandle using curl_easy_init() and have set some options eg. url, type of request, timeout, ssl verification etc.
I read that re-using same curlHandle increases performance and did little sample code writting as well, it happened to be true.
So I initialized handle once in the constructor of my class and then resuing the same handle every time just changing header or request, but if I do not use that curlHandle for 10 minutes and then if I try to reuse then it throws an exception that could not resolve host.
Is there the time limit for which you can use curlHandle?
Because I did not see any such mention in any SO question or curl documentation.
Or is it like after certain timeout you have to do curl_easy_init() again?
[EDIT]
I am using wolfSSL for SSL communication with cURL.
By default session-id caching is disabled in wolfSSL and enabled in curl
because of CURLOPT_SSL_SESSIONID_CACHE defaulting to 1.
Session-id times out after inactivity of 500 seconds( Approx. 8 minutes) whereas cURL tries to reuse the same session-id.
This is causing the failure in SSL_set_session of wolfSSL and this causes curl to fail after 8-10 minutes of inactivity.
Curl version used 7.49.1
No, there's no time limit for a curl handle and no, it is not supposed to cause any particular errors after having been idle for ten or more minutes. If you can reproduce this with a recent version of libcurl, it might be a bug...
I'm using libCURL to perform an HTTP GET request toward a device that responds with a continuous flow of data in a multipart HTTP response.
I'd like to handle the unfortunate but possible case where the device is disconnected/shutdown or is not reachable anymore on the network.
By default libCURL does not have a few seconds timeout as I need, so I tried:
setting the CURLOPT_CONNECTTIMEOUT options,
but this only works at connection stage, not while already receiving data.
setting the CURLOPT_TIMEOUT option,
but this seems to always force a timeout even when data is still received.
My question is: how can I properly handle a timeout with libCURL, in the case described above?
For your scenario instead of
curl_easy_setopt(curl, CURLOPT_TIMEOUT, <your timeout in seconds>);
use
curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1);
curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, <your timeout in seconds>);
The above two lines make sure that if the average speed drops below 1 byte per second, in a time frame of X seconds, then the operation is aborted (timeout).
See reference here.
curll_handle = Curl init()
curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 20 )
curl_easy_setopt(curl, CURLOPT_TIMEOUT, 30);
Request1:(curll_handle)
Curl_easy_perform(curll_handle)
For connection = 20 secs
Chal response request = 30 secs
request2:same handle used as above
curl_slist_append(headers, "Connection: Close");
Curl_easy_perform(curll_handle)
Questions:
Request 1:
will it use both timeouts(connection and response timeouts) for curl perform?
what is the total time will take? as per my understand 30 secs(CURLOPT_CONNECTTIMEOUT + CURLOPT_TIMEOUT)
Request 2:
if we use same request1 curl handle for request2. is it(req2) curl_easy_perform() will d0 both connection and response sequence requestS?
what the timeout value use for sencond request if use same req1 curl handle?
Request 1:
CURLOPT_TIMEOUT is the maximum time for the entire operation. It will not spend more time. You know it will be done within 30 seconds, successful or not.
CURLOPT_CONNECTTIMEOUT is the longest time you allow the "connection phase" to take. If the connection to the remote server is not done within 20 seconds, the transfer will return failure.
The times are set totally independent of each other. So if the connection phase takes 19 seconds, there is 11 seconds left for the transfer to complete or the maximum timeout will trigger.
Request 2:
Uses the same options and options are sticky and will be the same until changed, so it will have the exact same timeout values as request 1.
If the connection is kept and re-used from request 1, the CURLOPT_CONNECTTIMEOUT won't trigger since it is already connected.