C++ LibCurl retry on error - c++

I want to retry a curl connection in my C++ program for 5 times. When it fails 5 times in a row, it should stop the execution of the program. However, it stops after the first error at this point. I am able to catch the error, however I don't know how to execute the previous curl connection. E.g., with jQuery I can use something like $.ajax(this);. For LibCurl in C++ I am looking for a similar solution.
My current LibCurl code is shown below, note that I use multiple curl connections which all have other settings, therefore I would like a general approach which I can use for all my LibCurl errors within my LibcurlError function which is also included below.
curl = curl_easy_init();
if (curl) {
CurlResponse = "";
host = "http://google.com";
LibcurlHeaders = curl_slist_append(NULL, "Expect:");
if (ProxyAddress.length() > 0) {
curl_easy_setopt(curl, CURLOPT_PROXY, ProxyAddress.c_str());
}
curl_easy_setopt(curl, CURLOPT_URL, (host).c_str());
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER , 1);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST , 1);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, LibcurlHeaders);
res = curl_easy_perform(curl);
curl_slist_free_all(LibcurlHeaders);
if (res != CURLE_OK) {
//AT THIS POINT I WOULD LIKE TO RETRY FOR 5 TIMES WHICH I WOULD LIKE TO CATCH IN MY LibcurlError FUNCTION.
LibcurlError(curl_easy_strerror(res), host);
}
curl_easy_cleanup(curl);
}
curl_global_cleanup();
void LibcurlError(string error, string host) {
//IF FAILED FOR LESS THEN 5 TIMES IN A ROW -> RETRY CURL
//ELSE I WOULD LIKE TO EXECUTE MY ORIGINAL CODE WHICH IS STATED BELOW
Message = "LibCurl Error: ";
if (error == "Couldn't resolve host name") {
Message.append("Couldn't connect to the server of ");
if (host.find("google.com") != string::npos) {
Message.append("Google");
}
else {
Message.append("'" + host + "'");
}
}
else {
Message.append("'" + error + "'");
}
cout << Message << endl;
system("pause");
exit(0);
}

There is no CURL method that specifically does this because it can be accomplished by repeated calls to curl_easy_perform.
Here is how you would write the code in your question (the relevant part at least) using loops to retry the CURL request repeatedly:
#include <unistd.h>
#include <curl/curl.h>
/*
* This is the maximum number of times CURL will run
*/
const int max_attempts = 5;
curl = curl_easy_init();
if (curl) {
CurlResponse = "";
host = "http://google.com";
LibcurlHeaders = curl_slist_append(NULL, "Expect:");
if (ProxyAddress.length() > 0) {
curl_easy_setopt(curl, CURLOPT_PROXY, ProxyAddress.c_str());
}
curl_easy_setopt(curl, CURLOPT_URL, (host).c_str());
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER , 1);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST , 1);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, LibcurlHeaders);
for (int i = 1; i <= max_attempts &&
(res = curl_easy_perform(curl)) != CURLE_OK; i++) {
/*
* At this point, you would sleep
* for some seconds between requests
*/
const int sleep_secs = 1;
sleep(sleep_secs);
}
// As others have mentioned, you should delete this line:
//curl_slist_free_all(LibcurlHeaders);
if (res != CURLE_OK) {
// The max retries have all failed
LibcurlError(curl_easy_strerror(res), host);
}
else {
// The request has succeeded in the first `max_retries` attempts
// ...
}
curl_easy_cleanup(curl);
}
curl_global_cleanup();

Related

Libcurl C++ Ldap Request

i am new to cUrl.
What i need todo is a curl request to an ldap server.
So far the request works fine over the command line:
$curl ldap://XXXXXXXXX:389/DC=XXXXXXXXX,DC=DE?mail?sub?(sAMAccountName=XXXXXXXXX) --user s_ad_XXXXXXXXX:Password
After trying to get this done in my C++ CMake Project. I issued this error Code from
libcurl: libcurl: (39) LDAP local: ldap_search_ext Bad search filter
My code as snippet:
std::string readBuffer;
CURL* curl = curl_easy_init();
if (curl) {
CURLcode res;
char errbuf[CURL_ERROR_SIZE];
curl_easy_setopt(curl, CURLOPT_URL, "ldap://XXXXXXXXX:389/DC=XXXXXXXXX,DC=DE?mail?sub?(sAMAccountName=XXXXXXXXX) --user s_ad_XXXXXXXXX:Password");
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errbuf);
errbuf[0] = 0;
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); // this line prints into console, what curl is trying to do
curl_easy_setopt(curl, CURLOPT_FORBID_REUSE, 1L); // closes the connection, if not here the connection can be used more than once
// curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
res = curl_easy_perform(curl);
std::cout << res << std::endl;
curl_easy_cleanup(curl);
std::cout << readBuffer << std::endl;
if (res != CURLE_OK) {
size_t len = strlen(errbuf);
fprintf(stderr, "\nlibcurl: (%d) ", res);
if (len)
fprintf(stderr, "%s%s", errbuf,
((errbuf[len - 1] != '\n') ? "\n" : ""));
else
fprintf(stderr, "%s\n", curl_easy_strerror(res));
}
}
My Output:
Test curl library
* Trying XXXXXXXXXXXXXX
* TCP_NODELAY set
* Connected to XXXXXXXXXXXXXX
* LDAP local: ldap://XXXXXXXXX:389/DC=XXXXXXXXX,DC=DE?mail?sub?(sAMAccountName=XXXXXXXXX) --user s_ad_XXXXXXXXX:Password
* LDAP local: ldap_search_ext Bad search filter
* Closing connection 0
39
libcurl: (39) LDAP local: ldap_search_ext Bad search filter
I appricate every help! Thank you for taking the time reading this.
You cannot just bung everything into the CURLOPT_OPT_URL field. The --user bit should be separated into a separate parameter:
curl_easy_setopt(curl, CURLOPT_URL, "ldap://XXXXXXXXX:389/DC=XXXXXXXXX,DC=DE?mail?sub?(sAMAccountName=XXXXXXXXX)");
curl_easy_setopt(curl, CURLOPT_USERPWD, "s_ad_XXXXXXXXX:Password");

C++ cant use respond string from GET requests retrieved by ExpressJS API for anything else than printing

Scenario:
Im writing a c++ program which should retrieve files and strings from an expressJS API..
Downloading txt.files with curlRequests works perfectly but as soon as i try to retrieve plain strings, they can only be used for printing..
Problem: When trying to do anything with the response from my GET request (from the expressjs api), my response doesnt get treated as a string.
string myText = curlRequest.GetText("/templates/names");
string myTextB = "react.txt, scss.txt"
cout << myText << endl; // prints"react.txt, scss.txt"
cout << myTextB << endl; // prints "react.txt, scss.txt"
cout << stringHelper.GetSubstringPos(myText, "scss") << endl; // printsstring::npos
cout << stringHelper.GetSubstringPos(myTextB, "scss") << endl; // printsposition of 's' as expected
Here is my GetText function of the curlrequest.h in c++
string GetText(string ACTIONURL) {
CURL* curl;
CURLcode curlRes;
string res;
string url = templateCreator.APIURL + ACTIONURL;
curl_global_init(CURL_GLOBAL_DEFAULT);
curl = curl_easy_init();
if (curl) {
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1L);
curlRes = curl_easy_perform(curl);
res = curlRes;
if (curlRes == CURLE_HTTP_RETURNED_ERROR) {
res = "FAILED";
}
else if (curlRes != CURLE_OK) {
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(curlRes));
res = "FAILED";
}
curl_easy_cleanup(curl);
}
curl_global_cleanup();
return res;
}
Here is the route in expressjs (receiving the request and responding with a string)
router.get('/names', function(req, res, next) {
var str = "react.txt, scss.txt, example.txt";
res.send(str);
});
// I also tried sending a buffer but as its sended as string i face the same problem..
// C++ could decode the buffer ..
router.get('/buf', function(req, res, next) {
let data = 'hello world: (1 + 2 !== 4)';
let buff = new Buffer.from(data);
let base64data = buff.toString('base64');
console.log(base64data);
res.send(base64data);
});
Retrieving textfiles from my expressjs API is not a problem..
void GetFile(string ACTIONURL, string OUTDIR) {
CURL* curl;
FILE* fp;
CURLcode res;
string url = templateCreator.APIURL + ACTIONURL;
curl = curl_easy_init();
if (curl)
{
fopen_s(&fp, OUTDIR.c_str(), "wb");
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
fclose(fp);
}
return;
}
(After download I can read those line by line and store in a vector).
Still im hoping to get sending actual strings working..
Does anyone have a clue why im facing problems here?
I literally spent days on this unexpected 'small' issue already..
Thank you #n.'pronouns'm.
I Updated my function and realised that res = curlRes was a nobrainer..
Also the checks for valid response is gone for now.
//those 2 lines and a write_to_string function were missing and `res = curlRes` should do their job
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_to_string);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
The following 2 functions replace my GetText function now, and everything works as expected.
size_t write_to_string(void* ptr, size_t size, size_t count, void* stream) {
((string*)stream)->append((char*)ptr, 0, size * count);
return size * count;
}
string GetText(string ACTIONURL) {
CURL* curl;
CURLcode res;
string response;
curl = curl_easy_init();
if (curl) {
curl_easy_setopt(curl, CURLOPT_URL, templateCreator.APIURL + ACTIONURL.c_str());
curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_to_string);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
res = curl_easy_perform(curl);
if (res != CURLE_OK) {
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
}
curl_easy_cleanup(curl);
}
return response;
}
Thank you! I found the fix on 1 or 2 questions earlier too but was not aware that this was the actual problem. Working with strings is possible now!

How to include an h file locally, without visual studio?

I have a simple cpp code that uses curl.h as it's header file.
I use vscode, and not interested on linking via gui or w/e
How can I make the include statement work properly with the single project?
I got the header file from https://github.com/curl/curl/blob/master/include/curl/curl.h
Folder structure is like this:
/Folder
+-> curl.h
+-> curli.cpp
the code:
#include "curl.h"
using namespace std;
#define YOUR_URL "https://www.insertyourURLhere.com/"
#define USER_AND_PWD "user:password"
static string gs_strLastResponse;
// Callback to gather the response from the server. Comes in chunks (typically 16384 characters at a time), so needs to be stitched together.
size_t function_pt(void *ptr, size_t size, size_t nmemb, void * /*stream*/)
{
gs_strLastResponse += (const char*)ptr;
return size * nmemb;
}
bool CallServerWithCurl(string strData1, strData2, string& strErrorDescription)
{
CURL* curl = curl_easy_init();
if (curl == NULL)
{
strErrorDescription = "Unable to initialise Curl";
return false;
}
curl_easy_setopt(curl, CURLOPT_URL, (YOUR_URL + "get_result").c_str());
curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_easy_setopt(curl, CURLOPT_USERPWD, MY_USER_AND_PWD); // set user name and password for the authentication
char* data1 = curl_easy_escape(curl, strData1.c_str(), 0);
char* data2 = curl_easy_escape(curl, strData2.c_str(), 0);
string strArguments = "id=" + data1 + "&task=" + data2;
const char* my_data = strArguments.c_str();
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, my_data);
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, (long)strlen(my_data)); // if we don't provide POSTFIELDSIZE, libcurl will strlen() by itself
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); // enable verbose for easier tracing
gs_strLastResponse = "";
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, function_pt); // set a callback to capture the server's response
CURLcode res = curl_easy_perform(curl);
// we have to call twice, first call authenticates, second call does the work
res = curl_easy_perform(curl);
if (res != CURLE_OK)
{
strErrorDescription = "Curl call to server failed";
return false;
}
if (!DoSomethingWithServerResponse(gs_strLastResponse))
{
strErrorDescription = "Curl call to server returned an unexpected response";
return false;
}
// extract some transfer info
double speed_upload, total_time;
curl_easy_getinfo(curl, CURLINFO_SPEED_UPLOAD, &speed_upload);
curl_easy_getinfo(curl, CURLINFO_TOTAL_TIME, &total_time);
fprintf(stderr, "Speed: %.3f bytes/sec during %.3f seconds\n", speed_upload, total_time);
curl_easy_cleanup(curl);
return true;
}

libcurl - "curl_easy_cleanup()" is causing malloc error (C++)

When running my code (pertinent sections pasted below), I periodically get the following error:
program(34010,0x70000e58b000) malloc: *** error for object
0x7fc43d93fcf0: pointer being freed was not allocated set a breakpoint
in malloc_error_break to debug Signal: SIGABRT (signal SIGABRT)
I am running multi-threaded C++ code on a Macbook (OS-10.13) wherein different threads make use of the code in question simultaneously. To my knowledge, libcurl is indeed thread safe as long as I do not utilize the same "curl handle" (which I understand to be an instance of "CURL" aka "CURL *curl = curl_easy_init();") in two different threads at the same time. In my case, since each thread calls the function separately and initializes a new instance of a CURL object, I should be "safe", right? Hopefully there is something obvious that I'm missing that is causing me (or lib curl in this case) to attempt to free memory that has already been freed. If there is any more information I should have included (below) please don't hesitate to let me know.
The function that seg faults is
string http_lib::make_get_request(string url)
on the line that reads
curl_easy_cleanup(curl);
and sometimes (less often) on the line that reads
res = curl_easy_perform(curl);
Below is what I think would be the pertinent sections of my code:
size_t http_lib::CurlWrite_CallbackFunc_StdString(void *contents, size_t size, size_t nmemb, std::string *s)
{
size_t newLength = size*nmemb;
size_t oldLength = s->size();
try
{
s->resize(oldLength + newLength);
}
catch(std::bad_alloc &e)
{
//handle memory problem
return 0;
}
std::copy((char*)contents,(char*)contents+newLength,s->begin()+oldLength);
return size*nmemb;
}
string http_lib::make_post_request(string url, vector<string> headers, string post_params) {
CURL *curl;
CURLcode res;
curl = curl_easy_init();
string s;
if(curl)
{
struct curl_slist *chunk = NULL;
for(int i=0; i<headers.size(); i++){
/* Add a custom header */
chunk = curl_slist_append(chunk, headers[i].c_str());
}
/* set our custom set of headers */
res = curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, post_params.c_str());
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L); //only for https
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L); //only for https
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, CurlWrite_CallbackFunc_StdString);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &s);
if(networking_debug){
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); //verbose output
}
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if(res != CURLE_OK)
{
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
}
/* always cleanup */
curl_easy_cleanup(curl);
}
// Debug output
if (networking_debug){
cout<<"Response: " << s <<endl;
}
return s;
}
string http_lib::make_get_request(string url) {
//SslCurlWrapper sslObject;
CURL *curl;
CURLcode res;
curl = curl_easy_init();
string s;
if (curl) {
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
//tell libcurl to follow redirection
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L); //only for https
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L); //only for https
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, CurlWrite_CallbackFunc_StdString);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &s);
if(networking_debug){
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); //verbose output
}
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if (res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
}
if (networking_debug){
cout << "Response: " << s << endl;
}
return s;
}
In main() I have
int main(int argc, char *argv[]){
// Initialize http_lib (curl)
curl_global_init(CURL_GLOBAL_DEFAULT);
... spin up 10 or so threads that make get/post requests to https site (some requests utilize the make_post_request() function and others utilize make_get_requet() function).
}
CMAKE doesn't/didn't seem to want to use anything other than CURL_ROOT_DIR of "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include" for libcurl (aka curl).
Thus it was using the curl lib that mac (and/or Xcode) ships with. I haven't figured out what version that is, but I can say that not using it and instead using CURL version 7.57 is what fixed my issue.
I used "brew" package manager to
brew install curl
Doing so created /usr/local/Cellar/curl/7.57.0 directory and put all libs/includes in there.
Then I added
-I/usr/local/Cellar/curl/7.57.0/include -L/usr/local/Cellar/curl/7.57.0/lib
to my CMAKE CMAKE_CXX_FLAGS.
TLDR; Solution was to ensure I was using the newest version of the curl lib. Now that I am, no problem.

NaCl and curl - can't connect to server

I'm new to NaCl and now I'm starting to have some issues with this. I'm trying to load some data into C++ part of the application from C++ code and I know that I can't access files on HDD from NaCl. So, what I tried to do, was to use curl library to load this files via http from localhost.
So I set up NaCl environment, added curl - everything built fine, then I had an issue that curl couldn't resolve host name, but hopefully I overcame this issue, but then curls says that it can't connect to server (error code 7). I don't know what might be the issue.
I start chrome using this command:
google-chrome 127.0.0.1:5103/myapp --allow-nacl-socket-api=127.0.0.1
--no-sandbox --load-extension="/home/user/Desktop/nacl_sdk/pepper_49/getting_started/web/"
--enable-nacl --user-data-dir="/home/user/Desktop/chrome-dir/"
And code that is trying to load data from localhost is as follows:
static size_t data_write(char* buf, size_t size, size_t nmemb, void* userp) {
printf("start data_write\n");
if (userp) {
std::stringstream& data = *static_cast<std::stringstream*>(userp);
std::streamsize len = size * nmemb;
for (int i = 0; i < len; ++i) {
data << buf[i];
}
return len;
}
printf("end data_write\n");
return 0;
}
CURLcode curl_read(const std::string& url, std::stringstream& os, long timeout = 300) {
CURLcode code(CURLE_FAILED_INIT);
CURL* curl = curl_easy_init();
printf("CURL_READ\n");
if (curl) {
printf("curl is not null\n");
if (CURLE_OK == (code = curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION,
&data_write)) &&
CURLE_OK == (code = curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 1L)) &&
CURLE_OK ==
(code = curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L)) &&
CURLE_OK == (code = curl_easy_setopt(curl, CURLOPT_FILE, &os)) &&
CURLE_OK == (code = curl_easy_setopt(curl, CURLOPT_TIMEOUT, timeout)) &&
CURLE_OK == (code = curl_easy_setopt(curl, CURLOPT_URL, url.c_str()))) {
printf("START curl perform\n");
code = curl_easy_perform(curl);
printf("CURL ERROR (%d) = %s\n", code, curl_easy_strerror(code));
printf("END curl perform\n");
}
curl_easy_cleanup(curl);
}
return code;
}
And the output from console is:
Init curl
Loading data from: http://127.0.0.1/assets/model.obj
CURL_READ
curl is not null
START curl perform
CURL ERROR (7) = Couldn't connect to server
END curl perform
What might be the issue? Is it even possible to use curl in NaCl?