Requesting beginner-level guidance with using libCurl API - c++

I'm am trying to use libcurl (linked to a C++ program) for the first time, and need beginner-level help. I'm also largely unfamiliar with HTTP/HTML, etc. so please forgive me if my terminology belies that.
Using the executable curl, if I execute the following...
curl -k -u user:password https://confluence/pages/viewpage.action?pageId=42
...I get what looks like the legit contents of the webpage.
I would like to do the same from my C++ program using libcurl.
I've started with a minimal modification of basic example posted at https://curl.haxx.se/libcurl/c/simple.html:
#include <iostream>
#include <curl/curl.h>
int main(void)
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl)
{
curl_easy_setopt(curl, CURLOPT_URL, "https://confluence/pages/viewpage.action?pageId=42");
res = curl_easy_perform(curl);
if(CURLE_OK == res) { std::cout << "curl success" << std::endl; }
else { std::cout << "curl failure" << std::endl; }
curl_easy_cleanup(curl);
}
return 0;
}
This code results in the output:
curl failure
Can anyone guide me on how I can programmatically do what I did earlier with the curl executable? There are some obvious deficiencies with my sample code, i.e. the absence of a username and password, so I'd appreciate any guidance in the right direction. Thank you.
Update
The reason I used the -k option when executing the curl executable was because running the command without -k resulted in no webpage content being returned by curl. I just tried adding -k based on the help text and observed it worked. Sorry for my lack of understanding and ability to explain. I'd be grateful if an answerer can touch on these topics too, to help me understand.
Update and Closure
I'm a little embarrassed that I turned to StackOverflow without a little bit more effort on my part - apologies to the community for this poor question.
The (insecure) solution, from just a bit of elbow-grease is:
#include <iostream>
#include <curl/curl.h>
int main(void)
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl)
{
curl_easy_setopt(curl, CURLOPT_URL, "https://confluence/pages/viewpage.action?pageId=42");
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
res = curl_easy_perform(curl);
if(CURLE_OK == res) { std::cout << "curl success" << std::endl; }
else { std::cout << "curl failure" << std::endl; }
curl_easy_cleanup(curl);
}
return 0;
}

You are trying to access a secure website, it won't work unless you use secure socket(443) protocol.
Try using CURLOPT_USE_SSL e.g.
curl_easy_setopt(curl, CURLOPT_USE_SSL, "https://confluence/pages/viewpage.action?pageId=42");

Related

curl PATCH to update value works as a curl command but not in libcurl c++, any ideas what is wrong?

I am trying to replicate the following curl command in c++ code using the curl library but with no luck.
The curl command is (the url is an actual url I am just hiding it):
curl -iX PATCH '*URL*/attrs/topicData' \
-H 'Content-Type: application/json' \
-H 'Link: <http://context-provider:3000/data-models/ngsi-context.jsonld>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \
--data-raw '{
"value": "Hi, new data test",
"type": "Property"
}'
This works perfectly fine and updates the value as required. My issues is that I can't replicate it in c++ code. I am using the nlohmann json library just in case that helps.
My c++ code is:
json ent={
{"type","Property"},
{"value","updated successfully"}
};
curl_global_init(CURL_GLOBAL_DEFAULT);
std::string json_entity = ent.dump();
curl = curl_easy_init();
if (curl) {
// Add headers
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "Content-Type: application/json");
headers = curl_slist_append(headers, R"(Link: <http://context-provider:3000/data-models/ngsi-context.jsonld>; rel="http://w3.org/ns/json-ld#context"; type="application/ld+json")");
// Set custom headers
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
// Set URL
curl_easy_setopt(curl, CURLOPT_URL, "*URL*/attrs/topicData");
// Set request type
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PATCH");
// Set values
curl_easy_setopt(curl, CURLOPT_POSTFIELDS,json_entity);
// Perform the request which prints to stdout
result = curl_easy_perform(curl);
// Error check
if (result != CURLE_OK) {
std::cerr << "Error during curl request: "
<< curl_easy_strerror(result) << std::endl;
}
//Free header list
curl_slist_free_all(headers);
curl_easy_cleanup(curl);
}
else {
std::cerr << "Error initializing curl." << std::endl;
}
The error that I am getting is:
"type":"http://uri.etsi.org/ngsi-ld/errors/InvalidRequest",
"title":"Invalid request.",
"details":"Invalid request."
I think my issue is at set values command but I am not sure what the problem is.
Can anyone please advice me on what I am doing wrong?
CURLOPT_POSTFIELDS expects a char* but you are supplying a std::string.
This should be working better:
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, json_entity.data());

Problems with libcurl and JSON for Modern C++

I'm trying to teach myself C++ by writing a simple program that sends a cURL request to a JSON API, parses the data and then stores it either in a text document or database for a web application to access. I have done this task in PHP and figured C++ wouldn't be much harder but I can't even get cURL to return a string and display it.
I get this to compile with no errors, but the response "JSON data: " doesn't display anything where the JSON data should be.
Where did I go wrong? URL-to-API is the actual URL, so I believe I'm using a wrong setopt function, or not setting one. In PHP, "CURLOPT_RETURNTRANSFER" made it return as a string, but I get an error with it:
error: ‘CURLOPT_RETURNTRANSFER’ was not declared in this scope
curl_easy_setopt(curl, CURLOPT_RETURNTRANSFER, true);
I'm using g++ compiler on Ubuntu and added -lcurl to the command line argument.
#include <cstdio>
#include <cstdlib>
#include <iostream>
#include <string>
#include <curl/curl.h>
//#include "json.hpp"
using namespace std;
//using json = nlohmann::json;
size_t WriteCallback(char *contents, size_t size, size_t nmemb, void *userp) {
((std::string*)userp)->append((char*)contents, size * nmemb);
return size * nmemb;
}
string getJSON(string URL) {
CURL *curl;
CURLcode res;
string readBuffer;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_HEADER, 1);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, false);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, true); // follow redirect
//curl_easy_setopt(curl, CURLOPT_RETURNTRANSFER, true); // return as string
curl_easy_setopt(curl, CURLOPT_HEADER, false);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_URL, URL);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
return readBuffer;
}
return 0;
}
int main() {
string data = getJSON("URL-to-api");
cout << "JSON Data: \n" << data;
return 0;
}
When I uncomment the JSON for Modern C++ include and namespace line I get this error:
error This file requires compiler and library support for the ISO C++ 2011 standard. This support must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
Along with a bunch of errors for functions in that library. I just downloaded the most recent version of g++ before embarking on this project, so what do I need to do?
I'm using g++ 5.4.0 on Ubuntu.
UPDATE:
So I added a check under res = curl_easy_perform(curl) and it doesn't return the error message, and res gets displayed as 6. This seems to be much more difficult than it should be:
#include <cstdio>
#include <cstdlib>
#include <iostream>
#include <string>
#include <curl/curl.h>
//#include "json.hpp"
using namespace std;
//using json = nlohmann::json;
size_t WriteCallback(char *contents, size_t size, size_t nmemb, void *userp) {
((std::string*)userp)->append((char*)contents, size * nmemb);
return size * nmemb;
}
string getJSON(string URL) {
CURL *curl;
CURLcode res;
string readBuffer;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_HEADER, 1);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, false);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, true); // follow redirect
curl_easy_setopt(curl, CURLOPT_HEADER, false);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_URL, URL);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
res = curl_easy_perform(curl);
cout << res << endl;
if (!res) {
cout << "cURL didn't work\n";
}
/* always cleanup */
curl_easy_cleanup(curl);
curl = NULL;
return readBuffer;
}
}
int main() {
string data = getData("");
cout << "JSON Data: \n" << data;
return 0;
}
I get the following output when I run the program:
6
JSON Data:
In PHP "CURLOPT_RETURNTRANSFER" made it return as a string but I get an error:
error: ‘CURLOPT_RETURNTRANSFER’ was not declared in this scope
curl_easy_setopt(curl, CURLOPT_RETURNTRANSFER, true);
There is no CURLOPT_RETURNTRANSFER option documented for curl_easy_setopt(). I think that is an option specify to PHP's curl_exec() function, which doesn't exist in CURL itself. CURLOPT_WRITEFUNCTION is the correct way to go in this situation.
When I uncomment the JSON for Modern C++ include and namespace line I get:
error This file requires compiler and library support for the ISO C++ 2011 standard. This support must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
The error is self-explanatory. Your JSON library requires C++11 but you are not compiling with C++11 enabled. Some modern compilers still default to an older C++ version (usually C++98) and require you to explicitly enable C++11 (or later) when invoking the compiler on the command line, or in your project makefile configuration.
In the case of g++, the current version (8.2) defaults to (the GNU dialect of) C17 for C and C++14 for C++, if not specified otherwise via the -std parameter. Your version (5.4) defaults to (the GNU dialect of) C11 and C++98, respectively.
UPDATE: there are other mistakes in your code:
You are passing a std::string object to curl_easy_setopt() where a char* pointer is expected for CURLOPT_URL. You need to change this:
curl_easy_setopt(curl, CURLOPT_URL, URL);
To this instead:
curl_easy_setopt(curl, CURLOPT_URL, URL.c_str());
You are not testing the return value of curl_easy_perform() correctly. Per the documentation, curl_easy_perform() returns 0 (CURLE_OK) on success, and non-zero on error, so you need to change this:
if (!res)
To this instead:
if (res != CURLE_OK)
So I added a check under res = curl_easy_perform(curl) ..., and res gets displayed as 6.
That is CURLE_COULDNT_RESOLVE_HOST, which makes sense as your updated example is passing a blank URL to getJSON():
string data = getJSON(""); // should be "URL-to-api" instead!

Resume broken https download in libcurl

I have a binary file to be downloaded and using curl in linux terminal, the following command supports download resume on a broken request.
curl -C - -o sample1.bin https://speed.hetzner.de/100MB.bin
The above will resume a download that is canceled.
When i use libcurl in my cpp program to do the same, is there any api's that i can use to achieve the above result on a HTTPS broken request.
Thank you for you help.
NOTE: CURL_RESUME_FROM does not have support on HTTPS.
I think you could implement a retry system by yourself, for example:
CURL *curl;
curl = curl_easy_init();
//Set curl options as needed with curl_easy_setopt()
char* url;
int tries = 0;
bool done = false;
while (tries != 3 && !done) {
res = curl_easy_perform(curl);
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &http_code);
curl_easy_getinfo(curl, CURLINFO_TOTAL_TIME, &elapsed);
curl_easy_getinfo(curl, CURLINFO_EFFECTIVE_URL, &url);
if (res != CURLE_OK || http_code != 200)
tries++;
else
done = true;
}
// Check if any error has occurred
if (res != CURLE_OK || http_code != 200) {
// Could not perform request "
if (tries == 3) {
//Too many tries, remote host is overloaded or down
} else {
// Cannot perform CURL
}
}
// Curl succeeded
Also you can have a look to CURLOPT_LOW_SPEED_LIMIT and CURLOPT_LOW_SPEED_TIME to avoid any overhead in the server.
How about something "low-level" like passing an extra header? Let s be the size of the partially-downloaded file, just use Range: bytes=s-.
See Requesting_a_specific_range_from_a_server and CURLOPT_HTTPHEADER explained.
#include <curl/curl.h>
#include <string>
#include <sstream>
int dim=... //size of partial download sample1.bin
std::string s=std::to_string(dim); // <-- here s is the string representing the size of the partial download
CURL *curl = curl_easy_init();
struct curl_slist *list = NULL;
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL,"https://speed.hetzner.de/100MB.bin");
list = curl_slist_append(list, "Range: bytes="+s+"-"); //from where it left off to the end (or where it stops again)
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, list);
curl_easy_perform(curl);
curl_slist_free_all(list); /* free the list again */
}
Check out also Making HTTPS GET with libcurl.

how to use CURL to login site

I am very new to cURL, so hopefully could get some help. Currently, I am in a Window environment, and using Visual Studio.
I am trying to use cURL to access a DLink IP camera through the DLink website (https://mydlink.com/login). And grab the video stream by the IP camera to do some processing. But to do this, I have to first login. But I am not sure how to do it.
Below is my code.
int main()
{
CURL *curl;
CURLcode result;
char *url_1 = "https://mydlink.com/login";
char *postdata = "email=xyz#gmail.com&password=123456";
char *cookiefile = "tempcookie";
curl = curl_easy_init();
if( curl )
{
curl_easy_setopt(curl, CURLOPT_COOKIEFILE, cookiefile);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, dummy);
curl_easy_setopt(curl, CURLOPT_URL, url_1);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, postdata);
// Connect to target (login)
result = curl_easy_perform(curl);
if( result )
cout << "Cannot connect to site, check your url!\n";
else
{
//...
}
}
return 0;
}
Could someone please enlighten me, or provide some piece of code for it?
Thank you
1) Your example code is incomplete: you use dummy function which is not in your listing.
It is important that the dummy function returns size*nmemb (see manual for CURLOPT_WRITEFUNCTION), so it is difficult to say what went wrong.
2) you don't output your error code: please use curl_easy_strerror to decode your error in the result, then you would know why it failed.
3) if I supply my own "dummy" callback, then I get an HTML page without errors, and the page itself does not complain about wrong password or anything (which is strange, but it kind of works).
Here is my dummy:
size_t dummy(char *ptr, size_t size, size_t nmemb, void *userdata)
{
printf("%.*s", size*nmemb, ptr);
return size*nmemb;
}
I looked a bit further what mydlink.com is doing and it is doing acrobatics with the email address (like deciding if it is local, tries to guess a region etc), then manipulates cookies -- it is all in javascript, thus I am afraid one has to dig that Javascript in order to emulate proper login POST, or perhaps find some proper documentation about mydlink.com services, sorry.

CURL does not return proper error code

Here the output is supposed to be anything but 0.
Because 0 according to libcurl documentation is a success. http://curl.haxx.se/libcurl/c/libcurl-errors.html
curl_easy_setopt(curl_handle, CURLOPT_PROXY, "socks5://127.0.0.1:9050");
And clearly curl_easy_setopt is supposed to return 5 i.e CURLE_COULDNT_RESOLVE_PROXY
#include <iostream>
#include <string>
#include <curl/curl.h>
using namespace std;
int main()
{
CURL *curl_handle;
CURLcode err;
curl_global_init(CURL_GLOBAL_ALL);
curl_handle = curl_easy_init();
err = curl_easy_setopt(curl_handle, CURLOPT_PROXY, "socks5://127.0.0.1:9050");
cout << err;
}
Am i missing something here ?
Thanks for the replies guys.
I looked "CURLE_COULDNT_RESOLVE_PROXY" on google and came across a few examples.
Turns out i had to call curl_easy_perform(curl_handle) after every curl_easy_setopt to get the error codes.
err = curl_easy_perform(curl_handle);
cout << err;