Curl command line works, C++ curl library does not - c++

I'm trying to do some request via curl library of C++. I can successfully do my request and get the correct response via command line, but I cannot get the correct response via C++ code. My command line command looks like this
curl -X POST -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'Authorization: <some_hash_value>' -k <my_full_url> -data '<my_json_string>'
That works fine. Now I try to do the same request in C++ code. My code looks like this
void performRequest(const std::string& json, const void* userData, CallbackFunction callback)
{
struct curl_slist* headers = NULL;
headers = curl_slist_append(headers, "Accept: application/json");
headers = curl_slist_append(headers, "Content-Type: application/json");
headers = curl_slist_append(headers, (std::string("Authorization: ") + m_authorization).c_str());
CURL* curlHandle = curl_easy_init();
if (!curlHandle)
{
std::cerr << "Curl handler initialization failed";
}
curl_easy_setopt(curlHandle, CURLOPT_NOSIGNAL, 1);
curl_easy_setopt(curlHandle, CURLOPT_HTTPHEADER, headers);
// specify target URL, and note that this URL should include a file name, not only a directory
curl_easy_setopt(curlHandle, CURLOPT_URL, m_url.c_str());
// enable uploading
curl_easy_setopt(curlHandle, CURLOPT_UPLOAD, 1L);
// set HTTP method to POST
curl_easy_setopt(curlHandle, CURLOPT_CUSTOMREQUEST, "POST");
// set json data; I use EXACTLY the same string as in command line
curl_easy_setopt(curlHandle, CURLOPT_COPYPOSTFIELDS, json.c_str());
// set data size
curl_easy_setopt(curlHandle, CURLOPT_POSTFIELDSIZE_LARGE, json.size());
// set user data for getting it in response
curl_easy_setopt(curlHandle, CURLOPT_WRITEDATA, userData); // pointer to a custom struct
// set callback function for getting response
curl_easy_setopt(curlHandle, CURLOPT_WRITEFUNCTION, callback); // some callback
// send request
curl_easy_perform(curlHandle);
curl_easy_cleanup(curlHandle);
curl_slist_free_all(headers);
}
However, for some reason I get an error in the response from the server, from which I can assume that my code's request is not equivalent to command line's command. It seems that body is not sent. I cannot see my request Json body when I use CURLOPT_DEBUGFUNCTION for dumping debug info.
What is the problem here? What am I doing wrong? Any ideas?

Here is sample code that should work for you.
Notice that I:
Removed CURLOPT_UPLOAD as it does not seem as you are actually uploading something but rather just doing a simple POST.
Changed CURLOPT_CUSTOMREQUEST to CURLOPT_POST (not that it should matter), but I find it cleaner.
Reordered CURLOPT_POSTFIELDSIZE_LARGE and CURLOPT_COPYPOSTFIELDS
Removed the CURLOPT_WRITEDATA line for the sake of this sample code.
I have tested the following only by connecting to an instance of nc -l localhost 80
static size_t callback(char *ptr, size_t size, size_t nmemb, void *userdata)
{
string s(ptr);
cout << s << endl;
return size * nmemb;
}
int main(int argc, char** argv)
{
string m_authorization("PWNED");
string m_url("http://localhost");
string m_json("{}");
curl_global_init(CURL_GLOBAL_ALL);
CURL* curlHandle = curl_easy_init();
struct curl_slist* headers = nullptr;
headers = curl_slist_append(headers, "Accept: application/json");
headers = curl_slist_append(headers, "Content-Type: application/json");
headers = curl_slist_append(headers, (std::string("Authorization: ") + m_authorization).c_str());
curl_easy_setopt(curlHandle, CURLOPT_NOSIGNAL, 1);
curl_easy_setopt(curlHandle, CURLOPT_HTTPHEADER, headers);
// specify target URL, and note that this URL should include a file name, not only a directory
curl_easy_setopt(curlHandle, CURLOPT_URL, m_url.c_str());
// <= You are not uploading anything actually, this is a simple POST with payload
// enable uploading
// curl_easy_setopt(curlHandle, CURLOPT_UPLOAD, 1L);
// set HTTP method to POST
curl_easy_setopt(curlHandle, CURLOPT_POST, 1L);
// set data size before copy
curl_easy_setopt(curlHandle, CURLOPT_POSTFIELDSIZE_LARGE, m_json.size());
// set json data; I use EXACTLY the same string as in command line
curl_easy_setopt(curlHandle, CURLOPT_COPYPOSTFIELDS, m_json.c_str());
// set user data for getting it in response
// curl_easy_setopt(curlHandle, CURLOPT_WRITEDATA, userData); // pointer to a custom struct
// set callback function for getting response
curl_easy_setopt(curlHandle, CURLOPT_WRITEFUNCTION, callback); // some callback
// send request
curl_easy_perform(curlHandle);
curl_slist_free_all(headers);
curl_easy_cleanup(curlHandle);
curl_global_cleanup();
return 0;
}

In windows, you must init the winsock stuff with below function
curl_global_init(CURL_GLOBAL_ALL);

The problem was solved tanks to Rocki's and drew010's comments.
I have removed CURLOPT_CUSTOMREQUEST, CURLOPT_UPLOAD and CURLOPT_NOSIGNAL setting statements as there is no need of them.
I have also removed the line for setting CURLOPT_POSTFIELDSIZE_LARGE, although it works fine if it is set before setting CURLOPT_COPYPOSTFIELDS.
If the size has not been set prior to CURLOPT_COPYPOSTFIELDS, the data is assumed to be a zero terminated string; else the stored size informs the library about the byte count to copy. In any case, the size must not be changed after CURLOPT_COPYPOSTFIELDS, unless another CURLOPT_POSTFIELDS or CURLOPT_COPYPOSTFIELDS option is issued. (See: curl.haxx.se/libcurl/c/CURLOPT_COPYPOSTFIELDS.html)

Related

Creating a shared link for a Dropbox file using curl & C++

I want to create a shared link for a dropbox file using curl & C++ on a windows 10 desktop.
I've already manage to upload the file to a dropbox folder using curl & C++.
When I try to create the link with command line it works with
curl -X POST https://api.dropboxapi.com/2/sharing/create_shared_link_with_settings --header "Authorization: Bearer <TOKEN>" --header "Content-Type: application/json" --data "{\"path\":\"path_of_the_file\"}"
but when I use this code to do the same in C++ it hangs at < HTTP/1.1 100 Continue
Here is my code :
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if (curl) {
string readBuffer;
printf("Running curl test get shared link.\n");
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE); //no ssl
struct curl_slist *headers = NULL; // init to NULL is important
headers = curl_slist_append(headers, "Authorization: Bearer <TOKEN>");
headers = curl_slist_append(headers, "Content-Type: ");
headers = curl_slist_append(headers, "Dropbox-API-Arg: {\"path\":\"path_of_file\"}");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_POST, true);
curl_easy_setopt(curl, CURLOPT_VERBOSE, true);
curl_easy_setopt(curl, CURLOPT_URL, "https://api.dropboxapi.com/2/sharing/create_shared_link_with_settings");
// Perform the request, res will get the return code
res = curl_easy_perform(curl);
// Check for errors
if (res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
// always cleanup
curl_easy_cleanup(curl);
cout << readBuffer << endl;
printf("\nFinished curl test.\n");
}
curl_global_cleanup();
printf("Done get shared link!\n");
I've tried with content-type : application/json and adding fields but I can't reproduce what I'm doing with the command line
There are errors in the code, causing undefined behaviour.
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_easy_setopt(curl, CURLOPT_POST, true);
curl_easy_setopt(curl, CURLOPT_VERBOSE, true);
cUrl has a C API, C does not have overloads. Thus curl_easy_setopt is a multi-arg function, and the third argument type depends on the second argument value. CURLOPT_SSL_VERIFYPEER and CURLOPT_VERBOSE require long values, you pass the values as int. The size of the third argument is important exactly like this is in *printf functions. The proper calls must be
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(curl, CURLOPT_POST, 1L);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
The second. 100 Continue means you have not passed the data for the POST request. The server received request and waits for further data. See
libcurl example - http-post.c.
I can't reproduce what I'm doing with the command line
Add the --libcurl to the command line for getting a C code performing the same actions as in the command line.
Thank you S.M. Here is the code that works
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if (curl) {
string readBuffer;
printf("Running curl test get shared link.\n");
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L); //no ssl
struct curl_slist *headers = NULL; // init to NULL is important
headers = curl_slist_append(headers, "Authorization: Bearer <TOKEN>");
headers = curl_slist_append(headers, "Content-Type: application/json");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_POST, 1L);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "{\"path\":\"path_to_the_file"}");
curl_easy_setopt(curl, CURLOPT_URL, "https://api.dropboxapi.com/2/sharing/create_shared_link_with_settings");
// Perform the request, res will get the return code
res = curl_easy_perform(curl);
// Check for errors
if (res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
// always cleanup
curl_easy_cleanup(curl);
cout << readBuffer << endl;
printf("\nFinished curl test.\n");
}
curl_global_cleanup();
printf("Done get shared link!\n");

curl PATCH to update value works as a curl command but not in libcurl c++, any ideas what is wrong?

I am trying to replicate the following curl command in c++ code using the curl library but with no luck.
The curl command is (the url is an actual url I am just hiding it):
curl -iX PATCH '*URL*/attrs/topicData' \
-H 'Content-Type: application/json' \
-H 'Link: <http://context-provider:3000/data-models/ngsi-context.jsonld>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \
--data-raw '{
"value": "Hi, new data test",
"type": "Property"
}'
This works perfectly fine and updates the value as required. My issues is that I can't replicate it in c++ code. I am using the nlohmann json library just in case that helps.
My c++ code is:
json ent={
{"type","Property"},
{"value","updated successfully"}
};
curl_global_init(CURL_GLOBAL_DEFAULT);
std::string json_entity = ent.dump();
curl = curl_easy_init();
if (curl) {
// Add headers
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "Content-Type: application/json");
headers = curl_slist_append(headers, R"(Link: <http://context-provider:3000/data-models/ngsi-context.jsonld>; rel="http://w3.org/ns/json-ld#context"; type="application/ld+json")");
// Set custom headers
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
// Set URL
curl_easy_setopt(curl, CURLOPT_URL, "*URL*/attrs/topicData");
// Set request type
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PATCH");
// Set values
curl_easy_setopt(curl, CURLOPT_POSTFIELDS,json_entity);
// Perform the request which prints to stdout
result = curl_easy_perform(curl);
// Error check
if (result != CURLE_OK) {
std::cerr << "Error during curl request: "
<< curl_easy_strerror(result) << std::endl;
}
//Free header list
curl_slist_free_all(headers);
curl_easy_cleanup(curl);
}
else {
std::cerr << "Error initializing curl." << std::endl;
}
The error that I am getting is:
"type":"http://uri.etsi.org/ngsi-ld/errors/InvalidRequest",
"title":"Invalid request.",
"details":"Invalid request."
I think my issue is at set values command but I am not sure what the problem is.
Can anyone please advice me on what I am doing wrong?
CURLOPT_POSTFIELDS expects a char* but you are supplying a std::string.
This should be working better:
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, json_entity.data());

How to perform GET encoded JSON?

What I want to do is to perform CURL request with parameters and values by using GET method but using JSON.
I'm trying to perform the following:
curl -X GET \
-H "X-Parse-Application-Id: 12345_Example" \
-H "X-Parse-REST-API-Key: abcde_Example" \
-G \
--data-urlencode "where={ \"pin\":\"A string\" }" \
https://urlExample/classes/Pins
as you can see the where URL parameter constraining the value for keys should be encoded JSON.
This is my code:
std::size_t callback(
const char* in,
std::size_t size,
std::size_t num,
char* out)
{
std::string data(in, (std::size_t) size * num);
*((std::stringstream*) out) << data;
return size * num;
}
public: Json::Value query(const char* serverAddress, const char* applicationId, const char* restAPIKey) {
CURL* curl = curl_easy_init();
curl_slist* headerlist = NULL;
headerlist = curl_slist_append(headerlist, applicationId);
headerlist = curl_slist_append(headerlist, restAPIKey);
// Set HEADER.
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);
// Set remote URL.
curl_easy_setopt(curl, CURLOPT_URL, serverAddress);
// Don't bother trying IPv6, which would increase DNS resolution time.
curl_easy_setopt(curl, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4);
// Don't wait forever, time out after 10 seconds.
curl_easy_setopt(curl, CURLOPT_TIMEOUT, 10);
// Follow HTTP redirects if necessary.
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
// Response information.
int httpCode(0);
std::stringstream httpData;
// Hook up data handling function.
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, callback);
// Hook up data container (will be passed as the last parameter to the
// callback handling function). Can be any pointer type, since it will
// internally be passed as a void pointer.
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &httpData);
// Run our HTTP GET command, capture the HTTP response code, and clean up.
curl_easy_perform(curl);
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpCode);
curl_easy_cleanup(curl);
if (httpCode == 200) {
// Response looks good - done using Curl now. Try to parse the results.
Json::Value jsonData;
Json::CharReaderBuilder jsonReader;
std::string errs;
if (Json::parseFromStream(jsonReader, httpData, &jsonData, &errs)) {
return jsonData["results"];
}
else {
std::cout << "Could not parse HTTP data as JSON" << std::endl;
std::cout << "HTTP data was:\n" << httpData.str() << std::endl;
return NULL;
}
}
else {
std::cout << "Couldn't GET from " << serverAddress << " - exiting" << std::endl;
return NULL;
}
}
What should I include in my code in order to perform the GET method with encoded JSON?
According to the documentation of the Server API I'm using, when reading objects, this is what it says for curl:
back4app API Reference
READING OBJECTS:
To retrieve an object, you'll need to send a GET request to its class
endpoint with your app's credentials in the headers and the query
parameters in the URL parameters. This task can be easily accomplished
just by calling the appropriated method of your preferred Parse SDK.
Please check how to do it in the right panel of this documentation.
Request URL https://parseapi.back4app.com/classes/Pins
Method GET
Headers X-Parse-Application-Id:
BCrUQVkk80pCdeImSXoKXL5ZCtyyEZwbN7mAb11f
X-Parse-REST-API-Key: swrFFIXJlFudtF3HkZPtfybDFRTmS7sPwvGUzQ9w
Parameters A where URL parameter constraining the value for keys. It
should be encoded JSON.
Success Response Status 200 OK
Headers content-type: application/json;
Body a JSON object that contains a results field with a JSON array
that lists the objects.
EDIT:
Based on: Daniel Stenberg's answer I tried the following:
std::string temp = "where={ \"pin\":\"A string\" }";
char* encoded = curl_easy_escape(curl, temp.c_str(), temp.length());
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE_LARGE, std::strlen(encoded));
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, encoded);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "GET");
But no success. Should libcurl update their API and include such a feature for this case?
OK -- I am going to answer this one more time. This time correctly. I glossed over the fact that you posted the documentation in your question. Totally skipped it. No idea why my brain does that. Maybe it hates documentation and instinctively skips it.
So, the answer to your question is quite simple.
Keep your original code that's in your question (totally ignore the code that you posted in your Edit, it's totally wrong), but instead of doing this:
curl_easy_setopt(curl, CURLOPT_URL, serverAddress);
Do this:
const std::string whereQuery(curl_easy_escape(curl, "{ \"pin\":\"A string\" }", 0));
const std::string url("https://parseapi.back4app.com/classes/Pins?where=" + whereQuery);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
Sorry for dragging that out. I need to read questions better.

What is the correct BING Api POST url for batch url submission, using libcurl?

I am trying to submit a list of urls to BING webmaster. IN CPP
According to BING:
What is URL Submission API?
Easy to plug-in API solution that websites can call to notify Bing whenever
website contents is updated or created allowing instant crawling, indexing
and discovery of your site content.
I understand that need to send a POST request, JSON request sample:
POST /webmaster/api.svc/json/SubmitUrlbatch?​apikey=sampleapikeyEDECC1EA4AE341CC8B6 HTTP/1.1​
Content-Type: application/json; charset=utf-8​
Host: ssl.bing.com​
{
"siteUrl":"http://yoursite.com",​
"urlList":[
"http://yoursite.com/url1",
"http://yoursite.com/url2",
"http://yoursite.com/url3"
]
}
I have written the following, using libcurl to send a POST request.
std::string curl_post_json(const std::string url, const std::string json) {
CURL* curl;
CURLcode res;
std::string ret;
struct curl_slist* header = NULL;
std::string content_len = "Content-Length: " + std::to_string(json.size());
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if (curl) {
header = curl_slist_append(header, "Content-Type: application/json; charset=utf-8");
header = curl_slist_append(header, content_len.c_str());
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, header);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, json.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_response);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &ret);
res = curl_easy_perform(curl);
if (res != CURLE_OK) {
ret = curl_easy_strerror(res);
}
curl_easy_cleanup(curl);
}
curl_global_cleanup();
return ret;
note: write_response is a simple function (pointer) to copy response to string.
I use the following url:
https://ssl.bing.com/webmaster/api.svc/json/SubmitUrlbatch?apikey=<mykey>
but, receive:
{"ExceptionType"="System.InvalidOperationException","Message"="Authenticationfailed.","StackTrace"=null**strong text**}
What is the proper url for the POST submission?
I have the function working now. Needed to add the above.
header = curl_slist_append(header,"Host: ssl.bing.com");
Also the urlList field must not be empty, if it is empty an error response is returned.

libcurl post - underscore replaced with space (ubuntu)

I'm trying to send email and password in a post request to my express nodejs server using curl c++. The post data has '_' changed to ' ' when I log it from server.
char emailtext[50];
char passwordtext[50];
int emailstrlen = wcstombs(emailtext, email->getText(), 50);
int passwordstrlen = wcstombs(passwordtext, password->getText(), 50);
long totalsize = emailstrlen + passwordstrlen;
strcat(emailtext, ":");
strcat(emailtext, passwordtext);
// "myemail#yahoo.com:mypassword\0"
curl_global_init(CURL_GLOBAL_ALL);
CURLcode res;
CURL* curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
curl_easy_setopt(curl, CURLOPT_URL, "https://localhost:3000/login");
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
struct curl_slist *headers=NULL;
headers = curl_slist_append(headers, "Content-Type:text/plain; charset=utf-8");
cout << emailtext << endl;
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, emailtext);
/* pass our list of custom made headers */
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_perform(curl); /* post away! */
curl_slist_free_all(headers); /* free the header list */
curl_global_cleanup();
return true;
express nodejs server:
app.post('/login', bodyParser.text(), function (req, res) {
console.log("we got the post request for /login");
console.log("logging the body!");
console.log(req.body);
res.header('Content-type', 'text/plain');
return res.end('<h1>Hello, Secure World!</h1>');
});
say char* emailtext = "cool_stewj#yahoo.com:lololol"
output from log on server will then be "cool stewj#yahoo.com:lololol"
What's happening? I've tried url encoding the data which turned out to be silly and pointless. How do I get that underscore?
The terminal in ubuntu apparently doesn't show underscores, so any log output will have underscores replaced with spaces.
Btw that's a bad example of string code in beginning. Possible to just use one buffer. Just pointing it out.