C++ libcurl not posting data - c++

I am working on a C++ library which is making use of the curl library to perform an HTTP POST request to a webservice.
I have a function which is responsible for performing curl and processing the response, and this function calls another function to set curl up as it can be used from multiple places with the same setup.
Its successfully connects to the web service and I can see the response returned however, no post data is sent to the web service.
Below the code that I have to perform the HTTP request.
The first function is the function that does the curl perform, it calls a function to init the curl library and then return the pointer to it for use.
string response;
struct curl_slist *list = NULL;
const char * jsonString = ddEvent.getDDEventAsJSONString().c_str();
CURL *curl = this->initCurl(ddEvent, &response, list, &jsonString);
if (curl == NULL)
{
return false;
}
list = curl_slist_append(list, "Content-Type: application/json");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, list);
CURLcode res = curl_easy_perform(curl);
The function that initialises curl is as follows:
CURL * MyClass::initCurl(DDEvent ddEvent, string *response, struct curl_slist *list, const char **jsonString)
{
CURL *curl = NULL;
curl = curl_easy_init();
if (curl == NULL)
{
cout << "Failed to initialise curl" << endl;
return NULL;
}
stringstream url_stream;
url_stream << "http://192.168.1.123";
string url= dd_url_stream.str();
cout << "Using URL: " << url<< endl;
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 1);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 1);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, response);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &MyClass::curlResponseWriteCallback);
curl_easy_setopt(curl, CURLOPT_HEADER, 1);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
curl_easy_setopt(curl, CURLOPT_POST, 1);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, *jsonString);
return curl;
}
Also, the post data is a JSON string, its not a key/value posted form data.
Below is the output from the the verbose mode of curl
Using JSON string {"title":"HTTP Event Teesg","text":"Test warning event","alert_type":"success","tags":["simple_string_tag"]}
Using URL: http://192.168.1.123
* About to connect() to 192.168.1.123 port 80 (#0)
* Trying 192.168.1.123...
* Connected to 192.168.1.123 (192.168.1.123) port 80 (#0)
> POST / HTTP/1.1
Host: 192.168.1.123
Accept: */*
Content-Type: application/json
Content-Length: 0
< HTTP/1.1 200 OK
< Date: Sun, 04 Feb 2018 20:11:12 GMT
< Server: Apache/2.4.6 (CentOS) PHP/5.6.32
< X-Powered-By: PHP/5.6.32
< X-XSS-Protection: 1; mode=block
< Content-Length: 52
< Content-Type: text/html; charset=UTF-8
<

The Content-Length: 0 in the outgoing request suggests that the data you pass to CURLOPT_POSTFIELDS has zero length.
Note that CURLOPT_POSTFIELDS doesn't copy the data, it just points to your memory so you might want to make sure the data is still around for the entire time libcurl needs it. Since you haven't shown us the entire program we can't really tell you exactly how this happens for you.
Finally: why not just pass a 'char *' to initCurl instead of 'char **' that you need to dereference within the method?. It looks odd to me.

It seems that ddEvent.getDDEventAsJSONString() returns a temporary std::string object.
Save it in a local variable and then pass it as reference.
std::string jsonString = ddEvent.getDDEventAsJSONString();
this->initCurl(ddEvent, &response, list, jsonString);
CURL * MyClass::initCurl(DDEvent ddEvent, string *response, struct curl_slist *list, const std::string& jsonString)
You have to keep the string alive until you call curl_easy_perform() since CURLOPT_POSTFIELDS doesn't copy the string into libcurl.
Strings passed to libcurl as 'char *' arguments, are copied by the library; thus the string storage associated to the pointer argument may be overwritten after curl_easy_setopt returns. The only exception to this rule is really CURLOPT_POSTFIELDS, but the alternative that copies the string CURLOPT_COPYPOSTFIELDS has some usage characteristics you need to read up on.
There is no problem in your code where you call curl_easy_perform in the life scope of jsonString. However, if you intend to call the function afterward, instead of CURLOPT_POSTFIELDS use CURLOPT_COPYPOSTFIELDS which copy and memorize the data to post.

Related

How to perform GET encoded JSON?

What I want to do is to perform CURL request with parameters and values by using GET method but using JSON.
I'm trying to perform the following:
curl -X GET \
-H "X-Parse-Application-Id: 12345_Example" \
-H "X-Parse-REST-API-Key: abcde_Example" \
-G \
--data-urlencode "where={ \"pin\":\"A string\" }" \
https://urlExample/classes/Pins
as you can see the where URL parameter constraining the value for keys should be encoded JSON.
This is my code:
std::size_t callback(
const char* in,
std::size_t size,
std::size_t num,
char* out)
{
std::string data(in, (std::size_t) size * num);
*((std::stringstream*) out) << data;
return size * num;
}
public: Json::Value query(const char* serverAddress, const char* applicationId, const char* restAPIKey) {
CURL* curl = curl_easy_init();
curl_slist* headerlist = NULL;
headerlist = curl_slist_append(headerlist, applicationId);
headerlist = curl_slist_append(headerlist, restAPIKey);
// Set HEADER.
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);
// Set remote URL.
curl_easy_setopt(curl, CURLOPT_URL, serverAddress);
// Don't bother trying IPv6, which would increase DNS resolution time.
curl_easy_setopt(curl, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4);
// Don't wait forever, time out after 10 seconds.
curl_easy_setopt(curl, CURLOPT_TIMEOUT, 10);
// Follow HTTP redirects if necessary.
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
// Response information.
int httpCode(0);
std::stringstream httpData;
// Hook up data handling function.
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, callback);
// Hook up data container (will be passed as the last parameter to the
// callback handling function). Can be any pointer type, since it will
// internally be passed as a void pointer.
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &httpData);
// Run our HTTP GET command, capture the HTTP response code, and clean up.
curl_easy_perform(curl);
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpCode);
curl_easy_cleanup(curl);
if (httpCode == 200) {
// Response looks good - done using Curl now. Try to parse the results.
Json::Value jsonData;
Json::CharReaderBuilder jsonReader;
std::string errs;
if (Json::parseFromStream(jsonReader, httpData, &jsonData, &errs)) {
return jsonData["results"];
}
else {
std::cout << "Could not parse HTTP data as JSON" << std::endl;
std::cout << "HTTP data was:\n" << httpData.str() << std::endl;
return NULL;
}
}
else {
std::cout << "Couldn't GET from " << serverAddress << " - exiting" << std::endl;
return NULL;
}
}
What should I include in my code in order to perform the GET method with encoded JSON?
According to the documentation of the Server API I'm using, when reading objects, this is what it says for curl:
back4app API Reference
READING OBJECTS:
To retrieve an object, you'll need to send a GET request to its class
endpoint with your app's credentials in the headers and the query
parameters in the URL parameters. This task can be easily accomplished
just by calling the appropriated method of your preferred Parse SDK.
Please check how to do it in the right panel of this documentation.
Request URL https://parseapi.back4app.com/classes/Pins
Method GET
Headers X-Parse-Application-Id:
BCrUQVkk80pCdeImSXoKXL5ZCtyyEZwbN7mAb11f
X-Parse-REST-API-Key: swrFFIXJlFudtF3HkZPtfybDFRTmS7sPwvGUzQ9w
Parameters A where URL parameter constraining the value for keys. It
should be encoded JSON.
Success Response Status 200 OK
Headers content-type: application/json;
Body a JSON object that contains a results field with a JSON array
that lists the objects.
EDIT:
Based on: Daniel Stenberg's answer I tried the following:
std::string temp = "where={ \"pin\":\"A string\" }";
char* encoded = curl_easy_escape(curl, temp.c_str(), temp.length());
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE_LARGE, std::strlen(encoded));
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, encoded);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "GET");
But no success. Should libcurl update their API and include such a feature for this case?
OK -- I am going to answer this one more time. This time correctly. I glossed over the fact that you posted the documentation in your question. Totally skipped it. No idea why my brain does that. Maybe it hates documentation and instinctively skips it.
So, the answer to your question is quite simple.
Keep your original code that's in your question (totally ignore the code that you posted in your Edit, it's totally wrong), but instead of doing this:
curl_easy_setopt(curl, CURLOPT_URL, serverAddress);
Do this:
const std::string whereQuery(curl_easy_escape(curl, "{ \"pin\":\"A string\" }", 0));
const std::string url("https://parseapi.back4app.com/classes/Pins?where=" + whereQuery);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
Sorry for dragging that out. I need to read questions better.

What is the correct BING Api POST url for batch url submission, using libcurl?

I am trying to submit a list of urls to BING webmaster. IN CPP
According to BING:
What is URL Submission API?
Easy to plug-in API solution that websites can call to notify Bing whenever
website contents is updated or created allowing instant crawling, indexing
and discovery of your site content.
I understand that need to send a POST request, JSON request sample:
POST /webmaster/api.svc/json/SubmitUrlbatch?​apikey=sampleapikeyEDECC1EA4AE341CC8B6 HTTP/1.1​
Content-Type: application/json; charset=utf-8​
Host: ssl.bing.com​
{
"siteUrl":"http://yoursite.com",​
"urlList":[
"http://yoursite.com/url1",
"http://yoursite.com/url2",
"http://yoursite.com/url3"
]
}
I have written the following, using libcurl to send a POST request.
std::string curl_post_json(const std::string url, const std::string json) {
CURL* curl;
CURLcode res;
std::string ret;
struct curl_slist* header = NULL;
std::string content_len = "Content-Length: " + std::to_string(json.size());
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if (curl) {
header = curl_slist_append(header, "Content-Type: application/json; charset=utf-8");
header = curl_slist_append(header, content_len.c_str());
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, header);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, json.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_response);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &ret);
res = curl_easy_perform(curl);
if (res != CURLE_OK) {
ret = curl_easy_strerror(res);
}
curl_easy_cleanup(curl);
}
curl_global_cleanup();
return ret;
note: write_response is a simple function (pointer) to copy response to string.
I use the following url:
https://ssl.bing.com/webmaster/api.svc/json/SubmitUrlbatch?apikey=<mykey>
but, receive:
{"ExceptionType"="System.InvalidOperationException","Message"="Authenticationfailed.","StackTrace"=null**strong text**}
What is the proper url for the POST submission?
I have the function working now. Needed to add the above.
header = curl_slist_append(header,"Host: ssl.bing.com");
Also the urlList field must not be empty, if it is empty an error response is returned.

How to get the HTTP response string using Curl in C++

I'm very new to HTTP commands and the libcurl library. I know how to get the HTTP response code but not the HTTP response string. Following is the code snippet that I wrote to get the response code. Any help on how to get the response string will be highly appreciated!!!
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
CURLcode ret = curl_easy_perform(curl);
if (ret != CURLE_OK) {
LOG(INFO) << "Failed to perform the request. "
<< "Return code: " << ret;
return false;
}
std::unique_ptr<int64_t> httpCode(new int64_t);
// Get the last response code.
ret = curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, httpCode.get());
if (ret != CURLE_OK) {
LOG(INFO) << "curl_easy_getinfo failed to retrieve http code. "
<< "Return code: " << ret;
return false;
}
I tried doing this as well to get the HTTP response string in readBuffer.
static size_t WriteCallback(char *contents, size_t size, size_t nmemb, void *userp)
{
((std::string*)userp)->append((char*)contents, size * nmemb);
return size * nmemb;
}
std::string readBuffer;
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
CURLcode ret = curl_easy_perform(curl);
cout << readBuffer << "\n";
But the readBuffer is empty. I don't understand where I am going wrong. Any pointers on how to solve this will be really nice!
There doesn't look to be much wrong with your code. I ran the following code, based on yours, to read the front page of the BBC news website:
#include <iostream>
#include <string>
#include <curl/curl.h>
size_t WriteCallback(char *contents, size_t size, size_t nmemb, void *userp)
{
((std::string*)userp)->append((char*)contents, size * nmemb);
return size * nmemb;
}
int main(int argc, char * argv[])
{
curl_global_init(CURL_GLOBAL_ALL);
CURL* easyhandle = curl_easy_init();
std::string readBuffer;
curl_easy_setopt(easyhandle, CURLOPT_URL, "http://www.bbc.co.uk/news");
curl_easy_setopt(easyhandle, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(easyhandle, CURLOPT_PROXY, "http://my.proxy.net"); // replace with your actual proxy
curl_easy_setopt(easyhandle, CURLOPT_PROXYPORT, 8080L);
curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(easyhandle, CURLOPT_WRITEDATA, &readBuffer);
curl_easy_perform(easyhandle);
std::cout << readBuffer << std::endl;
return 0;
}
... and I got the full HTML response. NB I had to use a proxy, and I enabled verbose mode to see what was happening. Also NB; the HTTP server might be fussy about the URL you give it: if I replace http://www.bbc.co.uk/news with http://www.bbc.co.uk/news/ (i.e. with a trailing slash) then I get no data back (a response length of zero), as noted by the verbose curl output:
Host: www.bbc.co.uk
Accept: */*
Proxy-Connection: Keep-Alive
< HTTP/1.1 301 Moved Permanently
< X-Cache-Action: PASS (no-cache-control)
< Vary: Accept-Encoding
< X-Cache-Age: 0
< Content-Type: text/html;charset=utf-8
< Date: Mon, 10 Jul 2017 16:42:20 GMT
< Location: http://www.bbc.co.uk/news
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< Content-Length: 0
< X-Cache: MISS from barracuda1.[my proxy].net
< Connection: keep-alive
<
* Connection #0 to host [my proxy] left intact
Here, Content-Length is 0 and the WriteCallback() function is never called. Hope this helps.
For the numerical response code, getinfo with CURLINFO_RESPONSE_CODE is the way to go:
long response_code;
curl_easy_getinfo(handle, CURLINFO_RESPONSE_CODE,&response_code);
However there is no equivalent getinfo capture for the server's response text. If you need the server's text, inspect the raw HTTP headers. There are two ways to do this:
Enable writing headers to the payload with CURLOPT_HEADER, then extract the headers from the combined payload, splitting the body on \n\n
Set a header callback function with CURLOPT_HEADERFUNCTION and parse directly from that

C++ LibCurl makes program crash for some reason

I have trouble executing a LibCurl statement, at least it seems. I try to send off a 15MB file to a port where a script is listening that will process the file and return a response. However, before I get any response, the program crashes (only on 1 pptx file of 15MB, which is way larger than 'normal' other files)
However, does anyone see something strange in the LibCurl logs:
* timeout on name lookup is not supported
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 13621 (#0)
> PUT //File processing argument//
Host: localhost:13621
Accept: application/json
Accept: application/json
Content-Length: 15828910
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Tue, 14 Mar 2017 20:37:41 GMT
< Transfer-Encoding: chunked
< Server: Jetty(8.y.z-SNAPSHOT)
<
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact
Process returned -1073741571 (0xC00000FD) execution time : 12.682 s
The code I use:
curl = curl_easy_init();
if (curl) {
CurlResponse = "";
host = "localhost:13621";
LibcurlHeaders = curl_slist_append(LibcurlHeaders, "Expect:");
LibcurlHeaders = curl_slist_append(LibcurlHeaders, "Accept: application/json");
curl_easy_setopt(curl, CURLOPT_URL, (host).c_str());
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1);
curl_easy_setopt(curl, CURLOPT_READDATA, fd);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, LibcurlHeaders);
curl_easy_setopt(curl, CURLOPT_VERBOSE, CurlVerbose);
curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, (curl_off_t)file_info.st_size);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, LibcurlResponse);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &CurlResponse);
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
curl_global_cleanup();
Thanks for your reply

C++ CURL not returning data on specific URL

I want to send CGI command to IP camera.
I send e.a. CGI url to receive camera name and the function returns the right name, so I think my code is fine.
When I send another CGI url from my browser, it is working (camera returns "ok") but when I try to send it with my program, my camera is not returning anything and parameters are not set in camera.
here is my code:
string data; //will hold the url's contents
size_t writeCallback(char* buf, size_t size, size_t nmemb, void* up)
{ for (int c = 0; c<size*nmemb; c++)
{
data.push_back(buf[c]);
}
return size*nmemb; //tell curl how many bytes we handled
}
string send_CGI_command(string username, string password, string port, string URL)
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
curl_global_init(CURL_GLOBAL_ALL); //pretty obvious
curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_PORT , stoll (port,0,0) );
curl_easy_setopt(curl, CURLOPT_USERNAME , username.c_str());
curl_easy_setopt(curl, CURLOPT_USERPWD , (username + ":" + password).c_str());
curl_easy_setopt(curl, CURLOPT_HTTPAUTH , CURLAUTH_BASIC);
curl_easy_setopt(curl, CURLOPT_URL, (URL).c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &writeCallback);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_perform(curl);
curl_easy_cleanup(curl);
curl_global_cleanup();
return data;
}
}
I use this command to request the CGI:
string result = send_CGI_command(get_video_login_cam1(),get_video_password_cam1(),get_video_port_cam1(),"http://" + get_video_ip_cam1() + "/cgi-bin/configManager.cgi?action=setConfig&Locales.DSTEnable=true&Locales.DSTEnd.Day=0&Locales.DSTEnd.Hour=3&Locales.DSTEnd.Minute=0&Locales.DSTEnd.Month=10&Locales.DSTEnd.Week=-1&Locales.DSTEnd.Year=" + std::to_string(1900 + ltm->tm_year) + "&Locales.DSTStart.Day=0&Locales.DSTStart.Hour=2&Locales.DSTStart.Minute=0&Locales.DSTStart.Month=3&Locales.DSTStart.Week=-1&Locales.DSTStart.Year=" + std::to_string(1900 + ltm->tm_year) + "&Locales.TimeFormat=dd-MM-YYYY HH:mm:ss");
cout << endl << result << endl;
This is the debugger info:
* Trying 192.168.1.14...
* Connected to 192.168.1.14 (192.168.1.14) port 80 (#0)
* Server auth using Basic with user 'admin'
> GET /cgi-bin/configManager.cgi?action=setConfig&Locales.DSTEnable=true&Locales.DSTEnd.Day=0&Locales.DSTEnd.Hour=3&Locales.DSTEnd.Minute=0&Locales.DSTEnd.Month=10&Locales.DSTEnd.Week=-1&Locales.DSTEnd.Year=2016&Locales.DSTStart.Day=0&Locales.DSTStart.Hour=2&Locales.DSTStart.Minute=0&Locales.DSTStart.Month=3&Locales.DSTStart.Week=-1&Locales.DSTStart.Year=2016&Locales.TimeFormat=dd-MM-YYYY HH:mm:ss HTTP/1.1
Host: 192.168.1.14
Authorization: Basic YWRtaW46YWRtaW4=
Accept: */*
* Empty reply from server
* Connection #0 to host 192.168.1.14 left intact
I have no idea why my exact same CGI is working in browser and the exact same cgi does not returing anything with CURL. Like I said in beginning, another CGI command to get device name is working in browser and with same CURL code.
Thank you in advance for your suggestions.