How to perform GET encoded JSON? - c++

What I want to do is to perform CURL request with parameters and values by using GET method but using JSON.
I'm trying to perform the following:
curl -X GET \
-H "X-Parse-Application-Id: 12345_Example" \
-H "X-Parse-REST-API-Key: abcde_Example" \
-G \
--data-urlencode "where={ \"pin\":\"A string\" }" \
https://urlExample/classes/Pins
as you can see the where URL parameter constraining the value for keys should be encoded JSON.
This is my code:
std::size_t callback(
const char* in,
std::size_t size,
std::size_t num,
char* out)
{
std::string data(in, (std::size_t) size * num);
*((std::stringstream*) out) << data;
return size * num;
}
public: Json::Value query(const char* serverAddress, const char* applicationId, const char* restAPIKey) {
CURL* curl = curl_easy_init();
curl_slist* headerlist = NULL;
headerlist = curl_slist_append(headerlist, applicationId);
headerlist = curl_slist_append(headerlist, restAPIKey);
// Set HEADER.
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);
// Set remote URL.
curl_easy_setopt(curl, CURLOPT_URL, serverAddress);
// Don't bother trying IPv6, which would increase DNS resolution time.
curl_easy_setopt(curl, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4);
// Don't wait forever, time out after 10 seconds.
curl_easy_setopt(curl, CURLOPT_TIMEOUT, 10);
// Follow HTTP redirects if necessary.
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
// Response information.
int httpCode(0);
std::stringstream httpData;
// Hook up data handling function.
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, callback);
// Hook up data container (will be passed as the last parameter to the
// callback handling function). Can be any pointer type, since it will
// internally be passed as a void pointer.
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &httpData);
// Run our HTTP GET command, capture the HTTP response code, and clean up.
curl_easy_perform(curl);
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpCode);
curl_easy_cleanup(curl);
if (httpCode == 200) {
// Response looks good - done using Curl now. Try to parse the results.
Json::Value jsonData;
Json::CharReaderBuilder jsonReader;
std::string errs;
if (Json::parseFromStream(jsonReader, httpData, &jsonData, &errs)) {
return jsonData["results"];
}
else {
std::cout << "Could not parse HTTP data as JSON" << std::endl;
std::cout << "HTTP data was:\n" << httpData.str() << std::endl;
return NULL;
}
}
else {
std::cout << "Couldn't GET from " << serverAddress << " - exiting" << std::endl;
return NULL;
}
}
What should I include in my code in order to perform the GET method with encoded JSON?
According to the documentation of the Server API I'm using, when reading objects, this is what it says for curl:
back4app API Reference
READING OBJECTS:
To retrieve an object, you'll need to send a GET request to its class
endpoint with your app's credentials in the headers and the query
parameters in the URL parameters. This task can be easily accomplished
just by calling the appropriated method of your preferred Parse SDK.
Please check how to do it in the right panel of this documentation.
Request URL https://parseapi.back4app.com/classes/Pins
Method GET
Headers X-Parse-Application-Id:
BCrUQVkk80pCdeImSXoKXL5ZCtyyEZwbN7mAb11f
X-Parse-REST-API-Key: swrFFIXJlFudtF3HkZPtfybDFRTmS7sPwvGUzQ9w
Parameters A where URL parameter constraining the value for keys. It
should be encoded JSON.
Success Response Status 200 OK
Headers content-type: application/json;
Body a JSON object that contains a results field with a JSON array
that lists the objects.
EDIT:
Based on: Daniel Stenberg's answer I tried the following:
std::string temp = "where={ \"pin\":\"A string\" }";
char* encoded = curl_easy_escape(curl, temp.c_str(), temp.length());
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE_LARGE, std::strlen(encoded));
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, encoded);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "GET");
But no success. Should libcurl update their API and include such a feature for this case?

OK -- I am going to answer this one more time. This time correctly. I glossed over the fact that you posted the documentation in your question. Totally skipped it. No idea why my brain does that. Maybe it hates documentation and instinctively skips it.
So, the answer to your question is quite simple.
Keep your original code that's in your question (totally ignore the code that you posted in your Edit, it's totally wrong), but instead of doing this:
curl_easy_setopt(curl, CURLOPT_URL, serverAddress);
Do this:
const std::string whereQuery(curl_easy_escape(curl, "{ \"pin\":\"A string\" }", 0));
const std::string url("https://parseapi.back4app.com/classes/Pins?where=" + whereQuery);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
Sorry for dragging that out. I need to read questions better.

Related

cURL - Uploading string with a "+" will replace "+" with " "

Yeah weird title, I don't know what it should be.
I have a dll, and after a long time trying and finally figuring out how I could upload things to a webhook, it finally works. But, when I try to upload a string which contains a +, the + will be replaced with a " ".
Example:
Uploaded: FRvERUb9xgMlP4BS+bm+t+CLI6cG026vbtev2gSbBYM=
Received: FRvERUb9xgMlP4BS bm t CLI6cG026vbtev2gSbBYM=
I want to know how I can upload it so that the thing will also send the characters +.
The code for uploading:
void sendmsg() {
CURL* curl;
CURLcode res;
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if (curl) {
curl_easy_setopt(curl, CURLOPT_URL, "my webhook");
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "content=FRvERUb9xgMlP4BS+bm+t+CLI6cG026vbtev2gSbBYM=");
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
curl_global_cleanup();
}
This POST is a normal application/x-www-form-urlencoded kind (and libcurl will set that Content-Type by default when this option is used), which is commonly used by HTML forms.
You can use curl_easy_escape to url-encode your data, if necessary. It returns a pointer to an encoded string that can be passed as postdata.
if (curl) {
curl_easy_setopt(curl, CURLOPT_URL, "my webhook");
char* content = curl_easy_escape(curl, "FRvERUb9xgMlP4BS+bm+t+CLI6cG026vbtev2gSbBYM=", 0);
if (content) {
curl_easy_setopt(curl, CURLOPT_COPYPOSTFIELDS, ("content="s + content).c_str());
curl_free(content);
res = curl_easy_perform(curl);
}
curl_easy_cleanup(curl);
}

How to connect using CURL?

I'm learning how to use CURL properly, and according to all the examples (the documentation is a pain) my code should work, but for some reason sometimes it connects and other times it won't.
I did check if there was a firewall problem, or the antivirus interfering, but both are turn off and the problem persists.
The main idea is to connect to a local server (rpi), and in the future to an external server for backup/updates.
My code is as follows. Here's the callback function, and the actual function that does all the work, the different URLs are for example purposes.
static std::size_t callback(const char* in,std::size_t size, std::size_t num, std::string* out){
Silo* silo = new Silo();
const std::size_t totalBytes(size * num);
std::string data = std::to_string(totalBytes);
silo->Log("Total Bytes recive " + QString::fromStdString(data));
out->append(in, totalBytes);
return totalBytes;
}
void Server::RPI_Request(){
Silo* silo = new Silo();
//curl_global_init(CURL_GLOBAL_ALL);
CURL *curl = curl_easy_init();
const std::string url_A("http://date.jsontest.com/");
const std::string url_B("https://jsonplaceholder.typicode.com/todos/1");
const std::string url_C("https://www.google.com/");
const std::string url_D("https://stackoverflow.com/");
if (curl){
CURLcode res;
// set Ip Direction
curl_easy_setopt(curl, CURLOPT_URL, url_C.c_str() );
// Don't bother trying IPv6, which would increase DNS resolution time.
curl_easy_setopt(curl, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4);
// Don't wait forever, time out after 10 seconds.
silo->Log("antes de timeout");
curl_easy_setopt(curl, CURLOPT_TIMEOUT, 10);
// Follow HTTP redirects if necessary.
//curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
// Response information.
long httpCode(0);
std::unique_ptr<std::string> httpData(new std::string());
// Hook up data handling function.
silo->Log("antes de write function");
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, callback);
// Hook up data container (will be passed as the last parameter to the
// callback handling function). Can be any pointer type, since it will
// internally be passed as a void pointer.
curl_easy_setopt(curl, CURLOPT_WRITEDATA, httpData.get());
// Run our HTTP GET command, capture the HTTP response code, and clean up.
silo->Log("antes de easy perform");
res = curl_easy_perform(curl);
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpCode);
silo->Log("Respuesta de httpCode: " + QString::number(httpCode));
if (res != CURLE_OK){
silo->Log("Hay pedo no se conecto " + QString::fromStdString(url_C) );
} else {
silo->Log("Coneccion establecida con " + QString::fromStdString(url_C));
}
curl_easy_cleanup(curl);
//curl_global_cleanup();
}
}

curl PATCH to update value works as a curl command but not in libcurl c++, any ideas what is wrong?

I am trying to replicate the following curl command in c++ code using the curl library but with no luck.
The curl command is (the url is an actual url I am just hiding it):
curl -iX PATCH '*URL*/attrs/topicData' \
-H 'Content-Type: application/json' \
-H 'Link: <http://context-provider:3000/data-models/ngsi-context.jsonld>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \
--data-raw '{
"value": "Hi, new data test",
"type": "Property"
}'
This works perfectly fine and updates the value as required. My issues is that I can't replicate it in c++ code. I am using the nlohmann json library just in case that helps.
My c++ code is:
json ent={
{"type","Property"},
{"value","updated successfully"}
};
curl_global_init(CURL_GLOBAL_DEFAULT);
std::string json_entity = ent.dump();
curl = curl_easy_init();
if (curl) {
// Add headers
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "Content-Type: application/json");
headers = curl_slist_append(headers, R"(Link: <http://context-provider:3000/data-models/ngsi-context.jsonld>; rel="http://w3.org/ns/json-ld#context"; type="application/ld+json")");
// Set custom headers
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
// Set URL
curl_easy_setopt(curl, CURLOPT_URL, "*URL*/attrs/topicData");
// Set request type
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PATCH");
// Set values
curl_easy_setopt(curl, CURLOPT_POSTFIELDS,json_entity);
// Perform the request which prints to stdout
result = curl_easy_perform(curl);
// Error check
if (result != CURLE_OK) {
std::cerr << "Error during curl request: "
<< curl_easy_strerror(result) << std::endl;
}
//Free header list
curl_slist_free_all(headers);
curl_easy_cleanup(curl);
}
else {
std::cerr << "Error initializing curl." << std::endl;
}
The error that I am getting is:
"type":"http://uri.etsi.org/ngsi-ld/errors/InvalidRequest",
"title":"Invalid request.",
"details":"Invalid request."
I think my issue is at set values command but I am not sure what the problem is.
Can anyone please advice me on what I am doing wrong?
CURLOPT_POSTFIELDS expects a char* but you are supplying a std::string.
This should be working better:
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, json_entity.data());

What is the correct BING Api POST url for batch url submission, using libcurl?

I am trying to submit a list of urls to BING webmaster. IN CPP
According to BING:
What is URL Submission API?
Easy to plug-in API solution that websites can call to notify Bing whenever
website contents is updated or created allowing instant crawling, indexing
and discovery of your site content.
I understand that need to send a POST request, JSON request sample:
POST /webmaster/api.svc/json/SubmitUrlbatch?​apikey=sampleapikeyEDECC1EA4AE341CC8B6 HTTP/1.1​
Content-Type: application/json; charset=utf-8​
Host: ssl.bing.com​
{
"siteUrl":"http://yoursite.com",​
"urlList":[
"http://yoursite.com/url1",
"http://yoursite.com/url2",
"http://yoursite.com/url3"
]
}
I have written the following, using libcurl to send a POST request.
std::string curl_post_json(const std::string url, const std::string json) {
CURL* curl;
CURLcode res;
std::string ret;
struct curl_slist* header = NULL;
std::string content_len = "Content-Length: " + std::to_string(json.size());
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if (curl) {
header = curl_slist_append(header, "Content-Type: application/json; charset=utf-8");
header = curl_slist_append(header, content_len.c_str());
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, header);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, json.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_response);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &ret);
res = curl_easy_perform(curl);
if (res != CURLE_OK) {
ret = curl_easy_strerror(res);
}
curl_easy_cleanup(curl);
}
curl_global_cleanup();
return ret;
note: write_response is a simple function (pointer) to copy response to string.
I use the following url:
https://ssl.bing.com/webmaster/api.svc/json/SubmitUrlbatch?apikey=<mykey>
but, receive:
{"ExceptionType"="System.InvalidOperationException","Message"="Authenticationfailed.","StackTrace"=null**strong text**}
What is the proper url for the POST submission?
I have the function working now. Needed to add the above.
header = curl_slist_append(header,"Host: ssl.bing.com");
Also the urlList field must not be empty, if it is empty an error response is returned.

Curl command line works, C++ curl library does not

I'm trying to do some request via curl library of C++. I can successfully do my request and get the correct response via command line, but I cannot get the correct response via C++ code. My command line command looks like this
curl -X POST -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'Authorization: <some_hash_value>' -k <my_full_url> -data '<my_json_string>'
That works fine. Now I try to do the same request in C++ code. My code looks like this
void performRequest(const std::string& json, const void* userData, CallbackFunction callback)
{
struct curl_slist* headers = NULL;
headers = curl_slist_append(headers, "Accept: application/json");
headers = curl_slist_append(headers, "Content-Type: application/json");
headers = curl_slist_append(headers, (std::string("Authorization: ") + m_authorization).c_str());
CURL* curlHandle = curl_easy_init();
if (!curlHandle)
{
std::cerr << "Curl handler initialization failed";
}
curl_easy_setopt(curlHandle, CURLOPT_NOSIGNAL, 1);
curl_easy_setopt(curlHandle, CURLOPT_HTTPHEADER, headers);
// specify target URL, and note that this URL should include a file name, not only a directory
curl_easy_setopt(curlHandle, CURLOPT_URL, m_url.c_str());
// enable uploading
curl_easy_setopt(curlHandle, CURLOPT_UPLOAD, 1L);
// set HTTP method to POST
curl_easy_setopt(curlHandle, CURLOPT_CUSTOMREQUEST, "POST");
// set json data; I use EXACTLY the same string as in command line
curl_easy_setopt(curlHandle, CURLOPT_COPYPOSTFIELDS, json.c_str());
// set data size
curl_easy_setopt(curlHandle, CURLOPT_POSTFIELDSIZE_LARGE, json.size());
// set user data for getting it in response
curl_easy_setopt(curlHandle, CURLOPT_WRITEDATA, userData); // pointer to a custom struct
// set callback function for getting response
curl_easy_setopt(curlHandle, CURLOPT_WRITEFUNCTION, callback); // some callback
// send request
curl_easy_perform(curlHandle);
curl_easy_cleanup(curlHandle);
curl_slist_free_all(headers);
}
However, for some reason I get an error in the response from the server, from which I can assume that my code's request is not equivalent to command line's command. It seems that body is not sent. I cannot see my request Json body when I use CURLOPT_DEBUGFUNCTION for dumping debug info.
What is the problem here? What am I doing wrong? Any ideas?
Here is sample code that should work for you.
Notice that I:
Removed CURLOPT_UPLOAD as it does not seem as you are actually uploading something but rather just doing a simple POST.
Changed CURLOPT_CUSTOMREQUEST to CURLOPT_POST (not that it should matter), but I find it cleaner.
Reordered CURLOPT_POSTFIELDSIZE_LARGE and CURLOPT_COPYPOSTFIELDS
Removed the CURLOPT_WRITEDATA line for the sake of this sample code.
I have tested the following only by connecting to an instance of nc -l localhost 80
static size_t callback(char *ptr, size_t size, size_t nmemb, void *userdata)
{
string s(ptr);
cout << s << endl;
return size * nmemb;
}
int main(int argc, char** argv)
{
string m_authorization("PWNED");
string m_url("http://localhost");
string m_json("{}");
curl_global_init(CURL_GLOBAL_ALL);
CURL* curlHandle = curl_easy_init();
struct curl_slist* headers = nullptr;
headers = curl_slist_append(headers, "Accept: application/json");
headers = curl_slist_append(headers, "Content-Type: application/json");
headers = curl_slist_append(headers, (std::string("Authorization: ") + m_authorization).c_str());
curl_easy_setopt(curlHandle, CURLOPT_NOSIGNAL, 1);
curl_easy_setopt(curlHandle, CURLOPT_HTTPHEADER, headers);
// specify target URL, and note that this URL should include a file name, not only a directory
curl_easy_setopt(curlHandle, CURLOPT_URL, m_url.c_str());
// <= You are not uploading anything actually, this is a simple POST with payload
// enable uploading
// curl_easy_setopt(curlHandle, CURLOPT_UPLOAD, 1L);
// set HTTP method to POST
curl_easy_setopt(curlHandle, CURLOPT_POST, 1L);
// set data size before copy
curl_easy_setopt(curlHandle, CURLOPT_POSTFIELDSIZE_LARGE, m_json.size());
// set json data; I use EXACTLY the same string as in command line
curl_easy_setopt(curlHandle, CURLOPT_COPYPOSTFIELDS, m_json.c_str());
// set user data for getting it in response
// curl_easy_setopt(curlHandle, CURLOPT_WRITEDATA, userData); // pointer to a custom struct
// set callback function for getting response
curl_easy_setopt(curlHandle, CURLOPT_WRITEFUNCTION, callback); // some callback
// send request
curl_easy_perform(curlHandle);
curl_slist_free_all(headers);
curl_easy_cleanup(curlHandle);
curl_global_cleanup();
return 0;
}
In windows, you must init the winsock stuff with below function
curl_global_init(CURL_GLOBAL_ALL);
The problem was solved tanks to Rocki's and drew010's comments.
I have removed CURLOPT_CUSTOMREQUEST, CURLOPT_UPLOAD and CURLOPT_NOSIGNAL setting statements as there is no need of them.
I have also removed the line for setting CURLOPT_POSTFIELDSIZE_LARGE, although it works fine if it is set before setting CURLOPT_COPYPOSTFIELDS.
If the size has not been set prior to CURLOPT_COPYPOSTFIELDS, the data is assumed to be a zero terminated string; else the stored size informs the library about the byte count to copy. In any case, the size must not be changed after CURLOPT_COPYPOSTFIELDS, unless another CURLOPT_POSTFIELDS or CURLOPT_COPYPOSTFIELDS option is issued. (See: curl.haxx.se/libcurl/c/CURLOPT_COPYPOSTFIELDS.html)