"Failed writing body" CURLOPT_WRITEDATA - c++

The below code is to get response from a server using wsdl, here the problem is curl returns response but am unable to print it.
Error:
Failed writing body
Failed writing data
#include<stdio.h>
#include<string.h>
#include"../include/curl.h"
size_t write_data(void *ptr, size_t size, size_t count, void *stream)
{
/* ptr - your string variable.
stream - data chuck you received */
printf("%.*s", size, (char*)stream);
}
int main()
{
int res=0,i=0;
char buffer[4098]="",buff[128]="",buf[256]="",buf7[30]="",buf6[30]="",buf5[30]="";
char machineid[]="SUBANI";
char filename1[50]="";
int refno=0,paymode=0,taxtype=0;
FILE *fbc;
memset(filename1,0,sizeof(filename1));
sprintf(filename1,"/mnt/jffs2/Response_Details1.xml");
lk_dispclr();
lk_disptext(1,0,(unsigned char *)"Sending Request",0);
lk_disptext(2,0,(unsigned char *)"Please Wait",0);
memset(buffer,0,sizeof(buffer));
sprintf(buffer,"<?xml version=\"1.0\" encoding=\"utf-8\"?>\
<soap:Envelope xmlns:soap=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:log=\"http://wsdlclassess.application.sims.test.com\">\
<soap:Header>\
</soap:Header>\
<soap:Body>\
<log:loginMethod>\
<log:loginid>%s</log:loginid>\
<log:password>%s</log:password>\
</log:loginMethod>\
</soap:Body>\
</soap:Envelope>","raja","test");
res=GET_FILE1(buffer,filename1);
return 0;
}
int GET_FILE1(char *buffer,char *filename)
{
CURL *curl;
CURLcode res;
struct curl_slist *headers = NULL;
FILE *out_fd = (FILE *) 0;
char errorbuf[300] = "",tmpbuff[128]="";
char errmsg[256];
int Timeout=120; //Default timeout is = 2 mins
int buffer_size = 0;
char urlbuff[256]="";
char mstr[10240];
memset(urlbuff,0,sizeof(urlbuff));
memset(tmpbuff,0,sizeof(tmpbuff));
buffer_size = strlen(buffer);
strcpy(tmpbuff,"http://10.10.1.111:8081/test_server/services/application?wsdl");
tmpbuff[strlen(tmpbuff)]='\0';
curl = curl_easy_init();
if(curl)
{
out_fd = fopen (filename, "w");
curl_easy_setopt(curl, CURLOPT_FILE, out_fd);
printf("%s:Sign-In Request\n", __func__);
headers = curl_slist_append(headers, "Content-type:application/soap+xml; charset=utf-8; action=\"http://wsdlclassess.application.sims.test.com/loginMethod\"");
curl_easy_setopt(curl, CURLOPT_URL, tmpbuff);
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, mstr);
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, buffer_size);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, buffer);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(curl, CURLOPT_TIMEOUT, Timeout);
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER,errmsg);
printf("The Server%s:Performing Transaction.....\n",__func__);
res = curl_easy_perform(curl);
printf("res=after culreasey perform%d\n",res);
curl_slist_free_all(headers);
curl_easy_cleanup(curl);
printf("\nerrorbuf:%s\n",errmsg);
fclose(out_fd);
if(CURLE_OK != res)
{
puts("error occured is\n" );
//ppp_close();
return -1;
}
}
return 0;
}

The error is that you don't return the correct value from the function, in fact you don't return anything.
Also, the data provided to the function is actually the first ptr argument.
I agree that the documentation is not very clear, but it says:
The size of the data pointed to by ptr is size multiplied with nmemb, it will not be zero terminated.
The above line (emphasis mine) tells you that the data is in ptr which is the first argument in the function declaration provided in the documentation.
The documentation also states:
Return the number of bytes actually taken care of. If that amount differs from the amount passed to your function, it'll signal an error to the library. This will abort the transfer and return CURLE_WRITE_ERROR.
You don't return a value from the function, and so you have undefined behavior with a seemingly random value being returned causing the whole operation to fail. To fix this you should return size * count.
You also uses size to print the string, which is the size of the underlying type used (probably 1), your count variable is the number of characters read by CURL. To be fully working, without invoking more undefined behavior (since the data is not terminated) you should call printf like:
printf("%*.*s", size * count, size * count, ptr);

Related

libcurl downloads no data to buffer

I am using following code to download data from an url to memory (stream). Around 2% chance, the size of the stream is zero. I can download proper data from the same failing url if I try it another time. I am not sure if this is a network issue, CPU usage issue, or it's just the code not covering some corner cases. Please advice. Thanks!
static size_t write_data(char *ptr, size_t size, size_t nmemb, void *userdata)
{
std::vector<uchar> *stream = (std::vector<uchar>*)userdata;
size_t count = size * nmemb;
stream->insert(stream->end(), ptr, ptr + count);
return count;
}
static void CurlUrl(const char* img_url, std::vector<uchar>* stream) {
CURL *curl = curl_easy_init(); // curl_global_init is called eleswhere.
curl_easy_setopt(curl, CURLOPT_NOSIGNAL, 1);
curl_easy_setopt(curl, CURLOPT_URL, img_url);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, stream);
curl_easy_setopt(curl, CURLOPT_TIMEOUT, 10);
CURLcode res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
If it didn't deliver any download data into the buffer via the callback, it means that the transfer either failed or that there was exactly zero bytes to transfer.
Check the return code from curl_easy_perform() as it might actually tell you exactly what happened.
Use CURLOPT_VERBOSE to see what's going on if (1) is not enough.
Use CURLOPT_ERRORBUFFER to get a better error description if it fails if (2) is not enough.

How to read the JSON Error response returned from Server using Libcurl

I have a requirement where I have to read the error response from backend server which returns 500 Internal Server error. The error response is in JSON Format.
Below is the code snippet used in our application
INT CCurlHTTP::HTTPSPost(const CString& endPointUrl, const CString& urlparam,const CString& cookie){
CURL *curl;
CURLcode res;
struct curl_slist *headers=NULL;
char errbuf[CURL_ERROR_SIZE];
curl = curl_easy_init();
get_request req;
req.buffer =0;
req.len =0;
req.buflen =0;
if(curl)
{
//add url, headers, and parameters to the request
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "POST");
curl_easy_setopt(curl, CURLOPT_URL, endPointUrl);
curl_easy_setopt(curl, CURLOPT_DEFAULT_PROTOCOL, "https");
headers = curl_slist_append(headers, m_httpHeadAccept);
headers = curl_slist_append(headers, m_httpContentType);
//callback function used to save response
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, CurlWrite_CallbackFunc_String);
req.buffer = (unsigned char*) malloc(CHUNK_SIZE);
req.buflen = CHUNK_SIZE;
req.len = 0;
curl_easy_setopt(curl,CURLOPT_WRITEDATA, (void *)&req);
if (!cookie.IsEmpty())
{
headers = curl_slist_append(headers, m_DBAuthCertficate); //What is difference between this and line no 118?
CString pCookie = "DBAuthTicket=" + cookie;
curl_easy_setopt(curl,CURLOPT_COOKIE, pCookie);
}
else
{
headers = curl_slist_append(headers, m_OAuthToken);
}
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, urlparam);
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errbuf);
errbuf[0] = 0;
curl_easy_setopt(curl, CURLOPT_BUFFERSIZE, 512000);
curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1L);
res = curl_easy_perform(curl);
if(res != CURLE_OK)
{
/* if errors have occured, tell us wath's wrong with 'result'*/
m_response.Format("%s", curl_easy_strerror(res));
return res;
}
m_response = (char*)req.buffer;
m_errDescription = errbuf;
len = req.len;
buflen = req.buflen;
curl_easy_cleanup(curl);
free(req.buffer);
}
return res;
}
/****************************************************************************
Function: CurlWrite_CallbackFunc_String
Description: Read data from the connected URL
Return: String of data and size
****************************************************************************/
size_t CCurlHTTP::CurlWrite_CallbackFunc_String(void *contents, size_t size, size_t nmemb, void *userdata)
{
size_t rLen = size*nmemb;
get_request* req = (get_request*)userdata;
while(req->buflen < req->len + rLen + 1)
{
req->buffer = (unsigned char*)realloc(req->buffer,req->buflen + CHUNK_SIZE);
req->buflen += CHUNK_SIZE;
}
memcpy(&req->buffer[req->len], contents, rLen);
req->len += rLen;
req->buffer[req->len] = 0;
return rLen;
}
The above code works fine for the Success 200 OK Requests. It reads the JSON Response just fine. However, when I get a 500 Internal Server error, it does not read the JSON Error response that comes along with it. How do I read the JSON response in this scenario?
By setting the CURLOPT_FAILONERROR option to TRUE, you are telling curl_easy_perform() to fail immediately with CURLE_HTTP_RETURNED_ERROR on any HTTP response >= 400. It will not call the CURLOPT_WRITEFUNCTION callback, as it will simply close the connection and not even attempt to read the rest of the response.
To get the response data you want, simply remove the CURLOPT_FAILONERROR option. Curl's default behavior is to deliver the response data to you regardless of the HTTP response code. In which case, curl_easy_perform() will return CURLE_OK, and you can then retrieve the response code using curl_easy_getinfo(CURLINFO_RESPONSE_CODE) to check if the HTTP request was successful or not.
On a side note, since the code shown is written in C++, I would strongly advise you NOT to use a dynamic char[] buffer for get_request::buffer. Not only because you are not handling malloc()/realloc() failures at all, but also because manual memory management should be avoided in C++ in general. Use std::string or std::vector<char> instead, in which case you can eliminate get_request in this code altogether, eg:
INT CCurlHTTP::HTTPSPost(const CString& endPointUrl, const CString& urlparam,const CString& cookie){
...
std::string resp;
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &resp);
...
//curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1L);
res = curl_easy_perform(curl);
if (res != CURLE_OK)
{
/* if errors have occured, tell us what's wrong with 'result'*/
m_response.Format("%s", curl_easy_strerror(res));
curl_easy_cleanup(curl);
return res;
}
m_response = resp.c_str();
m_errDescription = errbuf;
len = resp.size();
buflen = resp.capacity();
curl_easy_cleanup(curl);
return res;
}
size_t CCurlHTTP::CurlWrite_CallbackFunc_String(void *contents, size_t size, size_t nmemb, void *userdata)
{
size_t rLen = size * nmemb;
static_cast<std::string*>(userdata)->append(static_cast<char*>(contents), rLen);
return rLen;
}

How to include an h file locally, without visual studio?

I have a simple cpp code that uses curl.h as it's header file.
I use vscode, and not interested on linking via gui or w/e
How can I make the include statement work properly with the single project?
I got the header file from https://github.com/curl/curl/blob/master/include/curl/curl.h
Folder structure is like this:
/Folder
+-> curl.h
+-> curli.cpp
the code:
#include "curl.h"
using namespace std;
#define YOUR_URL "https://www.insertyourURLhere.com/"
#define USER_AND_PWD "user:password"
static string gs_strLastResponse;
// Callback to gather the response from the server. Comes in chunks (typically 16384 characters at a time), so needs to be stitched together.
size_t function_pt(void *ptr, size_t size, size_t nmemb, void * /*stream*/)
{
gs_strLastResponse += (const char*)ptr;
return size * nmemb;
}
bool CallServerWithCurl(string strData1, strData2, string& strErrorDescription)
{
CURL* curl = curl_easy_init();
if (curl == NULL)
{
strErrorDescription = "Unable to initialise Curl";
return false;
}
curl_easy_setopt(curl, CURLOPT_URL, (YOUR_URL + "get_result").c_str());
curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_easy_setopt(curl, CURLOPT_USERPWD, MY_USER_AND_PWD); // set user name and password for the authentication
char* data1 = curl_easy_escape(curl, strData1.c_str(), 0);
char* data2 = curl_easy_escape(curl, strData2.c_str(), 0);
string strArguments = "id=" + data1 + "&task=" + data2;
const char* my_data = strArguments.c_str();
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, my_data);
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, (long)strlen(my_data)); // if we don't provide POSTFIELDSIZE, libcurl will strlen() by itself
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); // enable verbose for easier tracing
gs_strLastResponse = "";
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, function_pt); // set a callback to capture the server's response
CURLcode res = curl_easy_perform(curl);
// we have to call twice, first call authenticates, second call does the work
res = curl_easy_perform(curl);
if (res != CURLE_OK)
{
strErrorDescription = "Curl call to server failed";
return false;
}
if (!DoSomethingWithServerResponse(gs_strLastResponse))
{
strErrorDescription = "Curl call to server returned an unexpected response";
return false;
}
// extract some transfer info
double speed_upload, total_time;
curl_easy_getinfo(curl, CURLINFO_SPEED_UPLOAD, &speed_upload);
curl_easy_getinfo(curl, CURLINFO_TOTAL_TIME, &total_time);
fprintf(stderr, "Speed: %.3f bytes/sec during %.3f seconds\n", speed_upload, total_time);
curl_easy_cleanup(curl);
return true;
}

C++ CURL post file and then get response error

I'm trying to POST a file to an API and then check the response for various things. But whenever I try to POST a file that is larger than 0 bytes i get an error:
First-chance exception at 0x77AADF63 (ntdll.dll) in Test.exe:
0xC0000005: Access violation writing location 0x00000014.
The relevant code is:
std::string Network::Post(std::string url, std::map<std::string, std::string> requestBody, std::string file, int port) {
std::ostringstream response;
std::stringstream bodyStream;
struct stat file_info;
for (std::map<std::string, std::string>::iterator iterator = requestBody.begin(); iterator != requestBody.end(); iterator++) {
bodyStream << iterator->first << "=" << iterator->second << "&";
}
//Get file
FILE* file_handle = fopen(file.c_str(), "rb");
int t = fstat(_fileno(file_handle), &file_info);
CURL *ch;
curl_global_init(CURL_GLOBAL_ALL);
ch = curl_easy_init();
struct curl_slist *headers = NULL;
curl_slist_append(headers, "Content-Type: application/x-www-form-urlencoded");
std::string bodyStreamString = bodyStream.str();
const char* bodyStreamCharPtr = bodyStreamString.c_str();
curl_easy_setopt(ch, CURLOPT_URL, url.c_str());
curl_easy_setopt(ch, CURLOPT_PORT, port);
curl_easy_setopt(ch, CURLOPT_POSTFIELDS, bodyStreamCharPtr);
curl_easy_setopt(ch, CURLOPT_POSTFIELDSIZE, bodyStream.str().length());
curl_easy_setopt(ch, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(ch, CURLOPT_AUTOREFERER, 1);
curl_easy_setopt(ch, CURLOPT_FOLLOWLOCATION, 1);
curl_easy_setopt(ch, CURLOPT_COOKIEFILE, "cookies.dat");
curl_easy_setopt(ch, CURLOPT_COOKIEJAR, "cookies.dat");
curl_easy_setopt(ch, CURLOPT_WRITEFUNCTION, &ResponseToString);
curl_easy_setopt(ch, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(ch, CURLOPT_UPLOAD, 1L);
curl_easy_setopt(ch, CURLOPT_READDATA, file_handle);
curl_easy_setopt(ch, CURLOPT_INFILESIZE_LARGE, (curl_off_t)file_info.st_size);
curl_easy_setopt(ch, CURLOPT_WRITEDATA, &response);
int res = curl_easy_perform(ch);
curl_easy_cleanup(ch);
return response.str();
}
size_t Network::ResponseToString(char *ptr, size_t size, size_t nmemb, void *stream) {
std::ostringstream *s = (std::ostringstream*)stream;
size_t count = size * nmemb;
s->write(ptr, count);
return count;
}
Any idea what's happenning? Been stuck on it for a day or so, sigh :(
More details:
It just breaks at " int res = curl_easy_perform(ch);" , can't step into it.
If I remove the WRITEFUNCTION and WRITEDATA options , it works, but then I can't get the response.
The problem doesn't seem to be in the ResponseToString method either.
The problem is your WRITEFUNC.
I'm quite sure Network::ResponseToString is not static, and this cause the problem, since a non static function will pass "this" as first argument and then screw your function parameters, I suggest to avoid using a class member at all and use this (to be placed in the same file as Network::Post and before Post method definition):
static size_t ResponseToString(char *ptr, size_t size, size_t nmemb, std::ostringstream *stream) {
size_t count = size * nmemb;
stream->write(ptr, count);
return count;
}
Remember also to change:
curl_easy_setopt(ch, CURLOPT_WRITEFUNCTION, &ResponseToString);
in:
curl_easy_setopt(ch, CURLOPT_WRITEFUNCTION, ResponseToString);
Also, a lot of the headers you specify in curl opt declaration are not needed if your scope is a simple "post":
curl_easy_setopt(ch, CURLOPT_POSTFIELDSIZE, bodyStream.str().length());
curl_easy_setopt(ch, CURLOPT_UPLOAD, 1L);
curl_easy_setopt(ch, CURLOPT_READDATA, file_handle);
curl_easy_setopt(ch, CURLOPT_INFILESIZE_LARGE, (curl_off_t)file_info.st_size);
Decided to pipe all the requests from my app through Fiddler and I discovered that nothing was being sent and back came an error (sent 0 bytes out of n). So i poked around a bit and discovered the issue.
Aaand, the problem was my lack of a READFUNCTION. Solved it like this:
curl_easy_setopt(ch, CURLOPT_READDATA, file_handle);
curl_easy_setopt(ch, CURLOPT_READFUNCTION, &RequestToFile);
And this is the read function. Apparently it's required on windows.
size_t EasyCloud::Network::RequestToFile(void *ptr, size_t size, size_t nmemb, FILE *stream) {
curl_off_t nread;
size_t retcode = fread(ptr, size, nmemb, stream);
nread = (curl_off_t)retcode;
return retcode;
}

uploading file with libcurl

Take a look at the following code
static size_t reader(void *ptr, size_t size, size_t nmemb, FILE *stream) {
size_t retcode = fread(ptr, size, nmemb, stream);
cout << "*** We read " << retcode << " bytes from file" << endl;
return retcode;
}
void upload() { //upload() is called from ouside
FILE *pFile;
pFile = fopen("map.txt" , "r");
struct stat file_info;
stat("map.txt", &file_info);
size_t size = (size_t)file_info.st_size;
uploadFile(pFile, size);
}
bool uploadFile(void* data, size_t datasize) {
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if (curl) {
char *post_params = ...;
curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, post_params);
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, (long) strlen(post_params));
curl_easy_setopt(curl, CURLOPT_READFUNCTION, reader);
curl_easy_setopt(curl, CURLOPT_READDATA, data);
curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, (curl_off_t) datasize);
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
return true;
}
When the code is executed, the following is outputed
*** We read 490 bytes from file
*** We read 0 bytes from file
after that the app does nothing (even not exiting).
Can someone point out at what's wrong here?
Will be grateful for any help!!!
There's some serious confusions shown in this code. Let me try to explain:
CURLOPT_UPLOAD - this will ask libcurl to PUT the file when the protocol of choice is HTTP
CURLOPT_POSTFIELDS - tells libcurl to POST the data that is provided in the additional argument (which has the size set with CURLOPT_POSTFIELDSIZE)
CURLOPT_READFUNCTION - provides libcurl an alternative way to get data than CURLOPT_POSTFIELDS to allow a POST that reads the data from a file. When using CURLOPT_UPLOAD this is the only way to provide data.
So in the end the questions left for you are:
Do you want PUT or POST?
Do you want to provide the data as a string or do you want it provided with a callback?