How to get the HTTP response string using Curl in C++ - c++

Question

I'm very new to HTTP commands and the libcurl library. I know how to get the HTTP response code but not the HTTP response string. Following is the code snippet that I wrote to get the response code. Any help on how to get the response string will be highly appreciated!!!
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
CURLcode ret = curl_easy_perform(curl);
if (ret != CURLE_OK) {
LOG(INFO) << "Failed to perform the request. "
<< "Return code: " << ret;
return false;
}
std::unique_ptr<int64_t> httpCode(new int64_t);
// Get the last response code.
ret = curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, httpCode.get());
if (ret != CURLE_OK) {
LOG(INFO) << "curl_easy_getinfo failed to retrieve http code. "
<< "Return code: " << ret;
return false;
}
I tried doing this as well to get the HTTP response string in readBuffer.
static size_t WriteCallback(char *contents, size_t size, size_t nmemb, void *userp)
{
((std::string*)userp)->append((char*)contents, size * nmemb);
return size * nmemb;
}
std::string readBuffer;
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
CURLcode ret = curl_easy_perform(curl);
cout << readBuffer << "\n";
But the readBuffer is empty. I don't understand where I am going wrong. Any pointers on how to solve this will be really nice!

Answer 1

There doesn't look to be much wrong with your code. I ran the following code, based on yours, to read the front page of the BBC news website:
#include <iostream>
#include <string>
#include <curl/curl.h>
size_t WriteCallback(char *contents, size_t size, size_t nmemb, void *userp)
{
((std::string*)userp)->append((char*)contents, size * nmemb);
return size * nmemb;
}
int main(int argc, char * argv[])
{
curl_global_init(CURL_GLOBAL_ALL);
CURL* easyhandle = curl_easy_init();
std::string readBuffer;
curl_easy_setopt(easyhandle, CURLOPT_URL, "http://www.bbc.co.uk/news");
curl_easy_setopt(easyhandle, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(easyhandle, CURLOPT_PROXY, "http://my.proxy.net"); // replace with your actual proxy
curl_easy_setopt(easyhandle, CURLOPT_PROXYPORT, 8080L);
curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(easyhandle, CURLOPT_WRITEDATA, &readBuffer);
curl_easy_perform(easyhandle);
std::cout << readBuffer << std::endl;
return 0;
}
... and I got the full HTML response. NB I had to use a proxy, and I enabled verbose mode to see what was happening. Also NB; the HTTP server might be fussy about the URL you give it: if I replace http://www.bbc.co.uk/news with http://www.bbc.co.uk/news/ (i.e. with a trailing slash) then I get no data back (a response length of zero), as noted by the verbose curl output:
Host: www.bbc.co.uk
Accept: */*
Proxy-Connection: Keep-Alive
< HTTP/1.1 301 Moved Permanently
< X-Cache-Action: PASS (no-cache-control)
< Vary: Accept-Encoding
< X-Cache-Age: 0
< Content-Type: text/html;charset=utf-8
< Date: Mon, 10 Jul 2017 16:42:20 GMT
< Location: http://www.bbc.co.uk/news
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< Content-Length: 0
< X-Cache: MISS from barracuda1.[my proxy].net
< Connection: keep-alive
<
* Connection #0 to host [my proxy] left intact
Here, Content-Length is 0 and the WriteCallback() function is never called. Hope this helps.

Answer 2

For the numerical response code, getinfo with CURLINFO_RESPONSE_CODE is the way to go:
long response_code;
curl_easy_getinfo(handle, CURLINFO_RESPONSE_CODE,&response_code);
However there is no equivalent getinfo capture for the server's response text. If you need the server's text, inspect the raw HTTP headers. There are two ways to do this:
Enable writing headers to the payload with CURLOPT_HEADER, then extract the headers from the combined payload, splitting the body on \n\n
Set a header callback function with CURLOPT_HEADERFUNCTION and parse directly from that