Can one read a remote file as an istream with libcurl? - c++

Question

I'd like to use the libcurl library to open a remote date file and iterate through it with an istream. I've looked through the nice example in this thread but it writes the remote file to a local file. Instead I'd like to have the remote reads be pushed to an istream for subsequent programmatic manipulation. Is this possible? I would greatly appreciate help.
Best,
Aaron

Answer 1

Boost's IO Stream might be a better solution than STL's own stream. At least it is much simpler to create a boost stream. From boost's own docs:
#include <curl/curl.h>
#include <boost/iostreams/stream.hpp>
class CURLDevice
{
private:
CURL* handle;
public:
typedef char char_type;
typedef boost::iostreams::source_tag category;
CURLDevice()
{
handle = curl_easy_init();
}
CURLDevice(const std::string &url)
{
handle = curl_easy_init();
open( url );
}
~CURLDevice()
{
curl_easy_cleanup(handle);
}
void open(const std::string &url)
{
curl_easy_setopt(handle, CURLOPT_URL, url.c_str());
curl_easy_setopt(handle, CURLOPT_CONNECT_ONLY, 1);
curl_easy_perform(handle);
}
std::streamsize read(char* s, std::streamsize n)
{
size_t read;
CURLcode ret = curl_easy_recv(handle, s, n, &read);
if ( ret == CURLE_OK || ret == CURLE_AGAIN )
return read;
else
return -1;
}
};
typedef boost::iostreams::stream<CURLDevice> CURLStream;
int main(int argc, char **argv)
{
curl_global_init(CURL_GLOBAL_ALL);
{
CURLStream stream("http://google.com");
char buffer[256];
int sz;
do
{
sz = 256;
stream.read( buffer, sz );
sz = stream.gcount();
std::cout.write( buffer, sz );
}
while( sz > 0 );
}
curl_global_cleanup();
return 0;
}
Note: when I run the code above I get a segfault in CURL, this appears to be because I don't know exactly how to use curl itself.