download file from file:// prepended with libcurl in c++ - c++

I was asked for my library management c++ project to get access to library file only by its file:// included URL...
I used Libcurl because of same tag words of it when I was searching.
is downloading of a file through Libcurl (as it said it's compatible) ,a right way to do it?
command line tool and library
for transferring data with URLs
Supports...
DICT, FILE, ...
the libcurl code was
#include <curl/curl.h>
int downloadTO(const void* downloadURL, const void* OUTPUT_FILE_NAME)
{
CURL *curl_handle;
static const char *pagefilename = (char*)OUTPUT_FILE_NAME;
FILE *pagefile;
curl_global_init(CURL_GLOBAL_ALL);
/* init the curl session */
curl_handle = curl_easy_init();
/* set URL to get here */
curl_easy_setopt(curl_handle, CURLOPT_URL, (char*)downloadURL);
/* Switch on full protocol/debug output while testing */
curl_easy_setopt(curl_handle, CURLOPT_VERBOSE, 1L);
/* disable progress meter, set to 0L to enable and disable debug output */
curl_easy_setopt(curl_handle, CURLOPT_NOPROGRESS, 1L);
/* send all data to this function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data);
/* open the file */
pagefile = fopen(pagefilename, "wb");
if(pagefile)
{
/* write the page body to this file handle */
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, pagefile);
/* get it! */
curl_easy_perform(curl_handle);
/* close the header file */
fclose(pagefile);
}
/* cleanup curl stuff */
curl_easy_cleanup(curl_handle);
return 0;
}
can I also get access to the file with its file:// prepended URL with the sockets?

Related

How to download a zip file from a github repo using libcurl?

I have upload a zip file compressed with WinRAR containing a txt file into my github test Repo, how I could download the zip file using curl?
I tried:
static size_t write_data(void *ptr, size_t size, size_t nmemb, void *stream)
{
size_t written = fwrite(ptr, size, nmemb, (FILE *)stream);
return written;
}
void Download(std::string url)
{
CURL *curl_handle;
static const char *pagefilename = "data.zip";
FILE *pagefile;
curl_global_init(CURL_GLOBAL_ALL);
/* init the curl session */
curl_handle = curl_easy_init();
/* set URL to get here */
curl_easy_setopt(curl_handle, CURLOPT_URL, url.c_str());
/* Switch on full protocol/debug output while testing */
curl_easy_setopt(curl_handle, CURLOPT_VERBOSE, 1L);
/* disable progress meter, set to 0L to enable and disable debug output */
curl_easy_setopt(curl_handle, CURLOPT_NOPROGRESS, 1L);
/* send all data to this function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data);
/* open the file */
pagefile = fopen(pagefilename, "wb");
if(pagefile) {
/* write the page body to this file handle */
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, pagefile);
/* get it! */
CURLCode res;
res = curl_easy_perform(curl_handle);
/* close the header file */
fclose(pagefile);
}
/* cleanup curl stuff */
curl_easy_cleanup(curl_handle);
curl_global_cleanup();
return;
}
Download("https://github.com/R3uan3/test/files/9544940/test.zip");
CURLCode res return (CURLE_OK) but it create a file data.zip of 0 bytes, what's wrong?
The issue
I checked with
curl -I https://github.com/R3uan3/test/files/9544940/test.zip
It gets
HTTP/2 302
...
location: https://objects.githubusercontent.com/github-production-rep...
In other words: this is a redirect and you did not ask libcurl to follow redirects - so you only get that first response stored (which has a zero byte body).
The fix
You can make libcurl follow the redirect by adding this line to your code:
curl_easy_setopt(curl_handle, CURLOPT_FOLLOWLOCATION, 1L);

Using libcurl from C++/Windows to post to an API that requires a client certificate for authentication? Getting CURLE_SSL_CERTPROBLEM

I have searched here, but did not find anything that matched. What I have tested with curl.exe and that works fine. I then tried a small test program. Created a commandline Win32 project in VS2017 and copied in the code from simplessl.c. Problem is that no matter what certificates I try I always get CURLE_SSL_CERTPROBLEM. This points to the client certificate being wrong. I even tried with the 2048 bit certificate and key from here: https://fm4dd.com/openssl/certexamples.shtm Not sure how to troubleshoot.
The test program I used:
#include "pch.h"
#include <iostream>
#include <curl/curl.h>
int main()
{
std::cout << "Hello World!\n";
CURL *curl;
CURLcode res;
FILE *headerfile;
const char *pPassphrase = NULL;
static const char *pCertFile = "C:\\Prog\\TestCurl\\2048b-rsa-example-cert.pem";
static const char *pCACertFile = "cacert.pem";
static const char *pHeaderFile = "C:\\Prog\\TestCurl\\requestWith.txt";
const char *pKeyName;
const char *pKeyType;
const char *pEngine;
#ifdef USE_ENGINE
pKeyName = "rsa_test";
pKeyType = "ENG";
pEngine = "chil"; /* for nChiper HSM... */
#else
pKeyName = "C:\\Prog\\TestCurl\\2048b-rsa-example-keypair.pem";
pKeyType = "PEM";
pEngine = NULL;
#endif
headerfile = fopen(pHeaderFile, "wb");
curl_global_init(CURL_GLOBAL_DEFAULT);
curl = curl_easy_init();
if (curl) {
/* what call to write: */
//curl_easy_setopt(curl, CURLOPT_URL, "https://pintatesti.vero.fi/FIS/Return/IIT/Test/GetWithholdingPercentage/v1");
curl_easy_setopt(curl, CURLOPT_URL, "https://www.pedago.fi");
curl_easy_setopt(curl, CURLOPT_HEADERDATA, headerfile);
do { /* dummy loop, just to break out from */
if (pEngine) {
/* use crypto engine */
if (curl_easy_setopt(curl, CURLOPT_SSLENGINE, pEngine) != CURLE_OK) {
/* load the crypto engine */
fprintf(stderr, "can't set crypto engine\n");
break;
}
if (curl_easy_setopt(curl, CURLOPT_SSLENGINE_DEFAULT, 1L) != CURLE_OK) {
/* set the crypto engine as default */
/* only needed for the first time you load
a engine in a curl object... */
fprintf(stderr, "can't set crypto engine as default\n");
break;
}
}
/* cert is stored PEM coded in file... */
/* since PEM is default, we needn't set it for PEM */
curl_easy_setopt(curl, CURLOPT_SSLCERTTYPE, "PEM");
/* set the cert for client authentication */
curl_easy_setopt(curl, CURLOPT_SSLCERT, pCertFile);
/* sorry, for engine we must set the passphrase
(if the key has one...) */
if (pPassphrase)
curl_easy_setopt(curl, CURLOPT_KEYPASSWD, pPassphrase);
/* if we use a key stored in a crypto engine,
we must set the key type to "ENG" */
curl_easy_setopt(curl, CURLOPT_SSLKEYTYPE, pKeyType);
/* set the private key (file or ID in engine) */
curl_easy_setopt(curl, CURLOPT_SSLKEY, pKeyName);
/* set the file with the certs vaildating the server */
//curl_easy_setopt(curl, CURLOPT_CAINFO, pCACertFile);
/* disconnect if we can't validate server's cert */
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if (res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* we are done... */
} while (0);
/* always cleanup */
curl_easy_cleanup(curl);
}
curl_global_cleanup();
return 0;
}
Edit: Underlying error I get is " schannel: certificate format compatibility error" and only occurrence I can find is in schannel.c where it compares to a P12 type, but my certs are PEM?
Edit2: So it does work if I import the certificate into the local store, something I would definitely want to avoid! Also curl_easy_setopt(curl, CURLOPT_VERBOSE, 1); appears to be avery good idea! Now getting further.
Edit3: Got it working. Possibly related to: Having trouble sending client certificate in libcurl ssl request, what am I missing? which is from 2019. I would very much prefer to have the certificates in a file though.
Edit4: "libcurl/7.79.1 Schannel WinIDN"
I solved this with a little help from my friends. Libcurl when compiled as default on Windows will default to schannel. To override that you need to build for OpenSSL, thusly: nmake /f Makefile.vc mode=static DEBUG=yes WITH_SSL=static WITH_DEVEL=C:\Prog\OpenSSL-Win32_111L\. The important keywords are WITH_SSL=static or dynamic and WITH_DEVEL= path to where your OpenSSL files live. When built this way you can give file paths for certificate and key, ie: curl_easy_setopt(curl, CURLOPT_SSLCERT, m_Certificate.GetString());
This frees you of the messy certificate handling in Windows and lets you keep the certificates with the app itself.

Downloading a file via curl in c++ from dropbox

I want to download a file from a dropbox shared link using curl in a c++ program
I found a dropbox api pdf that showed me how to do it
#include <stdio.h>
#include <curl/curl.h>
int main (int argc, char *argv[])
{
CURL *curl;
CURLcode res;
/* In windows, this will init the winsock stuff */
curl_global_init(CURL_GLOBAL_ALL);
/* get a curl handle */
curl = curl_easy_init();
if(curl) {
printf ("Running curl test.\n");
struct curl_slist *headers=NULL; /* init to NULL is important */
headers = curl_slist_append(headers, "Authorization: Bearer
<ACCESS_TOKEN>");
headers = curl_slist_append(headers, "Content-Type:");
headers = curl_slist_append(headers, "Dropbox-API-Arg:
{\"path\":\"/test.txt\"}");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_URL,
"https://content.dropboxapi.com/2/files/download");
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "");
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if(res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
printf ("\nFinished curl test.\n");
}
curl_global_cleanup();
printf ("Done!\n");
return 0;
}
However, the comments supplied don't offer much explanation for me, and I can't get it to work.
I don't understand these three lines of code:
headers = curl_slist_append(headers, "Authorization: Bearer <ACCESS_TOKEN>");
headers = curl_slist_append(headers, "Content-Type:");
headers = curl_slist_append(headers, "Dropbox-API-Arg:{\"path\":\"/test.txt\"}");
I think I have to replace some stuff but I don't know what
"I think I have to replace some stuff but I don't know what" : Replace <ACCESS_TOKEN> with your actual access token.
You should also set the "Content-Type:" header to an appropriate value for the data you are fetching.
You must also change the value of the "Dropbox-API-Arg" header to match the file you are trying to get.
I finally found the solution to my problem.
Turns out I didn't have to use the Dropbox API
Here is the code
#include <iostream>
#include <curl/curl.h>
using namespace std;
size_t write_data(void *ptr, size_t size, size_t nmemb, FILE *stream)
{
size_t written;
written = fwrite(ptr, size, nmemb, stream);
return written;
}
int main(int argc, char** argv) {
CURL *curl;
FILE *fp;
const char* destination = "D:\\Desktop\\test.exe";
fp = fopen(destination, "wb");
curl = curl_easy_init();
/* A long parameter set to 1 tells the library to follow any Location: header
* that the server sends as part of an HTTP header in a 3xx response. The
*Location: header can specify a relative or an absolute URL to follow.
*/
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_URL, "https://www.dropbox.com/s/09nd26tdyto23yz/BankAccount.exe?dl=1"); // "dl=0"changed to "dl=1" to force download
// disabe the SSL peer certificate verification allowing the program to download the file from dropbox shared link
// in case it is not used it displays an error message stating "SSL peer certificate or SSH remote key was not OK"
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
CURLcode res;
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
fclose(fp);
if (res ==CURLE_OK)
cout << "OK";
else
cout << curl_easy_strerror(res);
return 0;
}
Thanks you guys for trying to help me. I appreciate

Getting cURL certificates to work with C++ and vcpkg install

so I have a class that makes an http call using curl in visual studios 2017 that was installed via vcpkg discussed: here, using curl_easy function calls:
string returnResponseAsString(string requestURL) {
CURL *curl_handle;
CURLcode res;
struct MemoryStruct chunk;
chunk.memory = (char *)malloc(1);
chunk.size = 0;
curl_global_init(CURL_GLOBAL_ALL);
/* init the curl session */
curl_handle = curl_easy_init();
/*Turn off SSL Verifcation*/
curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYPEER, FALSE);
/* specify URL to get */
curl_easy_setopt(curl_handle, CURLOPT_URL, requestURL.c_str());
/* send all data to this function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
/* we pass our 'chunk' struct to the callback function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void *)&chunk);
/* some servers don't like requests that are made without a user-agent*/
curl_easy_setopt(curl_handle, CURLOPT_USERAGENT, "libcurl-agent/1.0");
/* get it! */
res = curl_easy_perform(curl_handle);
/* check for errors */
if (res != CURLE_OK) {
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
}
else {
printf("%lu bytes retrieved\n", (long)chunk.size);
}
string response = chunk.memory;
/* cleanup curl stuff */
curl_easy_cleanup(curl_handle);
free(chunk.memory);
/* we're done with libcurl, so clean it up */
curl_global_cleanup();
return response;
}
If I dont include this line curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYPEER, FALSE); that turns off SSL verification, I get the error: curl_easy_perform() failed: Peer certificate cannot be authenticated with given CA certificates. I tried to install the certificates as per these instructions: link, but is says to place the downloaded certs in the same folder as the curl.exe. As far as I can tell there is no curl.exe installed by the vcpkg. I looked for .crt and I have an image of the found certs under vcpkg. Where Should I place the .crt file for authentication to work with visual studios?

Crash on curl_easy_perform() when uploading a file on CURL in C++

I'm having an issue with a crash when uploading a file via the curl library in C++. I'm using the exact demo code from this location: https://curl.haxx.se/libcurl/c/fileupload.html
The only thing I change in the code is the upload location, to upload to a local wamp server on windows, and the file to upload, which I've verified that its opening ok.
I'm running through visual studio 2014, and building CURL through DLL
The output from the program is:
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> PUT /replayupload.php HTTP/1.1
Host: 127.0.0.1
Accept: */*
Content-Length: 43
Expect: 100-continue
< HTTP/1.1 100 Continue
*then I get a crash at line 66 in the program. It seems the line:
res = curl_easy_perform(curl);
is causing the problem with an invalid parameter. I have verified that the curl variable is not null, but I'm finding it very difficult to get any more debug info than that, the call stack just references a memory address within the DLL.
I'm able to run the demo to just upload post variables and get a page, this runs fine without a crash. The crash only occurs when uploading a file.
my exact code is:
int main(void)
{
CURL *curl;
CURLcode res;
struct stat file_info;
double speed_upload, total_time;
FILE *fd;
fd = fopen("E:\\testfiles\\test.txt", "rb"); /* open file to upload */
if (!fd)
return 1; /* can't continue */
/* to get the file size */
if (fstat(_fileno(fd), &file_info) != 0)
return 1; /* can't continue */
curl = curl_easy_init();
if (curl) {
/* upload to this place */
curl_easy_setopt(curl, CURLOPT_URL,
"http://127.0.0.1/testupload.php");
/* tell it to "upload" to the URL */
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
/* set where to read from (on Windows you need to use READFUNCTION too) */
curl_easy_setopt(curl, CURLOPT_READDATA, fd);
/* and give the size of the upload (optional) */
curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE,
(curl_off_t)file_info.st_size);
/* enable verbose for easier tracing */
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
res = curl_easy_perform(curl);
/* Check for errors */
if (res != CURLE_OK) {
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
}
else {
/* now extract transfer info */
curl_easy_getinfo(curl, CURLINFO_SPEED_UPLOAD, &speed_upload);
curl_easy_getinfo(curl, CURLINFO_TOTAL_TIME, &total_time);
fprintf(stderr, "Speed: %.3f bytes/sec during %.3f seconds\n",
speed_upload, total_time);
}
/* always cleanup */
curl_easy_cleanup(curl);
}
fclose(fd);
return 0;
}
Thanks to Tkausl for spotting the line
/* set where to read from (on Windows you need to use READFUNCTION too) */
I added this line to my code
curl_easy_setopt(curl, CURLOPT_READFUNCTION, &fread);
And now everything seems to work.