I have upload a zip file compressed with WinRAR containing a txt file into my github test Repo, how I could download the zip file using curl?
I tried:
static size_t write_data(void *ptr, size_t size, size_t nmemb, void *stream)
{
size_t written = fwrite(ptr, size, nmemb, (FILE *)stream);
return written;
}
void Download(std::string url)
{
CURL *curl_handle;
static const char *pagefilename = "data.zip";
FILE *pagefile;
curl_global_init(CURL_GLOBAL_ALL);
/* init the curl session */
curl_handle = curl_easy_init();
/* set URL to get here */
curl_easy_setopt(curl_handle, CURLOPT_URL, url.c_str());
/* Switch on full protocol/debug output while testing */
curl_easy_setopt(curl_handle, CURLOPT_VERBOSE, 1L);
/* disable progress meter, set to 0L to enable and disable debug output */
curl_easy_setopt(curl_handle, CURLOPT_NOPROGRESS, 1L);
/* send all data to this function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data);
/* open the file */
pagefile = fopen(pagefilename, "wb");
if(pagefile) {
/* write the page body to this file handle */
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, pagefile);
/* get it! */
CURLCode res;
res = curl_easy_perform(curl_handle);
/* close the header file */
fclose(pagefile);
}
/* cleanup curl stuff */
curl_easy_cleanup(curl_handle);
curl_global_cleanup();
return;
}
Download("https://github.com/R3uan3/test/files/9544940/test.zip");
CURLCode res return (CURLE_OK) but it create a file data.zip of 0 bytes, what's wrong?
The issue
I checked with
curl -I https://github.com/R3uan3/test/files/9544940/test.zip
It gets
HTTP/2 302
...
location: https://objects.githubusercontent.com/github-production-rep...
In other words: this is a redirect and you did not ask libcurl to follow redirects - so you only get that first response stored (which has a zero byte body).
The fix
You can make libcurl follow the redirect by adding this line to your code:
curl_easy_setopt(curl_handle, CURLOPT_FOLLOWLOCATION, 1L);
Related
I want to download a file from a dropbox shared link using curl in a c++ program
I found a dropbox api pdf that showed me how to do it
#include <stdio.h>
#include <curl/curl.h>
int main (int argc, char *argv[])
{
CURL *curl;
CURLcode res;
/* In windows, this will init the winsock stuff */
curl_global_init(CURL_GLOBAL_ALL);
/* get a curl handle */
curl = curl_easy_init();
if(curl) {
printf ("Running curl test.\n");
struct curl_slist *headers=NULL; /* init to NULL is important */
headers = curl_slist_append(headers, "Authorization: Bearer
<ACCESS_TOKEN>");
headers = curl_slist_append(headers, "Content-Type:");
headers = curl_slist_append(headers, "Dropbox-API-Arg:
{\"path\":\"/test.txt\"}");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_URL,
"https://content.dropboxapi.com/2/files/download");
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "");
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if(res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
printf ("\nFinished curl test.\n");
}
curl_global_cleanup();
printf ("Done!\n");
return 0;
}
However, the comments supplied don't offer much explanation for me, and I can't get it to work.
I don't understand these three lines of code:
headers = curl_slist_append(headers, "Authorization: Bearer <ACCESS_TOKEN>");
headers = curl_slist_append(headers, "Content-Type:");
headers = curl_slist_append(headers, "Dropbox-API-Arg:{\"path\":\"/test.txt\"}");
I think I have to replace some stuff but I don't know what
"I think I have to replace some stuff but I don't know what" : Replace <ACCESS_TOKEN> with your actual access token.
You should also set the "Content-Type:" header to an appropriate value for the data you are fetching.
You must also change the value of the "Dropbox-API-Arg" header to match the file you are trying to get.
I finally found the solution to my problem.
Turns out I didn't have to use the Dropbox API
Here is the code
#include <iostream>
#include <curl/curl.h>
using namespace std;
size_t write_data(void *ptr, size_t size, size_t nmemb, FILE *stream)
{
size_t written;
written = fwrite(ptr, size, nmemb, stream);
return written;
}
int main(int argc, char** argv) {
CURL *curl;
FILE *fp;
const char* destination = "D:\\Desktop\\test.exe";
fp = fopen(destination, "wb");
curl = curl_easy_init();
/* A long parameter set to 1 tells the library to follow any Location: header
* that the server sends as part of an HTTP header in a 3xx response. The
*Location: header can specify a relative or an absolute URL to follow.
*/
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_URL, "https://www.dropbox.com/s/09nd26tdyto23yz/BankAccount.exe?dl=1"); // "dl=0"changed to "dl=1" to force download
// disabe the SSL peer certificate verification allowing the program to download the file from dropbox shared link
// in case it is not used it displays an error message stating "SSL peer certificate or SSH remote key was not OK"
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
CURLcode res;
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
fclose(fp);
if (res ==CURLE_OK)
cout << "OK";
else
cout << curl_easy_strerror(res);
return 0;
}
Thanks you guys for trying to help me. I appreciate
I was asked for my library management c++ project to get access to library file only by its file:// included URL...
I used Libcurl because of same tag words of it when I was searching.
is downloading of a file through Libcurl (as it said it's compatible) ,a right way to do it?
command line tool and library
for transferring data with URLs
Supports...
DICT, FILE, ...
the libcurl code was
#include <curl/curl.h>
int downloadTO(const void* downloadURL, const void* OUTPUT_FILE_NAME)
{
CURL *curl_handle;
static const char *pagefilename = (char*)OUTPUT_FILE_NAME;
FILE *pagefile;
curl_global_init(CURL_GLOBAL_ALL);
/* init the curl session */
curl_handle = curl_easy_init();
/* set URL to get here */
curl_easy_setopt(curl_handle, CURLOPT_URL, (char*)downloadURL);
/* Switch on full protocol/debug output while testing */
curl_easy_setopt(curl_handle, CURLOPT_VERBOSE, 1L);
/* disable progress meter, set to 0L to enable and disable debug output */
curl_easy_setopt(curl_handle, CURLOPT_NOPROGRESS, 1L);
/* send all data to this function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data);
/* open the file */
pagefile = fopen(pagefilename, "wb");
if(pagefile)
{
/* write the page body to this file handle */
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, pagefile);
/* get it! */
curl_easy_perform(curl_handle);
/* close the header file */
fclose(pagefile);
}
/* cleanup curl stuff */
curl_easy_cleanup(curl_handle);
return 0;
}
can I also get access to the file with its file:// prepended URL with the sockets?
How can you prevent / remove / stop the print/output to the command line in libcurl? For some reason, the following code works for URLs, but when I am hitting a page with an image, like http://192.168.123.123/banana.gif , then it crashes:
CURL *session;
session = curl_easy_init();
curl_easy_setopt(session, CURLOPT_URL, "http://192.168.123.123/banana.gif");
CURLcode curl_code = curl_easy_perform (session);
long http_code = 0;
curl_easy_getinfo(session, CURLINFO_RESPONSE_CODE, &http_code);
This is because you did not set the CURLOPT_WRITEDATA option:
The internal CURLOPT_WRITEFUNCTION will write the data to the FILE * given with this option, or to stdout if this option hasn't been set.
What you can do if you decide to completely ignore the response data is write it to /dev/null:
/* ... */
FILE *devnull = fopen("/dev/null", "w+");
curl_easy_setopt(session, CURLOPT_WRITEDATA, devnull);
CURLcode curl_code = curl_easy_perform(session);
fclose(devnull);
/* ... */
Another alternative is to use a NOOP write function:
curl_easy_setopt(session, CURLOPT_WRITEFUNCTION, noop_cb);
Where the write function simply returns the number of received bytes:
size_t noop_cb(void *ptr, size_t size, size_t nmemb, void *data) {
return size * nmemb;
}
Take a look at the following code
static size_t reader(void *ptr, size_t size, size_t nmemb, FILE *stream) {
size_t retcode = fread(ptr, size, nmemb, stream);
cout << "*** We read " << retcode << " bytes from file" << endl;
return retcode;
}
void upload() { //upload() is called from ouside
FILE *pFile;
pFile = fopen("map.txt" , "r");
struct stat file_info;
stat("map.txt", &file_info);
size_t size = (size_t)file_info.st_size;
uploadFile(pFile, size);
}
bool uploadFile(void* data, size_t datasize) {
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if (curl) {
char *post_params = ...;
curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, post_params);
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, (long) strlen(post_params));
curl_easy_setopt(curl, CURLOPT_READFUNCTION, reader);
curl_easy_setopt(curl, CURLOPT_READDATA, data);
curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, (curl_off_t) datasize);
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
return true;
}
When the code is executed, the following is outputed
*** We read 490 bytes from file
*** We read 0 bytes from file
after that the app does nothing (even not exiting).
Can someone point out at what's wrong here?
Will be grateful for any help!!!
There's some serious confusions shown in this code. Let me try to explain:
CURLOPT_UPLOAD - this will ask libcurl to PUT the file when the protocol of choice is HTTP
CURLOPT_POSTFIELDS - tells libcurl to POST the data that is provided in the additional argument (which has the size set with CURLOPT_POSTFIELDSIZE)
CURLOPT_READFUNCTION - provides libcurl an alternative way to get data than CURLOPT_POSTFIELDS to allow a POST that reads the data from a file. When using CURLOPT_UPLOAD this is the only way to provide data.
So in the end the questions left for you are:
Do you want PUT or POST?
Do you want to provide the data as a string or do you want it provided with a callback?
I am new to the libcurl and found a way to download a single file from the ftp server. Now my requirement is to download all files in a directory and i guess it was not supported by libcurl. Kindly suggest on libcurl how to download all files in directory or is there any other library similar to libcurl?
Thanks in advance.
Here is a sample piece of code.
static size_t GetFilesList_response(void *ptr, size_t size, size_t nmemb, void *data)
{
FILE *writehere = (FILE *)data;
return fwrite(ptr, size, nmemb, writehere);
}
bool FTPWithcURL::GetFilesList(char* tempFile)
{
CURL *curl;
CURLcode res;
FILE *ftpfile;
/* local file name to store the file as */
ftpfile = fopen(tempFile, "wb"); /* b is binary, needed on win32 */
curl = curl_easy_init();
if(curl)
{
curl_easy_setopt(curl, CURLOPT_URL, "ftp://ftp.example.com");
curl_easy_setopt(curl, CURLOPT_USERPWD, "username:password");
curl_easy_setopt(curl, CURLOPT_WRITEDATA, ftpfile);
// added to #Tombart suggestion
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, GetFilesList_response);
curl_easy_setopt(curl, CURLOPT_DIRLISTONLY, 1);
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
fclose(ftpfile); //
if(CURLE_OK != res)
return false;
return true;
}
You need the list of files on the FTP server. Which isn't straightforward as each FTP server might return a different format of file listing...
Anyway, the ftpgetresp.c example shows a way to do it, I think. FTP Custom CUSTOMREQUEST suggests another way.
Just use CURLOPT_WILDCARDMATCH feature.
Sample code:
https://curl.haxx.se/libcurl/c/ftp-wildcard.html