I wanna download *.txt files from server using curl, but unfortunately I can't understand how to do it, because I'm beginner in curl, I though to use recursive iterator from boost::filesystem, maybe you have any different ways to solve my problem? thank you)
boost::filesystem only works with file paths on the local machine, and UNC paths on a local network. You cannot use it to iterate remote files over the Internet.
If the files are on an HTTP server, libCURL cannot iterate the files directly, as that capability is not part of the HTTP protocol. Your only hope is if the HTTP server provides an HTML file (or other format) containing the file listing when you request a URL for the actual directory that the files reside in. However, for obvious security reasons, many HTTP servers disable this feature! Most servers will instead redirect to a specific file, like index.html, index.php, default.aspx, etc. But, if the HTTP server does allow to retreive a file listing, you would have to retrieve and parse that listing data manually in order to determine the URLs of the individual files, and then you can download them as needed.
If the files are on an FTP server, then that is more desirable, as directory listings are part of the FTP protocol, and libCURL can retrieve an FTP directory listing (make sure the requested URL ends with a backslash so libCURL knows a directory is being requested and not a specific file). However, you are responsible for parsing the listing data, which can be in any format the FTP server decides to send (and there are MANY MANY listing formats used by various online FTP servers!). Modern FTP servers support the FTP MLSD and MLST commands (as opposed to the original LIST command) to facilitate easier parsing of listing data, so you can try instructing libCURL to use those commands (via libCURL's CURLOPT_CUSTOMREQUEST option) if the FTP server supports those commands. Or, if you are interested in only the file names and not the file details (like timestamps, sizes, etc), you can use the FTP NLST command instead (via libCURL's CURLOPT_DIRLISTONLY option).
Related
I am trying to upload a file to the Shared Documents library of my SharePoint website. The files are of type PDF and HTML. I am running a Cold Fusion development environment and using CFHTTP commands to execute HTTP requests. I have been able push a POST command and a PUT command to the proper endpoints listed on this link below:
Link: https://learn.microsoft.com/en-us/graph/api/driveitem-createuploadsession?view=graph-rest-1.0#best-practices
I do not understand why but the first section that mentions the HTTP requests for creating an upload session is different than what was used in the example a little further. For my project, I am using the endpoint:
"/{variables.instance.microsoftGraphAPIURL}/drive/root:/{item-path}:/createUploadSession"
P.S. variables.instance.microsoftGraphAPIURL is a variable to a microsoft graph endpoint to our Sharepoint website
With better luck using PUT commands than POST commands for creating an Upload Session. I am able to receive an uploadURL, but the issue comes with trying to upload the file. For the file upload, I am trying to upload a file in the same directory with a file size of 114992 bytes. I keep getting "The Content-Range header length does not match the provided number of bytes." whenever I run my Put command to upload the file.
Thus, my Content-Range is "bytes 0-114991/114992" and my Content-Length is "114992". For the image below, I replaced the file with a pdf, but the original file was an HTML page at 114992 bytes. I want to use a resumable upload session to have one function for uploading image, HTML, and PDF files.
If anyone could tell me if there is an issue with my content headers or my upload session http request or anything else that is causing my issue, that would be amazing! Thank you.
I'm trying to host a static site through S3 and API Gateway (using S3 directly isn't an option due to security policy). One of the pages runs a client-side script to pull back a set of files from a specific folder on the server. I've set up the hosting following the Amazon tutorial.
For this to run, my script needs to be able to obtain the list of files for a specific folder.
If I was hosting the site on my own server using Apache, I could rely on the directory listing feature, where a GET on a folder with no index.html returns a file list. The tutorial suggests that this should be possible, but I can't seem to get it to work. If I submit a request to a particular {prefix}/{identifier}, I can retrieve the specific file, but sending a request to {prefix}/ returns an error.
Is there a way I can replicate directory listings so my Javascript can pull it down and read it, or is the only solution to write a server-side API in Lambda?
I want to download files that I have the full URL address using my c++ program, which library is the best to do this?
I was wondering for a command like:
system("download [URL] [DESTINATION]");
or
download(URL,DESTINATION);
I am using Windows, sorry that I forgot to mention.
You have not mentioned the operating system, anyways you can do that using system function in c.
#include <stdio.h>
int main()
{
system("wget url");
return 0;
}
Change url to get the file you needed.
have a look at curl
from curl site
curl is a command line tool for transferring data with URL syntax, supporting DICT, FILE, >FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, >SFTP, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a busload >of other useful tricks.
Its the best client side library available
http://curl.haxx.se/
libcurl is one of the widely used multi-protocol network transfer libraries. It can be used for downloading files or pages over HTTP and lot of other protocols.
The command line client curl is built on top of the libcurl library.
for a call to system("command"); you could use some command line tools that can achieve a download. wget and curl are such tools. the later even provides an API for C so you can write you own download(...) function.
I'm making an HTTP server in c++, I notice that the way apache works is if you request a directory without adding a forward slash at the end, firefox still somehow knows that it's a directory you are requesting (which seems impossible for firefox to do, which is why I'm assuming apache is doing a redirect).
Is that assumption right? Does apache check to see that you are requesting a directory and then does an http redirect to a request with the forward slash? If that is how apache works, how do I implement that in c++? Thanks to anyone who replies.
Determine if the resource represents a directory, if so reply with a:
HTTP/1.X 301 Moved Permanently
Location: URI-including-trailing-slash
Using 301 allows user agents to cache the redirect.
If you wanted to do this, you would:
call stat on the pathname
determine that it is a directory
send the necesssary HTTP response for a redirect
I'm not at all sure that you need to do this. Install the Firefox 'web developer' add-on to see exactly what goes back and forth.
Seriously, this should not be a problem. Suggestions for how to proceed:
Get the source code for Apache and look at what it does
Build a debug build of Apache and step through the code in a debugger in such a case; examine which pieces of code get run.
Install Wireshark (network analysis tool), Live HTTP Headers (Firefox extension) etc, and look at what's happening on the network
Read the relevant RFCs for HTTP - which presumably you should be keeping under your pillow anyway if you're writing a server.
Once you've done those things, it should be obvious how to do it. If you can't do those things, you should not be trying to develop a web server in C++.
The assumption is correct and make sure your response includes a Location header to the URL that allows directory listing and a legal 301/302 first line. It is not a C++ question, it is more of a HTTP protocol question, since you are trying to write a HTTP server, as one of the other posts suggests, read the RFC.
You should install Fiddler and observe the HTTP headers sent by other web servers.
Your question is impossible to answer precisely without more details, but you want to send an HTTP 3xx status code with a Location header.
I'm trying to serve static files for download in a django application, I figured that I'd put the static files in /media/files and have Apache set the content-type header to application/octet-stream (the files to download are going to be word files but I'll work out the details later).
To do this I activated mod_headers and then in the apache config did this:
<Location "/media/files">
Header set Content-Type "application/octet-stream"
</Location>
After doing this I restarted apache and tried a sample file but it doesn't work, I still get text/plain in the content type and the browser does not prompt me to download anything.
By the way I know it is recommended to use a different web server for static files but I don't have much control on the server I'm going to deploy, it has to be only Apache with mod_python.
There could be any number of problems (it takes a lot more information than you've provided to trace down some apache config problems) but here are some thoughts:
Are you absolutely certain this snippet is being applied to the right files (e.g., if there are multiple virtual servers, and you stuck this in the wrong one, well..)
Do you have rewriting going on that might prevent this from being seen as a match?
Are you setting the Content-Type header elsewhere?
Do you have content arbitration going on? Depending, that could override anything you do in the headers.
One thing you might try is to add some other header and see if it comes back. Also, try doing the request yourself with telnet or elsewise reducing the number of things between you and the server. Use the log files. They are there to help you. Good luck.