How do you copy a group of files from server machine to local hard disk through a C++ web application in one request? This is kind of downloading bulk files to your local machine from a server. I guess in Java you could do this through ZipInputStream and GZipInputStream.
Is it possible to achieve this entirely through server side code? Or will it require a client running on the local machine to carry out the bulk copying of files?
Say you have a Java servlet / ISAPI extension that accepts requests of the form
http://server:port/fileserver?f=FILE1&f=FILE2&.....&f=FILEN
On receipt of such a request, the server side code can, using zlib, pack all the files into a zip file and send the zip file as the HTTP response setting the Content-Type, Content-Length, Last-Modified,etc.
Further Note: If you are using ISAPI on IIS6 and above, you can also add this content into the IIS's kernel mode response cache.
If the user if aware of this, and the files are not nessesary for website rendering, you can push them into an archive and have a link from your site (or do something like sourceforge, redirect to the archive and the browser simply downloads it. For archiving you can use zlib. Just send Content-type as gzip and push data (here from stdin)
int ret;
/* avoid end-of-line conversions */
SET_BINARY_MODE(stdin);
SET_BINARY_MODE(stdout);
ret = def(stdin, stdout, Z_DEFAULT_COMPRESSION);
if (ret != Z_OK)
zerr(ret);
return ret;
If you are trying to have the whole page (HTML, CSS, JS, IMG) sent as one, all of those files can be inserted into HTML, even images. (see this).
Related
I'm using the following line to download a file, and when I do that, it's not downloading the most recent file.
HRESULT hr = URLDownloadToFile(NULL, _T("http://example.com/users.txt"), _T("users.txt"), 0, NULL);
On the first run, users.txt has 3 names in it, if you were to remove a name, and run it again it still downloads with 3 names.
I'm using remove("users.txt); to remove the file prior to download.
It is probably operating system specific, or at least you need a library for HTTP client side.
You need to read a lot more about the HTTP protocol. The formulation of your question makes me believe you don't understand much about it.
On some OSes (notably Linux and POSIX compliant ones), you can use libcurl (which is a good HTTP client free software library)
URLDownloadToFile seems to be a Windows specific thing. Did you carefully read its documentation? It is returning some error code. Do you handle hr correctly?
You can probably only get what the HTTP protocol (response from web server, for a GET HTTP request) gives you. Mostly, the MIME type of the content of the URL, the content size, and the content bytes (etc... including content encoding etc...). The fact that the content has 3 names is your understanding of it.
Try to read more about the HTTP protocol, and understand what is really going on. Are any cookies or sessions involved? Did you try to use something like telnet to manually make the HTTP exchange? Are you able to show it and understand it? What it the HTTP response code ?
If you have access to the server (e.g. using ssh) and are able to look into the log files, try to understand what exchanges happened and what HTTP status -i.e. error code- was sent back. Perhaps set up some Linux box locally for initial tests. Or setup some HTTP server locally and use http://localhost/ etc...
I wrote a simple HTTP responding C++ program using http://www.fastcgi.com library + nginx web-server. It works almost great. But I noticed that fcgi's function
int FCGX_Accept_r(FCGX_Request *request);
doesnt return before all the request transmitted to the server. The problem: i am unable to analyze parameters from URI before file transmission is finished: i want to analyze "?sid=" argument to reject transmission from invalid user. It would be a waste of bandwidth to allow anyone send 1GB files to a server.
As i know, web-server always send FCGI_BEGIN_REQUEST via FastCGI connection when new request started, then FCGI_PARAMS etc. So it would be good to react to content of some parameters before user transmitted all the (possibly unneeded and too large) data.
Also i will read more nginx documentation about its FastCGI abilities.
update:
Trying to send 2 GB file to nginx via HTML-form + sniffing the exchange between nginx and fastcgi: nginx doesn't even try to send a byte to FastCGI application before all the 2GB has come. So, this behavior is kinda normal. And if you want the functionality i am asking for, it turns out that you should develop your own nginx's module or simple web-server. Hm...
I think you can control by this :
location /xxx/upload {
access_by_lua '
//check cookie or something can recognize the req is valid or not
';
fcgi_pass XXXX;
}
check this link :
http://openresty.org/
Wonder what the community says about the most efficient (in terms of I/O and speed) solution for delivering multiple files back from a single request to a webservice would be. The client is not a web browser.
The options I see so far:
creating a zip archive and streaming it back to the client.
base64 encoding files an returning array of strings that would need to be decoded by the client.
Using Mime multipart/related and sending Mime headers for each file in iteration, also potentially streamed back to the client.
Maybe there are others I haven't considered?
CLARIFICATION:
Let's assume the files may be in the 10s of Megabytes, and that memory is around 4G but there are likely other processes and/or simultaneous requests.
I think you need to consider the bindings (streaming) and transports protocols (SOAP, REST). How large is the average file?
I am using ifstream to open a file and then read from it.
My program works fine when i give location of the local file
on my system.
for eg
/root/Desktop/abc.xxx works fine
But once the location is on the http server the file fails to
open.
for eg
http://192.168.0.10/abc.xxx fails to open.
Is there any alternate for ifstream when using a URL address?
thanks.
There are no utilities in the standard C++ library for accessing data via http protocol.
There are 3rd-party libraries though:
Libwww
libcurl
Another option is to have a virtual filesystem that maps remote http files as local files. This way you don't have to modify your application to access http. Something like http://okmij.org/ftp/HTTP-VFS.html
ifstream will not read files off of an HTTP Server. It will only read local files.
The f in ifstream is for file, not socket.
You need to make an HTTP GET Request and then stream the response, this is a totally different operation.
Consider using boost::asio, or similar. [Examples]
Update
Since the web server is on your local area network (judging by the IP address - not sure why people still insist in using those in these heady days of DNS, but that's by the by), you could probably mount the filesystem containing the desired file on your local machine, using NFS or similar. Then you'd be able to perform file operations, such a reading with ifstream.
ifstream only works with files accessible on the filesystem, not on web servers.
I am writing an application, similar to Seti#Home, that allows users to run processing on their home machine, and then upload the result to the central server.
However, the final result is maybe a 10K binary file. (Processing to achieve this output is several hours.)
What is the simplest reliable automatic method to upload this file to the central server? What do I need to do server-side to prevent blocking? Perhaps having the client send mail is simple and reliable? NB the client program is currently written in Python, if that matters.
Email is not a good solution; you will run into potential ISP blocking and other anti-spam mechanisms.
The easiest way is over HTTP via a simple webservice. Have a listener at your server that accepts the uploaded files as part of a HTTP POST and then dump them wherever they need to be post-processed.