I'd like to write some files directly to zip archive file (rather that creating them first on some folder and copy them to the archive on the second stage).
Therefore, i'm wondering if there's an option to set the ofstream to point directly on the file inside the archive.
for example, say i have archive in /tmp called data.zip, and inside it there's a file data1.log
can i do something like :
std::ofstream ostr("/tmp/data.zip/data1.log", std::ios::binary);
and start pushing data using the '<<' operator ?
thanks,
can i do something like :
std::ofstream ostr("/tmp/data.zip/data1.log", std::ios::binary);
and start pushing data using the '<<' operator ?
No, that's not possible.
Also note the std::ostream& operator<<(std::ostream&,const T&) operator is explicitly reserved for text formatted output, not to write binary data.
To achieve such you would need a std::streambuf implementation, that wraps the incoming character data to a file that is (finally?) compressed and added to the archive.
The C++ standard library has no notion how to magically interact with binary .zip files.
Also what would you mean "start pushing data"? A .zip archive also contains information about particular compressed file names and their relative paths.
How would you interact with the std::ofstream interface to specify which file's data to add actually?
You should research on c++ wrappers for the LZMA/7zip library, that let you control adding files to archives.
Related
(My previous questions was closed as a duplicate of Are there binary memory streams in C++ which is ridiculous, since i can't change the implementation of the library I'm using)
I'm using a library (Poco) to create zip files. It takes ostream as an input and writes the data of the zip file into it. Something like:
std::ofstream ofs("file.zip", std::ios::binary);
Compress compress(ofs);
// add data to compress ...
compress.close();
// now file.zip contains added file
This works. But I want to be able to create a zip in memory without creating a file. I tried using stringstream instead of ofstream, i get additionl newline characters in the data in the zip file is corrupted. Is there any other stream i can use?
(If someone still thinks it's a duplicate, I'm gonna need an explanation, since I don't see how this other question is helpful for me)
Use a std::stringstream -- that will create an in-memory string that you can write to as an ostream, and WILL NOT add extra newlines. If you later copy the string to an fstream that was opened in text mode (such as std::cout), then that process may add extra CR characters that are not in the string (nor in the original output).
If you are seeing extra characters corrupting your stream, they are coming from somewhere else -- something besides you compress call/lib is writing to the stream, or something with how you are looking at your stream is doing something.
If you're on linux, how about creating an anonymous file using memfd_create? You can then open /proc/self/fd/<fd> and do your stuff. Some implementations of std::ofstream may even provide a constructor that takes a FILE*, you can check if that's the case on your system.
I'm trying to merge a file with my executable, and read the merged file. I merge them with the Windows command;
copy /b Game.exe+Image.jpg TheGame.exe
Here's what I've tried:
std::ifstream f("Image.jpg");
if (f.good()) {
std::cout << "Found Image.jpg" << std::endl;
}
Image.jpg is in the same directory as the resulting executable file, and it works. However when I use the command to merge them and then delete the Image.jpg file it is not found (although it is merged with the executable.)
Any suggestions?
ifstream only works with external files. You deleted the file it is trying to open, so of course it will not find the file. What you are attempting cannot (easily) be done using binary merges. If you want to store a file inside of an executable, the correct approach is to store it in a resource instead. Read the following chapter on MSDN for more information.
Introduction to Resources
In particular, the following example shows how go create a new resource in an .exe file and write data into it. The example copies a resource from another .exe file, but you can write whatever you want. In tbis case, replace RT_DIALOG with RT_RCDATA, and write your image data.
Using Resources
I need iostream wrapper for zlib that allow to append data into end of existing stream.
gzopen() function in append mode follow the next rule. "When appending, gzopen does not test whether the file begins with a gzip stream, nor does it look for the end of the gzip streams to begin appending. gzopen will simply append a gzip stream to the existing file." E.g. opening of file with "a" option will create file with multiple compressed streams.
gzofstream wrapper for zlib translate std::ios_base::app in parameter list as "ab" mode of gzopen() function. As result, gzofstream will also create file with multiple streams.
However, in my application I need alternative behavior. Main module should open file in append mode (with std::ios_base::app option), write small data portion in the end of existing stream, and close the file. E.g., file should always contain single compressed stream after several open/close operation.
gzlog example from zlib has functionality close to required. However, it is pure C.
Can you propose some ready solutions for my task?
It seems that ready solution is not exists for this case: iostream wrappers don't support append operation.
I have implemented own code based on gzlog and pure C (not C++).
I am using Visual C++. How to copy the content of this file to another file?
UINT32 writeToLog(wstring log)
{
wfstream file1 (LOG_FILE_NAME, ios_base::out);
file1 << log;
file1.close();
// want to copy file1 to file2
return 0;
}
What exactly do you want to do? If you need a copy of the data, you can read it in and write it back out again. If you really need a copy of the file, you have to use OS specific calls.
In many cases, reading in the file data and then writing it out again to a different file is a close enough approximation to a copy - like this:
ifstream file1(...);
ofstream file2(...);
std::copy(istream_iterator<char>(file1),istream_iterator<char>(),ostream_iterator<char>(file2));
However that really isn't a copy - it's creating a new file with the same contents. It won't correctly handle hard links or symlinks, it won't correctly handle metadata and it will only 'copy' the default file stream.
If you need a file copy on Windows you should call one of CopyFile, CopyFileEx or CopyFileTransacted depending on your exact requirements.
Standard C++ has no file copying facility, other than reading the file into memory and writing it out again to a different file. As you are using Windows, you can use the CopyFile function - other OSs have similar, OS-specific functions.
The above code from Joe Gauterin did not work for me. I was trying to copy a .tga image file, so maybe something about istream_iterator<char> screwed it up. Instead I used:
ifstream file1(...);
ofstream file2(...);
char ch;
while(file1 && file1.get(ch))
{
file2.put(ch);
}
I'm writing a browser plugin, similiar to Flash and Java in that it starts downloading a file (.jar or .swf) as soon as it gets displayed. Java waits (I believe) until the entire jar files is loaded, but Flash does not. I want the same ability, but with a compressed archive file. I would like to access files in the archive as soon as the bytes necessary for their decompression are downloaded.
For example I'm downloading the archive into a memory buffer, and as soon as the first file is possible to decompress, I want to be able to decompress it (also to a memory buffer).
Are there any formats/libraries that support this?
EDIT: If possible, I'd prefer a single file format instead of separate ones for compression and archiving, like gz/bzip2 and tar.
There are 2 issues here
How to write the code.
What format to use.
On the file format, You can't use the .ZIP format because .ZIP puts the table of contents at the end of the file. That means you'd have to download the entire file before you can know what's in it. Zip has headers you can scan for but those headers are not the official list of what's in the file.
Zip explicitly puts the table of contents at the end because it allows fast adding a files.
Assume you have a zip file with contains files 'a', 'b', and 'c'. You want to update 'c'. It's perfectly valid in zip to read the table of contents, append the new c, write a new table of contents pointing to the new 'c' but the old 'c' is still in the file. If you scan for headers you'll end up seeing the old 'c' since it's still in the file.
This feature of appending was an explicit design goal of zip. It comes from the 1980s when a zip could span multiple floppy discs. If you needed to add a file it would suck to have to read all N discs just to re-write the entire zip file. So instead the format just lets you append updated files to the end which means it only needs the last disc. It just reads the old TOC, appends the new files, writes a new TOC.
Gzipped tar files don't have this problem. Tar files are stored header, file, header file, and the compression is on top of that so it's possible to decompress as the file it's downloaded and use the files as they become available. You can create gzipped tar files easily in windows using winrar (commercial) or 7-zip (free) and on linux, osx and cygwin use the tar command.
On the code to write,
O3D does this and is open source so you can look at the code
http://o3d.googlecode.com
The decompression code is in o3d/import/cross/...
It targets the NPAPI using some glue which can be found in o3d/plugin/cross
Check out the boost::zlib filters. They make using zlib a snap.
Here's the sample from the boost docs that will decompress a file and write it to the console:
#include <fstream>
#include <iostream>
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/zlib.hpp>
int main()
{
using namespace std;
ifstream file("hello.z", ios_base::in | ios_base::binary);
filtering_streambuf<input> in;
in.push(zlib_decompressor());
in.push(file);
boost::iostreams::copy(in, cout);
}
Sure, zlib for example uses z_stream for incremental compression and decompression via functions inflateInit, inflate, deflateInit, deflate. libzip2 has similar abilities.
For incremental extraction from the archive (as it gets deflated), look e.g. to the good old tar format.