I have a C++ application that requires using a standard compression format for a directory.
I thought about using zip format.
And therefore, zlib was obvious.
My problem is building the dictionary, i.e. compressing a directory containing some files with zlib to a standard zip file.
I have read that zlib does it but I don't understand how. I've checked minzip code. It seems to use gzip. I could build my own format but I want to be able to open these files using 7z for debugging (just in case).
What should I use in zlib to compress a directory?
[edit]
In minzip
I used minzip to compress 1 file into its gz version -> not zip but fine to me. (I will have that on various compilers in the world -- having a standard format is easier in case there is an issue on the client's platform)
Also, there is this code in main. It loops on a list of files and writes it to an output.
But where is the information on the file location in the archive?
do {
if (uncompr) {
if (copyout) {
file = gzopen(*argv, "rb");
if (file == NULL)
fprintf(stderr, "%s: can't gzopen %s\n", prog, *argv);
else
gz_uncompress(file, stdout);
} else {
file_uncompress(*argv);
}
} else {
if (copyout) {
FILE * in = fopen(*argv, "rb");
if (in == NULL) {
perror(*argv);
} else {
file = gzdopen(fileno(stdout), outmode);
if (file == NULL) error("can't gzdopen stdout");
gz_compress(in, file);
}
} else {
file_compress(*argv, outmode);
}
}
} while (argv++, --argc);
Sorry if this is obvious.
Look at minizip.c for an example of how to use minizip to create zip files. minizip is in the contrib directory of the zlib distribution, which means that it is third-party contributed code. It is not part of the zlib library.
Note that gzip is not zip. The zlib format is not zip. Raw deflate data is not zip. Those are all different formats. Raw deflate data is one of the compressed formats permitted within the zip format, and in fact is the most common. zlib is useful for processing zip files, but it does not do that by itself. zlib by itself provides only the compression and decompression engines and CRC calculation.
Do not mix together zip - standard for archive, which can containe multiple files in it, and zlib - which is used in zip to compress the single file/data stream. To compress a directory you should use minizip/infozip or any other library which is compatible with ZIP archive format.
I was misunderstanding a bit ZIP and the compression.
Finally I decided to implement the ZIP archive format following wikipedia, this page. And I used extensively HxD tool on windows and 7z to test a lot.
At the end I found out that the format of the archive is simple, and I support only 1 disk, N files with directories, and compression based on zlib.
I also use zlib for crc32 checksum.
so if someone has the problem in the future, see the zip format on wikipedia and use zlib for crc32, then if you want compress your chunks with zlib.
Thanks to all of you
Related
I need to compress the all of the contents of a directory into a single lz4 archive, i couldn't figure out a way to do that. I already have read all the available parameters, still no success. Kindly suggest me how can i achieve this.
Things like lz4, gzip, bzip2, and xz simply compress a stream of bytes. You need another utility, like tar, to convert a set of directories and files into a stream of bytes. The output of tar is then fed to the compressor. That's why you see archives with names like tar.gz or tar.xz.
How to use tar with lz4?
I like to think of myself as reasonably computer and google literate (I'm OK ar searching the web). However I've recently had the need to compress into .ngz format and for the life of me I can't find a program that will allow me to compress into that format. I can open the files with 7zip, but it wont allow me to create an ngz archive. Any help appreciated.
It is likely a gzipped cpio file. Try cpio -i < whatever.ngz. It will automatically recognize the gzip compression and decompress it, as well as extract the archive. cpio -oz < list > archive.ngz will make a new archive given the list of files in list.
I'm using zlib and C++ to compress/decompress files.
I can now deflate any file and put it to .z extension. But when inflating .z file I have to manually put the needed extension. Inflating works as it should and I can easily inflate .z file to stdout or file.
I 've read zlib manual but couldn't manage, how to solve my problem. I suppose I need to dig the archive's header for this information?
Any help would be appreciated, thanks!
The zlib library will compress and decompress both zlib and gzip streams. zlib streams have very compact header and trailer with no provision to store file name information. The gzip header and trailer on the other hand can store a file name, which can be used when decompressing. You should read the zlib documentation to see how to use the gzip format, and how to process the gzip header.
Is there a way to create a zip archive with the full folder content or multi-files.
I actually looked the example on the web but each time it's a compression of file only based on a buffer like for example : gzip_compressor() or gzwrite()
I can't give a full path in input but only a file buffer.
=> Then no folder compression nor multi-file compression ???
Please note that I would like to use zlib/gzip or boost (the only library i can link)
I think I missed something there...
Can you please help me ?
Marc.
There are several libraries out there to handle zip files. They use zlib for the compression, decompression, and crc32 operations. You should look at libzip and DotNetZip.
It looks like this will work. If you are on a Unix environment you may have to run the Java version:
http://www.7-zip.org/sdk.html
I need compress library for following needs:
1) Packing directory in one file with extension of my choice ( .pack for example )
2) Work with content in this file (directory) without unpaking it
3) Mechanism for encrypting
I already know about zlib, but for me it is ugly documented and not written anywhere what features are support from it
Packing directory in one file with extension of my choice ( .pack for example )
Most archivers don't require you to use a particular file extension. Regardless, one can still invoke file on a file to guess its content type.
Work with content in this file (directory) without unpaking it
It's possible to create a file list manually and prepend any archive file with that. Often, .sh installers for Linux contain a shell script in the beginning, followed by some magic line like __ARCHIVE_START__ followed by a binary archive. Hence it is possible to read lines from a file until __ARCHIVE_START__ line has been read. The rest of the file is an archive file.
Mechanism for encrypting
One easy way is to use different libraries for archiving and encrypting:
Bundle the files into one .tar.
Archive .tar into say .tar.xz.
Prepend the .tar.xz with file list followed by __ARCHIVE_START__ line.
Encrypt the file with any encryption library you please.
What you want is not a compression library. You want a compression, archiving, and encryption library or libraries. You need archiving to put a directory of files into a single file.
You can use zlib to do the compress part, but not archive or encrypt. zlib is documented in zlib.h and you can see a usage example there, as well as many examples in the source distribution package.
You can construct your own archiving format, or you can use existing ones for which there are libraries such as zip or tar, both of which use or can be directed to use zlib.
You can use OpenSSL for strong encryption.