I'm having a problem with QuaZip.
I have a program that extracts files from zip archives. The unzip library I was using hadn't been maintained in about 10 years and had problems with the archive contained more than about 65000 files, so I converted my program to use QuaZip.
It turns out that QuaZip has the same problem. I have an archive that 7-Zip shows has 67946 objects. QuaZip only sees 2397 files. ZipInfo also shows only 2397 files. It appears that these tools have not been updated for the newer zip64 format, although QuaZip does seem to use QuaZipFileInfo64 which presumably is there to support the ZIP64 format. Also, Wikipedia documents QuaZip as supporting the new format.
Calling getEntriesCount() for the quazip object returns 2397.
Using the JlCompress::getFileList code to get the list of files only returns 2397.
Not 100 percent sure what I am asking. Does QuaZip support more than 65500 files in an archive? If so, is there a reason why it would be acting as if it doesn't?
As I suspected in my last comment, the number of entries field in the central directory header is incorrect having overflowed on the creation. The solution I came up with was to remove the code in unzGoToNextFile that assumes that that entry is correct.
Related
I am using a header library in my code to set my output directory. I would be running almost 12000 executables in parallel and I want a common output folder which is one level up. Here is how I use it
global::directories().setOutputDir("./outputfolder/");
where outputfolder is a folder in the current directory with the executable. I would like this string to be one folder up directory (somethong like ../outputfolder)
Unfortunately setOutputDir takes inputs as string (http://www.palabos.org/documentation/develguide/globalDefs_8h_source.html)
I looked around and found that boost library would be the most appropriate way to achieve what I want. Just wanted to know if there can be a workaround using the standard C++ before I delve into the library
Thanks..
I'm compiling CEF (Chromium Embedded Framework) for our local html5 presentation.
I should say I'm very new for all this (CEF and C++).
I've already optimized cefclient project for the presentation, but I need to embed all html/js/css/etc files into project (reading from local storage is not an option).
As I understood, I should use .zip or .pak (renamed zip) files to embed. But how can I use them inside the project?
Should I use some lib for unzipping (zlib?) or there is another popular way? And how can I be sure that files will be compiled into project?
Sorry for such basic questions but there are very few information about this (or google hates me today).
Thank you for any help!
UPD: found great tool - WBEA (http://asterclick.drclue.net/WBEA.html), it looks like exactly what I want to, but works pretty slow (with JS).
UPD 2: It turns out that there are many ways to make HTML5 desktop application, for example Node-Webkit.
Here is an article that compares some of them http://clintberry.com/2013/html5-apps-desktop-2013/
You need:
Create zip file whitin your resources.
Embed it as win32 resource (after this step you will get correct executable with .zip file inside).
Create custom scheme handler to access this zip file.
CefZipReader class will be handly to implement handler from step 3.
Look around, may be something like what you want already exist somewhere.
This sounds very similar to self extracting installers.
No need to compile anything, just concatenate the zip to the end of the executable. All you need to do is find the offset at runtime from the start of the executable. This can be done easily by writing a large magic number and looking for it later.
Example Linux:
cat app magic_number data > new_app
Example Windows:
copy app.exe /B + magic.dat /B + data.dat /B new_app.exe
I need compress library for following needs:
1) Packing directory in one file with extension of my choice ( .pack for example )
2) Work with content in this file (directory) without unpaking it
3) Mechanism for encrypting
I already know about zlib, but for me it is ugly documented and not written anywhere what features are support from it
Packing directory in one file with extension of my choice ( .pack for example )
Most archivers don't require you to use a particular file extension. Regardless, one can still invoke file on a file to guess its content type.
Work with content in this file (directory) without unpaking it
It's possible to create a file list manually and prepend any archive file with that. Often, .sh installers for Linux contain a shell script in the beginning, followed by some magic line like __ARCHIVE_START__ followed by a binary archive. Hence it is possible to read lines from a file until __ARCHIVE_START__ line has been read. The rest of the file is an archive file.
Mechanism for encrypting
One easy way is to use different libraries for archiving and encrypting:
Bundle the files into one .tar.
Archive .tar into say .tar.xz.
Prepend the .tar.xz with file list followed by __ARCHIVE_START__ line.
Encrypt the file with any encryption library you please.
What you want is not a compression library. You want a compression, archiving, and encryption library or libraries. You need archiving to put a directory of files into a single file.
You can use zlib to do the compress part, but not archive or encrypt. zlib is documented in zlib.h and you can see a usage example there, as well as many examples in the source distribution package.
You can construct your own archiving format, or you can use existing ones for which there are libraries such as zip or tar, both of which use or can be directed to use zlib.
You can use OpenSSL for strong encryption.
How can I zip directory in C++. I read this question: How do I zip a directory of files using C++? But I'd prefer a way that uses something like gzip, zlib and boost(because I do not want to add new libs to the project). Winapi-way is also acceptable (if it exists). And I do not want to start new process.
I would like a code sample. Thanks in advance
You want zip but you don't want to use any libraires?
Do you want to be bound by a particular licence - if so then simply copy all the code from zlib and the zip add-on into your own code.
If you can't use their licence then get the specs and write your own clean room implementation - make sure that you haven't seen the zlib or zip code base though.
The other alternative is to bundle a freely available zip command line client and call it with a system() call
edit: if you mean you are already using zlib then minizip does the directory stuff - it's usually included with zlib in the contrib directory
You can use boost iostream which includes compression functionalities. Have a look at the documentation here: http://www.boost.org/doc/libs/1_46_1/libs/iostreams/doc/index.html
It seems that in fact in this case that won't work for a directory of files.
Thanks to the guys that helped me with my previous question (linked just for reference).
I can place the files fileTypeTest.cpp, libmagic.a, and magic in a directory, and I can compile with g++ -lmagic fileTypeTest.cpp fileTypeTest. Later, I'll be testing to see if it runs in Windows compiled with MinGW.
I'm planning on using libmagic in a small GUI application, and I'd like to compile it statically for distribution. My problem is that libmagic seems to require the external file, magic. (I'm actually using my own shortened and compiled version, magic_short.mgc, but I digress.)
A hacky solution would be to code the file into the application, creating (and deleting) the external file as needed. How can I avoid this?
added for clarity:
magic is a text file that describes properties of different filetypes. When asked to identify a file, libmagic searches through magic. There is a compiled version, magic.mgc that works faster. My application only needs to identify a handful of filetypes before deciding what to do with them, so I'll be using my own magic_short file to create magic_short.mgc.
This is tricky, I suppose you could do it this way... by the way, I have downloaded the libmagic source and looking at it...
There's a function in there called magic_read_entries within the minifile.c (this is the pure vanilla source that I downloaded from sourceforge where it is reading from the external file.
You could append the magic file (which is found in the /etc directory) to the end of the library code, like this cat magic >> libmagic.a. In my system, magic is 474443 bytes, libmagic.a is 38588 bytes.
In the magic.c file, you would need to change the magichandle_t* magic_init(unsigned flags) function, at the end of the function, add the line magic_read_entries and modify the function itself to read at the offset of the library itself to pull in the data, treat it as a pointer to pointer to char's (char **) and use that instead of reading from the file. Since you know where the offset is to the library data for reading, that should not be difficult.
Now the function magic_read_entries will no longer be used, as it is not going to be read from a file anymore. The function `magichandle_t* magic_init(unsigned flags)' will take care of loading the entries and you should be ok there.
If you need further help, let me know,
Edit:
I have used the old 'libmagic' from sourceforge.net and here is what I did:
Extracted the downloaded archive into my home directory, ungzipping/untarring the archive will create a folder called libmagic.
Create a folder within libmagic and call it Test
Copy the original magic.c and minifile.c into Test
Using the enclosed diff output highlighting the difference, apply it onto the magic.c source.
48a49,51
> #define MAGIC_DATA_OFFSET 0x971C
> #define MAGIC_STAT_LIB_NAME "libmagic.a"
>
125a129,130
> /* magic_read_entries is obsolete... */
> magic_read_entries(mh, MAGIC_STAT_LIB_NAME);
251c256,262
<
---
>
> if (!fseek(fp, MAGIC_DATA_OFFSET, SEEK_SET)){
> if (ftell(fp) != MAGIC_DATA_OFFSET) return 0;
> }else{
> return 0;
> }
>
Then issue make
The magic file (which I copied from /etc, under Slackware Linux 12.2) is concatenated to the libmagic.a file, i.e. cat magic >> libmagic.a. The SHA checksum for magic is (4abf536f2ada050ce945fbba796564342d6c9a61 magic),
here's the exact data for magic
(-rw-r--r-- 1 root root 474443 2007-06-03 00:52 /etc/file/magic) as found on my system.
Here's the diff for the minifile.c source, apply it and rebuild minifile executable by running make again.
40c40
< magic_read_entries(mh,"magic");
---
> /*magic_read_entries(mh,"magic");*/
It should work then. If not, you will need to adjust the offset into the library for reading by modifying the MAGIC_DATA_OFFSET. If you wish, I can stick up the magic data file into pastebin. Let me know.
Hope this helps,
Best regards,
Tom.
I can tell you how to compile a library in statically - you simply pass the path to the .a file on the end of your g++ command - .a files are just archives of compiled objects (.o). Using "ldd fileTypeTest" will show you the dynamically linked libraries - ${libdir}/libmagic.so shouldn't be in it.
As for linking in an external data file... I don't know - Can you not package the application (.deb|.rpm|.tar.bz2)? On windows, I'd write an installer using NSIS.
In the past I've built self extracting archives. Basically it is a .exe file consisting of a .zip archive and code to unzip it. download the .exe, run it, and poof! you can have as many files as you want.
http://en.wikipedia.org/wiki/Self-extracting_archive