I'm looking in "PhysicsFS" documentation and search for way to archive and compress data but can't find.Is it possible and if it is how i can do that
PhysicsFS zip support
PhysicsFS has support for reading files from a zip file mounted at an arbitrary point in the "virtual filesystem" that it provide. This effectively provide decompression from a ZIP archive.
However, PhysicsFS has no support to add or modify the content of a ZIP archive. It only permit to write uncompressed files in what is called "the write directory" in its documentation.
So, to summarize: PhysicsFS only support reading from ZIP archives, not writing to it.
For the compression you are on your own: just zip all the written files using an external compressor if you need so.
PhysicsFS usage
There is a small tutorial for PhysicsFS here.
It is very simple to use:
// initialize the lib
PHYSFS_init(NULL);
// "mount" a zip file in the root directory
PHYSFS_AddToSearchPath("myzip.zip", 1);
// set a directory for writing
PHYSFS_setWriteDir(const char *newDir);
// open a file for reading
PHYSFS_file* myfile = PHYSFS_openRead("myfile.txt");
// open a file for writing
PHYSFS_file* myfile = PHYSFS_openWrite("output_file.bin");
// get a file size
PHYSFS_sint64 file_size = PHYSFS_fileLength(myfile);
// read data from a file (decompress only if path is inside a zip mount point)
char* myBuf = new char[file_size];
int length_readed = PHYSFS_read(myfile, myBuf, 1, file_size);
// write data to a file (uncompressed)
char* myBuf = new char[new_file_size];
//...fill myBuf...
int length_writed = PHYSFS_write(myfile, myBuf, 1, new_file_size);
// close a file
PHYSFS_close(myfile);
// deinitialize the lib
PHYSFS_deinit();
Is it possible that you meant "PhysFS".
That is a library to read files from a zipfile. Basically like the Quake engine does.
Related
I've been using Java's standard library for writing Zip Archive files and, in that case, I do not have to know the files that are going to be stored into a zip file beforehand: I would simply create a new ZipEntry and then write into the ZipFile stream.
On the other hand, LibArchive and ZLib for C++ have to set the to-be-archived file information beforehand: in my case the data that I want to compress comes from an extern stream, and hence I cannot query the size of the stream itself. Is there a library/way to use those libraries such that the initial file information is not required?
EDIT Moreover, the following C++ code with LibArchive produces a valid ZIP file containing a file with no data.
a = archive_write_new();
archive_write_add_filter_gzip(a);
archive_write_set_format_pax_restricted(a);
archive_write_open_filename(a, path.c_str());
entry = archive_entry_new(); // Note 2
archive_entry_set_pathname(entry, entryName.c_str());
archive_entry_set_perm(entry, 0666);
archive_write_header(a, entry);
size_t s = archive_write_data(a, buff, len);
archive_entry_free(entry);
archive_write_close(a);
archive_write_free(a);
Check the sample on the official Zlib site. In sample bytes from source file compressed to the new file until EOF on source. File size of source file is not used.
The only thing you should change in sample to use your custom stream is this line:
strm.avail_in = fread(in, 1, CHUNK, source);
Use your function to get bytes from stream to fill in buffer.
To handle ZIP archive in Zlib use contrib/minizip. Here is a discussion on how to use it.
I'd like to write some files directly to zip archive file (rather that creating them first on some folder and copy them to the archive on the second stage).
Therefore, i'm wondering if there's an option to set the ofstream to point directly on the file inside the archive.
for example, say i have archive in /tmp called data.zip, and inside it there's a file data1.log
can i do something like :
std::ofstream ostr("/tmp/data.zip/data1.log", std::ios::binary);
and start pushing data using the '<<' operator ?
thanks,
can i do something like :
std::ofstream ostr("/tmp/data.zip/data1.log", std::ios::binary);
and start pushing data using the '<<' operator ?
No, that's not possible.
Also note the std::ostream& operator<<(std::ostream&,const T&) operator is explicitly reserved for text formatted output, not to write binary data.
To achieve such you would need a std::streambuf implementation, that wraps the incoming character data to a file that is (finally?) compressed and added to the archive.
The C++ standard library has no notion how to magically interact with binary .zip files.
Also what would you mean "start pushing data"? A .zip archive also contains information about particular compressed file names and their relative paths.
How would you interact with the std::ofstream interface to specify which file's data to add actually?
You should research on c++ wrappers for the LZMA/7zip library, that let you control adding files to archives.
I want to extract a file from vhdx file. There is a way to do this. -Mount the disk and read file from mounted location. But I want to write a CPP/C# code for this. I am able to get vhdx file details with MS provided API (visrtdisk.dll) which consists files size, GUID, etc, but I am not getting any API which can give me the disk structure (MBE/Partitions/FileSystem/etc) from given vhdx file.
Please see this C# library http://discutils.codeplex.com/, it is very comprehensive library and supports lot of file systems (including VHD, VHDx, ISO, EXT, HFS, HFSPlus etc.)
[The sample taken from home page and modified it for vhdx instead of vhd].
long diskSize = 30 * 1024 * 1024; //30MB
using (Stream vhdStream = File.Create(#"C:\TEMP\mydisk.vhdx"))
{
Disk disk = Disk.InitializeDynamic(vhdStream, diskSize);
BiosPartitionTable.Initialize(disk, WellKnownPartitionType.WindowsFat);
using (FatFileSystem fs = FatFileSystem.FormatPartition(disk, 0, null))
{
fs.CreateDirectory(#"TestDir\CHILD");
// do other things with the file system...
}
}
I'm trying to compress a text file with QT:
QFile inFile("d:\\build\\Directories\\Debug\\files\\developer.txt");
bool open_file_result = inFile.open(QIODevice::ReadOnly);
QByteArray ba = inFile.readAll();
QFile file("d:\\build\\Directories\\Debug\\files\\developer.gz");
bool open_zip_result = file.open(QIODevice::WriteOnly);
QDataStream out(&file);
out << qCompress(ba);
file.close();
open_file_result and open_zip_result are both true. I can also see the zip file (it also has a size, such as 50KB) but I can't open it. I'm getting the following error:
The archive is either in unknown format or damaged.
What am I doing wrong? If you have a better/another way to compress a text file, please tell me!
There is a difference between the Zip compression algorithm and the Zip container. You're confusing the two.
You need to do some research into the Zip container format that will help you locate and extract a zipped file within a Zip container. Once you've gotten that file, you can apply qUncompress to that file only.
Try looking at the QuaZip, which is a library that has been written for this purpose.
I'm writing a browser plugin, similiar to Flash and Java in that it starts downloading a file (.jar or .swf) as soon as it gets displayed. Java waits (I believe) until the entire jar files is loaded, but Flash does not. I want the same ability, but with a compressed archive file. I would like to access files in the archive as soon as the bytes necessary for their decompression are downloaded.
For example I'm downloading the archive into a memory buffer, and as soon as the first file is possible to decompress, I want to be able to decompress it (also to a memory buffer).
Are there any formats/libraries that support this?
EDIT: If possible, I'd prefer a single file format instead of separate ones for compression and archiving, like gz/bzip2 and tar.
There are 2 issues here
How to write the code.
What format to use.
On the file format, You can't use the .ZIP format because .ZIP puts the table of contents at the end of the file. That means you'd have to download the entire file before you can know what's in it. Zip has headers you can scan for but those headers are not the official list of what's in the file.
Zip explicitly puts the table of contents at the end because it allows fast adding a files.
Assume you have a zip file with contains files 'a', 'b', and 'c'. You want to update 'c'. It's perfectly valid in zip to read the table of contents, append the new c, write a new table of contents pointing to the new 'c' but the old 'c' is still in the file. If you scan for headers you'll end up seeing the old 'c' since it's still in the file.
This feature of appending was an explicit design goal of zip. It comes from the 1980s when a zip could span multiple floppy discs. If you needed to add a file it would suck to have to read all N discs just to re-write the entire zip file. So instead the format just lets you append updated files to the end which means it only needs the last disc. It just reads the old TOC, appends the new files, writes a new TOC.
Gzipped tar files don't have this problem. Tar files are stored header, file, header file, and the compression is on top of that so it's possible to decompress as the file it's downloaded and use the files as they become available. You can create gzipped tar files easily in windows using winrar (commercial) or 7-zip (free) and on linux, osx and cygwin use the tar command.
On the code to write,
O3D does this and is open source so you can look at the code
http://o3d.googlecode.com
The decompression code is in o3d/import/cross/...
It targets the NPAPI using some glue which can be found in o3d/plugin/cross
Check out the boost::zlib filters. They make using zlib a snap.
Here's the sample from the boost docs that will decompress a file and write it to the console:
#include <fstream>
#include <iostream>
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/zlib.hpp>
int main()
{
using namespace std;
ifstream file("hello.z", ios_base::in | ios_base::binary);
filtering_streambuf<input> in;
in.push(zlib_decompressor());
in.push(file);
boost::iostreams::copy(in, cout);
}
Sure, zlib for example uses z_stream for incremental compression and decompression via functions inflateInit, inflate, deflateInit, deflate. libzip2 has similar abilities.
For incremental extraction from the archive (as it gets deflated), look e.g. to the good old tar format.