Create .7z archive with custom header(Creating a .unity3d file) - compression

So, I recently came across the .unity3d file for a game a used to play, and unpacked it using a tool. (http://en.unity3d.netobf.com/) Now, I've made the tweaks the the game I needed to to make it run on a local server, and have come across the issue of how to compress the files back into a .unity3d file. I've reverse engineered the tool and determined that .unity3d files are LZMA compressed( just like a .7z archive ), but the header is "UnityWeb" instead of "7z". How might I achieve this?

7z is open source. If the only difference is indeed that header, then get the sources, find where the header is, change it and compile your own compression utility. Watch out for other constants describing the headers and signatures though (e.g. length of the signature). I'd suggest starting with line 9 of the file Xz.c (defining XZ_SIG and XZ_FOOTER_SIG).

Related

libtdms: How to access the metadata of the .tdms file in order to process it?

For reading .tdms file with c++ I downloaded libtdms from sourceforge.net. I use Visual Studio 22 on Windows so I also installed MinGW for executing the Makefile.
The library does what it should, reading the .tdms file, but I dont know how to access the data in order to process and visualize it. Do I have to look inside the variables of libtdms?
Is there an easier way of outputting some kind of table with all the informations without getting deep into the code. (not that easy as mechanical engineer)
How do I include the full library dictionary with its makefiles into a possible programm?
So far i only included single .h files when programming
My goal is to get measurements from the .tdms file (values, coordinates,...) an then use them for plotting datapoints 2D and 3D.
this kind of output would be helpful for processing the data further

How to serialize a diff of two folders optimally in C++

I'm trying to develop a file diff format for multiple files recursively in folders. Consider a source directory containing patched files and a destination directory containing original files. Write a size minimal diff file which expresses the difference between all files in the source and destination directory which can be applied to the original files in order to transform the original files into the patched files.
For this purpose I found the dtl library. Which algorithm or feature of the library should I use to write a file diff to the disk which I can then later read back and apply in order to patch the file? Any example code for this? I tried writing the result of the shortest edit script (SES) to the disk but I realized that I needed to specify the character and operation for every single byte. This of course makes the output file bigger than the entire comparison file, making this diff format entirely redundant since storing the entire target file instead would've saved more storage.
As another reference, this is very similar to how version control systems like git or svn operate but I don't want to use those since I'm mainly dealing with binary files and the simple requirement of creating and applying patches.
After doing some more search, I found the HDiffPatch project.
It worked fine apparently but it seems to take long on bigger folder comparisons:
diff usage: hdiffz [options] oldPath newPath outDiffFile
patch usage: hpatchz [options] oldPath diffFile outNewPath
EDIT:
Another good option is open-vcdiff but it only supports individual files.
use HDiffPatch: you can run hdiffz with "-s-48" for up speed;
or try "-s-32" , "-s-1k", "-s-128k" ...

Hiding application resources

I'm making a simple game with SFML 1.6 in C++. Of course, I have a lot of picture, level, and data files. Problem is, I don't want these files visible. Right now they're just plain picture files in a res/ subdirectory, and I want to either conceal them or encrypt them. Is it possible to put the raw data from the files into a resource file or something? Any solution is okay to me, I just don't want the files exposed to the user.
EDIT
Cross platform solutions best, but if they don't exist, that's okay, I'm working on windows. But I don't really want to use a library if it's not needed.
Most environments come with a resource compiler that converts images/icons/etc into string data and includes them in the source.
Another common technique is to copy them into the end of the final .exe as the last part of the build process. Then at run time, open the .exe as a file and read the data from some determined offset, see Embedding a filesystem in an executable?
The ideal way for this is to make your own archive format, which would contain all of your files' data along with some extra info needed to split files distinctly within it.

C++ file container (e.g. zip) for easy access

I have a lot of small files I need to ship with an application I build and I want to put this files into an archive to make copying and redistributing more easy.
I also really like the idea of having them all in one place so I need to compare the md5 of one file only in case something goes wrong.
I'm thinking about a class which can load the archive and return a list of files within the archive and load a file into memory if I need to access it.
I already searched the Internet for different methods of achieving what I want and found out about zlib and the lzma sdk.
Both didn't really appeal to me because I don't really found out how portable zlib is and I didn't like the lzma sdk as it is just to much and I don't want to blow up the application because of this problem. Another downside with zlib is that I don't have the C/C++ experience (I'm really new to C++) to get everything explained in the manual.
I also have to add that this is a time critical problem. I though some time about implementing a simple format like tar in a way I can easy access the files within my application but I just didn't find the time to do that yet.
So what I'm searching for is a library that allows me to access the files within an archive. I'd be glad if anybody could point me in the right direction here.
Thanks in advance,
Robin.
Edit: I need the archive to be accessed under linux and windows. Sorry I didn't mention that in the beginning.
For zipping, I've always been partial to ZipUtils, which makes the process easy and is built on top of the zlib and info-zip libraries.
The answer depends on whether you plan to modify the archive via code after the archive is initially built.
If you don't need to modify it, you can use TAR - it's a handy and simple format. If you want compression, you can implement tar.gz reader or find some library that does this (I believe there are some available, including open-source ones).
If your application needs random access to the data or it needs to modify the archive, then regular TAR or ZIP archives are not good. Virtual file system such as our SolFS or CodeBase file system will fit much better: virtual file systems are suited for frequent modifications of the storage, while archives target mainly write-once-read-many usage scenarios.
zlib is highly portable and very widely used. if you can't make sense of the C++ interface, there are alternatives for many other languages - see 'Related External Links' here.
Take another look before you search for something different.
If you're using Qt or Windows you can also pack data into the executable's resource area. You would only have to distribute the executable file using this technique. There's a well defined API already written and tested to access that data.
The zlib API is the way to go. Simple and portable. Lookat unzip.h header for APIs that access archive files. It is in C and very easy.
If the files are small, you can dump them into string literals (search for bin2h utility) and include in your project. Then change the code that read the files. If all files are currently read using ifstream class, simply changing it to istringstream class and recompile the code.
Try using Quazip - it's quite simple to use. You can use it as a stream from which you read the compressed file on the fly.

Game File Archive Format

I want to create a single data file that holds all the data that my game will need, and I want it to be compressed. I looked into tar and gzip, but I downloaded their sources and I don't know where to begin. Can somebody give me some pointers to how I can use these?
Unless you will always load all files from the archive, TAR/GZ might not be a very good idea, because you cannot extract specific files as you need them. This is the reason many games use ZIP archives, which do allow you to extract individual files as required (a good example is Quake, whose PK3 files are nothing but ZIP files with a different extension).
A bit of searching brought up Minizip, which is a ZIP library built on top of zlib. I couldn't find any separate documentation for it, but the header files seem to include a lot of comments, and I believe you can get off with it.
If you mean that you want your game to read out of the archive at runtime, then I recommend decompressing each time the game is run into a temporary folder, and then using the files as required. This can be achieved through using a library for decompressing whatever archive format you use. Look into zlib.