Are there good applications (software) that perform dictionary based compression algorithm (LZ77 and LZW). And it is better if the application show: Compression ratio, Compression and decompression Time.
I want to apply the compression in the text file and see the changes of file’s content after compressing.
Thanks
Probably the most widespread compression/decompression library is zlib, which uses LZ77. It is incredibly portable and runs on Linux and Windows. It also has a license with very few restrictions.
Starting with Windows XP, Windows supports LZ compression natively (see related functions).
Related
For a scientific project i need to compress video data. The video however doesn't contain natural video and the quality characteristics of the compression will be different than for natural footage (preservation of hard edges for example is more important than smooth gradients or color correctness).
I'm looking for a library that can be easily integrated in an existing c++ project and that let's me experiment with different video compression algorithms.
Any suggestions?
Look at FFmpeg. It is the the most mature open source tool for video compression and decompression. It comes with a command line tool, and with libraries for codecs and muxers/demuxers that can be statically or dynamically linked.
As satuon already answered, FFmpeg is the go-to solution for all things multimedia. However, I just wanted to suggest an easier path for you than trying to hook your program up to its libraries. It would probably be far easier for you to generate a sequence of raw RGB images within your program, dump each out to disc (perhaps using a ridiculously simple format like PPM), and then use FFmpeg from the command like to compress them into a proper movie.
This workflow might cut down on your prototyping and development time.
As for the specific video codec you will want to use, you have a plethora of options available to you. One of the most important considerations will be: Who needs to be able to play your video and what software will they have available?
Is this possible without any consistency problem.
Which popular compression algorithm will best suite for these kind of purpose ?
A compressed file system is the simplest way to do that. You can enable compression in NTFS, ZFS or BTRFS, depending on your OS. But the compression is not great.
A fast algorithm with an flexible container would also do. Zip seems workable: How to update one file in a zip archive
we have a C++ MFC application and C# Web Service. They are communicating over HTTP, but as they are exchanging text data, compression will help a lot. But for some reasons, we can't use an external library.
We basically need to compress a byte array on one side and decompress it on the other side.
What should we use to compress our data? The best scenario would be if there is something in MFC/win32 api. Or is there some simplistic code with at most LGPL license that we could integrate into our project?
As has already been said, the zlib is probably what you are looking for.
There are several algorithms within:
The deflate and inflate pair
zlib itself
lzo
The simpler is probably lzo (I advise to pass the uncompressed size on the side), but zlib isn't very complicated either and the compression rate can be parameterized (speed / size trade off) which can be a plus depending on your constraints.
For XML data (since you were speaking of web services), LZO gave me a ~4x compression factor.
Can't you just switch on HTTP compression? http://en.wikipedia.org/wiki/HTTP_compression
zlib has a very liberal license.
I'm working on a library to work with Mobipocket-format ebook files, and I have LZ77-style PalmDoc decompression and compression working. However, PalmDoc compression is only one of the two currently-used types of text compression being used on ebooks in the wild, the other being Dictionary Huffman aka huffcdic.
I've found a couple of implementations of the huffcdic decoding algorithm, but I'd like to be able to compress to the same format, and so far I haven't been able to find any examples of how to do that yet. Has someone else already figured this out and published the code?
i have been trying to use http://bazaar.launchpad.net/~kovid/calibre/trunk/view/head:/src/calibre/ebooks/compression/palmdoc.c but compression doesnt produce identical results, & there are 3 - 4 descrepencies also read one related thread LZ77 compression of palmdoc
This question on archiving PDF's got me wondering -- if I wanted to compress (for archival purposes) lots of files which are essentially small changes made on top of a master template (a letterhead), it seems like huge compression gains can be had with inter-file compression.
Do any of the standard compression/archiving formats support this? AFAIK, all the popular formats focus on compressing each single file.
Several formats do inter-file compression.
The oldest example is .tar.gz; a .tar has no compression but concatenates all the files together, with headers before each file, and a .gz can compress only one file. Both are applied in sequence, and it's a traditional format in the Unix world. .tar.bz2 is the same, only with bzip2 instead of gzip.
More recent examples are formats with optional "solid" compression (for instance, RAR and 7-Zip), which can internally concatenate all the files before compressing, if enabled by a command-line flag or GUI option.
Take a look at google's open-vcdiff.
http://code.google.com/p/open-vcdiff/
It is designed for calculating small compressed deltas and implements RFC 3284.
http://www.ietf.org/rfc/rfc3284.txt
Microsoft has an API for doing something similar, sans any semblance of a standard.
In general the algorithms you are looking for are ones based on Bentley/McIlroy:
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.11.8470
In particular these algorithms will be a win if the size of the template is larger than the window size (~32k) used by gzip or the block size (100-900k) used by bzip2.
They are used by Google internally inside of their BIGTABLE implementation to store compressed web pages for much the same reason you are seeking them.
Since LZW compression (which pretty much they all use) involves building a table of repeated characters as you go along, such as schema as you desire would limit you to having to decompress the entire archive at once.
If this is acceptable in your situation, it may be simpler to implement a method which just joins your files into one big file before compression.