I am trying to use pigz for parallel compress and decompress. I have found a way to compress in parallel using the following command:
tar cf - /input/dir | pigz > output_file.tar.gz
What is the equivalent command to decompress? I need to be able to specify the filename and the output directory path in the command for it to be useful for me.
Use pigz -dc for decompression to stdout. Then use > as you are in your example to direct the output to the desired path and file.
You can type just pigz for command options help.
You're probably looking for something along the lines of:
pigz -dc archive.tar.gz | tar xf -
but noting Mark Adler's (legendary, at this point) original post, pigz does not utilize multiple cores for decompression. However, it does utilize additional cores for reading, writing, and some additional calculations, which do yield a moderate performance increase over gzip. Enjoy!
tar -I pigz -xvf compressed_file.tar.gz
For some reason, pigz doesn't autocomplete ".gz" compressed files, but if you type the names of your files pigz finds them.
To decompress and keep your compressed files use: pigz -dk yourfilename.gz.
If you don't need the compressed versions use just pigz -d yourfilename.gz.
pigz --help shows all the options available.
Related
I'd like to use brotli to compress a list of files and directories. I'm able to do this with zip running
zip -r archive.zip *
and I'm looking for a similar command with brotli. I've tried
tar -cf archive.gz * && \
brotli -n -q 11 -o archive.zip archive.gz
but after decompression the zip doesn't have the same structure than with zip.
Your second command is actually right. Brotli, like Gzip/Bzip2/etc can only compress a single file.
What you must do is first package all of your files in a tarball:
tar -cvf output.tar /path/to/dir
And then compress the resulting tarball with Brotli:
brotli -j -Z output.tar
Which should leave you with a output.tar.br file (similar to *.tar.gz gzipped tarballs).
Have you tried Brotli-cli?
This comes with a lot of options to get files compressed using Brotli
you can try creating a .tar file instead of .gz
Currently I have a script that sets compression using the GZIP environment variable.
With gzip 1.8 I get a warning:
warning: GZIP environment variable is deprecated; use an alias or script
How should tar be invoked so the compression level be set?
Current command:
GZIP=-9 tar -zcf ... files to compress ...
You can specify the compression program using: -I / --use-compress-program=COMMAND.
So the original command:
GZIP=-9 tar -zcf ... files to compress ...
Becomes:
tar -I 'gzip -9' -cf ... files to compress ...
From this answer.
Depends on the OS, and the version of tar that ships with it. If it's BSD and/or macOS, you can just:
tar --options gzip:compression-level=9 -czf foo.tar.gz foo.txt
Otherwise, you could just take the compression stage into your own hands:
tar -cf - foo.txt | gzip -9 > foo.tar.gz
pigz supports the environment variable. It is compatible with gzip, so you could install it with the name gzip.
Some older versions of tar don't support the -I option to specify a compression command. So older versions cannot be used with something like -I 'gzip -9'.
A more portable and reliable solution is to handle the compression yourself with a pipe as mentioned in the other answer.
I need to compress the all of the contents of a directory into a single lz4 archive, i couldn't figure out a way to do that. I already have read all the available parameters, still no success. Kindly suggest me how can i achieve this.
Things like lz4, gzip, bzip2, and xz simply compress a stream of bytes. You need another utility, like tar, to convert a set of directories and files into a stream of bytes. The output of tar is then fed to the compressor. That's why you see archives with names like tar.gz or tar.xz.
How to use tar with lz4?
I have a tar.gz file which I have made by pigz (parallel gzip). I wanted to count the number of files inside the compressed file without decompressing it.
I use this command:
tar -tzf file.tar.gz
but I got an error:
tar: This does not look like a tar archive
tar: Skipping to next header
Is it because I used pigz instead of gzip? If yes how can I count the them now?
Since it is a tar and gzip archive you should use z option to use gzip. Then simply you can count lines with wc.
tar -tzf file.tar.gz | wc -l
you can use the tar -vv verbose option twice for full verbose, then grep the first character from file permissions. the ^ means only match first character (begin of line). the grep -c option count the lines.
drwxrwx--x directory
lrwxrwxrwx symlink
-rw-rw---- file
count regular files only
gzip -cd file.tar.gz | tar -tvv | grep -c ^-
I found the solution!
I used unpigz for those files and it changed the file extensions to .tar . after that I could use tar -tzf without any problems.
Thanks!
Is there a 7zip or winzip command to use so that the compression avoids thumbs.db files?
For 7zip,-x is the file exclusion option switch.
7z.exe a test.7z C:test* -r -x!thumbs.db
See Excluding Files at the following link.