I'd like to use brotli to compress a list of files and directories. I'm able to do this with zip running
zip -r archive.zip *
and I'm looking for a similar command with brotli. I've tried
tar -cf archive.gz * && \
brotli -n -q 11 -o archive.zip archive.gz
but after decompression the zip doesn't have the same structure than with zip.
Your second command is actually right. Brotli, like Gzip/Bzip2/etc can only compress a single file.
What you must do is first package all of your files in a tarball:
tar -cvf output.tar /path/to/dir
And then compress the resulting tarball with Brotli:
brotli -j -Z output.tar
Which should leave you with a output.tar.br file (similar to *.tar.gz gzipped tarballs).
Have you tried Brotli-cli?
This comes with a lot of options to get files compressed using Brotli
you can try creating a .tar file instead of .gz
Related
I have a situation where I need to use curl/wget to download a tar file from a website, based on users input. If they mention a build I need to download a tar file based on the release version, I have a logic already to switch between builds, Questions is how can i filter out a particular tar file from multiple files.
curl -s https://somewebsite/somerepo-folder/os-based.tar | grep os-based* > sample.txt
curl -s https://somewebsite/somerepo-folder/os-based2.tar
curl -s https://somewebsite/somerepo-folder/os-based2.tar
first curl downloads all files. Regex helps here, how can I place this along with curl?
if there is a mapping between the user-input and the tar file that you can think of, you can do something like this:
userInput=1
# some logic to map user-input with the tar filename to download
$tarFileName="os-based$userInput.tar"
wget "https://somewebsite/somerepo-folder/$tarFileName"
I have a large ~10GB zip file that was created using the standard Windows method (right click, select "send to compressed (zipped) folder"). I am able to unzip it just file on my Macbook.
I'm trying to unzip it on an EC2 machine. I know the file is a zip file because when I run file file.zip it says:
file.zip: Zip archive data, at least v2.0 to extract
Running unzip returns the following error:
Archive: file.zip
warning [file.zip]: 3082769992 extra bytes at beginning or within zipfile
(attempting to process anyway)
error [file.zip]: start of central directory not found;
zipfile corrupt.
(please check that you have transferred or created the zipfile in the
appropriate BINARY mode and that you have compiled UnZip properly)
Running tar xvf file.zip returns the following:
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Archive contains `<{\204\027\333"D\344\210\321o\331' where numeric off_t value expected
tar: Archive contains `S\354\202},F\3546\276[\265\273' where numeric time_t value expected
tar: Archive contains ``3c\254\372$:e' where numeric uid_t value expected
tar: Archive contains `\265\306\025+ÜĞL\352' where numeric gid_t value expected
...etc
Does anyone know what might be going wrong?
Actually, 7-zip should makes this well, you can install it by:
sudo apt-get install p7zip-full
Then, you can extract your zip file as follows:
7z e file.zip
If your zip archive has 88,000 files and you are dealing with ~10Gig of content, you will need an unzip program that supports the zip64 extension.
You can check if your unzip program supports zip64 like this
$ unzip -v | grep -i zip64
ZIP64_SUPPORT (archives using Zip64 for large files supported)
If it doesn't have ZIP64_SUPPORT, you are out of luck. I suspect your unzip doesn't support zip64.
Alternatives are to get a version of unzip that does support zip64 or use an alternative program, like z-zip.
Your Entire File did not zip most probably and you prematurely moved it. At Least that was the issue with me.
I was unable to install 7z on my machine due to no sudo access, but I managed to repair the archive using
zip -FF archive.zip --out archive_repaired.zip -fz
and unzip worked on the repaired archive.
I found the solution via this github issue
I am working on iMX6 Arm processor based hardware platform with embedded linux. I am using the tar command to compress a directory containing 3 files. When I try to decompress the folder to get back the three files I am facing different issues:
I get one or two files after decompressing.
The data in the original file and the decompressed file, there is data mismatch.
I am using these commands to create and decompress the tar file:
nohup tar -zcpf /home/root/upload_new_U1/unit1-`(date +%Y%m%d_%H-00)`.tar.gz upload_now_U1
tar -zxpf unit1-`(date +%Y%m%d_%H-00)`.tar.gz
Please help.
I have a tar.gz file which I have made by pigz (parallel gzip). I wanted to count the number of files inside the compressed file without decompressing it.
I use this command:
tar -tzf file.tar.gz
but I got an error:
tar: This does not look like a tar archive
tar: Skipping to next header
Is it because I used pigz instead of gzip? If yes how can I count the them now?
Since it is a tar and gzip archive you should use z option to use gzip. Then simply you can count lines with wc.
tar -tzf file.tar.gz | wc -l
you can use the tar -vv verbose option twice for full verbose, then grep the first character from file permissions. the ^ means only match first character (begin of line). the grep -c option count the lines.
drwxrwx--x directory
lrwxrwxrwx symlink
-rw-rw---- file
count regular files only
gzip -cd file.tar.gz | tar -tvv | grep -c ^-
I found the solution!
I used unpigz for those files and it changed the file extensions to .tar . after that I could use tar -tzf without any problems.
Thanks!
I am trying to use pigz for parallel compress and decompress. I have found a way to compress in parallel using the following command:
tar cf - /input/dir | pigz > output_file.tar.gz
What is the equivalent command to decompress? I need to be able to specify the filename and the output directory path in the command for it to be useful for me.
Use pigz -dc for decompression to stdout. Then use > as you are in your example to direct the output to the desired path and file.
You can type just pigz for command options help.
You're probably looking for something along the lines of:
pigz -dc archive.tar.gz | tar xf -
but noting Mark Adler's (legendary, at this point) original post, pigz does not utilize multiple cores for decompression. However, it does utilize additional cores for reading, writing, and some additional calculations, which do yield a moderate performance increase over gzip. Enjoy!
tar -I pigz -xvf compressed_file.tar.gz
For some reason, pigz doesn't autocomplete ".gz" compressed files, but if you type the names of your files pigz finds them.
To decompress and keep your compressed files use: pigz -dk yourfilename.gz.
If you don't need the compressed versions use just pigz -d yourfilename.gz.
pigz --help shows all the options available.