gzip, gunzip and zcat · Lesson 73

tar bundles; gzip compresses. It works on one file at a time: gzip big.txt replaces big.txt with big.txt.gz, keeping the permissions and date, and gunzip big.txt.gz (or gzip -d) does the reverse. Text compresses very well: log files often shrink to a tenth of their size. Files that are already compressed (JPEG, MP4, other .gz) hardly shrink at all.

~% seq 1 2000 > big.txt
~% ls -l big.txt
-rw-r--r--    1 root     root          8893 Oct  4 23:17 big.txt
~% gzip big.txt
~% ls -l big.txt*
-rw-r--r--    1 root     root          4218 Oct  4 23:17 big.txt.gz
~% gunzip big.txt.gz
~% ls big.txt*
big.txt

Keep, stdout, test

Replacing the original is the default, which surprises people. Three options change the behaviour. -k keeps the input file. -c writes the result to standard output and leaves the input alone, so you choose the name with a redirection: gzip -c big.txt > copy.gz. -t tests the integrity of a .gz and prints nothing if it is fine; its exit status says whether the file is damaged. gunzip refuses to overwrite an existing file unless you add -f.

~% gzip -k notes.txt          # notes.txt and notes.txt.gz
~% gzip -c big.txt > big2.gz  # big.txt untouched
~% gzip -t big2.gz && echo intact
intact
~% gunzip -k notes.txt.gz
gunzip: can't open 'notes.txt': File exists

zcat: reading without unpacking

Rotated logs are usually kept as .gz. You do not need to decompress them to read them: zcat file.gz (the same as gunzip -c) prints the original text to stdout, and the pipe does the rest. zcat old.log.gz | grep -c ERROR counts errors, zcat old.log.gz | tail shows the end. Nothing is written to disk and the .gz stays as it was. On a busy server that matters: decompressing a year of logs just to search them could fill the disk.

tar + gzip = .tar.gz

A .tar.gz (or .tgz) is simply a tar archive that has been gzipped. On most distributions GNU tar does both at once with z: tar czf site.tar.gz site and tar xzf site.tar.gz. The BusyBox tar in this VM was built without that option (tar: invalid option -- z), and it cannot read a .tar.gz directly either (tar: short read). That is a good excuse to learn the form that works everywhere: tar writes the archive to stdout with f -, gzip compresses the stream, and on the way back zcat feeds tar through stdin.

~% tar cf - site | gzip > site.tar.gz
~% zcat site.tar.gz | tar tf -
site/
site/index.html
~% mkdir -p restore
~% zcat site.tar.gz | tar xf - -C restore

BusyBox gzip accepts the levels -1 (fastest) to -9 (smallest) for compatibility, but this build produces the same output for all of them. It also happily compresses a .gz again into .gz.gz, which only wastes time: check the name before you run it.

Commands in this lesson

Command	What it does
`gzip file`	Replace file with file.gz.
`gzip -k file`	Compress and keep the original.
`gzip -c file > out.gz`	Compress to stdout; you choose the name.
`gunzip file.gz`	Decompress (same as gzip -d).
`gzip -t file.gz`	Test integrity; exit status 0 if fine.
`zcat file.gz`	Print the decompressed text.
`tar cf - dir \| gzip > a.tar.gz`	Make a .tar.gz with a pipe.
`zcat a.tar.gz \| tar xf -`	Unpack a .tar.gz with a pipe.

Quiz

After `gzip report.txt`, what is in the directory?

report.txt and report.txt.gz
Only report.txt.gz
Only report.txt, now smaller
report.gz

How do you count the lines containing ERROR in app.log.gz without creating any file?

gunzip app.log.gz; grep -c ERROR app.log
zcat app.log.gz | grep -c ERROR
grep -c ERROR app.log.gz
tar tf app.log.gz | grep ERROR

In this VM, how do you create docs.tar.gz from the docs directory?

tar czf docs.tar.gz docs
gzip docs
tar cf - docs | gzip > docs.tar.gz
zcat docs > docs.tar.gz

What does `-` mean in `tar cf - docs`?

Remove docs after archiving
Write the archive to standard output
Archive the previous directory
Exclude hidden files

`gzip -t backup.gz` prints nothing. What does that mean?

The file is intact (exit status 0)
The file is empty
gzip failed silently
The file was decompressed

Practice

Compress `/root/lab/l73/access.log` with gzip, replacing the original (only `access.log.gz` should remain).

`/root/lab/l73/old.log.gz` is a compressed log. Without decompressing it to disk, write the number of lines containing `ERROR` into `/root/lab/l73/errors.txt`.

Pack the directory `/root/lab/l73/site` into the compressed archive `/root/lab/l73/site.tar.gz`, with member names starting at `site/`.

Open this lesson in the app to do the tasks in a real Linux machine and have them checked.