Speeding up compression

We usually use compression when the only way to achieve faster data transmission is often reduce its size (especially  over the net). Canonical compression tools offer poor parallelism support, while multi-core system are wide adopted today.

If you want to keep working with versatile tool like gzip and step up to a higher level of performance, take a look at pigz (pronounced “pig-zee”, http://zlib.net/pigz , same lead programmer of zlib).

Command line is barely identical to gzip; by default it uses all available cores to spread calculation over. To limit core usage use the-p switch:

tar c  /data/path | pigz -p 3 > archive.tar.gz

uses 3 processes, if you run on a quad-core system, a processor  is made available for other tasks.

Lascia un commento