Efficiency benchmark

The headline (one-glance table)

Compressed and decompressed the same 1.83 MiB corpus five times per cell. Numbers are median wall-clock; stddev reflects the run-to-run noise of the laptop itself.

All times are wall-clock end-to-end. "level" = the tool's own knob, not a number pulled from thin air. The corpus file count is 304; total uncompressed bytes 1,827,165. The tar (no compression) ratio exceeds 100 % because tar stores per-file metadata (UID, GID, mode bits, 512-byte per-record padding); for small files that adds 50 % of overhead before any deflate runs.

Reading the table

1. Compress speed: zip < lha < gzip < tar < tar.gz

At default levels:

• zip -6 is the fastest compressor (~62 ms) — deflate-without-tar-blocking, with the zip-specific DEFLATE64 disabled by default.
• lha -lh5- is the second-fastest (~50 ms) and lands within 20 % of zip.
• gzip -6 -c on a raw tar stream takes ~170 ms — deflate is a more expensive encoder than LZHUF even at default.
• tar -czf adds another ~40 ms of framing on top of the underlying deflate (~210 ms total) — the cost of per-file records and 512-byte-aligned padding.
• tar -cf (no compression) is ~80 ms — just header overhead, no deflate at all.

So at default levels: zip is ~22 % faster than lha; lha is ~76 % faster than tar.gz. Compress-speed ordering is stable across corpus types in our testing (text source, mixed binary, random data) — the individual times shift, the rank doesn't.

2. Decompress speed: tar.gz ≈ zip < lha < tar

Decompress reverses the speed ranking in interesting ways:

• zip and tar.gz decompress in ~3-5 ms — deflate decoding is cheap.
• lha decompresses in ~40 ms — this is LZHUF, which has fixed Huffman tables and emits one bit at a time. It's slower than deflate decoding on this corpus, but the absolute time is still tiny.
• tar decompress in ~82 ms — same 512-byte per-record overhead on the way out.

For the workloads most people care about (decompress 100 times from a CDN) the absolute time is dominated by network latency, not algorithm choice. All three of tar.gz, zip, and lha decompress in single-digit ms on this corpus; the differences are well below the noise floor of disk seek + HTTP request latency.

3. Compression ratio: tar.gz < gzip ≈ lha < zip

For the same kenlm corpus:

• tar.gz -6 and gzip -6 -c both hit 455 KB / 24.93 % — the lower bound achievable with deflate on this corpus. -9 finds nothing further (already at the local minimum).
• lha -lh5- lands at 547 KB / 29.93 % — ~20 % larger than deflate, in exchange for ~2.2x faster compression. This is the deliberate trade-off baked into LZHUF's 1990 design: tuned for "fast unpack on a 286 with 1 MiB RAM", not for the tightest possible ratio on a 2020s workstation.
• zip -6 lands at 567 KB / 31.01 % — slightly worse than lha because zip's deflate config is older and DEFLATE64 (the wider-window variant) is off by default.
• tar (no compression) at 2.7 MB / 150 % — the per-record headers + 512-byte padding more than double the size of small files.

The ratio difference between tar.gz and lha is about 5 percentage points — ~91 KB on a 1.8 MiB input. On a fast CDN, that 91 KB re-downloads in <100 ms; on a slow CDN, it might take 1-2 seconds. That's the actual size-vs-speed trade on the wire.

4. -9 is a trap on this corpus

For already-redundant text (source code with repeating words and whitespace), gzip -9 and tar.gz -9 find nothing new compared to -6. The corpus isn't rich enough to benefit from the extra passes; -9 wastes 5-10x more CPU for ~0 % ratio gain.

If you have truly high-entropy content (a video fragment, a database dump, encrypted data), -9 will show a real gap. This corpus doesn't.

Which tool when

Deeper reading

Why lha -lh5- is faster than tar.gz -6 on compress

LZHUF (the algorithm behind .lh5-) uses a static-Huffman-table, fast-string-match design from 1990. deflate (gzip / zlib) uses LZ77 + dynamic Huffman and a sliding window that the encoder has to decide on. For input that doesn't fully fill the window (our 1.8 MiB corpus fits in a 32 KB window with room to spare), deflate spends most of its time scanning for matches; LZHUF doesn't. The decompressor side is closer to a tie because both algorithms have the same rough decode-loop structure.

On a corpus bigger than the deflate window (32 KB at -6), deflate starts to win on ratio because it can find longer-distance matches. Our 1.8 MiB corpus is small enough that ratio doesn't move much past -6, and the encode cost dominates. For a 100 MiB corpus, expect tar.gz -9 to pull ahead on ratio at ~3-5x the compress time of lha -lh5-.

Why tar (no compress) is so much bigger than input

tar records per-file metadata (UID, GID, mode, timestamps, filenames) plus a 512-byte-per-record padding. For our 1.8 MiB corpus with 304 small files, that's ~50 % of overhead before any deflate runs. tar -czf compresses most of it back out via the LZ77 backend, but raw tar is genuinely larger than its input for small-file corpora.

Why zip -6 is faster than tar.gz -6 on decompress

zip's archive format has a single central directory at the end of the file — the decoder can read it without streaming. tar.gz has to scan the tar header for every entry to find its size, then ask the deflate decoder to fill that exact size. zip's deflate can be wrapped around the central directory; tar.gz's deflate is per-record, and each record needs its own header parse. The single-digit ms gap here is small; on a corpus with many files (thousands), zip's lead grows.

Why is lha's decompress slower than deflate / zstd / lz4?

Decompression is where LZHUF (the algorithm behind .lh5- and .lh7-) shows its age. The 1990 design is fundamentally bit-serial:

• Static Huffman tables, bit-by-bit decode. The bitstream is read one bit at a time, and each bit determines the next table lookup. This is the decode loop in src/huf.c — the main decompressor hot path. There is no parallelism in this loop: bit n+1 cannot be read until bit n has been looked up.
• LZ77 back-references resolved serially. When the decoder reads a length/distance pair, it copies length bytes from distance ago. The slide window in src/slide.c is updated byte by byte, no SIMD memmove. On a 1.83 MiB corpus, this copy loop dominates the runtime.
• No SIMD anywhere in the source. grep -nE '(mmx|sse|avx|simd|__m128i|__builtin_ia32)' src/*.c returns zero hits. The vendored 1.14i code is straight ANSI C from 1995-2003, with no architecture-specific vectorization.

So the answer to the question is: the algorithm is the ceiling, not the implementation. LZHUF was tuned in 1990 for "fast unpack on a 286 with 1 MiB RAM", and the bit-serial decode loop was the natural fit. A SIMD-optimized lha port would gain perhaps 1.5-2x on decompress (the length-distance copy is the only vectorizable chunk), but the underlying bit-serial Huffman loop is fundamentally serial. To get more decompress speed, you have to switch algorithms, not micro-optimize LZHUF.

For comparison: zstd decompress is ~5-10x faster than lha at similar ratio, and lz4 is ~3-5x faster still. Both are tuned for modern SIMD CPUs (SSE2/AVX2 on x86_64, NEON on aarch64) and have decompress loops that are SIMD-friendly by design. lha predates MMX (1996) by six years and the original maintainers never had a vector unit to target.

When this matters: lha is the right answer for a static binary that has to live on a system without zstd/lz4 installed. lha is not the right answer for a high-throughput CDN that needs microsecond-scale decompress.

How to reproduce

On any Unix-like host (macOS, Linux, BSD):

git clone --depth 1 https://github.com/ljh-sh/lha.git
cd lha
bash scripts/build.sh
cp build/src/lha /usr/local/bin/

# corpus
test -d corpus/kenlm_src || cp -R ../kenlm/upstream/kenlm corpus/kenlm_src

# 3 runs each, median. The numbers in the table above used 5 runs
# (we report median ± sample-stdev) on an Apple M-series laptop.
for tool in \
  'lha  :lha c k.lh5- corpus/kenlm_src' \
  'tar  :tar -cf k.tar -C corpus kenlm_src' \
  'tgz6 :tar -czf k6.tgz -C corpus kenlm_src' \
  'tgz9 :tar -czf -9 k9.tgz -C corpus kenlm_src' \
  'zip6 :zip -qr -6 k6.zip kenlm_src' \
  'zip9 :zip -qr -9 k9.zip kenlm_src' ; do
  label=${tool%%:*}; cmd=${tool#*:}
  rm -f k.*
  for r in 1 2 3 4 5; do
    rm -rf corpus/kenlm_src; cp -R ../kenlm corpus/kenlm_src
    t=$( { time $cmd >/dev/null 2>&1; } 2>&1 | awk '/real/{print $NF}')
    sz=$(stat -c%s k.* 2>/dev/null | head -1)
    echo "$label  t=$t  sz=$sz"
  done
done

Numbers will differ by host (~10 % on different CPUs, ~5 % across macOS / Linux); the relative ordering (zip → lha → tar → tar.gz on compress time, tar.gz → zip → lha → tar on decompress, tar.gz → lha → zip → tar on ratio) is stable.