bzip2 llm rewrite (~1.5-2x speedup) - #811

Open

dweiss wants to merge 22 commits into

apache:masterfrom

dweiss:bzip2-llm-rewrite

Commits on Aug 27, 2026

bzip2 decoder rewrite, step 0: differential test harness and benchmarks

dweiss
and
claude
committed
7a108f1
bzip2 decoder rewrite, step 1: 64-bit bit reader and table-driven Huffman decoding

dweiss
and
claude
committed
13e1546
bzip2 decoder rewrite, step 2: packed inverse-BWT array and bulk output path

dweiss
and
claude
committed
82a019d
bzip2 decoder rewrite, step 3: multi-chain inverse BWT, CRC slicing-by-8, loop hygiene

dweiss
and
claude
committed
0917276
bzip2 decoder rewrite, step 3d/4: greedy table-section reads, refill tuning, Java 25 assessment

dweiss
and
claude
committed
e13fc5c

Commits on Aug 28, 2026

Add temporary 'real' benchmark.'
dweiss
committed
05e1734
Add a license for rat.
dweiss
committed
7221b1a
bzip2 decoder rewrite, step 5: bulk input for markable and concatenated sources

dweiss
and
claude
committed
afc8cfc
bzip2 decoder: document that a markable input's mark is used internally

dweiss
and
claude
committed
b452ea0
NOTES: results of porting the decoder changes to libbzip2

dweiss
and
claude
committed
73320ff
bzip2 decoder rewrite, step 6: block CRC through the java.util.zip.CRC32 intrinsic

dweiss
and
claude
committed
8827cfa
Revert "bzip2 decoder rewrite, step 6: block CRC through the java.util.zip.CRC32 intrinsic"
dweiss
committed
200d350

Commits on Aug 30, 2026

bzip2 encoder, step 7: word-at-a-time rotation compares, packed Huffman costs, bulk RLE1

dweiss
and
claude
committed
64e62a2
NOTES: record the suffix-array rotation sort experiment (2x slower, dropped)

dweiss
and
claude
committed
56633bf
bzip2, step 9: parallel block decompression on an ExecutorService

dweiss
and
claude
committed
61285e2
bzip2, step 10: parallel block compression on an ExecutorService

dweiss
and
claude
committed
6fbceea

Commits on Aug 31, 2026

bzip2 parallel decoder: cap the block-delimiter scan at 4 MiB

dweiss
and
claude
committed
01b3696
bzip2 parallel decoder: allocate the read() scratch byte eagerly

dweiss
and
claude
committed
8e934cd
Clean up performance-checking stuff used during development and regression test copies of previous implementations.
dweiss
committed
fd6ca56
bzip2 parallel decoder: correct the per-block memory note in the javadoc

dweiss
and
claude
committed
971da04
bzip2: changes.xml fixes

dweiss
and
claude
committed
e0d16e3
Improve delimiter scanning 1.
dweiss
committed
d559752