[lxy-9602]

Purpose

Linked issue: #158

Support realtime deduplicate compaction by remapping the latest deletions to compacted files at commit time, preventing deleted rows from reappearing. Share DV materialization between write and compaction providers.

Tests

• Unit tests for rewrite normalization, DV materialization and cleanup.
• Integration tests for concurrent commits/retries, rolling outputs, restart, full deletion, partition/bucket isolation and no-hit commits.

API and Format

No public API, storage format or protocol changes.

Documentation

Generative AI tooling

Generated-by: OpenAI Codex

[SteNicholas]

@lxy-9602, I left one comment for this pull request. PTAL.

[SteNicholas]

@lxy-9602, clean up intermediate compaction outputs. For a chain F1/F2 -> F3 -> F4, normalization removes F3 from the published manifest entries, but intermediate_files is only used for validation and never cleaned up. PrepareCommit() has already cleared the writer's compact_after_ through DrainIncrement(), so Close() cannot delete F3 either, and snapshot expiration cannot discover it because no snapshot references it. Each chained compaction therefore leaves the intermediate data file and its .offset/index sidecars orphaned. Please add cleanup after successful publication and extend the chained-compaction test to verify that these intermediate files are removed while final outputs remain intact.