Skip to content

[Subscription] Diagnose stalled replay and WAL footer recovery - #18970

Merged
jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/subscription-replay-diagnostics
Oct 22, 2026
Merged

jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/subscription-replay-diagnostics

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Summary

Add bounded, low-overhead diagnostics for subscription replay stalls and WAL footer recovery after abrupt termination.

What changed

  • Reuse the existing stalled-replay warning throttle and record the actual writer barrier that blocks progress.
  • Include bounded admission/memory, buffered-head, search-index, writer-progress, safe-frontier, retention, and WAL-iterator state in the stall snapshot.
  • Sample at most four writers per diagnostic group, prioritizing unknown and blocking writers.
  • Emit one bounded WARN only after a WAL footer is actually repaired, summarizing file size, entry count, writer progress, and a few samples.
  • Keep normal write/replay behavior unchanged: no per-entry logging, WAL format changes, extra fsync, or retention/ACK changes.

Validation

  • English full-reactor test-compile -DskipTests
  • Targeted WAL repair, prefetch queue, writer ordering, and progress iterator tests
  • Spotless formatting and git diff --check

The original production root cause still needs confirmation from the next reproduction using these diagnostics.

@jt2594838
jt2594838 merged commit 4f86b56 into apache:master Oct 22, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants