Skip to content

[Fix] Preserve query memory registrations during stale fragment cleanup - #18834

Merged
JackieTien97 merged 1 commit into
masterfrom
fix/memory-pool-query-registration-race
Oct 12, 2026
Merged

JackieTien97 merged 1 commit into
masterfrom
fix/memory-pool-query-registration-race

Conversation

@JackieTien97

Copy link
Copy Markdown
Contributor

Description

A delayed fragment cleanup can retain an old query-memory map while another fragment removes the last registration and a later fragment creates a replacement map. Removing the outer entry by query ID alone then deletes the replacement, causing MemoryPool.free() to fail when buffered results are consumed.

Only remove the outer entry when it still references the map being cleaned up. The identity check stays inside the existing registration lock, while memory-usage checks remain outside it.

Add a deterministic concurrent regression test that pauses an empty fragment's cleanup at the registration lock, replaces the query map, and verifies that the new reservation survives and can be freed. Additional tests cover missing fragments, other active fragments, deferred and repeated cleanup, and forced memory-leak reporting.

Validation

  • Before the production fix, the new concurrent test fails with expected:<1> but was:<0> because the replacement registration is removed.
  • English full-reactor build with focused unit tests passed: MemoryPoolTest (27 tests) and SharedTsBlockQueueTest (2 tests).
    • mvn clean test -Dtest=MemoryPoolTest,SharedTsBlockQueueTest -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false
  • Chinese full-reactor compilation passed:
    • mvn test-compile -DskipTests -P with-zh-locale
  • IoTDBTableViewQueryIT passed all four aligned/flush combinations in a clean worktree:
    • mvn clean verify -DskipUTs -Dit.test=IoTDBTableViewQueryIT -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -DDataNodeMaxHeapSize=1024 -DintegrationTest.forkCount=1 -pl integration-test -am -P with-integration-tests -P TableSimpleIT
  • Spotless formatting and git diff --check passed.

Checklist

  • Self-reviewed concurrent registration and deregistration.
  • Added comments explaining the identity check.
  • Added unit tests for the race and cleanup semantics.
  • Tested the affected query path in an IoTDB test cluster.

Key changed classes

  • MemoryPool
  • MemoryPoolTest

@codecov

codecov Bot commented Oct 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 45.90%. Comparing base (5c27559) to head (b0b96f6).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18834      +/-   ##
============================================
- Coverage     45.91%   45.90%   -0.01%     
  Complexity      712      712              
============================================
  Files          5502     5502              
  Lines        398764   398776      +12     
  Branches      51963    51968       +5     
============================================
- Hits         183090   183070      -20     
- Misses       215674   215706      +32     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JackieTien97
JackieTien97 merged commit 86b46e6 into master Oct 12, 2026
37 of 40 checks passed
@JackieTien97
JackieTien97 deleted the fix/memory-pool-query-registration-race branch October 12, 2026 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant