[hongzhi-gao]

Description

Write-memory accounting is spread across MemTables, TsFileProcessors and DataRegions, requiring callers to update several counters together. This change introduces named, categorized owner blocks that propagate usage to their accounting scope and tie rollback/release to the owner's lifecycle.

Accounting and lifecycle

• Add WriteMemoryBlock and WriteMemoryManager for MemTable arrays, Binary values, recovered totals, TsFile metadata and flush temporary reservations. Pending reservations count toward admission but become used MemTable memory only after admission succeeds. WAL failures roll back the admitted deltas.
• Replace positional write-memory deltas with named fields across row, mixed-row and split-tablet paths. MemTable release closes its blocks and updates the region total once; query-held TVLists retain their existing resource ownership.
• Preserve the WAL snapshot's aggregate memory field. Recovery places that aggregate in a separate category rather than guessing an array/Binary breakdown.
• Scope flush reservations to actual worker lifetime: propagate worker failures, cancel and await both workers, clear temporary references, then release the reservation. Budget reload retains outstanding reservations.
• Store each WAL queue entry's original charge and refund it on dequeue or interrupted enqueue.

Array-pool accounting and hot paths

Count idle arrays by actual shallow size under the existing per-type queue locks. A new ReadOnlyMemoryBlock exposes an exact snapshot to the existing memory-manager tree; readers acquire queue locks in a fixed order. This avoids an additional global atomic update on each array borrow/return. WriteMemoryBlock uses a monitor-guarded long; region totals remain atomic. Zero-delta writes skip reservation creation while retaining pressure checks.

SQL, configured memory limits, file/WAL formats and the existing admission policy are unchanged. Accounting follows write ownership, not total JVM heap liveness. New exception messages are provided in both English and Chinese.

Verification

Added 4 lifecycle integration tests, all passed in a real 1 ConfigNode + 1 DataNode environment using the repository’s fixtures, run through JUnitCore.

The tests cover:

• Array allocation and reuse: write across the 63/64/65-row boundaries, introduce sparse and new aligned columns, then repeat flushes and overwrite previously flushed data through unsequence writes.
• Rejection and retry: submit oversized WAL entries through row and Tablet writes, check that rejected writes leave no residual memory charge, and verify subsequent writes succeed.
• Restart and recovery: force a DataNode restart, check recovered data and write-memory accounting, then verify subsequent writes and flushes.
• Table-model split writes: write Tablets spanning multiple devices and time partitions, checking data and accounting across the resulting splits.

Assertions check both query results and live database write-memory metrics, including memory charged by writes and released after flush or rejection.

Performance observations

Compared with eb0e3f1f2080e34bf5593bfd1cf63b62ecdba2e0, using the same workload/configuration and three interleaved runs per version:

Workload (4 writers)	Median throughput	P50 RPC latency	P99 RPC latency
Non-aligned Tablet, 1024 rows	-0.24%	+3.88%	+0.10%
Aligned single row	+0.50%	-0.73%	+3.19%
Aligned Tablet, 1024 rows	+4.36%	-4.60%	-3.58%

Environment: shared Xeon Gold 6130 KVM host, 32 vCPUs, JDK 21, 1C1D/SimpleConsensus, ASYNC WAL, memory control enabled, 2 GiB DataNode heap, array size 64, MemTable threshold 32 MiB, compaction and timed flush disabled. Fixed per-writer warmup/measurement calls were 50k/100k for non-aligned Tablet, 125k/250k for aligned rows, and 20k/40k for aligned Tablet. Each run checked acknowledged row counts through queries after a final FLUSH.

These are controlled class-overlay comparisons, not two independent clean distribution builds. The throughput window excludes the final FLUSH; it does not measure synchronous durability throughput. Shared-host variability and existing QueryId waits affect single-row results. The repeated throughput regression observed during development did not recur in the final comparisons, but the latency increases above remain review considerations; these measurements do not establish a universal speedup.

---

This PR has:

• been self-reviewed, including concurrent reads, writes and read/write interaction.
• added comments explaining accounting ownership, reservation ordering and snapshot lock ordering.
• added unit/component tests for lifecycle and failure paths.
• added integration tests and run them against a real 1C1D environment.

Key changed/added classes

WriteMemoryBlock, WriteMemoryManager, AbstractMemTable, TsFileProcessor, DataRegionInfo, TsFileProcessorInfo, MemTableFlushTask, SystemInfo, MemoryControlledWALEntryQueue, PrimitiveArrayManager, ReadOnlyMemoryBlock, MemoryManager.

[Caideyipi]

Reviewed commit ef0d5bfa35c3 against the PR's merge base eb0e3f1f2080. I did not find an actionable regression introduced by this change.

The review covered row, mixed-row and split-tablet memory admission/WAL rollback; MemTable and metadata release; WAL snapshot format compatibility; idle-array accounting; and flush-worker failure/interruption cleanup.

Validation

89 focused tests passed:

• 59 DataNode tests across WriteMemoryBlockTest, MemTableMemoryBlockTest, WriteMemoryLifecycleTest, PrimitiveArrayManagerTest, MemTableFlushTaskTest, and TsFileProcessorTest.
• 3 tests in ReadOnlyMemoryBlockTest.
• 27 tests in AlignedTVListTest.

git diff --check also passed.

Full-reactor test compilation could not complete because the local JVM failed to reserve native memory. The integration tests in this PR were not rerun as part of this review.