Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
An aligned tablet that introduces many measurements repeatedly copied full-width value, bitmap and accounting arrays while holding the DataRegion write lock. This backport registers missing measurements in insertion order and extends the TVList in one batch, resizing the reordered arrays once. Writes without schema extension keep the single-pass path, and new columns retain implicit nulls and lazy value blocks.
Cherry-picks #18816 to
dev/1.3, including both1f05d5d3fb858519d580e4df6c1b766e47c03f63and5f27eb1ac3bc00f8568f672a14cc1ef9bdf581a3.The backport uses the 1.3 measurement IDs, memchunk/tablet APIs and memory-accounting method. The upstream regression tests exposed two gaps in 1.3's handling of lazy blocks: an existing bitmap could hide a null value block, and WAL sizing/writing could dereference null Binary entries or unmaterialized arrays. Handle implicit nulls before consulting bitmaps and serialize null blocks with the existing value placeholders and null flags, preserving the WAL layout. A cross-block WAL round-trip test covers every supported scalar type and verifies the serialized byte count and that serialization keeps the source value blocks lazy.
Regression coverage also includes existing/new/deleted and duplicate measurements, reordering with and without schema extension, nonzero tablet offsets, binary and primitive memory accounting, clone/clear/reuse, WAL recovery, handover, and a 10,000-column tablet followed by another write.
Validation
mvn spotless:apply -pl iotdb-core/datanodemvn test -pl iotdb-core/datanode -am -Dtest=AlignedWritableMemChunkTest,AlignedTVListTest,MemChunkDeserializeTest,MemTableFlushTaskTest,PrimitiveMemTableTest,AlignedTVListIteratorTest -Dsurefire.failIfNoSpecifiedTests=false— all 26 reactor modules built successfully; 67 tests passed in both configured Surefire executions with no failures, errors or skips. The initial run compiled DataNode and its dependencies from a fresh worktree; the final run recompiled the compatibility changes.git diff --checkThis PR has:
Key changed classes:
AlignedWritableMemChunk,AlignedTVList.