Repository navigation
Optimize positional field dispatch with compile-time indices - #741
Merged
liuzicheng1987 merged 6 commits intoOct 16, 2026
Merged
liuzicheng1987 merged 6 commits into
liuzicheng1987 merged 6 commits into
Conversation
added 6 commits
October 15, 2026 04:03
Select the known field directly through the existing ViewReader assignment path instead of scanning every field for each runtime index. Preserve validation, error collection, and cleanup behavior. Add regression coverage for multiple field errors, truncated input cleanup, empty objects, and runtime-index reader compatibility.
Replace the per-element field scan with a constexpr table of field-specific readers, preserving bounds checks, defaults, validation, and cleanup. Cover supported serialization formats and Generic; benchmark JSON and MessagePack with and without DefaultIfMissing.
Collaborator
|
Thanks for the contribution. |
liuzicheng1987
merged commit Oct 16, 2026
8866cac
into
munich-data-engineering:main
176 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Positional object readers already know which field comes next, but currently pass a runtime index through a scan of all reflected fields. Dispatch directly to the selected field instead.
ViewReaderoverload that reuses the existing assignment path.NoFieldNamesarray callbacks retain a runtime position, so their two shared view readers use a constexpr table of field-specific functions. Both ordinary andDefaultIfMissingpaths retain bounds checks, missing-field handling, validation, error collection, and cleanup.NoFieldNameschange are separate commits.The
NoFieldNameschange is covered in JSON, MessagePack, CBOR, FlexBuffers, UBJSON, Cereal, YAS, Boost.Serialization, and Generic. It does not enable unsupportedNoFieldNamescombinations: Avro's ordinary nested-record writer fails before reading, Cap’n Proto requires a struct root, and BSON/XML/TOML/YAML explicitly reject this processor.Validation:
The new benchmarks read a struct containing a vector of 100,000 deterministic records with either 2 or 16 float fields. Input generation, serialization, schema construction, and equality checks are outside the timer; parsing and output destruction are timed. These use ordinary reflect-cpp APIs, with
NoFieldNamesandDefaultIfMissingonly in their labeled cases.Both variants include #738, the reserve-capacity changes from #739 (
ce5db49). This isolates each format's dispatch change; bulk-vector I/O and adapter inlining are excluded. The PR branch itself remains based on upstream main and does not depend on #739.Wider records benefit substantially more. The two-field Boost case is about 4% slower (2.505 → 2.611 ms); this is a workload-dependent tradeoff, not a universal speedup.
Compile-time and code-size tradeoffs for the extension commits: three alternating release compilations of each new benchmark source file (after warmup, without PCH or linking) measured approximately unchanged compile time for YAS and NoFieldNames, +4.2% for Boost, +2.9% for Avro, and +4.0% for Cap’n Proto. Avro’s object text section grew 15.1%; other text-section changes were within 3%. NoFieldNames object-file size grew 16.3%, while its text-section size was essentially unchanged. These are source-file measurements, not full-project build or final executable sizes.
The PR introduces the shared test helper, shared benchmark helper, explicit compile-time dispatch test, and matching Cereal 2-/16-field benchmark. Added the
field_dispatchcases to the existing all-format benchmark executable and Linux/macOS/Windows benchmark summaries../build/benchmarks/all/reflect-cpp-all-format-benchmarks \ --benchmark_filter='field_dispatch|points_read_reflect_cpp_cereal' \ --benchmark_min_warmup_time=0.1 \ --benchmark_min_time=0.2s \ --benchmark_repetitions=5