Skip to content

[ntuple] set default initial page size to 16B (was 256B) - #23638

Merged
jblomer merged 1 commit into
root-project:masterfrom
jblomer:ntuple-initial-page-size
Oct 8, 2026
Merged

jblomer merged 1 commit into
root-project:masterfrom
jblomer:ntuple-initial-page-size

Conversation

@jblomer

@jblomer jblomer commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

This change has a negligible hit on performance (<1% for serializing 100k entries to 10k double columns uncompressed into the Null sink). It does help considerably keeping the page count low when wide and sparse EDMs are written under memory pressure (real world CMS SIM example: 3M --> 1.3M pages).

@jblomer jblomer self-assigned this Oct 7, 2026
@jblomer
jblomer force-pushed the ntuple-initial-page-size branch from ad1d8f6 to fa7024a Compare October 7, 2026 09:38
This change has a negligible hit on performance (<1% for serializing 100k
entries into 10k double columns uncompressed into the Null sink). It does
help considerably keeping the page count low when wide and sparse EDMs are
written under memory pressure (real world CMS SIM example: 3M --> 1.3M pages).
@pcanal pcanal changed the title [ntuple] set default initial page size to 16B [ntuple] set default initial page size to 16B (was 256B) Oct 7, 2026
@pcanal

pcanal commented Oct 7, 2026

Copy link
Copy Markdown
Member

I am at a tad bit confused on the unit of fInitialUnzippedPageSize, the title and comments are consistent in their use of byte but the RNTupleModel.cxx line:

    minPageBufferSize += r.size() * options.GetInitialUnzippedPageSize();

seems to use it as 'number' of elements. What am I misunderstanding?

On the other hand we also have:

// tree/ntuple/src/RColumn.cxx
42:   fInitialNElements = pageSink.GetWriteOptions().GetInitialUnzippedPageSize() / fElement->GetSize();

So on the surface it appears the usage might be inconsistent.

pcanal
pcanal previously requested changes Oct 7, 2026

@pcanal pcanal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the comment at #23638 (comment) which may or may not hint at other possible updates.

@jblomer

jblomer commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor Author

@pcanal In your referenced code snippet, r.size() is not the element size. It is the length of a vector with column representatives. We need one page for every column, so that calculation is correct. The unit is B.

@jblomer
jblomer requested a review from pcanal October 7, 2026 11:40
@pcanal

pcanal commented Oct 7, 2026

Copy link
Copy Markdown
Member

In your referenced code snippet, r.size() is not the element size. It is the length of a vector with column representatives. We need one page for every column, so that calculation is correct. The unit is B.

Doesn't that need to be also timed by the sizeof then ?

@jblomer

jblomer commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

In your referenced code snippet, r.size() is not the element size. It is the length of a vector with column representatives. We need one page for every column, so that calculation is correct. The unit is B.

Doesn't that need to be also timed by the sizeof then ?

Sorry, I don't understand that question. Can you please clarify?

@silverweed

Copy link
Copy Markdown
Contributor

Doesn't that need to be also timed by the sizeof then ?

The sizeof is taken into account by options.GetInitialUnzippedPageSize(): it's in the end minBufSize = nPages * minPageSize (where r.size() == nColumnReprs == nPages)

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

Test Results

    24 files      24 suites   3d 23h 9m 9s ⏱️
 3 884 tests  3 880 ✅ 0 💤 4 ❌
83 135 runs  83 131 ✅ 0 💤 4 ❌

For more details on these failures, see this check.

Results for commit fa7024a.

@pcanal

pcanal commented Oct 8, 2026

Copy link
Copy Markdown
Member

Doesn't that need to be also timed by the sizeof then ?
Sorry, I don't understand that question. Can you please clarify?

Re-reading the code clarified that the question was moot/misplaced.

@pcanal
pcanal dismissed their stale review October 8, 2026 09:41

Questions were clarified.

@pcanal pcanal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@jblomer
jblomer merged commit d8f7304 into root-project:master Oct 8, 2026
32 of 36 checks passed
@jblomer
jblomer deleted the ntuple-initial-page-size branch October 8, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants