Skip to content

perf: reduce per-request tracing overhead - #20

Merged
legendecas merged 1 commit into
nodejs:mainfrom
bengl:bengl/perf-hot-path
Oct 31, 2026
Merged

legendecas merged 1 commit into
nodejs:mainfrom
bengl:bengl/perf-hot-path

Conversation

@bengl

@bengl bengl commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Cuts the package's own per-request work, outside the serializer that #18 rewrites. Exported payloads, propagated headers, and span timing stay the same.

Spans. Span attributes were an object literal with __proto__: null, which V8 creates in dictionary mode, so every setAttribute was a hash-table store. The comment above it claimed the opposite. A class whose prototype is empty and has no prototype of its own keeps one shared fast-mode shape, inherits nothing, and still stores a key named __proto__ as an own property. Spans also allocated an empty events array and an unset status each, though most never get either. They now start from shared frozen defaults that addEvent and setStatus replace.

traceparent. Parsing split the header, ran three regexes, and returned another dictionary-mode literal. One anchored regex now checks the whole version-00 layout, and the parent is a small class instance. It accepts and rejects exactly what main does, both on a differential over 47 valid, malformed, and adversarial inputs and on the new table test.

IDs. Each ID was built by concatenating one two-character string per random byte. IDs are now slices of a hex string that encodes 128 random bytes at a time, so one native call covers about five spans. A slice is a view into its string, and a span can outlive its request in AsyncLocalStorage, so the chunk size matters. With one 8KB string per refill, a span kept alone pinned 8.6KB. With 128-byte chunks its two IDs pin at most two chunk strings, and the worst case is 897 bytes, under the 1,174 a span held on main. The CSPRNG, its 4KB refill, and the ID format are unchanged.

Listeners. The http server allocated a closure per request for its close check, and the client a closure plus two once() wrappers per response. Each listener is now one shared function that reads the span from the response, where the handler stores it, rather than through res.req, which application code can reassign.

fetch. Every undici request parsed its origin with new URL(). The handler now reuses the last origin's address and port.

Export. The exporter scanned the body with Buffer.byteLength for Content-Length, then passed the string to req.end(), which joins it to the headers and encodes it again. The body is now encoded to a Buffer once.

Benchmarks. New node:bench suites in benchmark/ time span construction, ID generation, and export on their own, so npm run bench reproduces the smaller rows below along with the request rows.

Performance. Node v26.12.1, main versus this PR:

Benchmark main This PR Change
http request (traced) 24.64k ops/s 27.49k ops/s +11.6%
fetch request (traced) 17.99k ops/s 19.09k ops/s +6.1%
http request (untraced) 30.73k ops/s 31.14k ops/s +1.3%, control
fetch request (untraced) 22.05k ops/s 22.02k ops/s -0.1%, control
client span construction 358 ns 76 ns 78.7% less
server span from a traceparent 485 ns 169 ns 65.3% less
IDs into a traceparent value 178 ns 88 ns 50.9% less
export spans to a collector 545k spans/s 703k spans/s +29.0%
Package CPU time, 50k traced http requests 242 ms 200 ms 17.5% less
Package CPU time, 50k traced fetch requests 249 ms 201 ms 19.4% less
Heap per span, spans held together 1,181 bytes 411 bytes 65.2% less
Heap per span kept alone, typical 1,176 bytes 681 bytes 42.1% less
Heap per span kept alone, worst case 1,174 bytes 897 bytes 23.6% less
  • Every row compares interleaved runs of main and this branch, on a machine shared with other work.
  • The first eight rows come from npm run bench. The request rows pool 22 runs per side, and the untraced rows show the noise floor. Against untraced, tracing overhead drops from 19.8% to 11.7% for http, and from 18.4% to 13.3% for fetch. The other suites are medians of nine runs per side.
  • The ID suite builds the header value and runs the check setHeader applies, because concatenated IDs also cost a flatten wherever they were first used.
  • The CPU-time rows sum every package frame, including the Node work called from them, in --cpu-prof profiles of 50k sequential requests to an in-process collector. They are medians of seven runs per side. About 90 ms of each is the serializer, which this PR leaves alone.
  • The heap rows hold 400k spans at once, or keep one span in every 199 or 201, so each kept span pins chunk strings no other kept span shares. Keeping one in every 200 happens to land every kept span on a chunk boundary, which gives the worst case.

Not taken. These were measured and dropped: a hand-written traceparent scanner, which was slower than the regex, a bigger random buffer, hoisting the port regex, and IDs encoded with one native call each, which won in a microbenchmark but not on the real request path. Deriving wall-clock time from the monotonic clock would save a clock read per span, but it changes span timing on long-running processes.

Merging. This branch merges cleanly with #18, #19, #21, and #22.

Tests. New tests interleave trace and span IDs so chunk boundaries fall mid-ID and check that no random bytes are reused, pin down the traceparent accept and reject set directly, and reassign res.req on both sides to check that spans still end. Node itself fails when res.req is reassigned before a response finishes, so the tests use the two paths where it does not: a client disconnect on the server, and a reassignment after the body ends on the client.

@bengl
bengl marked this pull request as ready for review October 30, 2026 18:35
@bengl
bengl requested a review from a team October 30, 2026 18:35

@AbhiPrasad AbhiPrasad left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for stuff like Client span construction, can we add the benchmark suites to the repo?

Tracing a request did more work than it needed to. Exported payloads,
propagated headers, and span timing stay the same.

- Keep span attributes in V8's fast mode. They were an object literal
  with __proto__: null, which V8 creates in dictionary mode, so every
  setAttribute was a hash-table store. A class with an empty,
  null-terminated prototype keeps one shared shape and still inherits
  nothing.
- Start spans from shared frozen events and status defaults. Most spans
  never add an event or set a status, but each one allocated both.
- Parse traceparent with one anchored regex instead of a split and three
  regexes, into a small class instance instead of a dictionary-mode
  literal. It accepts and rejects exactly the same inputs.
- Slice IDs from a hex string that encodes 128 random bytes at a time,
  instead of concatenating one two-character string per byte. A slice
  is a view into that string, so a span that outlives its request keeps
  up to two of these small strings alive, and still retains less memory
  than a span did before.
- Share response listeners. Each request allocated a closure, and each
  client response two once() wrappers. One function per event now reads
  the span from the response, where the handler stores it. Reading it
  through res.req instead would break if application code reassigned
  that property.
- Reuse the last parsed origin across fetch requests instead of calling
  new URL() for each one.
- Encode the export body to a Buffer once. Buffer.byteLength scanned
  the string for Content-Length, and req.end() encoded it again.
- Add node:bench suites that time span construction, ID generation,
  and export on their own.

On 50k traced requests, the package's CPU time drops about 18 percent
for http and 19 percent for fetch. The traced node:bench benchmarks
gain about 12 and 6 percent in throughput. Spans held together, as in
the export buffer, take 411 bytes each instead of 1,181. A span kept
alone, as a long-lived connection keeps its server span, takes at most
897 bytes instead of 1,174.

Signed-off-by: Bryan English <bryan@bryanenglish.com>
@bengl
bengl force-pushed the bengl/perf-hot-path branch from c0b02fc to 9d4c97a Compare October 30, 2026 19:22
@bengl

bengl commented Oct 30, 2026

Copy link
Copy Markdown
Member Author

@AbhiPrasad Good call. They're in now: benchmark/span.bench.js, benchmark/id.bench.js, and benchmark/export.bench.js. npm run bench will run them. The table uses their numbers now.

@legendecas
legendecas merged commit 7cdbacf into nodejs:main Oct 31, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants