[cesarfrancots]

#2541 and #2545 have merged, so this diff is now only this PR's three commits:

• perf(video-decode): pipeline the Media Foundation GPU readback
• fix(scripts): reject broken exports and start benchmarks inside their job, which fixes review feedback on scripts/capbench.py (that script comes from perf(windows): fence-waited readbacks, overlapped MF export decode, cheaper screen capture #2541)
• fix(video-decode): hold the Media Foundation sample until its surface is copied

Summary

Every Media Foundation frame is read back to the CPU, because the zero-copy path has been off since b971194. The decoder thread used to copy each sample to a staging texture and Map it straight away. As a result:

• The CPU sat blocked in Map while the GPU decoded and copied the sample.
• The GPU sat idle while the CPU copied the frame out.

Profiling the 3024×1964 screen track of the reference recording gave these per-frame costs:

Step	Time per frame
ReadSample	0.26 ms
Map (waiting for the GPU)	4.3 ms
memcpy	1.2 ms

In the 1080p60 export, waiting for the next frame's decode (4.4 s) took longer than rendering (3.2 s). Decoding was the critical path.

MediaFoundationDecoder::read_nv12 pipelines the readback:

• Ring of two staging textures. The ring is created with the decoder, so a failed allocation fails decoder creation instead of leaving a half-built ring.
• Copy queued before map. The copy of sample N+1 is queued before sample N is mapped. The GPU decodes and copies N+1 while the CPU copies N out.
• One copy per sample. Each sample is copied from Media Foundation's output surface straight into staging. Previously it also went through a pool texture that only the disabled zero-copy path needs.
• Same samples, same order. read_nv12 returns exactly what the one-at-a-time path did. It drains the queued sample before reporting end of stream or a read error, and a seek drops whatever is queued.
• Sample held until copied. Each IMFSample stays alive until the copy out of its texture slice is queued. Releasing a sample lets Media Foundation decode into that slice again, so releasing it earlier sometimes returned another frame's pixels. The first version of this branch had that race; the new test caught it intermittently (3 of 15 runs), and it passes 30 of 30 with the fix.

The rendering decoder now caches NV12Data directly. Its cached frames had zero-copy fields that were never filled; those are gone, along with the texture-read-failure tracking and two DecodedFrame zero-copy constructors that this branch was the last caller of. A failed Map is now reported as a read error. Previously it cached a black frame.

Verification

• Export output is byte-identical. The 1080p60 export of the reference recording has the same sha256 as on refactor(rendering): drive the Media Foundation decoder from one request state machine #2545 (a88dca3b…).
• New D3D11 test. pipelined_readback_returns_every_sample_in_order is #[ignore] like the existing device test, so CI doesn't run it. Run it with cargo test -p cap-video-decode --lib -- --include-ignored.
  • It compares every (pts, pixels) pair from read_nv12 against a one-at-a-time readback.
  • It checks the end-of-stream outcome and reading again after a seek.
  • It was mutation-checked. Each of these fails it: mapping the wrong staging texture, not dropping queued samples on seek, and returning a read error before the queued sample.
• cargo test -p cap-rendering --lib media_foundation: 13 pass.
• cargo clippy -p cap-rendering -p cap-video-decode --all-targets -- -D warnings is clean.
• Performance. These are interleaved A/B runs with scripts/capbench.py against the refactor(rendering): drive the Media Foundation decoder from one request state machine #2545 build (n=8). Each row is the median, with a bootstrap 95% CI.

Metric	#2545	This PR	Change
export-1080p60 wall	7.76 s	7.51 s	−3.3% [−8.2%, −0.3%] better
export-1080p60 fps	238.3	246.4	+3.4% [+0.3%, +8.9%] better
export-1080p60 CPU	15.7 s	15.0 s	−4.1% [−17.9%, +2.4%] same
export-4k60 wall	18.53 s	18.56 s	+0.2% [−1.7%, +2.5%] same
decode sequential	298.0 fps	286.4 fps	−3.9% [−11.9%, +2.6%] same
decode random access avg	70.1 ms	72.9 ms	+4.1% [−0.1%, +7.5%] same
peak commit, 1080p60 export	2588 MB	2584 MB	−0.2% same

The playback benchmarks run the FFmpeg decoder, because that is what the editor uses on Windows, so they do not exercise this change and are left out of the table. 4K export is limited by the encoder (encoder_send dominates the loop stats), so this change doesn't move it. Random access needs one extra sample decoded after each seek before the first frame comes back. It leans about 3 ms slower, but the CI still crosses zero. decode-display random access also varies by about 20% between sessions on the same build. The gain is modest; the clearest win is the 1080p export.

Notes for reviewers

• scripts/capbench.py (from perf(windows): fence-waited readbacks, overlapped MF export decode, cheaper screen capture #2541) now:

  • rejects an export as an invalid run if ffprobe can't read it, its frame count is wrong, or it falls outside the requested size;
  • starts each benchmark suspended and resumes it only after it joins the CPU-accounting job object.

• read_sample, MFDecodedFrame, FramePool and Nv12PlaneConverter are untouched and still public. Nothing in the repo reads frames through them any more, so they are the scaffolding for a future zero-copy path. Don't mix read_sample with read_nv12 on one decoder: read_nv12 keeps one sample queued. If zero-copy isn't coming back, they and the plane shader can be deleted in a follow-up.

• On the reference recording and the h264-decoder-lifecycle.mp4 fixture, Media Foundation ends each stream with MF_SOURCE_READERF_ERROR rather than end-of-stream. That was true before this PR too: the Stream error warnings at segment ends are not new.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no new actionable defects were found.

Summary

This PR gives Media Foundation two staging textures so the next frame’s GPU copy can overlap the current frame’s CPU copy.

• Media Foundation reads frames while the GPU prepares the next one.
• Windows benchmarks join their accounting job before they start.

_{Reviews (19) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile}

[vercel]

@cesarfrancots is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai

[cesarfrancots]

@greptileai review

[cesarfrancots]

@greptileai review

[cesarfrancots]

@richiemcilroy Greptile is at 5/5 on the current head (4f2a22c), and all review threads are resolved. Since the last round I merged the latest main (one conflict in crates/recording/src/cursor.rs, keeping main's cursor_id_for_image alongside this stack's cursor cache) and fixed Greptile's one new finding (capbench now rejects labels that aren't plain file names). #2541, #2545 and #2583 are all at 5/5 and mergeable. The only failing checks are the Vercel deploys, which need authorization for fork PRs.

[cesarfrancots]

@greptileai review

[cesarfrancots]

@greptileai review

[cesarfrancots]

@greptileai review

[cesarfrancots]

@greptileai review

[cesarfrancots]

@greptileai review