[lstein]

Summary

Opening a video the browser can't play (DivX/XviD .avi, MPEG-2, HEVC, 10-bit H.264…) used to mean watching a progress panel until ffmpeg finished the whole conversion. Now a full re-encode starts playing within a few seconds. When the conversion finishes, the player switches to the cached file at the same position, and from then on the scrubber works.

The single ffmpeg run uses the tee muxer to write three outputs from one encode:

1. The cached faststart MP4, as before.
2. A growing fragmented MP4, served by GET /streaming_video/… for Chrome and Firefox.
3. An HLS EVENT playlist with fMP4 segments, served by GET /streaming_hls/…/hls/{token}/{name} for Apple's player. Every browser on iOS and iPadOS, and Safari, refuses a growing MP4 of unknown length. (I measured it: the iPad sent the first range probe and then nothing more.)

Remuxes are never streamed, since they finish in seconds.

Player (video-player.js).

• Apple's player gets HLS first, then the growing MP4, then the progress panel. It's recognised by navigator.vendor === "Apple Computer, Inc." (every WebKit browser, iOS Chrome included) on an Apple navigator.platform, plus native HLS support. The platform test leaves out WebKit's Linux ports, which report the same vendor; iPadOS desktop mode reports MacIntel and still matches. Every other browser goes straight to the growing MP4, including desktop Chrome, though it also claims native HLS (see below).
• It shows a badge ("Converting… NN% — seeking works once it finishes") and keeps polling, because the poll is still the backend's liveness signal.
• On ready it swaps to the cached file and restores the position; a paused clip stays paused.
• A stream that dies after it has started playing waits for the finished file and resumes there.

Server. uvicorn.run now sets timeout_graceful_shutdown=5. A stream lasts as long as its conversion, and without the limit Ctrl+C waited for ffmpeg to finish.

Browser behaviour this was built against (measured)

• Chrome and Firefox play a growing fMP4 sent as a plain chunked 200.
  • Answering Range: bytes=0- with a 206 of what exists so far makes both fail.
  • So open-ended ranges get a 200; closed ranges get a 206 capped at 8 MB; a mid-file resume gets a 416.
• Chrome's native HLS player (desktop Chrome now claims HLS support). It is not used, because on a playlist that is still growing it behaves like a live player: it skipped large stretches (user report) or froze (reproduced with a fast-converting clip). These findings still shaped the HLS output:
  • Segment type: served as video/mp4, a segment fails with DEMUXER_ERROR_COULD_NOT_PARSE because Chrome parses the fragment as a whole file. They are served as video/iso.segment.
  • Playlist length: a live playlist with 1–2 segments fails outright, and 3 works (the spec's "three target durations from the live edge" rule). The playlist is offered only once it lists 3 segments, which is 6 s of video.
  • Start position: a live playlist otherwise starts near its newest segment. The route adds #EXT-X-START:TIME-OFFSET=0 (ffmpeg has no option for it).

Traps worth knowing about

• No codec-tag table in tee. Copied AAC from an .avi or .ts keeps its source tag, which the MP4 outputs reject, so the whole job fails. When streaming, audio is always re-encoded (cheap).
• flush_packets=1 on the fMP4 output. Without it, only the 28-byte ftyp reaches disk for a long time.
• HLS segment names are printf patterns. A %d anywhere in the cache path would break them. ffmpeg therefore runs from the cache directory with the HLS output named relative to it, and the executable, source and target are made absolute.
• The run token in HLS URLs is the HLS directory's mkdtemp name. Every run numbers its segments from index0 and the segments are cacheable, so without the token an edited video could be spliced from two runs.
• A failed HLS output is silent, because of onfail=ignore. The server clears hls_pending once the streamed MP4 holds 5 complete fragments and the playlist still isn't playable; the player then uses the MP4 instead of waiting. Both outputs get the same packets, and while streaming scene-cut keyframes are off (-sc_threshold 0), so MP4 fragment N closes when HLS segment N does. It doesn't use wall-clock time, because a slow encode would trip it. It doesn't use ffmpeg's out_time either: that follows the audio, which x264's lookahead leaves seconds ahead of the video at low frame rates, and a 5–10 fps clip with sound got given up on while its playlist was healthy.
• The HLS directory lingers 120 s after its run for a player that hasn't swapped yet. Stray .tmp files and directories are reaped by sweep_transcode_cache. Nothing new ends in .mp4, which prune and the byte budget act on.

Test plan

• Backend unit and route tests:
  • tee/HLS argument shape;
  • escaping and relative naming;
  • offer and give-up rules, including scripted progress where the playlist lags far behind out_time;
  • the fragment counter;
  • registry, token and linger lifecycle;
  • Range handling;
  • segment content type;
  • the start tag;
  • traversal and other-run tokens;
  • a segment deleted mid-request.
• Real-ffmpeg end-to-end tests:
  • a cache directory named with '[]|%05d;
  • mpeg4 + AAC .avi;
  • a dead HLS output;
  • a relative ffmpeg executable;
  • a remux that must not be streamed.
• Jest:
  • stream-then-swap;
  • the HLS → MP4 → panel order;
  • waiting for a slow playlist versus giving up on a dead one;
  • resume after a mid-play failure, with focus kept on the video;
  • the paused, closed and timeout cases.
• Real app driven with Playwright, using a 90 s 4K XviD AVI:
  • Chromium and Firefox both stream via the fMP4, starting in about 5 s;
  • a 180 s SD clip converting at about 20x plays smoothly in Chromium (it previously froze on HLS);
  • the swap keeps the position, and seeking to 80 s works afterwards.
• Manually verified in desktop Chrome on Ubuntu (fMP4 path).
• Manually verified on iPad (Chrome, which uses WebKit). This was before the last round of review fixes.
• Real ffmpeg: a 10 fps 720p clip with audio and the 4K clip both have their playlist offered and never given up on.
• Full suites: backend 1286 passed, frontend 982 passed, lint clean.
• Three rounds of fresh-context adversarial review; every confirmed finding fixed or listed below.

Known limitations (not addressed)

• Disk use during a re-encode: there are up to three copies (the kept .tmp, the fMP4 and the HLS directory), and the byte budget doesn't count them.
• Windows file locking: a playlist read can make ffmpeg's playlist rename fail, which skips one playlist update until the next segment. The lingering directory's removal can also fail while a segment is being read; the sweep reaps it later.
• Mid-run job reset: a job dropped and recreated under the same key during a run (a tab backgrounded for about 45 s) keeps writing into its replacement. The replacement's queued run then re-encodes until the watchdog kills it (about 45 s of wasted CPU), and an iPad can fall back to the panel. The root cause predates this branch.
• Drive-relative Windows album paths (C:photos) on Python 3.10/3.11 aren't made absolute.

🤖 Generated with Claude Code