Skip to content

Never reuse an existing Playwright fixture server - #15052

Merged
cderv merged 2 commits into
mainfrom
chore/playwright-fixture-port-collision
Oct 14, 2026
Merged

cderv merged 2 commits into
mainfrom
chore/playwright-fixture-port-collision

Conversation

@cderv

@cderv cderv commented Oct 14, 2026

Copy link
Copy Markdown
Member

Follow-up to #15013, which made the Playwright fixture server port configurable with QUARTO_PLAYWRIGHT_PORT.

When another process already answers on the fixture port, a local Playwright run uses it as its fixture server without checking what it serves. With two worktrees running Playwright at the same time, the second run tests the first worktree's rendered tests/docs/playwright HTML: a spec passed locally with its own rendered fixture deleted. An unrelated app answering 200 on the port makes every spec fail on missing elements instead, which looks like a regression in the feature under test.

Root cause

playwright.config.ts sets reuseExistingServer: !isCI on the fixture server. Locally, Playwright then reuses any server whose / answers with a status from 200 to 403. CI is not affected, as CI is set there and reuse was already off.

Fix

The fixture server is never reused. A busy port now fails the run at once with Playwright's http://127.0.0.1:<port> is already used error, and each concurrent run sets its own QUARTO_PLAYWRIGHT_PORT. Nothing starts the fixture server ahead of a run, so reuse only ever applied to this collision case; the cost is starting the server on each run (about 1.4 s on Windows).

The revealjs multiplex server keeps reuseExistingServer: !isCI. It starts on every run whatever specs are selected, and its port 1948 is fixed in revealjs/multiplex.qmd, so turning reuse off for it made any second concurrent run fail, even on its own fixture port. It serves no test fixtures, so sharing it cannot make a run test another checkout's HTML.

A process answering 404 on the fixture port is still not detected as busy: Playwright starts its own http.server, which on Windows binds over the occupied port, and the run times out after 120 s. That case never runs tests against the wrong content, so it is only documented in the Playwright testing rules.

Test plan

Local only, as CI keeps its current behavior:

  • Free port: html-callouts.spec.ts passes
  • Another python -m http.server 8765 serving a copy of the rendered fixture: the run stops at once with already used (before: 3 passed with the local fixture deleted)
  • Another run holding 8765 and 1948, this run with QUARTO_PLAYWRIGHT_PORT=8766: passes, including revealjs-multiplex.spec.ts
  • No server left listening after passing and failing runs

Checklist

I have (if applicable):

  • referenced the GitHub issue this PR closes
  • updated the appropriate changelog in the PR
  • ensured the present test suite passes
  • added new tests
  • created a separate documentation PR in Quarto's website repo and linked it to this PR
AI-assisted PR
  • AI tool used: Claude Code
  • Codebase grounding: local clone, Playwright source in node_modules
  • Human review: I have reviewed, tested, and verified the AI-generated content before submitting.

Note: autonomous AI agents submitting PRs without human oversight are not permitted — see the Code of Conduct.

cderv added 2 commits October 14, 2026 13:16
Locally, reuseExistingServer adopted any process answering 200-403 on the
fixture port without checking what it serves. When two worktrees run
Playwright at the same time, the second run tests the first worktree's
rendered tests/docs/playwright HTML: a spec passed with this checkout's
fixture deleted. An unrelated app answering 200 made every spec fail on
missing elements instead, which reads like a feature regression.

Nothing starts the fixture or multiplex server ahead of a run, so reuse
only ever applied to that collision case. A busy port now fails at once
with Playwright's "is already used" error. The cost is starting the
server on every invocation (about 1.4 s on Windows).

A process answering 404 is still not detected as busy: Playwright starts
its own http.server, which on Windows binds over the occupied port via
SO_REUSEADDR, and the run times out after 120 s. That case never runs
tests against the wrong content, so it is left as is and documented.
The multiplex webServer starts on every Playwright run, whatever specs
are selected, and its port 1948 is fixed in revealjs/multiplex.qmd.
With reuse off, a second concurrent run (another worktree) failed at
once with "http://127.0.0.1:1948 is already used", even on its own
QUARTO_PLAYWRIGHT_PORT, so concurrent runs became impossible.

The multiplex server only relays socket messages and serves no test
fixtures, so sharing another checkout's instance cannot make a run test
the wrong HTML. Verified: with 1948 held by a separate multiplex server,
revealjs-multiplex.spec.ts passes and a run on its own fixture port
passes alongside it. The fixture server stays never-reused.
@posit-snyk-bot

posit-snyk-bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@cderv
cderv merged commit d2177a6 into main Oct 14, 2026
51 checks passed
@cderv
cderv deleted the chore/playwright-fixture-port-collision branch October 14, 2026 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants