Repository navigation
Never reuse an existing Playwright fixture server - #15052
Merged
Merged
Conversation
Locally, reuseExistingServer adopted any process answering 200-403 on the fixture port without checking what it serves. When two worktrees run Playwright at the same time, the second run tests the first worktree's rendered tests/docs/playwright HTML: a spec passed with this checkout's fixture deleted. An unrelated app answering 200 made every spec fail on missing elements instead, which reads like a feature regression. Nothing starts the fixture or multiplex server ahead of a run, so reuse only ever applied to that collision case. A busy port now fails at once with Playwright's "is already used" error. The cost is starting the server on every invocation (about 1.4 s on Windows). A process answering 404 is still not detected as busy: Playwright starts its own http.server, which on Windows binds over the occupied port via SO_REUSEADDR, and the run times out after 120 s. That case never runs tests against the wrong content, so it is left as is and documented.
The multiplex webServer starts on every Playwright run, whatever specs are selected, and its port 1948 is fixed in revealjs/multiplex.qmd. With reuse off, a second concurrent run (another worktree) failed at once with "http://127.0.0.1:1948 is already used", even on its own QUARTO_PLAYWRIGHT_PORT, so concurrent runs became impossible. The multiplex server only relays socket messages and serves no test fixtures, so sharing another checkout's instance cannot make a run test the wrong HTML. Verified: with 1948 held by a separate multiplex server, revealjs-multiplex.spec.ts passes and a run on its own fixture port passes alongside it. The fixture server stays never-reused.
Collaborator
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #15013, which made the Playwright fixture server port configurable with
QUARTO_PLAYWRIGHT_PORT.When another process already answers on the fixture port, a local Playwright run uses it as its fixture server without checking what it serves. With two worktrees running Playwright at the same time, the second run tests the first worktree's rendered
tests/docs/playwrightHTML: a spec passed locally with its own rendered fixture deleted. An unrelated app answering 200 on the port makes every spec fail on missing elements instead, which looks like a regression in the feature under test.Root cause
playwright.config.tssetsreuseExistingServer: !isCIon the fixture server. Locally, Playwright then reuses any server whose/answers with a status from 200 to 403. CI is not affected, asCIis set there and reuse was already off.Fix
The fixture server is never reused. A busy port now fails the run at once with Playwright's
http://127.0.0.1:<port> is already usederror, and each concurrent run sets its ownQUARTO_PLAYWRIGHT_PORT. Nothing starts the fixture server ahead of a run, so reuse only ever applied to this collision case; the cost is starting the server on each run (about 1.4 s on Windows).The revealjs multiplex server keeps
reuseExistingServer: !isCI. It starts on every run whatever specs are selected, and its port 1948 is fixed inrevealjs/multiplex.qmd, so turning reuse off for it made any second concurrent run fail, even on its own fixture port. It serves no test fixtures, so sharing it cannot make a run test another checkout's HTML.A process answering 404 on the fixture port is still not detected as busy: Playwright starts its own
http.server, which on Windows binds over the occupied port, and the run times out after 120 s. That case never runs tests against the wrong content, so it is only documented in the Playwright testing rules.Test plan
Local only, as CI keeps its current behavior:
html-callouts.spec.tspassespython -m http.server 8765serving a copy of the rendered fixture: the run stops at once withalready used(before: 3 passed with the local fixture deleted)QUARTO_PLAYWRIGHT_PORT=8766: passes, includingrevealjs-multiplex.spec.tsChecklist
I have (if applicable):
AI-assisted PR
node_modulesNote: autonomous AI agents submitting PRs without human oversight are not permitted — see the Code of Conduct.