Skip to content

PPLT-6241: honor PERCY_SERVER_ADDRESS and harden CDP detection for Selenium 4.28+ - #277

Open
RaghavsBrowserStack wants to merge 2 commits into
masterfrom
PPLT-6241-cli-address-cdp
Open

RaghavsBrowserStack wants to merge 2 commits into
masterfrom
PPLT-6241-cli-address-cdp

Conversation

@RaghavsBrowserStack

@RaghavsBrowserStack RaghavsBrowserStack commented Oct 19, 2026 •

Copy link
Copy Markdown

Jira: https://browserstack.atlassian.net/browse/PPLT-6241

Problem

  1. CLI address (pre-existing). The SDK reads only PERCY_CLI_API, but percy exec --port N exports PERCY_SERVER_ADDRESS. On a non-default port the SDK disables itself, and the script exits 0 with an empty build.
  2. CDP on Remote (caused by Selenium 4.28+). execute_cdp_cmd now exists on every Remote driver, but only a ChromiumRemoteConnection (picked when the requested browserName is exactly chrome) understands it. On Automate with browserName: "Chrome", every CDP call raises Unrecognised command executeCdpCommand and falls back. It falls back cleanly but noisily, once per resize.
  3. Certification. The CI/dev env was capped at selenium<4.28, so 4.50 was untested.

Fix

  • The address resolves as PERCY_CLI_API → PERCY_SERVER_ADDRESS → http://localhost:5338.
  • _is_chrome() compares browserName case-insensitively in expose_closed_shadow_roots and change_window_dimension_and_wait.
  • When a driver rejects CDP ("Unrecognised command" / "unknown command" / "unknown method"), it goes into a WeakSet and CDP is skipped for it afterwards. Other CDP errors behave as before.
  • requirements.txt: the <4.28 cap is dropped (selenium>=4); pip still resolves 4.27.x on Python 3.8/3.9.

Testing

  • Python 3.10.11 + selenium 4.50.0: all suites pass (test_snapshot 96, units 51, robot 32, init 7, cache 4, driver_metadata 4). Coverage on percy/ is 100% and pylint scores 10/10. There are 11 new unit tests in tests/test_snapshot_units.py.
  • Address (percy exec --port 5506, PERCY_CLI_API unset, example-percy-selenium-python):
  • CDP (BrowserStack Win11 Chrome, browserName: "Chrome", percy_snapshot with responsive_snapshot_capture, widths 375/1280):
    • Before (2.2.1 + PERCY_CLI_API workaround): 54714705: 4 Unrecognised command errors (shadow DOM plus a failed CDP attempt for each resize)
    • Fixed: 54714683: 1 rejection (shadow DOM), after which every resize goes straight to set_window_size; snapshot taken at both widths

Summary by CodeRabbit

  • Bug Fixes
    • Improved compatibility with Chrome drivers that reject certain browser commands. Responsive resizing now falls back to standard window sizing when needed, and closed-shadow capture skips drivers that have rejected the relevant command.
    • The CLI API address now falls back to the configured server address before using the local default.
  • Chores
    • Updated the Selenium version constraint for CI and development.

Backwards compatibility

  • PERCY_CLI_API still takes precedence; PERCY_SERVER_ADDRESS is only a fallback.
  • The CDP resize path keeps the exact browserName == 'chrome' check from earlier releases, so drivers reporting e.g. Chrome still get a plain window resize.
  • setup.py keeps selenium>=3; only the CI/dev requirements change.

🤖 Generated with Claude Code

…lenium 4.28+

- Fall back to PERCY_SERVER_ADDRESS (exported by `percy exec --port`) when
  PERCY_CLI_API is unset.
- Compare browserName case-insensitively for CDP decisions.
- Remember drivers whose CDP command is rejected ("Unrecognised command") and
  skip CDP for them afterwards instead of failing every resize.
- Drop the <4.28 Selenium cap in the dev/CI requirements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Central YAML (base), Organization UI (inherited), Workspace UI (inherited)
  • Review profile: ASSERTIVE
  • Plan: Enterprise
  • Run ID: 39a52f41-5ff7-4bbf-b7ab-4306879687be
📥 Commits

Reviewing files that changed from the base of the PR and between 73409a1 and 5617a95.

📒 Files selected for processing: 2
  • percy/snapshot.py
  • tests/test_snapshot_units.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout: 7
  • GitHub Check: semgrep/ci
  • GitHub Check: Analyze (actions)
  • GitHub Check: Analyze (python)
  • GitHub Check: Lint
  • GitHub Check: Test (3.10)
  • GitHub Check: Test (3.8)
  • GitHub Check: Test (3.9)
🔇 Additional comments: 1
percy/snapshot.py: 1

924-925: 🎯 Functional Correctness

Keep lowercase-only CDP selection for responsive resizing.

"Chrome" drivers intentionally continue to use set_window_size. Do not change this branch to _is_chrome(driver) or update test_non_lowercase_chrome_keeps_window_resize.

📝 Walkthrough

The CLI API gains a server-address fallback. Snapshot capture tracks recognized CDP rejections, limits CDP use to eligible Chrome drivers, and falls back to set_window_size. Selenium constraints and unit tests are updated.

Changes: CLI API endpoint fallback

Layer / File(s) Summary
Resolve the CLI API address
percy/snapshot.py, tests/test_snapshot_units.py
The endpoint lookup checks PERCY_CLI_API, then PERCY_SERVER_ADDRESS, then http://localhost:5338. Tests cover precedence and defaults.

CDP rejection handling

Layer / File(s) Summary
Track CDP rejections
percy/snapshot.py, tests/test_snapshot_units.py
A weak set tracks hashable drivers with recognized CDP rejections. Closed-shadow capture skips rejected or non-Chrome drivers. Tests cover casing, shared rejection handling, and unhashable drivers.
Use the resize fallback
percy/snapshot.py, requirements.txt, tests/test_snapshot_units.py
Responsive resizing uses CDP for eligible Chrome drivers. Recognized CDP failures are recorded before the existing set_window_size fallback. The Selenium constraint now allows version 4 or later. Tests cover recognized and unrelated CDP failures.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Suggested reviewers: ninadbstack

Merge Risk: ⚪ Minimal · up to 5617a

No outstanding issue identified here prevents merging after normal checks.

Pre-merge checks: Passed 4
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title includes the valid Jira ID PPLT-6241 and clearly describes the address fallback and CDP detection changes.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Lowercasing browserName made drivers reporting "Chrome" switch from a real
window resize to CDP emulation, which earlier releases never did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@RaghavsBrowserStack
RaghavsBrowserStack marked this pull request as ready for review October 21, 2026 08:42
@RaghavsBrowserStack
RaghavsBrowserStack requested a review from a team as a code owner October 21, 2026 08:42

@RaghavsBrowserStack RaghavsBrowserStack left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review (automated) — 2 inline finding(s). Full report in the PR comment below. Verdict: Failed - see PR comment.

Comment thread requirements.txt
# across 3.8-3.10; revisit once that wedge is root-caused so CI can track
# current Selenium.
selenium>=4,<4.28
# CI/dev test env only — setup.py keeps `selenium>=3` for end users. Selenium

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] Comment misstates Selenium's Python floor: the 3.9 job now gets 4.36.0

Selenium 4.28.0–4.36.0 declare requires_python >=3.9; only 4.37.0+ needs 3.10. So the 3.9 CI job resolves 4.36.0, the version the removed comment said wedged geckodriver on the responsive test.

Suggestion: Correct the comment, and confirm Test (3.9) and Test (3.10) pass without hanging. If 3.9 wedges, use selenium>=4,<4.28; python_version<"3.10" plus selenium>=4; python_version>="3.10".

Reviewer: stack:percy-web-sdks-code-review

Comment thread percy/snapshot.py
# raise "Unrecognised command" on every call. That never recovers mid-session,
# so stop sending CDP to the driver instead of failing on each resize/snapshot.
_CDP_REJECTED_DRIVERS = WeakSet()
_CDP_REJECTION_MARKERS = ('unrecognised command', 'unknown command', 'unknown method')

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] Rejection markers are broader than a driver-level rejection

'unknown method' is generic. A single unsupported CDP method would disable CDP for the whole session, for both resize and shadow-DOM capture, because the memo is shared. Separately, under python -O Selenium's assert is stripped, so the rejection surfaces as a marker-less TypeError and the driver is never remembered (degraded, not broken).

Suggestion: Require executecdpcommand in the message, or drop 'unknown method'.

Reviewer: stack-code-reviewer, stack:percy-web-sdks-code-review

@RaghavsBrowserStack

Copy link
Copy Markdown
Author

Claude Code PR Review

PR: #277 • Head: 5617a95 • Reviewers: stack:percy-web-sdks-code-review, stack-code-reviewer, stack-percy-docs-reviewer

Summary

The CLI address now falls back to PERCY_SERVER_ADDRESS, which percy exec --port N exports. The SDK remembers drivers whose remote rejects CDP (Selenium 4.28+ "Unrecognised command executeCdpCommand") and stops sending them CDP for both the resize and closed-shadow-DOM paths. The test environment's selenium<4.28 cap is removed.

Review Table

Priority Category Check Status Notes
High Security No hardcoded secrets or credentials Pass No token or header logging; no new outbound targets
High Security Authentication/authorization checks present N/A SDK → local CLI only
High Security Input validation and sanitization N/A
High Security No IDOR — resource ownership validated N/A
High Security No SQL injection (parameterized queries) N/A
High Correctness Logic is correct, handles edge cases Pass Env precedence is correct (an empty PERCY_CLI_API falls through). The inner re-raise is caught by the outer except Exception and falls back to set_window_size (verified at snapshot.py:931).
High Correctness Error handling is explicit, no swallowed exceptions Pass Warns and continues as the contract requires. Unhashable or weakref-less drivers are handled.
High Correctness No race conditions or concurrency issues Pass Module-level WeakSet does not keep drivers alive
Medium Testing New code has corresponding tests Pass Env precedence, the rejection memo, sharing between the shadow-DOM and resize paths, non-rejection errors, unhashable drivers
Medium Testing Error paths and edge cases tested Pass
Medium Testing Existing tests still pass (no regressions) Fail Test (3.9) and Test (3.10) were still pending at review time. 3.9 now resolves Selenium 4.36.0, the version the removed comment said hung CI. See findings.
Medium Performance No N+1 queries or unbounded data fetching N/A
Medium Performance Long-running tasks use background jobs N/A
Medium Quality Follows existing codebase patterns Pass
Medium Quality Changes are focused (single concern) Pass The CI dependency bump is bundled in, but it is related
Low Quality Meaningful names, no dead code Pass
Low Quality Comments explain why, not what Fail requirements.txt comment is factually wrong, and the shadow-DOM guard comment is stale
Low Quality No unnecessary dependencies added Pass weakref is stdlib

Findings

  • File: .claude/knowledge/ → harness stacks/stack-domain-percy-web-sdks/knowledge/ (flow-snapshot-selenium.md, FEATURE-FLAGS.md, learnings.md)

  • Severity: High

  • Reviewer: stack-percy-docs-reviewer

  • Issue: This is a material contract change and the knowledge base doesn't reflect it. The new behaviour is the CLI address resolution order PERCY_CLI_API → PERCY_SERVER_ADDRESS → default, and the CDP rejection memo with its case rules (shadow DOM matches Chrome case-insensitively; resize keeps the exact 'chrome' match).

    • FEATURE-FLAGS.md lists neither PERCY_SERVER_ADDRESS nor PERCY_CLI_API.
    • learnings.md:10 wrongly says the Python port doesn't do CDP resize.
    • The CDP section of flow-snapshot-selenium.md covers JS only.

    The reviewer could not open the KB PR itself, for two reasons. Writes to the shared checkout were blocked in this background session. And roll-kb-pr.sh defaults to a non-existent stacks/stack-domain-percy-selenium-python/ folder instead of the shared stack-domain-percy-web-sdks stack. The code itself has no defect here.

  • Suggestion: Update the three files in stacks/stack-domain-percy-web-sdks/knowledge/, either by hand or by re-running the docs reviewer with PERCY_HARNESS_STACK_DIR=stacks/stack-domain-percy-web-sdks from a writable checkout. Add a drift item to audit the Ruby, Java and .NET ports for the same PERCY_SERVER_ADDRESS gap. Separately, fix roll-kb-pr.sh stack-dir detection for repos that share a multi-repo stack.

  • File: requirements.txt:1

  • Severity: Medium

  • Reviewer: stack:percy-web-sdks-code-review (corroborated by stack-code-reviewer)

  • Issue: The new comment says Selenium 4.28+ needs Python ≥ 3.10, so the 3.8 and 3.9 jobs resolve 4.27.x. That's wrong. PyPI requires_python is >=3.9 for 4.28.0–4.36.0 and >=3.10 only from 4.37.0, so the 3.9 job now installs 4.36.0. That is exactly the version the removed comment said wedged geckodriver on the responsive test. The wait_for_resize=False and settle changes may have fixed that, but this PR doesn't show it, and Test (3.9) and Test (3.10) were still pending at review time.

  • Suggestion: Correct the comment (4.28 needs ≥ 3.9, 4.37 needs ≥ 3.10). Confirm Test (3.9) and Test (3.10) go green without hanging before merge. If 3.9 wedges, add an environment marker: selenium>=4,<4.28; python_version<"3.10" / selenium>=4; python_version>="3.10".

  • File: percy/snapshot.py:53

  • Severity: Low

  • Reviewer: stack-code-reviewer

  • Issue: _CDP_REJECTION_MARKERS includes the generic 'unknown method'. A CDP-capable driver that rejects one method would then disable CDP for the whole session, and because the memo is shared, for both resize and shadow-DOM capture.

  • Suggestion: Match only driver-level rejection, e.g. require executecdpcommand in the message, or drop 'unknown method'.

  • File: percy/snapshot.py:511

  • Severity: Low

  • Reviewer: stack:percy-web-sdks-code-review

  • Issue: The comment still says the shadow-DOM guard mirrors the resize guard "(browserName == 'chrome')", but the two now deliberately differ: _is_chrome is case-insensitive here, while resize keeps the exact match.

  • Suggestion: Reword it to state the intentional asymmetry.

  • File: percy/snapshot.py:53

  • Severity: Low

  • Reviewer: stack:percy-web-sdks-code-review

  • Issue: Detection depends on Selenium's assert command_info is not None, "Unrecognised command …". Under python -O the assert is stripped, the error becomes a TypeError with no marker, and the memo never records the driver. Behaviour degrades to the existing per-call fallback but does not break.

  • Suggestion: Optional: accept it and note it in the comment, or handle that TypeError shape too.

  • File: README.md

  • Severity: Low

  • Reviewer: stack:percy-web-sdks-code-review

  • Issue: The README doesn't document PERCY_SERVER_ADDRESS or PERCY_CLI_API.

  • Suggestion: Optional: add them to the README's env section.

Checked and not a defect: one reviewer asked whether the inner raise in change_window_dimension_and_wait escapes. It doesn't: the outer except Exception catches it and falls back to set_window_size (snapshot.py:931–938).

Cross-SDK follow-up (non-gating): check the other non-JS ports (selenium-java/ruby/.NET, playwright-python/java/.NET, capybara). Any that read only PERCY_CLI_API will miss percy exec --port N.


Verdict: FAIL — the only High is the unremediated knowledge-base gap. The code change is sound. Fix the KB entries, then confirm the Python 3.9 and 3.10 CI jobs pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant