feat: replace Playwright MCP container with pre-installed @playwright/cli in agent image

[Copilot]

The Playwright MCP approach runs a separate Docker container, requiring host.docker.internal routing hacks, extra image pull overhead, and large MCP tool schemas that consume model context. @playwright/cli eliminates all of this by running directly inside the agent container.

Agent container

• Pre-installs @playwright/cli globally via npm
• Installs Chromium headless shell (--only-shell) with system deps via playwright-cli install-browser chromium --with-deps --only-shell
• Cleans npm cache and temp files post-install to minimize layer size

Smoke workflows (smoke-claude.md, smoke-codex.md)

• Remove tools: playwright: and network: allowed: playwright — no MCP container needed
• Replace Playwright test step with playwright-cli bash commands:

playwright-cli open https://github.com
playwright-cli eval "document.title"
playwright-cli close

• Lock files recompiled; mcr.microsoft.com/playwright/mcp removed from manifests

Documentation

Updated docs-site/playwright-testing.md with a new Agentic Workflows: Use @playwright/cli section documenting the CLI approach and a comparison table vs. the MCP container.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• https://api.github.com/repos/actions/github-script/git/ref/tags/v9
  • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv (http block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[github-actions]

Documentation Preview

Documentation build failed for this PR. View logs.

Built from commit 011c10f

[github-actions]

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	85.72%	85.81%	📈 +0.09%
Statements	85.67%	85.75%	📈 +0.08%
Functions	88.31%	88.31%	➡️ +0.00%
Branches	78.70%	78.74%	📈 +0.04%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
src/docker-manager.ts	87.3% → 87.6% (+0.29%)	87.1% → 87.4% (+0.28%)

---

Coverage comparison generated by scripts/ci/compare-coverage.ts

[Copilot]

Pull request overview

This PR migrates Playwright smoke testing from a separate Playwright MCP Docker container to a CLI-first approach by pre-installing @playwright/cli in the agent image, reducing networking complexity and MCP tool/schema overhead.

Changes:

• Pre-install @playwright/cli and Chromium headless shell in the agent container.
• Update smoke workflow markdown (and compiled lock workflows) to remove the Playwright MCP server/tooling and instead use playwright-cli via bash.
• Extend Playwright testing documentation to describe the CLI approach and compare it to the MCP container.

Show a summary per file

File	Description
containers/agent/Dockerfile	Installs @playwright/cli + Chromium shell during image build.
docs-site/src/content/docs/guides/playwright-testing.md	Adds CLI vs MCP guidance and example playwright-cli usage.
.github/workflows/smoke-claude.md	Removes Playwright tool/network allowlist and switches to playwright-cli instructions.
.github/workflows/smoke-codex.md	Removes Playwright tool/network allowlist and switches to playwright-cli instructions.
.github/workflows/smoke-claude.lock.yml	Compiled workflow updates removing Playwright MCP config/tooling.
.github/workflows/smoke-codex.lock.yml	Compiled workflow updates removing Playwright MCP config/tooling and image manifest entry.
.github/workflows/test-coverage-reporter.lock.yml	Compiled workflow updates (notably local AWF build/install + session-state collection changes).
.github/workflows/refactoring-scanner.lock.yml	Compiled workflow updates (local AWF build/install + session-state collection changes).
.github/workflows/export-audit.lock.yml	Compiled workflow updates (local AWF build/install + session-state collection changes).
.github/workflows/duplicate-code-detector.lock.yml	Compiled workflow updates (local AWF build/install + session-state collection changes).

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

containers/agent/Dockerfile:186

• Installing @playwright/cli without a version pin makes the agent image build non-reproducible and can introduce breakages from upstream changes (including transitive deps and postinstall behavior). Consider pinning to a known-good version (and bump it intentionally) to keep image builds deterministic.

RUN npm install -g @playwright/cli && \

docs-site/src/content/docs/guides/playwright-testing.md:41

• The example playwright-cli open https://myapp.localhost:3000 will resolve to loopback inside the container (per *.localhost semantics), not to the host machine’s dev server. For host-served apps, the docs should point to http://host.docker.internal:3000 (and ensure the workflow allows localhost/host ports), or otherwise clarify the networking expectations.

# Interact with the page
playwright-cli open https://myapp.localhost:3000
playwright-cli click "Get Started"

• Files reviewed: 10/10 changed files
• Comments generated: 2

[github-actions]

Smoke Test Results

✅ GitHub MCP: Last 2 merged PRs (#2465, #2439)
✅ Playwright: Page title verified (contains "GitHub")
✅ File Writing: Test file created successfully
✅ Bash Tools: File read-back verified

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude

[github-actions]

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.13	Python 3.12.3	❌ NO
Node.js	v24.14.1	v20.20.2	❌ NO
Go	go1.22.12	go1.22.12	✅ YES

Result: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot.

Tested by Smoke Chroot

[github-actions]

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	❌	N/A	❌ FAIL
Bun	hono	❌	N/A	❌ FAIL
C++	fmt	❌	N/A	❌ FAIL
C++	json	❌	N/A	❌ FAIL
Deno	oak	❌	N/A	❌ FAIL
Deno	std	❌	N/A	❌ FAIL
.NET	hello-world	❌	N/A	❌ FAIL
.NET	json-parse	❌	N/A	❌ FAIL
Go	color	❌	N/A	❌ FAIL
Go	env	❌	N/A	❌ FAIL
Go	uuid	❌	N/A	❌ FAIL
Java	gson	❌	N/A	❌ FAIL
Java	caffeine	❌	N/A	❌ FAIL
Node.js	clsx	❌	N/A	❌ FAIL
Node.js	execa	❌	N/A	❌ FAIL
Node.js	p-limit	❌	N/A	❌ FAIL
Rust	fd	❌	N/A	❌ FAIL
Rust	zoxide	❌	N/A	❌ FAIL

Overall: 0/8 ecosystems passed — ❌ FAIL

❌ Error Details

ALL_CLONES_FAILED: All repository clones failed because the gh CLI is not authenticated (GH_TOKEN environment variable is not set in this workflow run). Every clone attempt for all 8 ecosystems failed with:

gh: To use GitHub CLI in a GitHub Actions workflow, set the GH_TOKEN environment variable.

No tests could be executed. Please ensure GH_TOKEN is available in the workflow environment for gh repo clone to work.

Generated by Build Test Suite for issue #2466 · ● 143.8K · ◷

[github-actions]

Smoke Test: Services Connectivity

Check	Result
Redis PING	❌ timeout (no response)
PostgreSQL pg_isready	❌ no response
PostgreSQL SELECT 1	❌ skipped (pg unreachable)

Overall: FAIL — host.docker.internal services are not reachable from this runner environment.

🔌 Service connectivity validated by Smoke Services

[github-actions]

🔍 Smoke Test Results

Test	Status
GitHub MCP connectivity	✅
GitHub.com HTTP connectivity	❌ (pre-step data not available)
File write/read	❌ (pre-step data not available)

Overall: FAIL — workflow template variables (smoke-data outputs) were not substituted; HTTP and file tests could not be verified.

PR: feat: replace Playwright MCP container with pre-installed @playwright/cli in agent image
Author: @Copilot | Assignees: @lpcox, @Copilot

📰 BREAKING: Report filed by Smoke Copilot

[github-actions]

🔥 Smoke Test: Copilot BYOK (run 25601421718)

Test	Result
GitHub MCP (list PRs)	✅
GitHub.com HTTP connectivity	⚠️ template var unresolved
File write/read (smoke-test-copilot-byok-25601421718.txt)	✅
BYOK inference (api-proxy → api.githubcopilot.com)	✅

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com.

Overall: PASS (1 test skipped due to unresolved workflow template variable)

PR by @Copilot · Assignees: @lpcox @Copilot

🔑 BYOK report filed by Smoke Copilot BYOK

[github-actions]

Smoke Test Codex

• PR: Add first-class ARC/DinD compatibility mode for AWF
• PR: Fix chroot bind mounts for ARC/DinD split runner-daemon filesystems
• GitHub MCP/safeinputs-gh: ❌ unavailable
• Playwright/file/bash/build: ✅
• Tavily/discussion: ❌ unavailable
• Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex