Opencode Browser Click Challenge Harness

This harness runs agent trials against the browser-click workflow challenge. It is evidence tooling, not product runtime code.

The deterministic workflow example is:

examples/browser_click_workflow/

Default Behavior

By default the harness starts:

uv run wf-rpc-server --config examples/browser_click_workflow/wf.config.json --host 127.0.0.1 --port 8772

It waits until:

uv run wf --url http://127.0.0.1:8772/rpc status

passes, injects that URL into the prompt, runs opencode, then stops the server. Use --server-url to target an already-running server or --no-start-server to skip lifecycle management.

One Trial

From the repository root:

uv run python examples/agent_challenges/browser_click_challenge/run_opencode_trials.py `
  --model opencode/mimo-v2.5-free `
  --variant high `
  --trials 1

Results are written to:

examples/agent_challenges/browser_click_challenge/results/

Optional Opencode Server Attachment

--attach is opencode's server attach flag. It connects this non-interactive run to an already-running opencode server, for example:

--attach http://127.0.0.1:4096

It is not a direct MCP server URL. If that opencode server is configured with Playwright MCP tools, then the attached run can use those tools through opencode. One possible MCP server command for such an opencode setup is:

{
  "command": "npx",
  "args": ["-y", "@playwright/mcp@latest"]
}

The baseline challenge does not require Playwright MCP. The score is based on whether the agent used the workflow product path and produced the expected workflow output.

Required Agent Report

The prompt asks the agent to end with one fenced YAML block:

challenge_report:
  used_product_path: true
  used_helper_script: false
  workflow_file: "path/to/workflow.json-or-yaml"
  deployment_id: "browser_click_case_study.default"
  run_id: "run_..."
  before_clicked: false
  after_clicked: true
  run_failed: false
  leftover_processes: false
  notes: "short explanation"

The harness parses this report first. If the report is missing, it falls back to best-effort prose classification.

Classification

Each trial is classified as one of:

success: output shows workflow usage and before/after clicked states.
workflow_script: output shows a workflow run, but the agent drove it through a new helper script instead of the product-facing CLI/server path.
workflow_not_used: output appears to solve the task without wf, wf-rpc-server, deployment, or run evidence.
run_failed: output includes workflow usage but reports a failure.
timeout: the opencode process exceeded the configured timeout.
parse_error: the harness could not read opencode JSON/JSONL output.
unknown: no clear success or failure signal was found.

Committed tests cover harness logic only. They do not invoke opencode.