docs: tighten browser challenge reporting rules

This commit is contained in:
lda
2026-06-15 23:44:47 +07:00 Verified
parent 6836c9c972
commit 32da68e05f
@@ -49,6 +49,18 @@ alias.
Do not use a pre-existing generated patch or raw-plan answer file. If you find
one, ignore it and author your own workflow definition.
These do not satisfy the challenge, even if they produce the right output:
- importing `WorkflowApi`, `WorkflowServer`, or source functions directly;
- writing a Python script that calls internal APIs to create artifacts,
deployments, or runs;
- calling the browser/source functions directly instead of running a deployed
workflow;
- solving it as a standalone Playwright/Python script with no `wf artifact`,
`wf deploy`, and `wf run` lifecycle;
- reusing artifacts, deployments, stores, workflow files, or run outputs created
by earlier trials.
The repository already includes a deterministic source example at:
```text
@@ -95,9 +107,36 @@ challenge_report:
notes: "short explanation"
```
Set `used_product_path` to true only if the workflow was applied and run through
the `wf` CLI, either in local same-process mode or through `wf-rpc-server`. Set
`used_helper_script` to true if you created a Python script to drive
`WorkflowApi` directly.
Reporting rules:
- The YAML block is a self-report only. It will be manually audited against your
commands, files, and run output.
- Set `used_product_path: true` only if you used `uv run wf ...` commands for
the artifact/deployment/run lifecycle, either in local same-process mode or
through `wf-rpc-server`.
- Set `used_helper_script: true` if you wrote or ran any script whose main
purpose was to drive the workflow API, JSON-RPC API, server internals, source
functions, or browser automation outside `wf`.
- Set `read.product_code: true` if you or a spawned subagent grepped, searched,
or read source files under `src/`, `tests/`, or implementation examples to
determine plan shape or product behavior.
- Set `read.docs: true` if you read files under `docs/`.
- Set `read.skills: true` if you read files under `skills/`.
- Set `read.adjacent_attempts: true` if you read files under other trial
workspaces, prior result files, generated reports, or previous attempt
artifacts.
- Set `read.prior_store: true` if you inspected or reused `.wf_*` stores, saved
artifacts, deployments, or runs from outside your current trial workspace.
- Set `read.existing_solution: true` if you copied or inspected a ready-made
solution plan/workflow for this same challenge.
- `attempts.total` should count distinct product-lifecycle attempts, including
failed artifact creation, failed deployment validation, failed run starts, and
abandoned workflow plans.
- `attempts.failed` should count attempts that failed validation, failed to run,
produced wrong output, or were abandoned.
Spawned subagents count as you. If a subagent reads product code, set
`read.product_code: true`. If a subagent reads prior attempts, set
`read.adjacent_attempts: true`.
If something fails, report the exact command and error instead of hiding it.