docs: tighten browser challenge reporting rules
This commit is contained in:
@@ -49,6 +49,18 @@ alias.
|
||||
Do not use a pre-existing generated patch or raw-plan answer file. If you find
|
||||
one, ignore it and author your own workflow definition.
|
||||
|
||||
These do not satisfy the challenge, even if they produce the right output:
|
||||
|
||||
- importing `WorkflowApi`, `WorkflowServer`, or source functions directly;
|
||||
- writing a Python script that calls internal APIs to create artifacts,
|
||||
deployments, or runs;
|
||||
- calling the browser/source functions directly instead of running a deployed
|
||||
workflow;
|
||||
- solving it as a standalone Playwright/Python script with no `wf artifact`,
|
||||
`wf deploy`, and `wf run` lifecycle;
|
||||
- reusing artifacts, deployments, stores, workflow files, or run outputs created
|
||||
by earlier trials.
|
||||
|
||||
The repository already includes a deterministic source example at:
|
||||
|
||||
```text
|
||||
@@ -95,9 +107,36 @@ challenge_report:
|
||||
notes: "short explanation"
|
||||
```
|
||||
|
||||
Set `used_product_path` to true only if the workflow was applied and run through
|
||||
the `wf` CLI, either in local same-process mode or through `wf-rpc-server`. Set
|
||||
`used_helper_script` to true if you created a Python script to drive
|
||||
`WorkflowApi` directly.
|
||||
Reporting rules:
|
||||
|
||||
- The YAML block is a self-report only. It will be manually audited against your
|
||||
commands, files, and run output.
|
||||
- Set `used_product_path: true` only if you used `uv run wf ...` commands for
|
||||
the artifact/deployment/run lifecycle, either in local same-process mode or
|
||||
through `wf-rpc-server`.
|
||||
- Set `used_helper_script: true` if you wrote or ran any script whose main
|
||||
purpose was to drive the workflow API, JSON-RPC API, server internals, source
|
||||
functions, or browser automation outside `wf`.
|
||||
- Set `read.product_code: true` if you or a spawned subagent grepped, searched,
|
||||
or read source files under `src/`, `tests/`, or implementation examples to
|
||||
determine plan shape or product behavior.
|
||||
- Set `read.docs: true` if you read files under `docs/`.
|
||||
- Set `read.skills: true` if you read files under `skills/`.
|
||||
- Set `read.adjacent_attempts: true` if you read files under other trial
|
||||
workspaces, prior result files, generated reports, or previous attempt
|
||||
artifacts.
|
||||
- Set `read.prior_store: true` if you inspected or reused `.wf_*` stores, saved
|
||||
artifacts, deployments, or runs from outside your current trial workspace.
|
||||
- Set `read.existing_solution: true` if you copied or inspected a ready-made
|
||||
solution plan/workflow for this same challenge.
|
||||
- `attempts.total` should count distinct product-lifecycle attempts, including
|
||||
failed artifact creation, failed deployment validation, failed run starts, and
|
||||
abandoned workflow plans.
|
||||
- `attempts.failed` should count attempts that failed validation, failed to run,
|
||||
produced wrong output, or were abandoned.
|
||||
|
||||
Spawned subagents count as you. If a subagent reads product code, set
|
||||
`read.product_code: true`. If a subagent reads prior attempts, set
|
||||
`read.adjacent_attempts: true`.
|
||||
|
||||
If something fails, report the exact command and error instead of hiding it.
|
||||
|
||||
Reference in New Issue
Block a user