docs: plan draft explain and invalid intermediate authoring
This commit is contained in:
@@ -80,6 +80,14 @@ clear operator feedback before adding more architecture.
|
||||
capabilities without mutating or saving the draft workspace.
|
||||
- Completed: draft validation now preserves structured core validation issues
|
||||
and adds exact `wf draft bind` repair hints for missing state fields.
|
||||
- Planned: expand `wf explain` to cover draft/workflow validation codes such as
|
||||
`unknown_edge_destination`, `invalid_source_path`, and `patch_invalid`.
|
||||
Implementation plan:
|
||||
[`draft explain diagnostics`](superpowers/plans/2026-06-28-explain-draft-diagnostics.md).
|
||||
- Planned: let draft workspaces persist invalid intermediate route states so
|
||||
agents can add forward-routed target steps before final validation/save.
|
||||
Implementation plan:
|
||||
[`invalid intermediate draft authoring`](superpowers/plans/2026-06-28-draft-invalid-intermediate-authoring.md).
|
||||
- Keep status read-only; do not mutate registry, auth, config, or stores.
|
||||
|
||||
## Priority 2: Durable Run/Resume Hardening
|
||||
|
||||
@@ -0,0 +1,415 @@
|
||||
# Draft Invalid Intermediate Authoring Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Let draft workspaces persist patchable invalid intermediate graph states, especially forward routes to steps that will be added later, while keeping save/compile strict.
|
||||
|
||||
**Architecture:** Treat malformed JSON Patch as non-mutating, but treat valid draft edits that produce validation diagnostics as persisted revisions with `status: "invalid"`. Audit semantic helpers that currently raise before storage, then adjust only the forward-route path needed for iterative authoring. Final artifact save and compile continue to require valid drafts.
|
||||
|
||||
**Tech Stack:** Python 3.14, workflow draft workspace API, Typer CLI, pytest, Ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_api/draft_authoring.py`: allow forward-route semantic helper edits to reach draft workspace persistence instead of aborting.
|
||||
- Inspect `src/wf_artifacts/drafts/api.py`: confirm whether `patch_workflow_draft`
|
||||
returns the patched draft when validation status is invalid; change it only
|
||||
if the failing test proves the draft is lost.
|
||||
- Inspect `src/wf_artifacts/draft_workspaces/api.py`: confirm whether
|
||||
`patch_draft_workspace` stores invalid validation results with an incremented
|
||||
revision; change it only if the failing test proves the workspace is not
|
||||
persisted.
|
||||
- Modify `tests/wf_api/test_drafts_service.py`: focused API behavior.
|
||||
- Modify `tests/wf_cli/test_remote_target.py`: CLI/RPC behavior for forward route.
|
||||
- Modify `docs/wf_cli.md`, `skills/wf-cli/SKILL.md`, and `skills/wf-workflow/references/draft-workspaces.md`: explain invalid intermediate drafts.
|
||||
- Modify `docs/current_roadmap.md`: mark completion when done.
|
||||
|
||||
### Task 1: Capture The Current Forward-Route Failure
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/wf_api/test_drafts_service.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing API test**
|
||||
|
||||
Add a test that creates a draft from a simple capability, adds `wait` with a
|
||||
route to missing `collect`, and asserts the edit persists as invalid:
|
||||
|
||||
```python
|
||||
async def test_add_step_persists_invalid_forward_route(api_with_browser_specs) -> None:
|
||||
await api_with_browser_specs.create_draft_workspace_from_capability(
|
||||
workspace_id="browser",
|
||||
capability_name="local.browser_click.open_click_page",
|
||||
name="browser",
|
||||
)
|
||||
|
||||
result = await api_with_browser_specs.add_step_from_capability(
|
||||
workspace_id="browser",
|
||||
revision=1,
|
||||
step_id="wait",
|
||||
capability_name="local.browser_click.wait_for_click",
|
||||
route_from_step="call",
|
||||
routes={"ok": "collect"},
|
||||
input_map={
|
||||
"state.session_id": "session_id",
|
||||
"input.simulate": "simulate",
|
||||
"input.timeout_seconds": "timeout_seconds",
|
||||
},
|
||||
bind_outputs={"after": "state.after"},
|
||||
)
|
||||
|
||||
assert result["revision"] == 2
|
||||
assert result["status"] == "invalid"
|
||||
assert any(
|
||||
item["code"] == "unknown_edge_destination"
|
||||
for item in result["diagnostics"]
|
||||
)
|
||||
|
||||
stored = await api_with_browser_specs.get_draft_workspace(
|
||||
workspace_id="browser",
|
||||
include_draft=True,
|
||||
)
|
||||
assert stored["draft"]["steps"]["wait"]["use"] == "local.browser_click.wait_for_click"
|
||||
assert stored["draft"]["routes"]["wait"]["ok"] == "collect"
|
||||
```
|
||||
|
||||
Use the existing draft authoring API fixture/helper in
|
||||
`tests/wf_api/test_drafts_service.py`. If the file does not already provide
|
||||
browser-click specs, add a local helper that registers these three `NodeSpec`
|
||||
objects with the same names and outcomes used by the example source:
|
||||
|
||||
```python
|
||||
local.browser_click.open_click_page -> outcome ok
|
||||
local.browser_click.wait_for_click -> outcome ok
|
||||
local.browser_click.collect_snapshots -> outcome ok
|
||||
```
|
||||
|
||||
The helper should define only the schemas needed by this test:
|
||||
|
||||
```python
|
||||
open_click_page.output_schema.properties.before
|
||||
open_click_page.output_schema.properties.session_id
|
||||
wait_for_click.input_schema.properties.session_id
|
||||
wait_for_click.input_schema.properties.simulate
|
||||
wait_for_click.input_schema.properties.timeout_seconds
|
||||
wait_for_click.output_schema.properties.after
|
||||
collect_snapshots.input_schema.properties.session_id
|
||||
collect_snapshots.input_schema.properties.before
|
||||
collect_snapshots.input_schema.properties.after
|
||||
collect_snapshots.output_schema.properties.before
|
||||
collect_snapshots.output_schema.properties.after
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run and verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_drafts_service.py::test_add_step_persists_invalid_forward_route -q
|
||||
```
|
||||
|
||||
Expected: failure showing the edit does not persist or the helper raises.
|
||||
|
||||
- [ ] **Step 3: Record exact failure mode in a code comment**
|
||||
|
||||
If the failure is caused by a pre-persistence `ValueError` or `KeyError`, add
|
||||
one short comment in the implementation seam in the next task explaining why
|
||||
draft helpers must allow invalid persisted revisions.
|
||||
|
||||
### Task 2: Persist Valid Edits With Invalid Validation Status
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_api/draft_authoring.py`
|
||||
- Inspect/modify: `src/wf_artifacts/drafts/api.py`
|
||||
- Inspect/modify: `src/wf_artifacts/draft_workspaces/api.py`
|
||||
|
||||
- [ ] **Step 1: Identify where the edit is lost**
|
||||
|
||||
Run the failing test with verbose output:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_drafts_service.py::test_add_step_persists_invalid_forward_route -q -vv
|
||||
```
|
||||
|
||||
Expected: one concrete failure mode:
|
||||
|
||||
- helper raises before `patch_draft_workspace`, or
|
||||
- `patch_workflow_draft` returns no `draft`, or
|
||||
- workspace replacement rejects the invalid state.
|
||||
|
||||
- [ ] **Step 2: Fix only the observed loss point**
|
||||
|
||||
If `draft_authoring.py` raises before patching because a route target does not
|
||||
exist, remove that preflight for target existence and let validation report the
|
||||
diagnostic after the patch.
|
||||
|
||||
If `patch_workflow_draft` loses the patched draft on `KeyError`, change it so a
|
||||
valid patched `dict` is returned alongside validation diagnostics:
|
||||
|
||||
```python
|
||||
result = validate_workflow_draft(patched, node_defs=effective_node_defs)
|
||||
if result["status"] == "valid":
|
||||
patched = WorkflowDraft.model_validate(patched).model_dump(mode="json")
|
||||
return {"draft": patched, **result}
|
||||
```
|
||||
|
||||
Do not change the existing malformed JSON Patch branch:
|
||||
|
||||
```python
|
||||
if "draft" not in patched:
|
||||
# malformed JSON Patch remains non-mutating
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the focused API test**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_drafts_service.py::test_add_step_persists_invalid_forward_route -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```powershell
|
||||
git add src/wf_api/draft_authoring.py src/wf_artifacts/drafts/api.py src/wf_artifacts/draft_workspaces/api.py tests/wf_api/test_drafts_service.py
|
||||
git commit -m "fix: persist invalid draft forward routes"
|
||||
```
|
||||
|
||||
### Task 3: Prove Invalid Drafts Stay Strict At Boundaries
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/wf_api/test_drafts_service.py`
|
||||
- Modify: `tests/wf_cli/test_remote_target.py`
|
||||
|
||||
- [ ] **Step 1: Add API boundary assertions**
|
||||
|
||||
Extend the API test or add a second test:
|
||||
|
||||
```python
|
||||
async def test_invalid_forward_route_cannot_compile_or_save(api_with_browser_specs) -> None:
|
||||
await api_with_browser_specs.create_draft_workspace_from_capability(
|
||||
workspace_id="browser",
|
||||
capability_name="local.browser_click.open_click_page",
|
||||
name="browser",
|
||||
)
|
||||
await api_with_browser_specs.add_step_from_capability(
|
||||
workspace_id="browser",
|
||||
revision=1,
|
||||
step_id="wait",
|
||||
capability_name="local.browser_click.wait_for_click",
|
||||
route_from_step="call",
|
||||
routes={"ok": "collect"},
|
||||
input_map={"state.session_id": "session_id"},
|
||||
bind_outputs={"after": "state.after"},
|
||||
)
|
||||
|
||||
compiled = await api_with_browser_specs.compile_draft_workspace(
|
||||
workspace_id="browser"
|
||||
)
|
||||
|
||||
assert compiled["status"] == "invalid"
|
||||
assert any(
|
||||
item["code"] == "unknown_edge_destination"
|
||||
for item in compiled["diagnostics"]
|
||||
)
|
||||
```
|
||||
|
||||
If the API exposes `save_draft_workspace`, assert it returns invalid diagnostics
|
||||
instead of saving an artifact. Follow existing tests for the exact method name
|
||||
and assertion shape.
|
||||
|
||||
- [ ] **Step 2: Add CLI/RPC smoke coverage**
|
||||
|
||||
In `tests/wf_cli/test_remote_target.py`, add a command route test that invokes:
|
||||
|
||||
```python
|
||||
[
|
||||
"--url",
|
||||
rpc_url,
|
||||
"draft",
|
||||
"add-step",
|
||||
"browser",
|
||||
"--revision",
|
||||
"1",
|
||||
"--step",
|
||||
"wait",
|
||||
"--capability",
|
||||
"local.browser_click.wait_for_click",
|
||||
"--from-step",
|
||||
"call",
|
||||
"--route",
|
||||
"ok=collect",
|
||||
"--input",
|
||||
"state.session_id=session_id",
|
||||
"--bind-output",
|
||||
"after=state.after",
|
||||
]
|
||||
```
|
||||
|
||||
Assert:
|
||||
|
||||
```python
|
||||
assert result.exit_code == 0
|
||||
payload = json.loads(result.output)
|
||||
assert payload["status"] == "invalid"
|
||||
assert payload["revision"] == 2
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_drafts_service.py tests/wf_cli/test_remote_target.py -q
|
||||
```
|
||||
|
||||
Expected: PASS, except unrelated pre-existing failures must be documented in
|
||||
the implementation report if they occur.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```powershell
|
||||
git add tests/wf_api/test_drafts_service.py tests/wf_cli/test_remote_target.py
|
||||
git commit -m "test: cover invalid intermediate draft routes"
|
||||
```
|
||||
|
||||
### Task 4: Prove The Follow-Up Repair Path
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/wf_api/test_drafts_service.py`
|
||||
- Modify: `skills/wf-workflow/references/draft-workspaces.md`
|
||||
|
||||
- [ ] **Step 1: Add a repair-path test**
|
||||
|
||||
Add a test that:
|
||||
|
||||
1. persists `wait -> collect` while `collect` is missing,
|
||||
2. adds `collect` in the next revision, and
|
||||
3. validates the workspace as valid.
|
||||
|
||||
Use the exact current API helpers:
|
||||
|
||||
```python
|
||||
await api.add_step_from_capability(...)
|
||||
await api.add_step_from_capability(
|
||||
workspace_id="browser",
|
||||
revision=2,
|
||||
step_id="collect",
|
||||
capability_name="local.browser_click.collect_snapshots",
|
||||
route_from_step="wait",
|
||||
input_map={
|
||||
"state.session_id": "session_id",
|
||||
"state.before": "before",
|
||||
"state.after": "after",
|
||||
},
|
||||
bind_outputs={"before": "state.final_before", "after": "state.final_after"},
|
||||
)
|
||||
validated = await api.validate_draft_workspace(workspace_id="browser")
|
||||
assert validated["status"] == "valid"
|
||||
```
|
||||
|
||||
Use the same fixture/helper from Task 1. Do not read example solution files or
|
||||
copy plan JSON from `examples/browser_click_workflow`; this test is about the
|
||||
draft authoring API contract.
|
||||
|
||||
- [ ] **Step 2: Run and verify PASS**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_drafts_service.py::test_forward_route_becomes_valid_after_target_step_is_added -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 3: Document the sequence**
|
||||
|
||||
In `skills/wf-workflow/references/draft-workspaces.md`, add:
|
||||
|
||||
```markdown
|
||||
Forward routes in drafts are allowed as invalid intermediate state. If
|
||||
`wf draft add-step --route ok=collect` returns `status: invalid`, add the
|
||||
missing `collect` step next, then run `wf draft validate`. Do not save or
|
||||
compile until validation is valid.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```powershell
|
||||
git add tests/wf_api/test_drafts_service.py skills/wf-workflow/references/draft-workspaces.md
|
||||
git commit -m "docs: explain invalid intermediate draft routes"
|
||||
```
|
||||
|
||||
### Task 5: Final Docs And Verification
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/wf_cli.md`
|
||||
- Modify: `skills/wf-cli/SKILL.md`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
- Move: `docs/superpowers/plans/2026-06-28-draft-invalid-intermediate-authoring.md`
|
||||
|
||||
- [ ] **Step 1: Update docs**
|
||||
|
||||
In `docs/wf_cli.md`, under draft workspace commands, add:
|
||||
|
||||
```markdown
|
||||
Draft commands may return `status: invalid` after persisting an edit. That is
|
||||
normal for intermediate authoring. Repair diagnostics, run `wf draft validate`,
|
||||
then save/compile only after the workspace is valid.
|
||||
```
|
||||
|
||||
In `skills/wf-cli/SKILL.md`, add:
|
||||
|
||||
```markdown
|
||||
`status: invalid` from a draft edit is not always a command failure. Inspect
|
||||
diagnostics and continue repairing the same workspace unless the command
|
||||
reports a conflict or malformed patch.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update roadmap**
|
||||
|
||||
Add a completed bullet under Priority 1:
|
||||
|
||||
```markdown
|
||||
- Completed: draft workspaces can persist invalid intermediate route states,
|
||||
allowing agents to add missing target steps before final validation/save.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Archive this plan**
|
||||
|
||||
Move this file to:
|
||||
|
||||
```text
|
||||
docs/historical/superpowers/plans/2026-06-28-draft-invalid-intermediate-authoring.md
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run final verification**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_drafts_service.py tests/wf_cli/test_remote_target.py tests/docs -q
|
||||
uv run ruff check src/wf_api/draft_authoring.py src/wf_artifacts/drafts/api.py tests/wf_api/test_drafts_service.py tests/wf_cli/test_remote_target.py
|
||||
uv run ruff format --check src/wf_api/draft_authoring.py src/wf_artifacts/drafts/api.py tests/wf_api/test_drafts_service.py tests/wf_cli/test_remote_target.py
|
||||
uv run basedpyright --level error src/wf_api/draft_authoring.py src/wf_artifacts/drafts/api.py tests/wf_api/test_drafts_service.py tests/wf_cli/test_remote_target.py
|
||||
```
|
||||
|
||||
Expected: all new/focused tests pass. Any unrelated broad-suite failures must
|
||||
be reported with file/test names and reason.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```powershell
|
||||
git add docs/wf_cli.md skills/wf-cli/SKILL.md docs/current_roadmap.md docs/historical/superpowers/plans/2026-06-28-draft-invalid-intermediate-authoring.md
|
||||
git commit -m "docs: archive invalid draft route plan"
|
||||
```
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: forward-route persistence, strict save/compile boundaries,
|
||||
diagnostics, docs, and repair path are covered.
|
||||
- Placeholder scan: no placeholders remain.
|
||||
- Type consistency: task names use current `wf draft add-step`, `handle`,
|
||||
`branch`, `compile`, and `validate` vocabulary.
|
||||
@@ -0,0 +1,490 @@
|
||||
# Explain Draft Diagnostics Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Extend `wf explain` with draft/workflow validation codes using real enum/constant sources instead of copied strings.
|
||||
|
||||
**Architecture:** Keep `wf explain` exact-match and docs-backed. Add enum-backed explain entries for `wf_core.validation.issues.ValidationIssueCode` values, introduce constants for draft/store-only codes, and update tests/docs so agents can explain draft failures without reading source/tests.
|
||||
|
||||
**Tech Stack:** Python 3.14, Typer CLI, Pydantic models, pytest, Ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_cli/explain/entries.py`: import enum/constants and add cards.
|
||||
- Modify `src/wf_artifacts/drafts/api.py`: expose draft diagnostic code constants.
|
||||
- Modify `src/wf_artifacts/draft_workspaces/api.py`: expose workspace diagnostic code constants.
|
||||
- Modify `tests/wf_cli/test_explain.py`: cover new cards, enum-backed codes, and input-file extraction.
|
||||
- Modify `docs/wf_cli.md`: expand common diagnostics.
|
||||
- Modify `skills/wf-cli/SKILL.md` and `skills/wf-workflow/references/troubleshooting.md`: teach `wf explain` for draft codes.
|
||||
- Modify `docs/current_roadmap.md`: mark completion when done.
|
||||
|
||||
### Task 1: Add Constants For Non-Enum Draft Codes
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_artifacts/drafts/api.py`
|
||||
- Modify: `src/wf_artifacts/draft_workspaces/api.py`
|
||||
- Test: `tests/wf_cli/test_explain.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing import test**
|
||||
|
||||
Add this test to `tests/wf_cli/test_explain.py`:
|
||||
|
||||
```python
|
||||
def test_explain_registry_uses_exported_draft_codes() -> None:
|
||||
from wf_artifacts.draft_workspaces.api import REVISION_CONFLICT_CODE
|
||||
from wf_artifacts.drafts.api import (
|
||||
DRAFT_INVALID_CODE,
|
||||
PATCH_INVALID_CODE,
|
||||
UNKNOWN_OUTCOME_CODE,
|
||||
)
|
||||
|
||||
registry_codes = {
|
||||
entry.code for entry in DEFAULT_EXPLAIN_REGISTRY.list_full_entries()
|
||||
}
|
||||
|
||||
assert DRAFT_INVALID_CODE in registry_codes
|
||||
assert PATCH_INVALID_CODE in registry_codes
|
||||
assert UNKNOWN_OUTCOME_CODE in registry_codes
|
||||
assert REVISION_CONFLICT_CODE in registry_codes
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the test and verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_cli/test_explain.py::test_explain_registry_uses_exported_draft_codes -q
|
||||
```
|
||||
|
||||
Expected: import failure for the new constants.
|
||||
|
||||
- [ ] **Step 3: Add exported constants near producers**
|
||||
|
||||
In `src/wf_artifacts/drafts/api.py`, near the type aliases, add:
|
||||
|
||||
```python
|
||||
DRAFT_INVALID_CODE = "draft_invalid"
|
||||
PATCH_INVALID_CODE = "patch_invalid"
|
||||
DRAFT_NOT_OBJECT_CODE = "draft_not_object"
|
||||
UNKNOWN_OUTCOME_CODE = "unknown_outcome"
|
||||
```
|
||||
|
||||
Replace the matching string literals in this file:
|
||||
|
||||
```python
|
||||
code=PATCH_INVALID_CODE
|
||||
code=DRAFT_NOT_OBJECT_CODE
|
||||
code=DRAFT_INVALID_CODE
|
||||
code=UNKNOWN_OUTCOME_CODE
|
||||
```
|
||||
|
||||
In `src/wf_artifacts/draft_workspaces/api.py`, near the type aliases, add:
|
||||
|
||||
```python
|
||||
WORKSPACE_EXISTS_CODE = "workspace_exists"
|
||||
REVISION_CONFLICT_CODE = "revision_conflict"
|
||||
```
|
||||
|
||||
Replace the matching string literals in this file:
|
||||
|
||||
```python
|
||||
code=WORKSPACE_EXISTS_CODE
|
||||
code=REVISION_CONFLICT_CODE
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the focused test and verify it now reaches registry failure**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_cli/test_explain.py::test_explain_registry_uses_exported_draft_codes -q
|
||||
```
|
||||
|
||||
Expected: failure because the registry does not yet contain the new codes.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```powershell
|
||||
git add src/wf_artifacts/drafts/api.py src/wf_artifacts/draft_workspaces/api.py tests/wf_cli/test_explain.py
|
||||
git commit -m "refactor: name draft diagnostic codes"
|
||||
```
|
||||
|
||||
### Task 2: Add Enum-Backed Explain Cards
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_cli/explain/entries.py`
|
||||
- Modify: `tests/wf_cli/test_explain.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for workflow validation cards**
|
||||
|
||||
Add this test to `tests/wf_cli/test_explain.py`:
|
||||
|
||||
```python
|
||||
def test_explain_registry_covers_core_validation_codes() -> None:
|
||||
from wf_core.validation.issues import ValidationIssueCode
|
||||
|
||||
expected = {
|
||||
ValidationIssueCode.INVALID_SOURCE_PATH.value,
|
||||
ValidationIssueCode.INVALID_DESTINATION_PATH.value,
|
||||
ValidationIssueCode.UNKNOWN_EDGE_DESTINATION.value,
|
||||
ValidationIssueCode.UNDECLARED_EDGE_OUTCOME.value,
|
||||
ValidationIssueCode.MISSING_OUTCOME_EDGE.value,
|
||||
}
|
||||
registry_codes = {
|
||||
entry.code for entry in DEFAULT_EXPLAIN_REGISTRY.list_full_entries()
|
||||
}
|
||||
|
||||
assert expected <= registry_codes
|
||||
```
|
||||
|
||||
Add this behavior test:
|
||||
|
||||
```python
|
||||
def test_explain_unknown_edge_destination_mentions_forward_route_repair() -> None:
|
||||
card = DEFAULT_EXPLAIN_REGISTRY.get("unknown_edge_destination")
|
||||
|
||||
text = "\n".join(card.how_to_fix)
|
||||
|
||||
assert "wf draft handle" in text
|
||||
assert "wf draft branch" in text
|
||||
assert "target step first" in text.lower()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests and verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_cli/test_explain.py::test_explain_registry_covers_core_validation_codes tests/wf_cli/test_explain.py::test_explain_unknown_edge_destination_mentions_forward_route_repair -q
|
||||
```
|
||||
|
||||
Expected: unknown code / missing registry entries.
|
||||
|
||||
- [ ] **Step 3: Import real code sources**
|
||||
|
||||
At the top of `src/wf_cli/explain/entries.py`, add:
|
||||
|
||||
```python
|
||||
from wf_artifacts.draft_workspaces.api import REVISION_CONFLICT_CODE
|
||||
from wf_artifacts.drafts.api import (
|
||||
DRAFT_INVALID_CODE,
|
||||
PATCH_INVALID_CODE,
|
||||
UNKNOWN_OUTCOME_CODE,
|
||||
)
|
||||
from wf_core.validation.issues import ValidationIssueCode
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add the new cards**
|
||||
|
||||
Append these `ExplainCard` entries to `EXPLAIN_CARDS`:
|
||||
|
||||
```python
|
||||
ExplainCard(
|
||||
code=ValidationIssueCode.INVALID_SOURCE_PATH.value,
|
||||
summary="A workflow step reads from a path that is not declared or available.",
|
||||
why_it_happens=[
|
||||
"A step input binding points at input/state/context data that the draft schema does not declare.",
|
||||
"A literal placeholder or guessed path was used in a binding.",
|
||||
"A wrapper bootstrap included a field that is not present in the actual run input.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Run `wf schema InputPathBinding` to confirm binding shape.",
|
||||
"Inspect the draft input and state schemas.",
|
||||
"Use `wf draft set-input --merge` to repair step input bindings.",
|
||||
"Patch the draft input_schema/state_schema when the workflow genuinely needs a new path.",
|
||||
"Run `wf draft validate <workspace_id>` after the edit.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli.md#draft-workspaces",
|
||||
"docs/workflow_drafts.md",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code=ValidationIssueCode.INVALID_DESTINATION_PATH.value,
|
||||
summary="A workflow step writes to a state or output path that is not declared.",
|
||||
why_it_happens=[
|
||||
"A capability output is bound to a missing state_schema field.",
|
||||
"A workflow output projection points at a missing output_schema field.",
|
||||
"A draft patch changed output bindings without changing the matching schema.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"For capability output to state, prefer `wf draft bind --from local.FIELD --to state.FIELD`.",
|
||||
"For multiple output bindings, use `wf draft set-output --merge` when preserving existing mappings.",
|
||||
"Read any `repair_hint` returned by `wf draft validate` before writing JSON Patch.",
|
||||
"Run `wf draft validate <workspace_id>` after the edit.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli.md#draft-workspaces",
|
||||
"docs/workflow_drafts.md",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code=ValidationIssueCode.UNKNOWN_EDGE_DESTINATION.value,
|
||||
summary="A route or edge points at a step id that does not exist in the workflow.",
|
||||
why_it_happens=[
|
||||
"A draft route was added before the target step was created.",
|
||||
"A step id was misspelled in a route or edge.",
|
||||
"A raw plan edge references a node id that is absent from `nodes`.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"In draft authoring, create the target step first, then route to it.",
|
||||
"Use `wf draft handle <workspace_id> --step FROM --outcome OUTCOME --to TARGET` to repair one route.",
|
||||
"Use `wf draft branch <workspace_id> --step FROM --route OUTCOME=TARGET` for multiple route edits.",
|
||||
"For a complete graph authored at once, prefer `wf artifact create-from-plan` and validate the raw plan shape.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli.md#draft-workspaces",
|
||||
"docs/workflow_drafts.md",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code=ValidationIssueCode.UNDECLARED_EDGE_OUTCOME.value,
|
||||
summary="A route uses an outcome that the source step does not declare.",
|
||||
why_it_happens=[
|
||||
"The route outcome was guessed instead of read from capability metadata.",
|
||||
"A multi-outcome capability was wired with an incomplete or misspelled outcome map.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Run `wf cap inspect <capability>` and read the declared outcomes.",
|
||||
"Use `wf draft handle` or `wf draft branch` with the exact outcome names.",
|
||||
"Run `wf draft validate <workspace_id>` after route edits.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli.md#draft-workspaces",
|
||||
"docs/workflow_capabilities.md",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code=ValidationIssueCode.MISSING_OUTCOME_EDGE.value,
|
||||
summary="A step outcome has no route and the workflow cannot prove where execution goes next.",
|
||||
why_it_happens=[
|
||||
"A multi-outcome step was added without complete route coverage.",
|
||||
"A draft patch replaced a route map and dropped an existing outcome.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Run `wf cap inspect <capability>` to list declared outcomes.",
|
||||
"Use `wf draft branch --route OUTCOME=TARGET` for each missing outcome.",
|
||||
"Route terminal outcomes to `__end__` when the workflow should finish.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli.md#draft-workspaces",
|
||||
"docs/workflow_drafts.md",
|
||||
],
|
||||
),
|
||||
```
|
||||
|
||||
Also append these draft/workspace cards:
|
||||
|
||||
```python
|
||||
ExplainCard(
|
||||
code=UNKNOWN_OUTCOME_CODE,
|
||||
summary="A draft route uses an outcome that the source step cannot produce.",
|
||||
why_it_happens=[
|
||||
"The route outcome was guessed instead of read from the capability contract.",
|
||||
"A draft patch preserved an old outcome after the step capability changed.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Run `wf cap inspect <capability>` and read the declared outcomes.",
|
||||
"Use `wf draft handle <workspace_id> --step STEP --outcome OUTCOME --to TARGET` with a declared outcome.",
|
||||
"Use `wf draft branch <workspace_id> --step STEP --route OUTCOME=TARGET` when repairing multiple outcomes.",
|
||||
"Run `wf draft validate <workspace_id>` after route edits.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli.md#draft-workspaces",
|
||||
"docs/workflow_drafts.md",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code=DRAFT_INVALID_CODE,
|
||||
summary="A draft workspace contains an invalid draft shape or invalid workflow structure.",
|
||||
why_it_happens=[
|
||||
"The payload mixed draft-workspace shape with raw-plan shape.",
|
||||
"A JSON Patch produced a draft that does not satisfy the draft model.",
|
||||
"The draft model is syntactically valid but workflow validation found structural issues.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Run `wf schema draft` for draft workspace payloads.",
|
||||
"Run `wf schema raw` for `wf artifact create-from-plan` payloads.",
|
||||
"Run `wf draft validate <workspace_id>` and follow each diagnostic code.",
|
||||
"Use `wf explain <code>` for the nested diagnostics before patching again.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli.md#draft-workspaces",
|
||||
"docs/workflow_drafts.md",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code=PATCH_INVALID_CODE,
|
||||
summary="A draft patch is not a valid RFC 6902 JSON Patch or cannot be applied.",
|
||||
why_it_happens=[
|
||||
"The patch file used raw draft JSON instead of a JSON Patch operation list.",
|
||||
"A patch path points at a missing parent object.",
|
||||
"A patch operation is malformed or unsupported by the patch library.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Use focused commands such as `wf draft set-input`, `wf draft set-output`, `wf draft bind`, `wf draft handle`, and `wf draft branch` when possible.",
|
||||
"If using `wf draft patch`, make the file a JSON array of RFC 6902 operations.",
|
||||
"Run `wf schema draft` to inspect the draft shape before choosing patch paths.",
|
||||
"Retry with the current workspace revision.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli.md#draft-workspaces",
|
||||
"docs/workflow_drafts.md",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code=REVISION_CONFLICT_CODE,
|
||||
summary="A draft command used a stale workspace revision.",
|
||||
why_it_happens=[
|
||||
"Another edit advanced the draft workspace revision.",
|
||||
"The command was retried with an old `--revision` value.",
|
||||
"An agent copied a prior command transcript without fetching the current workspace.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Run `wf draft inspect <workspace_id>` to get the current revision.",
|
||||
"Repeat the edit with the current `--revision` value.",
|
||||
"Do not skip revision checks; they prevent overwriting another edit.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli.md#draft-workspaces",
|
||||
"docs/workflow_drafts.md",
|
||||
],
|
||||
),
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run explain tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_cli/test_explain.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```powershell
|
||||
git add src/wf_cli/explain/entries.py tests/wf_cli/test_explain.py
|
||||
git commit -m "feat: explain draft validation codes"
|
||||
```
|
||||
|
||||
### Task 3: Update User-Facing Docs And Skills
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/wf_cli.md`
|
||||
- Modify: `skills/wf-cli/SKILL.md`
|
||||
- Modify: `skills/wf-workflow/references/troubleshooting.md`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Update docs**
|
||||
|
||||
In `docs/wf_cli.md`, under `## Explain`, add a short list of draft examples:
|
||||
|
||||
```markdown
|
||||
Draft authoring diagnostics commonly include:
|
||||
|
||||
- `invalid_source_path`
|
||||
- `invalid_destination_path`
|
||||
- `unknown_edge_destination`
|
||||
- `unknown_outcome`
|
||||
- `patch_invalid`
|
||||
- `revision_conflict`
|
||||
```
|
||||
|
||||
In `skills/wf-cli/SKILL.md`, add:
|
||||
|
||||
```markdown
|
||||
For draft validation errors, run `wf explain <code>`. If routes point to a
|
||||
missing step, create the target step first or repair routes with
|
||||
`wf draft handle` / `wf draft branch`.
|
||||
```
|
||||
|
||||
In `skills/wf-workflow/references/troubleshooting.md`, add:
|
||||
|
||||
```markdown
|
||||
`unknown_edge_destination`: a route points to a missing step. Add the target
|
||||
step or repair the route; do not guess `draft step add` or `draft export`.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update roadmap**
|
||||
|
||||
Add a completed bullet under Priority 1:
|
||||
|
||||
```markdown
|
||||
- Completed: `wf explain` now covers draft/workflow validation codes such as
|
||||
`unknown_edge_destination`, `invalid_source_path`, and `patch_invalid`.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Verify docs and skills**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/docs tests/wf_cli/test_explain.py -q
|
||||
uv run ruff check src/wf_cli/explain/entries.py tests/wf_cli/test_explain.py
|
||||
uv run basedpyright --level error src/wf_cli/explain/entries.py tests/wf_cli/test_explain.py
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```powershell
|
||||
git add docs/wf_cli.md skills/wf-cli/SKILL.md skills/wf-workflow/references/troubleshooting.md docs/current_roadmap.md
|
||||
git commit -m "docs: teach draft explain codes"
|
||||
```
|
||||
|
||||
### Task 4: Final Verification
|
||||
|
||||
**Files:**
|
||||
- Move: `docs/superpowers/plans/2026-06-28-explain-draft-diagnostics.md`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Smoke the CLI**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run wf explain unknown_edge_destination --format compact
|
||||
uv run wf explain invalid_destination_path --format markdown
|
||||
uv run wf explain --list --format compact
|
||||
```
|
||||
|
||||
Expected: each command exits 0 and prints useful text.
|
||||
|
||||
- [ ] **Step 2: Archive the plan**
|
||||
|
||||
Move this plan to:
|
||||
|
||||
```text
|
||||
docs/historical/superpowers/plans/2026-06-28-explain-draft-diagnostics.md
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run final checks**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_cli/test_explain.py tests/docs -q
|
||||
uv run ruff check src/wf_cli/explain/entries.py tests/wf_cli/test_explain.py
|
||||
uv run ruff format --check src/wf_cli/explain/entries.py tests/wf_cli/test_explain.py
|
||||
uv run basedpyright --level error src/wf_cli/explain/entries.py tests/wf_cli/test_explain.py
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```powershell
|
||||
git add docs/current_roadmap.md docs/historical/superpowers/plans/2026-06-28-explain-draft-diagnostics.md
|
||||
git commit -m "docs: archive draft explain plan"
|
||||
```
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: all acceptance criteria from `2026-06-28-explain-draft-diagnostics.md` map to Tasks 1-4.
|
||||
- Placeholder scan: no placeholders remain.
|
||||
- Type consistency: enum-backed codes come from `ValidationIssueCode`; draft-only codes come from exported constants.
|
||||
@@ -0,0 +1,76 @@
|
||||
# Draft Invalid Intermediate Authoring Design
|
||||
|
||||
## Status
|
||||
|
||||
Planned.
|
||||
|
||||
## Problem
|
||||
|
||||
Draft workspaces are supposed to support iterative authoring, but some semantic
|
||||
helpers behave too much like final validation gates. In challenge runs, agents
|
||||
tried:
|
||||
|
||||
```powershell
|
||||
wf draft add-step browser_click `
|
||||
--revision 1 `
|
||||
--step wait `
|
||||
--capability local.browser_click.wait_for_click `
|
||||
--from-step call `
|
||||
--route ok=collect
|
||||
```
|
||||
|
||||
The intent is reasonable: add `wait`, route it to `collect`, then add `collect`.
|
||||
Today this can fail the whole command because the route points at a missing
|
||||
step. The agent loses the useful partial edit and has to rediscover the order:
|
||||
add steps first, then route later.
|
||||
|
||||
## Design
|
||||
|
||||
Draft workspace mutation should persist structurally patchable intermediate
|
||||
states even when validation reports draft-level errors. The workspace may become
|
||||
`status: "invalid"` with diagnostics, but the revision should still advance when
|
||||
the patch itself was valid and the resulting document can still be stored.
|
||||
|
||||
Strictness remains at boundaries:
|
||||
|
||||
- `wf draft save` must reject invalid drafts.
|
||||
- `wf draft compile` must reject invalid drafts.
|
||||
- `wf artifact create-from-plan` remains strict for raw plans.
|
||||
- malformed JSON Patch remains non-mutating and does not burn a revision.
|
||||
|
||||
This is a draft authoring behavior change, not a runtime behavior change.
|
||||
|
||||
## First Target
|
||||
|
||||
The first target is forward routes:
|
||||
|
||||
```text
|
||||
routes.<step>.<outcome> = "missing_step"
|
||||
```
|
||||
|
||||
Expected behavior:
|
||||
|
||||
- The edit is stored.
|
||||
- The workspace revision increments.
|
||||
- The workspace status becomes `invalid`.
|
||||
- Diagnostics include `unknown_edge_destination`.
|
||||
- The user can add the missing step in a later revision.
|
||||
- `wf draft validate` becomes valid after all missing route destinations exist.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not allow invalid drafts to be saved as artifacts.
|
||||
- Do not hide diagnostics or downgrade errors to success.
|
||||
- Do not add a `wf draft step ...` namespace in this slice.
|
||||
- Do not make `wf explain` fuzzy or command-discovery oriented.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- A focused test proves `wf draft add-step --route ok=collect` persists an
|
||||
invalid workspace when `collect` does not exist yet.
|
||||
- A follow-up edit that adds `collect` can make the same workspace valid.
|
||||
- `wf draft save` and `wf draft compile` continue to reject the invalid
|
||||
intermediate workspace before the missing step is added.
|
||||
- Diagnostics remain machine-readable and include the original code.
|
||||
- Docs/skills explain the intended sequence: add step, route with
|
||||
`wf draft handle` / `wf draft branch`, validate, then save.
|
||||
@@ -0,0 +1,92 @@
|
||||
# Explain Draft Diagnostics Design
|
||||
|
||||
## Status
|
||||
|
||||
Planned.
|
||||
|
||||
## Problem
|
||||
|
||||
`wf explain` currently covers deployment/source diagnostics but not the draft
|
||||
and workflow-core validation codes that agents now hit while authoring
|
||||
multi-step workflows. The missing explanations force agents to infer behavior
|
||||
from large schemas, source files, or tests.
|
||||
|
||||
The recent challenge runs exposed these recurring failures:
|
||||
|
||||
- `unknown_edge_destination`: agents route to a step that does not exist yet,
|
||||
for example `--route ok=collect`.
|
||||
- `invalid_source_path`: agents bind from `input.foo` or `state.foo` before
|
||||
that path exists in the draft schema/state.
|
||||
- `invalid_destination_path`: agents write capability output into undeclared
|
||||
state/output paths.
|
||||
- `draft_invalid` and `patch_invalid`: agents mix raw-plan shape, draft shape,
|
||||
and RFC 6902 patch shape.
|
||||
- `revision_conflict`: iterative draft commands use stale revision numbers.
|
||||
|
||||
## Design
|
||||
|
||||
Extend the docs-backed explain registry with draft/workflow validation cards.
|
||||
Where a code already exists in a real enum, use that enum instead of a bare
|
||||
string:
|
||||
|
||||
```python
|
||||
from wf_core.validation.issues import ValidationIssueCode
|
||||
|
||||
ValidationIssueCode.INVALID_SOURCE_PATH.value
|
||||
ValidationIssueCode.UNKNOWN_EDGE_DESTINATION.value
|
||||
```
|
||||
|
||||
For draft-store codes that are not enum-backed yet, introduce small constants
|
||||
near the producer before importing them into `wf_cli.explain.entries`.
|
||||
|
||||
The registry remains exact-match and docs-backed. It must not become fuzzy
|
||||
search or command discovery. If a command is unknown, that remains a CLI help
|
||||
problem unless the CLI emits a stable error code.
|
||||
|
||||
## Initial Code Set
|
||||
|
||||
Add explain cards for:
|
||||
|
||||
- `invalid_source_path`
|
||||
- `invalid_destination_path`
|
||||
- `unknown_edge_destination`
|
||||
- `undeclared_edge_outcome`
|
||||
- `missing_outcome_edge`
|
||||
- `unknown_outcome`
|
||||
- `draft_invalid`
|
||||
- `patch_invalid`
|
||||
- `revision_conflict`
|
||||
|
||||
## Required Guidance
|
||||
|
||||
`unknown_edge_destination` must explicitly mention draft authoring:
|
||||
|
||||
```text
|
||||
In a draft workspace, add the target step first, then route to it with
|
||||
wf draft handle or wf draft branch. If you are importing a complete graph,
|
||||
use wf artifact create-from-plan instead.
|
||||
```
|
||||
|
||||
`invalid_destination_path` must mention the focused helper:
|
||||
|
||||
```text
|
||||
For capability output to state, prefer:
|
||||
wf draft bind --from local.FIELD --to state.FIELD
|
||||
```
|
||||
|
||||
`draft_invalid` must distinguish draft shape from raw plan shape:
|
||||
|
||||
```text
|
||||
Use wf schema draft for draft workspaces and wf schema raw for
|
||||
artifact create-from-plan payloads.
|
||||
```
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- `wf explain --list` includes the new draft/workflow codes.
|
||||
- `wf explain unknown_edge_destination --format markdown` tells agents not to
|
||||
forward-route to missing steps in one `add-step` call.
|
||||
- Explain entries for `ValidationIssueCode` values import the enum, not copied
|
||||
string literals.
|
||||
- Existing explain parser behavior is unchanged.
|
||||
- Related docs links point to live docs, not `docs/superpowers/**` plans/specs.
|
||||
Reference in New Issue
Block a user