docs: archive completed superpowers plans

This commit is contained in:
lda
2026-06-04 22:29:32 +07:00 Unverified
parent e001b524d6
commit 0d34174a84
93 changed files with 88 additions and 10 deletions
@@ -0,0 +1,797 @@
# Capability Sources Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace the ad hoc broker/proxy/admin/stdlib split with a source registry that can project workflow node specs, MCP tools, prompts, and resources to the right surfaces.
**Architecture:** Add `CapabilitySource` as the source registry model, keep `SpecSource` behavior as a compatibility projection while the rest of the system migrates, then move stdlib and admin capabilities into explicit `wf.std`, `wf.mcp`, and `wf.admin` sources. Broker/proxy servers should project capabilities from sources instead of defining separate duplicate admin tools.
**Tech Stack:** Python 3.14, dataclasses, Pydantic models already used by node specs, FastMCP/MCP server decorators, pytest, ruff, basedpyright.
---
## File Structure
- Create `src/wf_mcp/broker/service/capability_sources.py`: canonical source model, visibility flags, permission flags, capability buckets, projection helpers.
- Modify `src/wf_mcp/broker/service/sources.py`: either delegate to the new model or become a small compatibility import.
- Modify `src/wf_mcp/broker/service/core.py`: store `capability_sources`, derive node-spec resolution from them, keep `spec_sources` and `specs_by_connection` as compatibility views if needed.
- Modify `src/wf_mcp/broker/service/builtins.py`: register `wf.std` stdlib specs and `wf.mcp` runtime specs through capability sources.
- Modify `src/wf_mcp/broker/tools.py`: project broker MCP tools from `wf.admin` source definitions.
- Create `src/wf_mcp/broker/admin_capabilities.py`: one reusable definition of broker/admin tool capabilities.
- Modify `src/wf_mcp/transparent_proxy/admin.py` and `src/wf_mcp/transparent_proxy/runtime.py`: project proxy admin tools from the same `wf.admin` capability definitions when admin MCP exposure is enabled.
- Modify `src/wf_mcp/shared/names.py`: move admin namespace toward `wf.admin` and use `LdaNamespace` where dotted names should be preserved.
- Modify `tests/wf_mcp/test_service.py`, `tests/wf_mcp/test_broker_server.py`, and `tests/wf_mcp/test_transparent_proxy.py`: prove projections and defaults.
- Modify `docs/wf_mcp_capability_sources.md`: update once implementation names are final.
---
### Task 1: Add Canonical Capability Source Model
**Files:**
- Create: `src/wf_mcp/broker/service/capability_sources.py`
- Modify: `src/wf_mcp/broker/service/sources.py`
- Test: `tests/wf_mcp/test_service.py`
- [ ] **Step 1: Write failing tests for source visibility and buckets**
Add tests in `tests/wf_mcp/test_service.py`:
```python
def test_service_sources_have_visibility_and_capability_buckets() -> None:
service = WfMcpService(store=FileStore(local_temp_root() / "source_shape_store"))
std_source = service.capability_sources["wf.std"]
mcp_source = service.capability_sources["wf.mcp"]
assert std_source.id == "wf.std"
assert std_source.kind == "system"
assert std_source.visibility.planner is True
assert std_source.visibility.mcp_client is True
assert std_source.visibility.admin_dashboard is True
assert "wf.std.runtime_error" in std_source.capabilities.node_specs
assert std_source.capabilities.tools == {}
assert mcp_source.id == "wf.mcp"
assert mcp_source.visibility.planner is True
assert mcp_source.permissions.calls_upstream is True
assert "wf.mcp.call_tool" in mcp_source.capabilities.node_specs
```
- [ ] **Step 2: Run the focused failing test**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_service.py::test_service_sources_have_visibility_and_capability_buckets -q
```
Expected: fail because `capability_sources` does not exist.
- [ ] **Step 3: Add the source model**
Create `src/wf_mcp/broker/service/capability_sources.py`:
```python
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Literal
from wf_authoring import NodeSpec
SourceKind = Literal["system", "connection"]
@dataclass(frozen=True, slots=True)
class SourceVisibility:
planner: bool = False
mcp_client: bool = False
admin_dashboard: bool = True
@dataclass(frozen=True, slots=True)
class SourcePermissions:
safe_for_workflow: bool = False
calls_upstream: bool = False
mutates_config: bool = False
mutates_auth: bool = False
@dataclass(slots=True)
class CapabilityBuckets:
tools: dict[str, Any] = field(default_factory=dict)
node_specs: dict[str, NodeSpec[Any, Any]] = field(default_factory=dict)
prompts: dict[str, Any] = field(default_factory=dict)
resources: dict[str, Any] = field(default_factory=dict)
@dataclass(slots=True)
class CapabilitySource:
id: str
kind: SourceKind
capabilities: CapabilityBuckets = field(default_factory=CapabilityBuckets)
enabled: bool = True
visibility: SourceVisibility = field(default_factory=SourceVisibility)
permissions: SourcePermissions = field(default_factory=SourcePermissions)
description: str | None = None
def as_status(self) -> dict[str, Any]:
return {
"id": self.id,
"kind": self.kind,
"enabled": self.enabled,
"visibility": {
"planner": self.visibility.planner,
"mcp_client": self.visibility.mcp_client,
"admin_dashboard": self.visibility.admin_dashboard,
},
"permissions": {
"safe_for_workflow": self.permissions.safe_for_workflow,
"calls_upstream": self.permissions.calls_upstream,
"mutates_config": self.permissions.mutates_config,
"mutates_auth": self.permissions.mutates_auth,
},
"description": self.description,
"tool_count": len(self.capabilities.tools),
"node_spec_count": len(self.capabilities.node_specs),
"prompt_count": len(self.capabilities.prompts),
"resource_count": len(self.capabilities.resources),
}
```
- [ ] **Step 4: Keep `SpecSource` as compatibility wrapper**
Modify `src/wf_mcp/broker/service/sources.py` so existing code can still import `SpecSource` while new code can move to `CapabilitySource`:
```python
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
from wf_authoring import NodeSpec
from .capability_sources import (
CapabilityBuckets,
CapabilitySource,
SourceKind,
SourcePermissions,
SourceVisibility,
)
@dataclass(slots=True)
class SpecSource:
id: str
kind: SourceKind
specs: dict[str, NodeSpec[Any, Any]] = field(default_factory=dict)
visible: bool = True
description: str | None = None
def as_capability_source(self) -> CapabilitySource:
return CapabilitySource(
id=self.id,
kind=self.kind,
capabilities=CapabilityBuckets(node_specs=self.specs),
visibility=SourceVisibility(
planner=self.visible,
mcp_client=False,
admin_dashboard=True,
),
permissions=SourcePermissions(safe_for_workflow=self.kind == "system"),
description=self.description,
)
def as_status(self) -> dict[str, Any]:
return self.as_capability_source().as_status()
```
- [ ] **Step 5: Run focused test**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_service.py::test_service_sources_have_visibility_and_capability_buckets -q
```
Expected: fail until service stores `capability_sources`; pass after Task 2.
---
### Task 2: Make `WfMcpService` Store Capability Sources
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- Modify: `src/wf_mcp/broker/service/specs.py`
- Test: `tests/wf_mcp/test_service.py`
- [ ] **Step 1: Add failing service projection tests**
Add:
```python
def test_service_spec_views_are_derived_from_capability_sources() -> None:
service = WfMcpService(store=FileStore(local_temp_root() / "source_view_store"))
assert "wf.std" in service.capability_sources
assert "wf.std" in service.spec_sources
assert "wf.std" in service.specs_by_connection
assert (
service.specs_by_connection["wf.std"]["wf.std.runtime_error"]
is service.capability_sources["wf.std"].capabilities.node_specs[
"wf.std.runtime_error"
]
)
```
- [ ] **Step 2: Run focused test**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_service.py::test_service_spec_views_are_derived_from_capability_sources -q
```
Expected: fail because compatibility views are not derived from capability sources.
- [ ] **Step 3: Update service fields and registration**
In `src/wf_mcp/broker/service/core.py`, replace the stored `spec_sources` field with canonical capability storage and add derived views:
```python
capability_sources: dict[str, CapabilitySource] = field(default_factory=dict)
@property
def spec_sources(self) -> dict[str, SpecSource]:
return {
source.id: SpecSource(
id=source.id,
kind=source.kind,
specs=source.capabilities.node_specs,
visible=source.enabled and source.visibility.planner,
description=source.description,
)
for source in self.capability_sources.values()
if source.capabilities.node_specs
}
@property
def specs_by_connection(self) -> dict[str, dict[str, NodeSpec[Any, Any]]]:
return {
source.id: source.capabilities.node_specs
for source in self.capability_sources.values()
if source.capabilities.node_specs
}
def register_capability_source(self, source: CapabilitySource) -> None:
self.capability_sources[source.id] = source
def register_spec_source(self, source: SpecSource) -> None:
self.register_capability_source(source.as_capability_source())
```
- [ ] **Step 4: Update spec resolution**
In `src/wf_mcp/broker/service/specs.py`, make `get_qualified_spec` read capability sources:
```python
from collections.abc import Mapping
from .capability_sources import CapabilitySource
def get_qualified_spec(
sources: Mapping[str, CapabilitySource],
qualified_name: str,
) -> NodeSpec[Any, Any]:
source_id, _ = qualified_name.rsplit(".", 1)
source = sources.get(source_id)
if (
source is None
or not source.enabled
or qualified_name not in source.capabilities.node_specs
):
raise KeyError(f"unknown qualified node {qualified_name!r}")
return source.capabilities.node_specs[qualified_name]
```
- [ ] **Step 5: Run focused service tests**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_service.py -q
```
Expected: pass.
---
### Task 3: Move All Authoring Ops Into `wf.std`
**Files:**
- Modify: `src/wf_mcp/broker/service/builtins.py`
- Test: `tests/wf_mcp/test_service.py`
- [ ] **Step 1: Add failing `wf.std` inventory test**
Add:
```python
def test_wf_std_source_contains_authoring_ops() -> None:
service = WfMcpService(store=FileStore(local_temp_root() / "stdlib_source_store"))
specs = service.capability_sources["wf.std"].capabilities.node_specs
expected = {
"wf.std.coalesce",
"wf.std.default_if_none",
"wf.std.constant",
"wf.std.pick_key",
"wf.std.truthy",
"wf.std.runtime_error",
"wf.std.first_item",
"wf.std.first_item_or_none",
"wf.std.first_item_maybe",
"wf.std.last_item",
"wf.std.last_item_or_none",
"wf.std.length",
"wf.std.is_empty",
}
assert expected <= set(specs)
```
- [ ] **Step 2: Run focused test**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_service.py::test_wf_std_source_contains_authoring_ops -q
```
Expected: fail because only `runtime_error` is currently registered.
- [ ] **Step 3: Register stdlib ops**
In `src/wf_mcp/broker/service/builtins.py`, import `wf_authoring.ops` symbols and qualify each one under `wf.std`:
```python
from wf_authoring import (
coalesce,
constant,
default_if_none,
first_item,
first_item_maybe,
first_item_or_none,
is_empty,
last_item,
last_item_or_none,
length,
pick_key,
runtime_error,
truthy,
)
def builtin_specs() -> dict[str, NodeSpec[Any, Any]]:
specs = [
coalesce,
default_if_none,
constant,
pick_key,
truthy,
runtime_error,
first_item,
first_item_or_none,
first_item_maybe,
last_item,
last_item_or_none,
length,
is_empty,
]
qualified_specs = [
qualify_spec(BUILTIN_CONNECTION_ID, _strip_authoring_prefix(spec))
for spec in specs
]
return {spec.name: spec for spec in qualified_specs}
def _strip_authoring_prefix(spec: NodeSpec[Any, Any]) -> NodeSpec[Any, Any]:
name = spec.name.removeprefix("authoring.")
return NodeSpec(
name=name,
input_model=spec.input_model,
output_model=spec.output_model,
outcomes=spec.outcomes,
fn=spec.fn,
description=spec.description,
is_async=spec.is_async,
accepts_context=spec.accepts_context,
input_schema_contract=spec.input_schema_contract,
output_schema_contract=spec.output_schema_contract,
)
```
- [ ] **Step 4: Run service tests**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_service.py -q
```
Expected: pass.
---
### Task 4: Add `wf.admin` Source Without MCP Exposure
**Files:**
- Create: `src/wf_mcp/broker/admin_capabilities.py`
- Modify: `src/wf_mcp/broker/service/core.py`
- Test: `tests/wf_mcp/test_service.py`
- [ ] **Step 1: Add failing admin source test**
Add:
```python
def test_wf_admin_source_exists_but_is_not_planner_visible() -> None:
service = WfMcpService(store=FileStore(local_temp_root() / "admin_source_store"))
source = service.capability_sources["wf.admin"]
assert source.kind == "system"
assert source.visibility.planner is False
assert source.visibility.mcp_client is False
assert source.visibility.admin_dashboard is True
assert source.permissions.mutates_config is True
assert "wf.admin.list_sources" in source.capabilities.tools
assert "wf.admin.disable_source" in source.capabilities.tools
assert "wf.admin" not in service.get_planner_catalog().snapshots
```
- [ ] **Step 2: Run focused test**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_service.py::test_wf_admin_source_exists_but_is_not_planner_visible -q
```
Expected: fail because `wf.admin` does not exist.
- [ ] **Step 3: Define admin capability objects**
Create `src/wf_mcp/broker/admin_capabilities.py`:
```python
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Awaitable, Callable
from .service.capability_sources import (
CapabilityBuckets,
CapabilitySource,
SourcePermissions,
SourceVisibility,
)
ADMIN_SOURCE_ID = "wf.admin"
@dataclass(frozen=True, slots=True)
class AdminTool:
name: str
description: str
handler_name: str
mutates_config: bool = False
mutates_auth: bool = False
def admin_source() -> CapabilitySource:
tools: dict[str, AdminTool] = {
"wf.admin.list_sources": AdminTool(
name="wf.admin.list_sources",
description="List broker capability sources.",
handler_name="list_sources",
),
"wf.admin.disable_source": AdminTool(
name="wf.admin.disable_source",
description="Disable a capability source.",
handler_name="disable_source",
mutates_config=True,
),
"wf.admin.enable_source": AdminTool(
name="wf.admin.enable_source",
description="Enable a capability source.",
handler_name="enable_source",
mutates_config=True,
),
}
return CapabilitySource(
id=ADMIN_SOURCE_ID,
kind="system",
capabilities=CapabilityBuckets(tools=tools),
visibility=SourceVisibility(
planner=False,
mcp_client=False,
admin_dashboard=True,
),
permissions=SourcePermissions(
safe_for_workflow=False,
calls_upstream=False,
mutates_config=True,
mutates_auth=True,
),
description="Privileged broker administration capabilities.",
)
```
- [ ] **Step 4: Install admin source**
In `WfMcpService.__post_init__`, register `admin_source()` after builtin sources:
```python
from ..admin_capabilities import admin_source
...
self.register_capability_source(admin_source())
```
- [ ] **Step 5: Run focused tests**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_service.py -q
```
Expected: pass.
---
### Task 5: Project Broker Admin Tools From `wf.admin`
**Files:**
- Modify: `src/wf_mcp/broker/tools.py`
- Test: `tests/wf_mcp/test_broker_server.py`
- [ ] **Step 1: Add test proving default broker exposes current public tools but has source-backed metadata**
Add to `tests/wf_mcp/test_broker_server.py`:
```python
def test_broker_admin_tools_are_backed_by_wf_admin_source() -> None:
service = WfMcpService(store=FileStore(local_temp_root() / "broker_admin_source"))
server = create_broker_server(service)
tools = asyncio.run(server.list_tools())
tool_names = {tool.name for tool in tools}
assert "list_spec_sources" in tool_names
assert "get_planner_catalog" in tool_names
assert "wf.admin.list_sources" in service.capability_sources[
"wf.admin"
].capabilities.tools
```
- [ ] **Step 2: Run focused broker test**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_broker_server.py::test_broker_admin_tools_are_backed_by_wf_admin_source -q
```
Expected: pass after Task 4; fail before Task 4.
- [ ] **Step 3: Keep current tool names as compatibility exports**
Do not rename public broker MCP tools in this task. Keep:
```text
list_connections
get_connection_statuses
refresh_connection_catalog
get_catalog
get_planner_catalog
list_spec_sources
read_broker_resource
render_broker_prompt
invoke_broker_method
call_broker_tool
get_broker_events
```
Add comments in `src/wf_mcp/broker/tools.py`:
```python
# These MCP tool names are compatibility exports. Their capability metadata
# belongs to the wf.admin source; future admin-enabled servers can project
# dotted wf.admin.* names from that source.
```
- [ ] **Step 4: Run focused broker tests**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_broker_server.py -q
```
Expected: pass.
---
### Task 6: Normalize Transparent Proxy Admin Naming Strategy
**Files:**
- Modify: `src/wf_mcp/shared/names.py`
- Modify: `src/wf_mcp/transparent_proxy/runtime.py`
- Test: `tests/wf_mcp/test_names.py`
- Test: `tests/wf_mcp/test_transparent_proxy.py`
- [ ] **Step 1: Add naming tests for admin namespace**
Add to `tests/wf_mcp/test_names.py`:
```python
def test_admin_namespace_is_distinct_from_wf_mcp_runtime_source() -> None:
assert ADMIN_NAMESPACE == "wf.admin"
assert is_admin_tool_name("wf.admin.list_connections") is True
assert is_admin_tool_name("wf.mcp.call_tool") is False
```
- [ ] **Step 2: Run focused naming test**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_names.py::test_admin_namespace_is_distinct_from_wf_mcp_runtime_source -q
```
Expected: fail because `ADMIN_NAMESPACE` is currently `wf.mcp`.
- [ ] **Step 3: Update namespace constants**
In `src/wf_mcp/shared/names.py`:
```python
ADMIN_NAMESPACE = "wf.admin"
def is_admin_tool_name(proxy_name: str) -> bool:
return proxy_name.startswith(f"{ADMIN_NAMESPACE}.") or proxy_name.startswith(
f"{ADMIN_NAMESPACE}_"
)
```
Use `LdaNamespace(ADMIN_NAMESPACE)` in `src/wf_mcp/transparent_proxy/runtime.py`:
```python
from ..shared.names import ADMIN_NAMESPACE, LdaNamespace
...
admin.add_transform(LdaNamespace(ADMIN_NAMESPACE))
```
- [ ] **Step 4: Update transparent proxy tests**
In `tests/wf_mcp/test_transparent_proxy.py`, update expected admin names from `wf.mcp_*` to dotted `wf.admin.*` if `LdaNamespace` preserves dotted names:
```python
assert "wf.admin.list_connections" in names
assert "wf.admin.get_connection_statuses" in names
assert "wf.admin.list_proxy_tools" in names
assert "wf.admin.get_proxy_tool" in names
```
Call tools by the new names:
```python
connections_result = await client.call_tool("wf.admin.list_connections")
proxy_tools_result = await client.call_tool("wf.admin.list_proxy_tools")
```
- [ ] **Step 5: Run transparent proxy tests**
Run:
```powershell
uv run --with pytest pytest tests\wf_mcp\test_names.py tests\wf_mcp\test_transparent_proxy.py -q
```
Expected: pass if `LdaNamespace` preserves dotted names. If FastMCP still emits underscore names, keep `wf.admin_*` as compatibility and document that dotted projection needs a deeper transform.
---
### Task 7: Update Docs And Full Verification
**Files:**
- Modify: `docs/wf_mcp_capability_sources.md`
- Modify: `docs/wf_mcp_architecture.md`
- Test: full verification commands
- [ ] **Step 1: Update docs with implemented names**
In `docs/wf_mcp_capability_sources.md`, update the migration section to mark implemented pieces:
```markdown
## Implemented Shape
- `CapabilitySource` owns source metadata and capability buckets.
- `wf.std` owns workflow stdlib node specs.
- `wf.mcp` owns workflow MCP runtime node specs.
- `wf.admin` owns privileged admin capability metadata.
- Broker and proxy MCP tool projection remains compatibility-first while the
admin projection stabilizes.
```
- [ ] **Step 2: Run full pytest**
Run:
```powershell
uv run --with pytest pytest -q
```
Expected: all tests pass, live-only tests may skip when env is absent.
- [ ] **Step 3: Run ruff**
Run:
```powershell
uv run ruff check src tests examples main.py
```
Expected: `All checks passed!`
- [ ] **Step 4: Run basedpyright errors**
Run:
```powershell
uv run basedpyright src tests examples main.py --level error
```
Expected: `0 errors`.
---
## Self-Review
Spec coverage:
- Capability source model is covered by Tasks 1 and 2.
- `wf.std` migration is covered by Task 3.
- `wf.admin` privileged source is covered by Task 4.
- Broker projection is covered by Task 5.
- Transparent proxy naming and admin namespace separation is covered by Task 6.
- Documentation and verification are covered by Task 7.
Known deliberate scope limits:
- Task 5 keeps current broker MCP public tool names as compatibility exports.
- Task 6 attempts dotted transparent-proxy admin names via `LdaNamespace`; if FastMCP still forces underscore naming, this plan keeps `wf.admin_*` compatibility and defers deeper transform work.
- Source enable/disable runtime behavior is not implemented in this plan beyond introducing `enabled`; it should be the next plan after the registry shape lands.
Placeholder scan:
- No `TBD`, `TODO`, or unspecified implementation steps remain.
- Each task includes exact paths, test names, commands, and expected outcomes.
Type consistency:
- `CapabilitySource`, `CapabilityBuckets`, `SourceVisibility`, and
`SourcePermissions` are introduced once and reused consistently.
- Compatibility views are named `spec_sources` and `specs_by_connection`.
@@ -0,0 +1,436 @@
# Workflow Artifacts V1 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add the first `wf_artifacts` package with immutable workflow artifact models, deployment bindings, dependency diagnostics, and a file-backed artifact store.
**Architecture:** `wf_artifacts` is a separate layer above `wf_core` and below future platform/MCP projections. It stores declarative workflow plans and dependency metadata, but does not execute workflows or import `wf_mcp`.
**Tech Stack:** Python 3.14, Pydantic v2 models, `pathlib`, JSON file storage, pytest.
---
## File Structure
- Create `src/wf_artifacts/__init__.py`: public facade for artifact models and stores.
- Create `src/wf_artifacts/models.py`: Pydantic models for artifacts, deployments, required capabilities, diagnostics, and drift policy.
- Create `src/wf_artifacts/store.py`: `WorkflowArtifactStore` protocol-like base class and `FileWorkflowArtifactStore`.
- Create `tests/artifacts/test_models.py`: serialization and validation tests for model shapes.
- Create `tests/artifacts/test_store.py`: file-store round-trip and latest-version tests.
## Task 1: Artifact Models
**Files:**
- Create: `src/wf_artifacts/models.py`
- Create: `src/wf_artifacts/__init__.py`
- Test: `tests/artifacts/test_models.py`
- [ ] **Step 1: Write failing model tests**
```python
from wf_artifacts import (
DependencyDiagnostic,
DiagnosticSeverity,
DriftPolicy,
RequiredCapability,
WorkflowArtifact,
WorkflowDeployment,
)
def test_workflow_artifact_serializes_required_capability_contract() -> None:
capability = RequiredCapability(
logical_source="context7",
capability_name="query-docs",
kind="tool",
input_schema_hash="sha256:input",
input_schema_snapshot={"type": "object", "properties": {}},
output_schema_hash="sha256:output",
output_schema_snapshot={"type": "object", "properties": {}},
observed_concrete_source="context7.default",
observed_at_epoch_ms=123,
)
artifact = WorkflowArtifact(
id="summarize_docs",
version=1,
title="Summarize Docs",
description="Summarize retrieved documentation.",
input_schema={"type": "object", "properties": {}},
output_schema={"type": "object", "properties": {}},
outcomes=("done", "failed"),
plan={"name": "summarize_docs", "nodes": [], "edges": []},
required_capabilities={"context7.query-docs": capability},
created_from_catalog_version="catalog-1",
)
dumped = artifact.model_dump(mode="json")
assert dumped["id"] == "summarize_docs"
assert dumped["version"] == 1
assert dumped["outcomes"] == ["done", "failed"]
assert dumped["required_capabilities"]["context7.query-docs"]["logical_source"] == "context7"
assert dumped["required_capabilities"]["context7.query-docs"]["input_schema_hash"] == "sha256:input"
def test_workflow_deployment_binds_logical_sources_to_concrete_sources() -> None:
deployment = WorkflowDeployment(
id="summarize_docs.personal",
artifact_id="summarize_docs",
artifact_version=1,
bindings={"context7": "context7.personal"},
drift_policy=DriftPolicy.BLOCK,
)
dumped = deployment.model_dump(mode="json")
assert dumped["id"] == "summarize_docs.personal"
assert dumped["artifact_id"] == "summarize_docs"
assert dumped["artifact_version"] == 1
assert dumped["bindings"]["context7"] == "context7.personal"
assert dumped["drift_policy"] == "block"
def test_dependency_diagnostic_is_structured() -> None:
diagnostic = DependencyDiagnostic(
severity=DiagnosticSeverity.ERROR,
code="capability_missing",
logical_ref="context7.query-docs",
bound_source="context7.default",
message="Bound source no longer exposes query-docs.",
repair_hint="Refresh catalog or bind context7 to another compatible source.",
)
dumped = diagnostic.model_dump(mode="json")
assert dumped["severity"] == "error"
assert dumped["code"] == "capability_missing"
assert dumped["logical_ref"] == "context7.query-docs"
assert dumped["bound_source"] == "context7.default"
assert dumped["repair_hint"].startswith("Refresh catalog")
```
- [ ] **Step 2: Run model tests to verify RED**
Run: `uv run --with pytest pytest tests\artifacts\test_models.py -q`
Expected: fail with `ModuleNotFoundError: No module named 'wf_artifacts'`.
- [ ] **Step 3: Implement minimal models**
```python
from __future__ import annotations
from enum import StrEnum
from typing import Any, Literal
from pydantic import BaseModel, Field
JsonObject = dict[str, Any]
class DriftPolicy(StrEnum):
BLOCK = "block"
WARN = "warn"
ALLOW = "allow"
class DiagnosticSeverity(StrEnum):
ERROR = "error"
WARNING = "warning"
class RequiredCapability(BaseModel):
logical_source: str
capability_name: str
kind: Literal["tool", "resource", "prompt", "node_spec", "workflow"]
input_schema_hash: str | None = None
input_schema_snapshot: JsonObject | None = None
output_schema_hash: str | None = None
output_schema_snapshot: JsonObject | None = None
observed_concrete_source: str | None = None
observed_at_epoch_ms: int | None = Field(default=None, ge=0)
class DependencyDiagnostic(BaseModel):
severity: DiagnosticSeverity
code: str
logical_ref: str
bound_source: str | None = None
message: str
repair_hint: str | None = None
class WorkflowArtifact(BaseModel):
id: str
version: int = Field(ge=1)
title: str
description: str | None = None
input_schema: JsonObject
output_schema: JsonObject
outcomes: tuple[str, ...]
plan: JsonObject
required_capabilities: dict[str, RequiredCapability] = Field(default_factory=dict)
workflow_dependencies: dict[str, int] = Field(default_factory=dict)
created_from_catalog_version: str | None = None
class WorkflowDeployment(BaseModel):
id: str
artifact_id: str
artifact_version: int = Field(ge=1)
bindings: dict[str, str] = Field(default_factory=dict)
drift_policy: DriftPolicy = DriftPolicy.BLOCK
```
- [ ] **Step 4: Export models**
```python
from .models import (
DependencyDiagnostic,
DiagnosticSeverity,
DriftPolicy,
RequiredCapability,
WorkflowArtifact,
WorkflowDeployment,
)
__all__ = [
"DependencyDiagnostic",
"DiagnosticSeverity",
"DriftPolicy",
"RequiredCapability",
"WorkflowArtifact",
"WorkflowDeployment",
]
```
- [ ] **Step 5: Run model tests to verify GREEN**
Run: `uv run --with pytest pytest tests\artifacts\test_models.py -q`
Expected: pass.
## Task 2: File Artifact Store
**Files:**
- Modify: `src/wf_artifacts/store.py`
- Modify: `src/wf_artifacts/__init__.py`
- Test: `tests/artifacts/test_store.py`
- [ ] **Step 1: Write failing store tests**
```python
from wf_artifacts import (
FileWorkflowArtifactStore,
WorkflowArtifact,
WorkflowDeployment,
)
def artifact(version: int) -> WorkflowArtifact:
return WorkflowArtifact(
id="summarize_docs",
version=version,
title=f"Summarize Docs v{version}",
description=None,
input_schema={"type": "object", "properties": {}},
output_schema={"type": "object", "properties": {}},
outcomes=("done",),
plan={"name": "summarize_docs", "nodes": [], "edges": []},
)
def test_file_store_round_trips_artifact_versions(tmp_path) -> None:
store = FileWorkflowArtifactStore(tmp_path)
store.save_artifact(artifact(1))
store.save_artifact(artifact(2))
loaded = store.get_artifact("summarize_docs", 2)
assert loaded.id == "summarize_docs"
assert loaded.version == 2
assert loaded.title == "Summarize Docs v2"
def test_file_store_resolves_latest_artifact_version(tmp_path) -> None:
store = FileWorkflowArtifactStore(tmp_path)
store.save_artifact(artifact(1))
store.save_artifact(artifact(3))
store.save_artifact(artifact(2))
latest = store.resolve_latest("summarize_docs")
assert latest.id == "summarize_docs"
assert latest.version == 3
def test_file_store_round_trips_deployment(tmp_path) -> None:
store = FileWorkflowArtifactStore(tmp_path)
deployment = WorkflowDeployment(
id="summarize_docs.personal",
artifact_id="summarize_docs",
artifact_version=1,
bindings={"context7": "context7.personal"},
)
store.save_deployment(deployment)
loaded = store.get_deployment("summarize_docs.personal")
assert loaded.id == "summarize_docs.personal"
assert loaded.artifact_id == "summarize_docs"
assert loaded.bindings["context7"] == "context7.personal"
```
- [ ] **Step 2: Run store tests to verify RED**
Run: `uv run --with pytest pytest tests\artifacts\test_store.py -q`
Expected: fail importing `FileWorkflowArtifactStore`.
- [ ] **Step 3: Implement file store**
```python
from __future__ import annotations
import json
from pathlib import Path
from .models import WorkflowArtifact, WorkflowDeployment
class WorkflowArtifactStore:
def save_artifact(self, artifact: WorkflowArtifact) -> None:
raise NotImplementedError
def get_artifact(self, artifact_id: str, version: int) -> WorkflowArtifact:
raise NotImplementedError
def list_artifacts(self) -> list[WorkflowArtifact]:
raise NotImplementedError
def resolve_latest(self, artifact_id: str) -> WorkflowArtifact:
raise NotImplementedError
def save_deployment(self, deployment: WorkflowDeployment) -> None:
raise NotImplementedError
def get_deployment(self, deployment_id: str) -> WorkflowDeployment:
raise NotImplementedError
class FileWorkflowArtifactStore(WorkflowArtifactStore):
def __init__(self, root: Path) -> None:
self.root = root
self.artifacts_dir.mkdir(parents=True, exist_ok=True)
self.deployments_dir.mkdir(parents=True, exist_ok=True)
@property
def artifacts_dir(self) -> Path:
return self.root / "workflows"
@property
def deployments_dir(self) -> Path:
return self.root / "deployments"
def save_artifact(self, artifact: WorkflowArtifact) -> None:
artifact_dir = self.artifacts_dir / artifact.id
artifact_dir.mkdir(parents=True, exist_ok=True)
path = artifact_dir / f"{artifact.version}.json"
path.write_text(
json.dumps(artifact.model_dump(mode="json"), indent=2),
encoding="utf-8",
)
def get_artifact(self, artifact_id: str, version: int) -> WorkflowArtifact:
path = self.artifacts_dir / artifact_id / f"{version}.json"
if not path.exists():
raise KeyError(f"unknown workflow artifact {artifact_id}@{version}")
return WorkflowArtifact.model_validate_json(path.read_text(encoding="utf-8"))
def list_artifacts(self) -> list[WorkflowArtifact]:
artifacts: list[WorkflowArtifact] = []
for path in sorted(self.artifacts_dir.glob("*/*.json")):
artifacts.append(
WorkflowArtifact.model_validate_json(path.read_text(encoding="utf-8"))
)
return artifacts
def resolve_latest(self, artifact_id: str) -> WorkflowArtifact:
versions = [
int(path.stem)
for path in (self.artifacts_dir / artifact_id).glob("*.json")
if path.stem.isdecimal()
]
if not versions:
raise KeyError(f"unknown workflow artifact {artifact_id!r}")
return self.get_artifact(artifact_id, max(versions))
def save_deployment(self, deployment: WorkflowDeployment) -> None:
path = self.deployments_dir / f"{deployment.id}.json"
path.write_text(
json.dumps(deployment.model_dump(mode="json"), indent=2),
encoding="utf-8",
)
def get_deployment(self, deployment_id: str) -> WorkflowDeployment:
path = self.deployments_dir / f"{deployment_id}.json"
if not path.exists():
raise KeyError(f"unknown workflow deployment {deployment_id!r}")
return WorkflowDeployment.model_validate_json(path.read_text(encoding="utf-8"))
```
- [ ] **Step 4: Export store types**
```python
from .store import FileWorkflowArtifactStore, WorkflowArtifactStore
__all__ = [
...
"FileWorkflowArtifactStore",
"WorkflowArtifactStore",
]
```
- [ ] **Step 5: Run store tests to verify GREEN**
Run: `uv run --with pytest pytest tests\artifacts -q`
Expected: pass.
## Task 3: Verification
**Files:**
- No production changes unless verification exposes issues.
- [ ] **Step 1: Run focused artifact tests**
Run: `uv run --with pytest pytest tests\artifacts -q`
Expected: all artifact tests pass.
- [ ] **Step 2: Run full test suite**
Run: `uv run --with pytest pytest -q`
Expected: existing tests and new artifact tests pass.
- [ ] **Step 3: Run lint/type checks**
Run: `uv run ruff check src tests examples main.py`
Expected: no lint errors.
Run: `uv run basedpyright src tests examples main.py --level error`
Expected: `0 errors`.
## Self-Review
- Spec coverage: implements the first artifact slice only: immutable models, deployments, dependency contracts, diagnostics, and file storage.
- Intentional gaps: no workflow execution, no dependency validation engine, no MCP projection, no native subgraph runtime.
- Placeholder scan: no TBD/TODO placeholders remain.
- Type consistency: tests and code use the same model names and fields.
@@ -0,0 +1,454 @@
# Unified MCP Surface And Protocol Proxy Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Converge broker mode and transparent proxy mode into one MCP server surface that supports direct upstream capability projection, stable workflow/admin tools, protocol-level proxying, and local broker notifications.
**Architecture:** Use one service/config/store layer and explicit projection layers. The unified server should expose upstream tools/resources/prompts directly when enabled, expose stable local workflow/admin tools, and route bidirectional MCP protocol features through well-named proxy components rather than ad hoc tool wrappers.
**Tech Stack:** FastMCP 3.x, MCP Python SDK, existing `WfMcpService`, transparent proxy runtime, capability source registry, pytest, basedpyright.
---
## Why This Plan Exists
The current project has two partial MCP surfaces:
- Broker mode exposes stable control/workflow tools such as
`create_workflow_artifact_from_plan`, `validate_workflow_deployment`, and
`run_workflow_deployment`.
- Transparent proxy mode exposes upstream capabilities directly through MCP
`tools/list` and `tools/call`.
That split is not enough. A real MCP proxy has more than tools:
- resources
- prompts
- tool calls
- notifications
- progress
- logging
- resource subscriptions
- elicitation
- sampling
- tasks
- capability/list-changed events
The unified server must not be a "tools-only proxy." It needs a protocol plan
that decides what is pass-through, what is projected, what is local, what is
unsupported, and what needs explicit client/server capability negotiation.
## Definitions
### Upstream Server
An MCP server configured as a connection, such as `everything.default`,
`context7.default`, or `serena.default`.
### Downstream Client
The MCP client connected to our server, such as Codex or MCP Inspector.
### Local Capability
A capability implemented by this project, such as `wf.workflow.run_deployment`
or `wf.admin.list_connections`.
### Proxy Capability
A capability discovered from an upstream server and projected through our
server, such as `everything.default_echo`.
### Protocol Proxy
Bidirectional routing for MCP protocol messages that are not ordinary
tools/resources/prompts list/read/get calls. Examples: sampling requests,
elicitation requests, progress notifications, task status notifications, and
resource update notifications.
## Target Surface
One MCP server instance should expose both proxy capabilities and local
capabilities.
```text
upstream tools:
everything.default_echo
context7.default_query-docs
upstream resources:
everything.default.instructions.md
upstream prompts:
everything.default.simple-prompt
stable workflow tools:
wf.workflow.list_artifacts
wf.workflow.create_artifact_from_plan
wf.workflow.save_deployment
wf.workflow.validate_deployment
wf.workflow.run_deployment
admin tools, only when explicitly enabled:
wf.admin.list_connections
wf.admin.refresh_connection_catalog
wf.admin.list_proxy_tools
wf.admin.reload_config
```
Compatibility broker tool names such as `get_planner_catalog` can remain during
migration, but namespaced tools should become the recommended interface.
## Protocol Coverage Matrix
| MCP Feature | Near-Term Behavior | Long-Term Behavior | Notes |
| --- | --- | --- | --- |
| `tools/list` | Project upstream tools plus local workflow/admin tools | Same, with pagination/search | Already partially supported in transparent proxy and broker separately. |
| `tools/call` | Forward upstream tool calls and execute local tools | Same, with tasks/progress support | Local tools should share handlers across modes. |
| `resources/list` | Project upstream resources | Same, plus local docs/resources | Resources-as-tools remains optional compatibility. |
| `resources/read` | Forward upstream reads | Same, with subscriptions | Needs namespacing and URI/local-name mapping. |
| `prompts/list` | Project upstream prompts | Same, plus local authoring manuals | Prompts-as-tools remains optional compatibility. |
| `prompts/get` | Forward upstream prompt rendering | Same | Must preserve arguments and metadata. |
| `notifications/progress` | Forward where the SDK/server surface supports it | First-class run/proxy progress bus | Local workflow runs should emit progress later. |
| `notifications/resources/updated` | Not reliable yet | Proxy subscriptions with lifecycle tracking | Requires subscription ownership and reload behavior. |
| `notifications/resources/list_changed` | Emit local changed events after refresh/reload if supported | Also forward upstream changes | Needed when tools/resources/prompts change. |
| `notifications/tools/list_changed` | Emit after config reload/catalog refresh if supported | Same | Important for clients that refresh tools. |
| `notifications/prompts/list_changed` | Emit after prompt catalog changes if supported | Same | Same shape as tool/resource changed. |
| `notifications/message` / logging | Proxy upstream logging where supported | Add local broker logging notifications | Everything server has logging examples. |
| Elicitation | Do not fake it as a tool | Route upstream elicitation to downstream client | Requires bidirectional request routing and capability checks. |
| Sampling | Do not fake it as a tool | Route upstream sampling to downstream client | Requires downstream client sampling support. |
| Tasks | Do not invent custom primary API | Use MCP Tasks for long-running runs where supported | Custom `start_run` only as compatibility fallback. |
| Ping | Support local ping and keep upstream health separately | Same | Upstream ping should be a health/admin operation, not necessarily forwarded blindly. |
## Current Risks
- FastMCP mount/unmount lifecycle is not complete enough for safe dynamic
unmount of all proxied capabilities.
- Transparent reload is currently best-effort.
- Bidirectional upstream requests such as elicitation/sampling require access to
the downstream client session, not just an upstream SDK client.
- Notifications must be scoped: a resource update from `everything.default`
should not look like a local broker config update.
- Clients vary. Codex may not immediately refresh tool lists. Inspector may show
more protocol features. The proxy must be robust even when clients ignore
optional notifications.
## Required Boundaries
### Shared Service Layer
`WfMcpService` or a sibling service owns:
- configured connections
- adapters
- auth/catalog stores
- capability sources
- workflow artifact store
- events
It should not own FastMCP decorators directly.
### Projection Layer
Projection modules register MCP-visible capabilities:
- upstream tools/resources/prompts
- local workflow tools
- local admin tools
- local docs/prompts/resources
Projection modules call shared handlers. They should not contain business logic.
### Protocol Routing Layer
Protocol routing handles bidirectional features:
- upstream-to-downstream elicitation
- upstream-to-downstream sampling
- upstream/local notifications
- tasks/progress
- subscriptions
This layer should be explicit. Do not bury protocol routing inside a generic
`call_tool` helper.
### Event/Notification Bus
Local events should be emitted once and then projected to:
- stored broker events
- MCP notifications where the client supports them
- future UI/dashboard streams
Examples:
```text
connection_registered
catalog_refresh_started
catalog_refresh_completed
tool_call_started
tool_call_completed
workflow_artifact_saved
workflow_deployment_saved
workflow_run_started
workflow_run_progress
workflow_run_completed
source_enabled
source_disabled
config_reloaded
```
## Prerequisites
- Complete `2026-05-12-workflow-artifact-hardening.md`.
- Keep artifact operations in shared callable functions, not duplicated
FastMCP decorators.
- Keep config mutation/admin operations behind explicit admin exposure.
- Document client capability assumptions for elicitation, sampling, tasks, and
notifications.
- Add tests against the fixture MCP server and the everything server where
practical.
## Phase 1: Inventory And Adapter Reality Check
**Files:**
- Create: `docs/mcp_protocol_proxy_inventory.md`
- Inspect: `src/wf_mcp/sdk/adapter.py`
- Inspect: `src/wf_mcp/transparent_proxy/runtime.py`
- Inspect: `src/wf_mcp/broker/artifact_tools.py`
- Test: no new tests required in this phase
- [ ] List which MCP SDK APIs are currently wrapped by `BackendAdapter`.
- [ ] List which FastMCP server APIs we currently use.
- [ ] List which features everything-server exposes that we can test:
- progress
- logging
- resource updates
- elicitation
- sampling
- tasks
- [ ] Identify SDK gaps before implementation. If an MCP feature is not
accessible through FastMCP/MCP SDK at our current version, document it instead
of inventing a fake abstraction.
## Phase 2: Extract Shared Workflow/Admin Handlers
**Files:**
- Create: `src/wf_mcp/workflow_surface/handlers.py`
- Create: `src/wf_mcp/admin_surface/handlers.py`
- Modify: `src/wf_mcp/broker/artifact_tools.py`
- Modify: `src/wf_mcp/broker/tools.py`
- Modify: `src/wf_mcp/transparent_proxy/admin.py`
- Test: `tests/wf_mcp/test_broker_server.py`
- Test: `tests/wf_mcp/test_transparent_proxy.py`
- [ ] Move workflow artifact list/save/inspect/validate/run logic into shared
handler functions/classes.
- [x] Move admin list/refresh/config/reload logic into shared handler
functions/classes.
- [x] Keep broker compatibility tool names working.
- [x] Keep transparent proxy admin tool names working.
- [ ] Do not change behavior in this phase; only remove duplicated logic and
create a single implementation path.
## Phase 3: Unified Server Factory
**Files:**
- Create: `src/wf_mcp/server/unified.py`
- Modify: `src/wf_mcp/cli.py`
- Modify: `src/wf_mcp/broker/server.py`
- Test: `tests/wf_mcp/test_unified_server.py`
- [x] Build one FastMCP server from `BrokerConfig`.
- [ ] Register local workflow tools with namespaced names:
- [x] `wf.workflow.list_artifacts`
- [x] `wf.workflow.create_artifact_from_plan`
- [x] `wf.workflow.save_artifact`
- [x] `wf.workflow.list_deployments`
- [x] `wf.workflow.save_deployment`
- [x] `wf.workflow.validate_deployment`
- [x] `wf.workflow.run_deployment`
- [x] Register admin tools only when admin exposure is enabled.
- [x] Project upstream tools using the transparent proxy path.
- [x] Keep existing `broker` and `proxy` CLI modes during migration.
- [x] Add `unified` CLI mode.
- [ ] Do not make unified default until manual Inspector/Codex tests pass.
## Phase 4: Namespacing And Collision Policy
**Files:**
- Modify: `src/wf_mcp/shared/names.py`
- Test: `tests/wf_mcp/test_names.py`
- Test: `tests/wf_mcp/test_unified_server.py`
- [ ] Use `wf.workflow.*` for stable workflow tools.
- [ ] Use `wf.admin.*` for privileged admin/control tools.
- [ ] Keep `wf.mcp.*` for workflow runtime helpers, not admin.
- [ ] Keep upstream proxy names collision-safe.
- [ ] Reject configured connection ids that collide with reserved local
namespaces.
- [ ] Decide whether compatibility broker names remain visible by default in
unified mode. Recommended: yes during migration, no after migration.
## Phase 5: Tool/Resource/Prompt Projection Parity
**Files:**
- Modify: `src/wf_mcp/transparent_proxy/runtime.py`
- Modify: unified server files from Phase 3.
- Test: `tests/wf_mcp/test_unified_server.py`
- [ ] Ensure upstream tools and stable workflow/admin tools both appear in
`tools/list`.
- [ ] Ensure upstream resources appear in `resources/list` and can be read.
- [ ] Ensure upstream prompts appear in `prompts/list` and can be rendered.
- [ ] Ensure resources-as-tools and prompts-as-tools remain optional projection
modes, not the only way to access resources/prompts.
- [ ] Ensure search/pagination includes stable local tools and upstream tools.
## Phase 6: Local Notification Bus
**Files:**
- Create: `src/wf_mcp/events/bus.py`
- Modify: `src/wf_mcp/broker/events.py`
- Modify: `src/wf_mcp/broker/service/core.py`
- Modify: unified server files from Phase 3.
- Test: `tests/wf_mcp/test_events.py`
- [x] Introduce an in-process event bus abstraction.
- [x] Keep the existing stored `McpEvent` list as one subscriber/sink.
- [ ] Add event kinds for workflow artifacts and deployments:
- [x] `workflow_artifact_saved`
- [x] `workflow_deployment_saved`
- [x] `workflow_run_started`
- [x] `workflow_run_completed`
- `workflow_run_failed`
- [ ] Add event kinds for capability changes:
- `source_enabled`
- `source_disabled`
- [x] `catalog_changed`
- [x] `tools_changed`
- [x] `resources_changed`
- [x] `prompts_changed`
- [ ] Do not emit MCP notifications yet unless the server/session API is
clearly available. This phase creates the source of truth.
## Phase 7: MCP Notifications
**Files:**
- Modify: unified server files from Phase 3.
- Modify: `src/wf_mcp/events/bus.py`
- Test: `tests/wf_mcp/test_unified_server.py`
- [ ] Emit MCP list-changed notifications when local or upstream catalogs
change, if supported:
- `notifications/tools/list_changed`
- `notifications/resources/list_changed`
- `notifications/prompts/list_changed`
- [ ] Emit local workflow progress notifications where supported.
- [ ] Proxy upstream logging notifications where supported.
- [ ] Ensure clients that ignore notifications can still poll/list manually.
- [ ] Add tests that assert notifications are requested/emitted through whatever
FastMCP/MCP SDK surface is available. If no testable surface exists, document
the limitation in `docs/mcp_protocol_proxy_inventory.md`.
## Phase 8: Elicitation And Sampling Routing
**Files:**
- Create: `src/wf_mcp/protocol/elicitation.py`
- Create: `src/wf_mcp/protocol/sampling.py`
- Modify: SDK adapter/session layer if supported.
- Test: `tests/wf_mcp/test_protocol_proxy.py`
- [ ] Determine how upstream MCP SDK exposes server-to-client elicitation
requests.
- [ ] Determine how FastMCP exposes downstream client elicitation responses.
- [ ] Route upstream elicitation requests to the downstream client only when the
downstream client advertised support.
- [ ] Route upstream sampling requests to the downstream client only when the
downstream client advertised support.
- [ ] Preserve request ids/correlation ids so responses return to the correct
upstream session.
- [ ] Return structured unsupported diagnostics when routing is impossible.
- [ ] Do not convert elicitation/sampling into normal tools as the primary
behavior.
## Phase 9: Tasks And Long-Running Workflow Runs
**Files:**
- Create: `src/wf_mcp/workflow_surface/runs.py`
- Modify: unified server files from Phase 3.
- Test: `tests/wf_mcp/test_workflow_tasks.py`
- [ ] Prefer MCP Tasks for long-running `wf.workflow.run_deployment` when the
client/server support task execution.
- [ ] Keep synchronous run behavior for short/manual local tests.
- [ ] Add a compatibility run store only if MCP Tasks are unavailable or
insufficient for Codex/Inspector.
- [ ] Map workflow interrupts to task status such as `input_required` only after
the runtime supports the needed resume model.
- [ ] Do not implement durable scheduling/cron here.
## Phase 10: Mode Migration
**Files:**
- Modify: `docs/wf_mcp_architecture.md`
- Modify: `docs/wf_mcp_capability_sources.md`
- Modify: `src/wf_mcp/cli.py`
- Test: `tests/wf_mcp/test_cli.py`
- [ ] Document unified mode as the recommended local mode once it passes manual
Codex and Inspector checks.
- [ ] Keep broker/proxy modes as compatibility modes.
- [ ] Mark compatibility broker tool names as legacy once namespaced local tools
work in unified mode.
- [ ] Do not delete compatibility modes until tests cover every important
surface.
> **Superseded on 2026-05-16:** the compatibility period is now considered long
> enough. The current plan is to retire the public broker/proxy mode split in
> [`2026-05-16-retire-legacy-mcp-modes.md`](2026-05-16-retire-legacy-mcp-modes.md)
> while keeping the useful internal implementation boundaries.
## Manual Verification Checklist
- [ ] Codex can list upstream tools.
- [ ] Codex can call `everything.default_echo` or equivalent upstream tool.
- [ ] Codex can call `wf.workflow.list_artifacts`.
- [ ] Codex can create an artifact from a plan.
- [ ] Codex can save a deployment.
- [ ] Codex can validate and run a deployment.
- [ ] Inspector shows upstream resources and prompts.
- [ ] Inspector shows local workflow tools with useful names/descriptions.
- [ ] If everything-server elicitation is triggered, the proxy either routes it
correctly or returns a clear unsupported diagnostic.
- [ ] If everything-server progress/logging is triggered, the proxy either
forwards it correctly or documents why not.
## Non-Goals
- No native `wf_core` subgraph implementation.
- No UI/dashboard implementation.
- No cron/scheduler implementation.
- No hidden conversion of every protocol feature into a tool.
- No pretending unsupported protocol features are proxied.
## Success Criteria
The project can run one local MCP server that gives an LLM client both:
- direct access to upstream MCP capabilities
- stable workflow/admin capabilities implemented by this project
The implementation must be explicit about unsupported protocol features and must
have a path to proxy elicitation, sampling, notifications, and tasks without
duplicating mode-specific logic.
@@ -0,0 +1,119 @@
# Workflow Artifact Hardening Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make saved workflow artifacts safe enough to rely on before merging MCP server modes or exposing workflow execution more broadly.
**Architecture:** Keep `wf_artifacts` provider-neutral. `wf_mcp` may project artifact operations, but validation, storage, artifact creation, and diagnostics should stay in `wf_artifacts` where possible.
**Tech Stack:** Python 3.14, Pydantic v2, pytest, `wf_core.Workflow` validation, existing `wf_mcp` broker service.
---
## Why This Comes First
The live Codex probe proved the artifact execution path works, but it also found
a real weakness: `create_workflow_artifact_from_plan` saved an invalid
`state_schema` shape, and the error surfaced only when `run_workflow_deployment`
tried to compile the workflow.
Before unifying broker and transparent proxy modes, artifact creation should
reject invalid plans earlier and return structured diagnostics.
## Scope
This plan hardens saved workflow creation and dependency validation. It does not
merge MCP server modes and does not add native subgraphs.
## Tasks
### Task 1: Validate Plan Shape During Artifact Creation
**Files:**
- Modify: `src/wf_artifacts/factory.py`
- Test: `tests/artifacts/test_factory.py`
- [ ] Add a failing test proving `create_workflow_artifact_from_plan` rejects a
plan that cannot become a `wf_core.Workflow`.
- [ ] Implement validation by constructing `wf_core.Workflow.model_validate`
from the plan fields.
- [ ] Keep the validation dependency one-way: `wf_artifacts` may import
`wf_core`, but `wf_core` must not import `wf_artifacts`.
- [ ] Return `ValueError` with a concise message containing the failing field
path or Pydantic error message.
- [ ] Run `uv run --with pytest pytest tests\artifacts\test_factory.py -q`.
### Task 2: Add Artifact Creation Diagnostics
**Files:**
- Modify: `src/wf_artifacts/models.py`
- Modify: `src/wf_artifacts/factory.py`
- Test: `tests/artifacts/test_factory.py`
- [ ] Decide whether creation failures should raise exceptions only or also
expose a `validate_workflow_artifact_plan(...) -> list[DependencyDiagnostic]`
style function.
- [ ] Recommended v1: add a separate `validate_workflow_artifact_plan(plan)`
that returns structured diagnostics, while the factory still raises on errors.
- [ ] Add tests for missing `input_schema`, missing `output_schema`, invalid
state schema, and missing start node.
### Task 3: Validate Direct Workflow Dependencies
**Files:**
- Modify: `src/wf_artifacts/validation.py`
- Test: `tests/artifacts/test_validation.py`
- [ ] Add artifact-store-aware validation for `workflow_dependencies`.
- [ ] Validate exact artifact-version pins.
- [ ] Return `dependency_missing` when a child artifact version does not exist.
- [ ] Return `dependency_cycle` when direct/transitive workflow artifact
dependencies cycle.
- [ ] Do not copy child dependency snapshots into the parent.
### Task 4: Improve Capability Contract Checking
**Files:**
- Modify: `src/wf_artifacts/validation.py`
- Test: `tests/artifacts/test_validation.py`
- Optional: `src/wf_mcp/broker/artifact_tools.py`
- [ ] Preserve current hash comparison behavior.
- [ ] Add tests for kind mismatch, for example required `tool` but available
`node_spec`.
- [ ] Decide whether `node_spec` can satisfy `tool` when the provider is an MCP
wrapper. Recommended: no implicit kind coercion in `wf_artifacts`; adapters
should present the available kind they mean to expose.
- [ ] Add diagnostics with `code="capability_kind_mismatch"`.
### Task 5: Document Current Runtime Limitations In Tool Responses
**Files:**
- Modify: `src/wf_mcp/broker/artifact_tools.py`
- Test: `tests/wf_mcp/test_broker_server.py`
- [ ] Keep interrupting artifacts rejected by `run_workflow_deployment`.
- [ ] Add `repair_hint` text explaining native subgraphs/nested resume are not
implemented yet.
- [ ] Add a test proving unsupported interrupt artifacts return a diagnostic
instead of raising.
## Verification
- [ ] `uv run --with pytest pytest tests\artifacts -q`
- [ ] `uv run --with pytest pytest tests\wf_mcp\test_broker_server.py -q`
- [ ] `uv run --with pytest pytest -q`
- [ ] `uv run ruff check src tests examples main.py`
- [ ] `uv run basedpyright src tests examples main.py --level error`
## Non-Goals
- No native `wf_core` subgraph implementation.
- No MCP Tasks implementation.
- No run history persistence.
- No broker/transparent mode merge in this plan.
@@ -0,0 +1,144 @@
# Retire Legacy MCP Modes Plan
> **Status:** completed retirement pass; kept as the decision record.
## Goal
Make one public MCP server surface exposed by `wf-mcp`.
`broker` mode and `proxy` mode were useful while the unified surface was being
built, but keeping all three as public launch modes now creates the wrong mental
model:
- broker mode suggests local workflow/admin tools are a separate product surface
- proxy mode suggests upstream projection is a separate product surface
- the combined server is already the real target: one server that exposes local
capabilities and proxied upstream capabilities together
The public product should have one server mode. The implementation may still
have multiple internal concern packages.
## Existing Documentation
This replaces the migration stance in
[`2026-05-12-unified-mcp-surface.md`](2026-05-12-unified-mcp-surface.md),
which said to keep broker/proxy as compatibility modes until coverage existed.
That compatibility period is now considered complete enough to end.
Related documents already cover adjacent plans:
- [`../../wf_mcp_architecture.md`](../../wf_mcp_architecture.md)
- concern boundaries and proxy mount lifecycle
- [`../../wf_mcp_capability_sources.md`](../../wf_mcp_capability_sources.md)
- source model, admin/workflow exposure, source inventory
- [`../../workflow_artifacts.md`](../../workflow_artifacts.md)
- artifacts, deployments, stable workflow control surface
- [`../../wf_mcp_proxy_reality_and_roadmap.md`](../../wf_mcp_proxy_reality_and_roadmap.md)
- what the proxy does well today and what should remain upstream-dependent
- [`../../mcp_protocol_proxy_inventory.md`](../../mcp_protocol_proxy_inventory.md)
- observed protocol behavior and relay gaps
No other major active plan was found to be undocumented while preparing this
pass.
## What Gets Retired
### Public CLI modes
Remove the public `serve --mode broker` and `serve --mode proxy` choices.
After this pass:
- `wf-mcp serve` runs the server surface
- users no longer choose among three product modes
- docs should describe one server behavior, not a mode matrix
### Public framing
Stop presenting broker/proxy as user-facing alternatives in docs and help text.
Where historical explanation is useful, call them legacy migration surfaces.
### Compatibility-only tests and docs
Delete or rewrite tests whose only purpose is to prove the old public mode split.
Keep behavior tests for the underlying capabilities when those behaviors still
exist through the public server.
## What Stays
### Internal concern packages
Do **not** flatten the codebase just because the public mode split disappears.
These packages still represent useful implementation boundaries:
- `wf_mcp.broker`
- `wf_mcp.transparent_proxy`
- `wf_mcp.server`
`transparent_proxy` is already partly a legacy package name, but the code inside
it still owns real proxy-mounting mechanics used by the server. Rename or
re-home that code only as a later cleanup if the package name becomes a real
source of confusion.
### Shared services
Keep the service/config/store/runtime objects that the public server already uses.
This pass is about removing duplicate **entrypoints**, not rewriting the
underlying architecture.
### Stable local capability names
Keep the source model and namespaces:
- `wf.workflow.*`
- `wf.admin.*`
- `wf.std.*`
- `wf.mcp.*`
- `<connection_id>.*`
The cleanup should reduce surfaces, not churn the capability vocabulary.
## Expected Code Changes
1. Simplify CLI mode selection so `serve` has one public behavior.
2. Remove or privatize old broker/proxy server launch functions that only exist
for the retired public modes.
3. Collapse docs/help text that still describe three user-facing modes.
4. Keep implementation reuse through the existing server path.
5. Update tests so they assert server behavior directly instead of branching on
legacy mode names.
## Non-Goals For This Pass
- No full rewrite of the proxy subsystem.
- No attempt to solve generic upstream notification relay.
- No safe unmount implementation for retired FastMCP providers.
- No renaming of every legacy internal package just to match the new public
shape.
- No workflow artifact redesign.
Those topics already have separate docs and should stay separate.
## Success Criteria
- `wf-mcp serve` has one public MCP server behavior.
- No public docs imply that broker/proxy are still supported product modes.
- The server continues to expose:
- proxied upstream capabilities
- stable local workflow tools
- optional admin tools
- The test suite passes with the old public mode split removed.
- Remaining roadmap docs still point to the real unresolved work instead of
making the reader rediscover why the combined server exists.
## Follow-On Work After This Pass
Once the public surface is singular, the next useful cleanup is not more mode
work. It is easier-to-explain capability projection:
1. keep improving source inventory and admin visibility
2. continue documenting proxy relay limitations explicitly
3. build workflow-facing wrapper artifacts on top of stable sources
That keeps the system moving toward a clean platform without pretending the
proxy layer is already a perfect MCP relay.
@@ -0,0 +1,68 @@
# Call Wrapper Artifacts Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Let `wf.workflow.call_capability` execute saved `WorkflowArtifact(kind="wrapper")` artifacts by their stable artifact node name without turning arbitrary saved workflows into node capabilities.
**Architecture:** Keep live `NodeSpec` execution unchanged. Extend the workflow surface resolution path so `workflow.<artifact_id>.v<version>` can resolve to a saved wrapper artifact, validate that it is wrapper-kind and interrupt-free, execute its stored plan through the existing workflow runner, and normalize the final workflow result into the same `qualified_name` / `outcome` / `output` payload shape as live capabilities.
**Tech Stack:** Python, Pydantic, `wf_artifacts`, `wf_core`, pytest.
---
### Task 1: Pin Wrapper-Artifact Call Semantics
**Files:**
- Modify: `tests/wf_mcp/test_service.py`
- [ ] **Step 1: Write the failing test**
Add a test that saves a wrapper artifact with a simple one-node plan, calls `WorkflowSurfaceHandlers.call_capability()` with `workflow.<id>.v<version>`, and asserts the returned `qualified_name`, `outcome`, and `output`.
- [ ] **Step 2: Run test to verify it fails**
Run: `uv run --with pytest pytest tests/wf_mcp/test_service.py -q`
Expected: FAIL because `call_capability()` only resolves live specs today.
### Task 2: Resolve Wrapper Artifacts in the Workflow Surface
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- [ ] **Step 1: Implement minimal wrapper-artifact resolution**
Add a small helper that:
- recognizes stable artifact node names
- loads the artifact from the store
- rejects non-wrapper artifacts
- rejects unsupported interrupt plans
- executes the artifact plan with the existing workflow runner
- converts the workflow run into the `call_capability` response payload
- [ ] **Step 2: Run focused tests**
Run: `uv run --with pytest pytest tests/wf_mcp/test_service.py -q`
Expected: PASS.
### Task 3: Verify the Whole Project
**Files:**
- No additional files.
- [ ] **Step 1: Run focused workflow-surface tests**
Run: `uv run --with pytest pytest tests/wf_mcp -q`
Expected: PASS.
- [ ] **Step 2: Run the full suite**
Run: `uv run --with pytest pytest -q`
Expected: PASS.
@@ -0,0 +1,55 @@
# Nested Authoring State Projection Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Let `wf_authoring` project nested authored state schemas into the flat exact-path state-field index that `wf_core` now supports.
**Architecture:** Keep authored JSON Schema nested for users and LLM clients. Add a focused flattening helper for state-field projection only, emitting both parent object paths and descendant paths. Resolve nested `BaseModel` metadata by authored path where available, while leaving non-`BaseModel` authored types schema-capable with default merge metadata.
**Tech Stack:** Python, Pydantic, pytest, existing `wf_authoring` schema adapter.
---
## File Structure
- Modify `src/wf_authoring/schemas.py`
- flatten nested schema properties into exact-path `StateField`s
- gather nested `BaseModel` metadata by authored path
- Modify `tests/authoring/helpers.py`
- add nested state models used by tests
- Modify `tests/authoring/test_schemas.py`
- pin nested projection behavior and nested metadata
- Update `docs/core_state_mapping_and_merge.md`
- note that authoring now projects nested authored models into the flat core index
## Tasks
### Task 1: Pin Nested Projection
- [ ] Add tests proving:
- nested authored state keeps parent and child declarations
- nested child metadata such as `append` lands on the exact child path
- parent object declaration remains independent from child declarations
- [ ] Run the focused authoring tests and confirm they fail under current top-level-only projection.
### Task 2: Implement Projection Helpers
- [ ] Add a schema-walking helper that yields `(path, property_schema)` for parent and descendant properties.
- [ ] Add nested `BaseModel` metadata traversal keyed by dotted path.
- [ ] Update `state_schema_from()` to build `StateField`s from the flattened path stream.
- [ ] Keep JSON Schema generation unchanged; flatten only the core `StateSchema.fields` index.
- [ ] Run the focused authoring tests and confirm they pass.
### Task 3: Document and Verify
- [ ] Update the core state mapping doc with the authoring projection rule.
- [ ] Run `uv run --with pytest pytest tests/authoring -q`
- [ ] Run `uv run --with pytest pytest -q`
- [ ] Run `uv run basedpyright --level error`
## Non-Goals
- custom metadata support for every Pydantic-supported type form
- changing `SchemaRef` shape
- reducer registries
- automatic deep merge behavior
@@ -0,0 +1,325 @@
# Nested Node-Local Mappings Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Let workflow maps address nested node-local input/output paths while preserving explicit state-mediated data flow and preparing state writes for future reducer libraries.
**Architecture:** Keep graph-facing paths unchanged. Add a small node-local path helper layer, validate non-overlapping write targets, construct nested node inputs from `in_map`, resolve nested node outputs from `out_map`, and refactor state writes into a prepared patch commit. Extract built-in merge-rule application behind a focused reducer-like module so future named reducer libraries can replace the dispatch seam without changing patch semantics.
**Tech Stack:** Python, Pydantic, pytest, existing `wf_core` path/state runtime.
---
## File Structure
- Modify `src/wf_core/paths.py`
- keep graph-path helpers
- add reusable path-overlap utility if it belongs at the generic path layer
- Create `src/wf_core/local_paths.py`
- node-local dotted path parsing
- nested get/set for node-local payloads
- overlap checks for write targets
- Create `src/wf_core/runtime/ops/merges.py`
- built-in exact-path merge-rule implementations
- future seam for named reducer registry
- Modify `src/wf_core/runtime/ops/state.py`
- replace per-write mutation loop with prepared patch commit
- delegate merge-rule application to `merges.py`
- Modify `src/wf_core/runtime/ops/nodes.py`
- build nested node inputs from `in_map`
- resolve nested node outputs into state patch writes
- Modify `src/wf_core/validation/steps.py`
- validate node-local top-level roots
- reject overlapping write targets
- Add/modify tests under `tests/core/`
- nested `in_map`
- nested `out_map`
- whole-object mapping still works
- overlapping write targets rejected
- missing nested output path fails
- merge dispatch remains behaviorally unchanged
---
### Task 1: Pin Node-Local Path Behavior
**Files:**
- Modify: `tests/core/test_validation.py`
- Modify: `tests/core/test_runtime.py`
- [ ] **Step 1: Add failing validation tests**
Cover:
```python
def test_validation_allows_nested_node_local_paths() -> None:
...
def test_validation_rejects_overlapping_node_input_destinations() -> None:
...
def test_validation_rejects_overlapping_state_write_destinations() -> None:
...
```
Expected rules:
- `state.person.name -> user.name` is valid when `user` exists in the node input schema
- `user -> state.person` and `user.name -> state.person.name` in one `out_map` is invalid because destination state paths overlap
- `state.person -> user` plus `state.person.name -> user.name` in one `in_map` is invalid because destination node-local paths overlap
- [ ] **Step 2: Add failing runtime tests**
Cover:
```python
def test_runtime_builds_nested_node_input_from_in_map() -> None:
...
def test_runtime_reads_nested_node_output_from_out_map() -> None:
...
def test_runtime_missing_nested_node_output_path_fails() -> None:
...
```
- [ ] **Step 3: Run focused tests and confirm failure**
Run:
```bash
uv run --with pytest pytest tests/core/test_validation.py tests/core/test_runtime.py -q
```
Expected: FAIL because node-local map sides are top-level-only today.
### Task 2: Add Node-Local Path Helpers
**Files:**
- Create: `src/wf_core/local_paths.py`
- Modify: `src/wf_core/validation/steps.py`
- [ ] **Step 1: Add minimal helper API**
Implement:
```python
def split_local_path(path: str) -> list[str]: ...
def get_local_value(payload: Mapping[str, Any], path: str) -> Any: ...
def set_local_value(payload: dict[str, Any], path: str, value: Any) -> None: ...
def paths_overlap(left: str, right: str) -> bool: ...
def has_overlapping_paths(paths: Iterable[str]) -> bool: ...
```
Rules:
- dotted local paths only
- no empty segments
- overlap means same path or ancestor/descendant path
- [ ] **Step 2: Update validation**
Use node-local path roots for schema checks:
```python
input_root = split_local_path(destination_path)[0]
output_root = split_local_path(source_path)[0]
```
Reject:
- overlapping `in_map` destination local paths
- overlapping `out_map` destination state paths
- [ ] **Step 3: Run focused validation tests**
Run:
```bash
uv run --with pytest pytest tests/core/test_validation.py -q
```
Expected: PASS for validation-specific cases.
### Task 3: Execute Nested Local Mappings
**Files:**
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Modify: `src/wf_core/runtime/ops/state.py`
- [ ] **Step 1: Build nested node inputs**
Replace flat input construction with:
```python
resolved_input: dict[str, Any] = {}
for source_path, destination_path in node.in_map.items():
value = safe_resolve_path(...)
set_local_value(resolved_input, destination_path, value)
```
- [ ] **Step 2: Resolve nested node outputs**
When preparing mapped output writes, use `get_local_value()` for each `out_map`
source path instead of indexing only top-level output keys.
- [ ] **Step 3: Preserve missing-path failures**
Raise `WorkflowExecutionError` when a mapped nested output path is missing.
- [ ] **Step 4: Run focused runtime tests**
Run:
```bash
uv run --with pytest pytest tests/core/test_runtime.py -q
```
Expected: PASS for nested mapping behavior.
### Task 4: Introduce Prepared Patch Commits and Extract Merge Dispatch
**Files:**
- Create: `src/wf_core/runtime/ops/merges.py`
- Modify: `src/wf_core/runtime/ops/state.py`
- Modify: `tests/core/test_state_ops.py`
- [ ] **Step 1: Add failing patch-level tests**
Cover:
```python
def test_state_patch_rejects_overlapping_destinations_before_mutation() -> None:
...
def test_builtin_merge_rules_preserve_existing_behavior() -> None:
...
```
- [ ] **Step 2: Extract built-in merge implementations**
Move the current strategy body out of `write_state_value()` into focused helpers:
```python
def apply_builtin_merge(
*,
strategy: str,
current_value: Any,
incoming_value: Any,
destination_path: str,
) -> Any: ...
```
Keep current semantics:
- `replace`
- `append`
- shallow `merge_object`
Add a docstring that this is the future seam for source-owned named reducers,
not custom reducer support yet.
- [ ] **Step 3: Prepare full write sets before mutation**
Refactor output mapping so it:
1. resolves all mapped output values
2. validates destination overlap
3. prepares the patch
4. applies merge behavior
No state changes should occur before all mapped output paths are known-good.
- [ ] **Step 4: Run state/runtime tests**
Run:
```bash
uv run --with pytest pytest tests/core/test_state_ops.py tests/core/test_runtime.py -q
```
Expected: PASS.
### Task 5: Keep Authoring and Docs Aligned
**Files:**
- Modify: `src/wf_authoring/builder/mapping.py`
- Modify: `docs/core_state_mapping_and_merge.md` if implementation details differ
- Modify: `docs/scratchpad.md` only if wording drift appears
- Modify/Add: authoring tests as needed
- [ ] **Step 1: Confirm authoring auto-maps remain top-level**
Automatic maps should stay conservative unless there is an explicit reason to
infer nested paths. The new feature is for explicit maps first.
- [ ] **Step 2: Add one authoring regression**
Prove that a builder can compile a workflow using explicit nested local map
paths without extra helper nodes.
- [ ] **Step 3: Update docs only for implementation drift**
The design doc already states the target behavior. Keep docs in sync with final
names and module boundaries, but do not broaden scope into nested state
declarations yet.
### Task 6: Verify the Whole Project
**Files:**
- No additional files.
- [ ] **Step 1: Run focused suites**
```bash
uv run --with pytest pytest tests/core tests/authoring -q
```
- [ ] **Step 2: Run the full suite**
```bash
uv run --with pytest pytest -q
```
- [ ] **Step 3: Run type checking**
```bash
uv run basedpyright --level error
```
Expected:
- tests pass
- any remaining basedpyright failures are called out explicitly if they come
from existing generated/build/doc-fixture noise rather than this work
---
## Deliberate Non-Goals
- nested declared state merge metadata
- reducer capability registry
- deep merge behavior
- native subgraphs
- parallel foreach
- automatic inference of nested maps from schemas
## Follow-On Plans
After this lands:
1. nested declared state paths with exact-path merge lookup
2. reducer capability model / registry seam
3. native subgraph design on the same map + patch boundary
4. async-only parallel foreach using patch combination rules
@@ -0,0 +1,61 @@
# Nested State Paths Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Let `wf_core` declare nested workflow state paths and apply merge behavior by exact destination path.
**Architecture:** Keep state declarations internally flat and path-keyed. Continue allowing undeclared state paths, but only exact declared paths receive typed merge behavior; parent declarations do not implicitly govern descendants. Reuse the state patch boundary added in Phase 1 and change only schema wording, validation coverage, runtime lookup, and docs.
**Tech Stack:** Python, Pydantic, pytest, existing `wf_core` state runtime.
---
## File Structure
- Modify `src/wf_core/models/schemas.py`
- clarify that `StateField` / `StateSchema` are path-keyed, not root-only
- Modify `src/wf_core/runtime/ops/state.py`
- resolve merge metadata by exact written state path
- Add `tests/core/test_nested_state_paths.py`
- exact nested path merge behavior
- ancestor declarations do not govern descendants
- undeclared nested paths still replace
- Update `docs/core_state_mapping_and_merge.md`
- mark nested declared state paths as implemented
- Update `docs/schema_validation.md`
- clarify that runtime state merge metadata is now exact-path capable
### Task 1: Pin Exact-Path State Behavior
- [ ] Add failing tests proving:
- `state.person.tags` uses a declaration for `"person.tags"` with `append`
- a declaration for `"person"` does not cause `state.person.tags` to inherit `merge_object`
- undeclared `state.person.tags` defaults to `replace`
- [ ] Run the focused tests and confirm the nested exact-path cases fail under the current root-only lookup.
### Task 2: Implement Exact-Path Lookup
- [ ] Update state model docstrings to describe path-keyed declarations.
- [ ] In `write_state_value()`, look up `workflow.state_schema.fields[".".join(parts)]` instead of only the first path segment.
- [ ] Keep undeclared paths as `replace`.
- [ ] Run the focused tests and confirm they pass.
### Task 3: Keep Docs Honest
- [ ] Update the core mapping design doc so Phase 2 is recorded as implemented, not future work.
- [ ] Update schema-validation docs to note exact-path state metadata without broadening payload validation claims.
### Task 4: Verify
- [ ] Run `uv run --with pytest pytest tests/core -q`
- [ ] Run `uv run --with pytest pytest -q`
- [ ] Run `uv run basedpyright --level error`
- [ ] Call out any residual type-check failures that are unrelated to this work.
## Non-Goals
- flatten nested authoring schemas from `wf_authoring`
- reducer registries or custom reducer capabilities
- changing `merge_object` from shallow to deep
- making parent declarations inherit into child paths
- `START` token model changes
@@ -0,0 +1,57 @@
# Reducer Authoring Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add Python authoring ergonomics for reducer definitions, including optional Pydantic config models.
**Architecture:** Keep `wf_core` reducer execution as the runtime layer. Add `wf_authoring.reducers` as the ergonomic layer that builds `ReducerDefinition` objects from Python functions, similar to how node authoring builds `NodeSpec`s. Config models are optional; when present, they generate `ReducerSpec.config_schema` and receive parsed config objects at call time.
**Tech Stack:** Python, Pydantic `BaseModel`, pytest, existing reducer runtime.
---
## File Structure
- Create `src/wf_authoring/reducers/`
- `callables.py`: reducer callable protocols
- `decorator.py`: `@reducer(...)`
- `catalog.py`: `ReducerCatalog`
- `__init__.py`: reducer authoring exports
- Modify `src/wf_authoring/__init__.py`
- export reducer authoring API
- Add `tests/authoring/test_reducers.py`
- plain reducer authoring
- configured reducer authoring with BaseModel config
- catalog specs/definitions
## Tasks
### Task 1: Pin Authoring API
- [ ] Add tests proving:
- `@reducer(name="wf.std.add")` wraps a two-arg callable
- `@reducer(name="wf.std.modulo_add", config_model=ModuloConfig)` wraps a callable receiving parsed config
- config model JSON Schema becomes `ReducerSpec.config_schema`
- `ReducerCatalog.from_reducers(...)` exposes definitions and specs
- [ ] Run focused tests and confirm failure before implementation.
### Task 2: Implement Reducer Authoring
- [ ] Add typed callable protocols for plain/config reducers.
- [ ] Add a small wrapper object that owns a `ReducerDefinition`.
- [ ] Add `@reducer(...)` overloads for bare and configured reducers.
- [ ] Add `ReducerCatalog`.
- [ ] Export from `wf_authoring`.
### Task 3: Verify
- [ ] Run `uv run --with pytest pytest tests/authoring/test_reducers.py -q`
- [ ] Run `uv run --with pytest pytest tests/authoring -q`
- [ ] Run full suite and basedpyright.
## Non-Goals
- MCP tools for authoring reducers
- LLM-authored reducer code
- external reducer packages
- replacing current built-in registration in this pass
@@ -0,0 +1,78 @@
# Reducer Capabilities Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace `merge_strategy` with named pure reducer references across core and authoring, using `wf.std.replace` as the default reducer.
**Architecture:** Introduce a small reducer registry in `wf_core`, register the current three built-ins as reducers, and have state writes resolve every declared reducer name through that registry. Keep undeclared state paths using default replace semantics. Migrate `wf_authoring.state_field()` and all state metadata/tests/docs to reducer names in the same pass so there is one merge concept in the codebase.
**Tech Stack:** Python, Pydantic, pytest, existing `wf_core` runtime and `wf_authoring` schema projection.
---
## File Structure
- Modify `src/wf_core/models/schemas.py`
- replace `merge_strategy` with `reducer`
- Replace/refactor `src/wf_core/runtime/ops/merges.py`
- reducer callable type
- built-in reducer functions
- default reducer registry
- reducer application helper
- Modify `src/wf_core/runtime/ops/state.py`
- resolve reducer names from state fields
- use default replace reducer for undeclared paths
- Modify `src/wf_authoring/schemas.py`
- expose `state_field(reducer=...)`
- project reducer metadata through flattened state paths
- Modify tests under `tests/core/`, `tests/authoring/`, and `tests/rewrite/`
- migrate old metadata
- add unknown reducer coverage
- Update docs mentioning `merge_strategy`
## Tasks
### Task 1: Pin Reducer Semantics
- [ ] Add tests proving:
- `StateField(type="string")` defaults to `wf.std.replace`
- `wf.std.append` preserves append behavior
- `wf.std.merge_object` preserves shallow object merge behavior
- unknown reducer names fail clearly
- exact nested state paths still use their own reducer
- [ ] Run focused core tests and confirm failure before implementation.
### Task 2: Replace Core Merge Strategy With Reducers
- [ ] Replace `merge_strategy` on `StateField` with `reducer`.
- [ ] Add reducer functions for `wf.std.replace`, `wf.std.append`, and `wf.std.merge_object`.
- [ ] Add a registry lookup path that raises for unknown reducer names.
- [ ] Update `write_state_value()` to resolve declared reducers and use `wf.std.replace` for undeclared paths.
- [ ] Run focused core tests and confirm reducer behavior is green.
### Task 3: Migrate Authoring
- [ ] Change `StateFieldMetadata` and `state_field()` to use `reducer`.
- [ ] Preserve nested metadata projection under reducer names.
- [ ] Update authoring/rewrite fixtures from `merge_strategy=` to `reducer=`.
- [ ] Run focused authoring tests and confirm they pass.
### Task 4: Update Docs
- [ ] Replace docs that describe `merge_strategy` with reducer terminology.
- [ ] Update examples to show reducer names, including the default replace reducer.
- [ ] Keep the design point that reducers are pure and source-owned.
### Task 5: Verify
- [ ] Run `uv run --with pytest pytest tests/core tests/authoring tests/rewrite -q`
- [ ] Run `uv run --with pytest pytest -q`
- [ ] Run `uv run basedpyright --level error`
## Non-Goals
- custom user-authored reducer registration through MCP/platform sources
- reducer parameters/configuration
- async reducers
- parallel foreach
- compatibility shims for `merge_strategy`
@@ -0,0 +1,63 @@
# ReducerRef Config Validation Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace string-only reducer references with `ReducerRef(name, config)` and validate reducer config before state mutation.
**Architecture:** Keep string reducer input as shorthand, normalize it to a `ReducerRef`, and add `config_schema` to reducer specs. Runtime reducer application resolves the reducer spec, validates config through the existing JSON Schema backend, then calls reducer functions with `(current, incoming, config)`.
**Tech Stack:** Python, Pydantic, jsonschema, pytest, existing reducer registry.
---
## File Structure
- Modify `src/wf_core/models/reducers.py`
- add `ReducerRef`
- add `config_schema` to `ReducerSpec`
- Modify `src/wf_core/models/schemas.py`
- make `StateField.reducer` a `ReducerRef` with string shorthand parsing
- Modify `src/wf_core/runtime/ops/merges.py`
- reducer callable accepts config
- validate config against reducer spec before merge
- Modify `src/wf_core/runtime/ops/state.py`
- pass `ReducerRef` to reducer application
- Modify `src/wf_artifacts/factory.py`
- infer reducer dependencies from `ReducerRef` objects and dict payloads
- Modify tests for core reducers and artifact dependency inference
## Tasks
### Task 1: Pin ReducerRef Behavior
- [ ] Add tests proving string shorthand normalizes to `ReducerRef(name=..., config={})`.
- [ ] Add tests proving object reducer payloads preserve config.
- [ ] Add tests proving invalid config fails before mutation.
- [ ] Run focused tests and confirm failure before implementation.
### Task 2: Implement ReducerRef and Config Validation
- [ ] Add `ReducerRef`.
- [ ] Add `ReducerSpec.config_schema`.
- [ ] Update reducer callables to accept config.
- [ ] Validate config before calling a reducer.
- [ ] Keep existing no-config reducers accepting `{}` only through their empty config schemas.
### Task 3: Update Artifact Dependency Inference
- [ ] Infer reducer dependency names from string reducers and object reducers.
- [ ] Keep dependency key by reducer name, not by reducer config.
- [ ] Run artifact factory tests.
### Task 4: Verify
- [ ] Run focused core/artifact tests.
- [ ] Run full suite.
- [ ] Run basedpyright.
## Non-Goals
- implementing `modulo_add`
- reducer decorator UX
- configurable reducer factories
- caching configured reducers
@@ -0,0 +1,64 @@
# Reducer Source Inventory Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Represent reducers as first-class source-owned capabilities and expose the built-in reducer catalog through existing source inventory.
**Architecture:** Add a small immutable `ReducerSpec` model in `wf_core`, keep runtime reducer execution where it is, and extend `wf_mcp` capability buckets with reducer ownership metadata. Register the three built-in reducer specs under `wf.std` so inventory becomes honest without adding reducer-authoring UX yet.
**Tech Stack:** Python, dataclasses/Pydantic, pytest, existing capability source inventory.
---
## File Structure
- Create `src/wf_core/models/reducers.py`
- reducer capability metadata
- Modify `src/wf_core/models/__init__.py` and `src/wf_core/__init__.py`
- export `ReducerSpec`
- Modify `src/wf_mcp/broker/service/capability_sources.py`
- add reducer bucket, counts, and inventory listing
- Modify `src/wf_mcp/broker/service/builtins.py`
- define/register built-in reducer specs under `wf.std`
- Modify `tests/wf_mcp/test_service.py`
- assert reducer inventory
- Update `docs/wf_mcp_capability_sources.md`
- document reducers under `wf.std`
- Update `docs/workflow_capabilities.md`
- name reducers as workflow-facing capabilities
## Tasks
### Task 1: Pin Inventory Behavior
- [ ] Add failing tests proving:
- `wf.std` owns built-in reducers
- source status exposes `reducer_count`
- source inventory exposes reducer names
- [ ] Run focused service tests and confirm failure before implementation.
### Task 2: Add ReducerSpec and Source Buckets
- [ ] Add `ReducerSpec` with `name`, `description`, and optional value-shape notes.
- [ ] Export `ReducerSpec` from core.
- [ ] Extend `CapabilityBuckets`, `as_status()`, and `as_inventory()` with reducers.
- [ ] Register `wf.std.replace`, `wf.std.append`, and `wf.std.merge_object` in built-ins.
- [ ] Run focused service tests and confirm they pass.
### Task 3: Update Docs
- [ ] Add reducers to the source vocabulary docs.
- [ ] Clarify that reducers are selected from sources; LLMs are not expected to author reducer code.
### Task 4: Verify
- [ ] Run `uv run --with pytest pytest tests/wf_mcp -q`
- [ ] Run `uv run --with pytest pytest -q`
- [ ] Run `uv run basedpyright --level error`
## Non-Goals
- reducer decorators
- reducer runtime dependency resolution from external sources
- reducer MCP tools
- non-built-in reducer libraries
@@ -0,0 +1,699 @@
# Workflow Draft Surface Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace the disposable draft prototype with the first real MCP-facing workflow draft surface: keyed, patchable JSON that adapts into `wf_authoring.WorkflowBuilder` instead of rebuilding graph semantics itself.
**Architecture:** The new draft layer remains a typed JSON seam in `wf_artifacts`, but delegates graph construction to `wf_authoring`. Draft parsing owns keyed presentation, validation, and stable patch paths; `WorkflowBuilder` owns graph construction. The MCP workflow surface keeps the same tool family while accepting the new draft shape.
**Tech Stack:** Python 3.14, Pydantic v2, `wf_authoring`, `wf_core`, `jsonpatch`, pytest, basedpyright, Ruff.
---
## File Structure
### Create
- `src/wf_artifacts/drafts/models.py`
- concrete Pydantic draft document models
- `src/wf_artifacts/drafts/adapter.py`
- thin JSON-draft-to-`WorkflowBuilder` adapter
- `src/wf_artifacts/drafts/api.py`
- public compile/validate/patch functions and diagnostics
- `tests/artifacts/test_draft_models.py`
- draft document validation
- `tests/artifacts/test_draft_adapter.py`
- keyed step/routes lowering through `WorkflowBuilder`
- `tests/artifacts/test_draft_api.py`
- public compile/validate/patch behavior
### Modify
- `src/wf_artifacts/drafts.py`
- replace module body with compatibility re-exports or remove once imports are updated
- `src/wf_artifacts/__init__.py`
- export public draft API/models from the package
- `src/wf_authoring/builder/core.py`
- add `use_ref(...)` for named external capabilities without local `NodeSpec`s
- `src/wf_authoring/ops/*`
- only if route helpers need a reusable public lowering entrypoint
- `src/wf_mcp/workflow_surface/handlers.py`
- keep using public draft API; no semantic duplication
- `tests/wf_mcp/test_workflow_surface.py`
- update draft fixtures to the new keyed document shape
- `tests/wf_mcp/test_server.py`
- confirm MCP schemas remain plain-object friendly
- `docs/workflow_drafts.md`
- replace prototype examples with the first real draft surface
- `docs/wf_mcp_end_to_end_runbook.md`
- update draft example
- `docs/wf_mcp_operator_manual.md`
- keep draft-first guidance accurate
- `docs/wf_mcp_troubleshooting.md`
- update patch-path examples
## Task 1: Split Draft Code Into Focused Modules
**Files:**
- Create: `src/wf_artifacts/drafts/models.py`
- Create: `src/wf_artifacts/drafts/api.py`
- Create: `src/wf_artifacts/drafts/adapter.py`
- Modify: `src/wf_artifacts/drafts.py`
- Modify: `src/wf_artifacts/__init__.py`
- Test: `tests/artifacts/test_draft_models.py`
- [ ] **Step 1: Write the failing model tests**
```python
from wf_artifacts.drafts import DraftUseStep, WorkflowDraft
def test_workflow_draft_uses_keyed_steps() -> None:
draft = WorkflowDraft.model_validate(
{
"name": "echo",
"input_schema": {},
"state_schema": {"fields": {}},
"output_schema": {},
"start": "echo",
"steps": {
"echo": {
"use": "demo.echo",
"in": {"input.text": "text"},
"out": {"echoed": "state.echoed"},
}
},
"routes": {"echo": {"ok": "__end__"}},
}
)
assert isinstance(draft.steps["echo"], DraftUseStep)
assert draft.steps["echo"].use == "demo.echo"
def test_draft_step_requires_exactly_one_kind_key() -> None:
result = WorkflowDraft.model_validate(
{
"name": "bad",
"input_schema": {},
"state_schema": {"fields": {}},
"output_schema": {},
"start": "bad",
"steps": {
"bad": {
"use": "demo.echo",
"join": {},
}
},
"routes": {},
}
)
```
The second test should be written with `pytest.raises(ValidationError)` and assert the authoring path identifies `steps.bad`.
- [ ] **Step 2: Run tests to verify they fail**
Run:
```bash
uv run --with pytest pytest tests/artifacts/test_draft_models.py -q
```
Expected: import errors or validation failures because keyed draft models do not exist yet.
- [ ] **Step 3: Implement minimal concrete draft models**
Create:
```python
# src/wf_artifacts/drafts/models.py
from __future__ import annotations
from typing import Annotated, Any, Literal
from pydantic import BaseModel, Field, model_validator
JsonObject = dict[str, Any]
STEP_KIND_KEYS = frozenset({"use", "foreach", "interrupt", "join"})
class DraftUseStep(BaseModel):
use: str
in_: dict[str, str] = Field(default_factory=dict, alias="in")
out: dict[str, str] = Field(default_factory=dict)
desc: str | None = None
retry: int | None = Field(default=None, ge=0)
timeout_seconds: int | None = Field(default=None, gt=0)
class DraftForeachPayload(BaseModel):
over: str
as_: str = Field(alias="as")
mode: Literal["serial", "parallel"] = "serial"
on_item_error: Literal["fail", "collect", "skip"] = "fail"
class DraftForeachStep(BaseModel):
foreach: DraftForeachPayload
class DraftInterruptPayload(BaseModel):
kind: str
request: dict[str, str] = Field(default_factory=dict)
resume: dict[str, str] = Field(default_factory=dict)
outcomes: list[str] = Field(default_factory=lambda: ["submitted"])
class DraftInterruptStep(BaseModel):
interrupt: DraftInterruptPayload
class DraftJoinStep(BaseModel):
join: JsonObject = Field(default_factory=dict)
DraftStep = Annotated[
DraftUseStep
| DraftForeachStep
| DraftInterruptStep
| DraftJoinStep,
Field(discriminator=None),
]
class WorkflowDraft(BaseModel):
name: str
input_schema: JsonObject
state_schema: JsonObject
output_schema: JsonObject
start: str
steps: dict[str, DraftStep]
routes: dict[str, dict[str, str]] = Field(default_factory=dict)
@model_validator(mode="before")
@classmethod
def _validate_step_kinds(cls, value: object) -> object:
if not isinstance(value, dict):
return value
steps = value.get("steps")
if not isinstance(steps, dict):
return value
for step_id, payload in steps.items():
if not isinstance(payload, dict):
continue
present = STEP_KIND_KEYS.intersection(payload)
if len(present) != 1:
raise ValueError(
f"steps.{step_id} must contain exactly one step kind key"
)
return value
```
Keep the public import path stable by re-exporting through `src/wf_artifacts/drafts.py` during the transition.
- [ ] **Step 4: Run tests to verify they pass**
Run:
```bash
uv run --with pytest pytest tests/artifacts/test_draft_models.py -q
```
Expected: pass.
- [ ] **Step 5: Commit**
```bash
git add src/wf_artifacts tests/artifacts/test_draft_models.py
git commit -m "refactor: add keyed workflow draft models"
```
## Task 2: Add `use_ref` And Thin Adapter Over `WorkflowBuilder`
**Files:**
- Create: `src/wf_artifacts/drafts/adapter.py`
- Modify only if needed: `src/wf_authoring/builder/core.py`
- Modify only if needed: `src/wf_authoring/ops/*`
- Test: `tests/artifacts/test_draft_adapter.py`
- [ ] **Step 1: Write failing adapter tests**
```python
from wf_artifacts.drafts import WorkflowDraft
from wf_artifacts.drafts.adapter import build_workflow_from_draft
def test_adapter_lowers_keyed_use_steps_and_routes_through_builder() -> None:
draft = WorkflowDraft.model_validate(
{
"name": "echo",
"input_schema": {},
"state_schema": {"fields": {}},
"output_schema": {},
"start": "echo",
"steps": {"echo": {"use": "demo.echo"}},
"routes": {"echo": {"ok": "__end__"}},
}
)
workflow = build_workflow_from_draft(draft)
assert workflow.nodes[0].id == "echo"
assert workflow.nodes[0].node == "demo.echo"
assert workflow.edges[0].from_ == "echo"
assert workflow.edges[0].outcome == "ok"
assert workflow.edges[0].to == "__end__"
def test_builder_use_ref_creates_external_node_use_without_node_def() -> None:
builder = WorkflowBuilder(
"echo",
input_schema={},
state_schema={"fields": {}},
output_schema={},
)
step = builder.use_ref("demo.echo", id="echo")
builder.set_entry_point(step)
builder.connect(step, "ok", "__end__")
workflow = builder.compile()
assert step.node == "demo.echo"
assert workflow.node_defs == []
```
- [ ] **Step 2: Run tests to verify they fail**
Run:
```bash
uv run --with pytest pytest tests/artifacts/test_draft_adapter.py -q
```
Expected: import error because `build_workflow_from_draft` does not exist.
- [ ] **Step 3: Implement the thin adapter**
First add:
```python
def use_ref(
self,
name: str,
*,
id: str | None = None,
in_map: MapArg | None = None,
out_map: MapArg | None = None,
desc: str | None = None,
) -> NodeUse:
...
```
`use_ref` creates a `NodeUse` for an already named external capability and does
not add a local `NodeDef`.
Then implement `build_workflow_from_draft(draft: WorkflowDraft) -> Workflow` so
it:
1. constructs a `WorkflowBuilder`
2. registers each draft step by stable id
3. uses existing `WorkflowBuilder` public methods for:
- `use_ref`
- `foreach`
- `interrupt`
- `join`
4. applies `routes`
5. calls explicit `start(...)`
6. returns `builder.build(...)`
Do **not** invent draft route sugar in this pass.
- [ ] **Step 4: Run tests to verify they pass**
Run:
```bash
uv run --with pytest pytest tests/artifacts/test_draft_adapter.py -q
```
Expected: pass.
- [ ] **Step 5: Commit**
```bash
git add src/wf_artifacts src/wf_authoring tests/artifacts/test_draft_adapter.py
git commit -m "feat: adapt workflow drafts through workflow builder"
```
## Task 3: Replace Prototype Public API
**Files:**
- Create: `src/wf_artifacts/drafts/api.py`
- Modify: `src/wf_artifacts/drafts.py`
- Modify: `src/wf_artifacts/__init__.py`
- Test: `tests/artifacts/test_draft_api.py`
- [ ] **Step 1: Write failing API tests**
```python
from wf_artifacts.drafts import compile_workflow_draft, patch_workflow_draft
def test_compile_workflow_draft_returns_raw_core_shape() -> None:
plan = compile_workflow_draft(_keyed_echo_draft())
assert plan["nodes"][0]["id"] == "echo"
assert plan["nodes"][0]["node"] == "demo.echo"
assert plan["edges"][0]["outcome"] == "ok"
def test_patch_workflow_draft_uses_stable_step_paths() -> None:
result = patch_workflow_draft(
_keyed_echo_draft(),
[
{
"op": "replace",
"path": "/steps/echo/in/input.text",
"value": "message",
}
],
)
assert result["status"] == "valid"
assert result["draft"]["steps"]["echo"]["in"]["input.text"] == "message"
```
- [ ] **Step 2: Run tests to verify they fail**
Run:
```bash
uv run --with pytest pytest tests/artifacts/test_draft_api.py -q
```
Expected: failures because the old prototype API still expects array `steps`.
- [ ] **Step 3: Implement the API**
Create:
```python
# src/wf_artifacts/drafts/api.py
def compile_workflow_draft(draft: JsonObject) -> JsonObject:
parsed = WorkflowDraft.model_validate(draft)
workflow = build_workflow_from_draft(parsed)
return workflow.model_dump(mode="json", by_alias=True, exclude={"node_defs"})
```
Keep:
- `validate_workflow_draft`
- `patch_workflow_draft`
- structured `DraftDiagnostic`
Update diagnostics to use keyed paths such as:
```text
steps.echo.in
routes.echo.error
```
Delete the old array-step prototype code after public tests are green.
- [ ] **Step 4: Run tests to verify they pass**
Run:
```bash
uv run --with pytest pytest tests/artifacts/test_draft_api.py -q
```
Expected: pass.
- [ ] **Step 5: Commit**
```bash
git add src/wf_artifacts tests/artifacts/test_draft_api.py
git commit -m "feat: replace draft prototype with keyed public api"
```
## Task 4: Update MCP Workflow Surface
**Files:**
- Modify: `tests/wf_mcp/test_workflow_surface.py`
- Modify: `tests/wf_mcp/test_server.py`
- Modify only if needed: `src/wf_mcp/workflow_surface/handlers.py`
- [ ] **Step 1: Update the failing MCP tests**
Replace old fixtures like:
```python
"steps": [{"id": "echo", "kind": "use", ...}]
```
with:
```python
"steps": {"echo": {"use": "demo.echo", ...}},
"routes": {"echo": {"ok": "__end__"}},
```
Keep assertions that:
- draft tools still expose plain object schemas to MCP clients
- `create_artifact_from_draft` still saves artifacts
- source binding normalization still works
- missing `wf.std` self-binding diagnostics still work
- [ ] **Step 2: Run tests to verify failures**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_server.py -q
```
Expected: failures wherever MCP handlers still assume the old prototype shape.
- [ ] **Step 3: Make minimal MCP adjustments**
Keep handlers thin:
```python
plan = compile_workflow_draft(draft)
```
No duplicate draft interpretation should appear in `wf_mcp`.
- [ ] **Step 4: Run tests to verify they pass**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_server.py -q
```
Expected: pass.
- [ ] **Step 5: Commit**
```bash
git add src/wf_mcp tests/wf_mcp
git commit -m "feat: accept keyed workflow drafts over mcp"
```
## Task 5: Add Outcome Validation When Capability Contracts Are Available
**Files:**
- Modify: `src/wf_artifacts/drafts/api.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
- [ ] **Step 1: Write failing outcome validation test**
```python
def test_draft_validation_rejects_unknown_capability_outcome_when_spec_is_known() -> None:
handlers = _handlers_with_demo_echo_spec()
draft = _keyed_echo_draft()
draft["routes"]["echo"] = {"typo": "__end__"}
result = asyncio.run(handlers.validate_draft(draft=draft))
assert result["status"] == "invalid"
assert result["diagnostics"][0]["path"] == "routes.echo.typo"
```
- [ ] **Step 2: Run test to verify it fails**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py -q
```
Expected: validation currently accepts the typo.
- [ ] **Step 3: Implement capability-aware outcome validation**
Pass an optional capability lookup into draft validation from MCP handlers.
Rules:
- validate outcome keys for `use` steps only when the capability is resolvable
- if the capability is unknown/unavailable, leave dependency validation to the
later artifact/deployment stages
- diagnostic path must identify the keyed route entry
Do not make `wf_artifacts` depend on `wf_mcp`; define a tiny callable/protocol
interface for lookup instead.
- [ ] **Step 4: Run tests to verify they pass**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py -q
```
Expected: pass.
- [ ] **Step 5: Commit**
```bash
git add src/wf_artifacts src/wf_mcp tests/wf_mcp/test_workflow_surface.py
git commit -m "feat: validate draft routes against known outcomes"
```
## Task 6: Update Documentation
**Files:**
- Modify: `docs/workflow_drafts.md`
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
- Modify: `docs/wf_mcp_operator_manual.md`
- Modify: `docs/wf_mcp_troubleshooting.md`
- [ ] **Step 1: Update docs to the real draft surface**
Replace prototype array examples with keyed examples:
```json
"steps": {
"echo": {
"use": "demo.echo_tool",
"in": {"input.text": "text"},
"out": {"echoed": "state.echoed"}
}
},
"routes": {
"echo": {
"ok": "__end__"
}
}
```
Document:
- exactly-one-kind-key rule
- stable keyed patch paths
- `route` as repeated condition-chain sugar
- `WorkflowBuilder` as the semantic owner beneath the JSON adapter
- outcome strings validated against capability contracts when available
- [ ] **Step 2: Run a targeted docs scan**
Run:
```bash
rg -n '\"steps\": \\[|\"kind\": \"use\"|/steps/0|create_artifact_from_draft' docs
```
Expected:
- no stale prototype examples in current docs
- `create_artifact_from_draft` still documented as the preferred path
- [ ] **Step 3: Commit**
```bash
git add docs
git commit -m "docs: describe keyed workflow draft surface"
```
## Task 7: Full Verification
**Files:**
- No new files
- [ ] **Step 1: Run focused verification**
```bash
uv run --with pytest pytest tests/artifacts tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_server.py -q
```
Expected: pass.
- [ ] **Step 2: Run full project tests**
```bash
uv run --with pytest pytest -q
```
Expected: pass.
- [ ] **Step 3: Run type checking**
```bash
uv run basedpyright --level error
```
Expected: `0 errors`.
- [ ] **Step 4: Run lint**
```bash
uvx ruff check src tests
```
Expected: pass.
- [ ] **Step 5: Commit final cleanup**
```bash
git add .
git commit -m "feat: ship keyed workflow draft authoring surface"
```
## Self-Review
### Spec Coverage
- keyed `steps`: Tasks 1-4
- compact `routes`: Tasks 1-4
- verb-keyed explicit step kinds: Task 1
- saved capability/workflow refs in `use`: Task 2, existing capability refs pass through unchanged
- stable patch paths: Tasks 3 and 6
- `wf_authoring` as semantic owner: Tasks 2 and 6
- outcome validation against declared contracts: Task 5
- prototype replacement rather than migration: Tasks 3, 4, 6
### Placeholder Scan
- no `TBD`
- no unspecified "add validation" placeholders
- every task has exact files, tests, commands, and expected behavior
### Type Consistency
- `WorkflowDraft`, `DraftUseStep`, `build_workflow_from_draft`, and public API
names stay consistent across all tasks
- patch examples use keyed paths consistently
- `routes` stays the only authored outcome-routing section
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,944 @@
# Core Path Bindings Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace loose core path/map strings with typed path objects and canonical list-of-struct node bindings while keeping deprecated shapes parse-compatible.
**Architecture:** Add immutable path value objects in `wf_core.paths`, then introduce canonical binding models in `wf_core.models.steps`. Runtime and validation move to the canonical bindings, while old `in_map`, `input_values`, `out_map`, and dict-shaped state fields are accepted only by model validators.
**Tech Stack:** Python 3.14, Pydantic v2, pytest, jsonschema, basedpyright, ruff.
**Current status as of 2026-05-20:** The implementation has moved past the
original checklist. Typed path values, canonical node bindings, canonical
runtime input/output handling, atomic state patches, canonical validation,
JSON-Schema-native state reducers, schema validation, and authoring canonical
emission are present in the tree. The remaining checklist item is the full
compatibility/regression pass in Task 9. If future code changes touch this area,
prefer adding focused tests to the existing `tests/core/test_*path*`,
`tests/core/test_*mapping*`, and `tests/authoring/test_builder.py` coverage
rather than reimplementing the earlier tasks.
---
## File Structure
- Modify `src/wf_core/paths.py`: own typed graph/state/local path objects and graph path resolution helpers.
- Modify `src/wf_core/local_paths.py`: keep compatibility wrappers over `LocalPath` plus local get/set helpers.
- Modify `src/wf_core/models/steps.py`: add `InputPathBinding`, `InputValueBinding`, `OutputBinding`, and canonical `NodeUse.input` / `NodeUse.output`.
- Modify `src/wf_core/models/conditions.py`: type condition path operands with `GraphSourcePath`.
- Modify `src/wf_core/models/schemas.py`: harden `SchemaRef` and add canonical state field declarations.
- Modify `src/wf_core/runtime/ops/nodes.py`: resolve canonical node input bindings.
- Modify `src/wf_core/runtime/ops/state.py`: apply canonical output bindings through an atomic state patch.
- Modify `src/wf_core/runtime/ops/schemas.py`: expose focused JSON Schema validation helpers.
- Modify `src/wf_core/validation/steps.py`: validate canonical bindings and typed paths.
- Modify `src/wf_authoring/dsl/paths.py`: emit core path objects while preserving ergonomic helpers.
- Modify `src/wf_authoring/dsl/conditions.py`: compile authoring expressions to core typed condition models.
- Add `tests/core/test_path_values.py`: path parsing, serialization, JSON Schema, and error tests.
- Add `tests/core/test_canonical_node_bindings.py`: canonical model parsing and deprecated compatibility tests.
- Add `tests/core/test_atomic_state_patches.py`: output binding, reducer, overlap, and atomicity tests.
- Update existing `tests/core/test_mapping_validation.py`, `tests/core/test_nested_mappings.py`, `tests/core/test_nested_state_paths.py`, and authoring tests as needed.
## Task 1: Add Typed Path Values
**Files:**
- Modify: `src/wf_core/paths.py`
- Modify: `src/wf_core/local_paths.py`
- Create: `tests/core/test_path_values.py`
- [ ] **Step 1: Write path value tests**
Add tests for parsing, string serialization, equality/hashability, invalid segments, root-only graph source reads, and no bare write state:
```python
import pytest
from pydantic import BaseModel, ValidationError
from wf_core.paths import GraphSourcePath, LocalPath, PathResolutionError, StatePath
def test_graph_source_path_accepts_root_and_nested_paths():
assert str(GraphSourcePath.parse("state")) == "state"
assert str(GraphSourcePath.parse("input.user")) == "input.user"
assert str(GraphSourcePath.context("loop_item")) == "context.loop_item"
def test_state_path_rejects_bare_state_write_target():
with pytest.raises(PathResolutionError, match="state path"):
StatePath.parse("state")
def test_local_path_supports_root_marker():
assert str(LocalPath.root()) == "."
assert str(LocalPath.of("user.name")) == "user.name"
@pytest.mark.parametrize("raw", ["", "state.", "state.items.0", "state.user-name"])
def test_paths_reject_invalid_segments(raw: str):
with pytest.raises(PathResolutionError):
GraphSourcePath.parse(raw)
def test_path_objects_are_hashable():
paths = {StatePath.of("person.name"), StatePath.of("person.name")}
assert len(paths) == 1
def test_pydantic_accepts_path_strings_and_serializes_strings():
class Payload(BaseModel):
source: GraphSourcePath
target: StatePath
local: LocalPath
payload = Payload.model_validate(
{"source": "input.user", "target": "state.person", "local": "user"}
)
assert payload.source == GraphSourcePath.input("user")
assert payload.model_dump(mode="json")["target"] == "state.person"
def test_pydantic_rejects_bad_path_string():
class Payload(BaseModel):
source: GraphSourcePath
with pytest.raises(ValidationError):
Payload.model_validate({"source": "output.foo"})
```
- [ ] **Step 2: Run path tests to verify they fail**
Run: `uv run --with pytest pytest tests/core/test_path_values.py -q`
Expected: failures because `GraphSourcePath`, `StatePath`, and `LocalPath` classes do not exist or do not validate strictly.
- [ ] **Step 3: Implement path value classes**
In `src/wf_core/paths.py`, add frozen dataclasses and shared parsing helpers. Keep existing helper function names as compatibility wrappers where practical.
Implementation shape:
```python
from dataclasses import dataclass
import re
from typing import Any, ClassVar, Literal
SEGMENT_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
@dataclass(frozen=True)
class LocalPath:
"""Node-local payload path. `.` means the whole local payload."""
parts: tuple[str, ...]
@classmethod
def root(cls) -> "LocalPath":
return cls(())
@classmethod
def of(cls, *fragments: str) -> "LocalPath":
return cls(_parse_fragments(*fragments, allow_empty=False))
@classmethod
def parse(cls, raw: str) -> "LocalPath":
if raw == ".":
return cls.root()
return cls.of(raw)
def __str__(self) -> str:
return "." if not self.parts else ".".join(self.parts)
```
Also add:
```python
GraphRoot = Literal["input", "state", "context"]
@dataclass(frozen=True)
class GraphSourcePath:
"""Readable workflow graph path rooted at input, state, or context."""
root: GraphRoot
parts: tuple[str, ...] = ()
@classmethod
def parse(cls, raw: str) -> "GraphSourcePath": ...
@classmethod
def input(cls, *fragments: str) -> "GraphSourcePath": ...
@classmethod
def state(cls, *fragments: str) -> "GraphSourcePath": ...
@classmethod
def context(cls, *fragments: str) -> "GraphSourcePath": ...
```
And:
```python
@dataclass(frozen=True)
class StatePath:
"""Writable workflow state path. Bare `state` is intentionally invalid."""
parts: tuple[str, ...]
@classmethod
def parse(cls, raw: str) -> "StatePath":
parsed = GraphSourcePath.parse(raw)
if parsed.root != "state" or not parsed.parts:
raise PathResolutionError("expected state path such as state.foo")
return cls(parsed.parts)
@classmethod
def of(cls, *fragments: str) -> "StatePath": ...
```
Add Pydantic `__get_pydantic_core_schema__` and `__get_pydantic_json_schema__` hooks for each class so strings validate into objects and serialize back to strings.
- [ ] **Step 4: Update local path wrappers**
In `src/wf_core/local_paths.py`, keep public functions but delegate parsing to `LocalPath.parse`:
```python
def split_local_path(path: str | LocalPath) -> list[str]:
"""Split one node-local path, accepting the new typed path object."""
parsed = path if isinstance(path, LocalPath) else LocalPath.parse(path)
return list(parsed.parts)
```
Update `paths_overlap` and `has_overlapping_paths` to accept `str | LocalPath`.
- [ ] **Step 5: Run path tests**
Run: `uv run --with pytest pytest tests/core/test_path_values.py -q`
Expected: all tests in `test_path_values.py` pass.
## Task 2: Add Canonical Node Binding Models
**Files:**
- Modify: `src/wf_core/models/steps.py`
- Test: `tests/core/test_canonical_node_bindings.py`
- [ ] **Step 1: Write canonical binding tests**
Create `tests/core/test_canonical_node_bindings.py`:
```python
import pytest
from pydantic import ValidationError
from wf_core.models.steps import NodeUse
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
def test_node_use_accepts_canonical_input_and_output_bindings():
node = NodeUse.model_validate(
{
"id": "echo",
"type": "node",
"node": "echo",
"input": [
{"target": "message", "path": "input.message"},
{"target": "mode", "value": None},
],
"output": [{"source": "echoed", "target": "state.echoed"}],
}
)
assert node.input[0].target == LocalPath.of("message")
assert node.input[0].path == GraphSourcePath.input("message")
assert node.input[1].value is None
assert node.output[0].target == StatePath.of("echoed")
def test_node_use_converts_old_maps_to_canonical_bindings():
node = NodeUse.model_validate(
{
"id": "echo",
"type": "node",
"node": "echo",
"in_map": {"input.message": "message"},
"input_values": {"mode": "fast"},
"out_map": {"echoed": "state.echoed"},
}
)
dumped = node.model_dump(mode="json")
assert "in_map" not in dumped
assert "input_values" not in dumped
assert "out_map" not in dumped
assert dumped["input"][0]["path"] == "input.message"
assert dumped["input"][1]["value"] == "fast"
assert dumped["output"][0]["target"] == "state.echoed"
def test_node_use_rejects_mixed_old_and_new_binding_styles():
with pytest.raises(ValidationError):
NodeUse.model_validate(
{
"id": "echo",
"type": "node",
"node": "echo",
"input": [{"target": "message", "path": "input.message"}],
"in_map": {"input.other": "other"},
}
)
def test_input_binding_rejects_path_and_value_together():
with pytest.raises(ValidationError):
NodeUse.model_validate(
{
"id": "bad",
"type": "node",
"node": "bad",
"input": [
{"target": "message", "path": "input.message", "value": "x"}
],
}
)
```
- [ ] **Step 2: Run binding tests to verify they fail**
Run: `uv run --with pytest pytest tests/core/test_canonical_node_bindings.py -q`
Expected: failures because canonical binding fields do not exist yet.
- [ ] **Step 3: Implement binding models**
In `src/wf_core/models/steps.py`, add:
```python
from pydantic import BaseModel, ConfigDict, Field, model_validator
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
class InputPathBinding(BaseModel):
"""Map one graph source path into one node-local input path."""
model_config = ConfigDict(extra="forbid")
target: LocalPath
path: GraphSourcePath
class InputValueBinding(BaseModel):
"""Map one static JSON-compatible value into one node-local input path."""
model_config = ConfigDict(extra="forbid")
target: LocalPath
value: object
InputBinding = Annotated[
InputPathBinding | InputValueBinding,
Field(union_mode="left_to_right"),
]
class OutputBinding(BaseModel):
"""Map one node-local output path into one workflow state path."""
model_config = ConfigDict(extra="forbid")
source: LocalPath
target: StatePath
```
Update `NodeUse`:
```python
class NodeUse(BaseModel):
...
input: list[InputBinding] = Field(default_factory=list)
output: list[OutputBinding] = Field(default_factory=list)
@model_validator(mode="before")
@classmethod
def _coerce_deprecated_maps(cls, data: object) -> object:
...
```
The validator should:
- If `input` or `output` is present, reject any of `in_map`, `input_values`, `out_map`.
- Convert `input_values` entries to `{"target": key, "value": value}` preserving order.
- Convert `in_map` entries to `{"target": destination, "path": source}` preserving order.
- Convert `out_map` entries to `{"source": source, "target": destination}` preserving order.
- Remove old keys from the normalized data.
- [ ] **Step 4: Run binding tests**
Run: `uv run --with pytest pytest tests/core/test_canonical_node_bindings.py -q`
Expected: all tests in `test_canonical_node_bindings.py` pass.
## Task 3: Move Runtime Node Input Resolution To Canonical Bindings
**Files:**
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_nested_mappings.py`
- Test: `tests/core/test_canonical_node_bindings.py`
- [ ] **Step 1: Add runtime tests for canonical input binding behavior**
In `tests/core/test_nested_mappings.py`, add a test that builds the existing minimal workflow style but uses `input` / `output` instead of old maps:
```python
def test_canonical_bindings_resolve_input_values_and_paths():
workflow = Workflow.model_validate(
{
"name": "canonical",
"input_schema": {"type": "object", "properties": {"message": {"type": "string"}}},
"state_schema": {"fields": {"echoed": {"type": "string"}}},
"output_schema": {"type": "object", "properties": {"echoed": {"type": "string"}}},
"start": "echo",
"node_defs": [
{
"name": "echo",
"input_schema": {
"type": "object",
"properties": {"message": {"type": "string"}, "mode": {"type": "string"}},
"required": ["message", "mode"],
},
"output_schema": {"type": "object", "properties": {"echoed": {"type": "string"}}},
"outcomes": ["ok"],
}
],
"nodes": [
{
"id": "echo",
"type": "node",
"node": "echo",
"input": [
{"target": "message", "path": "input.message"},
{"target": "mode", "value": "fast"},
],
"output": [{"source": "echoed", "target": "state.echoed"}],
}
],
"edges": [{"from": "echo", "outcome": "ok", "to": "__end__"}],
}
)
result = execute_workflow(
workflow,
{"message": "hi"},
registry={"echo": lambda payload, _ctx: {"echoed": f"{payload['mode']}:{payload['message']}"}},
)
assert result.output["echoed"] == "fast:hi"
```
- [ ] **Step 2: Run the focused test to verify failure**
Run: `uv run --with pytest pytest tests/core/test_nested_mappings.py::test_canonical_bindings_resolve_input_values_and_paths -q`
Expected: failure because runtime still reads `node.input_values`, `node.in_map`, and `node.out_map`.
- [ ] **Step 3: Update `_resolve_node_execution`**
In `src/wf_core/runtime/ops/nodes.py`, import binding classes and use `node.input`.
Implementation shape:
```python
from wf_core.models.steps import InputPathBinding, InputValueBinding
for binding in node.input:
if isinstance(binding, InputValueBinding):
value = binding.value
else:
value = safe_resolve_path(
str(binding.path),
state=run.state,
workflow_input=run.workflow_input,
context=context_values,
)
set_local_value(resolved_input, binding.target, value)
```
`set_local_value` should accept `LocalPath` after Task 1.
- [ ] **Step 4: Run canonical runtime test**
Run: `uv run --with pytest pytest tests/core/test_nested_mappings.py::test_canonical_bindings_resolve_input_values_and_paths -q`
Expected: pass.
## Task 4: Move Runtime Output Writes To Canonical Bindings And Atomic Patches
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_atomic_state_patches.py`
- [ ] **Step 1: Write atomic patch tests**
Create `tests/core/test_atomic_state_patches.py`:
```python
import pytest
from wf_core.errors import WorkflowExecutionError
from wf_core.models.workflow import Workflow
from wf_core.runtime.ops.state import apply_output_bindings
def _workflow() -> Workflow:
return Workflow.model_validate(
{
"name": "patch",
"input_schema": {"type": "object", "properties": {}},
"state_schema": {
"fields": {
"person": {"type": "object"},
"person.name": {"type": "string"},
}
},
"output_schema": {"type": "object", "properties": {}},
"start": "n",
"nodes": [],
"edges": [],
}
)
def test_output_bindings_commit_patch_atomically():
workflow = _workflow()
state = {"person": {"name": "old"}}
with pytest.raises(WorkflowExecutionError):
apply_output_bindings(
workflow,
[
{"source": "person.name", "target": "state.person.name"},
{"source": "missing", "target": "state.person.extra"},
],
{"person": {"name": "new"}},
state,
)
assert state["person"]["name"] == "old"
def test_output_bindings_reject_overlapping_write_targets():
workflow = _workflow()
state = {}
with pytest.raises(WorkflowExecutionError, match="overlapping"):
apply_output_bindings(
workflow,
[
{"source": "person", "target": "state.person"},
{"source": "person.name", "target": "state.person.name"},
],
{"person": {"name": "Ada"}},
state,
)
```
- [ ] **Step 2: Run atomic patch tests to verify failure**
Run: `uv run --with pytest pytest tests/core/test_atomic_state_patches.py -q`
Expected: failure because `apply_output_bindings` does not exist.
- [ ] **Step 3: Implement `apply_output_bindings`**
In `src/wf_core/runtime/ops/state.py`, add a canonical function:
```python
from wf_core.models.steps import OutputBinding
from wf_core.paths import StatePath
def apply_output_bindings(
workflow: Workflow,
bindings: Sequence[OutputBinding],
node_output: dict[str, Any],
state: dict[str, Any],
reducers: Mapping[str, ReducerDefinition] | None = None,
) -> dict[str, Any]:
"""Prepare and commit one atomic state patch from canonical output bindings."""
```
Function behavior:
- Validate no overlapping `binding.target`.
- Resolve every `binding.source` from `node_output` first.
- Build a prepared patch keyed by `StatePath`.
- Compute reducers into prepared merged values without mutating `state`.
- Commit all prepared values only after all prior steps succeed.
- Return JSON-friendly `dict[str, Any]` state changes using `str(path)` keys for now, until trace is separately migrated.
Keep `apply_output_map` as a compatibility wrapper that converts old map entries into `OutputBinding` and calls `apply_output_bindings`.
- [ ] **Step 4: Update node finalization**
In `src/wf_core/runtime/ops/nodes.py`, call `apply_output_bindings(workflow, node.output, result.output, run.state, reducers=reducers)` instead of `apply_output_map(...)`.
- [ ] **Step 5: Run state patch tests**
Run: `uv run --with pytest pytest tests/core/test_atomic_state_patches.py tests/core/test_nested_mappings.py -q`
Expected: pass.
## Task 5: Update Validation For Canonical Bindings
**Files:**
- Modify: `src/wf_core/validation/steps.py`
- Test: `tests/core/test_mapping_validation.py`
- Test: `tests/core/test_canonical_node_bindings.py`
- [ ] **Step 1: Add validation tests for canonical fields**
In `tests/core/test_mapping_validation.py`, add tests for invalid source paths, invalid destination paths, overlapping local input targets, and overlapping state output targets using canonical `input` / `output`.
Example:
```python
def test_validate_workflow_reports_overlapping_canonical_output_targets():
workflow = workflow_with_node(
node_use={
"id": "n",
"type": "node",
"node": "n",
"output": [
{"source": "person", "target": "state.person"},
{"source": "person.name", "target": "state.person.name"},
],
}
)
report = workflow.validate_structure()
assert any(issue.code == ValidationIssueCode.INVALID_DESTINATION_PATH for issue in report.issues)
```
Use the existing helper style in `tests/core/test_mapping_validation.py` rather than inventing a second full workflow factory if one already exists.
- [ ] **Step 2: Run mapping validation tests**
Run: `uv run --with pytest pytest tests/core/test_mapping_validation.py -q`
Expected: new canonical validation tests fail until validation reads `node.input` / `node.output`.
- [ ] **Step 3: Update `validate_node_use`**
In `src/wf_core/validation/steps.py`:
- Iterate `node.input`.
- For `InputValueBinding`, validate target local root against node input schema.
- For `InputPathBinding`, validate target and source graph path.
- Iterate `node.output`.
- Validate output source local root against node output schema.
- Validate destination `StatePath`.
- Use typed overlap helpers instead of raw map values.
- Keep issue paths readable, e.g. `nodes[0].input[1].target`.
- [ ] **Step 4: Run validation tests**
Run: `uv run --with pytest pytest tests/core/test_mapping_validation.py tests/core/test_canonical_node_bindings.py -q`
Expected: pass.
## Task 6: Add Canonical State Schema Fields
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Modify: `src/wf_core/runtime/ops/state.py`
- Modify: `src/wf_core/validation/steps.py`
- Test: `tests/core/test_nested_state_paths.py`
- Test: `tests/core/test_schema_validation.py`
- [ ] **Step 1: Write state schema canonical shape tests**
In `tests/core/test_nested_state_paths.py`, add:
```python
from wf_core.models.schemas import StateSchema
from wf_core.paths import StatePath
def test_state_schema_accepts_canonical_field_list():
schema = StateSchema.model_validate(
{
"fields": [
{"path": "state.person", "type": "object"},
{"path": "state.person.name", "type": "string", "reducer": "wf.std.replace"},
]
}
)
assert schema.fields[0].path == StatePath.of("person")
assert schema.field_map()["person.name"].type == "string"
def test_state_schema_accepts_deprecated_dict_shape():
schema = StateSchema.model_validate(
{"fields": {"person.name": {"type": "string"}}}
)
assert schema.model_dump(mode="json")["fields"][0]["path"] == "state.person.name"
```
- [ ] **Step 2: Run state schema tests to verify failure**
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py -q`
Expected: failure because `StateSchema.fields` is still a dict.
- [ ] **Step 3: Implement canonical `StateFieldDecl`**
In `src/wf_core/models/schemas.py`:
```python
class StateFieldDecl(BaseModel):
"""One declared state path plus validation and reducer metadata."""
path: StatePath
schema: SchemaRef = Field(default_factory=lambda: SchemaRef(type="object"))
reducer: ReducerRef = Field(default_factory=lambda: ReducerRef(name="wf.std.replace"))
trace: bool = True
default: Any = None
```
Preserve compatibility for old `type` directly on the field:
- For old dict values like `{"type": "string"}`, convert to `{"schema": {"type": "string"}}`.
- For canonical values, allow either `schema` or simple `type` as input if that keeps existing tests stable.
Update `StateSchema`:
```python
class StateSchema(BaseModel):
fields: list[StateFieldDecl] = Field(default_factory=list)
def field_map(self) -> dict[str, StateFieldDecl]:
return {".".join(field.path.parts): field for field in self.fields}
```
Add a model validator to accept old dict shape and normalize to list.
- [ ] **Step 4: Update callers of `workflow.state_schema.fields`**
Search: `rg 'state_schema\\.fields|\\.fields\\.get|set\\(workflow\\.state_schema\\.fields\\)' src tests`
Update code to use `workflow.state_schema.field_map()` when it needs lookup by rootless path.
Important updates:
- `src/wf_core/runtime/ops/state.py`
- `src/wf_core/validation/steps.py`
- any authoring or artifact code constructing state field maps.
- [ ] **Step 5: Run state schema tests**
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py -q`
Expected: pass.
## Task 7: Harden SchemaRef With JSON Schema Validation
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Modify: `src/wf_core/runtime/ops/schemas.py`
- Test: `tests/core/test_schema_validation.py`
- [ ] **Step 1: Add schema validation tests**
In `tests/core/test_schema_validation.py`, add tests:
```python
import pytest
from pydantic import ValidationError
from wf_core.models.schemas import SchemaRef
def test_schema_ref_accepts_valid_json_schema_with_defs():
schema = SchemaRef.model_validate(
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"$defs": {"Name": {"type": "string"}},
"properties": {"name": {"$ref": "#/$defs/Name"}},
}
)
assert schema.model_extra["$defs"]["Name"]["type"] == "string"
def test_schema_ref_rejects_invalid_json_schema():
with pytest.raises(ValidationError):
SchemaRef.model_validate({"type": 123})
```
- [ ] **Step 2: Run schema tests to verify failure**
Run: `uv run --with pytest pytest tests/core/test_schema_validation.py -q`
Expected: invalid schema is currently accepted.
- [ ] **Step 3: Add `jsonschema` validation**
In `src/wf_core/models/schemas.py`, import:
```python
from jsonschema import SchemaError
from jsonschema.validators import Draft202012Validator, validator_for
from pydantic import model_validator
```
Add an after validator to `SchemaRef`:
```python
@model_validator(mode="after")
def _validate_json_schema(self) -> "SchemaRef":
raw = self.model_dump(mode="python", exclude_none=True)
validator_cls = validator_for(raw, default=Draft202012Validator)
try:
validator_cls.check_schema(raw)
except SchemaError as exc:
raise ValueError(f"invalid JSON Schema: {exc.message}") from exc
return self
```
- [ ] **Step 4: Run schema tests**
Run: `uv run --with pytest pytest tests/core/test_schema_validation.py -q`
Expected: pass.
## Task 8: Update Authoring Helpers To Emit Canonical Bindings
**Files:**
- Modify: `src/wf_authoring/dsl/paths.py`
- Modify: `src/wf_authoring/dsl/conditions.py`
- Modify: `src/wf_authoring/builder/core.py`
- Test: `tests/authoring/test_builder.py`
- Test: `tests/authoring/test_conditions.py`
- Test: `tests/authoring/test_control_flow_examples.py`
- [ ] **Step 1: Add authoring tests for canonical dumps**
In `tests/authoring/test_builder.py`, add a test that builds a workflow and asserts the dumped node uses canonical `input` / `output`, not old maps:
```python
def test_builder_emits_canonical_node_bindings():
workflow = (
WorkflowBuilder("canonical")
.schemas(
input_schema={"type": "object", "properties": {"message": {"type": "string"}}},
state_schema={"fields": {"echoed": {"type": "string"}}},
output_schema={"type": "object", "properties": {"echoed": {"type": "string"}}},
)
.use(echo_node, id="echo", in_map={"input.message": "message"}, out_map={"echoed": "state.echoed"})
.start_at("echo")
.end("echo", "ok")
.build()
)
dumped_node = workflow.model_dump(mode="json")["nodes"][0]
assert "input" in dumped_node
assert "output" in dumped_node
assert "in_map" not in dumped_node
assert "out_map" not in dumped_node
```
Adapt helper names to the current builder API in the file.
- [ ] **Step 2: Run authoring builder tests**
Run: `uv run --with pytest pytest tests/authoring/test_builder.py tests/authoring/test_conditions.py -q`
Expected: new canonical dump test may fail until builder emits or model normalizes canonical shapes.
- [ ] **Step 3: Update path/condition authoring wrappers**
In `src/wf_authoring/dsl/paths.py`, make ergonomic helpers return wrappers around core path values or values accepted by core models. Preserve existing public behavior where possible:
```python
def state_path(*parts: str) -> GraphPath:
return GraphPath(str(GraphSourcePath.state(*parts)))
```
In `src/wf_authoring/dsl/conditions.py`, make `PathExpr` compile using `GraphSourcePath.parse` for `PathOperand`.
- [ ] **Step 4: Update builder to rely on canonical model normalization**
In `src/wf_authoring/builder/core.py`, either emit canonical binding dicts directly or keep passing old maps into `NodeUse.model_validate`. Prefer direct canonical emission where the builder already has enough structure.
Do not remove user-facing `in_map` / `out_map` builder parameters in this pass.
- [ ] **Step 5: Run authoring tests**
Run: `uv run --with pytest pytest tests/authoring -q`
Expected: authoring tests pass.
## Task 9: Full Compatibility And Regression Pass
**Files:**
- Modify docs/examples only if tests show stale serialized shapes.
- Test: full repo.
- [ ] **Step 1: Run core tests**
Run: `uv run --with pytest pytest tests/core tests/authoring tests/rewrite -q`
Expected: pass.
- [ ] **Step 2: Run artifact and MCP workflow-surface tests**
Run: `uv run --with pytest pytest tests/artifacts tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_workflow_wrappers.py tests/wf_mcp/test_mcp_workflow_surface_example.py -q`
Expected: pass.
- [ ] **Step 3: Run full test suite**
Run: `uv run --with pytest pytest -q`
Expected: pass, allowing any existing intentionally skipped environment-dependent tests.
- [ ] **Step 4: Run static checks**
Run:
```bash
uvx ruff check
uv run basedpyright --level error
```
Expected: ruff passes and basedpyright reports 0 errors.
- [ ] **Step 5: Format touched files**
Run:
```bash
uvx ruff format src/wf_core src/wf_authoring tests/core tests/authoring
```
Expected: files format cleanly.
## Self-Review Notes
- Spec coverage: typed paths, canonical bindings, parse-only compatibility, null/missing semantics, dynamic traversal deferral, state patch atomicity, reducer behavior, JSON Schema validation, authoring updates, and tracing shape are covered. Full trace migration is intentionally not implemented beyond returning string-keyed `state_changes` for compatibility.
- Placeholder scan: this plan avoids `TBD` and names concrete files, tests, commands, and behavior.
- Type consistency: `LocalPath`, `GraphSourcePath`, `StatePath`, `InputPathBinding`, `InputValueBinding`, `OutputBinding`, and `StateFieldDecl` are introduced before later tasks use them.
@@ -0,0 +1,262 @@
# JSON Schema State Reducers Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make `Workflow.state_schema` a normal JSON Schema object, with `reducer` as an explicit workflow extension keyword on field schemas.
**Architecture:** `StateSchema` should validate as JSON Schema first, then expose helper indexes for workflow runtime metadata. Runtime reducer lookup should compile from `properties` paths instead of requiring a separate path declaration list. Legacy `fields` inputs remain parse-only compatibility during the transition.
**Tech Stack:** Python, Pydantic v2, `jsonschema`, `wf_core` path models, pytest, basedpyright, ruff.
---
### Task 1: Add Canonical State Schema Tests
**Files:**
- Modify: `tests/core/test_nested_state_paths.py`
- Modify: `tests/core/test_schema_validation.py`
- [ ] **Step 1: Add a test for JSON Schema property reducers**
```python
def test_state_schema_uses_json_schema_properties_as_canonical_shape() -> None:
schema = StateSchema.model_validate(
{
"type": "object",
"properties": {
"person": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Display name",
"reducer": "wf.std.replace",
}
},
},
"count": {"type": "integer", "reducer": "wf.std.add"},
},
}
)
fields = schema.field_map()
assert fields["person.name"].validation_schema.type == "string"
assert fields["person.name"].reducer == ReducerRef(name="wf.std.replace")
assert fields["count"].reducer == ReducerRef(name="wf.std.add")
```
- [ ] **Step 2: Add a dump test proving the canonical output is still JSON Schema**
```python
def test_state_schema_dumps_canonical_json_schema_with_reducer_keyword() -> None:
schema = StateSchema.model_validate(
{
"type": "object",
"properties": {
"count": {"type": "integer", "reducer": "wf.std.add"}
},
}
)
dumped = schema.model_dump(mode="json")
assert dumped["type"] == "object"
assert dumped["properties"]["count"]["type"] == "integer"
assert dumped["properties"]["count"]["reducer"] == "wf.std.add"
Draft202012Validator.check_schema(dumped)
```
- [ ] **Step 3: Add a runtime reducer lookup test from canonical schema**
```python
def test_exact_nested_state_path_uses_reducer_from_json_schema_property() -> None:
workflow = _workflow_from_state_schema(
StateSchema.model_validate(
{
"type": "object",
"properties": {
"person": {
"type": "object",
"properties": {
"tags": {"type": "array", "reducer": "wf.std.append"}
},
}
},
}
)
)
state = {"person": {"tags": ["seed"]}}
write_state_value(workflow, state, "state.person.tags", ["next"])
assert state["person"]["tags"] == ["seed", "next"]
```
- [ ] **Step 4: Run focused tests and confirm failures**
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py -q`
Expected: new tests fail because `StateSchema` still serializes as `fields: [...]` and reducer lookup is compiled from field declarations only.
### Task 2: Implement JSON-Schema-Native `StateSchema`
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- [ ] **Step 1: Make `StateSchema` inherit JSON Schema fields directly**
`StateSchema` should expose common JSON Schema object fields:
```python
title: str | None = None
type: str | list[str] | None = "object"
properties: dict[str, Any] = Field(default_factory=dict)
required: list[str] = Field(default_factory=list)
```
- [ ] **Step 2: Preserve legacy `fields` as parse-only input**
Keep accepting:
```json
{
"fields": [
{ "path": "state.count", "type": "integer", "reducer": "wf.std.add" }
]
}
```
and:
```json
{ "fields": { "count": { "type": "integer", "reducer": "wf.std.add" } } }
```
by converting both into:
```json
{
"type": "object",
"properties": { "count": { "type": "integer", "reducer": "wf.std.add" } }
}
```
- [ ] **Step 3: Add `field_map()` as an internal compiled index**
`field_map()` should walk explicit object `properties` and return `StateFieldDecl` values keyed by rootless state path. It must:
- include every explicit property path
- parse `reducer` with `ReducerRef`
- default missing reducer to `wf.std.replace`
- preserve `trace` and `default` workflow extension keywords
- remove workflow extension keywords from `StateFieldDecl.validation_schema`
- [ ] **Step 4: Validate JSON Schema and extension keyword types**
Use `SchemaRef`/`jsonschema` validation for the complete state schema. Add explicit validation that:
- `reducer` is a string or `ReducerRef`-compatible object
- `trace` is a boolean when present
- `default` is allowed as JSON Schema/default metadata
### Task 3: Update Artifact Reducer Extraction
**Files:**
- Modify: `src/wf_artifacts/factory.py`
- [ ] **Step 1: Extract reducer dependencies from `state_schema.properties`**
Add a helper that walks explicit JSON Schema properties and yields reducer payloads from every property schema.
- [ ] **Step 2: Keep legacy `fields` extraction only as compatibility**
If `state_schema.fields` exists in old artifacts, continue reading it. Prefer canonical `properties` when present.
- [ ] **Step 3: Add tests through existing workflow surface/artifact tests**
Use an existing artifact/dependency test and assert a reducer declared at:
```json
state_schema.properties.count.reducer
```
is included in required capabilities.
### Task 4: Update Authoring Conversion
**Files:**
- Modify: `src/wf_authoring/schemas.py`
- Modify: `tests/authoring/test_schemas.py`
- [ ] **Step 1: Attach reducer metadata directly to generated property schemas**
When `state_schema_from(BaseModel)` sees `Annotated[..., state_field(reducer=...)]`, inject `reducer` and `trace` into that property schema instead of building a separate field map.
- [ ] **Step 2: Preserve model JSON Schema as the state schema**
Return `StateSchema.model_validate(schema_with_reducer_keywords)` so generated state schema remains JSON Schema-shaped.
### Task 5: Update Docs and Examples
**Files:**
- Modify: `docs/core_state_mapping_and_merge.md`
- Modify: `docs/workflow_drafts.md`
- Modify: `docs/wf_mcp_operator_manual.md`
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
- Modify: `examples/raw_canonical_workflow.py`
- [ ] **Step 1: Replace canonical `fields: [...]` examples**
Use JSON Schema:
```json
{
"type": "object",
"properties": {
"count": {
"type": "integer",
"description": "Counter value",
"reducer": "wf.std.add"
}
}
}
```
- [ ] **Step 2: Document extension semantics**
State clearly that `reducer` is not standard JSON Schema behavior. JSON Schema validators ignore it; `wf_core` reads it for workflow state writes.
### Task 6: Verification
**Files:**
- All touched files
- [ ] **Step 1: Run focused tests**
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py tests/authoring/test_schemas.py -q`
- [ ] **Step 2: Run full tests**
Run: `uv run --with pytest pytest -q`
- [ ] **Step 3: Run static checks**
Run:
```bash
uvx ruff check
uv run basedpyright --level error
```
---
## Self-Review
- Spec coverage: covers canonical JSON Schema state shape, reducer extension keyword, compatibility, runtime lookup, artifact dependency extraction, authoring generation, docs, and verification.
- Placeholder scan: no placeholders remain.
- Type consistency: `StateSchema`, `StateFieldDecl`, `ReducerRef`, and `SchemaRef` names match current code.
@@ -0,0 +1,280 @@
# Typed Source Artifact Contracts Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace persisted dict-key source/capability contract shapes with explicit list-of-struct models backed by `SourceRef` and `CapabilityRef`.
**Architecture:** Keep dot-joined strings as the JSON wire format for refs, but make Python/Pydantic fields use first-class ref objects. Accept old dict shapes as parse-only compatibility and dump the new list shapes. Runtime code should use helper indexes such as `binding_map()` and `required_capability_map()` instead of depending on serialized dict keys.
**Tech Stack:** Python, Pydantic v2, `wf_platform` refs, `wf_artifacts` models, pytest, ruff, basedpyright.
---
### Task 1: Make Platform Refs Pydantic Boundary Types
**Files:**
- Modify: `src/wf_platform/refs.py`
- Modify: `tests/refs/test_platform_refs.py`
- [ ] **Step 1: Add tests for Pydantic validation and serialization**
Add tests that prove:
```python
class Payload(BaseModel):
source: SourceRef
capability: CapabilityRef
payload = Payload.model_validate(
{"source": "demo.personal", "capability": "demo.personal.echo_tool"}
)
assert payload.source == SourceRef.parse("demo.personal")
assert payload.capability == CapabilityRef.parse("demo.personal.echo_tool")
assert payload.model_dump(mode="json") == {
"source": "demo.personal",
"capability": "demo.personal.echo_tool",
}
```
- [ ] **Step 2: Implement Pydantic core-schema hooks**
Use `__get_pydantic_core_schema__` on `SourceRef` and `CapabilityRef` so both accept existing instances or strings and serialize back to strings.
- [ ] **Step 3: Tighten segment validation modestly**
Reject whitespace-only refs and empty segments. Do not over-restrict valid MCP/source names yet; external systems can use dashes, underscores, and other non-empty string segments.
### Task 2: Convert Deployment Bindings to List-of-Struct
**Files:**
- Modify: `src/wf_artifacts/models.py`
- Modify: `tests/artifacts/test_models.py`
- Modify: `tests/artifacts/test_store.py`
- [ ] **Step 1: Add `SourceBinding`**
```python
class SourceBinding(BaseModel):
logical_source: SourceRef
concrete_source: SourceRef
```
- [ ] **Step 2: Change `WorkflowDeployment.bindings`**
Canonical model field:
```python
bindings: list[SourceBinding] = Field(default_factory=list)
```
Parse-only compatibility:
```json
{ "bindings": { "demo": "demo.personal" } }
```
should normalize to:
```json
{
"bindings": [{ "logical_source": "demo", "concrete_source": "demo.personal" }]
}
```
- [ ] **Step 3: Add `binding_map()`**
```python
def binding_map(self) -> dict[str, str]:
return {
str(binding.logical_source): str(binding.concrete_source)
for binding in self.bindings
}
```
Reject duplicate `logical_source` values during validation.
### Task 3: Convert Required Capabilities to List-of-Struct
**Files:**
- Modify: `src/wf_artifacts/models.py`
- Modify: `src/wf_artifacts/factory.py`
- Modify: `src/wf_artifacts/references.py`
- Modify: `tests/artifacts/test_models.py`
- Modify: `tests/artifacts/test_factory.py`
- [ ] **Step 1: Update `RequiredCapability`**
Canonical field:
```python
ref: CapabilityRef
```
Keep compatibility properties:
```python
@property
def logical_source(self) -> str: ...
@property
def capability_name(self) -> str: ...
```
Parse-only compatibility should accept old payloads with `logical_source` and `capability_name` and build `ref`.
- [ ] **Step 2: Change `WorkflowArtifact.required_capabilities`**
Canonical model field:
```python
required_capabilities: list[RequiredCapability] = Field(default_factory=list)
```
Parse-only compatibility:
```json
{
"required_capabilities": {
"demo.echo_tool": {
"logical_source": "demo",
"capability_name": "echo_tool",
"kind": "node_spec"
}
}
}
```
should normalize to a list and dump as:
```json
{
"required_capabilities": [
{
"ref": "demo.echo_tool",
"kind": "node_spec"
}
]
}
```
- [ ] **Step 3: Add `required_capability_map()`**
```python
def required_capability_map(self) -> dict[str, RequiredCapability]:
return {str(capability.ref): capability for capability in self.required_capabilities}
```
Reject duplicate `ref` values during validation.
### Task 4: Update Call Sites to Use Helper Maps
**Files:**
- Modify: `src/wf_artifacts/validation.py`
- Modify: `src/wf_artifacts/catalog.py`
- Modify: `src/wf_mcp/workflow_surface/runtime_dependencies.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `src/wf_mcp/workflow_surface/tools.py`
- Modify: `src/wf_mcp/broker/artifact_tools.py`
- Modify tests that directly index `.bindings` or `.required_capabilities`.
- [ ] **Step 1: Replace `deployment.bindings.get(...)`**
Use:
```python
bindings = deployment.binding_map()
bound_source_id = bindings.get(required.logical_source)
```
- [ ] **Step 2: Replace `artifact.required_capabilities.items()`**
Use:
```python
for logical_ref, required in artifact.required_capability_map().items():
...
```
- [ ] **Step 3: Keep API input compatibility**
MCP tools that accept `required_capabilities` from callers may still accept dict input, but constructed `WorkflowArtifact` should dump the canonical list shape.
### Task 5: Update Docs
**Files:**
- Modify: `docs/workflow_artifacts.md`
- [ ] **Step 1: Replace dict binding examples**
Use:
```json
{
"bindings": [
{ "logical_source": "context7", "concrete_source": "context7.default" }
]
}
```
- [ ] **Step 2: Replace dict required capability examples**
Use:
```json
{
"required_capabilities": [
{
"ref": "context7.query-docs",
"kind": "node_spec",
"input_schema_hash": "sha256:..."
}
]
}
```
State that old dict shapes are accepted at parse boundaries but not emitted by model dumps.
### Task 6: Verification
**Files:**
- All touched files
- [ ] **Step 1: Run focused tests**
Run:
```bash
uv run --with pytest pytest tests/refs/test_platform_refs.py tests/artifacts/test_models.py tests/artifacts/test_store.py tests/artifacts/test_factory.py tests/artifacts/test_validation.py -q
```
- [ ] **Step 2: Run full suite**
Run:
```bash
uv run --with pytest pytest -q
```
- [ ] **Step 3: Run static checks**
Run:
```bash
uvx ruff check
uv run basedpyright --level error
```
---
## Self-Review
- Spec coverage: covers typed refs, deployment binding list shape, required capability list shape, compatibility parsing, helper indexes, docs, and tests.
- Placeholder scan: no placeholders remain.
- Type consistency: `SourceRef`, `CapabilityRef`, `SourceBinding`, `RequiredCapability`, `WorkflowArtifact`, and `WorkflowDeployment` names match current code or are introduced in this plan.
@@ -0,0 +1,821 @@
# Wrapper Authoring Hints Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add enum-backed wrapper authoring hints to workflow capability inspection so LLM and human MCP clients can scaffold wrapper drafts without guessing raw workflow plans.
**Architecture:** Keep MCP request/response names as plain JSON strings, but compute wrapper hints in a pure workflow-surface helper. The helper uses typed Pydantic models with enum fields for confidence, outcome policy, candidate kinds, and missing-decision kinds, then `inspect_capability` attaches the hint payload to live NodeSpec and saved wrapper details.
**Tech Stack:** Python 3.14, Pydantic v2, FastMCP-facing dict payloads, pytest, ruff, basedpyright.
---
## File Structure
- Create `src/wf_mcp/workflow_surface/wrapper_hints.py`
- Owns enum-backed hint models.
- Owns pure functions that derive hints from one capability contract.
- Must not call MCP, stores, workflow runtime, or FastMCP.
- Modify `src/wf_mcp/workflow_surface/handlers.py`
- Imports the pure hint helper.
- Adds `wrapper_hints` to `inspect_capability` payloads.
- Modify `src/wf_mcp/workflow_surface/models.py`
- Adds JSON-schema-visible response models if MCP tool schemas need stronger documentation.
- Do this only after the helper shape stabilizes.
- Test `tests/wf_mcp/test_workflow_wrapper_hints.py`
- Focused unit tests for hint derivation.
- Test `tests/wf_mcp/test_workflow_surface.py`
- Integration tests proving `inspect_capability` includes hints.
- Docs `docs/workflow_capabilities.md`
- Explain that hints are scaffolding, not semantic guarantees.
## Vocabulary
Use enums for all classification/type fields. Do not return magic strings for these fields from helper internals.
```python
from enum import StrEnum
class WrapperHintConfidence(StrEnum):
HIGH = "high"
MEDIUM = "medium"
LOW = "low"
class WrapperOutcomePolicy(StrEnum):
PRESERVE_DECLARED = "preserve_declared"
MANUAL_MAPPING_REQUIRED = "manual_mapping_required"
class OutcomeCandidateKind(StrEnum):
BOOLEAN_CONTROL_FIELD = "boolean_control_field"
class MissingDecisionKind(StrEnum):
CHOOSE_OUTPUT_FIELDS = "choose_output_fields"
REVIEW_NESTED_OUTPUT = "review_nested_output"
CONFIRM_BOOLEAN_OUTCOMES = "confirm_boolean_outcomes"
CHOOSE_ERROR_MAPPING = "choose_error_mapping"
```
MCP JSON still serializes those enum values as strings.
## Task 1: Add Enum-Backed Hint Models
**Files:**
- Create: `src/wf_mcp/workflow_surface/wrapper_hints.py`
- Test: `tests/wf_mcp/test_workflow_wrapper_hints.py`
- [ ] **Step 1: Write failing model serialization test**
Create `tests/wf_mcp/test_workflow_wrapper_hints.py`:
```python
from __future__ import annotations
from wf_mcp.workflow_surface.wrapper_hints import (
MissingDecision,
MissingDecisionKind,
OutcomeCandidate,
OutcomeCandidateKind,
WrapperAuthoringHints,
WrapperHintConfidence,
WrapperOutcomePolicy,
)
def test_wrapper_hint_models_serialize_enum_fields_as_strings() -> None:
hints = WrapperAuthoringHints(
capability_name="demo.personal.echo_tool",
confidence=WrapperHintConfidence.MEDIUM,
declared_outcomes=["ok", "error"],
suggested_wrapper_outcomes=["ok", "error"],
outcome_policy=WrapperOutcomePolicy.PRESERVE_DECLARED,
input_schema={"type": "object", "properties": {}},
state_schema={"type": "object", "properties": {}},
output_schema={"type": "object", "properties": {}},
input_map={},
output_map={},
outcome_candidates=[
OutcomeCandidate(
kind=OutcomeCandidateKind.BOOLEAN_CONTROL_FIELD,
source="output.success",
candidate_outcomes=["success", "failure"],
confidence=WrapperHintConfidence.MEDIUM,
reason="top-level boolean field with control-like name",
automatic=False,
)
],
missing_decisions=[
MissingDecision(
kind=MissingDecisionKind.CONFIRM_BOOLEAN_OUTCOMES,
message="Confirm whether output.success should control routing.",
)
],
notes=["Hints are scaffolding, not semantic guarantees."],
)
dumped = hints.model_dump(mode="json")
assert dumped["confidence"] == "medium"
assert dumped["outcome_policy"] == "preserve_declared"
assert dumped["outcome_candidates"][0]["kind"] == "boolean_control_field"
assert dumped["missing_decisions"][0]["kind"] == "confirm_boolean_outcomes"
```
- [ ] **Step 2: Run model test to verify it fails**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py::test_wrapper_hint_models_serialize_enum_fields_as_strings -q
```
Expected: import failure because `wf_mcp.workflow_surface.wrapper_hints` does not exist.
- [ ] **Step 3: Implement model definitions**
Create `src/wf_mcp/workflow_surface/wrapper_hints.py`:
```python
from __future__ import annotations
from enum import StrEnum
from typing import Any
from pydantic import BaseModel, Field
JsonObject = dict[str, Any]
class WrapperHintConfidence(StrEnum):
"""Coarse confidence for generated wrapper scaffolding hints."""
HIGH = "high"
MEDIUM = "medium"
LOW = "low"
class WrapperOutcomePolicy(StrEnum):
"""How wrapper outcomes were chosen."""
PRESERVE_DECLARED = "preserve_declared"
MANUAL_MAPPING_REQUIRED = "manual_mapping_required"
class OutcomeCandidateKind(StrEnum):
"""Reason a field was offered as a possible outcome source."""
BOOLEAN_CONTROL_FIELD = "boolean_control_field"
class MissingDecisionKind(StrEnum):
"""Typed action item a human or LLM must decide before saving a wrapper."""
CHOOSE_OUTPUT_FIELDS = "choose_output_fields"
REVIEW_NESTED_OUTPUT = "review_nested_output"
CONFIRM_BOOLEAN_OUTCOMES = "confirm_boolean_outcomes"
CHOOSE_ERROR_MAPPING = "choose_error_mapping"
class OutcomeCandidate(BaseModel):
"""One possible outcome mapping that must not be applied automatically."""
kind: OutcomeCandidateKind
source: str = Field(description="Output path such as output.success.")
candidate_outcomes: list[str]
confidence: WrapperHintConfidence
reason: str
automatic: bool = False
class MissingDecision(BaseModel):
"""One explicit decision required before a wrapper should be saved."""
kind: MissingDecisionKind
message: str
class WrapperAuthoringHints(BaseModel):
"""Scaffold for creating a workflow wrapper around one capability."""
capability_name: str
confidence: WrapperHintConfidence
declared_outcomes: list[str]
suggested_wrapper_outcomes: list[str]
outcome_policy: WrapperOutcomePolicy
input_schema: JsonObject
state_schema: JsonObject
output_schema: JsonObject
input_map: dict[str, str]
output_map: dict[str, str]
outcome_candidates: list[OutcomeCandidate] = Field(default_factory=list)
missing_decisions: list[MissingDecision] = Field(default_factory=list)
notes: list[str] = Field(default_factory=list)
```
- [ ] **Step 4: Run model test to verify it passes**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py::test_wrapper_hint_models_serialize_enum_fields_as_strings -q
```
Expected: pass.
## Task 2: Derive Simple Wrapper Hints From Capability Schemas
**Files:**
- Modify: `src/wf_mcp/workflow_surface/wrapper_hints.py`
- Test: `tests/wf_mcp/test_workflow_wrapper_hints.py`
- [ ] **Step 1: Add failing simple-schema hint test**
Append to `tests/wf_mcp/test_workflow_wrapper_hints.py`:
```python
from wf_mcp.workflow_surface.wrapper_hints import wrapper_hints_for_capability
def test_wrapper_hints_scaffold_simple_object_input_and_output() -> None:
hints = wrapper_hints_for_capability(
capability_name="demo.personal.echo_tool",
input_schema={
"type": "object",
"properties": {"text": {"type": "string"}},
"required": ["text"],
},
output_schema={
"type": "object",
"properties": {"echoed": {"type": "string"}},
"required": ["echoed"],
},
outcomes=["ok"],
)
dumped = hints.model_dump(mode="json")
assert dumped["confidence"] == "high"
assert dumped["declared_outcomes"] == ["ok"]
assert dumped["suggested_wrapper_outcomes"] == ["ok"]
assert dumped["outcome_policy"] == "preserve_declared"
assert dumped["input_map"] == {"input.text": "text"}
assert dumped["output_map"] == {"echoed": "state.echoed"}
assert dumped["state_schema"]["properties"]["echoed"]["type"] == "string"
assert dumped["output_schema"]["properties"]["echoed"]["type"] == "string"
assert dumped["missing_decisions"] == []
```
- [ ] **Step 2: Run simple-schema test to verify it fails**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py::test_wrapper_hints_scaffold_simple_object_input_and_output -q
```
Expected: import failure or attribute error for `wrapper_hints_for_capability`.
- [ ] **Step 3: Implement `wrapper_hints_for_capability`**
Add to `src/wf_mcp/workflow_surface/wrapper_hints.py`:
```python
CONTROL_BOOLEAN_NAMES = {
"success",
"ok",
"failed",
"error",
"is_error",
"needs_input",
"requires_approval",
"approved",
"rejected",
"has_more",
"done",
"complete",
}
def wrapper_hints_for_capability(
*,
capability_name: str,
input_schema: JsonObject,
output_schema: JsonObject,
outcomes: list[str] | tuple[str, ...],
) -> WrapperAuthoringHints:
"""Derive conservative wrapper scaffolding for one workflow capability."""
input_properties = _object_properties(input_schema)
output_properties = _object_properties(output_schema)
input_map = {f"input.{name}": name for name in sorted(input_properties)}
output_map = {name: f"state.{name}" for name in sorted(output_properties)}
state_schema = {
"type": "object",
"properties": {
name: schema for name, schema in sorted(output_properties.items())
},
}
wrapper_output_schema = {
"type": "object",
"properties": {
name: schema for name, schema in sorted(output_properties.items())
},
}
missing_decisions = _missing_decisions_for_output(output_schema)
outcome_candidates = _boolean_outcome_candidates(output_properties)
if outcome_candidates:
missing_decisions.append(
MissingDecision(
kind=MissingDecisionKind.CONFIRM_BOOLEAN_OUTCOMES,
message="Confirm whether boolean output fields should control wrapper routing.",
)
)
confidence = _confidence_for_hint(
input_schema=input_schema,
output_schema=output_schema,
missing_decisions=missing_decisions,
outcome_candidates=outcome_candidates,
)
return WrapperAuthoringHints(
capability_name=capability_name,
confidence=confidence,
declared_outcomes=list(outcomes),
suggested_wrapper_outcomes=list(outcomes),
outcome_policy=WrapperOutcomePolicy.PRESERVE_DECLARED,
input_schema=input_schema,
state_schema=state_schema,
output_schema=wrapper_output_schema,
input_map=input_map,
output_map=output_map,
outcome_candidates=outcome_candidates,
missing_decisions=missing_decisions,
notes=[
"Hints are scaffolding, not semantic guarantees.",
"Declared outcomes are preserved; output-field outcome inference is not automatic.",
],
)
def _object_properties(schema: JsonObject) -> dict[str, JsonObject]:
properties = schema.get("properties")
if not isinstance(properties, dict):
return {}
return {
str(name): value
for name, value in properties.items()
if isinstance(value, dict)
}
```
- [ ] **Step 4: Implement confidence and missing decision helpers**
Add below `_object_properties` in `src/wf_mcp/workflow_surface/wrapper_hints.py`:
```python
def _missing_decisions_for_output(output_schema: JsonObject) -> list[MissingDecision]:
properties = _object_properties(output_schema)
if not properties:
return [
MissingDecision(
kind=MissingDecisionKind.CHOOSE_OUTPUT_FIELDS,
message="Capability output schema has no top-level object properties to map.",
)
]
decisions: list[MissingDecision] = []
for name, schema in sorted(properties.items()):
schema_type = schema.get("type")
if schema_type == "object" or schema_type == "array":
decisions.append(
MissingDecision(
kind=MissingDecisionKind.REVIEW_NESTED_OUTPUT,
message=f"Review output.{name}; nested or collection outputs may need explicit mapping.",
)
)
return decisions
def _boolean_outcome_candidates(
output_properties: dict[str, JsonObject],
) -> list[OutcomeCandidate]:
candidates: list[OutcomeCandidate] = []
for name, schema in sorted(output_properties.items()):
if schema.get("type") != "boolean":
continue
if name.casefold() not in CONTROL_BOOLEAN_NAMES:
continue
candidates.append(
OutcomeCandidate(
kind=OutcomeCandidateKind.BOOLEAN_CONTROL_FIELD,
source=f"output.{name}",
candidate_outcomes=_candidate_outcomes_for_boolean_name(name),
confidence=WrapperHintConfidence.MEDIUM,
reason="top-level boolean field with control-like name",
automatic=False,
)
)
return candidates
def _candidate_outcomes_for_boolean_name(name: str) -> list[str]:
normalized = name.casefold()
if normalized in {"success", "ok", "done", "complete"}:
return ["success", "failure"]
if normalized in {"failed", "error", "is_error"}:
return ["error", "ok"]
if normalized in {"approved", "rejected"}:
return ["approved", "rejected"]
if normalized in {"needs_input", "requires_approval"}:
return [normalized, "done"]
if normalized == "has_more":
return ["has_more", "done"]
return ["true", "false"]
def _confidence_for_hint(
*,
input_schema: JsonObject,
output_schema: JsonObject,
missing_decisions: list[MissingDecision],
outcome_candidates: list[OutcomeCandidate],
) -> WrapperHintConfidence:
if not _object_properties(input_schema) or not _object_properties(output_schema):
return WrapperHintConfidence.LOW
if any(
decision.kind == MissingDecisionKind.REVIEW_NESTED_OUTPUT
for decision in missing_decisions
):
return WrapperHintConfidence.LOW
if missing_decisions or outcome_candidates:
return WrapperHintConfidence.MEDIUM
return WrapperHintConfidence.HIGH
```
- [ ] **Step 5: Run wrapper hint tests**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py -q
```
Expected: pass.
## Task 3: Add Boolean Outcome Candidate Tests
**Files:**
- Modify: `tests/wf_mcp/test_workflow_wrapper_hints.py`
- Modify: `src/wf_mcp/workflow_surface/wrapper_hints.py` only if tests reveal gaps.
- [ ] **Step 1: Add candidate and non-candidate tests**
Append:
```python
def test_wrapper_hints_offer_boolean_outcome_candidates_without_auto_mapping() -> None:
hints = wrapper_hints_for_capability(
capability_name="demo.personal.submit",
input_schema={"type": "object", "properties": {"text": {"type": "string"}}},
output_schema={
"type": "object",
"properties": {
"success": {"type": "boolean"},
"message": {"type": "string"},
},
},
outcomes=["ok"],
)
dumped = hints.model_dump(mode="json")
candidate = dumped["outcome_candidates"][0]
assert dumped["confidence"] == "medium"
assert candidate["kind"] == "boolean_control_field"
assert candidate["source"] == "output.success"
assert candidate["candidate_outcomes"] == ["success", "failure"]
assert candidate["automatic"] is False
assert dumped["outcome_policy"] == "preserve_declared"
assert dumped["suggested_wrapper_outcomes"] == ["ok"]
assert dumped["missing_decisions"][0]["kind"] == "confirm_boolean_outcomes"
def test_wrapper_hints_do_not_treat_arbitrary_booleans_as_outcomes() -> None:
hints = wrapper_hints_for_capability(
capability_name="demo.personal.profile",
input_schema={"type": "object", "properties": {"user_id": {"type": "string"}}},
output_schema={
"type": "object",
"properties": {
"is_admin": {"type": "boolean"},
"name": {"type": "string"},
},
},
outcomes=["ok"],
)
dumped = hints.model_dump(mode="json")
assert dumped["confidence"] == "high"
assert dumped["outcome_candidates"] == []
assert dumped["missing_decisions"] == []
```
- [ ] **Step 2: Run tests**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py -q
```
Expected: pass. If arbitrary boolean fields produce candidates, fix `CONTROL_BOOLEAN_NAMES` filtering rather than deleting the test.
## Task 4: Add Complex Output Missing Decision Tests
**Files:**
- Modify: `tests/wf_mcp/test_workflow_wrapper_hints.py`
- Modify: `src/wf_mcp/workflow_surface/wrapper_hints.py` only if tests reveal gaps.
- [ ] **Step 1: Add nested-output and empty-output tests**
Append:
```python
def test_wrapper_hints_mark_nested_outputs_as_low_confidence() -> None:
hints = wrapper_hints_for_capability(
capability_name="demo.personal.search",
input_schema={"type": "object", "properties": {"query": {"type": "string"}}},
output_schema={
"type": "object",
"properties": {
"results": {
"type": "array",
"items": {
"type": "object",
"properties": {"title": {"type": "string"}},
},
}
},
},
outcomes=["ok"],
)
dumped = hints.model_dump(mode="json")
assert dumped["confidence"] == "low"
assert dumped["missing_decisions"][0]["kind"] == "review_nested_output"
assert dumped["output_map"] == {"results": "state.results"}
def test_wrapper_hints_mark_empty_output_schema_as_low_confidence() -> None:
hints = wrapper_hints_for_capability(
capability_name="demo.personal.no_output",
input_schema={"type": "object", "properties": {"text": {"type": "string"}}},
output_schema={"type": "object", "properties": {}},
outcomes=["ok"],
)
dumped = hints.model_dump(mode="json")
assert dumped["confidence"] == "low"
assert dumped["input_map"] == {"input.text": "text"}
assert dumped["output_map"] == {}
assert dumped["missing_decisions"][0]["kind"] == "choose_output_fields"
```
- [ ] **Step 2: Run tests**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py -q
```
Expected: pass.
## Task 5: Wire Hints Into `inspect_capability`
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
- [ ] **Step 1: Add failing live capability integration test**
In `tests/wf_mcp/test_workflow_surface.py`, after `test_workflow_surface_inspects_one_capability`, add:
```python
def test_workflow_surface_inspect_capability_includes_wrapper_hints() -> None:
service = WfMcpService(
store=FileStore(local_temp_root() / "surface_wrapper_hints_mcp"),
artifact_store=FileWorkflowArtifactStore(
local_temp_root() / "surface_wrapper_hints_artifacts"
),
)
service.register_connection(
ConnectionConfig(id="demo.personal", server="demo", account="personal")
)
service.register_specs("demo.personal", echo_tool)
handlers = WorkflowSurfaceHandlers(service)
payload = asyncio.run(
handlers.inspect_capability(qualified_name="demo.personal.echo_tool")
)
hints = payload["wrapper_hints"]
assert hints["capability_name"] == "demo.personal.echo_tool"
assert hints["declared_outcomes"] == ["ok"]
assert hints["input_map"] == {"input.text": "text"}
assert hints["output_map"] == {"echoed": "state.echoed"}
assert hints["outcome_policy"] == "preserve_declared"
```
- [ ] **Step 2: Run integration test to verify it fails**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py::test_workflow_surface_inspect_capability_includes_wrapper_hints -q
```
Expected: failure because `wrapper_hints` is absent.
- [ ] **Step 3: Attach hints for live NodeSpecs**
In `src/wf_mcp/workflow_surface/handlers.py`, import:
```python
from .wrapper_hints import wrapper_hints_for_capability
```
Then in `inspect_capability`, replace the live-detail return with:
```python
detail_payload = detail.model_dump(mode="json")
detail_payload["wrapper_hints"] = wrapper_hints_for_capability(
capability_name=detail.name,
input_schema=detail.input_schema,
output_schema=detail.output_schema,
outcomes=detail.outcomes,
).model_dump(mode="json")
return detail_payload
```
- [ ] **Step 4: Run integration test**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py::test_workflow_surface_inspect_capability_includes_wrapper_hints -q
```
Expected: pass.
## Task 6: Add Hints For Saved Wrapper Artifact Inspection
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
- [ ] **Step 1: Add failing saved-wrapper hint assertion**
In existing `test_workflow_surface_inspects_saved_wrapper_capability`, add:
```python
hints = payload["wrapper_hints"]
assert hints["capability_name"] == "workflow.echo_wrapper.v1"
assert hints["declared_outcomes"] == ["completed"]
assert hints["suggested_wrapper_outcomes"] == ["completed"]
assert hints["input_map"] == {"input.text": "text"}
assert hints["output_map"] == {"echoed": "state.echoed"}
```
- [ ] **Step 2: Run saved-wrapper test to verify it fails**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py::test_workflow_surface_inspects_saved_wrapper_capability -q
```
Expected: failure because wrapper detail lacks `wrapper_hints`.
- [ ] **Step 3: Attach hints for wrapper artifacts**
In `_wrapper_capability_detail`, add `wrapper_hints` to the returned dict:
```python
"wrapper_hints": wrapper_hints_for_capability(
capability_name=_artifact_capability_id(artifact),
input_schema=artifact.input_schema,
output_schema=artifact.output_schema,
outcomes=list(artifact.outcomes),
).model_dump(mode="json"),
```
- [ ] **Step 4: Run saved-wrapper test**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py::test_workflow_surface_inspects_saved_wrapper_capability -q
```
Expected: pass.
## Task 7: Document Hint Semantics
**Files:**
- Modify: `docs/workflow_capabilities.md`
- [ ] **Step 1: Add documentation section**
Add a section titled `## Wrapper Authoring Hints`:
```markdown
## Wrapper Authoring Hints
`wf.workflow.inspect_capability` returns `wrapper_hints` for planner-visible
capabilities. These hints are scaffolding for draft creation, not semantic
guarantees.
Declared capability outcomes are authoritative and are preserved by default.
Boolean output fields may appear as `outcome_candidates` when they have
control-like names such as `success`, `error`, `approved`, or `has_more`, but
they are never wired automatically. A wrapper author must explicitly confirm
whether those fields should become routing conditions.
`confidence` is intentionally coarse:
- `high`: simple object input/output schemas and no missing decisions.
- `medium`: usable scaffold with candidate decisions, such as boolean outcome
candidates.
- `low`: missing or nested output choices require explicit authoring.
`missing_decisions` is a typed list of decisions the author should resolve
before saving a wrapper. MCP clients should show these prominently rather than
treating the scaffold as complete.
```
- [ ] **Step 2: Run docs-adjacent tests**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/test_workflow_surface.py -q
```
Expected: pass.
## Task 8: Final Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused tests**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_workflow_surface_refs.py -q
```
Expected: pass.
- [ ] **Step 2: Run full tests if the workspace is not mid-edit**
Run:
```bash
uv run --with pytest pytest -q
```
Expected: pass, allowing intentional environment-dependent skips. If the user is actively editing `tests/rewrite`, run focused tests only and state that full-suite verification was deferred.
- [ ] **Step 3: Run static checks**
Run:
```bash
uvx ruff check src/wf_mcp/workflow_surface tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/test_workflow_surface.py
uv run basedpyright --level error src/wf_mcp/workflow_surface tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/test_workflow_surface.py
uvx ruff format --check src/wf_mcp/workflow_surface tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/test_workflow_surface.py
```
Expected: ruff passes, basedpyright has 0 errors, and formatting is clean.
## Self-Review
- Spec coverage: The plan covers enum-backed type fields, conservative outcome suggestions, boolean candidates, confidence/missing-decision UX, inspect-capability integration, and docs.
- Placeholder scan: No `TBD`, `TODO`, or unspecified implementation steps remain.
- Type consistency: `WrapperHintConfidence`, `WrapperOutcomePolicy`, `OutcomeCandidateKind`, and `MissingDecisionKind` are defined before use and serialize through Pydantic models.
- Scope check: This plan does not create or save wrappers automatically; it only adds hint payloads for authoring.
@@ -0,0 +1,531 @@
# Authoring Path Inputs Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make `wf_authoring` understand structural, string, iterable, and vararg path inputs consistently across DSL helpers and `WorkflowBuilder.use()`.
**Architecture:** Keep `wf_core` path objects as the canonical runtime model. Add one authoring coercion layer that turns ergonomic inputs into `GraphSourcePath`, `StatePath`, and `LocalPath`. Single-string helper calls parse TOML dotted-key expressions; varargs and iterables are literal path parts. Builder maps should normalize into typed path objects instead of string-to-string maps so the authoring layer stops being another dotted-string boundary.
**Tech Stack:** Python 3.14, stdlib `tomllib`, `wf_core.paths`, `wf_authoring.dsl`, `WorkflowBuilder`, pytest.
---
## Current State
The core now supports structural path objects:
```json
{ "root": "state", "parts": ["person.name", "three and four"] }
```
But `wf_authoring` still stores paths as strings:
- `wf_authoring.dsl.paths.GraphPath.value: str`
- `wf_authoring.dsl.mapping.PathArg = str | GraphPath`
- `WorkflowBuilder.use(..., in_map=..., out_map=...)` normalizes maps to `dict[str, str]`
- builder internals call `LocalPath.parse(...)`, `GraphSourcePath.parse(...)`, and `StatePath.parse(...)`
That means authoring helpers still risk ambiguity:
```python
state("person.name")
```
Today this is dotted shorthand. To express a literal field named `person.name`, users need structural/literal segment input.
---
## Semantics
### Single String Argument
Parse as TOML dotted-key expression:
```python
state("person.name")
# parts: ["person", "name"]
state('"person.name"')
# parts: ["person.name"]
state('person."three and four"')
# parts: ["person", "three and four"]
```
### Varargs
Treat each argument as a literal path segment:
```python
state("person.name", "email address")
# parts: ["person.name", "email address"]
state_path("oh", "my", "days")
# parts: ["oh", "my", "days"]
```
### Iterable Input
Treat iterable items as literal path segments:
```python
state(("person.name",))
# parts: ["person.name"]
input_path(["user", "email"])
# parts: ["user", "email"]
```
### Existing Path Objects
Pass through path objects without reparsing:
```python
state_path(StatePath(("person.name",)))
input_path(GraphSourcePath.input("user"))
```
### Structural Path Dicts
Accept structural core path dicts at authoring boundaries when data is already
model-shaped:
```python
state_path({"root": "state", "parts": ["person.name"]})
input_path({"root": "input", "parts": ["user", "email"]})
```
This keeps MCP / JSON-facing callers from converting canonical objects back
into display strings just to pass through `wf_authoring`.
---
## File Structure
- Create: `src/wf_authoring/dsl/path_inputs.py`
- Own `PathInput` type alias.
- Own TOML key-expression parser using `tomllib`.
- Own coercion functions for graph/local/state paths.
- Modify: `src/wf_authoring/dsl/paths.py`
- Make `GraphPath` wrap `GraphSourcePath`, not a string.
- Update `graph_path`, `input_path`, `state_path`, `context_path`.
- Modify: `src/wf_authoring/dsl/conditions.py`
- Use typed `GraphSourcePath` directly from `GraphPath` / `PathExpr`.
- Update `state(...)`, `input(...)`, and `context(...)` to accept `PathInput`.
- Modify: `src/wf_authoring/dsl/mapping.py`
- Expand `PathArg` to include structural/core path objects and iterable parts.
- Keep `bind_fields` / `bind_state` APIs stable, but normalize through the new coercers.
- Modify: `src/wf_authoring/builder/mapping.py`
- Normalize `MapArg` to typed paths, not strings.
- Keep legacy string map support.
- Modify: `src/wf_authoring/builder/core.py`
- Change `_canonical_input_bindings` / `_canonical_output_bindings` to accept typed path mappings.
- Stop reparsing paths from strings when already typed.
- Test:
- `tests/authoring/test_path_inputs.py`
- `tests/authoring/test_builder.py`
- `tests/authoring/test_conditions.py`
---
## Task 1: Add Path Input Coercion Module
**Files:**
- Create: `src/wf_authoring/dsl/path_inputs.py`
- Test: `tests/authoring/test_path_inputs.py`
- [ ] **Step 1: Write failing tests**
```python
from wf_authoring.dsl.path_inputs import (
coerce_graph_path,
coerce_local_path,
coerce_state_path,
)
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
def test_single_string_path_input_uses_toml_dotted_key_syntax() -> None:
assert coerce_state_path("person.name") == StatePath(("person", "name"))
assert coerce_state_path('"person.name"') == StatePath(("person.name",))
assert coerce_state_path('person."three and four"') == StatePath(
("person", "three and four")
)
def test_vararg_path_input_treats_parts_as_literal_segments() -> None:
assert coerce_state_path("person.name", "email address") == StatePath(
("person.name", "email address")
)
def test_iterable_path_input_treats_items_as_literal_segments() -> None:
assert coerce_local_path(("payload.text",)) == LocalPath(("payload.text",))
def test_existing_path_objects_pass_through() -> None:
source = GraphSourcePath("state", ("person.name",))
assert coerce_graph_path(source) is source
def test_structural_path_dicts_validate_through_core_models() -> None:
assert coerce_graph_path({"root": "state", "parts": ["person.name"]}) == (
GraphSourcePath("state", ("person.name",))
)
```
- [ ] **Step 2: Run tests to verify red**
```bash
uv run --with pytest pytest tests/authoring/test_path_inputs.py -q
```
Expected: fails because module does not exist.
- [ ] **Step 3: Implement parser using `tomllib`**
Implement:
```python
import tomllib
from collections.abc import Iterable, Mapping
from typing import TypeAlias
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
PathInput: TypeAlias = (
str
| Iterable[str]
| Mapping[str, object]
| GraphSourcePath
| StatePath
| LocalPath
)
```
Parser approach:
```python
def _parse_toml_key_expr(expr: str) -> tuple[str, ...]:
parsed = tomllib.loads(f"{expr} = true")
...
```
Walk the nested dict until the leaf value is `True`; each nested key is one path segment.
Rules:
- one `str` argument parses as TOML key expression
- multiple `str` arguments are literal segments
- one iterable argument is literal segments
- existing path object passes through when compatible
- structural dicts validate through the matching core path model
- invalid TOML raises `ValueError` with message mentioning TOML key expression
- [ ] **Step 4: Run tests to verify green**
```bash
uv run --with pytest pytest tests/authoring/test_path_inputs.py -q
```
Expected: all tests pass.
---
## Task 2: Make DSL Path Helpers Typed
**Files:**
- Modify: `src/wf_authoring/dsl/paths.py`
- Modify: `src/wf_authoring/dsl/conditions.py`
- Test: `tests/authoring/test_path_inputs.py`
- Test: `tests/authoring/test_conditions.py`
- [ ] **Step 1: Write failing helper tests**
Add:
```python
from wf_authoring import state, state_path
from wf_core.paths import GraphSourcePath
def test_state_path_helper_supports_toml_strings_and_literal_varargs() -> None:
assert state_path('"person.name"').path == GraphSourcePath(
"state", ("person.name",)
)
assert state_path("person.name", "email address").path == GraphSourcePath(
"state", ("person.name", "email address")
)
def test_state_expr_helper_uses_same_path_input_rules() -> None:
condition = state('"person.name"').eq("Ada").to_condition()
assert condition.left.path == GraphSourcePath("state", ("person.name",))
```
- [ ] **Step 2: Run tests to verify red**
```bash
uv run --with pytest pytest tests/authoring/test_path_inputs.py tests/authoring/test_conditions.py -q
```
Expected: old helpers either split incorrectly or do not accept these signatures.
- [ ] **Step 3: Update `GraphPath`**
Change:
```python
@dataclass(frozen=True, slots=True)
class GraphPath:
path: GraphSourcePath
@property
def value(self) -> str:
return str(self.path)
```
Keep `.value` as compatibility display output.
- [ ] **Step 4: Update helper signatures**
```python
def input_path(first: PathInput, *parts: str) -> GraphPath: ...
def state_path(first: PathInput, *parts: str) -> GraphPath: ...
def context_path(first: PathInput, *parts: str) -> GraphPath: ...
```
Use coercers from `path_inputs.py`.
- [ ] **Step 5: Update conditions**
Make `PathExpr` store `GraphSourcePath`, while keeping `.path` display property if needed:
```python
@dataclass(frozen=True, slots=True)
class PathExpr:
source: GraphSourcePath
@property
def path(self) -> str:
return str(self.source)
```
Use `PathOperand(path=self.source)` instead of reparsing strings.
- [ ] **Step 6: Run tests**
```bash
uv run --with pytest pytest tests/authoring/test_path_inputs.py tests/authoring/test_conditions.py -q
```
Expected: all pass.
---
## Task 3: Make Builder Maps Accept Typed Path Inputs
**Files:**
- Modify: `src/wf_authoring/builder/mapping.py`
- Modify: `src/wf_authoring/builder/core.py`
- Modify: `src/wf_authoring/dsl/mapping.py`
- Test: `tests/authoring/test_builder.py`
- [ ] **Step 1: Write failing builder tests**
Add:
```python
from wf_authoring import input_path, state_path
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
def test_builder_use_accepts_typed_paths_and_literal_iterable_paths() -> None:
builder = WorkflowBuilder(...)
step = builder.use(
auto_bind_node,
in_map={input_path('"text.with.dot"'): ("payload.text",)},
out_map={("payload.text",): state_path("state field")},
)
assert step.input[0].path == GraphSourcePath("input", ("text.with.dot",))
assert step.input[0].target == LocalPath(("payload.text",))
assert step.output[0].source == LocalPath(("payload.text",))
assert step.output[0].target == StatePath(("state field",))
```
Use existing builder test fixtures in `tests/authoring/test_builder.py`.
- [ ] **Step 2: Run tests to verify red**
```bash
uv run --with pytest pytest tests/authoring/test_builder.py -q
```
Expected: tuple/typed map values fail.
- [ ] **Step 3: Update map normalization**
In `builder/mapping.py`, introduce typed mapping aliases:
```python
InputMap = dict[GraphSourcePath, LocalPath]
OutputMap = dict[LocalPath, StatePath]
```
Add:
```python
normalize_input_mapping(mapping: MapArg | None) -> InputMap
normalize_output_mapping(mapping: MapArg | None) -> OutputMap
```
Rules:
- input map key = graph source path
- input map value = local path
- output map key = local path
- output map value = state path
Legacy strings still parse through the new coercers.
- [ ] **Step 4: Update builder core**
Change `_canonical_input_bindings`:
```python
def _canonical_input_bindings(
in_map: Mapping[GraphSourcePath, LocalPath],
input_values: Mapping[LocalPath, Any],
) -> list[InputBinding]:
```
Change `_canonical_output_bindings`:
```python
def _canonical_output_bindings(
out_map: Mapping[LocalPath, StatePath],
) -> list[OutputBinding]:
```
`InputValueBinding.target` should also accept typed/local path input.
- [ ] **Step 5: Update DSL mapping helpers**
`bind_fields(**mapping)` and `bind_state(**mapping)` can keep returning dicts, but values should be normalized display/typed consistently. Prefer returning typed path maps if that does not break tests; otherwise keep their public shape and let builder normalize.
- [ ] **Step 6: Run tests**
```bash
uv run --with pytest pytest tests/authoring/test_builder.py tests/authoring/test_demo_workflow.py tests/authoring/test_ops.py -q
```
Expected: all pass.
---
## Task 3.5: Foreach Boundary Check
**Files:**
- Inspect: `src/wf_authoring/builder/core.py`
- Inspect: `src/wf_core/models/steps.py` or current foreach model location
`WorkflowBuilder.foreach(over=...)` also accepts path-like input today, but the
core foreach model may still store the source path as a string. Do not let this
block `WorkflowBuilder.use()` map normalization.
- [ ] **Step 1: Inspect foreach field type**
If core foreach already accepts `GraphSourcePath`, normalize `over` through the
new graph-path coercer and add one focused test.
If core foreach still accepts only strings, keep the existing string
serialization path and leave a short comment at the call site:
```text
foreach path input should move to typed GraphSourcePath when the core foreach
model is upgraded.
```
- [ ] **Step 2: Avoid partial semantic claims**
Do not document foreach as fully structural until the core field is structural.
---
## Task 4: Docs and Examples
**Files:**
- Modify: `docs/structural_refs.md`
- Modify or create an authoring docs/example if one already exists.
- [ ] **Step 1: Add authoring examples**
Add:
```python
state("person.name") # TOML/dotted expression
state('"person.name"') # literal dotted field
state("person.name", "email") # literal segments
state(("person.name",)) # literal iterable
```
- [ ] **Step 2: Mention builder maps**
Add:
```python
g.use(
node,
in_map={input_path('"email.address"'): ("payload.email",)},
out_map={("result.score",): state_path("score")},
)
```
---
## Task 5: Verification
- [ ] **Step 1: Run focused authoring tests**
```bash
uv run --with pytest pytest tests/authoring -q
```
- [ ] **Step 2: Run full tests**
```bash
uv run --with pytest pytest -q
```
- [ ] **Step 3: Run checks**
```bash
uvx ruff check src/wf_authoring src/wf_core/paths.py tests/authoring
uv run basedpyright --level error src/wf_authoring src/wf_core/paths.py tests/authoring
```
---
## Self-Review Notes
- Do not roll a custom TOML parser. Use stdlib `tomllib`.
- Canonical saved JSON remains structural `root` / `parts`.
- Single strings are ergonomic expressions. Varargs and iterables are literal segments.
- Keep `.value` / `str(...)` as display compatibility only.
- `WorkflowBuilder.use()` should stop being a string-to-string path boundary.
- If Pydantic path models are not hashable enough for dict keys, normalize maps
into explicit binding-pair lists internally instead of falling back to display
strings.
@@ -0,0 +1,873 @@
# Builder Canonical Bindings Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make `WorkflowBuilder.use()` and `WorkflowBuilder.use_ref()` expose canonical `input` / `output` binding lists, while keeping `in_map`, `input_values`, and `out_map` as deprecated Python sugar.
**Architecture:** `wf_core.NodeUse` already stores canonical binding structs: `InputPathBinding`, `InputValueBinding`, and `OutputBinding`. The builder should accept those same structs/dicts directly, normalize them through core models, and reject mixed canonical/deprecated arguments. Map sugar remains for pleasant Python authoring, but JSON/MCP-facing callers should use binding lists so structural path dicts live inside structs, not as unhashable mapping keys.
**Tech Stack:** Python 3.14, Pydantic models from `wf_core.models.steps`, `wf_authoring.WorkflowBuilder`, pytest, basedpyright, ruff.
---
## Why the Previous Plan Did Not Finish This
`2026-05-21-authoring-path-inputs.md` focused on path coercion:
- single-string TOML path parsing
- iterable/vararg literal segments
- typed `GraphPath`
- map normalization from `dict[str, str]` toward typed paths
That plan made `in_map`, `input_values`, and `out_map` safer, but it did not change the public builder API shape. So the current state is still incomplete:
```python
g.use(node, in_map=..., input_values=..., out_map=...)
```
exists, but:
```python
g.use(node, input=[...], output=[...])
```
does not.
That matters because structural path dicts cannot be Python dict keys. The JSON/MCP-friendly shape must be list-of-structs:
```json
{
"input": [
{
"target": { "root": "local", "parts": ["payload.email"] },
"path": { "root": "input", "parts": ["email.address"] }
}
],
"output": [
{
"source": { "root": "local", "parts": ["result.score"] },
"target": { "root": "state", "parts": ["score"] }
}
]
}
```
---
## Current State
Core already has the right canonical models in `src/wf_core/models/steps.py`:
```python
class InputPathBinding(BaseModel):
target: LocalPath
path: GraphSourcePath
class InputValueBinding(BaseModel):
target: LocalPath
value: object
class OutputBinding(BaseModel):
source: LocalPath
target: StatePath
class NodeUse(BaseModel):
input: list[InputBinding] = Field(default_factory=list)
output: list[OutputBinding] = Field(default_factory=list)
```
Builder currently has only deprecated/sugar arguments in `src/wf_authoring/builder/core.py`:
```python
def use(
self,
spec: NodeSpec[Any, Any],
*,
id: str | None = None,
in_map: MapArg | None = None,
input_values: Mapping[Any, Any] | None = None,
out_map: MapArg | None = None,
desc: str | None = None,
) -> NodeUse:
...
```
This is the API gap.
---
## Public Semantics
### Canonical Builder Inputs
Add `input` and `output` parameters:
```python
g.use(
node,
input=[
{
"target": {"root": "local", "parts": ["payload.email"]},
"path": {"root": "input", "parts": ["email.address"]},
},
{
"target": {"root": "local", "parts": ["static.limit"]},
"value": 10,
},
],
output=[
{
"source": {"root": "local", "parts": ["result.score"]},
"target": {"root": "state", "parts": ["score"]},
},
],
)
```
Accepted item shapes:
- existing `InputPathBinding`
- existing `InputValueBinding`
- existing `OutputBinding`
- dicts that `InputPathBinding` / `InputValueBinding` / `OutputBinding` can validate
### Deprecated Sugar Inputs
Keep these for Python authors:
```python
g.use(node, in_map={state_path("text"): "payload.text"})
g.use(node, input_values={"limit": 10})
g.use(node, out_map={"result.score": state_path("score")})
```
But mark them as deprecated in docstrings and warn when explicitly used.
Auto-mapping still uses the same internal sugar when both canonical and deprecated args are absent.
### Mixing Rules
Reject ambiguous combinations:
- `input` cannot be mixed with `in_map`
- `input` cannot be mixed with `input_values`
- `output` cannot be mixed with `out_map`
Exact error examples:
```text
cannot mix canonical input with deprecated in_map/input_values
cannot mix canonical output with deprecated out_map
```
Use built-in `TypeError` for these authoring API misuse errors. This is similar
in spirit to Pydantic's user-error category: the caller supplied an invalid API
shape, not invalid workflow data.
### Structural Dict Key Rule
Do not support structural dicts as mapping keys. Python `dict` keys must be hashable, and adding `frozendict` support is not worth it.
If a user needs structural dict paths, they should use canonical binding lists:
```python
input=[{"target": {"root": "local", "parts": ["payload"]}, "path": {...}}]
```
Map sugar is for hashable Python authoring values only.
---
## File Structure
- Modify: `src/wf_authoring/builder/core.py`
- Add `input` / `output` parameters to `use()` and `use_ref()`.
- Add canonical/deprecated mixing checks.
- Use canonical binding normalization when provided.
- Warn when deprecated map-sugar args are explicitly used.
- Modify: `src/wf_authoring/builder/mapping.py`
- Add `InputBindingArg`, `OutputBindingArg` aliases.
- Add `normalize_input_bindings(...)`.
- Add `normalize_output_bindings(...)`.
- Keep map normalizers as deprecated/sugar internals.
- Modify: `docs/structural_refs.md`
- Document canonical `input` / `output` list usage for structural path dicts.
- State that structural dicts are not supported as map keys.
- Modify: `docs/authoring_sketch.md` or `docs/core_state_mapping_and_merge.md`
- Replace older “builder uses in_map/out_map” framing with “builder accepts canonical lists; maps are sugar.”
- Test:
- `tests/authoring/test_builder.py`
- Maybe `tests/authoring/test_path_inputs.py` only if structural dict errors belong there.
---
## Task 1: Add Canonical Binding Normalizers
**Files:**
- Modify: `src/wf_authoring/builder/mapping.py`
- Test: `tests/authoring/test_builder.py`
- [ ] **Step 1: Write failing tests for canonical dict bindings**
Add to `tests/authoring/test_builder.py`:
```python
def test_builder_use_accepts_canonical_binding_dicts_with_structural_paths() -> None:
builder = WorkflowBuilder(
name="canonical_binding_dicts",
input_schema=AutoBindInput,
state_schema=AutoBindState,
output_schema=AutoBindOutput,
)
step = builder.use(
auto_bind_node,
input=[
{
"target": {"root": "local", "parts": ["payload.text"]},
"path": {"root": "input", "parts": ["text.with.dot"]},
},
{
"target": {"root": "local", "parts": ["static.limit"]},
"value": 3,
},
],
output=[
{
"source": {"root": "local", "parts": ["payload.text"]},
"target": {"root": "state", "parts": ["text.with.dot"]},
}
],
)
assert isinstance(step.input[0], InputPathBinding)
assert step.input[0].path == GraphSourcePath("input", ("text.with.dot",))
assert step.input[0].target == LocalPath(("payload.text",))
assert isinstance(step.input[1], InputValueBinding)
assert step.input[1].target == LocalPath(("static.limit",))
assert step.input[1].value == 3
assert step.output[0].source == LocalPath(("payload.text",))
assert step.output[0].target == StatePath(("text.with.dot",))
```
Update imports:
```python
from wf_core.models.steps import InputPathBinding, InputValueBinding
```
- [ ] **Step 2: Run test to verify red**
```bash
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_use_accepts_canonical_binding_dicts_with_structural_paths -q
```
Expected: fails because `WorkflowBuilder.use()` has no `input` / `output` parameters.
- [ ] **Step 3: Add normalizer aliases and functions**
In `src/wf_authoring/builder/mapping.py`, add:
```python
from wf_core.models.steps import InputBinding, InputPathBinding, InputValueBinding, OutputBinding
InputBindingArg: TypeAlias = InputBinding | Mapping[str, object]
OutputBindingArg: TypeAlias = OutputBinding | Mapping[str, object]
```
Add:
```python
def normalize_input_bindings(bindings: Sequence[InputBindingArg] | None) -> list[InputBinding]:
"""Validate canonical input binding structs for WorkflowBuilder.use()."""
if bindings is None:
return []
normalized: list[InputBinding] = []
for binding in bindings:
if isinstance(binding, InputPathBinding | InputValueBinding):
normalized.append(binding)
continue
if not isinstance(binding, Mapping):
raise TypeError(f"unsupported input binding {binding!r}")
if "path" in binding:
normalized.append(InputPathBinding.model_validate(binding))
elif "value" in binding:
normalized.append(InputValueBinding.model_validate(binding))
else:
raise ValueError("input binding must contain either 'path' or 'value'")
return normalized
```
Add:
```python
def normalize_output_bindings(bindings: Sequence[OutputBindingArg] | None) -> list[OutputBinding]:
"""Validate canonical output binding structs for WorkflowBuilder.use()."""
if bindings is None:
return []
normalized: list[OutputBinding] = []
for binding in bindings:
if isinstance(binding, OutputBinding):
normalized.append(binding)
continue
if not isinstance(binding, Mapping):
raise TypeError(f"unsupported output binding {binding!r}")
normalized.append(OutputBinding.model_validate(binding))
return normalized
```
- [ ] **Step 4: Run focused normalizer-related test**
```bash
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_use_accepts_canonical_binding_dicts_with_structural_paths -q
```
Expected: still fails until builder signatures are updated.
---
## Task 2: Add `input` / `output` to `use()`
**Files:**
- Modify: `src/wf_authoring/builder/core.py`
- Test: `tests/authoring/test_builder.py`
- [ ] **Step 1: Update imports**
In `src/wf_authoring/builder/core.py`, import new aliases/functions:
```python
from .mapping import (
InputBindingArg,
OutputBindingArg,
normalize_input_bindings,
normalize_output_bindings,
)
```
- [ ] **Step 2: Update `use()` signature**
Change:
```python
def use(
self,
spec: NodeSpec[Any, Any],
*,
id: str | None = None,
in_map: MapArg | None = None,
input_values: Mapping[Any, Any] | None = None,
out_map: MapArg | None = None,
desc: str | None = None,
) -> NodeUse:
```
to:
```python
def use(
self,
spec: NodeSpec[Any, Any],
*,
id: str | None = None,
input: Sequence[InputBindingArg] | None = None,
output: Sequence[OutputBindingArg] | None = None,
in_map: MapArg | None = None,
input_values: Mapping[Any, Any] | None = None,
out_map: MapArg | None = None,
desc: str | None = None,
) -> NodeUse:
```
Import `Sequence` from `collections.abc`.
- [ ] **Step 3: Add mixing guard helper**
Add near the canonical binding helpers:
```python
def _reject_mixed_binding_styles(
*,
input: object | None,
output: object | None,
in_map: object | None,
input_values: object | None,
out_map: object | None,
) -> None:
"""Keep canonical binding lists and deprecated map sugar from mixing."""
if input is not None and (in_map is not None or input_values is not None):
raise TypeError("cannot mix canonical input with deprecated in_map/input_values")
if output is not None and out_map is not None:
raise TypeError("cannot mix canonical output with deprecated out_map")
```
- [ ] **Step 4: Use canonical bindings when provided**
In `use()`:
```python
_reject_mixed_binding_styles(
input=input,
output=output,
in_map=in_map,
input_values=input_values,
out_map=out_map,
)
if input is not None:
node_input = normalize_input_bindings(input)
else:
raw_in_map = auto_input_map(...) if in_map is None else in_map
node_input = _canonical_input_bindings(
normalize_input_mapping(raw_in_map),
normalize_input_values(input_values),
)
if output is not None:
node_output = normalize_output_bindings(output)
else:
raw_out_map = auto_output_map(...) if out_map is None else out_map
node_output = _canonical_output_bindings(normalize_output_mapping(raw_out_map))
```
Then pass:
```python
input=node_input,
output=node_output,
```
- [ ] **Step 5: Run focused test**
```bash
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_use_accepts_canonical_binding_dicts_with_structural_paths -q
```
Expected: pass.
---
## Task 3: Add `input` / `output` to `use_ref()`
**Files:**
- Modify: `src/wf_authoring/builder/core.py`
- Test: `tests/authoring/test_builder.py`
- [ ] **Step 1: Write failing test**
Add:
```python
def test_builder_use_ref_accepts_canonical_binding_dicts() -> None:
builder = WorkflowBuilder(
name="external_ref_canonical_bindings",
input_schema={},
state_schema={"fields": {}},
output_schema={},
)
step = builder.use_ref(
"demo.echo",
id="echo",
input=[
{
"target": {"root": "local", "parts": ["text"]},
"path": {"root": "input", "parts": ["text"]},
}
],
output=[
{
"source": {"root": "local", "parts": ["echoed"]},
"target": {"root": "state", "parts": ["echoed"]},
}
],
)
assert step.node == "demo.echo"
assert isinstance(step.input[0], InputPathBinding)
assert step.input[0].path == GraphSourcePath.input("text")
assert step.output[0].target == StatePath.of("echoed")
```
- [ ] **Step 2: Update `use_ref()` signature**
Add:
```python
input: Sequence[InputBindingArg] | None = None,
output: Sequence[OutputBindingArg] | None = None,
```
before deprecated map args.
- [ ] **Step 3: Use same mixing guard and normalization**
`use_ref()` has no auto-map fallback, so logic is simpler:
```python
_reject_mixed_binding_styles(...)
node_input = (
normalize_input_bindings(input)
if input is not None
else _canonical_input_bindings(
normalize_input_mapping(in_map),
normalize_input_values(input_values),
)
)
node_output = (
normalize_output_bindings(output)
if output is not None
else _canonical_output_bindings(normalize_output_mapping(out_map))
)
```
- [ ] **Step 4: Run focused test**
```bash
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_use_ref_accepts_canonical_binding_dicts -q
```
Expected: pass.
---
## Task 4: Deprecate Map Sugar Explicitly
**Files:**
- Modify: `src/wf_authoring/builder/core.py`
- Test: `tests/authoring/test_builder.py`
- [ ] **Step 1: Add warning helper**
Add:
```python
def _warn_deprecated_binding_sugar(
*,
in_map: object | None,
input_values: object | None,
out_map: object | None,
) -> None:
"""Warn when callers explicitly use map sugar instead of canonical bindings."""
used = [
name
for name, value in (
("in_map", in_map),
("input_values", input_values),
("out_map", out_map),
)
if value is not None
]
if not used:
return
warnings.warn(
f"{', '.join(used)} are deprecated WorkflowBuilder sugar; use canonical "
"input/output binding lists instead",
DeprecationWarning,
stacklevel=3,
)
```
Auto-mapping when args are omitted must not warn.
- [ ] **Step 2: Add warning tests**
Add:
```python
def test_builder_warns_when_explicit_deprecated_maps_are_used() -> None:
builder = WorkflowBuilder(
name="deprecated_maps",
input_schema=AutoBindInput,
state_schema=AutoBindState,
output_schema=AutoBindOutput,
)
with pytest.warns(DeprecationWarning, match="canonical input/output"):
builder.use(
auto_bind_node,
in_map={"input.text": "text"},
out_map={"text": "state.text"},
)
```
Add:
```python
def test_builder_auto_mapping_does_not_warn() -> None:
builder = WorkflowBuilder(
name="auto_map_no_warning",
input_schema=AutoBindInput,
state_schema=AutoBindState,
output_schema=AutoBindOutput,
)
with warnings.catch_warnings():
warnings.simplefilter("error", DeprecationWarning)
builder.use(auto_bind_node)
```
Import `warnings` in the test file.
- [ ] **Step 3: Call warning helper**
In `use()` and `use_ref()`, after the mixing guard:
```python
_warn_deprecated_binding_sugar(
in_map=in_map,
input_values=input_values,
out_map=out_map,
)
```
- [ ] **Step 4: Run focused warning tests**
```bash
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_warns_when_explicit_deprecated_maps_are_used tests/authoring/test_builder.py::test_builder_auto_mapping_does_not_warn -q
```
Expected: both pass.
---
## Task 5: Reject Mixed Styles and Dict Keys Clearly
**Files:**
- Modify: `src/wf_authoring/builder/core.py`
- Modify: `src/wf_authoring/builder/mapping.py`
- Test: `tests/authoring/test_builder.py`
- [ ] **Step 1: Add mixed-style tests**
Add:
```python
def test_builder_rejects_mixed_canonical_and_deprecated_input_styles() -> None:
builder = WorkflowBuilder(
name="mixed_input_styles",
input_schema=AutoBindInput,
state_schema=AutoBindState,
output_schema=AutoBindOutput,
)
with pytest.raises(TypeError, match="cannot mix canonical input"):
builder.use(
auto_bind_node,
input=[{"target": "text", "path": "input.text"}],
in_map={"input.text": "text"},
)
```
Add:
```python
def test_builder_rejects_mixed_canonical_and_deprecated_output_styles() -> None:
builder = WorkflowBuilder(
name="mixed_output_styles",
input_schema=AutoBindInput,
state_schema=AutoBindState,
output_schema=AutoBindOutput,
)
with pytest.raises(TypeError, match="cannot mix canonical output"):
builder.use(
auto_bind_node,
output=[{"source": "text", "target": "state.text"}],
out_map={"text": "state.text"},
)
```
- [ ] **Step 2: Add dict-key diagnostic test**
Python literal dicts cannot contain dict keys, so test the normalizer directly with a custom `Mapping` that yields a structural dict key:
```python
class _StructuralKeyMap:
def items(self):
return [
(
{"root": "input", "parts": ["email.address"]},
"payload.email",
)
]
def test_input_map_rejects_structural_dict_keys_with_clear_message() -> None:
with pytest.raises(TypeError, match="structural path dicts cannot be map keys"):
normalize_input_mapping(_StructuralKeyMap())
```
Import `normalize_input_mapping` from `wf_authoring.builder.mapping`.
- [ ] **Step 3: Implement dict-key guard**
In `normalize_input_mapping()`:
```python
def _reject_mapping_path_key(value: object, *, field_name: str) -> None:
if isinstance(value, Mapping):
raise TypeError(
f"structural path dicts cannot be map keys in {field_name}; "
"use canonical input/output binding lists instead"
)
```
Call it on source keys for input maps and source keys for output maps before coercion.
Do not reject structural dict values, because values are allowed:
```python
out_map={"result": {"root": "state", "parts": ["score"]}}
```
- [ ] **Step 4: Run focused tests**
```bash
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_rejects_mixed_canonical_and_deprecated_input_styles tests/authoring/test_builder.py::test_builder_rejects_mixed_canonical_and_deprecated_output_styles tests/authoring/test_builder.py::test_input_map_rejects_structural_dict_keys_with_clear_message -q
```
Expected: all pass.
---
## Task 6: Docs
**Files:**
- Modify: `docs/structural_refs.md`
- Modify: `docs/authoring_sketch.md`
- Modify: `docs/core_state_mapping_and_merge.md`
- [ ] **Step 1: Update structural refs authoring example**
In `docs/structural_refs.md`, replace the current map-sugar-first example with canonical binding list example:
```python
g.use(
node,
input=[
{
"target": {"root": "local", "parts": ["payload.email"]},
"path": {"root": "input", "parts": ["email.address"]},
}
],
output=[
{
"source": {"root": "local", "parts": ["result.score"]},
"target": {"root": "state", "parts": ["score"]},
}
],
)
```
Then state:
```text
`in_map`, `input_values`, and `out_map` remain deprecated Python sugar.
Structural path dicts are not valid map keys; use canonical binding lists when
working from JSON/MCP.
```
- [ ] **Step 2: Update authoring sketch**
In `docs/authoring_sketch.md`, update the API sketch from:
```python
use(node_spec, id=..., in_map=..., out_map=...)
```
to:
```python
use(node_spec, id=..., input=[...], output=[...])
```
Then mention:
```text
`in_map`, `input_values`, and `out_map` are compatibility sugar for Python
authors, not the preferred saved or MCP-facing shape.
```
- [ ] **Step 3: Update core mapping docs**
In `docs/core_state_mapping_and_merge.md`, ensure the docs say:
```text
The canonical public shape is list-of-binding structs. Deprecated map fields
are parse-only compatibility inputs at core level and Python sugar at builder
level.
```
---
## Task 7: Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused authoring builder tests**
```bash
uv run --with pytest pytest tests/authoring/test_builder.py -q
```
Expected: pass.
- [ ] **Step 2: Run authoring tests**
```bash
uv run --with pytest pytest tests/authoring -q
```
Expected: pass.
- [ ] **Step 3: Run full tests**
```bash
uv run --with pytest pytest -q
```
Expected: pass.
- [ ] **Step 4: Run lint/type checks**
```bash
uvx ruff check src/wf_authoring tests/authoring
uvx ruff format --check src/wf_authoring tests/authoring docs/structural_refs.md docs/authoring_sketch.md docs/core_state_mapping_and_merge.md
uv run basedpyright --level error src/wf_authoring tests/authoring
```
Expected:
- ruff check passes
- format check passes or reports only markdown files if ruff does not handle them
- basedpyright reports `0 errors`
---
## Self-Review Checklist
- `g.use(input=[...], output=[...])` exists.
- `g.use_ref(input=[...], output=[...])` exists.
- `input_values` still exists, but emits `DeprecationWarning` when explicitly used.
- `in_map` and `out_map` still exist, but emit `DeprecationWarning` when explicitly used.
- Auto-mapping does not warn.
- Canonical list inputs support structural path dicts inside binding structs.
- Structural dicts as map keys are rejected with a clear message.
- Saved/core `NodeUse` output remains canonical `input` / `output`; deprecated map fields do not reappear in dumps.
@@ -0,0 +1,399 @@
# MCP Frontend Structural Paths and Reducers Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Expose structural graph paths, canonical builder bindings, and structural reducer refs through the MCP/workflow frontend so LLM clients stop learning dotted-string separator conventions as canonical.
**Architecture:** Treat MCP as a frontend over the platform model, not the source of truth. The MCP tools should accept compatibility strings where existing users need them, but list/inspect/create responses should prefer canonical structs: `input` / `output` binding lists, path `{root, parts}` objects, and reducer refs with structural `ref` objects for configured reducers. Existing raw-plan escape hatches remain, but the recommended workflow-authoring path should produce canonical model-shaped JSON.
**Tech Stack:** Python 3.14, Pydantic v2, `wf_mcp.workflow_surface`, `wf_artifacts.drafts`, `wf_artifacts.factory`, `wf_core` models, pytest, basedpyright, ruff.
---
## Dependency
Run this plan **after**:
```text
2026-05-21-reducer-ref-structural-capability.md
```
because MCP should expose the final `ReducerRef` model shape, not invent a parallel frontend shape.
---
## Current Problems
MCP/workflow frontend still has several old shapes:
- workflow plans and draft APIs often show `in_map`, `input_values`, `out_map`
- state schema examples still use legacy `fields`
- reducer refs are often shown as dotted strings only
- inspect/list tool outputs may not clearly distinguish canonical structs from display strings
The result: an LLM client can build runnable workflows, but it learns the wrong authoring shape and then has to guess separator semantics.
---
## Target Frontend Shape
Recommended node use shape:
```json
{
"id": "echo",
"type": "node",
"node": "demo.echo",
"input": [
{
"target": { "root": "local", "parts": ["text"] },
"path": { "root": "input", "parts": ["text"] }
},
{
"target": { "root": "local", "parts": ["limit"] },
"value": 3
}
],
"output": [
{
"source": { "root": "local", "parts": ["echoed"] },
"target": { "root": "state", "parts": ["echoed"] }
}
]
}
```
Recommended configured reducer shape:
```json
{
"ref": { "source": "wf.std", "capability_key": "modulo_add" },
"config": { "modulus": 10 }
}
```
Compact unconfigured reducer shorthand remains accepted:
```json
"wf.std.add"
```
---
## File Structure
- Inspect/modify: `src/wf_mcp/workflow_surface/models.py`
- Request/response models for create/compile/validate/call workflow tools
- Inspect/modify: `src/wf_mcp/workflow_surface/handlers.py`
- create draft/workflow helpers
- source/capability inspection payloads
- Inspect/modify: `src/wf_mcp/workflow_surface/tools.py`
- MCP tool schemas/descriptions
- Inspect/modify: `src/wf_artifacts/drafts/models.py`
- draft step/input/output shape if drafts still generate raw maps
- Inspect/modify: `src/wf_artifacts/drafts/adapter.py`
- draft-to-builder compile path; should use canonical `input` / `output`
- Inspect/modify docs:
- `docs/wf_mcp_operator_manual.md`
- `docs/workflow_drafts.md`
- `docs/wf_mcp_end_to_end_runbook.md`
- `docs/structural_refs.md`
- Tests:
- `tests/wf_mcp/test_workflow_surface.py`
- `tests/wf_mcp/test_workflow_wrapper_hints.py`
- `tests/artifacts/test_draft_adapter.py`
- `tests/artifacts/test_draft_models.py`
- `tests/artifacts/test_draft_api.py`
---
## Task 1: Inventory MCP/Draft Surfaces That Emit Map Sugar
**Files:**
- Read-only first:
- `src/wf_mcp/workflow_surface/models.py`
- `src/wf_mcp/workflow_surface/handlers.py`
- `src/wf_artifacts/drafts/models.py`
- `src/wf_artifacts/drafts/adapter.py`
- [ ] **Step 1: Search old map fields**
Run:
```bash
rg -n '"in_map"|in_map|input_values|"out_map"|out_map|fields' src/wf_mcp src/wf_artifacts tests/wf_mcp tests/artifacts docs -g '*.py' -g '*.md'
```
- [ ] **Step 2: Categorize each hit**
Use these categories:
- compatibility input still accepted
- canonical output should be changed
- test fixture using old shape intentionally
- docs/example should migrate
- [ ] **Step 3: Write findings into this plan or a short docs note**
Add a small checklist under this task before implementation. Do not blindly replace all strings.
Findings from the first inventory pass:
- `src/wf_artifacts/drafts/adapter.py` is the highest-value runtime hit: it still
calls `WorkflowBuilder.use_ref(..., in_map=..., input_values=..., out_map=...)`
and `WorkflowBuilder.use(..., out_map=...)`, causing deprecation warnings from
MCP draft/workspace tests. This should be changed to canonical binding lists.
- Raw workflow-plan tests in `tests/wf_mcp/test_service.py`,
`tests/wf_mcp/test_broker_server.py`, `tests/wf_mcp/test_workflow_surface.py`,
and `tests/artifacts/test_factory.py` intentionally exercise raw-plan
compatibility. Do not bulk-rewrite those while raw-plan escape hatches remain.
- Draft model tests still use `state_schema.fields` as compatibility input. That
can stay as parse input, but new docs/examples should prefer JSON Schema
`properties`.
- Docs already explain parse-only compatibility in several places, but older
operator/runbook examples still need canonical `input` / `output` examples.
---
## Task 2: Draft Adapter Emits Canonical Builder Bindings
**Files:**
- Modify: `src/wf_artifacts/drafts/adapter.py`
- Modify: `tests/artifacts/test_draft_adapter.py`
- [ ] **Step 1: Add/adjust test**
Add a test proving a draft compiles through `WorkflowBuilder.use_ref(..., input=[...], output=[...])` or directly produces canonical `NodeUse.input` / `output`.
Expected assertion:
```python
node = workflow.nodes[0]
dumped = node.model_dump(mode="json")
assert "in_map" not in dumped
assert "out_map" not in dumped
assert dumped["input"][0]["target"] == {"root": "local", "parts": ["text"]}
assert dumped["input"][0]["path"] == {"root": "input", "parts": ["text"]}
```
- [ ] **Step 2: Update adapter**
Where it currently calls:
```python
builder.use_ref(..., in_map=step.in_, input_values=..., out_map=step.out)
```
convert draft structures into:
```python
input=[...]
output=[...]
```
If draft models still store maps, transform them into canonical binding dicts at the adapter boundary.
- [ ] **Step 3: Run draft adapter tests**
```bash
uv run --with pytest pytest tests/artifacts/test_draft_adapter.py -q
```
Expected: pass and no new deprecation warnings from the adapter.
---
## Task 3: Workflow Surface Requests Prefer Canonical Shapes
**Files:**
- Modify: `src/wf_mcp/workflow_surface/models.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `tests/wf_mcp/test_workflow_surface.py`
- [ ] **Step 1: Add schema tests for canonical binding request fields**
Find the create/compile draft request model and assert its JSON Schema includes:
```json
"input": {"type": "array", ...}
"output": {"type": "array", ...}
```
and does not force `in_map` / `out_map` as the primary example.
- [ ] **Step 2: Update Pydantic models**
Prefer these field names in MCP-facing request models:
```python
input: list[InputBindingLike] = Field(default_factory=list, description=...)
output: list[OutputBindingLike] = Field(default_factory=list, description=...)
```
If compatibility maps remain:
```python
in_map: dict[str, str] | None = Field(default=None, deprecated=True, description=...)
out_map: dict[str, str] | None = Field(default=None, deprecated=True, description=...)
```
If Pydantic `deprecated=True` causes schema issues, document deprecation in descriptions instead.
- [ ] **Step 3: Update handlers**
Handlers should pass canonical lists to builder/artifact APIs.
- [ ] **Step 4: Run workflow surface tests**
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py -q
```
Expected: pass.
---
## Task 4: Inspect/List Outputs Show Canonical Refs and Display Strings Separately
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `src/wf_platform/sources.py` if inventory models need fields
- Modify: tests in `tests/wf_mcp`
- [ ] **Step 1: Add response-shape assertions**
For source/capability inspection responses, assert reducers include enough info:
```json
{
"name": "wf.std.add",
"ref": { "source": "wf.std", "capability_key": "add" },
"description": "..."
}
```
Use `name` as display, `ref` as canonical.
- [ ] **Step 2: Update inventory models only if needed**
If `ReducerInventory` currently has only `name`, add:
```python
ref: CapabilityRef
```
or a serializable equivalent.
Keep old `name` for display.
- [ ] **Step 3: Run platform/MCP inventory tests**
```bash
uv run --with pytest pytest tests/platform/test_inventory.py tests/wf_mcp/test_service.py tests/wf_mcp/test_workflow_surface.py -q
```
Expected: pass.
---
## Task 5: Docs and MCP Tool Descriptions
**Files:**
- Modify: `docs/wf_mcp_operator_manual.md`
- Modify: `docs/workflow_drafts.md`
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
- Modify: `docs/structural_refs.md`
- Modify MCP tool descriptions in `src/wf_mcp/workflow_surface/tools.py` if needed
- [ ] **Step 1: Replace primary examples**
Replace examples that teach:
```json
"in_map": {"input.text": "text"}
```
with:
```json
"input": [{"target": {"root": "local", "parts": ["text"]}, "path": {"root": "input", "parts": ["text"]}}]
```
- [ ] **Step 2: Keep compatibility notes**
Add:
```text
`in_map`, `input_values`, and `out_map` are compatibility inputs. New MCP/JSON
clients should use `input` and `output` binding lists.
```
- [ ] **Step 3: Update reducer examples**
Show:
```json
"reducer": "wf.std.add"
```
for compact unconfigured reducers, and:
```json
"reducer": {
"ref": {"source": "wf.std", "capability_key": "modulo_add"},
"config": {"modulus": 10}
}
```
for configured reducers.
---
## Task 6: Verification
- [ ] **Step 1: Focused artifact/MCP tests**
```bash
uv run --with pytest pytest tests/artifacts/test_draft_adapter.py tests/artifacts/test_draft_models.py tests/artifacts/test_draft_api.py tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_workflow_wrapper_hints.py -q
```
- [ ] **Step 2: Full tests**
```bash
uv run --with pytest pytest -q
```
- [ ] **Step 3: Static checks**
```bash
uvx ruff check src/wf_mcp src/wf_artifacts tests/wf_mcp tests/artifacts
uvx ruff format --check src/wf_mcp src/wf_artifacts tests/wf_mcp tests/artifacts
uv run basedpyright --level error src/wf_mcp src/wf_artifacts tests/wf_mcp tests/artifacts
```
Expected:
- tests pass
- ruff passes
- basedpyright reports `0 errors`
---
## Self-Review Checklist
- MCP-facing examples prefer canonical binding lists.
- Compatibility maps remain accepted where documented.
- Draft adapter no longer emits deprecated builder sugar warnings.
- Reducer refs display `name` and canonical `ref` distinctly where inventory exposes them.
- No MCP handler reparses reducer dotted names by first/last dot.
@@ -0,0 +1,433 @@
# ReducerRef Structural Capability Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move reducer references from ambiguous dotted strings toward structural capability refs while preserving string reducer names as parse-only shorthand.
**Architecture:** Reducers are source-owned capabilities, not graph paths. `ReducerRef` should carry a structural `CapabilityRef` plus config, while old `name` strings continue to validate at compatibility boundaries. Runtime reducer lookup can keep using display names temporarily through a compatibility property; artifact dependency extraction should stop reparsing dotted reducer names manually.
**Tech Stack:** Python 3.14, Pydantic v2, `wf_platform.refs.CapabilityRef`, `wf_core.models.reducers.ReducerRef`, `wf_artifacts.factory`, pytest, basedpyright, ruff.
---
## Current State
`src/wf_core/models/reducers.py`:
```python
class ReducerRef(BaseModel):
name: str
config: dict[str, Any] = Field(default_factory=dict)
```
`src/wf_artifacts/factory.py` extracts reducer dependencies by reparsing the display name:
```python
reducer_ref = CapabilityRef.parse(reducer.name)
requirements[reducer.name] = RequiredCapability(ref=reducer_ref, kind="reducer")
```
This is the same separator problem in another domain. `wf.std.add` is a capability ref, not a graph path.
---
## Canonical Shape
New canonical reducer ref:
```json
{
"ref": { "source": "wf.std", "capability_key": "add" },
"config": {}
}
```
Compatibility inputs:
```json
"wf.std.add"
```
```json
{ "name": "wf.std.add", "config": { "modulus": 10 } }
```
For now, `ReducerRef.name` remains available as a display/registry key compatibility property. Runtime reducer registries are still keyed by strings such as `wf.std.add`.
---
## File Structure
- Modify: `src/wf_core/models/reducers.py`
- Add `ref: CapabilityRef`
- Keep `name` as computed/display compatibility property
- Parse old string and old `name` object shapes
- Dump canonical `ref` shape in JSON/Python model dumps
- Modify: `src/wf_artifacts/factory.py`
- Use `reducer.ref` for required capabilities
- Keep dependency key as `reducer.name` for now
- Modify tests:
- `tests/core/test_nested_state_paths.py`
- `tests/core/test_schema_validation.py`
- `tests/artifacts/test_factory.py`
- `tests/artifacts/test_validation.py` if needed
- Modify docs:
- `docs/structural_refs.md`
- `docs/core_state_mapping_and_merge.md`
---
## Task 1: Pin ReducerRef Compatibility and Canonical Dump
**Files:**
- Modify: `tests/core/test_nested_state_paths.py`
- [ ] **Step 1: Add reducer ref tests**
Add:
```python
from wf_platform import CapabilityRef
def test_reducer_ref_accepts_string_shorthand_and_dumps_structural_ref() -> None:
reducer = ReducerRef.model_validate("wf.std.add")
assert reducer.ref == CapabilityRef(source="wf.std", capability_key="add")
assert reducer.name == "wf.std.add"
assert reducer.model_dump(mode="json") == {
"ref": {"source": "wf.std", "capability_key": "add"},
"config": {},
}
def test_reducer_ref_accepts_legacy_name_object_with_config() -> None:
reducer = ReducerRef.model_validate({
"name": "wf.std.modulo_add",
"config": {"modulus": 10},
})
assert reducer.ref == CapabilityRef(source="wf.std", capability_key="modulo_add")
assert reducer.name == "wf.std.modulo_add"
assert reducer.config == {"modulus": 10}
def test_reducer_ref_accepts_canonical_ref_object() -> None:
reducer = ReducerRef.model_validate({
"ref": {"source": "wf.std", "capability_key": "append"},
})
assert reducer.name == "wf.std.append"
```
- [ ] **Step 2: Run focused tests to verify red**
```bash
uv run --with pytest pytest tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_string_shorthand_and_dumps_structural_ref tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_legacy_name_object_with_config tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_canonical_ref_object -q
```
Expected: fail because `ReducerRef` does not parse strings and has no `ref`.
---
## Task 2: Implement Structural ReducerRef
**Files:**
- Modify: `src/wf_core/models/reducers.py`
- [ ] **Step 1: Update imports**
Add:
```python
from collections.abc import Mapping
from pydantic import computed_field, model_validator
from wf_platform import CapabilityRef
```
- [ ] **Step 2: Change model fields**
Change `ReducerRef` to:
```python
class ReducerRef(BaseModel):
"""Reference to one reducer capability plus JSON-compatible configuration."""
ref: CapabilityRef
config: dict[str, Any] = Field(default_factory=dict)
```
- [ ] **Step 3: Add compatibility validator**
Add:
```python
@model_validator(mode="before")
@classmethod
def _coerce_legacy_shapes(cls, value: object) -> object:
if isinstance(value, str):
return {"ref": CapabilityRef.parse(value)}
if not isinstance(value, Mapping):
return value
data = dict(value)
if "ref" not in data and "name" in data:
data["ref"] = CapabilityRef.parse(str(data.pop("name")))
return data
```
Do not parse arbitrary dotted strings anywhere else.
- [ ] **Step 4: Add name compatibility property**
Add:
```python
@computed_field
@property
def name(self) -> str:
"""Display/registry compatibility key for existing reducer catalogs."""
return str(self.ref)
```
If `CapabilityRef.__str__` does not produce `source.capability_key`, use its display helper or add one there.
- [ ] **Step 5: Run reducer ref tests**
```bash
uv run --with pytest pytest tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_string_shorthand_and_dumps_structural_ref tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_legacy_name_object_with_config tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_canonical_ref_object -q
```
Expected: pass.
---
## Task 3: Update Reducer Field Serializers and Existing Expectations
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Modify tests that assert reducer dumps
- [ ] **Step 1: Inspect current reducer dump helper**
Current helper:
```python
def _dump_reducer_keyword(reducer: ReducerRef) -> str | dict[str, Any]:
if not reducer.config:
return reducer.name
return reducer.model_dump(mode="json")
```
Decide canonical output:
- For no-config reducers, keep string shorthand in JSON Schema `reducer` keyword for readability.
- For configured reducers, dump canonical object:
```json
{
"ref": { "source": "wf.std", "capability_key": "modulo_add" },
"config": { "modulus": 10 }
}
```
This keeps common schema compact while avoiding string parsing for config objects.
- [ ] **Step 2: Update helper**
Use:
```python
def _dump_reducer_keyword(reducer: ReducerRef) -> str | dict[str, Any]:
if not reducer.config:
return reducer.name
return reducer.model_dump(mode="json")
```
This may already work after `ReducerRef.model_dump()` changes. Keep the helper but update tests.
- [ ] **Step 3: Run schema tests**
```bash
uv run --with pytest pytest tests/core/test_schema_validation.py tests/core/test_nested_state_paths.py -q
```
Expected: pass after updating expectations for configured reducer dumps if needed.
---
## Task 4: Update Artifact Reducer Dependency Extraction
**Files:**
- Modify: `src/wf_artifacts/factory.py`
- Modify: `tests/artifacts/test_factory.py`
- [ ] **Step 1: Add/adjust artifact test**
In `tests/artifacts/test_factory.py`, ensure reducer dependencies assert structural refs:
```python
def test_create_workflow_artifact_from_plan_adds_reducer_dependencies() -> None:
...
reducer = artifact.required_capability_map()["wf.std.max"]
assert reducer.ref.source == "wf.std"
assert reducer.ref.capability_key == "max"
assert reducer.logical_source == "wf.std"
assert reducer.capability_name == "max"
assert reducer.kind == "reducer"
```
Add a configured reducer payload test:
```python
def test_create_workflow_artifact_from_plan_accepts_structural_reducer_ref() -> None:
plan = minimal_plan()
plan["state_schema"] = {
"type": "object",
"properties": {
"score": {
"type": "integer",
"reducer": {
"ref": {"source": "wf.std", "capability_key": "max"},
"config": {},
},
}
},
}
artifact = create_workflow_artifact_from_plan(...)
assert "wf.std.max" in artifact.required_capability_map()
```
- [ ] **Step 2: Update extraction**
Change:
```python
reducer_ref = CapabilityRef.parse(reducer.name)
requirements[reducer.name] = RequiredCapability(ref=reducer_ref, kind="reducer")
```
to:
```python
requirements[reducer.name] = RequiredCapability(ref=reducer.ref, kind="reducer")
```
- [ ] **Step 3: Run artifact tests**
```bash
uv run --with pytest pytest tests/artifacts/test_factory.py tests/artifacts/test_validation.py -q
```
Expected: pass.
---
## Task 5: Runtime Compatibility Check
**Files:**
- Tests only unless failures require runtime changes
- [ ] **Step 1: Run reducer runtime tests**
```bash
uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_atomic_state_patches.py -q
```
Expected: pass because `reducer.name` remains a compatibility registry key.
- [ ] **Step 2: If runtime fails**
Only if needed, update lookup code to use `reducer.name` as the compatibility string key. Do not make runtime registries structural in this pass.
---
## Task 6: Docs
**Files:**
- Modify: `docs/structural_refs.md`
- Modify: `docs/core_state_mapping_and_merge.md`
- [ ] **Step 1: Update reducer docs**
In `docs/structural_refs.md`, replace temporary wording with:
```text
Canonical configured reducer refs use `ref`:
{
"ref": {"source": "wf.std", "capability_key": "modulo_add"},
"config": {"modulus": 10}
}
String reducer names such as `wf.std.add` remain shorthand for unconfigured
reducers and compatibility display.
```
- [ ] **Step 2: Update merge docs**
In `docs/core_state_mapping_and_merge.md`, update examples with both compact and configured forms:
```json
"reducer": "wf.std.add"
```
and:
```json
"reducer": {
"ref": {"source": "wf.std", "capability_key": "modulo_add"},
"config": {"modulus": 10}
}
```
---
## Task 7: Verification
- [ ] **Step 1: Focused tests**
```bash
uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py tests/artifacts/test_factory.py tests/artifacts/test_validation.py -q
```
- [ ] **Step 2: Full tests**
```bash
uv run --with pytest pytest -q
```
- [ ] **Step 3: Static checks**
```bash
uvx ruff check src/wf_core src/wf_artifacts tests/core tests/artifacts
uvx ruff format --check src/wf_core src/wf_artifacts tests/core tests/artifacts
uv run basedpyright --level error src/wf_core src/wf_artifacts tests/core tests/artifacts
```
Expected:
- tests pass
- ruff passes
- basedpyright reports `0 errors`
---
## Self-Review Checklist
- `ReducerRef` canonical shape has `ref`, not only `name`.
- String shorthand still parses.
- Legacy `{name, config}` still parses.
- Runtime reducer lookup still works through `reducer.name`.
- Artifact dependency extraction uses `reducer.ref`.
- No graph path parser is used for reducer refs.
@@ -0,0 +1,570 @@
# State Schema and Reducer Ref Path Sweep Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Remove remaining dotted-string ambiguity from state field paths, then prepare `ReducerRef` to stop treating reducer capability names as opaque dotted strings.
**Architecture:** Do this in two independent passes. First, make `StateSchema` / `StateFieldDecl` preserve `StatePath` semantics internally and serialize state paths structurally where possible. Second, introduce a structural reducer capability ref while keeping string reducer names as parse-only shorthand. The state-schema pass is the immediate correctness fix; reducer refs are a follow-up because they touch artifacts/source dependencies.
**Tech Stack:** Python 3.14, Pydantic v2, `wf_core.paths.StatePath`, `wf_platform.refs.CapabilityRef`, `wf_core.models.schemas`, pytest, basedpyright, ruff.
---
## Why This Plan Exists
We just moved graph/node bindings toward structural paths:
```json
{ "root": "state", "parts": ["person.name"] }
```
But state schema indexing still builds rootless dotted strings in places:
```python
path = f"{prefix}.{name}"
StatePath.of(path)
```
That can corrupt JSON Schema property names containing dots:
```json
{
"type": "object",
"properties": {
"person.name": { "type": "string", "reducer": "wf.std.replace" }
}
}
```
The intended state path is:
```text
state -> "person.name"
```
not:
```text
state -> person -> name
```
Reducer refs have a similar-looking but different issue. `wf.std.add` is not a graph path; it is a capability ref. That cleanup should use `CapabilityRef`, not `StatePath`.
---
## Scope
### In Scope Now
- Keep exact JSON Schema property names as literal `StatePath.parts`.
- Add typed state-field indexing keyed by `StatePath`.
- Keep old string-keyed `field_map()` compatibility.
- Make `StateFieldDecl.path` dumps structural if the rest of the core path dump has already moved structural.
- Add tests proving literal dotted property names are not split.
- Document that state schema paths and reducer refs are different domains.
### Follow-Up Scope
- Change `ReducerRef` to carry a structural `CapabilityRef`.
- Keep `ReducerRef(name="wf.std.add")` shorthand as parse-only compatibility.
- Update artifact dependency extraction to use structural reducer refs.
Do not mix these two passes unless the state schema work forces a reducer model touch.
---
## Current State
Relevant files:
- `src/wf_core/models/schemas.py`
- `StateFieldDecl.path: StatePath`
- `StateFieldDecl._serialize_path()` currently returns `str(path)`
- `StateSchema.field_map()` returns `dict[str, StateFieldDecl]`
- `_iter_state_field_declarations(...)` builds `path` as dotted string
- `_set_state_property_schema(...)` receives `path_parts`
- `src/wf_core/models/reducers.py`
- `ReducerRef.name: str`
- `src/wf_authoring/schemas.py`
- `_iter_model_metadata(...)` builds rootless dotted strings from Pydantic field names
- `_flatten_state_properties(...)` and `_lookup_mutable_property_schema(...)` split strings with `.`
- `src/wf_core/runtime/ops/state.py`
- uses `workflow.state_schema.field_map()` and string keys
Important distinction:
```text
State path: graph data path, should use StatePath
Reducer name: source capability ref, should use CapabilityRef later
```
---
## Task 1: Pin Literal Dotted State Property Behavior
**Files:**
- Test: `tests/core/test_nested_state_paths.py`
- Test: `tests/core/test_schema_validation.py`
- [ ] **Step 1: Add failing StateSchema field-index test**
Add to `tests/core/test_nested_state_paths.py`:
```python
def test_state_schema_preserves_literal_dotted_property_names() -> None:
schema = StateSchema.model_validate({
"type": "object",
"properties": {
"person.name": {"type": "string", "reducer": "wf.std.replace"}
},
})
fields = schema.field_index()
assert set(fields) == {StatePath(("person.name",))}
assert fields[StatePath(("person.name",))].path == StatePath(("person.name",))
```
Expected failure: `field_index()` does not exist, or the path is split as `("person", "name")`.
- [ ] **Step 2: Add compatibility `field_map()` test**
Add:
```python
def test_state_schema_field_map_keeps_display_key_for_literal_dotted_property() -> None:
schema = StateSchema.model_validate({
"type": "object",
"properties": {
"person.name": {"type": "string", "reducer": "wf.std.replace"}
},
})
fields = schema.field_map()
assert set(fields) == {"person.name"}
assert fields["person.name"].path == StatePath(("person.name",))
```
This keeps old callers alive but makes the value typed/correct.
- [ ] **Step 3: Run focused tests to verify red**
```bash
uv run --with pytest pytest tests/core/test_nested_state_paths.py::test_state_schema_preserves_literal_dotted_property_names tests/core/test_nested_state_paths.py::test_state_schema_field_map_keeps_display_key_for_literal_dotted_property -q
```
Expected: fail before implementation.
---
## Task 2: Add Typed State Field Index
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Test: `tests/core/test_nested_state_paths.py`
- [ ] **Step 1: Add path-parts traversal helper**
In `src/wf_core/models/schemas.py`, replace string-prefix recursion with tuple path parts.
Add helper:
```python
def _append_state_part(prefix: tuple[str, ...], name: str) -> tuple[str, ...]:
"""Append one JSON Schema property name as one literal StatePath segment."""
return (*prefix, name)
```
- [ ] **Step 2: Add `field_index()`**
Add to `StateSchema`:
```python
def field_index(self) -> dict[StatePath, StateFieldDecl]:
"""Return reducer-aware declarations keyed by exact typed state path."""
root_schema = self.model_dump(mode="json", exclude_none=True)
return {
path: field
for path, field in _iter_state_field_declarations(
self.properties,
root_schema,
prefix=(),
)
}
```
- [ ] **Step 3: Make `field_map()` compatibility wrapper**
Change `field_map()` to:
```python
def field_map(self) -> dict[str, StateFieldDecl]:
"""Return reducer-aware declarations keyed by rootless display path."""
return {".".join(path.parts): field for path, field in self.field_index().items()}
```
Note: this display map is ambiguous for literal dotted segments, but values are correct. New runtime code should move to `field_index()`.
- [ ] **Step 4: Update `_iter_state_field_declarations` signature**
Change from string prefix:
```python
prefix: str
) -> Iterator[tuple[str, StateFieldDecl]]:
```
to typed prefix:
```python
prefix: tuple[str, ...]
) -> Iterator[tuple[StatePath, StateFieldDecl]]:
```
Inside loop:
```python
path_parts = _append_state_part(prefix, name)
path = StatePath(path_parts)
display_path = ".".join(path.parts)
```
Use `display_path` only in error messages and reducer validation labels.
- [ ] **Step 5: Update yielded `StateFieldDecl` construction**
Change:
```python
"path": StatePath.of(path),
```
to:
```python
"path": path,
```
- [ ] **Step 6: Update recursive calls**
Pass:
```python
prefix=path.parts
```
not a dotted string.
- [ ] **Step 7: Run focused tests**
```bash
uv run --with pytest pytest tests/core/test_nested_state_paths.py::test_state_schema_preserves_literal_dotted_property_names tests/core/test_nested_state_paths.py::test_state_schema_field_map_keeps_display_key_for_literal_dotted_property -q
```
Expected: pass.
---
## Task 3: Move Runtime Lookup to Typed State Paths
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- Test: `tests/core/test_nested_state_paths.py`
- Test: `tests/core/test_atomic_state_patches.py`
- [ ] **Step 1: Inspect current runtime lookup**
Current likely shape:
```python
state_fields = workflow.state_schema.field_map()
field = state_fields.get(".".join(path.parts))
```
This should move to `field_index()` where available.
- [ ] **Step 2: Update runtime type hints**
Change helpers from:
```python
state_fields: Mapping[str, StateFieldDecl]
```
to:
```python
state_fields: Mapping[StatePath, StateFieldDecl]
```
- [ ] **Step 3: Use typed lookup**
When resolving reducer for a write target:
```python
declared_field = state_fields.get(target)
```
where `target` is already a `StatePath`.
If code currently has only path parts, construct:
```python
target = StatePath(tuple(path_parts))
```
- [ ] **Step 4: Update affected-field overlap helper**
If `_affected_state_fields(...)` compares string prefixes, make it compare tuple parts:
```python
def _is_prefix(prefix: tuple[str, ...], parts: tuple[str, ...]) -> bool:
return parts[: len(prefix)] == prefix
```
This preserves exact path semantics without reparsing dotted display text.
- [ ] **Step 5: Run runtime-focused tests**
```bash
uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_atomic_state_patches.py -q
```
Expected: pass.
---
## Task 4: Structural `StateFieldDecl.path` Dump
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Test: `tests/core/test_nested_state_paths.py`
- Test: `tests/core/test_schema_validation.py`
- [ ] **Step 1: Check current expectations**
Existing tests may expect:
```python
{"path": "state.person.name"}
```
Decide based on current path model direction. Since `StatePath` now serializes structurally elsewhere, prefer:
```json
{ "path": { "root": "state", "parts": ["person.name"] } }
```
- [ ] **Step 2: Change serializer**
Remove this serializer:
```python
@field_serializer("path")
def _serialize_path(self, path: StatePath) -> str:
return str(path)
```
or change it to:
```python
@field_serializer("path")
def _serialize_path(self, path: StatePath) -> dict[str, str | list[str]]:
return StatePath._serialize(path)
```
Prefer removal if Pydantic uses the existing `StatePath` serializer correctly.
- [ ] **Step 3: Update tests**
Update or add:
```python
def test_state_field_decl_model_dump_serializes_path_structurally() -> None:
field = StateFieldDecl(path=StatePath(("person.name",)), schema={"type": "string"})
dumped = field.model_dump(mode="json")
assert dumped["path"] == {"root": "state", "parts": ["person.name"]}
```
- [ ] **Step 4: Run focused schema tests**
```bash
uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py -q
```
Expected: pass after expectation updates.
---
## Task 5: Authoring State Metadata Path Sweep
**Files:**
- Modify: `src/wf_authoring/schemas.py`
- Test: `tests/authoring/test_schemas.py`
- [ ] **Step 1: Add failing authoring test for literal dotted Pydantic field alias if possible**
If Pydantic field aliases are already used in this project, add:
```python
class DotAliasState(BaseModel):
person_name: Annotated[
str,
Field(alias="person.name"),
state_field(reducer="wf.std.replace"),
]
def test_state_schema_from_preserves_literal_dotted_alias_paths() -> None:
schema = state_schema_from(DotAliasState)
fields = schema.field_index()
assert StatePath(("person.name",)) in fields
```
If field aliases are not supported by the current authoring schema flow, document that Python model field names remain Python identifiers and alias path support is out of scope.
- [ ] **Step 2: Replace string path traversal with tuple parts**
In `src/wf_authoring/schemas.py`, update metadata collection helpers:
```python
def _iter_model_metadata(
model_type: type[BaseModel],
*,
prefix: tuple[str, ...] = (),
) -> Iterator[tuple[tuple[str, ...], StateFieldMetadata]]:
```
Use one literal segment per field name or alias:
```python
field_name = field_info.alias or name
path = (*prefix, field_name)
```
- [ ] **Step 3: Update lookup helpers to accept tuple parts**
Change:
```python
_lookup_mutable_property_schema(schema_payload, path)
_state_field_default(value, path, property_schema)
```
to tuple-based forms:
```python
_lookup_mutable_property_schema(schema_payload, path_parts)
_state_field_default(value, path_parts, property_schema)
```
Use field name lookup carefully; default lookup for nested aliases may need to remain conservative.
- [ ] **Step 4: Run authoring schema tests**
```bash
uv run --with pytest pytest tests/authoring/test_schemas.py -q
```
Expected: pass.
---
## Task 6: ReducerRef Capability Ref Plan Stub
**Files:**
- Modify: `docs/structural_refs.md`
- Create: `docs/superpowers/plans/YYYY-MM-DD-reducer-ref-structural-capability.md`
- [ ] **Step 1: Document reducer refs are capability refs**
In `docs/structural_refs.md`, add:
```text
Reducer refs are capability refs, not graph paths. `wf.std.add` is shorthand
for source `wf.std`, capability key `add`. The reducer cleanup should move
ReducerRef toward structural CapabilityRef while keeping string reducer names
as parse-only shorthand.
```
- [ ] **Step 2: Create follow-up plan stub**
Create a separate plan with only the intended boundary:
- `ReducerRef.name: str` remains compatibility display/shorthand for now.
- Add `ReducerRef.ref: CapabilityRef` or replace `name` with a `CapabilityRef` after artifact/source dependency code is ready.
- Update artifact dependency extraction from reducer refs.
- Keep configured reducers as `{ref/name, config}`.
Do not implement reducer structural refs in the state-schema path sweep unless the user explicitly asks to combine them.
---
## Task 7: Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused core tests**
```bash
uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_atomic_state_patches.py tests/core/test_schema_validation.py -q
```
Expected: pass.
- [ ] **Step 2: Run authoring schema tests**
```bash
uv run --with pytest pytest tests/authoring/test_schemas.py -q
```
Expected: pass.
- [ ] **Step 3: Run full tests**
```bash
uv run --with pytest pytest -q
```
Expected: pass.
- [ ] **Step 4: Run lint/type checks**
```bash
uvx ruff check src/wf_core src/wf_authoring tests/core tests/authoring
uvx ruff format --check src/wf_core src/wf_authoring tests/core tests/authoring
uv run basedpyright --level error src/wf_core src/wf_authoring tests/core tests/authoring
```
Expected:
- ruff check passes
- format check passes
- basedpyright reports `0 errors`
---
## Self-Review Checklist
- JSON Schema property names containing dots stay one `StatePath` segment.
- Runtime reducer lookup uses `StatePath`, not rootless dotted strings.
- `field_map()` remains available for compatibility but is not the preferred internal API.
- `StateFieldDecl.path` no longer forces string serialization if the project has moved to structural path JSON.
- Reducer refs are documented as capability refs, not graph paths.
- Reducer structural ref implementation is not accidentally mixed into the state schema path sweep.
@@ -0,0 +1,644 @@
# Structural Capability Refs Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Stop treating dotted capability names as authoritative data by making source/capability/workflow refs structural in saved artifacts and internal runtime paths.
**Architecture:** Public MCP/tool arguments may keep accepting old string names for compatibility, but model validation normalizes them into structural refs immediately. Saved artifacts, deployments, and generated JSON should write structured fields; display strings are derived only for UI/tool readability. Runtime resolution binds by explicit source fields, not by guessing where a dot-separated string should split.
**Tech Stack:** Python 3.14, Pydantic models, `wf_platform` refs, `wf_artifacts` models, `wf_mcp` workflow surface and broker service tests.
---
## Current Problem
Dotted names currently carry multiple meanings:
- `context7.default.query-docs` means provider/profile/capability.
- `wf.std.replace` means system source/capability.
- `workflow.echo_wrapper.v1` means workflow artifact/version.
- `demo.foo.bar` may mean source `demo` with capability key `foo.bar`, or source `demo.foo` with capability key `bar`.
No parser can infer the right boundary from the string alone. Dotted strings are still useful as display names and human-entered compatibility inputs, but they must not be the stored source of truth.
## Canonical Shapes
### Source Identity
Use `SourceRef` for concrete or logical source identity.
```json
{
"provider": "context7",
"profile": "default"
}
```
System sources may omit profile:
```json
{
"provider": "wf.std",
"profile": null
}
```
For the first implementation pass, this can still serialize through the current `SourceRef` string internally, but the plan should leave room for `profile: null`.
### Capability Ref
Use an object with a source and a local key:
```json
{
"source": "demo",
"capability_key": "foo.bar"
}
```
`capability_key` is the name inside a known source. It may contain dots. It is not parsed for source meaning.
### Workflow Artifact Capability Ref
Workflow artifacts are not external source capabilities. Store them separately:
```json
{
"artifact_id": "echo_wrapper",
"version": 1
}
```
The display string `workflow.echo_wrapper.v1` remains computable for list output and old input parsing.
---
## File Structure
- Modify `src/wf_platform/refs.py`
- Add structural capability ref input/output support.
- Keep compact-string parsing for old inputs.
- Make JSON serialization emit object shape for canonical saves.
- Modify `src/wf_artifacts/refs.py`
- Add Pydantic validation/serialization for `WorkflowCapabilityRef`.
- Accept old `workflow.<artifact_id>.v<version>` strings as parse-only input.
- Serialize new saves as `{"artifact_id": "...", "version": 1}`.
- Modify `src/wf_artifacts/models.py`
- Make `RequiredCapability.ref` canonical structural JSON.
- Keep `logical_source` and `capability_name` as compatibility accessors only.
- Keep accepting old dict/map/string shapes.
- Make `WorkflowDeployment.bindings` save as structural list, not dict.
- Modify `src/wf_artifacts/references.py`
- Replace string-concatenated `logical_ref` construction with structural `CapabilityRef`.
- Return display strings only where legacy maps still require them.
- Modify `src/wf_mcp/workflow_surface/refs.py`
- Parse workflow-surface ids into a union of structural refs.
- Keep old string input compatibility for MCP tool calls.
- Modify `src/wf_mcp/workflow_surface/runtime_dependencies.py`
- Keep source-prefix binding logic for legacy plan node strings.
- Add a path for structural plan node refs when the plan model supports them.
- Keep the current dotted-local-name regression test.
- Modify `src/wf_mcp/workflow_surface/handlers.py`
- When saving artifacts from plans/drafts/workspaces, emit structural required capability refs.
- Keep response display names for humans/LLMs.
- Add or modify tests:
- `tests/wf_platform/test_refs.py`
- `tests/wf_artifacts/test_refs.py`
- `tests/wf_mcp/test_workflow_surface_refs.py`
- `tests/wf_mcp/test_service.py`
- `tests/wf_mcp/test_workflow_surface.py`
---
## Task 1: Make `CapabilityRef` Serialize Structurally
**Files:**
- Modify: `src/wf_platform/refs.py`
- Test: `tests/wf_platform/test_refs.py`
- [ ] **Step 1: Write failing tests**
Add tests that prove three things:
```python
from pydantic import BaseModel
from wf_platform import CapabilityRef, SourceRef
class RefHolder(BaseModel):
ref: CapabilityRef
def test_capability_ref_accepts_legacy_string_input() -> None:
holder = RefHolder.model_validate({"ref": "demo.foo.bar"})
# Legacy parsing is best-effort only. It stays for old input compatibility.
assert str(holder.ref) == "demo.foo.bar"
def test_capability_ref_accepts_structural_input() -> None:
holder = RefHolder.model_validate(
{"ref": {"source": "demo", "capability_key": "foo.bar"}}
)
assert holder.ref.source == SourceRef.parse("demo")
assert holder.ref.name == "foo.bar"
def test_capability_ref_serializes_structurally() -> None:
holder = RefHolder(
ref=CapabilityRef(source=SourceRef.parse("demo"), name="foo.bar")
)
assert holder.model_dump(mode="json")["ref"] == {
"source": "demo",
"capability_key": "foo.bar",
}
```
- [ ] **Step 2: Run tests to verify red**
Run:
```bash
uv run --with pytest pytest tests/wf_platform/test_refs.py -q
```
Expected: structural input or structural serialization fails because `CapabilityRef` currently serializes as a string.
- [ ] **Step 3: Implement structural validation/serialization**
Update `CapabilityRef.__get_pydantic_core_schema__` to:
- accept existing `CapabilityRef`
- accept legacy string via `CapabilityRef.parse`
- accept dict `{"source": str, "capability_key": str}`
- serialize as dict `{"source": str(self.source), "capability_key": self.name}`
Keep `__str__` unchanged because display strings are still useful.
- [ ] **Step 4: Run tests to verify green**
Run:
```bash
uv run --with pytest pytest tests/wf_platform/test_refs.py -q
```
Expected: all tests pass.
---
## Task 2: Make `WorkflowCapabilityRef` Structural
**Files:**
- Modify: `src/wf_artifacts/refs.py`
- Test: `tests/wf_artifacts/test_refs.py`
- [ ] **Step 1: Write failing tests**
```python
from pydantic import BaseModel
from wf_artifacts import WorkflowCapabilityRef
class WorkflowRefHolder(BaseModel):
ref: WorkflowCapabilityRef
def test_workflow_capability_ref_accepts_legacy_string_input() -> None:
holder = WorkflowRefHolder.model_validate({"ref": "workflow.echo_wrapper.v1"})
assert holder.ref.artifact_id == "echo_wrapper"
assert holder.ref.version == 1
def test_workflow_capability_ref_accepts_structural_input() -> None:
holder = WorkflowRefHolder.model_validate(
{"ref": {"artifact_id": "echo_wrapper", "version": 1}}
)
assert holder.ref.artifact_id == "echo_wrapper"
assert holder.ref.version == 1
def test_workflow_capability_ref_serializes_structurally() -> None:
holder = WorkflowRefHolder(ref=WorkflowCapabilityRef("echo_wrapper", 1))
assert holder.model_dump(mode="json")["ref"] == {
"artifact_id": "echo_wrapper",
"version": 1,
}
```
- [ ] **Step 2: Run tests to verify red**
Run:
```bash
uv run --with pytest pytest tests/wf_artifacts/test_refs.py -q
```
Expected: Pydantic validation fails because `WorkflowCapabilityRef` has no schema hook.
- [ ] **Step 3: Implement Pydantic schema hook**
Add a Pydantic core schema method to `WorkflowCapabilityRef` that accepts legacy string and structural dict input, then serializes structurally.
- [ ] **Step 4: Run tests to verify green**
Run:
```bash
uv run --with pytest pytest tests/wf_artifacts/test_refs.py -q
```
Expected: all tests pass.
---
## Task 3: Save Required Capabilities in New Shape
**Files:**
- Modify: `src/wf_artifacts/models.py`
- Modify: `src/wf_artifacts/references.py`
- Test: `tests/wf_artifacts/test_models.py`
- Test: existing workflow-surface artifact creation tests
- [ ] **Step 1: Write failing model serialization tests**
Add tests proving old inputs parse and new outputs dump structurally:
```python
from wf_artifacts import RequiredCapability, WorkflowArtifact
def test_required_capability_accepts_legacy_logical_fields_but_dumps_ref_object() -> None:
capability = RequiredCapability.model_validate(
{
"logical_source": "demo",
"capability_name": "foo.bar",
"kind": "node_spec",
}
)
assert capability.logical_source == "demo"
assert capability.capability_name == "foo.bar"
assert capability.model_dump(mode="json")["ref"] == {
"source": "demo",
"capability_key": "foo.bar",
}
def test_workflow_artifact_accepts_legacy_required_capability_map_but_dumps_list() -> None:
artifact = WorkflowArtifact.model_validate(
{
"id": "echo",
"version": 1,
"title": "Echo",
"input_schema": {"type": "object", "properties": {}},
"output_schema": {"type": "object", "properties": {}},
"outcomes": ["completed"],
"plan": {"name": "echo", "nodes": [], "edges": []},
"required_capabilities": {
"demo.foo.bar": {"kind": "node_spec"},
},
}
)
dumped = artifact.model_dump(mode="json")
assert dumped["required_capabilities"][0]["ref"] == {
"source": "demo.foo",
"capability_key": "bar",
}
```
This test documents legacy string parsing as best-effort. New artifact creation should avoid this path when source bindings are known.
- [ ] **Step 2: Run tests to verify red**
Run:
```bash
uv run --with pytest pytest tests/wf_artifacts/test_models.py -q
```
Expected: dumps still contain old string refs.
- [ ] **Step 3: Update model serialization**
After Task 1, `RequiredCapability.ref` should dump structurally automatically. Ensure `WorkflowArtifact._reject_duplicate_required_capabilities` still works by using `str(capability.capability_ref())` only for internal duplicate checking.
- [ ] **Step 4: Update reference creation**
In `src/wf_artifacts/references.py`, replace this conceptual behavior:
```python
logical_ref = "demo.foo.bar"
RequiredCapability(ref=CapabilityRef.parse(logical_ref), ...)
```
with structural construction:
```python
capability_ref = CapabilityRef(
source=SourceRef.parse(logical_source),
name=capability_name,
)
RequiredCapability(ref=capability_ref, ...)
```
Only derive `str(capability_ref)` for compatibility map keys.
- [ ] **Step 5: Run tests to verify green**
Run:
```bash
uv run --with pytest pytest tests/wf_artifacts/test_models.py tests/wf_mcp/test_workflow_surface.py -q
```
Expected: artifact creation still works; saved dumps use structural refs.
---
## Task 4: Make Deployment Bindings Structural on Save
**Files:**
- Modify: `src/wf_artifacts/models.py`
- Test: `tests/wf_artifacts/test_models.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
- [ ] **Step 1: Write failing binding serialization test**
```python
from wf_artifacts import WorkflowDeployment
def test_workflow_deployment_accepts_legacy_binding_map_but_dumps_structural_list() -> None:
deployment = WorkflowDeployment.model_validate(
{
"id": "echo.personal",
"artifact_id": "echo",
"artifact_version": 1,
"bindings": {"demo": "demo.personal", "wf.std": "wf.std"},
}
)
dumped = deployment.model_dump(mode="json")
assert dumped["bindings"] == [
{"logical_source": "demo", "concrete_source": "demo.personal"},
{"logical_source": "wf.std", "concrete_source": "wf.std"},
]
```
- [ ] **Step 2: Run tests to verify red**
Run:
```bash
uv run --with pytest pytest tests/wf_artifacts/test_models.py -q
```
Expected: output may already be a list, but confirm `SourceRef` serialization remains stable.
- [ ] **Step 3: Keep current binding shape but document it as structural**
`SourceBinding` already separates logical and concrete source fields. Add docstrings explaining:
- `logical_source` is an artifact-local alias.
- `concrete_source` is the deployment-selected source id.
- neither field is a capability name.
- [ ] **Step 4: Run tests**
Run:
```bash
uv run --with pytest pytest tests/wf_artifacts/test_models.py tests/wf_mcp/test_workflow_surface.py -q
```
Expected: all tests pass.
---
## Task 5: Stop Parsing Workflow Capability Strings as Generic Capabilities
**Files:**
- Modify: `src/wf_mcp/workflow_surface/refs.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_workflow_surface_refs.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
- [ ] **Step 1: Write tests for separate ref domains**
```python
from wf_artifacts import WorkflowCapabilityRef
from wf_mcp.workflow_surface.refs import parse_workflow_surface_capability_id
from wf_platform import CapabilityRef
def test_workflow_surface_ref_parser_keeps_workflow_artifacts_separate() -> None:
parsed = parse_workflow_surface_capability_id("workflow.echo_wrapper.v1")
assert isinstance(parsed, WorkflowCapabilityRef)
assert parsed.artifact_id == "echo_wrapper"
assert parsed.version == 1
def test_workflow_surface_ref_parser_keeps_source_capabilities_structural() -> None:
parsed = parse_workflow_surface_capability_id(
{"source": "demo", "capability_key": "foo.bar"}
)
assert isinstance(parsed, CapabilityRef)
assert str(parsed.source) == "demo"
assert parsed.name == "foo.bar"
```
- [ ] **Step 2: Run tests to verify red**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface_refs.py -q
```
Expected: dict input fails because parser currently accepts strings only.
- [ ] **Step 3: Update parser input type**
Allow parser input as:
```python
str | dict[str, object]
```
Rules:
- if string starts with `workflow.`, try `WorkflowCapabilityRef.parse`
- if dict has `artifact_id` and `version`, parse as workflow artifact ref
- otherwise parse as `CapabilityRef`
- [ ] **Step 4: Run workflow surface tests**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface_refs.py tests/wf_mcp/test_workflow_surface.py -q
```
Expected: all tests pass.
---
## Task 6: Keep Runtime Binding Source-Aware
**Files:**
- Modify: `src/wf_mcp/workflow_surface/runtime_dependencies.py`
- Test: `tests/wf_mcp/test_service.py`
- [ ] **Step 1: Keep the dotted-local-name regression**
Ensure this test remains:
```python
def test_service_runs_logical_source_plan_with_dotted_local_name() -> None:
...
plan = _single_echo_plan("logical_dotted_local_name_plan", "demo.foo.bar")
...
bindings=[{"logical_source": "demo", "concrete_source": "demo.personal"}]
...
assert run.output["echoed"] == "hello"
```
- [ ] **Step 2: Add a unit test for longest binding prefix**
Add a focused test through public runtime behavior where both `demo` and `demo.pro` are bound, and `demo.pro.foo.bar` resolves through `demo.pro`.
- [ ] **Step 3: Keep `_bound_node_names` or move it to a shared helper**
If more than one module needs source-prefix binding, move the helper to a small module such as:
```text
src/wf_artifacts/bindings.py
```
Do not move it preemptively if runtime remains the only caller.
- [ ] **Step 4: Run tests**
Run:
```bash
uv run --with pytest pytest tests/wf_mcp/test_service.py -q
```
Expected: all tests pass.
---
## Task 7: Update Docs to State the Rule
**Files:**
- Modify: `docs/workflow_capabilities.md`
- Create or modify: `docs/structural_refs.md`
- [ ] **Step 1: Add the core rule**
Add this rule prominently:
```text
Qualified names are display strings. They are not authoritative identifiers.
Saved workflow artifacts and deployments should store structural refs.
Old strings are accepted at API boundaries only for compatibility.
```
- [ ] **Step 2: Add examples**
Include examples for:
```json
{ "source": "demo", "capability_key": "foo.bar" }
```
```json
{ "artifact_id": "echo_wrapper", "version": 1 }
```
```json
{ "logical_source": "demo", "concrete_source": "demo.personal" }
```
- [ ] **Step 3: Mention path refs are separate**
Add:
```text
Capability refs and graph paths are different domains. Path strings such as
state.person.name should migrate separately to path models.
```
---
## Task 8: Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused tests**
Run:
```bash
uv run --with pytest pytest tests/wf_platform tests/wf_artifacts tests/wf_mcp/test_service.py tests/wf_mcp/test_workflow_surface.py -q
```
Expected: all selected tests pass.
- [ ] **Step 2: Run full tests when workspace is stable**
Run:
```bash
uv run --with pytest pytest -q
```
Expected: full suite passes, except any explicitly user-owned temporary rewrite tests if the user says to ignore them.
- [ ] **Step 3: Run linters/type checker**
Run:
```bash
uvx ruff check
uv run basedpyright --level error
```
Expected: no new errors.
---
## Self-Review Notes
- This plan does not require changing all workflow plan node refs in one pass. Runtime accepts legacy plan strings while artifact metadata becomes structural first.
- This plan does not solve graph path ambiguity. Graph paths need their own migration to `GraphPath`/`LocalPath`.
- This plan does not force `profile` into every model immediately. It keeps `SourceRef` compatible and documents `profile` as future concrete source structure.
- This plan keeps MCP/client compatibility by accepting old string inputs and continuing to display derived names.
@@ -0,0 +1,219 @@
# Structural Graph Paths Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Save canonical graph paths as structural objects while keeping old dotted strings as parse-only compatibility input.
**Architecture:** The core already has first-class path types: `GraphSourcePath`, `StatePath`, and `LocalPath`. Update those Pydantic hooks to accept structural dict input and serialize structurally in JSON mode. Keep `str(path)` for display and legacy fields. Do not redesign `NodeUse.input` / `output`; those structs already replaced deprecated `in_map` / `out_map`.
**Tech Stack:** Python 3.14, Pydantic core schema hooks, `wf_core.paths`, `wf_core.models.steps`, pytest.
---
## Current State
Canonical node bindings already exist:
```json
{
"input": [{ "path": "input.message", "target": "message" }],
"output": [{ "source": "echoed", "target": "state.echoed" }]
}
```
Internally these parse to:
- `GraphSourcePath`
- `LocalPath`
- `StatePath`
The remaining problem is serialization. These path objects currently dump as strings, so saved JSON still relies on dot-separated path grammar.
## Canonical Shape
Graph source paths:
```json
{ "root": "state", "parts": ["person", "name"] }
```
State write paths:
```json
{ "root": "state", "parts": ["person", "name"] }
```
Local node paths:
```json
{ "root": "local", "parts": ["payload", "text"] }
```
Local root remains explicit:
```json
{ "root": "local", "parts": [] }
```
Old strings such as `"state.person.name"` and `"."` remain accepted input.
---
## Task 1: Add Structural Serialization for Path Types
**Files:**
- Modify: `src/wf_core/paths.py`
- Test: `tests/core/test_path_values.py`
- [ ] **Step 1: Update tests first**
Change `test_pydantic_accepts_path_strings_and_serializes_strings` into structural JSON expectations:
```python
dumped = payload.model_dump(mode="json")
assert dumped["source"] == {"root": "input", "parts": ["user"]}
assert dumped["target"] == {"root": "state", "parts": ["person"]}
assert dumped["local"] == {"root": "local", "parts": ["user"]}
```
Keep `model_dump()` expectations if useful for Python-mode compatibility only if the implementation intentionally keeps Python mode as strings. Otherwise assert structural dumps in both modes.
- [ ] **Step 2: Add structural input tests**
Add a test:
```python
payload = Payload.model_validate({
"source": {"root": "input", "parts": ["user.name"]},
"target": {"root": "state", "parts": ["person.name"]},
"local": {"root": "local", "parts": ["payload.text"]},
})
assert payload.source == GraphSourcePath.input("user.name")
assert payload.target == StatePath.of("person.name")
assert payload.local == LocalPath.of("payload.text")
```
This documents that structural `parts` are literal field names. Old string inputs still split on dots for compatibility, but structural parts such as `"user.name"` are not split again.
- [ ] **Step 3: Implement path serializers**
In `src/wf_core/paths.py`, update each path type:
- `LocalPath` accepts string, object instance, and dict `{"root": "local", "parts": list[str]}`
- `GraphSourcePath` accepts string, object instance, and dict `{"root": "input"|"state"|"context", "parts": list[str]}`
- `StatePath` accepts string, object instance, and dict `{"root": "state", "parts": list[str]}`
Serialize as dicts in JSON mode:
```python
{"root": "local", "parts": list(value.parts)}
{"root": value.root, "parts": list(value.parts)}
{"root": "state", "parts": list(value.parts)}
```
- [ ] **Step 4: Run focused path tests**
Run:
```bash
uv run --with pytest pytest tests/core/test_path_values.py -q
```
Expected: all tests pass.
---
## Task 2: Update Canonical Node Binding Dumps
**Files:**
- Test: `tests/core/test_canonical_node_bindings.py`
- Test: `tests/authoring/test_builder.py`
- [ ] **Step 1: Update canonical node dump expectations**
In `tests/core/test_canonical_node_bindings.py`, update JSON-mode expectations:
```python
assert dumped["input"][1]["path"] == {"root": "input", "parts": ["message"]}
assert dumped["input"][1]["target"] == {"root": "local", "parts": ["message"]}
assert dumped["output"][0]["source"] == {"root": "local", "parts": ["echoed"]}
assert dumped["output"][0]["target"] == {"root": "state", "parts": ["echoed"]}
```
Deprecated `in_map` / `out_map` inputs should continue parsing, but dumps must omit those old fields and emit structural paths.
- [ ] **Step 2: Update authoring serialization expectations**
In `tests/authoring/test_builder.py`, update any `model_dump(mode="json")` expectations that currently assert path strings.
- [ ] **Step 3: Run focused binding/authoring tests**
Run:
```bash
uv run --with pytest pytest tests/core/test_canonical_node_bindings.py tests/authoring/test_builder.py -q
```
Expected: all tests pass.
---
## Task 3: Update Docs
**Files:**
- Modify: `docs/structural_refs.md`
- Modify: any path/core docs if directly relevant.
- [ ] **Step 1: Add graph path note**
Extend the path note in `docs/structural_refs.md`:
```text
New canonical graph path JSON uses root/parts objects. Old strings are accepted
at parse boundaries for compatibility.
```
- [ ] **Step 2: Add examples**
Include examples:
```json
{"root": "input", "parts": ["message"]}
{"root": "state", "parts": ["echoed"]}
{"root": "local", "parts": []}
```
---
## Task 4: Verification
- [ ] **Step 1: Run focused tests**
```bash
uv run --with pytest pytest tests/core/test_path_values.py tests/core/test_canonical_node_bindings.py tests/authoring/test_builder.py -q
```
- [ ] **Step 2: Run full tests**
```bash
uv run --with pytest pytest -q
```
- [ ] **Step 3: Run checks**
```bash
uvx ruff check src/wf_core/paths.py tests/core/test_path_values.py tests/core/test_canonical_node_bindings.py tests/authoring/test_builder.py
uv run basedpyright --level error src/wf_core/paths.py tests/core/test_path_values.py tests/core/test_canonical_node_bindings.py tests/authoring/test_builder.py
```
---
## Self-Review Notes
- This plan does not revive `in_map` / `out_map`; those remain deprecated parse-only fields.
- This plan relaxes path segment validation. Structural `parts` preserve literal field names, including dots and spaces. Old dotted string inputs still split on dots for compatibility.
- This plan changes saved JSON shape for canonical path fields, so broad tests are required.
@@ -0,0 +1,604 @@
# Concurrent Foreach Barrier Write Semantics Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Enforce deterministic and explicit write semantics when concurrent foreach sibling item lineages commit at the barrier.
**Architecture:** Keep per-node patch building unchanged. Add a barrier-only validation step before replaying item patches: inspect all item patch destination paths, reject ambiguous sibling writes, and allow multi-writer paths only when the exact declared state path has a reducer whose metadata is `mergeable`. Barrier replay still happens in item-index order through existing reducer logic.
**Tech Stack:** Python 3.14, dataclasses, Pydantic v2 models, pytest, `StatePath`, `StateSchema.field_index()`, `StatePatch`, `ForeachBarrierState`, and existing reducer definitions.
---
## Semantics
This slice owns sibling write policy at a foreach barrier.
Allowed:
- A destination path written by exactly one item lineage uses normal state rules.
- A destination path written by multiple item lineages is allowed only if that exact declared state path has a `mergeable` reducer.
- Multi-writer reducer replay is deterministic item-index order.
- Multiple nodes inside the same item lineage may write multiple paths; that is item-local overlay behavior from Slice 2.
Rejected:
- Multiple sibling item lineages writing the same destination path with missing reducer/default replace.
- Multiple sibling item lineages writing the same destination path with an `exclusive` reducer such as `wf.std.replace`.
- Sibling item lineages writing ancestor/descendant paths such as `state.person` and `state.person.name`.
Important distinction:
- This is **not** normal node output validation. Node-level `build_output_patch(...)` still rejects overlapping output paths inside one node.
- This is **not** reducer implementation work. Existing reducers stay pure and domain-agnostic.
- This is **not** deep merge policy. Ancestor/descendant sibling writes are rejected for now.
---
## Files
- Modify: `src/wf_core/runtime/ops/state.py`
- Add barrier write validation helpers.
- Call them from `build_barrier_patch(...)` before replay.
- Test: `tests/core/test_concurrent_foreach.py`
- Add end-to-end concurrent foreach conflict tests.
- Test: `tests/core/test_atomic_state_patches.py`
- Add focused `build_barrier_patch(...)` unit tests if end-to-end setup becomes too noisy.
- Modify: `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`
- Mark write semantics as implemented.
- Modify: `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`
- Link this plan under Slice 3.
---
### Task 1: Add Focused Barrier Same-Path Tests
**Files:**
- Modify: `tests/core/test_atomic_state_patches.py`
- [ ] **Step 1: Add imports**
Ensure `tests/core/test_atomic_state_patches.py` imports:
```python
from wf_core.runtime.ops.state import StatePatch, build_barrier_patch
```
If `StatePatch` is already imported, only add `build_barrier_patch`.
- [ ] **Step 2: Add test for same-path writes without reducer**
Append:
```python
def test_barrier_rejects_sibling_same_path_writes_without_reducer() -> None:
workflow = Workflow(
name="barrier_conflict",
input_schema=SchemaRef(properties={}),
state_schema=StateSchema.from_field_map(
{"value": StateField(type="string")}
),
output_schema=SchemaRef(properties={}),
node_defs=[],
start="unused",
nodes=[],
edges=[],
)
with pytest.raises(WorkflowExecutionError, match="multiple sibling writes"):
build_barrier_patch(
workflow,
[
StatePatch(changes={"state.value": "a"}),
StatePatch(changes={"state.value": "b"}),
],
{},
)
```
- [ ] **Step 3: Add test for explicit replace still rejected**
Append:
```python
def test_barrier_rejects_sibling_same_path_writes_with_explicit_replace() -> None:
workflow = Workflow(
name="barrier_replace_conflict",
input_schema=SchemaRef(properties={}),
state_schema=StateSchema.from_field_map(
{
"value": StateField(
type="string",
reducer=ReducerRef(name="wf.std.replace"),
)
}
),
output_schema=SchemaRef(properties={}),
node_defs=[],
start="unused",
nodes=[],
edges=[],
)
with pytest.raises(WorkflowExecutionError, match="mergeable reducer"):
build_barrier_patch(
workflow,
[
StatePatch(changes={"state.value": "a"}),
StatePatch(changes={"state.value": "b"}),
],
{},
)
```
- [ ] **Step 4: Add test for mergeable reducer allowing same-path writes**
Append:
```python
def test_barrier_allows_sibling_same_path_writes_with_non_replace_reducer() -> None:
workflow = Workflow(
name="barrier_reducer",
input_schema=SchemaRef(properties={}),
state_schema=StateSchema.from_field_map(
{
"seen": StateField(
type="array",
reducer=ReducerRef(name="wf.std.append"),
)
}
),
output_schema=SchemaRef(properties={}),
node_defs=[],
start="unused",
nodes=[],
edges=[],
)
patch = build_barrier_patch(
workflow,
[
StatePatch(changes={"state.seen": "a"}),
StatePatch(changes={"state.seen": "b"}),
],
{},
)
assert patch.changes["state.seen"] == ["a", "b"]
```
- [ ] **Step 5: Run tests and verify failures**
Run:
```bash
uv run pytest tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_same_path_writes_without_reducer tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_same_path_writes_with_explicit_replace tests/core/test_atomic_state_patches.py::test_barrier_allows_sibling_same_path_writes_with_non_replace_reducer -q
```
Expected before implementation:
```text
two reject tests fail because current barrier accepts replace semantics
```
The reducer test may already pass.
---
### Task 2: Add Ancestor/Descendant Conflict Tests
**Files:**
- Modify: `tests/core/test_atomic_state_patches.py`
- [ ] **Step 1: Add ancestor/descendant conflict test**
Append:
```python
def test_barrier_rejects_sibling_ancestor_descendant_writes() -> None:
workflow = Workflow(
name="barrier_ancestor_conflict",
input_schema=SchemaRef(properties={}),
state_schema=StateSchema.from_field_map(
{
"person": StateField(
type="object",
properties={"name": {"type": "string"}},
),
"person.name": StateField(type="string"),
}
),
output_schema=SchemaRef(properties={}),
node_defs=[],
start="unused",
nodes=[],
edges=[],
)
with pytest.raises(WorkflowExecutionError, match="overlapping sibling writes"):
build_barrier_patch(
workflow,
[
StatePatch(changes={"state.person": {"name": "Ada"}}),
StatePatch(changes={"state.person.name": "Grace"}),
],
{},
)
```
- [ ] **Step 2: Add same-item ancestor/descendant note test only if needed**
Do **not** add a same-item ancestor/descendant test unless current behavior changes unexpectedly. Same-node output overlap is already rejected by `build_output_patch(...)`, and multi-node same-item writes are item-local overlay behavior. This slice only owns sibling conflicts.
- [ ] **Step 3: Run focused test and verify failure**
Run:
```bash
uv run pytest tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_ancestor_descendant_writes -q
```
Expected before implementation:
```text
FAILED because current barrier replays both writes
```
---
### Task 3: Implement Barrier Write Analysis
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- [ ] **Step 1: Add helper dataclass**
Near `StatePatch`, add:
```python
@dataclass(slots=True, frozen=True)
class _BarrierWrite:
"""One item-lineage write observed before a barrier commit."""
item_index: int
path: StatePath
source_key: str
```
- [ ] **Step 2: Add reducer policy predicate**
Add below `build_barrier_patch(...)` or near private helpers:
```python
def _allows_sibling_writes(
path: StatePath,
state_fields: Mapping[StatePath, StateFieldDecl],
reducers: Mapping[str, ReducerDefinition] | None,
) -> bool:
field = state_fields.get(path)
if field is None or field.reducer is None:
return False
return reducer_allows_sibling_writes(field.reducer, reducers)
```
If `StateFieldDecl.reducer` is never `None` for undeclared/default fields, inspect the actual model and adjust:
```python
return field.reducer.name != "wf.std.replace"
```
but preserve the rule: only a mergeable reducer allows sibling same-path writes.
- [ ] **Step 3: Add overlap predicate for barrier paths**
Add:
```python
def _state_paths_overlap(left: StatePath, right: StatePath) -> bool:
left_parts = left.parts
right_parts = right.parts
return left_parts == right_parts or _is_prefix(left_parts, right_parts) or _is_prefix(
right_parts,
left_parts,
)
```
- [ ] **Step 4: Add write collection helper**
Add:
```python
def _barrier_writes(item_patches: Sequence[StatePatch]) -> list[_BarrierWrite]:
writes: list[_BarrierWrite] = []
for item_index, item_patch in enumerate(item_patches):
for destination in item_patch.changes:
path = StatePath.parse(destination)
writes.append(
_BarrierWrite(
item_index=item_index,
path=path,
source_key=destination,
)
)
return writes
```
Important: `item_index` here is the order in `item_patches`, which `foreach.py` already passes sorted by real item index. Do not infer item ids from path strings.
- [ ] **Step 5: Add validation helper**
Add:
```python
def validate_barrier_writes(
item_patches: Sequence[StatePatch],
state_fields: Mapping[StatePath, StateFieldDecl],
*,
reducers: Mapping[str, ReducerDefinition] | None = None,
) -> None:
"""Reject ambiguous sibling writes before replaying a foreach barrier.
Normal node patch validation handles one node output. This helper handles
writes from different foreach item lineages that will commit together.
"""
writes = _barrier_writes(item_patches)
for index, left in enumerate(writes):
for right in writes[index + 1 :]:
if left.item_index == right.item_index:
continue
if left.path == right.path:
if _allows_sibling_writes(left.path, state_fields, reducers):
continue
raise WorkflowExecutionError(
"multiple sibling writes to "
f"{left.source_key!r} require a mergeable reducer"
)
if _state_paths_overlap(left.path, right.path):
raise WorkflowExecutionError(
"overlapping sibling writes are not supported at a foreach "
f"barrier: {left.source_key!r} and {right.source_key!r}"
)
```
- [ ] **Step 6: Call validation from `build_barrier_patch(...)`**
In `build_barrier_patch(...)`, after:
```python
state_fields = workflow.state_schema.field_index()
```
add:
```python
validate_barrier_writes(item_patches, state_fields, reducers=reducers)
```
- [ ] **Step 7: Verify focused unit tests**
Run:
```bash
uv run pytest tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_same_path_writes_without_reducer tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_same_path_writes_with_explicit_replace tests/core/test_atomic_state_patches.py::test_barrier_allows_sibling_same_path_writes_with_non_replace_reducer tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_ancestor_descendant_writes -q
```
Expected: pass.
---
### Task 4: Add End-To-End Concurrent Foreach Coverage
**Files:**
- Modify: `tests/core/test_concurrent_foreach.py`
- [ ] **Step 1: Add same-path no-reducer workflow helper**
Append:
```python
def _same_path_replace_workflow() -> Workflow:
foreach = ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": "state.items",
"as": "item",
"mode": "concurrent",
"concurrent": {"max_active": 2, "max_outstanding": 2},
}
)
return Workflow(
name="concurrent_foreach_replace_conflict",
input_schema=SchemaRef(
type="object",
properties={"items": {"type": "array"}},
),
state_schema=StateSchema.from_field_map(
{
"items": StateField(type="array"),
"winner": StateField(type="string"),
}
),
output_schema=SchemaRef(type="object", properties={}),
node_defs=[
NodeDef(
name="write_winner",
input_schema=SchemaRef(
type="object",
properties={"value": {}},
required=["value"],
),
output_schema=SchemaRef(
type="object",
properties={"winner": {}},
required=["winner"],
),
outcomes=["ok"],
)
],
start="each",
nodes=[
foreach,
NodeUse.model_validate(
{
"id": "write_winner",
"type": "node",
"node": "write_winner",
"input": [{"target": "value", "path": "context.item"}],
"output": [{"source": "winner", "target": "state.winner"}],
}
),
],
edges=[
Edge.model_validate(
{"from": "each", "outcome": "loop", "to": "write_winner"}
),
Edge.model_validate({"from": "write_winner", "outcome": "ok", "to": END}),
Edge.model_validate({"from": "each", "outcome": "done", "to": END}),
],
)
```
- [ ] **Step 2: Add end-to-end rejection test**
Append:
```python
def test_sync_concurrent_foreach_rejects_sibling_replace_writes() -> None:
workflow = _same_path_replace_workflow()
with pytest.raises(WorkflowExecutionError, match="mergeable reducer"):
execute_workflow(
workflow,
{"items": ["a", "b"]},
{
"write_winner": lambda payload, _ctx: {
"outcome": "ok",
"output": {"winner": payload["value"]},
}
},
)
```
- [ ] **Step 3: Strengthen existing happy path**
The existing `test_sync_concurrent_foreach_interleaves_items_and_commits_at_barrier`
already proves explicit `wf.std.append` allows sibling writes to `state.seen`.
Do not duplicate it.
- [ ] **Step 4: Run focused end-to-end tests**
Run:
```bash
uv run pytest tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_rejects_sibling_replace_writes tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_interleaves_items_and_commits_at_barrier -q
```
Expected: pass.
---
### Task 5: Update Docs
**Files:**
- Modify: `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`
- Modify: `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`
- [ ] **Step 1: Verify ADR merge rules current state**
In `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`, under
`## Merge and Reducer Rules`, verify that the implementation status is
documented:
```markdown
Current barrier validation enforces this policy for sibling foreach item
lineages. Same-path sibling writes require a `mergeable` reducer on the exact
destination state path. Ancestor/descendant sibling writes are rejected until a
future explicit deep merge policy exists.
```
- [ ] **Step 2: Verify roadmap Slice 3**
In `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`,
under Slice 3, verify the plan points to this slice:
```markdown
Plan:
- See [`2026-05-22-concurrent-foreach-barrier-write-semantics.md`](2026-05-22-concurrent-foreach-barrier-write-semantics.md).
```
If implementing immediately, also mark it as implemented in Current State after tests pass.
- [ ] **Step 3: Verify docs mentions**
Run:
```bash
rg -n "sibling writes|ancestor/descendant|mergeable reducer|barrier write" docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md
```
Expected: the ADR and roadmap both mention the semantics.
---
### Task 6: Verification
**Files:**
- No new source files.
- [ ] **Step 1: Run focused core tests**
Run:
```bash
uv run pytest tests/core/test_atomic_state_patches.py tests/core/test_concurrent_foreach.py tests/core/test_foreach_barrier_state.py -q
```
Expected: pass.
- [ ] **Step 2: Run authoring smoke tests**
Run:
```bash
uv run pytest tests/authoring/test_demo_workflow.py tests/authoring/test_builder.py tests/authoring/test_ops.py -q
```
Expected: pass.
- [ ] **Step 3: Run full suite**
Run:
```bash
uv run pytest -q
```
Expected: pass, allowing known intentional environment-only skips.
- [ ] **Step 4: Run lint/type/format checks**
Run:
```bash
uvx ruff check src tests
uvx ruff format --check src tests docs
uv run basedpyright --level error src tests
```
Expected: all pass with 0 type errors.
---
## Self-Review
- Spec coverage: the plan covers same-path sibling writes, mergeable reducer requirements, replace rejection, ancestor/descendant rejection, deterministic reducer order, end-to-end foreach behavior, and docs.
- Placeholder scan: all tasks include concrete code or exact commands; no TBD placeholders.
- Type consistency: the plan uses existing `StatePatch`, `StatePath`, `StateFieldDecl`, `ReducerRef`, `StateSchema.from_field_map`, and `WorkflowExecutionError`.
@@ -0,0 +1,648 @@
# Concurrent Foreach Item Overlays Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make concurrent foreach item frames read their own buffered state writes so multi-step item bodies are safe.
**Architecture:** Keep `RunState.state` as committed parent state. Store item-local overlay patches in the foreach parent barrier, keyed by item index/frame id. `state_view_for_frame(...)` should return parent state plus that item's staged overlay; sibling overlays remain invisible. This plan does not implement sibling write conflict policy; Slice 3 owns write semantics across different item lineages.
**Tech Stack:** Python 3.14, dataclasses, pytest, existing `StatePatch`, `ForeachBarrierState`, `safe_resolve_path`, and runtime scheduler modules.
---
## Boundary With Slice 3
This slice answers:
- Can node B in the same concurrent item read node A's buffered state write?
- Can multi-step item bodies run without reading stale parent state?
- Are sibling item overlays isolated from each other?
This slice does **not** answer:
- Should two sibling items be allowed to write the same state path without a reducer?
- Should ancestor/descendant sibling writes conflict?
- Should barrier trace `state_changes` show raw per-item inputs or final aggregate values?
Those are Slice 3 write semantics. Do not add broad write-conflict policy here except what is already enforced by `build_output_patch(...)` for a single node output.
---
## Files
- Modify: `src/wf_core/runtime/foreach_state.py`
- Accumulate successful item patches per item instead of storing only one patch.
- Expose helpers to get item overlay patches by frame/index.
- Modify: `src/wf_core/runtime/ops/overlays.py`
- Replace the current no-op seam with parent-state plus item-local staged writes.
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Build output patches against the frame-visible state view, not always `run.state`.
- Append item-local patches for concurrent item frames.
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Remove the single-node item-body guard.
- Keep `item_error.action != "fail"` unsupported.
- Test: `tests/core/test_concurrent_foreach.py`
- Add multi-step item-body tests.
- Test: `tests/core/test_foreach_barrier_state.py`
- Add item patch accumulation tests.
---
### Task 1: Add Failing Multi-Step Overlay Tests
**Files:**
- Modify: `tests/core/test_concurrent_foreach.py`
- [ ] **Step 1: Add a two-node item body test**
Append this test:
```python
def test_sync_concurrent_foreach_item_reads_own_buffered_write() -> None:
workflow = _workflow(
state_schema=StateSchema.from_field_map(
{
"items": StateField(type="array"),
"scratch": StateField(type="string"),
"seen": StateField(
type="array",
reducer=ReducerRef(name="wf.std.append"),
),
}
),
foreach=ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": "state.items",
"as": "item",
"mode": "concurrent",
"concurrent": {"max_active": 2, "max_outstanding": 2},
}
),
)
workflow.node_defs.extend(
[
NodeDef(
name="read_scratch",
input_schema=SchemaRef(
type="object",
properties={"scratch": {}},
required=["scratch"],
),
output_schema=SchemaRef(
type="object",
properties={"seen": {}},
required=["seen"],
),
outcomes=["ok"],
)
]
)
workflow.nodes.append(
NodeUse.model_validate(
{
"id": "read_scratch",
"type": "node",
"node": "read_scratch",
"input": [{"target": "scratch", "path": "state.scratch"}],
"output": [{"source": "seen", "target": "state.seen"}],
}
)
)
workflow.edges = [
Edge.model_validate({"from": "each", "outcome": "loop", "to": "record"}),
Edge.model_validate({"from": "record", "outcome": "ok", "to": "read_scratch"}),
Edge.model_validate({"from": "read_scratch", "outcome": "ok", "to": END}),
Edge.model_validate({"from": "each", "outcome": "done", "to": END}),
]
run = execute_workflow(
workflow,
{"items": ["a", "b", "c"]},
{
"record": lambda payload, _ctx: {
"outcome": "ok",
"output": {"scratch": f"scratch:{payload['value']}"},
},
"read_scratch": lambda payload, _ctx: {
"outcome": "ok",
"output": {"seen": payload["scratch"]},
},
},
)
assert run.state["seen"] == ["scratch:a", "scratch:b", "scratch:c"]
```
Important detail: do not mutate the generic `_workflow(...)` helper for this test.
Create a dedicated helper such as `_multi_step_overlay_workflow()` whose `record`
`NodeUse` writes `scratch` to `state.scratch`. The test is specifically about an
item-local write followed by an item-local read, so the workflow shape should be
self-contained and obvious.
- [ ] **Step 2: Add a sibling isolation test**
Append this test:
```python
def test_sync_concurrent_foreach_sibling_overlays_do_not_leak() -> None:
workflow = _workflow(
state_schema=StateSchema.from_field_map(
{
"items": StateField(type="array"),
"scratch": StateField(type="string"),
"seen": StateField(
type="array",
reducer=ReducerRef(name="wf.std.append"),
),
}
),
foreach=ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": "state.items",
"as": "item",
"mode": "concurrent",
"concurrent": {"max_active": 2, "max_outstanding": 2},
}
),
)
workflow.node_defs.extend(
[
NodeDef(
name="read_scratch",
input_schema=SchemaRef(
type="object",
properties={"scratch": {}},
required=["scratch"],
),
output_schema=SchemaRef(
type="object",
properties={"seen": {}},
required=["seen"],
),
outcomes=["ok"],
)
]
)
workflow.nodes.append(
NodeUse.model_validate(
{
"id": "read_scratch",
"type": "node",
"node": "read_scratch",
"input": [{"target": "scratch", "path": "state.scratch"}],
"output": [{"source": "seen", "target": "state.seen"}],
}
)
)
workflow.edges = [
Edge.model_validate({"from": "each", "outcome": "loop", "to": "record"}),
Edge.model_validate({"from": "record", "outcome": "ok", "to": "read_scratch"}),
Edge.model_validate({"from": "read_scratch", "outcome": "ok", "to": END}),
Edge.model_validate({"from": "each", "outcome": "done", "to": END}),
]
run = execute_workflow(
workflow,
{"items": ["a", "b"]},
{
"record": lambda payload, _ctx: {
"outcome": "ok",
"output": {"scratch": payload["value"]},
},
"read_scratch": lambda payload, _ctx: {
"outcome": "ok",
"output": {"seen": payload["scratch"]},
},
},
)
assert run.state["seen"] == ["a", "b"]
```
This catches the bad implementation where item `b` sees item `a`'s staged write or vice versa.
- [ ] **Step 3: Run tests and verify failure**
Run:
```bash
uv run pytest tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_item_reads_own_buffered_write tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_sibling_overlays_do_not_leak -q
```
Expected before implementation:
```text
FAILED with "concurrent foreach v1 only supports loop bodies with one node"
```
or, if the guard was already removed by another worker:
```text
FAILED because state.scratch is missing/stale
```
---
### Task 2: Accumulate Per-Item Patches
**Files:**
- Modify: `src/wf_core/runtime/foreach_state.py`
- Modify: `tests/core/test_foreach_barrier_state.py`
- [ ] **Step 1: Add patch accumulation tests**
Append to `tests/core/test_foreach_barrier_state.py`:
```python
def test_foreach_barrier_accumulates_multiple_patches_for_one_item() -> None:
barrier = ForeachBarrierState(mode="concurrent")
barrier.add_success_patch(
index=0,
frame_id="child-0",
patch=StatePatch(changes={"state.scratch": "a"}),
)
barrier.add_success_patch(
index=0,
frame_id="child-0",
patch=StatePatch(changes={"state.seen": "a"}),
)
result = barrier.pending_results[0]
assert result.patch.changes["state.scratch"] == "a"
assert result.patch.changes["state.seen"] == "a"
```
- [ ] **Step 2: Run test and verify failure**
Run:
```bash
uv run pytest tests/core/test_foreach_barrier_state.py::test_foreach_barrier_accumulates_multiple_patches_for_one_item -q
```
Expected before implementation:
```text
FAILED with "already recorded"
```
After this plan is implemented, both tests should pass.
- [ ] **Step 3: Replace duplicate rejection with patch merge**
In `src/wf_core/runtime/foreach_state.py`, change `add_success_patch(...)` to merge changes for the same item:
```python
def add_success_patch(
self, *, index: int, frame_id: str, patch: StatePatch
) -> None:
"""Buffer or extend successful item patches by item index.
A multi-step item body may produce multiple node patches. They are
accumulated for the same item lineage and replayed by the barrier in
item index order. Overlap inside one item remains governed by the normal
node output patch rules for each node; Slice 3 owns sibling conflict
policy at the barrier.
"""
existing = self.pending_results.get(index)
if existing is None:
self.pending_results[index] = PendingItemResult(
index=index,
frame_id=frame_id,
status="succeeded",
patch=patch,
)
return
if existing.frame_id != frame_id:
raise WorkflowExecutionError(
f"foreach item result for index {index!r} belongs to frame "
f"{existing.frame_id!r}, got {frame_id!r}"
)
existing.patch.changes.update(patch.changes)
```
This intentionally updates only `changes`; the barrier replays changes into a fresh staged state later. Do not try to merge `_prepared_writes` here.
- [ ] **Step 4: Update duplicate test**
Replace the prior duplicate-item-result test with a frame-mismatch test:
```python
def test_foreach_barrier_rejects_item_result_frame_mismatch() -> None:
barrier = ForeachBarrierState(mode="concurrent")
patch = StatePatch(changes={"state.count": 1})
barrier.add_success_patch(index=0, frame_id="child-0", patch=patch)
with pytest.raises(WorkflowExecutionError, match="belongs to frame"):
barrier.add_success_patch(index=0, frame_id="child-1", patch=patch)
```
- [ ] **Step 5: Verify barrier tests**
Run:
```bash
uv run pytest tests/core/test_foreach_barrier_state.py -q
```
Expected: pass.
---
### Task 3: Build Item-Local State Views
**Files:**
- Modify: `src/wf_core/runtime/ops/overlays.py`
- Test: `tests/core/test_concurrent_foreach.py`
- [ ] **Step 1: Implement overlay state view**
Replace `state_view_for_frame(...)` in `src/wf_core/runtime/ops/overlays.py`:
```python
from __future__ import annotations
from copy import deepcopy
from typing import Any
from wf_core.run_state import ExecutionFrame, RunState
from wf_core.runtime.foreach_state import ForeachBarrierState, item_frame_owner
from wf_core.runtime.ops.state import safe_set_nested_value
from wf_core.paths import StatePath
def state_view_for_frame(run: RunState, frame: ExecutionFrame) -> dict[str, Any]:
"""Return committed parent state plus this frame's item-local overlay.
Concurrent foreach item frames buffer writes in the parent barrier until the
foreach barrier commits. Later nodes in the same item must still read those
earlier writes, while sibling item frames must not see them.
"""
owner = item_frame_owner(frame)
if owner is None:
return run.state
parent_frame_id, foreach_node_id, item_index = owner
parent_frame = run.frames[parent_frame_id]
barrier = ForeachBarrierState.from_frame(parent_frame, foreach_node_id)
if barrier is None or barrier.mode != "concurrent":
return run.state
pending = barrier.pending_results.get(item_index)
if pending is None:
return run.state
state_view = deepcopy(run.state)
for destination, value in pending.patch.changes.items():
path = StatePath.parse(destination)
safe_set_nested_value(state_view, list(path.parts), value)
return state_view
```
Do not apply reducers here. The overlay view is an item-local read model, not the final parent commit. Reducers are applied at patch build time for each node and again at the barrier for aggregate commit.
- [ ] **Step 2: Verify overlay tests still fail on guard**
Run:
```bash
uv run pytest tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_item_reads_own_buffered_write -q
```
Expected: if the single-node guard is still present, failure remains the guard. If guard was removed by another worker, this may already pass.
---
### Task 4: Build Output Patches Against Frame State View
**Files:**
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_concurrent_foreach.py`
- [ ] **Step 1: Reuse the resolved state view for output patching**
Currently `_resolve_node_execution(...)` computes `state_view` but returns only input/context. Change it to return the state view too:
```python
) -> tuple[dict[str, Any], RuntimeContext, dict[str, Any]]:
```
Return:
```python
return resolved_input, context, state_view
```
Update both callers:
```python
resolved_input, context, state_view = _resolve_node_execution(...)
```
Then pass `state_view` into `_finalize_node_execution(...)`:
```python
state=state_view,
```
Add a parameter to `_finalize_node_execution(...)`:
```python
state_view: dict[str, Any],
```
And change `build_output_patch(...)` call from:
```python
run.state,
```
to:
```python
state_view,
```
This is required for node B in one item to build a patch using node A's staged value.
- [ ] **Step 2: Run focused overlay test**
Run:
```bash
uv run pytest tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_item_reads_own_buffered_write -q
```
Expected: still fails until the single-node guard is removed.
---
### Task 5: Lift The Single-Node Concurrent Body Restriction
**Files:**
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Modify: `tests/core/test_concurrent_foreach.py`
- [ ] **Step 1: Remove the old rejection test**
Delete or rewrite `test_sync_concurrent_foreach_rejects_multi_step_item_body_for_now`.
If preserving regression coverage is preferred, replace it with:
```python
def test_sync_concurrent_foreach_allows_multi_step_item_body_with_overlay() -> None:
# Use the same workflow shape as
# test_sync_concurrent_foreach_item_reads_own_buffered_write.
# Assert the workflow completes and output contains all expected values.
```
Prefer not duplicating the full workflow; reuse the dedicated helper introduced
for the overlay read/write tests:
```python
def _multi_step_overlay_workflow() -> Workflow:
...
```
- [ ] **Step 2: Remove validation call and helper**
In `src/wf_core/runtime/ops/foreach.py`, remove:
```python
_validate_single_node_loop_body(index, step)
```
Delete `_validate_single_node_loop_body(...)`.
Remove unused imports:
```python
from wf_core.models.steps import NodeUse
from wf_core.tokens import END
```
Keep graph traversal validation out of this slice. Normal workflow validation and runtime edge lookup still define whether graph topology is routable.
- [ ] **Step 3: Verify multi-step overlay tests**
Run:
```bash
uv run pytest tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_item_reads_own_buffered_write tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_sibling_overlays_do_not_leak -q
```
Expected: pass.
---
### Task 6: Document Overlay Semantics
**Files:**
- Modify: `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`
- Modify: `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`
- [ ] **Step 1: Update ADR current-state note**
In `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`, verify the
barrier commit section contains:
```markdown
Current sync execution supports item-local read overlays for concurrent foreach
item frames. `RunState.state` remains committed parent state, while
`state_view_for_frame` overlays the current item's buffered writes for reads by
later nodes in the same item lineage. Sibling overlays remain invisible until
the foreach barrier commits.
```
- [ ] **Step 2: Verify roadmap slice statuses**
In `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`,
verify Slice 1 is marked implemented and this plan is linked under Slice 2.
Use:
```markdown
Plan:
- See [`2026-05-22-concurrent-foreach-item-overlays.md`](2026-05-22-concurrent-foreach-item-overlays.md).
```
- [ ] **Step 3: Verify docs reference no stale limitation**
Run:
```bash
rg -n "one node|no-op overlay|multi-step concurrent item bodies are rejected" docs src tests
```
Expected: no stale claims except historical plan text in the already-completed V1 plan.
---
### Task 7: Verification
**Files:**
- No new files unless tests require helper extraction.
- [ ] **Step 1: Run focused core tests**
Run:
```bash
uv run pytest tests/core/test_concurrent_foreach.py tests/core/test_foreach_barrier_state.py tests/core/test_scheduler.py tests/core/test_atomic_state_patches.py -q
```
Expected: pass.
- [ ] **Step 2: Run authoring smoke tests**
Run:
```bash
uv run pytest tests/authoring/test_demo_workflow.py tests/authoring/test_builder.py tests/authoring/test_ops.py -q
```
Expected: pass.
- [ ] **Step 3: Run full suite**
Run:
```bash
uv run pytest -q
```
Expected: pass, allowing known intentional environment-only skips.
- [ ] **Step 4: Run lint/type/format checks**
Run:
```bash
uvx ruff check src tests
uvx ruff format --check src tests docs
uv run basedpyright --level error src tests
```
Expected: all pass with 0 type errors.
---
## Self-Review
- Spec coverage: the plan makes item-local overlays real, supports multi-step concurrent item bodies, keeps sibling overlays isolated, and explicitly defers sibling write conflict policy.
- Placeholder scan: no task uses “TBD” or “add tests” without concrete test content.
- Type consistency: the plan uses current names: `ForeachBarrierState`, `PendingItemResult`, `StatePatch`, `state_view_for_frame`, `item_frame_owner`, and `build_output_patch`.
@@ -0,0 +1,211 @@
# Concurrent Foreach Phase 4 Roadmap
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Split concurrent foreach execution into safe, independently testable runtime slices.
**Architecture:** `foreach(mode="concurrent")` is the workflow mode. Sync runtime should interleave admitted item frames one node call at a time; async runtime may later run admitted async node handlers simultaneously. All item writes must flow through `StatePatch` and commit at a foreach barrier, not directly from child frames into shared state.
**Tech Stack:** Python 3.14, dataclasses, Pydantic v2, pytest, `wf_core.runtime.scheduler`, `wf_core.runtime.foreach_state`, `wf_core.runtime.ops.state`.
---
## Current State
Already implemented:
- `ForeachNode.mode` accepts canonical `"concurrent"`.
- `ForeachConcurrentPolicy` exists with `max_active`, `max_outstanding`, and `interrupt="quiesce"`.
- Legacy `mode="parallel"` / `parallel={...}` parse into canonical concurrent shape.
- `ForeachItemErrorPolicy` exists with `fail`, `skip`, and `collect`.
- `completed_with_errors` is derived for `skip` and `collect`.
- `collect_to` is validated as a declared array state field.
- `StatePatch`, `build_output_patch(...)`, and `commit_state_patch(...)` exist.
- `ForeachBarrierState`, `PendingItemResult`, and `ItemErrorRecord` exist.
- Serial foreach progress now uses `ForeachBarrierState`.
- Sync `foreach(mode="concurrent")` runs with fail-only item policy, bounded
admission, deterministic interleaving, item-local overlays, and barrier
commits.
- Multi-step concurrent item bodies are supported for fail-only item policy.
- Barrier write validation rejects ambiguous sibling writes: same-path sibling
writes require a `mergeable` reducer, and ancestor/descendant
sibling writes are rejected.
- Concurrent `item_error.action="skip"` and `"collect"` are supported.
- `collect` writes ordered item error records to `collect_to`, writes an empty
list on clean success, and emits `completed_with_errors` only when failures
were collected.
- Async runtime batches ready concurrent-foreach item node handlers so handler
awaits may overlap up to admitted `max_active` work. State finalization and
traces still happen sequentially after handler results return.
## Non-Goals For Phase 4
- Do not implement Fork/Gather graph nodes.
- Do not turn `JoinNode` into a real barrier.
- Do not run sync node handlers in threads or processes.
- Do not add OpenTelemetry.
- Do not add platform-level source/tool semaphores.
- Do not add persistent run storage.
## Slice 1: Sync Concurrent Foreach, Fail-Only
Implement first because it proves the scheduler and frame admission model without async task orchestration or handled item failures.
Scope:
- `foreach(mode="concurrent", item_error.action="fail")` runs in sync runtime.
- Parent foreach admits up to `concurrent.max_active` item frames.
- Scheduler interleaves item frames one step at a time.
- Child output writes are buffered as per-item `StatePatch` objects.
- Barrier commits all successful item patches only when every item succeeds.
- Any item runtime failure fails the whole run.
- `skip` and `collect` remain runtime-unsupported for concurrent mode.
Plan:
- See [`2026-05-22-concurrent-foreach-v1-sync-fail-only.md`](2026-05-22-concurrent-foreach-v1-sync-fail-only.md).
## Slice 2: Item-Local Overlays
Implemented after Slice 1 because fail-only concurrent foreach needed
lineage-local reads before multi-step item bodies could be supported. Overlays
let later nodes in one item read earlier buffered writes from the same item
without exposing those writes to siblings.
Scope:
- `state_view_for_frame(...)` returns committed parent state plus current item
overlay for concurrent foreach item frames.
- Parent `RunState.state` remains unchanged until the barrier commits.
- Item patches accumulate across multiple nodes in the same item lineage.
- Multi-step concurrent item bodies are supported.
- Sibling item overlays remain isolated.
- Sibling write conflict policy remains deferred to Slice 3.
Plan:
- See [`2026-05-22-concurrent-foreach-item-overlays.md`](2026-05-22-concurrent-foreach-item-overlays.md).
## Slice 3: Barrier Commit Conflict Semantics
Implement after Slice 2 so conflict checks operate on real item-local overlays
and multi-step item patches.
Scope:
- Detect sibling lineage writes to the same state path.
- If exactly one lineage writes a destination path, default replace is allowed.
- If multiple sibling lineages write the same destination path, a declared
`mergeable` reducer is required.
- Ancestor/descendant writes across sibling lineages are conflicts unless an explicit future merge strategy covers them.
- Commit order is item index order, never completion order.
Files likely touched:
- `src/wf_core/runtime/foreach_state.py`
- `src/wf_core/runtime/ops/state.py`
- `src/wf_core/runtime/ops/foreach.py`
- `tests/core/test_concurrent_foreach.py`
Key tests:
- `test_concurrent_foreach_rejects_sibling_writes_without_reducer`
- `test_concurrent_foreach_applies_reducer_in_item_index_order`
- `test_concurrent_foreach_rejects_ancestor_descendant_write_conflict`
Plan:
- See [`2026-05-22-concurrent-foreach-barrier-write-semantics.md`](2026-05-22-concurrent-foreach-barrier-write-semantics.md).
## Slice 4: Item Error Policies
Implemented after barrier success commits became correct.
Scope:
- `item_error.action="skip"` continues after item runtime failures.
- `item_error.action="collect"` continues and writes structured errors to `collect_to`.
- Both emit `completed_with_errors` if at least one item failed.
- `collect` writes an empty list and emits `done` if all items succeed.
- Failed item frames remain `FAILED`; parent foreach decides whether the failure is handled.
Files likely touched:
- `src/wf_core/runtime/foreach_state.py`
- `src/wf_core/runtime/ops/foreach.py`
- `tests/core/test_concurrent_foreach_errors.py`
Key tests:
- `test_concurrent_foreach_skip_emits_completed_with_errors`
- `test_concurrent_foreach_collect_writes_ordered_error_records`
- `test_concurrent_foreach_collect_writes_empty_list_on_clean_success`
## Slice 5: Async Concurrent Foreach
Implemented after sync semantics stabilized.
Scope:
- Async runtime may have multiple async node handler calls in flight.
- `concurrent.max_active` caps admitted/running item work.
- Sync handlers are still called normally; no thread/process executor.
- Trace remains append-only chronological execution history.
- Barrier commit order remains item index order.
Files likely touched:
- `src/wf_core/runtime/engine.py`
- `src/wf_core/runtime/step.py`
- `src/wf_core/runtime/ops/nodes.py`
- `src/wf_core/runtime/ops/foreach.py`
- `tests/core/test_concurrent_foreach_async.py`
Key tests:
- `test_async_concurrent_foreach_respects_max_active`
- `test_async_concurrent_foreach_commits_in_item_index_order`
## Slice 6: Interrupt Quiescence
Implemented after async execution exists.
Scope:
- If any concurrent item interrupts, the whole run pauses.
- No new item frames are admitted after the interrupt.
- Already-started async node calls drain to pending results.
- The caller gets control only at a quiescent point.
- Pending results do not commit until resume/commit policy allows it.
- Item frames that route into an `InterruptNode` are prioritized before the
parent foreach can refill capacity.
- Already-started async handler calls drain at the batch boundary; state
finalization remains sequential.
Files likely touched:
- `src/wf_core/runtime/engine.py`
- `src/wf_core/runtime/preparation.py`
- `src/wf_core/runtime/ops/interrupts.py`
- `src/wf_core/runtime/ops/foreach.py`
- `tests/core/test_concurrent_foreach_interrupts.py`
Key tests:
- `test_concurrent_foreach_interrupt_returns_before_refill`
- `test_resume_prioritizes_interrupted_item_frame_before_siblings`
## Execution Order
1. Sync concurrent foreach, fail-only.
2. Item-local overlays for multi-step item bodies.
3. Barrier conflict semantics.
4. `skip` / `collect` item error policies.
5. Async concurrent foreach.
6. Interrupt quiescence.
## Self-Review
- Spec coverage: the roadmap covers scheduler admission, barrier commits, reducer conflicts, handled item failures, async handler execution, and interrupt quiescence.
- Placeholder scan: each slice has scope, likely files, and named tests; concrete code lives in the slice-specific plan.
- Type consistency: the roadmap uses canonical `concurrent`, `ForeachConcurrentPolicy`, `ForeachBarrierState`, `PendingItemResult`, and `StatePatch`.
@@ -0,0 +1,591 @@
# Concurrent Foreach Roadmap Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Implement concurrent foreach incrementally without breaking serial workflows or duplicating state-write logic.
**Architecture:** The work was split into four independently shippable layers: policy models, state patch extraction, barrier runtime state, and concurrent execution. Each layer preserves current serial behavior and adds tests before implementation. Sync runtime supports deterministic interleaving for concurrent foreach; async runtime can additionally run admitted async node handlers simultaneously.
**Tech Stack:** Python 3.14, Pydantic v2, dataclasses, pytest, basedpyright, ruff, existing `wf_core` scheduler/runtime modules.
## Implementation Status
- Phase 1 is implemented: foreach policy models, legacy `parallel` parse-only
compatibility, derived `completed_with_errors` outcomes, and collect
destination validation exist.
- Phase 2 is implemented: node output writes can be split into
`build_output_patch(...)` and `commit_state_patch(...)`; the old
`apply_output_bindings(...)` helper remains as the compatibility wrapper.
- Phase 3 is implemented: `wf_core.runtime.foreach_state` owns typed barrier
metadata and serial foreach progress now uses that metadata instead of ad hoc
`foreach_progress`.
- Phase 4 is implemented: `foreach(mode="concurrent")` supports sync
interleaving, async item-node batching, barrier commits, item error policies,
and quiescent interrupt handling.
- Phase 4 details live in the dedicated roadmap:
[`2026-05-22-concurrent-foreach-phase4-roadmap.md`](2026-05-22-concurrent-foreach-phase4-roadmap.md).
Start with
[`2026-05-22-concurrent-foreach-v1-sync-fail-only.md`](2026-05-22-concurrent-foreach-v1-sync-fail-only.md).
---
## Phase 1: Foreach Policy Models
**Goal:** Add the future policy shape while keeping runtime behavior serial-only.
**Files:**
- Modify: `src/wf_core/models/steps.py`
- Modify: `src/wf_core/validation/outcomes.py`
- Modify: `src/wf_core/validation/steps.py`
- Modify: `src/wf_authoring/builder/core.py`
- Test: `tests/core/test_foreach_policy.py`
- Test: `tests/authoring/test_builder.py`
- [ ] **Step 1: Add failing model tests**
Create `tests/core/test_foreach_policy.py`:
```python
from __future__ import annotations
import pytest
from pydantic import ValidationError
from wf_core.models.steps import ForeachNode
def test_serial_foreach_defaults_to_fail_item_policy() -> None:
node = ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": {"root": "state", "parts": ["items"]},
"as": "item",
}
)
assert node.mode == "serial"
assert node.item_error.action == "fail"
assert node.item_error.collect_to is None
assert node.concurrent is None
def test_collect_item_policy_requires_collect_to() -> None:
with pytest.raises(ValidationError, match="collect_to"):
ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": {"root": "state", "parts": ["items"]},
"as": "item",
"item_error": {"action": "collect"},
}
)
def test_concurrent_policy_requires_concurrent_mode() -> None:
with pytest.raises(ValidationError, match="concurrent policy"):
ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": {"root": "state", "parts": ["items"]},
"as": "item",
"concurrent": {"max_active": 4, "max_outstanding": 20},
}
)
def test_concurrent_policy_validates_capacity_order() -> None:
with pytest.raises(ValidationError, match="max_outstanding"):
ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": {"root": "state", "parts": ["items"]},
"as": "item",
"mode": "concurrent",
"concurrent": {"max_active": 10, "max_outstanding": 4},
}
)
```
- [ ] **Step 2: Add policy models**
In `src/wf_core/models/steps.py`, add:
```python
from typing import Self
class ForeachItemErrorPolicy(BaseModel):
"""Policy for runtime failures inside one foreach item lineage."""
model_config = ConfigDict(extra="forbid")
action: Literal["fail", "skip", "collect"] = "fail"
collect_to: StatePath | None = None
@model_validator(mode="after")
def _validate_collect_to(self) -> Self:
if self.action == "collect" and self.collect_to is None:
raise ValueError("collect item error policy requires collect_to")
if self.action != "collect" and self.collect_to is not None:
raise ValueError("collect_to is only valid when action='collect'")
return self
class ForeachConcurrentPolicy(BaseModel):
"""Concurrency policy for foreach frame admission."""
model_config = ConfigDict(extra="forbid")
max_active: int = Field(default=4, ge=1)
max_outstanding: int = Field(default=20, ge=1)
interrupt: Literal["quiesce"] = "quiesce"
@model_validator(mode="after")
def _validate_capacity(self) -> Self:
if self.max_outstanding < self.max_active:
raise ValueError("max_outstanding must be >= max_active")
return self
```
Update `ForeachNode`:
```python
item_error: ForeachItemErrorPolicy = Field(default_factory=ForeachItemErrorPolicy)
concurrent: ForeachConcurrentPolicy | None = None
on_item_error: Literal["fail", "collect", "skip"] | None = Field(
default=None,
exclude=True,
description="Deprecated parse-only shorthand; use item_error.action.",
)
```
Add a `model_validator(mode="before")` that converts old `on_item_error` into `item_error.action`.
Add a `model_validator(mode="after")` that enforces:
```python
if self.mode == "concurrent" and self.concurrent is None:
raise ValueError("concurrent foreach requires concurrent policy")
if self.mode == "serial" and self.concurrent is not None:
raise ValueError("concurrent policy is only valid when mode='concurrent'")
```
- [ ] **Step 3: Update derived outcomes**
In `src/wf_core/validation/outcomes.py`, update foreach outcome derivation:
```python
if step.type == "foreach":
outcomes = {"loop", "done"}
if step.item_error.action in {"skip", "collect"}:
outcomes.add("completed_with_errors")
return outcomes
```
- [ ] **Step 4: Validate collect destination schema**
In `src/wf_core/validation/steps.py`, when `node.item_error.action == "collect"`:
```python
destination_root = _state_destination_root(node.item_error.collect_to)
if destination_root is None or destination_root not in state_root_fields:
report.add(...)
```
Add a follow-up test that collect-to unknown state root reports a validation issue.
- [ ] **Step 5: Keep runtime unsupported**
In `src/wf_core/runtime/ops/foreach.py`, keep:
```python
if step.mode != "serial":
raise WorkflowExecutionError("concurrent foreach execution is not implemented yet")
```
Add a comment:
```python
# Policy models are accepted before execution support so saved workflows can
# validate shape, but runtime must reject concurrent until barrier commits exist.
```
- [ ] **Step 6: Verify phase**
Run:
```bash
uv run pytest tests/core/test_foreach_policy.py tests/authoring/test_builder.py -q
uvx ruff check src tests
uv run basedpyright --level error
```
Expected: all pass.
---
## Phase 2: State Patch Extraction
**Goal:** Split current node output writes into reusable “build patch” and “commit patch” operations without changing current serial behavior.
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_atomic_state_patches.py`
- Test: `tests/core/test_nested_state_paths.py`
- [ ] **Step 1: Add state patch model**
In `src/wf_core/runtime/ops/state.py`, add:
```python
from dataclasses import dataclass, field
from typing import Any
@dataclass(slots=True)
class StatePatch:
"""Validated state writes produced by one step before commit."""
changes: dict[str, Any] = field(default_factory=dict)
```
- [ ] **Step 2: Extract patch builder**
Refactor existing `apply_output_bindings(...)` into:
```python
def build_output_patch(
workflow: Workflow,
bindings: Sequence[OutputBinding],
output: Mapping[str, Any],
state: MutableMapping[str, Any],
*,
reducers: Mapping[str, ReducerDefinition] | None = None,
missing_field_message: str = "node output is missing required field {field}",
) -> StatePatch:
...
```
This function should:
- validate source paths
- validate destination paths
- calculate reducer-aware changes
- not mutate `state`
- [ ] **Step 3: Extract patch committer**
Add:
```python
def commit_state_patch(
state: MutableMapping[str, Any],
patch: StatePatch,
) -> dict[str, Any]:
"""Commit a validated patch to state and return committed changes."""
for path, value in patch.changes.items():
set_nested_value(state, split_state_path(path), value)
return dict(patch.changes)
```
Use the existing typed path helpers; do not reintroduce dotted-string parsing if a typed path helper exists.
- [ ] **Step 4: Preserve old API**
Keep `apply_output_bindings(...)` as a wrapper:
```python
patch = build_output_patch(...)
return commit_state_patch(state, patch)
```
Existing callers should keep working.
- [ ] **Step 5: Add equivalence tests**
Add tests that compare:
```python
old_changes = apply_output_bindings(...)
patch = build_output_patch(...)
new_changes = commit_state_patch(state2, patch)
assert old_changes["state.some_path"] == new_changes["state.some_path"]
assert state1["some_path"] == state2["some_path"]
```
Do not assert whole dict equality unless the test intentionally owns the full structure.
- [ ] **Step 6: Verify phase**
Run:
```bash
uv run pytest tests/core/test_atomic_state_patches.py tests/core/test_nested_state_paths.py tests/authoring/test_demo_workflow.py -q
uvx ruff check src tests
uv run basedpyright --level error
```
Expected: all pass; full suite should still pass before moving on.
---
## Phase 3: Foreach Barrier Runtime State
**Goal:** Add resumable barrier metadata and pending result structures without enabling concurrent execution.
**Files:**
- Modify: `src/wf_core/runtime/scheduler.py`
- Create: `src/wf_core/runtime/foreach_state.py`
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Test: `tests/core/test_foreach_barrier_state.py`
- [ ] **Step 1: Add pending result dataclasses**
Create `src/wf_core/runtime/foreach_state.py`:
```python
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Literal
from wf_core.runtime.ops.state import StatePatch
@dataclass(slots=True)
class ItemErrorRecord:
"""Structured runtime failure record for one foreach item."""
index: int
frame_id: str
node_id: str
error_type: str
message: str
item: Any = None
@dataclass(slots=True)
class PendingItemResult:
"""Buffered item result waiting for foreach barrier commit."""
index: int
frame_id: str
status: Literal["succeeded", "failed"]
patch: StatePatch = field(default_factory=StatePatch)
error: ItemErrorRecord | None = None
```
- [ ] **Step 2: Add foreach barrier state**
In the same file:
```python
@dataclass(slots=True)
class ForeachBarrierState:
"""Resumable state owned by one foreach parent frame."""
next_index: int = 0
active_frame_ids: tuple[str, ...] = ()
outstanding_frame_ids: tuple[str, ...] = ()
pending_results: dict[int, PendingItemResult] = field(default_factory=dict)
```
Add `to_metadata()` / `from_frame()` helpers. Wrong frame kind returns `None`; malformed metadata for a foreach parent raises `WorkflowExecutionError`.
- [ ] **Step 3: Move serial progress into typed state**
Current serial foreach uses:
```python
progress_map = frame.metadata.setdefault("foreach_progress", {})
```
Replace with typed barrier state, but keep behavior equivalent:
```python
barrier = ForeachBarrierState.from_frame(frame) or ForeachBarrierState()
loop_index = barrier.next_index
barrier.next_index += 1
frame.metadata["foreach_barrier"] = barrier.to_metadata()
```
- [ ] **Step 4: Add serialization tests**
Test:
```python
def test_foreach_barrier_state_round_trips_through_frame_metadata() -> None:
...
```
Assert specific fields:
```python
assert loaded.next_index == 2
assert loaded.outstanding_frame_ids == ("child-1",)
```
- [ ] **Step 5: Keep serial behavior passing**
Run:
```bash
uv run pytest tests/core/test_foreach_barrier_state.py tests/authoring/test_demo_workflow.py -q
```
Expected: pass.
---
## Phase 4: Concurrent Foreach Execution
**Goal:** Enable `foreach(mode="concurrent")` using policy limits, pending results, and barrier commits. Sync runtime interleaves admitted item frames one node call at a time; async runtime may run admitted async node handler calls simultaneously.
**Files:**
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Modify: `src/wf_core/runtime/step.py`
- Modify: `src/wf_core/runtime/engine.py`
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Modify: `src/wf_core/runtime/foreach_state.py`
- Test: `tests/core/test_concurrent_foreach.py`
- [ ] **Step 1: Add sync interleaving and async execution tests**
Create `tests/core/test_concurrent_foreach.py` with:
```python
def test_sync_runtime_interleaves_concurrent_foreach() -> None:
...
async def test_async_runtime_accepts_concurrent_foreach() -> None:
...
```
Historical expectation before Phase 4: both tests failed because runtime rejected
concurrent mode. Current implementation status: these tests should pass.
- [ ] **Step 2: Add capacity tests**
Use async node handlers that record start/completion order and block on `asyncio.Event`.
Test:
```python
async def test_concurrent_foreach_respects_max_active() -> None:
...
assert max_seen_active == 2
```
Use `max_active=2`.
- [ ] **Step 3: Add outstanding tests**
Use a node that blocks internally through a future block helper or controlled async wait.
Test:
```python
async def test_blocked_items_count_against_max_outstanding_not_active() -> None:
...
```
This may require a small test-only node that blocks through the runtime-supported internal wait. If internal blocking is not implemented yet, defer this test to subgraph/internal-wait work and keep `max_outstanding` tested through queued children.
- [ ] **Step 4: Add collect/skip tests**
Tests:
```python
async def test_concurrent_collect_writes_ordered_errors_and_emits_completed_with_errors() -> None:
...
async def test_concurrent_skip_emits_completed_with_errors_without_hidden_state() -> None:
...
```
Assert:
```python
assert run.state["document_errors"][0]["index"] == 1
assert run.trace[-1].outcome == "completed_with_errors"
```
- [ ] **Step 5: Add barrier commit ordering test**
Use nodes that complete out of order but write list-like results.
Assert committed state is ordered by item index, not completion order.
- [ ] **Step 6: Implement concurrent child scheduling**
In `step_foreach`, branch by mode:
```python
if step.mode == "serial":
return step_foreach_serial(...)
return step_foreach_concurrent(...)
```
`step_foreach_concurrent` should:
- inspect `ForeachBarrierState`
- start children while `active < max_active` and `outstanding < max_outstanding`
- block parent when waiting for children
- finish when all items terminal
- commit barrier patches in item index order
- emit `done` or `completed_with_errors`
- [ ] **Step 7: Add async node-call budget seam**
Add execution option shape only if needed by implementation:
```python
@dataclass(slots=True)
class RuntimeLimits:
max_active_node_calls: int = 16
```
If this is too large for the first concurrent pass, leave global node-call budget as follow-up and rely on foreach `max_active`.
- [ ] **Step 8: Verify phase**
Run:
```bash
uv run pytest tests/core/test_concurrent_foreach.py tests/authoring/test_demo_workflow.py -q
uv run pytest -q
uvx ruff check src tests
uv run basedpyright --level error
```
Expected: all pass.
---
## Implementation Order Recommendation
Ship these as separate commits/PRs:
1. Phase 1: policy shape and validation
2. Phase 2: patch extraction with no behavior change
3. Phase 3: barrier metadata with serial behavior unchanged
4. Phase 4: concurrent execution
Phase 4 depended on Phase 2 and Phase 3 because concurrent foreach needs patch extraction and resumable barrier state. Keep future fork/gather or native subgraph work layered on top of those runtime primitives instead of replacing them.
## Self-Review
- Spec coverage: ADR 0002 decisions are represented across the four phases.
- Intentional gaps: explicit Fork/Gather, lineage-token graph nodes, OpenTelemetry, platform source/tool caps, and full run persistence are not included.
- Risk control: phases 1-3 preserved serial behavior until phase 4 enabled `mode="concurrent"`.
@@ -0,0 +1,756 @@
# Scheduler Foundation Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add the internal scheduler foundation needed for future concurrent foreach and native subgraphs while preserving current serial workflow behavior.
**Architecture:** `RunState.frames` remains the source of frame lifecycle state, and a new serialized `ready_frame_ids` queue defines deterministic scheduling order. A new internal `wf_core.runtime.scheduler` module owns frame creation, enqueue/select, block/wake, and no-ready-frame resolution. Existing sync and async engines both use scheduler selection before `prepare_step`; node execution stays split between sync and async paths.
**Tech Stack:** Python 3.14, dataclasses, pytest, Pydantic workflow models, existing `wf_core` runtime modules.
---
## Files
- Modify: `src/wf_core/run_state.py`
- Add `FrameStatus.BLOCKED`.
- Add `RunState.ready_frame_ids: list[str]`.
- Create: `src/wf_core/runtime/scheduler.py`
- Internal scheduler helpers and typed block/foreach metadata helpers.
- Modify: `src/wf_core/runtime/ops/runs.py`
- Initialize root frame and ready queue.
- Modify: `src/wf_core/runtime/ops/flow.py`
- Re-enqueue normal frame advances and complete terminal frames through scheduler helpers.
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Use typed foreach metadata, explicit child frame creation, block parent, enqueue child.
- Modify: `src/wf_core/runtime/ops/frames.py`
- Keep context helpers; demote stack-collapse usage or leave compatibility wrappers that call scheduler helpers.
- Modify: `src/wf_core/runtime/preparation.py`
- Stop relying on stack-style frame collapse for selection.
- Resume interrupt by waking/enqueueing the resumed frame.
- Modify: `src/wf_core/runtime/engine.py`
- Use scheduler loop for sync and async resume.
- Modify: `src/wf_core/runtime/step.py`
- Keep `prepare_step()` boundary; ensure selected frame is already chosen by scheduler.
- Test: `tests/core/test_scheduler.py`
- Unit tests for scheduler helpers.
- Test: `tests/core/test_run_state.py`
- Add serialization/additive field checks if needed.
- Test: existing foreach/interrupt tests under `tests/authoring/` and `tests/core/`
- Add focused regressions where the behavior is not already covered.
---
### Task 1: Add RunState Scheduler Fields
**Files:**
- Modify: `src/wf_core/run_state.py`
- Test: `tests/core/test_scheduler.py`
- [ ] **Step 1: Write failing tests for ready queue serialization and frame status**
Create `tests/core/test_scheduler.py` with:
```python
from __future__ import annotations
from wf_core.run_state import FrameStatus, RunState, RunStatus
def test_run_state_serializes_ready_frame_ids() -> None:
run = RunState(
workflow_name="demo",
status=RunStatus.PENDING,
workflow_input={},
state={},
ready_frame_ids=["root"],
)
dumped = run.to_dict()
assert dumped["ready_frame_ids"] == ["root"]
def test_frame_status_has_blocked() -> None:
assert FrameStatus.BLOCKED == "blocked"
```
- [ ] **Step 2: Run failing tests**
Run:
```bash
uv run pytest tests/core/test_scheduler.py -q
```
Expected: fails because `ready_frame_ids` and `FrameStatus.BLOCKED` do not exist.
- [ ] **Step 3: Implement minimal state fields**
In `src/wf_core/run_state.py`:
```python
class FrameStatus(StrEnum):
PENDING = "pending"
RUNNING = "running"
BLOCKED = "blocked"
COMPLETED = "completed"
FAILED = "failed"
INTERRUPTED = "interrupted"
```
Add to `RunState`:
```python
ready_frame_ids: list[str] = field(default_factory=list)
```
- [ ] **Step 4: Verify**
Run:
```bash
uv run pytest tests/core/test_scheduler.py -q
```
Expected: pass.
---
### Task 2: Create Internal Scheduler Helpers
**Files:**
- Create: `src/wf_core/runtime/scheduler.py`
- Test: `tests/core/test_scheduler.py`
- [ ] **Step 1: Add failing scheduler helper tests**
Append tests:
```python
import pytest
from wf_core.errors import WorkflowExecutionError
from wf_core.run_state import ExecutionFrame
from wf_core.runtime.scheduler import (
add_frame,
block_frame_on_children,
enqueue_frame,
select_next_frame,
wake_frame,
)
def _run() -> RunState:
return RunState(
workflow_name="demo",
status=RunStatus.RUNNING,
workflow_input={},
state={},
)
def test_add_frame_rejects_duplicate_frame_ids() -> None:
run = _run()
add_frame(run, ExecutionFrame(id="root", kind="root", node_id="a"))
with pytest.raises(WorkflowExecutionError, match="duplicate frame id"):
add_frame(run, ExecutionFrame(id="root", kind="root", node_id="a"))
def test_enqueue_is_unique_and_priority_moves_to_front() -> None:
run = _run()
add_frame(run, ExecutionFrame(id="a", kind="root", node_id="a"))
add_frame(run, ExecutionFrame(id="b", kind="root", node_id="b"))
enqueue_frame(run, "a")
enqueue_frame(run, "b")
enqueue_frame(run, "a")
enqueue_frame(run, "a", front=True)
assert run.ready_frame_ids == ["a", "b"]
def test_select_next_frame_pops_and_marks_running() -> None:
run = _run()
add_frame(run, ExecutionFrame(id="root", kind="root", node_id="a"))
enqueue_frame(run, "root")
frame = select_next_frame(run)
assert frame is not None
assert frame.id == "root"
assert frame.status == FrameStatus.RUNNING
assert run.ready_frame_ids == []
assert run.current_frame_id == "root"
assert run.current_node_id == "a"
def test_blocked_frame_is_not_selectable_until_woken() -> None:
run = _run()
add_frame(run, ExecutionFrame(id="parent", kind="root", node_id="foreach"))
block_frame_on_children(run, "parent", ("child",))
assert select_next_frame(run) is None
wake_frame(run, "parent")
selected = select_next_frame(run)
assert selected is not None
assert selected.id == "parent"
```
- [ ] **Step 2: Run failing tests**
Run:
```bash
uv run pytest tests/core/test_scheduler.py -q
```
Expected: fails because `wf_core.runtime.scheduler` does not exist.
- [ ] **Step 3: Implement scheduler helpers**
Create `src/wf_core/runtime/scheduler.py`:
```python
from __future__ import annotations
from collections.abc import Sequence
from dataclasses import dataclass
from wf_core.errors import WorkflowExecutionError
from wf_core.run_state import ExecutionFrame, FrameStatus, RunState, RunStatus
@dataclass(slots=True, frozen=True)
class BlockedOnChildren:
"""Typed block reason for frames waiting on child frame completion."""
child_frame_ids: tuple[str, ...]
@classmethod
def from_frame(cls, frame: ExecutionFrame) -> "BlockedOnChildren | None":
raw = frame.metadata.get("blocked_on")
if raw is None:
return None
if not isinstance(raw, dict) or raw.get("type") != "child_frames":
raise WorkflowExecutionError(
f"malformed block reason for frame {frame.id!r}"
)
raw_ids = raw.get("frame_ids")
if not isinstance(raw_ids, list) or not all(
isinstance(item, str) for item in raw_ids
):
raise WorkflowExecutionError(
f"malformed child frame ids for frame {frame.id!r}"
)
return cls(tuple(raw_ids))
def to_metadata(self) -> dict[str, object]:
return {"type": "child_frames", "frame_ids": list(self.child_frame_ids)}
def add_frame(run: RunState, frame: ExecutionFrame, *, ready: bool = False) -> None:
"""Add a frame once; frame id reuse is always a runtime invariant error."""
if frame.id in run.frames:
raise WorkflowExecutionError(f"duplicate frame id {frame.id!r}")
run.frames[frame.id] = frame
if ready:
enqueue_frame(run, frame.id)
def enqueue_frame(run: RunState, frame_id: str, *, front: bool = False) -> None:
"""Put a pending frame in the ready queue without creating duplicates."""
frame = _frame(run, frame_id)
if frame.status != FrameStatus.PENDING:
raise WorkflowExecutionError(
f"cannot enqueue frame {frame_id!r} with status {frame.status!s}"
)
if frame_id in run.ready_frame_ids:
run.ready_frame_ids.remove(frame_id)
if front:
run.ready_frame_ids.insert(0, frame_id)
else:
run.ready_frame_ids.append(frame_id)
def select_next_frame(run: RunState) -> ExecutionFrame | None:
"""Select the next ready frame and expose it through compatibility cursor fields."""
while run.ready_frame_ids:
frame_id = run.ready_frame_ids.pop(0)
frame = _frame(run, frame_id)
if frame.status != FrameStatus.PENDING:
raise WorkflowExecutionError(
f"ready frame {frame_id!r} has status {frame.status!s}"
)
frame.status = FrameStatus.RUNNING
run.current_frame_id = frame.id
run.sync_from_current_frame()
return frame
return None
def mark_frame_pending(run: RunState, frame_id: str, *, front: bool = False) -> None:
"""Mark a live frame pending and enqueue it for future execution."""
frame = _frame(run, frame_id)
frame.status = FrameStatus.PENDING
enqueue_frame(run, frame_id, front=front)
def block_frame_on_children(
run: RunState, frame_id: str, child_frame_ids: Sequence[str]
) -> None:
"""Mark a frame blocked on child completion and remove it from the ready queue."""
frame = _frame(run, frame_id)
run.ready_frame_ids = [item for item in run.ready_frame_ids if item != frame_id]
frame.status = FrameStatus.BLOCKED
frame.metadata["blocked_on"] = BlockedOnChildren(
tuple(child_frame_ids)
).to_metadata()
def wake_frame(run: RunState, frame_id: str, *, front: bool = False) -> None:
"""Wake a blocked or interrupted frame and enqueue it as pending."""
frame = _frame(run, frame_id)
if frame.status not in {FrameStatus.BLOCKED, FrameStatus.INTERRUPTED}:
raise WorkflowExecutionError(
f"cannot wake frame {frame_id!r} with status {frame.status!s}"
)
frame.status = FrameStatus.PENDING
frame.metadata.pop("blocked_on", None)
enqueue_frame(run, frame_id, front=front)
def resolve_no_ready_frames(run: RunState) -> RunStatus:
"""Classify an empty ready queue into a terminal or blocked run state."""
if run.status == RunStatus.INTERRUPTED:
return RunStatus.INTERRUPTED
if any(frame.status == FrameStatus.FAILED for frame in run.frames.values()):
return RunStatus.FAILED
if run.frames and all(
frame.status == FrameStatus.COMPLETED for frame in run.frames.values()
):
return RunStatus.COMPLETED
if any(frame.status == FrameStatus.BLOCKED for frame in run.frames.values()):
raise WorkflowExecutionError("run has no ready frames and is deadlocked")
raise WorkflowExecutionError("run has no ready frames")
def _frame(run: RunState, frame_id: str) -> ExecutionFrame:
frame = run.frames.get(frame_id)
if frame is None:
raise WorkflowExecutionError(f"unknown frame id {frame_id!r}")
return frame
```
- [ ] **Step 4: Verify helper tests**
Run:
```bash
uv run pytest tests/core/test_scheduler.py -q
```
Expected: pass.
---
### Task 3: Initialize Root Through Scheduler
**Files:**
- Modify: `src/wf_core/runtime/ops/runs.py`
- Test: `tests/core/test_scheduler.py`
- [ ] **Step 1: Add root initialization test**
Append:
```python
from wf_core import SchemaRef, StateSchema, Workflow
from wf_core.runtime.ops.runs import create_run_state
def test_create_run_state_queues_root_frame() -> None:
workflow = Workflow(
name="demo",
input_schema=SchemaRef(properties={}),
state_schema=StateSchema(fields={}),
output_schema=SchemaRef(properties={}),
node_defs=[],
start="first",
nodes=[],
edges=[],
)
run = create_run_state(workflow, {})
assert run.current_frame_id == "root"
assert run.current_node_id == "first"
assert run.ready_frame_ids == ["root"]
assert run.frames["root"].status == FrameStatus.PENDING
```
- [ ] **Step 2: Run failing test if needed**
Run:
```bash
uv run pytest tests/core/test_scheduler.py::test_create_run_state_queues_root_frame -q
```
Expected: fails until root ready queue initialization exists.
- [ ] **Step 3: Update run creation**
In `src/wf_core/runtime/ops/runs.py`, create the root frame through `add_frame(..., ready=True)` and set compatibility cursor fields.
- [ ] **Step 4: Verify**
Run:
```bash
uv run pytest tests/core/test_scheduler.py -q
```
Expected: pass.
---
### Task 4: Re-Enqueue Normal Frame Advances
**Files:**
- Modify: `src/wf_core/runtime/ops/flow.py`
- Modify: `src/wf_core/runtime/step.py`
- Test: `tests/core/test_scheduler.py`
- [ ] **Step 1: Add normal advance test**
Append:
```python
from wf_core.runtime.ops.flow import advance_frame
def test_advance_frame_requeues_non_terminal_frame() -> None:
run = _run()
add_frame(run, ExecutionFrame(id="root", kind="root", node_id="a"))
run.current_frame_id = "root"
run.sync_from_current_frame()
frame = run.current_frame()
frame.status = FrameStatus.RUNNING
advance_frame(run, frame, outcome="ok", next_node_id="b")
assert frame.status == FrameStatus.PENDING
assert run.ready_frame_ids == ["root"]
assert run.current_node_id == "b"
```
- [ ] **Step 2: Run failing test**
Run:
```bash
uv run pytest tests/core/test_scheduler.py::test_advance_frame_requeues_non_terminal_frame -q
```
Expected: fails because `advance_frame` currently leaves non-terminal status as running and does not enqueue.
- [ ] **Step 3: Update `advance_frame`**
In `src/wf_core/runtime/ops/flow.py`, after setting node state:
```python
from wf_core.runtime.scheduler import mark_frame_pending
```
Use:
```python
if next_node_id == END:
frame.status = FrameStatus.COMPLETED
frame.finished_at_node_id = END
else:
frame.finished_at_node_id = None
mark_frame_pending(run, frame.id)
run.sync_from_current_frame()
```
- [ ] **Step 4: Verify focused tests**
Run:
```bash
uv run pytest tests/core/test_scheduler.py -q
```
Expected: pass.
---
### Task 5: Migrate Engine Loops To Scheduler Selection
**Files:**
- Modify: `src/wf_core/runtime/engine.py`
- Modify: `src/wf_core/runtime/preparation.py`
- Modify: `src/wf_core/runtime/step.py`
- Test: existing workflow tests
- [ ] **Step 1: Add regression tests if existing coverage is insufficient**
Before adding new tests, run:
```bash
uv run pytest tests/authoring/test_demo_workflow.py tests/core/test_scheduler.py -q
```
Expected before implementation: likely failures after Task 4 until engine selection is updated.
- [ ] **Step 2: Update engine loops**
Change `resume_workflow` and `resume_workflow_async` to:
```python
while True:
frame = select_next_frame(run)
if frame is None:
status = resolve_no_ready_frames(run)
if status == RunStatus.COMPLETED:
break
return run
step_workflow(...)
if run.status == RunStatus.INTERRUPTED:
return run
```
Use the same selection flow in the async loop before `await step_workflow_async(...)`.
- [ ] **Step 3: Remove stack-style frame collapse from step preparation**
In `prepare_step`, remove old stack-style frame collapse and keep it focused on resolving the selected frames node. It should still return `None` for interrupted/end states.
- [ ] **Step 4: Verify serial workflows**
Run:
```bash
uv run pytest tests/authoring/test_demo_workflow.py tests/core/test_scheduler.py -q
```
Expected: pass.
---
### Task 6: Make Serial Foreach Use Block/Wake
**Files:**
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Modify: `src/wf_core/runtime/ops/frames.py`
- Create or extend: `tests/core/test_scheduler.py`
- Test: `tests/authoring/test_demo_workflow.py`
- [ ] **Step 1: Add foreach block/wake regression test**
Add a focused test using an existing demo workflow or a small builder workflow that asserts:
```python
foreach_entries = [entry for entry in run.trace if entry.step_type == "foreach"]
assert foreach_entries
assert all(frame.status != FrameStatus.BLOCKED for frame in run.frames.values())
assert any(frame.kind == "foreach_iteration" for frame in run.frames.values())
```
Then add a lower-level test if needed for child completion waking parent:
```python
def test_child_completion_wakes_blocked_parent() -> None:
run = _run()
add_frame(run, ExecutionFrame(id="parent", kind="root", node_id="foreach"))
add_frame(
run,
ExecutionFrame(
id="child",
kind="foreach_iteration",
node_id="__end__",
parent_frame_id="parent",
),
)
block_frame_on_children(run, "parent", ("child",))
run.frames["child"].status = FrameStatus.COMPLETED
wake_parent_if_children_complete(run, "child")
assert run.frames["parent"].status == FrameStatus.PENDING
assert run.ready_frame_ids == ["parent"]
```
- [ ] **Step 2: Implement typed foreach metadata**
In `src/wf_core/runtime/scheduler.py` or a small sibling module if the file grows too large:
```python
@dataclass(slots=True, frozen=True)
class ForeachIterationMetadata:
foreach_node_id: str
loop_index: int
loop_item: object
loop_alias: str
@classmethod
def from_frame(cls, frame: ExecutionFrame) -> "ForeachIterationMetadata | None":
if frame.kind != "foreach_iteration":
return None
...
def to_metadata(self) -> dict[str, object]:
...
```
Wrong frame kind returns `None`; malformed foreach iteration metadata raises `WorkflowExecutionError`.
- [ ] **Step 3: Implement parent wake helper**
Add:
```python
def wake_parent_if_children_complete(run: RunState, child_frame_id: str) -> None:
child = _frame(run, child_frame_id)
parent_id = child.parent_frame_id
if parent_id is None:
return
parent = _frame(run, parent_id)
block = BlockedOnChildren.from_frame(parent)
if block is None:
return
if all(run.frames[item].status == FrameStatus.COMPLETED for item in block.child_frame_ids):
wake_frame(run, parent_id)
```
- [ ] **Step 4: Update foreach child creation**
In `step_foreach`, replace direct `run.frames[child_id] = ...` with `add_frame(..., ready=True)`, then call `block_frame_on_children(run, frame.id, (child_id,))`. Parent should not stay in ready queue while child runs.
- [ ] **Step 5: Verify foreach behavior**
Run:
```bash
uv run pytest tests/authoring/test_demo_workflow.py tests/core/test_scheduler.py -q
```
Expected: pass.
---
### Task 7: Resume Interrupt Through Ready Queue
**Files:**
- Modify: `src/wf_core/runtime/preparation.py`
- Modify: `src/wf_core/runtime/ops/interrupts.py`
- Test: existing interrupt tests or new focused tests
- [ ] **Step 1: Find existing interrupt tests**
Run:
```bash
rg -n "interrupt|resume_payload|resume_outcome" tests src -g '*.py'
```
- [ ] **Step 2: Add/adjust test for resume priority**
Add a focused test that creates an interrupted frame plus another ready frame and verifies resume places interrupted frame first:
```python
def test_resume_wakes_interrupted_frame_at_front() -> None:
run = _run()
add_frame(run, ExecutionFrame(id="waiting", kind="root", node_id="ask"))
add_frame(run, ExecutionFrame(id="sibling", kind="root", node_id="work"))
run.frames["waiting"].status = FrameStatus.INTERRUPTED
run.frames["sibling"].status = FrameStatus.PENDING
run.ready_frame_ids = ["sibling"]
wake_frame(run, "waiting", front=True)
assert run.ready_frame_ids == ["waiting", "sibling"]
```
- [ ] **Step 3: Update resume code**
After `resume_interrupt(...)`, wake/enqueue the resumed frame at the front. Keep one outstanding `run.interrupt`.
- [ ] **Step 4: Verify interrupt tests**
Run focused interrupt tests found in Step 1 plus:
```bash
uv run pytest tests/core/test_scheduler.py -q
```
Expected: pass.
---
### Task 8: Full Verification
**Files:**
- Potentially update docs if implementation differs from ADR.
- [ ] **Step 1: Run focused workflow tests**
Run:
```bash
uv run pytest tests/core/test_scheduler.py tests/authoring/test_demo_workflow.py -q
```
Expected: pass.
- [ ] **Step 2: Run full test suite**
Run:
```bash
uv run pytest -q
```
Expected: pass, except known environment-only skips.
- [ ] **Step 3: Run lint/type checks**
Run:
```bash
uvx ruff check src tests
uv run basedpyright --level error
```
Expected: ruff passes and basedpyright reports 0 errors.
- [ ] **Step 4: Review docs**
Check:
```bash
git diff -- CONTEXT.md docs/adr/0001-scheduler-foundation-before-concurrent-foreach.md docs/current_roadmap.md
```
Expected: docs remain aligned with implemented first-pass behavior.
---
## Self-Review
- Spec coverage: ADR decisions are covered by tasks for `BLOCKED`, ready queue, scheduler helpers, sync/async engine migration, serial foreach block/wake, interrupt resume priority, and verification.
- Intentional gaps: no `foreach(mode="concurrent")`, no `ForeachConcurrentPolicy`, no lineage patches, no BarrierNode, and no public scheduler exports.
- Type consistency: helper names are stable across tasks: `add_frame`, `enqueue_frame`, `select_next_frame`, `mark_frame_pending`, `block_frame_on_children`, `wake_frame`, `wake_parent_if_children_complete`, and `resolve_no_ready_frames`.
@@ -0,0 +1,748 @@
# Lineage State Runtime Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add runtime scopes and lineages before native subgraphs so sibling branches/items can own isolated replayable state writes and later merge through barriers.
**Architecture:** Implements [`../specs/2026-05-24-lineage-state-runtime-design.md`](../specs/2026-05-24-lineage-state-runtime-design.md). Keep `RunState.state` as the committed root-scope compatibility state, add `RuntimeScope`, `LineageState`, and `StateWrite`, and migrate concurrent foreach from foreach-specific overlays to lineage-backed state views. A frame says where execution is; a scope says which workflow state root execution belongs to; a lineage says which pending writes that execution can see.
**Tech Stack:** Python 3.14, dataclasses, existing `StatePatch`, `StatePath`, `ReducerRef`, `ForeachBarrierState`, pytest, basedpyright, ruff.
---
## Current Implementation Status
This plan is being implemented incrementally. The full `RuntimeScope` /
`LineageState` storage model below is still future work, but the runtime now has
the compatibility subset needed before native subgraphs:
- `StateWrite` exists and records `incoming_value` for replay plus
`visible_value` for same-lineage reads.
- `StatePatch` stores ordered `writes` while preserving `changes` as the
trace/compatibility view.
- `LineageStateView` materializes committed state plus visible lineage writes.
- Concurrent foreach item overlays read `StateWrite.visible_value`.
- Foreach pending result metadata persists write records and `lineage_id`.
- `ExecutionFrame` and `RuntimeContext` carry `scope_id`, `lineage_id`, and
`parent_lineage_id`.
- Concurrent foreach child frames receive deterministic, opaque lineage ids,
including nested foreach frames.
- `RunState` has root scope/lineage storage, scope-aware state views, and
generic non-root node writes buffer into `RunState.lineages`.
- New concurrent foreach item writes are stored in `RunState.lineages`.
`ForeachBarrierState` now keeps scheduling/result metadata plus compatibility
patches for old serialized barrier data.
Direct commits now go through a scope-root commit decision: top-level frames
commit to root state, and prepared native-child root frames commit to their
child scope state. Descendant item/branch lineages still buffer writes until a
barrier or future gather commits them.
Remaining work should avoid jumping straight into a broad rewrite. Native
subgraph scaffolding and non-interrupting prepared-child execution are now
present (`SubgraphNode`, structural `WorkflowRef`, terminal workflow outcomes,
authoring helpers, and `PreparedSubgraph`). Child graphs execute through their
own scope/lineage and map output back at completion. Interrupt bubbling and
saved/deployed workflow resolution remain later work.
---
## File Structure
- Modify: `src/wf_core/run_state.py`
- Add `RuntimeScope`, `StateWrite`, `LineageState`.
- Add `ExecutionFrame.scope_id` and `ExecutionFrame.lineage_id`.
- Add `RunState.scopes` and `RunState.lineages`.
- Modify: `src/wf_core/runtime/ops/state.py`
- Change `StatePatch` from only path-value maps to ordered `StateWrite` records while preserving `changes` as a compatibility/trace view.
- Create: `src/wf_core/runtime/lineage.py`
- Own scope/lineage lookup, state view materialization, append writes, and conversion of completed lineage writes into barrier patches.
- Modify: `src/wf_core/runtime/ops/runs.py`
- Initialize root scope and root lineage.
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Resolve node input from frame scope/lineage view and buffer non-root writes into lineage records.
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Create concurrent item lineages and commit completed lineage writes through the barrier.
- Modify: `src/wf_core/runtime/foreach_state.py`
- Store completed lineage ids in pending item results; keep old patch metadata parse-compatible.
- Modify: `src/wf_core/runtime/ops/overlays.py`
- Reduce to a compatibility facade over lineage state views.
- Test: `tests/core/test_lineage_state.py`
- Unit tests for root scope/lineage, state views, write records, and non-root write buffering.
- Test: `tests/core/test_atomic_state_patches.py`
- Tests for ordered `StateWrite` records and compatibility `changes`.
- Test: `tests/core/test_concurrent_foreach.py`
- Regression tests for sibling isolation, same-item visibility, and deterministic barrier commits.
- Docs: `docs/wf_core_architecture.md`, `docs/current_roadmap.md`, `docs/superpowers/specs/2026-05-24-native-subgraphs-design.md`
- Document scope/lineage as the prerequisite for native subgraphs.
---
## Core Shape
Target runtime state types:
```python
@dataclass(slots=True)
class StateWrite:
"""One reducer-aware write record owned by a lineage or patch."""
path: StatePath
incoming_value: Any
visible_value: Any
reducer: ReducerRef
@dataclass(slots=True)
class RuntimeScope:
"""Committed state root for one workflow activation."""
id: str
workflow_name: str
committed_state: dict[str, Any] = field(default_factory=dict)
@dataclass(slots=True)
class LineageState:
"""Pending ordered writes visible to frames in one lineage."""
id: str
scope_id: str
parent_id: str | None = None
writes: list[StateWrite] = field(default_factory=list)
```
Target patch shape:
```python
@dataclass(slots=True)
class StatePatch:
"""Validated state writes produced by one step before commit."""
writes: list[StateWrite] = dataclass_field(default_factory=list)
_staged_state: dict[str, Any] = dataclass_field(default_factory=dict, repr=False)
@property
def changes(self) -> dict[str, Any]:
return {str(write.path): write.incoming_value for write in self.writes}
@property
def visible_values(self) -> dict[str, Any]:
return {str(write.path): write.visible_value for write in self.writes}
```
Compatibility requirement: existing tests and callers that read `patch.changes`
should continue to work. New lineage code must use ordered `writes`, not
flattened final values.
---
## Task 1: Add Ordered StateWrite Records to StatePatch
Status: implemented as the compatibility shape. `StatePatch.changes` remains a
stored compatibility dict rather than a derived-only property for now.
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- Test: `tests/core/test_atomic_state_patches.py`
- [ ] **Step 1: Add failing tests**
Append:
```python
def test_output_patch_records_incoming_and_visible_values() -> None:
workflow = _workflow_with_state_field(
path="state.count",
schema={"type": "integer"},
reducer="wf.std.add",
)
state = {"count": 2}
patch = build_output_patch(
workflow,
[OutputBinding.model_validate({"source": "delta", "target": "state.count"})],
{"delta": 3},
state,
)
assert patch.changes["state.count"] == 3
assert patch.visible_values["state.count"] == 5
assert patch.writes[0].incoming_value == 3
assert patch.writes[0].visible_value == 5
def test_barrier_replays_incoming_values_not_lineage_visible_values() -> None:
workflow = _workflow_with_state_field(
path="state.number",
schema={"type": "integer"},
reducer="wf.std.add",
)
patch = build_barrier_patch(
workflow,
[
StatePatch(
writes=[
StateWrite(
path=StatePath(("number",)),
incoming_value=3,
visible_value=5,
reducer=ReducerRef(name="wf.std.add"),
)
]
),
StatePatch(
writes=[
StateWrite(
path=StatePath(("number",)),
incoming_value=1,
visible_value=3,
reducer=ReducerRef(name="wf.std.add"),
)
]
),
],
{"number": 2},
)
assert patch.changes["state.number"] == 6
assert patch.visible_values["state.number"] == 6
```
Add imports:
```python
from wf_core.models.reducers import ReducerRef
from wf_core.paths import StatePath
from wf_core.run_state import StateWrite
```
- [ ] **Step 2: Run expected failing tests**
Run:
```bash
uv run pytest tests/core/test_atomic_state_patches.py::test_output_patch_records_incoming_and_visible_values tests/core/test_atomic_state_patches.py::test_barrier_replays_incoming_values_not_lineage_visible_values -q
```
Expected: failure because `StateWrite`, `StatePatch.writes`, and `visible_values`
do not exist.
- [ ] **Step 3: Implement `StateWrite` and patch views**
In `src/wf_core/run_state.py`, add `StateWrite` near runtime dataclasses:
```python
@dataclass(slots=True)
class StateWrite:
path: StatePath
incoming_value: Any
visible_value: Any
reducer: ReducerRef
```
Import `ReducerRef` and `StatePath`.
In `src/wf_core/runtime/ops/state.py`, change `StatePatch` to store ordered
writes and expose `changes` / `visible_values` properties. Keep `_staged_state`.
- [ ] **Step 4: Build write records in `build_output_patch`**
When `prepare_state_value(...)` returns a merged value, also capture the reducer
used for the destination. If needed, extract reducer lookup from
`prepare_state_value(...)` into a helper:
```python
def reducer_for_state_path(
path: StatePath,
state_fields: Mapping[StatePath, StateFieldDecl],
) -> ReducerRef:
field = state_fields.get(path)
return field.reducer if field and field.reducer else ReducerRef(name="wf.std.replace")
```
Create:
```python
StateWrite(
path=destination_path,
incoming_value=value,
visible_value=merged_value,
reducer=reducer,
)
```
- [ ] **Step 5: Replay incoming values in `build_barrier_patch`**
Update `build_barrier_patch(...)` to iterate over `item_patch.writes`, not over
`item_patch.changes.items()`. Replay `write.incoming_value` against the staged
state. The resulting barrier patch should contain one `StateWrite` per final
destination with both incoming and visible values set to the final committed
aggregate value, because the barrier is the public commit point.
- [ ] **Step 6: Run atomic patch tests**
Run:
```bash
uv run pytest tests/core/test_atomic_state_patches.py -q
```
Expected: pass.
---
## Task 2: Add Runtime Scopes and Root Lineage
Status: partially implemented. Frames and runtime context carry `scope_id`,
`lineage_id`, and `parent_lineage_id`, but `RunState.scopes`,
`RunState.lineages`, `RuntimeScope`, and `LineageState` are not implemented yet.
This is deliberate; foreach still stores pending writes in barrier metadata.
**Files:**
- Modify: `src/wf_core/run_state.py`
- Modify: `src/wf_core/runtime/ops/runs.py`
- Test: `tests/core/test_lineage_state.py`
- [ ] **Step 1: Add root initialization test**
Create `tests/core/test_lineage_state.py` with a local minimal workflow helper
that builds a tiny core `Workflow`. Then add:
```python
def test_create_run_state_initializes_root_scope_and_lineage() -> None:
workflow = minimal_workflow()
run = create_run_state(workflow, {"value": "seed"})
assert run.scopes["root"].id == "root"
assert run.scopes["root"].workflow_name == workflow.name
assert run.scopes["root"].committed_state["value"] == "seed"
assert run.lineages["root"].id == "root"
assert run.lineages["root"].scope_id == "root"
assert run.lineages["root"].parent_id is None
assert run.lineages["root"].writes == []
assert run.frames["root"].scope_id == "root"
assert run.frames["root"].lineage_id == "root"
```
- [ ] **Step 2: Run expected failing test**
Run:
```bash
uv run pytest tests/core/test_lineage_state.py::test_create_run_state_initializes_root_scope_and_lineage -q
```
Expected: failure because `scopes`, `lineages`, `scope_id`, and `lineage_id`
do not exist.
- [ ] **Step 3: Add dataclasses and fields**
In `src/wf_core/run_state.py`, add `RuntimeScope` and `LineageState`. Add
`scope_id: str = "root"` and `lineage_id: str = "root"` to `ExecutionFrame`.
Add `scopes` and `lineages` to `RunState`.
- [ ] **Step 4: Initialize root scope and lineage**
In `src/wf_core/runtime/ops/runs.py`, initialize:
```python
run = RunState(
workflow_name=workflow.name,
status=RunStatus.PENDING,
workflow_input=dict(workflow_input),
state=state,
scopes={
"root": RuntimeScope(
id="root",
workflow_name=workflow.name,
committed_state=state,
)
},
lineages={"root": LineageState(id="root", scope_id="root")},
current_frame_id="root",
current_node_id=workflow.start,
)
```
The root scope may share the same dict object as `RunState.state` during this
migration.
- [ ] **Step 5: Run focused test**
Run:
```bash
uv run pytest tests/core/test_lineage_state.py -q
```
Expected: pass.
---
## Task 3: Add Lineage Runtime Helpers
Status: partially implemented. `LineageStateView` and
`lineage_writes_for_frame(run, frame)` exist in `src/wf_core/runtime/lineage.py`,
backed by current foreach metadata.
**Files:**
- Create: `src/wf_core/runtime/lineage.py`
- Test: `tests/core/test_lineage_state.py`
- [ ] **Step 1: Add helper tests**
Append:
```python
def test_lineage_state_view_applies_visible_values_only_for_reads() -> None:
workflow = minimal_workflow()
run = create_run_state(workflow, {"number": 2})
add_lineage(run, scope_id="root", lineage_id="branch", parent_id="root")
append_lineage_writes(
run,
scope_id="root",
lineage_id="branch",
writes=[
StateWrite(
path=StatePath(("number",)),
incoming_value=3,
visible_value=5,
reducer=ReducerRef(name="wf.std.add"),
)
],
)
view = lineage_state_view(run, scope_id="root", lineage_id="branch")
assert view["number"] == 5
assert run.state["number"] == 2
def test_lineage_write_patch_preserves_incoming_values_for_barrier_replay() -> None:
workflow = minimal_workflow()
run = create_run_state(workflow, {"number": 2})
add_lineage(run, scope_id="root", lineage_id="branch", parent_id="root")
append_lineage_writes(
run,
scope_id="root",
lineage_id="branch",
writes=[
StateWrite(
path=StatePath(("number",)),
incoming_value=3,
visible_value=5,
reducer=ReducerRef(name="wf.std.add"),
)
],
)
patch = lineage_patch(run, scope_id="root", lineage_id="branch")
assert patch.writes[0].incoming_value == 3
assert patch.writes[0].visible_value == 5
```
- [ ] **Step 2: Run expected failing tests**
Run:
```bash
uv run pytest tests/core/test_lineage_state.py -q
```
Expected: import failure for `wf_core.runtime.lineage`.
- [ ] **Step 3: Implement `runtime.lineage`**
Create helpers:
```python
def add_lineage(
run: RunState, *, scope_id: str, lineage_id: str, parent_id: str
) -> None: ...
def append_lineage_writes(
run: RunState,
*,
scope_id: str,
lineage_id: str,
writes: Sequence[StateWrite],
) -> None: ...
def lineage_patch(run: RunState, *, scope_id: str, lineage_id: str) -> StatePatch: ...
def lineage_state_view(
run: RunState, *, scope_id: str, lineage_id: str
) -> dict[str, Any]: ...
```
`lineage_state_view(...)` should deep-copy `run.scopes[scope_id].committed_state`
and apply `write.visible_value` from ancestor/current lineage writes in order.
`lineage_patch(...)` should return ordered writes with incoming values intact.
- [ ] **Step 4: Run focused tests**
Run:
```bash
uv run pytest tests/core/test_lineage_state.py -q
```
Expected: pass.
---
## Task 4: Route Node Reads and Non-Root Writes Through Lineage
**Files:**
- Modify: `src/wf_core/runtime/ops/overlays.py`
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_lineage_state.py`
- [ ] **Step 1: Add non-root write buffering test**
Add a test with a one-node workflow that reads `state.value`, writes
`state.value`, and runs the frame with `lineage_id="child"`. Assert:
```python
assert result.state_changes == {}
assert run.state["value"] == "root"
assert run.lineages["child"].writes[0].incoming_value == "root-child"
assert lineage_state_view(run, scope_id="root", lineage_id="child")["value"] == "root-child"
```
- [ ] **Step 2: Run expected failing test**
Run:
```bash
uv run pytest tests/core/test_lineage_state.py::test_non_root_lineage_node_writes_are_buffered_not_committed -q
```
Expected: failure because node execution still commits or cannot read through
lineage.
- [ ] **Step 3: Update overlay facade**
`state_view_for_frame(run, frame)` should call:
```python
lineage_state_view(run, scope_id=frame.scope_id, lineage_id=frame.lineage_id)
```
For root scope/root lineage it may return `run.state` directly as an optimization.
- [ ] **Step 4: Update node finalization**
In `_finalize_node_execution(...)`:
- if `is_root_lineage_frame(frame)`, commit patch to `run.state`
- otherwise append `patch.writes` to the frame lineage and return empty committed
`state_changes`
- [ ] **Step 5: Run focused tests**
Run:
```bash
uv run pytest tests/core/test_lineage_state.py tests/core/test_atomic_state_patches.py -q
```
Expected: pass.
---
## Task 5: Migrate Concurrent Foreach to Lineages
Status: implemented for new concurrent foreach results. Concurrent foreach
child frames have lineage ids, nested item lineages are tested, item writes are
stored in `RunState.lineages`, and pending item results persist `lineage_id`.
`ForeachBarrierState.patch` remains as a compatibility fallback.
**Files:**
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Modify: `src/wf_core/runtime/foreach_state.py`
- Test: `tests/core/test_concurrent_foreach.py`
- Test: `tests/core/test_foreach_barrier_state.py`
- [ ] **Step 1: Add item lineage regression**
Add:
```python
def test_concurrent_foreach_item_frames_use_distinct_lineages() -> None:
workflow = _workflow(mode="concurrent", concurrent={"max_active": 2})
run = execute_workflow(workflow, {"items": ["a", "b"]}, {"record": _record_handler})
item_frames = [frame for frame in run.frames.values() if frame.kind == "foreach_iteration"]
assert len(item_frames) == 2
assert item_frames[0].lineage_id != "root"
assert item_frames[1].lineage_id != "root"
assert item_frames[0].lineage_id != item_frames[1].lineage_id
```
- [ ] **Step 2: Add same-item read regression**
Add or keep a multi-step concurrent foreach test where item node 1 writes
`state.scratch`, item node 2 reads `state.scratch`, and siblings do not see each
other's scratch.
- [ ] **Step 3: Store lineage id on pending item result**
Add `lineage_id: str | None = None` to `PendingItemResult`, parse it from
metadata, and serialize it back. Keep old `patch` parse compatibility.
- [ ] **Step 4: Create item lineages on admission**
In `_admit_concurrent_children(...)`, before adding the child frame:
```python
child_lineage_id = child_id
add_lineage(
run,
scope_id=frame.scope_id,
lineage_id=child_lineage_id,
parent_id=frame.lineage_id,
)
```
Pass `scope_id=frame.scope_id` and `lineage_id=child_lineage_id` to the child
`ExecutionFrame`.
- [ ] **Step 5: Record completed lineage ids**
When a child completes, record `child.lineage_id` on the barrier pending result.
Do not copy flattened visible values into the barrier.
- [ ] **Step 6: Build barrier from lineage patches**
In `_finish_concurrent_foreach(...)`, construct `success_patches` from
`lineage_patch(run, scope_id=frame.scope_id, lineage_id=result.lineage_id)` for
new results. Keep existing `result.patch` fallback for old metadata.
- [ ] **Step 7: Run foreach tests**
Run:
```bash
uv run pytest tests/core/test_concurrent_foreach.py tests/core/test_concurrent_foreach_async.py tests/core/test_foreach_barrier_state.py -q
```
Expected: pass.
---
## Task 6: Remove Foreach-Specific Overlay Coupling
**Files:**
- Modify: `src/wf_core/runtime/ops/overlays.py`
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core`
- [ ] **Step 1: Remove foreach imports from node/overlay state path**
Ensure `ops/nodes.py` and `ops/overlays.py` do not import
`ForeachBarrierState` or `item_frame_owner`.
- [ ] **Step 2: Run core tests**
Run:
```bash
uv run pytest tests/core -q
```
Expected: pass.
---
## Task 7: Update Docs
**Files:**
- Modify: `docs/wf_core_architecture.md`
- Modify: `docs/current_roadmap.md`
- Modify: `docs/superpowers/specs/2026-05-24-native-subgraphs-design.md`
- [ ] **Step 1: Document scope/lineage**
Add a section explaining:
- scope is workflow state root
- frame is scheduler position
- lineage is pending write ownership
- concurrent foreach uses child lineages
- native subgraphs require child scopes
- [ ] **Step 2: Update native subgraph spec**
Ensure it says native subgraphs depend on child runtime scopes plus lineages,
not lineage alone.
- [ ] **Step 3: Run doc red-flag scan**
Run:
```bash
rg -n "U[N]RESOLVED|I[N]COMPLETE|F[I]LL_ME|D[E]CIDE_ME" docs/wf_core_architecture.md docs/current_roadmap.md docs/superpowers/specs/2026-05-24-native-subgraphs-design.md
```
Expected: no output.
---
## Task 8: Full Verification
- [ ] **Step 1: Run tests**
```bash
uv run pytest -q
```
- [ ] **Step 2: Run type check**
```bash
uv run basedpyright --level error src tests examples
```
- [ ] **Step 3: Run lint**
```bash
uvx ruff check src tests examples
```
- [ ] **Step 4: Run format check**
```bash
uvx ruff format --check src tests examples
```
---
## Self-Review
Spec coverage:
- Scope is represented explicitly and is available for native subgraph state
roots.
- Lineage stores ordered replayable writes, not full state and not only visible
values.
- `StatePatch` preserves incoming values for gather/barrier replay.
- Same-lineage reads use visible values.
- Concurrent foreach is the first migration target.
Type consistency:
- `StateWrite.incoming_value` is replay/trace input.
- `StateWrite.visible_value` is same-lineage read value.
- `RuntimeScope.committed_state` is scope-local committed state.
- `ExecutionFrame.scope_id` and `lineage_id` select visibility.
@@ -0,0 +1,108 @@
# Native Subgraph Interrupt Resume Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Let a prepared native child workflow interrupt its parent run and later resume inside the original child scope.
**Architecture:** Add a typed internal interrupt route that distinguishes the public parent-subgraph identity from the actual interrupted child frame. Make interrupt request/resume operations scope-aware, then have engine resume select the prepared child workflow and reducers before applying child resume bindings. The parent `SubgraphNode` remains blocked until the child reaches its ordinary terminal outcome.
**Tech Stack:** Python 3.14, dataclasses, Pydantic workflow models, pytest, Ruff, basedpyright.
---
### Task 1: Child Interrupt Contract
**Files:**
- Modify: `src/wf_core/run_state.py`
- Modify: `tests/core/test_subgraph_step.py`
- [x] **Step 1: Write failing child interrupt/resume tests**
Add a child workflow containing an `InterruptNode` followed by a node or terminal step. Execute it through a parent `SubgraphNode` and assert:
```python
assert run.status == RunStatus.INTERRUPTED
assert run.interrupt is not None
assert run.interrupt.node_id == "child"
assert run.interrupt.payload["question"] == "confirm?"
assert run.frames["root"].status == FrameStatus.BLOCKED
resumed = resume_workflow(
parent,
run,
{},
resume_payload={"answer": "yes"},
subgraphs={"child.workflow": prepared_child},
)
assert resumed.status == RunStatus.COMPLETED
assert resumed.output["answer"] == "yes"
```
- [x] **Step 2: Run tests and observe the existing explicit rejection**
Run: `uv run pytest -q tests/core/test_subgraph_step.py`
Expected: FAIL because child `InterruptNode` execution currently raises that child interrupts are unsupported.
- [x] **Step 3: Add structural route state**
Add `InterruptRoute` to `run_state.py` containing the interrupted child
`frame_id`, `node_id`, `scope_id`, `lineage_id`, and `workflow_ref`. Add an
optional `route` field to `InterruptRequest`; root interrupt requests continue
to use `route=None`.
### Task 2: Scope-Aware Interrupt Creation and Resume
**Files:**
- Modify: `src/wf_core/runtime/ops/handlers.py`
- Modify: `src/wf_core/runtime/ops/interrupts.py`
- Modify: `src/wf_core/runtime/preparation.py`
- Modify: `src/wf_core/runtime/step.py`
- Modify: `src/wf_core/runtime/engine.py`
- [x] **Step 1: Permit child interrupts and build their payload from child scope**
Remove the explicit child rejection. Build child interrupt request bindings
from `state_view_for_frame(...)` and `scope_input_for_frame(...)`, not from the
root compatibility dictionaries. For a non-root scope, find the owning parent
subgraph frame for public identity and attach `InterruptRoute` for resume.
- [x] **Step 2: Resume through the routed child workflow**
When an interrupted request has `route`, restore the routed child as the
current frame, resolve its `PreparedSubgraph`, build the child workflow index,
and apply `resume` output bindings into the child scope using normal
scope-aware patch commit logic. Root interrupts retain the existing path.
- [x] **Step 3: Verify parent completion behavior**
After child resume, scheduling must continue child execution first. Only after
the child finishes may the blocked parent subgraph frame wake and map child
output into parent state.
### Task 3: Verification and Documentation
**Files:**
- Modify: `docs/wf_core_architecture.md`
- Modify: `docs/current_roadmap.md`
- [x] **Step 1: Update documented limitations**
Document that prepared child interrupts now bubble and resume locally, while
artifact/deployment resolution for nested children remains outside core.
- [x] **Step 2: Run focused verification**
Run:
```powershell
uv run pytest -q tests/core/test_subgraph_step.py tests/core/test_concurrent_foreach_interrupts.py
uvx ruff check src/wf_core tests/core/test_subgraph_step.py tests/core/test_concurrent_foreach_interrupts.py
uvx ruff format --check src/wf_core tests/core/test_subgraph_step.py tests/core/test_concurrent_foreach_interrupts.py
uv run basedpyright --level error src/wf_core tests/core/test_subgraph_step.py tests/core/test_concurrent_foreach_interrupts.py
```
Expected: all commands exit successfully.
@@ -0,0 +1,397 @@
# Saved Subgraph Platform Resolution Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Execute non-interrupting saved child workflow artifacts natively from a parent deployment while resolving all descendant dependencies through the parent deployment binding environment.
**Architecture:** `wf_core` remains unaware of artifact storage: it only looks up caller-prepared `PreparedSubgraph` dependencies by structural workflow-ref display key. A new focused workflow-surface resolver loads exact child artifact versions, traverses descendants with cycle detection, validates descendant capabilities and existing interrupt limitations, and prepares child workflows for `WfMcpService` execution. Future child deployment overrides remain outside this slice and must be keyed by subgraph use site, not artifact identity.
**Tech Stack:** Python 3.14, Pydantic v2, `wf_core` native subgraphs, `wf_artifacts` stores/deployment diagnostics, `wf_mcp` workflow surface, pytest, ruff, basedpyright.
---
## File Structure
- Modify `src/wf_core/runtime/subgraphs.py`: accept a caller-prepared structural saved `WorkflowRef`; do not load artifacts.
- Create `src/wf_mcp/workflow_surface/saved_subgraphs.py`: own saved-child traversal, diagnostic production, and preparation of executable child dependencies.
- Modify `src/wf_mcp/workflow_surface/handlers.py`: include descendant dependency/interrupt diagnostics in deployment validation.
- Modify `src/wf_mcp/broker/service/core.py`: supply prepared saved children to `execute_workflow_async`.
- Modify `tests/core/test_subgraph_step.py`: cover core execution of an already-prepared saved ref.
- Create `tests/wf_mcp/test_saved_subgraphs.py`: cover deployment-bound saved child execution and unrunnable descendant cases.
- Modify `docs/current_roadmap.md` and `docs/workflow_artifacts.md`: record the new runnable saved-child path and remaining persisted-resume limitation.
### Task 1: Core Accepts Prepared Saved References
**Files:**
- Modify: `src/wf_core/runtime/subgraphs.py`
- Test: `tests/core/test_subgraph_step.py`
- [ ] **Step 1: Write the failing core test**
Add a test that constructs a parent `SubgraphNode` with:
```python
workflow=WorkflowRef(artifact_id="child", version=1)
```
and supplies:
```python
subgraphs={
"workflow.child.v1": PreparedSubgraph(
workflow=child,
registry={"echo": echo_handler},
reducers={},
)
}
```
Assert the parent run completes and maps the child output into parent state.
- [ ] **Step 2: Run the core test to verify it fails**
Run:
```bash
uv run pytest -q tests/core/test_subgraph_step.py
```
Expected: FAIL because `resolve_prepared_subgraph()` currently rejects a saved structural ref before checking supplied prepared dependencies.
- [ ] **Step 3: Make prepared dependency lookup structural**
Update `resolve_prepared_subgraph()`:
```python
def resolve_prepared_subgraph(
ref: WorkflowRef,
subgraphs: Mapping[str, PreparedSubgraph[HandlerT]] | None,
) -> PreparedSubgraph[HandlerT]:
"""Resolve a caller-prepared child; artifact loading is not a core concern."""
key = ref.name if ref.name is not None else ref.display
prepared = None if subgraphs is None else subgraphs.get(key)
if prepared is None:
raise WorkflowExecutionError(
f"no prepared child workflow registered for {ref.display!r}"
)
return prepared
```
- [ ] **Step 4: Run the core test to verify it passes**
Run:
```bash
uv run pytest -q tests/core/test_subgraph_step.py
```
Expected: PASS.
### Task 2: Traverse and Prepare Saved Child Artifacts
**Files:**
- Create: `src/wf_mcp/workflow_surface/saved_subgraphs.py`
- Test: `tests/wf_mcp/test_saved_subgraphs.py`
- [ ] **Step 1: Write failing traversal tests**
Add focused tests for a helper that receives a root artifact plan containing a
structural `SubgraphNode` ref and a `FileWorkflowArtifactStore`:
```python
resolution = resolve_saved_subgraph_tree(
root_artifact=parent,
artifact_store=artifact_store,
)
assert resolution.artifacts_by_ref["workflow.child.v1"].id == "child"
assert resolution.diagnostics == []
```
Add tests asserting:
```python
assert resolution.diagnostics[0].code == "workflow_dependency_missing"
assert resolution.diagnostics[0].code == "workflow_dependency_cycle"
```
for a missing child and a parent/child cycle respectively.
- [ ] **Step 2: Run tests to verify they fail**
Run:
```bash
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
```
Expected: FAIL because `saved_subgraphs.py` and its resolver do not exist.
- [ ] **Step 3: Implement saved-child tree discovery**
Create a typed resolution object:
```python
@dataclass(frozen=True, slots=True)
class SavedSubgraphTree:
"""Saved descendant artifacts keyed by structural workflow-ref display."""
artifacts_by_ref: dict[str, WorkflowArtifact]
diagnostics: list[DependencyDiagnostic]
```
Implement:
```python
def resolve_saved_subgraph_tree(
*,
root_artifact: WorkflowArtifact,
artifact_store: WorkflowArtifactStore,
) -> SavedSubgraphTree:
"""Load exact saved descendants and report missing refs or cycles."""
```
Parse each artifact plan as `RawWorkflowPlan`, visit only `SubgraphNode`
instances whose `workflow.artifact_id` and `.version` are present, load the
exact artifact, and recurse. Keep an active artifact stack of
`(artifact_id, version)` values so only recursion cycles fail; repeated reuse
of the same child in separate branches is allowed.
Construct direct diagnostics without inventing a fake capability:
```python
DependencyDiagnostic(
severity=DiagnosticSeverity.ERROR,
code="workflow_dependency_missing",
logical_ref=ref.display,
message=f"Saved child workflow {ref.display!r} is unavailable.",
repair_hint="Save the referenced artifact version or update the parent graph.",
)
```
Use analogous text for `workflow_dependency_cycle`.
- [ ] **Step 4: Run traversal tests to verify they pass**
Run:
```bash
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
```
Expected: traversal tests PASS.
### Task 3: Validate Descendants Under One Deployment Environment
**Files:**
- Modify: `src/wf_mcp/workflow_surface/saved_subgraphs.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_saved_subgraphs.py`
- [ ] **Step 1: Write failing public-validation tests**
Add tests that save a parent artifact referencing a child artifact whose plan
uses logical node `demo.echo_tool`. Save one parent deployment:
```python
WorkflowDeployment(
id="parent.personal",
artifact_id="parent",
artifact_version=1,
bindings={"demo": "demo.personal"},
)
```
Assert:
```python
result = asyncio.run(handlers.validate_deployment(deployment_id="parent.personal"))
assert result["status"] == "runnable"
```
Add descendant failure assertions:
```python
assert result["diagnostics"][0]["code"] == "binding_missing"
assert result["diagnostics"][0]["logical_ref"] == "demo.echo_tool"
```
and for an interrupting saved child:
```python
assert result["status"] == "unrunnable"
assert result["diagnostics"][0]["code"] == "unsupported_interrupt"
```
The interrupt diagnostic must be reported before execution because
`run_deployment` is still one-shot.
- [ ] **Step 2: Run validation tests to verify they fail**
Run:
```bash
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
```
Expected: FAIL because `_deployment_validation()` validates only the root artifact.
- [ ] **Step 3: Add descendant validation composition**
Add a helper in `saved_subgraphs.py`:
```python
def validate_saved_subgraph_tree(
*,
tree: SavedSubgraphTree,
deployment: WorkflowDeployment,
sources: list[AvailableSource],
unsupported_interrupt: Callable[[WorkflowArtifact], DependencyDiagnostic | None],
) -> list[DependencyDiagnostic]:
"""Validate descendants in the root deployment environment."""
```
It should begin with tree discovery diagnostics, then for each loaded child
call `validate_deployment_dependencies(...)`, and finally append the existing
unsupported-interrupt diagnostic for that child when present.
Update `WorkflowSurfaceHandlers._deployment_validation()` to discover the
saved tree and extend the root diagnostic list with descendant diagnostics.
Preserve the root artifact interrupt check in `run_deployment()`; it remains
the existing surface behavior.
- [ ] **Step 4: Run validation tests to verify they pass**
Run:
```bash
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
```
Expected: descendant validation tests PASS.
### Task 4: Execute Prepared Saved Children
**Files:**
- Modify: `src/wf_mcp/workflow_surface/saved_subgraphs.py`
- Modify: `src/wf_mcp/broker/service/core.py`
- Test: `tests/wf_mcp/test_saved_subgraphs.py`
- [ ] **Step 1: Write failing end-to-end execution tests**
Use the parent deployment and child artifact from Task 3. Assert:
```python
payload = asyncio.run(
handlers.run_deployment(
deployment_id="parent.personal",
workflow_input={"text": "hello"},
)
)
assert payload["status"] == "completed"
assert payload["output"]["echoed"] == "hello"
```
Add a nested parent -> middle -> child test where only the parent deployment
contains `{"demo": "demo.personal"}`, and assert the grandchild node executes
through the inherited binding.
- [ ] **Step 2: Run execution tests to verify they fail**
Run:
```bash
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
```
Expected: FAIL because the service does not provide prepared saved children to core.
- [ ] **Step 3: Prepare executable children and supply them to core**
Add:
```python
def prepare_saved_subgraphs(
*,
tree: SavedSubgraphTree,
deployment: WorkflowDeployment | None,
sources: dict[str, CapabilitySource],
compile_plan: Callable[[RawWorkflowPlan, dict[str, str] | None], Workflow],
) -> dict[str, PreparedSubgraph[AsyncRegistryHandler]]:
"""Compile loaded descendants with the parent deployment bindings."""
```
For each child artifact, parse its plan, resolve its node/reducer runtime
dependencies with `resolve_runtime_dependencies(...)`, compile it, and return
the dependency under its structural ref display key:
```python
prepared[workflow_ref_display] = PreparedSubgraph(
workflow=compile_plan(plan, dependencies.node_name_bindings),
registry=dependencies.node_registry,
reducers=dependencies.reducers,
)
```
Update `WfMcpService.run_workflow_from_plan()` to resolve the saved tree for
`runtime_artifact` when an artifact store exists, prepare loaded children, and
pass:
```python
subgraphs=prepared_subgraphs
```
to `execute_workflow_async(...)`.
- [ ] **Step 4: Run saved-child tests to verify they pass**
Run:
```bash
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
```
Expected: all saved-subgraph tests PASS.
### Task 5: Documentation and Full Verification
**Files:**
- Modify: `docs/current_roadmap.md`
- Modify: `docs/workflow_artifacts.md`
- [ ] **Step 1: Document current support**
Record:
- Non-interrupting saved child artifacts now run natively through deployments.
- Descendant logical dependencies inherit the root deployment binding environment.
- Missing/cyclic/interrupting saved descendants are reported as unrunnable.
- Explicit per-child deployment overrides and persisted saved-interrupt resume remain future work.
- [ ] **Step 2: Run focused and full verification**
Run:
```bash
uv run pytest -q tests/core/test_subgraph_step.py tests/wf_mcp/test_saved_subgraphs.py
uv run pytest -q
uvx ruff check src/wf_core src/wf_mcp tests/core/test_subgraph_step.py tests/wf_mcp/test_saved_subgraphs.py
uvx ruff format --check src/wf_core/runtime/subgraphs.py src/wf_mcp/workflow_surface/saved_subgraphs.py src/wf_mcp/workflow_surface/handlers.py src/wf_mcp/broker/service/core.py tests/core/test_subgraph_step.py tests/wf_mcp/test_saved_subgraphs.py
uv run basedpyright --level error src/wf_core src/wf_mcp tests/core/test_subgraph_step.py tests/wf_mcp/test_saved_subgraphs.py
```
Expected: all commands pass, with the repository's intentionally skipped live
integration test remaining skipped unless its environment is provided.
## Self-Review
- Spec coverage: the plan covers exact artifact loading, inherited bindings,
cycle/missing diagnostics, preserved interrupt rejection, and native execution.
- Boundary check: artifact traversal and dependency preparation stay in
`wf_mcp`; `wf_core` accepts only caller-prepared structural refs.
- Future compatibility: no child deployment field is added; per-use-site
override remains an additive future platform feature.
@@ -0,0 +1,52 @@
# Stateful Transparent Proxy Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Preserve one upstream MCP session for visible proxy operations made by one connected downstream client, while documenting that generic upstream list/resource notifications are still not relayed.
**Architecture:** The workflow execution pool already owns background/offline sessions. The transparent proxy should use FastMCP's `StatefulProxyClient`, which is designed for Playwright-like upstreams and scopes reuse to one downstream MCP session. This slice does not unify interactive proxy sessions with deployment runtimes and does not implement arbitrary notification rebroadcast.
**Tech Stack:** Python 3.14, FastMCP `StatefulProxyClient` and `FastMCPProxy`, pytest fixture MCP server, ruff, basedpyright.
---
### Task 1: Prove Stateful Proxy Behavior
**Files:**
- Modify: `tests/fixtures/mcp_echo_server.py`
- Modify: `tests/wf_mcp/test_proxy.py`
- Modify: `tests/wf_mcp/test_protocol_relay.py`
- [ ] Add fixture tools that store and read a value in the upstream server process.
- [ ] Add a proxy test that writes the value through one proxied request and reads it through another request in the same downstream client session.
- [ ] Change protocol-relay coverage to assert that `tools/list_changed`, `resources/list_changed`, `prompts/list_changed`, and `resources/updated` are not yet relayed. Keep string-valued logging forwarding as a strict expected-failure tripwire because the installed FastMCP `StatefulProxyClient` handler currently assumes mapping-valued MCP log data.
- [ ] Run the focused tests and confirm they fail before proxy construction changes.
### Task 2: Use FastMCP Stateful Proxy Sessions
**Files:**
- Modify: `src/wf_mcp/proxy/mounts.py`
- [ ] Replace `create_proxy(Client(...))` with `StatefulProxyClient(...)` and `FastMCPProxy(client_factory=client.new_stateful, ...)`.
- [ ] Preserve the existing `ProxyNamespace` and `ResourceLinkNamespace` transforms exactly as mounted-provider output transforms.
- [ ] Add a docstring/comment stating that FastMCP owns the interactive session lifecycle and this is intentionally separate from offline workflow execution sessions.
### Task 3: Verification
**Files:**
- Test: `tests/wf_mcp/test_proxy.py`
- Test: `tests/wf_mcp/test_protocol_relay.py`
- [ ] Run focused proxy/protocol tests.
- [ ] Run `uv run pytest -q`.
- [ ] Run `uvx ruff check`.
- [ ] Run `uv run basedpyright --level error`.
## Scope Boundary
- Interactive visible proxy calls share state within one downstream MCP client session.
- Deployment execution continues to use the owned runtime pool because scheduled/background runs may exist without a downstream client session.
- Generic upstream notification relay remains separate work. FastMCP's stateful path is intended to forward logs/progress/elicitation, but string-valued MCP log data currently exposes an upstream FastMCP handler bug and remains documented with an expected-failure test.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,25 @@
# Superseded OpenAPI Generated-Client Plan
This plan is intentionally retired.
The generated-client approach made runtime execution depend on parsing
`openapi-python-client` generated Python functions to recover parameter-name
mappings such as:
```text
OpenAPI public name: petId
generated Python kwarg: pet_id
```
That dependency direction is too fragile. Do not continue the generated-client
tasks from this file's old history.
Use the replacement plan instead:
```text
docs/superpowers/plans/2026-05-27-openapi-core-capability-source.md
```
The replacement plan uses the OpenAPI document as source of truth,
`openapi-core` for validation/unmarshalling, and a small generic `httpx`
request builder for execution.
@@ -0,0 +1,638 @@
# OpenAPI Core Capability Source Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Let a saved OpenAPI document become a workflow capability source whose operations appear as workflow-facing `NodeSpec`s and execute through spec-driven HTTP requests, without parsing generated Python client code.
**Architecture:** The OpenAPI document is the source of truth for inventory, JSON Schemas, operation paths, parameters, request bodies, and response validation. `openapi-core` validates/unmarshals OpenAPI requests and responses; a small local `httpx` adapter builds and sends HTTP requests from public OpenAPI-shaped payloads. No generated Python client runtime, no AST parsing, and no case-conversion dependency.
**Tech Stack:** Python 3.14, `openapi-core`, `httpx`, existing `jsonschema`, `wf_authoring.NodeSpec`, `wf_platform.CapabilitySource`, `wf_mcp` service registration.
---
## Why This Replaces The Generated-Client Plan
The first plan made the ugly part parameter-name recovery:
```text
OpenAPI/public input: path.petId, header.X-Trace-ID
generated Python fn: pet_id, x_trace_id
```
Because `openapi-python-client` did not expose a stable operation manifest for those mappings, the implementation parsed generated endpoint functions. That is the wrong dependency direction. We should not inspect generated Python to recover metadata already present in the OpenAPI document.
The revised runtime shape is:
```text
workflow input
-> OpenAPI-shaped request parts
-> openapi-core validates/unmarshals request
-> local httpx request builder sends request
-> openapi-core validates/unmarshals response
-> generic workflow outcome
```
This makes validation and execution boring. Outcome mapping remains intentionally generic until saved wrappers add business semantics.
## Non-Goals
- Do not generate Python clients for v1 runtime execution.
- Do not parse generated Python, generated docstrings, or generated function signatures.
- Do not invent a full OpenAPI validator. Use `openapi-core` for request/response validation where possible and `jsonschema` for existing schema-boundary checks.
- Do not make every HTTP status a business outcome. Raw OpenAPI operations expose generic transport outcomes.
- Do not implement custom auth UX in this slice. Leave auth as explicit configuration fields and later integrate with the existing auth/store layer.
- Do not expose every OpenAPI operation as a top-level MCP tool. Expose operations as workflow capabilities first.
## V1 Public Payload Shape
Workflow inputs stay OpenAPI-shaped:
```json
{
"path": {"petId": "pet-1"},
"query": {"includeOwner": true},
"header": {"X-Trace-ID": "abc"},
"cookie": {},
"body": {"name": "Ada"}
}
```
No `petId -> pet_id` translation exists because there is no generated Python function.
## V1 Outcome Semantics
Every raw OpenAPI operation node exposes:
```text
ok
http_error
unexpected_status
validation_error
transport_error
```
Rules:
- `ok`: response status is a declared 2xx response and response validation passes.
- `http_error`: response status is a declared non-2xx response and response validation passes.
- `unexpected_status`: response status is not declared and no `default` response covers it.
- `validation_error`: request or response does not match the OpenAPI document.
- `transport_error`: HTTP client raises before a response exists.
Output shape:
```json
{
"status_code": 200,
"headers": {},
"body": {},
"validation_errors": []
}
```
`body` is JSON when the response is JSON, text for text responses, bytes/base64 later if needed. Keep binary response support out of v1 unless a test fixture forces it.
## Planned File Structure
- Keep: `src/wf_openapi/__init__.py`
- Public exports for the optional OpenAPI capability-source package.
- Keep/modify: `src/wf_openapi/models.py`
- Operation/source/execution models. Remove generated-client metadata.
- Keep/modify: `src/wf_openapi/spec.py`
- Load OpenAPI documents, normalize operations, merge inherited path-item parameters with operation-local overrides.
- Keep/modify: `src/wf_openapi/schemas.py`
- Produce JSON Schema contracts from effective OpenAPI operation inputs/outputs.
- Replace: `src/wf_openapi/executor.py`
- Generic `httpx` + `openapi-core` operation executor.
- Remove or repurpose: `src/wf_openapi/codegen.py`
- Delete generated-client runtime helpers. If kept temporarily, it must not be used by runtime/source tests.
- Create: `src/wf_openapi/request.py`
- Build method, URL, headers, cookies, query params, and JSON body from OpenAPI-shaped payload.
- Create: `src/wf_openapi/validation.py`
- Thin adapter between local request/response objects and `openapi-core` protocols.
- Keep/modify: `src/wf_openapi/source.py`
- Build `CapabilitySource` and `NodeSpec`s using generic execution config.
- Tests:
- `tests/openapi/test_spec_inventory.py`
- `tests/openapi/test_schemas.py` if split becomes useful.
- `tests/openapi/test_request_builder.py`
- `tests/openapi/test_executor.py`
- `tests/openapi/test_source.py`
---
## Task 1: Dependency And Plan Reset
**Files:**
- Modify: `pyproject.toml`
- Modify: `uv.lock`
- Modify: `docs/superpowers/plans/2026-05-27-openapi-capability-source.md`
- Test: `tests/openapi/test_codegen_executor.py` may be removed or replaced later.
- [ ] **Step 1: Replace runtime dependency**
In `pyproject.toml`, remove `openapi-python-client` unless another committed package already uses it. Add:
```toml
"openapi-core>=0.19",
```
Keep `httpx` if already present transitively or directly; add it directly if `wf_openapi` imports it.
- [ ] **Step 2: Refresh lockfile**
Run:
```bash
uv lock
```
Expected: lockfile updates successfully.
- [ ] **Step 3: Mark generated-client plan superseded**
Keep the superseded note at the top of:
```text
docs/superpowers/plans/2026-05-27-openapi-capability-source.md
```
Expected: future agents do not continue Task 5/6 AST parsing work.
- [ ] **Step 4: Verify import availability**
Run:
```bash
uv run python -c "import openapi_core, httpx; print(openapi_core.__name__, httpx.__name__)"
```
Expected: prints `openapi_core httpx`.
---
## Task 2: Remove Generated-Client Runtime Coupling
**Files:**
- Modify: `src/wf_openapi/codegen.py`
- Modify: `src/wf_openapi/executor.py`
- Modify: `tests/openapi/test_codegen_executor.py`
- [ ] **Step 1: Write failing guard test**
Add a test that proves runtime no longer imports generated-client metadata:
```python
def test_openapi_runtime_does_not_require_generated_manifest() -> None:
from wf_openapi.executor import OpenApiExecutionConfig
config = OpenApiExecutionConfig(base_url="https://api.example.test")
assert config.base_url == "https://api.example.test"
assert not hasattr(config, "generated_package")
assert not hasattr(config, "operation_modules")
assert not hasattr(config, "parameter_arguments")
```
Run:
```bash
uv run pytest -q tests/openapi/test_codegen_executor.py::test_openapi_runtime_does_not_require_generated_manifest
```
Expected before implementation: FAIL because generated-client fields still exist.
- [ ] **Step 2: Simplify execution config**
Replace generated-client config with:
```python
from dataclasses import dataclass
@dataclass(frozen=True, slots=True)
class OpenApiExecutionConfig:
"""Runtime config for spec-driven OpenAPI HTTP execution."""
base_url: str
timeout_seconds: float = 30.0
```
Do not include generated package/module/parameter mapping fields.
- [ ] **Step 3: Remove generated manifest helpers from runtime path**
Delete or quarantine:
```python
GeneratedOperationMetadata
load_generated_operation_manifest
generate_openapi_client
```
If `codegen.py` remains, its module docstring must say it is experimental/offline tooling and not used by source/executor runtime.
- [ ] **Step 4: Run focused tests**
Run:
```bash
uv run pytest -q tests/openapi
uv run ruff check src/wf_openapi tests/openapi
uv run basedpyright --level error src/wf_openapi tests/openapi
```
Expected: generated-client tests that no longer match are removed/replaced; remaining tests pass.
---
## Task 3: Generic Request Builder
**Files:**
- Create: `src/wf_openapi/request.py`
- Test: `tests/openapi/test_request_builder.py`
- [ ] **Step 1: Write request builder tests**
Create `tests/openapi/test_request_builder.py`:
```python
from wf_openapi.request import build_http_request_parts
from wf_openapi.spec import load_openapi_operations
FIXTURE = "tests/openapi/fixtures/petstore_minimal.openapi.json"
def test_build_http_request_parts_uses_public_openapi_names() -> None:
operation = next(op for op in load_openapi_operations(FIXTURE) if op.name == "get_pet")
parts = build_http_request_parts(
operation,
base_url="https://api.example.test/v1",
payload={
"path": {"petId": "pet-1"},
"query": {"includeOwner": True},
"header": {"X-Trace-ID": "trace-1"},
},
)
assert parts.method == "GET"
assert parts.url == "https://api.example.test/v1/pets/pet-1"
assert parts.params["includeOwner"] is True
assert parts.headers["X-Trace-ID"] == "trace-1"
```
Expected before implementation: FAIL because `wf_openapi.request` does not exist.
- [ ] **Step 2: Implement request parts**
Create:
```python
from dataclasses import dataclass, field
from typing import Any, Mapping
from urllib.parse import quote
from wf_openapi.models import OpenApiOperation
@dataclass(frozen=True, slots=True)
class HttpRequestParts:
"""OpenAPI-shaped request parts ready for httpx."""
method: str
url: str
params: dict[str, Any] = field(default_factory=dict)
headers: dict[str, str] = field(default_factory=dict)
cookies: dict[str, str] = field(default_factory=dict)
json: Any | None = None
def build_http_request_parts(
operation: OpenApiOperation,
*,
base_url: str,
payload: Mapping[str, Any],
) -> HttpRequestParts:
"""Build an HTTP request without renaming public OpenAPI fields."""
path_values = _mapping(payload, "path")
path = operation.path
for parameter in operation.effective_parameters:
if parameter.get("in") != "path":
continue
name = parameter["name"]
if name not in path_values:
raise ValueError(f"missing path parameter {name!r}")
path = path.replace("{" + name + "}", quote(str(path_values[name]), safe=""))
return HttpRequestParts(
method=operation.method.upper(),
url=base_url.rstrip("/") + path,
params=dict(_mapping(payload, "query")),
headers={str(k): str(v) for k, v in _mapping(payload, "header").items()},
cookies={str(k): str(v) for k, v in _mapping(payload, "cookie").items()},
json=payload.get("body"),
)
def _mapping(payload: Mapping[str, Any], key: str) -> Mapping[str, Any]:
value = payload.get(key, {})
if not isinstance(value, Mapping):
raise ValueError(f"{key} must be an object")
return value
```
- [ ] **Step 3: Add edge tests**
Add tests for:
```text
missing path parameter -> ValueError
non-object query/header/cookie/path -> ValueError
body passes through as json payload
```
- [ ] **Step 4: Verify**
Run:
```bash
uv run pytest -q tests/openapi/test_request_builder.py
```
Expected: PASS.
---
## Task 4: openapi-core Validation Adapter
**Files:**
- Create: `src/wf_openapi/validation.py`
- Test: `tests/openapi/test_validation.py`
- [ ] **Step 1: Write validation tests**
Create tests that load the fixture and validate a request built from public payload:
```python
from wf_openapi.request import build_http_request_parts
from wf_openapi.spec import load_openapi, load_openapi_operations
from wf_openapi.validation import validate_openapi_request
FIXTURE = "tests/openapi/fixtures/petstore_minimal.openapi.json"
def test_validate_openapi_request_accepts_public_payload() -> None:
document = load_openapi(FIXTURE)
operation = next(op for op in load_openapi_operations(FIXTURE) if op.name == "get_pet")
parts = build_http_request_parts(
operation,
base_url="https://api.example.test",
payload={"path": {"petId": "pet-1"}},
)
result = validate_openapi_request(document, parts)
assert result.valid is True
assert result.errors == []
```
Expected before implementation: FAIL because validation adapter does not exist.
- [ ] **Step 2: Implement minimal protocol objects**
Implement local request/response protocol adapters required by `openapi-core`. Keep them in `validation.py` and document that they are intentionally thin protocol shims.
The adapter must carry:
```text
method
full_url_pattern or path pattern if required by openapi-core
parameters/path/query/header/cookie
body
mimetype
```
If `openapi-core` requires a different protocol shape, adapt only this file.
- [ ] **Step 3: Validate response path**
Add:
```python
def validate_openapi_response(document, request_parts, response_parts) -> ValidationResult:
...
```
Test declared `200` response and undeclared status behavior.
- [ ] **Step 4: Verify**
Run:
```bash
uv run pytest -q tests/openapi/test_validation.py
uv run basedpyright --level error src/wf_openapi/validation.py tests/openapi/test_validation.py
```
Expected: PASS.
---
## Task 5: Generic HTTP Executor
**Files:**
- Modify: `src/wf_openapi/executor.py`
- Test: `tests/openapi/test_executor.py`
- [ ] **Step 1: Write executor tests with mocked transport**
Use `httpx.MockTransport`:
```python
import httpx
from wf_openapi.executor import OpenApiExecutionConfig, call_openapi_operation
from wf_openapi.spec import load_openapi, load_openapi_operations
FIXTURE = "tests/openapi/fixtures/petstore_minimal.openapi.json"
async def test_call_openapi_operation_maps_success() -> None:
document = load_openapi(FIXTURE)
operation = next(op for op in load_openapi_operations(FIXTURE) if op.name == "get_pet")
async def handler(request: httpx.Request) -> httpx.Response:
assert request.url.path == "/pets/pet-1"
return httpx.Response(200, json={"id": "pet-1"})
result = await call_openapi_operation(
document,
operation,
OpenApiExecutionConfig(base_url="https://api.example.test"),
{"path": {"petId": "pet-1"}},
client=httpx.AsyncClient(transport=httpx.MockTransport(handler)),
)
assert result.outcome == "ok"
assert result.value.status_code == 200
assert result.value.body["id"] == "pet-1"
```
Expected before implementation: FAIL because executor still uses generated client or wrong signature.
- [ ] **Step 2: Implement executor**
`call_openapi_operation(...)` should:
```text
build request parts
validate/unmarshal request
send with httpx.AsyncClient
parse response body by content-type
validate/unmarshal response
return NodeReturn with generic outcome
```
Transport exceptions become `transport_error`. Validation failures become `validation_error`.
- [ ] **Step 3: Add outcome tests**
Add tests for:
```text
declared non-2xx -> http_error
undeclared status -> unexpected_status
invalid request -> validation_error
httpx transport exception -> transport_error
```
- [ ] **Step 4: Verify**
Run:
```bash
uv run pytest -q tests/openapi/test_executor.py
```
Expected: PASS.
---
## Task 6: Source Integration
**Files:**
- Modify: `src/wf_openapi/source.py`
- Test: `tests/openapi/test_source.py`
- [ ] **Step 1: Write source execution test**
Build a `CapabilitySource`, get `source.capabilities.node_specs["petstore.default.get_pet"]`, call its async handler with public payload, and use `httpx.MockTransport` through runtime/config injection.
Expected before implementation: FAIL because source still uses generated metadata or does not pass executor dependencies.
- [ ] **Step 2: Update source builder**
`build_openapi_capability_source(...)` should accept:
```python
document_path: Path
source_id: str
base_url: str
```
It should not accept:
```text
generated_package
operation_modules
parameter_arguments
```
- [ ] **Step 3: Preserve schema contracts**
Ensure each `NodeSpec` still exposes:
```text
input_schema_contract from operation input schema
output_schema_contract from operation output schema
outcomes = ("ok", "http_error", "unexpected_status", "validation_error", "transport_error")
```
- [ ] **Step 4: Verify**
Run:
```bash
uv run pytest -q tests/openapi/test_source.py
```
Expected: PASS.
---
## Task 7: Docs And Final Cleanup
**Files:**
- Create: `docs/openapi_capability_source.md`
- Modify: `docs/current_roadmap.md`
- Delete or rewrite: generated-client-only tests/files if no longer used.
- [ ] **Step 1: Document the boundary**
Create `docs/openapi_capability_source.md` with:
```markdown
# OpenAPI Capability Sources
OpenAPI sources expose raw API operations as workflow capabilities.
The OpenAPI document is the source of truth. Runtime execution uses a generic
httpx request builder and openapi-core validation. The runtime does not parse
generated Python clients and does not rename public OpenAPI fields.
Raw OpenAPI nodes expose generic transport outcomes. Saved wrappers should add
business-specific outcomes such as `not_found`, `rate_limited`, or
`needs_input`.
```
- [ ] **Step 2: Note deferred auth/body/binary support**
Document:
```text
auth integration: future
binary/multipart request bodies: future
rich outcome mapping: wrappers, not raw operations
```
- [ ] **Step 3: Final verification**
Run:
```bash
uv run pytest -q tests/openapi
uv run ruff check src/wf_openapi tests/openapi
uv run ruff format --check src/wf_openapi tests/openapi
uv run basedpyright --level error src/wf_openapi tests/openapi
```
Expected: all pass.
---
## Self-Review
- The plan no longer requires generated Python client parsing.
- Public OpenAPI names remain public workflow names.
- Validation is library-backed through `openapi-core`, not hand-rolled.
- HTTP execution is locally owned but small and testable with `httpx.MockTransport`.
- Outcome mapping stays generic and wrapper-friendly.
- Auth, multipart/binary, and business outcomes are deferred explicitly.
@@ -0,0 +1,306 @@
# Draft Output Binding Docs Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Clarify the two different `output` binding shapes in workflow drafts so MCP/LLM clients stop applying step-level `source`/`target` bindings to top-level workflow output projection.
**Architecture:** This is docs-only for now. The core model is internally coherent: step-level `output` uses `OutputBinding` (`source` local -> `target` state), while top-level workflow `output` uses input-binding shape (`path` graph -> `target` local output payload). Update the main draft docs, runbook, and schema-facing descriptions to teach this explicitly without changing runtime behavior.
**Tech Stack:** Markdown docs, Pydantic field descriptions, pytest schema/docs tests, ruff, basedpyright.
---
## Scope
Do:
- Add a clear “Two Outputs, Different Shapes” docs section.
- Show exact JSON for both step-level output and top-level workflow output.
- Explain the legacy fallback: empty top-level `output` projects same-name top-level state fields.
- Update MCP model field descriptions so schema viewers see “uses `path`, not `source`” for top-level output.
- Add docs/test assertions that this guidance is exported.
Do not:
- Rename fields to `writes` / `returns`.
- Change validation behavior.
- Remove legacy same-name output fallback.
- Add automatic MCP content block extraction.
## Files
- Modify: `docs/workflow_drafts.md`
- Add the primary explanation and examples.
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
- Add a short warning in the draft patching section.
- Modify: `docs/workflow_capabilities.md`
- Mention that `next_actions.patch_examples` may include top-level output projection examples.
- Modify: `src/wf_artifacts/drafts/models.py`
- Improve `WorkflowDraft.output` field description.
- Modify: `src/wf_core/models/workflow.py`
- Improve `Workflow.output` field description.
- Modify: `tests/wf_mcp/server/test_docs.py`
- Assert exported docs include the new guidance.
- Modify if needed: `tests/wf_mcp/server/test_config.py`
- Assert schema descriptions include “path, not source” if exposed in tool schema.
## Task 1: Document The Two Output Shapes
**Files:**
- Modify: `docs/workflow_drafts.md`
- [ ] **Step 1: Add docs section**
After the “Important details” list or before “Explicit Outputs And Error Outcomes”, add:
```markdown
## Two Outputs, Different Shapes
Drafts have two fields named `output`, but they do different jobs.
### Step-Level `steps.<id>.output`
Step output writes a node's local return payload into workflow state. It uses
`source` / `target`:
```json
{
"source": { "root": "local", "parts": ["text"] },
"target": { "root": "state", "parts": ["result_text"] }
}
```
Read this as:
```text
node output.text -> state.result_text
```
### Top-Level `output`
Top-level workflow output projects graph values into the final public workflow
output payload. It uses input-binding shape: `path` / `target`, not
`source` / `target`.
```json
{
"path": { "root": "state", "parts": ["result_text"] },
"target": { "root": "local", "parts": ["result_text"] }
}
```
Read this as:
```text
state.result_text -> workflow output.result_text
```
If top-level `output` is empty, the runtime keeps the legacy same-name fallback:
for every field in `output_schema`, it copies the top-level state field with the
same name when present. That fallback is convenient, but explicit output
projection is clearer for new workflows.
```
- [ ] **Step 2: Run grep check**
Run:
```powershell
rg -n "Two Outputs, Different Shapes|path.*not.*source|state.result_text -> workflow output.result_text" docs/workflow_drafts.md
```
Expected: all terms appear.
## Task 2: Update Runbook Warning And Example
**Files:**
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
- [ ] **Step 1: Add warning near draft patching section**
Near “Patch Or Validate The Workspace”, add:
```markdown
When patching output bindings, keep the two levels separate:
- Step-level `steps.<id>.output` uses `source` local -> `target` state.
- Top-level `output` uses `path` graph -> `target` local output payload.
For explicit final output projection from state, use:
```json
{
"path": { "root": "state", "parts": ["result_text"] },
"target": { "root": "local", "parts": ["result_text"] }
}
```
Do not use `source` at top level. `source` belongs to step output bindings.
```
- [ ] **Step 2: Run grep check**
Run:
```powershell
rg -n "Do not use `source` at top level|steps.<id>.output|result_text" docs/wf_mcp_end_to_end_runbook.md
```
Expected: all terms appear.
## Task 3: Update Field Descriptions
**Files:**
- Modify: `src/wf_artifacts/drafts/models.py`
- Modify: `src/wf_core/models/workflow.py`
- [ ] **Step 1: Update `WorkflowDraft.output` field description**
Change:
```python
output: list[InputBinding] = Field(default_factory=list)
```
to:
```python
output: list[InputBinding] = Field(
default_factory=list,
description=(
"Top-level workflow output projection. Uses input-binding shape: "
"`path` reads from input/state/context and `target` writes to the "
"local public output payload. Do not use step output `source` here."
),
)
```
- [ ] **Step 2: Update `Workflow.output` field description**
In `src/wf_core/models/workflow.py`, extend the `output` description to include:
```python
"Use `path`, not `source`; `source` belongs to step-level node output bindings."
```
Keep the existing legacy fallback explanation.
- [ ] **Step 3: Run focused schema/type checks**
Run:
```powershell
uv run basedpyright --level error src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py
uv run ruff check src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py
uv run ruff format --check src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py
```
Expected: pass.
## Task 4: Add Exported Docs Test
**Files:**
- Modify: `tests/wf_mcp/server/test_docs.py`
- [ ] **Step 1: Add docs resource assertion**
In the docs resource test that reads workflow authoring/draft docs, assert:
```python
assert "Two Outputs, Different Shapes" in text
assert "Do not use step output `source` here" in text or "Do not use `source` at top level" in text
```
Use the existing variable names in the file. Do not assert whole payload dict equality.
- [ ] **Step 2: Run focused docs test**
Run:
```powershell
uv run pytest tests/wf_mcp/server/test_docs.py -q
```
Expected: pass.
## Task 5: Optional Schema Description Test
**Files:**
- Modify if needed: `tests/wf_mcp/server/test_config.py`
- [ ] **Step 1: Inspect whether draft output field description is exposed**
Run:
```powershell
uv run pytest tests/wf_mcp/server/test_config.py -q
```
If this test already inspects `create_draft_workspace` request schemas, add:
```python
output_description = minimal_request["properties"]["output"]["description"]
assert "path" in output_description
assert "source" in output_description
```
If the schema nests the description differently, skip this test change and rely
on `test_docs.py`.
## Task 6: Final Verification
**Files:**
- All touched docs and Python files.
- [ ] **Step 1: Run focused tests**
Run:
```powershell
uv run pytest tests/wf_mcp/server/test_docs.py tests/wf_mcp/server/test_config.py -q
```
Expected: pass.
- [ ] **Step 2: Run touched-file lint/type checks**
Run:
```powershell
uv run ruff check src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py tests/wf_mcp/server/test_docs.py tests/wf_mcp/server/test_config.py
uv run ruff format --check src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py tests/wf_mcp/server/test_docs.py tests/wf_mcp/server/test_config.py
uv run basedpyright --level error src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py tests/wf_mcp/server/test_docs.py tests/wf_mcp/server/test_config.py
```
Expected: pass.
- [ ] **Step 3: Optional full suite**
Run when time allows:
```powershell
uv run pytest -q
```
Expected current baseline: full suite passes with the existing skip/xfail count.
## Notes For Opencode
- This is docs/description work only.
- Do not rename model fields.
- Do not remove fallback same-name projection.
- Do not auto-extract MCP content blocks.
- The exact mental model to teach is:
```text
steps.call.output: local node output -> workflow state
workflow output: input/state/context graph path -> public output payload
```
@@ -0,0 +1,518 @@
# Next Actions Model Refactor Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move wrapper-draft `next_actions` from handler-local dict helpers into reusable typed workflow-surface models and constructors.
**Architecture:** Create `src/wf_mcp/workflow_surface/next_actions.py` as the single home for advisory guidance models. Keep the current `create_draft_workspace_from_capability` JSON output stable while adding generic `NextActions` / `NextActionPatchExample` types and `NextActions.from_wrapper_hints(...)`. Do not add deployment/run guidance in this pass.
**Tech Stack:** Python 3.14, Pydantic v2, pytest, ruff, basedpyright.
---
## Scope
Do:
- Create `workflow_surface/next_actions.py`.
- Move `WrapperDraftPatchExample` and `WrapperDraftNextActions` into generic models.
- Replace handler-local `_wrapper_draft_next_actions` and `_wrapper_draft_patch_examples` with `NextActions.from_wrapper_hints(...)`.
- Keep existing output fields stable:
- `can_save_now`
- `recommended_next_tool`
- `reason`
- `patch_examples`
- `warnings`
- Add `can_continue` as an additive field.
- Preserve all existing tests and add targeted serialization/model tests.
Do not:
- Add `next_actions` to `validate_deployment`, `run_deployment`, or `resume_run`.
- Enforce `can_save_now`.
- Change MCP tool names.
- Add automatic semantic mapping or MCP content extraction.
## Files
- Create: `src/wf_mcp/workflow_surface/next_actions.py`
- Generic guidance models and wrapper-hints constructor.
- Modify: `src/wf_mcp/workflow_surface/models.py`
- Import/reuse `NextActions` for `CreateDraftWorkspaceFromCapabilityResult`.
- Remove local wrapper-specific next-action models.
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Import `NextActions`.
- Replace private dict helpers with `NextActions.from_wrapper_hints(...).model_dump(mode="json")`.
- Test: `tests/wf_mcp/workflow_surface/test_next_actions.py`
- Direct model/constructor tests.
- Modify: `tests/wf_mcp/workflow_surface/test_drafts.py`
- Add assertions for `can_continue`.
- Modify if needed: `tests/wf_mcp/server/test_config.py`
- Keep schema assertions passing with generic model names.
## Task 1: Add Direct NextActions Model Tests
**Files:**
- Create: `tests/wf_mcp/workflow_surface/test_next_actions.py`
- [ ] **Step 1: Create failing tests**
Create `tests/wf_mcp/workflow_surface/test_next_actions.py`:
```python
from __future__ import annotations
from wf_mcp.workflow_surface.next_actions import NextActions, NextActionTool
def test_next_actions_from_high_confidence_wrapper_hints() -> None:
hints = {
"confidence": "high",
"missing_decisions": [],
"notes": ["Hints are scaffolding, not semantic guarantees."],
}
actions = NextActions.from_wrapper_hints(
workspace_id="echo_wrapper",
revision=3,
hints=hints,
)
dumped = actions.model_dump(mode="json")
assert dumped["can_continue"] is True
assert dumped["can_save_now"] is True
assert dumped["recommended_next_tool"] == (
NextActionTool.VALIDATE_DRAFT_WORKSPACE.value
)
assert "high confidence" in dumped["reason"]
assert dumped["patch_examples"] == []
assert dumped["warnings"] == []
def test_next_actions_from_low_confidence_wrapper_hints() -> None:
hints = {
"confidence": "low",
"missing_decisions": [{"kind": "review_nested_output"}],
"notes": ["Raw MCP content blocks are not workflow-shaped."],
}
actions = NextActions.from_wrapper_hints(
workspace_id="content_wrapper",
revision=5,
hints=hints,
)
dumped = actions.model_dump(mode="json")
assert dumped["can_continue"] is True
assert dumped["can_save_now"] is False
assert dumped["recommended_next_tool"] == (
NextActionTool.PATCH_DRAFT_WORKSPACE.value
)
assert "missing wrapper decisions" in dumped["reason"]
assert dumped["patch_examples"][0]["request"]["workspace_id"] == "content_wrapper"
assert dumped["patch_examples"][0]["request"]["revision"] == 5
assert dumped["warnings"] == ["Raw MCP content blocks are not workflow-shaped."]
```
Run:
```powershell
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
```
Expected: fail because `wf_mcp.workflow_surface.next_actions` does not exist.
## Task 2: Create `next_actions.py`
**Files:**
- Create: `src/wf_mcp/workflow_surface/next_actions.py`
- [ ] **Step 1: Add generic models and constructor**
Create `src/wf_mcp/workflow_surface/next_actions.py`:
```python
from __future__ import annotations
from enum import StrEnum
from typing import Any, Self
from pydantic import BaseModel, Field
from .wrapper_hints import WrapperAuthoringHints
class NextActionTool(StrEnum):
"""Stable MCP workflow tools that guidance may recommend."""
PATCH_DRAFT_WORKSPACE = "wf.workflow.patch_draft_workspace"
VALIDATE_DRAFT_WORKSPACE = "wf.workflow.validate_draft_workspace"
VALIDATE_DEPLOYMENT = "wf.workflow.validate_deployment"
RUN_DEPLOYMENT = "wf.workflow.run_deployment"
RESUME_RUN = "wf.workflow.resume_run"
READ_RUN_TRACE = "wf.workflow.read_run_trace"
class NextActionPatchExample(BaseModel):
"""Concrete example request for a recommended workflow tool."""
description: str = Field(description="Human-readable reason for this example.")
tool: NextActionTool = Field(description="MCP workflow tool to call.")
request: dict[str, Any] = Field(
description="JSON request payload to pass to the tool."
)
class NextActions(BaseModel):
"""Advisory continuation hints for MCP workflow clients."""
can_continue: bool = Field(
description=(
"Whether there is an obvious next workflow-surface tool call. "
"Advisory only."
)
)
can_save_now: bool | None = Field(
default=None,
description=(
"Advisory wrapper-authoring signal. False means review is "
"recommended before saving; the server does not enforce it."
),
)
recommended_next_tool: NextActionTool | None = Field(
default=None,
description="Suggested next MCP workflow tool, if one is obvious.",
)
reason: str = Field(description="Short explanation for the recommendation.")
patch_examples: list[NextActionPatchExample] = Field(
default_factory=list,
description="Concrete JSON Patch examples for common missing decisions.",
)
warnings: list[str] = Field(
default_factory=list,
description="Non-blocking warnings copied from low-confidence hints.",
)
@classmethod
def from_wrapper_hints(
cls,
*,
workspace_id: str,
revision: int,
hints: WrapperAuthoringHints | dict[str, Any],
) -> Self:
"""Create guidance after create_draft_workspace_from_capability."""
hint_payload = _hint_payload(hints)
confidence = str(hint_payload.get("confidence", "low"))
missing_decisions = hint_payload.get("missing_decisions")
notes = [
str(note) for note in hint_payload.get("notes", []) if isinstance(note, str)
]
has_missing = (
isinstance(missing_decisions, list) and len(missing_decisions) > 0
)
can_save_now = confidence == "high" and not has_missing
if can_save_now:
return cls(
can_continue=True,
can_save_now=True,
recommended_next_tool=NextActionTool.VALIDATE_DRAFT_WORKSPACE,
reason=(
"Wrapper hints are high confidence and have no missing decisions."
),
patch_examples=[],
warnings=[],
)
return cls(
can_continue=True,
can_save_now=False,
recommended_next_tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
reason="Review missing wrapper decisions before saving.",
patch_examples=_wrapper_draft_patch_examples(
workspace_id=workspace_id,
revision=revision,
hints=hint_payload,
),
warnings=notes,
)
```
Then add below:
```python
def _hint_payload(hints: WrapperAuthoringHints | dict[str, Any]) -> dict[str, Any]:
"""Return a JSON-compatible wrapper hint payload."""
if isinstance(hints, WrapperAuthoringHints):
return hints.model_dump(mode="json")
return dict(hints)
def _wrapper_draft_patch_examples(
*,
workspace_id: str,
revision: int,
hints: dict[str, Any],
) -> list[NextActionPatchExample]:
"""Return conservative JSON Patch examples without guessing semantics."""
examples: list[NextActionPatchExample] = []
missing_decisions = hints.get("missing_decisions")
if not isinstance(missing_decisions, list):
return examples
decision_kinds = {
str(decision.get("kind"))
for decision in missing_decisions
if isinstance(decision, dict)
}
if {"choose_output_fields", "review_nested_output"} & decision_kinds:
examples.append(
NextActionPatchExample(
description=(
"Replace output bindings after choosing which capability "
"outputs should be written to workflow state."
),
tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
request={
"workspace_id": workspace_id,
"revision": revision,
"patch": [
{
"op": "replace",
"path": "/draft/steps/call/output",
"value": [],
}
],
},
)
)
if "confirm_boolean_outcomes" in decision_kinds:
examples.append(
NextActionPatchExample(
description=(
"Review boolean output candidates before adding routing; "
"do not route on boolean fields automatically."
),
tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
request={
"workspace_id": workspace_id,
"revision": revision,
"patch": [],
},
)
)
return examples
```
- [ ] **Step 2: Run direct tests**
Run:
```powershell
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
```
Expected: pass.
## Task 3: Use Generic Models In MCP Result Schema
**Files:**
- Modify: `src/wf_mcp/workflow_surface/models.py`
- Modify: `tests/wf_mcp/server/test_config.py`
- [ ] **Step 1: Replace local models**
In `src/wf_mcp/workflow_surface/models.py`:
1. Import:
```python
from .next_actions import NextActions
```
2. Delete local classes:
```python
class WrapperDraftPatchExample(...)
class WrapperDraftNextActions(...)
```
3. Change:
```python
next_actions: WrapperDraftNextActions = Field(...)
```
to:
```python
next_actions: NextActions = Field(
description=(
"Advisory next step guidance derived from wrapper_hints. "
"The server does not enforce can_save_now."
)
)
```
- [ ] **Step 2: Update schema test for additive field**
In `tests/wf_mcp/server/test_config.py`, keep existing assertions and add:
```python
assert "can_continue" in next_actions_schema["properties"]
```
Run:
```powershell
uv run pytest tests/wf_mcp/server/test_config.py -q
```
Expected: pass after handler is updated in Task 4. If this fails only because handler runtime does not return `can_continue`, proceed to Task 4.
## Task 4: Replace Handler Helpers
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `tests/wf_mcp/workflow_surface/test_drafts.py`
- [ ] **Step 1: Add output assertions for additive field**
In `tests/wf_mcp/workflow_surface/test_drafts.py`, in both next action assertions, add:
```python
assert next_actions["can_continue"] is True
```
Run:
```powershell
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py::test_workflow_surface_creates_draft_workspace_from_capability_hints tests/wf_mcp/workflow_surface/test_drafts.py::test_workflow_surface_low_confidence_draft_returns_patch_guidance -q
```
Expected: fail because current dict helper does not emit `can_continue`.
- [ ] **Step 2: Import and use `NextActions`**
In `src/wf_mcp/workflow_surface/handlers.py`, add:
```python
from .next_actions import NextActions
```
Change the return in `create_draft_workspace_from_capability` to:
```python
"next_actions": NextActions.from_wrapper_hints(
workspace_id=workspace_id,
revision=int(result["revision"]),
hints=hints,
).model_dump(mode="json"),
```
- [ ] **Step 3: Delete private helpers**
Remove from `handlers.py`:
```python
_wrapper_draft_next_actions
_wrapper_draft_patch_examples
```
Run:
```powershell
rg -n "_wrapper_draft_next_actions|_wrapper_draft_patch_examples" src/wf_mcp/workflow_surface/handlers.py
```
Expected: no matches.
- [ ] **Step 4: Run focused behavior tests**
Run:
```powershell
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/server/test_config.py -q
```
Expected: pass.
## Task 5: Update Docs If Needed
**Files:**
- Modify if needed: `docs/workflow_capabilities.md`
- Modify if needed: `docs/superpowers/specs/2026-05-31-workflow-surface-next-actions-design.md`
- [ ] **Step 1: Check docs already match implementation**
Run:
```powershell
rg -n "can_continue|NextActions|next_actions" docs/workflow_capabilities.md docs/superpowers/specs/2026-05-31-workflow-surface-next-actions-design.md
```
If `workflow_capabilities.md` does not mention `can_continue`, add:
```markdown
`next_actions.can_continue` is advisory. It says whether the response has an
obvious workflow-surface tool to call next.
```
## Task 6: Final Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused tests**
Run:
```powershell
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py -q
```
Expected: pass.
- [ ] **Step 2: Run lint/format checks**
Run:
```powershell
uv run ruff check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
uv run ruff format --check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
```
Expected: pass.
- [ ] **Step 3: Run touched-file type check**
Run:
```powershell
uv run basedpyright --level error src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
```
Expected: `0 errors`.
- [ ] **Step 4: Optional full suite**
Run when time allows:
```powershell
uv run pytest -q
```
Expected current baseline: full suite passes with the existing skip/xfail count.
## Notes For Opencode
- Keep this as a refactor plus additive `can_continue`.
- Do not add deployment/run guidance yet.
- Do not enforce `can_save_now`.
- Keep JSON output stable for existing fields.
- `next_actions` is UX guidance. Diagnostics and validation remain source of truth.
@@ -0,0 +1,629 @@
# Next Actions Model Refactor Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move MCP workflow guidance UX into reusable `NextActions` models so wrapper draft guidance stops living as ad-hoc dict helpers in `handlers.py`.
**Architecture:** Add `src/wf_mcp/workflow_surface/next_actions.py` as the single owner of next-action enums, response models, and the first constructor: `NextActions.from_wrapper_hints(...)`. Keep the existing `create_draft_workspace_from_capability` JSON stable, with `can_continue` as the only additive field. Do not add deployment/run guidance in this pass; this is the foundation for those later constructors.
**Tech Stack:** Python 3.14, Pydantic v2, FastMCP schema generation, pytest, ruff, basedpyright.
---
## File Structure
- Create `src/wf_mcp/workflow_surface/next_actions.py`
- Owns `NextActionTool`, `NextActionPatchExample`, and `NextActions`.
- Owns wrapper-hint guidance policy currently in `handlers.py`.
- Contains docstrings explaining that `next_actions` is advisory, not authority.
- Modify `src/wf_mcp/workflow_surface/models.py`
- Reuse the generic models for MCP response schemas.
- Keep compatibility aliases for `WrapperDraftPatchExample` and `WrapperDraftNextActions` if local imports or generated schemas still reference the old names.
- Modify `src/wf_mcp/workflow_surface/handlers.py`
- Remove `_wrapper_draft_next_actions(...)` and `_wrapper_draft_patch_examples(...)`.
- Call `NextActions.from_wrapper_hints(...).model_dump(mode="json")`.
- Create `tests/wf_mcp/workflow_surface/test_next_actions.py`
- Unit-test the generic model without the full draft store.
- Modify `tests/wf_mcp/workflow_surface/test_drafts.py`
- Keep integration coverage through `create_draft_workspace_from_capability`.
- Assert old fields still exist and `can_continue` is additive.
- Modify `tests/wf_mcp/server/test_config.py`
- Assert the MCP output schema documents `can_continue` and still documents `can_save_now`.
- Modify `docs/workflow_capabilities.md`
- Add one short note that `next_actions` is advisory guidance and not validation authority.
## Scope Boundaries
- Do not add `NextActions.from_deployment_validation(...)` in this pass.
- Do not add `NextActions.from_run_result(...)` in this pass.
- Do not change current wrapper patch example semantics.
- Do not infer MCP content block extraction or boolean routing.
- Do not make `can_save_now` authoritative.
---
### Task 1: Add Generic Next-Actions Unit Tests
**Files:**
- Create: `tests/wf_mcp/workflow_surface/test_next_actions.py`
- [ ] **Step 1: Write failing tests for high-confidence wrapper hints**
Create `tests/wf_mcp/workflow_surface/test_next_actions.py`:
```python
from __future__ import annotations
from wf_mcp.workflow_surface.next_actions import NextActionTool, NextActions
def test_next_actions_from_high_confidence_wrapper_hints_can_validate() -> None:
actions = NextActions.from_wrapper_hints(
workspace_id="echo_workspace",
revision=3,
hints={
"confidence": "high",
"missing_decisions": [],
"notes": [],
},
)
dumped = actions.model_dump(mode="json")
assert dumped["can_continue"] is True
assert dumped["can_save_now"] is True
assert dumped["recommended_next_tool"] == (
NextActionTool.VALIDATE_DRAFT_WORKSPACE.value
)
assert "high confidence" in dumped["reason"]
assert dumped["patch_examples"] == []
assert dumped["warnings"] == []
```
- [ ] **Step 2: Write failing tests for low-confidence wrapper hints**
Append to `tests/wf_mcp/workflow_surface/test_next_actions.py`:
```python
def test_next_actions_from_low_confidence_wrapper_hints_can_patch() -> None:
actions = NextActions.from_wrapper_hints(
workspace_id="echo_workspace",
revision=4,
hints={
"confidence": "low",
"missing_decisions": [
{
"kind": "review_nested_output",
"message": "Review nested output fields before mapping.",
},
{
"kind": "confirm_boolean_outcomes",
"message": "Boolean fields may be data, not outcomes.",
},
],
"notes": ["Raw MCP tool output is not workflow-shaped."],
},
)
dumped = actions.model_dump(mode="json")
assert dumped["can_continue"] is True
assert dumped["can_save_now"] is False
assert dumped["recommended_next_tool"] == NextActionTool.PATCH_DRAFT_WORKSPACE.value
assert "missing wrapper decisions" in dumped["reason"]
assert dumped["warnings"][0] == "Raw MCP tool output is not workflow-shaped."
assert len(dumped["patch_examples"]) == 2
assert dumped["patch_examples"][0]["tool"] == (
NextActionTool.PATCH_DRAFT_WORKSPACE.value
)
assert dumped["patch_examples"][0]["request"]["workspace_id"] == "echo_workspace"
assert dumped["patch_examples"][0]["request"]["revision"] == 4
assert dumped["patch_examples"][0]["request"]["patch"][0]["path"] == (
"/draft/steps/call/output"
)
assert dumped["patch_examples"][1]["request"]["patch"] == []
```
- [ ] **Step 3: Run the focused test to verify it fails**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
```
Expected: FAIL with `ModuleNotFoundError: No module named 'wf_mcp.workflow_surface.next_actions'`.
---
### Task 2: Implement `next_actions.py`
**Files:**
- Create: `src/wf_mcp/workflow_surface/next_actions.py`
- [ ] **Step 1: Create the generic models and wrapper-hints constructor**
Create `src/wf_mcp/workflow_surface/next_actions.py`:
```python
from __future__ import annotations
from enum import StrEnum
from typing import Any, Self
from pydantic import BaseModel, Field
from .wrapper_hints import WrapperAuthoringHints
class NextActionTool(StrEnum):
"""Stable MCP workflow tools that guidance may recommend."""
PATCH_DRAFT_WORKSPACE = "wf.workflow.patch_draft_workspace"
VALIDATE_DRAFT_WORKSPACE = "wf.workflow.validate_draft_workspace"
VALIDATE_DEPLOYMENT = "wf.workflow.validate_deployment"
RUN_DEPLOYMENT = "wf.workflow.run_deployment"
RESUME_RUN = "wf.workflow.resume_run"
READ_RUN_TRACE = "wf.workflow.read_run_trace"
class NextActionPatchExample(BaseModel):
"""Concrete example request for a recommended MCP workflow tool."""
description: str = Field(description="Human-readable reason for this example.")
tool: NextActionTool = Field(description="MCP workflow tool to call.")
request: dict[str, Any] = Field(
description="JSON request payload to pass to the tool."
)
class NextActions(BaseModel):
"""Advisory continuation hints for MCP workflow clients.
This object is guidance, not authority. Validation diagnostics and runtime
status remain the source of truth; clients should treat this as a compact
answer to "what tool should I call next?"
"""
can_continue: bool = Field(
description=(
"Whether there is an obvious next workflow-surface tool call. "
"Advisory only."
)
)
can_save_now: bool | None = Field(
default=None,
description=(
"Advisory wrapper-authoring signal. False means review is "
"recommended before saving; the server does not enforce this."
),
)
recommended_next_tool: NextActionTool | None = Field(
default=None,
description="Suggested next MCP workflow tool, if one is obvious.",
)
reason: str = Field(description="Short explanation for the recommendation.")
patch_examples: list[NextActionPatchExample] = Field(
default_factory=list,
description="Concrete JSON Patch examples for common missing decisions.",
)
warnings: list[str] = Field(
default_factory=list,
description="Non-blocking warnings copied from low-confidence hints.",
)
@classmethod
def from_wrapper_hints(
cls,
*,
workspace_id: str,
revision: int,
hints: WrapperAuthoringHints | dict[str, Any],
) -> Self:
"""Create guidance after bootstrapping a wrapper draft workspace."""
payload = (
hints.model_dump(mode="json") if isinstance(hints, WrapperAuthoringHints)
else hints
)
confidence = str(payload.get("confidence", "low"))
missing_decisions = payload.get("missing_decisions")
notes = [
str(note) for note in payload.get("notes", []) if isinstance(note, str)
]
has_missing = (
isinstance(missing_decisions, list) and len(missing_decisions) > 0
)
can_save_now = confidence == "high" and not has_missing
if can_save_now:
return cls(
can_continue=True,
can_save_now=True,
recommended_next_tool=NextActionTool.VALIDATE_DRAFT_WORKSPACE,
reason="Wrapper hints are high confidence and have no missing decisions.",
patch_examples=[],
warnings=[],
)
return cls(
can_continue=True,
can_save_now=False,
recommended_next_tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
reason="Review missing wrapper decisions before saving.",
patch_examples=_wrapper_draft_patch_examples(
workspace_id=workspace_id,
revision=revision,
hints=payload,
),
warnings=notes,
)
def _wrapper_draft_patch_examples(
*,
workspace_id: str,
revision: int,
hints: dict[str, Any],
) -> list[NextActionPatchExample]:
"""Return conservative JSON Patch examples without guessing semantics."""
examples: list[NextActionPatchExample] = []
missing_decisions = hints.get("missing_decisions")
if not isinstance(missing_decisions, list):
return examples
decision_kinds = {
str(decision.get("kind"))
for decision in missing_decisions
if isinstance(decision, dict)
}
if {"choose_output_fields", "review_nested_output"} & decision_kinds:
examples.append(
NextActionPatchExample(
description=(
"Replace output bindings after choosing which capability "
"outputs should be written to workflow state."
),
tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
request={
"workspace_id": workspace_id,
"revision": revision,
"patch": [
{
"op": "replace",
"path": "/draft/steps/call/output",
"value": [],
}
],
},
)
)
if "confirm_boolean_outcomes" in decision_kinds:
examples.append(
NextActionPatchExample(
description=(
"Review boolean output candidates before adding routing; "
"do not route on boolean fields automatically."
),
tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
request={
"workspace_id": workspace_id,
"revision": revision,
"patch": [],
},
)
)
return examples
```
- [ ] **Step 2: Run the focused unit test**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
```
Expected: PASS.
---
### Task 3: Replace Wrapper-Specific MCP Models with Generic Models
**Files:**
- Modify: `src/wf_mcp/workflow_surface/models.py`
- [ ] **Step 1: Import generic next-action models**
Near the other local imports in `src/wf_mcp/workflow_surface/models.py`, add:
```python
from .next_actions import NextActionPatchExample, NextActions
```
- [ ] **Step 2: Remove wrapper-specific model class bodies**
Delete the current `WrapperDraftPatchExample` and `WrapperDraftNextActions` class definitions.
Replace them with compatibility aliases immediately before `CreateDraftWorkspaceFromCapabilityResult`:
```python
# Compatibility aliases for older imports. The JSON fields are now generic
# workflow-surface guidance, not wrapper-only policy.
WrapperDraftPatchExample = NextActionPatchExample
WrapperDraftNextActions = NextActions
```
- [ ] **Step 3: Keep `CreateDraftWorkspaceFromCapabilityResult` typed with the alias**
Leave the field shape unchanged except that the alias now points at the generic type:
```python
class CreateDraftWorkspaceFromCapabilityResult(DraftWorkspaceResult):
"""Draft workspace result plus wrapper hints and advisory next actions."""
wrapper_hints: dict[str, Any] = Field(
description=(
"The wrapper_hints payload used before applying request overrides. "
"Use this to patch uncertain maps or schemas by revision."
)
)
next_actions: WrapperDraftNextActions = Field(
description=(
"Advisory next step guidance derived from wrapper_hints. "
"The server does not enforce can_save_now."
)
)
```
- [ ] **Step 4: Run model import check**
Run:
```bash
uv run python -c "from wf_mcp.workflow_surface.models import WrapperDraftNextActions, CreateDraftWorkspaceFromCapabilityResult; print(WrapperDraftNextActions.__name__)"
```
Expected output includes:
```text
NextActions
```
---
### Task 4: Replace Handler Dict Helpers with `NextActions`
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- [ ] **Step 1: Import `NextActions`**
Add to the local imports in `src/wf_mcp/workflow_surface/handlers.py`:
```python
from .next_actions import NextActions
```
- [ ] **Step 2: Update `create_draft_workspace_from_capability`**
Replace the `next_actions` part of the return value with:
```python
return {
**result,
"wrapper_hints": hints,
"next_actions": NextActions.from_wrapper_hints(
workspace_id=workspace_id,
revision=int(result["revision"]),
hints=hints,
).model_dump(mode="json"),
}
```
- [ ] **Step 3: Delete handler-local guidance helpers**
Delete these functions from `src/wf_mcp/workflow_surface/handlers.py`:
```python
def _wrapper_draft_next_actions(...)
def _wrapper_draft_patch_examples(...)
```
Do not delete `_source_id_for_capability(...)`, which follows those helpers.
- [ ] **Step 4: Run focused draft tests**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py -q
```
Expected: PASS except for assertions that still need additive `can_continue` checks in Task 5.
---
### Task 5: Update Integration and Schema Tests
**Files:**
- Modify: `tests/wf_mcp/workflow_surface/test_drafts.py`
- Modify: `tests/wf_mcp/server/test_config.py`
- [ ] **Step 1: Add `can_continue` assertions to draft integration tests**
In `tests/wf_mcp/workflow_surface/test_drafts.py`, find the test that asserts high-confidence `next_actions`.
Add this assertion near the other `next_actions` assertions:
```python
assert next_actions["can_continue"] is True
```
Keep these existing assertions:
```python
assert next_actions["can_save_now"] is True
assert next_actions["recommended_next_tool"] == (
"wf.workflow.validate_draft_workspace"
)
assert "high confidence" in next_actions["reason"]
assert next_actions["patch_examples"] == []
assert next_actions["warnings"] == []
```
- [ ] **Step 2: Add `can_continue` assertion to low-confidence integration test**
In the low-confidence/content-block test in `tests/wf_mcp/workflow_surface/test_drafts.py`, add:
```python
assert next_actions["can_continue"] is True
```
Keep field-level assertions. Do not assert whole dict equality.
- [ ] **Step 3: Update MCP schema test**
In `tests/wf_mcp/server/test_config.py`, find the `create_draft_workspace_from_capability` output schema test.
Add:
```python
next_actions_schema = result_schema["properties"]["next_actions"]
next_action_properties = next_actions_schema["properties"]
assert "can_continue" in next_action_properties
assert "Advisory" in next_action_properties["can_continue"]["description"]
assert "can_save_now" in next_action_properties
assert "Advisory" in next_action_properties["can_save_now"]["description"]
```
If the existing test already has `next_actions_schema`, reuse it. Do not assert the entire generated schema.
- [ ] **Step 4: Run focused tests**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py -q
```
Expected: PASS.
---
### Task 6: Add a Small Documentation Note
**Files:**
- Modify: `docs/workflow_capabilities.md`
- [ ] **Step 1: Add an advisory-guidance note**
Find the section that mentions `next_actions` or wrapper draft guidance. Add this paragraph:
```markdown
`next_actions` is advisory guidance, not validation authority. It gives MCP
clients a compact "what should I call next?" pointer, while diagnostics,
artifact validation, deployment validation, and runtime status remain the
source of truth.
```
- [ ] **Step 2: Verify docs reference still works**
Run:
```bash
uv run pytest tests/wf_mcp/server/test_docs.py -q
```
Expected: PASS.
---
### Task 7: Full Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused tests**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py tests/wf_mcp/server/test_docs.py -q
```
Expected: PASS.
- [ ] **Step 2: Run formatting check on touched files**
Run:
```bash
uv run ruff format --check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
```
Expected: PASS.
- [ ] **Step 3: Run lint on touched files**
Run:
```bash
uv run ruff check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
```
Expected: PASS.
- [ ] **Step 4: Run type check**
Run:
```bash
uv run basedpyright --level error
```
Expected: `0 errors`.
- [ ] **Step 5: Run the full test suite if time allows**
Run:
```bash
uv run pytest -q
```
Expected: current suite status should remain unchanged from the baseline, currently about `723 passed, 1 skipped, 1 xfailed`.
---
## Self-Review Checklist
- Spec coverage:
- `next_actions.py` is created.
- Generic models exist.
- Wrapper-hint constructor exists.
- Handler-local dict helpers are removed.
- Existing JSON fields remain stable.
- Deployment/run guidance is intentionally deferred.
- Placeholder scan:
- No `TBD`.
- No open-ended "add appropriate validation".
- Each code step includes exact snippets.
- Type consistency:
- `NextActionTool.PATCH_DRAFT_WORKSPACE.value` serializes to the existing string.
- `WrapperDraftNextActions` remains importable as an alias.
- `CreateDraftWorkspaceFromCapabilityResult.next_actions` keeps the same public response location.
## Handoff Notes
This plan is intentionally small. It should not change workflow behavior or draft semantics. If any test fails outside `next_actions` schema/serialization, stop and inspect before broadening the change.
@@ -0,0 +1,481 @@
# Workflow Lifecycle Operator Docs Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make the MCP-facing workflow lifecycle understandable end-to-end for LLM clients and human operators.
**Architecture:** This is a documentation and schema-test pass, not a runtime redesign. The docs should describe the current stable workflow control surface: discover capabilities, create a draft workspace, save an artifact, save/validate/delete a deployment, run, inspect trace slices, and resume interrupted durable runs. Keep examples aligned with the actual MCP tool names and current compact-response philosophy.
**Tech Stack:** Markdown docs, Python tests, FastMCP tool metadata, `uv run pytest`, `uv run ruff`, `uv run basedpyright --level error`.
---
## Scope
Implement documentation for the current workflow lifecycle. Do not add new MCP tools unless a test proves a documented flow cannot be expressed with existing tools.
Current tools to document as primary path:
- `wf.workflow.list_capabilities`
- `wf.workflow.inspect_capability`
- `wf.workflow.create_draft_workspace_from_capability`
- `wf.workflow.patch_draft_workspace`
- `wf.workflow.validate_draft_workspace`
- `wf.workflow.create_artifact_from_workspace`
- `wf.workflow.save_deployment`
- `wf.workflow.validate_deployment`
- `wf.workflow.run_deployment`
- `wf.workflow.inspect_run`
- `wf.workflow.read_run_trace`
- `wf.workflow.resume_run`
- `wf.workflow.delete_deployment`
Important behavior to make explicit:
- `validate_deployment` is stored-catalog validation by default.
- `validate_deployment(live_check=true)` contacts required upstream sources and can spawn stdio/network work.
- `run_deployment` returns compact status and `trace_count`; do not request trace by default.
- `read_run_trace` is the explicit bounded debug path.
- `resume_run` works for durable stopped runs when the run is interrupted and dependencies remain compatible.
- `delete_deployment` removes only mutable deployment bindings. It does not delete immutable workflow artifacts or durable run records.
## Files
- Modify: `docs/wf_mcp_operator_manual.md`
- Primary mental model and short command flow.
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
- Full happy path with concrete MCP tool payloads.
- Modify: `docs/wf_mcp_troubleshooting.md`
- Failure-mode lookup for validation, live checks, run, resume, and cleanup.
- Modify: `docs/durable_run_operations.md`
- Clarify run/resume/trace lifecycle and relationship to deployment deletion.
- Modify: `docs/README.md`
- Ensure the docs index points readers to the operator manual/runbook/troubleshooting in the right order.
- Modify: `tests/wf_mcp/server/test_docs.py`
- Add low-cost assertions that the exported docs resources include the new lifecycle terms.
- Modify only if needed: `src/wf_mcp/documentation.py`
- Do not add new docs resources unless existing resources do not expose the updated docs.
## Task 1: Update The Operator Manual Primary Path
**Files:**
- Modify: `docs/wf_mcp_operator_manual.md`
- [ ] **Step 1: Find the current workflow tool family section**
Run:
```powershell
rg -n "Workflow Tools|workflow tools|validate_deployment|run_deployment|delete_deployment" docs/wf_mcp_operator_manual.md
```
Expected: existing sections mention workflow discovery, deployment validation, run, and resume.
- [ ] **Step 2: Add a concise primary-path checklist**
Add or update a section named exactly:
```markdown
## Primary Workflow Lifecycle
```
Use this content, adjusting surrounding prose only for fit:
```markdown
## Primary Workflow Lifecycle
Use this path when an LLM client needs to build, test, and run a saved workflow.
1. Discover workflow-ready capabilities with `wf.workflow.list_capabilities`.
2. Inspect the selected capability with `wf.workflow.inspect_capability`.
3. Create a patchable draft with `wf.workflow.create_draft_workspace_from_capability`.
4. Patch the draft with `wf.workflow.patch_draft_workspace`.
5. Validate the draft with `wf.workflow.validate_draft_workspace`.
6. Save an immutable workflow or wrapper artifact with `wf.workflow.create_artifact_from_workspace` or `wf.workflow.create_wrapper_from_workspace`.
7. Save a mutable deployment with `wf.workflow.save_deployment`.
8. Validate the deployment with `wf.workflow.validate_deployment`.
9. Optionally call `wf.workflow.validate_deployment` with `live_check=true` before a real run.
10. Run with `wf.workflow.run_deployment`.
11. If the run returns `interrupted`, resume with `wf.workflow.resume_run`.
12. Inspect stopped runs with `wf.workflow.inspect_run`; read bounded trace slices with `wf.workflow.read_run_trace`.
13. Delete temporary deployments with `wf.workflow.delete_deployment`.
Artifacts are immutable saved definitions. Deployments are mutable environment bindings. Runs are durable stopped execution records. Deleting a deployment does not delete artifacts or existing run records.
```
- [ ] **Step 3: Document `live_check` next to deployment validation**
Find the `validate_deployment` section and add:
```markdown
By default, `validate_deployment` validates against the broker's current source inventory and saved catalog snapshots. This is cheap and side-effect-light.
Pass `live_check=true` only when you explicitly want to contact each required upstream source. A live check may spawn stdio MCP servers or perform network I/O. Live-check failures are returned as `source_unreachable` diagnostics.
```
- [ ] **Step 4: Document `delete_deployment` in the tool table**
Add a table row near deployment tools:
```markdown
| Delete a temporary deployment binding | `wf.workflow.delete_deployment` |
```
Also add:
```markdown
`delete_deployment` removes the saved deployment binding only. It does not delete workflow artifacts, wrapper artifacts, or run checkpoints.
```
- [ ] **Step 5: Run a docs grep sanity check**
Run:
```powershell
rg -n "Primary Workflow Lifecycle|live_check|source_unreachable|delete_deployment" docs/wf_mcp_operator_manual.md
```
Expected: all four terms appear.
## Task 2: Update The End-To-End Runbook With Concrete Payloads
**Files:**
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
- [ ] **Step 1: Locate the happy-path flow**
Run:
```powershell
rg -n "create_draft_workspace_from_capability|save_deployment|validate_deployment|run_deployment|resume_run" docs/wf_mcp_end_to_end_runbook.md
```
Expected: existing examples for draft creation, deployment validation, and run execution.
- [ ] **Step 2: Add a compact lifecycle summary near the top**
Add:
```markdown
## Minimal Lifecycle Summary
The shortest dependable lifecycle is:
```text
list_capabilities
inspect_capability
create_draft_workspace_from_capability
patch_draft_workspace
validate_draft_workspace
create_artifact_from_workspace
save_deployment
validate_deployment
run_deployment
inspect_run or read_run_trace only when needed
resume_run only when status is interrupted
delete_deployment for temporary deployments
```
Do not expect a newly saved workflow to appear as a new MCP tool in an existing client session. Use `run_deployment` and `call_capability` as stable front doors.
```
- [ ] **Step 3: Add an explicit live validation example**
Near the deployment validation example, add:
```markdown
### Optional Live Source Check
Use this before a real run when you need to know whether the bound upstream source can currently answer.
```yaml
tool: wf.workflow.validate_deployment
arguments:
deployment_id: "example.personal"
live_check: true
```
Expected successful shape:
```json
{
"deployment_id": "example.personal",
"status": "runnable",
"diagnostics": []
}
```
If a bound upstream source is down, expect `status="unrunnable"` and a diagnostic with `code="source_unreachable"`.
```
- [ ] **Step 4: Add a cleanup example**
Near the end of the runbook, add:
```markdown
### Cleanup Temporary Deployments
Temporary test deployments can be removed without touching immutable artifacts.
```yaml
tool: wf.workflow.delete_deployment
arguments:
deployment_id: "example.personal"
```
Expected:
```json
{
"deployment_id": "example.personal",
"deleted": true
}
```
```
- [ ] **Step 5: Run grep sanity check**
Run:
```powershell
rg -n "Minimal Lifecycle Summary|Optional Live Source Check|delete_deployment|source_unreachable" docs/wf_mcp_end_to_end_runbook.md
```
Expected: all four terms appear.
## Task 3: Update Troubleshooting For Live Checks And Cleanup
**Files:**
- Modify: `docs/wf_mcp_troubleshooting.md`
- [ ] **Step 1: Locate validation diagnostics**
Run:
```powershell
rg -n "binding_missing|source_missing|source_disabled|capability_missing|schema_changed|source_unreachable|delete_deployment" docs/wf_mcp_troubleshooting.md
```
Expected: existing sections for static diagnostics; `source_unreachable` may be missing.
- [ ] **Step 2: Add `source_unreachable` section**
Add this section after `source_disabled` or near other deployment validation diagnostics:
```markdown
## `validate_deployment(live_check=true)` Says `source_unreachable`
Meaning: static deployment validation found a matching saved source/catalog, but the live upstream source could not answer when contacted.
Common causes:
- stdio MCP server command is missing or exits during startup
- network MCP server is offline
- auth/config changed outside the broker
- source process starts too slowly and hits the live-check timeout
What to do:
1. Check the connection with `wf.admin.get_connection_statuses`.
2. Refresh or reload the config if the source was recently enabled.
3. Fix the source command/auth/network outside the workflow artifact.
4. Run `wf.workflow.validate_deployment` again with `live_check=true`.
Do not fix this by editing the workflow artifact unless the source capability itself changed. This is an environment problem, not workflow business logic.
```
- [ ] **Step 3: Add deployment cleanup troubleshooting**
Add:
```markdown
## Test Deployment Clutter
Symptom: `wf.workflow.list_deployments` shows temporary deployments from earlier tests or LLM attempts.
Use:
```yaml
tool: wf.workflow.delete_deployment
arguments:
deployment_id: "test_alias_check"
```
This deletes only the mutable deployment binding. Saved artifacts and durable run records remain.
```
- [ ] **Step 4: Run grep sanity check**
Run:
```powershell
rg -n "source_unreachable|Test Deployment Clutter|delete_deployment|test_alias_check" docs/wf_mcp_troubleshooting.md
```
Expected: all four terms appear.
## Task 4: Update Durable Run Docs For Deployment Deletion Boundaries
**Files:**
- Modify: `docs/durable_run_operations.md`
- [ ] **Step 1: Locate run/deployment lifecycle text**
Run:
```powershell
rg -n "deployment|run_deployment|resume_run|inspect_run|read_run_trace|delete_deployment" docs/durable_run_operations.md
```
Expected: current run/resume docs; `delete_deployment` may be missing.
- [ ] **Step 2: Add deployment deletion boundary note**
Add under the run lifecycle or deployment section:
```markdown
## Deployment Deletion Boundary
`wf.workflow.delete_deployment` removes the mutable deployment binding. It does not delete:
- immutable workflow artifacts
- wrapper artifacts
- stored run records
- run checkpoints
Existing run records keep the pinned deployment and artifact environment captured at run time. Deleting a deployment prevents future runs through that deployment id, but it does not erase historical stopped-run inspection data.
```
- [ ] **Step 3: Reconfirm trace guidance**
Ensure the doc says:
```markdown
Use `inspect_run` for compact stopped-run summaries. Use `read_run_trace` only when trace entries are needed, and always request a bounded `trace_range`.
```
- [ ] **Step 4: Run grep sanity check**
Run:
```powershell
rg -n "Deployment Deletion Boundary|delete_deployment|read_run_trace|trace_range" docs/durable_run_operations.md
```
Expected: all four terms appear.
## Task 5: Update Docs Index And Exported Docs Tests
**Files:**
- Modify: `docs/README.md`
- Modify: `tests/wf_mcp/server/test_docs.py`
- [ ] **Step 1: Update docs index ordering**
In `docs/README.md`, ensure these entries exist and read clearly:
```markdown
- [`wf_mcp_operator_manual.md`](wf_mcp_operator_manual.md): start here for the MCP-facing workflow lifecycle and tool families.
- [`wf_mcp_end_to_end_runbook.md`](wf_mcp_end_to_end_runbook.md): concrete tool-call runbook from capability discovery through deployment, run, resume, and cleanup.
- [`wf_mcp_troubleshooting.md`](wf_mcp_troubleshooting.md): diagnostics and repair steps for source, deployment, run, and resume failures.
- [`durable_run_operations.md`](durable_run_operations.md): durable run records, compact inspection, bounded traces, and resume semantics.
```
- [ ] **Step 2: Add docs resource assertions**
Open `tests/wf_mcp/server/test_docs.py` and find the test that reads `wf://docs/operator-manual` or docs resources.
Add field-level assertions instead of whole-dict assertions:
```python
assert "Primary Workflow Lifecycle" in manual_text
assert "live_check" in manual_text
assert "delete_deployment" in manual_text
```
If the test currently uses a different variable name than `manual_text`, use the existing variable. Do not assert whole payload dict equality.
- [ ] **Step 3: Run focused docs tests**
Run:
```powershell
uv run pytest tests/wf_mcp/server/test_docs.py -q
```
Expected: all tests in that file pass.
## Task 6: Final Verification
**Files:**
- All modified docs and tests.
- [ ] **Step 1: Run docs grep checks**
Run:
```powershell
rg -n "Primary Workflow Lifecycle|live_check|source_unreachable|delete_deployment|Deployment Deletion Boundary" docs
```
Expected: terms appear in the intended docs, not only in historical plans.
- [ ] **Step 2: Run focused tests**
Run:
```powershell
uv run pytest tests/wf_mcp/server/test_docs.py tests/wf_mcp/server/test_tools.py tests/wf_mcp/workflow_surface/test_deployments.py -q
```
Expected: pass.
- [ ] **Step 3: Run lint/format checks for touched files**
Run:
```powershell
uv run ruff check tests/wf_mcp/server/test_docs.py
uv run ruff format --check tests/wf_mcp/server/test_docs.py
```
Expected: pass.
- [ ] **Step 4: Run type check for touched Python files**
Run:
```powershell
uv run basedpyright --level error tests/wf_mcp/server/test_docs.py
```
Expected: `0 errors`.
- [ ] **Step 5: Optional full test suite**
Run when time allows:
```powershell
uv run pytest -q
```
Expected current baseline: full suite passes with the existing skip/xfail count.
## Notes For Opencode
- Keep this docs-first. Do not add new runtime behavior unless a doc assertion proves the current docs cannot represent the actual tool surface.
- Prefer concise examples over giant JSON payloads.
- Use exact current tool names with the `wf.workflow.*` namespace.
- Do not document raw MCP proxy tools as the workflow authoring path.
- Do not claim `delete_deployment` deletes artifacts or runs.
- Do not tell users to expect saved workflows to appear as new MCP tools in existing client sessions.
@@ -0,0 +1,485 @@
# Wrapper Draft Next Actions Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add advisory `next_actions` to `wf.workflow.create_draft_workspace_from_capability` so LLM clients can continue safely even when they cannot easily read the docs.
**Architecture:** Keep `wrapper_hints` as the source of truth for scaffold confidence, missing decisions, and mapping warnings. Add a small typed result object that converts those hints into concrete next-tool guidance. `can_save_now` is advisory only; do not block saving or enforce policy in this slice.
**Tech Stack:** Python 3.14, Pydantic v2, FastMCP tool schemas, pytest, ruff, basedpyright.
---
## Scope
Add `next_actions` to `CreateDraftWorkspaceFromCapabilityResult` and handler payloads.
Do:
- Return machine-readable guidance from `create_draft_workspace_from_capability`.
- Make `can_save_now` advisory only.
- Recommend `wf.workflow.patch_draft_workspace` when `wrapper_hints.missing_decisions` is non-empty or confidence is low.
- Recommend `wf.workflow.validate_draft_workspace` when the scaffold looks safe enough to validate.
- Include patch examples for common missing decisions.
- Document the field and add schema tests.
Do not:
- Block `create_artifact_from_workspace`.
- Add a new tool.
- Infer raw MCP `content[0].text` automatically.
- Treat boolean output candidates as real routing semantics.
- Replace `wrapper_hints`; `next_actions` should summarize and guide, not duplicate every hint.
## Files
- Modify: `src/wf_mcp/workflow_surface/models.py`
- Add typed `WrapperDraftNextActions` and `WrapperDraftPatchExample` Pydantic models.
- Add `next_actions` field to `CreateDraftWorkspaceFromCapabilityResult`.
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Add helper that derives `next_actions` from `wrapper_hints` and workspace id/revision.
- Include `next_actions` in `create_draft_workspace_from_capability` result.
- Test: `tests/wf_mcp/workflow_surface/test_drafts.py`
- Assert high-confidence draft returns validate/save guidance.
- Assert low-confidence content-block draft returns patch guidance and advisory `can_save_now=false`.
- Test: `tests/wf_mcp/server/test_config.py`
- Assert output schema exposes `next_actions` with field descriptions.
- Modify: `docs/workflow_capabilities.md`
- Explain `next_actions` as advisory continuation hints.
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
- Mention that clients can follow `next_actions` after draft creation.
## Data Shape
Add this output shape:
```json
{
"next_actions": {
"can_save_now": true,
"recommended_next_tool": "wf.workflow.validate_draft_workspace",
"reason": "Wrapper hints are high confidence and have no missing decisions.",
"patch_examples": [],
"warnings": []
}
}
```
Low-confidence example:
```json
{
"next_actions": {
"can_save_now": false,
"recommended_next_tool": "wf.workflow.patch_draft_workspace",
"reason": "Review missing wrapper decisions before saving.",
"patch_examples": [
{
"description": "Replace the output bindings after choosing workflow state fields.",
"tool": "wf.workflow.patch_draft_workspace",
"request": {
"workspace_id": "echo_wrapper",
"revision": 1,
"patch": [
{
"op": "replace",
"path": "/draft/steps/call/output",
"value": []
}
]
}
}
],
"warnings": [
"Raw MCP content blocks are not workflow-shaped. Use an explicit wrapper or extraction node."
]
}
}
```
## Task 1: Add Result Models And Schema Test
**Files:**
- Modify: `src/wf_mcp/workflow_surface/models.py`
- Modify: `tests/wf_mcp/server/test_config.py`
- [ ] **Step 1: Add failing schema assertions**
In `tests/wf_mcp/server/test_config.py`, inside the test that already inspects `create_draft_workspace_from_capability` output schema, add:
```python
assert "next_actions" in from_capability_output["properties"]
next_actions_schema = from_capability_output["properties"]["next_actions"]
assert "recommended_next_tool" in next_actions_schema["properties"]
assert "patch_examples" in next_actions_schema["properties"]
assert "advisory" in next_actions_schema["properties"]["can_save_now"]["description"]
```
Run:
```powershell
uv run pytest tests/wf_mcp/server/test_config.py -q
```
Expected: fail because `next_actions` is not in the output schema.
- [ ] **Step 2: Add Pydantic result models**
In `src/wf_mcp/workflow_surface/models.py`, add near `CreateDraftWorkspaceFromCapabilityResult`:
```python
class WrapperDraftPatchExample(BaseModel):
"""Concrete patch-workspace example for a likely next authoring edit."""
description: str = Field(description="Human-readable reason for this patch.")
tool: str = Field(description="MCP tool to call for this example.")
request: dict[str, Any] = Field(
description="JSON request payload to pass to the tool."
)
class WrapperDraftNextActions(BaseModel):
"""Advisory continuation hints after bootstrapping a wrapper draft."""
can_save_now: bool = Field(
description=(
"Advisory only. False means the scaffold likely needs review before "
"saving, but the server does not enforce this."
)
)
recommended_next_tool: str = Field(
description=(
"Suggested next MCP tool, usually wf.workflow.validate_draft_workspace "
"or wf.workflow.patch_draft_workspace."
)
)
reason: str = Field(description="Short explanation for the recommendation.")
patch_examples: list[WrapperDraftPatchExample] = Field(
default_factory=list,
description="Concrete JSON Patch examples for common missing decisions.",
)
warnings: list[str] = Field(
default_factory=list,
description="Non-blocking warnings copied from low-confidence wrapper hints.",
)
```
Then update:
```python
class CreateDraftWorkspaceFromCapabilityResult(DraftWorkspaceResult):
"""Draft workspace result plus wrapper hints and advisory next actions."""
wrapper_hints: dict[str, Any] = Field(...)
next_actions: WrapperDraftNextActions = Field(
description=(
"Advisory next step guidance derived from wrapper_hints. "
"The server does not enforce can_save_now."
)
)
```
- [ ] **Step 3: Run schema test**
Run:
```powershell
uv run pytest tests/wf_mcp/server/test_config.py -q
```
Expected: schema assertion passes, but runtime tests may still fail later until handler returns `next_actions`.
## Task 2: Derive Next Actions In The Handler
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `tests/wf_mcp/workflow_surface/test_drafts.py`
- [ ] **Step 1: Add high-confidence behavior test**
In `tests/wf_mcp/workflow_surface/test_drafts.py`, extend `test_workflow_surface_creates_draft_workspace_from_capability_hints`:
```python
next_actions = result["next_actions"]
assert next_actions["can_save_now"] is True
assert next_actions["recommended_next_tool"] == "wf.workflow.validate_draft_workspace"
assert "high confidence" in next_actions["reason"]
assert next_actions["patch_examples"] == []
assert next_actions["warnings"] == []
```
Run:
```powershell
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py::test_workflow_surface_creates_draft_workspace_from_capability_hints -q
```
Expected: fail because handler does not return `next_actions`.
- [ ] **Step 2: Add low-confidence behavior test**
Find or create a content-block capability test near existing wrapper hint tests. Use an output schema shaped like:
```python
content_output_schema = {
"type": "object",
"properties": {
"content": {
"type": "array",
"items": {
"type": "object",
"properties": {
"type": {"type": "string"},
"text": {"type": "string"},
},
},
}
},
}
```
Register a NodeSpec/capability with that output schema, call `create_draft_workspace_from_capability`, then assert:
```python
next_actions = result["next_actions"]
assert next_actions["can_save_now"] is False
assert next_actions["recommended_next_tool"] == "wf.workflow.patch_draft_workspace"
assert "missing wrapper decisions" in next_actions["reason"]
assert next_actions["patch_examples"][0]["tool"] == "wf.workflow.patch_draft_workspace"
assert next_actions["patch_examples"][0]["request"]["workspace_id"] == "content_wrapper"
assert next_actions["patch_examples"][0]["request"]["revision"] == result["revision"]
assert next_actions["warnings"]
```
Run:
```powershell
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py -q
```
Expected: fail until implementation exists.
- [ ] **Step 3: Implement helper in handler**
In `src/wf_mcp/workflow_surface/handlers.py`, add a helper near other draft helpers:
```python
def _wrapper_draft_next_actions(
*,
workspace_id: str,
revision: int,
hints: dict[str, Any],
) -> dict[str, Any]:
"""Convert wrapper_hints into advisory next-tool guidance for MCP clients."""
confidence = str(hints.get("confidence", "low"))
missing_decisions = hints.get("missing_decisions")
notes = [str(note) for note in hints.get("notes", []) if isinstance(note, str)]
has_missing = isinstance(missing_decisions, list) and len(missing_decisions) > 0
can_save_now = confidence == "high" and not has_missing
if can_save_now:
return {
"can_save_now": True,
"recommended_next_tool": "wf.workflow.validate_draft_workspace",
"reason": "Wrapper hints are high confidence and have no missing decisions.",
"patch_examples": [],
"warnings": [],
}
return {
"can_save_now": False,
"recommended_next_tool": "wf.workflow.patch_draft_workspace",
"reason": "Review missing wrapper decisions before saving.",
"patch_examples": _wrapper_draft_patch_examples(
workspace_id=workspace_id,
revision=revision,
hints=hints,
),
"warnings": notes,
}
```
Add:
```python
def _wrapper_draft_patch_examples(
*,
workspace_id: str,
revision: int,
hints: dict[str, Any],
) -> list[dict[str, Any]]:
"""Return conservative JSON Patch examples without guessing semantics."""
examples: list[dict[str, Any]] = []
missing_decisions = hints.get("missing_decisions")
if not isinstance(missing_decisions, list):
return examples
decision_kinds = {
str(decision.get("kind"))
for decision in missing_decisions
if isinstance(decision, dict)
}
if {
"choose_output_fields",
"review_nested_output",
} & decision_kinds:
examples.append(
{
"description": (
"Replace output bindings after choosing which capability "
"outputs should be written to workflow state."
),
"tool": "wf.workflow.patch_draft_workspace",
"request": {
"workspace_id": workspace_id,
"revision": revision,
"patch": [
{
"op": "replace",
"path": "/draft/steps/call/output",
"value": [],
}
],
},
}
)
if "confirm_boolean_outcomes" in decision_kinds:
examples.append(
{
"description": (
"Review boolean output candidates before adding routing; "
"do not route on boolean fields automatically."
),
"tool": "wf.workflow.patch_draft_workspace",
"request": {
"workspace_id": workspace_id,
"revision": revision,
"patch": [],
},
}
)
return examples
```
Then change the return in `create_draft_workspace_from_capability`:
```python
return {
**result,
"wrapper_hints": hints,
"next_actions": _wrapper_draft_next_actions(
workspace_id=workspace_id,
revision=int(result["revision"]),
hints=hints,
),
}
```
- [ ] **Step 4: Run behavior tests**
Run:
```powershell
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py -q
```
Expected: pass.
## Task 3: Document Next Actions
**Files:**
- Modify: `docs/workflow_capabilities.md`
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
- [ ] **Step 1: Update workflow capability docs**
In `docs/workflow_capabilities.md`, near the `wrapper_hints` section, add:
```markdown
`create_draft_workspace_from_capability` also returns `next_actions`.
This is advisory guidance for clients that cannot easily read the full docs.
It summarizes whether the scaffold is safe-looking enough to validate, which
tool to call next, and concrete patch examples for common missing decisions.
`next_actions.can_save_now` is not enforced. A caller can still save a low
confidence draft, but the field exists to make that risk explicit.
```
- [ ] **Step 2: Update runbook**
In `docs/wf_mcp_end_to_end_runbook.md`, near the draft-from-capability example, add:
```markdown
After `create_draft_workspace_from_capability`, inspect `next_actions`.
If `recommended_next_tool` is `wf.workflow.patch_draft_workspace`, apply or
adapt the returned `patch_examples` before saving. If it recommends
`wf.workflow.validate_draft_workspace`, validate the draft before creating an
artifact.
```
- [ ] **Step 3: Grep docs**
Run:
```powershell
rg -n "next_actions|can_save_now|patch_examples" docs/workflow_capabilities.md docs/wf_mcp_end_to_end_runbook.md
```
Expected: all three terms appear.
## Task 4: Final Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused tests**
Run:
```powershell
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py tests/wf_mcp/test_workflow_wrapper_hints.py -q
```
Expected: pass.
- [ ] **Step 2: Run touched-file lint and format check**
Run:
```powershell
uv run ruff check src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
uv run ruff format --check src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
```
Expected: pass.
- [ ] **Step 3: Run touched-file type check**
Run:
```powershell
uv run basedpyright --level error src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
```
Expected: `0 errors`.
- [ ] **Step 4: Optional full suite**
Run when time allows:
```powershell
uv run pytest -q
```
Expected current baseline: full suite passes with the existing skip/xfail count.
## Notes For Opencode
- `can_save_now` is advisory. Do not enforce it.
- Do not add a new save gate.
- Do not generate semantic routes from boolean fields.
- Keep patch examples conservative; empty patch examples are acceptable when the missing decision cannot be safely represented.
- Avoid whole-dict assertions. Assert individual fields.
- Existing docs may be long; keep additions short.
@@ -0,0 +1,707 @@
# Deployment And Run Next Actions Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add advisory `next_actions` guidance to deployment validation and run lifecycle responses so MCP clients know the safest next workflow tool to call.
**Architecture:** Extend the existing `src/wf_mcp/workflow_surface/next_actions.py` module with deployment/run constructors instead of putting more UX policy in `handlers.py`. Thread the resulting `NextActions` object into `validate_deployment(...)` and `_run_payload(...)`, which covers `run_deployment`, `resume_run`, `inspect_run`, and `read_run_trace`. Keep diagnostics and runtime status authoritative; `next_actions` remains advisory.
**Tech Stack:** Python 3.14, Pydantic v2, pytest, ruff, basedpyright.
---
## File Structure
- Modify `src/wf_mcp/workflow_surface/next_actions.py`
- Add `NextActions.from_deployment_validation(...)`.
- Add `NextActions.from_run_result(...)`.
- Add tiny private helpers for diagnostic code inspection and bounded trace examples.
- Modify `src/wf_mcp/workflow_surface/handlers.py`
- Add `next_actions` to `validate_deployment(...)`.
- Add optional `next_actions` construction inside `_run_payload(...)`.
- Do not duplicate branching logic in individual handler methods.
- Modify `tests/wf_mcp/workflow_surface/test_next_actions.py`
- Unit-test deployment/run constructors directly.
- Modify `tests/wf_mcp/workflow_surface/test_deployments.py`
- Assert `validate_deployment` returns useful next actions for runnable and unrunnable deployments.
- Modify `tests/wf_mcp/workflow_surface/test_runs.py`
- Assert run lifecycle responses return next actions for completed and failed runs.
- Add interrupt/resume tests only if an existing fixture/helper makes this small; otherwise leave interrupt-specific coverage to constructor unit tests.
- Modify `tests/wf_mcp/server/test_config.py`
- Assert output schemas for `validate_deployment` and `run_deployment` include `next_actions`.
- Modify `docs/workflow_capabilities.md`
- Add a short note that deployment/run responses now include advisory `next_actions`.
## Scope Boundaries
- Do not create new workflow tools.
- Do not make `next_actions` required for correctness.
- Do not read or return full traces automatically.
- Do not add auto-repair behavior.
- Do not change existing status strings, diagnostics, or run payload fields.
- Do not implement persisted resume changes in this pass.
---
### Task 1: Add Constructor Unit Tests
**Files:**
- Modify: `tests/wf_mcp/workflow_surface/test_next_actions.py`
- [ ] **Step 1: Add imports**
At the top of `tests/wf_mcp/workflow_surface/test_next_actions.py`, add:
```python
from wf_artifacts import DependencyDiagnostic, DiagnosticSeverity
```
Keep the existing `NextActionTool` / `NextActions` import.
- [ ] **Step 2: Add deployment validation constructor tests**
Append to `tests/wf_mcp/workflow_surface/test_next_actions.py`:
```python
def test_next_actions_from_runnable_deployment_recommends_run() -> None:
actions = NextActions.from_deployment_validation(
deployment_id="echo.personal",
diagnostics=[],
)
dumped = actions.model_dump(mode="json")
assert dumped["can_continue"] is True
assert dumped["recommended_next_tool"] == NextActionTool.RUN_DEPLOYMENT.value
assert "run_deployment" in dumped["reason"]
assert dumped["warnings"] == []
def test_next_actions_from_unrunnable_deployment_recommends_validation_retry() -> None:
diagnostic = DependencyDiagnostic(
severity=DiagnosticSeverity.ERROR,
code="source_unreachable",
logical_ref="demo.echo_tool",
bound_source="demo.personal",
message="Live check for upstream source 'demo.personal' failed.",
repair_hint="Start or reconnect the source.",
)
actions = NextActions.from_deployment_validation(
deployment_id="echo.personal",
diagnostics=[diagnostic],
)
dumped = actions.model_dump(mode="json")
assert dumped["can_continue"] is True
assert dumped["recommended_next_tool"] == NextActionTool.VALIDATE_DEPLOYMENT.value
assert "fix or reconnect" in dumped["reason"]
assert dumped["warnings"][0] == "source_unreachable: demo.personal"
```
- [ ] **Step 3: Add run result constructor tests**
Append to `tests/wf_mcp/workflow_surface/test_next_actions.py`:
```python
def test_next_actions_from_completed_run_has_no_required_next_tool() -> None:
actions = NextActions.from_run_result(
run_id="run_123",
status="completed",
trace_count=2,
diagnostics=[],
)
dumped = actions.model_dump(mode="json")
assert dumped["can_continue"] is False
assert dumped["recommended_next_tool"] is None
assert "completed" in dumped["reason"]
assert dumped["patch_examples"] == []
def test_next_actions_from_failed_run_recommends_bounded_trace() -> None:
actions = NextActions.from_run_result(
run_id="run_123",
status="failed",
trace_count=12,
diagnostics=[],
)
dumped = actions.model_dump(mode="json")
assert dumped["can_continue"] is True
assert dumped["recommended_next_tool"] == NextActionTool.READ_RUN_TRACE.value
assert "bounded trace" in dumped["reason"]
assert dumped["patch_examples"][0]["tool"] == NextActionTool.READ_RUN_TRACE.value
assert dumped["patch_examples"][0]["request"]["run_id"] == "run_123"
assert dumped["patch_examples"][0]["request"]["trace_range"]["start"] == 0
assert dumped["patch_examples"][0]["request"]["trace_range"]["limit"] == 25
def test_next_actions_from_interrupted_run_recommends_resume() -> None:
actions = NextActions.from_run_result(
run_id="run_123",
status="interrupted",
trace_count=3,
diagnostics=[],
)
dumped = actions.model_dump(mode="json")
assert dumped["can_continue"] is True
assert dumped["recommended_next_tool"] == NextActionTool.RESUME_RUN.value
assert "resume_run" in dumped["reason"]
assert dumped["patch_examples"] == []
```
- [ ] **Step 4: Run the new tests to verify they fail**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
```
Expected: FAIL with `AttributeError` for missing `from_deployment_validation` and `from_run_result`.
---
### Task 2: Implement Deployment/Run Constructors
**Files:**
- Modify: `src/wf_mcp/workflow_surface/next_actions.py`
- [ ] **Step 1: Add type-only imports**
In `src/wf_mcp/workflow_surface/next_actions.py`, add:
```python
from collections.abc import Sequence
```
Do not import `DependencyDiagnostic` directly unless needed at runtime. The constructors can accept diagnostics as objects or dicts to keep coupling low.
- [ ] **Step 2: Add `from_deployment_validation`**
Inside `class NextActions`, after `from_wrapper_hints(...)`, add:
```python
@classmethod
def from_deployment_validation(
cls,
*,
deployment_id: str,
diagnostics: Sequence[object],
) -> Self:
"""Create guidance after validate_deployment."""
if not diagnostics:
return cls(
can_continue=True,
can_save_now=None,
recommended_next_tool=NextActionTool.RUN_DEPLOYMENT,
reason=(
f"Deployment {deployment_id!r} is runnable; call "
"wf.workflow.run_deployment with workflow_input."
),
patch_examples=[],
warnings=[],
)
codes = {_diagnostic_field(diagnostic, "code") for diagnostic in diagnostics}
warnings = [_diagnostic_warning(diagnostic) for diagnostic in diagnostics]
if "source_unreachable" in codes:
reason = (
"One or more live sources are unreachable; fix or reconnect the "
"source, then rerun wf.workflow.validate_deployment with live_check=true."
)
elif "source_missing" in codes or "binding_missing" in codes:
reason = (
"Deployment bindings or sources are missing; inspect the deployment "
"and save corrected bindings before running."
)
elif "capability_missing" in codes or "schema_changed" in codes:
reason = (
"A required capability is missing or drifted; inspect capabilities "
"or refresh sources, then validate again."
)
else:
reason = (
"Deployment is not runnable; inspect diagnostics, repair the "
"deployment or sources, then validate again."
)
return cls(
can_continue=True,
can_save_now=None,
recommended_next_tool=NextActionTool.VALIDATE_DEPLOYMENT,
reason=reason,
patch_examples=[],
warnings=warnings,
)
```
- [ ] **Step 3: Add `from_run_result`**
Inside `class NextActions`, after `from_deployment_validation(...)`, add:
```python
@classmethod
def from_run_result(
cls,
*,
run_id: str | None,
status: str,
trace_count: int,
diagnostics: Sequence[object],
) -> Self:
"""Create guidance after run_deployment, inspect_run, resume_run, or read_run_trace."""
warnings = [_diagnostic_warning(diagnostic) for diagnostic in diagnostics]
if status == "interrupted" and run_id is not None:
return cls(
can_continue=True,
can_save_now=None,
recommended_next_tool=NextActionTool.RESUME_RUN,
reason=(
"Run is interrupted; call wf.workflow.resume_run with this "
"run_id and the interrupt response payload."
),
patch_examples=[],
warnings=warnings,
)
if status in {"failed", "unrunnable"}:
examples = (
[_bounded_trace_example(run_id=run_id, trace_count=trace_count)]
if run_id is not None and trace_count > 0
else []
)
return cls(
can_continue=bool(examples),
can_save_now=None,
recommended_next_tool=(
NextActionTool.READ_RUN_TRACE if examples else None
),
reason=(
"Run failed; read a bounded trace slice for debugging."
if examples
else "Run failed before producing trace entries; inspect diagnostics and error."
),
patch_examples=examples,
warnings=warnings,
)
if status == "completed":
examples = (
[_bounded_trace_example(run_id=run_id, trace_count=trace_count)]
if run_id is not None and trace_count > 0
else []
)
return cls(
can_continue=False,
can_save_now=None,
recommended_next_tool=None,
reason=(
"Run completed. No required next workflow tool; use read_run_trace "
"with a bounded trace_range only if debugging."
),
patch_examples=examples,
warnings=warnings,
)
return cls(
can_continue=False,
can_save_now=None,
recommended_next_tool=None,
reason=f"Run status {status!r} has no obvious next workflow tool.",
patch_examples=[],
warnings=warnings,
)
```
- [ ] **Step 4: Add private diagnostic helpers**
Below `_wrapper_draft_patch_examples(...)`, add:
```python
def _diagnostic_field(diagnostic: object, field: str) -> str | None:
"""Read a diagnostic field from either a Pydantic model or a JSON dict."""
if isinstance(diagnostic, dict):
value = diagnostic.get(field)
else:
value = getattr(diagnostic, field, None)
return value if isinstance(value, str) else None
def _diagnostic_warning(diagnostic: object) -> str:
"""Format one compact diagnostic warning for next_actions."""
code = _diagnostic_field(diagnostic, "code") or "diagnostic"
bound_source = _diagnostic_field(diagnostic, "bound_source")
logical_ref = _diagnostic_field(diagnostic, "logical_ref")
if bound_source:
return f"{code}: {bound_source}"
if logical_ref:
return f"{code}: {logical_ref}"
return code
def _bounded_trace_example(
*,
run_id: str,
trace_count: int,
) -> NextActionPatchExample:
"""Return a safe read_run_trace request; never suggest full trace reads."""
return NextActionPatchExample(
description=(
"Read a bounded debug trace slice. Increase start/limit only when needed."
),
tool=NextActionTool.READ_RUN_TRACE,
request={
"run_id": run_id,
"trace_range": {
"start": 0,
"limit": min(trace_count, 25),
},
},
)
```
- [ ] **Step 5: Run constructor tests**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
```
Expected: PASS.
---
### Task 3: Thread NextActions Through Deployment Validation
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `tests/wf_mcp/workflow_surface/test_deployments.py`
- [ ] **Step 1: Add deployment assertions**
In `test_workflow_surface_validate_deployment_live_check_is_opt_in`, after current payload assertions, add:
```python
assert payload["next_actions"]["can_continue"] is True
assert payload["next_actions"]["recommended_next_tool"] == (
"wf.workflow.run_deployment"
)
```
In `test_workflow_surface_validates_deployment_dependencies`, after diagnostic assertions, add:
```python
assert payload["next_actions"]["can_continue"] is True
assert payload["next_actions"]["recommended_next_tool"] == (
"wf.workflow.validate_deployment"
)
assert payload["next_actions"]["warnings"][0] == "source_missing: context7.personal"
```
If the exact warning uses `context7` instead of `context7.personal`, keep the assertion stable by checking fields separately:
```python
assert payload["next_actions"]["warnings"]
assert "source_missing" in payload["next_actions"]["warnings"][0]
```
Prefer the exact assertion only if the implementation returns `bound_source`.
- [ ] **Step 2: Update `validate_deployment` return payload**
In `src/wf_mcp/workflow_surface/handlers.py`, replace the return body in `validate_deployment(...)` with:
```python
status = "unrunnable" if diagnostics else "runnable"
diagnostic_payloads = [
diagnostic.model_dump(mode="json") for diagnostic in diagnostics
]
return {
"deployment_id": deployment.id,
"artifact_id": artifact.id,
"artifact_version": artifact.version,
"status": status,
"diagnostics": diagnostic_payloads,
"next_actions": NextActions.from_deployment_validation(
deployment_id=deployment.id,
diagnostics=diagnostics,
).model_dump(mode="json"),
}
```
- [ ] **Step 3: Run deployment tests**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface/test_deployments.py tests/wf_mcp/workflow_surface/test_next_actions.py -q
```
Expected: PASS.
---
### Task 4: Thread NextActions Through Run Payloads
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `tests/wf_mcp/workflow_surface/test_runs.py`
- [ ] **Step 1: Add completed-run assertions**
In `test_workflow_surface_runs_non_interrupting_deployment`, after current run payload assertions, add:
```python
assert payload["next_actions"]["can_continue"] is False
assert payload["next_actions"]["recommended_next_tool"] is None
assert "completed" in payload["next_actions"]["reason"]
```
After the `inspected` assertions in the same test, add:
```python
assert inspected["next_actions"]["can_continue"] is False
assert inspected["next_actions"]["recommended_next_tool"] is None
```
- [ ] **Step 2: Add failed-run assertions**
In `test_workflow_surface_failed_deployment_exposes_error_on_run_and_inspect`, after current assertions, add:
```python
assert payload["next_actions"]["recommended_next_tool"] is None
assert "before producing trace" in payload["next_actions"]["reason"]
assert inspected["next_actions"]["recommended_next_tool"] is None
```
This failing artifact currently has `trace_count == 0`, so the safe guidance is diagnostics/error, not `read_run_trace`.
- [ ] **Step 3: Add trace-detail assertion**
In `test_workflow_surface_run_deployment_can_include_trace_detail`, after the current trace assertions, add:
```python
assert payload["next_actions"]["patch_examples"][0]["request"]["trace_range"][
"limit"
] == 1
```
This confirms completed runs may include bounded trace guidance without making it required.
- [ ] **Step 4: Update `_run_payload`**
In `src/wf_mcp/workflow_surface/handlers.py`, inside `_run_payload(...)`, add `next_actions` to the base `payload` dict:
```python
"next_actions": NextActions.from_run_result(
run_id=run_id,
status=status,
trace_count=trace_count,
diagnostics=diagnostics or [],
).model_dump(mode="json"),
```
The resulting base payload should include:
```python
payload = {
"deployment_id": deployment.id,
"artifact_id": artifact.id,
"artifact_version": artifact.version,
"status": status,
"run_id": run_id,
"resume_readiness": resume_readiness,
"interrupt": interrupt,
"outcome": outcome,
"error": error,
"output": output,
"diagnostics": [
diagnostic.model_dump(mode="json") for diagnostic in diagnostics or []
],
"trace_count": trace_count,
"next_actions": NextActions.from_run_result(
run_id=run_id,
status=status,
trace_count=trace_count,
diagnostics=diagnostics or [],
).model_dump(mode="json"),
}
```
- [ ] **Step 5: Run run tests**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface/test_runs.py tests/wf_mcp/workflow_surface/test_next_actions.py -q
```
Expected: PASS.
---
### Task 5: Update MCP Output Schema Tests
**Files:**
- Modify: `tests/wf_mcp/server/test_config.py`
- Modify: `tests/wf_mcp/server/test_tools.py`
- [ ] **Step 1: Add schema assertions for deployment/run outputs**
In `tests/wf_mcp/server/test_config.py`, find the existing workflow-surface schema test that checks `create_draft_workspace_from_capability` output contains `next_actions`.
Add assertions for `wf.workflow.validate_deployment` and `wf.workflow.run_deployment` output schemas. Use the same local `by_name` / tool lookup style already in that test file:
```python
validate_deployment = by_name["wf.workflow.validate_deployment"]
run_deployment = by_name["wf.workflow.run_deployment"]
validate_output = validate_deployment.outputSchema
run_output = run_deployment.outputSchema
assert "next_actions" in validate_output["properties"]
assert "recommended_next_tool" in validate_output["properties"]["next_actions"][
"properties"
]
assert "next_actions" in run_output["properties"]
assert "recommended_next_tool" in run_output["properties"]["next_actions"][
"properties"
]
```
If this test file uses `tool.outputSchema` through dict access instead of attributes, follow the existing style in the file. Do not assert the whole schema.
- [ ] **Step 2: Add tool description assertion only if output schema exists**
In `tests/wf_mcp/server/test_tools.py`, add a small assertion that `run_deployment.description` or the output schema describes `next_actions` only if that file already inspects output schemas. Do not add fragile full-schema assertions.
If `test_tools.py` only checks input schemas and titles, skip this step.
- [ ] **Step 3: Run server tests**
Run:
```bash
uv run pytest tests/wf_mcp/server/test_config.py tests/wf_mcp/server/test_tools.py -q
```
Expected: PASS.
---
### Task 6: Update Docs
**Files:**
- Modify: `docs/workflow_capabilities.md`
- [ ] **Step 1: Add deployment/run guidance note**
Add this paragraph near the existing `next_actions` explanation:
```markdown
Deployment validation and run lifecycle responses also expose `next_actions`.
For runnable deployments this points to `wf.workflow.run_deployment`; for
unrunnable deployments it points back to validation after the caller repairs
bindings, sources, or schema drift. Failed runs never suggest reading an
unbounded trace; trace guidance always uses a bounded `trace_range`.
```
- [ ] **Step 2: Run docs tests**
Run:
```bash
uv run pytest tests/wf_mcp/server/test_docs.py -q
```
Expected: PASS.
---
### Task 7: Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused test set**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_deployments.py tests/wf_mcp/workflow_surface/test_runs.py tests/wf_mcp/server/test_config.py tests/wf_mcp/server/test_tools.py tests/wf_mcp/server/test_docs.py -q
```
Expected: PASS.
- [ ] **Step 2: Run formatting check on touched files**
Run:
```bash
uv run ruff format --check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_deployments.py tests/wf_mcp/workflow_surface/test_runs.py tests/wf_mcp/server/test_config.py tests/wf_mcp/server/test_tools.py
```
Expected: PASS.
- [ ] **Step 3: Run lint on touched files**
Run:
```bash
uv run ruff check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_deployments.py tests/wf_mcp/workflow_surface/test_runs.py tests/wf_mcp/server/test_config.py tests/wf_mcp/server/test_tools.py
```
Expected: PASS.
- [ ] **Step 4: Run type check**
Run:
```bash
uv run basedpyright --level error
```
Expected: `0 errors`.
- [ ] **Step 5: Optional full suite**
Run:
```bash
uv run pytest -q
```
Expected: full suite remains green with the existing skip/xfail count.
---
## Self-Review Checklist
- `next_actions` is present on `validate_deployment` responses.
- `next_actions` is present on all `_run_payload(...)` responses.
- Completed runs do not require another tool.
- Failed runs recommend bounded trace only when a trace exists.
- Interrupted runs recommend `wf.workflow.resume_run`.
- Diagnostics remain authoritative; guidance only summarizes.
- No unbounded trace read is suggested.
- Existing fields remain unchanged.
## Notes For Opencode
- Keep this as an additive UX change.
- Do not make any runtime/deployment behavior depend on `next_actions`.
- Do not add a new tool.
- Do not broaden trace payloads.
- If server schema tests are awkward, prefer field-level assertions over whole-schema equality.
@@ -0,0 +1,215 @@
# Move RawWorkflowPlan to wf_api.models
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Extract `RawWorkflowPlan` from `wf_mcp.models` to canonical `wf_api.models`, keeping a compatibility shim in `wf_mcp.models`.
**Architecture:** `RawWorkflowPlan` is a standalone Pydantic model with no dependencies on other `wf_mcp.models` definitions. It depends only on `pydantic` and `wf_core` (Edge, InputBinding, Step). This makes it safe to move without entanglement. The `wf_mcp.models` module will re-export from `wf_api.models` as a shim.
**Tech Stack:** Python 3.14, Pydantic v2, pytest, ruff, basedpyright
---
### Task 1: Create `src/wf_api/models.py` with RawWorkflowPlan
**Files:**
- Create: `src/wf_api/models.py`
- [ ] **Step 1: Create the file with the model**
```python
from __future__ import annotations
from typing import Any
from pydantic import BaseModel, Field
from wf_core import Edge
from wf_core.models.steps import InputBinding, Step
class RawWorkflowPlan(BaseModel):
"""Raw authoring plan using the same graph step and edge models as core."""
name: str
input_schema: dict[str, Any]
state_schema: dict[str, Any]
output_schema: dict[str, Any]
outcomes: list[str] = Field(
default_factory=lambda: ["ok"],
description=(
"Declared public workflow outcomes. Legacy plans without this field "
"default to ok."
),
)
output: list[InputBinding] = Field(
default_factory=list,
description=(
"Optional root workflow output bindings. Sources read graph paths "
"such as state.result and targets write the public output payload."
),
)
start: str
nodes: list[Step]
edges: list[Edge]
```
- [ ] **Step 2: Verify the file was created correctly**
Run: `python -c "from wf_api.models import RawWorkflowPlan; print(RawWorkflowPlan.__name__)"`
Expected: `RawWorkflowPlan`
---
### Task 2: Replace `wf_mcp.models.RawWorkflowPlan` definition with shim
**Files:**
- Modify: `src/wf_mcp/models.py`
- [ ] **Step 1: Replace the RawWorkflowPlan class definition with a re-export**
Replace the `RawWorkflowPlan` class block (lines 45-68) with:
```python
# RawWorkflowPlan moved to wf_api.models; re-exported here for backward compat.
from wf_api.models import RawWorkflowPlan # noqa: F401
```
Keep the original import block for pydantic, Edge, InputBinding, Step — they are still used indirectly via the re-export. Actually, after removing the class definition, `BaseModel`, `Field`, `Edge`, `InputBinding`, `Step` are no longer needed by this file. Remove those imports if no other class in the file uses them.
Check: The remaining classes in `wf_mcp/models.py` are `ConnectionConfig`, `AuthRecord`, `CatalogSnapshot`, `BrokerConfig`, `dump_catalog_snapshot`. These use `dataclass`, `field`, `Path`, `Any`, `CatalogNodeEntry`, `CatalogPromptEntry`, `CatalogResourceEntry`. They do NOT use `BaseModel`, `Field`, `Edge`, `InputBinding`, `Step`.
So remove: `from pydantic import BaseModel, Field`, `from wf_core import Edge`, `from wf_core.models.steps import InputBinding, Step`.
- [ ] **Step 2: Run ruff on the file**
Run: `uv run ruff check src/wf_mcp/models.py`
Expected: no errors
- [ ] **Step 3: Run basedpyright on the file**
Run: `uv run basedpyright --level error src/wf_mcp/models.py`
Expected: no errors
---
### Task 3: Update `wf_mcp/__init__.py` to import from shim
**Files:**
- Verify: `src/wf_mcp/__init__.py`
No change needed — `wf_mcp/__init__.py` already imports `RawWorkflowPlan` from `.models`, and the shim re-exports it. Verify this still works.
- [ ] **Step 1: Verify the import chain works**
Run: `python -c "from wf_mcp import RawWorkflowPlan; print(RawWorkflowPlan.__name__)"`
Expected: `RawWorkflowPlan`
---
### Task 4: Add focused tests
**Files:**
- Create: `tests/wf_api/test_raw_workflow_plan_extraction.py`
- [ ] **Step 1: Write the tests**
```python
from __future__ import annotations
def test_canonical_import_from_wf_api_models() -> None:
from wf_api.models import RawWorkflowPlan
assert RawWorkflowPlan.__name__ == "RawWorkflowPlan"
def test_compat_import_from_wf_mcp_models() -> None:
from wf_mcp.models import RawWorkflowPlan as CompatPlan
assert CompatPlan.__name__ == "RawWorkflowPlan"
def test_canonical_and_compat_are_identical() -> None:
from wf_api.models import RawWorkflowPlan as Canonical
from wf_mcp.models import RawWorkflowPlan as Compat
assert Canonical is Compat
```
Note: The import direction rule is already covered by `tests/wf_api/test_import_direction.py::test_wf_api_has_no_wf_mcp_imports`. No need to duplicate.
- [ ] **Step 2: Run the new tests**
Run: `uv run pytest tests/wf_api/test_raw_workflow_plan_extraction.py -v`
Expected: all 3 PASS
---
### Task 5: Update test imports to use canonical path
**Files:**
- Modify: `tests/wf_mcp/service/conftest.py`
- Modify: `tests/wf_mcp/workflow_surface/test_runs.py`
- [ ] **Step 1: Update `tests/wf_mcp/service/conftest.py`**
Change line 8 from:
```python
from wf_mcp.models import AuthRecord, ConnectionConfig, RawWorkflowPlan
```
to:
```python
from wf_api.models import RawWorkflowPlan
from wf_mcp.models import AuthRecord, ConnectionConfig
```
- [ ] **Step 2: Update `tests/wf_mcp/workflow_surface/test_runs.py`**
Change line 32 from:
```python
from wf_mcp.models import RawWorkflowPlan
```
to:
```python
from wf_api.models import RawWorkflowPlan
```
- [ ] **Step 3: Run ruff on touched files**
Run: `uv run ruff check tests/wf_mcp/service/conftest.py tests/wf_mcp/workflow_surface/test_runs.py`
Expected: no errors
- [ ] **Step 4: Run basedpyright on touched files**
Run: `uv run basedpyright --level error tests/wf_mcp/service/conftest.py tests/wf_mcp/workflow_surface/test_runs.py`
Expected: no errors
---
### Task 6: Run full test suite and verify
- [ ] **Step 1: Run pytest**
Run: `uv run pytest -q`
Expected: all tests pass
- [ ] **Step 2: Run ruff on all touched files**
Run: `uv run ruff check src/wf_api/models.py src/wf_mcp/models.py src/wf_mcp/__init__.py tests/wf_api/test_raw_workflow_plan_extraction.py tests/wf_mcp/service/conftest.py tests/wf_mcp/workflow_surface/test_runs.py`
Expected: no errors
- [ ] **Step 3: Run basedpyright on touched files**
Run: `uv run basedpyright --level error src/wf_api/models.py src/wf_mcp/models.py`
Expected: no errors
@@ -0,0 +1,585 @@
# wf_api Extraction Roadmap
> **For agentic workers:** This is an architecture roadmap, not a deterministic implementation checklist. Use it to choose and scope future focused implementation plans. Do not execute multiple slices at once.
**Goal:** Extract a protocol-neutral workflow application API from `wf_mcp` while preserving current process-local behavior and avoiding a large semantic rewrite.
> Current update: the original `WorkflowApiBackend` seam was useful for proving
> dependency direction, but has been collapsed. `WorkflowApi` now composes
> domain services directly from `WorkflowOperationContext`; MCP owns only
> context construction and tool schemas.
**Architecture:** `wf_api` becomes the long-lived in-process application service layer. `wf_mcp`, `wf_cli`, and future HTTP/UI adapters call `wf_api`; `wf_api` must not import `wf_mcp`.
**Current State:** Slice 1 originally introduced `wf_api.WorkflowApi`, a
`WorkflowApiBackend` protocol, and an MCP adapter backend. Later slices removed
that double-delegation seam: `WorkflowApi` now composes domain services directly
from `WorkflowOperationContext`. Both CLI and MCP workflow tools call
`WorkflowApi`; `wf_api` imports no `wf_mcp` modules. `WorkflowSurfaceHandlers`
is now a thin MCP compatibility subclass rather than the operation
implementation.
Slice 3 moved the protocol-neutral workflow helpers into `wf_api`: constants,
capability refs, wrapper hints, next actions, raw workflow plan model, runtime
dependency resolution, saved subgraph preparation, and durable run lifecycle
helpers. The old `wf_mcp.workflow_surface.*` module paths remain compatibility
shims for those helpers.
**Current Constraint:** `WfMcpService` still acts as a compatibility facade over
focused broker services. Recent slices have extracted source/catalog, runtime,
upstream transport, events, connection sync, and content access; the remaining
work is to keep shrinking facade responsibilities while preserving process-local
behavior.
---
## Target Shape
```text
wf_core = workflow execution kernel
wf_authoring = Python authoring sugar and NodeSpec construction
wf_artifacts = saved workflow/deployment/run models and stores
wf_platform = source/capability/event platform primitives
wf_api = process-local workflow application service/use cases
wf_mcp = MCP adapter and MCP runtime/proxy/admin surface
wf_cli = CLI adapter
```
Desired dependency direction:
```text
wf_cli ─┐
├──> wf_api ───> wf_artifacts / wf_platform / wf_core / wf_authoring
wf_mcp ─┘
```
Forbidden dependency direction:
```text
wf_api -> wf_mcp
```
Process-local behavior remains the default:
```python
config = load_broker_config(path)
service = build_service_from_config(config)
api = WorkflowApi(WfMcpWorkflowApiBackend(service))
```
No FastAPI, daemon, socket, auth redesign, or network boundary is required for this extraction.
## Slice 1: Dependency Direction Cleanup
### Goal
Make both CLI and MCP call a protocol-neutral `WorkflowApi`, while `WorkflowApi`
does not import `wf_mcp`.
### Allowed
- Create `src/wf_api/`.
- Add `WorkflowApiBackend` protocol.
- Add `WorkflowApi`.
- Add `WfMcpWorkflowApiBackend` adapter around `WfMcpService`.
- Update `wf_cli.context` to construct `WorkflowApi`.
- Update MCP workflow tool registration to use `WorkflowApi`.
- Keep compatibility aliases if existing imports need a transition.
- Keep payloads and behavior unchanged.
### Not Allowed
- Do not split `WorkflowApi` by domain yet.
- Do not move every helper module yet.
- Do not rename events yet.
- Do not redesign stores.
- Do not add FastAPI.
- Do not change response payloads.
- Do not change command/tool names.
### Implemented Shape
```text
src/wf_api/
__init__.py
backend.py # WorkflowApiBackend high-level operation protocol
service.py # WorkflowApi thin delegating facade
src/wf_mcp/workflow_surface/
handlers.py # existing implementation; now backend plumbing
tools.py # MCP adapter; calls WorkflowApi
src/wf_mcp/broker/service/
workflow_api_backend.py # WfMcpWorkflowApiBackend
```
`WorkflowApiBackend` currently exposes high-level workflow operations that mirror
the old workflow surface (`list_capabilities`, `create_draft_workspace`, `run_deployment`,
and so on). That is intentionally not the final clean domain API. It keeps
behavior and payloads stable while introducing the dependency seam. Later slices
can replace selected `dict[str, Any]` method boundaries with stronger domain
models after callers are routed through `WorkflowApi`.
Live source validation remains behind the MCP backend adapter because it touches
MCP connections, adapters, and auth. `wf_api` owns the operation name, but the
current backend owns the live-check implementation.
### Success Criteria
- `wf_api` imports no `wf_mcp` modules. **Done.**
- `wf_cli` uses `WorkflowApi`. **Done.**
- `wf_mcp.workflow_surface.tools` uses `WorkflowApi`. **Done.**
- Existing MCP workflow-surface tests pass. **Done at implementation time.**
- Existing CLI tests pass. **Done at implementation time.**
- Behavior and payloads are unchanged. **Intended and guarded by tests.**
## Slice 2: Stabilize API Names And Compatibility Shims
### Goal
Make naming honest without breaking callers.
### Docs-First Current Slice
Before renames, document the new ownership:
- `wf_api.WorkflowApi` is the application-facing process-local API.
- `wf_api.WorkflowApiBackend` is the high-level backend protocol.
- `wf_mcp.broker.service.WfMcpWorkflowApiBackend` adapts the current MCP service
stack into the backend protocol.
- `wf_mcp.workflow_surface.WorkflowSurfaceHandlers` is legacy/internal
implementation plumbing. New adapter code should not treat it as the canonical
API.
### Later Likely Work
- Rename `WorkflowSurfaceHandlers` usage to `WorkflowApi` in tests and CLI code.
- If a class rename is chosen, keep a temporary import shim:
```python
from wf_api import WorkflowApi as WorkflowSurfaceHandlers
```
- Update docs to say:
```text
MCP tools and CLI commands are adapters over wf_api.
```
- Rename test fixture helpers from `handlers(...)` to `api(...)` if useful.
### If/Then
- If shims create confusion, remove them quickly after imports are migrated.
- If too many downstream imports still expect `WorkflowSurfaceHandlers`, keep the shim for one release/work session and document it as deprecated.
### Success Criteria
- New code imports `WorkflowApi`.
- Old name remains only in compatibility modules or is gone.
- Docs describe `wf_api` as the application service layer.
## Slice 3: Move Protocol-Neutral Workflow Surface Modules
### Goal
Move helper modules that are not MCP-specific out of `wf_mcp.workflow_surface`.
### Completed Moves
```text
wf_mcp.workflow_surface.constants -> wf_api.constants
wf_mcp.workflow_surface.refs -> wf_api.refs
wf_mcp.workflow_surface.wrapper_hints -> wf_api.wrapper_hints
wf_mcp.workflow_surface.next_actions -> wf_api.next_actions
wf_mcp.models.RawWorkflowPlan -> wf_api.models.RawWorkflowPlan
wf_mcp.workflow_surface.runtime_dependencies -> wf_api.runtime_dependencies
wf_mcp.workflow_surface.saved_subgraphs -> wf_api.saved_subgraphs
wf_mcp.workflow_surface.run_lifecycle -> wf_api.run_lifecycle
```
The full `wf_mcp.workflow_surface.models` module did not move. It still holds
MCP tool request/response schemas such as `TraceRange` and workflow tool result
models. Move or split those only when the MCP schema boundary is clearer.
### If/Then
- If a module imports MCP connection/adapters/auth, do not move it in this slice.
- If imports become circular, leave a shim in the old location and move one module at a time.
- If a helper is really platform vocabulary, consider `wf_platform` instead of `wf_api`.
### Success Criteria
- `wf_api` owns protocol-neutral workflow API helpers. **Done.**
- `wf_mcp.workflow_surface` keeps MCP adapter/schema code plus compatibility
shims. **Mostly done.**
- Tests still pass with import-only or near-import-only changes. **Done at
implementation time.**
## Slice 4: Split The Big API By Domain
### Goal
Reduce the large `WorkflowApi` class after the package boundary is correct.
### Slice 4A: Operation Context Scaffolding
Do not move handler methods first. `WorkflowSurfaceHandlers` methods currently
reach through `self.service` for stores, capability sources, events, live source
calls, adapter lookup, and catalog helpers. Moving method bodies before defining
that dependency surface would either make `wf_api` import `wf_mcp` or produce a
fake split where every domain service still depends on the whole MCP service.
First introduce a small protocol-neutral operation context in `wf_api`:
```text
src/wf_api/
operation_context.py # protocols/dataclass for stores, sources, events, live calls
```
The exact names may change, but the context should answer these questions:
- How does workflow API code access artifact, draft workspace, deployment, and run stores?
- How does it read planner-visible `CapabilitySource` objects?
- How does it record artifact/deployment/run lifecycle events without importing MCP event types?
- How does it perform live source validation or live capability calls without importing MCP adapters/auth?
- Which existing `WfMcpService` helpers are still required by moved domain methods?
`WfMcpWorkflowApiBackend` or another MCP-owned adapter can build this context
from `WfMcpService`. The context is scaffolding only: Slice 4A should not move
capability/draft/artifact/deployment/run method bodies yet and should not change
public payloads.
### Candidate Shape
```text
src/wf_api/
service.py # facade that composes domain services
operation_context.py
capabilities.py
drafts.py
artifacts.py
deployments.py
runs.py
```
Possible facade:
```python
class WorkflowApi:
capabilities: CapabilityApi
drafts: DraftApi
artifacts: ArtifactApi
deployments: DeploymentApi
runs: RunApi
```
Compatibility can keep flat methods:
```python
async def list_capabilities(...):
return await self.capabilities.list_capabilities(...)
```
### Planned Domain Split Order
After Slice 4A proves the operation-context seam, split method groups in small
behavior-preserving slices:
#### Slice 4B: Drafts First
Move stateless draft and draft workspace operations first:
```text
validate_draft
compile_draft
patch_draft
list_draft_workspaces
create_draft_workspace
get_draft_workspace
delete_draft_workspace
validate_draft_workspace
patch_draft_workspace
set_draft_name
set_draft_route
set_step_input_map
set_step_output_map
create_minimal_draft_workspace
```
Reason: drafts mostly use the draft workspace store, workflow draft compiler,
wrapper hints, and deterministic patch helpers. They have the lowest live-source
and durable-runtime coupling.
Leave `create_draft_workspace_from_capability` in the MCP-backed handler during
4B. It depends on `inspect_capability`, wrapper hints, and capability source
inspection, so it should move with either a small follow-up capability bootstrap
slice or Slice 4E.
#### Slice 4C: Artifacts And Deployments
Move saved artifact and deployment operations next:
```text
list_artifacts
save_artifact
create_artifact_from_plan
create_artifact_from_draft
create_artifact_from_workspace
create_wrapper_from_workspace
inspect_artifact
list_deployments
inspect_deployment
save_deployment
delete_deployment
validate_deployment
```
Reason: this group is store-heavy and introduces dependency validation, saved
subgraph tree resolution, and event recording. It should move only after drafts
prove the context seam.
#### Slice 4D: Runs
Move run lifecycle operations after artifacts/deployments:
```text
run_deployment
resume_run
inspect_run
read_run_trace
```
Reason: runs are runtime-sensitive. They touch durable checkpoints, pinned
dependency environments, resume readiness, prepared saved subgraphs, trace
slicing, and compact next-action guidance. This should not be the first method
move.
#### Slice 4E: Capabilities Last
Move workflow capability operations last:
```text
list_capabilities
inspect_capability
call_capability
create_draft_workspace_from_capability
```
Reason: capabilities look simple but are the messiest boundary. They combine
planner-visible source inventory, wrapper artifacts, direct wrapper calls,
external live source calls, source visibility, and schema/wrapper hints.
`create_draft_workspace_from_capability` also belongs here because it is driven
by `inspect_capability` wrapper hints. Keep them in the MCP-backed
implementation until the other domain services are stable.
#### After Slice 4E: Helper Promotion Cleanup
Once the handler is mostly a compatibility adapter, promote duplicated helper
symbols into stable homes instead of leaving long-term cross-domain private
imports:
```text
raw_plan_from_artifact -> wf_api.artifact_plans or wf_artifacts
artifact_capability_id -> wf_api artifact/capability refs helper
available_sources_from_capability_sources -> wf_api source snapshot helper
```
This should be a cleanup slice, not part of 4E unless required to avoid circular
imports or behavior drift.
### If/Then
- If the context starts mirroring all of `WfMcpService`, stop and split it into
smaller protocols rather than creating a new god object.
- If callers benefit from flat methods, keep the facade flat and split internals only.
- If domain APIs are clean enough, expose nested services later.
- If a method spans domains, keep it in the facade until a better boundary appears.
- If live source calls cannot be abstracted cleanly yet, leave capability
calling in the MCP backend and move drafts/artifacts first.
### Success Criteria
- `wf_api` has an explicit operation context/protocol seam that imports no
`wf_mcp` modules.
- `WfMcpWorkflowApiBackend` can adapt `WfMcpService` into that seam.
- No workflow method behavior changes in Slice 4A.
- Each domain file is readable on its own.
- Public payloads remain unchanged.
- `wf_cli` and `wf_mcp` do not care about the internal split.
## Slice 5: Move Listing/Event Primitives To Better Homes
### Goal
Remove remaining protocol-neutral utilities from MCP-named packages.
### Current Recommendation
Split this into two different concerns. Listing/helper consolidation is
behavior-preserving cleanup and should happen first. Event migration changes
domain vocabulary and should remain separate until lifecycle event semantics are
clearer.
### Slice 5A/5B: Listing And Workflow Helper Consolidation
Concrete plan:
```text
docs/superpowers/plans/2026-06-02-wf-api-slice-5a-5b-helper-consolidation.md
```
Planned moves:
```text
wf_api.capabilities._matches_query -> wf_api.listing.matches_query
wf_api.artifacts._matches_query -> wf_api.listing.matches_query
wf_api.capabilities._paged_list_payload -> wf_api.listing.paged_list_payload
wf_api.artifacts._paged_list_payload -> wf_api.listing.paged_list_payload
wf_mcp.workflow_surface.handlers fallback -> wf_api.listing.paged_list_payload
wf_api.runs._raw_plan_from_artifact -> wf_api.artifact_plans.raw_plan_from_artifact
wf_api.capabilities._raw_plan_from_artifact -> wf_api.artifact_plans.raw_plan_from_artifact
wf_api.capabilities._artifact_capability_id -> wf_api.artifact_refs.artifact_capability_id
wf_api.artifacts._artifact_capability_id -> wf_api.artifact_refs.artifact_capability_id
wf_api.{drafts,artifacts,capabilities} requirement helpers
-> wf_api.capability_requirements
```
Do not move `wf_mcp.shared.pagination` in this slice. It is still used by proxy
tool search/listing code, so treating it as dead workflow-surface debt would be
incorrect.
### Post-5 Helper Cleanup: Workflow Surface Test Thinning
Concrete plan:
```text
docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning.md
```
Intent:
```text
wf_api tests = canonical application behavior tests
wf_mcp.workflow_surface tests = adapter/schema/live-source/integration smoke tests
```
Do not replace stronger workflow-surface integration tests with weaker unit
tests. Only remove a handler test when an equal-or-stronger `wf_api` test exists
and at least one handler-level smoke/delegation test still protects the adapter
path.
### Candidate Moves
```text
wf_mcp.shared.listing.matches_query -> wf_platform.listing or wf_api.listing
wf_mcp.shared.listing.paged_list_payload -> wf_platform.listing or wf_api.listing
wf_mcp.events.McpEvent -> wf_platform.events.DomainEvent
wf_mcp.events.EventBus -> wf_platform.events.EventBus
```
### If/Then
- If events are only used by MCP admin/proxy code, leave them in `wf_mcp`.
- If events describe artifact/deployment/run lifecycle, move or fork them into `wf_platform`.
- If renaming `McpEvent` causes churn, introduce `DomainEvent` first and keep `McpEvent` as an alias temporarily.
### Success Criteria
- Workflow API lifecycle events no longer require MCP naming.
- Listing helpers used by CLI/API are not imported from `wf_mcp`.
## Slice 6: Store Construction And Config Boundary
### Goal
Make store/runtime construction less MCP-owned.
### Current Problem
`build_service_from_config` and `WfMcpService.__post_init__` currently build or own protocol-neutral stores. CLI uses this path because it is pragmatic, but long term config/store construction should be reusable without MCP server assumptions.
### Candidate Work
- Extract config-to-store construction into a neutral builder.
- Keep MCP connection config in `wf_mcp`.
- Let `wf_cli` and future API/server adapters reuse the neutral store builder.
### If/Then
- If extraction creates too many config model moves, defer it.
- If CLI needs only current config behavior, keep using `wf_mcp` builder until FastAPI/UI pressure appears.
### Success Criteria
- Artifact/draft/run stores can be constructed without starting MCP concepts.
- Current `wf_mcp.config.json` remains supported.
## Slice 7: Future HTTP/FastAPI Adapter
### Goal
Expose `WorkflowApi` over HTTP only after the in-process API is stable.
### Not Yet
Do not start this until:
- `wf_api` exists.
- CLI and MCP both call `wf_api`.
- Run/draft/deployment payloads are stable enough.
- Auth and multi-client lifecycle questions are explicit.
### Future Shape
```text
wf_http or wf_server
routes/
capabilities.py
drafts.py
artifacts.py
deployments.py
runs.py
```
Routes should be thin:
```python
@router.post("/runs")
async def start_run(...):
return await api.run_deployment(...)
```
### Success Criteria
- HTTP is an adapter, not a new source of workflow logic.
- Process-local API remains usable without HTTP.
## Open Questions
1. Should `WorkflowApiBackend` be one protocol or several domain protocols?
- Recommendation for Slice 1: one protocol. Split later only if it hurts.
2. Should live source validation live in `wf_api`?
- Recommendation: `wf_api` owns the operation, backend owns the live-check implementation.
3. Should `wf_api` expose flat methods or nested services?
- Recommendation for Slice 1: flat methods for compatibility. Consider nested internals later.
4. Should `WorkflowSurfaceHandlers` disappear immediately?
- Recommendation: no. Keep a short-lived shim if it reduces churn.
5. Should `wf_cli` stop importing `wf_mcp` after Slice 1?
- Not fully. It may still use `wf_mcp` config/service construction until store/config extraction happens.
## Immediate Next Plan Status
Slice 1 implementation plan exists and was executed:
```text
docs/superpowers/plans/2026-06-01-wf-api-slice-1-dependency-direction.md
```
The next implementation plan should cover Slice 2 only. Prefer docs and
compatibility naming first; do not move helper modules until Slice 3.
@@ -0,0 +1,503 @@
# wf_api Slice 3A: Refs And Constants Move Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move the protocol-neutral workflow API refs and constants helpers from `wf_mcp.workflow_surface` into `wf_api`, while preserving old imports as compatibility shims.
**Architecture:** `wf_api` is now the canonical home for process-local workflow API helpers that are not MCP-specific. This slice moves only `constants.py` and `refs.py` because both are small and import only `wf_artifacts` / `wf_platform`. `wf_mcp.workflow_surface.constants` and `wf_mcp.workflow_surface.refs` remain thin re-export shims so existing imports keep working.
**Tech Stack:** Python 3.14+, Pydantic-backed refs from `wf_artifacts` and `wf_platform`, pytest, ruff, basedpyright.
---
## Scope
### In Scope
- Create `src/wf_api/constants.py`.
- Create `src/wf_api/refs.py`.
- Re-export the new helpers from `src/wf_api/__init__.py`.
- Replace `wf_mcp.workflow_surface.constants` with a compatibility shim.
- Replace `wf_mcp.workflow_surface.refs` with a compatibility shim.
- Update `src/wf_mcp/workflow_surface/handlers.py` to import canonical helpers from `wf_api`.
- Add/adjust tests for canonical imports and shim compatibility.
- Keep `wf_api` free of `wf_mcp` imports.
### Out Of Scope
- Do not move `models.py`.
- Do not move `next_actions.py`.
- Do not move `wrapper_hints.py`.
- Do not move `run_lifecycle.py`.
- Do not move `runtime_dependencies.py`.
- Do not move `saved_subgraphs.py`.
- Do not rename `WorkflowSurfaceHandlers`.
- Do not change public payloads, tool names, command names, or parsing behavior.
---
## File Structure
### New Canonical Files
| File | Responsibility |
| --- | --- |
| `src/wf_api/constants.py` | Canonical workflow API literals used by draft/helper code. |
| `src/wf_api/refs.py` | Canonical parser for workflow-surface capability IDs. |
### Compatibility Shims
| File | Responsibility |
| --- | --- |
| `src/wf_mcp/workflow_surface/constants.py` | Re-export constants from `wf_api.constants`; no local logic. |
| `src/wf_mcp/workflow_surface/refs.py` | Re-export refs from `wf_api.refs`; no local logic. |
### Modified Consumers
| File | Change |
| --- | --- |
| `src/wf_api/__init__.py` | Re-export moved helpers. |
| `src/wf_mcp/workflow_surface/handlers.py` | Import constants and parser from `wf_api`. |
| `tests/wf_mcp/test_workflow_surface_refs.py` | Keep shim compatibility tests and add canonical import tests. |
| `tests/wf_api/test_import_direction.py` | Existing guard should continue to pass. |
---
## Task 1: Add Canonical `wf_api.constants`
**Files:**
- Create: `src/wf_api/constants.py`
- [ ] **Step 1: Create `src/wf_api/constants.py`**
Write this exact file:
```python
"""Protocol-neutral workflow API literals used by draft/helper code."""
DEFAULT_CALL_STEP_ID = "call"
DEFAULT_ERROR_STEP_ID = "tool_error"
DEFAULT_OK_OUTCOME = "ok"
DEFAULT_ERROR_OUTCOME = "error"
RUNTIME_ERROR_CAPABILITY = "wf.std.runtime_error"
__all__ = [
"DEFAULT_CALL_STEP_ID",
"DEFAULT_ERROR_OUTCOME",
"DEFAULT_ERROR_STEP_ID",
"DEFAULT_OK_OUTCOME",
"RUNTIME_ERROR_CAPABILITY",
]
```
- [ ] **Step 2: Run import smoke check**
Run:
```powershell
uv run python -c "from wf_api.constants import DEFAULT_CALL_STEP_ID, RUNTIME_ERROR_CAPABILITY; print(DEFAULT_CALL_STEP_ID, RUNTIME_ERROR_CAPABILITY)"
```
Expected output:
```text
call wf.std.runtime_error
```
---
## Task 2: Add Canonical `wf_api.refs`
**Files:**
- Create: `src/wf_api/refs.py`
- [ ] **Step 1: Create `src/wf_api/refs.py`**
Write this exact file:
```python
from __future__ import annotations
from typing import Any, TypeAlias
from wf_artifacts import WorkflowCapabilityRef
from wf_platform import CapabilityRef
WorkflowSurfaceCapabilityId: TypeAlias = CapabilityRef | WorkflowCapabilityRef
def parse_workflow_surface_capability_id(
value: str | dict[str, Any],
) -> WorkflowSurfaceCapabilityId:
"""Parse a workflow API capability id into its real domain ref.
API callers still pass strings at protocol boundaries. Internally,
workflow-facing capability ids are either live source capabilities or saved
wrapper artifacts, so this parser avoids inventing a third identifier model.
"""
if isinstance(value, dict):
if "artifact_id" in value and "version" in value:
return WorkflowCapabilityRef._validate(value)
return CapabilityRef._validate(value)
try:
return WorkflowCapabilityRef.parse(value)
except ValueError:
return CapabilityRef.parse(value)
```
- [ ] **Step 2: Run import smoke check**
Run:
```powershell
uv run python -c "from wf_api.refs import parse_workflow_surface_capability_id; print(parse_workflow_surface_capability_id('workflow.echo_wrapper.v2'))"
```
Expected output:
```text
workflow.echo_wrapper.v2
```
---
## Task 3: Re-export Helpers From `wf_api`
**Files:**
- Modify: `src/wf_api/__init__.py`
- [ ] **Step 1: Update imports and `__all__`**
Change `src/wf_api/__init__.py` to include these imports:
```python
from .constants import (
DEFAULT_CALL_STEP_ID,
DEFAULT_ERROR_OUTCOME,
DEFAULT_ERROR_STEP_ID,
DEFAULT_OK_OUTCOME,
RUNTIME_ERROR_CAPABILITY,
)
from .refs import WorkflowSurfaceCapabilityId, parse_workflow_surface_capability_id
```
Ensure `__all__` includes:
```python
__all__ = [
"DEFAULT_CALL_STEP_ID",
"DEFAULT_ERROR_OUTCOME",
"DEFAULT_ERROR_STEP_ID",
"DEFAULT_OK_OUTCOME",
"RUNTIME_ERROR_CAPABILITY",
"TraceRange",
"WorkflowApi",
"WorkflowApiBackend",
"WorkflowSurfaceCapabilityId",
"parse_workflow_surface_capability_id",
]
```
- [ ] **Step 2: Run import smoke check**
Run:
```powershell
uv run python -c "from wf_api import DEFAULT_OK_OUTCOME, parse_workflow_surface_capability_id; print(DEFAULT_OK_OUTCOME, parse_workflow_surface_capability_id('demo.personal.echo_tool'))"
```
Expected output:
```text
ok demo.personal.echo_tool
```
---
## Task 4: Convert Old Workflow-Surface Modules To Shims
**Files:**
- Modify: `src/wf_mcp/workflow_surface/constants.py`
- Modify: `src/wf_mcp/workflow_surface/refs.py`
- [ ] **Step 1: Replace `src/wf_mcp/workflow_surface/constants.py`**
Replace the file with this shim:
```python
"""Compatibility shim for workflow API constants.
New code should import these literals from `wf_api.constants`. This module stays
so older MCP workflow-surface imports keep working during extraction.
"""
from wf_api.constants import (
DEFAULT_CALL_STEP_ID,
DEFAULT_ERROR_OUTCOME,
DEFAULT_ERROR_STEP_ID,
DEFAULT_OK_OUTCOME,
RUNTIME_ERROR_CAPABILITY,
)
__all__ = [
"DEFAULT_CALL_STEP_ID",
"DEFAULT_ERROR_OUTCOME",
"DEFAULT_ERROR_STEP_ID",
"DEFAULT_OK_OUTCOME",
"RUNTIME_ERROR_CAPABILITY",
]
```
- [ ] **Step 2: Replace `src/wf_mcp/workflow_surface/refs.py`**
Replace the file with this shim:
```python
"""Compatibility shim for workflow API capability refs.
New code should import from `wf_api.refs`. This module stays so older MCP
workflow-surface imports keep working during extraction.
"""
from wf_api.refs import (
WorkflowSurfaceCapabilityId,
parse_workflow_surface_capability_id,
)
__all__ = [
"WorkflowSurfaceCapabilityId",
"parse_workflow_surface_capability_id",
]
```
- [ ] **Step 3: Run shim import smoke check**
Run:
```powershell
uv run python -c "from wf_mcp.workflow_surface.constants import DEFAULT_CALL_STEP_ID; from wf_mcp.workflow_surface.refs import parse_workflow_surface_capability_id; print(DEFAULT_CALL_STEP_ID, parse_workflow_surface_capability_id('workflow.echo_wrapper.v2'))"
```
Expected output:
```text
call workflow.echo_wrapper.v2
```
---
## Task 5: Update Canonical Imports In `WorkflowSurfaceHandlers`
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- [ ] **Step 1: Change constants import**
Replace:
```python
from .constants import (
DEFAULT_CALL_STEP_ID,
DEFAULT_ERROR_OUTCOME,
DEFAULT_ERROR_STEP_ID,
DEFAULT_OK_OUTCOME,
RUNTIME_ERROR_CAPABILITY,
)
```
with:
```python
from wf_api.constants import (
DEFAULT_CALL_STEP_ID,
DEFAULT_ERROR_OUTCOME,
DEFAULT_ERROR_STEP_ID,
DEFAULT_OK_OUTCOME,
RUNTIME_ERROR_CAPABILITY,
)
```
- [ ] **Step 2: Change refs import**
Replace:
```python
from .refs import parse_workflow_surface_capability_id
```
with:
```python
from wf_api.refs import parse_workflow_surface_capability_id
```
- [ ] **Step 3: Run import smoke check**
Run:
```powershell
uv run python -c "from wf_mcp.workflow_surface.handlers import WorkflowSurfaceHandlers; print(WorkflowSurfaceHandlers.__name__)"
```
Expected output:
```text
WorkflowSurfaceHandlers
```
---
## Task 6: Update Ref Tests For Canonical And Shim Imports
**Files:**
- Modify: `tests/wf_mcp/test_workflow_surface_refs.py`
- [ ] **Step 1: Update imports**
Change the top imports to:
```python
from wf_api.refs import parse_workflow_surface_capability_id
from wf_artifacts import WorkflowCapabilityRef
from wf_mcp.workflow_surface.refs import (
parse_workflow_surface_capability_id as parse_workflow_surface_capability_id_shim,
)
from wf_platform import CapabilityRef
```
- [ ] **Step 2: Add shim compatibility test**
Append this test to the file:
```python
def test_workflow_surface_refs_shim_reexports_canonical_parser() -> None:
assert parse_workflow_surface_capability_id_shim is parse_workflow_surface_capability_id
```
- [ ] **Step 3: Add constants shim compatibility test**
Append this test to the file:
```python
def test_workflow_surface_constants_shim_reexports_canonical_literals() -> None:
from wf_api.constants import DEFAULT_CALL_STEP_ID
from wf_mcp.workflow_surface.constants import (
DEFAULT_CALL_STEP_ID as DEFAULT_CALL_STEP_ID_SHIM,
)
assert DEFAULT_CALL_STEP_ID_SHIM == DEFAULT_CALL_STEP_ID
```
- [ ] **Step 4: Run focused tests**
Run:
```powershell
uv run pytest tests/wf_mcp/test_workflow_surface_refs.py tests/wf_api/test_import_direction.py -q
```
Expected: all tests pass.
---
## Task 7: Search For Remaining Canonical Import Opportunities
**Files:**
- Inspect only unless the search finds new low-risk direct consumers.
- [ ] **Step 1: Search old imports**
Run:
```powershell
rg -n "from \\.constants|from \\.refs|from wf_mcp\\.workflow_surface\\.(constants|refs)" src tests
```
Expected remaining matches:
```text
src/wf_mcp/workflow_surface/constants.py
src/wf_mcp/workflow_surface/refs.py
tests/wf_mcp/test_workflow_surface_refs.py
```
If any other production module imports the old paths, update it to import from
`wf_api.constants` or `wf_api.refs`.
- [ ] **Step 2: Search for accidental `wf_api -> wf_mcp` imports**
Run:
```powershell
rg -n "wf_mcp" src/wf_api tests/wf_api
```
Expected matches only in test text/docstrings for the import-direction guard,
not in `src/wf_api/*.py`.
---
## Task 8: Verification
- [ ] **Step 1: Run focused tests**
```powershell
uv run pytest tests/wf_api tests/wf_mcp/test_workflow_surface_refs.py tests/wf_mcp/workflow_surface -q
```
Expected: all pass.
- [ ] **Step 2: Run CLI context smoke tests**
```powershell
uv run pytest tests/wf_cli/test_context.py -q
```
Expected: pass.
- [ ] **Step 3: Run ruff on touched files**
```powershell
uv run ruff check src/wf_api src/wf_mcp/workflow_surface/constants.py src/wf_mcp/workflow_surface/refs.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/test_workflow_surface_refs.py tests/wf_api
```
Expected: all checks pass.
- [ ] **Step 4: Run basedpyright on touched files**
```powershell
uv run basedpyright --level error src/wf_api src/wf_mcp/workflow_surface/constants.py src/wf_mcp/workflow_surface/refs.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/test_workflow_surface_refs.py tests/wf_api
```
Expected: `0 errors`.
- [ ] **Step 5: Optional full suite**
Run this if time allows:
```powershell
uv run pytest -q
```
Expected: full suite passes with the projects existing skipped/xfailed counts.
---
## Self-Review Checklist
- `wf_api.constants` imports no `wf_mcp`.
- `wf_api.refs` imports no `wf_mcp`.
- Old `wf_mcp.workflow_surface.constants` import path still works.
- Old `wf_mcp.workflow_surface.refs` import path still works.
- `WorkflowSurfaceHandlers` imports the canonical `wf_api` helpers.
- No public payload shape changed.
- No behavior changed.
- No other workflow-surface helper moved in this slice.
@@ -0,0 +1,548 @@
# wf_api Slice 3B: Guidance Helpers Move Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move protocol-neutral wrapper guidance helpers from `wf_mcp.workflow_surface` into `wf_api`, while preserving old imports as compatibility shims.
**Architecture:** `wrapper_hints.py` and `next_actions.py` are coupled guidance helpers: `next_actions` imports `WrapperAuthoringHints`, and both describe workflow authoring UX rather than MCP transport behavior. This slice moves them together to avoid a half-moved dependency. The old `wf_mcp.workflow_surface` modules remain thin re-export shims so MCP schemas/tests and older imports keep working.
**Tech Stack:** Python 3.14+, Pydantic v2 models, pytest, ruff, basedpyright.
---
## Scope
### In Scope
- Create `src/wf_api/wrapper_hints.py`.
- Create `src/wf_api/next_actions.py`.
- Re-export selected public guidance types/functions from `src/wf_api/__init__.py`.
- Replace `wf_mcp.workflow_surface.wrapper_hints` with a compatibility shim.
- Replace `wf_mcp.workflow_surface.next_actions` with a compatibility shim.
- Update `src/wf_mcp/workflow_surface/handlers.py` to import canonical helpers from `wf_api`.
- Update `src/wf_mcp/workflow_surface/models.py` to import canonical next-action models from `wf_api`.
- Update direct tests to use canonical imports while preserving shim compatibility tests.
- Keep `wf_api` free of `wf_mcp` imports.
### Out Of Scope
- Do not move `models.py`.
- Do not move `run_lifecycle.py`.
- Do not move `runtime_dependencies.py`.
- Do not move `saved_subgraphs.py`.
- Do not rename `WorkflowSurfaceHandlers`.
- Do not change wrapper hint behavior.
- Do not change next action behavior.
- Do not change public payloads, MCP tool names, CLI command names, or JSON schema field names.
---
## File Structure
### New Canonical Files
| File | Responsibility |
| --- | --- |
| `src/wf_api/wrapper_hints.py` | Wrapper scaffolding hints, confidence, missing-decision models, and conservative schema mapping helper. |
| `src/wf_api/next_actions.py` | Advisory next-action models and factory helpers for wrapper/deployment/run responses. |
### Compatibility Shims
| File | Responsibility |
| --- | --- |
| `src/wf_mcp/workflow_surface/wrapper_hints.py` | Re-export wrapper hint helpers from `wf_api.wrapper_hints`. |
| `src/wf_mcp/workflow_surface/next_actions.py` | Re-export next action helpers from `wf_api.next_actions`. |
### Modified Consumers
| File | Change |
| --- | --- |
| `src/wf_api/__init__.py` | Re-export public guidance helpers. |
| `src/wf_mcp/workflow_surface/handlers.py` | Import `NextActions` and wrapper hint helpers from `wf_api`. |
| `src/wf_mcp/workflow_surface/models.py` | Import `NextActionPatchExample` and `NextActions` from `wf_api.next_actions`. |
| `tests/wf_mcp/test_workflow_wrapper_hints.py` | Use canonical `wf_api.wrapper_hints`; add shim identity test. |
| `tests/wf_mcp/workflow_surface/test_next_actions.py` | Use canonical `wf_api.next_actions`; add shim identity test. |
---
## Task 1: Create Canonical `wf_api.wrapper_hints`
**Files:**
- Create: `src/wf_api/wrapper_hints.py`
- [ ] **Step 1: Copy existing implementation**
Create `src/wf_api/wrapper_hints.py` by copying the complete current contents of:
```text
src/wf_mcp/workflow_surface/wrapper_hints.py
```
Do not change behavior. The copied module must not import `wf_mcp`.
- [ ] **Step 2: Add `__all__` at the end**
Append this block to the copied file:
```python
__all__ = [
"MissingDecision",
"MissingDecisionKind",
"OutcomeCandidate",
"OutcomeCandidateKind",
"WrapperAuthoringHints",
"WrapperHintConfidence",
"WrapperOutcomePolicy",
"workflow_output_schema_for_authoring",
"wrapper_hints_for_capability",
]
```
- [ ] **Step 3: Run import smoke check**
Run:
```powershell
uv run python -c "from wf_api.wrapper_hints import WrapperAuthoringHints, wrapper_hints_for_capability; print(WrapperAuthoringHints.__name__, wrapper_hints_for_capability.__name__)"
```
Expected output:
```text
WrapperAuthoringHints wrapper_hints_for_capability
```
---
## Task 2: Create Canonical `wf_api.next_actions`
**Files:**
- Create: `src/wf_api/next_actions.py`
- [ ] **Step 1: Copy existing implementation**
Create `src/wf_api/next_actions.py` by copying the complete current contents of:
```text
src/wf_mcp/workflow_surface/next_actions.py
```
- [ ] **Step 2: Keep local wrapper hint import canonical**
Ensure the copied file imports wrapper hints from the new `wf_api` package via:
```python
from .wrapper_hints import WrapperAuthoringHints
```
The copied module must not import `wf_mcp`.
- [ ] **Step 3: Add `__all__` at the end**
Append this block to the copied file:
```python
__all__ = [
"NextActionPatchExample",
"NextActionTool",
"NextActions",
]
```
- [ ] **Step 4: Run import smoke check**
Run:
```powershell
uv run python -c "from wf_api.next_actions import NextActionTool, NextActions; print(NextActionTool.RUN_DEPLOYMENT, NextActions.__name__)"
```
Expected output:
```text
wf.workflow.run_deployment NextActions
```
---
## Task 3: Re-export Guidance Helpers From `wf_api`
**Files:**
- Modify: `src/wf_api/__init__.py`
- [ ] **Step 1: Add imports**
Add these imports:
```python
from .next_actions import NextActionPatchExample, NextActionTool, NextActions
from .wrapper_hints import (
MissingDecision,
MissingDecisionKind,
OutcomeCandidate,
OutcomeCandidateKind,
WrapperAuthoringHints,
WrapperHintConfidence,
WrapperOutcomePolicy,
workflow_output_schema_for_authoring,
wrapper_hints_for_capability,
)
```
- [ ] **Step 2: Add names to `__all__`**
Ensure `__all__` includes these names:
```python
"MissingDecision",
"MissingDecisionKind",
"NextActionPatchExample",
"NextActionTool",
"NextActions",
"OutcomeCandidate",
"OutcomeCandidateKind",
"WrapperAuthoringHints",
"WrapperHintConfidence",
"WrapperOutcomePolicy",
"workflow_output_schema_for_authoring",
"wrapper_hints_for_capability",
```
- [ ] **Step 3: Run top-level import smoke check**
Run:
```powershell
uv run python -c "from wf_api import NextActions, WrapperAuthoringHints; print(NextActions.__name__, WrapperAuthoringHints.__name__)"
```
Expected output:
```text
NextActions WrapperAuthoringHints
```
---
## Task 4: Convert Old Workflow-Surface Guidance Modules To Shims
**Files:**
- Modify: `src/wf_mcp/workflow_surface/wrapper_hints.py`
- Modify: `src/wf_mcp/workflow_surface/next_actions.py`
- [ ] **Step 1: Replace `src/wf_mcp/workflow_surface/wrapper_hints.py`**
Replace the file with this shim:
```python
"""Compatibility shim for workflow API wrapper authoring hints.
New code should import from `wf_api.wrapper_hints`. This module stays so older
MCP workflow-surface imports keep working during extraction.
"""
from wf_api.wrapper_hints import (
MissingDecision,
MissingDecisionKind,
OutcomeCandidate,
OutcomeCandidateKind,
WrapperAuthoringHints,
WrapperHintConfidence,
WrapperOutcomePolicy,
workflow_output_schema_for_authoring,
wrapper_hints_for_capability,
)
__all__ = [
"MissingDecision",
"MissingDecisionKind",
"OutcomeCandidate",
"OutcomeCandidateKind",
"WrapperAuthoringHints",
"WrapperHintConfidence",
"WrapperOutcomePolicy",
"workflow_output_schema_for_authoring",
"wrapper_hints_for_capability",
]
```
- [ ] **Step 2: Replace `src/wf_mcp/workflow_surface/next_actions.py`**
Replace the file with this shim:
```python
"""Compatibility shim for workflow API next-action guidance.
New code should import from `wf_api.next_actions`. This module stays so older
MCP workflow-surface imports keep working during extraction.
"""
from wf_api.next_actions import NextActionPatchExample, NextActionTool, NextActions
__all__ = [
"NextActionPatchExample",
"NextActionTool",
"NextActions",
]
```
- [ ] **Step 3: Run shim import smoke check**
Run:
```powershell
uv run python -c "from wf_mcp.workflow_surface.wrapper_hints import WrapperAuthoringHints; from wf_mcp.workflow_surface.next_actions import NextActions; print(WrapperAuthoringHints.__name__, NextActions.__name__)"
```
Expected output:
```text
WrapperAuthoringHints NextActions
```
---
## Task 5: Update Production Imports To Canonical Paths
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `src/wf_mcp/workflow_surface/models.py`
- [ ] **Step 1: Update `handlers.py` next action import**
Replace:
```python
from .next_actions import NextActions
```
with:
```python
from wf_api.next_actions import NextActions
```
- [ ] **Step 2: Update `handlers.py` wrapper hint import**
Replace:
```python
from .wrapper_hints import (
workflow_output_schema_for_authoring,
wrapper_hints_for_capability,
)
```
with:
```python
from wf_api.wrapper_hints import (
workflow_output_schema_for_authoring,
wrapper_hints_for_capability,
)
```
- [ ] **Step 3: Update `models.py` next action import**
Replace:
```python
from .next_actions import NextActionPatchExample, NextActions
```
with:
```python
from wf_api.next_actions import NextActionPatchExample, NextActions
```
- [ ] **Step 4: Run production import smoke check**
Run:
```powershell
uv run python -c "from wf_mcp.workflow_surface.handlers import WorkflowSurfaceHandlers; from wf_mcp.workflow_surface.models import WrapperDraftNextActions; print(WorkflowSurfaceHandlers.__name__, WrapperDraftNextActions.__name__)"
```
Expected output:
```text
WorkflowSurfaceHandlers NextActions
```
---
## Task 6: Update Direct Tests And Add Shim Compatibility Tests
**Files:**
- Modify: `tests/wf_mcp/test_workflow_wrapper_hints.py`
- Modify: `tests/wf_mcp/workflow_surface/test_next_actions.py`
- [ ] **Step 1: Update wrapper hints test imports**
In `tests/wf_mcp/test_workflow_wrapper_hints.py`, replace imports from:
```python
from wf_mcp.workflow_surface.wrapper_hints import (
```
with:
```python
from wf_api.wrapper_hints import (
```
- [ ] **Step 2: Add wrapper hints shim identity test**
Append this test to `tests/wf_mcp/test_workflow_wrapper_hints.py`:
```python
def test_workflow_surface_wrapper_hints_shim_reexports_canonical_helper() -> None:
from wf_api.wrapper_hints import wrapper_hints_for_capability
from wf_mcp.workflow_surface.wrapper_hints import (
wrapper_hints_for_capability as wrapper_hints_for_capability_shim,
)
assert wrapper_hints_for_capability_shim is wrapper_hints_for_capability
```
- [ ] **Step 3: Update next actions test imports**
In `tests/wf_mcp/workflow_surface/test_next_actions.py`, replace:
```python
from wf_mcp.workflow_surface.next_actions import NextActionTool, NextActions
```
with:
```python
from wf_api.next_actions import NextActionTool, NextActions
```
- [ ] **Step 4: Add next actions shim identity test**
Append this test to `tests/wf_mcp/workflow_surface/test_next_actions.py`:
```python
def test_workflow_surface_next_actions_shim_reexports_canonical_model() -> None:
from wf_api.next_actions import NextActions
from wf_mcp.workflow_surface.next_actions import NextActions as NextActionsShim
assert NextActionsShim is NextActions
```
- [ ] **Step 5: Run focused tests**
Run:
```powershell
uv run pytest tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_api/test_import_direction.py -q
```
Expected: all pass.
---
## Task 7: Search For Remaining Canonical Import Opportunities
**Files:**
- Inspect only unless the search finds new low-risk direct consumers.
- [ ] **Step 1: Search old imports**
Run:
```powershell
rg -n "from \\.next_actions|from \\.wrapper_hints|from wf_mcp\\.workflow_surface\\.(next_actions|wrapper_hints)" src tests
```
Expected remaining matches:
```text
src/wf_mcp/workflow_surface/next_actions.py
src/wf_mcp/workflow_surface/wrapper_hints.py
tests/wf_mcp/test_workflow_wrapper_hints.py
tests/wf_mcp/workflow_surface/test_next_actions.py
```
If any other production module imports the old paths, update it to import from
`wf_api.next_actions` or `wf_api.wrapper_hints`.
- [ ] **Step 2: Search for accidental `wf_api -> wf_mcp` imports**
Run:
```powershell
rg -n "from wf_mcp|import wf_mcp|wf_mcp\\." src/wf_api
```
Expected: no matches.
---
## Task 8: Verification
- [ ] **Step 1: Run focused tests**
```powershell
uv run pytest tests/wf_api tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface tests/wf_mcp/server/test_config.py -q
```
Expected: all pass.
- [ ] **Step 2: Run CLI tests that assert next_actions/wrapper_hints payloads**
```powershell
uv run pytest tests/wf_cli/test_discovery_lifecycle.py tests/wf_cli/test_run_deploy.py -q
```
Expected: all pass.
- [ ] **Step 3: Run ruff on touched files**
```powershell
uv run ruff check src/wf_api src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/wrapper_hints.py src/wf_mcp/workflow_surface/handlers.py src/wf_mcp/workflow_surface/models.py tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_api
```
Expected: all checks pass.
- [ ] **Step 4: Run basedpyright on touched files**
```powershell
uv run basedpyright --level error src/wf_api src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/wrapper_hints.py src/wf_mcp/workflow_surface/handlers.py src/wf_mcp/workflow_surface/models.py tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_api
```
Expected: `0 errors`.
- [ ] **Step 5: Optional full suite**
Run this if time allows:
```powershell
uv run pytest -q
```
Expected: full suite passes with the projects existing skipped/xfailed counts.
---
## Self-Review Checklist
- `wf_api.wrapper_hints` imports no `wf_mcp`.
- `wf_api.next_actions` imports no `wf_mcp`.
- Old `wf_mcp.workflow_surface.wrapper_hints` import path still works.
- Old `wf_mcp.workflow_surface.next_actions` import path still works.
- `WorkflowSurfaceHandlers` imports canonical guidance helpers from `wf_api`.
- `wf_mcp.workflow_surface.models` imports canonical next-action models from `wf_api`.
- Wrapper hint behavior is unchanged.
- Next action behavior is unchanged.
- No public payload shape changed.
- No other workflow-surface helper moved in this slice.
@@ -0,0 +1,535 @@
# wf_api Slice 4A: Operation Context Scaffolding Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add a protocol-neutral workflow operation context seam so future domain services can be extracted from `WorkflowSurfaceHandlers` without depending on `WfMcpService`.
**Architecture:** This is scaffolding only. `wf_api.operation_context` defines small protocols and a `WorkflowOperationContext` dataclass for stores, capability sources, event recording, workflow runtime operations, and optional live source checking. `wf_mcp.broker.service.workflow_operation_context` adapts the current `WfMcpService` into that context. No workflow handler methods move in this slice and no public payloads change.
**Tech Stack:** Python 3.14+, `typing.Protocol`, dataclasses, existing `wf_artifacts`, `wf_platform`, `wf_authoring`, `wf_core`, pytest, ruff, basedpyright.
---
## Scope
### In Scope
- Create `src/wf_api/operation_context.py`.
- Define focused protocols instead of a new god object.
- Create `src/wf_mcp/broker/service/workflow_operation_context.py` to adapt `WfMcpService`.
- Add tests proving:
- `wf_api.operation_context` imports no `wf_mcp`.
- a `WfMcpService` can be adapted into `WorkflowOperationContext`.
- context stores/sources point to the existing service objects.
- event recording and runtime methods delegate to the service.
- Add docstrings explaining this is scaffolding for later domain splits.
### Out Of Scope
- Do not move methods out of `WorkflowSurfaceHandlers`.
- Do not change `WorkflowApiBackend`.
- Do not change MCP tool request/response models.
- Do not change public payloads.
- Do not rename `WorkflowSurfaceHandlers`.
- Do not add FastAPI/HTTP.
- Do not hide or remove compatibility shims.
---
## Design
### New `wf_api.operation_context`
The context is not a service locator. It is the smallest explicit set of
capabilities that extracted domain services will need.
Protocol groups:
- `WorkflowEventRecorder`: record one lifecycle event object.
- `WorkflowSpecProvider`: look up a qualified node spec and expose capability sources.
- `WorkflowArtifactCataloger`: produce saved artifact catalog entries.
- `WorkflowRuntimeRunner`: run and resume compiled workflow plans.
- `WorkflowLiveSourceChecker`: optional live-source validation hook.
Dataclass:
```python
@dataclass(frozen=True, slots=True)
class WorkflowOperationContext:
artifact_store: WorkflowArtifactStore | None
draft_workspace_store: DraftWorkspaceStore | None
run_store: RunStore | None
capability_sources: Mapping[str, CapabilitySource]
events: WorkflowEventRecorder
specs: WorkflowSpecProvider
artifacts: WorkflowArtifactCataloger
runtime: WorkflowRuntimeRunner
live_sources: WorkflowLiveSourceChecker | None = None
```
This will look somewhat broad, but each field is a small protocol or simple
store reference. If implementation pressure makes this mirror all of
`WfMcpService`, stop and split protocols further.
### MCP Adapter
`wf_mcp.broker.service.workflow_operation_context.context_from_service(service)`
builds the context from the current `WfMcpService`.
This adapter may import `wf_mcp`; `wf_api` must not.
---
## Task 1: Add `wf_api.operation_context`
**Files:**
- Create: `src/wf_api/operation_context.py`
- [ ] **Step 1: Create `src/wf_api/operation_context.py`**
Write this file:
```python
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass
from typing import Any, Protocol
from wf_artifacts import (
DraftWorkspaceStore,
RunStore,
WorkflowArtifact,
WorkflowArtifactCatalogEntry,
WorkflowArtifactStore,
WorkflowDeployment,
)
from wf_authoring import AsyncRegistryHandler
from wf_core import AsyncNodeHandler, RunState, Workflow
from wf_core.runtime.ops.merges import ReducerDefinition
from wf_platform import CapabilitySource
from .models import RawWorkflowPlan
from .saved_subgraphs import SavedSubgraphTree
class WorkflowEventRecorder(Protocol):
"""Records workflow lifecycle events without exposing MCP event types."""
def record_event(self, event: object) -> None:
"""Record one event object supplied by an adapter-owned event factory."""
class WorkflowSpecProvider(Protocol):
"""Provides planner-visible capability sources and qualified node specs."""
@property
def capability_sources(self) -> Mapping[str, CapabilitySource]:
"""Planner-visible capability sources keyed by source id."""
def get_qualified_spec(self, qualified_name: str) -> object:
"""Return the node spec for one fully qualified capability name."""
class WorkflowArtifactCataloger(Protocol):
"""Formats saved workflow artifacts for list/detail surfaces."""
def workflow_artifact_catalog_entry(
self, artifact: WorkflowArtifact
) -> WorkflowArtifactCatalogEntry:
"""Return the catalog entry representation for one saved artifact."""
class WorkflowRuntimeRunner(Protocol):
"""Runs and resumes workflow plans using an adapter-owned runtime backend."""
async def run_workflow_from_plan(
self,
plan: RawWorkflowPlan,
*,
workflow_input: dict[str, Any],
node_name_bindings: dict[str, str] | None = None,
registry: dict[str, AsyncRegistryHandler] | None = None,
reducers: dict[str, ReducerDefinition] | None = None,
prepared_subgraphs: dict[str, object] | None = None,
) -> RunState:
"""Execute one raw workflow plan and return its run state."""
async def resume_workflow_from_plan(
self,
plan: RawWorkflowPlan,
*,
run: RunState,
resume_payload: dict[str, Any] | None,
resume_outcome: str,
node_name_bindings: dict[str, str] | None = None,
registry: dict[str, AsyncRegistryHandler] | None = None,
reducers: dict[str, ReducerDefinition] | None = None,
prepared_subgraphs: dict[str, object] | None = None,
) -> RunState:
"""Resume one interrupted raw workflow plan and return its run state."""
class WorkflowLiveSourceChecker(Protocol):
"""Optional hook for validating live external source availability."""
async def available_sources(self) -> list[object]:
"""Return source availability records understood by the caller."""
@dataclass(frozen=True, slots=True)
class WorkflowOperationContext:
"""Protocol-neutral dependencies needed by workflow API operations.
This is scaffolding for splitting the large MCP-backed handler into domain
services. Keep this shape explicit; do not add arbitrary access to the whole
MCP service.
"""
artifact_store: WorkflowArtifactStore | None
draft_workspace_store: DraftWorkspaceStore | None
run_store: RunStore | None
capability_sources: Mapping[str, CapabilitySource]
events: WorkflowEventRecorder
specs: WorkflowSpecProvider
artifacts: WorkflowArtifactCataloger
runtime: WorkflowRuntimeRunner
live_sources: WorkflowLiveSourceChecker | None = None
__all__ = [
"WorkflowArtifactCataloger",
"WorkflowEventRecorder",
"WorkflowLiveSourceChecker",
"WorkflowOperationContext",
"WorkflowRuntimeRunner",
"WorkflowSpecProvider",
]
```
- [ ] **Step 2: Run import smoke check**
```powershell
uv run python -c "from wf_api.operation_context import WorkflowOperationContext; print(WorkflowOperationContext.__name__)"
```
Expected:
```text
WorkflowOperationContext
```
---
## Task 2: Re-export Operation Context Types
**Files:**
- Modify: `src/wf_api/__init__.py`
- [ ] **Step 1: Add imports**
Add:
```python
from .operation_context import (
WorkflowArtifactCataloger,
WorkflowEventRecorder,
WorkflowLiveSourceChecker,
WorkflowOperationContext,
WorkflowRuntimeRunner,
WorkflowSpecProvider,
)
```
- [ ] **Step 2: Add names to `__all__`**
Add:
```python
"WorkflowArtifactCataloger",
"WorkflowEventRecorder",
"WorkflowLiveSourceChecker",
"WorkflowOperationContext",
"WorkflowRuntimeRunner",
"WorkflowSpecProvider",
```
- [ ] **Step 3: Run top-level import smoke check**
```powershell
uv run python -c "from wf_api import WorkflowOperationContext, WorkflowRuntimeRunner; print(WorkflowOperationContext.__name__, WorkflowRuntimeRunner.__name__)"
```
Expected:
```text
WorkflowOperationContext WorkflowRuntimeRunner
```
---
## Task 3: Add MCP Adapter For Operation Context
**Files:**
- Create: `src/wf_mcp/broker/service/workflow_operation_context.py`
- [ ] **Step 1: Create adapter file**
Write this file:
```python
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
from wf_api.operation_context import (
WorkflowArtifactCataloger,
WorkflowEventRecorder,
WorkflowLiveSourceChecker,
WorkflowOperationContext,
WorkflowRuntimeRunner,
WorkflowSpecProvider,
)
from .core import WfMcpService
@dataclass(frozen=True, slots=True)
class WfMcpWorkflowEventRecorder(WorkflowEventRecorder):
"""Adapter-owned event recorder backed by WfMcpService."""
service: WfMcpService
def record_event(self, event: object) -> None:
self.service._record_event(event) # noqa: SLF001
@dataclass(frozen=True, slots=True)
class WfMcpWorkflowSpecProvider(WorkflowSpecProvider):
"""Adapter-owned spec provider backed by WfMcpService."""
service: WfMcpService
@property
def capability_sources(self):
return self.service.capability_sources
def get_qualified_spec(self, qualified_name: str) -> object:
return self.service._get_qualified_spec(qualified_name) # noqa: SLF001
@dataclass(frozen=True, slots=True)
class WfMcpWorkflowArtifactCataloger(WorkflowArtifactCataloger):
"""Adapter-owned artifact catalog formatter backed by WfMcpService."""
service: WfMcpService
def workflow_artifact_catalog_entry(self, artifact):
return self.service.workflow_artifact_catalog_entry(artifact)
@dataclass(frozen=True, slots=True)
class WfMcpWorkflowRuntimeRunner(WorkflowRuntimeRunner):
"""Adapter-owned runtime runner backed by WfMcpService."""
service: WfMcpService
async def run_workflow_from_plan(self, plan, **kwargs):
return await self.service.run_workflow_from_plan(plan, **kwargs)
async def resume_workflow_from_plan(self, plan, **kwargs):
return await self.service.resume_workflow_from_plan(plan, **kwargs)
@dataclass(frozen=True, slots=True)
class WfMcpWorkflowLiveSourceChecker(WorkflowLiveSourceChecker):
"""Placeholder live source checker; real live checks remain in handlers today."""
service: WfMcpService
async def available_sources(self) -> list[object]:
# Existing live source availability logic still lives near handlers.
# Slice 4A only creates the seam; it does not move live-check behavior.
return []
def context_from_service(service: WfMcpService) -> WorkflowOperationContext:
"""Adapt the current MCP service stack into a protocol-neutral context."""
specs = WfMcpWorkflowSpecProvider(service)
return WorkflowOperationContext(
artifact_store=service.artifact_store,
draft_workspace_store=service.draft_workspace_store,
run_store=service.run_store,
capability_sources=specs.capability_sources,
events=WfMcpWorkflowEventRecorder(service),
specs=specs,
artifacts=WfMcpWorkflowArtifactCataloger(service),
runtime=WfMcpWorkflowRuntimeRunner(service),
live_sources=WfMcpWorkflowLiveSourceChecker(service),
)
__all__ = [
"WfMcpWorkflowArtifactCataloger",
"WfMcpWorkflowEventRecorder",
"WfMcpWorkflowLiveSourceChecker",
"WfMcpWorkflowRuntimeRunner",
"WfMcpWorkflowSpecProvider",
"context_from_service",
]
```
- [ ] **Step 2: Run adapter import smoke check**
```powershell
uv run python -c "from wf_mcp.broker.service.workflow_operation_context import context_from_service; print(context_from_service.__name__)"
```
Expected:
```text
context_from_service
```
---
## Task 4: Add Focused Tests
**Files:**
- Create: `tests/wf_api/test_operation_context.py`
- [ ] **Step 1: Write tests**
Write this file:
```python
from __future__ import annotations
import ast
import json
from pathlib import Path
from wf_api.operation_context import WorkflowOperationContext
from wf_cli.context import load_cli_context
from wf_mcp.broker.service.workflow_operation_context import context_from_service
def test_wf_api_operation_context_imports_no_wf_mcp() -> None:
path = Path(__file__).resolve().parents[2] / "src" / "wf_api" / "operation_context.py"
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
violations: list[str] = []
for node in ast.walk(tree):
if isinstance(node, ast.ImportFrom) and node.module is not None:
if node.module.startswith("wf_mcp"):
violations.append(f"{node.lineno}: from {node.module} import ...")
elif isinstance(node, ast.Import):
for alias in node.names:
if alias.name.startswith("wf_mcp"):
violations.append(f"{node.lineno}: import {alias.name}")
assert violations == []
def test_context_from_service_exposes_existing_store_objects(tmp_path: Path) -> None:
config_path = tmp_path / "wf_mcp.config.json"
config_path.write_text(
json.dumps(
{
"store_root": ".wf_mcp_store",
"connections": [
{
"id": "demo.personal",
"server": "demo",
"account": "personal",
}
],
}
),
encoding="utf-8",
)
cli_context = load_cli_context(config_path)
operation_context = context_from_service(cli_context.service)
assert isinstance(operation_context, WorkflowOperationContext)
assert operation_context.artifact_store is cli_context.service.artifact_store
assert operation_context.draft_workspace_store is cli_context.service.draft_workspace_store
assert operation_context.run_store is cli_context.service.run_store
assert operation_context.capability_sources is cli_context.service.capability_sources
def test_context_from_service_delegates_specs_and_events(tmp_path: Path) -> None:
config_path = tmp_path / "wf_mcp.config.json"
config_path.write_text(
json.dumps({"store_root": ".wf_mcp_store", "connections": []}),
encoding="utf-8",
)
cli_context = load_cli_context(config_path)
operation_context = context_from_service(cli_context.service)
event = object()
operation_context.events.record_event(event)
assert cli_context.service.events[-1] is event
```
- [ ] **Step 2: Run focused tests**
```powershell
uv run pytest tests/wf_api/test_operation_context.py tests/wf_api/test_import_direction.py -q
```
Expected: all pass.
---
## Task 5: Verification
- [ ] **Step 1: Run focused tests**
```powershell
uv run pytest tests/wf_api/test_operation_context.py tests/wf_api/test_import_direction.py tests/wf_cli/test_context.py -q
```
Expected: all pass.
- [ ] **Step 2: Run ruff on touched files**
```powershell
uv run ruff check src/wf_api/operation_context.py src/wf_api/__init__.py src/wf_mcp/broker/service/workflow_operation_context.py tests/wf_api/test_operation_context.py
```
Expected: all checks pass.
- [ ] **Step 3: Run basedpyright on touched files**
```powershell
uv run basedpyright --level error src/wf_api/operation_context.py src/wf_api/__init__.py src/wf_mcp/broker/service/workflow_operation_context.py tests/wf_api/test_operation_context.py
```
Expected: `0 errors`.
- [ ] **Step 4: Optional full suite**
```powershell
uv run pytest -q
```
Expected: full suite passes with the projects existing skipped/xfailed counts.
---
## Self-Review Checklist
- `wf_api.operation_context` imports no `wf_mcp`.
- The context is not a broad wrapper around all of `WfMcpService`.
- No `WorkflowSurfaceHandlers` method body moved.
- No public payload changed.
- MCP-owned adapter code is the only new code that imports `WfMcpService`.
- Live source behavior remains unchanged; the live-source protocol is scaffolding only.
@@ -0,0 +1,509 @@
# wf_api Slice 4B: Draft Service Extraction Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move draft validation, draft workspace editing, and minimal draft bootstrapping out of `WorkflowSurfaceHandlers` into a protocol-neutral `wf_api.drafts.WorkflowDraftApi`.
**Architecture:** `WorkflowDraftApi` depends on `WorkflowOperationContext`, not `WfMcpService`. `WorkflowSurfaceHandlers` keeps the public method names and delegates the draft subset to `WorkflowDraftApi`. This is the first real method-body extraction, so the slice intentionally excludes methods that save artifacts, record events, or call `inspect_capability`.
**Tech Stack:** Python 3.14+, `wf_api.operation_context`, `wf_artifacts` draft helpers, `wf_core` path/binding models, pytest, ruff, basedpyright.
---
## Scope
### Move In This Slice
Move these methods from `WorkflowSurfaceHandlers` to `wf_api.drafts.WorkflowDraftApi`:
```text
validate_draft
compile_draft
patch_draft
list_draft_workspaces
create_draft_workspace
get_draft_workspace
delete_draft_workspace
validate_draft_workspace
patch_draft_workspace
set_draft_name
set_draft_route
set_step_input_map
set_step_output_map
create_minimal_draft_workspace
```
Move these draft-only helper functions to `wf_api.drafts`:
```text
_required_capabilities_for_plan
_required_capability_payloads
_observed_node_specs
_draft_input_maps
_draft_output_map
_draft_input_bindings_payload
_draft_output_bindings_payload
_graph_path_payload
_local_path_payload
_state_path_payload
_escape_json_pointer
```
### Do Not Move In This Slice
Do not move:
```text
create_draft_workspace_from_capability
create_artifact_from_draft
create_artifact_from_workspace
create_wrapper_from_workspace
```
Reasons:
- `create_draft_workspace_from_capability` depends on `inspect_capability`, which belongs to the capability domain. Move it later after the capability-inspection seam is explicit.
- artifact-from-draft/workspace methods save artifacts and record events. Move them with artifacts/deployments in Slice 4C.
Temporary duplication is allowed for private helper functions used by both
draft preview and artifact creation. If a helper still has live callers in
`WorkflowSurfaceHandlers` after draft delegation, keep the old copy until Slice
4C moves the artifact/deployment methods. Do not make `wf_api` import
`wf_mcp` just to avoid duplication.
### Invariants
- No public payload changes.
- No MCP tool schema changes.
- `WorkflowSurfaceHandlers` still exposes the same methods.
- `wf_api` imports no `wf_mcp`.
- Draft methods delegate through `WorkflowDraftApi`.
---
## Task 1: Create `wf_api.drafts`
**Files:**
- Create: `src/wf_api/drafts.py`
- [ ] **Step 1: Create `WorkflowDraftApi` skeleton**
Create `src/wf_api/drafts.py` with imports and class skeleton:
```python
from __future__ import annotations
from collections.abc import Sequence
from typing import Any
from wf_artifacts import (
DraftWorkspaceStore,
RequiredCapability,
build_workflow_artifact_from_plan,
compile_workflow_draft,
create_draft_workspace as create_draft_workspace_record,
get_draft_workspace as get_draft_workspace_record,
patch_draft_workspace as patch_draft_workspace_record,
patch_workflow_draft,
validate_workflow_draft,
)
from wf_core.models.steps import (
InputBinding,
InputPathBinding,
InputValueBinding,
OutputBinding,
)
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
from wf_platform import CapabilityRef, NodeSpecInventory
from .constants import (
DEFAULT_CALL_STEP_ID,
DEFAULT_ERROR_OUTCOME,
DEFAULT_ERROR_STEP_ID,
DEFAULT_OK_OUTCOME,
RUNTIME_ERROR_CAPABILITY,
)
from .operation_context import WorkflowOperationContext
class WorkflowDraftApi:
"""Draft validation and workspace editing operations.
This service deliberately excludes artifact persistence and capability
inspection. Those domains still live in the MCP-backed handler until later
extraction slices.
"""
def __init__(self, context: WorkflowOperationContext) -> None:
self.context = context
```
- [ ] **Step 2: Export the draft service**
Add `WorkflowDraftApi` to `src/wf_api/__init__.py` so future adapters can use
the canonical import path:
```python
from .drafts import WorkflowDraftApi
```
Also add `"WorkflowDraftApi"` to `__all__`.
- [ ] **Step 3: Add store helper**
Add:
```python
def _draft_store(self) -> DraftWorkspaceStore:
if self.context.draft_workspace_store is None:
raise KeyError("draft workspace store is not configured")
return self.context.draft_workspace_store
```
- [ ] **Step 4: Add outcome lookup helper**
Add:
```python
def _outcomes_for_capability(self, qualified_name: str) -> tuple[str, ...] | None:
try:
spec = self.context.specs.get_qualified_spec(qualified_name)
except KeyError:
return None
outcomes = getattr(spec, "outcomes", None)
return tuple(outcomes) if outcomes is not None else None
```
---
## Task 2: Move Stateless Draft Methods
**Files:**
- Modify: `src/wf_api/drafts.py`
- [ ] **Step 1: Add `validate_draft`**
```python
async def validate_draft(self, *, draft: dict[str, Any]) -> dict[str, Any]:
return validate_workflow_draft(
draft,
outcome_lookup=self._outcomes_for_capability,
)
```
- [ ] **Step 2: Add `compile_draft`**
```python
async def compile_draft(self, *, draft: dict[str, Any]) -> dict[str, Any]:
plan = compile_workflow_draft(draft)
return {
"compiled_plan": plan,
"required_capabilities": _required_capability_payloads(
_required_capabilities_for_plan(
plan,
source_bindings=None,
context=self.context,
)
),
}
```
- [ ] **Step 3: Add `patch_draft`**
```python
async def patch_draft(
self,
*,
draft: dict[str, Any],
patch: list[dict[str, Any]],
) -> dict[str, Any]:
return patch_workflow_draft(draft, patch)
```
---
## Task 3: Move Draft Workspace Methods
**Files:**
- Modify: `src/wf_api/drafts.py`
- [ ] **Step 1: Add workspace CRUD and validation methods**
Move these method bodies exactly from `WorkflowSurfaceHandlers`, replacing
`self._draft_store()` with the new `WorkflowDraftApi._draft_store()`:
```text
list_draft_workspaces
create_draft_workspace
get_draft_workspace
delete_draft_workspace
validate_draft_workspace
patch_draft_workspace
```
Keep behavior and return payloads identical.
- [ ] **Step 2: Add patch convenience methods**
Move these method bodies exactly:
```text
set_draft_name
set_draft_route
set_step_input_map
set_step_output_map
```
They should call `self.patch_draft_workspace(...)` inside `WorkflowDraftApi`.
---
## Task 4: Move Minimal Draft Bootstrap
**Files:**
- Modify: `src/wf_api/drafts.py`
- [ ] **Step 1: Add `create_minimal_draft_workspace`**
Move `WorkflowSurfaceHandlers.create_minimal_draft_workspace` into
`WorkflowDraftApi` unchanged except:
- use `self._outcomes_for_capability(...)`
- use `self.create_draft_workspace(...)`
- keep the existing comments about provider-specific error envelopes
- [ ] **Step 2: Add helper functions**
Move these helper functions from `handlers.py` to the bottom of `wf_api.drafts`:
```text
_draft_input_maps
_draft_output_map
_draft_input_bindings_payload
_draft_output_bindings_payload
_graph_path_payload
_local_path_payload
_state_path_payload
_escape_json_pointer
```
Do not change their behavior.
---
## Task 5: Move Required-Capability Draft Helpers
**Files:**
- Modify: `src/wf_api/drafts.py`
- [ ] **Step 1: Move `_required_capabilities_for_plan`**
Move the helper from `handlers.py` and change its signature from:
```python
def _required_capabilities_for_plan(
plan: dict[str, Any],
*,
source_bindings: dict[str, str] | None,
service: WfMcpService,
) -> dict[str, RequiredCapability]:
```
to:
```python
def _required_capabilities_for_plan(
plan: dict[str, Any],
*,
source_bindings: dict[str, str] | None,
context: WorkflowOperationContext,
) -> dict[str, RequiredCapability]:
```
Inside it, call `_observed_node_specs(context)` instead of
`_observed_node_specs(service)`.
- [ ] **Step 2: Move `_observed_node_specs`**
Change its signature from:
```python
def _observed_node_specs(service: WfMcpService) -> dict[str, NodeSpecInventory]:
```
to:
```python
def _observed_node_specs(
context: WorkflowOperationContext,
) -> dict[str, NodeSpecInventory]:
```
Loop over `context.capability_sources.values()`.
- [ ] **Step 3: Move `_required_capability_payloads`**
Move it unchanged.
---
## Task 6: Wire `WorkflowSurfaceHandlers` To Delegate Draft Methods
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- [ ] **Step 1: Add imports**
Add:
```python
from wf_api.drafts import WorkflowDraftApi
from wf_mcp.broker.service.workflow_operation_context import context_from_service
```
- [ ] **Step 2: Instantiate draft service**
In `WorkflowSurfaceHandlers.__init__`, add:
```python
self._drafts = WorkflowDraftApi(context_from_service(service))
```
- [ ] **Step 3: Replace moved method bodies with delegates**
For each moved method, keep the same signature and replace the body with a call
to `self._drafts`.
Example:
```python
async def validate_draft(self, *, draft: dict[str, Any]) -> dict[str, Any]:
return await self._drafts.validate_draft(draft=draft)
```
Apply this pattern to every method in the Slice 4B move list.
- [ ] **Step 4: Remove moved helper functions from `handlers.py`**
Delete only helper functions that are no longer used in `handlers.py`:
```text
_draft_input_maps
_draft_output_map
_draft_input_bindings_payload
_draft_output_bindings_payload
_graph_path_payload
_local_path_payload
_state_path_payload
```
Remove `_escape_json_pointer` only if no remaining references exist in `handlers.py`.
Do not remove from `handlers.py` yet unless `rg` proves there are no remaining
callers:
```text
_required_capabilities_for_plan
_required_capability_payloads
_observed_node_specs
```
Artifact creation currently still uses these helpers. It is acceptable for
`wf_api.drafts` and `handlers.py` to each have a copy until Slice 4C moves the
artifact/deployment methods. If the implementor can safely share them from
`wf_api` without creating an MCP import or changing behavior, that is allowed,
but not required for this slice.
---
## Task 7: Add Focused Tests
**Files:**
- Create: `tests/wf_api/test_drafts_service.py`
- [ ] **Step 1: Write direct service tests**
Create tests that build a `WfMcpService` through existing test helpers or
`load_cli_context`, adapt it with `context_from_service`, then instantiate
`WorkflowDraftApi`.
Cover:
- `patch_draft` applies a JSON patch.
- `create_draft_workspace` creates a workspace.
- `patch_draft_workspace` updates revision.
- `validate_draft_workspace` refreshes status.
- `create_minimal_draft_workspace` returns the same shape as before for a simple `wf.std` capability or a registered test spec.
- [ ] **Step 2: Add delegation smoke test**
In an existing workflow-surface draft test file or a new focused test, assert
that `WorkflowSurfaceHandlers.validate_draft(...)` and
`WorkflowDraftApi.validate_draft(...)` return equivalent status/diagnostics for
the same draft.
Do not assert entire dict equality; compare stable fields individually.
---
## Task 8: Verification
- [ ] **Step 1: Run draft tests**
```powershell
uv run pytest tests/wf_api/test_drafts_service.py tests/wf_mcp/workflow_surface/test_drafts.py -q
```
Expected: all pass.
- [ ] **Step 2: Run MCP schema/config tests**
```powershell
uv run pytest tests/wf_mcp/server/test_config.py tests/wf_mcp/workflow_surface -q
```
Expected: all pass.
- [ ] **Step 3: Run ruff on touched files**
```powershell
uv run ruff check src/wf_api/drafts.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_drafts_service.py
```
Expected: all checks pass.
- [ ] **Step 4: Run basedpyright on touched files**
```powershell
uv run basedpyright --level error src/wf_api/drafts.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_drafts_service.py
```
Expected: `0 errors`.
- [ ] **Step 5: Optional full suite**
```powershell
uv run pytest -q
```
Expected: full suite passes with the projects existing skipped/xfailed counts.
---
## Self-Review Checklist
- `wf_api.drafts` imports no `wf_mcp`.
- `WorkflowSurfaceHandlers` public draft method signatures are unchanged.
- Moved methods delegate through `WorkflowDraftApi`.
- `create_draft_workspace_from_capability` remains in handlers.
- artifact-saving methods remain in handlers.
- No public payload shape changed.
- No MCP schema changed.
- No new dependency on the whole `WfMcpService` inside `wf_api`.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,542 @@
# wf CLI Docs And Skill Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Create real user-facing `wf` CLI documentation and a small agent skill, then update `wf explain` cards to reference those docs instead of planning specs.
**Architecture:** Treat `docs/superpowers/*` as planning history only, not runtime/user guidance. The canonical CLI reference should live at `docs/wf_cli.md`; the optional repo-local skill should live at `skills/wf-cli/SKILL.md` and point agents to the real doc plus the safest command flow.
**Tech Stack:** Markdown docs, existing `wf_cli.explain` registry, pytest, ruff.
---
## Scope
Create:
```text
docs/wf_cli.md
skills/wf-cli/SKILL.md
```
Modify:
```text
docs/README.md
src/wf_cli/explain/entries.py
tests/wf_cli/test_explain.py
```
Do not add command aliases in this slice. `wf cap` and `wf deploy` are the real registered command names.
Do not link `wf explain` runtime guidance to:
```text
docs/superpowers/specs/*
docs/superpowers/plans/*
```
Those are planning artifacts, not user/operator docs.
## Task 1: Add Real CLI User Documentation
**Files:**
- Create: `docs/wf_cli.md`
- Modify: `docs/README.md`
- [ ] **Step 1: Create `docs/wf_cli.md`**
Create `docs/wf_cli.md`:
```markdown
# wf CLI
`wf` is the workflow platform command-line interface. It is a second front door
beside MCP: useful for shell-driven authoring, local validation, file-based
patches, and agent workflows that do better with commands than giant MCP
schemas.
`wf` uses the same config/store stack as the MCP server in v1:
```bash
wf --config wf_mcp.config.json <command>
```
If `--config` is omitted, `wf_mcp.config.json` is used.
## Output Policy
JSON is the default output format for every command.
List/discovery commands may support:
```text
--format json # complete machine-readable payload
--format ids # one identifier per line
--format compact # one concise line per item
```
Detail and mutation commands are JSON-only unless documented otherwise.
There is no `table` format in v1.
## Lifecycle
The normal CLI workflow is:
1. Inspect capabilities.
2. Create a draft workspace from a capability.
3. Inspect or patch the draft.
4. Validate the draft.
5. Save an artifact.
6. Save a deployment with source bindings.
7. Validate the deployment.
8. Run the deployment.
9. Read bounded trace detail only when debugging.
## Capability Discovery
List capabilities:
```bash
wf cap list
wf cap list --source wf.std --format ids
wf cap list --query echo --format compact
```
Inspect one capability:
```bash
wf cap inspect wf.std.concat
```
`inspect` returns the full contract, including `wrapper_hints` when available.
Hints are scaffolding, not semantic guarantees.
## Draft Workspaces
Create a draft from a capability:
```bash
wf draft create-from-capability concat_ws wf.std.concat --name concat_ws
```
List and inspect drafts:
```bash
wf draft list --format compact
wf draft inspect concat_ws
wf draft inspect concat_ws --include-draft
```
Patch a draft with RFC 6902 JSON Patch:
```bash
wf draft patch concat_ws \
--revision 1 \
--input '[{"op":"replace","path":"/name","value":"concat_ws_v2"}]'
```
Validate:
```bash
wf draft validate concat_ws
```
Save as an artifact:
```bash
wf draft save concat_ws \
--artifact concat_ws \
--version 1 \
--title "Concat Workflow" \
--outcome ok \
--binding wf.std=wf.std
```
Use `--kind wrapper` when saving a callable wrapper artifact:
```bash
wf draft save concat_ws \
--artifact concat_wrapper \
--version 1 \
--title "Concat Wrapper" \
--kind wrapper \
--outcome ok \
--binding wf.std=wf.std
```
## Artifacts
List and inspect artifacts:
```bash
wf artifact list --format ids
wf artifact list --kind wrapper --format compact
wf artifact inspect concat_ws 1
```
Artifacts are immutable saved workflow definitions. List output is compact by
design; use `inspect` for full details.
## Deployments
Save a deployment from flags:
```bash
wf deploy save concat_ws.default \
--artifact concat_ws \
--version 1 \
--binding wf.std=wf.std
```
Save a deployment from JSON:
```bash
wf deploy save --input-file deployment.json
```
List, inspect, validate, and delete:
```bash
wf deploy list --format compact
wf deploy inspect concat_ws.default
wf deploy validate concat_ws.default
wf deploy validate concat_ws.default --live
wf deploy delete concat_ws.default
```
`--live` performs opt-in upstream liveness checks. Static validation can pass
even when a live external source is temporarily unreachable.
## Runs And Traces
Start a deployment:
```bash
wf run start concat_ws.default \
--input '{"items":["red","blue"],"separator":" + "}'
```
Inspect a run without trace detail:
```bash
wf run inspect run_123
```
Read a bounded trace slice:
```bash
wf run trace run_123 --from 0 --limit 25
```
Trace output can be large. Always request a bounded range.
## Explain
Explain stable diagnostic/error codes:
```bash
wf explain source_missing
wf explain deployment_unrunnable --format markdown
wf explain --input-file validation-output.json
wf explain --list --format compact
```
`wf explain` is exact-match and docs-backed. It is not fuzzy search and does not
generate prose.
## Common Diagnostics
### `source_missing`
A required logical source is not available or not bound.
Check:
```bash
wf deploy inspect <deployment_id>
wf cap list
wf deploy validate <deployment_id> --live
```
### `binding_missing`
A deployment is missing a required logical-to-concrete source binding.
Check artifact requirements, then save the deployment with all required
bindings:
```bash
wf deploy save <deployment_id> \
--artifact <artifact_id> \
--version <version> \
--binding <logical>=<concrete>
```
### `capability_missing`
The bound source does not expose a required capability.
Check:
```bash
wf cap list --source <source_id>
wf deploy inspect <deployment_id>
```
### `schema_changed`
A saved dependency schema no longer matches the live capability. Inspect the
live capability, patch the draft or wrapper, and save a new artifact version.
### `deployment_unrunnable`
The deployment failed validation and should not be run yet.
Check:
```bash
wf deploy validate <deployment_id>
wf explain --input-file validation-output.json
```
## Known Limits
- The CLI reuses `wf_mcp` service/config/store wiring in v1.
- Config loading registers stores and connections, but not arbitrary in-memory
test `NodeSpec` functions.
- Targeted draft editing helpers such as `wf draft step add` are not in v1.
- `wf` does not replace MCP resources/prompts or interactive MCP clients.
```
- [ ] **Step 2: Add CLI doc to docs index**
Modify `docs/README.md` under `## Current Overview` or a new `## CLI` section:
```markdown
- [`wf_cli.md`](wf_cli.md): workflow platform CLI commands, output formats,
lifecycle flow, and common diagnostics.
```
- [ ] **Step 3: Verify doc links manually**
Run:
```bash
Test-Path docs/wf_cli.md
Select-String -Path docs/README.md -Pattern 'wf_cli.md'
```
Expected: both show the new doc exists and is indexed.
## Task 2: Add Repo-Local Agent Skill
**Files:**
- Create: `skills/wf-cli/SKILL.md`
- [ ] **Step 1: Create `skills/wf-cli/SKILL.md`**
Create `skills/wf-cli/SKILL.md`:
```markdown
---
name: wf-cli
description: Use when authoring, validating, deploying, running, or debugging workflows through the repo-local `wf` CLI.
---
# wf CLI
Use the `wf` CLI when an agent needs a shell-friendly workflow lifecycle:
1. Discover capabilities.
2. Create or patch a draft workspace.
3. Validate the draft.
4. Save an artifact.
5. Save and validate a deployment.
6. Run the deployment.
7. Read bounded trace slices only when debugging.
Canonical docs:
- `docs/wf_cli.md`
- `docs/workflow_capabilities.md`
- `docs/workflow_drafts.md`
- `docs/workflow_artifacts.md`
- `docs/durable_run_operations.md`
## Core Commands
```bash
wf cap list --format ids
wf cap inspect <capability>
wf draft create-from-capability <workspace_id> <capability>
wf draft inspect <workspace_id> --include-draft
wf draft patch <workspace_id> --revision <n> --input-file patch.json
wf draft validate <workspace_id>
wf draft save <workspace_id> --artifact <artifact_id> --version <n> --title <title>
wf deploy save <deployment_id> --artifact <artifact_id> --version <n> --binding <logical>=<concrete>
wf deploy validate <deployment_id>
wf run start <deployment_id> --input-file input.json
wf run trace <run_id> --from 0 --limit 25
```
## Rules
- Prefer `--input-file` for large JSON.
- Prefer `--format ids` or `--format compact` for discovery.
- Do not request unbounded traces.
- Do not treat wrapper hints as semantic guarantees.
- If validation fails, run `wf explain <code>` or `wf explain --input-file <validation-output.json>`.
- Do not use docs under `docs/superpowers/` as user-facing runtime guidance.
```
- [ ] **Step 2: Do not wire skill installation**
Do not add marketplace/plugin installation logic in this slice. This repo-local
skill is a source document for future packaging; it is not automatically active
until a user installs or copies it into their agent environment.
## Task 3: Update `wf explain` Doc References
**Files:**
- Modify: `src/wf_cli/explain/entries.py`
- Test: `tests/wf_cli/test_explain.py`
- [ ] **Step 1: Add tests that all explain doc refs are user-facing and existing**
Append to `tests/wf_cli/test_explain.py`:
```python
from pathlib import Path
def test_explain_related_docs_do_not_point_to_planning_artifacts() -> None:
for entry in DEFAULT_EXPLAIN_REGISTRY.list_full_entries():
for related_doc in entry.related_docs:
assert "docs/superpowers/" not in related_doc
def test_explain_related_doc_files_exist() -> None:
repo_root = Path(__file__).resolve().parents[2]
for entry in DEFAULT_EXPLAIN_REGISTRY.list_full_entries():
for related_doc in entry.related_docs:
path_text = related_doc.split("#", 1)[0]
if path_text.startswith("docs/"):
assert (repo_root / path_text).exists(), path_text
```
- [ ] **Step 2: Add registry full-entry accessor**
Modify `src/wf_cli/explain/registry.py`:
```python
def list_full_entries(self) -> list[ExplainCard]:
"""Return full cards for internal validation/tests."""
return list(self._entries.values())
```
- [ ] **Step 3: Update explain cards to use `docs/wf_cli.md`**
Modify `src/wf_cli/explain/entries.py`:
```python
related_docs=[
"docs/wf_cli.md#deployments",
"docs/workflow_capabilities.md",
],
```
Use these mappings:
```text
source_missing -> docs/wf_cli.md#common-diagnostics, docs/workflow_capabilities.md
source_unreachable -> docs/wf_cli.md#deployments, docs/wf_mcp_troubleshooting.md
binding_missing -> docs/wf_cli.md#deployments, docs/workflow_artifacts.md
capability_missing -> docs/wf_cli.md#capability-discovery, docs/workflow_capabilities.md
schema_changed -> docs/wf_cli.md#common-diagnostics, docs/schema_validation.md
deployment_unrunnable -> docs/wf_cli.md#common-diagnostics, docs/current_roadmap.md
```
- [ ] **Step 4: Run explain tests**
Run:
```bash
uv run pytest tests/wf_cli/test_explain.py -q
```
Expected: pass.
## Task 4: Verification
**Files:**
- No new files unless lint/format requires cleanup.
- [ ] **Step 1: Run focused CLI tests**
Run:
```bash
uv run pytest tests/wf_cli -q
```
Expected: all CLI tests pass.
- [ ] **Step 2: Run focused lint**
Run:
```bash
uv run ruff check src/wf_cli tests/wf_cli
```
Expected: no lint errors.
- [ ] **Step 3: Run focused format check**
Run:
```bash
uv run ruff format --check src/wf_cli tests/wf_cli
```
Expected: no formatting changes required. If this fails, run:
```bash
uv run ruff format src/wf_cli tests/wf_cli
```
Then rerun the format check.
- [ ] **Step 4: Run docs path check**
Run:
```bash
Test-Path docs/wf_cli.md
Test-Path skills/wf-cli/SKILL.md
rg -n "docs/superpowers/" src/wf_cli/explain docs/wf_cli.md skills/wf-cli/SKILL.md
```
Expected:
- `docs/wf_cli.md` exists.
- `skills/wf-cli/SKILL.md` exists.
- `rg` finds no `docs/superpowers/` runtime guidance references in those paths.
## Self-Review Checklist
- [ ] `wf explain` cards no longer link to `docs/superpowers/specs` or `docs/superpowers/plans`.
- [ ] `docs/wf_cli.md` contains the lifecycle that the CLI actually supports today.
- [ ] `docs/README.md` points to the new CLI doc.
- [ ] The skill is repo-local documentation only; no install/packaging behavior was added.
- [ ] Tests verify explain-card doc refs are not stale.
@@ -0,0 +1,955 @@
# wf CLI Explain Registry Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add a docs-backed `wf explain` command that explains stable workflow diagnostic/error codes from exact codes or CLI/MCP-style error JSON.
**Architecture:** Keep explanation data in a small protocol-neutral registry under `wf_cli.explain`, then make `src/wf_cli/commands/explain.py` a thin Typer boundary. This slice does not change run/deploy behavior and does not implement fuzzy search, generated prose, or a large FAQ.
**Tech Stack:** Python 3.14, Typer, Pydantic v2, pytest `CliRunner`, existing `wf_cli.io.emit_json`.
---
## File Structure
Create:
```text
src/wf_cli/explain/
__init__.py # public exports for the explain registry package
entries.py # curated explanation cards; no runtime workflow logic
models.py # Pydantic models for cards and list summaries
parser.py # extracts stable codes from direct strings and JSON payloads
registry.py # exact lookup/list API over curated cards
tests/wf_cli/test_explain.py
```
Modify:
```text
src/wf_cli/commands/explain.py
```
Do not modify:
```text
src/wf_cli/commands/runs.py
src/wf_cli/commands/deployments.py
src/wf_mcp/
src/wf_core/
```
## Behavior Contract
Supported commands:
```bash
wf explain source_missing
wf explain source_missing --format json
wf explain source_missing --format markdown
wf explain source_missing --format compact
wf explain --input-file error.json
wf explain --stdin
wf explain --list
```
Supported initial codes:
```text
source_missing
source_unreachable
binding_missing
capability_missing
schema_changed
deployment_unrunnable
```
JSON output rules:
- `wf explain <code>` returns one full card object.
- `wf explain --input-file error.json` returns `{"entries": [full_card, ...]}` because input JSON can contain multiple diagnostics.
- `wf explain --stdin` follows the same shape as `--input-file`.
- `wf explain --list` returns a lean index: `{"entries": [{"code": "...", "summary": "..."}]}`.
Markdown output rules:
- One card renders as a small markdown document.
- Multiple cards render as multiple markdown documents separated by one blank line.
- List output renders a bullet list.
Compact output rules:
- One line per card: `code: summary`.
- List output uses the same one-line shape.
Input extraction rules:
- Direct string code: `source_missing`
- JSON object with `code`: `{"code": "source_missing"}`
- JSON object with `error.code`: `{"error": {"code": "deployment_unrunnable"}}`
- JSON object with diagnostics: `{"diagnostics": [{"code": "source_missing"}, {"code": "schema_changed"}]}`
- JSON list of diagnostics: `[{"code": "source_missing"}, {"code": "schema_changed"}]`
- Preserve first-seen order and dedupe repeated codes.
- Unknown codes should fail with a Typer error that includes the unknown code.
## Task 1: Add Registry Models And Entries
**Files:**
- Create: `src/wf_cli/explain/models.py`
- Create: `src/wf_cli/explain/entries.py`
- Create: `src/wf_cli/explain/__init__.py`
- Test: `tests/wf_cli/test_explain.py`
- [ ] **Step 1: Write failing model/registry smoke tests**
Create `tests/wf_cli/test_explain.py` with these first tests:
```python
from __future__ import annotations
from wf_cli.explain import DEFAULT_EXPLAIN_REGISTRY
def test_explain_registry_returns_full_card_for_known_code() -> None:
card = DEFAULT_EXPLAIN_REGISTRY.get("source_missing")
assert card.code == "source_missing"
assert "source" in card.summary.lower()
assert card.why_it_happens
assert card.how_to_fix
assert card.related_docs
def test_explain_registry_list_is_lean() -> None:
entries = DEFAULT_EXPLAIN_REGISTRY.list_entries()
source_missing = next(entry for entry in entries if entry.code == "source_missing")
assert source_missing.code == "source_missing"
assert "source" in source_missing.summary.lower()
assert not hasattr(source_missing, "how_to_fix")
```
- [ ] **Step 2: Run the failing tests**
Run:
```bash
uv run pytest tests/wf_cli/test_explain.py -q
```
Expected: fail because `wf_cli.explain` does not exist.
- [ ] **Step 3: Implement Pydantic models**
Create `src/wf_cli/explain/models.py`:
```python
from __future__ import annotations
from pydantic import BaseModel, Field
class ExplainCard(BaseModel):
"""Human-curated help for one stable workflow diagnostic/error code."""
code: str = Field(min_length=1, description="Stable diagnostic or CLI error code.")
summary: str = Field(min_length=1, description="One-sentence explanation.")
why_it_happens: list[str] = Field(
description="Common causes, ordered from most likely to least likely."
)
how_to_fix: list[str] = Field(
description="Concrete next steps an agent or user can try."
)
related_docs: list[str] = Field(
default_factory=list,
description="Documentation resource IDs or file references.",
)
class ExplainSummary(BaseModel):
"""Lean index entry for `wf explain --list`."""
code: str = Field(min_length=1)
summary: str = Field(min_length=1)
```
- [ ] **Step 4: Implement curated entries**
Create `src/wf_cli/explain/entries.py`:
```python
from __future__ import annotations
from .models import ExplainCard
EXPLAIN_CARDS: tuple[ExplainCard, ...] = (
ExplainCard(
code="source_missing",
summary="A required logical source is not available or not bound.",
why_it_happens=[
"The artifact requires a logical source that the deployment did not bind.",
"The concrete source was removed, renamed, disabled, or never registered.",
"A saved wrapper or workflow depends on a source that is absent in this config.",
],
how_to_fix=[
"Run `wf deploy inspect <deployment_id>` and check the bindings.",
"Run `wf cap list` to confirm the concrete source is available.",
"Save the deployment again with the missing logical source bound.",
"Run `wf deploy validate <deployment_id> --live` after changing bindings.",
],
related_docs=[
"docs/wf_cli_usage.md#deployment-validation",
"docs/workflow_capabilities.md",
],
),
ExplainCard(
code="source_unreachable",
summary="A concrete source exists in config but could not be reached.",
why_it_happens=[
"The upstream MCP server or local process failed during liveness checks.",
"The source command, URL, authentication, or environment is invalid.",
"The source is slow or hung and exceeded the bounded liveness timeout.",
],
how_to_fix=[
"Check the source command or URL in the active config.",
"Start or restart the upstream server.",
"Run validation without `--live` if you only need static deployment checks.",
"Run `wf deploy validate <deployment_id> --live` again after fixing the source.",
],
related_docs=[
"docs/wf_cli_usage.md#deployment-validation",
"docs/wf_mcp_unified_proxy_plan.md",
],
),
ExplainCard(
code="binding_missing",
summary="A deployment is missing a required logical-to-concrete source binding.",
why_it_happens=[
"The artifact was saved with required capabilities under a logical source.",
"The deployment was saved without a binding for that logical source.",
"A binding field was misspelled or placed under the wrong payload key.",
],
how_to_fix=[
"Inspect the artifact requirements.",
"Inspect the deployment bindings.",
"Save the deployment with `bindings` entries that map each logical source.",
"Use `wf deploy validate <deployment_id>` to confirm the binding set.",
],
related_docs=[
"docs/wf_cli_usage.md#save-and-validate-a-deployment",
"docs/workflow_capabilities.md#sources",
],
),
ExplainCard(
code="capability_missing",
summary="A required capability is not present on the bound source.",
why_it_happens=[
"The upstream source no longer exposes the tool or node spec.",
"The workflow was bound to the wrong account/profile/source.",
"The capability was renamed after the artifact was saved.",
],
how_to_fix=[
"Run `wf cap list` and search for the expected capability.",
"Inspect the deployment bindings for the affected logical source.",
"Rebind to a concrete source that exposes the capability.",
"Rebuild or patch the artifact if the capability was intentionally renamed.",
],
related_docs=[
"docs/workflow_capabilities.md",
"docs/wf_cli_usage.md#capability-discovery",
],
),
ExplainCard(
code="schema_changed",
summary="A saved dependency schema no longer matches the live capability.",
why_it_happens=[
"The upstream tool or node spec changed its input/output schema.",
"The deployment is bound to a different source profile than the one used before.",
"A wrapper assumes fields that the live capability no longer declares.",
],
how_to_fix=[
"Inspect the live capability.",
"Compare it with the saved artifact dependency summary.",
"Patch the draft or wrapper to match the new schema.",
"Save a new artifact version and deployment after validating the change.",
],
related_docs=[
"docs/workflow_capabilities.md#dependency-validation",
"docs/schema_validation.md",
],
),
ExplainCard(
code="deployment_unrunnable",
summary="The deployment failed validation and should not be run yet.",
why_it_happens=[
"One or more required sources, capabilities, schemas, or bindings are invalid.",
"The deployment points at an artifact version that cannot be resolved.",
"Live validation found an upstream source or capability problem.",
],
how_to_fix=[
"Run `wf deploy validate <deployment_id>` and read the diagnostics.",
"Run `wf explain --input-file <validation-output.json>` for diagnostic details.",
"Fix source bindings or rebuild the artifact version.",
"Re-run validation before starting the deployment.",
],
related_docs=[
"docs/wf_cli_usage.md#deployment-validation",
"docs/current_roadmap.md",
],
),
)
```
- [ ] **Step 5: Implement registry**
Create `src/wf_cli/explain/registry.py`:
```python
from __future__ import annotations
from collections.abc import Iterable
from .entries import EXPLAIN_CARDS
from .models import ExplainCard, ExplainSummary
class UnknownExplainCode(KeyError):
"""Raised when a diagnostic code is not present in the curated registry."""
class ExplainRegistry:
"""Exact-match registry for docs-backed explanation cards."""
def __init__(self, entries: Iterable[ExplainCard] = EXPLAIN_CARDS) -> None:
self._entries = {entry.code: entry for entry in entries}
def get(self, code: str) -> ExplainCard:
"""Return a full explanation card for one stable code."""
try:
return self._entries[code]
except KeyError as exc:
raise UnknownExplainCode(code) from exc
def list_entries(self) -> list[ExplainSummary]:
"""Return lean summaries for discovery output."""
return [
ExplainSummary(code=entry.code, summary=entry.summary)
for entry in self._entries.values()
]
DEFAULT_EXPLAIN_REGISTRY = ExplainRegistry()
```
- [ ] **Step 6: Export public API**
Create `src/wf_cli/explain/__init__.py`:
```python
"""Docs-backed explanation registry for workflow CLI diagnostics."""
from .models import ExplainCard, ExplainSummary
from .registry import DEFAULT_EXPLAIN_REGISTRY, ExplainRegistry, UnknownExplainCode
__all__ = [
"DEFAULT_EXPLAIN_REGISTRY",
"ExplainCard",
"ExplainRegistry",
"ExplainSummary",
"UnknownExplainCode",
]
```
- [ ] **Step 7: Run tests**
Run:
```bash
uv run pytest tests/wf_cli/test_explain.py -q
```
Expected: pass.
## Task 2: Add JSON/String Code Parser
**Files:**
- Create: `src/wf_cli/explain/parser.py`
- Modify: `src/wf_cli/explain/__init__.py`
- Test: `tests/wf_cli/test_explain.py`
- [ ] **Step 1: Add parser tests**
Replace the import block at the top of `tests/wf_cli/test_explain.py` with:
```python
from __future__ import annotations
import pytest
from wf_cli.explain import (
DEFAULT_EXPLAIN_REGISTRY,
ExplainInputError,
extract_explain_codes,
parse_explain_input,
)
```
Then append these tests to `tests/wf_cli/test_explain.py`:
```python
def test_parse_explain_input_accepts_direct_code() -> None:
assert parse_explain_input("source_missing") == ["source_missing"]
def test_parse_explain_input_extracts_error_code() -> None:
raw = '{"error": {"code": "deployment_unrunnable"}}'
assert parse_explain_input(raw) == ["deployment_unrunnable"]
def test_parse_explain_input_extracts_diagnostic_codes_in_order() -> None:
raw = """
{
"diagnostics": [
{"code": "source_missing"},
{"code": "schema_changed"},
{"code": "source_missing"}
]
}
"""
assert parse_explain_input(raw) == ["source_missing", "schema_changed"]
def test_extract_explain_codes_accepts_diagnostic_list() -> None:
value = [{"code": "binding_missing"}, {"code": "capability_missing"}]
assert extract_explain_codes(value) == ["binding_missing", "capability_missing"]
def test_parse_explain_input_rejects_json_without_codes() -> None:
with pytest.raises(ExplainInputError, match="no explainable code"):
parse_explain_input('{"status": "failed"}')
```
- [ ] **Step 2: Run parser tests to verify failure**
Run:
```bash
uv run pytest tests/wf_cli/test_explain.py -q
```
Expected: fail because `parser.py` and exports do not exist.
- [ ] **Step 3: Implement parser**
Create `src/wf_cli/explain/parser.py`:
```python
from __future__ import annotations
import json
from typing import Any
class ExplainInputError(ValueError):
"""Raised when `wf explain` input cannot be reduced to stable codes."""
def parse_explain_input(raw: str) -> list[str]:
"""Parse a direct code or JSON payload into first-seen unique codes."""
stripped = raw.strip()
if not stripped:
raise ExplainInputError("explain input is empty")
if stripped.startswith("{") or stripped.startswith("["):
try:
value = json.loads(stripped)
except json.JSONDecodeError as exc:
raise ExplainInputError(f"invalid JSON explain input: {exc.msg}") from exc
return extract_explain_codes(value)
return [stripped]
def extract_explain_codes(value: Any) -> list[str]:
"""Extract known diagnostic-code shapes without guessing or fuzzy matching."""
codes: list[str] = []
_collect_codes(value, codes)
deduped = _dedupe(codes)
if not deduped:
raise ExplainInputError("no explainable code found in input")
return deduped
def _collect_codes(value: Any, codes: list[str]) -> None:
if isinstance(value, str):
codes.append(value)
return
if isinstance(value, list):
for item in value:
_collect_codes(item, codes)
return
if not isinstance(value, dict):
return
code = value.get("code")
if isinstance(code, str):
codes.append(code)
error = value.get("error")
if isinstance(error, dict):
error_code = error.get("code")
if isinstance(error_code, str):
codes.append(error_code)
diagnostics = value.get("diagnostics")
if isinstance(diagnostics, list):
for diagnostic in diagnostics:
_collect_codes(diagnostic, codes)
def _dedupe(codes: list[str]) -> list[str]:
seen: set[str] = set()
result: list[str] = []
for code in codes:
if code in seen:
continue
seen.add(code)
result.append(code)
return result
```
- [ ] **Step 4: Export parser helpers**
Modify `src/wf_cli/explain/__init__.py`:
```python
"""Docs-backed explanation registry for workflow CLI diagnostics."""
from .models import ExplainCard, ExplainSummary
from .parser import ExplainInputError, extract_explain_codes, parse_explain_input
from .registry import DEFAULT_EXPLAIN_REGISTRY, ExplainRegistry, UnknownExplainCode
__all__ = [
"DEFAULT_EXPLAIN_REGISTRY",
"ExplainCard",
"ExplainInputError",
"ExplainRegistry",
"ExplainSummary",
"UnknownExplainCode",
"extract_explain_codes",
"parse_explain_input",
]
```
- [ ] **Step 5: Run parser tests**
Run:
```bash
uv run pytest tests/wf_cli/test_explain.py -q
```
Expected: pass.
## Task 3: Implement `wf explain` Command And Rendering
**Files:**
- Modify: `src/wf_cli/commands/explain.py`
- Modify: `src/wf_cli/app.py`
- Test: `tests/wf_cli/test_explain.py`
- Test: `tests/wf_cli/test_app.py`
- [ ] **Step 1: Add CLI tests**
Replace the import block at the top of `tests/wf_cli/test_explain.py` with:
```python
from __future__ import annotations
import json
import pytest
from typer.testing import CliRunner
from wf_cli.app import app
from wf_cli.explain import (
DEFAULT_EXPLAIN_REGISTRY,
ExplainInputError,
extract_explain_codes,
parse_explain_input,
)
```
Then append these tests to `tests/wf_cli/test_explain.py`:
```python
runner = CliRunner()
def test_wf_explain_code_outputs_full_json_card() -> None:
result = runner.invoke(app, ["explain", "source_missing"])
assert result.exit_code == 0
payload = json.loads(result.output)
assert payload["code"] == "source_missing"
assert payload["summary"]
assert payload["why_it_happens"]
assert payload["how_to_fix"]
def test_wf_explain_list_outputs_lean_json_index() -> None:
result = runner.invoke(app, ["explain", "--list"])
assert result.exit_code == 0
payload = json.loads(result.output)
first = payload["entries"][0]
assert first["code"]
assert first["summary"]
assert "how_to_fix" not in first
def test_wf_explain_markdown_format() -> None:
result = runner.invoke(app, ["explain", "source_missing", "--format", "markdown"])
assert result.exit_code == 0
assert "# source_missing" in result.output
assert "## How To Fix" in result.output
def test_wf_explain_compact_format() -> None:
result = runner.invoke(app, ["explain", "source_missing", "--format", "compact"])
assert result.exit_code == 0
assert result.output.strip().startswith("source_missing: ")
def test_wf_explain_input_file_outputs_multiple_cards(tmp_path) -> None:
error_file = tmp_path / "error.json"
error_file.write_text(
json.dumps(
{
"diagnostics": [
{"code": "source_missing"},
{"code": "schema_changed"},
{"code": "source_missing"},
]
}
),
encoding="utf-8",
)
result = runner.invoke(app, ["explain", "--input-file", str(error_file)])
assert result.exit_code == 0
payload = json.loads(result.output)
assert len(payload["entries"]) == 2
assert payload["entries"][0]["code"] == "source_missing"
assert payload["entries"][1]["code"] == "schema_changed"
def test_wf_explain_stdin_outputs_multiple_cards() -> None:
result = runner.invoke(
app,
["explain", "--stdin"],
input='{"error": {"code": "deployment_unrunnable"}}',
)
assert result.exit_code == 0
payload = json.loads(result.output)
assert payload["entries"][0]["code"] == "deployment_unrunnable"
def test_wf_explain_unknown_code_fails_clearly() -> None:
result = runner.invoke(app, ["explain", "not_a_real_code"])
assert result.exit_code != 0
assert "not_a_real_code" in result.output
def test_wf_explain_list_rejects_other_input_modes() -> None:
result = runner.invoke(app, ["explain", "source_missing", "--list"])
assert result.exit_code != 0
assert "--list cannot be combined" in result.output
```
Append this test to `tests/wf_cli/test_app.py`:
```python
def test_wf_explain_help_shows_input_modes() -> None:
result = runner.invoke(app, ["explain", "--help"])
assert result.exit_code == 0
assert "--input-file" in result.output
assert "--stdin" in result.output
assert "--list" in result.output
```
- [ ] **Step 2: Run CLI tests to verify failure**
Run:
```bash
uv run pytest tests/wf_cli/test_explain.py tests/wf_cli/test_app.py -q
```
Expected: fail because `wf explain` has no callback implementation.
- [ ] **Step 3: Implement command**
Replace `src/wf_cli/commands/explain.py` with:
```python
from __future__ import annotations
from enum import StrEnum
from pathlib import Path
from typing import Annotated
import typer
from wf_cli.explain import (
DEFAULT_EXPLAIN_REGISTRY,
ExplainCard,
ExplainInputError,
ExplainSummary,
UnknownExplainCode,
parse_explain_input,
)
from wf_cli.io import emit_json
class ExplainFormat(StrEnum):
"""Output formats supported by `wf explain`."""
JSON = "json"
MARKDOWN = "markdown"
COMPACT = "compact"
def explain_command(
code: Annotated[
str | None,
typer.Argument(help="Diagnostic/error code, or JSON payload containing codes."),
] = None,
input_file: Annotated[
Path | None,
typer.Option("--input-file", help="Read diagnostic/error JSON from a file."),
] = None,
read_stdin: Annotated[
bool,
typer.Option("--stdin", help="Read diagnostic/error JSON from standard input."),
] = False,
list_entries: Annotated[
bool,
typer.Option("--list", help="List known explanation codes."),
] = False,
output_format: Annotated[
ExplainFormat,
typer.Option("--format", help="Output format."),
] = ExplainFormat.JSON,
) -> None:
"""Explain exact workflow diagnostic codes without generated prose."""
try:
if list_entries:
if code is not None or input_file is not None or read_stdin:
raise ExplainInputError(
"--list cannot be combined with code, --input-file, or --stdin"
)
_emit_summaries(DEFAULT_EXPLAIN_REGISTRY.list_entries(), output_format)
return
codes = _read_codes(code=code, input_file=input_file, read_stdin=read_stdin)
cards = [DEFAULT_EXPLAIN_REGISTRY.get(item) for item in codes]
except (ExplainInputError, UnknownExplainCode) as exc:
raise typer.BadParameter(_error_message(exc)) from exc
if len(cards) == 1 and input_file is None and not read_stdin:
_emit_card(cards[0], output_format)
else:
_emit_cards(cards, output_format)
def _read_codes(
*,
code: str | None,
input_file: Path | None,
read_stdin: bool,
) -> list[str]:
"""Resolve the mutually exclusive input modes supported by `wf explain`."""
selected = sum(value is not None for value in (code, input_file)) + int(read_stdin)
if selected == 0:
raise ExplainInputError("provide a code, --input-file, --stdin, or --list")
if selected > 1:
raise ExplainInputError("code, --input-file, and --stdin are mutually exclusive")
if code is not None:
return parse_explain_input(code)
if input_file is not None:
try:
return parse_explain_input(input_file.read_text(encoding="utf-8"))
except OSError as exc:
message = f"could not read input file {input_file!s}: {exc}"
raise ExplainInputError(message) from exc
return parse_explain_input(typer.get_text_stream("stdin").read())
def _emit_card(card: ExplainCard, output_format: ExplainFormat) -> None:
if output_format is ExplainFormat.JSON:
emit_json(card.model_dump(mode="json"))
return
if output_format is ExplainFormat.MARKDOWN:
print(_card_markdown(card))
return
print(_card_compact(card))
def _emit_cards(cards: list[ExplainCard], output_format: ExplainFormat) -> None:
if output_format is ExplainFormat.JSON:
emit_json({"entries": [card.model_dump(mode="json") for card in cards]})
return
if output_format is ExplainFormat.MARKDOWN:
print("\n\n".join(_card_markdown(card) for card in cards))
return
print("\n".join(_card_compact(card) for card in cards))
def _emit_summaries(
summaries: list[ExplainSummary],
output_format: ExplainFormat,
) -> None:
if output_format is ExplainFormat.JSON:
emit_json(
{"entries": [summary.model_dump(mode="json") for summary in summaries]}
)
return
if output_format is ExplainFormat.MARKDOWN:
print("\n".join(f"- `{item.code}`: {item.summary}" for item in summaries))
return
print("\n".join(f"{item.code}: {item.summary}" for item in summaries))
def _card_markdown(card: ExplainCard) -> str:
lines = [
f"# {card.code}",
"",
card.summary,
"",
"## Why It Happens",
*[f"- {item}" for item in card.why_it_happens],
"",
"## How To Fix",
*[f"- {item}" for item in card.how_to_fix],
]
if card.related_docs:
lines.extend(["", "## Related Docs", *[f"- {item}" for item in card.related_docs]])
return "\n".join(lines)
def _card_compact(card: ExplainCard) -> str:
return f"{card.code}: {card.summary}"
def _error_message(exc: Exception) -> str:
if isinstance(exc, UnknownExplainCode):
return f"unknown explain code: {exc.args[0]}"
return str(exc)
```
Then modify `src/wf_cli/app.py` so `explain` is registered as a direct command,
not as a Typer sub-app:
```python
app.add_typer(caps.app, name="cap")
app.add_typer(drafts.app, name="draft")
app.add_typer(artifacts.app, name="artifact")
app.add_typer(deployments.app, name="deploy")
app.add_typer(runs.app, name="run")
app.add_typer(docs.app, name="docs")
app.add_typer(schema.app, name="schema")
app.command("explain")(explain.explain_command)
```
Reason: `wf explain <code> --format markdown` is a single command with options,
not a command group. The Typer sub-app callback pattern with
`invoke_without_command=True` can misparse callback options as subcommand names
when registered via `add_typer()`.
- [ ] **Step 4: Run CLI tests**
Run:
```bash
uv run pytest tests/wf_cli/test_explain.py tests/wf_cli/test_app.py -q
```
Expected: pass.
## Task 4: Verify Focused Slice
**Files:**
- No new files unless formatting changes are required.
- [ ] **Step 1: Run focused CLI tests**
Run:
```bash
uv run pytest tests/wf_cli -q
```
Expected: all `tests/wf_cli` tests pass.
- [ ] **Step 2: Run focused lint**
Run:
```bash
uv run ruff check src/wf_cli tests/wf_cli
```
Expected: no lint errors.
- [ ] **Step 3: Run focused format check**
Run:
```bash
uv run ruff format --check src/wf_cli tests/wf_cli
```
Expected: no formatting changes required. If this fails, run:
```bash
uv run ruff format src/wf_cli tests/wf_cli
```
Then rerun the format check.
- [ ] **Step 4: Run type check**
Run:
```bash
uv run basedpyright --level error
```
Expected: `0 errors`.
## Self-Review Checklist
- [ ] `wf explain --list` returns lean summaries, not full cards.
- [ ] `wf explain <code>` returns a full card.
- [ ] `wf explain --input-file` and `wf explain --stdin` support multiple cards.
- [ ] Unknown codes fail clearly and include the unknown code string.
- [ ] No fuzzy matching or generated prose was added.
- [ ] No run/deploy behavior changed.
- [ ] The new package has docstrings around the registry/parser boundary.
@@ -0,0 +1,715 @@
# wf CLI Foundation Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add the `wf_cli` package, Typer entrypoint, shared CLI context/config loader, and JSON IO helpers without implementing workflow commands yet.
**Architecture:** Create a protocol-neutral `wf_cli` package as a second front door beside `wf_mcp`. The first slice wires Typer command groups and reusable helpers only; later plans will add `deploy/run`, `explain`, and draft authoring commands. The CLI may construct `wf_mcp` service/handler objects through `wf_cli.context` for v1, but command modules should not directly own MCP-specific setup.
**Tech Stack:** Python 3.14, Typer, Pydantic v2, pytest, ruff, basedpyright.
---
## File Structure
- Modify `pyproject.toml`
- Add `typer>=0.16`.
- Add script entrypoint `wf = "wf_cli.app:main"`.
- Create `src/wf_cli/__init__.py`
- Minimal package marker and future exports.
- Create `src/wf_cli/app.py`
- Owns the Typer root app.
- Registers lifecycle command groups.
- Exposes `main()`.
- Create `src/wf_cli/context.py`
- Owns config loading and service/handler construction.
- Reuses `wf_mcp.broker.load_broker_config` and `build_service_from_config` for v1.
- Create `src/wf_cli/io.py`
- Owns JSON input parsing from inline JSON and files.
- Owns JSON output formatting.
- Provides a small `CliInputError` for bad CLI payloads.
- Create `src/wf_cli/commands/__init__.py`
- Exports command group apps.
- Create command modules:
- `src/wf_cli/commands/caps.py`
- `src/wf_cli/commands/drafts.py`
- `src/wf_cli/commands/artifacts.py`
- `src/wf_cli/commands/deployments.py`
- `src/wf_cli/commands/runs.py`
- `src/wf_cli/commands/docs.py`
- `src/wf_cli/commands/schema.py`
- `src/wf_cli/commands/explain.py`
- Create tests:
- `tests/wf_cli/test_app.py`
- `tests/wf_cli/test_context.py`
- `tests/wf_cli/test_io.py`
## Scope Boundaries
- Do not implement real workflow commands in this slice.
- Do not duplicate MCP workflow logic.
- Do not add draft mutation helpers yet.
- Do not add `wf explain` registry entries yet.
- Do not add subprocess CLI tests yet; use Typer `CliRunner` and direct function tests.
---
### Task 1: Add Typer Dependency And Script
**Files:**
- Modify: `pyproject.toml`
- [ ] **Step 1: Add dependency and entrypoint**
In `pyproject.toml`, add `typer>=0.16` to `[project].dependencies`:
```toml
dependencies = [
"fastmcp>=3.2.4",
"httpx>=0.28",
"jsonpatch>=1.33",
"jsonschema>=4.26",
"mcp[cli,rich]>=1",
"openapi-core>=0.19",
"pydantic>=2",
"typer>=0.16",
]
```
In `[project.scripts]`, add `wf` while preserving `wf-mcp`:
```toml
[project.scripts]
wf = "wf_cli.app:main"
wf-mcp = "wf_mcp.cli:main"
```
- [ ] **Step 2: Sync dependencies if needed**
Run:
```bash
uv lock
```
Expected: `uv.lock` updates if Typer is not already present transitively.
If `uv lock` cannot access the network, stop and report the dependency-lock blocker. Do not manually edit `uv.lock`.
---
### Task 2: Add App Skeleton Tests
**Files:**
- Create: `tests/wf_cli/test_app.py`
- [ ] **Step 1: Write failing Typer app tests**
Create `tests/wf_cli/test_app.py`:
```python
from __future__ import annotations
from typer.testing import CliRunner
from wf_cli.app import app
runner = CliRunner()
def test_wf_help_lists_lifecycle_groups() -> None:
result = runner.invoke(app, ["--help"])
assert result.exit_code == 0
assert "cap" in result.output
assert "draft" in result.output
assert "artifact" in result.output
assert "deploy" in result.output
assert "run" in result.output
assert "schema" in result.output
assert "explain" in result.output
def test_wf_run_group_help_exists() -> None:
result = runner.invoke(app, ["run", "--help"])
assert result.exit_code == 0
assert "Run workflow deployments" in result.output
```
- [ ] **Step 2: Run tests to verify they fail**
Run:
```bash
uv run pytest tests/wf_cli/test_app.py -q
```
Expected: FAIL because `wf_cli` does not exist.
---
### Task 3: Create Typer App And Command Groups
**Files:**
- Create: `src/wf_cli/__init__.py`
- Create: `src/wf_cli/app.py`
- Create: `src/wf_cli/commands/__init__.py`
- Create: `src/wf_cli/commands/caps.py`
- Create: `src/wf_cli/commands/drafts.py`
- Create: `src/wf_cli/commands/artifacts.py`
- Create: `src/wf_cli/commands/deployments.py`
- Create: `src/wf_cli/commands/runs.py`
- Create: `src/wf_cli/commands/docs.py`
- Create: `src/wf_cli/commands/schema.py`
- Create: `src/wf_cli/commands/explain.py`
- [ ] **Step 1: Create package marker**
Create `src/wf_cli/__init__.py`:
```python
"""Workflow platform command-line interface."""
```
- [ ] **Step 2: Create command group modules**
Create `src/wf_cli/commands/caps.py`:
```python
from __future__ import annotations
import typer
app = typer.Typer(
name="cap",
help="Inspect and call workflow capabilities.",
no_args_is_help=True,
)
```
Create `src/wf_cli/commands/drafts.py`:
```python
from __future__ import annotations
import typer
app = typer.Typer(
name="draft",
help="Create, inspect, patch, validate, and save draft workflows.",
no_args_is_help=True,
)
```
Create `src/wf_cli/commands/artifacts.py`:
```python
from __future__ import annotations
import typer
app = typer.Typer(
name="artifact",
help="List and inspect saved workflow artifacts.",
no_args_is_help=True,
)
```
Create `src/wf_cli/commands/deployments.py`:
```python
from __future__ import annotations
import typer
app = typer.Typer(
name="deploy",
help="Save, inspect, validate, and delete workflow deployments.",
no_args_is_help=True,
)
```
Create `src/wf_cli/commands/runs.py`:
```python
from __future__ import annotations
import typer
app = typer.Typer(
name="run",
help="Run workflow deployments and inspect durable runs.",
no_args_is_help=True,
)
```
Create `src/wf_cli/commands/docs.py`:
```python
from __future__ import annotations
import typer
app = typer.Typer(
name="docs",
help="List and read workflow documentation resources.",
no_args_is_help=True,
)
```
Create `src/wf_cli/commands/schema.py`:
```python
from __future__ import annotations
import typer
app = typer.Typer(
name="schema",
help="Print expected input shapes for wf commands.",
no_args_is_help=True,
)
```
Create `src/wf_cli/commands/explain.py`:
```python
from __future__ import annotations
import typer
app = typer.Typer(
name="explain",
help="Explain workflow diagnostic and CLI error codes.",
no_args_is_help=True,
)
```
- [ ] **Step 3: Create command package exports**
Create `src/wf_cli/commands/__init__.py`:
```python
"""Typer command groups for the wf CLI."""
from . import artifacts, caps, deployments, docs, drafts, explain, runs, schema
__all__ = [
"artifacts",
"caps",
"deployments",
"docs",
"drafts",
"explain",
"runs",
"schema",
]
```
- [ ] **Step 4: Create root app**
Create `src/wf_cli/app.py`:
```python
from __future__ import annotations
from typing import Annotated
import typer
from .commands import artifacts, caps, deployments, docs, drafts, explain, runs, schema
app = typer.Typer(
name="wf",
help="Workflow platform CLI.",
no_args_is_help=True,
)
@app.callback()
def root(
config: Annotated[
str,
typer.Option(
"--config",
help="Path to workflow/MCP config JSON.",
),
] = "wf_mcp.config.json",
) -> None:
"""Run workflow platform commands."""
# The root callback owns global options only. Command modules should load
# context explicitly so tests can call command functions without Typer state.
_ = config
app.add_typer(caps.app, name="cap")
app.add_typer(drafts.app, name="draft")
app.add_typer(artifacts.app, name="artifact")
app.add_typer(deployments.app, name="deploy")
app.add_typer(runs.app, name="run")
app.add_typer(docs.app, name="docs")
app.add_typer(schema.app, name="schema")
app.add_typer(explain.app, name="explain")
def main() -> None:
"""Console script entrypoint for `wf`."""
app()
```
- [ ] **Step 5: Run app tests**
Run:
```bash
uv run pytest tests/wf_cli/test_app.py -q
```
Expected: PASS.
---
### Task 4: Add JSON IO Tests
**Files:**
- Create: `tests/wf_cli/test_io.py`
- [ ] **Step 1: Write failing IO tests**
Create `tests/wf_cli/test_io.py`:
```python
from __future__ import annotations
import json
import pytest
from wf_cli.io import CliInputError, emit_json, parse_json_input
def test_parse_json_input_reads_inline_json() -> None:
payload = parse_json_input(input_json='{"text": "hello"}', input_file=None)
assert payload["text"] == "hello"
def test_parse_json_input_reads_file(tmp_path) -> None:
path = tmp_path / "payload.json"
path.write_text('{"text": "from file"}', encoding="utf-8")
payload = parse_json_input(input_json=None, input_file=path)
assert payload["text"] == "from file"
def test_parse_json_input_rejects_both_inline_and_file(tmp_path) -> None:
path = tmp_path / "payload.json"
path.write_text("{}", encoding="utf-8")
with pytest.raises(CliInputError, match="mutually exclusive"):
parse_json_input(input_json="{}", input_file=path)
def test_parse_json_input_rejects_invalid_json() -> None:
with pytest.raises(CliInputError, match="invalid JSON"):
parse_json_input(input_json="{", input_file=None)
def test_emit_json_writes_pretty_json(capsys) -> None:
emit_json({"ok": True, "items": [1]})
captured = capsys.readouterr()
payload = json.loads(captured.out)
assert payload["ok"] is True
assert payload["items"][0] == 1
```
- [ ] **Step 2: Run tests to verify they fail**
Run:
```bash
uv run pytest tests/wf_cli/test_io.py -q
```
Expected: FAIL because `wf_cli.io` does not exist.
---
### Task 5: Implement JSON IO Helpers
**Files:**
- Create: `src/wf_cli/io.py`
- [ ] **Step 1: Create IO helpers**
Create `src/wf_cli/io.py`:
```python
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
class CliInputError(ValueError):
"""Raised when CLI JSON/file input cannot be parsed safely."""
def parse_json_input(
*,
input_json: str | None,
input_file: Path | None,
) -> dict[str, Any]:
"""Parse exactly one JSON object from inline JSON or a file path."""
if input_json is not None and input_file is not None:
raise CliInputError("--input and --input-file are mutually exclusive")
if input_json is None and input_file is None:
return {}
raw = input_json if input_json is not None else _read_input_file(input_file)
try:
payload = json.loads(raw)
except json.JSONDecodeError as exc:
raise CliInputError(f"invalid JSON input: {exc.msg}") from exc
if not isinstance(payload, dict):
raise CliInputError("JSON input must be an object")
return payload
def emit_json(payload: Any) -> None:
"""Write JSON output in the CLI default machine-readable format."""
print(json.dumps(payload, indent=2, sort_keys=True))
def _read_input_file(path: Path | None) -> str:
"""Read a required JSON input file."""
if path is None:
raise CliInputError("input file path is required")
try:
return path.read_text(encoding="utf-8")
except OSError as exc:
raise CliInputError(f"could not read input file {path!s}: {exc}") from exc
```
- [ ] **Step 2: Run IO tests**
Run:
```bash
uv run pytest tests/wf_cli/test_io.py -q
```
Expected: PASS.
---
### Task 6: Add CLI Context Tests
**Files:**
- Create: `tests/wf_cli/test_context.py`
- [ ] **Step 1: Write failing context tests**
Create `tests/wf_cli/test_context.py`:
```python
from __future__ import annotations
import json
from wf_cli.context import load_cli_context
from tests.wf_mcp.test_support import local_temp_root
def test_load_cli_context_builds_service_and_handlers() -> None:
tmp_path = local_temp_root() / "wf_cli_context"
tmp_path.mkdir(parents=True, exist_ok=True)
config_path = tmp_path / "wf_mcp.config.json"
config_path.write_text(
json.dumps(
{
"store_root": ".wf_mcp_store",
"connections": [
{
"id": "demo.personal",
"server": "demo",
"account": "personal",
}
],
}
),
encoding="utf-8",
)
context = load_cli_context(config_path)
assert context.config_path == config_path
assert context.service.connections.list_all()[0].id == "demo.personal"
assert context.handlers.service is context.service
```
- [ ] **Step 2: Run test to verify it fails**
Run:
```bash
uv run pytest tests/wf_cli/test_context.py -q
```
Expected: FAIL because `wf_cli.context` does not exist.
---
### Task 7: Implement CLI Context Loader
**Files:**
- Create: `src/wf_cli/context.py`
- [ ] **Step 1: Create context loader**
Create `src/wf_cli/context.py`:
```python
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from wf_mcp.broker import build_service_from_config, load_broker_config
from wf_mcp.broker.service import WfMcpService
from wf_mcp.workflow_surface import WorkflowSurfaceHandlers
@dataclass(frozen=True)
class CliContext:
"""Protocol-neutral CLI handle over the current workflow service stack.
V1 intentionally reuses wf_mcp service construction because that is where
config, store, source, artifact, draft, and run wiring currently lives. Keep
this dependency behind context.py so later extraction does not affect every
command module.
"""
config_path: Path
service: WfMcpService
handlers: WorkflowSurfaceHandlers
def load_cli_context(config_path: str | Path) -> CliContext:
"""Load config and build workflow-surface handlers for CLI commands."""
resolved_config_path = Path(config_path)
config = load_broker_config(resolved_config_path)
service = build_service_from_config(config)
return CliContext(
config_path=resolved_config_path,
service=service,
handlers=WorkflowSurfaceHandlers(service),
)
```
- [ ] **Step 2: Run context tests**
Run:
```bash
uv run pytest tests/wf_cli/test_context.py -q
```
Expected: PASS.
---
### Task 8: Run Foundation Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused CLI tests**
Run:
```bash
uv run pytest tests/wf_cli/test_app.py tests/wf_cli/test_io.py tests/wf_cli/test_context.py -q
```
Expected: PASS.
- [ ] **Step 2: Run Typer help manually through uv**
Run:
```bash
uv run wf --help
```
Expected: exit 0 and output lists lifecycle groups including `deploy`, `run`, `draft`, and `explain`.
- [ ] **Step 3: Run lint on touched files**
Run:
```bash
uv run ruff check src/wf_cli tests/wf_cli
```
Expected: PASS.
- [ ] **Step 4: Run format check on touched files**
Run:
```bash
uv run ruff format --check src/wf_cli tests/wf_cli
```
Expected: PASS.
- [ ] **Step 5: Run type check**
Run:
```bash
uv run basedpyright --level error
```
Expected: `0 errors`.
---
## Self-Review Checklist
- `wf_cli` is a new package, not under `wf_mcp`.
- `wf` script exists and `wf-mcp` still exists.
- Typer is the only CLI framework used in `wf_cli`.
- Command groups exist but do not pretend to implement workflow behavior.
- Shared config/service construction lives in `wf_cli.context`.
- JSON parsing/printing lives in `wf_cli.io`.
- No workflow logic is duplicated from MCP handlers.
- No app-domain command groups (`scenario`, `risk`, `decision`, etc.) were added.
## Notes For Opencode
- This is a foundation slice. Do not implement `deploy validate` or `run start` here.
- If Typer dependency locking fails because of network access, stop and report it.
- Keep command modules boring and small.
- Do not move stores out of `wf_mcp` yet; only hide the dependency behind `wf_cli.context`.
@@ -0,0 +1,678 @@
# wf CLI Run And Deploy Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Implement the first useful `wf` CLI vertical slice: `wf deploy validate`, `wf run start`, `wf run inspect`, and `wf run trace`.
**Architecture:** Keep command modules thin Typer wrappers over `WorkflowSurfaceHandlers` loaded through `wf_cli.context`. Use real file-backed stores in tests so CLI behavior matches the MCP workflow surface. Keep JSON output as the default and reuse `wf_cli.io` for payload parsing/printing.
**Tech Stack:** Python 3.14, Typer, Pydantic v2, pytest, ruff, basedpyright.
---
## File Structure
- Modify `src/wf_cli/app.py`
- Store root `--config` in Typer context so subcommands can load the requested config.
- Modify `src/wf_cli/context.py`
- Add `config_path_from_context(ctx)` helper.
- Modify `src/wf_cli/commands/deployments.py`
- Add `validate` command.
- Modify `src/wf_cli/commands/runs.py`
- Add `start`, `inspect`, and `trace` commands.
- Create `tests/wf_cli/test_run_deploy.py`
- End-to-end CLI tests using `CliRunner`.
- Seed artifact/deployment data into the configured store.
## Scope Boundaries
- Do not implement `wf deploy save`, `wf deploy delete`, or `wf deploy inspect`.
- Do not implement `wf run resume`.
- Do not implement `wf cap`, `wf draft`, `wf explain`, or `wf schema`.
- Do not add table/ids/compact formatting.
- Do not add stdin input yet; use `--input` and `--input-file`.
- Do not mock `WorkflowSurfaceHandlers`; use the real service/store path.
---
### Task 1: Make Global `--config` Available To Commands
**Files:**
- Modify: `src/wf_cli/app.py`
- Modify: `src/wf_cli/context.py`
- Modify: `tests/wf_cli/test_app.py`
- [ ] **Step 1: Add a failing test for config propagation**
Append to `tests/wf_cli/test_app.py`:
```python
def test_root_callback_stores_config_path() -> None:
result = runner.invoke(app, ["--config", "custom.json", "run", "--help"])
assert result.exit_code == 0
assert "Run workflow deployments" in result.output
```
This mostly locks down that root `--config` remains accepted before subcommands.
- [ ] **Step 2: Update root callback to use Typer context**
In `src/wf_cli/app.py`, change the callback signature to:
```python
@app.callback()
def root(
ctx: typer.Context,
config: Annotated[
str,
typer.Option(
"--config",
help="Path to workflow/MCP config JSON.",
),
] = "wf_mcp.config.json",
) -> None:
"""Run workflow platform commands."""
ctx.obj = {"config_path": config}
```
Keep `import typer` already present.
- [ ] **Step 3: Add config helper**
In `src/wf_cli/context.py`, add:
```python
import typer
```
Then add below `CliContext`:
```python
def config_path_from_context(ctx: typer.Context) -> str:
"""Return the root --config path captured by the Typer callback."""
obj = ctx.obj if isinstance(ctx.obj, dict) else {}
value = obj.get("config_path", "wf_mcp.config.json")
return value if isinstance(value, str) else "wf_mcp.config.json"
```
- [ ] **Step 4: Run app/context tests**
Run:
```bash
uv run pytest tests/wf_cli/test_app.py tests/wf_cli/test_context.py -q
```
Expected: PASS.
---
### Task 2: Add CLI Run/Deploy Test Fixture Helpers
**Files:**
- Create: `tests/wf_cli/test_run_deploy.py`
- [ ] **Step 1: Create fixture helpers**
Create `tests/wf_cli/test_run_deploy.py` with this opening:
```python
from __future__ import annotations
import json
from pathlib import Path
from typer.testing import CliRunner
from wf_artifacts import FileWorkflowArtifactStore, WorkflowDeployment
from wf_cli.app import app
from wf_mcp.models import ConnectionConfig
from tests.wf_mcp.test_support import echo_tool, local_temp_root
from tests.wf_mcp.workflow_surface.conftest import echo_artifact
runner = CliRunner()
def _write_config(root: Path) -> Path:
config_path = root / "wf_mcp.config.json"
config_path.write_text(
json.dumps(
{
"store_root": ".wf_mcp_store",
"connections": [
{
"id": "demo.personal",
"server": "demo",
"account": "personal",
}
],
}
),
encoding="utf-8",
)
return config_path
def _seed_echo_deployment(root: Path) -> Path:
config_path = _write_config(root)
store_root = root / ".wf_mcp_store"
artifact_store = FileWorkflowArtifactStore(store_root)
artifact_store.save_artifact(echo_artifact())
artifact_store.save_deployment(
WorkflowDeployment(
id="echo.personal",
artifact_id="echo",
artifact_version=1,
bindings=[{"logical_source": "demo", "concrete_source": "demo.personal"}],
)
)
return config_path
```
This fixture writes the same config/store shape that `wf_cli.context` loads.
- [ ] **Step 2: Add failing deploy validate test**
Append:
```python
def test_wf_deploy_validate_outputs_json() -> None:
root = local_temp_root() / "wf_cli_deploy_validate"
root.mkdir(parents=True, exist_ok=True)
config_path = _seed_echo_deployment(root)
result = runner.invoke(
app,
["--config", str(config_path), "deploy", "validate", "echo.personal"],
)
assert result.exit_code == 0
payload = json.loads(result.output)
assert payload["deployment_id"] == "echo.personal"
assert payload["status"] == "runnable"
assert payload["next_actions"]["recommended_next_tool"] == (
"wf.workflow.run_deployment"
)
```
- [ ] **Step 3: Run test to verify it fails**
Run:
```bash
uv run pytest tests/wf_cli/test_run_deploy.py::test_wf_deploy_validate_outputs_json -q
```
Expected: FAIL because `deploy validate` does not exist.
---
### Task 3: Implement `wf deploy validate`
**Files:**
- Modify: `src/wf_cli/commands/deployments.py`
- [ ] **Step 1: Add imports and command**
Replace `src/wf_cli/commands/deployments.py` with:
```python
from __future__ import annotations
import asyncio
from typing import Annotated
import typer
from wf_cli.context import config_path_from_context, load_cli_context
from wf_cli.io import emit_json
app = typer.Typer(
name="deploy",
help="Save, inspect, validate, and delete workflow deployments.",
no_args_is_help=True,
)
@app.command("validate")
def validate_deployment(
ctx: typer.Context,
deployment_id: Annotated[str, typer.Argument(help="Deployment id to validate.")],
live: Annotated[
bool,
typer.Option(
"--live",
help="Also perform opt-in upstream liveness checks.",
),
] = False,
) -> None:
"""Validate one saved workflow deployment."""
context = load_cli_context(config_path_from_context(ctx))
payload = asyncio.run(
context.handlers.validate_deployment(
deployment_id=deployment_id,
live_check=live,
)
)
emit_json(payload)
```
- [ ] **Step 2: Run deploy test**
Run:
```bash
uv run pytest tests/wf_cli/test_run_deploy.py::test_wf_deploy_validate_outputs_json -q
```
Expected: PASS.
---
### Task 4: Add Run Command Tests
**Files:**
- Modify: `tests/wf_cli/test_run_deploy.py`
- [ ] **Step 1: Add run start test**
Append:
```python
def test_wf_run_start_accepts_inline_json_input() -> None:
root = local_temp_root() / "wf_cli_run_start"
root.mkdir(parents=True, exist_ok=True)
config_path = _seed_echo_deployment(root)
result = runner.invoke(
app,
[
"--config",
str(config_path),
"run",
"start",
"echo.personal",
"--input",
'{"text": "hello"}',
],
)
assert result.exit_code == 0
payload = json.loads(result.output)
assert payload["status"] == "completed"
assert payload["output"]["echoed"] == "hello"
assert isinstance(payload["run_id"], str)
assert payload["next_actions"]["can_continue"] is False
```
- [ ] **Step 2: Add run start file input test**
Append:
```python
def test_wf_run_start_accepts_input_file() -> None:
root = local_temp_root() / "wf_cli_run_start_file"
root.mkdir(parents=True, exist_ok=True)
config_path = _seed_echo_deployment(root)
input_path = root / "input.json"
input_path.write_text('{"text": "from file"}', encoding="utf-8")
result = runner.invoke(
app,
[
"--config",
str(config_path),
"run",
"start",
"echo.personal",
"--input-file",
str(input_path),
],
)
assert result.exit_code == 0
payload = json.loads(result.output)
assert payload["status"] == "completed"
assert payload["output"]["echoed"] == "from file"
```
- [ ] **Step 3: Add inspect and trace test**
Append:
```python
def test_wf_run_inspect_and_trace_existing_run() -> None:
root = local_temp_root() / "wf_cli_run_inspect_trace"
root.mkdir(parents=True, exist_ok=True)
config_path = _seed_echo_deployment(root)
start = runner.invoke(
app,
[
"--config",
str(config_path),
"run",
"start",
"echo.personal",
"--input",
'{"text": "hello"}',
],
)
run_id = json.loads(start.output)["run_id"]
inspected = runner.invoke(
app,
["--config", str(config_path), "run", "inspect", run_id],
)
traced = runner.invoke(
app,
[
"--config",
str(config_path),
"run",
"trace",
run_id,
"--from",
"0",
"--limit",
"1",
],
)
assert inspected.exit_code == 0
inspected_payload = json.loads(inspected.output)
assert inspected_payload["run_id"] == run_id
assert inspected_payload["status"] == "completed"
assert "trace" not in inspected_payload
assert traced.exit_code == 0
traced_payload = json.loads(traced.output)
assert traced_payload["run_id"] == run_id
assert traced_payload["trace_start"] == 0
assert traced_payload["trace_limit"] == 1
assert traced_payload["trace"][0]["node_id"] == "echo"
```
- [ ] **Step 4: Run tests to verify they fail**
Run:
```bash
uv run pytest tests/wf_cli/test_run_deploy.py -q
```
Expected: deploy test passes, run tests fail because run commands do not exist.
---
### Task 5: Implement Run Commands
**Files:**
- Modify: `src/wf_cli/commands/runs.py`
- [ ] **Step 1: Replace run command module**
Replace `src/wf_cli/commands/runs.py` with:
```python
from __future__ import annotations
import asyncio
from pathlib import Path
from typing import Annotated
import typer
from wf_cli.context import config_path_from_context, load_cli_context
from wf_cli.io import CliInputError, emit_json, parse_json_input
from wf_mcp.workflow_surface import TraceRange
app = typer.Typer(
name="run",
help="Run workflow deployments and inspect durable runs.",
no_args_is_help=True,
)
@app.command("start")
def start_run(
ctx: typer.Context,
deployment_id: Annotated[str, typer.Argument(help="Deployment id to run.")],
input_json: Annotated[
str | None,
typer.Option("--input", help="Workflow input JSON object."),
] = None,
input_file: Annotated[
Path | None,
typer.Option("--input-file", help="Path to workflow input JSON object."),
] = None,
trace_from: Annotated[
int | None,
typer.Option("--trace-from", min=0, help="Optional trace slice start."),
] = None,
trace_limit: Annotated[
int | None,
typer.Option("--trace-limit", min=1, max=100, help="Optional trace slice limit."),
] = None,
) -> None:
"""Start one workflow deployment."""
try:
workflow_input = parse_json_input(input_json=input_json, input_file=input_file)
except CliInputError as exc:
raise typer.BadParameter(str(exc)) from exc
context = load_cli_context(config_path_from_context(ctx))
trace_range = _optional_trace_range(start=trace_from, limit=trace_limit)
payload = asyncio.run(
context.handlers.run_deployment(
deployment_id=deployment_id,
workflow_input=workflow_input,
trace_range=trace_range,
)
)
emit_json(payload)
@app.command("inspect")
def inspect_run(
ctx: typer.Context,
run_id: Annotated[str, typer.Argument(help="Durable run id to inspect.")],
) -> None:
"""Inspect a durable run without trace entries."""
context = load_cli_context(config_path_from_context(ctx))
emit_json(asyncio.run(context.handlers.inspect_run(run_id=run_id)))
@app.command("trace")
def trace_run(
ctx: typer.Context,
run_id: Annotated[str, typer.Argument(help="Durable run id to trace.")],
trace_from: Annotated[
int,
typer.Option("--from", min=0, help="Zero-based trace start offset."),
] = 0,
limit: Annotated[
int,
typer.Option("--limit", min=1, max=100, help="Maximum trace entries."),
] = 25,
) -> None:
"""Read a bounded debug trace slice."""
context = load_cli_context(config_path_from_context(ctx))
payload = asyncio.run(
context.handlers.read_run_trace(
run_id=run_id,
trace_range=TraceRange(start=trace_from, limit=limit),
)
)
emit_json(payload)
def _optional_trace_range(*, start: int | None, limit: int | None) -> TraceRange | None:
"""Build a trace range only when the caller requested trace detail."""
if start is None and limit is None:
return None
return TraceRange(start=start or 0, limit=limit or 25)
```
- [ ] **Step 2: Run CLI run/deploy tests**
Run:
```bash
uv run pytest tests/wf_cli/test_run_deploy.py -q
```
Expected: PASS.
---
### Task 6: Verify Help And Input Error Behavior
**Files:**
- Modify: `tests/wf_cli/test_app.py`
- Modify: `tests/wf_cli/test_run_deploy.py`
- [ ] **Step 1: Add help assertions**
Append to `tests/wf_cli/test_app.py`:
```python
def test_wf_deploy_validate_help_exists() -> None:
result = runner.invoke(app, ["deploy", "validate", "--help"])
assert result.exit_code == 0
assert "Validate one saved workflow deployment" in result.output
def test_wf_run_start_help_exists() -> None:
result = runner.invoke(app, ["run", "start", "--help"])
assert result.exit_code == 0
assert "--input-file" in result.output
```
- [ ] **Step 2: Add bad JSON assertion**
Append to `tests/wf_cli/test_run_deploy.py`:
```python
def test_wf_run_start_reports_bad_json() -> None:
root = local_temp_root() / "wf_cli_run_bad_json"
root.mkdir(parents=True, exist_ok=True)
config_path = _seed_echo_deployment(root)
result = runner.invoke(
app,
[
"--config",
str(config_path),
"run",
"start",
"echo.personal",
"--input",
"{",
],
)
assert result.exit_code != 0
assert "invalid JSON" in result.output
```
- [ ] **Step 3: Run tests**
Run:
```bash
uv run pytest tests/wf_cli/test_app.py tests/wf_cli/test_run_deploy.py -q
```
Expected: PASS.
---
### Task 7: Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run all CLI tests**
Run:
```bash
uv run pytest tests/wf_cli -q
```
Expected: PASS.
- [ ] **Step 2: Run focused workflow surface tests**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface/test_deployments.py tests/wf_mcp/workflow_surface/test_runs.py -q
```
Expected: PASS.
- [ ] **Step 3: Run lint**
Run:
```bash
uv run ruff check src/wf_cli tests/wf_cli
```
Expected: PASS.
- [ ] **Step 4: Run format check**
Run:
```bash
uv run ruff format --check src/wf_cli tests/wf_cli
```
Expected: PASS.
- [ ] **Step 5: Run type check**
Run:
```bash
uv run basedpyright --level error
```
Expected: `0 errors`.
---
## Self-Review Checklist
- `wf deploy validate` returns handler JSON with `next_actions`.
- `wf run start` accepts `--input` and `--input-file`.
- `wf run inspect` omits trace entries.
- `wf run trace` requires bounded `--from` / `--limit`.
- Commands load the root `--config` path.
- Command modules stay thin and do not duplicate workflow logic.
- No unrelated CLI groups or draft authoring commands were implemented.
## Notes For Opencode
- Keep this to run/deploy commands only.
- Do not implement `wf run resume` in this slice.
- Do not add `--format`; JSON stays default.
- Use real file-backed stores in tests.
- If Typer callback context is annoying, keep the smallest helper in `wf_cli.context` rather than passing global state.
@@ -0,0 +1,761 @@
# wf_api Remove Double Delegation Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Collapse workflow calls from `WorkflowApi -> WfMcpWorkflowApiBackend -> WorkflowSurfaceHandlers -> domain services` to `WorkflowApi -> domain services`.
**Architecture:** `WorkflowApi` becomes the protocol-neutral application facade that composes `WorkflowCapabilityApi`, `WorkflowDraftApi`, `WorkflowArtifactApi`, `WorkflowDeploymentApi`, and `WorkflowRunApi` from a `WorkflowOperationContext`. MCP and CLI construct `WorkflowApi(context_from_service(service))` directly. `WorkflowSurfaceHandlers` remains only as a temporary compatibility shim for legacy imports/tests, and `WfMcpWorkflowApiBackend` / `WorkflowApiBackend` are removed.
**Tech Stack:** Python 3.14, `wf_api`, `wf_mcp`, dataclasses, pytest, ruff, basedpyright.
---
## Current Chain
```text
wf_mcp tools / wf_cli
-> WorkflowApi
-> WfMcpWorkflowApiBackend
-> WorkflowSurfaceHandlers
-> WorkflowCapabilityApi / WorkflowDraftApi / WorkflowArtifactApi / WorkflowDeploymentApi / WorkflowRunApi
```
This creates two mechanical delegation layers. Adding one workflow operation currently requires touching at least `WorkflowApi`, `WorkflowApiBackend`, `WfMcpWorkflowApiBackend`, and usually `WorkflowSurfaceHandlers`.
## Target Chain
```text
wf_mcp tools / wf_cli
-> WorkflowApi
-> WorkflowCapabilityApi / WorkflowDraftApi / WorkflowArtifactApi / WorkflowDeploymentApi / WorkflowRunApi
```
Legacy imports of `WorkflowSurfaceHandlers` may still work, but only as a thin wrapper around `WorkflowApi`.
## Files
- Modify: `src/wf_api/models.py`
- Modify: `src/wf_api/__init__.py`
- Modify: `src/wf_api/service.py`
- Delete: `src/wf_api/backend.py`
- Delete: `src/wf_mcp/broker/service/workflow_api_backend.py`
- Modify: `src/wf_cli/context.py`
- Modify: `src/wf_cli/commands/runs.py`
- Modify: `src/wf_mcp/workflow_surface/tools.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `tests/wf_cli/test_context.py`
- Modify: `tests/wf_api/test_cli_context_uses_api.py`
- Add: `tests/wf_api/test_direct_service.py`
- Add: `tests/wf_api/test_no_double_delegation.py`
- Modify docs: `docs/current_roadmap.md`, `docs/wf_mcp_architecture.md`, `docs/superpowers/plans/2026-06-01-wf-api-extraction-roadmap.md`
---
### Task 1: Move TraceRange Out of `backend.py`
**Files:**
- Modify: `src/wf_api/models.py`
- Modify: `src/wf_api/__init__.py`
- Modify: `src/wf_cli/commands/runs.py`
- Modify: `src/wf_mcp/workflow_surface/tools.py`
- [ ] **Step 1: Write import regression test**
Add to `tests/wf_api/test_raw_workflow_plan_extraction.py`:
```python
def test_trace_range_exports_from_wf_api_models() -> None:
from wf_api import TraceRange
from wf_api.models import TraceRange as CanonicalTraceRange
assert TraceRange is CanonicalTraceRange
assert TraceRange(start=1, limit=2).start == 1
assert TraceRange(start=1, limit=2).limit == 2
```
- [ ] **Step 2: Run failing test**
Run:
```bash
uv run pytest tests\wf_api\test_raw_workflow_plan_extraction.py::test_trace_range_exports_from_wf_api_models -q
```
Expected: fail because `wf_api.models.TraceRange` does not exist yet.
- [ ] **Step 3: Add `TraceRange` to `wf_api.models`**
In `src/wf_api/models.py`, add imports:
```python
from dataclasses import dataclass
```
Then add before `RawWorkflowPlan`:
```python
@dataclass(frozen=True, slots=True)
class TraceRange:
"""Caller-bounded debug trace slice for durable deployment runs."""
start: int = 0
limit: int = 25
```
- [ ] **Step 4: Update `wf_api.__init__` export**
Change:
```python
from .backend import TraceRange, WorkflowApiBackend
```
to:
```python
from .models import RawWorkflowPlan, TraceRange
```
Remove `"WorkflowApiBackend"` from `__all__`. Keep `"TraceRange"`.
If `RawWorkflowPlan` was not exported before, include it only if already expected by tests; do not add new public API unless the existing file already imports it elsewhere.
- [ ] **Step 5: Update imports that referenced `wf_api.backend.TraceRange`**
In `src/wf_cli/commands/runs.py`, replace:
```python
from wf_api.backend import TraceRange
```
with:
```python
from wf_api import TraceRange
```
In `src/wf_mcp/workflow_surface/tools.py`, remove:
```python
from wf_api.backend import TraceRange as ApiTraceRange
```
Also remove `_to_api_trace_range()`. Later tasks pass MCP `TraceRange` directly because `WorkflowRunApi` validates trace ranges structurally through `TraceRangeLike`.
- [ ] **Step 6: Verify Task 1**
Run:
```bash
uv run pytest tests\wf_api\test_raw_workflow_plan_extraction.py::test_trace_range_exports_from_wf_api_models tests\wf_cli\test_run_deploy.py tests\wf_mcp\server\test_tools.py -q
uv run ruff check src\wf_api\models.py src\wf_api\__init__.py src\wf_cli\commands\runs.py src\wf_mcp\workflow_surface\tools.py tests\wf_api\test_raw_workflow_plan_extraction.py
uv run ruff format --check src\wf_api\models.py src\wf_api\__init__.py src\wf_cli\commands\runs.py src\wf_mcp\workflow_surface\tools.py tests\wf_api\test_raw_workflow_plan_extraction.py
```
Expected: tests pass, lint pass, format pass.
---
### Task 2: Make `WorkflowApi` Compose Domain Services Directly
**Files:**
- Modify: `src/wf_api/service.py`
- Add: `tests/wf_api/test_direct_service.py`
- [ ] **Step 1: Write direct-composition tests**
Create `tests/wf_api/test_direct_service.py`:
```python
from __future__ import annotations
import asyncio
from wf_artifacts import FileWorkflowArtifactStore
from wf_api import WorkflowApi
from wf_api.artifacts import WorkflowArtifactApi
from wf_api.capabilities import WorkflowCapabilityApi
from wf_api.deployments import WorkflowDeploymentApi
from wf_api.drafts import WorkflowDraftApi
from wf_api.runs import WorkflowRunApi
from wf_mcp.broker import WfMcpService
from wf_mcp.broker.service.workflow_operation_context import context_from_service
from wf_mcp.models import ConnectionConfig
from wf_mcp.storage import FileStore
from tests.wf_mcp.test_support import echo_tool, local_temp_root
def _api() -> WorkflowApi:
root = local_temp_root() / "wf_api_direct_composition"
service = WfMcpService(
store=FileStore(root / "mcp"),
artifact_store=FileWorkflowArtifactStore(root),
)
service.register_connection(
ConnectionConfig(id="demo.personal", server="demo", account="personal")
)
service.register_specs("demo.personal", echo_tool)
return WorkflowApi(context_from_service(service))
def test_workflow_api_composes_domain_services() -> None:
api = _api()
assert isinstance(api.capabilities, WorkflowCapabilityApi)
assert isinstance(api.drafts, WorkflowDraftApi)
assert isinstance(api.artifacts, WorkflowArtifactApi)
assert isinstance(api.deployments, WorkflowDeploymentApi)
assert isinstance(api.runs, WorkflowRunApi)
assert not hasattr(api, "backend")
def test_workflow_api_direct_capability_call() -> None:
api = _api()
result = asyncio.run(
api.call_capability(
qualified_name="demo.personal.echo_tool",
payload={"text": "hello"},
)
)
assert result["kind"] == "node_spec"
assert result["outcome"] == "ok"
assert result["output"] == {"echoed": "hello"}
```
- [ ] **Step 2: Run failing tests**
Run:
```bash
uv run pytest tests\wf_api\test_direct_service.py -q
```
Expected: fail because `WorkflowApi` still expects a `WorkflowApiBackend`.
- [ ] **Step 3: Rewrite `WorkflowApi.__init__`**
In `src/wf_api/service.py`, replace:
```python
from .backend import TraceRange, WorkflowApiBackend
```
with:
```python
from .artifacts import WorkflowArtifactApi
from .capabilities import WorkflowCapabilityApi
from .deployments import WorkflowDeploymentApi
from .drafts import WorkflowDraftApi
from .models import TraceRange
from .operation_context import WorkflowOperationContext
from .runs import TraceRangeLike, WorkflowRunApi
```
Replace the class docstring and constructor:
```python
class WorkflowApi:
"""Protocol-neutral workflow application facade.
This facade owns the stable application entry point. It composes the
domain APIs from a WorkflowOperationContext so MCP, CLI, and future HTTP
callers share one operation surface without importing wf_mcp.
"""
def __init__(self, context: WorkflowOperationContext) -> None:
self.context = context
self.capabilities = WorkflowCapabilityApi(context)
self.drafts = WorkflowDraftApi(context)
self.artifacts = WorkflowArtifactApi(context)
self.deployments = WorkflowDeploymentApi(context)
self.runs = WorkflowRunApi(context)
```
- [ ] **Step 4: Replace backend delegations with domain service delegations**
In `src/wf_api/service.py`, replace these groups:
Capabilities:
```python
self.backend.list_capabilities(...) -> self.capabilities.list_capabilities(...)
self.backend.inspect_capability(...) -> self.capabilities.inspect_capability(...)
self.backend.call_capability(...) -> self.capabilities.call_capability(...)
self.backend.create_draft_workspace_from_capability(...) -> self.capabilities.create_draft_workspace_from_capability(...)
```
Artifacts:
```python
self.backend.list_artifacts(...) -> self.artifacts.list_artifacts(...)
self.backend.inspect_artifact(...) -> self.artifacts.inspect_artifact(...)
self.backend.save_artifact(...) -> self.artifacts.save_artifact(...)
self.backend.create_artifact_from_plan(...) -> self.artifacts.create_artifact_from_plan(...)
self.backend.create_artifact_from_draft(...) -> self.artifacts.create_artifact_from_draft(...)
self.backend.create_artifact_from_workspace(...) -> self.artifacts.create_artifact_from_workspace(...)
self.backend.create_wrapper_from_workspace(...) -> self.artifacts.create_wrapper_from_workspace(...)
```
Drafts:
```python
self.backend.validate_draft(...) -> self.drafts.validate_draft(...)
self.backend.compile_draft(...) -> self.drafts.compile_draft(...)
self.backend.patch_draft(...) -> self.drafts.patch_draft(...)
self.backend.list_draft_workspaces() -> self.drafts.list_draft_workspaces()
self.backend.create_draft_workspace(...) -> self.drafts.create_draft_workspace(...)
self.backend.get_draft_workspace(...) -> self.drafts.get_draft_workspace(...)
self.backend.delete_draft_workspace(...) -> self.drafts.delete_draft_workspace(...)
self.backend.validate_draft_workspace(...) -> self.drafts.validate_draft_workspace(...)
self.backend.patch_draft_workspace(...) -> self.drafts.patch_draft_workspace(...)
self.backend.set_draft_name(...) -> self.drafts.set_draft_name(...)
self.backend.set_draft_route(...) -> self.drafts.set_draft_route(...)
self.backend.set_step_input_map(...) -> self.drafts.set_step_input_map(...)
self.backend.set_step_output_map(...) -> self.drafts.set_step_output_map(...)
self.backend.create_minimal_draft_workspace(...) -> self.drafts.create_minimal_draft_workspace(...)
```
Deployments:
```python
self.backend.list_deployments() -> self.deployments.list_deployments()
self.backend.inspect_deployment(...) -> self.deployments.inspect_deployment(...)
self.backend.save_deployment(...) -> self.deployments.save_deployment(...)
self.backend.delete_deployment(...) -> self.deployments.delete_deployment(...)
self.backend.validate_deployment(...) -> self.deployments.validate_deployment(...)
```
Runs:
```python
self.backend.run_deployment(...) -> self.runs.run_deployment(...)
self.backend.resume_run(...) -> self.runs.resume_run(...)
self.backend.inspect_run(...) -> self.runs.inspect_run(...)
self.backend.read_run_trace(...) -> self.runs.read_run_trace(...)
```
For run methods, change type hints from `TraceRange | None` to `TraceRangeLike | None` and `TraceRange` to `TraceRangeLike` so MCP Pydantic `TraceRange` and CLI dataclass `TraceRange` both remain accepted structurally.
- [ ] **Step 5: Verify Task 2**
Run:
```bash
uv run pytest tests\wf_api\test_direct_service.py tests\wf_api\test_capability_api.py tests\wf_api\test_drafts_service.py tests\wf_api\test_artifact_api.py tests\wf_api\test_deployment_api.py tests\wf_api\test_run_api.py -q
uv run ruff check src\wf_api\service.py tests\wf_api\test_direct_service.py
uv run ruff format --check src\wf_api\service.py tests\wf_api\test_direct_service.py
```
Expected: direct service tests and domain API tests pass.
---
### Task 3: Update CLI and MCP Tool Construction
**Files:**
- Modify: `src/wf_cli/context.py`
- Modify: `tests/wf_cli/test_context.py`
- Modify: `src/wf_mcp/workflow_surface/tools.py`
- Add: `tests/wf_api/test_no_double_delegation.py`
- [ ] **Step 1: Write no-backend-chain test**
Create `tests/wf_api/test_no_double_delegation.py`:
```python
from __future__ import annotations
import ast
from pathlib import Path
def _imports_module(path: Path, module_name: str) -> bool:
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
for node in ast.walk(tree):
if isinstance(node, ast.ImportFrom) and node.module == module_name:
return True
if isinstance(node, ast.Import):
if any(alias.name == module_name for alias in node.names):
return True
return False
def test_cli_and_mcp_tools_do_not_import_backend_adapter() -> None:
root = Path(__file__).resolve().parents[2]
assert not _imports_module(
root / "src" / "wf_cli" / "context.py",
"wf_mcp.broker.service.workflow_api_backend",
)
assert not _imports_module(
root / "src" / "wf_mcp" / "workflow_surface" / "tools.py",
"wf_mcp.broker.service.workflow_api_backend",
)
```
- [ ] **Step 2: Run failing no-backend-chain test**
Run:
```bash
uv run pytest tests\wf_api\test_no_double_delegation.py -q
```
Expected: fail because CLI context and MCP tools still import `WfMcpWorkflowApiBackend`.
- [ ] **Step 3: Update CLI context**
In `src/wf_cli/context.py`, remove:
```python
from wf_mcp.broker.service.workflow_api_backend import WfMcpWorkflowApiBackend
```
Add:
```python
from wf_mcp.broker.service.workflow_operation_context import context_from_service
```
Change `load_cli_context()`:
```python
handlers=WorkflowApi(context_from_service(service)),
```
- [ ] **Step 4: Update CLI context test**
In `tests/wf_cli/test_context.py`, replace the private backend-chain assertion:
```python
assert context.handlers.backend._handlers.service is context.service # type: ignore[attr-defined]
```
with:
```python
assert context.handlers.context.artifact_store is context.service.artifact_store
assert context.handlers.context.draft_workspace_store is context.service.draft_workspace_store
assert context.handlers.context.run_store is context.service.run_store
```
This tests the public context seam instead of the deleted backend chain.
- [ ] **Step 5: Update MCP workflow tools**
In `src/wf_mcp/workflow_surface/tools.py`, remove:
```python
from wf_mcp.broker.service.workflow_api_backend import WfMcpWorkflowApiBackend
```
Add:
```python
from wf_mcp.broker.service.workflow_operation_context import context_from_service
```
Change:
```python
handlers = WorkflowApi(WfMcpWorkflowApiBackend(service))
```
to:
```python
handlers = WorkflowApi(context_from_service(service))
```
For run tools, pass `request.trace_range` or `trace_range` directly to `handlers.*`. Remove conversions through `ApiTraceRange`.
- [ ] **Step 6: Verify Task 3**
Run:
```bash
uv run pytest tests\wf_api\test_no_double_delegation.py tests\wf_cli\test_context.py tests\wf_cli tests\wf_mcp\server\test_tools.py tests\wf_mcp\workflow_surface -q
uv run ruff check src\wf_cli\context.py src\wf_mcp\workflow_surface\tools.py tests\wf_cli\test_context.py tests\wf_api\test_no_double_delegation.py
uv run ruff format --check src\wf_cli\context.py src\wf_mcp\workflow_surface\tools.py tests\wf_cli\test_context.py tests\wf_api\test_no_double_delegation.py
```
Expected: CLI and MCP workflow tool tests pass.
---
### Task 4: Shrink `WorkflowSurfaceHandlers` to Compatibility Shim
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `tests/wf_api/test_direct_service.py` or add a small handler shim test
- [ ] **Step 1: Add compatibility shim test**
Add to `tests/wf_api/test_direct_service.py`:
```python
def test_workflow_surface_handlers_is_compatibility_shim() -> None:
from wf_api import WorkflowApi
from wf_mcp.workflow_surface import WorkflowSurfaceHandlers
root = local_temp_root() / "workflow_surface_handler_shim"
service = WfMcpService(
store=FileStore(root / "mcp"),
artifact_store=FileWorkflowArtifactStore(root),
)
handlers = WorkflowSurfaceHandlers(service)
assert isinstance(handlers, WorkflowApi)
assert handlers.service is service
assert handlers.context.artifact_store is service.artifact_store
```
- [ ] **Step 2: Run failing compatibility test**
Run:
```bash
uv run pytest tests\wf_api\test_direct_service.py::test_workflow_surface_handlers_is_compatibility_shim -q
```
Expected: fail because `WorkflowSurfaceHandlers` is not a `WorkflowApi` subclass yet.
- [ ] **Step 3: Replace `WorkflowSurfaceHandlers` implementation**
Replace `src/wf_mcp/workflow_surface/handlers.py` with:
```python
from __future__ import annotations
from wf_api import WorkflowApi
from ..broker.service.workflow_operation_context import context_from_service
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from ..broker.service import WfMcpService
class WorkflowSurfaceHandlers(WorkflowApi):
"""Compatibility wrapper for old wf_mcp.workflow_surface imports.
New code should construct `WorkflowApi(context_from_service(service))`
directly. This shim keeps tests and legacy broker artifact tools working
while the MCP surface is migrated.
"""
def __init__(self, service: WfMcpService) -> None:
self.service = service
super().__init__(context_from_service(service))
__all__ = ["WorkflowSurfaceHandlers"]
```
This file should no longer import domain services directly.
- [ ] **Step 4: Verify handler compatibility**
Run:
```bash
uv run pytest tests\wf_api\test_direct_service.py::test_workflow_surface_handlers_is_compatibility_shim tests\wf_mcp\workflow_surface tests\wf_mcp\test_saved_subgraphs.py tests\wf_mcp\broker -q
uv run ruff check src\wf_mcp\workflow_surface\handlers.py tests\wf_api\test_direct_service.py
uv run ruff format --check src\wf_mcp\workflow_surface\handlers.py tests\wf_api\test_direct_service.py
```
Expected: old handler tests pass through the shim.
---
### Task 5: Delete Backend Protocol and Adapter
**Files:**
- Delete: `src/wf_api/backend.py`
- Delete: `src/wf_mcp/broker/service/workflow_api_backend.py`
- Modify: `src/wf_api/__init__.py`
- Modify docs that describe the old backend chain
- [ ] **Step 1: Delete backend files**
Delete:
```text
src/wf_api/backend.py
src/wf_mcp/broker/service/workflow_api_backend.py
```
- [ ] **Step 2: Remove public backend export**
In `src/wf_api/__init__.py`, ensure there is no import or `__all__` entry for `WorkflowApiBackend`.
- [ ] **Step 3: Search for live backend references**
Run:
```bash
rg -n "WorkflowApiBackend|WfMcpWorkflowApiBackend|workflow_api_backend|\\.backend" src tests
```
Expected: no live source/test references.
Historical docs under `docs/superpowers/plans/2026-06-01-*` may still mention the old slice. Do not rewrite historical plans except the active roadmap files named in Task 6.
- [ ] **Step 4: Verify deletion**
Run:
```bash
uv run pytest tests\wf_api\test_import_direction.py tests\wf_api\test_no_double_delegation.py tests\wf_api\test_cli_context_uses_api.py -q
uv run ruff check src\wf_api src\wf_cli\context.py src\wf_mcp\workflow_surface src\wf_mcp\broker\service tests\wf_api
uv run ruff format --check src\wf_api src\wf_cli\context.py src\wf_mcp\workflow_surface src\wf_mcp\broker\service tests\wf_api
```
Expected: tests pass, lint pass, format pass.
---
### Task 6: Update Active Docs
**Files:**
- Modify: `docs/current_roadmap.md`
- Modify: `docs/wf_mcp_architecture.md`
- Modify: `docs/superpowers/plans/2026-06-01-wf-api-extraction-roadmap.md`
- [ ] **Step 1: Update `docs/current_roadmap.md`**
Replace the bullet that says the next useful slice is removing double-delegation with:
```markdown
- Double-delegation has been removed: CLI and MCP workflow tools construct
`WorkflowApi(context_from_service(service))` directly. `WorkflowSurfaceHandlers`
remains only as a temporary compatibility shim for older imports.
```
- [ ] **Step 2: Update `docs/wf_mcp_architecture.md`**
Find the architecture text that contains:
```text
wf_api.WorkflowApi ───> WorkflowApiBackend
```
Replace that diagram/text with:
```text
wf_mcp.workflow_surface.tools
-> wf_api.WorkflowApi
-> wf_api domain services
-> WorkflowOperationContext
-> WfMcpService adapters/stores/runtime
```
Add:
```markdown
`WorkflowSurfaceHandlers` is a compatibility shim only. New entrypoints should
construct `WorkflowApi(context_from_service(service))` directly.
```
- [ ] **Step 3: Update active extraction roadmap**
In `docs/superpowers/plans/2026-06-01-wf-api-extraction-roadmap.md`, add a current-state note near the top:
```markdown
> Current update: the original `WorkflowApiBackend` seam was useful for proving
> dependency direction, but has been collapsed. `WorkflowApi` now composes
> domain services directly from `WorkflowOperationContext`; MCP owns only
> context construction and tool schemas.
```
Do not rewrite the historical task bodies. They describe prior slices.
- [ ] **Step 4: Verify docs**
Run:
```bash
git diff --check -- docs\current_roadmap.md docs\wf_mcp_architecture.md docs\superpowers\plans\2026-06-01-wf-api-extraction-roadmap.md
```
Expected: no whitespace errors.
---
### Task 7: Final Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused workflow API/MCP/CLI tests**
Run:
```bash
uv run pytest tests\wf_api tests\wf_cli tests\wf_mcp\workflow_surface tests\wf_mcp\server\test_tools.py tests\wf_mcp\test_saved_subgraphs.py -q
```
Expected: selected tests pass.
- [ ] **Step 2: Run full suite**
Run:
```bash
uv run pytest -q
```
Expected: full suite passes with known skip/xfail counts.
- [ ] **Step 3: Run lint and format checks**
Run:
```bash
uv run ruff check src\wf_api src\wf_cli src\wf_mcp tests\wf_api tests\wf_cli tests\wf_mcp
uv run ruff format --check src\wf_api src\wf_cli src\wf_mcp tests\wf_api tests\wf_cli tests\wf_mcp
```
Expected: all checks pass.
- [ ] **Step 4: Run typecheck**
Run:
```bash
uv run basedpyright --level error
```
Expected: `0 errors, 0 warnings, 0 notes`. If the command exits nonzero only because of the known workspace enumeration warning, report that exactly.
- [ ] **Step 5: Final reference check**
Run:
```bash
rg -n "WorkflowApiBackend|WfMcpWorkflowApiBackend|workflow_api_backend|\\.backend" src tests docs\current_roadmap.md docs\wf_mcp_architecture.md
```
Expected: no references in live source/tests/current docs.
---
## Self-Review
- Spec coverage: The plan removes `WorkflowApiBackend`, deletes `WfMcpWorkflowApiBackend`, updates CLI/MCP construction, keeps handler compatibility, moves `TraceRange`, and updates active docs.
- Placeholder scan: No `TODO`/`TBD` placeholders remain.
- Type consistency: `WorkflowApi` accepts `WorkflowOperationContext`; run trace methods accept `TraceRangeLike`; `TraceRange` is a convenience DTO exported from `wf_api.models`.
- Scope check: This does not remove all `WorkflowSurfaceHandlers` tests or legacy imports. It reduces handlers to a shim; deleting the shim is a later cleanup once `wf_mcp.broker.artifact_tools` and legacy tests stop importing it.
@@ -0,0 +1,682 @@
# wf_api Slice 4C: Artifacts And Deployments Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move saved artifact and deployment operations out of `WorkflowSurfaceHandlers` into protocol-neutral `wf_api` domain services.
**Architecture:** Add `WorkflowArtifactApi` and `WorkflowDeploymentApi` that depend on `WorkflowOperationContext`, not `WfMcpService`. Keep `WorkflowSurfaceHandlers` public method signatures unchanged and delegate artifact/deployment methods to the new services. Extend operation-context protocols for event emission and live source checks so `wf_api` does not import MCP event factories, adapters, auth, or connections.
**Tech Stack:** Python 3.14+, `wf_api.operation_context`, `wf_api.drafts`, `wf_artifacts`, `wf_platform`, `wf_core`, pytest, ruff, basedpyright.
---
## Scope
### Move In This Slice
Move these methods from `WorkflowSurfaceHandlers`:
```text
list_artifacts
save_artifact
create_artifact_from_plan
create_artifact_from_draft
create_artifact_from_workspace
create_wrapper_from_workspace
inspect_artifact
list_deployments
inspect_deployment
save_deployment
delete_deployment
validate_deployment
```
Move or duplicate only the helpers required by those methods:
```text
_available_sources
_suggested_self_bindings
_observed_node_specs
_capability_name
_artifact_capability_id
_deployment_summary
```
### Do Not Move In This Slice
Do not move:
```text
list_capabilities
inspect_capability
call_capability
_wrapper_artifact_for_capability_name
_wrapper_capability_summaries
_wrapper_capability_detail
_call_wrapper_artifact
run_deployment
resume_run
inspect_run
read_run_trace
_raw_plan_from_artifact
_run_payload
_interrupt_payload
```
Reasons:
- Capability methods still combine live source specs, saved wrappers, and direct test calls. Move them in Slice 4E.
- Run methods depend on durable run checkpoints, runtime preparation, trace slicing, and saved subgraph execution. Move them in Slice 4D.
- `_raw_plan_from_artifact` is still needed by wrapper capability calls and run methods. Leave it in `handlers.py` until those domains move or extract it separately.
### Invariants
- No public payload changes.
- No MCP tool schema changes.
- `WorkflowSurfaceHandlers` still exposes the same methods.
- `wf_api` imports no `wf_mcp`.
- Event construction stays adapter-owned.
- Live upstream checks stay adapter-owned.
- Temporary private helper duplication is allowed when capability/run methods still need a helper in `handlers.py`.
---
## Task 1: Extend Operation Context For Events And Live Checks
**Files:**
- Modify: `src/wf_api/operation_context.py`
- Create: `src/wf_mcp/broker/service/workflow_live_checks.py`
- Modify: `src/wf_mcp/broker/service/workflow_operation_context.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_api/test_operation_context.py`
- [ ] **Step 1: Add protocol methods**
In `src/wf_api/operation_context.py`, update `WorkflowEventRecorder`:
```python
class WorkflowEventRecorder(Protocol):
"""Records workflow lifecycle events without exposing MCP event types."""
def record_event(self, event: object) -> None:
"""Record one adapter-native event object."""
...
def record_workflow_event(
self,
event_type: str,
*,
capability_id: str,
payload: dict[str, Any],
) -> None:
"""Record one workflow lifecycle event by protocol-neutral fields."""
...
```
Update `WorkflowLiveSourceChecker`:
```python
class WorkflowLiveSourceChecker(Protocol):
"""Optional hook for validating live external source availability."""
async def deployment_diagnostics(
self,
*,
deployment: WorkflowDeployment,
artifacts: Sequence[WorkflowArtifact],
) -> list[DependencyDiagnostic]:
"""Return opt-in live-source diagnostics for a deployment tree."""
...
```
Add imports:
```python
from collections.abc import Mapping, Sequence
from wf_artifacts import DependencyDiagnostic, WorkflowDeployment
```
Remove or keep `available_sources()` only if still used by tests. New moved code should use `deployment_diagnostics(...)`.
- [ ] **Step 2: Move MCP live-check helper out of handlers**
Create `src/wf_mcp/broker/service/workflow_live_checks.py` and move these
handler-level live-check pieces into it:
```text
LIVE_SOURCE_CHECK_TIMEOUT_SECONDS
_LIVE_SOURCE_CHECK_FAILURES
live_source_diagnostics
_required_live_sources
```
Rename `_live_source_diagnostics(...)` to public module-private-adapter helper
`live_source_diagnostics(...)`.
The new module should own the MCP-only imports:
```python
import asyncio
import anyio
import httpx
from mcp.client.streamable_http import StreamableHTTPError
from mcp.shared.exceptions import McpError
from wf_artifacts import DependencyDiagnostic, DiagnosticSeverity, WorkflowArtifact, WorkflowDeployment
from wf_mcp.broker.service.adapters import require_adapter
from wf_mcp.broker.service.core import WfMcpService
```
Keep the existing docstring explaining that live checks perform opt-in upstream
I/O. This split is required to avoid a circular import:
```text
handlers.py -> workflow_operation_context.py -> handlers.py
```
After the move, update `handlers.py` to import `live_source_diagnostics` from
the new module for as long as `validate_deployment` still lives in handlers.
When Task 4 delegates `validate_deployment`, remove that handler import if it
is unused.
- [ ] **Step 3: Implement MCP adapter methods**
In `src/wf_mcp/broker/service/workflow_operation_context.py`, import:
```python
from collections.abc import Sequence
from wf_artifacts import DependencyDiagnostic, WorkflowArtifact, WorkflowDeployment
from wf_mcp.events import make_event
from wf_mcp.broker.service.workflow_live_checks import live_source_diagnostics
```
Then update event recorder:
```python
def record_workflow_event(
self,
event_type: str,
*,
capability_id: str,
payload: dict[str, Any],
) -> None:
self.service._record_event( # noqa: SLF001
make_event(event_type, capability_id=capability_id, payload=payload)
)
```
Update live source checker:
```python
async def deployment_diagnostics(
self,
*,
deployment: WorkflowDeployment,
artifacts: Sequence[WorkflowArtifact],
) -> list[DependencyDiagnostic]:
return await live_source_diagnostics(
self.service,
deployment=deployment,
artifacts=artifacts,
)
```
Do not import handler modules from `workflow_operation_context.py`.
- [ ] **Step 4: Update operation context tests**
In `tests/wf_api/test_operation_context.py`, add a test that calls:
```python
operation_context.events.record_workflow_event(
"workflow_artifact_saved",
capability_id="workflow.demo.v1",
payload={"artifact_id": "demo", "version": 1},
)
```
Then assert the service recorded an event with stable fields individually. Do not assert full dict equality.
- [ ] **Step 5: Run focused tests**
```powershell
uv run pytest tests/wf_api/test_operation_context.py -q
```
Expected: pass.
---
## Task 2: Create `wf_api.artifacts`
**Files:**
- Create: `src/wf_api/artifacts.py`
- Modify: `src/wf_api/__init__.py`
- Test: `tests/wf_api/test_artifact_api.py`
- [ ] **Step 1: Create service skeleton**
Create `src/wf_api/artifacts.py`:
```python
from __future__ import annotations
from collections.abc import Sequence
from typing import Any
from wf_artifacts import (
ArtifactKind,
RequiredCapability,
WorkflowArtifact,
create_workflow_artifact_from_plan as build_workflow_artifact_from_plan,
)
from wf_platform import CapabilityRef, NodeSpecInventory
from .drafts import WorkflowDraftApi
from .models import RawWorkflowPlan
from .operation_context import WorkflowOperationContext
class WorkflowArtifactApi:
"""Saved workflow artifact operations.
Event construction is intentionally delegated through
WorkflowOperationContext so this module stays protocol-neutral.
"""
def __init__(self, context: WorkflowOperationContext) -> None:
self.context = context
self.drafts = WorkflowDraftApi(context)
def _artifact_store(self):
if self.context.artifact_store is None:
raise KeyError("workflow artifact store is not configured")
return self.context.artifact_store
```
- [ ] **Step 2: Move artifact methods**
Move these bodies from `WorkflowSurfaceHandlers`, replacing `self.service` access:
```text
list_artifacts
save_artifact
create_artifact_from_plan
create_artifact_from_draft
inspect_artifact
```
Required replacements:
```python
self.service.artifact_store -> self._artifact_store()
self.service.workflow_artifact_catalog_entry(artifact) -> self.context.artifacts.workflow_artifact_catalog_entry(artifact)
self.service._record_event(make_event(...)) -> self.context.events.record_workflow_event(...)
_observed_node_specs(self.service) -> _observed_node_specs(self.context)
```
Keep return payloads byte-for-byte equivalent except for dictionary ordering.
- [ ] **Step 3: Move workspace artifact methods**
Move:
```text
create_artifact_from_workspace
create_wrapper_from_workspace
```
Use `self.context.draft_workspace_store` through `self.drafts` or a local store helper. Preserve current behavior:
- validate workspace draft first
- return `saved: False` with diagnostics when invalid
- call `create_artifact_from_draft(...)` when valid
- wrapper path passes `kind="wrapper"`
- [ ] **Step 4: Add helpers**
Add private helpers to `wf_api.artifacts`:
```text
_required_capability_payloads
_suggested_self_bindings
_observed_node_specs
_plan_nodes
_artifact_capability_id
```
Duplicate `_required_capability_payloads`, `_observed_node_specs`, and `_plan_nodes` from `wf_api.drafts` for now instead of importing private draft helpers. We can consolidate after Slice 4C if duplication becomes annoying.
Do not remove `_artifact_capability_id` from `handlers.py`; wrapper capability methods still need it until Slice 4E.
- [ ] **Step 5: Export artifact service**
In `src/wf_api/__init__.py`:
```python
from .artifacts import WorkflowArtifactApi
```
Add `"WorkflowArtifactApi"` to `__all__`.
- [ ] **Step 6: Add focused tests**
Create `tests/wf_api/test_artifact_api.py` with tests that instantiate `WorkflowArtifactApi(context_from_service(service))`:
- `save_artifact` stores a `WorkflowArtifact` and returns `saved: True`.
- `create_artifact_from_plan` saves an artifact and includes observed node specs.
- `create_artifact_from_workspace` returns `saved: False` when workspace validation fails.
- `create_wrapper_from_workspace` saves `kind == "wrapper"`.
- Handler delegation for `inspect_artifact` returns the same stable fields as direct `WorkflowArtifactApi.inspect_artifact`.
Use field-by-field assertions unless asserting a known closed model shape.
- [ ] **Step 7: Run artifact tests**
```powershell
uv run pytest tests/wf_api/test_artifact_api.py tests/wf_api/test_drafts_service.py -q
```
Expected: pass.
---
## Task 3: Create `wf_api.deployments`
**Files:**
- Create: `src/wf_api/deployments.py`
- Modify: `src/wf_api/__init__.py`
- Test: `tests/wf_api/test_deployment_api.py`
- [ ] **Step 1: Create service skeleton**
Create `src/wf_api/deployments.py`:
```python
from __future__ import annotations
from typing import Any
from wf_artifacts import (
AvailableCapability,
AvailableSource,
DependencyDiagnostic,
WorkflowArtifact,
WorkflowDeployment,
hash_json_schema,
validate_deployment_dependencies,
)
from wf_platform import CapabilitySource
from .next_actions import NextActions
from .operation_context import WorkflowOperationContext
from .saved_subgraphs import resolve_saved_subgraph_tree, validate_saved_subgraph_tree
class WorkflowDeploymentApi:
"""Saved deployment operations and dependency validation."""
def __init__(self, context: WorkflowOperationContext) -> None:
self.context = context
def _artifact_store(self):
if self.context.artifact_store is None:
raise KeyError("workflow artifact store is not configured")
return self.context.artifact_store
```
- [ ] **Step 2: Move deployment methods**
Move these bodies from `WorkflowSurfaceHandlers`:
```text
list_deployments
inspect_deployment
save_deployment
delete_deployment
validate_deployment
```
Required replacements:
```python
self.service.artifact_store -> self._artifact_store()
self.service._record_event(make_event(...)) -> self.context.events.record_workflow_event(...)
_available_sources(self.service) -> _available_sources(self.context.capability_sources)
```
For `validate_deployment(live_check=True)`, use:
```python
if live_check and self.context.live_sources is not None:
diagnostics.extend(
await self.context.live_sources.deployment_diagnostics(
deployment=deployment,
artifacts=[artifact, *tree.artifacts_by_ref.values()],
)
)
```
If `live_check=True` and `live_sources is None`, preserve static validation only. Do not invent a new warning payload in this slice.
- [ ] **Step 3: Move deployment validation helper**
Move `_deployment_validation` logic into `WorkflowDeploymentApi` as a private method:
```python
def _deployment_validation(
self,
deployment_id: str,
) -> tuple[WorkflowDeployment, WorkflowArtifact, list[DependencyDiagnostic], SavedSubgraphTree]:
...
```
Use `self._artifact_store()` and `_available_sources(self.context.capability_sources)`.
- [ ] **Step 4: Add helper functions**
Add private helpers:
```text
_available_sources
_capability_name
_deployment_summary
```
Adapt `_available_sources` to accept `Mapping[str, CapabilitySource]` instead of `WfMcpService`.
Do not remove `_available_sources` or `_capability_name` from `handlers.py` if run/capability methods still use them.
- [ ] **Step 5: Export deployment service**
In `src/wf_api/__init__.py`:
```python
from .deployments import WorkflowDeploymentApi
```
Add `"WorkflowDeploymentApi"` to `__all__`.
- [ ] **Step 6: Add focused tests**
Create `tests/wf_api/test_deployment_api.py` with tests that instantiate `WorkflowDeploymentApi(context_from_service(service))`:
- `save_deployment` stores and returns stable deployment fields.
- `list_deployments` returns compact summaries.
- `delete_deployment` removes one deployment.
- `validate_deployment(live_check=False)` returns `runnable` for a valid binding.
- `validate_deployment(live_check=True)` calls the operation-context live checker. A simple fake context may be easier than MCP service setup for this test.
- Handler delegation for `validate_deployment` returns the same stable status/diagnostic fields as direct `WorkflowDeploymentApi.validate_deployment`.
- [ ] **Step 7: Run deployment tests**
```powershell
uv run pytest tests/wf_api/test_deployment_api.py tests/wf_mcp/workflow_surface/test_deployments.py -q
```
Expected: pass.
---
## Task 4: Wire `WorkflowSurfaceHandlers`
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- [ ] **Step 1: Add imports**
```python
from wf_api.artifacts import WorkflowArtifactApi
from wf_api.deployments import WorkflowDeploymentApi
```
- [ ] **Step 2: Instantiate services**
In `WorkflowSurfaceHandlers.__init__`, avoid building multiple independent contexts:
```python
context = context_from_service(service)
self._drafts = WorkflowDraftApi(context)
self._artifacts = WorkflowArtifactApi(context)
self._deployments = WorkflowDeploymentApi(context)
```
- [ ] **Step 3: Replace moved artifact methods with delegates**
Replace bodies for:
```text
list_artifacts
save_artifact
create_artifact_from_plan
create_artifact_from_draft
create_artifact_from_workspace
create_wrapper_from_workspace
inspect_artifact
```
Example:
```python
async def inspect_artifact(self, *, artifact_id: str, version: int) -> dict[str, Any]:
return await self._artifacts.inspect_artifact(
artifact_id=artifact_id,
version=version,
)
```
- [ ] **Step 4: Replace moved deployment methods with delegates**
Replace bodies for:
```text
list_deployments
inspect_deployment
save_deployment
delete_deployment
validate_deployment
```
Example:
```python
async def validate_deployment(
self,
*,
deployment_id: str,
live_check: bool = False,
) -> dict[str, Any]:
return await self._deployments.validate_deployment(
deployment_id=deployment_id,
live_check=live_check,
)
```
- [ ] **Step 5: Remove only unused imports/helpers**
After delegation, run:
```powershell
rg -n "_available_sources|_suggested_self_bindings|_observed_node_specs|_deployment_summary|_artifact_capability_id|_capability_name" src/wf_mcp/workflow_surface/handlers.py
```
Remove a helper from `handlers.py` only if it has no remaining caller there.
Expected likely result:
- `_suggested_self_bindings`, `_observed_node_specs`, `_deployment_summary` can probably be removed.
- `_artifact_capability_id`, `_capability_name`, `_available_sources` may still be needed by capability/run methods. Keep them if referenced.
---
## Task 5: Verification
- [ ] **Step 1: Run focused wf_api tests**
```powershell
uv run pytest tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py tests/wf_api/test_drafts_service.py -q
```
Expected: pass.
- [ ] **Step 2: Run workflow surface tests**
```powershell
uv run pytest tests/wf_mcp/workflow_surface -q
```
Expected: pass.
- [ ] **Step 3: Run import-direction test**
```powershell
uv run pytest tests/wf_api/test_import_direction.py -q
```
Expected: pass; `wf_api` has no `wf_mcp` imports.
- [ ] **Step 4: Run ruff on touched files**
```powershell
uv run ruff check src/wf_api/artifacts.py src/wf_api/deployments.py src/wf_api/operation_context.py src/wf_api/__init__.py src/wf_mcp/broker/service/workflow_operation_context.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py
```
Expected: all checks pass.
- [ ] **Step 5: Run basedpyright on touched files**
```powershell
uv run basedpyright --level error src/wf_api/artifacts.py src/wf_api/deployments.py src/wf_api/operation_context.py src/wf_mcp/broker/service/workflow_operation_context.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py
```
Expected: `0 errors`.
- [ ] **Step 6: Optional full suite**
```powershell
uv run pytest -q
```
Expected: full suite passes with the projects existing skipped/xfailed counts.
---
## Self-Review Checklist
- `wf_api.artifacts` imports no `wf_mcp`.
- `wf_api.deployments` imports no `wf_mcp`.
- Event construction remains in `wf_mcp.broker.service.workflow_operation_context`.
- Live upstream adapter/auth probing remains in `wf_mcp`.
- `WorkflowSurfaceHandlers` public artifact/deployment method signatures are unchanged.
- `create_draft_workspace_from_capability` still lives in `WorkflowSurfaceHandlers`.
- Capability methods still live in `WorkflowSurfaceHandlers`.
- Run methods still live in `WorkflowSurfaceHandlers`.
- No public payload shape changed.
- No MCP schema changed.
- Temporary helper duplication is documented and deliberate.
@@ -0,0 +1,574 @@
# wf_api Slice 4D: Run Lifecycle Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move deployment run, resume, stopped-run inspection, and bounded trace reading out of `WorkflowSurfaceHandlers` into a protocol-neutral `wf_api.runs.WorkflowRunApi`.
**Architecture:** `WorkflowRunApi` depends on `WorkflowOperationContext` and `WorkflowDeploymentApi`, not `WfMcpService`. Runtime execution remains adapter-owned through `WorkflowRuntimeRunner`; run persistence and payload shaping move into `wf_api`. Keep MCP Pydantic request models at the MCP boundary and pass only a structural trace range into `wf_api`.
**Tech Stack:** Python 3.14+, `wf_api.operation_context`, `wf_api.deployments`, `wf_api.run_lifecycle`, `wf_api.saved_subgraphs`, `wf_artifacts` run store models, `wf_core.RunState`, pytest, ruff, basedpyright.
---
## Scope
### Move In This Slice
Move these methods from `WorkflowSurfaceHandlers` to `wf_api.runs.WorkflowRunApi`:
```text
run_deployment
resume_run
inspect_run
read_run_trace
```
Move or duplicate only the helpers needed by those methods:
```text
_run_store
_raw_plan_from_artifact
_plan_field
_run_payload
_interrupt_payload
```
### Do Not Move In This Slice
Do not move:
```text
list_capabilities
inspect_capability
call_capability
_wrapper_artifact_for_capability_name
_wrapper_capability_summaries
_wrapper_capability_detail
_call_wrapper_artifact
```
Reasons:
- Capability methods still own wrapper discovery and direct test calls.
- `_raw_plan_from_artifact` is still needed by wrapper direct calls in `handlers.py`; duplicate it temporarily in `wf_api.runs` or move it to a small shared `wf_api` helper only if that does not widen the slice.
### Invariants
- No public payload changes.
- No MCP tool schema changes.
- `WorkflowSurfaceHandlers` public run method signatures stay unchanged.
- `wf_api` imports no `wf_mcp`.
- Runtime event construction remains adapter-owned in `WfMcpService`.
- `run_deployment` still persists stopped runs.
- `resume_run` still revalidates pinned dependency environments before mutating state.
- Trace payloads remain opt-in and bounded by `trace_range`.
---
## Task 1: Align Runtime Protocol With Actual Runtime Calls
**Files:**
- Modify: `src/wf_api/operation_context.py`
- Modify: `src/wf_mcp/broker/service/workflow_operation_context.py`
- Test: `tests/wf_api/test_operation_context.py`
- [ ] **Step 1: Update `WorkflowRuntimeRunner` protocol**
In `src/wf_api/operation_context.py`, replace the older generic runtime kwargs with the current deployment-aware shape:
```python
from wf_api.saved_subgraphs import SavedSubgraphTree
```
```python
class WorkflowRuntimeRunner(Protocol):
"""Runs and resumes workflow plans using an adapter-owned runtime backend."""
async def run_workflow_from_plan(
self,
plan: RawWorkflowPlan,
workflow_input: dict[str, Any],
deployment: WorkflowDeployment | None = None,
artifact: WorkflowArtifact | None = None,
saved_subgraph_tree: SavedSubgraphTree | None = None,
) -> RunState:
"""Execute one raw workflow plan and return its run state."""
...
async def resume_workflow_from_plan(
self,
plan: RawWorkflowPlan,
run: RunState,
*,
resume_payload: dict[str, Any],
resume_outcome: str,
deployment: WorkflowDeployment | None = None,
artifact: WorkflowArtifact | None = None,
saved_subgraph_tree: SavedSubgraphTree | None = None,
) -> RunState:
"""Resume one interrupted raw workflow plan and return its run state."""
...
```
Remove unused imports from the protocol file if `AsyncRegistryHandler` or
`ReducerDefinition` are no longer needed.
- [ ] **Step 2: Give adapter methods explicit signatures**
In `src/wf_mcp/broker/service/workflow_operation_context.py`, replace `**kwargs`
runtime adapter methods with explicit signatures matching the protocol:
```python
async def run_workflow_from_plan(
self,
plan,
workflow_input,
deployment=None,
artifact=None,
saved_subgraph_tree=None,
):
return await self.service.run_workflow_from_plan(
plan,
workflow_input,
deployment=deployment,
artifact=artifact,
saved_subgraph_tree=saved_subgraph_tree,
)
```
Do the same for `resume_workflow_from_plan(...)`.
- [ ] **Step 3: Run operation-context tests**
```powershell
uv run pytest tests/wf_api/test_operation_context.py -q
```
Expected: pass.
---
## Task 2: Create `wf_api.runs`
**Files:**
- Create: `src/wf_api/runs.py`
- Modify: `src/wf_api/__init__.py`
- Test: `tests/wf_api/test_run_api.py`
- [ ] **Step 1: Create service skeleton and trace range protocol**
Create `src/wf_api/runs.py`:
```python
from __future__ import annotations
from dataclasses import asdict
from typing import Any, Protocol
from wf_artifacts import (
DependencyDiagnostic,
RunStore,
WorkflowArtifact,
WorkflowDeployment,
)
from wf_core import RunState
from .deployments import WorkflowDeploymentApi, _available_sources
from .models import RawWorkflowPlan
from .next_actions import NextActions
from .run_lifecycle import (
create_pinned_environment,
has_blocking_diagnostics,
load_stored_run,
mark_resume_blocked,
persist_stopped_run,
restore_interrupted_run,
validate_pinned_resume_environment,
)
from .saved_subgraphs import saved_subgraph_tree_from_snapshots
from .operation_context import WorkflowOperationContext
class TraceRangeLike(Protocol):
"""Small structural trace range accepted from MCP, CLI, or HTTP adapters."""
start: int
limit: int
class WorkflowRunApi:
"""Deployment run lifecycle operations.
Runtime execution stays behind WorkflowOperationContext.runtime so wf_api
does not depend on MCP service internals.
"""
def __init__(self, context: WorkflowOperationContext) -> None:
self.context = context
self.deployments = WorkflowDeploymentApi(context)
def _run_store(self) -> RunStore:
if self.context.run_store is None:
raise KeyError("workflow run store is not configured")
return self.context.run_store
```
Use `TraceRangeLike | None` for run methods. This lets handler methods pass
their MCP Pydantic `TraceRange` without importing it into `wf_api`.
- [ ] **Step 2: Export run service**
In `src/wf_api/__init__.py`:
```python
from .runs import WorkflowRunApi
```
Add `"WorkflowRunApi"` to `__all__`.
---
## Task 3: Move Run Methods
**Files:**
- Modify: `src/wf_api/runs.py`
- [ ] **Step 1: Move `run_deployment`**
Move the current handler body into `WorkflowRunApi.run_deployment(...)`.
Required replacements:
```python
self._deployments.deployment_validation(...) -> self.deployments.deployment_validation(...)
self.service.run_workflow_from_plan(...) -> self.context.runtime.run_workflow_from_plan(...)
self._run_store() -> self._run_store()
```
Call runtime with the same arguments:
```python
run = await self.context.runtime.run_workflow_from_plan(
plan,
workflow_input,
deployment=deployment,
artifact=artifact,
saved_subgraph_tree=tree,
)
```
- [ ] **Step 2: Move `resume_run`**
Move the current handler body into `WorkflowRunApi.resume_run(...)`.
Required replacements:
```python
validate_pinned_resume_environment(..., sources=_available_sources(self.service))
```
becomes:
```python
validate_pinned_resume_environment(
record=record,
sources=_available_sources(self.context.capability_sources),
)
```
Call runtime with:
```python
run = await self.context.runtime.resume_workflow_from_plan(
plan,
stopped_run,
resume_payload=resume_payload,
resume_outcome=resume_outcome,
deployment=environment.deployment,
artifact=environment.root_artifact,
saved_subgraph_tree=tree,
)
```
- [ ] **Step 3: Move stopped-run readers**
Move:
```text
inspect_run
read_run_trace
```
Preserve current payload shape:
- `inspect_run` returns no trace list.
- `read_run_trace` returns only `trace_range.start : start + limit`.
- both include `trace_count`.
- both include `next_actions` via `_run_payload`.
---
## Task 4: Move Run Helpers
**Files:**
- Modify: `src/wf_api/runs.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- [ ] **Step 1: Add private helpers to `wf_api.runs`**
Move or duplicate these helpers into `src/wf_api/runs.py`:
```text
_raw_plan_from_artifact
_plan_field
_run_payload
_interrupt_payload
```
Keep the trace comment inside `_run_payload`:
```python
# Trace entries can grow quickly, so the public run tool only includes
# a bounded debug slice when the caller explicitly asks for a range.
```
This comment is important because trace bloat is a public UX boundary.
- [ ] **Step 2: Keep handler copies only if needed**
After handler delegation, run:
```powershell
rg -n "_raw_plan_from_artifact|_run_payload|_interrupt_payload|_plan_field" src/wf_mcp/workflow_surface/handlers.py
```
Expected:
- `_raw_plan_from_artifact` likely remains because `_call_wrapper_artifact` still uses it.
- `_plan_field` remains if `_raw_plan_from_artifact` remains.
- `_run_payload` and `_interrupt_payload` should be removable if no handler run methods remain.
Remove only helpers with no remaining handler callers.
---
## Task 5: Wire `WorkflowSurfaceHandlers`
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- [ ] **Step 1: Add import**
```python
from wf_api.runs import WorkflowRunApi
```
- [ ] **Step 2: Instantiate run service**
In `WorkflowSurfaceHandlers.__init__`, reuse the same context object:
```python
context = context_from_service(service)
self._drafts = WorkflowDraftApi(context)
self._artifacts = WorkflowArtifactApi(context)
self._deployments = WorkflowDeploymentApi(context)
self._runs = WorkflowRunApi(context)
```
Do not call `context_from_service(service)` separately for every domain service.
- [ ] **Step 3: Replace run method bodies with delegates**
Replace:
```text
run_deployment
resume_run
inspect_run
read_run_trace
```
Example:
```python
async def inspect_run(self, *, run_id: str) -> dict[str, Any]:
"""Return one durable stopped-run summary without debug trace entries."""
return await self._runs.inspect_run(run_id=run_id)
```
For `trace_range`, pass the MCP model object through directly:
```python
return await self._runs.run_deployment(
deployment_id=deployment_id,
workflow_input=workflow_input,
trace_range=trace_range,
)
```
`WorkflowRunApi` accepts it structurally through `TraceRangeLike`.
- [ ] **Step 4: Remove now-unused imports**
After replacing run methods, remove imports from `handlers.py` only if `ruff`
confirms they are unused. Likely candidates:
```text
dataclasses.asdict
RunStore
run_lifecycle helpers
saved_subgraph_tree_from_snapshots
```
Do not remove `SavedSubgraphTree`, `direct_wrapper_interrupt_diagnostic`,
`resolve_saved_subgraph_tree`, or `_raw_plan_from_artifact` if wrapper/capability
methods still need them.
---
## Task 6: Add Focused Run API Tests
**Files:**
- Create: `tests/wf_api/test_run_api.py`
- [ ] **Step 1: Cover unrunnable deployment path**
Create a test that saves a deployment with missing/unbound requirements and
asserts:
```python
result = asyncio.run(api.run_deployment(...))
assert result["status"] == "unrunnable"
assert result["run_id"] is None
assert result["trace_count"] == 0
assert result["diagnostics"][0]["code"]
```
- [ ] **Step 2: Cover completed run persistence**
Use existing test helpers (`echo_tool`, local temp store patterns) to register a
valid source, save an artifact/deployment, run it, and assert:
```python
assert result["status"] == "completed"
assert isinstance(result["run_id"], str)
assert result["resume_readiness"] == "not_applicable"
assert result["trace_count"] >= 1
```
Then load the run from the run store and assert it exists.
- [ ] **Step 3: Cover inspect and bounded trace**
After a completed run:
```python
summary = asyncio.run(api.inspect_run(run_id=run_id))
trace = asyncio.run(api.read_run_trace(run_id=run_id, trace_range=SimpleTraceRange(start=0, limit=1)))
```
Assert:
```python
assert "trace" not in summary
assert trace["trace_start"] == 0
assert trace["trace_limit"] == 1
assert len(trace["trace"]) <= 1
assert trace["trace_count"] == summary["trace_count"]
```
Define local helper:
```python
@dataclass(frozen=True)
class SimpleTraceRange:
start: int
limit: int
```
- [ ] **Step 4: Cover handler delegation**
Add one smoke test comparing stable fields from:
```python
handler_result = asyncio.run(WorkflowSurfaceHandlers(service).inspect_run(run_id=run_id))
api_result = asyncio.run(WorkflowRunApi(context_from_service(service)).inspect_run(run_id=run_id))
```
Compare `status`, `run_id`, `trace_count`, and `resume_readiness` individually.
Do not duplicate every old workflow-surface run test. `wf_api` should own run
behavior; `wf_mcp` should keep only adapter/schema/delegation coverage.
---
## Task 7: Verification
- [ ] **Step 1: Run focused run tests**
```powershell
uv run pytest tests/wf_api/test_run_api.py tests/wf_mcp/workflow_surface/test_runs.py -q
```
Expected: pass.
- [ ] **Step 2: Run deployment/artifact tests because runs reuse them**
```powershell
uv run pytest tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py tests/wf_api/test_operation_context.py -q
```
Expected: pass.
- [ ] **Step 3: Run import-direction test**
```powershell
uv run pytest tests/wf_api/test_import_direction.py -q
```
Expected: pass; `wf_api` has no `wf_mcp` imports.
- [ ] **Step 4: Run ruff on touched files**
```powershell
uv run ruff check src/wf_api/runs.py src/wf_api/operation_context.py src/wf_api/__init__.py src/wf_mcp/broker/service/workflow_operation_context.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_run_api.py
```
Expected: all checks pass.
- [ ] **Step 5: Run basedpyright on touched files**
```powershell
uv run basedpyright --level error src/wf_api/runs.py src/wf_api/operation_context.py src/wf_mcp/broker/service/workflow_operation_context.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_run_api.py
```
Expected: `0 errors`.
- [ ] **Step 6: Optional full suite**
```powershell
uv run pytest -q
```
Expected: full suite passes with the projects existing skipped/xfailed counts.
---
## Self-Review Checklist
- `wf_api.runs` imports no `wf_mcp`.
- Runtime execution goes through `WorkflowOperationContext.runtime`.
- Run persistence uses `WorkflowOperationContext.run_store`.
- `WorkflowSurfaceHandlers` public run signatures are unchanged.
- `TraceRange` stays structural at the `wf_api` layer.
- Trace list remains opt-in and bounded.
- `resume_run` still blocks when pinned dependency validation fails.
- Capability direct wrapper calls still work because handler keeps `_raw_plan_from_artifact` if needed.
- No public payload shape changed.
- No MCP schema changed.
@@ -0,0 +1,663 @@
# wf_api Slice 4E: Capabilities Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move workflow capability discovery, inspection, direct capability calls, wrapper capability projection, and capability-backed draft bootstrap out of `WorkflowSurfaceHandlers` into `wf_api.capabilities.WorkflowCapabilityApi`.
**Architecture:** `WorkflowCapabilityApi` depends on `WorkflowOperationContext`, `WorkflowDraftApi`, and existing `wf_api` helper modules. It must not depend on `WfMcpService`, MCP events, MCP tools, or MCP request models. `WorkflowSurfaceHandlers` becomes a thin adapter/delegator for workflow operations.
**Tech Stack:** Python 3.14+, `wf_api.operation_context`, `wf_api.drafts`, `wf_api.runs`, `wf_api.wrapper_hints`, `wf_api.refs`, `wf_artifacts`, `wf_authoring`, `wf_core.RuntimeContext`, pytest, ruff, basedpyright.
---
## Scope
### Move In This Slice
Move these methods from `WorkflowSurfaceHandlers`:
```text
list_capabilities
inspect_capability
call_capability
create_draft_workspace_from_capability
```
Move these wrapper/capability private methods:
```text
_wrapper_artifact_for_capability_name
_wrapper_capability_summaries
_wrapper_capability_detail
_call_wrapper_artifact
```
Move or duplicate only the helpers needed by those methods:
```text
_schema_field_names
_source_id_for_capability
_artifact_capability_id
_raw_plan_from_artifact
_required_capability_payloads
_draft_name_from_capability
```
### Do Not Move In This Slice
Do not move MCP tool registration, MCP request/response Pydantic models, proxy/admin/broker runtime code, or CLI code.
### Invariants
- No public payload changes.
- No MCP tool schema changes.
- `WorkflowSurfaceHandlers` public capability method signatures stay unchanged.
- `wf_api` imports no `wf_mcp`.
- Direct raw NodeSpec calls still use `build_async_registry`.
- Direct wrapper calls still reject unsupported interrupting wrappers through `direct_wrapper_interrupt_diagnostic`.
- Full saved workflows still run through deployments, not direct capability calls.
- `create_draft_workspace_from_capability` keeps using inspect-capability wrapper hints.
---
## Design Notes
Capability extraction is the final domain split because it spans several concepts:
- live planner-visible source NodeSpecs
- saved wrapper artifacts projected as workflow-facing capabilities
- direct capability REPL calls
- wrapper calls through workflow execution
- wrapper-hint-driven draft bootstrap
Keep this in one `WorkflowCapabilityApi` for now. Do not create five tiny services unless tests prove the file is too large after extraction.
Temporary private helper duplication is allowed. A later cleanup can promote common helpers such as `artifact_capability_id`, `raw_plan_from_artifact`, and source snapshots into better shared modules. Do not widen this slice just to make helper names perfect.
---
## Task 1: Create `wf_api.capabilities`
**Files:**
- Create: `src/wf_api/capabilities.py`
- Modify: `src/wf_api/__init__.py`
- Test: `tests/wf_api/test_capability_api.py`
- [ ] **Step 1: Create service skeleton**
Create `src/wf_api/capabilities.py`:
```python
from __future__ import annotations
from collections.abc import Sequence
from typing import Any
from wf_artifacts import (
DependencyDiagnostic,
DiagnosticSeverity,
WorkflowArtifact,
WorkflowCapabilityRef,
)
from wf_authoring import build_async_registry
from wf_core import RuntimeContext
from wf_core.models.steps import InputBinding, OutputBinding
from wf_core.paths import GraphSourcePath
from wf_platform import CapabilitySource, page_items
from .drafts import WorkflowDraftApi
from .models import RawWorkflowPlan
from .operation_context import WorkflowOperationContext
from .refs import parse_workflow_surface_capability_id
from .saved_subgraphs import direct_wrapper_interrupt_diagnostic
from .wrapper_hints import (
workflow_output_schema_for_authoring,
wrapper_hints_for_capability,
)
class WorkflowCapabilityApi:
"""Workflow-facing capability discovery, inspection, and REPL calls.
This service owns the source/wrapper projection, while adapter-specific MCP
tool schemas stay outside wf_api.
"""
def __init__(self, context: WorkflowOperationContext) -> None:
self.context = context
self.drafts = WorkflowDraftApi(context)
```
- [ ] **Step 2: Add local list helpers**
Add local helpers rather than importing `wf_mcp.shared`:
```python
def _matches_query(*values: object, query: str | None) -> bool:
if query is None:
return True
needle = query.strip().casefold()
if not needle:
return True
return any(needle in str(value).casefold() for value in values if value is not None)
def _paged_list_payload(
key: str,
items: Sequence[dict[str, Any]],
*,
cursor: str | None,
limit: int,
) -> dict[str, Any]:
page = page_items(items, cursor=cursor, limit=limit)
return {key: list(page.items), "next_cursor": page.next_cursor, "total": page.total}
```
This duplicates current list behavior without importing MCP shared helpers into
`wf_api`.
- [ ] **Step 3: Export capability service**
In `src/wf_api/__init__.py`:
```python
from .capabilities import WorkflowCapabilityApi
```
Add `"WorkflowCapabilityApi"` to `__all__`.
---
## Task 2: Move Discovery And Inspection
**Files:**
- Modify: `src/wf_api/capabilities.py`
- [ ] **Step 1: Move `list_capabilities`**
Move the existing handler body into:
```python
async def list_capabilities(
self,
*,
query: str | None = None,
source_id: str | None = None,
cursor: str | None = None,
limit: int = 50,
) -> dict[str, Any]:
...
```
Required replacements:
```python
self.service.capability_sources -> self.context.capability_sources
self._wrapper_capability_summaries(...) -> self._wrapper_capability_summaries(...)
matches_query(...) -> _matches_query(...)
paged_list_payload(...) -> _paged_list_payload(...)
```
Preserve sorting and response shape.
- [ ] **Step 2: Move `inspect_capability`**
Move the existing handler body into:
```python
async def inspect_capability(self, *, qualified_name: str) -> dict[str, Any]:
...
```
Required replacements:
```python
self.service.capability_sources -> self.context.capability_sources
self._wrapper_capability_detail(...) -> self._wrapper_capability_detail(...)
```
Preserve:
- enabled/planner visibility filtering
- wrapper detail fallback
- `KeyError(f"unknown workflow capability {qualified_name!r}")`
- `wrapper_hints` payload
- [ ] **Step 3: Add helper functions**
Move or duplicate:
```text
_schema_field_names
_artifact_capability_id
_required_capability_payloads
```
Do not import private helpers from `wf_api.artifacts` or `wf_api.runs` in this slice unless the import is already public. Private duplication is acceptable here.
---
## Task 3: Move Wrapper Capability Projection
**Files:**
- Modify: `src/wf_api/capabilities.py`
- [ ] **Step 1: Add artifact store helper**
Add:
```python
def _artifact_store(self):
return self.context.artifact_store
```
Do not raise from this helper. Existing wrapper projection returns no wrapper
rows/details when artifact store is absent.
- [ ] **Step 2: Move `_wrapper_artifact_for_capability_name`**
Move the existing method, replacing:
```python
self.service.artifact_store -> self.context.artifact_store
```
Preserve current behavior:
- invalid capability ids return `None`
- non-wrapper artifacts return `None`
- missing artifact store returns `None`
- missing artifact id/version returns `None`
- [ ] **Step 3: Move `_wrapper_capability_summaries`**
Move the method and replace:
```python
matches_query(...) -> _matches_query(...)
```
Preserve `source_id not in {None, "workflow"}` filtering and the existing row shape.
- [ ] **Step 4: Move `_wrapper_capability_detail`**
Move the method unchanged except helper references now point to local functions.
Preserve:
- `kind == "wrapper_artifact"`
- `required_capabilities`
- `wrapper_hints`
- output/input schema fields
---
## Task 4: Move Direct Capability Calls
**Files:**
- Modify: `src/wf_api/capabilities.py`
- [ ] **Step 1: Move `call_capability`**
Move the existing handler body into:
```python
async def call_capability(
self,
*,
qualified_name: str,
payload: dict[str, Any],
deployment_id: str | None = None,
) -> dict[str, Any]:
...
```
Required replacements:
```python
self.service._get_qualified_spec(qualified_name) -> self.context.specs.get_qualified_spec(qualified_name)
self.service.capability_sources -> self.context.capability_sources
self._call_wrapper_artifact(...) -> self._call_wrapper_artifact(...)
```
Preserve direct NodeSpec call behavior:
- build handler with `build_async_registry(spec)[spec.name]`
- pass `RuntimeContext(current_node_id=spec.name)`
- catch `Exception` and return `capability_call_failed` diagnostic payload
- successful response returns `kind: "node_spec"` and empty diagnostics
- [ ] **Step 2: Move `_call_wrapper_artifact`**
Move the wrapper call method into `WorkflowCapabilityApi`.
Required replacements:
```python
self.service.artifact_store -> self.context.artifact_store
self.service.run_workflow_from_plan(...) -> self.context.runtime.run_workflow_from_plan(...)
```
Call runtime with the same deployment/artifact arguments:
```python
run = await self.context.runtime.run_workflow_from_plan(
plan,
payload,
deployment=deployment,
artifact=artifact,
)
```
Preserve:
- `direct_wrapper_interrupt_diagnostic` rejection
- deployment target validation
- `kind: "wrapper_artifact"`
- `outcome: run.status.value`
- `output: run.output`
- [ ] **Step 3: Move raw plan helper**
Move or duplicate:
```text
_raw_plan_from_artifact
_plan_field
```
Do not import private `_raw_plan_from_artifact` from `wf_api.runs` unless you
first make it public. Keeping a local copy is acceptable for this slice.
---
## Task 5: Move Capability-Backed Draft Bootstrap
**Files:**
- Modify: `src/wf_api/capabilities.py`
- [ ] **Step 1: Move `create_draft_workspace_from_capability`**
Move the existing handler body into `WorkflowCapabilityApi`.
Required replacements:
```python
capability = await self.inspect_capability(...)
result = await self.drafts.create_minimal_draft_workspace(...)
```
Preserve:
- wrapper-hint defaults
- explicit `input` overrides `input_map`
- explicit `output` overrides `output_map`
- returned `wrapper_hints`
- returned `next_actions`
- [ ] **Step 2: Move `_draft_name_from_capability`**
Move or duplicate:
```python
def _draft_name_from_capability(capability_name: str) -> str:
"""Return a stable draft name when caller does not provide one."""
return capability_name.replace(".", "_").replace("-", "_")
```
---
## Task 6: Wire `WorkflowSurfaceHandlers`
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- [ ] **Step 1: Add import and instance**
Add:
```python
from wf_api.capabilities import WorkflowCapabilityApi
```
In `WorkflowSurfaceHandlers.__init__`, reuse the existing operation context:
```python
context = context_from_service(service)
self._capabilities = WorkflowCapabilityApi(context)
self._drafts = WorkflowDraftApi(context)
self._artifacts = WorkflowArtifactApi(context)
self._deployments = WorkflowDeploymentApi(context)
self._runs = WorkflowRunApi(context)
```
- [ ] **Step 2: Delegate moved methods**
Replace method bodies for:
```text
list_capabilities
inspect_capability
call_capability
create_draft_workspace_from_capability
```
Example:
```python
async def inspect_capability(self, *, qualified_name: str) -> dict[str, Any]:
"""Return one planner-visible workflow capability contract."""
return await self._capabilities.inspect_capability(qualified_name=qualified_name)
```
- [ ] **Step 3: Remove moved private methods**
Remove these from `handlers.py` after delegation:
```text
_wrapper_artifact_for_capability_name
_wrapper_capability_summaries
_wrapper_capability_detail
_call_wrapper_artifact
```
Then run:
```powershell
rg -n "_schema_field_names|_source_id_for_capability|_artifact_capability_id|_raw_plan_from_artifact|_plan_field|_draft_name_from_capability|_required_capability_payloads" src/wf_mcp/workflow_surface/handlers.py
```
Remove each helper only if it has no remaining handler caller. The target after
4E should be close to zero private workflow-domain helpers in `handlers.py`.
- [ ] **Step 4: Prune imports**
Use `ruff check` to remove unused imports. Likely candidates:
```text
DependencyDiagnostic
DiagnosticSeverity
WorkflowArtifact
WorkflowCapabilityRef
CapabilitySource
build_async_registry
RuntimeContext
direct_wrapper_interrupt_diagnostic
workflow_output_schema_for_authoring
wrapper_hints_for_capability
parse_workflow_surface_capability_id
matches_query
paged_list_payload
```
Do not remove imports still needed by method signatures such as `InputBinding`,
`OutputBinding`, `GraphSourcePath`, `TraceRange`, or `RawWorkflowPlan`.
---
## Task 7: Add Focused Capability API Tests
**Files:**
- Create: `tests/wf_api/test_capability_api.py`
- [ ] **Step 1: Cover live source capability listing and inspection**
Build a service with `echo_tool`, adapt with `context_from_service`, instantiate
`WorkflowCapabilityApi`, and assert:
```python
listed = asyncio.run(api.list_capabilities())
assert listed["total"] >= 1
assert any(item["name"] == "demo.personal.echo_tool" for item in listed["capabilities"])
detail = asyncio.run(api.inspect_capability(qualified_name="demo.personal.echo_tool"))
assert detail["name"] == "demo.personal.echo_tool"
assert "wrapper_hints" in detail
```
- [ ] **Step 2: Cover direct NodeSpec call**
Call:
```python
result = asyncio.run(
api.call_capability(
qualified_name="demo.personal.echo_tool",
payload={"text": "hello"},
)
)
```
Assert stable fields:
```python
assert result["kind"] == "node_spec"
assert result["outcome"] == "ok"
assert result["diagnostics"] == []
```
- [ ] **Step 3: Cover saved wrapper projection**
Save a wrapper artifact and assert:
- `list_capabilities(source_id="workflow")` includes `kind == "wrapper_artifact"`
- `inspect_capability(qualified_name="workflow.<id>.v<version>")` returns wrapper detail
- `call_capability(...)` executes the wrapper through runtime and returns `kind == "wrapper_artifact"`
Use existing artifact helpers where possible. Do not duplicate entire run tests.
- [ ] **Step 4: Cover capability-backed draft bootstrap**
Call `create_draft_workspace_from_capability(...)` and assert:
```python
assert result["workspace_id"] == "echo_ws"
assert result["revision"] == 1
assert "wrapper_hints" in result
assert "next_actions" in result
```
Fetch the workspace through `WorkflowDraftApi` and assert the draft uses the
expected capability name.
- [ ] **Step 5: Cover handler delegation smoke**
Compare stable fields from handler and direct API for one method:
```python
handler_result = asyncio.run(handlers.inspect_capability(qualified_name=name))
api_result = asyncio.run(api.inspect_capability(qualified_name=name))
assert handler_result["name"] == api_result["name"]
assert handler_result["kind"] == api_result["kind"]
```
Do not duplicate every capability behavior test in both layers.
---
## Task 8: Verification
- [ ] **Step 1: Run focused capability tests**
```powershell
uv run pytest tests/wf_api/test_capability_api.py tests/wf_mcp/workflow_surface/test_capabilities.py -q
```
If `tests/wf_mcp/workflow_surface/test_capabilities.py` does not exist, run the
closest existing workflow-surface capability tests discovered by `rg -n "call_capability|inspect_capability|list_capabilities" tests/wf_mcp`.
Expected: pass.
- [ ] **Step 2: Run adjacent API tests**
```powershell
uv run pytest tests/wf_api/test_drafts_service.py tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py tests/wf_api/test_run_api.py -q
```
Expected: pass.
- [ ] **Step 3: Run import-direction test**
```powershell
uv run pytest tests/wf_api/test_import_direction.py -q
```
Expected: pass; `wf_api` has no `wf_mcp` imports.
- [ ] **Step 4: Run ruff on touched files**
```powershell
uv run ruff check src/wf_api/capabilities.py src/wf_api/__init__.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_capability_api.py
```
Expected: all checks pass.
- [ ] **Step 5: Run basedpyright on touched files**
```powershell
uv run basedpyright --level error src/wf_api/capabilities.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_capability_api.py
```
Expected: `0 errors`.
- [ ] **Step 6: Optional full suite**
```powershell
uv run pytest -q
```
Expected: full suite passes with the projects existing skipped/xfailed counts.
---
## Self-Review Checklist
- `wf_api.capabilities` imports no `wf_mcp`.
- `WorkflowSurfaceHandlers` public capability signatures are unchanged.
- `create_draft_workspace_from_capability` moved with capability inspection.
- Direct NodeSpec calls still work.
- Saved wrapper discovery and direct wrapper calls still work.
- Full saved workflows still require deployments.
- No public payload shape changed.
- No MCP schema changed.
- Handler is now mostly a thin compatibility adapter over `wf_api` domain services.
## Follow-Up Cleanup After 4E
After this slice lands, consider a cleanup plan to promote shared helpers:
```text
wf_api.runs._raw_plan_from_artifact -> wf_api.artifact_plans.raw_plan_from_artifact
wf_api.capabilities._artifact_capability_id -> wf_api.artifact_refs.artifact_capability_id
wf_api.deployments._available_sources -> wf_api.source_snapshots.available_sources_from_capability_sources
```
Do not do that cleanup inside 4E unless it is required to remove circular imports
or duplicate behavior bugs.
@@ -0,0 +1,836 @@
# wf_api Slice 5A/5B Helper Consolidation Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Consolidate duplicated workflow API listing, artifact-plan, source-snapshot, and dependency-summary helpers into protocol-neutral `wf_api` modules without changing public payloads.
**Architecture:** Slice 5A moves workflow list helpers out of MCP-named modules for workflow API callers. Slice 5B promotes duplicated private helpers from `wf_api.capabilities`, `wf_api.artifacts`, `wf_api.drafts`, `wf_api.runs`, and `wf_api.deployments` into focused `wf_api` helper modules. `wf_mcp.shared.pagination` stays untouched because proxy/admin code still uses it.
**Tech Stack:** Python 3.14, Pydantic v2, `wf_api`, `wf_artifacts`, `wf_platform.page_items`, pytest, ruff, basedpyright.
---
## Scope
### In Scope
- Create `src/wf_api/listing.py` for `matches_query` and `paged_list_payload`.
- Create `src/wf_api/artifact_plans.py` for `raw_plan_from_artifact`, `plan_field`, and `plan_nodes`.
- Create `src/wf_api/artifact_refs.py` for `artifact_capability_id`.
- Create `src/wf_api/capability_requirements.py` for `required_capability_payloads`, `observed_node_specs`, and `required_capabilities_for_plan`.
- Create `src/wf_api/source_snapshots.py` for deployment/run source snapshot helpers if currently duplicated in `deployments.py` and `runs.py`.
- Update `src/wf_api/{capabilities,artifacts,drafts,runs,deployments}.py` to import the shared helpers.
- Update `src/wf_mcp/workflow_surface/handlers.py` no-store `list_artifacts` fallback to use `wf_api.listing.paged_list_payload`.
- Add focused `wf_api` tests for the promoted helpers.
### Out of Scope
- Do not move or delete `wf_mcp.shared.pagination`; `src/wf_mcp/proxy/tools.py` still uses it.
- Do not move event primitives in this slice.
- Do not delete `wf_mcp.workflow_surface.*` compatibility shims.
- Do not change MCP tool names, response payload shapes, pagination semantics, artifact IDs, or capability IDs.
- Do not move request/response Pydantic models out of `wf_mcp.workflow_surface.models`.
## File Map
| File | Responsibility |
| --- | --- |
| `src/wf_api/listing.py` | Workflow API list filtering and common paged response payloads. |
| `src/wf_api/artifact_plans.py` | Safe extraction of `RawWorkflowPlan` and plan node dictionaries from saved artifacts. |
| `src/wf_api/artifact_refs.py` | Stable workflow artifact capability IDs. |
| `src/wf_api/capability_requirements.py` | Required-capability payloads and observed NodeSpec inventory projection. |
| `src/wf_api/source_snapshots.py` | Serializable source snapshots used by deployment validation and run resume checks. |
| `src/wf_api/__init__.py` | Optional re-exports for public helper symbols. |
| `src/wf_api/capabilities.py` | Remove local duplicates; import shared helpers. |
| `src/wf_api/artifacts.py` | Remove local duplicates; import shared helpers. |
| `src/wf_api/drafts.py` | Remove local duplicates; import shared helpers. |
| `src/wf_api/runs.py` | Remove local `raw_plan_from_artifact`; import shared helpers. |
| `src/wf_api/deployments.py` | Import source snapshot helper if duplicated there. |
| `src/wf_mcp/workflow_surface/handlers.py` | Stop importing workflow list helper from `wf_mcp.shared`. |
| `tests/wf_api/test_listing.py` | Unit tests for query matching and paged payload shape. |
| `tests/wf_api/test_artifact_helpers.py` | Unit tests for plan extraction, artifact refs, and dependency helper outputs. |
| `tests/wf_api/test_import_direction.py` | Existing guard; must continue passing. |
---
## Task 1: Add `wf_api.listing`
**Files:**
- Create: `src/wf_api/listing.py`
- Test: `tests/wf_api/test_listing.py`
- Modify: `src/wf_api/__init__.py`
- [ ] **Step 1: Write focused listing tests**
Create `tests/wf_api/test_listing.py`:
```python
from __future__ import annotations
from wf_api.listing import matches_query, paged_list_payload
def test_matches_query_accepts_empty_or_missing_query() -> None:
assert matches_query("Alpha", query=None) is True
assert matches_query("Alpha", query=" ") is True
def test_matches_query_searches_non_none_values_case_insensitively() -> None:
assert matches_query(None, "Demo Echo", query="echo") is True
assert matches_query(None, "Demo Echo", query="missing") is False
def test_paged_list_payload_preserves_common_shape() -> None:
payload = paged_list_payload(
"nodes",
[{"name": "a"}, {"name": "b"}, {"name": "c"}],
cursor=None,
limit=2,
)
assert payload["nodes"] == [{"name": "a"}, {"name": "b"}]
assert payload["total"] == 3
assert payload["next_cursor"] is not None
```
- [ ] **Step 2: Run the tests and verify import failure**
Run:
```bash
uv run pytest tests/wf_api/test_listing.py -q
```
Expected: fail because `wf_api.listing` does not exist.
- [ ] **Step 3: Add `src/wf_api/listing.py`**
```python
from __future__ import annotations
from collections.abc import Sequence
from typing import Any, TypeVar
from wf_platform import page_items
T = TypeVar("T")
def matches_query(*values: object, query: str | None) -> bool:
"""Return whether a compact discovery row matches a human search query."""
if query is None:
return True
needle = query.strip().casefold()
if not needle:
return True
return any(needle in str(value).casefold() for value in values if value is not None)
def paged_list_payload(
key: str,
items: Sequence[T],
*,
cursor: str | None,
limit: int,
) -> dict[str, Any]:
"""Build the shared workflow API list response shape."""
page = page_items(items, cursor=cursor, limit=limit)
return {
key: list(page.items),
"next_cursor": page.next_cursor,
"total": page.total,
}
```
- [ ] **Step 4: Export helpers from `src/wf_api/__init__.py`**
Add imports:
```python
from .listing import matches_query, paged_list_payload
```
Add to `__all__`:
```python
"matches_query",
"paged_list_payload",
```
- [ ] **Step 5: Run listing tests**
Run:
```bash
uv run pytest tests/wf_api/test_listing.py -q
```
Expected: pass.
---
## Task 2: Route Current Workflow API Listing Calls Through `wf_api.listing`
**Files:**
- Modify: `src/wf_api/capabilities.py`
- Modify: `src/wf_api/artifacts.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: existing `tests/wf_api/test_capability_api.py`, `tests/wf_api/test_artifact_api.py`
- [ ] **Step 1: Replace local listing helpers in `src/wf_api/capabilities.py`**
Remove local `_matches_query` and `_paged_list_payload`.
Add:
```python
from .listing import matches_query, paged_list_payload
```
Replace calls:
```python
_matches_query(...)
```
with:
```python
matches_query(...)
```
Replace calls:
```python
_paged_list_payload(...)
```
with:
```python
paged_list_payload(...)
```
- [ ] **Step 2: Replace local listing helpers in `src/wf_api/artifacts.py`**
Remove local `_matches_query`, `_paged_list_payload`, the local `TypeVar`, and now-unused `wf_platform.page_items` import.
Add:
```python
from .listing import matches_query, paged_list_payload
```
Replace local helper calls the same way as Task 2 Step 1.
- [ ] **Step 3: Stop workflow handler fallback from importing `wf_mcp.shared` listing**
In `src/wf_mcp/workflow_surface/handlers.py`, replace:
```python
from ..shared import paged_list_payload
```
with:
```python
from wf_api.listing import paged_list_payload
```
The no-store fallback must keep returning:
```python
return paged_list_payload("nodes", [], cursor=cursor, limit=limit)
```
- [ ] **Step 4: Run focused tests**
Run:
```bash
uv run pytest tests/wf_api/test_listing.py tests/wf_api/test_capability_api.py tests/wf_api/test_artifact_api.py tests/wf_api/test_import_direction.py -q
```
Expected: pass.
---
## Task 3: Add Artifact Plan And Artifact Ref Helpers
**Files:**
- Create: `src/wf_api/artifact_plans.py`
- Create: `src/wf_api/artifact_refs.py`
- Test: `tests/wf_api/test_artifact_helpers.py`
- [ ] **Step 1: Add failing tests for artifact helper behavior**
Create `tests/wf_api/test_artifact_helpers.py` with the imports below. If a helper fixture for artifacts already exists in nearby tests, use it; otherwise construct the minimal `WorkflowArtifact` inline with valid fields copied from existing `tests/wf_api/test_artifact_api.py`.
```python
from __future__ import annotations
import pytest
from wf_api.artifact_plans import plan_field, plan_nodes, raw_plan_from_artifact
from wf_api.artifact_refs import artifact_capability_id
def test_artifact_capability_id_uses_workflow_ref_shape(echo_artifact) -> None:
assert artifact_capability_id(echo_artifact) == (
f"workflow.{echo_artifact.id}.v{echo_artifact.version}"
)
def test_raw_plan_from_artifact_preserves_required_plan_fields(echo_artifact) -> None:
plan = raw_plan_from_artifact(echo_artifact)
assert plan.name == echo_artifact.plan["name"]
assert plan.start == echo_artifact.plan["start"]
assert len(plan.nodes) == len(echo_artifact.plan["nodes"])
def test_plan_field_reports_missing_field(echo_artifact) -> None:
broken = echo_artifact.model_copy(
update={"plan": {key: value for key, value in echo_artifact.plan.items() if key != "start"}}
)
with pytest.raises(ValueError, match="missing plan field 'start'"):
plan_field(broken, "start")
def test_plan_nodes_returns_only_dict_nodes(echo_artifact) -> None:
artifact = echo_artifact.model_copy(
update={"plan": {**echo_artifact.plan, "nodes": [{"id": "a"}, "bad"]}}
)
assert plan_nodes(artifact) == [{"id": "a"}]
```
If there is no reusable `echo_artifact` fixture, add a private helper in this test file instead of importing fixtures across packages.
- [ ] **Step 2: Run the tests and verify import failure**
Run:
```bash
uv run pytest tests/wf_api/test_artifact_helpers.py -q
```
Expected: fail because modules do not exist.
- [ ] **Step 3: Create `src/wf_api/artifact_refs.py`**
```python
from __future__ import annotations
from wf_artifacts import WorkflowArtifact, WorkflowCapabilityRef
def artifact_capability_id(artifact: WorkflowArtifact) -> str:
"""Return the stable workflow capability name for a saved artifact."""
return str(
WorkflowCapabilityRef(
artifact_id=artifact.id,
version=artifact.version,
)
)
```
- [ ] **Step 4: Create `src/wf_api/artifact_plans.py`**
```python
from __future__ import annotations
from typing import Any
from wf_artifacts import WorkflowArtifact
from .models import RawWorkflowPlan
def raw_plan_from_artifact(artifact: WorkflowArtifact) -> RawWorkflowPlan:
"""Validate the stored raw workflow plan shape expected by runtime calls."""
return RawWorkflowPlan.model_validate(
{
"name": plan_field(artifact, "name"),
"input_schema": plan_field(artifact, "input_schema"),
"state_schema": plan_field(artifact, "state_schema"),
"output_schema": plan_field(artifact, "output_schema"),
"outcomes": artifact.plan.get("outcomes", ["ok"]),
"output": artifact.plan.get("output", []),
"start": plan_field(artifact, "start"),
"nodes": plan_field(artifact, "nodes"),
"edges": plan_field(artifact, "edges"),
}
)
def plan_field(artifact: WorkflowArtifact, field_name: str) -> Any:
"""Return one required raw-plan field with an artifact-specific error."""
try:
return artifact.plan[field_name]
except KeyError as exc:
raise ValueError(
f"workflow artifact {artifact.id}@{artifact.version} "
f"is missing plan field {field_name!r}"
) from exc
def plan_nodes(artifact: WorkflowArtifact) -> list[dict[str, Any]]:
"""Return only dict-shaped node entries from a saved raw plan."""
nodes = artifact.plan.get("nodes", [])
return [node for node in nodes if isinstance(node, dict)]
```
- [ ] **Step 5: Export helper modules if desired**
In `src/wf_api/__init__.py`, export only stable helper names if the package already re-exports helpers. If the file is intentionally selective, skip this step and keep imports module-qualified.
- [ ] **Step 6: Run helper tests**
Run:
```bash
uv run pytest tests/wf_api/test_artifact_helpers.py -q
```
Expected: pass.
---
## Task 4: Replace Duplicate Artifact Plan/Ref Helpers In Domain APIs
**Files:**
- Modify: `src/wf_api/capabilities.py`
- Modify: `src/wf_api/artifacts.py`
- Modify: `src/wf_api/runs.py`
- Test: existing capability/artifact/run API tests
- [ ] **Step 1: Update `src/wf_api/capabilities.py` imports**
Add:
```python
from .artifact_plans import raw_plan_from_artifact
from .artifact_refs import artifact_capability_id
```
Remove local `_raw_plan_from_artifact`, `_plan_field`, and `_artifact_capability_id`.
Replace:
```python
_raw_plan_from_artifact(...)
_artifact_capability_id(...)
```
with:
```python
raw_plan_from_artifact(...)
artifact_capability_id(...)
```
- [ ] **Step 2: Update `src/wf_api/artifacts.py` imports**
Add:
```python
from .artifact_plans import plan_nodes
from .artifact_refs import artifact_capability_id
```
Remove local `_plan_nodes` and `_artifact_capability_id`.
Replace calls with `plan_nodes(...)` and `artifact_capability_id(...)`.
- [ ] **Step 3: Update `src/wf_api/runs.py` imports**
Add:
```python
from .artifact_plans import raw_plan_from_artifact
```
Remove local `_raw_plan_from_artifact` and `_plan_field`.
Replace calls with `raw_plan_from_artifact(...)`.
- [ ] **Step 4: Run focused tests**
Run:
```bash
uv run pytest tests/wf_api/test_artifact_helpers.py tests/wf_api/test_capability_api.py tests/wf_api/test_artifact_api.py tests/wf_api/test_run_api.py tests/wf_api/test_import_direction.py -q
```
Expected: pass.
---
## Task 5: Add Shared Capability Requirement Helpers
**Files:**
- Create: `src/wf_api/capability_requirements.py`
- Modify: `src/wf_api/drafts.py`
- Modify: `src/wf_api/artifacts.py`
- Modify: `src/wf_api/capabilities.py`
- Test: `tests/wf_api/test_artifact_helpers.py`
- [ ] **Step 1: Add tests for requirement payload and observed specs**
Append to `tests/wf_api/test_artifact_helpers.py`:
```python
from wf_api.capability_requirements import (
observed_node_specs,
required_capability_payloads,
)
def test_required_capability_payloads_sorts_by_name(required_capabilities) -> None:
payload = required_capability_payloads(required_capabilities)
assert list(payload) == sorted(required_capabilities)
first = next(iter(payload.values()))
assert "ref" in first
assert "kind" in first
def test_observed_node_specs_projects_enabled_context_specs(operation_context) -> None:
observed = observed_node_specs(operation_context)
assert isinstance(observed, dict)
assert all(hasattr(detail, "name") for detail in observed.values())
```
If `required_capabilities` or `operation_context` fixtures do not exist, create explicit local helpers by copying the smallest valid setup from `tests/wf_api/test_artifact_api.py` or `tests/wf_api/test_capability_api.py`. Do not import private helpers from production modules.
- [ ] **Step 2: Create `src/wf_api/capability_requirements.py`**
```python
from __future__ import annotations
from typing import Any
from wf_artifacts import (
RequiredCapability,
WorkflowArtifact,
create_workflow_artifact_from_plan as build_workflow_artifact_from_plan,
)
from wf_platform import CapabilityRef, NodeSpecInventory
from .artifact_plans import plan_nodes
from .operation_context import WorkflowOperationContext
def required_capability_payloads(
requirements: dict[str, RequiredCapability],
) -> dict[str, dict[str, Any]]:
"""Return deterministic JSON payloads for required capabilities."""
return {
name: capability.model_dump(mode="json")
for name, capability in sorted(requirements.items())
}
def observed_node_specs(
context: WorkflowOperationContext,
) -> dict[str, NodeSpecInventory]:
"""Project current executable specs into serializable observed contracts."""
observed: dict[str, NodeSpecInventory] = {}
for source in context.capability_sources.values():
inventory = source.as_inventory()
observed.update(
{detail.name: detail for detail in inventory.capabilities.node_spec_details}
)
return observed
def required_capabilities_for_plan(
plan: dict[str, Any],
*,
source_bindings: dict[str, str] | None,
context: WorkflowOperationContext,
) -> dict[str, RequiredCapability]:
"""Infer a draft dependency summary without persisting an artifact."""
artifact = build_workflow_artifact_from_plan(
artifact_id="draft_preview",
version=1,
title="Draft Preview",
plan=plan,
outcomes=("completed",),
source_bindings=source_bindings,
observed_node_specs=observed_node_specs(context),
)
requirements = artifact.required_capability_map()
for node in plan_nodes(artifact):
raw_ref = node.get("node")
if not isinstance(raw_ref, str) or raw_ref in requirements:
continue
try:
parsed = CapabilityRef.parse(raw_ref)
except ValueError:
continue
requirements[raw_ref] = RequiredCapability(
ref=parsed,
kind="node_spec",
)
return requirements
```
- [ ] **Step 3: Update `src/wf_api/drafts.py`**
Import:
```python
from .capability_requirements import (
observed_node_specs,
required_capabilities_for_plan,
required_capability_payloads,
)
```
Replace:
```python
_required_capability_payloads(...)
_required_capabilities_for_plan(...)
_observed_node_specs(...)
```
with:
```python
required_capability_payloads(...)
required_capabilities_for_plan(...)
observed_node_specs(...)
```
Remove local `_required_capabilities_for_plan`, `_required_capability_payloads`, `_observed_node_specs`, and `_plan_nodes` if no longer used.
- [ ] **Step 4: Update `src/wf_api/artifacts.py`**
Import:
```python
from .capability_requirements import (
observed_node_specs,
required_capability_payloads,
)
```
Replace local helper calls and remove local duplicate helper definitions.
- [ ] **Step 5: Update `src/wf_api/capabilities.py`**
Import:
```python
from .capability_requirements import required_capability_payloads
```
Replace local helper calls and remove the local duplicate helper definition.
- [ ] **Step 6: Run focused tests**
Run:
```bash
uv run pytest tests/wf_api/test_artifact_helpers.py tests/wf_api/test_drafts_service.py tests/wf_api/test_artifact_api.py tests/wf_api/test_capability_api.py tests/wf_api/test_import_direction.py -q
```
Expected: pass.
---
## Task 6: Add Source Snapshot Helper If Duplicated
**Files:**
- Create: `src/wf_api/source_snapshots.py`
- Modify: `src/wf_api/deployments.py`
- Modify: `src/wf_api/runs.py`
- Test: existing deployment/run API tests
- [ ] **Step 1: Inspect current source snapshot helper names**
Run:
```bash
rg -n "_available_sources|AvailableSource|AvailableCapability|capability_name" src/wf_api src/wf_mcp/workflow_surface
```
Expected: identify whether `_available_sources` still exists in `src/wf_api/deployments.py` and is imported by `src/wf_api/runs.py`.
- [ ] **Step 2: Create `src/wf_api/source_snapshots.py` only if a helper exists**
If `_available_sources` exists, move it as:
```python
from __future__ import annotations
from collections.abc import Mapping
from wf_artifacts import AvailableCapability, AvailableSource
from wf_platform import CapabilitySource
def available_sources_from_capability_sources(
sources: Mapping[str, CapabilitySource],
) -> dict[str, AvailableSource]:
"""Project live capability sources into pinned resume-validation snapshots."""
return {
source_id: AvailableSource(
id=source.id,
capabilities={
name: AvailableCapability(name=name)
for name in source.capabilities.node_specs
},
)
for source_id, source in sources.items()
}
```
If the existing helper carries more fields than `name`, preserve those fields exactly. Do not simplify the payload.
- [ ] **Step 3: Update deployment/run imports**
Replace duplicated or cross-domain imports with:
```python
from .source_snapshots import available_sources_from_capability_sources
```
Use it wherever resume/deployment validation needs current source snapshots.
- [ ] **Step 4: Run focused tests**
Run:
```bash
uv run pytest tests/wf_api/test_deployment_api.py tests/wf_api/test_run_api.py tests/wf_api/test_import_direction.py -q
```
Expected: pass.
---
## Task 7: Remove Duplicate Private Helpers And Guard Imports
**Files:**
- Modify: `src/wf_api/capabilities.py`
- Modify: `src/wf_api/artifacts.py`
- Modify: `src/wf_api/drafts.py`
- Modify: `src/wf_api/runs.py`
- Modify: `src/wf_api/deployments.py`
- Test: import-direction guard
- [ ] **Step 1: Search for leftover duplicated helpers**
Run:
```bash
rg -n "def _matches_query|def _paged_list_payload|def _raw_plan_from_artifact|def _plan_field|def _artifact_capability_id|def _required_capability_payloads|def _observed_node_specs|def _plan_nodes|def _available_sources" src/wf_api src/wf_mcp/workflow_surface
```
Expected:
- No duplicate helper definitions in domain API modules.
- `src/wf_mcp/shared/listing.py` may still define `matches_query` and `paged_list_payload`; leave it alone unless no MCP code imports it.
- `src/wf_mcp/shared/pagination.py` must remain.
- [ ] **Step 2: Search for forbidden workflow listing import**
Run:
```bash
rg -n "from \.\.shared import paged_list_payload|from wf_mcp.shared import paged_list_payload" src/wf_mcp/workflow_surface src/wf_api
```
Expected: no matches.
- [ ] **Step 3: Verify `wf_api` still imports no `wf_mcp`**
Run:
```bash
uv run pytest tests/wf_api/test_import_direction.py -q
```
Expected: pass.
---
## Task 8: Final Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused wf_api workflow tests**
Run:
```bash
uv run pytest tests/wf_api/test_listing.py tests/wf_api/test_artifact_helpers.py tests/wf_api/test_drafts_service.py tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py tests/wf_api/test_run_api.py tests/wf_api/test_capability_api.py tests/wf_api/test_import_direction.py -q
```
Expected: pass.
- [ ] **Step 2: Run adapter-focused workflow surface tests**
Run:
```bash
uv run pytest tests/wf_mcp/workflow_surface tests/wf_mcp/test_server.py -q
```
Expected: pass. If `tests/wf_mcp/workflow_surface` does not exist in this checkout, run the nearest existing workflow-surface test files discovered with `rg -n "WorkflowSurfaceHandlers|register_workflow_tools" tests/wf_mcp`.
- [ ] **Step 3: Run lint and type checks**
Run:
```bash
uv run ruff check src/wf_api src/wf_mcp/workflow_surface tests/wf_api
uv run ruff format --check src/wf_api src/wf_mcp/workflow_surface tests/wf_api
uv run basedpyright --level error
```
Expected: all pass with zero new diagnostics.
- [ ] **Step 4: Optional full suite**
Run:
```bash
uv run pytest -q
```
Expected: existing suite status remains at least as good as before this slice.
---
## Handoff Report Requirements
When done, report:
- Files created.
- Files modified.
- Exact helpers moved and their new canonical module.
- Any helpers intentionally left in place and why.
- Verification commands and outputs.
- Any deviations from this plan.
## Self-Review
- Spec coverage: covers roadmap Slice 5 listing cleanup and post-Slice-4 helper promotion. Event primitives are explicitly deferred because their semantics are larger than this helper cleanup.
- Placeholder scan: no `TBD`, no unspecified edge handling, no “write tests for above” without concrete examples.
- Type consistency: helper names are stable and public names omit leading underscores; domain modules should import from `wf_api.*`, never `wf_mcp.*`.
@@ -0,0 +1,511 @@
# wf_api Store Ownership Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make workflow artifact, draft workspace, and run store ownership explicit so `wf_api`, CLI, MCP, and future HTTP entrypoints can share stores without relying on `WfMcpService.__post_init__`.
**Architecture:** Add a protocol-neutral `WorkflowStores` bundle in `wf_api` that groups the three workflow stores. MCP config construction remains responsible for creating file-backed stores from `BrokerConfig.store_root`; `WfMcpService` receives stores but no longer manufactures them from its MCP `Store`. Existing process-local behavior stays intact through `build_service_from_config`.
**Tech Stack:** Python 3.14, dataclasses, `wf_api`, `wf_artifacts`, `wf_mcp`, pytest, ruff, basedpyright.
---
## Current Problem
`src/wf_mcp/broker/service/core.py` currently does this in `WfMcpService.__post_init__`:
```python
if self.artifact_store is None:
self.artifact_store = FileWorkflowArtifactStore(_store_root(self.store))
if self.draft_workspace_store is None:
self.draft_workspace_store = FileDraftWorkspaceStore(_store_root(self.store))
if self.run_store is None:
self.run_store = FileRunStore(_store_root(self.store))
```
That makes a protocol-specific service decide protocol-neutral workflow persistence. It also hides missing-store tests because `WfMcpService(store=FileStore(...))` silently creates workflow stores.
`src/wf_mcp/broker/config.py::build_service_from_config` already does the right thing by passing all three stores explicitly. This slice preserves that behavior and removes the fallback.
## Target Ownership Rule
- `wf_artifacts` owns store protocols and file store implementations.
- `wf_api` may group protocol-neutral workflow stores into a small DTO.
- `wf_mcp` owns MCP config loading and calls the DTO factory for file-backed process-local stores.
- `WfMcpService` owns broker state, connections, adapters, source catalogs, events, and execution wiring.
- `WfMcpService` does not create workflow artifact/draft/run stores by guessing from the MCP catalog/auth store.
## Files
- Create: `src/wf_api/stores.py`
- Modify: `src/wf_api/__init__.py`
- Modify: `src/wf_mcp/broker/config.py`
- Modify: `src/wf_mcp/broker/service/core.py`
- Modify: direct `WfMcpService(...)` tests only where they rely on implicit workflow stores
- Test: `tests/wf_api/test_stores.py`
- Test: `tests/wf_mcp/service/test_catalog.py`
- Test: `tests/wf_mcp/test_broker_server.py`
---
### Task 1: Add Protocol-Neutral Store Bundle
**Files:**
- Create: `src/wf_api/stores.py`
- Modify: `src/wf_api/__init__.py`
- Test: `tests/wf_api/test_stores.py`
- [ ] **Step 1: Write failing store bundle tests**
Create `tests/wf_api/test_stores.py`:
```python
from __future__ import annotations
from wf_api.stores import WorkflowStores, file_workflow_stores
from wf_artifacts import (
FileDraftWorkspaceStore,
FileRunStore,
FileWorkflowArtifactStore,
)
from tests.wf_mcp.test_support import local_temp_root
def test_file_workflow_stores_constructs_all_three_file_stores() -> None:
root = local_temp_root() / "wf_api_file_workflow_stores"
stores = file_workflow_stores(root)
assert isinstance(stores, WorkflowStores)
assert isinstance(stores.artifact_store, FileWorkflowArtifactStore)
assert isinstance(stores.draft_workspace_store, FileDraftWorkspaceStore)
assert isinstance(stores.run_store, FileRunStore)
assert stores.artifact_store.root == root
assert stores.draft_workspace_store.root == root
assert stores.run_store.root == root
def test_wf_api_exports_workflow_stores() -> None:
from wf_api import WorkflowStores as ExportedWorkflowStores
from wf_api import file_workflow_stores as exported_file_workflow_stores
assert ExportedWorkflowStores is WorkflowStores
assert exported_file_workflow_stores is file_workflow_stores
```
- [ ] **Step 2: Run failing tests**
Run:
```bash
uv run pytest tests\wf_api\test_stores.py -q
```
Expected: import failure for `wf_api.stores`.
- [ ] **Step 3: Implement `wf_api.stores`**
Create `src/wf_api/stores.py`:
```python
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from wf_artifacts import (
DraftWorkspaceStore,
FileDraftWorkspaceStore,
FileRunStore,
FileWorkflowArtifactStore,
RunStore,
WorkflowArtifactStore,
)
@dataclass(frozen=True, slots=True)
class WorkflowStores:
"""Protocol-neutral persistence dependencies for workflow APIs."""
artifact_store: WorkflowArtifactStore
draft_workspace_store: DraftWorkspaceStore
run_store: RunStore
def file_workflow_stores(root: str | Path) -> WorkflowStores:
"""Create process-local file-backed workflow stores under one root."""
store_root = Path(root)
return WorkflowStores(
artifact_store=FileWorkflowArtifactStore(store_root),
draft_workspace_store=FileDraftWorkspaceStore(store_root),
run_store=FileRunStore(store_root),
)
__all__ = ["WorkflowStores", "file_workflow_stores"]
```
- [ ] **Step 4: Export from `wf_api`**
Update `src/wf_api/__init__.py`:
```python
from .stores import WorkflowStores, file_workflow_stores
```
Add both names to `__all__`.
- [ ] **Step 5: Verify Task 1**
Run:
```bash
uv run pytest tests\wf_api\test_stores.py -q
uv run ruff check src\wf_api\stores.py tests\wf_api\test_stores.py
uv run ruff format --check src\wf_api\stores.py tests\wf_api\test_stores.py
```
Expected: tests pass, lint pass, format pass.
---
### Task 2: Move Config Store Construction Through the Bundle
**Files:**
- Modify: `src/wf_mcp/broker/config.py`
- Test: `tests/wf_mcp/test_broker_server.py`
- [ ] **Step 1: Strengthen config construction test**
Find `tests/wf_mcp/test_broker_server.py::test_build_service_from_config_uses_store_root_for_artifacts`.
Update it to assert all three stores use the configured root:
```python
def test_build_service_from_config_uses_store_root_for_workflow_stores() -> None:
store_root = local_temp_root() / "broker_config_workflow_stores"
config = BrokerConfig(store_root=store_root, connections=[])
service = build_service_from_config(config)
assert isinstance(service.artifact_store, FileWorkflowArtifactStore)
assert isinstance(service.draft_workspace_store, FileDraftWorkspaceStore)
assert isinstance(service.run_store, FileRunStore)
assert service.artifact_store.root == store_root
assert service.draft_workspace_store.root == store_root
assert service.run_store.root == store_root
```
Ensure the test imports:
```python
from wf_artifacts import FileDraftWorkspaceStore, FileRunStore, FileWorkflowArtifactStore
```
- [ ] **Step 2: Run the focused test**
Run:
```bash
uv run pytest tests\wf_mcp\test_broker_server.py::test_build_service_from_config_uses_store_root_for_workflow_stores -q
```
Expected: pass before the config refactor, proving current behavior is covered.
- [ ] **Step 3: Update `build_service_from_config` to use `file_workflow_stores`**
In `src/wf_mcp/broker/config.py`, replace direct file store imports:
```python
from wf_api import file_workflow_stores
```
Remove:
```python
from wf_artifacts import (
FileDraftWorkspaceStore,
FileRunStore,
FileWorkflowArtifactStore,
)
```
Then update `build_service_from_config`:
```python
def build_service_from_config(config: BrokerConfig) -> WfMcpService:
"""Create a broker service with SDK adapters for configured connections."""
runtime_factory = PersistentSessionFactory()
workflow_stores = file_workflow_stores(config.store_root)
service = WfMcpService(
store=FileStore(config.store_root),
artifact_store=workflow_stores.artifact_store,
draft_workspace_store=workflow_stores.draft_workspace_store,
run_store=workflow_stores.run_store,
# Discovery can use short-lived SDK sessions. Workflow execution needs
# a persistent runtime so stateful MCP servers keep session/page state
# across sequential workflow nodes.
tool_executor=McpRuntimePool(runtime_factory.create),
)
```
- [ ] **Step 4: Verify Task 2**
Run:
```bash
uv run pytest tests\wf_mcp\test_broker_server.py::test_build_service_from_config_uses_store_root_for_workflow_stores -q
uv run ruff check src\wf_mcp\broker\config.py tests\wf_mcp\test_broker_server.py
uv run ruff format --check src\wf_mcp\broker\config.py tests\wf_mcp\test_broker_server.py
```
Expected: tests pass, lint pass, format pass.
---
### Task 3: Remove Implicit Workflow Store Creation from WfMcpService
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- Modify: `tests/wf_mcp/service/test_catalog.py`
- [ ] **Step 1: Replace the old default-store test**
Find `tests/wf_mcp/service/test_catalog.py::test_service_installs_default_draft_workspace_store`.
Replace it with:
```python
def test_service_does_not_install_workflow_stores_implicitly() -> None:
root = local_temp_root() / "service_no_implicit_workflow_stores"
service = WfMcpService(store=FileStore(root))
assert service.artifact_store is None
assert service.draft_workspace_store is None
assert service.run_store is None
```
- [ ] **Step 2: Run failing test**
Run:
```bash
uv run pytest tests\wf_mcp\service\test_catalog.py::test_service_does_not_install_workflow_stores_implicitly -q
```
Expected: fail because `WfMcpService.__post_init__` still creates stores.
- [ ] **Step 3: Remove implicit creation from `WfMcpService.__post_init__`**
In `src/wf_mcp/broker/service/core.py`, remove the imports:
```python
FileDraftWorkspaceStore,
FileRunStore,
FileWorkflowArtifactStore,
```
Remove the `_store_root` helper entirely:
```python
def _store_root(store: Store) -> Path:
"""Return the file root for stores that expose one, else use local default."""
root = getattr(store, "root", None)
return root if isinstance(root, Path) else Path(".wf_mcp_store")
```
Update `WfMcpService.__post_init__` to:
```python
def __post_init__(self) -> None:
"""Install broker-local system specs when enabled.
Workflow stores are injected by entrypoint/config construction. This service
must not guess workflow persistence from the MCP catalog/auth store because
CLI, MCP, and future HTTP frontends may share or swap those stores.
"""
if self.include_builtin_specs:
for source in builtin_sources().values():
self.register_capability_source(source)
self.register_capability_source(admin_source())
```
If `Path` becomes unused in `core.py`, remove `from pathlib import Path`.
- [ ] **Step 4: Verify Task 3**
Run:
```bash
uv run pytest tests\wf_mcp\service\test_catalog.py::test_service_does_not_install_workflow_stores_implicitly -q
uv run ruff check src\wf_mcp\broker\service\core.py tests\wf_mcp\service\test_catalog.py
uv run ruff format --check src\wf_mcp\broker\service\core.py tests\wf_mcp\service\test_catalog.py
```
Expected: tests pass, lint pass, format pass.
---
### Task 4: Fix Direct Service Tests That Need Workflow Stores
**Files:**
- Modify only tests that fail after Task 3.
- [ ] **Step 1: Run targeted workflow API/service tests**
Run:
```bash
uv run pytest tests\wf_api tests\wf_mcp\workflow_surface tests\wf_mcp\test_broker_server.py tests\wf_mcp\service -q
```
Expected: if failures appear, they should be tests that constructed `WfMcpService(store=...)` but then used workflow artifact/draft/run operations.
- [ ] **Step 2: Patch only failing tests by injecting stores explicitly**
For any failing direct `WfMcpService(...)` test that needs workflow stores, use this pattern:
```python
from wf_api import file_workflow_stores
root = local_temp_root() / "test_specific_name"
workflow_stores = file_workflow_stores(root)
service = WfMcpService(
store=FileStore(root / "mcp"),
artifact_store=workflow_stores.artifact_store,
draft_workspace_store=workflow_stores.draft_workspace_store,
run_store=workflow_stores.run_store,
)
```
Do not add stores to tests that only exercise broker catalog/admin/source behavior.
- [ ] **Step 3: Keep no-store behavior tests intact**
Tests like these should keep `artifact_store=None` through `WorkflowOperationContext` or direct service construction because they prove graceful no-store behavior:
```python
assert result["nodes"] == []
assert result["deployments"] == []
```
Do not “fix” these by injecting stores unless the test is explicitly about stored workflow data.
- [ ] **Step 4: Verify targeted tests**
Run:
```bash
uv run pytest tests\wf_api tests\wf_mcp\workflow_surface tests\wf_mcp\test_broker_server.py tests\wf_mcp\service -q
uv run ruff check tests\wf_api tests\wf_mcp\workflow_surface tests\wf_mcp\test_broker_server.py tests\wf_mcp\service
uv run ruff format --check tests\wf_api tests\wf_mcp\workflow_surface tests\wf_mcp\test_broker_server.py tests\wf_mcp\service
```
Expected: targeted tests pass and no broad fixture churn.
---
### Task 5: Document the Store Ownership Rule
**Files:**
- Modify: `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md`
- Modify: `docs/current_roadmap.md` if it already has a `wf_api` section
- [ ] **Step 1: Update the extraction map**
In `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md`, under `### Store Ownership Ambiguity`, replace the section with:
```markdown
### Store Ownership
- `wf_artifacts` owns workflow store protocols and file-backed implementations.
- `wf_api.stores.WorkflowStores` groups the artifact, draft workspace, and run stores as protocol-neutral API dependencies.
- MCP config construction creates file-backed workflow stores from `BrokerConfig.store_root` and injects them into `WfMcpService`.
- `WfMcpService.__post_init__` no longer creates workflow stores from its MCP `Store`; direct service tests must inject stores when they exercise workflow persistence.
- Future HTTP/API entrypoints should construct or receive the same `WorkflowStores` bundle instead of importing `wf_mcp`.
```
- [ ] **Step 2: Update roadmap only if there is a matching section**
If `docs/current_roadmap.md` has a `wf_api` or API extraction section, add:
```markdown
- Workflow store ownership is explicit: entrypoints construct/inject `WorkflowStores`; `WfMcpService` no longer guesses stores from the MCP store root.
```
If there is no matching section, skip this file.
- [ ] **Step 3: Verify docs are not stale**
Run:
```bash
rg -n "_store_root\\(|creates default `FileWorkflowArtifactStore`|installs default.*store" src docs tests
```
Expected: no current docs/tests claim `WfMcpService` installs default workflow stores. Historical plans may still mention old implementation; leave historical plans alone unless they are current roadmap/research docs.
---
### Task 6: Final Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused suite**
Run:
```bash
uv run pytest tests\wf_api tests\wf_mcp\workflow_surface tests\wf_mcp\test_broker_server.py tests\wf_mcp\service -q
```
Expected: all selected tests pass.
- [ ] **Step 2: Run full suite**
Run:
```bash
uv run pytest -q
```
Expected: full suite passes with the repos known skip/xfail counts.
- [ ] **Step 3: Run lint and format checks**
Run:
```bash
uv run ruff check src\wf_api src\wf_mcp tests\wf_api tests\wf_mcp
uv run ruff format --check src\wf_api src\wf_mcp tests\wf_api tests\wf_mcp
```
Expected: all checks pass.
- [ ] **Step 4: Run typecheck**
Run:
```bash
uv run basedpyright --level error
```
Expected: `0 errors, 0 warnings, 0 notes`. If the command exits nonzero only because of the known workspace enumeration warning, report that exactly.
---
## Self-Review
- Spec coverage: This plan covers explicit store creation, `WfMcpService.__post_init__` cleanup, config behavior preservation, test migration, and docs.
- Placeholder scan: No `TODO`/`TBD` placeholders remain. Historical-plan references are explicitly scoped.
- Type consistency: `WorkflowStores` uses protocol types from `wf_artifacts`; `file_workflow_stores()` returns file-backed implementations; `WfMcpService` field types do not change.
- Scope check: This does not implement FastAPI, persisted-run conflict handling, or store locking. Those are separate slices.
@@ -0,0 +1,69 @@
# wf_mcp Workflow Surface Test Thinning Ledger
This ledger records every `wf_mcp.workflow_surface` test removed or kept during
the thinning pass. Do not delete a test unless the `replacement` column points
to equal-or-stronger coverage.
| Test | Decision | Replacement / Reason |
| --- | --- | --- |
| test_artifacts::test_workflow_surface_lists_artifact_catalog_entries | keep | Handler adapter smoke test for compact artifact listing; reduced to minimal assertions. |
| test_artifacts::test_workflow_surface_pages_and_filters_artifact_catalog_entries | remove | Covered by WorkflowArtifactApi list tests plus wf_api.listing pagination tests; handler keeps one list_artifacts smoke test. |
| test_capabilities::test_workflow_surface_lists_planner_visible_capabilities | keep | Handler list smoke test; reduced to minimal adapter-path assertions. |
| test_capabilities::test_workflow_surface_filters_stdlib_capabilities_by_source | remove | Covered by WorkflowCapabilityApi source/query filtering; handler list smoke remains. |
| test_capabilities::test_workflow_surface_call_capability_returns_structured_error | keep | Protected: MCP/service behavior — structured error on capability call failure. |
| test_capabilities::test_workflow_surface_lists_saved_wrapper_capabilities | remove | Covered by WorkflowCapabilityApi saved wrapper list tests. |
| test_capabilities::test_workflow_surface_inspects_one_capability | keep | Handler inspect smoke test; ensures adapter path works for single capability inspection. |
| test_capabilities::test_workflow_surface_inspect_capability_includes_wrapper_hints | keep | Protected: MCP/service behavior — wrapper hints detail through handler. |
| test_capabilities::test_workflow_surface_inspects_saved_wrapper_capability | remove | Covered by WorkflowCapabilityApi saved wrapper inspect tests. |
| test_capabilities::test_workflow_surface_does_not_auto_map_raw_mcp_content_blocks | keep | Protected: MCP content-block mapping behavior. |
| test_deployments::test_workflow_surface_validates_deployment_dependencies | keep | Handler-level dependency validation has stronger next-action assertions than wf_api. |
| test_deployments::test_workflow_surface_validate_deployment_live_check_is_opt_in | keep | Protected: MCP service adapter — live check is opt-in. |
| test_deployments::test_workflow_surface_validate_deployment_live_check_reports_unreachable_source | keep | Protected: MCP service adapter — unreachable source reporting. |
| test_deployments::test_workflow_surface_validate_deployment_live_check_reports_missing_connection | keep | Protected: MCP service adapter — missing connection reporting. |
| test_deployments::test_workflow_surface_records_artifact_and_deployment_save_events | keep | Protected: service event recording. |
| test_deployments::test_workflow_surface_save_deployment_accepts_deployment_id_alias | keep | Protected: request alias normalization. |
| test_deployments::test_workflow_surface_deletes_deployment | keep | Protected: delete event recording. |
| test_deployments::test_workflow_surface_save_deployment_rejects_id_and_deployment_id | keep | Protected: XOR validation on id/deployment_id. |
| test_deployments::test_workflow_surface_lists_compact_deployment_summaries_and_inspects_detail | keep | Protected: compact-vs-detail response shape. |
| test_drafts::test_workflow_surface_validates_draft_without_saving | remove | Covered by WorkflowDraftApi.validate_draft via test_delegation_smoke_validate_draft_equivalence. |
| test_drafts::test_workflow_surface_rejects_unknown_draft_route_outcome_when_spec_is_known | keep | Live outcome lookup through handler; not duplicated in wf_api. |
| test_drafts::test_workflow_surface_creates_artifact_from_draft_with_binding_suggestions | keep | Binding suggestions and artifact persistence through handler. |
| test_drafts::test_workflow_surface_draft_artifact_requires_std_self_binding | keep | binding_missing diagnostic through handler/deployment integration. |
| test_drafts::test_workflow_surface_patches_draft_without_saving | remove | Covered by WorkflowDraftApi.patch_draft. |
| test_drafts::test_workflow_surface_creates_and_gets_draft_workspace | remove | Covered by WorkflowDraftApi.create_draft_workspace and get_draft_workspace assertions in tests/wf_api/test_drafts_service.py. |
| test_drafts::test_workflow_surface_lists_draft_workspaces | remove | Covered by tests/wf_api/test_drafts_service.py::test_list_draft_workspaces_returns_sorted_summaries_without_drafts. |
| test_drafts::test_workflow_surface_deletes_draft_workspace | remove | Covered by tests/wf_api/test_drafts_service.py::test_delete_draft_workspace_is_idempotent. |
| test_drafts::test_workflow_surface_patch_helpers_update_draft_workspace | remove | Covered by tests/wf_api/test_drafts_service.py::test_draft_workspace_patch_helpers_update_revision_and_bindings. |
| test_drafts::test_workflow_surface_validates_draft_workspace_with_live_outcomes | keep | Live outcome lookup through handler/service stack. |
| test_drafts::test_workflow_surface_patches_draft_workspace_by_revision | remove | Covered by WorkflowDraftApi.patch_draft_workspace. |
| test_drafts::test_workflow_surface_creates_minimal_draft_workspace_with_error_route | keep | MCP request model parsing and error route generation. |
| test_drafts::test_workflow_surface_minimal_draft_honors_explicit_error_message_source | keep | Explicit error_message_source handling. |
| test_drafts::test_minimal_draft_request_accepts_structural_error_message_source | keep | MCP Pydantic model validation (CreateMinimalDraftWorkspaceRequest). |
| test_drafts::test_workflow_surface_accepts_canonical_bindings_for_minimal_workspace | keep | Canonical InputPathBinding/OutputBinding through handler. |
| test_drafts::test_workflow_surface_creates_draft_workspace_from_capability_hints | keep | Wrapper hints and next-actions through handler. |
| test_drafts::test_workflow_surface_creates_artifact_from_workspace | keep | Artifact persistence with schema snapshots through handler. |
| test_drafts::test_workflow_surface_workspace_artifact_infers_raw_concrete_dependency | keep | Source dependency inference through handler. |
| test_drafts::test_workflow_surface_creates_wrapper_from_workspace | keep | Wrapper creation through handler. |
| test_drafts::test_workflow_surface_low_confidence_draft_returns_patch_guidance | keep | Next-action guidance with patch examples through handler. |
| test_runs::test_raw_workflow_plan_uses_core_step_and_edge_models | keep | DEVIATION: Plan said wf_api covers this, but test_raw_workflow_plan_extraction.py only tests imports (canonical, compat, identity). The surface test is the only one exercising actual RawWorkflowPlan step/edge model parsing. Kept to avoid coverage gap. |
| test_runs::test_workflow_surface_runs_non_interrupting_deployment | keep | Persisted run records, trace slicing, response model, next-actions. |
| test_runs::test_workflow_surface_failed_deployment_exposes_error_on_run_and_inspect | keep | Failed run error exposure and inspect_run. |
| test_runs::test_workflow_surface_run_deployment_can_include_trace_detail | keep | Protected: MCP TraceRange, RunDeploymentResult model validation. |
| test_runs::test_workflow_surface_run_deployment_can_read_empty_trace_range | keep | Protected: empty trace range behavior. |
| test_runs::test_workflow_surface_runs_deployment_with_bound_node_spec_dependency | keep | Protected: logical source binding. |
| test_runs::test_workflow_surface_runs_artifact_created_from_concrete_node_ref | keep | Protected: concrete node ref artifact creation and run. |
| test_runs::test_workflow_surface_detects_drift_from_saved_node_spec_snapshot | keep | Protected: schema drift detection. |
| test_runs::test_workflow_surface_runs_deployment_with_bound_reducer_dependency | keep | Protected: reducer dependency integration. |
| test_wrappers::test_workflow_surface_creates_wrapper_artifact_from_plan | keep | Handler integration: wrapper artifact creation from plan. |
| test_wrappers::test_workflow_surface_creates_artifact_with_logical_node_refs | keep | Handler integration: logical node ref resolution. |
| test_wrappers::test_workflow_surface_calls_saved_wrapper_artifact | keep | Handler integration: saved wrapper direct call. |
| test_wrappers::test_workflow_surface_calls_live_node_spec_with_self_describing_response | keep | Handler integration: live node spec call. |
| test_wrappers::test_workflow_surface_calls_saved_wrapper_artifact_with_deployment_bindings | keep | Protected: saved wrapper call with deployment bindings. |
| test_next_actions::test_next_actions_from_high_confidence_wrapper_hints_can_validate | keep | NextActions model unit test. |
| test_next_actions::test_next_actions_from_low_confidence_wrapper_hints_can_patch | keep | NextActions model unit test. |
| test_next_actions::test_next_actions_from_runnable_deployment_recommends_run | keep | NextActions model unit test. |
| test_next_actions::test_next_actions_from_unrunnable_deployment_recommends_validation_retry | keep | NextActions model unit test. |
| test_next_actions::test_next_actions_from_completed_run_has_no_required_next_tool | keep | NextActions model unit test. |
| test_next_actions::test_next_actions_from_failed_run_recommends_bounded_trace | keep | NextActions model unit test. |
| test_next_actions::test_next_actions_from_interrupted_run_recommends_resume | keep | NextActions model unit test. |
| test_next_actions::test_workflow_surface_next_actions_shim_reexports_canonical_model | keep | Shim re-export verification. |
@@ -0,0 +1,545 @@
# wf_mcp Workflow Surface Test Thinning Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Reduce duplicated `wf_mcp.workflow_surface` behavior tests now covered by `wf_api` tests while preserving adapter, schema, live-source, run, wrapper, and end-to-end coverage.
**Architecture:** `wf_api` tests are the canonical behavior tests for application-service logic. `wf_mcp.workflow_surface` tests should prove that `WorkflowSurfaceHandlers` exposes that behavior through the MCP-owned adapter boundary and should keep any test that exercises MCP-specific request models, live checks, service event recording, tool schema, or realistic integration paths.
**Tech Stack:** Python 3.14, pytest, `wf_api`, `wf_mcp.workflow_surface`, `WorkflowSurfaceHandlers`, ruff, basedpyright.
---
## Coverage Policy
Use this rule for every deletion:
```text
Only remove a wf_mcp.workflow_surface test when an equal-or-stronger wf_api test
already covers the behavior and a smaller handler smoke/delegation test still
proves the adapter path.
```
Keep tests that cover:
- MCP request/response Pydantic models such as `TraceRange`, `RunDeploymentResult`, and `CreateMinimalDraftWorkspaceRequest`.
- live source checks through `WfMcpService` adapters.
- service event recording.
- handler-to-service wiring.
- saved wrapper calls with deployment bindings.
- source binding, schema drift, reducer dependency, subgraph, or durable run integration.
- next-action guidance generated through real handler operations.
Thin tests that only repeat:
- list filtering/pagination details already covered by `WorkflowCapabilityApi` or `WorkflowArtifactApi`.
- basic inspect/list payload details already covered by `wf_api` domain tests.
- basic draft workspace CRUD details already covered by `WorkflowDraftApi`.
- basic artifact/deployment CRUD details already covered by `WorkflowArtifactApi` or `WorkflowDeploymentApi`.
## File Map
| File | Planned role |
| --- | --- |
| `tests/wf_mcp/workflow_surface/test_artifacts.py` | Thin to one handler adapter smoke test for compact artifact listing. |
| `tests/wf_mcp/workflow_surface/test_capabilities.py` | Thin list/filter/inspect duplicates; keep wrapper-hints, MCP content-block, direct call error, and saved wrapper coverage if not stronger in `wf_api`. |
| `tests/wf_mcp/workflow_surface/test_deployments.py` | Keep live-check, events, alias/XOR, delete event, and compact-vs-detail tests. Maybe remove only duplicate dependency validation if covered by `wf_api`. |
| `tests/wf_mcp/workflow_surface/test_drafts.py` | Keep most tests; remove only the simplest validate/patch/list CRUD duplicates if `wf_api` has equal coverage. |
| `tests/wf_mcp/workflow_surface/test_runs.py` | Keep run/trace/model tests. Do not thin in first pass except the raw plan model extraction test if duplicated in `tests/wf_api/test_raw_workflow_plan_extraction.py`. |
| `tests/wf_mcp/workflow_surface/test_wrappers.py` | Keep all tests in first pass; these are handler integration paths and direct capability REPL behavior. |
| `tests/wf_api/*` | Do not weaken. Add missing behavior tests here before removing handler duplicates. |
---
## Task 1: Build A Deletion Ledger Before Editing
**Files:**
- Create: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
- [ ] **Step 1: Create the ledger file**
Create `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`:
```markdown
# wf_mcp Workflow Surface Test Thinning Ledger
This ledger records every `wf_mcp.workflow_surface` test removed or kept during
the thinning pass. Do not delete a test unless the `replacement` column points
to equal-or-stronger coverage.
| Test | Decision | Replacement / Reason |
| --- | --- | --- |
```
- [ ] **Step 2: Populate the initial ledger with all workflow-surface tests**
Run:
```bash
rg -n "^def test_" tests/wf_mcp/workflow_surface
```
Append each test name to the ledger with `Decision` set to `unclassified`.
Expected: the ledger contains every test from:
```text
tests/wf_mcp/workflow_surface/test_artifacts.py
tests/wf_mcp/workflow_surface/test_capabilities.py
tests/wf_mcp/workflow_surface/test_deployments.py
tests/wf_mcp/workflow_surface/test_drafts.py
tests/wf_mcp/workflow_surface/test_runs.py
tests/wf_mcp/workflow_surface/test_wrappers.py
tests/wf_mcp/workflow_surface/test_next_actions.py
```
- [ ] **Step 3: Mark protected tests**
Mark these as `keep` unless a later task explicitly adds stronger coverage:
```text
tests/wf_mcp/workflow_surface/test_capabilities.py::test_workflow_surface_call_capability_returns_structured_error
tests/wf_mcp/workflow_surface/test_capabilities.py::test_workflow_surface_inspect_capability_includes_wrapper_hints
tests/wf_mcp/workflow_surface/test_capabilities.py::test_workflow_surface_does_not_auto_map_raw_mcp_content_blocks
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_validate_deployment_live_check_is_opt_in
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_validate_deployment_live_check_reports_unreachable_source
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_validate_deployment_live_check_reports_missing_connection
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_records_artifact_and_deployment_save_events
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_save_deployment_accepts_deployment_id_alias
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_save_deployment_rejects_id_and_deployment_id
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_run_deployment_can_include_trace_detail
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_run_deployment_can_read_empty_trace_range
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_runs_deployment_with_bound_node_spec_dependency
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_runs_artifact_created_from_concrete_node_ref
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_detects_drift_from_saved_node_spec_snapshot
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_runs_deployment_with_bound_reducer_dependency
tests/wf_mcp/workflow_surface/test_wrappers.py
```
Reason: these cover adapter wiring, MCP/service behavior, direct wrapper calls,
trace schema behavior, binding logic, reducer dependencies, or important
integration seams.
---
## Task 2: Thin Artifact Listing Duplicates
**Files:**
- Modify: `tests/wf_mcp/workflow_surface/test_artifacts.py`
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
- Test: `tests/wf_api/test_artifact_api.py`, `tests/wf_mcp/workflow_surface/test_artifacts.py`
- [ ] **Step 1: Confirm wf_api artifact coverage**
Run:
```bash
rg -n "list_artifacts|pagination|kind|query|plan\" not in" tests/wf_api/test_artifact_api.py tests/wf_api/test_listing.py
```
Expected: `wf_api` coverage exists for empty list payload, compact artifact rows,
query/kind behavior, and pagination helper shape. If it does not, add the
missing assertion to `tests/wf_api/test_artifact_api.py` before deleting any
handler test.
- [ ] **Step 2: Keep one handler smoke test**
Keep `test_workflow_surface_lists_artifact_catalog_entries` and reduce it only
if needed to these adapter-boundary assertions:
```python
def test_workflow_surface_lists_artifact_catalog_entries() -> None:
artifact_store = FileWorkflowArtifactStore(local_temp_root() / "surface_artifacts")
artifact_store.save_artifact(artifact())
h = handlers(artifact_store)
payload = asyncio.run(h.list_artifacts())
nodes = payload["nodes"]
assert payload["total"] == 1
assert payload["next_cursor"] is None
assert nodes[0]["name"] == "workflow.summarize_docs.v1"
assert nodes[0]["artifact_id"] == "summarize_docs"
assert "plan" not in nodes[0]
```
- [ ] **Step 3: Remove duplicate pagination/filter handler test**
Delete:
```text
test_workflow_surface_pages_and_filters_artifact_catalog_entries
```
Mark it in the ledger:
```text
remove | Covered by WorkflowArtifactApi list tests plus wf_api.listing pagination tests; handler keeps one list_artifacts smoke test.
```
- [ ] **Step 4: Run artifact tests**
Run:
```bash
uv run pytest tests/wf_api/test_artifact_api.py tests/wf_api/test_listing.py tests/wf_mcp/workflow_surface/test_artifacts.py -q
```
Expected: pass.
---
## Task 3: Thin Capability List/Inspect Duplicates Conservatively
**Files:**
- Modify: `tests/wf_mcp/workflow_surface/test_capabilities.py`
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
- Test: `tests/wf_api/test_capability_api.py`, `tests/wf_mcp/workflow_surface/test_capabilities.py`
- [ ] **Step 1: Confirm wf_api capability coverage**
Run:
```bash
rg -n "list_capabilities|inspect_capability|saved_wrapper|wrapper_hints|runtime_error|content blocks" tests/wf_api/test_capability_api.py tests/wf_mcp/workflow_surface/test_capabilities.py
```
Expected: `wf_api` covers planner-visible listing, source filtering, unknown
inspect, saved wrapper list/inspect, and direct handler delegation smoke.
- [ ] **Step 2: Keep one handler list smoke test**
Keep `test_workflow_surface_lists_planner_visible_capabilities`, but keep it
compact. It should assert adapter path and summary shape only:
```python
def test_workflow_surface_lists_planner_visible_capabilities() -> None:
h = handlers(FileWorkflowArtifactStore(local_temp_root() / "surface_caps"))
payload = asyncio.run(h.list_capabilities(limit=2))
first = payload["capabilities"][0]
assert len(payload["capabilities"]) == 2
assert payload["total"] >= 2
assert payload["next_cursor"] == "2"
assert first["kind"] == "node_spec"
assert "input_schema" not in first
```
- [ ] **Step 3: Remove duplicate source-filter test if wf_api has it**
Delete:
```text
test_workflow_surface_filters_stdlib_capabilities_by_source
```
Only delete it if `tests/wf_api/test_capability_api.py` contains:
```text
test_list_capabilities_filters_by_source
```
Mark the ledger:
```text
remove | Covered by WorkflowCapabilityApi source/query filtering; handler list smoke remains.
```
- [ ] **Step 4: Remove duplicate saved-wrapper list/inspect tests if wf_api has them**
Delete these only if `tests/wf_api/test_capability_api.py` has equivalent saved
wrapper list and inspect tests:
```text
test_workflow_surface_lists_saved_wrapper_capabilities
test_workflow_surface_inspects_saved_wrapper_capability
```
Mark the ledger:
```text
remove | Covered by WorkflowCapabilityApi saved wrapper list/inspect tests.
```
- [ ] **Step 5: Keep MCP-specific and guidance-sensitive capability tests**
Keep these tests unchanged:
```text
test_workflow_surface_call_capability_returns_structured_error
test_workflow_surface_inspect_capability_includes_wrapper_hints
test_workflow_surface_does_not_auto_map_raw_mcp_content_blocks
```
Do not remove `test_workflow_surface_inspects_one_capability` unless there is
still another handler-level inspect smoke test after this task.
- [ ] **Step 6: Run capability tests**
Run:
```bash
uv run pytest tests/wf_api/test_capability_api.py tests/wf_mcp/workflow_surface/test_capabilities.py -q
```
Expected: pass.
---
## Task 4: Thin Deployment Tests Only Where Purely Duplicated
**Files:**
- Modify: `tests/wf_mcp/workflow_surface/test_deployments.py`
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
- Test: `tests/wf_api/test_deployment_api.py`, `tests/wf_mcp/workflow_surface/test_deployments.py`
- [ ] **Step 1: Keep MCP live-check and event tests**
Do not delete:
```text
test_workflow_surface_validate_deployment_live_check_is_opt_in
test_workflow_surface_validate_deployment_live_check_reports_unreachable_source
test_workflow_surface_validate_deployment_live_check_reports_missing_connection
test_workflow_surface_records_artifact_and_deployment_save_events
test_workflow_surface_save_deployment_accepts_deployment_id_alias
test_workflow_surface_deletes_deployment
test_workflow_surface_save_deployment_rejects_id_and_deployment_id
test_workflow_surface_lists_compact_deployment_summaries_and_inspects_detail
```
Reason: these exercise MCP service adapters, event recording, request alias
normalization, and compact-vs-detail response shape.
- [ ] **Step 2: Evaluate basic dependency validation duplicate**
Check whether `tests/wf_api/test_deployment_api.py` covers dependency validation
next-actions for an unrunnable deployment:
```bash
rg -n "source_missing|binding_missing|next_actions|unrunnable" tests/wf_api/test_deployment_api.py
```
If the `wf_api` test has equal next-action assertions, delete:
```text
test_workflow_surface_validates_deployment_dependencies
```
If it does not, keep the handler test and mark it:
```text
keep | Handler-level dependency validation still has stronger next-action assertions than wf_api.
```
- [ ] **Step 3: Run deployment tests**
Run:
```bash
uv run pytest tests/wf_api/test_deployment_api.py tests/wf_mcp/workflow_surface/test_deployments.py -q
```
Expected: pass.
---
## Task 5: Thin Draft Tests With High Caution
**Files:**
- Modify: `tests/wf_mcp/workflow_surface/test_drafts.py`
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
- Test: `tests/wf_api/test_drafts_service.py`, `tests/wf_mcp/workflow_surface/test_drafts.py`
- [ ] **Step 1: Identify pure draft API duplicates**
Run:
```bash
rg -n "validate_draft|patch_draft|list_draft_workspaces|delete_draft_workspace|create_minimal_draft_workspace|create_draft_workspace_from_capability|create_artifact_from_workspace|create_wrapper_from_workspace" tests/wf_api/test_drafts_service.py tests/wf_api/test_artifact_api.py tests/wf_api/test_capability_api.py tests/wf_mcp/workflow_surface/test_drafts.py
```
Expected: most simple draft workspace CRUD and patch helper behavior exists in
`tests/wf_api/test_drafts_service.py`.
- [ ] **Step 2: Remove only pure CRUD duplicates**
Candidates for removal if `wf_api` tests cover equal behavior:
```text
test_workflow_surface_validates_draft_without_saving
test_workflow_surface_patches_draft_without_saving
test_workflow_surface_creates_and_gets_draft_workspace
test_workflow_surface_lists_draft_workspaces
test_workflow_surface_deletes_draft_workspace
test_workflow_surface_patch_helpers_update_draft_workspace
test_workflow_surface_patches_draft_workspace_by_revision
```
For each removed test, add a ledger row with the exact `wf_api` replacement test.
- [ ] **Step 3: Keep guidance, model, and artifact integration tests**
Do not delete these in this pass:
```text
test_workflow_surface_rejects_unknown_draft_route_outcome_when_spec_is_known
test_workflow_surface_creates_artifact_from_draft_with_binding_suggestions
test_workflow_surface_draft_artifact_requires_std_self_binding
test_workflow_surface_validates_draft_workspace_with_live_outcomes
test_workflow_surface_creates_minimal_draft_workspace_with_error_route
test_workflow_surface_minimal_draft_honors_explicit_error_message_source
test_minimal_draft_request_accepts_structural_error_message_source
test_workflow_surface_accepts_canonical_bindings_for_minimal_workspace
test_workflow_surface_creates_draft_workspace_from_capability_hints
test_workflow_surface_creates_artifact_from_workspace
test_workflow_surface_workspace_artifact_infers_raw_concrete_dependency
test_workflow_surface_creates_wrapper_from_workspace
test_workflow_surface_low_confidence_draft_returns_patch_guidance
```
Reason: these cover live outcome lookup, request model parsing, wrapper hints,
next actions, artifact persistence, and source dependency inference.
- [ ] **Step 4: Run draft tests**
Run:
```bash
uv run pytest tests/wf_api/test_drafts_service.py tests/wf_api/test_artifact_api.py tests/wf_api/test_capability_api.py tests/wf_mcp/workflow_surface/test_drafts.py -q
```
Expected: pass.
---
## Task 6: Keep Run And Wrapper Tests Mostly Intact
**Files:**
- Modify: `tests/wf_mcp/workflow_surface/test_runs.py`
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
- Test: `tests/wf_api/test_run_api.py`, `tests/wf_mcp/workflow_surface/test_runs.py`, `tests/wf_mcp/workflow_surface/test_wrappers.py`
- [ ] **Step 1: Remove raw plan model duplicate only**
If `tests/wf_api/test_raw_workflow_plan_extraction.py` covers core step and edge
model parsing, delete:
```text
tests/wf_mcp/workflow_surface/test_runs.py::test_raw_workflow_plan_uses_core_step_and_edge_models
```
Mark the ledger:
```text
remove | RawWorkflowPlan extraction is canonical in wf_api model tests.
```
- [ ] **Step 2: Keep run deployment behavior tests**
Keep all remaining tests in `tests/wf_mcp/workflow_surface/test_runs.py`.
Reason: they cover persisted run records, trace slicing with MCP `TraceRange`,
response Pydantic model validation, logical source binding, schema drift, and
reducer dependency integration through the handler/service stack.
- [ ] **Step 3: Keep wrapper tests**
Do not delete tests in `tests/wf_mcp/workflow_surface/test_wrappers.py` in this
pass.
Reason: wrapper direct calls and deployment-bound wrapper calls are meaningful
handler integration tests even if `WorkflowCapabilityApi` also has lower-level
coverage.
- [ ] **Step 4: Run run/wrapper tests**
Run:
```bash
uv run pytest tests/wf_api/test_run_api.py tests/wf_api/test_raw_workflow_plan_extraction.py tests/wf_mcp/workflow_surface/test_runs.py tests/wf_mcp/workflow_surface/test_wrappers.py -q
```
Expected: pass.
---
## Task 7: Final Review And Verification
**Files:**
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
- Test: all workflow-surface and wf_api tests
- [ ] **Step 1: Ensure ledger has no unclassified rows**
Run:
```bash
rg -n "unclassified" docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md
```
Expected: no matches.
- [ ] **Step 2: Run full focused API and adapter tests**
Run:
```bash
uv run pytest tests/wf_api tests/wf_mcp/workflow_surface tests/wf_mcp/server/test_tools.py tests/wf_mcp/server/test_config.py -q
```
Expected: pass.
- [ ] **Step 3: Run lint and type checks**
Run:
```bash
uv run ruff check tests/wf_api tests/wf_mcp/workflow_surface
uv run ruff format --check tests/wf_api tests/wf_mcp/workflow_surface
uv run basedpyright --level error
```
Expected:
- Ruff commands pass.
- Basedpyright reports `0 errors`; if it exits nonzero only because workspace
enumeration exceeds 10 seconds, report that exact output as an environment
issue rather than a type failure.
- [ ] **Step 4: Optional full suite**
Run:
```bash
uv run pytest -q
```
Expected: existing suite status remains at least as good as before this pass.
---
## Handoff Report Requirements
When done, report:
- Tests removed, grouped by file.
- Tests kept intentionally, with reasons for any controversial keeps.
- Any new or strengthened `wf_api` tests.
- Ledger path.
- Verification commands and exact outputs.
- Deviations from the plan.
## Self-Review
- Spec coverage: the plan preserves “good tests” by requiring a ledger and exact replacement coverage before any deletion.
- Placeholder scan: no deferred implementation slots; each deletion candidate has a guard and replacement rule.
- Type consistency: all referenced test paths and test names were taken from the current tree inspection.
@@ -0,0 +1,605 @@
# WfMcpService ConnectionService Extraction Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move connection registration and config reconciliation out of `WfMcpService` into a focused `ConnectionService`.
**Architecture:** `ConnectionService` owns the broker-local `ConnectionRegistry` and emits connection lifecycle events. `SourceCatalogService` still owns capability sources, so `ConnectionService` binds to it after both services are constructed. `WfMcpService` remains a compatibility facade with `.connections`, `register_connection()`, and `sync_connections_from_config()` delegating to the new service.
**Tech Stack:** Python 3.14, dataclasses, pytest, ruff, basedpyright, existing `wf_mcp` broker service modules.
---
## File Structure
- Create `src/wf_mcp/broker/service/connection_service.py`
- Owns `ConnectionRegistry`.
- Validates connection IDs and reserved IDs.
- Registers connections and hydrates source catalog snapshots.
- Reconciles config reload changes.
- Modify `src/wf_mcp/broker/service/core.py`
- Removes direct `ConnectionRegistry` field from `WfMcpService`.
- Constructs `ConnectionService`, passes its lookup/list callbacks into `SourceCatalogService`, then binds the source catalog back to `ConnectionService`.
- Keeps compatibility property/method delegates.
- Create `tests/wf_mcp/service/test_connection_service.py`
- Direct tests for `ConnectionService`.
- Service facade smoke test for `.connections` compatibility.
- Modify `docs/current_roadmap.md`
- Mark the connection-service extraction as the current/complete slice after implementation.
- Optionally modify `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md`
- Add one ownership note if the file still tracks `WfMcpService` decomposition.
---
### Task 1: Add Direct ConnectionService Tests
**Files:**
- Create: `tests/wf_mcp/service/test_connection_service.py`
- [ ] **Step 1: Create direct tests for the new service boundary**
Create `tests/wf_mcp/service/test_connection_service.py` with:
```python
from __future__ import annotations
from wf_mcp.broker.service.connection_service import ConnectionService
from wf_mcp.broker.service.events import BrokerEventRecorder
from wf_mcp.broker.service.source_catalog import SourceCatalogService
from wf_mcp.events import EventBus
from wf_mcp.models import BrokerConfig, ConnectionConfig
from wf_mcp.runtime import ToolExecutor
from wf_mcp.storage import FileStore
from ..test_support import local_temp_root
def _source_catalog(service: ConnectionService) -> SourceCatalogService:
store = FileStore(local_temp_root() / "connection_service_catalog")
def _tool_executor_for(_connection: ConnectionConfig) -> ToolExecutor:
raise AssertionError("tool executor should not be needed in these tests")
catalog = SourceCatalogService(
store=store,
connection_lookup=service.get,
connection_list_enabled=service.list_enabled,
connection_list_all=service.list_all,
tool_executor_for=_tool_executor_for,
load_auth=lambda _connection_id: None,
emit_event=service.events.record_event,
)
service.bind_source_catalog(catalog)
return catalog
def test_connection_service_rejects_reserved_connection_ids() -> None:
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
_source_catalog(service)
for connection_id in ("wf.admin", "wf.mcp"):
try:
service.register_connection(
ConnectionConfig(id=connection_id, server="wf", account="reserved")
)
except ValueError as exc:
assert connection_id in str(exc)
assert "reserved by wf-mcp" in str(exc)
else:
raise AssertionError(f"expected {connection_id!r} to be rejected")
def test_connection_service_registers_connection_and_empty_source() -> None:
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
catalog = _source_catalog(service)
service.register_connection(
ConnectionConfig(id="demo.personal", server="demo", account="personal")
)
assert service.get("demo.personal").server == "demo"
assert [connection.id for connection in service.list_enabled()] == ["demo.personal"]
source = catalog.capability_sources["demo.personal"]
assert source.enabled is True
assert source.description == "No catalog loaded for demo.personal."
assert service.events.list_events()[0].kind == "connection_registered"
assert service.events.list_events()[0].connection_id == "demo.personal"
def test_connection_service_sync_removes_retired_connections_and_sources() -> None:
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
catalog = _source_catalog(service)
service.register_connection(
ConnectionConfig(id="demo.personal", server="demo", account="personal")
)
service.sync_connections_from_config(
BrokerConfig(store_root=local_temp_root(), connections=[])
)
assert service.list_all() == []
assert "demo.personal" not in catalog.capability_sources
def test_connection_service_sync_updates_existing_source_enabled_flag() -> None:
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
catalog = _source_catalog(service)
service.register_connection(
ConnectionConfig(id="demo.personal", server="demo", account="personal")
)
service.sync_connections_from_config(
BrokerConfig(
store_root=local_temp_root(),
connections=[
ConnectionConfig(
id="demo.personal",
server="demo",
account="personal",
enabled=False,
)
],
)
)
assert service.get("demo.personal").enabled is False
assert catalog.capability_sources["demo.personal"].enabled is False
```
- [ ] **Step 2: Run the direct test and confirm it fails before implementation**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
```
Expected: import failure for `wf_mcp.broker.service.connection_service`.
---
### Task 2: Implement ConnectionService
**Files:**
- Create: `src/wf_mcp/broker/service/connection_service.py`
- [ ] **Step 1: Add the service implementation**
Create `src/wf_mcp/broker/service/connection_service.py` with:
```python
from __future__ import annotations
from dataclasses import dataclass, field
from ...connections import ConnectionRegistry, parse_connection_id
from ...models import BrokerConfig, ConnectionConfig
from ...shared.names import RESERVED_CONNECTION_IDS
from .events import BrokerEventRecorder
from .source_catalog import SourceCatalogService
@dataclass(slots=True)
class ConnectionService:
"""Own broker connection registration and config reconciliation.
SourceCatalogService needs connection lookup callbacks during construction,
while registering a connection needs source-catalog hydration. The catalog is
therefore bound after both services exist; `_source_catalog()` makes that
construction cycle explicit and fail-fast.
"""
events: BrokerEventRecorder
connections: ConnectionRegistry = field(default_factory=ConnectionRegistry)
source_catalog: SourceCatalogService | None = None
def bind_source_catalog(self, source_catalog: SourceCatalogService) -> None:
self.source_catalog = source_catalog
def get(self, connection_id: str) -> ConnectionConfig:
return self.connections.get(connection_id)
def list_all(self) -> list[ConnectionConfig]:
return self.connections.list_all()
def list_enabled(self) -> list[ConnectionConfig]:
return self.connections.list_enabled()
def register_connection(self, connection: ConnectionConfig) -> None:
self._validate_connection_id(connection.id)
self.connections.register(connection)
self._source_catalog().hydrate_connection_source_from_snapshot(connection)
self.events.record_kind(
"connection_registered",
connection_id=connection.id,
payload={"server": connection.server, "account": connection.account},
)
def sync_connections_from_config(self, config: BrokerConfig) -> None:
"""Reconcile registry/source state after the public server reloads config."""
source_catalog = self._source_catalog()
next_ids = {connection.id for connection in config.connections}
previous_ids = set(self.connections.connections)
for connection_id in previous_ids - next_ids:
del self.connections.connections[connection_id]
source_catalog.capability_sources.pop(connection_id, None)
for connection in config.connections:
self._validate_connection_id(connection.id)
self.connections.register(connection)
source = source_catalog.capability_sources.get(connection.id)
if source is None:
source_catalog.hydrate_connection_source_from_snapshot(connection)
else:
source.enabled = connection.enabled
def _source_catalog(self) -> SourceCatalogService:
if self.source_catalog is None:
raise RuntimeError("ConnectionService requires a bound SourceCatalogService")
return self.source_catalog
@staticmethod
def _validate_connection_id(connection_id: str) -> None:
parse_connection_id(connection_id)
if connection_id in RESERVED_CONNECTION_IDS:
raise ValueError(f"connection id {connection_id!r} is reserved by wf-mcp")
```
- [ ] **Step 2: Run the direct tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
```
Expected: all tests pass.
- [ ] **Step 3: Run ruff on the new files**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/connection_service.py tests/wf_mcp/service/test_connection_service.py
```
Expected: all checks pass.
---
### Task 3: Wire WfMcpService Through ConnectionService
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- [ ] **Step 1: Update imports and dataclass fields**
In `src/wf_mcp/broker/service/core.py`:
Remove:
```python
from ...connections import ConnectionRegistry, parse_connection_id
from ...shared.names import RESERVED_CONNECTION_IDS
```
Replace with:
```python
from ...connections import ConnectionRegistry
```
Add:
```python
from .connection_service import ConnectionService
```
In `WfMcpService`, remove the dataclass field:
```python
connections: ConnectionRegistry = field(default_factory=ConnectionRegistry)
```
Add this init-false field near the other service fields:
```python
connection_service: ConnectionService = field(init=False)
```
- [ ] **Step 2: Construct and bind the connection service**
In `__post_init__`, replace the source-catalog construction block with this shape:
```python
self.events = BrokerEventRecorder(self.event_bus)
self.connection_service = ConnectionService(events=self.events)
self.upstream = UpstreamTransportService(
store=self.store,
event_sink=self.events.record_event,
tool_executor=self.tool_executor,
)
self.source_catalog = SourceCatalogService(
store=self.store,
connection_lookup=self.connection_service.get,
connection_list_enabled=self.connection_service.list_enabled,
connection_list_all=self.connection_service.list_all,
tool_executor_for=self.upstream.tool_executor_for,
load_auth=self.upstream.load_auth,
emit_event=self.events.record_event,
default_catalog_max_age_seconds=self.default_catalog_max_age_seconds,
)
self.connection_service.bind_source_catalog(self.source_catalog)
```
- [ ] **Step 3: Preserve `.connections` compatibility as a property**
Add this property below `capability_sources` or above it:
```python
@property
def connections(self) -> ConnectionRegistry:
"""Compatibility view of the broker connection registry.
Connection lifecycle ownership has moved to ConnectionService. Keep this
property because admin handlers, CLI helpers, and tests still inspect the
registry through the service facade.
"""
return self.connection_service.connections
```
- [ ] **Step 4: Replace connection lifecycle method bodies with delegates**
Replace `register_connection` with:
```python
def register_connection(self, connection: ConnectionConfig) -> None:
self.connection_service.register_connection(connection)
```
Replace `sync_connections_from_config` with:
```python
def sync_connections_from_config(self, config: BrokerConfig) -> None:
self.connection_service.sync_connections_from_config(config)
```
- [ ] **Step 5: Run focused service tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/service/test_catalog.py tests/wf_mcp/service/test_sources.py tests/wf_mcp/test_events.py -q
```
Expected: all selected tests pass.
---
### Task 4: Add Facade Compatibility Tests
**Files:**
- Modify: `tests/wf_mcp/service/test_connection_service.py`
- [ ] **Step 1: Add WfMcpService compatibility coverage**
Append these imports:
```python
from wf_mcp.broker import WfMcpService
```
Append these tests:
```python
def test_wfmcpservice_exposes_connection_registry_from_connection_service() -> None:
service = WfMcpService(store=FileStore(local_temp_root() / "connection_facade"))
service.register_connection(
ConnectionConfig(id="demo.personal", server="demo", account="personal")
)
assert service.connections is service.connection_service.connections
assert service.connections.get("demo.personal").account == "personal"
assert "demo.personal" in service.capability_sources
def test_wfmcpservice_sync_connections_delegates_to_connection_service() -> None:
service = WfMcpService(store=FileStore(local_temp_root() / "connection_sync"))
service.register_connection(
ConnectionConfig(id="demo.personal", server="demo", account="personal")
)
service.sync_connections_from_config(
BrokerConfig(
store_root=local_temp_root(),
connections=[
ConnectionConfig(
id="demo.work",
server="demo",
account="work",
enabled=True,
)
],
)
)
assert [connection.id for connection in service.connections.list_all()] == [
"demo.work"
]
assert "demo.personal" not in service.capability_sources
assert "demo.work" in service.capability_sources
```
- [ ] **Step 2: Run the compatibility tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
```
Expected: all tests pass.
---
### Task 5: Clean Imports and Verify Call Sites
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- Possibly modify files only if ruff reports stale imports.
- [ ] **Step 1: Search for stale direct ownership assumptions**
Run:
```bash
rg -n 'parse_connection_id|RESERVED_CONNECTION_IDS|connection_service|ConnectionRegistry|connections: ConnectionRegistry' src/wf_mcp/broker/service tests/wf_mcp/service
```
Expected:
- `parse_connection_id` and `RESERVED_CONNECTION_IDS` appear in `connection_service.py`, not `core.py`.
- `connections: ConnectionRegistry` appears in `connection_service.py`, not `core.py`.
- `connection_service` appears in `core.py` and direct tests.
- [ ] **Step 2: Run ruff on modified service files**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/connection_service.py tests/wf_mcp/service/test_connection_service.py
```
Expected: all checks pass. If ruff reports unused imports in `core.py`, remove only those imports.
- [ ] **Step 3: Run basedpyright on modified source**
Run:
```bash
uv run basedpyright --level error
```
Expected: 0 errors.
---
### Task 6: Update Roadmap and Extraction Map
**Files:**
- Modify: `docs/current_roadmap.md`
- Modify if present/relevant: `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md`
- [ ] **Step 1: Update `docs/current_roadmap.md`**
Find the bullet that says:
```markdown
- Next planned service extraction: move connection registration/config reconciliation
into a `ConnectionService`. That slice should own reserved connection-id
rejection, `register_connection`, `sync_connections_from_config`, and source
hydration coordination while leaving `WfMcpService` as a compatibility
coordinator.
```
Replace it with:
```markdown
- Connection ownership now lives in `ConnectionService`: it owns the broker
`ConnectionRegistry`, reserved connection-id rejection, `register_connection`,
and `sync_connections_from_config`. `WfMcpService.connections` remains a
compatibility property while source hydration still belongs to
`SourceCatalogService`.
```
- [ ] **Step 2: Update the extraction map if it contains the WfMcpService split notes**
Run:
```bash
rg -n 'ConnectionService|connection registration|sync_connections_from_config|WfMcpService' docs/superpowers/research/2026-06-01-wf-api-extraction-map.md
```
If the file exists and contains the service split section, add this short note near the other extracted-service bullets:
```markdown
- Connection registration/config reload reconciliation is now owned by
`wf_mcp.broker.service.connection_service.ConnectionService`. The service owns
the `ConnectionRegistry`; `WfMcpService.connections` is only a compatibility
property.
```
- [ ] **Step 3: Run docs grep to verify roadmap wording**
Run:
```bash
rg -n 'ConnectionService|Connection ownership|Next planned service extraction' docs/current_roadmap.md docs/superpowers/research/2026-06-01-wf-api-extraction-map.md
```
Expected:
- `docs/current_roadmap.md` mentions completed `ConnectionService` ownership.
- No stale "Next planned service extraction" wording for this same slice remains.
---
### Task 7: Final Verification
**Files:**
- No new files.
- [ ] **Step 1: Run focused tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/service/test_catalog.py tests/wf_mcp/service/test_sources.py tests/wf_mcp/service/test_events.py tests/wf_mcp/test_broker_server.py tests/wf_mcp/test_admin_surface.py -q
```
Expected: all selected tests pass.
- [ ] **Step 2: Run full test suite**
Run:
```bash
uv run pytest -q
```
Expected: full suite passes with the existing skipped/xfailed counts only.
- [ ] **Step 3: Run final static checks**
Run:
```bash
uv run ruff check
uv run ruff format --check
uv run basedpyright --level error
```
Expected:
- ruff check passes.
- ruff format check passes for Python files.
- basedpyright reports 0 errors.
If markdown format checks complain about preview-only markdown behavior, do not rewrite unrelated markdown. Report it as a formatting-tool limitation and keep the code checks green.
---
## Self-Review
- Spec coverage: The plan moves reserved ID validation, `register_connection`, and `sync_connections_from_config` into `ConnectionService`; preserves `.connections` compatibility; keeps source hydration coordination explicit through a post-construction bind.
- Placeholder scan: No `TBD`, generic "add tests", or unfilled implementation steps remain.
- Type consistency: `ConnectionService` exposes `get`, `list_all`, and `list_enabled` so `SourceCatalogService` can use bound methods without depending on `WfMcpService`.
- Risk: The construction cycle between connection lookup and source hydration is intentionally represented by `bind_source_catalog()`. The fail-fast `_source_catalog()` guard prevents silent use before binding.
@@ -0,0 +1,691 @@
# WfMcpService Event Recorder Extraction Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Extract broker event recording and catalog-change event fanout from `WfMcpService` into a focused `BrokerEventRecorder` while preserving existing event history, subscribers, and public service methods.
**Architecture:** Add `BrokerEventRecorder` under `wf_mcp.broker.service`. It wraps the existing `EventBus`, owns `record_event`, `record_kind`, `record_catalog_change_events`, and `list_events`. `WfMcpService` keeps `event_bus` for compatibility with server/proxy wiring and keeps `_record_event` / `_record_catalog_change_events` as delegates until later cleanup.
**Tech Stack:** Python 3.14, dataclasses, existing `wf_mcp.events.EventBus`/`McpEvent`, pytest, ruff, basedpyright.
---
## Scope
Move now:
- Direct `event_bus.publish(...)` usage.
- `list_events`.
- `_record_event`, as `BrokerEventRecorder.record_event`.
- `_record_catalog_change_events`, as `BrokerEventRecorder.record_catalog_change_events`.
- Event construction helper for simple workflow/API events, as `BrokerEventRecorder.record_kind`.
Keep now:
- `WfMcpService.event_bus` dataclass field for server/proxy compatibility.
- `WfMcpService.list_events`, `_record_event`, and `_record_catalog_change_events` as delegates.
- Event kind strings and payload shapes.
- Existing `wf_api.operation_context.WorkflowEventRecorder` protocol.
Do not do in this slice:
- Do not move `EventBus` itself.
- Do not change notification projection.
- Do not convert all event callers to a new domain event enum.
- Do not remove private delegate methods from `WfMcpService`.
---
## Target File Structure
- Create `src/wf_mcp/broker/service/events.py`
- Defines `BrokerEventRecorder`.
- Owns catalog-change fanout logic.
- Has docstrings stating this is broker-local event recording, not MCP notification delivery.
- Modify `src/wf_mcp/broker/service/core.py`
- Add `events: BrokerEventRecorder = field(init=False)`.
- Construct it from `event_bus` in `__post_init__`.
- Pass `self.events.record_event` into `UpstreamTransportService`, `SourceCatalogService`, and `WorkflowRuntimeService`.
- Delegate public/private event methods.
- Modify `src/wf_mcp/broker/service/workflow_operation_context.py`
- `WfMcpWorkflowEventRecorder` should hold `BrokerEventRecorder`, not call `service._record_event`.
- Add tests in `tests/wf_mcp/service/test_event_recorder.py`.
- Update docs:
- `docs/current_roadmap.md`.
- `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` if stale.
---
## Task 1: Add BrokerEventRecorder Skeleton
**Files:**
- Create: `src/wf_mcp/broker/service/events.py`
- Create: `tests/wf_mcp/service/test_event_recorder.py`
- [ ] **Step 1: Write direct event recorder tests**
Create `tests/wf_mcp/service/test_event_recorder.py`:
```python
from __future__ import annotations
from wf_mcp.broker.service.events import BrokerEventRecorder
from wf_mcp.events import EventBus, make_event
def test_broker_event_recorder_records_existing_event() -> None:
bus = EventBus()
recorder = BrokerEventRecorder(bus)
event = make_event("connection_registered", connection_id="demo.personal")
recorder.record_event(event)
assert recorder.list_events()[0] is event
assert bus.list_events()[0] is event
def test_broker_event_recorder_builds_simple_event() -> None:
recorder = BrokerEventRecorder(EventBus())
recorder.record_kind(
"workflow_artifact_saved",
capability_id="echo",
payload={"version": 1},
)
event = recorder.list_events()[0]
assert event.kind == "workflow_artifact_saved"
assert event.capability_id == "echo"
assert event.payload["version"] == 1
```
- [ ] **Step 2: Run the tests and verify they fail**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_event_recorder.py::test_broker_event_recorder_records_existing_event tests/wf_mcp/service/test_event_recorder.py::test_broker_event_recorder_builds_simple_event -q
```
Expected: import failure because `wf_mcp.broker.service.events` does not exist.
- [ ] **Step 3: Create BrokerEventRecorder**
Create `src/wf_mcp/broker/service/events.py`:
```python
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
from wf_mcp.events import EventBus, McpEvent, make_event
@dataclass(slots=True)
class BrokerEventRecorder:
"""Broker-local event recorder backed by the existing EventBus.
This class records and fans out local service events. MCP notifications are
still projected by subscribers/resources elsewhere; this is only the broker
event emission boundary.
"""
event_bus: EventBus
def record_event(self, event: McpEvent) -> None:
self.event_bus.publish(event)
def record_kind(
self,
event_type: str,
*,
connection_id: str | None = None,
capability_id: str | None = None,
workflow_name: str | None = None,
payload: dict[str, Any] | None = None,
) -> None:
self.record_event(
make_event(
event_type,
connection_id=connection_id,
capability_id=capability_id,
workflow_name=workflow_name,
payload=payload or {},
)
)
def list_events(self) -> list[McpEvent]:
return self.event_bus.list_events()
```
- [ ] **Step 4: Run the tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_event_recorder.py::test_broker_event_recorder_records_existing_event tests/wf_mcp/service/test_event_recorder.py::test_broker_event_recorder_builds_simple_event -q
```
Expected: pass.
- [ ] **Step 5: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/events.py tests/wf_mcp/service/test_event_recorder.py
```
Expected: pass.
---
## Task 2: Move Catalog-Change Fanout Into BrokerEventRecorder
**Files:**
- Modify: `src/wf_mcp/broker/service/events.py`
- Test: `tests/wf_mcp/service/test_event_recorder.py`
- [ ] **Step 1: Write direct catalog fanout test**
Append to `tests/wf_mcp/service/test_event_recorder.py`:
```python
from wf_mcp.models import CatalogSnapshot
def test_broker_event_recorder_records_catalog_change_fanout() -> None:
recorder = BrokerEventRecorder(EventBus())
snapshot = CatalogSnapshot(
connection_id="demo.personal",
nodes=[
{
"qualified_name": "demo.personal.echo",
"connection_id": "demo.personal",
"local_name": "echo",
"input_schema": {},
"output_schema": {},
"outcomes": ["ok"],
}
],
resources=[
{
"qualified_name": "demo.personal.resource.welcome",
"connection_id": "demo.personal",
"local_name": "welcome",
"uri": "demo://welcome",
}
],
prompts=[
{
"qualified_name": "demo.personal.prompt.welcome",
"connection_id": "demo.personal",
"local_name": "welcome",
}
],
fetched_at_epoch_ms=1,
max_age_seconds=300,
)
recorder.record_catalog_change_events(
"demo.personal",
snapshot,
reason="catalog_refresh",
)
events = recorder.list_events()
event_kinds = [event.kind for event in events]
assert event_kinds == [
"tools_changed",
"resources_changed",
"prompts_changed",
"catalog_changed",
]
assert events[-1].payload["reason"] == "catalog_refresh"
assert events[-1].payload["node_count"] == 1
assert events[-1].payload["resource_count"] == 1
assert events[-1].payload["prompt_count"] == 1
```
If `CatalogSnapshot` requires additional fields in this repo, inspect an existing fixture in `tests/wf_mcp/service/test_events.py` and use the same minimal shape. Do not weaken the assertion to only “some event exists.”
- [ ] **Step 2: Run the fanout test and verify it fails**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_event_recorder.py::test_broker_event_recorder_records_catalog_change_fanout -q
```
Expected: fail because `record_catalog_change_events` does not exist.
- [ ] **Step 3: Implement catalog fanout**
In `src/wf_mcp/broker/service/events.py`, import:
```python
from wf_mcp.models import CatalogSnapshot
```
Add:
```python
def record_catalog_change_events(
self,
connection_id: str,
snapshot: CatalogSnapshot,
*,
reason: str,
) -> None:
"""Emit local change events that future MCP notifications can project."""
counts = {
"node_count": len(snapshot.nodes),
"resource_count": len(snapshot.resources),
"prompt_count": len(snapshot.prompts),
}
if snapshot.nodes:
self.record_kind(
"tools_changed",
connection_id=connection_id,
payload={"reason": reason, "node_count": counts["node_count"]},
)
if snapshot.resources:
self.record_kind(
"resources_changed",
connection_id=connection_id,
payload={
"reason": reason,
"resource_count": counts["resource_count"],
},
)
if snapshot.prompts:
self.record_kind(
"prompts_changed",
connection_id=connection_id,
payload={"reason": reason, "prompt_count": counts["prompt_count"]},
)
self.record_kind(
"catalog_changed",
connection_id=connection_id,
payload={"reason": reason, **counts},
)
```
- [ ] **Step 4: Run event recorder tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_event_recorder.py -q
```
Expected: all tests in the file pass.
- [ ] **Step 5: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/events.py tests/wf_mcp/service/test_event_recorder.py
```
Expected: pass.
---
## Task 3: Wire BrokerEventRecorder Into WfMcpService
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- Test: `tests/wf_mcp/test_events.py`
- Test: `tests/wf_mcp/service/test_event_recorder.py`
- [ ] **Step 1: Add service identity test**
Append to `tests/wf_mcp/service/test_event_recorder.py`:
```python
from wf_mcp.broker import WfMcpService
from wf_mcp.storage import FileStore
from ..test_support import local_temp_root
def test_wfmcpservice_uses_broker_event_recorder() -> None:
bus = EventBus()
service = WfMcpService(
store=FileStore(local_temp_root() / "service_event_recorder"),
event_bus=bus,
)
service._record_event( # noqa: SLF001
make_event("connection_registered", connection_id="demo.personal")
)
assert service.events.event_bus is bus
assert service.list_events()[0].kind == "connection_registered"
```
- [ ] **Step 2: Run the service identity test and verify it fails**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_event_recorder.py::test_wfmcpservice_uses_broker_event_recorder -q
```
Expected: fail because `service.events` does not exist.
- [ ] **Step 3: Add events field to WfMcpService**
In `src/wf_mcp/broker/service/core.py`, import:
```python
from .events import BrokerEventRecorder
```
Add dataclass field:
```python
events: BrokerEventRecorder = field(init=False)
```
At the top of `__post_init__`, before constructing `UpstreamTransportService`, add:
```python
self.events = BrokerEventRecorder(self.event_bus)
```
Update service construction callbacks:
```python
self.upstream = UpstreamTransportService(
store=self.store,
event_sink=self.events.record_event,
tool_executor=self.tool_executor,
)
```
```python
emit_event=self.events.record_event,
```
```python
emit_event=self.events.record_event,
```
- [ ] **Step 4: Delegate event methods**
Replace:
```python
def list_events(self) -> list[McpEvent]:
return self.event_bus.list_events()
def _record_event(self, event: McpEvent) -> None:
self.event_bus.publish(event)
```
with:
```python
def list_events(self) -> list[McpEvent]:
return self.events.list_events()
def _record_event(self, event: McpEvent) -> None:
self.events.record_event(event)
```
Replace `_record_catalog_change_events` body with:
```python
self.events.record_catalog_change_events(
connection_id,
snapshot,
reason=reason,
)
```
Keep the method signature and docstring.
- [ ] **Step 5: Run service event tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_event_recorder.py::test_wfmcpservice_uses_broker_event_recorder tests/wf_mcp/test_events.py -q
```
Expected: pass.
- [ ] **Step 6: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/events.py tests/wf_mcp/service/test_event_recorder.py
```
Expected: pass.
---
## Task 4: Update WorkflowOperationContext Event Adapter
**Files:**
- Modify: `src/wf_mcp/broker/service/workflow_operation_context.py`
- Test: `tests/wf_api/test_operation_context.py`
- [ ] **Step 1: Strengthen context event adapter test**
In `tests/wf_api/test_operation_context.py`, add or update a test:
```python
def test_context_event_recorder_uses_broker_event_recorder(tmp_path: Path) -> None:
service = WfMcpService(store=FileStore(tmp_path / "context_events"))
context = context_from_service(service)
context.events.record_workflow_event(
"workflow_artifact_saved",
capability_id="echo",
payload={"version": 1},
)
assert service.events.list_events()[-1].kind == "workflow_artifact_saved"
assert service.events.list_events()[-1].capability_id == "echo"
```
If this file already has an equivalent `record_workflow_event` test, update its assertions to read through `service.events.list_events()` instead of only `service.list_events()`.
- [ ] **Step 2: Run the context event test**
Run:
```bash
uv run pytest tests/wf_api/test_operation_context.py::test_context_event_recorder_uses_broker_event_recorder -q
```
If the test name was updated instead of added, run the actual updated test name.
Expected: fail until the adapter stops calling private service methods, or pass if Task 3 delegates already cover it.
- [ ] **Step 3: Update WfMcpWorkflowEventRecorder**
In `src/wf_mcp/broker/service/workflow_operation_context.py`, import:
```python
from .events import BrokerEventRecorder
```
Change:
```python
class WfMcpWorkflowEventRecorder(WorkflowEventRecorder):
"""Adapter-owned event recorder backed by WfMcpService."""
service: WfMcpService
```
to:
```python
class WfMcpWorkflowEventRecorder(WorkflowEventRecorder):
"""Adapter-owned event recorder backed by BrokerEventRecorder."""
events: BrokerEventRecorder
```
Replace method bodies:
```python
def record_event(self, event: Any) -> None:
self.events.record_event(event)
def record_workflow_event(
self,
event_type: str,
*,
capability_id: str,
payload: dict[str, Any],
) -> None:
self.events.record_kind(
event_type,
capability_id=capability_id,
payload=payload,
)
```
In `context_from_service`, change:
```python
events=WfMcpWorkflowEventRecorder(service),
```
to:
```python
events=WfMcpWorkflowEventRecorder(service.events),
```
- [ ] **Step 4: Run operation context tests**
Run:
```bash
uv run pytest tests/wf_api/test_operation_context.py -q
```
Expected: pass.
- [ ] **Step 5: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/workflow_operation_context.py tests/wf_api/test_operation_context.py
```
Expected: pass.
---
## Task 5: Clean Imports, Docs, and Verify
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- Modify: `src/wf_mcp/broker/service/workflow_operation_context.py`
- Modify: `docs/current_roadmap.md`
- Modify: `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` if stale.
- [ ] **Step 1: Remove stale imports**
After the move, `src/wf_mcp/broker/service/core.py` should no longer import `make_event` directly unless it is still used for local resource/local prompt/connection registration events.
If `make_event` is still used only in these service-local cases:
```python
connection_registered
resource_read_completed for local docs
prompt_get_completed for local docs
```
leave it for now. Those local cases can move in a later connection/resource admin extraction.
`src/wf_mcp/broker/service/workflow_operation_context.py` should no longer import `make_event`.
- [ ] **Step 2: Add roadmap note**
In `docs/current_roadmap.md`, under the service extraction bullets, add:
```markdown
- Broker event recording is being separated from broker coordination.
`BrokerEventRecorder` now owns EventBus publication, event history reads,
simple event construction, and catalog-change fanout. `WfMcpService` keeps
delegate methods for compatibility.
```
- [ ] **Step 3: Update extraction map if stale**
If `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` says `WfMcpService` directly owns event recording/fanout, add:
```markdown
Event recording ownership is now split: `BrokerEventRecorder` owns EventBus
publication, simple event construction, event history reads, and catalog-change
fanout. `WfMcpService` remains the coordinator and compatibility façade.
```
- [ ] **Step 4: Run focused verification**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_event_recorder.py tests/wf_mcp/test_events.py tests/wf_mcp/service/test_events.py tests/wf_api/test_operation_context.py tests/wf_mcp/workflow_surface/test_deployments.py -q
```
Expected: all selected tests pass.
- [ ] **Step 5: Run full verification**
Run:
```bash
uv run pytest -q
uv run ruff check src/wf_mcp/broker/service src/wf_api tests/wf_mcp/service tests/wf_api
uv run ruff format --check src/wf_mcp/broker/service src/wf_api tests/wf_mcp/service tests/wf_api docs/current_roadmap.md
uv run basedpyright --level error
```
Expected:
- pytest passes.
- ruff check passes.
- ruff format check passes.
- basedpyright reports `0 errors`. If the known workspace enumeration warning causes a nonzero exit despite `0 errors`, record the exact output.
---
## Non-Goals and Follow-Up Slices
This plan intentionally leaves these for later:
1. **Connection service extraction:** move `register_connection`, reserved-id policy, and config reconciliation.
2. **Local resource/prompt admin extraction:** move local docs event emission out of `WfMcpService.read_resource` and `render_prompt`.
3. **Final service rename:** once `WfMcpService` mostly wires implementation services, consider a clearer broker coordinator name.
---
## Self-Review
- Spec coverage: The plan extracts event recording and catalog-change fanout while preserving existing event bus compatibility and event payload shapes.
- Placeholder scan: No placeholder implementation steps are left. The one import cleanup step explicitly says when to keep `make_event` because local admin event emission remains in `WfMcpService`.
- Type consistency: `BrokerEventRecorder` consistently wraps `EventBus`; `WfMcpService.events` holds the recorder; workflow operation context depends on the recorder instead of private service methods.
@@ -0,0 +1,709 @@
# WfMcpService Resource/Prompt Access Extraction Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Extract the local-docs-aware resource read and prompt render logic from `WfMcpService` into a focused `ContentAccessService` while preserving existing MCP admin, CLI, and workflow behavior.
**Architecture:** Add `ContentAccessService` under `wf_mcp.broker.service`. It owns the "is this a local doc? if so return inline, else look up catalog entry and delegate upstream" branching that currently lives in `WfMcpService.read_resource` and `WfMcpService.render_prompt`. `WfMcpService` remains the compatibility coordinator; its `read_resource` and `render_prompt` become one-line delegates.
**Why a new service instead of folding into `UpstreamTransportService`:** The local-docs check depends on `SourceCatalogService`, not on upstream adapters. Putting it in `UpstreamTransportService` would add a cross-cutting catalog dependency to a transport-focused service. A thin content-access layer keeps the boundary clean.
**Tech Stack:** Python 3.14, dataclasses, `SourceCatalogService`, `UpstreamTransportService`, `ConnectionService`, pytest, ruff, basedpyright.
---
## Scope
Move now:
- `WfMcpService.read_resource` branching logic (local docs check + upstream delegation).
- `WfMcpService.render_prompt` branching logic (local docs check + upstream delegation).
Keep now:
- `invoke_method`. Already a clean delegate to `upstream`; no local-docs branching. Does not belong in this service.
- `send_notification`. Same reasoning.
- `refresh_connection_catalog`. Different semantics; stays on `WfMcpService` / `UpstreamTransportService`.
- All other `WfMcpService` delegates and compatibility properties.
- Event recording internals on `BrokerEventRecorder`.
Do not do in this slice:
- Do not move `invoke_method` or `send_notification` into `ContentAccessService`.
- Do not move `refresh_connection_catalog`.
- Do not rename `WfMcpService`.
- Do not change adapter implementations.
- Do not change MCP tool schemas or CLI behavior.
---
## Target File Structure
- Create `src/wf_mcp/broker/service/content_access.py`
- Owns `ContentAccessService`.
- Depends on `SourceCatalogService` (for local docs lookup and catalog entry resolution), `UpstreamTransportService` (for upstream reads/renders), `ConnectionService` (for connection lookup), and `BrokerEventRecorder` (for local-docs event recording).
- Contains docstrings stating this is a broker-internal service, not a protocol-neutral API.
- Modify `src/wf_mcp/broker/service/core.py`
- Add `content_access: ContentAccessService = field(init=False)`.
- Construct it after `source_catalog`, `upstream`, `connection_service`, and `events` in `__post_init__`.
- Replace `read_resource` and `render_prompt` bodies with one-line delegates.
- Add tests in `tests/wf_mcp/service/test_content_access.py`.
- Update docs:
- `docs/superpowers/research/2026-06-02-wfmcpservice-remaining-responsibilities.md` if stale.
---
## Task 1: Add ContentAccessService Skeleton
**Files:**
- Create: `src/wf_mcp/broker/service/content_access.py`
- Create: `tests/wf_mcp/service/test_content_access.py`
- [ ] **Step 1: Write a direct local-resource test**
Create `tests/wf_mcp/service/test_content_access.py`:
```python
from __future__ import annotations
import asyncio
import pytest
from wf_mcp.broker import WfMcpService
from wf_mcp.broker.service.content_access import ContentAccessService
from wf_mcp.broker.service.connection_service import ConnectionService
from wf_mcp.broker.service.events import BrokerEventRecorder
from wf_mcp.broker.service.source_catalog import SourceCatalogService
from wf_mcp.broker.service.upstream_transport import UpstreamTransportService
from wf_mcp.events import EventBus
from wf_mcp.models import ConnectionConfig
from wf_mcp.storage import FileStore
from wf_platform import (
CapabilityBuckets,
CapabilitySource,
DocumentationPrompt,
DocumentationResource,
SourceVisibility,
)
from ..test_support import FakeAdapter, local_temp_root
def _make_content_access(
*,
store_root: str = "content_access_default",
) -> tuple[ContentAccessService, BrokerEventRecorder]:
store = FileStore(local_temp_root() / store_root)
events = BrokerEventRecorder(EventBus())
connection_service = ConnectionService(events=events)
upstream = UpstreamTransportService(
store=store,
event_sink=events.record_event,
)
source_catalog = SourceCatalogService(
store=store,
connection_lookup=connection_service.get,
connection_list_enabled=connection_service.list_enabled,
connection_list_all=connection_service.list_all,
tool_executor_for=upstream.tool_executor_for,
load_auth=upstream.load_auth,
emit_event=events.record_event,
)
connection_service.bind_source_catalog(source_catalog)
_register_local_docs(source_catalog)
content_access = ContentAccessService(
source_catalog=source_catalog,
upstream=upstream,
connection_service=connection_service,
event_sink=events.record_event,
)
return content_access, events
def _register_local_docs(source_catalog: SourceCatalogService) -> None:
"""Install deterministic local docs without depending on repo Markdown files."""
source_catalog.register_capability_source(
CapabilitySource(
id="test.docs",
kind="system",
capabilities=CapabilityBuckets(
resources={
"test.docs.example": DocumentationResource(
name="test.docs.example",
uri="wf://docs/example",
title="Example Doc",
description="Test documentation resource.",
mime_type="text/markdown",
text="# Example",
)
},
prompts={
"test.docs.guide": DocumentationPrompt(
name="test.docs.guide",
title="Guide Prompt",
description="Test documentation prompt.",
text="Use the test docs.",
)
},
),
visibility=SourceVisibility(planner=True),
)
)
def test_content_access_reads_local_documentation_resource() -> None:
content_access, events = _make_content_access()
result = asyncio.run(
content_access.read_resource("test.docs.example")
)
assert result["contents"][0]["uri"] == "wf://docs/example"
assert result["contents"][0]["text"] == "# Example"
assert "resource_read_completed" in [e.kind for e in events.list_events()]
```
- [ ] **Step 2: Run the test and verify it fails**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_reads_local_documentation_resource -q
```
Expected: import failure because `content_access.py` does not exist.
- [ ] **Step 3: Create the skeleton service**
Create `src/wf_mcp/broker/service/content_access.py`:
```python
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from typing import Any
from wf_mcp.events import McpEvent, make_event
from .connection_service import ConnectionService
from .source_catalog import SourceCatalogService
from .upstream_transport import UpstreamTransportService
EventSink = Callable[[McpEvent], None]
@dataclass(slots=True)
class ContentAccessService:
"""Own local-docs-aware resource reads and prompt renders for the broker.
This service checks SourceCatalogService for local documentation entries
before falling back to upstream transport. It is broker-internal, not a
protocol-neutral content API.
"""
source_catalog: SourceCatalogService
upstream: UpstreamTransportService
connection_service: ConnectionService
event_sink: EventSink
async def read_resource(self, qualified_name: str) -> dict[str, Any]:
local_resource = self.source_catalog.local_documentation_resource(
qualified_name
)
if local_resource is not None:
self.event_sink(
make_event(
"resource_read_completed",
capability_id=qualified_name,
payload={"uri": local_resource.uri, "source": "local"},
)
)
return {
"contents": [
{
"uri": local_resource.uri,
"mimeType": local_resource.mime_type,
"text": local_resource.text,
}
]
}
resource = self.source_catalog.get_resource(qualified_name)
connection = self.connection_service.get(resource.connection_id)
return await self.upstream.read_resource(
connection,
qualified_name,
resource.uri,
)
async def render_prompt(
self,
qualified_name: str,
*,
arguments: dict[str, str] | None = None,
) -> dict[str, Any]:
local_prompt = self.source_catalog.local_documentation_prompt(qualified_name)
if local_prompt is not None:
self.event_sink(
make_event(
"prompt_get_completed",
capability_id=qualified_name,
payload={
"argument_keys": sorted((arguments or {}).keys()),
"source": "local",
},
)
)
return {
"description": local_prompt.description,
"messages": [
{
"role": "user",
"content": {
"type": "text",
"text": local_prompt.text,
},
}
],
}
prompt = self.source_catalog.get_prompt(qualified_name)
connection = self.connection_service.get(prompt.connection_id)
return await self.upstream.render_prompt(
connection,
qualified_name,
prompt.local_name,
arguments,
)
```
- [ ] **Step 4: Run the test**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_reads_local_documentation_resource -q
```
Expected: pass.
- [ ] **Step 5: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/content_access.py tests/wf_mcp/service/test_content_access.py
```
Expected: pass.
---
## Task 2: Add Direct Upstream Resource and Prompt Tests
**Files:**
- Modify: `tests/wf_mcp/service/test_content_access.py`
- [ ] **Step 1: Add upstream resource read test**
Append to `tests/wf_mcp/service/test_content_access.py`:
```python
def test_content_access_reads_upstream_resource_with_events() -> None:
service = WfMcpService(
store=FileStore(local_temp_root() / "content_upstream_resource")
)
service.register_connection(
ConnectionConfig(id="demo.personal", server="demo", account="personal")
)
service.register_adapter("demo", FakeAdapter())
asyncio.run(service.refresh_connection_catalog("demo.personal"))
result = asyncio.run(
service.content_access.read_resource("demo.personal.resource.welcome")
)
assert result["contents"][0]["text"] == "Welcome from the fake adapter resource."
event_kinds = [e.kind for e in service.list_events()]
assert "resource_read_started" in event_kinds
assert "resource_read_completed" in event_kinds
def test_content_access_renders_upstream_prompt_with_events() -> None:
service = WfMcpService(
store=FileStore(local_temp_root() / "content_upstream_prompt")
)
service.register_connection(
ConnectionConfig(id="demo.personal", server="demo", account="personal")
)
service.register_adapter("demo", FakeAdapter())
asyncio.run(service.refresh_connection_catalog("demo.personal"))
result = asyncio.run(
service.content_access.render_prompt(
"demo.personal.prompt.summarize",
arguments={"text": "hello world"},
)
)
assert "hello world" in result["messages"][0]["content"]["text"]
event_kinds = [e.kind for e in service.list_events()]
assert "prompt_get_started" in event_kinds
assert "prompt_get_completed" in event_kinds
```
- [ ] **Step 2: Run the tests and verify they fail**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_reads_upstream_resource_with_events tests/wf_mcp/service/test_content_access.py::test_content_access_renders_upstream_prompt_with_events -q
```
Expected: fail because `service.content_access` does not exist on `WfMcpService`.
- [ ] **Step 3: Wire ContentAccessService into WfMcpService**
In `src/wf_mcp/broker/service/core.py`, import:
```python
from .content_access import ContentAccessService
```
Add field:
```python
content_access: ContentAccessService = field(init=False)
```
In `__post_init__`, after constructing `upstream`, `source_catalog`, `connection_service`, and `events`:
```python
self.content_access = ContentAccessService(
source_catalog=self.source_catalog,
upstream=self.upstream,
connection_service=self.connection_service,
event_sink=self.events.record_event,
)
```
- [ ] **Step 4: Run the tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_reads_upstream_resource_with_events tests/wf_mcp/service/test_content_access.py::test_content_access_renders_upstream_prompt_with_events -q
```
Expected: pass.
- [ ] **Step 5: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/content_access.py tests/wf_mcp/service/test_content_access.py
```
Expected: pass.
---
## Task 3: Delegate WfMcpService.read_resource and render_prompt
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- Test: `tests/wf_mcp/service/test_events.py`
- Test: `tests/wf_mcp/service/test_content_access.py`
- [ ] **Step 1: Replace read_resource body with delegate**
In `src/wf_mcp/broker/service/core.py`, replace:
```python
async def read_resource(self, qualified_name: str) -> dict[str, Any]:
local_resource = self.source_catalog.local_documentation_resource(
qualified_name
)
if local_resource is not None:
self._record_event(
make_event(
"resource_read_completed",
capability_id=qualified_name,
payload={"uri": local_resource.uri, "source": "local"},
)
)
return {
"contents": [
{
"uri": local_resource.uri,
"mimeType": local_resource.mime_type,
"text": local_resource.text,
}
]
}
resource = self.get_resource(qualified_name)
connection = self.connections.get(resource.connection_id)
return await self.upstream.read_resource(
connection,
qualified_name,
resource.uri,
)
```
with:
```python
async def read_resource(self, qualified_name: str) -> dict[str, Any]:
return await self.content_access.read_resource(qualified_name)
```
- [ ] **Step 2: Replace render_prompt body with delegate**
Replace:
```python
async def render_prompt(
self,
qualified_name: str,
*,
arguments: dict[str, str] | None = None,
) -> dict[str, Any]:
local_prompt = self.source_catalog.local_documentation_prompt(qualified_name)
if local_prompt is not None:
self._record_event(
make_event(
"prompt_get_completed",
capability_id=qualified_name,
payload={
"argument_keys": sorted((arguments or {}).keys()),
"source": "local",
},
)
)
return {
"description": local_prompt.description,
"messages": [
{
"role": "user",
"content": {
"type": "text",
"text": local_prompt.text,
},
}
],
}
prompt = self.get_prompt(qualified_name)
connection = self.connections.get(prompt.connection_id)
return await self.upstream.render_prompt(
connection,
qualified_name,
prompt.local_name,
arguments,
)
```
with:
```python
async def render_prompt(
self,
qualified_name: str,
*,
arguments: dict[str, str] | None = None,
) -> dict[str, Any]:
return await self.content_access.render_prompt(
qualified_name,
arguments=arguments,
)
```
- [ ] **Step 3: Run the existing event/proxy tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_events.py::test_service_can_proxy_resource_reads_and_prompt_gets -q
```
Expected: pass. This proves the delegate preserves existing behavior.
- [ ] **Step 4: Run full content_access tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_content_access.py -q
```
Expected: pass.
- [ ] **Step 5: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/core.py
```
Expected: pass.
---
## Task 4: Add Local Prompt Test and Edge-Case Coverage
**Files:**
- Modify: `tests/wf_mcp/service/test_content_access.py`
- [ ] **Step 1: Add local prompt test**
Append:
```python
def test_content_access_renders_local_documentation_prompt() -> None:
content_access, events = _make_content_access(
store_root="content_access_local_prompt"
)
result = asyncio.run(
content_access.render_prompt("test.docs.guide")
)
assert result["description"] == "Test documentation prompt."
assert result["messages"][0]["role"] == "user"
assert result["messages"][0]["content"]["text"] == "Use the test docs."
assert "prompt_get_completed" in [e.kind for e in events.list_events()]
```
- [ ] **Step 2: Run the local prompt test**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_renders_local_documentation_prompt -q
```
Expected: pass.
- [ ] **Step 3: Add missing-resource error test**
Append:
```python
def test_content_access_raises_on_unknown_resource() -> None:
content_access, _ = _make_content_access(
store_root="content_access_missing_resource"
)
with pytest.raises(KeyError):
asyncio.run(content_access.read_resource("nonexistent.resource"))
```
- [ ] **Step 4: Run the error test**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_raises_on_unknown_resource -q
```
Expected: pass (KeyError from catalog lookup).
- [ ] **Step 5: Run ruff**
Run:
```bash
uv run ruff check tests/wf_mcp/service/test_content_access.py
```
Expected: pass.
---
## Task 5: Clean Unused Imports and Verify
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- Modify: `docs/superpowers/research/2026-06-02-wfmcpservice-remaining-responsibilities.md` if stale.
- [ ] **Step 1: Remove stale imports from core.py**
After the delegate change, `core.py` may no longer use `make_event` directly (verify with grep). If `make_event` is still used by `_record_catalog_change_events` or other methods, keep it. Only remove imports that are truly unused after the extraction.
Run:
```bash
rg -n "make_event" src/wf_mcp/broker/service/core.py
```
If only the removed `read_resource`/`render_prompt` bodies used `make_event`, remove the import. Otherwise keep it.
- [ ] **Step 2: Update research doc if stale**
If `docs/superpowers/research/2026-06-02-wfmcpservice-remaining-responsibilities.md` still lists `read_resource` and `render_prompt` as "Mixed real responsibility", update their classification:
```markdown
| `read_resource` | Delegates to `ContentAccessService` | Compatibility facade | Keep for admin/MCP callers; new code should call `content_access`. |
| `render_prompt` | Delegates to `ContentAccessService` | Compatibility facade | Keep for admin/MCP callers; new code should call `content_access`. |
```
- [ ] **Step 3: Run focused verification**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_content_access.py tests/wf_mcp/service/test_events.py -q
```
Expected: all selected tests pass.
- [ ] **Step 4: Run full verification**
Run:
```bash
uv run pytest -q
uv run ruff check src/wf_mcp/broker/service tests/wf_mcp/service
uv run ruff format --check src/wf_mcp/broker/service tests/wf_mcp/service
uv run basedpyright --level error
```
Expected:
- pytest passes.
- ruff check passes.
- ruff format check passes.
- basedpyright reports `0 errors`. If the known workspace enumeration warning causes a nonzero exit despite `0 errors`, record the exact output.
---
## Non-Goals and Follow-Up Slices
This plan intentionally leaves these for later:
1. **`invoke_method`/`send_notification`:** Already clean delegates to `upstream`. No local-docs branching. Moving them would add coupling without reducing complexity.
2. **`refresh_connection_catalog`:** Different semantics (catalog refresh, not content access). Stays on `WfMcpService` / `UpstreamTransportService`.
3. **Connection registry extraction:** Already done in `ConnectionService`; do not change it in this slice.
4. **Event recorder extraction:** Already done in `BrokerEventRecorder`; do not change it in this slice.
5. **`workflow_artifact_catalog_entry`:** Not broker content access; handle in a separate small cleanup.
---
## Self-Review
- Spec coverage: The plan extracts the local-docs-aware resource/prompt branching logic while preserving current public `WfMcpService` methods as one-line delegates. Upstream event semantics are unchanged.
- Placeholder scan: No placeholder implementation tasks are left. Local docs tests use deterministic test-only resource and prompt keys.
- Type consistency: `ContentAccessService` depends on `SourceCatalogService`, `UpstreamTransportService`, `ConnectionService`, and an `EventSink` callback. `WfMcpService.content_access` is exposed as a public field so callers (admin handlers, tests) can use it directly.
@@ -0,0 +1,891 @@
# WfMcpService Runtime Extraction Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Extract workflow compile/prepare/run/resume responsibilities from `WfMcpService` into a focused runtime implementation service while preserving existing public service methods and MCP/CLI behavior.
**Architecture:** Add `WorkflowRuntimeService` under `wf_mcp.broker.service`. It depends on `SourceCatalogService`, an optional artifact store, and an event emitter. `WfMcpService` remains the broker coordinator and compatibility façade; its runtime methods become thin delegates. This follows the previous `SourceCatalogService` extraction and keeps transport/auth/catalog refresh responsibilities out of this slice.
**Tech Stack:** Python 3.14, dataclasses, `wf_core` runtime APIs, `wf_api.runtime_dependencies`, `wf_api.saved_subgraphs`, pytest, ruff, basedpyright.
---
## Scope
Move now:
- `compile_plan`.
- `_prepare_workflow_runtime`, renamed to `prepare_workflow_runtime` on the new service.
- `run_workflow_from_plan`.
- `resume_workflow_from_plan`.
- Runtime event emission for `workflow_run_started`, `workflow_run_completed`, and `workflow_run_resumed`.
Keep now:
- Existing `WfMcpService.compile_plan`, `run_workflow_from_plan`, and `resume_workflow_from_plan` public method names as delegates.
- Existing source/catalog behavior in `SourceCatalogService`.
- Connection/adapters/auth/upstream I/O on `WfMcpService`.
- Catalog refresh on `WfMcpService`.
- Resource/prompt/raw method calls on `WfMcpService`.
- Event bus implementation on `WfMcpService`.
Do not do in this slice:
- Do not introduce a protocol-neutral runtime service in `wf_api`.
- Do not move `WorkflowOperationContext` itself.
- Do not change MCP tool schemas, CLI commands, run payload shape, or saved-run lifecycle models.
- Do not rename `WfMcpService`.
---
## Target File Structure
- Create `src/wf_mcp/broker/service/workflow_runtime.py`
- Owns runtime compile/prepare/run/resume.
- Has docstrings explaining that durable resume currently rebuilds dependencies from current in-memory service state.
- Depends on `SourceCatalogService`, optional `WorkflowArtifactStore`, and event emitter callback.
- Modify `src/wf_mcp/broker/service/core.py`
- Add `workflow_runtime: WorkflowRuntimeService = field(init=False)`.
- Construct it in `__post_init__` after `source_catalog`.
- Keep existing runtime methods as delegates.
- Remove runtime-only imports after the move.
- Modify `src/wf_mcp/broker/service/workflow_operation_context.py`
- `WfMcpWorkflowRuntimeRunner` should call `service.workflow_runtime` directly.
- Add direct runtime tests in `tests/wf_mcp/service/test_workflow_runtime.py`
- Component-level compile/run tests.
- Compatibility tests that `WfMcpService` still delegates and emits the same events.
- Update docs:
- `docs/current_roadmap.md`
- `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` if stale.
---
## Task 1: Add WorkflowRuntimeService Skeleton and Compile Test
**Files:**
- Create: `src/wf_mcp/broker/service/workflow_runtime.py`
- Create: `tests/wf_mcp/service/test_workflow_runtime.py`
- Modify: `src/wf_mcp/broker/service/core.py`
- [ ] **Step 1: Write the direct compile test**
Create `tests/wf_mcp/service/test_workflow_runtime.py`:
```python
from __future__ import annotations
from wf_core import NodeUse
from wf_mcp.broker.service.source_catalog import SourceCatalogService
from wf_mcp.broker.service.workflow_runtime import WorkflowRuntimeService
from wf_mcp.models import ConnectionConfig
from wf_mcp.storage import FileStore
from wf_platform import CapabilityBuckets, CapabilitySource, SourceVisibility
from ..test_support import echo_tool, local_temp_root
from .conftest import single_echo_plan
def _unused_tool_executor(connection: ConnectionConfig):
raise AssertionError("tool executor should not be used by direct compile tests")
def _source_catalog() -> SourceCatalogService:
connection = ConnectionConfig(
id="demo.personal",
server="demo",
account="personal",
)
catalog = SourceCatalogService(
store=FileStore(local_temp_root() / "runtime_source_catalog"),
connection_lookup=lambda connection_id: connection,
connection_list_enabled=lambda: [connection],
connection_list_all=lambda: [connection],
tool_executor_for=_unused_tool_executor,
load_auth=lambda connection_id: None,
emit_event=lambda event: None,
)
catalog.register_capability_source(
CapabilitySource(
id="demo.personal",
kind="connection",
capabilities=CapabilityBuckets(
node_specs={"demo.personal.echo_tool": echo_tool}
),
visibility=SourceVisibility(planner=True),
)
)
return catalog
def test_workflow_runtime_service_compiles_plan_directly() -> None:
runtime = WorkflowRuntimeService(
source_catalog=_source_catalog(),
artifact_store=None,
emit_event=lambda event: None,
)
workflow = runtime.compile_plan(
single_echo_plan("runtime_compile", "demo.echo_tool"),
{"demo.echo_tool": "demo.personal.echo_tool"},
)
node = workflow.nodes[0]
assert isinstance(node, NodeUse)
assert node.node == "demo.personal.echo_tool"
assert "demo.personal.echo_tool" in workflow.node_defs
```
- [ ] **Step 2: Run the compile test and verify it fails**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_compiles_plan_directly -q
```
Expected: import failure because `wf_mcp.broker.service.workflow_runtime` does not exist.
- [ ] **Step 3: Create WorkflowRuntimeService with compile_plan**
Create `src/wf_mcp/broker/service/workflow_runtime.py`:
```python
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from typing import Any
from wf_artifacts import WorkflowArtifactStore
from wf_authoring import NodeSpec
from wf_core import NodeUse, Workflow
from wf_api.models import RawWorkflowPlan
from ...events import McpEvent
from .source_catalog import SourceCatalogService
EventEmitter = Callable[[McpEvent], None]
@dataclass(slots=True)
class WorkflowRuntimeService:
"""Compile and execute workflow plans against broker-owned runtime deps.
This service is still an MCP broker implementation detail. It receives
source/catalog state from `SourceCatalogService`, but it does not own
connections, adapters, auth, or upstream discovery.
"""
source_catalog: SourceCatalogService
artifact_store: WorkflowArtifactStore | None
emit_event: EventEmitter
def compile_plan(
self,
plan: RawWorkflowPlan,
node_name_bindings: dict[str, str] | None = None,
) -> Workflow:
node_defs: dict[str, Any] = {}
bindings = node_name_bindings or {}
for step in plan.nodes:
if not isinstance(step, NodeUse):
continue
qualified_name = bindings.get(step.node, step.node)
spec: NodeSpec[Any, Any] = self.source_catalog.get_qualified_spec(
qualified_name
)
node_defs[qualified_name] = spec.to_node_def()
nodes = []
for node in plan.nodes:
payload = node.model_dump(by_alias=True)
if isinstance(node, NodeUse):
payload["node"] = bindings.get(node.node, node.node)
nodes.append(payload)
payload = {
"name": plan.name,
"input_schema": plan.input_schema,
"state_schema": plan.state_schema,
"output_schema": plan.output_schema,
"output": [binding.model_dump(mode="json") for binding in plan.output],
"outcomes": plan.outcomes,
"start": plan.start,
"node_defs": [node.model_dump() for node in node_defs.values()],
"nodes": nodes,
"edges": [edge.model_dump(by_alias=True) for edge in plan.edges],
}
return Workflow.model_validate(payload)
```
- [ ] **Step 4: Run the compile test**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_compiles_plan_directly -q
```
Expected: pass.
- [ ] **Step 5: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/workflow_runtime.py tests/wf_mcp/service/test_workflow_runtime.py
```
Expected: pass.
---
## Task 2: Wire Runtime Service Into WfMcpService as a Delegate
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- Test: `tests/wf_mcp/service/test_workflow_runtime.py`
- [ ] **Step 1: Add compatibility identity and delegate tests**
Append to `tests/wf_mcp/service/test_workflow_runtime.py`:
```python
from wf_mcp.broker import WfMcpService
def test_wfmcpservice_constructs_workflow_runtime_with_source_catalog() -> None:
service = WfMcpService(store=FileStore(local_temp_root() / "runtime_delegate"))
assert service.workflow_runtime.source_catalog is service.source_catalog
assert service.workflow_runtime.artifact_store is service.artifact_store
def test_wfmcpservice_compile_plan_delegates_to_workflow_runtime() -> None:
service = WfMcpService(store=FileStore(local_temp_root() / "runtime_compile_delegate"))
service.register_connection(
ConnectionConfig(id="demo.personal", server="demo", account="personal")
)
service.register_specs("demo.personal", echo_tool)
workflow = service.compile_plan(
single_echo_plan("runtime_delegate_compile", "demo.echo_tool"),
{"demo.echo_tool": "demo.personal.echo_tool"},
)
assert "demo.personal.echo_tool" in workflow.node_defs
```
- [ ] **Step 2: Run the new tests and verify they fail**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_wfmcpservice_constructs_workflow_runtime_with_source_catalog tests/wf_mcp/service/test_workflow_runtime.py::test_wfmcpservice_compile_plan_delegates_to_workflow_runtime -q
```
Expected: first test fails because `workflow_runtime` does not exist.
- [ ] **Step 3: Construct workflow_runtime in WfMcpService**
In `src/wf_mcp/broker/service/core.py`, import:
```python
from .workflow_runtime import WorkflowRuntimeService
```
Add the dataclass field:
```python
workflow_runtime: WorkflowRuntimeService = field(init=False)
```
In `__post_init__`, after `self.source_catalog = SourceCatalogService(...)`, add:
```python
self.workflow_runtime = WorkflowRuntimeService(
source_catalog=self.source_catalog,
artifact_store=self.artifact_store,
emit_event=self._record_event,
)
```
- [ ] **Step 4: Delegate compile_plan**
Replace `WfMcpService.compile_plan` body with:
```python
return self.workflow_runtime.compile_plan(plan, node_name_bindings)
```
Keep the method signature unchanged.
- [ ] **Step 5: Run delegate tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_wfmcpservice_constructs_workflow_runtime_with_source_catalog tests/wf_mcp/service/test_workflow_runtime.py::test_wfmcpservice_compile_plan_delegates_to_workflow_runtime -q
```
Expected: both pass.
- [ ] **Step 6: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/workflow_runtime.py tests/wf_mcp/service/test_workflow_runtime.py
```
Expected: pass.
---
## Task 3: Move Runtime Preparation
**Files:**
- Modify: `src/wf_mcp/broker/service/workflow_runtime.py`
- Modify: `src/wf_mcp/broker/service/core.py`
- Test: `tests/wf_mcp/service/test_workflow_runtime.py`
- [ ] **Step 1: Add a direct preparation test**
Append:
```python
def test_workflow_runtime_service_prepares_node_registry_and_reducers() -> None:
runtime = WorkflowRuntimeService(
source_catalog=_source_catalog(),
artifact_store=None,
emit_event=lambda event: None,
)
workflow, registry, reducers, prepared_subgraphs = runtime.prepare_workflow_runtime(
single_echo_plan("runtime_prepare", "demo.echo_tool"),
deployment=None,
artifact=None,
)
assert "demo.personal.echo_tool" in workflow.node_defs
assert "demo.personal.echo_tool" in registry
assert isinstance(reducers, dict)
assert prepared_subgraphs == {}
```
- [ ] **Step 2: Run the preparation test and verify it fails**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_prepares_node_registry_and_reducers -q
```
Expected: fail because `prepare_workflow_runtime` does not exist.
- [ ] **Step 3: Move _prepare_workflow_runtime into WorkflowRuntimeService**
In `src/wf_mcp/broker/service/workflow_runtime.py`, add imports:
```python
from wf_artifacts import WorkflowArtifact, WorkflowDeployment
from wf_api.runtime_dependencies import resolve_runtime_dependencies
from wf_api.saved_subgraphs import (
SavedSubgraphTree,
prepare_saved_subgraphs,
resolve_saved_subgraph_tree,
)
```
Add the method:
```python
def prepare_workflow_runtime(
self,
plan: RawWorkflowPlan,
*,
deployment: WorkflowDeployment | None,
artifact: WorkflowArtifact | None,
saved_subgraph_tree: SavedSubgraphTree | None = None,
) -> tuple[Workflow, dict[str, Any], dict[str, Any], dict[str, Any]]:
"""Resolve bindings once into the executable pieces core expects.
Saved-run resume still rebuilds prepared dependencies from the current
in-memory broker state. Durable resume will need a stricter snapshot,
but this keeps the current platform boundary explicit.
"""
plan_node_names = [
node.node for node in plan.nodes if isinstance(node, NodeUse)
]
runtime_artifact = artifact or WorkflowArtifact(
id=plan.name,
version=1,
title=plan.name,
input_schema=plan.input_schema,
output_schema=plan.output_schema,
outcomes=("completed",),
plan=plan.model_dump(mode="json", by_alias=True),
)
dependencies = resolve_runtime_dependencies(
artifact=runtime_artifact,
deployment=deployment,
sources=self.source_catalog.capability_sources,
plan_node_names=plan_node_names,
)
prepared_subgraphs = {}
if saved_subgraph_tree is not None:
prepared_subgraphs = prepare_saved_subgraphs(
tree=saved_subgraph_tree,
deployment=deployment,
sources=self.source_catalog.capability_sources,
compile_plan=self.compile_plan,
)
elif artifact is not None and self.artifact_store is not None:
tree = resolve_saved_subgraph_tree(
root_artifact=artifact,
artifact_store=self.artifact_store,
)
prepared_subgraphs = prepare_saved_subgraphs(
tree=tree,
deployment=deployment,
sources=self.source_catalog.capability_sources,
compile_plan=self.compile_plan,
)
workflow = self.compile_plan(plan, dependencies.node_name_bindings)
return (
workflow,
dependencies.node_registry,
dependencies.reducers,
prepared_subgraphs,
)
```
- [ ] **Step 4: Delegate _prepare_workflow_runtime**
In `src/wf_mcp/broker/service/core.py`, replace `_prepare_workflow_runtime` body with:
```python
return self.workflow_runtime.prepare_workflow_runtime(
plan,
deployment=deployment,
artifact=artifact,
saved_subgraph_tree=saved_subgraph_tree,
)
```
Keep the private method signature unchanged for compatibility with any tests or internal callers.
- [ ] **Step 5: Run preparation and hydrated runtime regression tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_prepares_node_registry_and_reducers tests/wf_mcp/service/test_catalog.py::test_service_hydrates_planner_specs_from_stored_catalog -q
```
Expected: both pass.
- [ ] **Step 6: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/workflow_runtime.py tests/wf_mcp/service/test_workflow_runtime.py
```
Expected: pass.
---
## Task 4: Move Run and Resume Execution
**Files:**
- Modify: `src/wf_mcp/broker/service/workflow_runtime.py`
- Modify: `src/wf_mcp/broker/service/core.py`
- Test: `tests/wf_mcp/service/test_workflow_runtime.py`
- Test: `tests/wf_api/test_run_api.py`
- [ ] **Step 1: Add a direct run test with event assertions**
Append:
```python
import asyncio
def test_workflow_runtime_service_runs_plan_and_emits_events() -> None:
events = []
runtime = WorkflowRuntimeService(
source_catalog=_source_catalog(),
artifact_store=None,
emit_event=events.append,
)
run = asyncio.run(
runtime.run_workflow_from_plan(
single_echo_plan("runtime_run", "demo.echo_tool"),
{"text": "hello"},
)
)
assert run.output["echoed"] == "hello"
assert [event.type for event in events] == [
"workflow_run_started",
"workflow_run_completed",
]
assert events[1].payload["status"] == "completed"
```
- [ ] **Step 2: Run the direct run test and verify it fails**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_runs_plan_and_emits_events -q
```
Expected: fail because `WorkflowRuntimeService.run_workflow_from_plan` does not exist.
- [ ] **Step 3: Add run and resume methods to WorkflowRuntimeService**
In `src/wf_mcp/broker/service/workflow_runtime.py`, add imports:
```python
from wf_core import (
RunState,
execute_workflow_result_async,
resume_workflow_result_async,
)
from ...events import make_event
```
Add:
```python
async def run_workflow_from_plan(
self,
plan: RawWorkflowPlan,
workflow_input: dict[str, Any],
deployment: WorkflowDeployment | None = None,
artifact: WorkflowArtifact | None = None,
saved_subgraph_tree: SavedSubgraphTree | None = None,
) -> RunState:
self.emit_event(
make_event(
"workflow_run_started",
workflow_name=plan.name,
payload={"input_keys": sorted(workflow_input.keys())},
)
)
workflow, registry, reducers, prepared_subgraphs = (
self.prepare_workflow_runtime(
plan,
deployment=deployment,
artifact=artifact,
saved_subgraph_tree=saved_subgraph_tree,
)
)
run = await execute_workflow_result_async(
workflow,
workflow_input,
registry,
reducers=reducers,
subgraphs=prepared_subgraphs,
)
self.emit_event(
make_event(
"workflow_run_completed",
workflow_name=plan.name,
payload={"status": run.status.value},
)
)
return run
async def resume_workflow_from_plan(
self,
plan: RawWorkflowPlan,
run: RunState,
*,
resume_payload: dict[str, Any],
resume_outcome: str = "submitted",
deployment: WorkflowDeployment | None = None,
artifact: WorkflowArtifact | None = None,
saved_subgraph_tree: SavedSubgraphTree | None = None,
) -> RunState:
"""Resume one stopped run using its prepared runtime dependency boundary."""
workflow, registry, reducers, prepared_subgraphs = (
self.prepare_workflow_runtime(
plan,
deployment=deployment,
artifact=artifact,
saved_subgraph_tree=saved_subgraph_tree,
)
)
resumed = await resume_workflow_result_async(
workflow,
run,
registry,
resume_payload=resume_payload,
resume_outcome=resume_outcome,
reducers=reducers,
subgraphs=prepared_subgraphs,
)
self.emit_event(
make_event(
"workflow_run_resumed",
workflow_name=plan.name,
payload={"status": resumed.status.value},
)
)
return resumed
```
- [ ] **Step 4: Delegate WfMcpService run/resume**
Replace `WfMcpService.run_workflow_from_plan` body with:
```python
return await self.workflow_runtime.run_workflow_from_plan(
plan,
workflow_input,
deployment=deployment,
artifact=artifact,
saved_subgraph_tree=saved_subgraph_tree,
)
```
Replace `WfMcpService.resume_workflow_from_plan` body with:
```python
return await self.workflow_runtime.resume_workflow_from_plan(
plan,
run,
resume_payload=resume_payload,
resume_outcome=resume_outcome,
deployment=deployment,
artifact=artifact,
saved_subgraph_tree=saved_subgraph_tree,
)
```
Keep public signatures unchanged.
- [ ] **Step 5: Run direct and API run tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_runs_plan_and_emits_events tests/wf_api/test_run_api.py -q
```
Expected: pass.
- [ ] **Step 6: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/workflow_runtime.py tests/wf_mcp/service/test_workflow_runtime.py
```
Expected: pass.
---
## Task 5: Point WorkflowOperationContext Runtime Adapter at workflow_runtime
**Files:**
- Modify: `src/wf_mcp/broker/service/workflow_operation_context.py`
- Test: `tests/wf_api/test_operation_context.py`
- Test: `tests/wf_api/test_run_api.py`
- [ ] **Step 1: Add an adapter identity test**
In `tests/wf_api/test_operation_context.py`, add:
```python
def test_context_runtime_runner_uses_workflow_runtime_service() -> None:
service = WfMcpService(store=FileStore(local_temp_root() / "context_runtime"))
context = context_from_service(service)
assert getattr(context.runtime, "runtime") is service.workflow_runtime
```
If this file does not have `local_temp_root`, use the same temp-store helper style already used by its neighboring tests.
- [ ] **Step 2: Run the adapter test and verify it fails**
Run:
```bash
uv run pytest tests/wf_api/test_operation_context.py::test_context_runtime_runner_uses_workflow_runtime_service -q
```
Expected: fail because `WfMcpWorkflowRuntimeRunner` stores `service`, not `runtime`.
- [ ] **Step 3: Update runtime adapter**
In `src/wf_mcp/broker/service/workflow_operation_context.py`, import:
```python
from .workflow_runtime import WorkflowRuntimeService
```
Change:
```python
class WfMcpWorkflowRuntimeRunner(WorkflowRuntimeRunner):
"""Adapter-owned runtime runner backed by WfMcpService."""
service: WfMcpService
```
to:
```python
class WfMcpWorkflowRuntimeRunner(WorkflowRuntimeRunner):
"""Adapter-owned runtime runner backed by WorkflowRuntimeService."""
runtime: WorkflowRuntimeService
```
Replace calls from `self.service.run_workflow_from_plan(...)` and `self.service.resume_workflow_from_plan(...)` to `self.runtime.run_workflow_from_plan(...)` and `self.runtime.resume_workflow_from_plan(...)`.
In `context_from_service`, change:
```python
runtime=WfMcpWorkflowRuntimeRunner(service),
```
to:
```python
runtime=WfMcpWorkflowRuntimeRunner(service.workflow_runtime),
```
- [ ] **Step 4: Run context and run API tests**
Run:
```bash
uv run pytest tests/wf_api/test_operation_context.py::test_context_runtime_runner_uses_workflow_runtime_service tests/wf_api/test_run_api.py -q
```
Expected: pass.
- [ ] **Step 5: Run ruff**
Run:
```bash
uv run ruff check src/wf_mcp/broker/service/workflow_operation_context.py tests/wf_api/test_operation_context.py
```
Expected: pass.
---
## Task 6: Clean Imports, Docs, and Verify
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- Modify: `src/wf_mcp/broker/service/workflow_runtime.py`
- Modify: `docs/current_roadmap.md`
- Modify: `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` if stale.
- [ ] **Step 1: Remove stale runtime imports from core.py**
After the move, `src/wf_mcp/broker/service/core.py` should no longer import runtime-only names such as:
```python
from wf_core import NodeUse, Workflow, execute_workflow_result_async, resume_workflow_result_async
from wf_api.runtime_dependencies import resolve_runtime_dependencies
from wf_api.saved_subgraphs import prepare_saved_subgraphs, resolve_saved_subgraph_tree
```
Keep names still needed for type annotations, public signatures, or non-runtime service methods:
```python
from wf_core import RunState
from wf_api.models import RawWorkflowPlan
from wf_api.saved_subgraphs import SavedSubgraphTree
```
- [ ] **Step 2: Add roadmap note**
In `docs/current_roadmap.md`, under the wf_api/service extraction bullets, add:
```markdown
- Workflow runtime execution is being separated from broker coordination.
`WorkflowRuntimeService` now owns plan compilation, dependency preparation,
run, and resume; `WfMcpService` keeps delegate methods for compatibility.
```
- [ ] **Step 3: Update extraction map if stale**
If `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` says `WfMcpService` directly owns workflow runtime execution, add or update a note:
```markdown
Workflow runtime ownership is now split: `WorkflowRuntimeService` owns plan
compilation, dependency preparation, run, and resume. `WfMcpService` remains the
broker coordinator and compatibility façade.
```
Do not edit the file if it already describes this state.
- [ ] **Step 4: Run focused verification**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py tests/wf_mcp/service/test_catalog.py::test_service_hydrates_planner_specs_from_stored_catalog tests/wf_api/test_operation_context.py tests/wf_api/test_run_api.py tests/wf_mcp/workflow_surface/test_runs.py -q
```
Expected: all selected tests pass.
- [ ] **Step 5: Run full verification**
Run:
```bash
uv run pytest -q
uv run ruff check src/wf_mcp/broker/service src/wf_api tests/wf_mcp/service tests/wf_api
uv run ruff format --check src/wf_mcp/broker/service src/wf_api tests/wf_mcp/service tests/wf_api docs/current_roadmap.md
uv run basedpyright --level error
```
Expected:
- pytest passes.
- ruff check passes.
- ruff format check passes.
- basedpyright reports `0 errors`. If the known workspace enumeration warning causes a nonzero exit despite `0 errors`, record the exact output.
---
## Non-Goals and Follow-Up Slices
This plan intentionally leaves these slices for later:
1. **Transport/upstream service extraction:** move connection lookup, adapter lookup, auth loading, resource reads, prompt rendering, raw method calls, and notifications.
2. **Event recorder extraction:** turn `_record_event` and catalog change event emission into an injected event recorder implementation.
3. **WfMcpService rename:** once most implementations are extracted, rename the remaining coordinator to a clearer broker runtime name if the public import impact is acceptable.
4. **Protocol-neutral API expansion:** decide whether runtime execution belongs behind a `wf_api` implementation protocol once CLI/HTTP need a shared process boundary.
---
## Self-Review
- Spec coverage: The plan extracts compile/prepare/run/resume and preserves current public service methods, context adaptation, saved subgraph preparation, and run payload behavior.
- Placeholder scan: No placeholders or vague “write tests” steps remain; each task has explicit code snippets and commands.
- Type consistency: `WorkflowRuntimeService` receives `SourceCatalogService`, `WorkflowArtifactStore | None`, and `EventEmitter`; later tasks use the same names and signatures.
@@ -0,0 +1,29 @@
# Workflow Skill Docs Mirror Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add a repo-local `wf-workflow` skill with agent-facing references and expose those same skill files through the MCP documentation source.
**Architecture:** `docs/` remains project-facing documentation. `skills/wf-workflow/` becomes a self-contained agent-facing operating manual with distilled references. `src/wf_mcp/documentation.py` exposes the skill files as documentation resources under `wf://skills/...` without changing existing `wf://docs/...` resources.
**Tech Stack:** Markdown skills, `wf_mcp.documentation`, `wf_platform.DocumentationResource`, pytest.
---
## Tasks
- [ ] Add a failing docs resource test for `wf://skills/wf-workflow/SKILL.md` and one reference.
- [ ] Create `skills/wf-workflow/SKILL.md`.
- [ ] Create `skills/wf-workflow/references/workflow-lifecycle.md`.
- [ ] Create `skills/wf-workflow/references/capabilities-and-wrappers.md`.
- [ ] Create `skills/wf-workflow/references/draft-workspaces.md`.
- [ ] Create `skills/wf-workflow/references/troubleshooting.md`.
- [ ] Update `src/wf_mcp/documentation.py` to load the skill files as resources.
- [ ] Verify focused docs tests and lint.
## Guardrails
- Do not replace the existing `skills/wf-cli` skill.
- Do not make `SKILL.md` a link-only file.
- Do not symlink skill references to `docs/`; checked-in distilled files are more portable.
- Do not remove existing `wf://docs/...` resources.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,519 @@
# Persisted Run/Resume Hardening Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Lock down the persisted run/resume contract with missing regression tests and add a fail-fast required-store factory for durable API frontends.
**Architecture:** The current V1 run/resume behavior mostly exists in `wf_api.runs` and `wf_api.run_lifecycle`. This plan avoids redesigning `RunState`, retry, checkpoint cadence, or transactional storage. It adds tests for weakly covered contract rules, then introduces a protocol-neutral durable-context helper that future HTTP/API frontends can use before constructing `WorkflowApi`.
**Tech Stack:** Python 3.14, pytest, Pydantic v2, `wf_api`, `wf_artifacts`, `wf_core`, existing MCP-backed test fixtures.
---
## File Map
- Modify `tests/wf_api/test_run_api.py`
- Add contract tests for rejected resume of non-interrupted stored runs.
- Add contract test proving deleted deployments do not break stored run inspection.
- Strengthen trace-range validation test to prove store lookup is not touched.
- Create `src/wf_api/durable_context.py`
- Add required-store validation for durable frontends.
- Add `durable_workflow_api(context)` helper returning `WorkflowApi`.
- Modify `src/wf_api/__init__.py`
- Export the durable-context helpers.
- Create `tests/wf_api/test_durable_context.py`
- Cover success and missing-store failures.
- Modify `docs/superpowers/specs/2026-06-03-persisted-run-resume-contract.md`
- Mark these V1 hardening items as implemented after tests/helper land.
- Modify `docs/current_roadmap.md`
- Add a short note that durable required-store factory exists; transactional backend remains future work.
Out of scope:
- No transactional run store.
- No concurrent resume compare-and-swap.
- No paged run listing/checkpoint listing.
- No automatic retry/timeout policy.
- No HTTP server routes.
---
## Task 1: Add Missing Run Contract Tests
**Files:**
- Modify: `tests/wf_api/test_run_api.py`
- [ ] **Step 1: Add imports for stored-run helpers**
At the top of `tests/wf_api/test_run_api.py`, extend imports:
```python
from wf_artifacts import (
FileWorkflowArtifactStore,
FileRunStore,
ResumeReadiness,
WorkflowDeployment,
)
```
If `ResumeReadiness` is already imported later by another task, keep one import only.
- [ ] **Step 2: Add test for non-interrupted resume rejection**
Append this test after `test_run_api_completed_run_persists`:
```python
def test_run_api_rejects_resume_for_completed_run() -> None:
root = local_temp_root() / "run_api_resume_completed_rejected"
service, _ = _service_with_echo(root)
context = context_from_service(service)
api = WorkflowRunApi(context)
result = asyncio.run(
api.run_deployment(
deployment_id="echo.personal",
workflow_input={"text": "hello"},
)
)
with pytest.raises(ValueError, match="is not interrupted"):
asyncio.run(
api.resume_run(
run_id=result["run_id"],
resume_payload={"answer": "ignored"},
)
)
```
This locks down: `resume_run` only resumes stored runs whose latest status is `interrupted`.
- [ ] **Step 3: Add test that deleting deployment does not erase stored inspection**
Append this test after the non-interrupted resume test:
```python
def test_run_api_inspect_uses_pinned_environment_after_deployment_deleted() -> None:
root = local_temp_root() / "run_api_inspect_after_deployment_deleted"
service, artifact_store = _service_with_echo(root)
context = context_from_service(service)
api = WorkflowRunApi(context)
result = asyncio.run(
api.run_deployment(
deployment_id="echo.personal",
workflow_input={"text": "hello"},
)
)
artifact_store.delete_deployment("echo.personal")
summary = asyncio.run(api.inspect_run(run_id=result["run_id"]))
assert summary["status"] == "completed"
assert summary["run_id"] == result["run_id"]
assert summary["deployment_id"] == "echo.personal"
assert summary["artifact_id"] == "echo"
assert summary["output"]["echoed"] == "hello"
```
This locks down: stored run inspection reads pinned environment from `WorkflowRunRecord`, not mutable deployment storage.
- [ ] **Step 4: Strengthen trace-range-before-store validation**
In `test_run_api_rejects_invalid_trace_range_before_store_lookup`, replace the setup with a service whose run store raises if accessed:
```python
class ExplodingRunStore(FileRunStore):
def get_run(self, run_id: str):
raise AssertionError("run store must not be read before trace_range validation")
```
Then build the service explicitly:
```python
root = local_temp_root() / "run_api_invalid_trace_range"
service, _ = _service_with_echo(root)
service.run_store = ExplodingRunStore(root / "exploding_runs")
context = context_from_service(service)
api = WorkflowRunApi(context)
```
Keep the two existing `pytest.raises(ValueError, ...)` assertions unchanged. The test must still pass, proving invalid trace ranges fail before store lookup.
- [ ] **Step 5: Run focused run API tests**
Run:
```bash
uv run pytest tests/wf_api/test_run_api.py -q
```
Expected:
```text
all tests in tests/wf_api/test_run_api.py pass
```
- [ ] **Step 6: Commit Task 1**
```bash
git add tests/wf_api/test_run_api.py
git commit -m "test: lock down durable run resume contract"
```
---
## Task 2: Add Required-Store Durable Context Helper
**Files:**
- Create: `src/wf_api/durable_context.py`
- Modify: `src/wf_api/__init__.py`
- Test: `tests/wf_api/test_durable_context.py`
- [ ] **Step 1: Write failing durable-context tests**
Create `tests/wf_api/test_durable_context.py`:
```python
from __future__ import annotations
import pytest
from wf_api import WorkflowApi
from wf_api.durable_context import durable_workflow_api, require_workflow_stores
from wf_api.operation_context import WorkflowOperationContext
from wf_api.stores import file_workflow_stores
from wf_mcp.broker import WfMcpService
from wf_mcp.broker.service.workflow_operation_context import context_from_service
from wf_mcp.storage import FileStore
def test_require_workflow_stores_returns_existing_store_bundle(tmp_path) -> None:
stores = file_workflow_stores(tmp_path / "workflow_stores")
service = WfMcpService(
store=FileStore(tmp_path / "mcp"),
artifact_store=stores.artifact_store,
draft_workspace_store=stores.draft_workspace_store,
run_store=stores.run_store,
)
context = context_from_service(service)
required = require_workflow_stores(context)
assert required.artifact_store is stores.artifact_store
assert required.draft_workspace_store is stores.draft_workspace_store
assert required.run_store is stores.run_store
def test_require_workflow_stores_rejects_missing_store(tmp_path) -> None:
service = WfMcpService(
store=FileStore(tmp_path / "mcp"),
artifact_store=None,
draft_workspace_store=None,
run_store=None,
)
context = context_from_service(service)
with pytest.raises(ValueError, match="durable workflow API requires stores"):
require_workflow_stores(context)
def test_durable_workflow_api_returns_workflow_api_with_same_context(tmp_path) -> None:
stores = file_workflow_stores(tmp_path / "workflow_stores")
service = WfMcpService(
store=FileStore(tmp_path / "mcp"),
artifact_store=stores.artifact_store,
draft_workspace_store=stores.draft_workspace_store,
run_store=stores.run_store,
)
context = context_from_service(service)
api = durable_workflow_api(context)
assert isinstance(api, WorkflowApi)
assert api.context is context
```
- [ ] **Step 2: Run tests and verify they fail**
Run:
```bash
uv run pytest tests/wf_api/test_durable_context.py -q
```
Expected:
```text
FAIL with ModuleNotFoundError: No module named 'wf_api.durable_context'
```
- [ ] **Step 3: Implement durable context helper**
Create `src/wf_api/durable_context.py`:
```python
from __future__ import annotations
from .operation_context import WorkflowOperationContext
from .service import WorkflowApi
from .stores import WorkflowStores
def require_workflow_stores(context: WorkflowOperationContext) -> WorkflowStores:
"""Return required stores or fail before constructing durable frontends.
`WorkflowOperationContext` keeps stores optional for compatibility tests and
lightweight MCP surfaces. Durable API surfaces need all stores up front so a
run cannot start without somewhere to persist artifacts, drafts, and stopped
execution state.
"""
missing = []
if context.artifact_store is None:
missing.append("artifact_store")
if context.draft_workspace_store is None:
missing.append("draft_workspace_store")
if context.run_store is None:
missing.append("run_store")
if missing:
raise ValueError(
"durable workflow API requires stores: " + ", ".join(missing)
)
return WorkflowStores(
artifact_store=context.artifact_store,
draft_workspace_store=context.draft_workspace_store,
run_store=context.run_store,
)
def durable_workflow_api(context: WorkflowOperationContext) -> WorkflowApi:
"""Construct a WorkflowApi only after durable store dependencies exist."""
require_workflow_stores(context)
return WorkflowApi(context)
__all__ = ["durable_workflow_api", "require_workflow_stores"]
```
- [ ] **Step 4: Export helpers from wf_api**
Modify `src/wf_api/__init__.py` to import and export:
```python
from .durable_context import durable_workflow_api, require_workflow_stores
```
Add these names to `__all__`:
```python
"durable_workflow_api",
"require_workflow_stores",
```
- [ ] **Step 5: Run durable-context tests**
Run:
```bash
uv run pytest tests/wf_api/test_durable_context.py -q
```
Expected:
```text
3 passed
```
- [ ] **Step 6: Run wf_api import-direction test**
Run:
```bash
uv run pytest tests/wf_api/test_import_direction.py -q
```
Expected:
```text
1 passed
```
- [ ] **Step 7: Commit Task 2**
```bash
git add src/wf_api/durable_context.py src/wf_api/__init__.py tests/wf_api/test_durable_context.py
git commit -m "feat: add durable workflow API store guard"
```
---
## Task 3: Document Implemented Hardening
**Files:**
- Modify: `docs/superpowers/specs/2026-06-03-persisted-run-resume-contract.md`
- Modify: `docs/current_roadmap.md`
- [ ] **Step 1: Update spec current gaps**
In `docs/superpowers/specs/2026-06-03-persisted-run-resume-contract.md`, under `## Current Gaps / Next Implementation Work`, replace item 1 with:
```markdown
1. **Required stores for durable API**
- Implemented for process-local frontends through
`wf_api.durable_context.require_workflow_stores()` and
`wf_api.durable_context.durable_workflow_api()`.
- `WorkflowOperationContext` still allows optional stores for MCP test and
compatibility paths.
```
Under `## Implementation Order`, replace item 1 with:
```markdown
1. Contract regression tests now cover:
- non-interrupted `resume_run` rejection
- deleted deployment does not erase existing run inspection
- trace range validates before store lookup
- blocked resume writes no checkpoint (`tests/wf_mcp/test_saved_subgraphs.py`)
```
- [ ] **Step 2: Update current roadmap**
In `docs/current_roadmap.md`, find the durable/persisted run section and add:
```markdown
- `wf_api.durable_context` now provides a required-store guard for future
durable HTTP/API frontends. It preserves the current process-local behavior
while failing fast if artifact, draft, or run stores are missing.
```
If no durable/persisted run section exists, add it under the current `wf_api` or runtime roadmap notes.
- [ ] **Step 3: Run docs grep sanity check**
Run:
```bash
rg -n "durable_workflow_api|require_workflow_stores|non-interrupted|blocked resume writes no checkpoint" docs
```
Expected:
```text
matches in persisted-run spec and/or current roadmap
```
- [ ] **Step 4: Commit Task 3**
```bash
git add docs/superpowers/specs/2026-06-03-persisted-run-resume-contract.md docs/current_roadmap.md
git commit -m "docs: record durable run hardening status"
```
---
## Task 4: Final Verification
**Files:**
- Verify all touched code/tests/docs.
- [ ] **Step 1: Run focused tests**
Run:
```bash
uv run pytest tests/wf_api/test_run_api.py tests/wf_api/test_durable_context.py tests/wf_api/test_import_direction.py tests/wf_mcp/test_saved_subgraphs.py -q
```
Expected:
```text
all selected tests pass
```
- [ ] **Step 2: Run broader wf_api suite**
Run:
```bash
uv run pytest tests/wf_api -q
```
Expected:
```text
all wf_api tests pass
```
- [ ] **Step 3: Run ruff**
Run:
```bash
uv run ruff check src/wf_api tests/wf_api tests/wf_mcp/test_saved_subgraphs.py
uv run ruff format --check src/wf_api tests/wf_api tests/wf_mcp/test_saved_subgraphs.py
```
Expected:
```text
All checks passed
```
- [ ] **Step 4: Run basedpyright**
Run:
```bash
uv run basedpyright --level error
```
Expected:
```text
0 errors, 0 warnings, 0 notes
```
Known caveat: this repo may still exit nonzero with the workspace enumeration warning even when it reports `0 errors`.
- [ ] **Step 5: Final report**
Report:
```text
Implemented persisted run/resume hardening:
- added missing contract regression tests
- added wf_api durable required-store guard
- updated persisted-run spec and roadmap
Verification:
- focused tests: ...
- wf_api tests: ...
- ruff: ...
- basedpyright: ...
```
---
## Self-Review
Spec coverage:
- `resume_run` only resumes interrupted runs: Task 1.
- Pinned environment survives deployment deletion: Task 1.
- Trace range validates before store lookup: Task 1.
- Blocked resume writes no checkpoint: already covered by `tests/wf_mcp/test_saved_subgraphs.py`; referenced in Task 3 and final focused verification.
- Required-store context/factory for durable API surfaces: Task 2.
- Transactional backend, CAS resume guard, run listing, progress, retry/timeout: explicitly out of scope.
Placeholder scan:
- No placeholder implementation steps.
- Code snippets include exact functions and expected assertions.
Type consistency:
- `WorkflowStores`, `WorkflowOperationContext`, and `WorkflowApi` names match current `src/wf_api`.
- `TraceRangeLike` remains unchanged; this plan does not alter MCP/CLI schemas.
@@ -0,0 +1,565 @@
# Remote CLI Lifecycle RPC Methods Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make the `wf` draft/artifact/deploy commands work against `client.target.kind = "rpc_http"` instead of failing fast as local-only commands.
**Architecture:** Extend the fixed JSON-RPC method set, then extend `RpcWorkflowApiClient` to structurally support every `WorkflowApi` method used by the CLI. Only after the client and server both support the methods should the CLI command modules switch from local-only context to target-aware context. Keep this as remote workflow lifecycle support, not source registry/auth/MCP hosting.
**Tech Stack:** Python 3.14, Pydantic v2, fastapi-jsonrpc, httpx, Typer, pytest, ruff, basedpyright.
---
## Scope
In scope:
- Add missing JSON-RPC methods for draft workspace, artifact, and deployment operations used by CLI.
- Add matching `RpcWorkflowApiClient` methods.
- Route `wf draft`, `wf artifact`, and `wf deploy` through `load_cli_context_from_typer`.
- Add remote CLI integration tests for create draft → validate → save artifact → save deployment → validate deployment.
- Keep existing local CLI behavior and old `wf_mcp.config.json` compatibility.
Out of scope:
- Source registry.
- MCP/OpenAPI source config.
- `/mcp` hosting.
- Auth.
- SQL stores.
- Remote docs/schema/explain commands.
- Streaming/progress.
---
## Required Method Coverage
The client must implement every method used by these command modules:
```text
src/wf_cli/commands/drafts.py
list_draft_workspaces
get_draft_workspace
create_draft_workspace_from_capability
patch_draft_workspace
validate_draft_workspace
create_wrapper_from_workspace
create_artifact_from_workspace
src/wf_cli/commands/artifacts.py
list_artifacts
inspect_artifact
src/wf_cli/commands/deployments.py
validate_deployment
list_deployments
inspect_deployment
save_deployment
delete_deployment
```
Do not route a command module to target-aware context until the client has the
methods that module calls.
---
## Task 1: Add Missing RPC DTOs and Server Methods for Artifacts/Deployments
**Files:**
- Modify: `src/wf_transport_rpc_http/models.py`
- Modify: `src/wf_transport_rpc_http/app.py`
- Modify: `src/wf_transport_rpc_http/__init__.py`
- Modify: `tests/wf_transport_rpc_http/test_app.py`
- [ ] **Step 1: Add focused RPC app tests**
Append a test that seeds an artifact/deployment through `server.api`, then calls:
```text
workflow.artifacts.list
workflow.artifacts.inspect
workflow.deployments.list
workflow.deployments.inspect
workflow.deployments.delete
```
Assertions:
```python
assert listed_artifacts["result"]["nodes"]
assert inspected_artifact["result"]["artifact_id"] == "rpc_lifecycle"
assert listed_deployments["result"]["deployments"]
assert inspected_deployment["result"]["deployment_id"] == "rpc_lifecycle.default"
assert deleted["result"]["deployment_id"] == "rpc_lifecycle.default"
```
Use the existing `_constant_plan()` helper in `tests/wf_transport_rpc_http/test_app.py` if possible. Do not bypass the RPC app for the operations under test.
- [ ] **Step 2: Run the new test and verify method-not-found failures**
Run:
```bash
uv run pytest tests/wf_transport_rpc_http/test_app.py::test_rpc_artifact_and_deployment_catalog_methods -q
```
Expected: fail with JSON-RPC method-not-found for the newly required methods.
- [ ] **Step 3: Add DTOs**
In `src/wf_transport_rpc_http/models.py`, add:
```python
class ListArtifactsParams(RpcParamsModel):
query: str | None = None
kind: str | None = None
cursor: str | None = None
limit: int = Field(default=50, ge=1, le=100)
class InspectArtifactParams(RpcParamsModel):
artifact_id: str = Field(min_length=1)
version: int = Field(ge=1)
class ListDeploymentsParams(RpcParamsModel):
pass
class InspectDeploymentParams(RpcParamsModel):
deployment_id: str = Field(min_length=1)
class DeleteDeploymentParams(RpcParamsModel):
deployment_id: str = Field(min_length=1)
```
Export these from `src/wf_transport_rpc_http/__init__.py`.
- [ ] **Step 4: Register artifact/deployment methods**
In `src/wf_transport_rpc_http/app.py`, import the new DTOs and register:
```text
workflow.artifacts.list -> server.api.list_artifacts
workflow.artifacts.inspect -> server.api.inspect_artifact
workflow.deployments.list -> server.api.list_deployments
workflow.deployments.inspect -> server.api.inspect_deployment
workflow.deployments.delete -> server.api.delete_deployment
```
Use the same expected-error handling as existing methods:
```python
except (ValueError, KeyError, LookupError, FileNotFoundError) as exc:
raise_workflow_rpc_error(exc)
```
For `ListArtifactsParams.kind`, pass the string through as `kind=params.kind`.
If basedpyright complains because `WorkflowApi.list_artifacts` expects
`ArtifactKind | None`, narrow with:
```python
kind = params.kind if params.kind in {"workflow", "wrapper"} else None
```
and reject invalid values in the DTO if needed.
- [ ] **Step 5: Run focused transport tests**
Run:
```bash
uv run pytest tests/wf_transport_rpc_http/test_app.py -q
uv run ruff check src/wf_transport_rpc_http tests/wf_transport_rpc_http
uv run basedpyright --level error src/wf_transport_rpc_http tests/wf_transport_rpc_http
```
Expected: tests pass, ruff clean, basedpyright 0 errors.
---
## Task 2: Add Draft Workspace RPC Methods
**Files:**
- Modify: `src/wf_transport_rpc_http/models.py`
- Modify: `src/wf_transport_rpc_http/app.py`
- Modify: `src/wf_transport_rpc_http/__init__.py`
- Modify: `tests/wf_transport_rpc_http/test_app.py`
- [ ] **Step 1: Add focused draft workspace RPC test**
Append a test that calls:
```text
workflow.draft_workspaces.create_from_capability
workflow.draft_workspaces.list
workflow.draft_workspaces.get
workflow.draft_workspaces.validate
workflow.draft_workspaces.patch
workflow.draft_workspaces.create_artifact
workflow.draft_workspaces.create_wrapper
```
Use a small patch such as changing the draft name or title through the existing
workspace patch format. Assert:
```python
assert created["result"]["workspace_id"] == "remote_ws"
assert listed["result"]["workspaces"]
assert fetched["result"]["workspace_id"] == "remote_ws"
assert validated["result"]["status"] in {"valid", "invalid"}
assert patched["result"]["revision"] == created["result"]["revision"] + 1
assert artifact["result"]["artifact_id"] == "remote_artifact"
assert wrapper["result"]["artifact_id"] == "remote_wrapper"
```
If wrapper creation requires an output-capable draft and the simple capability
draft cannot satisfy it, keep `create_wrapper` covered by client method tests
and document why the RPC app integration test only covers `create_artifact`.
- [ ] **Step 2: Run the new test and verify method-not-found failures**
Run:
```bash
uv run pytest tests/wf_transport_rpc_http/test_app.py::test_rpc_draft_workspace_methods -q
```
Expected: fail with method-not-found for missing workspace methods.
- [ ] **Step 3: Add DTOs**
Add to `src/wf_transport_rpc_http/models.py`:
```python
class ListDraftWorkspacesParams(RpcParamsModel):
pass
class GetDraftWorkspaceParams(RpcParamsModel):
workspace_id: str = Field(min_length=1)
include_draft: bool = False
class PatchDraftWorkspaceParams(RpcParamsModel):
workspace_id: str = Field(min_length=1)
revision: int = Field(ge=1)
patch: list[dict[str, Any]]
class ValidateDraftWorkspaceParams(RpcParamsModel):
workspace_id: str = Field(min_length=1)
class CreateArtifactFromWorkspaceParams(RpcParamsModel):
workspace_id: str = Field(min_length=1)
artifact_id: str = Field(min_length=1)
version: int = Field(ge=1)
title: str = Field(min_length=1)
outcomes: list[str]
kind: str = "workflow"
description: str | None = None
required_capabilities: dict[str, dict[str, Any]] | None = None
source_bindings: dict[str, str] | None = None
created_from_catalog_version: str | None = None
class CreateWrapperFromWorkspaceParams(RpcParamsModel):
workspace_id: str = Field(min_length=1)
artifact_id: str = Field(min_length=1)
version: int = Field(ge=1)
title: str = Field(min_length=1)
outcomes: list[str]
description: str | None = None
required_capabilities: dict[str, dict[str, Any]] | None = None
source_bindings: dict[str, str] | None = None
created_from_catalog_version: str | None = None
```
Export these from `src/wf_transport_rpc_http/__init__.py`.
- [ ] **Step 4: Register draft workspace methods**
In `src/wf_transport_rpc_http/app.py`, register:
```text
workflow.draft_workspaces.list
workflow.draft_workspaces.get
workflow.draft_workspaces.create_from_capability
workflow.draft_workspaces.patch
workflow.draft_workspaces.validate
workflow.draft_workspaces.create_artifact
workflow.draft_workspaces.create_wrapper
```
Map them to the matching `server.api` methods:
```python
server.api.list_draft_workspaces()
server.api.get_draft_workspace(...)
server.api.create_draft_workspace_from_capability(...)
server.api.patch_draft_workspace(...)
server.api.validate_draft_workspace(...)
server.api.create_artifact_from_workspace(...)
server.api.create_wrapper_from_workspace(...)
```
The existing `workflow.drafts.create_from_capability` method may stay for
backward compatibility. It can call the same API method as
`workflow.draft_workspaces.create_from_capability`.
- [ ] **Step 5: Run focused transport tests**
Run:
```bash
uv run pytest tests/wf_transport_rpc_http/test_app.py -q
uv run ruff check src/wf_transport_rpc_http tests/wf_transport_rpc_http
uv run basedpyright --level error src/wf_transport_rpc_http tests/wf_transport_rpc_http
```
Expected: tests pass, ruff clean, basedpyright 0 errors.
---
## Task 3: Extend RpcWorkflowApiClient
**Files:**
- Modify: `src/wf_transport_rpc_http/client.py`
- Modify: `tests/wf_transport_rpc_http/test_client.py`
- [ ] **Step 1: Add client tests for the newly exposed methods**
Add tests covering:
```text
list_artifacts / inspect_artifact
list_deployments / inspect_deployment / validate_deployment / delete_deployment
list_draft_workspaces / get_draft_workspace / validate_draft_workspace
create_draft_workspace_from_capability
patch_draft_workspace
create_artifact_from_workspace
create_wrapper_from_workspace if feasible
```
Use `httpx.ASGITransport(app=create_rpc_app(server))` like the existing client
tests. Seed artifacts/deployments through `server.api` where the method under
test is only read/list/inspect. For create/patch workspace tests, call through
the client.
- [ ] **Step 2: Run client tests and verify failures**
Run:
```bash
uv run pytest tests/wf_transport_rpc_http/test_client.py -q
```
Expected: fail because `RpcWorkflowApiClient` lacks the new methods.
- [ ] **Step 3: Implement client methods**
Add one client method for each CLI-used `WorkflowApi` method listed in
"Required Method Coverage".
Wire methods to the JSON-RPC names from Tasks 1-2. Examples:
```python
async def list_artifacts(...):
return await self._call("workflow.artifacts.list", {...})
async def get_draft_workspace(...):
return await self._call("workflow.draft_workspaces.get", {...})
async def create_artifact_from_workspace(...):
return await self._call("workflow.draft_workspaces.create_artifact", {...})
```
Keep signatures close to `WorkflowApi` so basedpyright accepts CLI command
calls without casts.
- [ ] **Step 4: Run client and type checks**
Run:
```bash
uv run pytest tests/wf_transport_rpc_http/test_client.py -q
uv run basedpyright --level error src/wf_transport_rpc_http src/wf_cli
```
Expected: client tests pass and basedpyright reports 0 errors.
---
## Task 4: Route Draft/Artifact/Deploy CLI Through Target-Aware Context
**Files:**
- Modify: `src/wf_cli/commands/drafts.py`
- Modify: `src/wf_cli/commands/artifacts.py`
- Modify: `src/wf_cli/commands/deployments.py`
- Modify: `tests/wf_cli/test_remote_target.py`
- [ ] **Step 1: Write remote CLI lifecycle test**
Add a test that starts an in-process JSON-RPC app and invokes Typer commands
with:
```text
wf --config wf.json --url http://test/rpc draft create-from-capability ...
wf --config wf.json --url http://test/rpc draft validate ...
wf --config wf.json --url http://test/rpc draft save ...
wf --config wf.json --url http://test/rpc artifact inspect ...
wf --config wf.json --url http://test/rpc deploy save ...
wf --config wf.json --url http://test/rpc deploy validate ...
```
Use the same `httpx.AsyncClient` monkeypatch pattern already present in
`tests/wf_cli/test_remote_target.py`.
Assertions:
```python
assert created.exit_code == 0
assert validated.exit_code == 0
assert saved_artifact.exit_code == 0
assert inspected_artifact.exit_code == 0
assert saved_deployment.exit_code == 0
assert validated_deployment.exit_code == 0
```
Also assert key output fragments:
```python
assert '"workspace_id": "remote_ws"' in created.output
assert '"status": "valid"' in validated.output
assert '"artifact_id": "remote_artifact"' in saved_artifact.output
assert '"deployment_id": "remote_artifact.default"' in saved_deployment.output
assert '"status": "runnable"' in validated_deployment.output
```
- [ ] **Step 2: Run test and verify failure**
Run:
```bash
uv run pytest tests/wf_cli/test_remote_target.py::test_wf_remote_draft_artifact_deploy_lifecycle -q
```
Expected: fail because command modules still use local-only context.
- [ ] **Step 3: Switch command imports**
In `src/wf_cli/commands/drafts.py`, `artifacts.py`, and `deployments.py`,
replace:
```python
from wf_cli.context import load_local_cli_context_from_typer as load_cli_context
```
with:
```python
from wf_cli.context import load_cli_context_from_typer as load_cli_context
```
Keep the local `load_cli_context` alias name so existing tests that monkeypatch
`wf_cli.commands.<module>.load_cli_context` keep working.
- [ ] **Step 4: Run CLI tests**
Run:
```bash
uv run pytest tests/wf_cli/test_remote_target.py tests/wf_cli/test_run_deploy.py tests/wf_cli/test_discovery_lifecycle.py -q
uv run basedpyright --level error src/wf_cli src/wf_transport_rpc_http
```
Expected: tests pass and basedpyright reports 0 errors.
---
## Task 5: Documentation and Verification
**Files:**
- Modify: `docs/superpowers/specs/2026-06-03-workflow-config-targets-and-sources.md`
- Modify: `docs/current_roadmap.md`
- [ ] **Step 1: Update spec status**
Update the implementation status to say:
```markdown
- remote JSON-RPC client support now covers capability, draft workspace,
artifact, deployment, and run CLI commands
- draft/artifact/deploy commands no longer fail fast for `rpc_http` targets
```
Remove or update the older local-only/fail-fast status line.
- [ ] **Step 2: Update roadmap**
Update the roadmap note to say the basic remote CLI lifecycle is wired:
```markdown
selected `wf` commands can target JSON-RPC HTTP
```
should become:
```markdown
the basic `wf` lifecycle can target JSON-RPC HTTP: capability discovery,
draft workspace authoring, artifact/deployment operations, run, inspect, and
bounded trace.
```
- [ ] **Step 3: Run focused verification**
Run:
```bash
uv run pytest tests/wf_transport_rpc_http tests/wf_cli/test_remote_target.py tests/wf_cli/test_run_deploy.py tests/wf_cli/test_discovery_lifecycle.py -q
uv run ruff check src/wf_transport_rpc_http src/wf_cli tests/wf_transport_rpc_http tests/wf_cli
uv run ruff format --check src/wf_transport_rpc_http src/wf_cli tests/wf_transport_rpc_http tests/wf_cli
uv run basedpyright --level error src/wf_transport_rpc_http src/wf_cli tests/wf_transport_rpc_http tests/wf_cli
```
Expected: pass, except basedpyright may still exit non-zero for the known
workspace enumeration warning only when run at full workspace scope. For the
focused command above, expect 0 errors.
- [ ] **Step 4: Run full verification**
Run:
```bash
uv run pytest -q
uv run ruff check
uv run ruff format --check
uv run basedpyright --level error
```
Expected:
- pytest passes with current skip/xfail count
- ruff passes
- basedpyright reports `0 errors`; if it exits 1 only due to workspace
enumeration warning, report that exactly
---
## Self-Review Notes
This plan intentionally adds server RPC methods before client methods and client
methods before CLI routing. That sequence prevents the previous half-migration
problem where commands could receive a partial client.
Do not remove old local tests or monkeypatch seams. The command modules can keep
a local import alias named `load_cli_context` for compatibility with existing
tests, but the imported helper should become target-aware after Task 4.
@@ -0,0 +1,577 @@
# Source Admin API Surface Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add a protocol-neutral read-only source/admin surface in `wf_api` and make MCP admin source tools delegate through it.
**Architecture:** `WorkflowApiSurface` stays workflow-lifecycle-only. Source/admin operations get a sibling `WorkflowSourceAdminSurface` plus `WorkflowSourceAdminApi` implementation over `WorkflowOperationContext.specs.capability_sources`. The MCP admin handler becomes an adapter over this neutral API while connection/raw MCP methods remain MCP-broker-owned.
**Tech Stack:** Python 3.14, dataclasses, Protocols, Pydantic-backed platform models, pytest, ruff, basedpyright.
---
## Current Findings
- Source catalog internals already moved out of the old god service into `src/wf_mcp/broker/service/source_catalog.py`.
- `wf_api` does not currently have a source/admin API module.
- MCP admin source tools still flow through:
```text
wf_mcp.admin_surface.tools
-> BrokerAdminHandlers
-> WfMcpService.list_source_summaries / inspect_source
-> SourceCatalogService
```
- `WorkflowOperationContext.specs.capability_sources` already exposes the source inventory `wf_api` needs for read-only source listing and inspection.
## Scope
In scope:
- `list_sources(cursor=None, limit=50) -> dict`
- `inspect_source(source_id: str) -> dict`
- Protocol-neutral `WorkflowSourceAdminSurface`
- Local implementation `WorkflowSourceAdminApi`
- MCP `BrokerAdminHandlers.list_sources()` and `.inspect_source()` delegate through `WorkflowSourceAdminApi`
- Focused tests proving payload compatibility with existing MCP source output
Out of scope for this slice:
- Adding/removing/updating sources
- Store-backed source registry
- Connection status, catalog refresh, raw method invocation
- JSON-RPC transport methods for source admin
- CLI `wf source ...` commands
Those are follow-up slices once this neutral seam exists.
## File Structure
- Create `src/wf_api/source_admin.py`
- Owns `WorkflowSourceAdminApi`.
- Uses only `WorkflowOperationContext` and platform source models.
- Imports no `wf_mcp`.
- Modify `src/wf_api/surface.py`
- Adds sibling protocol `WorkflowSourceAdminSurface`.
- Does not make `WorkflowApiSurface` inherit it.
- Modify `src/wf_api/__init__.py`
- Exports `WorkflowSourceAdminApi` and `WorkflowSourceAdminSurface`.
- Modify `src/wf_mcp/admin_surface/handlers/broker.py`
- Construct `WorkflowSourceAdminApi(context_from_service(service))`.
- Delegate `list_sources` and `inspect_source` to it.
- Keep connection/catalog/resource/raw methods unchanged.
- Make those two source methods async, matching the async MCP tool boundary.
- Modify `src/wf_mcp/admin_surface/tools.py`
- Await async source handler methods.
- Create `tests/wf_api/test_source_admin_api.py`
- Direct neutral API tests.
- Modify `tests/wf_mcp/test_admin_surface.py`
- Add adapter smoke coverage for the neutral source admin delegation.
- Modify `docs/current_roadmap.md`
- Mark the read-only neutral source/admin seam as completed.
---
### Task 1: Add failing `wf_api` source admin tests
**Files:**
- Create: `tests/wf_api/test_source_admin_api.py`
- [ ] **Step 1: Write direct tests**
Create `tests/wf_api/test_source_admin_api.py`:
```python
from __future__ import annotations
from typing import Any
import pytest
from wf_api import WorkflowSourceAdminApi
from wf_api.operation_context import WorkflowOperationContext
from wf_authoring import NodeSpec
from wf_platform import (
CapabilityBuckets,
CapabilitySource,
SourcePermissions,
SourceVisibility,
)
class DummyEvents:
def record_event(self, event: object) -> None:
pass
def record_workflow_event(
self,
event_type: str,
*,
capability_id: str,
payload: dict[str, Any],
) -> None:
pass
class DummyRuntime:
async def run_workflow_from_plan(self, *args: Any, **kwargs: Any) -> object:
raise AssertionError("source admin tests must not run workflows")
async def resume_workflow_from_plan(
self,
*args: Any,
**kwargs: Any,
) -> object:
raise AssertionError("source admin tests must not resume workflows")
class StaticSpecProvider:
def __init__(self, sources: dict[str, CapabilitySource]) -> None:
self._sources = sources
@property
def capability_sources(self) -> dict[str, CapabilitySource]:
return self._sources
def get_qualified_spec(self, qualified_name: str) -> NodeSpec[Any, Any]:
raise KeyError(f"unknown capability {qualified_name!r}")
def _api(*sources: CapabilitySource) -> WorkflowSourceAdminApi:
provider = StaticSpecProvider({source.id: source for source in sources})
return WorkflowSourceAdminApi(
WorkflowOperationContext(
artifact_store=None,
draft_workspace_store=None,
run_store=None,
events=DummyEvents(),
specs=provider,
runtime=DummyRuntime(),
live_sources=None,
)
)
def _source(source_id: str, *, enabled: bool = True) -> CapabilitySource:
return CapabilitySource(
id=source_id,
kind="connection",
enabled=enabled,
capabilities=CapabilityBuckets(),
visibility=SourceVisibility(
planner=True,
mcp_client=True,
admin_dashboard=True,
),
permissions=SourcePermissions(calls_upstream=True),
description=f"{source_id} source",
)
@pytest.mark.asyncio
async def test_source_admin_lists_compact_sources_in_id_order() -> None:
api = _api(_source("zeta.personal"), _source("alpha.personal", enabled=False))
payload = await api.list_sources(limit=10)
assert payload["total"] == 2
assert payload["next_cursor"] is None
assert [source["id"] for source in payload["sources"]] == [
"alpha.personal",
"zeta.personal",
]
assert payload["sources"][0]["enabled"] is False
assert payload["sources"][1]["description"] == "zeta.personal source"
@pytest.mark.asyncio
async def test_source_admin_pages_sources() -> None:
api = _api(_source("a"), _source("b"), _source("c"))
first = await api.list_sources(limit=2)
second = await api.list_sources(cursor=first["next_cursor"], limit=2)
assert [source["id"] for source in first["sources"]] == ["a", "b"]
assert first["next_cursor"] == "2"
assert [source["id"] for source in second["sources"]] == ["c"]
assert second["next_cursor"] is None
@pytest.mark.asyncio
async def test_source_admin_inspects_full_source_inventory() -> None:
api = _api(_source("demo.personal"))
payload = await api.inspect_source(source_id="demo.personal")
assert payload["id"] == "demo.personal"
assert payload["kind"] == "connection"
assert payload["description"] == "demo.personal source"
assert payload["visibility"]["planner"] is True
assert payload["permissions"]["calls_upstream"] is True
@pytest.mark.asyncio
async def test_source_admin_inspect_unknown_source_raises_clear_key_error() -> None:
api = _api(_source("demo.personal"))
with pytest.raises(KeyError, match="unknown source 'missing.source'"):
await api.inspect_source(source_id="missing.source")
```
- [ ] **Step 2: Run tests and verify failure**
Run:
```bash
uv run pytest tests/wf_api/test_source_admin_api.py -q
```
Expected: FAIL because `WorkflowSourceAdminApi` is not exported yet.
---
### Task 2: Implement `WorkflowSourceAdminApi`
**Files:**
- Create: `src/wf_api/source_admin.py`
- Modify: `src/wf_api/__init__.py`
- [ ] **Step 1: Add source admin API**
Create `src/wf_api/source_admin.py`:
```python
from __future__ import annotations
from typing import Any
from wf_platform import page_items
from .operation_context import WorkflowOperationContext
class WorkflowSourceAdminApi:
"""Read-only protocol-neutral source inventory operations.
This is a sibling to WorkflowApi, not part of WorkflowApiSurface, because
source administration is server/platform management rather than workflow
lifecycle execution.
"""
def __init__(self, context: WorkflowOperationContext) -> None:
self.context = context
async def list_sources(
self,
*,
cursor: str | None = None,
limit: int = 50,
) -> dict[str, Any]:
summaries = [
source.as_status().model_dump(mode="json")
for source in sorted(
self.context.specs.capability_sources.values(),
key=lambda source: source.id,
)
]
page = page_items(summaries, cursor=cursor, limit=limit)
return {
"sources": list(page.items),
"next_cursor": page.next_cursor,
"total": page.total,
}
async def inspect_source(self, *, source_id: str) -> dict[str, Any]:
try:
source = self.context.specs.capability_sources[source_id]
except KeyError as exc:
raise KeyError(f"unknown source {source_id!r}") from exc
return source.as_inventory().model_dump(mode="json")
```
- [ ] **Step 2: Export from `wf_api`**
Modify `src/wf_api/__init__.py`:
```python
from .source_admin import WorkflowSourceAdminApi
```
Add `"WorkflowSourceAdminApi"` to `__all__`.
- [ ] **Step 3: Run direct tests**
Run:
```bash
uv run pytest tests/wf_api/test_source_admin_api.py -q
```
Expected: PASS.
---
### Task 3: Add sibling surface protocol
**Files:**
- Modify: `src/wf_api/surface.py`
- Modify: `src/wf_api/__init__.py`
- Test: `tests/wf_api/test_source_admin_api.py`
- [ ] **Step 1: Add protocol conformance test**
Append to `tests/wf_api/test_source_admin_api.py`:
```python
from wf_api import WorkflowSourceAdminSurface
def test_source_admin_api_satisfies_surface_protocol() -> None:
api: WorkflowSourceAdminSurface = _api(_source("demo.personal"))
assert api is not None
```
- [ ] **Step 2: Add protocol**
In `src/wf_api/surface.py`, add this class near the other surface protocols:
```python
class WorkflowSourceAdminSurface(Protocol):
"""Read-only source/admin methods exposed by platform frontends."""
async def list_sources(
self,
*,
cursor: str | None = None,
limit: int = 50,
) -> dict[str, Any]: ...
async def inspect_source(
self,
*,
source_id: str,
) -> dict[str, Any]: ...
```
Add `"WorkflowSourceAdminSurface"` to `__all__`.
Do not add it as a base class of `WorkflowApiSurface`.
- [ ] **Step 3: Export protocol**
Modify `src/wf_api/__init__.py`:
```python
from .surface import WorkflowSourceAdminSurface
```
Add `"WorkflowSourceAdminSurface"` to `__all__`.
- [ ] **Step 4: Run tests and type check**
Run:
```bash
uv run pytest tests/wf_api/test_source_admin_api.py tests/wf_api/test_import_direction.py -q
uv run basedpyright --level error src/wf_api tests/wf_api/test_source_admin_api.py
```
Expected: tests PASS, basedpyright reports 0 errors.
---
### Task 4: Delegate MCP admin source tools through `wf_api`
**Files:**
- Modify: `src/wf_mcp/admin_surface/handlers/broker.py`
- Modify: `src/wf_mcp/admin_surface/tools.py`
- Test: `tests/wf_mcp/test_admin_surface.py`
- [ ] **Step 1: Add adapter smoke assertion**
In `tests/wf_mcp/test_admin_surface.py`, inside
`test_broker_admin_handlers_list_connections_and_events`, add:
```python
sources = _run(handlers.list_sources(limit=100))
source_ids = {source["id"] for source in sources["sources"]}
assert "wf.std" in source_ids
assert "wf.docs" in source_ids
assert sources["total"] >= 2
```
This test uses the existing `_run()` helper.
- [ ] **Step 2: Run the updated test and verify failure**
Run:
```bash
uv run pytest tests/wf_mcp/test_admin_surface.py::test_broker_admin_handlers_list_connections_and_events -q
```
Expected before delegation: FAIL because `BrokerAdminHandlers.list_sources()`
is still sync and returns a dict, not an awaitable. This failure drives the async
source-handler cleanup.
- [ ] **Step 3: Change handler implementation**
Modify `src/wf_mcp/admin_surface/handlers/broker.py`:
```python
from wf_api import WorkflowSourceAdminApi, WorkflowSourceAdminSurface
from wf_mcp.broker.service.workflow_operation_context import context_from_service
```
Update `__init__`:
```python
def __init__(self, service: WfMcpService) -> None:
self.service = service
self.sources: WorkflowSourceAdminSurface = WorkflowSourceAdminApi(
context_from_service(service)
)
```
Update source methods:
```python
async def list_sources(
self,
*,
cursor: str | None = None,
limit: int = 50,
) -> dict[str, Any]:
return await self.sources.list_sources(cursor=cursor, limit=limit)
async def inspect_source(self, source_id: str) -> dict[str, Any]:
return await self.sources.inspect_source(source_id=source_id)
```
Do not add an `asyncio.run()` bridge. The handler is called from async MCP tools,
so source methods should be async at this boundary.
- [ ] **Step 4: Await source handler calls in MCP tools**
In `src/wf_mcp/admin_surface/tools.py`, update:
```python
return await handlers.list_sources(cursor=cursor, limit=limit)
```
and:
```python
return await handlers.inspect_source(source_id)
```
- [ ] **Step 5: Run MCP admin tests**
Run:
```bash
uv run pytest tests/wf_mcp/test_admin_surface.py tests/wf_mcp/server/test_config.py tests/wf_mcp/test_broker_server.py -q
```
Expected: PASS.
---
### Task 5: Documentation and verification
**Files:**
- Modify: `docs/current_roadmap.md`
- Modify: `docs/superpowers/specs/2026-06-03-cli-api-alignment-notes.md`
- [ ] **Step 1: Update roadmap**
In `docs/current_roadmap.md`, under **Durable API service shape** or
**CLI/API alignment**, add:
```markdown
- Completed: read-only source inventory now has a protocol-neutral
`WorkflowSourceAdminApi` / `WorkflowSourceAdminSurface`; MCP admin source
tools delegate through it while connection/raw MCP operations remain
broker-owned.
```
- [ ] **Step 2: Update CLI/API notes**
In `docs/superpowers/specs/2026-06-03-cli-api-alignment-notes.md`, under
**Next Slices**, replace the source/admin item with:
```markdown
1. **Source/admin transport and CLI commands**
- Build JSON-RPC methods and `wf source ...` commands over
`WorkflowSourceAdminSurface`.
- Keep mutation out until the store-backed source registry is designed.
```
- [ ] **Step 3: Run verification**
Run:
```bash
uv run pytest tests/wf_api/test_source_admin_api.py tests/wf_mcp/test_admin_surface.py tests/wf_mcp/server/test_config.py tests/wf_mcp/test_broker_server.py -q
uv run ruff check src/wf_api src/wf_mcp/admin_surface tests/wf_api/test_source_admin_api.py tests/wf_mcp/test_admin_surface.py tests/wf_mcp/server/test_config.py tests/wf_mcp/test_broker_server.py
uv run ruff format --check src/wf_api src/wf_mcp/admin_surface tests/wf_api/test_source_admin_api.py tests/wf_mcp/test_admin_surface.py tests/wf_mcp/server/test_config.py tests/wf_mcp/test_broker_server.py
uv run basedpyright --level error src/wf_api src/wf_mcp/admin_surface tests/wf_api/test_source_admin_api.py tests/wf_mcp/test_admin_surface.py
```
Expected:
- pytest PASS
- ruff check PASS
- ruff format PASS
- basedpyright 0 errors
- [ ] **Step 4: Commit**
```bash
git add src/wf_api/source_admin.py src/wf_api/surface.py src/wf_api/__init__.py src/wf_mcp/admin_surface/handlers/broker.py src/wf_mcp/admin_surface/tools.py tests/wf_api/test_source_admin_api.py tests/wf_mcp/test_admin_surface.py docs/current_roadmap.md docs/superpowers/specs/2026-06-03-cli-api-alignment-notes.md
git commit -m "feat: add source admin api surface"
```
---
## Follow-Up Slices
1. **Source/admin JSON-RPC transport**
- Add fixed methods such as `workflow.sources.list` and
`workflow.sources.inspect`.
- Add a dedicated RPC source-admin client or mixin, but keep the lifecycle
`RpcWorkflowApiClient` contract clear.
2. **CLI `wf source` commands**
- Add `wf source list` and `wf source inspect`.
- Use the same target-aware context pattern as workflow lifecycle commands,
but the CLI context may need a second handler field for source admin.
3. **Store-backed source registry**
- Config remains bootstrap.
- Server-owned dynamic source changes persist through a source registry store.
- Source identity stays structural: source id, provider/account/profile, and
concrete transport details are not inferred from dotted display names.
4. **Mutable source admin**
- Add source create/update/delete only after store persistence and validation
rules exist.
- Enforce duplicate id behavior and liveness/validation diagnostics before
a source can become runnable.
## Self-Review
- Spec coverage: read-only source list/inspect, neutral surface, MCP adapter, and docs are covered.
- Placeholder scan: no TBD/TODO placeholders.
- Type consistency: `WorkflowSourceAdminApi`, `WorkflowSourceAdminSurface`, `source_id`, `cursor`, and `limit` names are consistent across tasks.
- Risk: `BrokerAdminHandlers.list_sources` / `inspect_source` become async. The plan updates the direct handler test and the MCP tool wrappers that call them.
@@ -0,0 +1,421 @@
# Source Registry Next Slices Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move the source registry toward a generic platform/API boundary, then wire startup merge and finally mutation without locking MCP-specific assumptions into `wf_api`.
**Architecture:** Split generic registry mechanics from MCP-specific source entries. `wf_api` should own the generic registry file/store concepts once they no longer import MCP validators. `wf_mcp` should own MCP source entry models and conversion into `ConnectionConfig` / broker services. Startup merge happens after the split so config-vs-store precedence is implemented against the right abstractions.
**Tech Stack:** Python 3.14, Pydantic v2, existing `wf_mcp.source_registry`, `wf_api`, `wf_config`, `WfMcpService`, pytest, ruff, basedpyright.
---
## Current State
Slice 1 created `src/wf_mcp/source_registry.py` with:
- `SourceRegistryModel`
- `StdioSourceTransport`
- `HttpSourceTransport`
- `McpSourceRegistryEntry`
- `SourceRegistryFile`
- `SourceRegistryStore`
- `FileSourceRegistryStore`
This is useful and tested, but still MCP-shaped:
- id validation uses `wf_mcp.connections.parse_connection_id`
- reserved ids come from `wf_mcp.shared.names`
- source entry type is `McpSourceRegistryEntry`
- transport definitions are MCP transports
Slice 2A then moved generic registry mechanics to `wf_api.source_registry`,
while `wf_mcp.source_registry` kept MCP-specific entries and transports. Slice
2B added `registry_entry_to_connection_config()`.
The next executable slice is startup merge:
`docs/superpowers/plans/2026-06-03-source-registry-startup-merge.md`.
## Slice Order
1. **Slice 2A: Generic Registry Mechanics**
- **Status: complete.**
- Move generic validation/store/file mechanics to `wf_api`.
- Keep MCP entry/transport validation in `wf_mcp`.
- Prefer boring helpers over deep Pydantic generics where that keeps the
boundary easier to type-check.
- Do not change runtime behavior.
2. **Slice 2B: MCP Entry Conversion**
- **Status: complete.**
- Add conversion helpers between `McpSourceRegistryEntry` and
`ConnectionConfig`.
- Keep config merge out of scope.
3. **Slice 3: Startup Merge**
- **Status: complete.**
- Load registry at broker/server startup.
- Merge config and registry with config precedence.
- Emit events/diagnostics for shadowed registry entries.
4. **Slice 4: Read Desired Registry Through Admin**
- **Status: complete.**
- Expose desired registry entries separately from observed source inventory.
- `WorkflowSourceRegistryApi` provides neutral read-only access.
- JSON-RPC methods `workflow.admin.source_registry.list` / `.inspect`.
- CLI commands `wf admin registry list` / `wf admin registry inspect`.
- Local/static servers report unavailable instead of empty.
- Concrete MCP-backed `WorkflowServer` construction remains future work.
5. **Slice 5: Mutation Commands**
- **Status: complete.**
- Add add/update/enable/disable/remove operations.
- Use registry store, validation, and atomic writes.
- Keep auth/catalog cleanup deferred.
- JSON-RPC/CLI calls work for targets that expose the registry-admin surface;
local/static servers report unavailable and concrete MCP-backed
`WorkflowServer` construction remains future work.
6. **Slice 6: Config Ownership Policy**
- **Status: complete.**
- Replace implicit config-shadowing with explicit `locked` / `seed`
ownership policy.
- `locked` config entries remain operator-owned and shadow/reject registry
mutation for the same id.
- `seed` config entries bootstrap missing store entries, then the store owns
later admin changes.
- Update startup merge diagnostics and registry admin payloads so users can
see why a source is mutable or shadowed.
---
## Slice 2A: Generic Registry Mechanics
### Goal
Move generic registry store mechanics out of `wf_mcp` without pretending MCP
source entries are generic.
### Target Shape
Create `src/wf_api/source_registry.py` with protocol-neutral mechanics only:
```python
from __future__ import annotations
import json
import re
from collections.abc import Callable
from pathlib import Path
from typing import Generic, Protocol, TypeVar
from uuid import uuid4
from pydantic import BaseModel, ConfigDict
class SourceRegistryBaseModel(BaseModel):
"""Base model for persisted source registry state; reject misspelled fields."""
model_config = ConfigDict(extra="forbid")
SOURCE_REGISTRY_ID_PATTERN = r"^[A-Za-z0-9_][A-Za-z0-9_.-]*$"
def validate_source_registry_id(value: str) -> str:
"""Validate ids that are safe as registry keys and filesystem path segments.
This helper intentionally does not parse provider/account meaning. MCP can
layer stricter `parse_connection_id` validation on top while other future
source families can reuse the safe-id rule.
"""
if not re.fullmatch(SOURCE_REGISTRY_ID_PATTERN, value):
raise ValueError(
"source id must start with alphanumeric or underscore and contain "
"only [A-Za-z0-9_.-]"
)
return value
def validate_unique_source_ids(entries: list[object]) -> None:
"""Reject duplicate `id` fields without owning the entry model shape."""
seen: set[str] = set()
for entry in entries:
source_id = getattr(entry, "id", None)
if not isinstance(source_id, str):
raise ValueError("source registry entries must expose string id")
if source_id in seen:
raise ValueError(f"duplicate source id {source_id!r}")
seen.add(source_id)
RegistryT = TypeVar("RegistryT", bound=BaseModel)
class SourceRegistryStore(Protocol[RegistryT]):
def load_registry(self) -> RegistryT: ...
def save_registry(self, registry: RegistryT) -> None: ...
class AtomicJsonRegistryStore(Generic[RegistryT]):
"""Filesystem implementation for small desired-registry documents."""
def __init__(
self,
root: Path,
*,
filename: str,
registry_type: type[RegistryT],
empty_factory: Callable[[], RegistryT],
corrupt_label: str,
) -> None:
self.root = root
self.filename = filename
self.registry_type = registry_type
self.empty_factory = empty_factory
self.corrupt_label = corrupt_label
self.root.mkdir(parents=True, exist_ok=True)
@property
def path(self) -> Path:
return self.root / self.filename
def load_registry(self) -> RegistryT:
if not self.path.exists():
return self.empty_factory()
try:
data = json.loads(self.path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc:
raise ValueError(f"{self.corrupt_label} is corrupted: {self.path}") from exc
return self.registry_type.model_validate(data)
def save_registry(self, registry: RegistryT) -> None:
validated = self.registry_type.model_validate(registry.model_dump(mode="json"))
payload = json.dumps(validated.model_dump(mode="json"), indent=2)
tmp_path = self.path.with_name(f"{self.path.name}.{uuid4().hex}.tmp")
tmp_path.write_text(payload, encoding="utf-8")
tmp_path.replace(self.path)
```
Then `src/wf_mcp/source_registry.py` should import these generic pieces and keep:
- `StdioSourceTransport`
- `HttpSourceTransport`
- `SourceTransport`
- `McpSourceRegistryEntry`
- `SourceRegistryFile` or `McpSourceRegistryFile` as the MCP registry document
model, using `validate_unique_source_ids(self.sources)`
- `FileSourceRegistryStore` as the MCP concrete store wrapper around
`AtomicJsonRegistryStore[SourceRegistryFile]`
### Tests
Move generic tests to `tests/wf_api/test_source_registry.py`:
- safe id validation accepts normal registry ids and rejects unsafe ids
- duplicate id helper rejects repeated ids using a fake entry object
- missing file returns empty generic registry
- save/load round trip
- corrupted JSON wraps as `ValueError`
Keep MCP tests in `tests/wf_mcp/test_source_registry.py`:
- MCP id validation
- reserved id rejection
- stdio/http transport models
- `McpFileSourceRegistryStore` round trip
### Acceptance Criteria
- `wf_api.source_registry` imports no `wf_mcp`.
- Existing `wf_mcp.source_registry` public behavior remains compatible.
- Tests pass.
- No startup/runtime behavior changes.
---
## Slice 2B: MCP Entry Conversion
### Goal
Add explicit conversion helpers so startup merge can convert registry entries
into broker connection configs without duplicating field logic.
Status: complete. The implemented helper preserves entry metadata, `auth_ref`,
profile, transport details, and a `source_registry` marker.
### New Helpers
In `src/wf_mcp/source_registry.py`:
```python
def registry_entry_to_connection_config(
entry: McpSourceRegistryEntry,
) -> ConnectionConfig:
return ConnectionConfig(
id=entry.id,
server=entry.provider,
account=entry.account,
enabled=entry.enabled,
metadata={
**entry.metadata,
"auth_ref": entry.auth_ref,
"profile": entry.profile,
"transport": entry.transport.model_dump(mode="json"),
"source_registry": True,
},
)
```
And optionally:
```python
def connection_config_to_registry_entry(
connection: ConnectionConfig,
) -> McpSourceRegistryEntry | None:
...
```
Only add reverse conversion if an implementation needs it. Do not guess unknown
transport metadata.
### Tests
- registry entry converts to `ConnectionConfig`
- provider/account/profile/transport metadata are preserved
- disabled entry creates disabled connection config
### Acceptance Criteria
- Conversion is explicit and tested.
- No startup/runtime behavior changes yet.
---
## Slice 3: Startup Merge
### Goal
Load desired dynamic registry state during service construction and merge it
with config-defined connections/sources.
### Merge Rules
1. Built-in reserved ids always win.
2. Config-defined entries win over registry entries with the same id.
3. Registry entries fill ids not present in config.
### Implementation Direction
- Broker config construction should create a `McpFileSourceRegistryStore`.
- `WfMcpService` or `ConnectionService` should accept an optional registry store.
- On startup/reload:
- load config connections
- load registry entries
- convert registry entries to `ConnectionConfig`
- merge with config precedence
- register merged connections
- emit an event for registry entries shadowed by config
### Tests
- absent registry preserves current config-only behavior
- registry-only connection appears after service construction
- config shadows same-id registry entry
- invalid registry fails startup clearly
- disabled registry source hydrates disabled
### Acceptance Criteria
- Existing legacy config behavior is unchanged when no registry file exists.
- Dynamic registry entries persist across service recreation.
- Shadowed entries do not silently override config.
---
## Slice 4: Read Desired Registry Through Admin
### Goal
Expose desired registry entries distinctly from observed source inventory.
Status: complete for API/transport/CLI plumbing. `WorkflowSourceRegistryApi`
provides neutral read-only access. JSON-RPC methods
`workflow.admin.source_registry.list` / `.inspect` are registered. CLI commands
`wf admin registry list` / `wf admin registry inspect` are available for targets
that expose the surface. Local/static servers report
`source_registry_unavailable`. Concrete MCP-backed `WorkflowServer` construction
remains future work.
### Why
`wf source list` currently reports runtime/observed source inventory. Registry
entries are desired server-owned configuration. Users need both views when a
source exists in registry but is disabled, shadowed, or not hydrated.
### Candidate Commands
- `wf admin registry list`
- `wf admin registry inspect SOURCE_ID`
or:
- `wf source registry list`
- `wf source registry inspect SOURCE_ID`
Pick one naming shape in the implementation plan.
### Acceptance Criteria
- Desired registry view is not confused with observed source inventory.
- Shadowed/disabled state is visible.
- No mutation yet.
---
## Slice 5: Mutation Commands
### Goal
Add safe registry mutation.
Status: complete. Implementation:
[2026-06-04 source registry mutations](../plans/2026-06-04-source-registry-mutations.md).
### Operations
- add source
- update source
- enable source
- disable source
- remove source
### Rules
- Validate full registry before commit.
- Write atomically.
- Do not mutate config files.
- Do not delete auth/catalog files in v1.
- Prefer disable over remove for sources referenced by deployments.
- Optional live validation can be a flag, not required for save.
### Acceptance Criteria
- RPC methods exist.
- CLI commands exist.
- Mutations persist across process restart for targets backed by a registry
store; local/static servers report unavailable.
- Validation errors are actionable.
---
## Self-Review
- This is a multi-slice roadmap, not a single execution plan for all mutation work.
- Slice 2A is the immediate next implementation target and resolves the location problem.
- Startup merge is intentionally after generic/MCP split and conversion helpers.
- Config ownership policy is intentionally after mutation commands, because it
changes precedence semantics rather than introducing persistence.
@@ -0,0 +1,351 @@
# Source Registry Startup Merge Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Load persisted dynamic MCP source registry entries during broker/server construction and merge them with config-defined connections without changing behavior when the registry file is absent.
**Architecture:** Keep parsing/loading config separate from runtime hydration. `wf_mcp.source_registry` owns MCP registry entries and conversion to `ConnectionConfig`; `ConnectionService` owns connection reconciliation and source-catalog hydration; `build_service_from_config()` wires the default file store from `BrokerConfig.store_root`.
**Tech Stack:** Python 3.14, dataclasses, Pydantic v2, `wf_mcp.source_registry`, `ConnectionService`, `WfMcpService`, `build_service_from_config`, pytest, ruff, basedpyright.
---
## Preconditions
Completed before this plan:
- Generic registry mechanics exist in `wf_api.source_registry`.
- MCP-specific registry models remain in `wf_mcp.source_registry`.
- `registry_entry_to_connection_config()` exists and is tested.
- No startup/runtime merge uses the registry yet.
## Merge Rules
1. Config-defined connections win over registry entries with the same id.
2. Registry entries fill ids not present in config.
3. Reserved ids remain rejected by existing `ConnectionService` validation.
4. Shadowed registry entries must be visible through an event/diagnostic, not silently ignored.
5. Missing `source_registry.json` must preserve current config-only behavior.
Do not mutate config files. Do not delete auth/catalog files.
---
## Task 1: Add Merge Helper Tests First
- [ ] Add focused tests in `tests/wf_mcp/service/test_connection_service.py`.
- [ ] Import:
```python
from wf_mcp.source_registry import (
FileSourceRegistryStore,
McpSourceRegistryEntry,
SourceRegistryFile,
StdioSourceTransport,
)
```
- [ ] Add helper:
```python
def _registry_entry(
source_id: str = "demo.registry",
*,
enabled: bool = True,
) -> McpSourceRegistryEntry:
return McpSourceRegistryEntry(
id=source_id,
kind="mcp",
enabled=enabled,
provider="demo",
account=source_id.rsplit(".", 1)[-1],
transport=StdioSourceTransport(command="demo-server"),
)
```
- [ ] Add `test_connection_service_sync_merges_registry_entries`.
Expected behavior:
```python
store = FileSourceRegistryStore(tmp_path)
store.save_registry(SourceRegistryFile(sources=[_registry_entry()]))
service.sync_connections_from_config(
BrokerConfig(store_root=tmp_path, connections=[]),
source_registry_store=store,
)
assert [connection.id for connection in service.list_all()] == ["demo.registry"]
assert "demo.registry" in catalog.capability_sources
```
- [ ] Add `test_connection_service_sync_config_shadows_registry_entry`.
Expected behavior:
```python
store.save_registry(SourceRegistryFile(sources=[_registry_entry("demo.same")]))
service.sync_connections_from_config(
BrokerConfig(
store_root=tmp_path,
connections=[
ConnectionConfig(id="demo.same", server="demo", account="config"),
],
),
source_registry_store=store,
)
assert service.get("demo.same").account == "config"
assert service.events.list_events()[-1].kind == "source_registry_ignored_config_shadow"
assert service.events.list_events()[-1].connection_id == "demo.same"
```
- [ ] Add `test_connection_service_sync_registry_disabled_entry_hydrates_disabled_source`.
Expected behavior:
```python
store.save_registry(SourceRegistryFile(sources=[_registry_entry(enabled=False)]))
service.sync_connections_from_config(
BrokerConfig(store_root=tmp_path, connections=[]),
source_registry_store=store,
)
assert service.get("demo.registry").enabled is False
assert catalog.capability_sources["demo.registry"].enabled is False
```
- [ ] Run:
```bash
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
```
Expected: new tests fail until Task 2.
---
## Task 2: Implement Registry-Aware Connection Reconciliation
- [ ] Update `src/wf_mcp/broker/service/connection_service.py`.
- [ ] Import:
```python
from ...source_registry import (
SourceRegistryStore,
registry_entry_to_connection_config,
)
```
- [ ] Change `sync_connections_from_config` signature:
```python
def sync_connections_from_config(
self,
config: BrokerConfig,
*,
source_registry_store: SourceRegistryStore | None = None,
) -> None:
```
- [ ] Build a merged connection list before existing remove/update/register logic:
```python
connections = list(config.connections)
config_ids = {connection.id for connection in connections}
if source_registry_store is not None:
registry = source_registry_store.load_registry()
for entry in registry.sources:
if entry.id in config_ids:
self.events.record_kind(
"source_registry_ignored_config_shadow",
connection_id=entry.id,
payload={
"server": entry.provider,
"account": entry.account,
"reason": "config_connection_takes_precedence",
},
)
continue
connections.append(registry_entry_to_connection_config(entry))
```
- [ ] Reuse the existing reconciliation logic against the merged `connections` list.
- [ ] Keep `BrokerConfig` immutable; do not mutate `config.connections`.
- [ ] Keep existing event behavior for registered/updated/removed connections.
- [ ] Add a short comment near the merge explaining config precedence.
- [ ] Run:
```bash
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
```
Expected: all connection service tests pass.
---
## Task 3: Wire WfMcpService Facade
- [ ] Update `src/wf_mcp/broker/service/core.py`.
- [ ] Import `SourceRegistryStore` from `wf_mcp.source_registry`.
- [ ] Change facade method signature:
```python
def sync_connections_from_config(
self,
config: BrokerConfig,
*,
source_registry_store: SourceRegistryStore | None = None,
) -> None:
self.connection_service.sync_connections_from_config(
config,
source_registry_store=source_registry_store,
)
```
- [ ] Add/adjust a facade test in `tests/wf_mcp/service/test_connection_service.py` proving `WfMcpService.sync_connections_from_config(..., source_registry_store=store)` delegates and hydrates a registry connection.
---
## Task 4: Wire Build From Config
- [ ] Update `src/wf_mcp/broker/config.py`.
- [ ] Import `FileSourceRegistryStore`.
- [ ] Construct the store in `build_service_from_config(config)`:
```python
source_registry_store = FileSourceRegistryStore(config.store_root)
```
- [ ] Replace the manual connection loop with one registry-aware sync call:
```python
service.sync_connections_from_config(
config,
source_registry_store=source_registry_store,
)
for connection in service.connections.list_all():
if connection.server not in service.adapters:
service.register_adapter(connection.server, McpSdkAdapter())
```
Important:
- Preserve adapter registration for config-defined and registry-defined connections.
- Do not load the registry in `load_broker_config`; it should only parse config files.
- Do not save the registry during startup.
---
## Task 5: Add Build-Service Integration Tests
- [ ] Update `tests/wf_mcp/test_broker_server.py` or `tests/wf_mcp/server/test_config.py`.
- [ ] Add `test_build_service_from_config_loads_source_registry_entries`.
Setup:
```python
config = BrokerConfig(store_root=tmp_path, connections=[])
FileSourceRegistryStore(tmp_path).save_registry(
SourceRegistryFile(sources=[_registry_entry("fixture.registry")])
)
service = build_service_from_config(config)
```
Assertions:
```python
assert service.connections.get("fixture.registry").server == "fixture"
assert "fixture" in service.adapters
assert "fixture.registry" in service.capability_sources
```
- [ ] Add `test_build_service_from_config_config_shadows_registry`.
Assertions:
```python
assert service.connections.get("fixture.same").account == "config"
assert any(
event.kind == "source_registry_ignored_config_shadow"
and event.connection_id == "fixture.same"
for event in service.list_events()
)
```
- [ ] Add `test_build_service_from_config_absent_registry_preserves_existing_behavior` if no existing test already covers this. It can extend the current `test_build_service_from_config_registers_connections`.
---
## Task 6: Update Docs
- [ ] Update `docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md`.
Mark startup merge as complete or in-progress, depending on final status:
```md
### Slice 3: Startup Merge
Status: complete. Broker/service construction now loads `source_registry.json`,
merges config-defined connections with dynamic registry entries, preserves config
precedence, and emits `source_registry_ignored_config_shadow` for shadowed
registry entries.
```
- [ ] Update `docs/current_roadmap.md`.
Replace the source-registry note with:
```md
- Source registry startup merge is implemented: absent registry preserves
config-only behavior, registry-only entries hydrate as dynamic connections,
and config entries shadow same-id registry entries with an event.
```
- [ ] Update `docs/superpowers/plans/2026-06-03-source-registry-next-slices.md`.
Mark Slice 2A and Slice 2B complete if not already done, and mark Slice 3 complete after implementation.
---
## Task 7: Verify
- [ ] Run focused tests:
```bash
uv run pytest tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/test_broker_server.py tests/wf_mcp/server/test_config.py -q
```
- [ ] Run source registry tests:
```bash
uv run pytest tests/wf_api/test_source_registry.py tests/wf_mcp/test_source_registry.py -q
```
- [ ] Run quality checks:
```bash
uv run ruff check src/wf_mcp/source_registry.py src/wf_mcp/broker/config.py src/wf_mcp/broker/service/connection_service.py src/wf_mcp/broker/service/core.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/test_broker_server.py tests/wf_mcp/server/test_config.py
uv run basedpyright --level error src/wf_mcp/source_registry.py src/wf_mcp/broker/config.py src/wf_mcp/broker/service/connection_service.py src/wf_mcp/broker/service/core.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/test_broker_server.py tests/wf_mcp/server/test_config.py
```
- [ ] If focused tests pass, run the full suite if time permits:
```bash
uv run pytest -q
```
---
## Acceptance Criteria
- Missing registry file keeps existing config-only behavior.
- Registry-only entries become registered connections and hydrated source catalog entries.
- Config entries shadow same-id registry entries.
- Shadowing emits `source_registry_ignored_config_shadow`.
- Disabled registry entries hydrate disabled connection/source state.
- `load_broker_config()` remains config-only parsing.
- No registry mutation commands are added in this slice.
- `wf_api` still imports no `wf_mcp`.
@@ -0,0 +1,433 @@
# Source Registry Store Slice 1 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add validated source registry models and a filesystem store for desired server-owned source configuration, without wiring it into startup or mutation commands yet.
**Architecture:** The new registry is desired configuration state and stays separate from existing auth/catalog storage. Models live in `wf_mcp` for this first slice because current connection/source registry semantics are still MCP-provider-specific; the file-store interface is small enough to move later. Runtime merge, RPC/CLI mutation, and config reconciliation are explicitly deferred.
**Tech Stack:** Python 3.14, Pydantic v2, existing `wf_mcp.connections.parse_connection_id`, existing `RESERVED_CONNECTION_IDS`, pytest, ruff, basedpyright.
---
## Scope
In scope:
- Typed `SourceRegistryFile` model.
- Typed `McpSourceRegistryEntry` model.
- Typed `StdioSourceTransport` and `HttpSourceTransport` models.
- Duplicate id validation.
- Reserved id validation.
- ID validation using existing connection id rules.
- `SourceRegistryStore` protocol.
- `FileSourceRegistryStore` using `<store_root>/source_registry.json`.
- Atomic filesystem writes.
- Tests for load missing file, save/load round trip, validation errors, and path.
Out of scope:
- Startup merge with config.
- Runtime hydration from registry.
- Mutating API/CLI/RPC commands.
- Auth record changes.
- Catalog deletion or cleanup.
- SQL/remote stores.
## File Structure
- Create `src/wf_mcp/source_registry.py`
- Pydantic models and validation.
- Protocol and file store.
- No dependency on `WfMcpService`.
- Modify `src/wf_mcp/storage/__init__.py`
- No change in this slice unless the implementor decides a store export is needed.
- Prefer exporting from `wf_mcp.source_registry`, not overloading `wf_mcp.storage`.
- Create `tests/wf_mcp/test_source_registry.py`
- Direct model/store tests.
- Modify `docs/current_roadmap.md`
- Mark Slice 1 complete after implementation.
---
### Task 1: Add failing source registry model tests
**Files:**
- Create: `tests/wf_mcp/test_source_registry.py`
- [ ] **Step 1: Write model validation tests**
Create `tests/wf_mcp/test_source_registry.py`:
```python
from __future__ import annotations
import pytest
from wf_mcp.source_registry import (
HttpSourceTransport,
McpSourceRegistryEntry,
SourceRegistryFile,
StdioSourceTransport,
)
def _entry(source_id: str = "github.work") -> McpSourceRegistryEntry:
return McpSourceRegistryEntry(
id=source_id,
provider="github",
account="work",
transport=StdioSourceTransport(
command="npx",
args=("-y", "@modelcontextprotocol/server-github"),
env={"GITHUB_TOKEN": "${GITHUB_TOKEN}"},
),
auth_ref=source_id,
metadata={"purpose": "tests"},
)
def test_source_registry_entry_keeps_identity_and_transport_structural() -> None:
entry = _entry()
assert entry.id == "github.work"
assert entry.provider == "github"
assert entry.account == "work"
assert entry.profile is None
assert entry.transport.kind == "stdio"
assert entry.transport.command == "npx"
assert entry.auth_ref == "github.work"
def test_source_registry_accepts_http_transport() -> None:
entry = McpSourceRegistryEntry(
id="github.http",
provider="github",
account="work",
transport=HttpSourceTransport(url="https://example.test/mcp"),
)
assert entry.transport.kind == "http"
assert str(entry.transport.url) == "https://example.test/mcp"
def test_source_registry_rejects_duplicate_ids() -> None:
with pytest.raises(ValueError, match="duplicate source id 'github.work'"):
SourceRegistryFile(sources=[_entry("github.work"), _entry("github.work")])
def test_source_registry_rejects_reserved_ids() -> None:
with pytest.raises(ValueError, match="reserved"):
_entry("wf.admin")
def test_source_registry_rejects_unsafe_ids() -> None:
with pytest.raises(ValueError, match="connection id"):
_entry("../bad")
```
- [ ] **Step 2: Run tests and verify failure**
Run:
```bash
uv run pytest tests/wf_mcp/test_source_registry.py -q
```
Expected: FAIL because `wf_mcp.source_registry` does not exist yet.
---
### Task 2: Implement source registry models
**Files:**
- Create: `src/wf_mcp/source_registry.py`
- Test: `tests/wf_mcp/test_source_registry.py`
- [ ] **Step 1: Add models and validators**
Create `src/wf_mcp/source_registry.py`:
```python
from __future__ import annotations
import json
from collections.abc import Mapping, Sequence
from pathlib import Path
from typing import Annotated, Literal, Protocol
from pydantic import AnyHttpUrl, BaseModel, ConfigDict, Field, field_validator, model_validator
from .connections import parse_connection_id
from .shared.names import RESERVED_CONNECTION_IDS
class SourceRegistryModel(BaseModel):
"""Base model for persisted source registry state; reject misspelled fields."""
model_config = ConfigDict(extra="forbid")
class StdioSourceTransport(SourceRegistryModel):
kind: Literal["stdio"] = "stdio"
command: str = Field(min_length=1)
args: tuple[str, ...] = ()
env: dict[str, str] = Field(default_factory=dict)
class HttpSourceTransport(SourceRegistryModel):
kind: Literal["http"] = "http"
url: AnyHttpUrl
headers: dict[str, str] = Field(default_factory=dict)
SourceTransport = Annotated[
StdioSourceTransport | HttpSourceTransport,
Field(discriminator="kind"),
]
class McpSourceRegistryEntry(SourceRegistryModel):
"""Desired MCP source configuration persisted by server-owned mutation."""
id: str
kind: Literal["mcp"] = "mcp"
enabled: bool = True
provider: str = Field(min_length=1)
account: str = Field(min_length=1)
profile: str | None = None
transport: SourceTransport
auth_ref: str | None = None
metadata: dict[str, object] = Field(default_factory=dict)
@field_validator("id")
@classmethod
def validate_id(cls, value: str) -> str:
parse_connection_id(value)
if value in RESERVED_CONNECTION_IDS:
raise ValueError(f"source id {value!r} is reserved")
return value
class SourceRegistryFile(SourceRegistryModel):
version: Literal[1] = 1
sources: list[McpSourceRegistryEntry] = Field(default_factory=list)
@model_validator(mode="after")
def validate_unique_source_ids(self) -> SourceRegistryFile:
seen: set[str] = set()
for source in self.sources:
if source.id in seen:
raise ValueError(f"duplicate source id {source.id!r}")
seen.add(source.id)
return self
def source_map(self) -> dict[str, McpSourceRegistryEntry]:
return {source.id: source for source in self.sources}
class SourceRegistryStore(Protocol):
"""Persistence boundary for desired server-owned source configuration."""
def load_registry(self) -> SourceRegistryFile:
"""Return the stored registry, or an empty registry when absent."""
...
def save_registry(self, registry: SourceRegistryFile) -> None:
"""Persist one validated registry atomically."""
...
```
- [ ] **Step 2: Run model tests**
Run:
```bash
uv run pytest tests/wf_mcp/test_source_registry.py -q
```
Expected: model tests PASS except store tests are not added yet.
---
### Task 3: Add failing file store tests
**Files:**
- Modify: `tests/wf_mcp/test_source_registry.py`
- [ ] **Step 1: Append file store tests**
Append:
```python
from pathlib import Path
from wf_mcp.source_registry import FileSourceRegistryStore
def test_file_source_registry_store_loads_empty_registry_when_missing(
tmp_path: Path,
) -> None:
store = FileSourceRegistryStore(tmp_path)
registry = store.load_registry()
assert registry.version == 1
assert registry.sources == []
assert store.path == tmp_path / "source_registry.json"
def test_file_source_registry_store_round_trips_registry(tmp_path: Path) -> None:
store = FileSourceRegistryStore(tmp_path)
registry = SourceRegistryFile(sources=[_entry("github.work")])
store.save_registry(registry)
loaded = store.load_registry()
assert loaded.source_map()["github.work"].provider == "github"
assert loaded.source_map()["github.work"].transport.kind == "stdio"
def test_file_source_registry_store_validates_loaded_registry(tmp_path: Path) -> None:
store = FileSourceRegistryStore(tmp_path)
store.path.write_text(
'{"version": 1, "sources": [{"id": "wf.admin", "provider": "wf", '
'"account": "admin", "transport": {"kind": "stdio", "command": "x"}}]}',
encoding="utf-8",
)
with pytest.raises(ValueError, match="reserved"):
store.load_registry()
```
- [ ] **Step 2: Run tests and verify failure**
Run:
```bash
uv run pytest tests/wf_mcp/test_source_registry.py -q
```
Expected: FAIL because `FileSourceRegistryStore` does not exist yet.
---
### Task 4: Implement file store
**Files:**
- Modify: `src/wf_mcp/source_registry.py`
- Test: `tests/wf_mcp/test_source_registry.py`
- [ ] **Step 1: Add file store implementation**
Append to `src/wf_mcp/source_registry.py`:
```python
class FileSourceRegistryStore:
"""Filesystem implementation for desired source registry state."""
def __init__(self, root: Path) -> None:
self.root = root
self.root.mkdir(parents=True, exist_ok=True)
@property
def path(self) -> Path:
return self.root / "source_registry.json"
def load_registry(self) -> SourceRegistryFile:
if not self.path.exists():
return SourceRegistryFile()
data = json.loads(self.path.read_text(encoding="utf-8"))
return SourceRegistryFile.model_validate(data)
def save_registry(self, registry: SourceRegistryFile) -> None:
# Validate again at the store boundary so callers cannot persist stale or
# partially constructed model-like objects after mutation.
validated = SourceRegistryFile.model_validate(
registry.model_dump(mode="json")
)
payload = json.dumps(validated.model_dump(mode="json"), indent=2)
tmp_path = self.path.with_name(f"{self.path.name}.tmp")
tmp_path.write_text(payload, encoding="utf-8")
tmp_path.replace(self.path)
```
- [ ] **Step 2: Add `__all__`**
At the bottom of `src/wf_mcp/source_registry.py`, add:
```python
__all__ = [
"FileSourceRegistryStore",
"HttpSourceTransport",
"McpSourceRegistryEntry",
"SourceRegistryFile",
"SourceRegistryStore",
"SourceTransport",
"StdioSourceTransport",
]
```
- [ ] **Step 3: Run tests**
Run:
```bash
uv run pytest tests/wf_mcp/test_source_registry.py -q
```
Expected: PASS.
---
### Task 5: Documentation and verification
**Files:**
- Modify: `docs/current_roadmap.md`
- [ ] **Step 1: Update roadmap**
In `docs/current_roadmap.md`, near the store-backed source registry note, add:
```markdown
- First source registry implementation slice complete: validated registry
models plus `FileSourceRegistryStore` exist, but startup merge and mutation
commands are still deferred.
```
- [ ] **Step 2: Run verification**
Run:
```bash
uv run pytest tests/wf_mcp/test_source_registry.py -q
uv run ruff check src/wf_mcp/source_registry.py tests/wf_mcp/test_source_registry.py
uv run ruff format --check src/wf_mcp/source_registry.py tests/wf_mcp/test_source_registry.py
uv run basedpyright --level error src/wf_mcp/source_registry.py tests/wf_mcp/test_source_registry.py
```
Expected:
- pytest PASS
- ruff check PASS
- ruff format PASS
- basedpyright 0 errors
- [ ] **Step 3: Commit**
```bash
git add src/wf_mcp/source_registry.py tests/wf_mcp/test_source_registry.py docs/current_roadmap.md
git commit -m "feat: add source registry file store"
```
---
## Self-Review
- Spec coverage: Slice 1 only is covered: models, validation, file store, tests, docs.
- Placeholder scan: no TBD/TODO placeholders.
- Type consistency: `SourceRegistryFile`, `McpSourceRegistryEntry`, `SourceRegistryStore`, and `FileSourceRegistryStore` names are consistent.
- Deferred work is explicit: startup merge, runtime hydration, RPC/CLI mutation, and auth/catalog cleanup are not part of this slice.
@@ -0,0 +1,220 @@
# CodeRabbit Cleanup Follow-Ups Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Resolve non-blocking cleanup findings from the June 4 CodeRabbit review without mixing broad refactors into the focused correctness patch.
**Architecture:** Keep behavior unchanged unless a task explicitly says otherwise. Prefer documentation, small helpers, and narrowly scoped tests over cross-package refactors. Do not move modules or rename public APIs in this follow-up.
**Tech Stack:** Python 3.14, Pydantic v2, Typer, FastAPI JSON-RPC, pytest, ruff, basedpyright.
---
## Scope
This plan intentionally excludes the already-fixed review items:
- CLI config exception tuple syntax in `src/wf_cli/context.py`
- CLI source-registry file JSON object validation and exception chaining
- RPC method module structure-test coverage
- stale long-lived API and CLI/API alignment doc statuses
- typed `tmp_path` annotations in `tests/wf_config/test_config_models.py`
---
### Task 1: Document Source-Registry Mutation Asymmetry
**Files:**
- Modify: `docs/superpowers/plans/2026-06-04-source-registry-mutations.md`
- [ ] **Step 1: Add rationale under shadow handling**
Find the section mentioning:
```markdown
A future `allow_shadow` flag can relax add; do not add it in this slice.
```
Append:
```markdown
Rationale: `add` rejects config-shadowed ids to prevent silent no-ops: adding a
registry entry that cannot activate while config owns the same id. Existing
shadowed registry entries can still be updated, enabled, disabled, or removed so
operators can prepare store state for config removal or future `seed` ownership
policy.
```
- [ ] **Step 2: Verify doc diff**
Run:
```bash
git diff -- docs/superpowers/plans/2026-06-04-source-registry-mutations.md
```
Expected: only the rationale paragraph changed.
---
### Task 2: Clarify RPC Target Selection Precedence
**Files:**
- Modify: `docs/superpowers/plans/2026-06-03-workflow-config-and-rpc-cli-target.md`
- [ ] **Step 1: Add precedence table near the plan introduction**
Add:
```markdown
### Target Selection Precedence
1. `--url` CLI override selects an RPC HTTP target.
2. `--local` CLI override selects the in-process local target.
3. Config file `client.target` selects the configured target.
4. Missing target config defaults to local.
CLI overrides intentionally win over config so one-off diagnostics can point at
a different server without editing the config file.
```
- [ ] **Step 2: Verify no code references are changed**
Run:
```bash
git diff --stat
```
Expected: only the plan document is changed by this task.
---
### Task 3: Extract Source-Registry RPC Availability Helper
**Files:**
- Modify: `src/wf_transport_rpc_http/methods_source_registry.py`
- Test: `tests/wf_transport_rpc_http/test_source_registry_rpc.py`
- [ ] **Step 1: Add a helper**
In `src/wf_transport_rpc_http/methods_source_registry.py`, add:
```python
def _require_source_registry_admin(
server: WorkflowServer,
*,
operation: str,
) -> WorkflowSourceRegistrySurface:
admin = server.source_registry_admin
if admin is None:
raise WorkflowRpcError(
data={
"code": "source_registry_unavailable",
"message": (
f"source registry admin {operation} are not available "
"for this server"
),
}
)
return admin
```
Import `WorkflowSourceRegistrySurface` from `wf_api` if needed.
- [ ] **Step 2: Replace repeated `None` checks**
Use:
```python
admin = _require_source_registry_admin(server, operation="reads")
```
for list/inspect, and:
```python
admin = _require_source_registry_admin(server, operation="mutations")
```
for add/update/enable/disable/remove.
- [ ] **Step 3: Run source-registry RPC tests**
Run:
```bash
uv run pytest tests/wf_transport_rpc_http/test_source_registry_rpc.py -q
```
Expected: all tests pass.
---
### Task 4: Clarify Chronological Event Ordering
**Files:**
- Modify: `src/wf_api/admin.py`
- Test: existing admin API tests if present
- [ ] **Step 1: Add a comment above `list_events`**
Add a short comment/docstring note near `WorkflowAdminApi.list_events`:
```python
# Preserve provider order for events; event providers are expected to return
# chronological order and callers may rely on that ordering for diagnostics.
```
Do not sort event payloads in this task.
- [ ] **Step 2: Run admin API tests**
Run:
```bash
uv run pytest tests/wf_api -q
```
Expected: all `wf_api` tests pass.
---
### Task 5: Record Shared ID Pattern Follow-Up
**Files:**
- Modify: `docs/current_roadmap.md`
- [ ] **Step 1: Add a small platform cleanup bullet**
Under the platform cleanup/architecture section, add:
```markdown
- Cleanup candidate: consolidate store/source registry id validation patterns
(`SOURCE_REGISTRY_ID_PATTERN`, `STORE_ID_PATTERN`) only after another package
needs the same rule. Today they intentionally stay close to their stores.
```
- [ ] **Step 2: Verify docs only**
Run:
```bash
git diff -- docs/current_roadmap.md
```
Expected: only the cleanup bullet changed.
---
## Final Verification
Run:
```bash
uv run pytest tests/wf_transport_rpc_http/test_source_registry_rpc.py tests/wf_api -q
uv run ruff check src/wf_transport_rpc_http/methods_source_registry.py src/wf_api/admin.py docs/superpowers/plans/2026-06-04-source-registry-mutations.md docs/superpowers/plans/2026-06-03-workflow-config-and-rpc-cli-target.md docs/current_roadmap.md
uv run basedpyright --level error src/wf_transport_rpc_http/methods_source_registry.py src/wf_api/admin.py
git diff --check
```
Expected: pytest exits 0, ruff exits 0, basedpyright exits 0, and `git diff --check` reports no whitespace errors.
@@ -0,0 +1,722 @@
# Source Config Ownership Policy Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace implicit "config always shadows registry" behavior with explicit config ownership policy for MCP source entries: `locked` vs `seed`.
**Architecture:** Keep v1 safety by making `locked` the default for existing config connections. Add `seed` as an explicit opt-in that materializes a missing store entry, then lets the store-backed registry own future admin changes. Do this in the MCP broker config path first; neutral `wf_config.server.sources` only models built-in stdlib sources today and should not grow MCP transport config in this slice.
**Tech Stack:** Python 3.14, Pydantic v2, wf_mcp broker config/service, wf_mcp source registry, pytest, ruff, basedpyright.
---
## Current Context
Relevant current behavior:
- `src/wf_mcp/broker/service/connection_service.py::ConnectionService.sync_connections_from_config()` loads config connections first, then ignores same-id registry entries.
- Same-id registry entries emit `source_registry_ignored_config_shadow`.
- `src/wf_mcp/source_registry.py::McpSourceRegistryEntry` is the persisted desired source entry.
- `src/wf_mcp/models.py::ConnectionConfig` is the broker runtime config model used by legacy MCP config.
- `src/wf_mcp/control.py::BrokerConfigFile.to_runtime()` converts config-file connection declarations into `ConnectionConfig`.
Intended new policy:
- `locked`: config owns the source id. Same-id registry entries remain shadowed.
- `seed`: config bootstraps a missing registry entry. Once the registry entry exists, registry owns later runtime state for that id.
- Backward compatibility: existing config with no policy behaves as `locked`.
Out of scope:
- Do not add MCP source transports to neutral `wf_config.server.sources`.
- Do not add auth/catalog cleanup.
- Do not implement live remount without reload.
- Do not alter built-in reserved ids (`wf.std`, `wf.recipes`, `wf.admin`).
---
### Task 1: Add Policy Field to Broker Runtime and Config Models
**Files:**
- Modify: `src/wf_mcp/models.py`
- Modify: `src/wf_mcp/control.py`
- Test: `tests/wf_mcp/test_broker_config.py` or nearest existing broker config test file
- [ ] **Step 1: Inspect existing model shape**
Run:
```bash
rg -n "class ConnectionConfig|class .*Connection" src/wf_mcp/models.py src/wf_mcp/control.py tests/wf_mcp -g '*.py'
```
Expected: locate `ConnectionConfig` and the Pydantic config-file model that constructs it.
- [ ] **Step 2: Add a policy type and field to runtime config**
In `src/wf_mcp/models.py`, add a type alias near `ConnectionConfig`:
```python
SourceConfigOwnership = Literal["locked", "seed"]
```
Add to `ConnectionConfig`:
```python
source_config_ownership: SourceConfigOwnership = "locked"
```
If `Literal` is not imported, import it from `typing`.
- [ ] **Step 3: Add field to config-file connection model**
In `src/wf_mcp/control.py`, add the same field to the config-file connection model:
```python
source_config_ownership: SourceConfigOwnership = "locked"
```
When converting to `ConnectionConfig`, pass:
```python
source_config_ownership=self.source_config_ownership
```
If the config-file model is named differently, update the exact class that owns `id`, `server`, and `account`.
- [ ] **Step 4: Add config parsing tests**
In the existing broker config test file, add:
```python
def test_broker_config_connection_defaults_to_locked(tmp_path: Path) -> None:
config_path = tmp_path / "wf_mcp.config.json"
config_path.write_text(
json.dumps(
{
"store_root": str(tmp_path / "store"),
"connections": [
{"id": "demo.default", "server": "demo", "account": "default"}
],
}
),
encoding="utf-8",
)
config = load_broker_config(config_path)
assert config.connections[0].source_config_ownership == "locked"
def test_broker_config_connection_accepts_seed_policy(tmp_path: Path) -> None:
config_path = tmp_path / "wf_mcp.config.json"
config_path.write_text(
json.dumps(
{
"store_root": str(tmp_path / "store"),
"connections": [
{
"id": "demo.default",
"server": "demo",
"account": "default",
"source_config_ownership": "seed",
}
],
}
),
encoding="utf-8",
)
config = load_broker_config(config_path)
assert config.connections[0].source_config_ownership == "seed"
```
Import `json`, `Path`, and `load_broker_config` as needed.
- [ ] **Step 5: Run focused tests**
Run:
```bash
uv run pytest tests/wf_mcp/test_broker_config.py -q
```
Expected: all tests pass.
---
### Task 2: Convert Seed Config Connections to Registry Entries
**Files:**
- Modify: `src/wf_mcp/source_registry.py`
- Test: `tests/wf_mcp/test_source_registry.py`
- [ ] **Step 1: Add conversion helper**
In `src/wf_mcp/source_registry.py`, add:
```python
def connection_config_to_registry_entry(
connection: ConnectionConfig,
) -> McpSourceRegistryEntry:
"""Materialize a seed config connection into persisted registry state.
Seed config is bootstrap-only. The registry entry must carry enough source
identity to become the future desired-state owner after first startup.
"""
transport = connection.metadata.get("transport")
if not isinstance(transport, dict):
raise ValueError(
f"seed connection {connection.id!r} requires metadata.transport"
)
profile = connection.metadata.get("profile")
auth_ref = connection.metadata.get("auth_ref")
return McpSourceRegistryEntry.model_validate(
{
"id": connection.id,
"enabled": connection.enabled,
"provider": connection.server,
"account": connection.account,
"profile": profile if isinstance(profile, str) else None,
"transport": transport,
"auth_ref": auth_ref if isinstance(auth_ref, str) else None,
"metadata": {
key: value
for key, value in connection.metadata.items()
if key not in {"transport", "profile", "auth_ref", "source_registry"}
},
}
)
```
Also export it in `__all__`.
- [ ] **Step 2: Add conversion test**
In `tests/wf_mcp/test_source_registry.py`, add:
```python
def test_connection_config_to_registry_entry_preserves_transport_metadata() -> None:
connection = ConnectionConfig(
id="github.work",
server="github",
account="work",
enabled=False,
metadata={
"transport": {"kind": "stdio", "command": "npx", "args": ["server"]},
"profile": "corp",
"auth_ref": "secret://github/work",
"region": "us",
},
)
entry = connection_config_to_registry_entry(connection)
assert entry.id == "github.work"
assert entry.provider == "github"
assert entry.account == "work"
assert entry.enabled is False
assert entry.profile == "corp"
assert entry.auth_ref == "secret://github/work"
assert entry.transport.kind == "stdio"
assert entry.metadata["region"] == "us"
```
Add imports for `ConnectionConfig` and `connection_config_to_registry_entry`.
- [ ] **Step 3: Add missing transport failure test**
```python
def test_connection_config_to_registry_entry_requires_transport_metadata() -> None:
connection = ConnectionConfig(id="github.work", server="github", account="work")
with pytest.raises(ValueError, match="requires metadata.transport"):
connection_config_to_registry_entry(connection)
```
- [ ] **Step 4: Run source registry tests**
Run:
```bash
uv run pytest tests/wf_mcp/test_source_registry.py -q
```
Expected: all tests pass.
---
### Task 3: Implement Locked vs Seed Startup Merge
**Files:**
- Modify: `src/wf_mcp/broker/service/connection_service.py`
- Test: `tests/wf_mcp/service/test_connection_service.py`
- [ ] **Step 1: Update imports**
In `src/wf_mcp/broker/service/connection_service.py`, import:
```python
from ...source_registry import (
SourceRegistryFile,
SourceRegistryStore,
connection_config_to_registry_entry,
registry_entry_to_connection_config,
)
```
If `SourceRegistryStore` and `registry_entry_to_connection_config` are already imported, extend the existing import.
- [ ] **Step 2: Rewrite registry merge block**
Inside `sync_connections_from_config`, replace the current registry loop with this logic:
```python
connections = list(config.connections)
config_by_id = {connection.id: connection for connection in connections}
registry_entries = {}
registry_changed = False
if source_registry_store is not None:
registry = source_registry_store.load_registry()
registry_entries = registry.source_map()
for connection in connections:
if connection.source_config_ownership != "seed":
continue
if connection.id in registry_entries:
continue
seeded = connection_config_to_registry_entry(connection)
registry_entries[seeded.id] = seeded
registry_changed = True
self.events.record_kind(
"source_registry_seeded_from_config",
connection_id=seeded.id,
payload={"server": seeded.provider, "account": seeded.account},
)
if registry_changed:
source_registry_store.save_registry(
SourceRegistryFile(sources=list(registry_entries.values()))
)
merged_connections: list[ConnectionConfig] = []
for connection in connections:
registry_entry = registry_entries.get(connection.id)
if connection.source_config_ownership == "seed" and registry_entry is not None:
merged_connections.append(registry_entry_to_connection_config(registry_entry))
continue
merged_connections.append(connection)
merged_ids = {connection.id for connection in merged_connections}
for entry in registry_entries.values():
config_connection = config_by_id.get(entry.id)
if config_connection is not None:
if config_connection.source_config_ownership == "locked":
self.events.record_kind(
"source_registry_ignored_config_shadow",
connection_id=entry.id,
payload={
"server": entry.provider,
"account": entry.account,
"reason": "locked_config_connection_takes_precedence",
},
)
continue
if entry.id not in merged_ids:
merged_connections.append(registry_entry_to_connection_config(entry))
connections = merged_connections
```
Important notes:
- `locked` keeps current behavior.
- `seed` with no store entry writes a store entry, then uses that store entry.
- `seed` with an existing store entry uses the store entry, not config.
- Registry-only entries still hydrate as before.
- [ ] **Step 3: Add locked behavior regression test**
In `tests/wf_mcp/service/test_connection_service.py`, keep or add:
```python
def test_connection_service_sync_locked_config_shadows_registry_entry() -> None:
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
catalog = _source_catalog(service)
store = FileSourceRegistryStore(local_temp_root() / "locked_shadow")
store.save_registry(
SourceRegistryFile(
sources=[
_registry_entry(
"demo.default",
provider="registry",
account="stored",
)
]
)
)
config = BrokerConfig(
store_root=local_temp_root(),
connections=[
ConnectionConfig(
id="demo.default",
server="config",
account="locked",
source_config_ownership="locked",
)
],
)
service.bind_source_catalog(catalog)
service.sync_connections_from_config(config, source_registry_store=store)
connection = service.get("demo.default")
assert connection.server == "config"
assert connection.account == "locked"
assert any(
event.kind == "source_registry_ignored_config_shadow"
for event in service.events.list_events()
)
```
Use existing helpers if names differ.
- [ ] **Step 4: Add seed materialization test**
```python
def test_connection_service_sync_seed_config_materializes_registry_entry() -> None:
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
catalog = _source_catalog(service)
store_root = local_temp_root() / "seed_materialized"
store = FileSourceRegistryStore(store_root)
config = BrokerConfig(
store_root=local_temp_root(),
connections=[
ConnectionConfig(
id="demo.default",
server="demo",
account="default",
metadata={"transport": {"kind": "stdio", "command": "demo-server"}},
source_config_ownership="seed",
)
],
)
service.bind_source_catalog(catalog)
service.sync_connections_from_config(config, source_registry_store=store)
registry = store.load_registry()
assert registry.sources[0].id == "demo.default"
assert registry.sources[0].provider == "demo"
assert service.get("demo.default").metadata["source_registry"] is True
assert any(
event.kind == "source_registry_seeded_from_config"
for event in service.events.list_events()
)
```
- [ ] **Step 5: Add seed existing-store-wins test**
```python
def test_connection_service_sync_seed_existing_registry_entry_wins() -> None:
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
catalog = _source_catalog(service)
store = FileSourceRegistryStore(local_temp_root() / "seed_existing")
store.save_registry(
SourceRegistryFile(
sources=[
_registry_entry(
"demo.default",
provider="registry",
account="stored",
)
]
)
)
config = BrokerConfig(
store_root=local_temp_root(),
connections=[
ConnectionConfig(
id="demo.default",
server="config",
account="seed",
metadata={"transport": {"kind": "stdio", "command": "config-server"}},
source_config_ownership="seed",
)
],
)
service.bind_source_catalog(catalog)
service.sync_connections_from_config(config, source_registry_store=store)
connection = service.get("demo.default")
assert connection.server == "registry"
assert connection.account == "stored"
```
- [ ] **Step 6: Run connection-service tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
```
Expected: all tests pass.
---
### Task 4: Update Registry Mutation Shadow Checks
**Files:**
- Modify: `src/wf_mcp/broker/service/source_registry_admin.py`
- Test: `tests/wf_mcp/service/test_source_registry_admin.py`
- [ ] **Step 1: Add config ownership lookup helper**
In `SourceRegistryAdminProvider`, add:
```python
def _config_connection(self, source_id: str) -> ConnectionConfig | None:
for connection in self.config_connections:
if connection.id == source_id:
return connection
return None
```
- [ ] **Step 2: Change `config_source_ids` if needed**
Keep `config_source_ids()` unchanged for read payload compatibility:
```python
def config_source_ids(self) -> set[str]:
return {connection.id for connection in self.config_connections}
```
- [ ] **Step 3: Update `add_registry_entry` shadow rejection**
Replace the current config id check with:
```python
config_connection = self._config_connection(source_id)
if (
config_connection is not None
and config_connection.source_config_ownership == "locked"
):
raise ValueError(
f"cannot add {source_id!r}: id is locked by a config connection"
)
```
For `seed`, adding remains allowed only if no registry entry already exists.
- [ ] **Step 4: Add locked add rejection test**
In `tests/wf_mcp/service/test_source_registry_admin.py`, add:
```python
def test_add_rejects_locked_config_shadow(tmp_path: Path) -> None:
provider = _provider(tmp_path, config_ids=frozenset({"github.work"}))
with pytest.raises(ValueError, match="locked by a config connection"):
provider.add_registry_entry(_entry_dict("github.work"))
```
If `_provider` cannot pass policy, update it to build `ConnectionConfig(..., source_config_ownership="locked")`.
- [ ] **Step 5: Add seed add allowed test**
Update `_provider` to accept config connections or config policy, then add:
```python
def test_add_allows_seed_config_shadow_when_registry_missing(tmp_path: Path) -> None:
provider = _provider(
tmp_path,
config_connections=[
ConnectionConfig(
id="github.work",
server="github",
account="work",
source_config_ownership="seed",
)
],
)
result = provider.add_registry_entry(_entry_dict("github.work"))
assert result.id == "github.work"
```
- [ ] **Step 6: Run source registry admin tests**
Run:
```bash
uv run pytest tests/wf_mcp/service/test_source_registry_admin.py -q
```
Expected: all tests pass.
---
### Task 5: Expose Ownership in Admin Registry Payloads
**Files:**
- Modify: `src/wf_api/source_registry_admin.py`
- Modify: `src/wf_mcp/broker/service/source_registry_admin.py`
- Test: `tests/wf_api/test_source_registry_admin_api.py`
- Test: `tests/wf_mcp/service/test_source_registry_admin.py`
- [ ] **Step 1: Extend provider protocol**
In `WorkflowSourceRegistryProvider`, add:
```python
def config_source_ownership(self) -> Mapping[str, str]: ...
```
- [ ] **Step 2: Implement provider method**
In `SourceRegistryAdminProvider`, add:
```python
def config_source_ownership(self) -> dict[str, str]:
return {
connection.id: connection.source_config_ownership
for connection in self.config_connections
}
```
- [ ] **Step 3: Include ownership in summary and inspect payloads**
In `WorkflowSourceRegistryApi`, compute:
```python
ownership = self._provider.config_source_ownership()
```
Update `_entry_summary` signature to accept `ownership: Mapping[str, str]`.
Include:
```python
"config_ownership": ownership.get(entry_id),
"mutable": ownership.get(entry_id) != "locked",
```
For inspect payloads, include the same fields at top level:
```python
"config_ownership": ownership.get(source_id),
"mutable": ownership.get(source_id) != "locked",
```
Keep existing `shadowed_by_config` for compatibility.
- [ ] **Step 4: Update fake providers in tests**
In `tests/wf_api/test_source_registry_admin_api.py`, add:
```python
def config_source_ownership(self) -> dict[str, str]:
return {source_id: "locked" for source_id in self._config_ids}
```
or allow the fake to accept an ownership mapping.
- [ ] **Step 5: Add API payload test**
```python
def test_list_registry_entries_reports_config_ownership_and_mutability() -> None:
api = WorkflowSourceRegistryApi(
provider=FakeRegistryProvider(
[FakeRegistryEntry(id="github.work")],
config_ids={"github.work"},
)
)
payload = asyncio.run(api.list_registry_entries())
entry = payload["entries"][0]
assert entry["shadowed_by_config"] is True
assert entry["config_ownership"] == "locked"
assert entry["mutable"] is False
```
- [ ] **Step 6: Run API/provider tests**
Run:
```bash
uv run pytest tests/wf_api/test_source_registry_admin_api.py tests/wf_mcp/service/test_source_registry_admin.py -q
```
Expected: all tests pass.
---
### Task 6: Update Docs and Roadmap Status
**Files:**
- Modify: `docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md`
- Modify: `docs/superpowers/plans/2026-06-03-source-registry-next-slices.md`
- Modify: `docs/current_roadmap.md`
- [ ] **Step 1: Update spec Slice 6 status**
In `docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md`, change Slice 6 status from planned to complete and summarize:
```markdown
Status: complete. MCP broker config connections now support
`source_config_ownership="locked" | "seed"`. `locked` preserves v1 shadowing.
`seed` materializes missing store entries and lets existing registry entries
own future runtime state.
```
- [ ] **Step 2: Update next-slices plan**
In `docs/superpowers/plans/2026-06-03-source-registry-next-slices.md`, mark Slice 6 complete.
- [ ] **Step 3: Update current roadmap**
In `docs/current_roadmap.md`, replace the planned wording with implemented wording:
```markdown
Config ownership policy is implemented for MCP broker config connections:
`locked` entries stay operator-owned, while `seed` entries bootstrap missing
store entries and then let the store own later admin changes.
```
- [ ] **Step 4: Run doc diff check**
Run:
```bash
git diff -- docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md docs/superpowers/plans/2026-06-03-source-registry-next-slices.md docs/current_roadmap.md
```
Expected: docs match the implemented slice and do not claim neutral `wf_config.server.sources` supports MCP source ownership yet.
---
## Final Verification
Run:
```bash
uv run pytest tests/wf_mcp/test_source_registry.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/service/test_source_registry_admin.py tests/wf_api/test_source_registry_admin_api.py -q
uv run ruff check src/wf_mcp/models.py src/wf_mcp/control.py src/wf_mcp/source_registry.py src/wf_mcp/broker/service/connection_service.py src/wf_mcp/broker/service/source_registry_admin.py src/wf_api/source_registry_admin.py tests/wf_mcp/test_source_registry.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/service/test_source_registry_admin.py tests/wf_api/test_source_registry_admin_api.py
uv run basedpyright --level error src/wf_mcp/models.py src/wf_mcp/control.py src/wf_mcp/source_registry.py src/wf_mcp/broker/service/connection_service.py src/wf_mcp/broker/service/source_registry_admin.py src/wf_api/source_registry_admin.py tests/wf_mcp/test_source_registry.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/service/test_source_registry_admin.py tests/wf_api/test_source_registry_admin_api.py
git diff --check
```
Expected: pytest exits 0, ruff exits 0, basedpyright exits 0, and `git diff --check` reports no whitespace errors.
## Self-Review
- This plan intentionally keeps `locked` as the default for backward compatibility.
- This plan does not add MCP source entries to neutral `wf_config` because current neutral source config only supports built-in sources.
- This plan requires `seed` config connections to carry `metadata.transport`; without transport, the config cannot be materialized into a durable registry entry.
- The admin payload adds `config_ownership` and `mutable` without removing `shadowed_by_config`, preserving compatibility.
@@ -0,0 +1,347 @@
# Source Registry Admin Reads Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Expose the persisted desired source registry through read-only admin APIs, JSON-RPC, and CLI without confusing it with observed runtime source inventory.
**Architecture:** Desired registry state is server/platform configuration. It is not workflow lifecycle state and not observed catalog/source inventory. Add a neutral read-only `WorkflowSourceRegistryApi` over a provider protocol in `wf_api`; implement the provider in `wf_mcp` using `FileSourceRegistryStore` and optional config connection ids for shadow information. Keep mutations out of scope.
**Tech Stack:** Python 3.14, Pydantic v2, `wf_api`, `wf_mcp.source_registry`, `wf_transport_rpc_http`, `wf_cli`, Typer, pytest, ruff, basedpyright.
---
## Naming Decision
Use admin/config naming:
- JSON-RPC:
- `workflow.admin.source_registry.list`
- `workflow.admin.source_registry.inspect`
- CLI:
- `wf admin registry list`
- `wf admin registry inspect SOURCE_ID`
Do **not** use `wf source registry ...` in this slice. `wf source list` already means observed/hydrated source inventory. Registry reads are desired server-owned configuration state.
---
## Payload Shape
List payload:
```json
{
"entries": [
{
"id": "github.work",
"kind": "mcp",
"enabled": true,
"provider": "github",
"account": "work",
"profile": null,
"transport_kind": "stdio",
"auth_ref": "github.work",
"shadowed_by_config": false
}
],
"next_cursor": null,
"total": 1
}
```
Inspect payload:
```json
{
"entry": {
"id": "github.work",
"kind": "mcp",
"enabled": true,
"provider": "github",
"account": "work",
"profile": null,
"transport": {"kind": "stdio", "command": "npx", "args": [], "env": {}},
"auth_ref": "github.work",
"metadata": {}
},
"shadowed_by_config": false
}
```
Notes:
- List returns summaries; inspect returns full entry detail.
- `shadowed_by_config` is advisory. If a caller has no config provider, return
`false` rather than guessing.
- Do not include auth secret payloads. `auth_ref` is only an id/reference.
---
## Task 1: Add Neutral Source Registry Admin API
- [ ] Create `src/wf_api/source_registry_admin.py`.
- [ ] Define:
```python
from __future__ import annotations
from collections.abc import Mapping, Sequence, Set
from dataclasses import asdict, is_dataclass
from typing import Any, Protocol
from wf_platform import page_items
class WorkflowSourceRegistryProvider(Protocol):
"""Provides desired source registry state for read-only admin frontends."""
def list_registry_entries(self) -> Sequence[Mapping[str, Any] | object]: ...
def config_source_ids(self) -> Set[str]: ...
```
- [ ] Define `WorkflowSourceRegistryApi` with:
```python
async def list_registry_entries(
self,
*,
cursor: str | None = None,
limit: int = 50,
) -> dict[str, Any]: ...
async def inspect_registry_entry(self, *, source_id: str) -> dict[str, Any]: ...
```
- [ ] Normalize provider objects using the same style as `wf_api.admin._payload`:
mapping, dataclass, or Pydantic `model_dump(mode="json")`.
- [ ] Add helpers:
```python
def _entry_summary(entry: dict[str, Any], shadowed_ids: set[str]) -> dict[str, Any]:
transport = entry.get("transport")
transport_kind = transport.get("kind") if isinstance(transport, Mapping) else None
return {
"id": entry["id"],
"kind": entry["kind"],
"enabled": entry["enabled"],
"provider": entry.get("provider"),
"account": entry.get("account"),
"profile": entry.get("profile"),
"transport_kind": transport_kind,
"auth_ref": entry.get("auth_ref"),
"shadowed_by_config": entry["id"] in shadowed_ids,
}
```
- [ ] `inspect_registry_entry()` should raise `KeyError(f"unknown registry source {source_id!r}")` when missing.
- [ ] Export `WorkflowSourceRegistryApi` and `WorkflowSourceRegistryProvider` from `src/wf_api/__init__.py`.
- [ ] Add `WorkflowSourceRegistrySurface` to `src/wf_api/surface.py` and `__all__`.
### Tests
- [ ] Create `tests/wf_api/test_source_registry_admin_api.py`.
- [ ] Test list returns compact summaries in id order.
- [ ] Test pagination.
- [ ] Test inspect returns full entry and shadow flag.
- [ ] Test unknown inspect raises clear `KeyError`.
- [ ] Test the concrete API satisfies `WorkflowSourceRegistrySurface`.
---
## Task 2: Add MCP Provider for Registry Reads
- [ ] Create `src/wf_mcp/broker/service/source_registry_admin.py`.
- [ ] Define:
```python
from __future__ import annotations
from collections.abc import Sequence
from dataclasses import dataclass, field
from ...models import ConnectionConfig
from ...source_registry import SourceRegistryStore
@dataclass(slots=True)
class SourceRegistryAdminProvider:
"""Read desired MCP source registry state without mutating it."""
source_registry_store: SourceRegistryStore
config_connections: Sequence[ConnectionConfig] = field(default_factory=tuple)
def list_registry_entries(self) -> list[object]:
return list(self.source_registry_store.load_registry().sources)
def config_source_ids(self) -> set[str]:
return {connection.id for connection in self.config_connections}
```
- [ ] Keep this provider read-only.
- [ ] Do not load auth records or catalog snapshots here.
### Tests
- [ ] Create `tests/wf_mcp/service/test_source_registry_admin.py`.
- [ ] Test provider lists entries from `FileSourceRegistryStore`.
- [ ] Test provider reports config-shadowed ids.
---
## Task 3: Wire Server Context
- [ ] Update `src/wf_server/context.py`.
- [ ] Add a nullable `source_registry_admin` field to `WorkflowServer`:
```python
source_registry_admin: WorkflowSourceRegistryApi | None = None
```
- [ ] For the local/static server builder, leave it as `None`. Local/static server has no file-backed MCP source registry.
- [ ] This field is used by JSON-RPC registration; missing support should return a structured RPC error.
### Tests
- [ ] Update `tests/wf_server/test_local_static_server.py` if needed to assert local/static construction still works.
---
## Task 4: Wire MCP/CLI Server Construction
- [ ] Identify the current `WorkflowServer` construction path for JSON-RPC HTTP target servers.
- [ ] When constructing a server from broker config, create:
```python
source_registry_admin = WorkflowSourceRegistryApi(
SourceRegistryAdminProvider(
source_registry_store=FileSourceRegistryStore(config.store_root),
config_connections=config.connections,
)
)
```
- [ ] Attach it to `WorkflowServer`.
- [ ] Do not alter source registry startup merge behavior in this slice.
### Tests
- [ ] Add or update a JSON-RPC server construction test that seeds `source_registry.json` and asserts `server.source_registry_admin` is not `None`.
---
## Task 5: Add JSON-RPC Methods and Client Mixin
- [ ] Add `src/wf_transport_rpc_http/methods_source_registry.py`.
- [ ] Register:
```python
workflow.admin.source_registry.list
workflow.admin.source_registry.inspect
```
- [ ] If `server.source_registry_admin is None`, raise `WorkflowRpcError` with:
```json
{
"code": "source_registry_unavailable",
"message": "source registry admin reads are not available for this server"
}
```
- [ ] Add `src/wf_transport_rpc_http/client_source_registry.py`.
- [ ] Add `RpcSourceRegistryClientMixin` with:
```python
async def list_registry_entries(self, *, cursor: str | None = None, limit: int = 50) -> dict[str, Any]: ...
async def inspect_registry_entry(self, *, source_id: str) -> dict[str, Any]: ...
```
- [ ] Include the mixin in `src/wf_transport_rpc_http/client.py`.
- [ ] Register methods in `src/wf_transport_rpc_http/app.py`.
### Tests
- [ ] Add focused JSON-RPC method tests.
- [ ] Add client tests proving the mixin calls the correct method names.
- [ ] Add a local/static unavailable test.
---
## Task 6: Add CLI Commands
- [ ] Add `src/wf_cli/commands/source_registry.py`.
- [ ] Register it under the existing `wf admin` app:
```bash
wf admin registry list
wf admin registry inspect SOURCE_ID
```
- [ ] Use the target-aware CLI context, not local-only context.
- [ ] Output JSON by default, following existing CLI command conventions.
- [ ] Do not add mutation flags.
### Tests
- [ ] Add CLI tests:
- local/server target lists entries
- inspect returns full entry
- missing entry exits nonzero or returns structured error, matching current CLI patterns
---
## Task 7: Docs
- [ ] Update `docs/current_roadmap.md`.
- [ ] Update `docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md`.
- [ ] Update `docs/superpowers/plans/2026-06-03-source-registry-next-slices.md`.
- [ ] Add a short note to CLI docs if there is a current CLI command reference:
```md
`wf admin registry list` shows desired persisted registry entries. `wf source list`
shows observed/hydrated source inventory. Use both when debugging disabled,
shadowed, or not-yet-hydrated sources.
```
---
## Task 8: Verify
- [ ] Focused tests:
```bash
uv run pytest tests/wf_api/test_source_registry_admin_api.py tests/wf_mcp/service/test_source_registry_admin.py -q
```
- [ ] RPC/CLI tests:
```bash
uv run pytest tests/wf_transport_rpc_http tests/wf_cli -q
```
- [ ] Existing source registry tests:
```bash
uv run pytest tests/wf_api/test_source_registry.py tests/wf_mcp/test_source_registry.py -q
```
- [ ] Quality:
```bash
uv run ruff check src/wf_api src/wf_mcp src/wf_transport_rpc_http src/wf_cli tests/wf_api tests/wf_mcp tests/wf_transport_rpc_http tests/wf_cli
uv run basedpyright --level error src/wf_api src/wf_mcp src/wf_transport_rpc_http src/wf_cli tests/wf_api tests/wf_mcp tests/wf_transport_rpc_http tests/wf_cli
```
---
## Acceptance Criteria
- Desired registry reads are available separately from observed source inventory.
- List is compact and paginated.
- Inspect returns full persisted entry details.
- Shadowed-by-config status is visible.
- Local/static servers report registry reads unavailable instead of pretending to have an empty registry.
- No mutation commands are added.
- `wf source list` behavior is unchanged.
@@ -0,0 +1,365 @@
# Source Registry Mutations Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add safe source registry mutation operations so server-owned MCP sources can be added, updated, enabled, disabled, and removed without editing config files by hand.
**Architecture:** Mutations target only the persisted desired registry, never config files. `wf_api` stays protocol-neutral by accepting/returning registry entry dictionaries. The MCP provider owns validation through `McpSourceRegistryEntry` / `SourceRegistryFile` and persistence through `SourceRegistryStore`. Runtime hydration still happens through the existing startup/reload merge path.
**Tech Stack:** Python 3.14, Pydantic v2, `wf_api.source_registry_admin`, `wf_mcp.source_registry`, `wf_transport_rpc_http`, `wf_cli`, Typer, pytest, ruff, basedpyright.
---
## Scope
Add these operations:
- add source registry entry
- update source registry entry
- enable source registry entry
- disable source registry entry
- remove source registry entry
Out of scope:
- no auth record creation/deletion
- no catalog snapshot deletion
- no config file mutation
- no live transport validation by default
- no raw proxy remount behavior changes
---
## Semantics
### Config Shadowing
Config-defined connections still win over registry entries during startup/reload.
For mutation v1:
- `add` rejects a source id that already exists in config to avoid silently adding a registry entry with no runtime effect.
- `update`, `enable`, `disable`, and `remove` may operate on existing registry entries even if they are currently shadowed by config.
- A future `allow_shadow` flag can relax `add`; do not add it in this slice.
Rationale: `add` rejects config-shadowed ids to prevent silent no-ops: adding a
registry entry that cannot activate while config owns the same id. Existing
shadowed registry entries can still be updated, enabled, disabled, or removed so
operators can prepare store state for config removal or future `seed` ownership
policy.
### Full-Registry Validation
Every mutation must:
1. load the current registry
2. build a new `SourceRegistryFile`
3. validate the full file
4. save one atomic replacement
5. return the updated entry or removal payload
Never mutate the loaded registry object in-place and then save after partial validation. Build a new list first.
### Events
Mutations should emit events if an event recorder is available:
- `source_registry_entry_added`
- `source_registry_entry_updated`
- `source_registry_entry_enabled`
- `source_registry_entry_disabled`
- `source_registry_entry_removed`
Payload should include at least:
```json
{"source_id": "github.work", "provider": "github", "account": "work"}
```
If adding event wiring is awkward for this slice, keep mutation payloads correct and document event wiring as future work. Do not block mutation safety on event polish.
---
## Task 1: Extend Neutral API Provider Protocol
- [ ] Update `src/wf_api/source_registry_admin.py`.
- [ ] Add mutation methods to `WorkflowSourceRegistryProvider`:
```python
def add_registry_entry(self, entry: Mapping[str, Any]) -> Mapping[str, Any] | object: ...
def update_registry_entry(
self,
source_id: str,
patch: Mapping[str, Any],
) -> Mapping[str, Any] | object: ...
def set_registry_entry_enabled(
self,
source_id: str,
enabled: bool,
) -> Mapping[str, Any] | object: ...
def remove_registry_entry(self, source_id: str) -> Mapping[str, Any] | object: ...
```
- [ ] Add async API methods:
```python
async def add_registry_entry(self, *, entry: dict[str, Any]) -> dict[str, Any]: ...
async def update_registry_entry(
self,
*,
source_id: str,
patch: dict[str, Any],
) -> dict[str, Any]: ...
async def enable_registry_entry(self, *, source_id: str) -> dict[str, Any]: ...
async def disable_registry_entry(self, *, source_id: str) -> dict[str, Any]: ...
async def remove_registry_entry(self, *, source_id: str) -> dict[str, Any]: ...
```
- [ ] Return shapes:
```python
{"entry": <full entry>, "shadowed_by_config": bool}
{"removed": true, "source_id": "..."}
```
- [ ] Reuse existing `_payload()` and shadow helper logic.
- [ ] Update `WorkflowSourceRegistrySurface` in `src/wf_api/surface.py`.
### Tests
- [ ] Extend `tests/wf_api/test_source_registry_admin_api.py`.
- [ ] Add fake mutable provider.
- [ ] Test each API method delegates and returns normalized payloads.
- [ ] Test remove payload.
---
## Task 2: Implement MCP Mutation Provider
- [ ] Update `src/wf_mcp/broker/service/source_registry_admin.py`.
- [ ] Keep `SourceRegistryAdminProvider` as the read/write provider.
- [ ] Add optional event sink if practical:
```python
from collections.abc import Callable
from ...events import McpEvent
event_sink: Callable[[McpEvent], None] | None = None
```
If this creates too much coupling, skip event sink and document it.
- [ ] Add helpers:
```python
def _load(self) -> SourceRegistryFile: ...
def _save(self, sources: list[McpSourceRegistryEntry]) -> SourceRegistryFile: ...
def _entry_map(self, registry: SourceRegistryFile) -> dict[str, McpSourceRegistryEntry]: ...
def _require_entry(self, source_id: str) -> McpSourceRegistryEntry: ...
```
- [ ] `add_registry_entry(entry)`:
- reject if `entry["id"]` is in `config_source_ids()`
- validate with `McpSourceRegistryEntry.model_validate(entry)`
- reject duplicate existing registry id
- save full `SourceRegistryFile`
- return added entry
- [ ] `update_registry_entry(source_id, patch)`:
- require existing registry entry
- reject changing `id` in v1 unless it equals `source_id`
- merge existing full JSON entry with patch
- validate with `McpSourceRegistryEntry`
- save full file
- return updated entry
- [ ] `set_registry_entry_enabled(source_id, enabled)`:
- require existing registry entry
- update `enabled`
- save full file
- return updated entry
- [ ] `remove_registry_entry(source_id)`:
- require existing registry entry
- save full file without the entry
- return `{"removed": True, "source_id": source_id}`
- do not delete auth/catalog
### Tests
- [ ] Extend or create tests in `tests/wf_mcp/service/test_source_registry_admin.py`.
- [ ] Test add persists and round-trips through store.
- [ ] Test add rejects config-shadowed id.
- [ ] Test add rejects duplicate registry id.
- [ ] Test update persists provider/account/transport changes.
- [ ] Test update rejects id change.
- [ ] Test enable/disable persist.
- [ ] Test remove persists absence and does not touch unrelated entries.
- [ ] Test missing source raises clear `KeyError`.
- [ ] Test malformed payload raises validation error with actionable message.
---
## Task 3: Add RPC Mutation Methods
- [ ] Update `src/wf_transport_rpc_http/models.py`.
- [ ] Add params:
```python
class AddRegistryEntryParams(RpcParamsModel):
entry: dict[str, Any]
class UpdateRegistryEntryParams(RpcParamsModel):
source_id: str = Field(min_length=1)
patch: dict[str, Any]
class RegistryEntryIdParams(RpcParamsModel):
source_id: str = Field(min_length=1)
```
- [ ] Update `src/wf_transport_rpc_http/methods_source_registry.py`.
- [ ] Register:
```text
workflow.admin.source_registry.add
workflow.admin.source_registry.update
workflow.admin.source_registry.enable
workflow.admin.source_registry.disable
workflow.admin.source_registry.remove
```
- [ ] Keep unavailable behavior identical to read methods:
```json
{"code": "source_registry_unavailable", ...}
```
- [ ] Use `Params(...)` for required params, matching inspect methods.
- [ ] Update `src/wf_transport_rpc_http/client_source_registry.py`.
- [ ] Add matching client methods.
### Tests
- [ ] Extend `tests/wf_transport_rpc_http/test_source_registry_rpc.py`.
- [ ] Add positive method tests using a fake `WorkflowServer` with `WorkflowSourceRegistryApi`.
- [ ] Add unavailable mutation test on local/static server.
- [ ] Add client method tests for method names and payloads.
---
## Task 4: Add CLI Mutation Commands
- [ ] Update `src/wf_cli/commands/source_registry.py`.
- [ ] Add commands:
```bash
wf admin registry add --input '{"id":"github.work",...}'
wf admin registry update SOURCE_ID --patch '{"enabled":false}'
wf admin registry enable SOURCE_ID
wf admin registry disable SOURCE_ID
wf admin registry remove SOURCE_ID
```
- [ ] Support `--input-file` for add and `--patch-file` for update if existing CLI helpers make it easy.
- [ ] JSON inline input is required for v1. Do not design a large flag matrix for every MCP field yet.
- [ ] For remove, require `--confirm` to avoid accidental deletion:
```bash
wf admin registry remove github.work --confirm
```
- [ ] If `source_registry_admin is None`, keep the existing "not available" behavior.
### Tests
- [ ] Extend `tests/wf_cli/test_source_registry.py`.
- [ ] Test help includes add/update/enable/disable/remove.
- [ ] Test local/static unavailable for one mutation command.
- [ ] Test remove without `--confirm` fails.
- [ ] Test commands delegate to a fake context where possible, or use RPC client monkeypatch patterns already used in CLI tests.
---
## Task 5: Wire Real MCP Server Provider If Available
Current state: `WorkflowServer.source_registry_admin` exists, but local/static servers set it to `None`. There may still be no concrete MCP-backed `WorkflowServer` construction path.
- [ ] If a concrete MCP-backed `WorkflowServer` construction path exists, wire:
```python
WorkflowSourceRegistryApi(
provider=SourceRegistryAdminProvider(
source_registry_store=FileSourceRegistryStore(config.store_root),
config_connections=config.connections,
)
)
```
- [ ] If no such path exists, do not invent it in this slice. Keep docs explicit that RPC/CLI mutation methods require a target exposing `source_registry_admin`.
---
## Task 6: Docs
- [ ] Update `docs/current_roadmap.md`.
- [ ] Update `docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md`.
- [ ] Update `docs/superpowers/plans/2026-06-03-source-registry-next-slices.md`.
- [ ] Document:
- registry mutations change desired state only
- config files are not mutated
- auth/catalog are not deleted on remove
- startup/reload is still how runtime hydration sees changes
- config-shadowed add is rejected in v1
---
## Task 7: Verify
- [ ] Focused API/provider tests:
```bash
uv run pytest tests/wf_api/test_source_registry_admin_api.py tests/wf_mcp/service/test_source_registry_admin.py -q
```
- [ ] RPC/CLI tests:
```bash
uv run pytest tests/wf_transport_rpc_http/test_source_registry_rpc.py tests/wf_cli/test_source_registry.py -q
```
- [ ] Existing registry/startup tests:
```bash
uv run pytest tests/wf_api/test_source_registry.py tests/wf_mcp/test_source_registry.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/test_broker_server.py -q
```
- [ ] Quality:
```bash
uv run ruff check src/wf_api/source_registry_admin.py src/wf_mcp/broker/service/source_registry_admin.py src/wf_transport_rpc_http src/wf_cli tests/wf_api/test_source_registry_admin_api.py tests/wf_mcp/service/test_source_registry_admin.py tests/wf_transport_rpc_http/test_source_registry_rpc.py tests/wf_cli/test_source_registry.py
uv run basedpyright --level error src/wf_api/source_registry_admin.py src/wf_mcp/broker/service/source_registry_admin.py src/wf_transport_rpc_http src/wf_cli tests/wf_api/test_source_registry_admin_api.py tests/wf_mcp/service/test_source_registry_admin.py tests/wf_transport_rpc_http/test_source_registry_rpc.py tests/wf_cli/test_source_registry.py
```
---
## Acceptance Criteria
- Registry add/update/enable/disable/remove work through neutral API methods.
- MCP provider validates all writes through `McpSourceRegistryEntry` / `SourceRegistryFile`.
- Whole registry is validated before every save.
- Config files are not mutated.
- Auth/catalog files are not deleted.
- Adding an id owned by config is rejected in v1.
- Existing shadowed registry entries can still be updated or removed.
- RPC methods and CLI commands exist.
- Remove requires explicit CLI confirmation.
- Observed source inventory behavior is unchanged until reload/startup rehydrates runtime state.