docs: archive completed superpowers plans
This commit is contained in:
@@ -0,0 +1,797 @@
|
||||
# Capability Sources Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace the ad hoc broker/proxy/admin/stdlib split with a source registry that can project workflow node specs, MCP tools, prompts, and resources to the right surfaces.
|
||||
|
||||
**Architecture:** Add `CapabilitySource` as the source registry model, keep `SpecSource` behavior as a compatibility projection while the rest of the system migrates, then move stdlib and admin capabilities into explicit `wf.std`, `wf.mcp`, and `wf.admin` sources. Broker/proxy servers should project capabilities from sources instead of defining separate duplicate admin tools.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, Pydantic models already used by node specs, FastMCP/MCP server decorators, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create `src/wf_mcp/broker/service/capability_sources.py`: canonical source model, visibility flags, permission flags, capability buckets, projection helpers.
|
||||
- Modify `src/wf_mcp/broker/service/sources.py`: either delegate to the new model or become a small compatibility import.
|
||||
- Modify `src/wf_mcp/broker/service/core.py`: store `capability_sources`, derive node-spec resolution from them, keep `spec_sources` and `specs_by_connection` as compatibility views if needed.
|
||||
- Modify `src/wf_mcp/broker/service/builtins.py`: register `wf.std` stdlib specs and `wf.mcp` runtime specs through capability sources.
|
||||
- Modify `src/wf_mcp/broker/tools.py`: project broker MCP tools from `wf.admin` source definitions.
|
||||
- Create `src/wf_mcp/broker/admin_capabilities.py`: one reusable definition of broker/admin tool capabilities.
|
||||
- Modify `src/wf_mcp/transparent_proxy/admin.py` and `src/wf_mcp/transparent_proxy/runtime.py`: project proxy admin tools from the same `wf.admin` capability definitions when admin MCP exposure is enabled.
|
||||
- Modify `src/wf_mcp/shared/names.py`: move admin namespace toward `wf.admin` and use `LdaNamespace` where dotted names should be preserved.
|
||||
- Modify `tests/wf_mcp/test_service.py`, `tests/wf_mcp/test_broker_server.py`, and `tests/wf_mcp/test_transparent_proxy.py`: prove projections and defaults.
|
||||
- Modify `docs/wf_mcp_capability_sources.md`: update once implementation names are final.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Canonical Capability Source Model
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/broker/service/capability_sources.py`
|
||||
- Modify: `src/wf_mcp/broker/service/sources.py`
|
||||
- Test: `tests/wf_mcp/test_service.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for source visibility and buckets**
|
||||
|
||||
Add tests in `tests/wf_mcp/test_service.py`:
|
||||
|
||||
```python
|
||||
def test_service_sources_have_visibility_and_capability_buckets() -> None:
|
||||
service = WfMcpService(store=FileStore(local_temp_root() / "source_shape_store"))
|
||||
|
||||
std_source = service.capability_sources["wf.std"]
|
||||
mcp_source = service.capability_sources["wf.mcp"]
|
||||
|
||||
assert std_source.id == "wf.std"
|
||||
assert std_source.kind == "system"
|
||||
assert std_source.visibility.planner is True
|
||||
assert std_source.visibility.mcp_client is True
|
||||
assert std_source.visibility.admin_dashboard is True
|
||||
assert "wf.std.runtime_error" in std_source.capabilities.node_specs
|
||||
assert std_source.capabilities.tools == {}
|
||||
|
||||
assert mcp_source.id == "wf.mcp"
|
||||
assert mcp_source.visibility.planner is True
|
||||
assert mcp_source.permissions.calls_upstream is True
|
||||
assert "wf.mcp.call_tool" in mcp_source.capabilities.node_specs
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the focused failing test**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_service.py::test_service_sources_have_visibility_and_capability_buckets -q
|
||||
```
|
||||
|
||||
Expected: fail because `capability_sources` does not exist.
|
||||
|
||||
- [ ] **Step 3: Add the source model**
|
||||
|
||||
Create `src/wf_mcp/broker/service/capability_sources.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Literal
|
||||
|
||||
from wf_authoring import NodeSpec
|
||||
|
||||
SourceKind = Literal["system", "connection"]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SourceVisibility:
|
||||
planner: bool = False
|
||||
mcp_client: bool = False
|
||||
admin_dashboard: bool = True
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SourcePermissions:
|
||||
safe_for_workflow: bool = False
|
||||
calls_upstream: bool = False
|
||||
mutates_config: bool = False
|
||||
mutates_auth: bool = False
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class CapabilityBuckets:
|
||||
tools: dict[str, Any] = field(default_factory=dict)
|
||||
node_specs: dict[str, NodeSpec[Any, Any]] = field(default_factory=dict)
|
||||
prompts: dict[str, Any] = field(default_factory=dict)
|
||||
resources: dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class CapabilitySource:
|
||||
id: str
|
||||
kind: SourceKind
|
||||
capabilities: CapabilityBuckets = field(default_factory=CapabilityBuckets)
|
||||
enabled: bool = True
|
||||
visibility: SourceVisibility = field(default_factory=SourceVisibility)
|
||||
permissions: SourcePermissions = field(default_factory=SourcePermissions)
|
||||
description: str | None = None
|
||||
|
||||
def as_status(self) -> dict[str, Any]:
|
||||
return {
|
||||
"id": self.id,
|
||||
"kind": self.kind,
|
||||
"enabled": self.enabled,
|
||||
"visibility": {
|
||||
"planner": self.visibility.planner,
|
||||
"mcp_client": self.visibility.mcp_client,
|
||||
"admin_dashboard": self.visibility.admin_dashboard,
|
||||
},
|
||||
"permissions": {
|
||||
"safe_for_workflow": self.permissions.safe_for_workflow,
|
||||
"calls_upstream": self.permissions.calls_upstream,
|
||||
"mutates_config": self.permissions.mutates_config,
|
||||
"mutates_auth": self.permissions.mutates_auth,
|
||||
},
|
||||
"description": self.description,
|
||||
"tool_count": len(self.capabilities.tools),
|
||||
"node_spec_count": len(self.capabilities.node_specs),
|
||||
"prompt_count": len(self.capabilities.prompts),
|
||||
"resource_count": len(self.capabilities.resources),
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Keep `SpecSource` as compatibility wrapper**
|
||||
|
||||
Modify `src/wf_mcp/broker/service/sources.py` so existing code can still import `SpecSource` while new code can move to `CapabilitySource`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from wf_authoring import NodeSpec
|
||||
|
||||
from .capability_sources import (
|
||||
CapabilityBuckets,
|
||||
CapabilitySource,
|
||||
SourceKind,
|
||||
SourcePermissions,
|
||||
SourceVisibility,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class SpecSource:
|
||||
id: str
|
||||
kind: SourceKind
|
||||
specs: dict[str, NodeSpec[Any, Any]] = field(default_factory=dict)
|
||||
visible: bool = True
|
||||
description: str | None = None
|
||||
|
||||
def as_capability_source(self) -> CapabilitySource:
|
||||
return CapabilitySource(
|
||||
id=self.id,
|
||||
kind=self.kind,
|
||||
capabilities=CapabilityBuckets(node_specs=self.specs),
|
||||
visibility=SourceVisibility(
|
||||
planner=self.visible,
|
||||
mcp_client=False,
|
||||
admin_dashboard=True,
|
||||
),
|
||||
permissions=SourcePermissions(safe_for_workflow=self.kind == "system"),
|
||||
description=self.description,
|
||||
)
|
||||
|
||||
def as_status(self) -> dict[str, Any]:
|
||||
return self.as_capability_source().as_status()
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run focused test**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_service.py::test_service_sources_have_visibility_and_capability_buckets -q
|
||||
```
|
||||
|
||||
Expected: fail until service stores `capability_sources`; pass after Task 2.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Make `WfMcpService` Store Capability Sources
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Modify: `src/wf_mcp/broker/service/specs.py`
|
||||
- Test: `tests/wf_mcp/test_service.py`
|
||||
|
||||
- [ ] **Step 1: Add failing service projection tests**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def test_service_spec_views_are_derived_from_capability_sources() -> None:
|
||||
service = WfMcpService(store=FileStore(local_temp_root() / "source_view_store"))
|
||||
|
||||
assert "wf.std" in service.capability_sources
|
||||
assert "wf.std" in service.spec_sources
|
||||
assert "wf.std" in service.specs_by_connection
|
||||
assert (
|
||||
service.specs_by_connection["wf.std"]["wf.std.runtime_error"]
|
||||
is service.capability_sources["wf.std"].capabilities.node_specs[
|
||||
"wf.std.runtime_error"
|
||||
]
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run focused test**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_service.py::test_service_spec_views_are_derived_from_capability_sources -q
|
||||
```
|
||||
|
||||
Expected: fail because compatibility views are not derived from capability sources.
|
||||
|
||||
- [ ] **Step 3: Update service fields and registration**
|
||||
|
||||
In `src/wf_mcp/broker/service/core.py`, replace the stored `spec_sources` field with canonical capability storage and add derived views:
|
||||
|
||||
```python
|
||||
capability_sources: dict[str, CapabilitySource] = field(default_factory=dict)
|
||||
|
||||
@property
|
||||
def spec_sources(self) -> dict[str, SpecSource]:
|
||||
return {
|
||||
source.id: SpecSource(
|
||||
id=source.id,
|
||||
kind=source.kind,
|
||||
specs=source.capabilities.node_specs,
|
||||
visible=source.enabled and source.visibility.planner,
|
||||
description=source.description,
|
||||
)
|
||||
for source in self.capability_sources.values()
|
||||
if source.capabilities.node_specs
|
||||
}
|
||||
|
||||
@property
|
||||
def specs_by_connection(self) -> dict[str, dict[str, NodeSpec[Any, Any]]]:
|
||||
return {
|
||||
source.id: source.capabilities.node_specs
|
||||
for source in self.capability_sources.values()
|
||||
if source.capabilities.node_specs
|
||||
}
|
||||
|
||||
def register_capability_source(self, source: CapabilitySource) -> None:
|
||||
self.capability_sources[source.id] = source
|
||||
|
||||
def register_spec_source(self, source: SpecSource) -> None:
|
||||
self.register_capability_source(source.as_capability_source())
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update spec resolution**
|
||||
|
||||
In `src/wf_mcp/broker/service/specs.py`, make `get_qualified_spec` read capability sources:
|
||||
|
||||
```python
|
||||
from collections.abc import Mapping
|
||||
|
||||
from .capability_sources import CapabilitySource
|
||||
|
||||
|
||||
def get_qualified_spec(
|
||||
sources: Mapping[str, CapabilitySource],
|
||||
qualified_name: str,
|
||||
) -> NodeSpec[Any, Any]:
|
||||
source_id, _ = qualified_name.rsplit(".", 1)
|
||||
source = sources.get(source_id)
|
||||
if (
|
||||
source is None
|
||||
or not source.enabled
|
||||
or qualified_name not in source.capabilities.node_specs
|
||||
):
|
||||
raise KeyError(f"unknown qualified node {qualified_name!r}")
|
||||
return source.capabilities.node_specs[qualified_name]
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run focused service tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_service.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Move All Authoring Ops Into `wf.std`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/builtins.py`
|
||||
- Test: `tests/wf_mcp/test_service.py`
|
||||
|
||||
- [ ] **Step 1: Add failing `wf.std` inventory test**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def test_wf_std_source_contains_authoring_ops() -> None:
|
||||
service = WfMcpService(store=FileStore(local_temp_root() / "stdlib_source_store"))
|
||||
specs = service.capability_sources["wf.std"].capabilities.node_specs
|
||||
|
||||
expected = {
|
||||
"wf.std.coalesce",
|
||||
"wf.std.default_if_none",
|
||||
"wf.std.constant",
|
||||
"wf.std.pick_key",
|
||||
"wf.std.truthy",
|
||||
"wf.std.runtime_error",
|
||||
"wf.std.first_item",
|
||||
"wf.std.first_item_or_none",
|
||||
"wf.std.first_item_maybe",
|
||||
"wf.std.last_item",
|
||||
"wf.std.last_item_or_none",
|
||||
"wf.std.length",
|
||||
"wf.std.is_empty",
|
||||
}
|
||||
assert expected <= set(specs)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run focused test**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_service.py::test_wf_std_source_contains_authoring_ops -q
|
||||
```
|
||||
|
||||
Expected: fail because only `runtime_error` is currently registered.
|
||||
|
||||
- [ ] **Step 3: Register stdlib ops**
|
||||
|
||||
In `src/wf_mcp/broker/service/builtins.py`, import `wf_authoring.ops` symbols and qualify each one under `wf.std`:
|
||||
|
||||
```python
|
||||
from wf_authoring import (
|
||||
coalesce,
|
||||
constant,
|
||||
default_if_none,
|
||||
first_item,
|
||||
first_item_maybe,
|
||||
first_item_or_none,
|
||||
is_empty,
|
||||
last_item,
|
||||
last_item_or_none,
|
||||
length,
|
||||
pick_key,
|
||||
runtime_error,
|
||||
truthy,
|
||||
)
|
||||
|
||||
|
||||
def builtin_specs() -> dict[str, NodeSpec[Any, Any]]:
|
||||
specs = [
|
||||
coalesce,
|
||||
default_if_none,
|
||||
constant,
|
||||
pick_key,
|
||||
truthy,
|
||||
runtime_error,
|
||||
first_item,
|
||||
first_item_or_none,
|
||||
first_item_maybe,
|
||||
last_item,
|
||||
last_item_or_none,
|
||||
length,
|
||||
is_empty,
|
||||
]
|
||||
qualified_specs = [
|
||||
qualify_spec(BUILTIN_CONNECTION_ID, _strip_authoring_prefix(spec))
|
||||
for spec in specs
|
||||
]
|
||||
return {spec.name: spec for spec in qualified_specs}
|
||||
|
||||
|
||||
def _strip_authoring_prefix(spec: NodeSpec[Any, Any]) -> NodeSpec[Any, Any]:
|
||||
name = spec.name.removeprefix("authoring.")
|
||||
return NodeSpec(
|
||||
name=name,
|
||||
input_model=spec.input_model,
|
||||
output_model=spec.output_model,
|
||||
outcomes=spec.outcomes,
|
||||
fn=spec.fn,
|
||||
description=spec.description,
|
||||
is_async=spec.is_async,
|
||||
accepts_context=spec.accepts_context,
|
||||
input_schema_contract=spec.input_schema_contract,
|
||||
output_schema_contract=spec.output_schema_contract,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run service tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_service.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add `wf.admin` Source Without MCP Exposure
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/broker/admin_capabilities.py`
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Test: `tests/wf_mcp/test_service.py`
|
||||
|
||||
- [ ] **Step 1: Add failing admin source test**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def test_wf_admin_source_exists_but_is_not_planner_visible() -> None:
|
||||
service = WfMcpService(store=FileStore(local_temp_root() / "admin_source_store"))
|
||||
source = service.capability_sources["wf.admin"]
|
||||
|
||||
assert source.kind == "system"
|
||||
assert source.visibility.planner is False
|
||||
assert source.visibility.mcp_client is False
|
||||
assert source.visibility.admin_dashboard is True
|
||||
assert source.permissions.mutates_config is True
|
||||
assert "wf.admin.list_sources" in source.capabilities.tools
|
||||
assert "wf.admin.disable_source" in source.capabilities.tools
|
||||
assert "wf.admin" not in service.get_planner_catalog().snapshots
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run focused test**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_service.py::test_wf_admin_source_exists_but_is_not_planner_visible -q
|
||||
```
|
||||
|
||||
Expected: fail because `wf.admin` does not exist.
|
||||
|
||||
- [ ] **Step 3: Define admin capability objects**
|
||||
|
||||
Create `src/wf_mcp/broker/admin_capabilities.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Awaitable, Callable
|
||||
|
||||
from .service.capability_sources import (
|
||||
CapabilityBuckets,
|
||||
CapabilitySource,
|
||||
SourcePermissions,
|
||||
SourceVisibility,
|
||||
)
|
||||
|
||||
ADMIN_SOURCE_ID = "wf.admin"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AdminTool:
|
||||
name: str
|
||||
description: str
|
||||
handler_name: str
|
||||
mutates_config: bool = False
|
||||
mutates_auth: bool = False
|
||||
|
||||
|
||||
def admin_source() -> CapabilitySource:
|
||||
tools: dict[str, AdminTool] = {
|
||||
"wf.admin.list_sources": AdminTool(
|
||||
name="wf.admin.list_sources",
|
||||
description="List broker capability sources.",
|
||||
handler_name="list_sources",
|
||||
),
|
||||
"wf.admin.disable_source": AdminTool(
|
||||
name="wf.admin.disable_source",
|
||||
description="Disable a capability source.",
|
||||
handler_name="disable_source",
|
||||
mutates_config=True,
|
||||
),
|
||||
"wf.admin.enable_source": AdminTool(
|
||||
name="wf.admin.enable_source",
|
||||
description="Enable a capability source.",
|
||||
handler_name="enable_source",
|
||||
mutates_config=True,
|
||||
),
|
||||
}
|
||||
return CapabilitySource(
|
||||
id=ADMIN_SOURCE_ID,
|
||||
kind="system",
|
||||
capabilities=CapabilityBuckets(tools=tools),
|
||||
visibility=SourceVisibility(
|
||||
planner=False,
|
||||
mcp_client=False,
|
||||
admin_dashboard=True,
|
||||
),
|
||||
permissions=SourcePermissions(
|
||||
safe_for_workflow=False,
|
||||
calls_upstream=False,
|
||||
mutates_config=True,
|
||||
mutates_auth=True,
|
||||
),
|
||||
description="Privileged broker administration capabilities.",
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Install admin source**
|
||||
|
||||
In `WfMcpService.__post_init__`, register `admin_source()` after builtin sources:
|
||||
|
||||
```python
|
||||
from ..admin_capabilities import admin_source
|
||||
|
||||
...
|
||||
|
||||
self.register_capability_source(admin_source())
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_service.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Project Broker Admin Tools From `wf.admin`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/tools.py`
|
||||
- Test: `tests/wf_mcp/test_broker_server.py`
|
||||
|
||||
- [ ] **Step 1: Add test proving default broker exposes current public tools but has source-backed metadata**
|
||||
|
||||
Add to `tests/wf_mcp/test_broker_server.py`:
|
||||
|
||||
```python
|
||||
def test_broker_admin_tools_are_backed_by_wf_admin_source() -> None:
|
||||
service = WfMcpService(store=FileStore(local_temp_root() / "broker_admin_source"))
|
||||
server = create_broker_server(service)
|
||||
|
||||
tools = asyncio.run(server.list_tools())
|
||||
tool_names = {tool.name for tool in tools}
|
||||
|
||||
assert "list_spec_sources" in tool_names
|
||||
assert "get_planner_catalog" in tool_names
|
||||
assert "wf.admin.list_sources" in service.capability_sources[
|
||||
"wf.admin"
|
||||
].capabilities.tools
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run focused broker test**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_broker_server.py::test_broker_admin_tools_are_backed_by_wf_admin_source -q
|
||||
```
|
||||
|
||||
Expected: pass after Task 4; fail before Task 4.
|
||||
|
||||
- [ ] **Step 3: Keep current tool names as compatibility exports**
|
||||
|
||||
Do not rename public broker MCP tools in this task. Keep:
|
||||
|
||||
```text
|
||||
list_connections
|
||||
get_connection_statuses
|
||||
refresh_connection_catalog
|
||||
get_catalog
|
||||
get_planner_catalog
|
||||
list_spec_sources
|
||||
read_broker_resource
|
||||
render_broker_prompt
|
||||
invoke_broker_method
|
||||
call_broker_tool
|
||||
get_broker_events
|
||||
```
|
||||
|
||||
Add comments in `src/wf_mcp/broker/tools.py`:
|
||||
|
||||
```python
|
||||
# These MCP tool names are compatibility exports. Their capability metadata
|
||||
# belongs to the wf.admin source; future admin-enabled servers can project
|
||||
# dotted wf.admin.* names from that source.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused broker tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_broker_server.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Normalize Transparent Proxy Admin Naming Strategy
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/shared/names.py`
|
||||
- Modify: `src/wf_mcp/transparent_proxy/runtime.py`
|
||||
- Test: `tests/wf_mcp/test_names.py`
|
||||
- Test: `tests/wf_mcp/test_transparent_proxy.py`
|
||||
|
||||
- [ ] **Step 1: Add naming tests for admin namespace**
|
||||
|
||||
Add to `tests/wf_mcp/test_names.py`:
|
||||
|
||||
```python
|
||||
def test_admin_namespace_is_distinct_from_wf_mcp_runtime_source() -> None:
|
||||
assert ADMIN_NAMESPACE == "wf.admin"
|
||||
assert is_admin_tool_name("wf.admin.list_connections") is True
|
||||
assert is_admin_tool_name("wf.mcp.call_tool") is False
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run focused naming test**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_names.py::test_admin_namespace_is_distinct_from_wf_mcp_runtime_source -q
|
||||
```
|
||||
|
||||
Expected: fail because `ADMIN_NAMESPACE` is currently `wf.mcp`.
|
||||
|
||||
- [ ] **Step 3: Update namespace constants**
|
||||
|
||||
In `src/wf_mcp/shared/names.py`:
|
||||
|
||||
```python
|
||||
ADMIN_NAMESPACE = "wf.admin"
|
||||
|
||||
|
||||
def is_admin_tool_name(proxy_name: str) -> bool:
|
||||
return proxy_name.startswith(f"{ADMIN_NAMESPACE}.") or proxy_name.startswith(
|
||||
f"{ADMIN_NAMESPACE}_"
|
||||
)
|
||||
```
|
||||
|
||||
Use `LdaNamespace(ADMIN_NAMESPACE)` in `src/wf_mcp/transparent_proxy/runtime.py`:
|
||||
|
||||
```python
|
||||
from ..shared.names import ADMIN_NAMESPACE, LdaNamespace
|
||||
|
||||
...
|
||||
|
||||
admin.add_transform(LdaNamespace(ADMIN_NAMESPACE))
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update transparent proxy tests**
|
||||
|
||||
In `tests/wf_mcp/test_transparent_proxy.py`, update expected admin names from `wf.mcp_*` to dotted `wf.admin.*` if `LdaNamespace` preserves dotted names:
|
||||
|
||||
```python
|
||||
assert "wf.admin.list_connections" in names
|
||||
assert "wf.admin.get_connection_statuses" in names
|
||||
assert "wf.admin.list_proxy_tools" in names
|
||||
assert "wf.admin.get_proxy_tool" in names
|
||||
```
|
||||
|
||||
Call tools by the new names:
|
||||
|
||||
```python
|
||||
connections_result = await client.call_tool("wf.admin.list_connections")
|
||||
proxy_tools_result = await client.call_tool("wf.admin.list_proxy_tools")
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run transparent proxy tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest tests\wf_mcp\test_names.py tests\wf_mcp\test_transparent_proxy.py -q
|
||||
```
|
||||
|
||||
Expected: pass if `LdaNamespace` preserves dotted names. If FastMCP still emits underscore names, keep `wf.admin_*` as compatibility and document that dotted projection needs a deeper transform.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Update Docs And Full Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/wf_mcp_capability_sources.md`
|
||||
- Modify: `docs/wf_mcp_architecture.md`
|
||||
- Test: full verification commands
|
||||
|
||||
- [ ] **Step 1: Update docs with implemented names**
|
||||
|
||||
In `docs/wf_mcp_capability_sources.md`, update the migration section to mark implemented pieces:
|
||||
|
||||
```markdown
|
||||
## Implemented Shape
|
||||
|
||||
- `CapabilitySource` owns source metadata and capability buckets.
|
||||
- `wf.std` owns workflow stdlib node specs.
|
||||
- `wf.mcp` owns workflow MCP runtime node specs.
|
||||
- `wf.admin` owns privileged admin capability metadata.
|
||||
- Broker and proxy MCP tool projection remains compatibility-first while the
|
||||
admin projection stabilizes.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run full pytest**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
Expected: all tests pass, live-only tests may skip when env is absent.
|
||||
|
||||
- [ ] **Step 3: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run ruff check src tests examples main.py
|
||||
```
|
||||
|
||||
Expected: `All checks passed!`
|
||||
|
||||
- [ ] **Step 4: Run basedpyright errors**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run basedpyright src tests examples main.py --level error
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
Spec coverage:
|
||||
|
||||
- Capability source model is covered by Tasks 1 and 2.
|
||||
- `wf.std` migration is covered by Task 3.
|
||||
- `wf.admin` privileged source is covered by Task 4.
|
||||
- Broker projection is covered by Task 5.
|
||||
- Transparent proxy naming and admin namespace separation is covered by Task 6.
|
||||
- Documentation and verification are covered by Task 7.
|
||||
|
||||
Known deliberate scope limits:
|
||||
|
||||
- Task 5 keeps current broker MCP public tool names as compatibility exports.
|
||||
- Task 6 attempts dotted transparent-proxy admin names via `LdaNamespace`; if FastMCP still forces underscore naming, this plan keeps `wf.admin_*` compatibility and defers deeper transform work.
|
||||
- Source enable/disable runtime behavior is not implemented in this plan beyond introducing `enabled`; it should be the next plan after the registry shape lands.
|
||||
|
||||
Placeholder scan:
|
||||
|
||||
- No `TBD`, `TODO`, or unspecified implementation steps remain.
|
||||
- Each task includes exact paths, test names, commands, and expected outcomes.
|
||||
|
||||
Type consistency:
|
||||
|
||||
- `CapabilitySource`, `CapabilityBuckets`, `SourceVisibility`, and
|
||||
`SourcePermissions` are introduced once and reused consistently.
|
||||
- Compatibility views are named `spec_sources` and `specs_by_connection`.
|
||||
@@ -0,0 +1,436 @@
|
||||
# Workflow Artifacts V1 Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add the first `wf_artifacts` package with immutable workflow artifact models, deployment bindings, dependency diagnostics, and a file-backed artifact store.
|
||||
|
||||
**Architecture:** `wf_artifacts` is a separate layer above `wf_core` and below future platform/MCP projections. It stores declarative workflow plans and dependency metadata, but does not execute workflows or import `wf_mcp`.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2 models, `pathlib`, JSON file storage, pytest.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create `src/wf_artifacts/__init__.py`: public facade for artifact models and stores.
|
||||
- Create `src/wf_artifacts/models.py`: Pydantic models for artifacts, deployments, required capabilities, diagnostics, and drift policy.
|
||||
- Create `src/wf_artifacts/store.py`: `WorkflowArtifactStore` protocol-like base class and `FileWorkflowArtifactStore`.
|
||||
- Create `tests/artifacts/test_models.py`: serialization and validation tests for model shapes.
|
||||
- Create `tests/artifacts/test_store.py`: file-store round-trip and latest-version tests.
|
||||
|
||||
## Task 1: Artifact Models
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_artifacts/models.py`
|
||||
- Create: `src/wf_artifacts/__init__.py`
|
||||
- Test: `tests/artifacts/test_models.py`
|
||||
|
||||
- [ ] **Step 1: Write failing model tests**
|
||||
|
||||
```python
|
||||
from wf_artifacts import (
|
||||
DependencyDiagnostic,
|
||||
DiagnosticSeverity,
|
||||
DriftPolicy,
|
||||
RequiredCapability,
|
||||
WorkflowArtifact,
|
||||
WorkflowDeployment,
|
||||
)
|
||||
|
||||
|
||||
def test_workflow_artifact_serializes_required_capability_contract() -> None:
|
||||
capability = RequiredCapability(
|
||||
logical_source="context7",
|
||||
capability_name="query-docs",
|
||||
kind="tool",
|
||||
input_schema_hash="sha256:input",
|
||||
input_schema_snapshot={"type": "object", "properties": {}},
|
||||
output_schema_hash="sha256:output",
|
||||
output_schema_snapshot={"type": "object", "properties": {}},
|
||||
observed_concrete_source="context7.default",
|
||||
observed_at_epoch_ms=123,
|
||||
)
|
||||
artifact = WorkflowArtifact(
|
||||
id="summarize_docs",
|
||||
version=1,
|
||||
title="Summarize Docs",
|
||||
description="Summarize retrieved documentation.",
|
||||
input_schema={"type": "object", "properties": {}},
|
||||
output_schema={"type": "object", "properties": {}},
|
||||
outcomes=("done", "failed"),
|
||||
plan={"name": "summarize_docs", "nodes": [], "edges": []},
|
||||
required_capabilities={"context7.query-docs": capability},
|
||||
created_from_catalog_version="catalog-1",
|
||||
)
|
||||
|
||||
dumped = artifact.model_dump(mode="json")
|
||||
|
||||
assert dumped["id"] == "summarize_docs"
|
||||
assert dumped["version"] == 1
|
||||
assert dumped["outcomes"] == ["done", "failed"]
|
||||
assert dumped["required_capabilities"]["context7.query-docs"]["logical_source"] == "context7"
|
||||
assert dumped["required_capabilities"]["context7.query-docs"]["input_schema_hash"] == "sha256:input"
|
||||
|
||||
|
||||
def test_workflow_deployment_binds_logical_sources_to_concrete_sources() -> None:
|
||||
deployment = WorkflowDeployment(
|
||||
id="summarize_docs.personal",
|
||||
artifact_id="summarize_docs",
|
||||
artifact_version=1,
|
||||
bindings={"context7": "context7.personal"},
|
||||
drift_policy=DriftPolicy.BLOCK,
|
||||
)
|
||||
|
||||
dumped = deployment.model_dump(mode="json")
|
||||
|
||||
assert dumped["id"] == "summarize_docs.personal"
|
||||
assert dumped["artifact_id"] == "summarize_docs"
|
||||
assert dumped["artifact_version"] == 1
|
||||
assert dumped["bindings"]["context7"] == "context7.personal"
|
||||
assert dumped["drift_policy"] == "block"
|
||||
|
||||
|
||||
def test_dependency_diagnostic_is_structured() -> None:
|
||||
diagnostic = DependencyDiagnostic(
|
||||
severity=DiagnosticSeverity.ERROR,
|
||||
code="capability_missing",
|
||||
logical_ref="context7.query-docs",
|
||||
bound_source="context7.default",
|
||||
message="Bound source no longer exposes query-docs.",
|
||||
repair_hint="Refresh catalog or bind context7 to another compatible source.",
|
||||
)
|
||||
|
||||
dumped = diagnostic.model_dump(mode="json")
|
||||
|
||||
assert dumped["severity"] == "error"
|
||||
assert dumped["code"] == "capability_missing"
|
||||
assert dumped["logical_ref"] == "context7.query-docs"
|
||||
assert dumped["bound_source"] == "context7.default"
|
||||
assert dumped["repair_hint"].startswith("Refresh catalog")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run model tests to verify RED**
|
||||
|
||||
Run: `uv run --with pytest pytest tests\artifacts\test_models.py -q`
|
||||
|
||||
Expected: fail with `ModuleNotFoundError: No module named 'wf_artifacts'`.
|
||||
|
||||
- [ ] **Step 3: Implement minimal models**
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import StrEnum
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
JsonObject = dict[str, Any]
|
||||
|
||||
|
||||
class DriftPolicy(StrEnum):
|
||||
BLOCK = "block"
|
||||
WARN = "warn"
|
||||
ALLOW = "allow"
|
||||
|
||||
|
||||
class DiagnosticSeverity(StrEnum):
|
||||
ERROR = "error"
|
||||
WARNING = "warning"
|
||||
|
||||
|
||||
class RequiredCapability(BaseModel):
|
||||
logical_source: str
|
||||
capability_name: str
|
||||
kind: Literal["tool", "resource", "prompt", "node_spec", "workflow"]
|
||||
input_schema_hash: str | None = None
|
||||
input_schema_snapshot: JsonObject | None = None
|
||||
output_schema_hash: str | None = None
|
||||
output_schema_snapshot: JsonObject | None = None
|
||||
observed_concrete_source: str | None = None
|
||||
observed_at_epoch_ms: int | None = Field(default=None, ge=0)
|
||||
|
||||
|
||||
class DependencyDiagnostic(BaseModel):
|
||||
severity: DiagnosticSeverity
|
||||
code: str
|
||||
logical_ref: str
|
||||
bound_source: str | None = None
|
||||
message: str
|
||||
repair_hint: str | None = None
|
||||
|
||||
|
||||
class WorkflowArtifact(BaseModel):
|
||||
id: str
|
||||
version: int = Field(ge=1)
|
||||
title: str
|
||||
description: str | None = None
|
||||
input_schema: JsonObject
|
||||
output_schema: JsonObject
|
||||
outcomes: tuple[str, ...]
|
||||
plan: JsonObject
|
||||
required_capabilities: dict[str, RequiredCapability] = Field(default_factory=dict)
|
||||
workflow_dependencies: dict[str, int] = Field(default_factory=dict)
|
||||
created_from_catalog_version: str | None = None
|
||||
|
||||
|
||||
class WorkflowDeployment(BaseModel):
|
||||
id: str
|
||||
artifact_id: str
|
||||
artifact_version: int = Field(ge=1)
|
||||
bindings: dict[str, str] = Field(default_factory=dict)
|
||||
drift_policy: DriftPolicy = DriftPolicy.BLOCK
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Export models**
|
||||
|
||||
```python
|
||||
from .models import (
|
||||
DependencyDiagnostic,
|
||||
DiagnosticSeverity,
|
||||
DriftPolicy,
|
||||
RequiredCapability,
|
||||
WorkflowArtifact,
|
||||
WorkflowDeployment,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"DependencyDiagnostic",
|
||||
"DiagnosticSeverity",
|
||||
"DriftPolicy",
|
||||
"RequiredCapability",
|
||||
"WorkflowArtifact",
|
||||
"WorkflowDeployment",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run model tests to verify GREEN**
|
||||
|
||||
Run: `uv run --with pytest pytest tests\artifacts\test_models.py -q`
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 2: File Artifact Store
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/store.py`
|
||||
- Modify: `src/wf_artifacts/__init__.py`
|
||||
- Test: `tests/artifacts/test_store.py`
|
||||
|
||||
- [ ] **Step 1: Write failing store tests**
|
||||
|
||||
```python
|
||||
from wf_artifacts import (
|
||||
FileWorkflowArtifactStore,
|
||||
WorkflowArtifact,
|
||||
WorkflowDeployment,
|
||||
)
|
||||
|
||||
|
||||
def artifact(version: int) -> WorkflowArtifact:
|
||||
return WorkflowArtifact(
|
||||
id="summarize_docs",
|
||||
version=version,
|
||||
title=f"Summarize Docs v{version}",
|
||||
description=None,
|
||||
input_schema={"type": "object", "properties": {}},
|
||||
output_schema={"type": "object", "properties": {}},
|
||||
outcomes=("done",),
|
||||
plan={"name": "summarize_docs", "nodes": [], "edges": []},
|
||||
)
|
||||
|
||||
|
||||
def test_file_store_round_trips_artifact_versions(tmp_path) -> None:
|
||||
store = FileWorkflowArtifactStore(tmp_path)
|
||||
store.save_artifact(artifact(1))
|
||||
store.save_artifact(artifact(2))
|
||||
|
||||
loaded = store.get_artifact("summarize_docs", 2)
|
||||
|
||||
assert loaded.id == "summarize_docs"
|
||||
assert loaded.version == 2
|
||||
assert loaded.title == "Summarize Docs v2"
|
||||
|
||||
|
||||
def test_file_store_resolves_latest_artifact_version(tmp_path) -> None:
|
||||
store = FileWorkflowArtifactStore(tmp_path)
|
||||
store.save_artifact(artifact(1))
|
||||
store.save_artifact(artifact(3))
|
||||
store.save_artifact(artifact(2))
|
||||
|
||||
latest = store.resolve_latest("summarize_docs")
|
||||
|
||||
assert latest.id == "summarize_docs"
|
||||
assert latest.version == 3
|
||||
|
||||
|
||||
def test_file_store_round_trips_deployment(tmp_path) -> None:
|
||||
store = FileWorkflowArtifactStore(tmp_path)
|
||||
deployment = WorkflowDeployment(
|
||||
id="summarize_docs.personal",
|
||||
artifact_id="summarize_docs",
|
||||
artifact_version=1,
|
||||
bindings={"context7": "context7.personal"},
|
||||
)
|
||||
|
||||
store.save_deployment(deployment)
|
||||
loaded = store.get_deployment("summarize_docs.personal")
|
||||
|
||||
assert loaded.id == "summarize_docs.personal"
|
||||
assert loaded.artifact_id == "summarize_docs"
|
||||
assert loaded.bindings["context7"] == "context7.personal"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run store tests to verify RED**
|
||||
|
||||
Run: `uv run --with pytest pytest tests\artifacts\test_store.py -q`
|
||||
|
||||
Expected: fail importing `FileWorkflowArtifactStore`.
|
||||
|
||||
- [ ] **Step 3: Implement file store**
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from .models import WorkflowArtifact, WorkflowDeployment
|
||||
|
||||
|
||||
class WorkflowArtifactStore:
|
||||
def save_artifact(self, artifact: WorkflowArtifact) -> None:
|
||||
raise NotImplementedError
|
||||
|
||||
def get_artifact(self, artifact_id: str, version: int) -> WorkflowArtifact:
|
||||
raise NotImplementedError
|
||||
|
||||
def list_artifacts(self) -> list[WorkflowArtifact]:
|
||||
raise NotImplementedError
|
||||
|
||||
def resolve_latest(self, artifact_id: str) -> WorkflowArtifact:
|
||||
raise NotImplementedError
|
||||
|
||||
def save_deployment(self, deployment: WorkflowDeployment) -> None:
|
||||
raise NotImplementedError
|
||||
|
||||
def get_deployment(self, deployment_id: str) -> WorkflowDeployment:
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
class FileWorkflowArtifactStore(WorkflowArtifactStore):
|
||||
def __init__(self, root: Path) -> None:
|
||||
self.root = root
|
||||
self.artifacts_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.deployments_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@property
|
||||
def artifacts_dir(self) -> Path:
|
||||
return self.root / "workflows"
|
||||
|
||||
@property
|
||||
def deployments_dir(self) -> Path:
|
||||
return self.root / "deployments"
|
||||
|
||||
def save_artifact(self, artifact: WorkflowArtifact) -> None:
|
||||
artifact_dir = self.artifacts_dir / artifact.id
|
||||
artifact_dir.mkdir(parents=True, exist_ok=True)
|
||||
path = artifact_dir / f"{artifact.version}.json"
|
||||
path.write_text(
|
||||
json.dumps(artifact.model_dump(mode="json"), indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
def get_artifact(self, artifact_id: str, version: int) -> WorkflowArtifact:
|
||||
path = self.artifacts_dir / artifact_id / f"{version}.json"
|
||||
if not path.exists():
|
||||
raise KeyError(f"unknown workflow artifact {artifact_id}@{version}")
|
||||
return WorkflowArtifact.model_validate_json(path.read_text(encoding="utf-8"))
|
||||
|
||||
def list_artifacts(self) -> list[WorkflowArtifact]:
|
||||
artifacts: list[WorkflowArtifact] = []
|
||||
for path in sorted(self.artifacts_dir.glob("*/*.json")):
|
||||
artifacts.append(
|
||||
WorkflowArtifact.model_validate_json(path.read_text(encoding="utf-8"))
|
||||
)
|
||||
return artifacts
|
||||
|
||||
def resolve_latest(self, artifact_id: str) -> WorkflowArtifact:
|
||||
versions = [
|
||||
int(path.stem)
|
||||
for path in (self.artifacts_dir / artifact_id).glob("*.json")
|
||||
if path.stem.isdecimal()
|
||||
]
|
||||
if not versions:
|
||||
raise KeyError(f"unknown workflow artifact {artifact_id!r}")
|
||||
return self.get_artifact(artifact_id, max(versions))
|
||||
|
||||
def save_deployment(self, deployment: WorkflowDeployment) -> None:
|
||||
path = self.deployments_dir / f"{deployment.id}.json"
|
||||
path.write_text(
|
||||
json.dumps(deployment.model_dump(mode="json"), indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
def get_deployment(self, deployment_id: str) -> WorkflowDeployment:
|
||||
path = self.deployments_dir / f"{deployment_id}.json"
|
||||
if not path.exists():
|
||||
raise KeyError(f"unknown workflow deployment {deployment_id!r}")
|
||||
return WorkflowDeployment.model_validate_json(path.read_text(encoding="utf-8"))
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Export store types**
|
||||
|
||||
```python
|
||||
from .store import FileWorkflowArtifactStore, WorkflowArtifactStore
|
||||
|
||||
__all__ = [
|
||||
...
|
||||
"FileWorkflowArtifactStore",
|
||||
"WorkflowArtifactStore",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run store tests to verify GREEN**
|
||||
|
||||
Run: `uv run --with pytest pytest tests\artifacts -q`
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 3: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- No production changes unless verification exposes issues.
|
||||
|
||||
- [ ] **Step 1: Run focused artifact tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests\artifacts -q`
|
||||
|
||||
Expected: all artifact tests pass.
|
||||
|
||||
- [ ] **Step 2: Run full test suite**
|
||||
|
||||
Run: `uv run --with pytest pytest -q`
|
||||
|
||||
Expected: existing tests and new artifact tests pass.
|
||||
|
||||
- [ ] **Step 3: Run lint/type checks**
|
||||
|
||||
Run: `uv run ruff check src tests examples main.py`
|
||||
|
||||
Expected: no lint errors.
|
||||
|
||||
Run: `uv run basedpyright src tests examples main.py --level error`
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: implements the first artifact slice only: immutable models, deployments, dependency contracts, diagnostics, and file storage.
|
||||
- Intentional gaps: no workflow execution, no dependency validation engine, no MCP projection, no native subgraph runtime.
|
||||
- Placeholder scan: no TBD/TODO placeholders remain.
|
||||
- Type consistency: tests and code use the same model names and fields.
|
||||
@@ -0,0 +1,454 @@
|
||||
# Unified MCP Surface And Protocol Proxy Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Converge broker mode and transparent proxy mode into one MCP server surface that supports direct upstream capability projection, stable workflow/admin tools, protocol-level proxying, and local broker notifications.
|
||||
|
||||
**Architecture:** Use one service/config/store layer and explicit projection layers. The unified server should expose upstream tools/resources/prompts directly when enabled, expose stable local workflow/admin tools, and route bidirectional MCP protocol features through well-named proxy components rather than ad hoc tool wrappers.
|
||||
|
||||
**Tech Stack:** FastMCP 3.x, MCP Python SDK, existing `WfMcpService`, transparent proxy runtime, capability source registry, pytest, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Why This Plan Exists
|
||||
|
||||
The current project has two partial MCP surfaces:
|
||||
|
||||
- Broker mode exposes stable control/workflow tools such as
|
||||
`create_workflow_artifact_from_plan`, `validate_workflow_deployment`, and
|
||||
`run_workflow_deployment`.
|
||||
- Transparent proxy mode exposes upstream capabilities directly through MCP
|
||||
`tools/list` and `tools/call`.
|
||||
|
||||
That split is not enough. A real MCP proxy has more than tools:
|
||||
|
||||
- resources
|
||||
- prompts
|
||||
- tool calls
|
||||
- notifications
|
||||
- progress
|
||||
- logging
|
||||
- resource subscriptions
|
||||
- elicitation
|
||||
- sampling
|
||||
- tasks
|
||||
- capability/list-changed events
|
||||
|
||||
The unified server must not be a "tools-only proxy." It needs a protocol plan
|
||||
that decides what is pass-through, what is projected, what is local, what is
|
||||
unsupported, and what needs explicit client/server capability negotiation.
|
||||
|
||||
## Definitions
|
||||
|
||||
### Upstream Server
|
||||
|
||||
An MCP server configured as a connection, such as `everything.default`,
|
||||
`context7.default`, or `serena.default`.
|
||||
|
||||
### Downstream Client
|
||||
|
||||
The MCP client connected to our server, such as Codex or MCP Inspector.
|
||||
|
||||
### Local Capability
|
||||
|
||||
A capability implemented by this project, such as `wf.workflow.run_deployment`
|
||||
or `wf.admin.list_connections`.
|
||||
|
||||
### Proxy Capability
|
||||
|
||||
A capability discovered from an upstream server and projected through our
|
||||
server, such as `everything.default_echo`.
|
||||
|
||||
### Protocol Proxy
|
||||
|
||||
Bidirectional routing for MCP protocol messages that are not ordinary
|
||||
tools/resources/prompts list/read/get calls. Examples: sampling requests,
|
||||
elicitation requests, progress notifications, task status notifications, and
|
||||
resource update notifications.
|
||||
|
||||
## Target Surface
|
||||
|
||||
One MCP server instance should expose both proxy capabilities and local
|
||||
capabilities.
|
||||
|
||||
```text
|
||||
upstream tools:
|
||||
everything.default_echo
|
||||
context7.default_query-docs
|
||||
|
||||
upstream resources:
|
||||
everything.default.instructions.md
|
||||
|
||||
upstream prompts:
|
||||
everything.default.simple-prompt
|
||||
|
||||
stable workflow tools:
|
||||
wf.workflow.list_artifacts
|
||||
wf.workflow.create_artifact_from_plan
|
||||
wf.workflow.save_deployment
|
||||
wf.workflow.validate_deployment
|
||||
wf.workflow.run_deployment
|
||||
|
||||
admin tools, only when explicitly enabled:
|
||||
wf.admin.list_connections
|
||||
wf.admin.refresh_connection_catalog
|
||||
wf.admin.list_proxy_tools
|
||||
wf.admin.reload_config
|
||||
```
|
||||
|
||||
Compatibility broker tool names such as `get_planner_catalog` can remain during
|
||||
migration, but namespaced tools should become the recommended interface.
|
||||
|
||||
## Protocol Coverage Matrix
|
||||
|
||||
| MCP Feature | Near-Term Behavior | Long-Term Behavior | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `tools/list` | Project upstream tools plus local workflow/admin tools | Same, with pagination/search | Already partially supported in transparent proxy and broker separately. |
|
||||
| `tools/call` | Forward upstream tool calls and execute local tools | Same, with tasks/progress support | Local tools should share handlers across modes. |
|
||||
| `resources/list` | Project upstream resources | Same, plus local docs/resources | Resources-as-tools remains optional compatibility. |
|
||||
| `resources/read` | Forward upstream reads | Same, with subscriptions | Needs namespacing and URI/local-name mapping. |
|
||||
| `prompts/list` | Project upstream prompts | Same, plus local authoring manuals | Prompts-as-tools remains optional compatibility. |
|
||||
| `prompts/get` | Forward upstream prompt rendering | Same | Must preserve arguments and metadata. |
|
||||
| `notifications/progress` | Forward where the SDK/server surface supports it | First-class run/proxy progress bus | Local workflow runs should emit progress later. |
|
||||
| `notifications/resources/updated` | Not reliable yet | Proxy subscriptions with lifecycle tracking | Requires subscription ownership and reload behavior. |
|
||||
| `notifications/resources/list_changed` | Emit local changed events after refresh/reload if supported | Also forward upstream changes | Needed when tools/resources/prompts change. |
|
||||
| `notifications/tools/list_changed` | Emit after config reload/catalog refresh if supported | Same | Important for clients that refresh tools. |
|
||||
| `notifications/prompts/list_changed` | Emit after prompt catalog changes if supported | Same | Same shape as tool/resource changed. |
|
||||
| `notifications/message` / logging | Proxy upstream logging where supported | Add local broker logging notifications | Everything server has logging examples. |
|
||||
| Elicitation | Do not fake it as a tool | Route upstream elicitation to downstream client | Requires bidirectional request routing and capability checks. |
|
||||
| Sampling | Do not fake it as a tool | Route upstream sampling to downstream client | Requires downstream client sampling support. |
|
||||
| Tasks | Do not invent custom primary API | Use MCP Tasks for long-running runs where supported | Custom `start_run` only as compatibility fallback. |
|
||||
| Ping | Support local ping and keep upstream health separately | Same | Upstream ping should be a health/admin operation, not necessarily forwarded blindly. |
|
||||
|
||||
## Current Risks
|
||||
|
||||
- FastMCP mount/unmount lifecycle is not complete enough for safe dynamic
|
||||
unmount of all proxied capabilities.
|
||||
- Transparent reload is currently best-effort.
|
||||
- Bidirectional upstream requests such as elicitation/sampling require access to
|
||||
the downstream client session, not just an upstream SDK client.
|
||||
- Notifications must be scoped: a resource update from `everything.default`
|
||||
should not look like a local broker config update.
|
||||
- Clients vary. Codex may not immediately refresh tool lists. Inspector may show
|
||||
more protocol features. The proxy must be robust even when clients ignore
|
||||
optional notifications.
|
||||
|
||||
## Required Boundaries
|
||||
|
||||
### Shared Service Layer
|
||||
|
||||
`WfMcpService` or a sibling service owns:
|
||||
|
||||
- configured connections
|
||||
- adapters
|
||||
- auth/catalog stores
|
||||
- capability sources
|
||||
- workflow artifact store
|
||||
- events
|
||||
|
||||
It should not own FastMCP decorators directly.
|
||||
|
||||
### Projection Layer
|
||||
|
||||
Projection modules register MCP-visible capabilities:
|
||||
|
||||
- upstream tools/resources/prompts
|
||||
- local workflow tools
|
||||
- local admin tools
|
||||
- local docs/prompts/resources
|
||||
|
||||
Projection modules call shared handlers. They should not contain business logic.
|
||||
|
||||
### Protocol Routing Layer
|
||||
|
||||
Protocol routing handles bidirectional features:
|
||||
|
||||
- upstream-to-downstream elicitation
|
||||
- upstream-to-downstream sampling
|
||||
- upstream/local notifications
|
||||
- tasks/progress
|
||||
- subscriptions
|
||||
|
||||
This layer should be explicit. Do not bury protocol routing inside a generic
|
||||
`call_tool` helper.
|
||||
|
||||
### Event/Notification Bus
|
||||
|
||||
Local events should be emitted once and then projected to:
|
||||
|
||||
- stored broker events
|
||||
- MCP notifications where the client supports them
|
||||
- future UI/dashboard streams
|
||||
|
||||
Examples:
|
||||
|
||||
```text
|
||||
connection_registered
|
||||
catalog_refresh_started
|
||||
catalog_refresh_completed
|
||||
tool_call_started
|
||||
tool_call_completed
|
||||
workflow_artifact_saved
|
||||
workflow_deployment_saved
|
||||
workflow_run_started
|
||||
workflow_run_progress
|
||||
workflow_run_completed
|
||||
source_enabled
|
||||
source_disabled
|
||||
config_reloaded
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Complete `2026-05-12-workflow-artifact-hardening.md`.
|
||||
- Keep artifact operations in shared callable functions, not duplicated
|
||||
FastMCP decorators.
|
||||
- Keep config mutation/admin operations behind explicit admin exposure.
|
||||
- Document client capability assumptions for elicitation, sampling, tasks, and
|
||||
notifications.
|
||||
- Add tests against the fixture MCP server and the everything server where
|
||||
practical.
|
||||
|
||||
## Phase 1: Inventory And Adapter Reality Check
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `docs/mcp_protocol_proxy_inventory.md`
|
||||
- Inspect: `src/wf_mcp/sdk/adapter.py`
|
||||
- Inspect: `src/wf_mcp/transparent_proxy/runtime.py`
|
||||
- Inspect: `src/wf_mcp/broker/artifact_tools.py`
|
||||
- Test: no new tests required in this phase
|
||||
|
||||
- [ ] List which MCP SDK APIs are currently wrapped by `BackendAdapter`.
|
||||
- [ ] List which FastMCP server APIs we currently use.
|
||||
- [ ] List which features everything-server exposes that we can test:
|
||||
- progress
|
||||
- logging
|
||||
- resource updates
|
||||
- elicitation
|
||||
- sampling
|
||||
- tasks
|
||||
- [ ] Identify SDK gaps before implementation. If an MCP feature is not
|
||||
accessible through FastMCP/MCP SDK at our current version, document it instead
|
||||
of inventing a fake abstraction.
|
||||
|
||||
## Phase 2: Extract Shared Workflow/Admin Handlers
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Create: `src/wf_mcp/admin_surface/handlers.py`
|
||||
- Modify: `src/wf_mcp/broker/artifact_tools.py`
|
||||
- Modify: `src/wf_mcp/broker/tools.py`
|
||||
- Modify: `src/wf_mcp/transparent_proxy/admin.py`
|
||||
- Test: `tests/wf_mcp/test_broker_server.py`
|
||||
- Test: `tests/wf_mcp/test_transparent_proxy.py`
|
||||
|
||||
- [ ] Move workflow artifact list/save/inspect/validate/run logic into shared
|
||||
handler functions/classes.
|
||||
- [x] Move admin list/refresh/config/reload logic into shared handler
|
||||
functions/classes.
|
||||
- [x] Keep broker compatibility tool names working.
|
||||
- [x] Keep transparent proxy admin tool names working.
|
||||
- [ ] Do not change behavior in this phase; only remove duplicated logic and
|
||||
create a single implementation path.
|
||||
|
||||
## Phase 3: Unified Server Factory
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/server/unified.py`
|
||||
- Modify: `src/wf_mcp/cli.py`
|
||||
- Modify: `src/wf_mcp/broker/server.py`
|
||||
- Test: `tests/wf_mcp/test_unified_server.py`
|
||||
|
||||
- [x] Build one FastMCP server from `BrokerConfig`.
|
||||
- [ ] Register local workflow tools with namespaced names:
|
||||
- [x] `wf.workflow.list_artifacts`
|
||||
- [x] `wf.workflow.create_artifact_from_plan`
|
||||
- [x] `wf.workflow.save_artifact`
|
||||
- [x] `wf.workflow.list_deployments`
|
||||
- [x] `wf.workflow.save_deployment`
|
||||
- [x] `wf.workflow.validate_deployment`
|
||||
- [x] `wf.workflow.run_deployment`
|
||||
- [x] Register admin tools only when admin exposure is enabled.
|
||||
- [x] Project upstream tools using the transparent proxy path.
|
||||
- [x] Keep existing `broker` and `proxy` CLI modes during migration.
|
||||
- [x] Add `unified` CLI mode.
|
||||
- [ ] Do not make unified default until manual Inspector/Codex tests pass.
|
||||
|
||||
## Phase 4: Namespacing And Collision Policy
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/shared/names.py`
|
||||
- Test: `tests/wf_mcp/test_names.py`
|
||||
- Test: `tests/wf_mcp/test_unified_server.py`
|
||||
|
||||
- [ ] Use `wf.workflow.*` for stable workflow tools.
|
||||
- [ ] Use `wf.admin.*` for privileged admin/control tools.
|
||||
- [ ] Keep `wf.mcp.*` for workflow runtime helpers, not admin.
|
||||
- [ ] Keep upstream proxy names collision-safe.
|
||||
- [ ] Reject configured connection ids that collide with reserved local
|
||||
namespaces.
|
||||
- [ ] Decide whether compatibility broker names remain visible by default in
|
||||
unified mode. Recommended: yes during migration, no after migration.
|
||||
|
||||
## Phase 5: Tool/Resource/Prompt Projection Parity
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/transparent_proxy/runtime.py`
|
||||
- Modify: unified server files from Phase 3.
|
||||
- Test: `tests/wf_mcp/test_unified_server.py`
|
||||
|
||||
- [ ] Ensure upstream tools and stable workflow/admin tools both appear in
|
||||
`tools/list`.
|
||||
- [ ] Ensure upstream resources appear in `resources/list` and can be read.
|
||||
- [ ] Ensure upstream prompts appear in `prompts/list` and can be rendered.
|
||||
- [ ] Ensure resources-as-tools and prompts-as-tools remain optional projection
|
||||
modes, not the only way to access resources/prompts.
|
||||
- [ ] Ensure search/pagination includes stable local tools and upstream tools.
|
||||
|
||||
## Phase 6: Local Notification Bus
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/events/bus.py`
|
||||
- Modify: `src/wf_mcp/broker/events.py`
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Modify: unified server files from Phase 3.
|
||||
- Test: `tests/wf_mcp/test_events.py`
|
||||
|
||||
- [x] Introduce an in-process event bus abstraction.
|
||||
- [x] Keep the existing stored `McpEvent` list as one subscriber/sink.
|
||||
- [ ] Add event kinds for workflow artifacts and deployments:
|
||||
- [x] `workflow_artifact_saved`
|
||||
- [x] `workflow_deployment_saved`
|
||||
- [x] `workflow_run_started`
|
||||
- [x] `workflow_run_completed`
|
||||
- `workflow_run_failed`
|
||||
- [ ] Add event kinds for capability changes:
|
||||
- `source_enabled`
|
||||
- `source_disabled`
|
||||
- [x] `catalog_changed`
|
||||
- [x] `tools_changed`
|
||||
- [x] `resources_changed`
|
||||
- [x] `prompts_changed`
|
||||
- [ ] Do not emit MCP notifications yet unless the server/session API is
|
||||
clearly available. This phase creates the source of truth.
|
||||
|
||||
## Phase 7: MCP Notifications
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: unified server files from Phase 3.
|
||||
- Modify: `src/wf_mcp/events/bus.py`
|
||||
- Test: `tests/wf_mcp/test_unified_server.py`
|
||||
|
||||
- [ ] Emit MCP list-changed notifications when local or upstream catalogs
|
||||
change, if supported:
|
||||
- `notifications/tools/list_changed`
|
||||
- `notifications/resources/list_changed`
|
||||
- `notifications/prompts/list_changed`
|
||||
- [ ] Emit local workflow progress notifications where supported.
|
||||
- [ ] Proxy upstream logging notifications where supported.
|
||||
- [ ] Ensure clients that ignore notifications can still poll/list manually.
|
||||
- [ ] Add tests that assert notifications are requested/emitted through whatever
|
||||
FastMCP/MCP SDK surface is available. If no testable surface exists, document
|
||||
the limitation in `docs/mcp_protocol_proxy_inventory.md`.
|
||||
|
||||
## Phase 8: Elicitation And Sampling Routing
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/protocol/elicitation.py`
|
||||
- Create: `src/wf_mcp/protocol/sampling.py`
|
||||
- Modify: SDK adapter/session layer if supported.
|
||||
- Test: `tests/wf_mcp/test_protocol_proxy.py`
|
||||
|
||||
- [ ] Determine how upstream MCP SDK exposes server-to-client elicitation
|
||||
requests.
|
||||
- [ ] Determine how FastMCP exposes downstream client elicitation responses.
|
||||
- [ ] Route upstream elicitation requests to the downstream client only when the
|
||||
downstream client advertised support.
|
||||
- [ ] Route upstream sampling requests to the downstream client only when the
|
||||
downstream client advertised support.
|
||||
- [ ] Preserve request ids/correlation ids so responses return to the correct
|
||||
upstream session.
|
||||
- [ ] Return structured unsupported diagnostics when routing is impossible.
|
||||
- [ ] Do not convert elicitation/sampling into normal tools as the primary
|
||||
behavior.
|
||||
|
||||
## Phase 9: Tasks And Long-Running Workflow Runs
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/workflow_surface/runs.py`
|
||||
- Modify: unified server files from Phase 3.
|
||||
- Test: `tests/wf_mcp/test_workflow_tasks.py`
|
||||
|
||||
- [ ] Prefer MCP Tasks for long-running `wf.workflow.run_deployment` when the
|
||||
client/server support task execution.
|
||||
- [ ] Keep synchronous run behavior for short/manual local tests.
|
||||
- [ ] Add a compatibility run store only if MCP Tasks are unavailable or
|
||||
insufficient for Codex/Inspector.
|
||||
- [ ] Map workflow interrupts to task status such as `input_required` only after
|
||||
the runtime supports the needed resume model.
|
||||
- [ ] Do not implement durable scheduling/cron here.
|
||||
|
||||
## Phase 10: Mode Migration
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/wf_mcp_architecture.md`
|
||||
- Modify: `docs/wf_mcp_capability_sources.md`
|
||||
- Modify: `src/wf_mcp/cli.py`
|
||||
- Test: `tests/wf_mcp/test_cli.py`
|
||||
|
||||
- [ ] Document unified mode as the recommended local mode once it passes manual
|
||||
Codex and Inspector checks.
|
||||
- [ ] Keep broker/proxy modes as compatibility modes.
|
||||
- [ ] Mark compatibility broker tool names as legacy once namespaced local tools
|
||||
work in unified mode.
|
||||
- [ ] Do not delete compatibility modes until tests cover every important
|
||||
surface.
|
||||
|
||||
> **Superseded on 2026-05-16:** the compatibility period is now considered long
|
||||
> enough. The current plan is to retire the public broker/proxy mode split in
|
||||
> [`2026-05-16-retire-legacy-mcp-modes.md`](2026-05-16-retire-legacy-mcp-modes.md)
|
||||
> while keeping the useful internal implementation boundaries.
|
||||
|
||||
## Manual Verification Checklist
|
||||
|
||||
- [ ] Codex can list upstream tools.
|
||||
- [ ] Codex can call `everything.default_echo` or equivalent upstream tool.
|
||||
- [ ] Codex can call `wf.workflow.list_artifacts`.
|
||||
- [ ] Codex can create an artifact from a plan.
|
||||
- [ ] Codex can save a deployment.
|
||||
- [ ] Codex can validate and run a deployment.
|
||||
- [ ] Inspector shows upstream resources and prompts.
|
||||
- [ ] Inspector shows local workflow tools with useful names/descriptions.
|
||||
- [ ] If everything-server elicitation is triggered, the proxy either routes it
|
||||
correctly or returns a clear unsupported diagnostic.
|
||||
- [ ] If everything-server progress/logging is triggered, the proxy either
|
||||
forwards it correctly or documents why not.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- No native `wf_core` subgraph implementation.
|
||||
- No UI/dashboard implementation.
|
||||
- No cron/scheduler implementation.
|
||||
- No hidden conversion of every protocol feature into a tool.
|
||||
- No pretending unsupported protocol features are proxied.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
The project can run one local MCP server that gives an LLM client both:
|
||||
|
||||
- direct access to upstream MCP capabilities
|
||||
- stable workflow/admin capabilities implemented by this project
|
||||
|
||||
The implementation must be explicit about unsupported protocol features and must
|
||||
have a path to proxy elicitation, sampling, notifications, and tasks without
|
||||
duplicating mode-specific logic.
|
||||
@@ -0,0 +1,119 @@
|
||||
# Workflow Artifact Hardening Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make saved workflow artifacts safe enough to rely on before merging MCP server modes or exposing workflow execution more broadly.
|
||||
|
||||
**Architecture:** Keep `wf_artifacts` provider-neutral. `wf_mcp` may project artifact operations, but validation, storage, artifact creation, and diagnostics should stay in `wf_artifacts` where possible.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, pytest, `wf_core.Workflow` validation, existing `wf_mcp` broker service.
|
||||
|
||||
---
|
||||
|
||||
## Why This Comes First
|
||||
|
||||
The live Codex probe proved the artifact execution path works, but it also found
|
||||
a real weakness: `create_workflow_artifact_from_plan` saved an invalid
|
||||
`state_schema` shape, and the error surfaced only when `run_workflow_deployment`
|
||||
tried to compile the workflow.
|
||||
|
||||
Before unifying broker and transparent proxy modes, artifact creation should
|
||||
reject invalid plans earlier and return structured diagnostics.
|
||||
|
||||
## Scope
|
||||
|
||||
This plan hardens saved workflow creation and dependency validation. It does not
|
||||
merge MCP server modes and does not add native subgraphs.
|
||||
|
||||
## Tasks
|
||||
|
||||
### Task 1: Validate Plan Shape During Artifact Creation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/factory.py`
|
||||
- Test: `tests/artifacts/test_factory.py`
|
||||
|
||||
- [ ] Add a failing test proving `create_workflow_artifact_from_plan` rejects a
|
||||
plan that cannot become a `wf_core.Workflow`.
|
||||
- [ ] Implement validation by constructing `wf_core.Workflow.model_validate`
|
||||
from the plan fields.
|
||||
- [ ] Keep the validation dependency one-way: `wf_artifacts` may import
|
||||
`wf_core`, but `wf_core` must not import `wf_artifacts`.
|
||||
- [ ] Return `ValueError` with a concise message containing the failing field
|
||||
path or Pydantic error message.
|
||||
- [ ] Run `uv run --with pytest pytest tests\artifacts\test_factory.py -q`.
|
||||
|
||||
### Task 2: Add Artifact Creation Diagnostics
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/models.py`
|
||||
- Modify: `src/wf_artifacts/factory.py`
|
||||
- Test: `tests/artifacts/test_factory.py`
|
||||
|
||||
- [ ] Decide whether creation failures should raise exceptions only or also
|
||||
expose a `validate_workflow_artifact_plan(...) -> list[DependencyDiagnostic]`
|
||||
style function.
|
||||
- [ ] Recommended v1: add a separate `validate_workflow_artifact_plan(plan)`
|
||||
that returns structured diagnostics, while the factory still raises on errors.
|
||||
- [ ] Add tests for missing `input_schema`, missing `output_schema`, invalid
|
||||
state schema, and missing start node.
|
||||
|
||||
### Task 3: Validate Direct Workflow Dependencies
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/validation.py`
|
||||
- Test: `tests/artifacts/test_validation.py`
|
||||
|
||||
- [ ] Add artifact-store-aware validation for `workflow_dependencies`.
|
||||
- [ ] Validate exact artifact-version pins.
|
||||
- [ ] Return `dependency_missing` when a child artifact version does not exist.
|
||||
- [ ] Return `dependency_cycle` when direct/transitive workflow artifact
|
||||
dependencies cycle.
|
||||
- [ ] Do not copy child dependency snapshots into the parent.
|
||||
|
||||
### Task 4: Improve Capability Contract Checking
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/validation.py`
|
||||
- Test: `tests/artifacts/test_validation.py`
|
||||
- Optional: `src/wf_mcp/broker/artifact_tools.py`
|
||||
|
||||
- [ ] Preserve current hash comparison behavior.
|
||||
- [ ] Add tests for kind mismatch, for example required `tool` but available
|
||||
`node_spec`.
|
||||
- [ ] Decide whether `node_spec` can satisfy `tool` when the provider is an MCP
|
||||
wrapper. Recommended: no implicit kind coercion in `wf_artifacts`; adapters
|
||||
should present the available kind they mean to expose.
|
||||
- [ ] Add diagnostics with `code="capability_kind_mismatch"`.
|
||||
|
||||
### Task 5: Document Current Runtime Limitations In Tool Responses
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/artifact_tools.py`
|
||||
- Test: `tests/wf_mcp/test_broker_server.py`
|
||||
|
||||
- [ ] Keep interrupting artifacts rejected by `run_workflow_deployment`.
|
||||
- [ ] Add `repair_hint` text explaining native subgraphs/nested resume are not
|
||||
implemented yet.
|
||||
- [ ] Add a test proving unsupported interrupt artifacts return a diagnostic
|
||||
instead of raising.
|
||||
|
||||
## Verification
|
||||
|
||||
- [ ] `uv run --with pytest pytest tests\artifacts -q`
|
||||
- [ ] `uv run --with pytest pytest tests\wf_mcp\test_broker_server.py -q`
|
||||
- [ ] `uv run --with pytest pytest -q`
|
||||
- [ ] `uv run ruff check src tests examples main.py`
|
||||
- [ ] `uv run basedpyright src tests examples main.py --level error`
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- No native `wf_core` subgraph implementation.
|
||||
- No MCP Tasks implementation.
|
||||
- No run history persistence.
|
||||
- No broker/transparent mode merge in this plan.
|
||||
@@ -0,0 +1,144 @@
|
||||
# Retire Legacy MCP Modes Plan
|
||||
|
||||
> **Status:** completed retirement pass; kept as the decision record.
|
||||
|
||||
## Goal
|
||||
|
||||
Make one public MCP server surface exposed by `wf-mcp`.
|
||||
|
||||
`broker` mode and `proxy` mode were useful while the unified surface was being
|
||||
built, but keeping all three as public launch modes now creates the wrong mental
|
||||
model:
|
||||
|
||||
- broker mode suggests local workflow/admin tools are a separate product surface
|
||||
- proxy mode suggests upstream projection is a separate product surface
|
||||
- the combined server is already the real target: one server that exposes local
|
||||
capabilities and proxied upstream capabilities together
|
||||
|
||||
The public product should have one server mode. The implementation may still
|
||||
have multiple internal concern packages.
|
||||
|
||||
## Existing Documentation
|
||||
|
||||
This replaces the migration stance in
|
||||
[`2026-05-12-unified-mcp-surface.md`](2026-05-12-unified-mcp-surface.md),
|
||||
which said to keep broker/proxy as compatibility modes until coverage existed.
|
||||
That compatibility period is now considered complete enough to end.
|
||||
|
||||
Related documents already cover adjacent plans:
|
||||
|
||||
- [`../../wf_mcp_architecture.md`](../../wf_mcp_architecture.md)
|
||||
- concern boundaries and proxy mount lifecycle
|
||||
- [`../../wf_mcp_capability_sources.md`](../../wf_mcp_capability_sources.md)
|
||||
- source model, admin/workflow exposure, source inventory
|
||||
- [`../../workflow_artifacts.md`](../../workflow_artifacts.md)
|
||||
- artifacts, deployments, stable workflow control surface
|
||||
- [`../../wf_mcp_proxy_reality_and_roadmap.md`](../../wf_mcp_proxy_reality_and_roadmap.md)
|
||||
- what the proxy does well today and what should remain upstream-dependent
|
||||
- [`../../mcp_protocol_proxy_inventory.md`](../../mcp_protocol_proxy_inventory.md)
|
||||
- observed protocol behavior and relay gaps
|
||||
|
||||
No other major active plan was found to be undocumented while preparing this
|
||||
pass.
|
||||
|
||||
## What Gets Retired
|
||||
|
||||
### Public CLI modes
|
||||
|
||||
Remove the public `serve --mode broker` and `serve --mode proxy` choices.
|
||||
|
||||
After this pass:
|
||||
|
||||
- `wf-mcp serve` runs the server surface
|
||||
- users no longer choose among three product modes
|
||||
- docs should describe one server behavior, not a mode matrix
|
||||
|
||||
### Public framing
|
||||
|
||||
Stop presenting broker/proxy as user-facing alternatives in docs and help text.
|
||||
Where historical explanation is useful, call them legacy migration surfaces.
|
||||
|
||||
### Compatibility-only tests and docs
|
||||
|
||||
Delete or rewrite tests whose only purpose is to prove the old public mode split.
|
||||
Keep behavior tests for the underlying capabilities when those behaviors still
|
||||
exist through the public server.
|
||||
|
||||
## What Stays
|
||||
|
||||
### Internal concern packages
|
||||
|
||||
Do **not** flatten the codebase just because the public mode split disappears.
|
||||
These packages still represent useful implementation boundaries:
|
||||
|
||||
- `wf_mcp.broker`
|
||||
- `wf_mcp.transparent_proxy`
|
||||
- `wf_mcp.server`
|
||||
|
||||
`transparent_proxy` is already partly a legacy package name, but the code inside
|
||||
it still owns real proxy-mounting mechanics used by the server. Rename or
|
||||
re-home that code only as a later cleanup if the package name becomes a real
|
||||
source of confusion.
|
||||
|
||||
### Shared services
|
||||
|
||||
Keep the service/config/store/runtime objects that the public server already uses.
|
||||
This pass is about removing duplicate **entrypoints**, not rewriting the
|
||||
underlying architecture.
|
||||
|
||||
### Stable local capability names
|
||||
|
||||
Keep the source model and namespaces:
|
||||
|
||||
- `wf.workflow.*`
|
||||
- `wf.admin.*`
|
||||
- `wf.std.*`
|
||||
- `wf.mcp.*`
|
||||
- `<connection_id>.*`
|
||||
|
||||
The cleanup should reduce surfaces, not churn the capability vocabulary.
|
||||
|
||||
## Expected Code Changes
|
||||
|
||||
1. Simplify CLI mode selection so `serve` has one public behavior.
|
||||
2. Remove or privatize old broker/proxy server launch functions that only exist
|
||||
for the retired public modes.
|
||||
3. Collapse docs/help text that still describe three user-facing modes.
|
||||
4. Keep implementation reuse through the existing server path.
|
||||
5. Update tests so they assert server behavior directly instead of branching on
|
||||
legacy mode names.
|
||||
|
||||
## Non-Goals For This Pass
|
||||
|
||||
- No full rewrite of the proxy subsystem.
|
||||
- No attempt to solve generic upstream notification relay.
|
||||
- No safe unmount implementation for retired FastMCP providers.
|
||||
- No renaming of every legacy internal package just to match the new public
|
||||
shape.
|
||||
- No workflow artifact redesign.
|
||||
|
||||
Those topics already have separate docs and should stay separate.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- `wf-mcp serve` has one public MCP server behavior.
|
||||
- No public docs imply that broker/proxy are still supported product modes.
|
||||
- The server continues to expose:
|
||||
- proxied upstream capabilities
|
||||
- stable local workflow tools
|
||||
- optional admin tools
|
||||
- The test suite passes with the old public mode split removed.
|
||||
- Remaining roadmap docs still point to the real unresolved work instead of
|
||||
making the reader rediscover why the combined server exists.
|
||||
|
||||
## Follow-On Work After This Pass
|
||||
|
||||
Once the public surface is singular, the next useful cleanup is not more mode
|
||||
work. It is easier-to-explain capability projection:
|
||||
|
||||
1. keep improving source inventory and admin visibility
|
||||
2. continue documenting proxy relay limitations explicitly
|
||||
3. build workflow-facing wrapper artifacts on top of stable sources
|
||||
|
||||
That keeps the system moving toward a clean platform without pretending the
|
||||
proxy layer is already a perfect MCP relay.
|
||||
@@ -0,0 +1,68 @@
|
||||
# Call Wrapper Artifacts Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Let `wf.workflow.call_capability` execute saved `WorkflowArtifact(kind="wrapper")` artifacts by their stable artifact node name without turning arbitrary saved workflows into node capabilities.
|
||||
|
||||
**Architecture:** Keep live `NodeSpec` execution unchanged. Extend the workflow surface resolution path so `workflow.<artifact_id>.v<version>` can resolve to a saved wrapper artifact, validate that it is wrapper-kind and interrupt-free, execute its stored plan through the existing workflow runner, and normalize the final workflow result into the same `qualified_name` / `outcome` / `output` payload shape as live capabilities.
|
||||
|
||||
**Tech Stack:** Python, Pydantic, `wf_artifacts`, `wf_core`, pytest.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Pin Wrapper-Artifact Call Semantics
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/test_service.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Add a test that saves a wrapper artifact with a simple one-node plan, calls `WorkflowSurfaceHandlers.call_capability()` with `workflow.<id>.v<version>`, and asserts the returned `qualified_name`, `outcome`, and `output`.
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/wf_mcp/test_service.py -q`
|
||||
|
||||
Expected: FAIL because `call_capability()` only resolves live specs today.
|
||||
|
||||
### Task 2: Resolve Wrapper Artifacts in the Workflow Surface
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
|
||||
- [ ] **Step 1: Implement minimal wrapper-artifact resolution**
|
||||
|
||||
Add a small helper that:
|
||||
|
||||
- recognizes stable artifact node names
|
||||
- loads the artifact from the store
|
||||
- rejects non-wrapper artifacts
|
||||
- rejects unsupported interrupt plans
|
||||
- executes the artifact plan with the existing workflow runner
|
||||
- converts the workflow run into the `call_capability` response payload
|
||||
|
||||
- [ ] **Step 2: Run focused tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/wf_mcp/test_service.py -q`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 3: Verify the Whole Project
|
||||
|
||||
**Files:**
|
||||
|
||||
- No additional files.
|
||||
|
||||
- [ ] **Step 1: Run focused workflow-surface tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/wf_mcp -q`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 2: Run the full suite**
|
||||
|
||||
Run: `uv run --with pytest pytest -q`
|
||||
|
||||
Expected: PASS.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Nested Authoring State Projection Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Let `wf_authoring` project nested authored state schemas into the flat exact-path state-field index that `wf_core` now supports.
|
||||
|
||||
**Architecture:** Keep authored JSON Schema nested for users and LLM clients. Add a focused flattening helper for state-field projection only, emitting both parent object paths and descendant paths. Resolve nested `BaseModel` metadata by authored path where available, while leaving non-`BaseModel` authored types schema-capable with default merge metadata.
|
||||
|
||||
**Tech Stack:** Python, Pydantic, pytest, existing `wf_authoring` schema adapter.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_authoring/schemas.py`
|
||||
- flatten nested schema properties into exact-path `StateField`s
|
||||
- gather nested `BaseModel` metadata by authored path
|
||||
- Modify `tests/authoring/helpers.py`
|
||||
- add nested state models used by tests
|
||||
- Modify `tests/authoring/test_schemas.py`
|
||||
- pin nested projection behavior and nested metadata
|
||||
- Update `docs/core_state_mapping_and_merge.md`
|
||||
- note that authoring now projects nested authored models into the flat core index
|
||||
|
||||
## Tasks
|
||||
|
||||
### Task 1: Pin Nested Projection
|
||||
|
||||
- [ ] Add tests proving:
|
||||
- nested authored state keeps parent and child declarations
|
||||
- nested child metadata such as `append` lands on the exact child path
|
||||
- parent object declaration remains independent from child declarations
|
||||
- [ ] Run the focused authoring tests and confirm they fail under current top-level-only projection.
|
||||
|
||||
### Task 2: Implement Projection Helpers
|
||||
|
||||
- [ ] Add a schema-walking helper that yields `(path, property_schema)` for parent and descendant properties.
|
||||
- [ ] Add nested `BaseModel` metadata traversal keyed by dotted path.
|
||||
- [ ] Update `state_schema_from()` to build `StateField`s from the flattened path stream.
|
||||
- [ ] Keep JSON Schema generation unchanged; flatten only the core `StateSchema.fields` index.
|
||||
- [ ] Run the focused authoring tests and confirm they pass.
|
||||
|
||||
### Task 3: Document and Verify
|
||||
|
||||
- [ ] Update the core state mapping doc with the authoring projection rule.
|
||||
- [ ] Run `uv run --with pytest pytest tests/authoring -q`
|
||||
- [ ] Run `uv run --with pytest pytest -q`
|
||||
- [ ] Run `uv run basedpyright --level error`
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- custom metadata support for every Pydantic-supported type form
|
||||
- changing `SchemaRef` shape
|
||||
- reducer registries
|
||||
- automatic deep merge behavior
|
||||
@@ -0,0 +1,325 @@
|
||||
# Nested Node-Local Mappings Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Let workflow maps address nested node-local input/output paths while preserving explicit state-mediated data flow and preparing state writes for future reducer libraries.
|
||||
|
||||
**Architecture:** Keep graph-facing paths unchanged. Add a small node-local path helper layer, validate non-overlapping write targets, construct nested node inputs from `in_map`, resolve nested node outputs from `out_map`, and refactor state writes into a prepared patch commit. Extract built-in merge-rule application behind a focused reducer-like module so future named reducer libraries can replace the dispatch seam without changing patch semantics.
|
||||
|
||||
**Tech Stack:** Python, Pydantic, pytest, existing `wf_core` path/state runtime.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_core/paths.py`
|
||||
- keep graph-path helpers
|
||||
- add reusable path-overlap utility if it belongs at the generic path layer
|
||||
- Create `src/wf_core/local_paths.py`
|
||||
- node-local dotted path parsing
|
||||
- nested get/set for node-local payloads
|
||||
- overlap checks for write targets
|
||||
- Create `src/wf_core/runtime/ops/merges.py`
|
||||
- built-in exact-path merge-rule implementations
|
||||
- future seam for named reducer registry
|
||||
- Modify `src/wf_core/runtime/ops/state.py`
|
||||
- replace per-write mutation loop with prepared patch commit
|
||||
- delegate merge-rule application to `merges.py`
|
||||
- Modify `src/wf_core/runtime/ops/nodes.py`
|
||||
- build nested node inputs from `in_map`
|
||||
- resolve nested node outputs into state patch writes
|
||||
- Modify `src/wf_core/validation/steps.py`
|
||||
- validate node-local top-level roots
|
||||
- reject overlapping write targets
|
||||
- Add/modify tests under `tests/core/`
|
||||
- nested `in_map`
|
||||
- nested `out_map`
|
||||
- whole-object mapping still works
|
||||
- overlapping write targets rejected
|
||||
- missing nested output path fails
|
||||
- merge dispatch remains behaviorally unchanged
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Pin Node-Local Path Behavior
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/core/test_validation.py`
|
||||
- Modify: `tests/core/test_runtime.py`
|
||||
|
||||
- [ ] **Step 1: Add failing validation tests**
|
||||
|
||||
Cover:
|
||||
|
||||
```python
|
||||
def test_validation_allows_nested_node_local_paths() -> None:
|
||||
...
|
||||
|
||||
|
||||
def test_validation_rejects_overlapping_node_input_destinations() -> None:
|
||||
...
|
||||
|
||||
|
||||
def test_validation_rejects_overlapping_state_write_destinations() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
Expected rules:
|
||||
|
||||
- `state.person.name -> user.name` is valid when `user` exists in the node input schema
|
||||
- `user -> state.person` and `user.name -> state.person.name` in one `out_map` is invalid because destination state paths overlap
|
||||
- `state.person -> user` plus `state.person.name -> user.name` in one `in_map` is invalid because destination node-local paths overlap
|
||||
|
||||
- [ ] **Step 2: Add failing runtime tests**
|
||||
|
||||
Cover:
|
||||
|
||||
```python
|
||||
def test_runtime_builds_nested_node_input_from_in_map() -> None:
|
||||
...
|
||||
|
||||
|
||||
def test_runtime_reads_nested_node_output_from_out_map() -> None:
|
||||
...
|
||||
|
||||
|
||||
def test_runtime_missing_nested_node_output_path_fails() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run focused tests and confirm failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_validation.py tests/core/test_runtime.py -q
|
||||
```
|
||||
|
||||
Expected: FAIL because node-local map sides are top-level-only today.
|
||||
|
||||
### Task 2: Add Node-Local Path Helpers
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_core/local_paths.py`
|
||||
- Modify: `src/wf_core/validation/steps.py`
|
||||
|
||||
- [ ] **Step 1: Add minimal helper API**
|
||||
|
||||
Implement:
|
||||
|
||||
```python
|
||||
def split_local_path(path: str) -> list[str]: ...
|
||||
def get_local_value(payload: Mapping[str, Any], path: str) -> Any: ...
|
||||
def set_local_value(payload: dict[str, Any], path: str, value: Any) -> None: ...
|
||||
def paths_overlap(left: str, right: str) -> bool: ...
|
||||
def has_overlapping_paths(paths: Iterable[str]) -> bool: ...
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- dotted local paths only
|
||||
- no empty segments
|
||||
- overlap means same path or ancestor/descendant path
|
||||
|
||||
- [ ] **Step 2: Update validation**
|
||||
|
||||
Use node-local path roots for schema checks:
|
||||
|
||||
```python
|
||||
input_root = split_local_path(destination_path)[0]
|
||||
output_root = split_local_path(source_path)[0]
|
||||
```
|
||||
|
||||
Reject:
|
||||
|
||||
- overlapping `in_map` destination local paths
|
||||
- overlapping `out_map` destination state paths
|
||||
|
||||
- [ ] **Step 3: Run focused validation tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_validation.py -q
|
||||
```
|
||||
|
||||
Expected: PASS for validation-specific cases.
|
||||
|
||||
### Task 3: Execute Nested Local Mappings
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||
|
||||
- [ ] **Step 1: Build nested node inputs**
|
||||
|
||||
Replace flat input construction with:
|
||||
|
||||
```python
|
||||
resolved_input: dict[str, Any] = {}
|
||||
for source_path, destination_path in node.in_map.items():
|
||||
value = safe_resolve_path(...)
|
||||
set_local_value(resolved_input, destination_path, value)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Resolve nested node outputs**
|
||||
|
||||
When preparing mapped output writes, use `get_local_value()` for each `out_map`
|
||||
source path instead of indexing only top-level output keys.
|
||||
|
||||
- [ ] **Step 3: Preserve missing-path failures**
|
||||
|
||||
Raise `WorkflowExecutionError` when a mapped nested output path is missing.
|
||||
|
||||
- [ ] **Step 4: Run focused runtime tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_runtime.py -q
|
||||
```
|
||||
|
||||
Expected: PASS for nested mapping behavior.
|
||||
|
||||
### Task 4: Introduce Prepared Patch Commits and Extract Merge Dispatch
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_core/runtime/ops/merges.py`
|
||||
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||
- Modify: `tests/core/test_state_ops.py`
|
||||
|
||||
- [ ] **Step 1: Add failing patch-level tests**
|
||||
|
||||
Cover:
|
||||
|
||||
```python
|
||||
def test_state_patch_rejects_overlapping_destinations_before_mutation() -> None:
|
||||
...
|
||||
|
||||
|
||||
def test_builtin_merge_rules_preserve_existing_behavior() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Extract built-in merge implementations**
|
||||
|
||||
Move the current strategy body out of `write_state_value()` into focused helpers:
|
||||
|
||||
```python
|
||||
def apply_builtin_merge(
|
||||
*,
|
||||
strategy: str,
|
||||
current_value: Any,
|
||||
incoming_value: Any,
|
||||
destination_path: str,
|
||||
) -> Any: ...
|
||||
```
|
||||
|
||||
Keep current semantics:
|
||||
|
||||
- `replace`
|
||||
- `append`
|
||||
- shallow `merge_object`
|
||||
|
||||
Add a docstring that this is the future seam for source-owned named reducers,
|
||||
not custom reducer support yet.
|
||||
|
||||
- [ ] **Step 3: Prepare full write sets before mutation**
|
||||
|
||||
Refactor output mapping so it:
|
||||
|
||||
1. resolves all mapped output values
|
||||
2. validates destination overlap
|
||||
3. prepares the patch
|
||||
4. applies merge behavior
|
||||
|
||||
No state changes should occur before all mapped output paths are known-good.
|
||||
|
||||
- [ ] **Step 4: Run state/runtime tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_state_ops.py tests/core/test_runtime.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 5: Keep Authoring and Docs Aligned
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_authoring/builder/mapping.py`
|
||||
- Modify: `docs/core_state_mapping_and_merge.md` if implementation details differ
|
||||
- Modify: `docs/scratchpad.md` only if wording drift appears
|
||||
- Modify/Add: authoring tests as needed
|
||||
|
||||
- [ ] **Step 1: Confirm authoring auto-maps remain top-level**
|
||||
|
||||
Automatic maps should stay conservative unless there is an explicit reason to
|
||||
infer nested paths. The new feature is for explicit maps first.
|
||||
|
||||
- [ ] **Step 2: Add one authoring regression**
|
||||
|
||||
Prove that a builder can compile a workflow using explicit nested local map
|
||||
paths without extra helper nodes.
|
||||
|
||||
- [ ] **Step 3: Update docs only for implementation drift**
|
||||
|
||||
The design doc already states the target behavior. Keep docs in sync with final
|
||||
names and module boundaries, but do not broaden scope into nested state
|
||||
declarations yet.
|
||||
|
||||
### Task 6: Verify the Whole Project
|
||||
|
||||
**Files:**
|
||||
|
||||
- No additional files.
|
||||
|
||||
- [ ] **Step 1: Run focused suites**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core tests/authoring -q
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the full suite**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run type checking**
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- tests pass
|
||||
- any remaining basedpyright failures are called out explicitly if they come
|
||||
from existing generated/build/doc-fixture noise rather than this work
|
||||
|
||||
---
|
||||
|
||||
## Deliberate Non-Goals
|
||||
|
||||
- nested declared state merge metadata
|
||||
- reducer capability registry
|
||||
- deep merge behavior
|
||||
- native subgraphs
|
||||
- parallel foreach
|
||||
- automatic inference of nested maps from schemas
|
||||
|
||||
## Follow-On Plans
|
||||
|
||||
After this lands:
|
||||
|
||||
1. nested declared state paths with exact-path merge lookup
|
||||
2. reducer capability model / registry seam
|
||||
3. native subgraph design on the same map + patch boundary
|
||||
4. async-only parallel foreach using patch combination rules
|
||||
@@ -0,0 +1,61 @@
|
||||
# Nested State Paths Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Let `wf_core` declare nested workflow state paths and apply merge behavior by exact destination path.
|
||||
|
||||
**Architecture:** Keep state declarations internally flat and path-keyed. Continue allowing undeclared state paths, but only exact declared paths receive typed merge behavior; parent declarations do not implicitly govern descendants. Reuse the state patch boundary added in Phase 1 and change only schema wording, validation coverage, runtime lookup, and docs.
|
||||
|
||||
**Tech Stack:** Python, Pydantic, pytest, existing `wf_core` state runtime.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_core/models/schemas.py`
|
||||
- clarify that `StateField` / `StateSchema` are path-keyed, not root-only
|
||||
- Modify `src/wf_core/runtime/ops/state.py`
|
||||
- resolve merge metadata by exact written state path
|
||||
- Add `tests/core/test_nested_state_paths.py`
|
||||
- exact nested path merge behavior
|
||||
- ancestor declarations do not govern descendants
|
||||
- undeclared nested paths still replace
|
||||
- Update `docs/core_state_mapping_and_merge.md`
|
||||
- mark nested declared state paths as implemented
|
||||
- Update `docs/schema_validation.md`
|
||||
- clarify that runtime state merge metadata is now exact-path capable
|
||||
|
||||
### Task 1: Pin Exact-Path State Behavior
|
||||
|
||||
- [ ] Add failing tests proving:
|
||||
- `state.person.tags` uses a declaration for `"person.tags"` with `append`
|
||||
- a declaration for `"person"` does not cause `state.person.tags` to inherit `merge_object`
|
||||
- undeclared `state.person.tags` defaults to `replace`
|
||||
- [ ] Run the focused tests and confirm the nested exact-path cases fail under the current root-only lookup.
|
||||
|
||||
### Task 2: Implement Exact-Path Lookup
|
||||
|
||||
- [ ] Update state model docstrings to describe path-keyed declarations.
|
||||
- [ ] In `write_state_value()`, look up `workflow.state_schema.fields[".".join(parts)]` instead of only the first path segment.
|
||||
- [ ] Keep undeclared paths as `replace`.
|
||||
- [ ] Run the focused tests and confirm they pass.
|
||||
|
||||
### Task 3: Keep Docs Honest
|
||||
|
||||
- [ ] Update the core mapping design doc so Phase 2 is recorded as implemented, not future work.
|
||||
- [ ] Update schema-validation docs to note exact-path state metadata without broadening payload validation claims.
|
||||
|
||||
### Task 4: Verify
|
||||
|
||||
- [ ] Run `uv run --with pytest pytest tests/core -q`
|
||||
- [ ] Run `uv run --with pytest pytest -q`
|
||||
- [ ] Run `uv run basedpyright --level error`
|
||||
- [ ] Call out any residual type-check failures that are unrelated to this work.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- flatten nested authoring schemas from `wf_authoring`
|
||||
- reducer registries or custom reducer capabilities
|
||||
- changing `merge_object` from shallow to deep
|
||||
- making parent declarations inherit into child paths
|
||||
- `START` token model changes
|
||||
@@ -0,0 +1,57 @@
|
||||
# Reducer Authoring Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add Python authoring ergonomics for reducer definitions, including optional Pydantic config models.
|
||||
|
||||
**Architecture:** Keep `wf_core` reducer execution as the runtime layer. Add `wf_authoring.reducers` as the ergonomic layer that builds `ReducerDefinition` objects from Python functions, similar to how node authoring builds `NodeSpec`s. Config models are optional; when present, they generate `ReducerSpec.config_schema` and receive parsed config objects at call time.
|
||||
|
||||
**Tech Stack:** Python, Pydantic `BaseModel`, pytest, existing reducer runtime.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create `src/wf_authoring/reducers/`
|
||||
- `callables.py`: reducer callable protocols
|
||||
- `decorator.py`: `@reducer(...)`
|
||||
- `catalog.py`: `ReducerCatalog`
|
||||
- `__init__.py`: reducer authoring exports
|
||||
- Modify `src/wf_authoring/__init__.py`
|
||||
- export reducer authoring API
|
||||
- Add `tests/authoring/test_reducers.py`
|
||||
- plain reducer authoring
|
||||
- configured reducer authoring with BaseModel config
|
||||
- catalog specs/definitions
|
||||
|
||||
## Tasks
|
||||
|
||||
### Task 1: Pin Authoring API
|
||||
|
||||
- [ ] Add tests proving:
|
||||
- `@reducer(name="wf.std.add")` wraps a two-arg callable
|
||||
- `@reducer(name="wf.std.modulo_add", config_model=ModuloConfig)` wraps a callable receiving parsed config
|
||||
- config model JSON Schema becomes `ReducerSpec.config_schema`
|
||||
- `ReducerCatalog.from_reducers(...)` exposes definitions and specs
|
||||
- [ ] Run focused tests and confirm failure before implementation.
|
||||
|
||||
### Task 2: Implement Reducer Authoring
|
||||
|
||||
- [ ] Add typed callable protocols for plain/config reducers.
|
||||
- [ ] Add a small wrapper object that owns a `ReducerDefinition`.
|
||||
- [ ] Add `@reducer(...)` overloads for bare and configured reducers.
|
||||
- [ ] Add `ReducerCatalog`.
|
||||
- [ ] Export from `wf_authoring`.
|
||||
|
||||
### Task 3: Verify
|
||||
|
||||
- [ ] Run `uv run --with pytest pytest tests/authoring/test_reducers.py -q`
|
||||
- [ ] Run `uv run --with pytest pytest tests/authoring -q`
|
||||
- [ ] Run full suite and basedpyright.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- MCP tools for authoring reducers
|
||||
- LLM-authored reducer code
|
||||
- external reducer packages
|
||||
- replacing current built-in registration in this pass
|
||||
@@ -0,0 +1,78 @@
|
||||
# Reducer Capabilities Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace `merge_strategy` with named pure reducer references across core and authoring, using `wf.std.replace` as the default reducer.
|
||||
|
||||
**Architecture:** Introduce a small reducer registry in `wf_core`, register the current three built-ins as reducers, and have state writes resolve every declared reducer name through that registry. Keep undeclared state paths using default replace semantics. Migrate `wf_authoring.state_field()` and all state metadata/tests/docs to reducer names in the same pass so there is one merge concept in the codebase.
|
||||
|
||||
**Tech Stack:** Python, Pydantic, pytest, existing `wf_core` runtime and `wf_authoring` schema projection.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_core/models/schemas.py`
|
||||
- replace `merge_strategy` with `reducer`
|
||||
- Replace/refactor `src/wf_core/runtime/ops/merges.py`
|
||||
- reducer callable type
|
||||
- built-in reducer functions
|
||||
- default reducer registry
|
||||
- reducer application helper
|
||||
- Modify `src/wf_core/runtime/ops/state.py`
|
||||
- resolve reducer names from state fields
|
||||
- use default replace reducer for undeclared paths
|
||||
- Modify `src/wf_authoring/schemas.py`
|
||||
- expose `state_field(reducer=...)`
|
||||
- project reducer metadata through flattened state paths
|
||||
- Modify tests under `tests/core/`, `tests/authoring/`, and `tests/rewrite/`
|
||||
- migrate old metadata
|
||||
- add unknown reducer coverage
|
||||
- Update docs mentioning `merge_strategy`
|
||||
|
||||
## Tasks
|
||||
|
||||
### Task 1: Pin Reducer Semantics
|
||||
|
||||
- [ ] Add tests proving:
|
||||
- `StateField(type="string")` defaults to `wf.std.replace`
|
||||
- `wf.std.append` preserves append behavior
|
||||
- `wf.std.merge_object` preserves shallow object merge behavior
|
||||
- unknown reducer names fail clearly
|
||||
- exact nested state paths still use their own reducer
|
||||
- [ ] Run focused core tests and confirm failure before implementation.
|
||||
|
||||
### Task 2: Replace Core Merge Strategy With Reducers
|
||||
|
||||
- [ ] Replace `merge_strategy` on `StateField` with `reducer`.
|
||||
- [ ] Add reducer functions for `wf.std.replace`, `wf.std.append`, and `wf.std.merge_object`.
|
||||
- [ ] Add a registry lookup path that raises for unknown reducer names.
|
||||
- [ ] Update `write_state_value()` to resolve declared reducers and use `wf.std.replace` for undeclared paths.
|
||||
- [ ] Run focused core tests and confirm reducer behavior is green.
|
||||
|
||||
### Task 3: Migrate Authoring
|
||||
|
||||
- [ ] Change `StateFieldMetadata` and `state_field()` to use `reducer`.
|
||||
- [ ] Preserve nested metadata projection under reducer names.
|
||||
- [ ] Update authoring/rewrite fixtures from `merge_strategy=` to `reducer=`.
|
||||
- [ ] Run focused authoring tests and confirm they pass.
|
||||
|
||||
### Task 4: Update Docs
|
||||
|
||||
- [ ] Replace docs that describe `merge_strategy` with reducer terminology.
|
||||
- [ ] Update examples to show reducer names, including the default replace reducer.
|
||||
- [ ] Keep the design point that reducers are pure and source-owned.
|
||||
|
||||
### Task 5: Verify
|
||||
|
||||
- [ ] Run `uv run --with pytest pytest tests/core tests/authoring tests/rewrite -q`
|
||||
- [ ] Run `uv run --with pytest pytest -q`
|
||||
- [ ] Run `uv run basedpyright --level error`
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- custom user-authored reducer registration through MCP/platform sources
|
||||
- reducer parameters/configuration
|
||||
- async reducers
|
||||
- parallel foreach
|
||||
- compatibility shims for `merge_strategy`
|
||||
@@ -0,0 +1,63 @@
|
||||
# ReducerRef Config Validation Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace string-only reducer references with `ReducerRef(name, config)` and validate reducer config before state mutation.
|
||||
|
||||
**Architecture:** Keep string reducer input as shorthand, normalize it to a `ReducerRef`, and add `config_schema` to reducer specs. Runtime reducer application resolves the reducer spec, validates config through the existing JSON Schema backend, then calls reducer functions with `(current, incoming, config)`.
|
||||
|
||||
**Tech Stack:** Python, Pydantic, jsonschema, pytest, existing reducer registry.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_core/models/reducers.py`
|
||||
- add `ReducerRef`
|
||||
- add `config_schema` to `ReducerSpec`
|
||||
- Modify `src/wf_core/models/schemas.py`
|
||||
- make `StateField.reducer` a `ReducerRef` with string shorthand parsing
|
||||
- Modify `src/wf_core/runtime/ops/merges.py`
|
||||
- reducer callable accepts config
|
||||
- validate config against reducer spec before merge
|
||||
- Modify `src/wf_core/runtime/ops/state.py`
|
||||
- pass `ReducerRef` to reducer application
|
||||
- Modify `src/wf_artifacts/factory.py`
|
||||
- infer reducer dependencies from `ReducerRef` objects and dict payloads
|
||||
- Modify tests for core reducers and artifact dependency inference
|
||||
|
||||
## Tasks
|
||||
|
||||
### Task 1: Pin ReducerRef Behavior
|
||||
|
||||
- [ ] Add tests proving string shorthand normalizes to `ReducerRef(name=..., config={})`.
|
||||
- [ ] Add tests proving object reducer payloads preserve config.
|
||||
- [ ] Add tests proving invalid config fails before mutation.
|
||||
- [ ] Run focused tests and confirm failure before implementation.
|
||||
|
||||
### Task 2: Implement ReducerRef and Config Validation
|
||||
|
||||
- [ ] Add `ReducerRef`.
|
||||
- [ ] Add `ReducerSpec.config_schema`.
|
||||
- [ ] Update reducer callables to accept config.
|
||||
- [ ] Validate config before calling a reducer.
|
||||
- [ ] Keep existing no-config reducers accepting `{}` only through their empty config schemas.
|
||||
|
||||
### Task 3: Update Artifact Dependency Inference
|
||||
|
||||
- [ ] Infer reducer dependency names from string reducers and object reducers.
|
||||
- [ ] Keep dependency key by reducer name, not by reducer config.
|
||||
- [ ] Run artifact factory tests.
|
||||
|
||||
### Task 4: Verify
|
||||
|
||||
- [ ] Run focused core/artifact tests.
|
||||
- [ ] Run full suite.
|
||||
- [ ] Run basedpyright.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- implementing `modulo_add`
|
||||
- reducer decorator UX
|
||||
- configurable reducer factories
|
||||
- caching configured reducers
|
||||
@@ -0,0 +1,64 @@
|
||||
# Reducer Source Inventory Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Represent reducers as first-class source-owned capabilities and expose the built-in reducer catalog through existing source inventory.
|
||||
|
||||
**Architecture:** Add a small immutable `ReducerSpec` model in `wf_core`, keep runtime reducer execution where it is, and extend `wf_mcp` capability buckets with reducer ownership metadata. Register the three built-in reducer specs under `wf.std` so inventory becomes honest without adding reducer-authoring UX yet.
|
||||
|
||||
**Tech Stack:** Python, dataclasses/Pydantic, pytest, existing capability source inventory.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create `src/wf_core/models/reducers.py`
|
||||
- reducer capability metadata
|
||||
- Modify `src/wf_core/models/__init__.py` and `src/wf_core/__init__.py`
|
||||
- export `ReducerSpec`
|
||||
- Modify `src/wf_mcp/broker/service/capability_sources.py`
|
||||
- add reducer bucket, counts, and inventory listing
|
||||
- Modify `src/wf_mcp/broker/service/builtins.py`
|
||||
- define/register built-in reducer specs under `wf.std`
|
||||
- Modify `tests/wf_mcp/test_service.py`
|
||||
- assert reducer inventory
|
||||
- Update `docs/wf_mcp_capability_sources.md`
|
||||
- document reducers under `wf.std`
|
||||
- Update `docs/workflow_capabilities.md`
|
||||
- name reducers as workflow-facing capabilities
|
||||
|
||||
## Tasks
|
||||
|
||||
### Task 1: Pin Inventory Behavior
|
||||
|
||||
- [ ] Add failing tests proving:
|
||||
- `wf.std` owns built-in reducers
|
||||
- source status exposes `reducer_count`
|
||||
- source inventory exposes reducer names
|
||||
- [ ] Run focused service tests and confirm failure before implementation.
|
||||
|
||||
### Task 2: Add ReducerSpec and Source Buckets
|
||||
|
||||
- [ ] Add `ReducerSpec` with `name`, `description`, and optional value-shape notes.
|
||||
- [ ] Export `ReducerSpec` from core.
|
||||
- [ ] Extend `CapabilityBuckets`, `as_status()`, and `as_inventory()` with reducers.
|
||||
- [ ] Register `wf.std.replace`, `wf.std.append`, and `wf.std.merge_object` in built-ins.
|
||||
- [ ] Run focused service tests and confirm they pass.
|
||||
|
||||
### Task 3: Update Docs
|
||||
|
||||
- [ ] Add reducers to the source vocabulary docs.
|
||||
- [ ] Clarify that reducers are selected from sources; LLMs are not expected to author reducer code.
|
||||
|
||||
### Task 4: Verify
|
||||
|
||||
- [ ] Run `uv run --with pytest pytest tests/wf_mcp -q`
|
||||
- [ ] Run `uv run --with pytest pytest -q`
|
||||
- [ ] Run `uv run basedpyright --level error`
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- reducer decorators
|
||||
- reducer runtime dependency resolution from external sources
|
||||
- reducer MCP tools
|
||||
- non-built-in reducer libraries
|
||||
@@ -0,0 +1,699 @@
|
||||
# Workflow Draft Surface Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace the disposable draft prototype with the first real MCP-facing workflow draft surface: keyed, patchable JSON that adapts into `wf_authoring.WorkflowBuilder` instead of rebuilding graph semantics itself.
|
||||
|
||||
**Architecture:** The new draft layer remains a typed JSON seam in `wf_artifacts`, but delegates graph construction to `wf_authoring`. Draft parsing owns keyed presentation, validation, and stable patch paths; `WorkflowBuilder` owns graph construction. The MCP workflow surface keeps the same tool family while accepting the new draft shape.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, `wf_authoring`, `wf_core`, `jsonpatch`, pytest, basedpyright, Ruff.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
### Create
|
||||
|
||||
- `src/wf_artifacts/drafts/models.py`
|
||||
- concrete Pydantic draft document models
|
||||
- `src/wf_artifacts/drafts/adapter.py`
|
||||
- thin JSON-draft-to-`WorkflowBuilder` adapter
|
||||
- `src/wf_artifacts/drafts/api.py`
|
||||
- public compile/validate/patch functions and diagnostics
|
||||
- `tests/artifacts/test_draft_models.py`
|
||||
- draft document validation
|
||||
- `tests/artifacts/test_draft_adapter.py`
|
||||
- keyed step/routes lowering through `WorkflowBuilder`
|
||||
- `tests/artifacts/test_draft_api.py`
|
||||
- public compile/validate/patch behavior
|
||||
|
||||
### Modify
|
||||
|
||||
- `src/wf_artifacts/drafts.py`
|
||||
- replace module body with compatibility re-exports or remove once imports are updated
|
||||
- `src/wf_artifacts/__init__.py`
|
||||
- export public draft API/models from the package
|
||||
- `src/wf_authoring/builder/core.py`
|
||||
- add `use_ref(...)` for named external capabilities without local `NodeSpec`s
|
||||
- `src/wf_authoring/ops/*`
|
||||
- only if route helpers need a reusable public lowering entrypoint
|
||||
- `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- keep using public draft API; no semantic duplication
|
||||
- `tests/wf_mcp/test_workflow_surface.py`
|
||||
- update draft fixtures to the new keyed document shape
|
||||
- `tests/wf_mcp/test_server.py`
|
||||
- confirm MCP schemas remain plain-object friendly
|
||||
- `docs/workflow_drafts.md`
|
||||
- replace prototype examples with the first real draft surface
|
||||
- `docs/wf_mcp_end_to_end_runbook.md`
|
||||
- update draft example
|
||||
- `docs/wf_mcp_operator_manual.md`
|
||||
- keep draft-first guidance accurate
|
||||
- `docs/wf_mcp_troubleshooting.md`
|
||||
- update patch-path examples
|
||||
|
||||
## Task 1: Split Draft Code Into Focused Modules
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_artifacts/drafts/models.py`
|
||||
- Create: `src/wf_artifacts/drafts/api.py`
|
||||
- Create: `src/wf_artifacts/drafts/adapter.py`
|
||||
- Modify: `src/wf_artifacts/drafts.py`
|
||||
- Modify: `src/wf_artifacts/__init__.py`
|
||||
- Test: `tests/artifacts/test_draft_models.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing model tests**
|
||||
|
||||
```python
|
||||
from wf_artifacts.drafts import DraftUseStep, WorkflowDraft
|
||||
|
||||
|
||||
def test_workflow_draft_uses_keyed_steps() -> None:
|
||||
draft = WorkflowDraft.model_validate(
|
||||
{
|
||||
"name": "echo",
|
||||
"input_schema": {},
|
||||
"state_schema": {"fields": {}},
|
||||
"output_schema": {},
|
||||
"start": "echo",
|
||||
"steps": {
|
||||
"echo": {
|
||||
"use": "demo.echo",
|
||||
"in": {"input.text": "text"},
|
||||
"out": {"echoed": "state.echoed"},
|
||||
}
|
||||
},
|
||||
"routes": {"echo": {"ok": "__end__"}},
|
||||
}
|
||||
)
|
||||
|
||||
assert isinstance(draft.steps["echo"], DraftUseStep)
|
||||
assert draft.steps["echo"].use == "demo.echo"
|
||||
|
||||
|
||||
def test_draft_step_requires_exactly_one_kind_key() -> None:
|
||||
result = WorkflowDraft.model_validate(
|
||||
{
|
||||
"name": "bad",
|
||||
"input_schema": {},
|
||||
"state_schema": {"fields": {}},
|
||||
"output_schema": {},
|
||||
"start": "bad",
|
||||
"steps": {
|
||||
"bad": {
|
||||
"use": "demo.echo",
|
||||
"join": {},
|
||||
}
|
||||
},
|
||||
"routes": {},
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
The second test should be written with `pytest.raises(ValidationError)` and assert the authoring path identifies `steps.bad`.
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/artifacts/test_draft_models.py -q
|
||||
```
|
||||
|
||||
Expected: import errors or validation failures because keyed draft models do not exist yet.
|
||||
|
||||
- [ ] **Step 3: Implement minimal concrete draft models**
|
||||
|
||||
Create:
|
||||
|
||||
```python
|
||||
# src/wf_artifacts/drafts/models.py
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Annotated, Any, Literal
|
||||
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
|
||||
JsonObject = dict[str, Any]
|
||||
STEP_KIND_KEYS = frozenset({"use", "foreach", "interrupt", "join"})
|
||||
|
||||
|
||||
class DraftUseStep(BaseModel):
|
||||
use: str
|
||||
in_: dict[str, str] = Field(default_factory=dict, alias="in")
|
||||
out: dict[str, str] = Field(default_factory=dict)
|
||||
desc: str | None = None
|
||||
retry: int | None = Field(default=None, ge=0)
|
||||
timeout_seconds: int | None = Field(default=None, gt=0)
|
||||
|
||||
|
||||
class DraftForeachPayload(BaseModel):
|
||||
over: str
|
||||
as_: str = Field(alias="as")
|
||||
mode: Literal["serial", "parallel"] = "serial"
|
||||
on_item_error: Literal["fail", "collect", "skip"] = "fail"
|
||||
|
||||
|
||||
class DraftForeachStep(BaseModel):
|
||||
foreach: DraftForeachPayload
|
||||
|
||||
|
||||
class DraftInterruptPayload(BaseModel):
|
||||
kind: str
|
||||
request: dict[str, str] = Field(default_factory=dict)
|
||||
resume: dict[str, str] = Field(default_factory=dict)
|
||||
outcomes: list[str] = Field(default_factory=lambda: ["submitted"])
|
||||
|
||||
|
||||
class DraftInterruptStep(BaseModel):
|
||||
interrupt: DraftInterruptPayload
|
||||
|
||||
|
||||
class DraftJoinStep(BaseModel):
|
||||
join: JsonObject = Field(default_factory=dict)
|
||||
|
||||
|
||||
DraftStep = Annotated[
|
||||
DraftUseStep
|
||||
| DraftForeachStep
|
||||
| DraftInterruptStep
|
||||
| DraftJoinStep,
|
||||
Field(discriminator=None),
|
||||
]
|
||||
|
||||
|
||||
class WorkflowDraft(BaseModel):
|
||||
name: str
|
||||
input_schema: JsonObject
|
||||
state_schema: JsonObject
|
||||
output_schema: JsonObject
|
||||
start: str
|
||||
steps: dict[str, DraftStep]
|
||||
routes: dict[str, dict[str, str]] = Field(default_factory=dict)
|
||||
|
||||
@model_validator(mode="before")
|
||||
@classmethod
|
||||
def _validate_step_kinds(cls, value: object) -> object:
|
||||
if not isinstance(value, dict):
|
||||
return value
|
||||
steps = value.get("steps")
|
||||
if not isinstance(steps, dict):
|
||||
return value
|
||||
for step_id, payload in steps.items():
|
||||
if not isinstance(payload, dict):
|
||||
continue
|
||||
present = STEP_KIND_KEYS.intersection(payload)
|
||||
if len(present) != 1:
|
||||
raise ValueError(
|
||||
f"steps.{step_id} must contain exactly one step kind key"
|
||||
)
|
||||
return value
|
||||
```
|
||||
|
||||
Keep the public import path stable by re-exporting through `src/wf_artifacts/drafts.py` during the transition.
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/artifacts/test_draft_models.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/wf_artifacts tests/artifacts/test_draft_models.py
|
||||
git commit -m "refactor: add keyed workflow draft models"
|
||||
```
|
||||
|
||||
## Task 2: Add `use_ref` And Thin Adapter Over `WorkflowBuilder`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_artifacts/drafts/adapter.py`
|
||||
- Modify only if needed: `src/wf_authoring/builder/core.py`
|
||||
- Modify only if needed: `src/wf_authoring/ops/*`
|
||||
- Test: `tests/artifacts/test_draft_adapter.py`
|
||||
|
||||
- [ ] **Step 1: Write failing adapter tests**
|
||||
|
||||
```python
|
||||
from wf_artifacts.drafts import WorkflowDraft
|
||||
from wf_artifacts.drafts.adapter import build_workflow_from_draft
|
||||
|
||||
|
||||
def test_adapter_lowers_keyed_use_steps_and_routes_through_builder() -> None:
|
||||
draft = WorkflowDraft.model_validate(
|
||||
{
|
||||
"name": "echo",
|
||||
"input_schema": {},
|
||||
"state_schema": {"fields": {}},
|
||||
"output_schema": {},
|
||||
"start": "echo",
|
||||
"steps": {"echo": {"use": "demo.echo"}},
|
||||
"routes": {"echo": {"ok": "__end__"}},
|
||||
}
|
||||
)
|
||||
|
||||
workflow = build_workflow_from_draft(draft)
|
||||
|
||||
assert workflow.nodes[0].id == "echo"
|
||||
assert workflow.nodes[0].node == "demo.echo"
|
||||
assert workflow.edges[0].from_ == "echo"
|
||||
assert workflow.edges[0].outcome == "ok"
|
||||
assert workflow.edges[0].to == "__end__"
|
||||
|
||||
|
||||
def test_builder_use_ref_creates_external_node_use_without_node_def() -> None:
|
||||
builder = WorkflowBuilder(
|
||||
"echo",
|
||||
input_schema={},
|
||||
state_schema={"fields": {}},
|
||||
output_schema={},
|
||||
)
|
||||
|
||||
step = builder.use_ref("demo.echo", id="echo")
|
||||
builder.set_entry_point(step)
|
||||
builder.connect(step, "ok", "__end__")
|
||||
workflow = builder.compile()
|
||||
|
||||
assert step.node == "demo.echo"
|
||||
assert workflow.node_defs == []
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/artifacts/test_draft_adapter.py -q
|
||||
```
|
||||
|
||||
Expected: import error because `build_workflow_from_draft` does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement the thin adapter**
|
||||
|
||||
First add:
|
||||
|
||||
```python
|
||||
def use_ref(
|
||||
self,
|
||||
name: str,
|
||||
*,
|
||||
id: str | None = None,
|
||||
in_map: MapArg | None = None,
|
||||
out_map: MapArg | None = None,
|
||||
desc: str | None = None,
|
||||
) -> NodeUse:
|
||||
...
|
||||
```
|
||||
|
||||
`use_ref` creates a `NodeUse` for an already named external capability and does
|
||||
not add a local `NodeDef`.
|
||||
|
||||
Then implement `build_workflow_from_draft(draft: WorkflowDraft) -> Workflow` so
|
||||
it:
|
||||
|
||||
1. constructs a `WorkflowBuilder`
|
||||
2. registers each draft step by stable id
|
||||
3. uses existing `WorkflowBuilder` public methods for:
|
||||
- `use_ref`
|
||||
- `foreach`
|
||||
- `interrupt`
|
||||
- `join`
|
||||
4. applies `routes`
|
||||
5. calls explicit `start(...)`
|
||||
6. returns `builder.build(...)`
|
||||
|
||||
Do **not** invent draft route sugar in this pass.
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/artifacts/test_draft_adapter.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/wf_artifacts src/wf_authoring tests/artifacts/test_draft_adapter.py
|
||||
git commit -m "feat: adapt workflow drafts through workflow builder"
|
||||
```
|
||||
|
||||
## Task 3: Replace Prototype Public API
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_artifacts/drafts/api.py`
|
||||
- Modify: `src/wf_artifacts/drafts.py`
|
||||
- Modify: `src/wf_artifacts/__init__.py`
|
||||
- Test: `tests/artifacts/test_draft_api.py`
|
||||
|
||||
- [ ] **Step 1: Write failing API tests**
|
||||
|
||||
```python
|
||||
from wf_artifacts.drafts import compile_workflow_draft, patch_workflow_draft
|
||||
|
||||
|
||||
def test_compile_workflow_draft_returns_raw_core_shape() -> None:
|
||||
plan = compile_workflow_draft(_keyed_echo_draft())
|
||||
|
||||
assert plan["nodes"][0]["id"] == "echo"
|
||||
assert plan["nodes"][0]["node"] == "demo.echo"
|
||||
assert plan["edges"][0]["outcome"] == "ok"
|
||||
|
||||
|
||||
def test_patch_workflow_draft_uses_stable_step_paths() -> None:
|
||||
result = patch_workflow_draft(
|
||||
_keyed_echo_draft(),
|
||||
[
|
||||
{
|
||||
"op": "replace",
|
||||
"path": "/steps/echo/in/input.text",
|
||||
"value": "message",
|
||||
}
|
||||
],
|
||||
)
|
||||
|
||||
assert result["status"] == "valid"
|
||||
assert result["draft"]["steps"]["echo"]["in"]["input.text"] == "message"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/artifacts/test_draft_api.py -q
|
||||
```
|
||||
|
||||
Expected: failures because the old prototype API still expects array `steps`.
|
||||
|
||||
- [ ] **Step 3: Implement the API**
|
||||
|
||||
Create:
|
||||
|
||||
```python
|
||||
# src/wf_artifacts/drafts/api.py
|
||||
def compile_workflow_draft(draft: JsonObject) -> JsonObject:
|
||||
parsed = WorkflowDraft.model_validate(draft)
|
||||
workflow = build_workflow_from_draft(parsed)
|
||||
return workflow.model_dump(mode="json", by_alias=True, exclude={"node_defs"})
|
||||
```
|
||||
|
||||
Keep:
|
||||
|
||||
- `validate_workflow_draft`
|
||||
- `patch_workflow_draft`
|
||||
- structured `DraftDiagnostic`
|
||||
|
||||
Update diagnostics to use keyed paths such as:
|
||||
|
||||
```text
|
||||
steps.echo.in
|
||||
routes.echo.error
|
||||
```
|
||||
|
||||
Delete the old array-step prototype code after public tests are green.
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/artifacts/test_draft_api.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/wf_artifacts tests/artifacts/test_draft_api.py
|
||||
git commit -m "feat: replace draft prototype with keyed public api"
|
||||
```
|
||||
|
||||
## Task 4: Update MCP Workflow Surface
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/test_workflow_surface.py`
|
||||
- Modify: `tests/wf_mcp/test_server.py`
|
||||
- Modify only if needed: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
|
||||
- [ ] **Step 1: Update the failing MCP tests**
|
||||
|
||||
Replace old fixtures like:
|
||||
|
||||
```python
|
||||
"steps": [{"id": "echo", "kind": "use", ...}]
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
"steps": {"echo": {"use": "demo.echo", ...}},
|
||||
"routes": {"echo": {"ok": "__end__"}},
|
||||
```
|
||||
|
||||
Keep assertions that:
|
||||
|
||||
- draft tools still expose plain object schemas to MCP clients
|
||||
- `create_artifact_from_draft` still saves artifacts
|
||||
- source binding normalization still works
|
||||
- missing `wf.std` self-binding diagnostics still work
|
||||
|
||||
- [ ] **Step 2: Run tests to verify failures**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_server.py -q
|
||||
```
|
||||
|
||||
Expected: failures wherever MCP handlers still assume the old prototype shape.
|
||||
|
||||
- [ ] **Step 3: Make minimal MCP adjustments**
|
||||
|
||||
Keep handlers thin:
|
||||
|
||||
```python
|
||||
plan = compile_workflow_draft(draft)
|
||||
```
|
||||
|
||||
No duplicate draft interpretation should appear in `wf_mcp`.
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_server.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/wf_mcp tests/wf_mcp
|
||||
git commit -m "feat: accept keyed workflow drafts over mcp"
|
||||
```
|
||||
|
||||
## Task 5: Add Outcome Validation When Capability Contracts Are Available
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/drafts/api.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Test: `tests/wf_mcp/test_workflow_surface.py`
|
||||
|
||||
- [ ] **Step 1: Write failing outcome validation test**
|
||||
|
||||
```python
|
||||
def test_draft_validation_rejects_unknown_capability_outcome_when_spec_is_known() -> None:
|
||||
handlers = _handlers_with_demo_echo_spec()
|
||||
draft = _keyed_echo_draft()
|
||||
draft["routes"]["echo"] = {"typo": "__end__"}
|
||||
|
||||
result = asyncio.run(handlers.validate_draft(draft=draft))
|
||||
|
||||
assert result["status"] == "invalid"
|
||||
assert result["diagnostics"][0]["path"] == "routes.echo.typo"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py -q
|
||||
```
|
||||
|
||||
Expected: validation currently accepts the typo.
|
||||
|
||||
- [ ] **Step 3: Implement capability-aware outcome validation**
|
||||
|
||||
Pass an optional capability lookup into draft validation from MCP handlers.
|
||||
|
||||
Rules:
|
||||
|
||||
- validate outcome keys for `use` steps only when the capability is resolvable
|
||||
- if the capability is unknown/unavailable, leave dependency validation to the
|
||||
later artifact/deployment stages
|
||||
- diagnostic path must identify the keyed route entry
|
||||
|
||||
Do not make `wf_artifacts` depend on `wf_mcp`; define a tiny callable/protocol
|
||||
interface for lookup instead.
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/wf_artifacts src/wf_mcp tests/wf_mcp/test_workflow_surface.py
|
||||
git commit -m "feat: validate draft routes against known outcomes"
|
||||
```
|
||||
|
||||
## Task 6: Update Documentation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/workflow_drafts.md`
|
||||
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
|
||||
- Modify: `docs/wf_mcp_operator_manual.md`
|
||||
- Modify: `docs/wf_mcp_troubleshooting.md`
|
||||
|
||||
- [ ] **Step 1: Update docs to the real draft surface**
|
||||
|
||||
Replace prototype array examples with keyed examples:
|
||||
|
||||
```json
|
||||
"steps": {
|
||||
"echo": {
|
||||
"use": "demo.echo_tool",
|
||||
"in": {"input.text": "text"},
|
||||
"out": {"echoed": "state.echoed"}
|
||||
}
|
||||
},
|
||||
"routes": {
|
||||
"echo": {
|
||||
"ok": "__end__"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Document:
|
||||
|
||||
- exactly-one-kind-key rule
|
||||
- stable keyed patch paths
|
||||
- `route` as repeated condition-chain sugar
|
||||
- `WorkflowBuilder` as the semantic owner beneath the JSON adapter
|
||||
- outcome strings validated against capability contracts when available
|
||||
|
||||
- [ ] **Step 2: Run a targeted docs scan**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n '\"steps\": \\[|\"kind\": \"use\"|/steps/0|create_artifact_from_draft' docs
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- no stale prototype examples in current docs
|
||||
- `create_artifact_from_draft` still documented as the preferred path
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add docs
|
||||
git commit -m "docs: describe keyed workflow draft surface"
|
||||
```
|
||||
|
||||
## Task 7: Full Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- No new files
|
||||
|
||||
- [ ] **Step 1: Run focused verification**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/artifacts tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_server.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run full project tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run type checking**
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 4: Run lint**
|
||||
|
||||
```bash
|
||||
uvx ruff check src tests
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Commit final cleanup**
|
||||
|
||||
```bash
|
||||
git add .
|
||||
git commit -m "feat: ship keyed workflow draft authoring surface"
|
||||
```
|
||||
|
||||
## Self-Review
|
||||
|
||||
### Spec Coverage
|
||||
|
||||
- keyed `steps`: Tasks 1-4
|
||||
- compact `routes`: Tasks 1-4
|
||||
- verb-keyed explicit step kinds: Task 1
|
||||
- saved capability/workflow refs in `use`: Task 2, existing capability refs pass through unchanged
|
||||
- stable patch paths: Tasks 3 and 6
|
||||
- `wf_authoring` as semantic owner: Tasks 2 and 6
|
||||
- outcome validation against declared contracts: Task 5
|
||||
- prototype replacement rather than migration: Tasks 3, 4, 6
|
||||
|
||||
### Placeholder Scan
|
||||
|
||||
- no `TBD`
|
||||
- no unspecified "add validation" placeholders
|
||||
- every task has exact files, tests, commands, and expected behavior
|
||||
|
||||
### Type Consistency
|
||||
|
||||
- `WorkflowDraft`, `DraftUseStep`, `build_workflow_from_draft`, and public API
|
||||
names stay consistent across all tasks
|
||||
- patch examples use keyed paths consistently
|
||||
- `routes` stays the only authored outcome-routing section
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,944 @@
|
||||
# Core Path Bindings Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace loose core path/map strings with typed path objects and canonical list-of-struct node bindings while keeping deprecated shapes parse-compatible.
|
||||
|
||||
**Architecture:** Add immutable path value objects in `wf_core.paths`, then introduce canonical binding models in `wf_core.models.steps`. Runtime and validation move to the canonical bindings, while old `in_map`, `input_values`, `out_map`, and dict-shaped state fields are accepted only by model validators.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, pytest, jsonschema, basedpyright, ruff.
|
||||
|
||||
**Current status as of 2026-05-20:** The implementation has moved past the
|
||||
original checklist. Typed path values, canonical node bindings, canonical
|
||||
runtime input/output handling, atomic state patches, canonical validation,
|
||||
JSON-Schema-native state reducers, schema validation, and authoring canonical
|
||||
emission are present in the tree. The remaining checklist item is the full
|
||||
compatibility/regression pass in Task 9. If future code changes touch this area,
|
||||
prefer adding focused tests to the existing `tests/core/test_*path*`,
|
||||
`tests/core/test_*mapping*`, and `tests/authoring/test_builder.py` coverage
|
||||
rather than reimplementing the earlier tasks.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_core/paths.py`: own typed graph/state/local path objects and graph path resolution helpers.
|
||||
- Modify `src/wf_core/local_paths.py`: keep compatibility wrappers over `LocalPath` plus local get/set helpers.
|
||||
- Modify `src/wf_core/models/steps.py`: add `InputPathBinding`, `InputValueBinding`, `OutputBinding`, and canonical `NodeUse.input` / `NodeUse.output`.
|
||||
- Modify `src/wf_core/models/conditions.py`: type condition path operands with `GraphSourcePath`.
|
||||
- Modify `src/wf_core/models/schemas.py`: harden `SchemaRef` and add canonical state field declarations.
|
||||
- Modify `src/wf_core/runtime/ops/nodes.py`: resolve canonical node input bindings.
|
||||
- Modify `src/wf_core/runtime/ops/state.py`: apply canonical output bindings through an atomic state patch.
|
||||
- Modify `src/wf_core/runtime/ops/schemas.py`: expose focused JSON Schema validation helpers.
|
||||
- Modify `src/wf_core/validation/steps.py`: validate canonical bindings and typed paths.
|
||||
- Modify `src/wf_authoring/dsl/paths.py`: emit core path objects while preserving ergonomic helpers.
|
||||
- Modify `src/wf_authoring/dsl/conditions.py`: compile authoring expressions to core typed condition models.
|
||||
- Add `tests/core/test_path_values.py`: path parsing, serialization, JSON Schema, and error tests.
|
||||
- Add `tests/core/test_canonical_node_bindings.py`: canonical model parsing and deprecated compatibility tests.
|
||||
- Add `tests/core/test_atomic_state_patches.py`: output binding, reducer, overlap, and atomicity tests.
|
||||
- Update existing `tests/core/test_mapping_validation.py`, `tests/core/test_nested_mappings.py`, `tests/core/test_nested_state_paths.py`, and authoring tests as needed.
|
||||
|
||||
## Task 1: Add Typed Path Values
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/paths.py`
|
||||
- Modify: `src/wf_core/local_paths.py`
|
||||
- Create: `tests/core/test_path_values.py`
|
||||
|
||||
- [ ] **Step 1: Write path value tests**
|
||||
|
||||
Add tests for parsing, string serialization, equality/hashability, invalid segments, root-only graph source reads, and no bare write state:
|
||||
|
||||
```python
|
||||
import pytest
|
||||
from pydantic import BaseModel, ValidationError
|
||||
|
||||
from wf_core.paths import GraphSourcePath, LocalPath, PathResolutionError, StatePath
|
||||
|
||||
|
||||
def test_graph_source_path_accepts_root_and_nested_paths():
|
||||
assert str(GraphSourcePath.parse("state")) == "state"
|
||||
assert str(GraphSourcePath.parse("input.user")) == "input.user"
|
||||
assert str(GraphSourcePath.context("loop_item")) == "context.loop_item"
|
||||
|
||||
|
||||
def test_state_path_rejects_bare_state_write_target():
|
||||
with pytest.raises(PathResolutionError, match="state path"):
|
||||
StatePath.parse("state")
|
||||
|
||||
|
||||
def test_local_path_supports_root_marker():
|
||||
assert str(LocalPath.root()) == "."
|
||||
assert str(LocalPath.of("user.name")) == "user.name"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("raw", ["", "state.", "state.items.0", "state.user-name"])
|
||||
def test_paths_reject_invalid_segments(raw: str):
|
||||
with pytest.raises(PathResolutionError):
|
||||
GraphSourcePath.parse(raw)
|
||||
|
||||
|
||||
def test_path_objects_are_hashable():
|
||||
paths = {StatePath.of("person.name"), StatePath.of("person.name")}
|
||||
assert len(paths) == 1
|
||||
|
||||
|
||||
def test_pydantic_accepts_path_strings_and_serializes_strings():
|
||||
class Payload(BaseModel):
|
||||
source: GraphSourcePath
|
||||
target: StatePath
|
||||
local: LocalPath
|
||||
|
||||
payload = Payload.model_validate(
|
||||
{"source": "input.user", "target": "state.person", "local": "user"}
|
||||
)
|
||||
assert payload.source == GraphSourcePath.input("user")
|
||||
assert payload.model_dump(mode="json")["target"] == "state.person"
|
||||
|
||||
|
||||
def test_pydantic_rejects_bad_path_string():
|
||||
class Payload(BaseModel):
|
||||
source: GraphSourcePath
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
Payload.model_validate({"source": "output.foo"})
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run path tests to verify they fail**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_path_values.py -q`
|
||||
|
||||
Expected: failures because `GraphSourcePath`, `StatePath`, and `LocalPath` classes do not exist or do not validate strictly.
|
||||
|
||||
- [ ] **Step 3: Implement path value classes**
|
||||
|
||||
In `src/wf_core/paths.py`, add frozen dataclasses and shared parsing helpers. Keep existing helper function names as compatibility wrappers where practical.
|
||||
|
||||
Implementation shape:
|
||||
|
||||
```python
|
||||
from dataclasses import dataclass
|
||||
import re
|
||||
from typing import Any, ClassVar, Literal
|
||||
|
||||
SEGMENT_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LocalPath:
|
||||
"""Node-local payload path. `.` means the whole local payload."""
|
||||
|
||||
parts: tuple[str, ...]
|
||||
|
||||
@classmethod
|
||||
def root(cls) -> "LocalPath":
|
||||
return cls(())
|
||||
|
||||
@classmethod
|
||||
def of(cls, *fragments: str) -> "LocalPath":
|
||||
return cls(_parse_fragments(*fragments, allow_empty=False))
|
||||
|
||||
@classmethod
|
||||
def parse(cls, raw: str) -> "LocalPath":
|
||||
if raw == ".":
|
||||
return cls.root()
|
||||
return cls.of(raw)
|
||||
|
||||
def __str__(self) -> str:
|
||||
return "." if not self.parts else ".".join(self.parts)
|
||||
```
|
||||
|
||||
Also add:
|
||||
|
||||
```python
|
||||
GraphRoot = Literal["input", "state", "context"]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GraphSourcePath:
|
||||
"""Readable workflow graph path rooted at input, state, or context."""
|
||||
|
||||
root: GraphRoot
|
||||
parts: tuple[str, ...] = ()
|
||||
|
||||
@classmethod
|
||||
def parse(cls, raw: str) -> "GraphSourcePath": ...
|
||||
|
||||
@classmethod
|
||||
def input(cls, *fragments: str) -> "GraphSourcePath": ...
|
||||
|
||||
@classmethod
|
||||
def state(cls, *fragments: str) -> "GraphSourcePath": ...
|
||||
|
||||
@classmethod
|
||||
def context(cls, *fragments: str) -> "GraphSourcePath": ...
|
||||
```
|
||||
|
||||
And:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class StatePath:
|
||||
"""Writable workflow state path. Bare `state` is intentionally invalid."""
|
||||
|
||||
parts: tuple[str, ...]
|
||||
|
||||
@classmethod
|
||||
def parse(cls, raw: str) -> "StatePath":
|
||||
parsed = GraphSourcePath.parse(raw)
|
||||
if parsed.root != "state" or not parsed.parts:
|
||||
raise PathResolutionError("expected state path such as state.foo")
|
||||
return cls(parsed.parts)
|
||||
|
||||
@classmethod
|
||||
def of(cls, *fragments: str) -> "StatePath": ...
|
||||
```
|
||||
|
||||
Add Pydantic `__get_pydantic_core_schema__` and `__get_pydantic_json_schema__` hooks for each class so strings validate into objects and serialize back to strings.
|
||||
|
||||
- [ ] **Step 4: Update local path wrappers**
|
||||
|
||||
In `src/wf_core/local_paths.py`, keep public functions but delegate parsing to `LocalPath.parse`:
|
||||
|
||||
```python
|
||||
def split_local_path(path: str | LocalPath) -> list[str]:
|
||||
"""Split one node-local path, accepting the new typed path object."""
|
||||
parsed = path if isinstance(path, LocalPath) else LocalPath.parse(path)
|
||||
return list(parsed.parts)
|
||||
```
|
||||
|
||||
Update `paths_overlap` and `has_overlapping_paths` to accept `str | LocalPath`.
|
||||
|
||||
- [ ] **Step 5: Run path tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_path_values.py -q`
|
||||
|
||||
Expected: all tests in `test_path_values.py` pass.
|
||||
|
||||
## Task 2: Add Canonical Node Binding Models
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/models/steps.py`
|
||||
- Test: `tests/core/test_canonical_node_bindings.py`
|
||||
|
||||
- [ ] **Step 1: Write canonical binding tests**
|
||||
|
||||
Create `tests/core/test_canonical_node_bindings.py`:
|
||||
|
||||
```python
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from wf_core.models.steps import NodeUse
|
||||
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
|
||||
|
||||
|
||||
def test_node_use_accepts_canonical_input_and_output_bindings():
|
||||
node = NodeUse.model_validate(
|
||||
{
|
||||
"id": "echo",
|
||||
"type": "node",
|
||||
"node": "echo",
|
||||
"input": [
|
||||
{"target": "message", "path": "input.message"},
|
||||
{"target": "mode", "value": None},
|
||||
],
|
||||
"output": [{"source": "echoed", "target": "state.echoed"}],
|
||||
}
|
||||
)
|
||||
|
||||
assert node.input[0].target == LocalPath.of("message")
|
||||
assert node.input[0].path == GraphSourcePath.input("message")
|
||||
assert node.input[1].value is None
|
||||
assert node.output[0].target == StatePath.of("echoed")
|
||||
|
||||
|
||||
def test_node_use_converts_old_maps_to_canonical_bindings():
|
||||
node = NodeUse.model_validate(
|
||||
{
|
||||
"id": "echo",
|
||||
"type": "node",
|
||||
"node": "echo",
|
||||
"in_map": {"input.message": "message"},
|
||||
"input_values": {"mode": "fast"},
|
||||
"out_map": {"echoed": "state.echoed"},
|
||||
}
|
||||
)
|
||||
|
||||
dumped = node.model_dump(mode="json")
|
||||
assert "in_map" not in dumped
|
||||
assert "input_values" not in dumped
|
||||
assert "out_map" not in dumped
|
||||
assert dumped["input"][0]["path"] == "input.message"
|
||||
assert dumped["input"][1]["value"] == "fast"
|
||||
assert dumped["output"][0]["target"] == "state.echoed"
|
||||
|
||||
|
||||
def test_node_use_rejects_mixed_old_and_new_binding_styles():
|
||||
with pytest.raises(ValidationError):
|
||||
NodeUse.model_validate(
|
||||
{
|
||||
"id": "echo",
|
||||
"type": "node",
|
||||
"node": "echo",
|
||||
"input": [{"target": "message", "path": "input.message"}],
|
||||
"in_map": {"input.other": "other"},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_input_binding_rejects_path_and_value_together():
|
||||
with pytest.raises(ValidationError):
|
||||
NodeUse.model_validate(
|
||||
{
|
||||
"id": "bad",
|
||||
"type": "node",
|
||||
"node": "bad",
|
||||
"input": [
|
||||
{"target": "message", "path": "input.message", "value": "x"}
|
||||
],
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run binding tests to verify they fail**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_canonical_node_bindings.py -q`
|
||||
|
||||
Expected: failures because canonical binding fields do not exist yet.
|
||||
|
||||
- [ ] **Step 3: Implement binding models**
|
||||
|
||||
In `src/wf_core/models/steps.py`, add:
|
||||
|
||||
```python
|
||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
|
||||
|
||||
|
||||
class InputPathBinding(BaseModel):
|
||||
"""Map one graph source path into one node-local input path."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
target: LocalPath
|
||||
path: GraphSourcePath
|
||||
|
||||
|
||||
class InputValueBinding(BaseModel):
|
||||
"""Map one static JSON-compatible value into one node-local input path."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
target: LocalPath
|
||||
value: object
|
||||
|
||||
|
||||
InputBinding = Annotated[
|
||||
InputPathBinding | InputValueBinding,
|
||||
Field(union_mode="left_to_right"),
|
||||
]
|
||||
|
||||
|
||||
class OutputBinding(BaseModel):
|
||||
"""Map one node-local output path into one workflow state path."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
source: LocalPath
|
||||
target: StatePath
|
||||
```
|
||||
|
||||
Update `NodeUse`:
|
||||
|
||||
```python
|
||||
class NodeUse(BaseModel):
|
||||
...
|
||||
input: list[InputBinding] = Field(default_factory=list)
|
||||
output: list[OutputBinding] = Field(default_factory=list)
|
||||
|
||||
@model_validator(mode="before")
|
||||
@classmethod
|
||||
def _coerce_deprecated_maps(cls, data: object) -> object:
|
||||
...
|
||||
```
|
||||
|
||||
The validator should:
|
||||
|
||||
- If `input` or `output` is present, reject any of `in_map`, `input_values`, `out_map`.
|
||||
- Convert `input_values` entries to `{"target": key, "value": value}` preserving order.
|
||||
- Convert `in_map` entries to `{"target": destination, "path": source}` preserving order.
|
||||
- Convert `out_map` entries to `{"source": source, "target": destination}` preserving order.
|
||||
- Remove old keys from the normalized data.
|
||||
|
||||
- [ ] **Step 4: Run binding tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_canonical_node_bindings.py -q`
|
||||
|
||||
Expected: all tests in `test_canonical_node_bindings.py` pass.
|
||||
|
||||
## Task 3: Move Runtime Node Input Resolution To Canonical Bindings
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||
- Test: `tests/core/test_nested_mappings.py`
|
||||
- Test: `tests/core/test_canonical_node_bindings.py`
|
||||
|
||||
- [ ] **Step 1: Add runtime tests for canonical input binding behavior**
|
||||
|
||||
In `tests/core/test_nested_mappings.py`, add a test that builds the existing minimal workflow style but uses `input` / `output` instead of old maps:
|
||||
|
||||
```python
|
||||
def test_canonical_bindings_resolve_input_values_and_paths():
|
||||
workflow = Workflow.model_validate(
|
||||
{
|
||||
"name": "canonical",
|
||||
"input_schema": {"type": "object", "properties": {"message": {"type": "string"}}},
|
||||
"state_schema": {"fields": {"echoed": {"type": "string"}}},
|
||||
"output_schema": {"type": "object", "properties": {"echoed": {"type": "string"}}},
|
||||
"start": "echo",
|
||||
"node_defs": [
|
||||
{
|
||||
"name": "echo",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {"message": {"type": "string"}, "mode": {"type": "string"}},
|
||||
"required": ["message", "mode"],
|
||||
},
|
||||
"output_schema": {"type": "object", "properties": {"echoed": {"type": "string"}}},
|
||||
"outcomes": ["ok"],
|
||||
}
|
||||
],
|
||||
"nodes": [
|
||||
{
|
||||
"id": "echo",
|
||||
"type": "node",
|
||||
"node": "echo",
|
||||
"input": [
|
||||
{"target": "message", "path": "input.message"},
|
||||
{"target": "mode", "value": "fast"},
|
||||
],
|
||||
"output": [{"source": "echoed", "target": "state.echoed"}],
|
||||
}
|
||||
],
|
||||
"edges": [{"from": "echo", "outcome": "ok", "to": "__end__"}],
|
||||
}
|
||||
)
|
||||
|
||||
result = execute_workflow(
|
||||
workflow,
|
||||
{"message": "hi"},
|
||||
registry={"echo": lambda payload, _ctx: {"echoed": f"{payload['mode']}:{payload['message']}"}},
|
||||
)
|
||||
|
||||
assert result.output["echoed"] == "fast:hi"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the focused test to verify failure**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_nested_mappings.py::test_canonical_bindings_resolve_input_values_and_paths -q`
|
||||
|
||||
Expected: failure because runtime still reads `node.input_values`, `node.in_map`, and `node.out_map`.
|
||||
|
||||
- [ ] **Step 3: Update `_resolve_node_execution`**
|
||||
|
||||
In `src/wf_core/runtime/ops/nodes.py`, import binding classes and use `node.input`.
|
||||
|
||||
Implementation shape:
|
||||
|
||||
```python
|
||||
from wf_core.models.steps import InputPathBinding, InputValueBinding
|
||||
|
||||
|
||||
for binding in node.input:
|
||||
if isinstance(binding, InputValueBinding):
|
||||
value = binding.value
|
||||
else:
|
||||
value = safe_resolve_path(
|
||||
str(binding.path),
|
||||
state=run.state,
|
||||
workflow_input=run.workflow_input,
|
||||
context=context_values,
|
||||
)
|
||||
set_local_value(resolved_input, binding.target, value)
|
||||
```
|
||||
|
||||
`set_local_value` should accept `LocalPath` after Task 1.
|
||||
|
||||
- [ ] **Step 4: Run canonical runtime test**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_nested_mappings.py::test_canonical_bindings_resolve_input_values_and_paths -q`
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 4: Move Runtime Output Writes To Canonical Bindings And Atomic Patches
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||
- Test: `tests/core/test_atomic_state_patches.py`
|
||||
|
||||
- [ ] **Step 1: Write atomic patch tests**
|
||||
|
||||
Create `tests/core/test_atomic_state_patches.py`:
|
||||
|
||||
```python
|
||||
import pytest
|
||||
|
||||
from wf_core.errors import WorkflowExecutionError
|
||||
from wf_core.models.workflow import Workflow
|
||||
from wf_core.runtime.ops.state import apply_output_bindings
|
||||
|
||||
|
||||
def _workflow() -> Workflow:
|
||||
return Workflow.model_validate(
|
||||
{
|
||||
"name": "patch",
|
||||
"input_schema": {"type": "object", "properties": {}},
|
||||
"state_schema": {
|
||||
"fields": {
|
||||
"person": {"type": "object"},
|
||||
"person.name": {"type": "string"},
|
||||
}
|
||||
},
|
||||
"output_schema": {"type": "object", "properties": {}},
|
||||
"start": "n",
|
||||
"nodes": [],
|
||||
"edges": [],
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_output_bindings_commit_patch_atomically():
|
||||
workflow = _workflow()
|
||||
state = {"person": {"name": "old"}}
|
||||
|
||||
with pytest.raises(WorkflowExecutionError):
|
||||
apply_output_bindings(
|
||||
workflow,
|
||||
[
|
||||
{"source": "person.name", "target": "state.person.name"},
|
||||
{"source": "missing", "target": "state.person.extra"},
|
||||
],
|
||||
{"person": {"name": "new"}},
|
||||
state,
|
||||
)
|
||||
|
||||
assert state["person"]["name"] == "old"
|
||||
|
||||
|
||||
def test_output_bindings_reject_overlapping_write_targets():
|
||||
workflow = _workflow()
|
||||
state = {}
|
||||
|
||||
with pytest.raises(WorkflowExecutionError, match="overlapping"):
|
||||
apply_output_bindings(
|
||||
workflow,
|
||||
[
|
||||
{"source": "person", "target": "state.person"},
|
||||
{"source": "person.name", "target": "state.person.name"},
|
||||
],
|
||||
{"person": {"name": "Ada"}},
|
||||
state,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run atomic patch tests to verify failure**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_atomic_state_patches.py -q`
|
||||
|
||||
Expected: failure because `apply_output_bindings` does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement `apply_output_bindings`**
|
||||
|
||||
In `src/wf_core/runtime/ops/state.py`, add a canonical function:
|
||||
|
||||
```python
|
||||
from wf_core.models.steps import OutputBinding
|
||||
from wf_core.paths import StatePath
|
||||
|
||||
|
||||
def apply_output_bindings(
|
||||
workflow: Workflow,
|
||||
bindings: Sequence[OutputBinding],
|
||||
node_output: dict[str, Any],
|
||||
state: dict[str, Any],
|
||||
reducers: Mapping[str, ReducerDefinition] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Prepare and commit one atomic state patch from canonical output bindings."""
|
||||
```
|
||||
|
||||
Function behavior:
|
||||
|
||||
- Validate no overlapping `binding.target`.
|
||||
- Resolve every `binding.source` from `node_output` first.
|
||||
- Build a prepared patch keyed by `StatePath`.
|
||||
- Compute reducers into prepared merged values without mutating `state`.
|
||||
- Commit all prepared values only after all prior steps succeed.
|
||||
- Return JSON-friendly `dict[str, Any]` state changes using `str(path)` keys for now, until trace is separately migrated.
|
||||
|
||||
Keep `apply_output_map` as a compatibility wrapper that converts old map entries into `OutputBinding` and calls `apply_output_bindings`.
|
||||
|
||||
- [ ] **Step 4: Update node finalization**
|
||||
|
||||
In `src/wf_core/runtime/ops/nodes.py`, call `apply_output_bindings(workflow, node.output, result.output, run.state, reducers=reducers)` instead of `apply_output_map(...)`.
|
||||
|
||||
- [ ] **Step 5: Run state patch tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_atomic_state_patches.py tests/core/test_nested_mappings.py -q`
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 5: Update Validation For Canonical Bindings
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/validation/steps.py`
|
||||
- Test: `tests/core/test_mapping_validation.py`
|
||||
- Test: `tests/core/test_canonical_node_bindings.py`
|
||||
|
||||
- [ ] **Step 1: Add validation tests for canonical fields**
|
||||
|
||||
In `tests/core/test_mapping_validation.py`, add tests for invalid source paths, invalid destination paths, overlapping local input targets, and overlapping state output targets using canonical `input` / `output`.
|
||||
|
||||
Example:
|
||||
|
||||
```python
|
||||
def test_validate_workflow_reports_overlapping_canonical_output_targets():
|
||||
workflow = workflow_with_node(
|
||||
node_use={
|
||||
"id": "n",
|
||||
"type": "node",
|
||||
"node": "n",
|
||||
"output": [
|
||||
{"source": "person", "target": "state.person"},
|
||||
{"source": "person.name", "target": "state.person.name"},
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
report = workflow.validate_structure()
|
||||
|
||||
assert any(issue.code == ValidationIssueCode.INVALID_DESTINATION_PATH for issue in report.issues)
|
||||
```
|
||||
|
||||
Use the existing helper style in `tests/core/test_mapping_validation.py` rather than inventing a second full workflow factory if one already exists.
|
||||
|
||||
- [ ] **Step 2: Run mapping validation tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_mapping_validation.py -q`
|
||||
|
||||
Expected: new canonical validation tests fail until validation reads `node.input` / `node.output`.
|
||||
|
||||
- [ ] **Step 3: Update `validate_node_use`**
|
||||
|
||||
In `src/wf_core/validation/steps.py`:
|
||||
|
||||
- Iterate `node.input`.
|
||||
- For `InputValueBinding`, validate target local root against node input schema.
|
||||
- For `InputPathBinding`, validate target and source graph path.
|
||||
- Iterate `node.output`.
|
||||
- Validate output source local root against node output schema.
|
||||
- Validate destination `StatePath`.
|
||||
- Use typed overlap helpers instead of raw map values.
|
||||
- Keep issue paths readable, e.g. `nodes[0].input[1].target`.
|
||||
|
||||
- [ ] **Step 4: Run validation tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_mapping_validation.py tests/core/test_canonical_node_bindings.py -q`
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 6: Add Canonical State Schema Fields
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/models/schemas.py`
|
||||
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||
- Modify: `src/wf_core/validation/steps.py`
|
||||
- Test: `tests/core/test_nested_state_paths.py`
|
||||
- Test: `tests/core/test_schema_validation.py`
|
||||
|
||||
- [ ] **Step 1: Write state schema canonical shape tests**
|
||||
|
||||
In `tests/core/test_nested_state_paths.py`, add:
|
||||
|
||||
```python
|
||||
from wf_core.models.schemas import StateSchema
|
||||
from wf_core.paths import StatePath
|
||||
|
||||
|
||||
def test_state_schema_accepts_canonical_field_list():
|
||||
schema = StateSchema.model_validate(
|
||||
{
|
||||
"fields": [
|
||||
{"path": "state.person", "type": "object"},
|
||||
{"path": "state.person.name", "type": "string", "reducer": "wf.std.replace"},
|
||||
]
|
||||
}
|
||||
)
|
||||
|
||||
assert schema.fields[0].path == StatePath.of("person")
|
||||
assert schema.field_map()["person.name"].type == "string"
|
||||
|
||||
|
||||
def test_state_schema_accepts_deprecated_dict_shape():
|
||||
schema = StateSchema.model_validate(
|
||||
{"fields": {"person.name": {"type": "string"}}}
|
||||
)
|
||||
|
||||
assert schema.model_dump(mode="json")["fields"][0]["path"] == "state.person.name"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run state schema tests to verify failure**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py -q`
|
||||
|
||||
Expected: failure because `StateSchema.fields` is still a dict.
|
||||
|
||||
- [ ] **Step 3: Implement canonical `StateFieldDecl`**
|
||||
|
||||
In `src/wf_core/models/schemas.py`:
|
||||
|
||||
```python
|
||||
class StateFieldDecl(BaseModel):
|
||||
"""One declared state path plus validation and reducer metadata."""
|
||||
|
||||
path: StatePath
|
||||
schema: SchemaRef = Field(default_factory=lambda: SchemaRef(type="object"))
|
||||
reducer: ReducerRef = Field(default_factory=lambda: ReducerRef(name="wf.std.replace"))
|
||||
trace: bool = True
|
||||
default: Any = None
|
||||
```
|
||||
|
||||
Preserve compatibility for old `type` directly on the field:
|
||||
|
||||
- For old dict values like `{"type": "string"}`, convert to `{"schema": {"type": "string"}}`.
|
||||
- For canonical values, allow either `schema` or simple `type` as input if that keeps existing tests stable.
|
||||
|
||||
Update `StateSchema`:
|
||||
|
||||
```python
|
||||
class StateSchema(BaseModel):
|
||||
fields: list[StateFieldDecl] = Field(default_factory=list)
|
||||
|
||||
def field_map(self) -> dict[str, StateFieldDecl]:
|
||||
return {".".join(field.path.parts): field for field in self.fields}
|
||||
```
|
||||
|
||||
Add a model validator to accept old dict shape and normalize to list.
|
||||
|
||||
- [ ] **Step 4: Update callers of `workflow.state_schema.fields`**
|
||||
|
||||
Search: `rg 'state_schema\\.fields|\\.fields\\.get|set\\(workflow\\.state_schema\\.fields\\)' src tests`
|
||||
|
||||
Update code to use `workflow.state_schema.field_map()` when it needs lookup by rootless path.
|
||||
|
||||
Important updates:
|
||||
|
||||
- `src/wf_core/runtime/ops/state.py`
|
||||
- `src/wf_core/validation/steps.py`
|
||||
- any authoring or artifact code constructing state field maps.
|
||||
|
||||
- [ ] **Step 5: Run state schema tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py -q`
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 7: Harden SchemaRef With JSON Schema Validation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/models/schemas.py`
|
||||
- Modify: `src/wf_core/runtime/ops/schemas.py`
|
||||
- Test: `tests/core/test_schema_validation.py`
|
||||
|
||||
- [ ] **Step 1: Add schema validation tests**
|
||||
|
||||
In `tests/core/test_schema_validation.py`, add tests:
|
||||
|
||||
```python
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from wf_core.models.schemas import SchemaRef
|
||||
|
||||
|
||||
def test_schema_ref_accepts_valid_json_schema_with_defs():
|
||||
schema = SchemaRef.model_validate(
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"type": "object",
|
||||
"$defs": {"Name": {"type": "string"}},
|
||||
"properties": {"name": {"$ref": "#/$defs/Name"}},
|
||||
}
|
||||
)
|
||||
|
||||
assert schema.model_extra["$defs"]["Name"]["type"] == "string"
|
||||
|
||||
|
||||
def test_schema_ref_rejects_invalid_json_schema():
|
||||
with pytest.raises(ValidationError):
|
||||
SchemaRef.model_validate({"type": 123})
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run schema tests to verify failure**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_schema_validation.py -q`
|
||||
|
||||
Expected: invalid schema is currently accepted.
|
||||
|
||||
- [ ] **Step 3: Add `jsonschema` validation**
|
||||
|
||||
In `src/wf_core/models/schemas.py`, import:
|
||||
|
||||
```python
|
||||
from jsonschema import SchemaError
|
||||
from jsonschema.validators import Draft202012Validator, validator_for
|
||||
from pydantic import model_validator
|
||||
```
|
||||
|
||||
Add an after validator to `SchemaRef`:
|
||||
|
||||
```python
|
||||
@model_validator(mode="after")
|
||||
def _validate_json_schema(self) -> "SchemaRef":
|
||||
raw = self.model_dump(mode="python", exclude_none=True)
|
||||
validator_cls = validator_for(raw, default=Draft202012Validator)
|
||||
try:
|
||||
validator_cls.check_schema(raw)
|
||||
except SchemaError as exc:
|
||||
raise ValueError(f"invalid JSON Schema: {exc.message}") from exc
|
||||
return self
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run schema tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_schema_validation.py -q`
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 8: Update Authoring Helpers To Emit Canonical Bindings
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_authoring/dsl/paths.py`
|
||||
- Modify: `src/wf_authoring/dsl/conditions.py`
|
||||
- Modify: `src/wf_authoring/builder/core.py`
|
||||
- Test: `tests/authoring/test_builder.py`
|
||||
- Test: `tests/authoring/test_conditions.py`
|
||||
- Test: `tests/authoring/test_control_flow_examples.py`
|
||||
|
||||
- [ ] **Step 1: Add authoring tests for canonical dumps**
|
||||
|
||||
In `tests/authoring/test_builder.py`, add a test that builds a workflow and asserts the dumped node uses canonical `input` / `output`, not old maps:
|
||||
|
||||
```python
|
||||
def test_builder_emits_canonical_node_bindings():
|
||||
workflow = (
|
||||
WorkflowBuilder("canonical")
|
||||
.schemas(
|
||||
input_schema={"type": "object", "properties": {"message": {"type": "string"}}},
|
||||
state_schema={"fields": {"echoed": {"type": "string"}}},
|
||||
output_schema={"type": "object", "properties": {"echoed": {"type": "string"}}},
|
||||
)
|
||||
.use(echo_node, id="echo", in_map={"input.message": "message"}, out_map={"echoed": "state.echoed"})
|
||||
.start_at("echo")
|
||||
.end("echo", "ok")
|
||||
.build()
|
||||
)
|
||||
|
||||
dumped_node = workflow.model_dump(mode="json")["nodes"][0]
|
||||
assert "input" in dumped_node
|
||||
assert "output" in dumped_node
|
||||
assert "in_map" not in dumped_node
|
||||
assert "out_map" not in dumped_node
|
||||
```
|
||||
|
||||
Adapt helper names to the current builder API in the file.
|
||||
|
||||
- [ ] **Step 2: Run authoring builder tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/authoring/test_builder.py tests/authoring/test_conditions.py -q`
|
||||
|
||||
Expected: new canonical dump test may fail until builder emits or model normalizes canonical shapes.
|
||||
|
||||
- [ ] **Step 3: Update path/condition authoring wrappers**
|
||||
|
||||
In `src/wf_authoring/dsl/paths.py`, make ergonomic helpers return wrappers around core path values or values accepted by core models. Preserve existing public behavior where possible:
|
||||
|
||||
```python
|
||||
def state_path(*parts: str) -> GraphPath:
|
||||
return GraphPath(str(GraphSourcePath.state(*parts)))
|
||||
```
|
||||
|
||||
In `src/wf_authoring/dsl/conditions.py`, make `PathExpr` compile using `GraphSourcePath.parse` for `PathOperand`.
|
||||
|
||||
- [ ] **Step 4: Update builder to rely on canonical model normalization**
|
||||
|
||||
In `src/wf_authoring/builder/core.py`, either emit canonical binding dicts directly or keep passing old maps into `NodeUse.model_validate`. Prefer direct canonical emission where the builder already has enough structure.
|
||||
|
||||
Do not remove user-facing `in_map` / `out_map` builder parameters in this pass.
|
||||
|
||||
- [ ] **Step 5: Run authoring tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/authoring -q`
|
||||
|
||||
Expected: authoring tests pass.
|
||||
|
||||
## Task 9: Full Compatibility And Regression Pass
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify docs/examples only if tests show stale serialized shapes.
|
||||
- Test: full repo.
|
||||
|
||||
- [ ] **Step 1: Run core tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core tests/authoring tests/rewrite -q`
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run artifact and MCP workflow-surface tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/artifacts tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_workflow_wrappers.py tests/wf_mcp/test_mcp_workflow_surface_example.py -q`
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run full test suite**
|
||||
|
||||
Run: `uv run --with pytest pytest -q`
|
||||
|
||||
Expected: pass, allowing any existing intentionally skipped environment-dependent tests.
|
||||
|
||||
- [ ] **Step 4: Run static checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uvx ruff check
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: ruff passes and basedpyright reports 0 errors.
|
||||
|
||||
- [ ] **Step 5: Format touched files**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uvx ruff format src/wf_core src/wf_authoring tests/core tests/authoring
|
||||
```
|
||||
|
||||
Expected: files format cleanly.
|
||||
|
||||
## Self-Review Notes
|
||||
|
||||
- Spec coverage: typed paths, canonical bindings, parse-only compatibility, null/missing semantics, dynamic traversal deferral, state patch atomicity, reducer behavior, JSON Schema validation, authoring updates, and tracing shape are covered. Full trace migration is intentionally not implemented beyond returning string-keyed `state_changes` for compatibility.
|
||||
- Placeholder scan: this plan avoids `TBD` and names concrete files, tests, commands, and behavior.
|
||||
- Type consistency: `LocalPath`, `GraphSourcePath`, `StatePath`, `InputPathBinding`, `InputValueBinding`, `OutputBinding`, and `StateFieldDecl` are introduced before later tasks use them.
|
||||
@@ -0,0 +1,262 @@
|
||||
# JSON Schema State Reducers Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make `Workflow.state_schema` a normal JSON Schema object, with `reducer` as an explicit workflow extension keyword on field schemas.
|
||||
|
||||
**Architecture:** `StateSchema` should validate as JSON Schema first, then expose helper indexes for workflow runtime metadata. Runtime reducer lookup should compile from `properties` paths instead of requiring a separate path declaration list. Legacy `fields` inputs remain parse-only compatibility during the transition.
|
||||
|
||||
**Tech Stack:** Python, Pydantic v2, `jsonschema`, `wf_core` path models, pytest, basedpyright, ruff.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Canonical State Schema Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/core/test_nested_state_paths.py`
|
||||
- Modify: `tests/core/test_schema_validation.py`
|
||||
|
||||
- [ ] **Step 1: Add a test for JSON Schema property reducers**
|
||||
|
||||
```python
|
||||
def test_state_schema_uses_json_schema_properties_as_canonical_shape() -> None:
|
||||
schema = StateSchema.model_validate(
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"person": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"description": "Display name",
|
||||
"reducer": "wf.std.replace",
|
||||
}
|
||||
},
|
||||
},
|
||||
"count": {"type": "integer", "reducer": "wf.std.add"},
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
fields = schema.field_map()
|
||||
assert fields["person.name"].validation_schema.type == "string"
|
||||
assert fields["person.name"].reducer == ReducerRef(name="wf.std.replace")
|
||||
assert fields["count"].reducer == ReducerRef(name="wf.std.add")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add a dump test proving the canonical output is still JSON Schema**
|
||||
|
||||
```python
|
||||
def test_state_schema_dumps_canonical_json_schema_with_reducer_keyword() -> None:
|
||||
schema = StateSchema.model_validate(
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"count": {"type": "integer", "reducer": "wf.std.add"}
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
dumped = schema.model_dump(mode="json")
|
||||
assert dumped["type"] == "object"
|
||||
assert dumped["properties"]["count"]["type"] == "integer"
|
||||
assert dumped["properties"]["count"]["reducer"] == "wf.std.add"
|
||||
Draft202012Validator.check_schema(dumped)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add a runtime reducer lookup test from canonical schema**
|
||||
|
||||
```python
|
||||
def test_exact_nested_state_path_uses_reducer_from_json_schema_property() -> None:
|
||||
workflow = _workflow_from_state_schema(
|
||||
StateSchema.model_validate(
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"person": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"tags": {"type": "array", "reducer": "wf.std.append"}
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
)
|
||||
)
|
||||
state = {"person": {"tags": ["seed"]}}
|
||||
|
||||
write_state_value(workflow, state, "state.person.tags", ["next"])
|
||||
|
||||
assert state["person"]["tags"] == ["seed", "next"]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused tests and confirm failures**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py -q`
|
||||
|
||||
Expected: new tests fail because `StateSchema` still serializes as `fields: [...]` and reducer lookup is compiled from field declarations only.
|
||||
|
||||
### Task 2: Implement JSON-Schema-Native `StateSchema`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/models/schemas.py`
|
||||
|
||||
- [ ] **Step 1: Make `StateSchema` inherit JSON Schema fields directly**
|
||||
|
||||
`StateSchema` should expose common JSON Schema object fields:
|
||||
|
||||
```python
|
||||
title: str | None = None
|
||||
type: str | list[str] | None = "object"
|
||||
properties: dict[str, Any] = Field(default_factory=dict)
|
||||
required: list[str] = Field(default_factory=list)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Preserve legacy `fields` as parse-only input**
|
||||
|
||||
Keep accepting:
|
||||
|
||||
```json
|
||||
{
|
||||
"fields": [
|
||||
{ "path": "state.count", "type": "integer", "reducer": "wf.std.add" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
and:
|
||||
|
||||
```json
|
||||
{ "fields": { "count": { "type": "integer", "reducer": "wf.std.add" } } }
|
||||
```
|
||||
|
||||
by converting both into:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "object",
|
||||
"properties": { "count": { "type": "integer", "reducer": "wf.std.add" } }
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add `field_map()` as an internal compiled index**
|
||||
|
||||
`field_map()` should walk explicit object `properties` and return `StateFieldDecl` values keyed by rootless state path. It must:
|
||||
|
||||
- include every explicit property path
|
||||
- parse `reducer` with `ReducerRef`
|
||||
- default missing reducer to `wf.std.replace`
|
||||
- preserve `trace` and `default` workflow extension keywords
|
||||
- remove workflow extension keywords from `StateFieldDecl.validation_schema`
|
||||
|
||||
- [ ] **Step 4: Validate JSON Schema and extension keyword types**
|
||||
|
||||
Use `SchemaRef`/`jsonschema` validation for the complete state schema. Add explicit validation that:
|
||||
|
||||
- `reducer` is a string or `ReducerRef`-compatible object
|
||||
- `trace` is a boolean when present
|
||||
- `default` is allowed as JSON Schema/default metadata
|
||||
|
||||
### Task 3: Update Artifact Reducer Extraction
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/factory.py`
|
||||
|
||||
- [ ] **Step 1: Extract reducer dependencies from `state_schema.properties`**
|
||||
|
||||
Add a helper that walks explicit JSON Schema properties and yields reducer payloads from every property schema.
|
||||
|
||||
- [ ] **Step 2: Keep legacy `fields` extraction only as compatibility**
|
||||
|
||||
If `state_schema.fields` exists in old artifacts, continue reading it. Prefer canonical `properties` when present.
|
||||
|
||||
- [ ] **Step 3: Add tests through existing workflow surface/artifact tests**
|
||||
|
||||
Use an existing artifact/dependency test and assert a reducer declared at:
|
||||
|
||||
```json
|
||||
state_schema.properties.count.reducer
|
||||
```
|
||||
|
||||
is included in required capabilities.
|
||||
|
||||
### Task 4: Update Authoring Conversion
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_authoring/schemas.py`
|
||||
- Modify: `tests/authoring/test_schemas.py`
|
||||
|
||||
- [ ] **Step 1: Attach reducer metadata directly to generated property schemas**
|
||||
|
||||
When `state_schema_from(BaseModel)` sees `Annotated[..., state_field(reducer=...)]`, inject `reducer` and `trace` into that property schema instead of building a separate field map.
|
||||
|
||||
- [ ] **Step 2: Preserve model JSON Schema as the state schema**
|
||||
|
||||
Return `StateSchema.model_validate(schema_with_reducer_keywords)` so generated state schema remains JSON Schema-shaped.
|
||||
|
||||
### Task 5: Update Docs and Examples
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/core_state_mapping_and_merge.md`
|
||||
- Modify: `docs/workflow_drafts.md`
|
||||
- Modify: `docs/wf_mcp_operator_manual.md`
|
||||
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
|
||||
- Modify: `examples/raw_canonical_workflow.py`
|
||||
|
||||
- [ ] **Step 1: Replace canonical `fields: [...]` examples**
|
||||
|
||||
Use JSON Schema:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"count": {
|
||||
"type": "integer",
|
||||
"description": "Counter value",
|
||||
"reducer": "wf.std.add"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Document extension semantics**
|
||||
|
||||
State clearly that `reducer` is not standard JSON Schema behavior. JSON Schema validators ignore it; `wf_core` reads it for workflow state writes.
|
||||
|
||||
### Task 6: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py tests/authoring/test_schemas.py -q`
|
||||
|
||||
- [ ] **Step 2: Run full tests**
|
||||
|
||||
Run: `uv run --with pytest pytest -q`
|
||||
|
||||
- [ ] **Step 3: Run static checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uvx ruff check
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: covers canonical JSON Schema state shape, reducer extension keyword, compatibility, runtime lookup, artifact dependency extraction, authoring generation, docs, and verification.
|
||||
- Placeholder scan: no placeholders remain.
|
||||
- Type consistency: `StateSchema`, `StateFieldDecl`, `ReducerRef`, and `SchemaRef` names match current code.
|
||||
@@ -0,0 +1,280 @@
|
||||
# Typed Source Artifact Contracts Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace persisted dict-key source/capability contract shapes with explicit list-of-struct models backed by `SourceRef` and `CapabilityRef`.
|
||||
|
||||
**Architecture:** Keep dot-joined strings as the JSON wire format for refs, but make Python/Pydantic fields use first-class ref objects. Accept old dict shapes as parse-only compatibility and dump the new list shapes. Runtime code should use helper indexes such as `binding_map()` and `required_capability_map()` instead of depending on serialized dict keys.
|
||||
|
||||
**Tech Stack:** Python, Pydantic v2, `wf_platform` refs, `wf_artifacts` models, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Make Platform Refs Pydantic Boundary Types
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_platform/refs.py`
|
||||
- Modify: `tests/refs/test_platform_refs.py`
|
||||
|
||||
- [ ] **Step 1: Add tests for Pydantic validation and serialization**
|
||||
|
||||
Add tests that prove:
|
||||
|
||||
```python
|
||||
class Payload(BaseModel):
|
||||
source: SourceRef
|
||||
capability: CapabilityRef
|
||||
|
||||
payload = Payload.model_validate(
|
||||
{"source": "demo.personal", "capability": "demo.personal.echo_tool"}
|
||||
)
|
||||
|
||||
assert payload.source == SourceRef.parse("demo.personal")
|
||||
assert payload.capability == CapabilityRef.parse("demo.personal.echo_tool")
|
||||
assert payload.model_dump(mode="json") == {
|
||||
"source": "demo.personal",
|
||||
"capability": "demo.personal.echo_tool",
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement Pydantic core-schema hooks**
|
||||
|
||||
Use `__get_pydantic_core_schema__` on `SourceRef` and `CapabilityRef` so both accept existing instances or strings and serialize back to strings.
|
||||
|
||||
- [ ] **Step 3: Tighten segment validation modestly**
|
||||
|
||||
Reject whitespace-only refs and empty segments. Do not over-restrict valid MCP/source names yet; external systems can use dashes, underscores, and other non-empty string segments.
|
||||
|
||||
### Task 2: Convert Deployment Bindings to List-of-Struct
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/models.py`
|
||||
- Modify: `tests/artifacts/test_models.py`
|
||||
- Modify: `tests/artifacts/test_store.py`
|
||||
|
||||
- [ ] **Step 1: Add `SourceBinding`**
|
||||
|
||||
```python
|
||||
class SourceBinding(BaseModel):
|
||||
logical_source: SourceRef
|
||||
concrete_source: SourceRef
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Change `WorkflowDeployment.bindings`**
|
||||
|
||||
Canonical model field:
|
||||
|
||||
```python
|
||||
bindings: list[SourceBinding] = Field(default_factory=list)
|
||||
```
|
||||
|
||||
Parse-only compatibility:
|
||||
|
||||
```json
|
||||
{ "bindings": { "demo": "demo.personal" } }
|
||||
```
|
||||
|
||||
should normalize to:
|
||||
|
||||
```json
|
||||
{
|
||||
"bindings": [{ "logical_source": "demo", "concrete_source": "demo.personal" }]
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add `binding_map()`**
|
||||
|
||||
```python
|
||||
def binding_map(self) -> dict[str, str]:
|
||||
return {
|
||||
str(binding.logical_source): str(binding.concrete_source)
|
||||
for binding in self.bindings
|
||||
}
|
||||
```
|
||||
|
||||
Reject duplicate `logical_source` values during validation.
|
||||
|
||||
### Task 3: Convert Required Capabilities to List-of-Struct
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/models.py`
|
||||
- Modify: `src/wf_artifacts/factory.py`
|
||||
- Modify: `src/wf_artifacts/references.py`
|
||||
- Modify: `tests/artifacts/test_models.py`
|
||||
- Modify: `tests/artifacts/test_factory.py`
|
||||
|
||||
- [ ] **Step 1: Update `RequiredCapability`**
|
||||
|
||||
Canonical field:
|
||||
|
||||
```python
|
||||
ref: CapabilityRef
|
||||
```
|
||||
|
||||
Keep compatibility properties:
|
||||
|
||||
```python
|
||||
@property
|
||||
def logical_source(self) -> str: ...
|
||||
|
||||
@property
|
||||
def capability_name(self) -> str: ...
|
||||
```
|
||||
|
||||
Parse-only compatibility should accept old payloads with `logical_source` and `capability_name` and build `ref`.
|
||||
|
||||
- [ ] **Step 2: Change `WorkflowArtifact.required_capabilities`**
|
||||
|
||||
Canonical model field:
|
||||
|
||||
```python
|
||||
required_capabilities: list[RequiredCapability] = Field(default_factory=list)
|
||||
```
|
||||
|
||||
Parse-only compatibility:
|
||||
|
||||
```json
|
||||
{
|
||||
"required_capabilities": {
|
||||
"demo.echo_tool": {
|
||||
"logical_source": "demo",
|
||||
"capability_name": "echo_tool",
|
||||
"kind": "node_spec"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
should normalize to a list and dump as:
|
||||
|
||||
```json
|
||||
{
|
||||
"required_capabilities": [
|
||||
{
|
||||
"ref": "demo.echo_tool",
|
||||
"kind": "node_spec"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add `required_capability_map()`**
|
||||
|
||||
```python
|
||||
def required_capability_map(self) -> dict[str, RequiredCapability]:
|
||||
return {str(capability.ref): capability for capability in self.required_capabilities}
|
||||
```
|
||||
|
||||
Reject duplicate `ref` values during validation.
|
||||
|
||||
### Task 4: Update Call Sites to Use Helper Maps
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/validation.py`
|
||||
- Modify: `src/wf_artifacts/catalog.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/runtime_dependencies.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/tools.py`
|
||||
- Modify: `src/wf_mcp/broker/artifact_tools.py`
|
||||
- Modify tests that directly index `.bindings` or `.required_capabilities`.
|
||||
|
||||
- [ ] **Step 1: Replace `deployment.bindings.get(...)`**
|
||||
|
||||
Use:
|
||||
|
||||
```python
|
||||
bindings = deployment.binding_map()
|
||||
bound_source_id = bindings.get(required.logical_source)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Replace `artifact.required_capabilities.items()`**
|
||||
|
||||
Use:
|
||||
|
||||
```python
|
||||
for logical_ref, required in artifact.required_capability_map().items():
|
||||
...
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Keep API input compatibility**
|
||||
|
||||
MCP tools that accept `required_capabilities` from callers may still accept dict input, but constructed `WorkflowArtifact` should dump the canonical list shape.
|
||||
|
||||
### Task 5: Update Docs
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/workflow_artifacts.md`
|
||||
|
||||
- [ ] **Step 1: Replace dict binding examples**
|
||||
|
||||
Use:
|
||||
|
||||
```json
|
||||
{
|
||||
"bindings": [
|
||||
{ "logical_source": "context7", "concrete_source": "context7.default" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Replace dict required capability examples**
|
||||
|
||||
Use:
|
||||
|
||||
```json
|
||||
{
|
||||
"required_capabilities": [
|
||||
{
|
||||
"ref": "context7.query-docs",
|
||||
"kind": "node_spec",
|
||||
"input_schema_hash": "sha256:..."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
State that old dict shapes are accepted at parse boundaries but not emitted by model dumps.
|
||||
|
||||
### Task 6: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/refs/test_platform_refs.py tests/artifacts/test_models.py tests/artifacts/test_store.py tests/artifacts/test_factory.py tests/artifacts/test_validation.py -q
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run full suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run static checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uvx ruff check
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: covers typed refs, deployment binding list shape, required capability list shape, compatibility parsing, helper indexes, docs, and tests.
|
||||
- Placeholder scan: no placeholders remain.
|
||||
- Type consistency: `SourceRef`, `CapabilityRef`, `SourceBinding`, `RequiredCapability`, `WorkflowArtifact`, and `WorkflowDeployment` names match current code or are introduced in this plan.
|
||||
@@ -0,0 +1,821 @@
|
||||
# Wrapper Authoring Hints Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add enum-backed wrapper authoring hints to workflow capability inspection so LLM and human MCP clients can scaffold wrapper drafts without guessing raw workflow plans.
|
||||
|
||||
**Architecture:** Keep MCP request/response names as plain JSON strings, but compute wrapper hints in a pure workflow-surface helper. The helper uses typed Pydantic models with enum fields for confidence, outcome policy, candidate kinds, and missing-decision kinds, then `inspect_capability` attaches the hint payload to live NodeSpec and saved wrapper details.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, FastMCP-facing dict payloads, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create `src/wf_mcp/workflow_surface/wrapper_hints.py`
|
||||
- Owns enum-backed hint models.
|
||||
- Owns pure functions that derive hints from one capability contract.
|
||||
- Must not call MCP, stores, workflow runtime, or FastMCP.
|
||||
- Modify `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Imports the pure hint helper.
|
||||
- Adds `wrapper_hints` to `inspect_capability` payloads.
|
||||
- Modify `src/wf_mcp/workflow_surface/models.py`
|
||||
- Adds JSON-schema-visible response models if MCP tool schemas need stronger documentation.
|
||||
- Do this only after the helper shape stabilizes.
|
||||
- Test `tests/wf_mcp/test_workflow_wrapper_hints.py`
|
||||
- Focused unit tests for hint derivation.
|
||||
- Test `tests/wf_mcp/test_workflow_surface.py`
|
||||
- Integration tests proving `inspect_capability` includes hints.
|
||||
- Docs `docs/workflow_capabilities.md`
|
||||
- Explain that hints are scaffolding, not semantic guarantees.
|
||||
|
||||
## Vocabulary
|
||||
|
||||
Use enums for all classification/type fields. Do not return magic strings for these fields from helper internals.
|
||||
|
||||
```python
|
||||
from enum import StrEnum
|
||||
|
||||
|
||||
class WrapperHintConfidence(StrEnum):
|
||||
HIGH = "high"
|
||||
MEDIUM = "medium"
|
||||
LOW = "low"
|
||||
|
||||
|
||||
class WrapperOutcomePolicy(StrEnum):
|
||||
PRESERVE_DECLARED = "preserve_declared"
|
||||
MANUAL_MAPPING_REQUIRED = "manual_mapping_required"
|
||||
|
||||
|
||||
class OutcomeCandidateKind(StrEnum):
|
||||
BOOLEAN_CONTROL_FIELD = "boolean_control_field"
|
||||
|
||||
|
||||
class MissingDecisionKind(StrEnum):
|
||||
CHOOSE_OUTPUT_FIELDS = "choose_output_fields"
|
||||
REVIEW_NESTED_OUTPUT = "review_nested_output"
|
||||
CONFIRM_BOOLEAN_OUTCOMES = "confirm_boolean_outcomes"
|
||||
CHOOSE_ERROR_MAPPING = "choose_error_mapping"
|
||||
```
|
||||
|
||||
MCP JSON still serializes those enum values as strings.
|
||||
|
||||
## Task 1: Add Enum-Backed Hint Models
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/workflow_surface/wrapper_hints.py`
|
||||
- Test: `tests/wf_mcp/test_workflow_wrapper_hints.py`
|
||||
|
||||
- [ ] **Step 1: Write failing model serialization test**
|
||||
|
||||
Create `tests/wf_mcp/test_workflow_wrapper_hints.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_mcp.workflow_surface.wrapper_hints import (
|
||||
MissingDecision,
|
||||
MissingDecisionKind,
|
||||
OutcomeCandidate,
|
||||
OutcomeCandidateKind,
|
||||
WrapperAuthoringHints,
|
||||
WrapperHintConfidence,
|
||||
WrapperOutcomePolicy,
|
||||
)
|
||||
|
||||
|
||||
def test_wrapper_hint_models_serialize_enum_fields_as_strings() -> None:
|
||||
hints = WrapperAuthoringHints(
|
||||
capability_name="demo.personal.echo_tool",
|
||||
confidence=WrapperHintConfidence.MEDIUM,
|
||||
declared_outcomes=["ok", "error"],
|
||||
suggested_wrapper_outcomes=["ok", "error"],
|
||||
outcome_policy=WrapperOutcomePolicy.PRESERVE_DECLARED,
|
||||
input_schema={"type": "object", "properties": {}},
|
||||
state_schema={"type": "object", "properties": {}},
|
||||
output_schema={"type": "object", "properties": {}},
|
||||
input_map={},
|
||||
output_map={},
|
||||
outcome_candidates=[
|
||||
OutcomeCandidate(
|
||||
kind=OutcomeCandidateKind.BOOLEAN_CONTROL_FIELD,
|
||||
source="output.success",
|
||||
candidate_outcomes=["success", "failure"],
|
||||
confidence=WrapperHintConfidence.MEDIUM,
|
||||
reason="top-level boolean field with control-like name",
|
||||
automatic=False,
|
||||
)
|
||||
],
|
||||
missing_decisions=[
|
||||
MissingDecision(
|
||||
kind=MissingDecisionKind.CONFIRM_BOOLEAN_OUTCOMES,
|
||||
message="Confirm whether output.success should control routing.",
|
||||
)
|
||||
],
|
||||
notes=["Hints are scaffolding, not semantic guarantees."],
|
||||
)
|
||||
|
||||
dumped = hints.model_dump(mode="json")
|
||||
|
||||
assert dumped["confidence"] == "medium"
|
||||
assert dumped["outcome_policy"] == "preserve_declared"
|
||||
assert dumped["outcome_candidates"][0]["kind"] == "boolean_control_field"
|
||||
assert dumped["missing_decisions"][0]["kind"] == "confirm_boolean_outcomes"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run model test to verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py::test_wrapper_hint_models_serialize_enum_fields_as_strings -q
|
||||
```
|
||||
|
||||
Expected: import failure because `wf_mcp.workflow_surface.wrapper_hints` does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement model definitions**
|
||||
|
||||
Create `src/wf_mcp/workflow_surface/wrapper_hints.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import StrEnum
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
JsonObject = dict[str, Any]
|
||||
|
||||
|
||||
class WrapperHintConfidence(StrEnum):
|
||||
"""Coarse confidence for generated wrapper scaffolding hints."""
|
||||
|
||||
HIGH = "high"
|
||||
MEDIUM = "medium"
|
||||
LOW = "low"
|
||||
|
||||
|
||||
class WrapperOutcomePolicy(StrEnum):
|
||||
"""How wrapper outcomes were chosen."""
|
||||
|
||||
PRESERVE_DECLARED = "preserve_declared"
|
||||
MANUAL_MAPPING_REQUIRED = "manual_mapping_required"
|
||||
|
||||
|
||||
class OutcomeCandidateKind(StrEnum):
|
||||
"""Reason a field was offered as a possible outcome source."""
|
||||
|
||||
BOOLEAN_CONTROL_FIELD = "boolean_control_field"
|
||||
|
||||
|
||||
class MissingDecisionKind(StrEnum):
|
||||
"""Typed action item a human or LLM must decide before saving a wrapper."""
|
||||
|
||||
CHOOSE_OUTPUT_FIELDS = "choose_output_fields"
|
||||
REVIEW_NESTED_OUTPUT = "review_nested_output"
|
||||
CONFIRM_BOOLEAN_OUTCOMES = "confirm_boolean_outcomes"
|
||||
CHOOSE_ERROR_MAPPING = "choose_error_mapping"
|
||||
|
||||
|
||||
class OutcomeCandidate(BaseModel):
|
||||
"""One possible outcome mapping that must not be applied automatically."""
|
||||
|
||||
kind: OutcomeCandidateKind
|
||||
source: str = Field(description="Output path such as output.success.")
|
||||
candidate_outcomes: list[str]
|
||||
confidence: WrapperHintConfidence
|
||||
reason: str
|
||||
automatic: bool = False
|
||||
|
||||
|
||||
class MissingDecision(BaseModel):
|
||||
"""One explicit decision required before a wrapper should be saved."""
|
||||
|
||||
kind: MissingDecisionKind
|
||||
message: str
|
||||
|
||||
|
||||
class WrapperAuthoringHints(BaseModel):
|
||||
"""Scaffold for creating a workflow wrapper around one capability."""
|
||||
|
||||
capability_name: str
|
||||
confidence: WrapperHintConfidence
|
||||
declared_outcomes: list[str]
|
||||
suggested_wrapper_outcomes: list[str]
|
||||
outcome_policy: WrapperOutcomePolicy
|
||||
input_schema: JsonObject
|
||||
state_schema: JsonObject
|
||||
output_schema: JsonObject
|
||||
input_map: dict[str, str]
|
||||
output_map: dict[str, str]
|
||||
outcome_candidates: list[OutcomeCandidate] = Field(default_factory=list)
|
||||
missing_decisions: list[MissingDecision] = Field(default_factory=list)
|
||||
notes: list[str] = Field(default_factory=list)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run model test to verify it passes**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py::test_wrapper_hint_models_serialize_enum_fields_as_strings -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 2: Derive Simple Wrapper Hints From Capability Schemas
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/wrapper_hints.py`
|
||||
- Test: `tests/wf_mcp/test_workflow_wrapper_hints.py`
|
||||
|
||||
- [ ] **Step 1: Add failing simple-schema hint test**
|
||||
|
||||
Append to `tests/wf_mcp/test_workflow_wrapper_hints.py`:
|
||||
|
||||
```python
|
||||
from wf_mcp.workflow_surface.wrapper_hints import wrapper_hints_for_capability
|
||||
|
||||
|
||||
def test_wrapper_hints_scaffold_simple_object_input_and_output() -> None:
|
||||
hints = wrapper_hints_for_capability(
|
||||
capability_name="demo.personal.echo_tool",
|
||||
input_schema={
|
||||
"type": "object",
|
||||
"properties": {"text": {"type": "string"}},
|
||||
"required": ["text"],
|
||||
},
|
||||
output_schema={
|
||||
"type": "object",
|
||||
"properties": {"echoed": {"type": "string"}},
|
||||
"required": ["echoed"],
|
||||
},
|
||||
outcomes=["ok"],
|
||||
)
|
||||
|
||||
dumped = hints.model_dump(mode="json")
|
||||
|
||||
assert dumped["confidence"] == "high"
|
||||
assert dumped["declared_outcomes"] == ["ok"]
|
||||
assert dumped["suggested_wrapper_outcomes"] == ["ok"]
|
||||
assert dumped["outcome_policy"] == "preserve_declared"
|
||||
assert dumped["input_map"] == {"input.text": "text"}
|
||||
assert dumped["output_map"] == {"echoed": "state.echoed"}
|
||||
assert dumped["state_schema"]["properties"]["echoed"]["type"] == "string"
|
||||
assert dumped["output_schema"]["properties"]["echoed"]["type"] == "string"
|
||||
assert dumped["missing_decisions"] == []
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run simple-schema test to verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py::test_wrapper_hints_scaffold_simple_object_input_and_output -q
|
||||
```
|
||||
|
||||
Expected: import failure or attribute error for `wrapper_hints_for_capability`.
|
||||
|
||||
- [ ] **Step 3: Implement `wrapper_hints_for_capability`**
|
||||
|
||||
Add to `src/wf_mcp/workflow_surface/wrapper_hints.py`:
|
||||
|
||||
```python
|
||||
CONTROL_BOOLEAN_NAMES = {
|
||||
"success",
|
||||
"ok",
|
||||
"failed",
|
||||
"error",
|
||||
"is_error",
|
||||
"needs_input",
|
||||
"requires_approval",
|
||||
"approved",
|
||||
"rejected",
|
||||
"has_more",
|
||||
"done",
|
||||
"complete",
|
||||
}
|
||||
|
||||
|
||||
def wrapper_hints_for_capability(
|
||||
*,
|
||||
capability_name: str,
|
||||
input_schema: JsonObject,
|
||||
output_schema: JsonObject,
|
||||
outcomes: list[str] | tuple[str, ...],
|
||||
) -> WrapperAuthoringHints:
|
||||
"""Derive conservative wrapper scaffolding for one workflow capability."""
|
||||
input_properties = _object_properties(input_schema)
|
||||
output_properties = _object_properties(output_schema)
|
||||
input_map = {f"input.{name}": name for name in sorted(input_properties)}
|
||||
output_map = {name: f"state.{name}" for name in sorted(output_properties)}
|
||||
state_schema = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
name: schema for name, schema in sorted(output_properties.items())
|
||||
},
|
||||
}
|
||||
wrapper_output_schema = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
name: schema for name, schema in sorted(output_properties.items())
|
||||
},
|
||||
}
|
||||
missing_decisions = _missing_decisions_for_output(output_schema)
|
||||
outcome_candidates = _boolean_outcome_candidates(output_properties)
|
||||
if outcome_candidates:
|
||||
missing_decisions.append(
|
||||
MissingDecision(
|
||||
kind=MissingDecisionKind.CONFIRM_BOOLEAN_OUTCOMES,
|
||||
message="Confirm whether boolean output fields should control wrapper routing.",
|
||||
)
|
||||
)
|
||||
confidence = _confidence_for_hint(
|
||||
input_schema=input_schema,
|
||||
output_schema=output_schema,
|
||||
missing_decisions=missing_decisions,
|
||||
outcome_candidates=outcome_candidates,
|
||||
)
|
||||
return WrapperAuthoringHints(
|
||||
capability_name=capability_name,
|
||||
confidence=confidence,
|
||||
declared_outcomes=list(outcomes),
|
||||
suggested_wrapper_outcomes=list(outcomes),
|
||||
outcome_policy=WrapperOutcomePolicy.PRESERVE_DECLARED,
|
||||
input_schema=input_schema,
|
||||
state_schema=state_schema,
|
||||
output_schema=wrapper_output_schema,
|
||||
input_map=input_map,
|
||||
output_map=output_map,
|
||||
outcome_candidates=outcome_candidates,
|
||||
missing_decisions=missing_decisions,
|
||||
notes=[
|
||||
"Hints are scaffolding, not semantic guarantees.",
|
||||
"Declared outcomes are preserved; output-field outcome inference is not automatic.",
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
def _object_properties(schema: JsonObject) -> dict[str, JsonObject]:
|
||||
properties = schema.get("properties")
|
||||
if not isinstance(properties, dict):
|
||||
return {}
|
||||
return {
|
||||
str(name): value
|
||||
for name, value in properties.items()
|
||||
if isinstance(value, dict)
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Implement confidence and missing decision helpers**
|
||||
|
||||
Add below `_object_properties` in `src/wf_mcp/workflow_surface/wrapper_hints.py`:
|
||||
|
||||
```python
|
||||
def _missing_decisions_for_output(output_schema: JsonObject) -> list[MissingDecision]:
|
||||
properties = _object_properties(output_schema)
|
||||
if not properties:
|
||||
return [
|
||||
MissingDecision(
|
||||
kind=MissingDecisionKind.CHOOSE_OUTPUT_FIELDS,
|
||||
message="Capability output schema has no top-level object properties to map.",
|
||||
)
|
||||
]
|
||||
decisions: list[MissingDecision] = []
|
||||
for name, schema in sorted(properties.items()):
|
||||
schema_type = schema.get("type")
|
||||
if schema_type == "object" or schema_type == "array":
|
||||
decisions.append(
|
||||
MissingDecision(
|
||||
kind=MissingDecisionKind.REVIEW_NESTED_OUTPUT,
|
||||
message=f"Review output.{name}; nested or collection outputs may need explicit mapping.",
|
||||
)
|
||||
)
|
||||
return decisions
|
||||
|
||||
|
||||
def _boolean_outcome_candidates(
|
||||
output_properties: dict[str, JsonObject],
|
||||
) -> list[OutcomeCandidate]:
|
||||
candidates: list[OutcomeCandidate] = []
|
||||
for name, schema in sorted(output_properties.items()):
|
||||
if schema.get("type") != "boolean":
|
||||
continue
|
||||
if name.casefold() not in CONTROL_BOOLEAN_NAMES:
|
||||
continue
|
||||
candidates.append(
|
||||
OutcomeCandidate(
|
||||
kind=OutcomeCandidateKind.BOOLEAN_CONTROL_FIELD,
|
||||
source=f"output.{name}",
|
||||
candidate_outcomes=_candidate_outcomes_for_boolean_name(name),
|
||||
confidence=WrapperHintConfidence.MEDIUM,
|
||||
reason="top-level boolean field with control-like name",
|
||||
automatic=False,
|
||||
)
|
||||
)
|
||||
return candidates
|
||||
|
||||
|
||||
def _candidate_outcomes_for_boolean_name(name: str) -> list[str]:
|
||||
normalized = name.casefold()
|
||||
if normalized in {"success", "ok", "done", "complete"}:
|
||||
return ["success", "failure"]
|
||||
if normalized in {"failed", "error", "is_error"}:
|
||||
return ["error", "ok"]
|
||||
if normalized in {"approved", "rejected"}:
|
||||
return ["approved", "rejected"]
|
||||
if normalized in {"needs_input", "requires_approval"}:
|
||||
return [normalized, "done"]
|
||||
if normalized == "has_more":
|
||||
return ["has_more", "done"]
|
||||
return ["true", "false"]
|
||||
|
||||
|
||||
def _confidence_for_hint(
|
||||
*,
|
||||
input_schema: JsonObject,
|
||||
output_schema: JsonObject,
|
||||
missing_decisions: list[MissingDecision],
|
||||
outcome_candidates: list[OutcomeCandidate],
|
||||
) -> WrapperHintConfidence:
|
||||
if not _object_properties(input_schema) or not _object_properties(output_schema):
|
||||
return WrapperHintConfidence.LOW
|
||||
if any(
|
||||
decision.kind == MissingDecisionKind.REVIEW_NESTED_OUTPUT
|
||||
for decision in missing_decisions
|
||||
):
|
||||
return WrapperHintConfidence.LOW
|
||||
if missing_decisions or outcome_candidates:
|
||||
return WrapperHintConfidence.MEDIUM
|
||||
return WrapperHintConfidence.HIGH
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run wrapper hint tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 3: Add Boolean Outcome Candidate Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/test_workflow_wrapper_hints.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/wrapper_hints.py` only if tests reveal gaps.
|
||||
|
||||
- [ ] **Step 1: Add candidate and non-candidate tests**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_wrapper_hints_offer_boolean_outcome_candidates_without_auto_mapping() -> None:
|
||||
hints = wrapper_hints_for_capability(
|
||||
capability_name="demo.personal.submit",
|
||||
input_schema={"type": "object", "properties": {"text": {"type": "string"}}},
|
||||
output_schema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"success": {"type": "boolean"},
|
||||
"message": {"type": "string"},
|
||||
},
|
||||
},
|
||||
outcomes=["ok"],
|
||||
)
|
||||
|
||||
dumped = hints.model_dump(mode="json")
|
||||
candidate = dumped["outcome_candidates"][0]
|
||||
|
||||
assert dumped["confidence"] == "medium"
|
||||
assert candidate["kind"] == "boolean_control_field"
|
||||
assert candidate["source"] == "output.success"
|
||||
assert candidate["candidate_outcomes"] == ["success", "failure"]
|
||||
assert candidate["automatic"] is False
|
||||
assert dumped["outcome_policy"] == "preserve_declared"
|
||||
assert dumped["suggested_wrapper_outcomes"] == ["ok"]
|
||||
assert dumped["missing_decisions"][0]["kind"] == "confirm_boolean_outcomes"
|
||||
|
||||
|
||||
def test_wrapper_hints_do_not_treat_arbitrary_booleans_as_outcomes() -> None:
|
||||
hints = wrapper_hints_for_capability(
|
||||
capability_name="demo.personal.profile",
|
||||
input_schema={"type": "object", "properties": {"user_id": {"type": "string"}}},
|
||||
output_schema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"is_admin": {"type": "boolean"},
|
||||
"name": {"type": "string"},
|
||||
},
|
||||
},
|
||||
outcomes=["ok"],
|
||||
)
|
||||
|
||||
dumped = hints.model_dump(mode="json")
|
||||
|
||||
assert dumped["confidence"] == "high"
|
||||
assert dumped["outcome_candidates"] == []
|
||||
assert dumped["missing_decisions"] == []
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py -q
|
||||
```
|
||||
|
||||
Expected: pass. If arbitrary boolean fields produce candidates, fix `CONTROL_BOOLEAN_NAMES` filtering rather than deleting the test.
|
||||
|
||||
## Task 4: Add Complex Output Missing Decision Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/test_workflow_wrapper_hints.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/wrapper_hints.py` only if tests reveal gaps.
|
||||
|
||||
- [ ] **Step 1: Add nested-output and empty-output tests**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_wrapper_hints_mark_nested_outputs_as_low_confidence() -> None:
|
||||
hints = wrapper_hints_for_capability(
|
||||
capability_name="demo.personal.search",
|
||||
input_schema={"type": "object", "properties": {"query": {"type": "string"}}},
|
||||
output_schema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"results": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {"title": {"type": "string"}},
|
||||
},
|
||||
}
|
||||
},
|
||||
},
|
||||
outcomes=["ok"],
|
||||
)
|
||||
|
||||
dumped = hints.model_dump(mode="json")
|
||||
|
||||
assert dumped["confidence"] == "low"
|
||||
assert dumped["missing_decisions"][0]["kind"] == "review_nested_output"
|
||||
assert dumped["output_map"] == {"results": "state.results"}
|
||||
|
||||
|
||||
def test_wrapper_hints_mark_empty_output_schema_as_low_confidence() -> None:
|
||||
hints = wrapper_hints_for_capability(
|
||||
capability_name="demo.personal.no_output",
|
||||
input_schema={"type": "object", "properties": {"text": {"type": "string"}}},
|
||||
output_schema={"type": "object", "properties": {}},
|
||||
outcomes=["ok"],
|
||||
)
|
||||
|
||||
dumped = hints.model_dump(mode="json")
|
||||
|
||||
assert dumped["confidence"] == "low"
|
||||
assert dumped["input_map"] == {"input.text": "text"}
|
||||
assert dumped["output_map"] == {}
|
||||
assert dumped["missing_decisions"][0]["kind"] == "choose_output_fields"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 5: Wire Hints Into `inspect_capability`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Test: `tests/wf_mcp/test_workflow_surface.py`
|
||||
|
||||
- [ ] **Step 1: Add failing live capability integration test**
|
||||
|
||||
In `tests/wf_mcp/test_workflow_surface.py`, after `test_workflow_surface_inspects_one_capability`, add:
|
||||
|
||||
```python
|
||||
def test_workflow_surface_inspect_capability_includes_wrapper_hints() -> None:
|
||||
service = WfMcpService(
|
||||
store=FileStore(local_temp_root() / "surface_wrapper_hints_mcp"),
|
||||
artifact_store=FileWorkflowArtifactStore(
|
||||
local_temp_root() / "surface_wrapper_hints_artifacts"
|
||||
),
|
||||
)
|
||||
service.register_connection(
|
||||
ConnectionConfig(id="demo.personal", server="demo", account="personal")
|
||||
)
|
||||
service.register_specs("demo.personal", echo_tool)
|
||||
handlers = WorkflowSurfaceHandlers(service)
|
||||
|
||||
payload = asyncio.run(
|
||||
handlers.inspect_capability(qualified_name="demo.personal.echo_tool")
|
||||
)
|
||||
|
||||
hints = payload["wrapper_hints"]
|
||||
assert hints["capability_name"] == "demo.personal.echo_tool"
|
||||
assert hints["declared_outcomes"] == ["ok"]
|
||||
assert hints["input_map"] == {"input.text": "text"}
|
||||
assert hints["output_map"] == {"echoed": "state.echoed"}
|
||||
assert hints["outcome_policy"] == "preserve_declared"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run integration test to verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py::test_workflow_surface_inspect_capability_includes_wrapper_hints -q
|
||||
```
|
||||
|
||||
Expected: failure because `wrapper_hints` is absent.
|
||||
|
||||
- [ ] **Step 3: Attach hints for live NodeSpecs**
|
||||
|
||||
In `src/wf_mcp/workflow_surface/handlers.py`, import:
|
||||
|
||||
```python
|
||||
from .wrapper_hints import wrapper_hints_for_capability
|
||||
```
|
||||
|
||||
Then in `inspect_capability`, replace the live-detail return with:
|
||||
|
||||
```python
|
||||
detail_payload = detail.model_dump(mode="json")
|
||||
detail_payload["wrapper_hints"] = wrapper_hints_for_capability(
|
||||
capability_name=detail.name,
|
||||
input_schema=detail.input_schema,
|
||||
output_schema=detail.output_schema,
|
||||
outcomes=detail.outcomes,
|
||||
).model_dump(mode="json")
|
||||
return detail_payload
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run integration test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py::test_workflow_surface_inspect_capability_includes_wrapper_hints -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 6: Add Hints For Saved Wrapper Artifact Inspection
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Test: `tests/wf_mcp/test_workflow_surface.py`
|
||||
|
||||
- [ ] **Step 1: Add failing saved-wrapper hint assertion**
|
||||
|
||||
In existing `test_workflow_surface_inspects_saved_wrapper_capability`, add:
|
||||
|
||||
```python
|
||||
hints = payload["wrapper_hints"]
|
||||
assert hints["capability_name"] == "workflow.echo_wrapper.v1"
|
||||
assert hints["declared_outcomes"] == ["completed"]
|
||||
assert hints["suggested_wrapper_outcomes"] == ["completed"]
|
||||
assert hints["input_map"] == {"input.text": "text"}
|
||||
assert hints["output_map"] == {"echoed": "state.echoed"}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run saved-wrapper test to verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py::test_workflow_surface_inspects_saved_wrapper_capability -q
|
||||
```
|
||||
|
||||
Expected: failure because wrapper detail lacks `wrapper_hints`.
|
||||
|
||||
- [ ] **Step 3: Attach hints for wrapper artifacts**
|
||||
|
||||
In `_wrapper_capability_detail`, add `wrapper_hints` to the returned dict:
|
||||
|
||||
```python
|
||||
"wrapper_hints": wrapper_hints_for_capability(
|
||||
capability_name=_artifact_capability_id(artifact),
|
||||
input_schema=artifact.input_schema,
|
||||
output_schema=artifact.output_schema,
|
||||
outcomes=list(artifact.outcomes),
|
||||
).model_dump(mode="json"),
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run saved-wrapper test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py::test_workflow_surface_inspects_saved_wrapper_capability -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 7: Document Hint Semantics
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/workflow_capabilities.md`
|
||||
|
||||
- [ ] **Step 1: Add documentation section**
|
||||
|
||||
Add a section titled `## Wrapper Authoring Hints`:
|
||||
|
||||
```markdown
|
||||
## Wrapper Authoring Hints
|
||||
|
||||
`wf.workflow.inspect_capability` returns `wrapper_hints` for planner-visible
|
||||
capabilities. These hints are scaffolding for draft creation, not semantic
|
||||
guarantees.
|
||||
|
||||
Declared capability outcomes are authoritative and are preserved by default.
|
||||
Boolean output fields may appear as `outcome_candidates` when they have
|
||||
control-like names such as `success`, `error`, `approved`, or `has_more`, but
|
||||
they are never wired automatically. A wrapper author must explicitly confirm
|
||||
whether those fields should become routing conditions.
|
||||
|
||||
`confidence` is intentionally coarse:
|
||||
|
||||
- `high`: simple object input/output schemas and no missing decisions.
|
||||
- `medium`: usable scaffold with candidate decisions, such as boolean outcome
|
||||
candidates.
|
||||
- `low`: missing or nested output choices require explicit authoring.
|
||||
|
||||
`missing_decisions` is a typed list of decisions the author should resolve
|
||||
before saving a wrapper. MCP clients should show these prominently rather than
|
||||
treating the scaffold as complete.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run docs-adjacent tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/test_workflow_surface.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 8: Final Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_workflow_surface_refs.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run full tests if the workspace is not mid-edit**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
Expected: pass, allowing intentional environment-dependent skips. If the user is actively editing `tests/rewrite`, run focused tests only and state that full-suite verification was deferred.
|
||||
|
||||
- [ ] **Step 3: Run static checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uvx ruff check src/wf_mcp/workflow_surface tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/test_workflow_surface.py
|
||||
uv run basedpyright --level error src/wf_mcp/workflow_surface tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/test_workflow_surface.py
|
||||
uvx ruff format --check src/wf_mcp/workflow_surface tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/test_workflow_surface.py
|
||||
```
|
||||
|
||||
Expected: ruff passes, basedpyright has 0 errors, and formatting is clean.
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: The plan covers enum-backed type fields, conservative outcome suggestions, boolean candidates, confidence/missing-decision UX, inspect-capability integration, and docs.
|
||||
- Placeholder scan: No `TBD`, `TODO`, or unspecified implementation steps remain.
|
||||
- Type consistency: `WrapperHintConfidence`, `WrapperOutcomePolicy`, `OutcomeCandidateKind`, and `MissingDecisionKind` are defined before use and serialize through Pydantic models.
|
||||
- Scope check: This plan does not create or save wrappers automatically; it only adds hint payloads for authoring.
|
||||
@@ -0,0 +1,531 @@
|
||||
# Authoring Path Inputs Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make `wf_authoring` understand structural, string, iterable, and vararg path inputs consistently across DSL helpers and `WorkflowBuilder.use()`.
|
||||
|
||||
**Architecture:** Keep `wf_core` path objects as the canonical runtime model. Add one authoring coercion layer that turns ergonomic inputs into `GraphSourcePath`, `StatePath`, and `LocalPath`. Single-string helper calls parse TOML dotted-key expressions; varargs and iterables are literal path parts. Builder maps should normalize into typed path objects instead of string-to-string maps so the authoring layer stops being another dotted-string boundary.
|
||||
|
||||
**Tech Stack:** Python 3.14, stdlib `tomllib`, `wf_core.paths`, `wf_authoring.dsl`, `WorkflowBuilder`, pytest.
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
The core now supports structural path objects:
|
||||
|
||||
```json
|
||||
{ "root": "state", "parts": ["person.name", "three and four"] }
|
||||
```
|
||||
|
||||
But `wf_authoring` still stores paths as strings:
|
||||
|
||||
- `wf_authoring.dsl.paths.GraphPath.value: str`
|
||||
- `wf_authoring.dsl.mapping.PathArg = str | GraphPath`
|
||||
- `WorkflowBuilder.use(..., in_map=..., out_map=...)` normalizes maps to `dict[str, str]`
|
||||
- builder internals call `LocalPath.parse(...)`, `GraphSourcePath.parse(...)`, and `StatePath.parse(...)`
|
||||
|
||||
That means authoring helpers still risk ambiguity:
|
||||
|
||||
```python
|
||||
state("person.name")
|
||||
```
|
||||
|
||||
Today this is dotted shorthand. To express a literal field named `person.name`, users need structural/literal segment input.
|
||||
|
||||
---
|
||||
|
||||
## Semantics
|
||||
|
||||
### Single String Argument
|
||||
|
||||
Parse as TOML dotted-key expression:
|
||||
|
||||
```python
|
||||
state("person.name")
|
||||
# parts: ["person", "name"]
|
||||
|
||||
state('"person.name"')
|
||||
# parts: ["person.name"]
|
||||
|
||||
state('person."three and four"')
|
||||
# parts: ["person", "three and four"]
|
||||
```
|
||||
|
||||
### Varargs
|
||||
|
||||
Treat each argument as a literal path segment:
|
||||
|
||||
```python
|
||||
state("person.name", "email address")
|
||||
# parts: ["person.name", "email address"]
|
||||
|
||||
state_path("oh", "my", "days")
|
||||
# parts: ["oh", "my", "days"]
|
||||
```
|
||||
|
||||
### Iterable Input
|
||||
|
||||
Treat iterable items as literal path segments:
|
||||
|
||||
```python
|
||||
state(("person.name",))
|
||||
# parts: ["person.name"]
|
||||
|
||||
input_path(["user", "email"])
|
||||
# parts: ["user", "email"]
|
||||
```
|
||||
|
||||
### Existing Path Objects
|
||||
|
||||
Pass through path objects without reparsing:
|
||||
|
||||
```python
|
||||
state_path(StatePath(("person.name",)))
|
||||
input_path(GraphSourcePath.input("user"))
|
||||
```
|
||||
|
||||
### Structural Path Dicts
|
||||
|
||||
Accept structural core path dicts at authoring boundaries when data is already
|
||||
model-shaped:
|
||||
|
||||
```python
|
||||
state_path({"root": "state", "parts": ["person.name"]})
|
||||
input_path({"root": "input", "parts": ["user", "email"]})
|
||||
```
|
||||
|
||||
This keeps MCP / JSON-facing callers from converting canonical objects back
|
||||
into display strings just to pass through `wf_authoring`.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create: `src/wf_authoring/dsl/path_inputs.py`
|
||||
- Own `PathInput` type alias.
|
||||
- Own TOML key-expression parser using `tomllib`.
|
||||
- Own coercion functions for graph/local/state paths.
|
||||
|
||||
- Modify: `src/wf_authoring/dsl/paths.py`
|
||||
- Make `GraphPath` wrap `GraphSourcePath`, not a string.
|
||||
- Update `graph_path`, `input_path`, `state_path`, `context_path`.
|
||||
|
||||
- Modify: `src/wf_authoring/dsl/conditions.py`
|
||||
- Use typed `GraphSourcePath` directly from `GraphPath` / `PathExpr`.
|
||||
- Update `state(...)`, `input(...)`, and `context(...)` to accept `PathInput`.
|
||||
|
||||
- Modify: `src/wf_authoring/dsl/mapping.py`
|
||||
- Expand `PathArg` to include structural/core path objects and iterable parts.
|
||||
- Keep `bind_fields` / `bind_state` APIs stable, but normalize through the new coercers.
|
||||
|
||||
- Modify: `src/wf_authoring/builder/mapping.py`
|
||||
- Normalize `MapArg` to typed paths, not strings.
|
||||
- Keep legacy string map support.
|
||||
|
||||
- Modify: `src/wf_authoring/builder/core.py`
|
||||
- Change `_canonical_input_bindings` / `_canonical_output_bindings` to accept typed path mappings.
|
||||
- Stop reparsing paths from strings when already typed.
|
||||
|
||||
- Test:
|
||||
- `tests/authoring/test_path_inputs.py`
|
||||
- `tests/authoring/test_builder.py`
|
||||
- `tests/authoring/test_conditions.py`
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add Path Input Coercion Module
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_authoring/dsl/path_inputs.py`
|
||||
- Test: `tests/authoring/test_path_inputs.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests**
|
||||
|
||||
```python
|
||||
from wf_authoring.dsl.path_inputs import (
|
||||
coerce_graph_path,
|
||||
coerce_local_path,
|
||||
coerce_state_path,
|
||||
)
|
||||
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
|
||||
|
||||
|
||||
def test_single_string_path_input_uses_toml_dotted_key_syntax() -> None:
|
||||
assert coerce_state_path("person.name") == StatePath(("person", "name"))
|
||||
assert coerce_state_path('"person.name"') == StatePath(("person.name",))
|
||||
assert coerce_state_path('person."three and four"') == StatePath(
|
||||
("person", "three and four")
|
||||
)
|
||||
|
||||
|
||||
def test_vararg_path_input_treats_parts_as_literal_segments() -> None:
|
||||
assert coerce_state_path("person.name", "email address") == StatePath(
|
||||
("person.name", "email address")
|
||||
)
|
||||
|
||||
|
||||
def test_iterable_path_input_treats_items_as_literal_segments() -> None:
|
||||
assert coerce_local_path(("payload.text",)) == LocalPath(("payload.text",))
|
||||
|
||||
|
||||
def test_existing_path_objects_pass_through() -> None:
|
||||
source = GraphSourcePath("state", ("person.name",))
|
||||
assert coerce_graph_path(source) is source
|
||||
|
||||
|
||||
def test_structural_path_dicts_validate_through_core_models() -> None:
|
||||
assert coerce_graph_path({"root": "state", "parts": ["person.name"]}) == (
|
||||
GraphSourcePath("state", ("person.name",))
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify red**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_path_inputs.py -q
|
||||
```
|
||||
|
||||
Expected: fails because module does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement parser using `tomllib`**
|
||||
|
||||
Implement:
|
||||
|
||||
```python
|
||||
import tomllib
|
||||
from collections.abc import Iterable, Mapping
|
||||
from typing import TypeAlias
|
||||
|
||||
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
|
||||
|
||||
PathInput: TypeAlias = (
|
||||
str
|
||||
| Iterable[str]
|
||||
| Mapping[str, object]
|
||||
| GraphSourcePath
|
||||
| StatePath
|
||||
| LocalPath
|
||||
)
|
||||
```
|
||||
|
||||
Parser approach:
|
||||
|
||||
```python
|
||||
def _parse_toml_key_expr(expr: str) -> tuple[str, ...]:
|
||||
parsed = tomllib.loads(f"{expr} = true")
|
||||
...
|
||||
```
|
||||
|
||||
Walk the nested dict until the leaf value is `True`; each nested key is one path segment.
|
||||
|
||||
Rules:
|
||||
|
||||
- one `str` argument parses as TOML key expression
|
||||
- multiple `str` arguments are literal segments
|
||||
- one iterable argument is literal segments
|
||||
- existing path object passes through when compatible
|
||||
- structural dicts validate through the matching core path model
|
||||
- invalid TOML raises `ValueError` with message mentioning TOML key expression
|
||||
|
||||
- [ ] **Step 4: Run tests to verify green**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_path_inputs.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Make DSL Path Helpers Typed
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_authoring/dsl/paths.py`
|
||||
- Modify: `src/wf_authoring/dsl/conditions.py`
|
||||
- Test: `tests/authoring/test_path_inputs.py`
|
||||
- Test: `tests/authoring/test_conditions.py`
|
||||
|
||||
- [ ] **Step 1: Write failing helper tests**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from wf_authoring import state, state_path
|
||||
from wf_core.paths import GraphSourcePath
|
||||
|
||||
|
||||
def test_state_path_helper_supports_toml_strings_and_literal_varargs() -> None:
|
||||
assert state_path('"person.name"').path == GraphSourcePath(
|
||||
"state", ("person.name",)
|
||||
)
|
||||
assert state_path("person.name", "email address").path == GraphSourcePath(
|
||||
"state", ("person.name", "email address")
|
||||
)
|
||||
|
||||
|
||||
def test_state_expr_helper_uses_same_path_input_rules() -> None:
|
||||
condition = state('"person.name"').eq("Ada").to_condition()
|
||||
|
||||
assert condition.left.path == GraphSourcePath("state", ("person.name",))
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify red**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_path_inputs.py tests/authoring/test_conditions.py -q
|
||||
```
|
||||
|
||||
Expected: old helpers either split incorrectly or do not accept these signatures.
|
||||
|
||||
- [ ] **Step 3: Update `GraphPath`**
|
||||
|
||||
Change:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class GraphPath:
|
||||
path: GraphSourcePath
|
||||
|
||||
@property
|
||||
def value(self) -> str:
|
||||
return str(self.path)
|
||||
```
|
||||
|
||||
Keep `.value` as compatibility display output.
|
||||
|
||||
- [ ] **Step 4: Update helper signatures**
|
||||
|
||||
```python
|
||||
def input_path(first: PathInput, *parts: str) -> GraphPath: ...
|
||||
def state_path(first: PathInput, *parts: str) -> GraphPath: ...
|
||||
def context_path(first: PathInput, *parts: str) -> GraphPath: ...
|
||||
```
|
||||
|
||||
Use coercers from `path_inputs.py`.
|
||||
|
||||
- [ ] **Step 5: Update conditions**
|
||||
|
||||
Make `PathExpr` store `GraphSourcePath`, while keeping `.path` display property if needed:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class PathExpr:
|
||||
source: GraphSourcePath
|
||||
|
||||
@property
|
||||
def path(self) -> str:
|
||||
return str(self.source)
|
||||
```
|
||||
|
||||
Use `PathOperand(path=self.source)` instead of reparsing strings.
|
||||
|
||||
- [ ] **Step 6: Run tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_path_inputs.py tests/authoring/test_conditions.py -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Make Builder Maps Accept Typed Path Inputs
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_authoring/builder/mapping.py`
|
||||
- Modify: `src/wf_authoring/builder/core.py`
|
||||
- Modify: `src/wf_authoring/dsl/mapping.py`
|
||||
- Test: `tests/authoring/test_builder.py`
|
||||
|
||||
- [ ] **Step 1: Write failing builder tests**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from wf_authoring import input_path, state_path
|
||||
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
|
||||
|
||||
|
||||
def test_builder_use_accepts_typed_paths_and_literal_iterable_paths() -> None:
|
||||
builder = WorkflowBuilder(...)
|
||||
step = builder.use(
|
||||
auto_bind_node,
|
||||
in_map={input_path('"text.with.dot"'): ("payload.text",)},
|
||||
out_map={("payload.text",): state_path("state field")},
|
||||
)
|
||||
|
||||
assert step.input[0].path == GraphSourcePath("input", ("text.with.dot",))
|
||||
assert step.input[0].target == LocalPath(("payload.text",))
|
||||
assert step.output[0].source == LocalPath(("payload.text",))
|
||||
assert step.output[0].target == StatePath(("state field",))
|
||||
```
|
||||
|
||||
Use existing builder test fixtures in `tests/authoring/test_builder.py`.
|
||||
|
||||
- [ ] **Step 2: Run tests to verify red**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_builder.py -q
|
||||
```
|
||||
|
||||
Expected: tuple/typed map values fail.
|
||||
|
||||
- [ ] **Step 3: Update map normalization**
|
||||
|
||||
In `builder/mapping.py`, introduce typed mapping aliases:
|
||||
|
||||
```python
|
||||
InputMap = dict[GraphSourcePath, LocalPath]
|
||||
OutputMap = dict[LocalPath, StatePath]
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
normalize_input_mapping(mapping: MapArg | None) -> InputMap
|
||||
normalize_output_mapping(mapping: MapArg | None) -> OutputMap
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- input map key = graph source path
|
||||
- input map value = local path
|
||||
- output map key = local path
|
||||
- output map value = state path
|
||||
|
||||
Legacy strings still parse through the new coercers.
|
||||
|
||||
- [ ] **Step 4: Update builder core**
|
||||
|
||||
Change `_canonical_input_bindings`:
|
||||
|
||||
```python
|
||||
def _canonical_input_bindings(
|
||||
in_map: Mapping[GraphSourcePath, LocalPath],
|
||||
input_values: Mapping[LocalPath, Any],
|
||||
) -> list[InputBinding]:
|
||||
```
|
||||
|
||||
Change `_canonical_output_bindings`:
|
||||
|
||||
```python
|
||||
def _canonical_output_bindings(
|
||||
out_map: Mapping[LocalPath, StatePath],
|
||||
) -> list[OutputBinding]:
|
||||
```
|
||||
|
||||
`InputValueBinding.target` should also accept typed/local path input.
|
||||
|
||||
- [ ] **Step 5: Update DSL mapping helpers**
|
||||
|
||||
`bind_fields(**mapping)` and `bind_state(**mapping)` can keep returning dicts, but values should be normalized display/typed consistently. Prefer returning typed path maps if that does not break tests; otherwise keep their public shape and let builder normalize.
|
||||
|
||||
- [ ] **Step 6: Run tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_builder.py tests/authoring/test_demo_workflow.py tests/authoring/test_ops.py -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3.5: Foreach Boundary Check
|
||||
|
||||
**Files:**
|
||||
|
||||
- Inspect: `src/wf_authoring/builder/core.py`
|
||||
- Inspect: `src/wf_core/models/steps.py` or current foreach model location
|
||||
|
||||
`WorkflowBuilder.foreach(over=...)` also accepts path-like input today, but the
|
||||
core foreach model may still store the source path as a string. Do not let this
|
||||
block `WorkflowBuilder.use()` map normalization.
|
||||
|
||||
- [ ] **Step 1: Inspect foreach field type**
|
||||
|
||||
If core foreach already accepts `GraphSourcePath`, normalize `over` through the
|
||||
new graph-path coercer and add one focused test.
|
||||
|
||||
If core foreach still accepts only strings, keep the existing string
|
||||
serialization path and leave a short comment at the call site:
|
||||
|
||||
```text
|
||||
foreach path input should move to typed GraphSourcePath when the core foreach
|
||||
model is upgraded.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Avoid partial semantic claims**
|
||||
|
||||
Do not document foreach as fully structural until the core field is structural.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Docs and Examples
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/structural_refs.md`
|
||||
- Modify or create an authoring docs/example if one already exists.
|
||||
|
||||
- [ ] **Step 1: Add authoring examples**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
state("person.name") # TOML/dotted expression
|
||||
state('"person.name"') # literal dotted field
|
||||
state("person.name", "email") # literal segments
|
||||
state(("person.name",)) # literal iterable
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Mention builder maps**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
g.use(
|
||||
node,
|
||||
in_map={input_path('"email.address"'): ("payload.email",)},
|
||||
out_map={("result.score",): state_path("score")},
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Verification
|
||||
|
||||
- [ ] **Step 1: Run focused authoring tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring -q
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run full tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run checks**
|
||||
|
||||
```bash
|
||||
uvx ruff check src/wf_authoring src/wf_core/paths.py tests/authoring
|
||||
uv run basedpyright --level error src/wf_authoring src/wf_core/paths.py tests/authoring
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Notes
|
||||
|
||||
- Do not roll a custom TOML parser. Use stdlib `tomllib`.
|
||||
- Canonical saved JSON remains structural `root` / `parts`.
|
||||
- Single strings are ergonomic expressions. Varargs and iterables are literal segments.
|
||||
- Keep `.value` / `str(...)` as display compatibility only.
|
||||
- `WorkflowBuilder.use()` should stop being a string-to-string path boundary.
|
||||
- If Pydantic path models are not hashable enough for dict keys, normalize maps
|
||||
into explicit binding-pair lists internally instead of falling back to display
|
||||
strings.
|
||||
@@ -0,0 +1,873 @@
|
||||
# Builder Canonical Bindings Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make `WorkflowBuilder.use()` and `WorkflowBuilder.use_ref()` expose canonical `input` / `output` binding lists, while keeping `in_map`, `input_values`, and `out_map` as deprecated Python sugar.
|
||||
|
||||
**Architecture:** `wf_core.NodeUse` already stores canonical binding structs: `InputPathBinding`, `InputValueBinding`, and `OutputBinding`. The builder should accept those same structs/dicts directly, normalize them through core models, and reject mixed canonical/deprecated arguments. Map sugar remains for pleasant Python authoring, but JSON/MCP-facing callers should use binding lists so structural path dicts live inside structs, not as unhashable mapping keys.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic models from `wf_core.models.steps`, `wf_authoring.WorkflowBuilder`, pytest, basedpyright, ruff.
|
||||
|
||||
---
|
||||
|
||||
## Why the Previous Plan Did Not Finish This
|
||||
|
||||
`2026-05-21-authoring-path-inputs.md` focused on path coercion:
|
||||
|
||||
- single-string TOML path parsing
|
||||
- iterable/vararg literal segments
|
||||
- typed `GraphPath`
|
||||
- map normalization from `dict[str, str]` toward typed paths
|
||||
|
||||
That plan made `in_map`, `input_values`, and `out_map` safer, but it did not change the public builder API shape. So the current state is still incomplete:
|
||||
|
||||
```python
|
||||
g.use(node, in_map=..., input_values=..., out_map=...)
|
||||
```
|
||||
|
||||
exists, but:
|
||||
|
||||
```python
|
||||
g.use(node, input=[...], output=[...])
|
||||
```
|
||||
|
||||
does not.
|
||||
|
||||
That matters because structural path dicts cannot be Python dict keys. The JSON/MCP-friendly shape must be list-of-structs:
|
||||
|
||||
```json
|
||||
{
|
||||
"input": [
|
||||
{
|
||||
"target": { "root": "local", "parts": ["payload.email"] },
|
||||
"path": { "root": "input", "parts": ["email.address"] }
|
||||
}
|
||||
],
|
||||
"output": [
|
||||
{
|
||||
"source": { "root": "local", "parts": ["result.score"] },
|
||||
"target": { "root": "state", "parts": ["score"] }
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
Core already has the right canonical models in `src/wf_core/models/steps.py`:
|
||||
|
||||
```python
|
||||
class InputPathBinding(BaseModel):
|
||||
target: LocalPath
|
||||
path: GraphSourcePath
|
||||
|
||||
|
||||
class InputValueBinding(BaseModel):
|
||||
target: LocalPath
|
||||
value: object
|
||||
|
||||
|
||||
class OutputBinding(BaseModel):
|
||||
source: LocalPath
|
||||
target: StatePath
|
||||
|
||||
|
||||
class NodeUse(BaseModel):
|
||||
input: list[InputBinding] = Field(default_factory=list)
|
||||
output: list[OutputBinding] = Field(default_factory=list)
|
||||
```
|
||||
|
||||
Builder currently has only deprecated/sugar arguments in `src/wf_authoring/builder/core.py`:
|
||||
|
||||
```python
|
||||
def use(
|
||||
self,
|
||||
spec: NodeSpec[Any, Any],
|
||||
*,
|
||||
id: str | None = None,
|
||||
in_map: MapArg | None = None,
|
||||
input_values: Mapping[Any, Any] | None = None,
|
||||
out_map: MapArg | None = None,
|
||||
desc: str | None = None,
|
||||
) -> NodeUse:
|
||||
...
|
||||
```
|
||||
|
||||
This is the API gap.
|
||||
|
||||
---
|
||||
|
||||
## Public Semantics
|
||||
|
||||
### Canonical Builder Inputs
|
||||
|
||||
Add `input` and `output` parameters:
|
||||
|
||||
```python
|
||||
g.use(
|
||||
node,
|
||||
input=[
|
||||
{
|
||||
"target": {"root": "local", "parts": ["payload.email"]},
|
||||
"path": {"root": "input", "parts": ["email.address"]},
|
||||
},
|
||||
{
|
||||
"target": {"root": "local", "parts": ["static.limit"]},
|
||||
"value": 10,
|
||||
},
|
||||
],
|
||||
output=[
|
||||
{
|
||||
"source": {"root": "local", "parts": ["result.score"]},
|
||||
"target": {"root": "state", "parts": ["score"]},
|
||||
},
|
||||
],
|
||||
)
|
||||
```
|
||||
|
||||
Accepted item shapes:
|
||||
|
||||
- existing `InputPathBinding`
|
||||
- existing `InputValueBinding`
|
||||
- existing `OutputBinding`
|
||||
- dicts that `InputPathBinding` / `InputValueBinding` / `OutputBinding` can validate
|
||||
|
||||
### Deprecated Sugar Inputs
|
||||
|
||||
Keep these for Python authors:
|
||||
|
||||
```python
|
||||
g.use(node, in_map={state_path("text"): "payload.text"})
|
||||
g.use(node, input_values={"limit": 10})
|
||||
g.use(node, out_map={"result.score": state_path("score")})
|
||||
```
|
||||
|
||||
But mark them as deprecated in docstrings and warn when explicitly used.
|
||||
|
||||
Auto-mapping still uses the same internal sugar when both canonical and deprecated args are absent.
|
||||
|
||||
### Mixing Rules
|
||||
|
||||
Reject ambiguous combinations:
|
||||
|
||||
- `input` cannot be mixed with `in_map`
|
||||
- `input` cannot be mixed with `input_values`
|
||||
- `output` cannot be mixed with `out_map`
|
||||
|
||||
Exact error examples:
|
||||
|
||||
```text
|
||||
cannot mix canonical input with deprecated in_map/input_values
|
||||
cannot mix canonical output with deprecated out_map
|
||||
```
|
||||
|
||||
Use built-in `TypeError` for these authoring API misuse errors. This is similar
|
||||
in spirit to Pydantic's user-error category: the caller supplied an invalid API
|
||||
shape, not invalid workflow data.
|
||||
|
||||
### Structural Dict Key Rule
|
||||
|
||||
Do not support structural dicts as mapping keys. Python `dict` keys must be hashable, and adding `frozendict` support is not worth it.
|
||||
|
||||
If a user needs structural dict paths, they should use canonical binding lists:
|
||||
|
||||
```python
|
||||
input=[{"target": {"root": "local", "parts": ["payload"]}, "path": {...}}]
|
||||
```
|
||||
|
||||
Map sugar is for hashable Python authoring values only.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify: `src/wf_authoring/builder/core.py`
|
||||
- Add `input` / `output` parameters to `use()` and `use_ref()`.
|
||||
- Add canonical/deprecated mixing checks.
|
||||
- Use canonical binding normalization when provided.
|
||||
- Warn when deprecated map-sugar args are explicitly used.
|
||||
|
||||
- Modify: `src/wf_authoring/builder/mapping.py`
|
||||
- Add `InputBindingArg`, `OutputBindingArg` aliases.
|
||||
- Add `normalize_input_bindings(...)`.
|
||||
- Add `normalize_output_bindings(...)`.
|
||||
- Keep map normalizers as deprecated/sugar internals.
|
||||
|
||||
- Modify: `docs/structural_refs.md`
|
||||
- Document canonical `input` / `output` list usage for structural path dicts.
|
||||
- State that structural dicts are not supported as map keys.
|
||||
|
||||
- Modify: `docs/authoring_sketch.md` or `docs/core_state_mapping_and_merge.md`
|
||||
- Replace older “builder uses in_map/out_map” framing with “builder accepts canonical lists; maps are sugar.”
|
||||
|
||||
- Test:
|
||||
- `tests/authoring/test_builder.py`
|
||||
- Maybe `tests/authoring/test_path_inputs.py` only if structural dict errors belong there.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add Canonical Binding Normalizers
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_authoring/builder/mapping.py`
|
||||
- Test: `tests/authoring/test_builder.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for canonical dict bindings**
|
||||
|
||||
Add to `tests/authoring/test_builder.py`:
|
||||
|
||||
```python
|
||||
def test_builder_use_accepts_canonical_binding_dicts_with_structural_paths() -> None:
|
||||
builder = WorkflowBuilder(
|
||||
name="canonical_binding_dicts",
|
||||
input_schema=AutoBindInput,
|
||||
state_schema=AutoBindState,
|
||||
output_schema=AutoBindOutput,
|
||||
)
|
||||
|
||||
step = builder.use(
|
||||
auto_bind_node,
|
||||
input=[
|
||||
{
|
||||
"target": {"root": "local", "parts": ["payload.text"]},
|
||||
"path": {"root": "input", "parts": ["text.with.dot"]},
|
||||
},
|
||||
{
|
||||
"target": {"root": "local", "parts": ["static.limit"]},
|
||||
"value": 3,
|
||||
},
|
||||
],
|
||||
output=[
|
||||
{
|
||||
"source": {"root": "local", "parts": ["payload.text"]},
|
||||
"target": {"root": "state", "parts": ["text.with.dot"]},
|
||||
}
|
||||
],
|
||||
)
|
||||
|
||||
assert isinstance(step.input[0], InputPathBinding)
|
||||
assert step.input[0].path == GraphSourcePath("input", ("text.with.dot",))
|
||||
assert step.input[0].target == LocalPath(("payload.text",))
|
||||
assert isinstance(step.input[1], InputValueBinding)
|
||||
assert step.input[1].target == LocalPath(("static.limit",))
|
||||
assert step.input[1].value == 3
|
||||
assert step.output[0].source == LocalPath(("payload.text",))
|
||||
assert step.output[0].target == StatePath(("text.with.dot",))
|
||||
```
|
||||
|
||||
Update imports:
|
||||
|
||||
```python
|
||||
from wf_core.models.steps import InputPathBinding, InputValueBinding
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify red**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_use_accepts_canonical_binding_dicts_with_structural_paths -q
|
||||
```
|
||||
|
||||
Expected: fails because `WorkflowBuilder.use()` has no `input` / `output` parameters.
|
||||
|
||||
- [ ] **Step 3: Add normalizer aliases and functions**
|
||||
|
||||
In `src/wf_authoring/builder/mapping.py`, add:
|
||||
|
||||
```python
|
||||
from wf_core.models.steps import InputBinding, InputPathBinding, InputValueBinding, OutputBinding
|
||||
|
||||
InputBindingArg: TypeAlias = InputBinding | Mapping[str, object]
|
||||
OutputBindingArg: TypeAlias = OutputBinding | Mapping[str, object]
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def normalize_input_bindings(bindings: Sequence[InputBindingArg] | None) -> list[InputBinding]:
|
||||
"""Validate canonical input binding structs for WorkflowBuilder.use()."""
|
||||
if bindings is None:
|
||||
return []
|
||||
normalized: list[InputBinding] = []
|
||||
for binding in bindings:
|
||||
if isinstance(binding, InputPathBinding | InputValueBinding):
|
||||
normalized.append(binding)
|
||||
continue
|
||||
if not isinstance(binding, Mapping):
|
||||
raise TypeError(f"unsupported input binding {binding!r}")
|
||||
if "path" in binding:
|
||||
normalized.append(InputPathBinding.model_validate(binding))
|
||||
elif "value" in binding:
|
||||
normalized.append(InputValueBinding.model_validate(binding))
|
||||
else:
|
||||
raise ValueError("input binding must contain either 'path' or 'value'")
|
||||
return normalized
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def normalize_output_bindings(bindings: Sequence[OutputBindingArg] | None) -> list[OutputBinding]:
|
||||
"""Validate canonical output binding structs for WorkflowBuilder.use()."""
|
||||
if bindings is None:
|
||||
return []
|
||||
normalized: list[OutputBinding] = []
|
||||
for binding in bindings:
|
||||
if isinstance(binding, OutputBinding):
|
||||
normalized.append(binding)
|
||||
continue
|
||||
if not isinstance(binding, Mapping):
|
||||
raise TypeError(f"unsupported output binding {binding!r}")
|
||||
normalized.append(OutputBinding.model_validate(binding))
|
||||
return normalized
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused normalizer-related test**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_use_accepts_canonical_binding_dicts_with_structural_paths -q
|
||||
```
|
||||
|
||||
Expected: still fails until builder signatures are updated.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Add `input` / `output` to `use()`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_authoring/builder/core.py`
|
||||
- Test: `tests/authoring/test_builder.py`
|
||||
|
||||
- [ ] **Step 1: Update imports**
|
||||
|
||||
In `src/wf_authoring/builder/core.py`, import new aliases/functions:
|
||||
|
||||
```python
|
||||
from .mapping import (
|
||||
InputBindingArg,
|
||||
OutputBindingArg,
|
||||
normalize_input_bindings,
|
||||
normalize_output_bindings,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `use()` signature**
|
||||
|
||||
Change:
|
||||
|
||||
```python
|
||||
def use(
|
||||
self,
|
||||
spec: NodeSpec[Any, Any],
|
||||
*,
|
||||
id: str | None = None,
|
||||
in_map: MapArg | None = None,
|
||||
input_values: Mapping[Any, Any] | None = None,
|
||||
out_map: MapArg | None = None,
|
||||
desc: str | None = None,
|
||||
) -> NodeUse:
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
def use(
|
||||
self,
|
||||
spec: NodeSpec[Any, Any],
|
||||
*,
|
||||
id: str | None = None,
|
||||
input: Sequence[InputBindingArg] | None = None,
|
||||
output: Sequence[OutputBindingArg] | None = None,
|
||||
in_map: MapArg | None = None,
|
||||
input_values: Mapping[Any, Any] | None = None,
|
||||
out_map: MapArg | None = None,
|
||||
desc: str | None = None,
|
||||
) -> NodeUse:
|
||||
```
|
||||
|
||||
Import `Sequence` from `collections.abc`.
|
||||
|
||||
- [ ] **Step 3: Add mixing guard helper**
|
||||
|
||||
Add near the canonical binding helpers:
|
||||
|
||||
```python
|
||||
def _reject_mixed_binding_styles(
|
||||
*,
|
||||
input: object | None,
|
||||
output: object | None,
|
||||
in_map: object | None,
|
||||
input_values: object | None,
|
||||
out_map: object | None,
|
||||
) -> None:
|
||||
"""Keep canonical binding lists and deprecated map sugar from mixing."""
|
||||
if input is not None and (in_map is not None or input_values is not None):
|
||||
raise TypeError("cannot mix canonical input with deprecated in_map/input_values")
|
||||
if output is not None and out_map is not None:
|
||||
raise TypeError("cannot mix canonical output with deprecated out_map")
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Use canonical bindings when provided**
|
||||
|
||||
In `use()`:
|
||||
|
||||
```python
|
||||
_reject_mixed_binding_styles(
|
||||
input=input,
|
||||
output=output,
|
||||
in_map=in_map,
|
||||
input_values=input_values,
|
||||
out_map=out_map,
|
||||
)
|
||||
|
||||
if input is not None:
|
||||
node_input = normalize_input_bindings(input)
|
||||
else:
|
||||
raw_in_map = auto_input_map(...) if in_map is None else in_map
|
||||
node_input = _canonical_input_bindings(
|
||||
normalize_input_mapping(raw_in_map),
|
||||
normalize_input_values(input_values),
|
||||
)
|
||||
|
||||
if output is not None:
|
||||
node_output = normalize_output_bindings(output)
|
||||
else:
|
||||
raw_out_map = auto_output_map(...) if out_map is None else out_map
|
||||
node_output = _canonical_output_bindings(normalize_output_mapping(raw_out_map))
|
||||
```
|
||||
|
||||
Then pass:
|
||||
|
||||
```python
|
||||
input=node_input,
|
||||
output=node_output,
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run focused test**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_use_accepts_canonical_binding_dicts_with_structural_paths -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Add `input` / `output` to `use_ref()`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_authoring/builder/core.py`
|
||||
- Test: `tests/authoring/test_builder.py`
|
||||
|
||||
- [ ] **Step 1: Write failing test**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def test_builder_use_ref_accepts_canonical_binding_dicts() -> None:
|
||||
builder = WorkflowBuilder(
|
||||
name="external_ref_canonical_bindings",
|
||||
input_schema={},
|
||||
state_schema={"fields": {}},
|
||||
output_schema={},
|
||||
)
|
||||
|
||||
step = builder.use_ref(
|
||||
"demo.echo",
|
||||
id="echo",
|
||||
input=[
|
||||
{
|
||||
"target": {"root": "local", "parts": ["text"]},
|
||||
"path": {"root": "input", "parts": ["text"]},
|
||||
}
|
||||
],
|
||||
output=[
|
||||
{
|
||||
"source": {"root": "local", "parts": ["echoed"]},
|
||||
"target": {"root": "state", "parts": ["echoed"]},
|
||||
}
|
||||
],
|
||||
)
|
||||
|
||||
assert step.node == "demo.echo"
|
||||
assert isinstance(step.input[0], InputPathBinding)
|
||||
assert step.input[0].path == GraphSourcePath.input("text")
|
||||
assert step.output[0].target == StatePath.of("echoed")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `use_ref()` signature**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
input: Sequence[InputBindingArg] | None = None,
|
||||
output: Sequence[OutputBindingArg] | None = None,
|
||||
```
|
||||
|
||||
before deprecated map args.
|
||||
|
||||
- [ ] **Step 3: Use same mixing guard and normalization**
|
||||
|
||||
`use_ref()` has no auto-map fallback, so logic is simpler:
|
||||
|
||||
```python
|
||||
_reject_mixed_binding_styles(...)
|
||||
|
||||
node_input = (
|
||||
normalize_input_bindings(input)
|
||||
if input is not None
|
||||
else _canonical_input_bindings(
|
||||
normalize_input_mapping(in_map),
|
||||
normalize_input_values(input_values),
|
||||
)
|
||||
)
|
||||
node_output = (
|
||||
normalize_output_bindings(output)
|
||||
if output is not None
|
||||
else _canonical_output_bindings(normalize_output_mapping(out_map))
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused test**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_use_ref_accepts_canonical_binding_dicts -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Deprecate Map Sugar Explicitly
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_authoring/builder/core.py`
|
||||
- Test: `tests/authoring/test_builder.py`
|
||||
|
||||
- [ ] **Step 1: Add warning helper**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def _warn_deprecated_binding_sugar(
|
||||
*,
|
||||
in_map: object | None,
|
||||
input_values: object | None,
|
||||
out_map: object | None,
|
||||
) -> None:
|
||||
"""Warn when callers explicitly use map sugar instead of canonical bindings."""
|
||||
used = [
|
||||
name
|
||||
for name, value in (
|
||||
("in_map", in_map),
|
||||
("input_values", input_values),
|
||||
("out_map", out_map),
|
||||
)
|
||||
if value is not None
|
||||
]
|
||||
if not used:
|
||||
return
|
||||
warnings.warn(
|
||||
f"{', '.join(used)} are deprecated WorkflowBuilder sugar; use canonical "
|
||||
"input/output binding lists instead",
|
||||
DeprecationWarning,
|
||||
stacklevel=3,
|
||||
)
|
||||
```
|
||||
|
||||
Auto-mapping when args are omitted must not warn.
|
||||
|
||||
- [ ] **Step 2: Add warning tests**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def test_builder_warns_when_explicit_deprecated_maps_are_used() -> None:
|
||||
builder = WorkflowBuilder(
|
||||
name="deprecated_maps",
|
||||
input_schema=AutoBindInput,
|
||||
state_schema=AutoBindState,
|
||||
output_schema=AutoBindOutput,
|
||||
)
|
||||
|
||||
with pytest.warns(DeprecationWarning, match="canonical input/output"):
|
||||
builder.use(
|
||||
auto_bind_node,
|
||||
in_map={"input.text": "text"},
|
||||
out_map={"text": "state.text"},
|
||||
)
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def test_builder_auto_mapping_does_not_warn() -> None:
|
||||
builder = WorkflowBuilder(
|
||||
name="auto_map_no_warning",
|
||||
input_schema=AutoBindInput,
|
||||
state_schema=AutoBindState,
|
||||
output_schema=AutoBindOutput,
|
||||
)
|
||||
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("error", DeprecationWarning)
|
||||
builder.use(auto_bind_node)
|
||||
```
|
||||
|
||||
Import `warnings` in the test file.
|
||||
|
||||
- [ ] **Step 3: Call warning helper**
|
||||
|
||||
In `use()` and `use_ref()`, after the mixing guard:
|
||||
|
||||
```python
|
||||
_warn_deprecated_binding_sugar(
|
||||
in_map=in_map,
|
||||
input_values=input_values,
|
||||
out_map=out_map,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused warning tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_warns_when_explicit_deprecated_maps_are_used tests/authoring/test_builder.py::test_builder_auto_mapping_does_not_warn -q
|
||||
```
|
||||
|
||||
Expected: both pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Reject Mixed Styles and Dict Keys Clearly
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_authoring/builder/core.py`
|
||||
- Modify: `src/wf_authoring/builder/mapping.py`
|
||||
- Test: `tests/authoring/test_builder.py`
|
||||
|
||||
- [ ] **Step 1: Add mixed-style tests**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def test_builder_rejects_mixed_canonical_and_deprecated_input_styles() -> None:
|
||||
builder = WorkflowBuilder(
|
||||
name="mixed_input_styles",
|
||||
input_schema=AutoBindInput,
|
||||
state_schema=AutoBindState,
|
||||
output_schema=AutoBindOutput,
|
||||
)
|
||||
|
||||
with pytest.raises(TypeError, match="cannot mix canonical input"):
|
||||
builder.use(
|
||||
auto_bind_node,
|
||||
input=[{"target": "text", "path": "input.text"}],
|
||||
in_map={"input.text": "text"},
|
||||
)
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def test_builder_rejects_mixed_canonical_and_deprecated_output_styles() -> None:
|
||||
builder = WorkflowBuilder(
|
||||
name="mixed_output_styles",
|
||||
input_schema=AutoBindInput,
|
||||
state_schema=AutoBindState,
|
||||
output_schema=AutoBindOutput,
|
||||
)
|
||||
|
||||
with pytest.raises(TypeError, match="cannot mix canonical output"):
|
||||
builder.use(
|
||||
auto_bind_node,
|
||||
output=[{"source": "text", "target": "state.text"}],
|
||||
out_map={"text": "state.text"},
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add dict-key diagnostic test**
|
||||
|
||||
Python literal dicts cannot contain dict keys, so test the normalizer directly with a custom `Mapping` that yields a structural dict key:
|
||||
|
||||
```python
|
||||
class _StructuralKeyMap:
|
||||
def items(self):
|
||||
return [
|
||||
(
|
||||
{"root": "input", "parts": ["email.address"]},
|
||||
"payload.email",
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def test_input_map_rejects_structural_dict_keys_with_clear_message() -> None:
|
||||
with pytest.raises(TypeError, match="structural path dicts cannot be map keys"):
|
||||
normalize_input_mapping(_StructuralKeyMap())
|
||||
```
|
||||
|
||||
Import `normalize_input_mapping` from `wf_authoring.builder.mapping`.
|
||||
|
||||
- [ ] **Step 3: Implement dict-key guard**
|
||||
|
||||
In `normalize_input_mapping()`:
|
||||
|
||||
```python
|
||||
def _reject_mapping_path_key(value: object, *, field_name: str) -> None:
|
||||
if isinstance(value, Mapping):
|
||||
raise TypeError(
|
||||
f"structural path dicts cannot be map keys in {field_name}; "
|
||||
"use canonical input/output binding lists instead"
|
||||
)
|
||||
```
|
||||
|
||||
Call it on source keys for input maps and source keys for output maps before coercion.
|
||||
|
||||
Do not reject structural dict values, because values are allowed:
|
||||
|
||||
```python
|
||||
out_map={"result": {"root": "state", "parts": ["score"]}}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_builder.py::test_builder_rejects_mixed_canonical_and_deprecated_input_styles tests/authoring/test_builder.py::test_builder_rejects_mixed_canonical_and_deprecated_output_styles tests/authoring/test_builder.py::test_input_map_rejects_structural_dict_keys_with_clear_message -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Docs
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/structural_refs.md`
|
||||
- Modify: `docs/authoring_sketch.md`
|
||||
- Modify: `docs/core_state_mapping_and_merge.md`
|
||||
|
||||
- [ ] **Step 1: Update structural refs authoring example**
|
||||
|
||||
In `docs/structural_refs.md`, replace the current map-sugar-first example with canonical binding list example:
|
||||
|
||||
```python
|
||||
g.use(
|
||||
node,
|
||||
input=[
|
||||
{
|
||||
"target": {"root": "local", "parts": ["payload.email"]},
|
||||
"path": {"root": "input", "parts": ["email.address"]},
|
||||
}
|
||||
],
|
||||
output=[
|
||||
{
|
||||
"source": {"root": "local", "parts": ["result.score"]},
|
||||
"target": {"root": "state", "parts": ["score"]},
|
||||
}
|
||||
],
|
||||
)
|
||||
```
|
||||
|
||||
Then state:
|
||||
|
||||
```text
|
||||
`in_map`, `input_values`, and `out_map` remain deprecated Python sugar.
|
||||
Structural path dicts are not valid map keys; use canonical binding lists when
|
||||
working from JSON/MCP.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update authoring sketch**
|
||||
|
||||
In `docs/authoring_sketch.md`, update the API sketch from:
|
||||
|
||||
```python
|
||||
use(node_spec, id=..., in_map=..., out_map=...)
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
use(node_spec, id=..., input=[...], output=[...])
|
||||
```
|
||||
|
||||
Then mention:
|
||||
|
||||
```text
|
||||
`in_map`, `input_values`, and `out_map` are compatibility sugar for Python
|
||||
authors, not the preferred saved or MCP-facing shape.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update core mapping docs**
|
||||
|
||||
In `docs/core_state_mapping_and_merge.md`, ensure the docs say:
|
||||
|
||||
```text
|
||||
The canonical public shape is list-of-binding structs. Deprecated map fields
|
||||
are parse-only compatibility inputs at core level and Python sugar at builder
|
||||
level.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused authoring builder tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_builder.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run authoring tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run full tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 4: Run lint/type checks**
|
||||
|
||||
```bash
|
||||
uvx ruff check src/wf_authoring tests/authoring
|
||||
uvx ruff format --check src/wf_authoring tests/authoring docs/structural_refs.md docs/authoring_sketch.md docs/core_state_mapping_and_merge.md
|
||||
uv run basedpyright --level error src/wf_authoring tests/authoring
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- ruff check passes
|
||||
- format check passes or reports only markdown files if ruff does not handle them
|
||||
- basedpyright reports `0 errors`
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `g.use(input=[...], output=[...])` exists.
|
||||
- `g.use_ref(input=[...], output=[...])` exists.
|
||||
- `input_values` still exists, but emits `DeprecationWarning` when explicitly used.
|
||||
- `in_map` and `out_map` still exist, but emit `DeprecationWarning` when explicitly used.
|
||||
- Auto-mapping does not warn.
|
||||
- Canonical list inputs support structural path dicts inside binding structs.
|
||||
- Structural dicts as map keys are rejected with a clear message.
|
||||
- Saved/core `NodeUse` output remains canonical `input` / `output`; deprecated map fields do not reappear in dumps.
|
||||
+399
@@ -0,0 +1,399 @@
|
||||
# MCP Frontend Structural Paths and Reducers Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Expose structural graph paths, canonical builder bindings, and structural reducer refs through the MCP/workflow frontend so LLM clients stop learning dotted-string separator conventions as canonical.
|
||||
|
||||
**Architecture:** Treat MCP as a frontend over the platform model, not the source of truth. The MCP tools should accept compatibility strings where existing users need them, but list/inspect/create responses should prefer canonical structs: `input` / `output` binding lists, path `{root, parts}` objects, and reducer refs with structural `ref` objects for configured reducers. Existing raw-plan escape hatches remain, but the recommended workflow-authoring path should produce canonical model-shaped JSON.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, `wf_mcp.workflow_surface`, `wf_artifacts.drafts`, `wf_artifacts.factory`, `wf_core` models, pytest, basedpyright, ruff.
|
||||
|
||||
---
|
||||
|
||||
## Dependency
|
||||
|
||||
Run this plan **after**:
|
||||
|
||||
```text
|
||||
2026-05-21-reducer-ref-structural-capability.md
|
||||
```
|
||||
|
||||
because MCP should expose the final `ReducerRef` model shape, not invent a parallel frontend shape.
|
||||
|
||||
---
|
||||
|
||||
## Current Problems
|
||||
|
||||
MCP/workflow frontend still has several old shapes:
|
||||
|
||||
- workflow plans and draft APIs often show `in_map`, `input_values`, `out_map`
|
||||
- state schema examples still use legacy `fields`
|
||||
- reducer refs are often shown as dotted strings only
|
||||
- inspect/list tool outputs may not clearly distinguish canonical structs from display strings
|
||||
|
||||
The result: an LLM client can build runnable workflows, but it learns the wrong authoring shape and then has to guess separator semantics.
|
||||
|
||||
---
|
||||
|
||||
## Target Frontend Shape
|
||||
|
||||
Recommended node use shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "echo",
|
||||
"type": "node",
|
||||
"node": "demo.echo",
|
||||
"input": [
|
||||
{
|
||||
"target": { "root": "local", "parts": ["text"] },
|
||||
"path": { "root": "input", "parts": ["text"] }
|
||||
},
|
||||
{
|
||||
"target": { "root": "local", "parts": ["limit"] },
|
||||
"value": 3
|
||||
}
|
||||
],
|
||||
"output": [
|
||||
{
|
||||
"source": { "root": "local", "parts": ["echoed"] },
|
||||
"target": { "root": "state", "parts": ["echoed"] }
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Recommended configured reducer shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"ref": { "source": "wf.std", "capability_key": "modulo_add" },
|
||||
"config": { "modulus": 10 }
|
||||
}
|
||||
```
|
||||
|
||||
Compact unconfigured reducer shorthand remains accepted:
|
||||
|
||||
```json
|
||||
"wf.std.add"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Inspect/modify: `src/wf_mcp/workflow_surface/models.py`
|
||||
- Request/response models for create/compile/validate/call workflow tools
|
||||
|
||||
- Inspect/modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- create draft/workflow helpers
|
||||
- source/capability inspection payloads
|
||||
|
||||
- Inspect/modify: `src/wf_mcp/workflow_surface/tools.py`
|
||||
- MCP tool schemas/descriptions
|
||||
|
||||
- Inspect/modify: `src/wf_artifacts/drafts/models.py`
|
||||
- draft step/input/output shape if drafts still generate raw maps
|
||||
|
||||
- Inspect/modify: `src/wf_artifacts/drafts/adapter.py`
|
||||
- draft-to-builder compile path; should use canonical `input` / `output`
|
||||
|
||||
- Inspect/modify docs:
|
||||
- `docs/wf_mcp_operator_manual.md`
|
||||
- `docs/workflow_drafts.md`
|
||||
- `docs/wf_mcp_end_to_end_runbook.md`
|
||||
- `docs/structural_refs.md`
|
||||
|
||||
- Tests:
|
||||
- `tests/wf_mcp/test_workflow_surface.py`
|
||||
- `tests/wf_mcp/test_workflow_wrapper_hints.py`
|
||||
- `tests/artifacts/test_draft_adapter.py`
|
||||
- `tests/artifacts/test_draft_models.py`
|
||||
- `tests/artifacts/test_draft_api.py`
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Inventory MCP/Draft Surfaces That Emit Map Sugar
|
||||
|
||||
**Files:**
|
||||
|
||||
- Read-only first:
|
||||
- `src/wf_mcp/workflow_surface/models.py`
|
||||
- `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- `src/wf_artifacts/drafts/models.py`
|
||||
- `src/wf_artifacts/drafts/adapter.py`
|
||||
|
||||
- [ ] **Step 1: Search old map fields**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n '"in_map"|in_map|input_values|"out_map"|out_map|fields' src/wf_mcp src/wf_artifacts tests/wf_mcp tests/artifacts docs -g '*.py' -g '*.md'
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Categorize each hit**
|
||||
|
||||
Use these categories:
|
||||
|
||||
- compatibility input still accepted
|
||||
- canonical output should be changed
|
||||
- test fixture using old shape intentionally
|
||||
- docs/example should migrate
|
||||
|
||||
- [ ] **Step 3: Write findings into this plan or a short docs note**
|
||||
|
||||
Add a small checklist under this task before implementation. Do not blindly replace all strings.
|
||||
|
||||
Findings from the first inventory pass:
|
||||
|
||||
- `src/wf_artifacts/drafts/adapter.py` is the highest-value runtime hit: it still
|
||||
calls `WorkflowBuilder.use_ref(..., in_map=..., input_values=..., out_map=...)`
|
||||
and `WorkflowBuilder.use(..., out_map=...)`, causing deprecation warnings from
|
||||
MCP draft/workspace tests. This should be changed to canonical binding lists.
|
||||
- Raw workflow-plan tests in `tests/wf_mcp/test_service.py`,
|
||||
`tests/wf_mcp/test_broker_server.py`, `tests/wf_mcp/test_workflow_surface.py`,
|
||||
and `tests/artifacts/test_factory.py` intentionally exercise raw-plan
|
||||
compatibility. Do not bulk-rewrite those while raw-plan escape hatches remain.
|
||||
- Draft model tests still use `state_schema.fields` as compatibility input. That
|
||||
can stay as parse input, but new docs/examples should prefer JSON Schema
|
||||
`properties`.
|
||||
- Docs already explain parse-only compatibility in several places, but older
|
||||
operator/runbook examples still need canonical `input` / `output` examples.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Draft Adapter Emits Canonical Builder Bindings
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/drafts/adapter.py`
|
||||
- Modify: `tests/artifacts/test_draft_adapter.py`
|
||||
|
||||
- [ ] **Step 1: Add/adjust test**
|
||||
|
||||
Add a test proving a draft compiles through `WorkflowBuilder.use_ref(..., input=[...], output=[...])` or directly produces canonical `NodeUse.input` / `output`.
|
||||
|
||||
Expected assertion:
|
||||
|
||||
```python
|
||||
node = workflow.nodes[0]
|
||||
dumped = node.model_dump(mode="json")
|
||||
assert "in_map" not in dumped
|
||||
assert "out_map" not in dumped
|
||||
assert dumped["input"][0]["target"] == {"root": "local", "parts": ["text"]}
|
||||
assert dumped["input"][0]["path"] == {"root": "input", "parts": ["text"]}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update adapter**
|
||||
|
||||
Where it currently calls:
|
||||
|
||||
```python
|
||||
builder.use_ref(..., in_map=step.in_, input_values=..., out_map=step.out)
|
||||
```
|
||||
|
||||
convert draft structures into:
|
||||
|
||||
```python
|
||||
input=[...]
|
||||
output=[...]
|
||||
```
|
||||
|
||||
If draft models still store maps, transform them into canonical binding dicts at the adapter boundary.
|
||||
|
||||
- [ ] **Step 3: Run draft adapter tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/artifacts/test_draft_adapter.py -q
|
||||
```
|
||||
|
||||
Expected: pass and no new deprecation warnings from the adapter.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Workflow Surface Requests Prefer Canonical Shapes
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/models.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Modify: `tests/wf_mcp/test_workflow_surface.py`
|
||||
|
||||
- [ ] **Step 1: Add schema tests for canonical binding request fields**
|
||||
|
||||
Find the create/compile draft request model and assert its JSON Schema includes:
|
||||
|
||||
```json
|
||||
"input": {"type": "array", ...}
|
||||
"output": {"type": "array", ...}
|
||||
```
|
||||
|
||||
and does not force `in_map` / `out_map` as the primary example.
|
||||
|
||||
- [ ] **Step 2: Update Pydantic models**
|
||||
|
||||
Prefer these field names in MCP-facing request models:
|
||||
|
||||
```python
|
||||
input: list[InputBindingLike] = Field(default_factory=list, description=...)
|
||||
output: list[OutputBindingLike] = Field(default_factory=list, description=...)
|
||||
```
|
||||
|
||||
If compatibility maps remain:
|
||||
|
||||
```python
|
||||
in_map: dict[str, str] | None = Field(default=None, deprecated=True, description=...)
|
||||
out_map: dict[str, str] | None = Field(default=None, deprecated=True, description=...)
|
||||
```
|
||||
|
||||
If Pydantic `deprecated=True` causes schema issues, document deprecation in descriptions instead.
|
||||
|
||||
- [ ] **Step 3: Update handlers**
|
||||
|
||||
Handlers should pass canonical lists to builder/artifact APIs.
|
||||
|
||||
- [ ] **Step 4: Run workflow surface tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Inspect/List Outputs Show Canonical Refs and Display Strings Separately
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Modify: `src/wf_platform/sources.py` if inventory models need fields
|
||||
- Modify: tests in `tests/wf_mcp`
|
||||
|
||||
- [ ] **Step 1: Add response-shape assertions**
|
||||
|
||||
For source/capability inspection responses, assert reducers include enough info:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "wf.std.add",
|
||||
"ref": { "source": "wf.std", "capability_key": "add" },
|
||||
"description": "..."
|
||||
}
|
||||
```
|
||||
|
||||
Use `name` as display, `ref` as canonical.
|
||||
|
||||
- [ ] **Step 2: Update inventory models only if needed**
|
||||
|
||||
If `ReducerInventory` currently has only `name`, add:
|
||||
|
||||
```python
|
||||
ref: CapabilityRef
|
||||
```
|
||||
|
||||
or a serializable equivalent.
|
||||
|
||||
Keep old `name` for display.
|
||||
|
||||
- [ ] **Step 3: Run platform/MCP inventory tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/platform/test_inventory.py tests/wf_mcp/test_service.py tests/wf_mcp/test_workflow_surface.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Docs and MCP Tool Descriptions
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/wf_mcp_operator_manual.md`
|
||||
- Modify: `docs/workflow_drafts.md`
|
||||
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
|
||||
- Modify: `docs/structural_refs.md`
|
||||
- Modify MCP tool descriptions in `src/wf_mcp/workflow_surface/tools.py` if needed
|
||||
|
||||
- [ ] **Step 1: Replace primary examples**
|
||||
|
||||
Replace examples that teach:
|
||||
|
||||
```json
|
||||
"in_map": {"input.text": "text"}
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```json
|
||||
"input": [{"target": {"root": "local", "parts": ["text"]}, "path": {"root": "input", "parts": ["text"]}}]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Keep compatibility notes**
|
||||
|
||||
Add:
|
||||
|
||||
```text
|
||||
`in_map`, `input_values`, and `out_map` are compatibility inputs. New MCP/JSON
|
||||
clients should use `input` and `output` binding lists.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update reducer examples**
|
||||
|
||||
Show:
|
||||
|
||||
```json
|
||||
"reducer": "wf.std.add"
|
||||
```
|
||||
|
||||
for compact unconfigured reducers, and:
|
||||
|
||||
```json
|
||||
"reducer": {
|
||||
"ref": {"source": "wf.std", "capability_key": "modulo_add"},
|
||||
"config": {"modulus": 10}
|
||||
}
|
||||
```
|
||||
|
||||
for configured reducers.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Verification
|
||||
|
||||
- [ ] **Step 1: Focused artifact/MCP tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/artifacts/test_draft_adapter.py tests/artifacts/test_draft_models.py tests/artifacts/test_draft_api.py tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_workflow_wrapper_hints.py -q
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Full tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Static checks**
|
||||
|
||||
```bash
|
||||
uvx ruff check src/wf_mcp src/wf_artifacts tests/wf_mcp tests/artifacts
|
||||
uvx ruff format --check src/wf_mcp src/wf_artifacts tests/wf_mcp tests/artifacts
|
||||
uv run basedpyright --level error src/wf_mcp src/wf_artifacts tests/wf_mcp tests/artifacts
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- tests pass
|
||||
- ruff passes
|
||||
- basedpyright reports `0 errors`
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- MCP-facing examples prefer canonical binding lists.
|
||||
- Compatibility maps remain accepted where documented.
|
||||
- Draft adapter no longer emits deprecated builder sugar warnings.
|
||||
- Reducer refs display `name` and canonical `ref` distinctly where inventory exposes them.
|
||||
- No MCP handler reparses reducer dotted names by first/last dot.
|
||||
@@ -0,0 +1,433 @@
|
||||
# ReducerRef Structural Capability Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Move reducer references from ambiguous dotted strings toward structural capability refs while preserving string reducer names as parse-only shorthand.
|
||||
|
||||
**Architecture:** Reducers are source-owned capabilities, not graph paths. `ReducerRef` should carry a structural `CapabilityRef` plus config, while old `name` strings continue to validate at compatibility boundaries. Runtime reducer lookup can keep using display names temporarily through a compatibility property; artifact dependency extraction should stop reparsing dotted reducer names manually.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, `wf_platform.refs.CapabilityRef`, `wf_core.models.reducers.ReducerRef`, `wf_artifacts.factory`, pytest, basedpyright, ruff.
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
`src/wf_core/models/reducers.py`:
|
||||
|
||||
```python
|
||||
class ReducerRef(BaseModel):
|
||||
name: str
|
||||
config: dict[str, Any] = Field(default_factory=dict)
|
||||
```
|
||||
|
||||
`src/wf_artifacts/factory.py` extracts reducer dependencies by reparsing the display name:
|
||||
|
||||
```python
|
||||
reducer_ref = CapabilityRef.parse(reducer.name)
|
||||
requirements[reducer.name] = RequiredCapability(ref=reducer_ref, kind="reducer")
|
||||
```
|
||||
|
||||
This is the same separator problem in another domain. `wf.std.add` is a capability ref, not a graph path.
|
||||
|
||||
---
|
||||
|
||||
## Canonical Shape
|
||||
|
||||
New canonical reducer ref:
|
||||
|
||||
```json
|
||||
{
|
||||
"ref": { "source": "wf.std", "capability_key": "add" },
|
||||
"config": {}
|
||||
}
|
||||
```
|
||||
|
||||
Compatibility inputs:
|
||||
|
||||
```json
|
||||
"wf.std.add"
|
||||
```
|
||||
|
||||
```json
|
||||
{ "name": "wf.std.add", "config": { "modulus": 10 } }
|
||||
```
|
||||
|
||||
For now, `ReducerRef.name` remains available as a display/registry key compatibility property. Runtime reducer registries are still keyed by strings such as `wf.std.add`.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify: `src/wf_core/models/reducers.py`
|
||||
- Add `ref: CapabilityRef`
|
||||
- Keep `name` as computed/display compatibility property
|
||||
- Parse old string and old `name` object shapes
|
||||
- Dump canonical `ref` shape in JSON/Python model dumps
|
||||
|
||||
- Modify: `src/wf_artifacts/factory.py`
|
||||
- Use `reducer.ref` for required capabilities
|
||||
- Keep dependency key as `reducer.name` for now
|
||||
|
||||
- Modify tests:
|
||||
- `tests/core/test_nested_state_paths.py`
|
||||
- `tests/core/test_schema_validation.py`
|
||||
- `tests/artifacts/test_factory.py`
|
||||
- `tests/artifacts/test_validation.py` if needed
|
||||
|
||||
- Modify docs:
|
||||
- `docs/structural_refs.md`
|
||||
- `docs/core_state_mapping_and_merge.md`
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Pin ReducerRef Compatibility and Canonical Dump
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/core/test_nested_state_paths.py`
|
||||
|
||||
- [ ] **Step 1: Add reducer ref tests**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from wf_platform import CapabilityRef
|
||||
|
||||
|
||||
def test_reducer_ref_accepts_string_shorthand_and_dumps_structural_ref() -> None:
|
||||
reducer = ReducerRef.model_validate("wf.std.add")
|
||||
|
||||
assert reducer.ref == CapabilityRef(source="wf.std", capability_key="add")
|
||||
assert reducer.name == "wf.std.add"
|
||||
assert reducer.model_dump(mode="json") == {
|
||||
"ref": {"source": "wf.std", "capability_key": "add"},
|
||||
"config": {},
|
||||
}
|
||||
|
||||
|
||||
def test_reducer_ref_accepts_legacy_name_object_with_config() -> None:
|
||||
reducer = ReducerRef.model_validate({
|
||||
"name": "wf.std.modulo_add",
|
||||
"config": {"modulus": 10},
|
||||
})
|
||||
|
||||
assert reducer.ref == CapabilityRef(source="wf.std", capability_key="modulo_add")
|
||||
assert reducer.name == "wf.std.modulo_add"
|
||||
assert reducer.config == {"modulus": 10}
|
||||
|
||||
|
||||
def test_reducer_ref_accepts_canonical_ref_object() -> None:
|
||||
reducer = ReducerRef.model_validate({
|
||||
"ref": {"source": "wf.std", "capability_key": "append"},
|
||||
})
|
||||
|
||||
assert reducer.name == "wf.std.append"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run focused tests to verify red**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_string_shorthand_and_dumps_structural_ref tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_legacy_name_object_with_config tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_canonical_ref_object -q
|
||||
```
|
||||
|
||||
Expected: fail because `ReducerRef` does not parse strings and has no `ref`.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Implement Structural ReducerRef
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/models/reducers.py`
|
||||
|
||||
- [ ] **Step 1: Update imports**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from collections.abc import Mapping
|
||||
from pydantic import computed_field, model_validator
|
||||
from wf_platform import CapabilityRef
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Change model fields**
|
||||
|
||||
Change `ReducerRef` to:
|
||||
|
||||
```python
|
||||
class ReducerRef(BaseModel):
|
||||
"""Reference to one reducer capability plus JSON-compatible configuration."""
|
||||
|
||||
ref: CapabilityRef
|
||||
config: dict[str, Any] = Field(default_factory=dict)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add compatibility validator**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
@model_validator(mode="before")
|
||||
@classmethod
|
||||
def _coerce_legacy_shapes(cls, value: object) -> object:
|
||||
if isinstance(value, str):
|
||||
return {"ref": CapabilityRef.parse(value)}
|
||||
if not isinstance(value, Mapping):
|
||||
return value
|
||||
data = dict(value)
|
||||
if "ref" not in data and "name" in data:
|
||||
data["ref"] = CapabilityRef.parse(str(data.pop("name")))
|
||||
return data
|
||||
```
|
||||
|
||||
Do not parse arbitrary dotted strings anywhere else.
|
||||
|
||||
- [ ] **Step 4: Add name compatibility property**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
@computed_field
|
||||
@property
|
||||
def name(self) -> str:
|
||||
"""Display/registry compatibility key for existing reducer catalogs."""
|
||||
return str(self.ref)
|
||||
```
|
||||
|
||||
If `CapabilityRef.__str__` does not produce `source.capability_key`, use its display helper or add one there.
|
||||
|
||||
- [ ] **Step 5: Run reducer ref tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_string_shorthand_and_dumps_structural_ref tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_legacy_name_object_with_config tests/core/test_nested_state_paths.py::test_reducer_ref_accepts_canonical_ref_object -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Update Reducer Field Serializers and Existing Expectations
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/models/schemas.py`
|
||||
- Modify tests that assert reducer dumps
|
||||
|
||||
- [ ] **Step 1: Inspect current reducer dump helper**
|
||||
|
||||
Current helper:
|
||||
|
||||
```python
|
||||
def _dump_reducer_keyword(reducer: ReducerRef) -> str | dict[str, Any]:
|
||||
if not reducer.config:
|
||||
return reducer.name
|
||||
return reducer.model_dump(mode="json")
|
||||
```
|
||||
|
||||
Decide canonical output:
|
||||
|
||||
- For no-config reducers, keep string shorthand in JSON Schema `reducer` keyword for readability.
|
||||
- For configured reducers, dump canonical object:
|
||||
|
||||
```json
|
||||
{
|
||||
"ref": { "source": "wf.std", "capability_key": "modulo_add" },
|
||||
"config": { "modulus": 10 }
|
||||
}
|
||||
```
|
||||
|
||||
This keeps common schema compact while avoiding string parsing for config objects.
|
||||
|
||||
- [ ] **Step 2: Update helper**
|
||||
|
||||
Use:
|
||||
|
||||
```python
|
||||
def _dump_reducer_keyword(reducer: ReducerRef) -> str | dict[str, Any]:
|
||||
if not reducer.config:
|
||||
return reducer.name
|
||||
return reducer.model_dump(mode="json")
|
||||
```
|
||||
|
||||
This may already work after `ReducerRef.model_dump()` changes. Keep the helper but update tests.
|
||||
|
||||
- [ ] **Step 3: Run schema tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_schema_validation.py tests/core/test_nested_state_paths.py -q
|
||||
```
|
||||
|
||||
Expected: pass after updating expectations for configured reducer dumps if needed.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Update Artifact Reducer Dependency Extraction
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/factory.py`
|
||||
- Modify: `tests/artifacts/test_factory.py`
|
||||
|
||||
- [ ] **Step 1: Add/adjust artifact test**
|
||||
|
||||
In `tests/artifacts/test_factory.py`, ensure reducer dependencies assert structural refs:
|
||||
|
||||
```python
|
||||
def test_create_workflow_artifact_from_plan_adds_reducer_dependencies() -> None:
|
||||
...
|
||||
reducer = artifact.required_capability_map()["wf.std.max"]
|
||||
assert reducer.ref.source == "wf.std"
|
||||
assert reducer.ref.capability_key == "max"
|
||||
assert reducer.logical_source == "wf.std"
|
||||
assert reducer.capability_name == "max"
|
||||
assert reducer.kind == "reducer"
|
||||
```
|
||||
|
||||
Add a configured reducer payload test:
|
||||
|
||||
```python
|
||||
def test_create_workflow_artifact_from_plan_accepts_structural_reducer_ref() -> None:
|
||||
plan = minimal_plan()
|
||||
plan["state_schema"] = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"score": {
|
||||
"type": "integer",
|
||||
"reducer": {
|
||||
"ref": {"source": "wf.std", "capability_key": "max"},
|
||||
"config": {},
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
artifact = create_workflow_artifact_from_plan(...)
|
||||
|
||||
assert "wf.std.max" in artifact.required_capability_map()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update extraction**
|
||||
|
||||
Change:
|
||||
|
||||
```python
|
||||
reducer_ref = CapabilityRef.parse(reducer.name)
|
||||
requirements[reducer.name] = RequiredCapability(ref=reducer_ref, kind="reducer")
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
requirements[reducer.name] = RequiredCapability(ref=reducer.ref, kind="reducer")
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run artifact tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/artifacts/test_factory.py tests/artifacts/test_validation.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Runtime Compatibility Check
|
||||
|
||||
**Files:**
|
||||
|
||||
- Tests only unless failures require runtime changes
|
||||
|
||||
- [ ] **Step 1: Run reducer runtime tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_atomic_state_patches.py -q
|
||||
```
|
||||
|
||||
Expected: pass because `reducer.name` remains a compatibility registry key.
|
||||
|
||||
- [ ] **Step 2: If runtime fails**
|
||||
|
||||
Only if needed, update lookup code to use `reducer.name` as the compatibility string key. Do not make runtime registries structural in this pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Docs
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/structural_refs.md`
|
||||
- Modify: `docs/core_state_mapping_and_merge.md`
|
||||
|
||||
- [ ] **Step 1: Update reducer docs**
|
||||
|
||||
In `docs/structural_refs.md`, replace temporary wording with:
|
||||
|
||||
```text
|
||||
Canonical configured reducer refs use `ref`:
|
||||
|
||||
{
|
||||
"ref": {"source": "wf.std", "capability_key": "modulo_add"},
|
||||
"config": {"modulus": 10}
|
||||
}
|
||||
|
||||
String reducer names such as `wf.std.add` remain shorthand for unconfigured
|
||||
reducers and compatibility display.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update merge docs**
|
||||
|
||||
In `docs/core_state_mapping_and_merge.md`, update examples with both compact and configured forms:
|
||||
|
||||
```json
|
||||
"reducer": "wf.std.add"
|
||||
```
|
||||
|
||||
and:
|
||||
|
||||
```json
|
||||
"reducer": {
|
||||
"ref": {"source": "wf.std", "capability_key": "modulo_add"},
|
||||
"config": {"modulus": 10}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Verification
|
||||
|
||||
- [ ] **Step 1: Focused tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py tests/artifacts/test_factory.py tests/artifacts/test_validation.py -q
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Full tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Static checks**
|
||||
|
||||
```bash
|
||||
uvx ruff check src/wf_core src/wf_artifacts tests/core tests/artifacts
|
||||
uvx ruff format --check src/wf_core src/wf_artifacts tests/core tests/artifacts
|
||||
uv run basedpyright --level error src/wf_core src/wf_artifacts tests/core tests/artifacts
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- tests pass
|
||||
- ruff passes
|
||||
- basedpyright reports `0 errors`
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `ReducerRef` canonical shape has `ref`, not only `name`.
|
||||
- String shorthand still parses.
|
||||
- Legacy `{name, config}` still parses.
|
||||
- Runtime reducer lookup still works through `reducer.name`.
|
||||
- Artifact dependency extraction uses `reducer.ref`.
|
||||
- No graph path parser is used for reducer refs.
|
||||
+570
@@ -0,0 +1,570 @@
|
||||
# State Schema and Reducer Ref Path Sweep Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Remove remaining dotted-string ambiguity from state field paths, then prepare `ReducerRef` to stop treating reducer capability names as opaque dotted strings.
|
||||
|
||||
**Architecture:** Do this in two independent passes. First, make `StateSchema` / `StateFieldDecl` preserve `StatePath` semantics internally and serialize state paths structurally where possible. Second, introduce a structural reducer capability ref while keeping string reducer names as parse-only shorthand. The state-schema pass is the immediate correctness fix; reducer refs are a follow-up because they touch artifacts/source dependencies.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, `wf_core.paths.StatePath`, `wf_platform.refs.CapabilityRef`, `wf_core.models.schemas`, pytest, basedpyright, ruff.
|
||||
|
||||
---
|
||||
|
||||
## Why This Plan Exists
|
||||
|
||||
We just moved graph/node bindings toward structural paths:
|
||||
|
||||
```json
|
||||
{ "root": "state", "parts": ["person.name"] }
|
||||
```
|
||||
|
||||
But state schema indexing still builds rootless dotted strings in places:
|
||||
|
||||
```python
|
||||
path = f"{prefix}.{name}"
|
||||
StatePath.of(path)
|
||||
```
|
||||
|
||||
That can corrupt JSON Schema property names containing dots:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"person.name": { "type": "string", "reducer": "wf.std.replace" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The intended state path is:
|
||||
|
||||
```text
|
||||
state -> "person.name"
|
||||
```
|
||||
|
||||
not:
|
||||
|
||||
```text
|
||||
state -> person -> name
|
||||
```
|
||||
|
||||
Reducer refs have a similar-looking but different issue. `wf.std.add` is not a graph path; it is a capability ref. That cleanup should use `CapabilityRef`, not `StatePath`.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope Now
|
||||
|
||||
- Keep exact JSON Schema property names as literal `StatePath.parts`.
|
||||
- Add typed state-field indexing keyed by `StatePath`.
|
||||
- Keep old string-keyed `field_map()` compatibility.
|
||||
- Make `StateFieldDecl.path` dumps structural if the rest of the core path dump has already moved structural.
|
||||
- Add tests proving literal dotted property names are not split.
|
||||
- Document that state schema paths and reducer refs are different domains.
|
||||
|
||||
### Follow-Up Scope
|
||||
|
||||
- Change `ReducerRef` to carry a structural `CapabilityRef`.
|
||||
- Keep `ReducerRef(name="wf.std.add")` shorthand as parse-only compatibility.
|
||||
- Update artifact dependency extraction to use structural reducer refs.
|
||||
|
||||
Do not mix these two passes unless the state schema work forces a reducer model touch.
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
Relevant files:
|
||||
|
||||
- `src/wf_core/models/schemas.py`
|
||||
- `StateFieldDecl.path: StatePath`
|
||||
- `StateFieldDecl._serialize_path()` currently returns `str(path)`
|
||||
- `StateSchema.field_map()` returns `dict[str, StateFieldDecl]`
|
||||
- `_iter_state_field_declarations(...)` builds `path` as dotted string
|
||||
- `_set_state_property_schema(...)` receives `path_parts`
|
||||
|
||||
- `src/wf_core/models/reducers.py`
|
||||
- `ReducerRef.name: str`
|
||||
|
||||
- `src/wf_authoring/schemas.py`
|
||||
- `_iter_model_metadata(...)` builds rootless dotted strings from Pydantic field names
|
||||
- `_flatten_state_properties(...)` and `_lookup_mutable_property_schema(...)` split strings with `.`
|
||||
|
||||
- `src/wf_core/runtime/ops/state.py`
|
||||
- uses `workflow.state_schema.field_map()` and string keys
|
||||
|
||||
Important distinction:
|
||||
|
||||
```text
|
||||
State path: graph data path, should use StatePath
|
||||
Reducer name: source capability ref, should use CapabilityRef later
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Pin Literal Dotted State Property Behavior
|
||||
|
||||
**Files:**
|
||||
|
||||
- Test: `tests/core/test_nested_state_paths.py`
|
||||
- Test: `tests/core/test_schema_validation.py`
|
||||
|
||||
- [ ] **Step 1: Add failing StateSchema field-index test**
|
||||
|
||||
Add to `tests/core/test_nested_state_paths.py`:
|
||||
|
||||
```python
|
||||
def test_state_schema_preserves_literal_dotted_property_names() -> None:
|
||||
schema = StateSchema.model_validate({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"person.name": {"type": "string", "reducer": "wf.std.replace"}
|
||||
},
|
||||
})
|
||||
|
||||
fields = schema.field_index()
|
||||
|
||||
assert set(fields) == {StatePath(("person.name",))}
|
||||
assert fields[StatePath(("person.name",))].path == StatePath(("person.name",))
|
||||
```
|
||||
|
||||
Expected failure: `field_index()` does not exist, or the path is split as `("person", "name")`.
|
||||
|
||||
- [ ] **Step 2: Add compatibility `field_map()` test**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def test_state_schema_field_map_keeps_display_key_for_literal_dotted_property() -> None:
|
||||
schema = StateSchema.model_validate({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"person.name": {"type": "string", "reducer": "wf.std.replace"}
|
||||
},
|
||||
})
|
||||
|
||||
fields = schema.field_map()
|
||||
|
||||
assert set(fields) == {"person.name"}
|
||||
assert fields["person.name"].path == StatePath(("person.name",))
|
||||
```
|
||||
|
||||
This keeps old callers alive but makes the value typed/correct.
|
||||
|
||||
- [ ] **Step 3: Run focused tests to verify red**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_nested_state_paths.py::test_state_schema_preserves_literal_dotted_property_names tests/core/test_nested_state_paths.py::test_state_schema_field_map_keeps_display_key_for_literal_dotted_property -q
|
||||
```
|
||||
|
||||
Expected: fail before implementation.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Add Typed State Field Index
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/models/schemas.py`
|
||||
- Test: `tests/core/test_nested_state_paths.py`
|
||||
|
||||
- [ ] **Step 1: Add path-parts traversal helper**
|
||||
|
||||
In `src/wf_core/models/schemas.py`, replace string-prefix recursion with tuple path parts.
|
||||
|
||||
Add helper:
|
||||
|
||||
```python
|
||||
def _append_state_part(prefix: tuple[str, ...], name: str) -> tuple[str, ...]:
|
||||
"""Append one JSON Schema property name as one literal StatePath segment."""
|
||||
return (*prefix, name)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add `field_index()`**
|
||||
|
||||
Add to `StateSchema`:
|
||||
|
||||
```python
|
||||
def field_index(self) -> dict[StatePath, StateFieldDecl]:
|
||||
"""Return reducer-aware declarations keyed by exact typed state path."""
|
||||
root_schema = self.model_dump(mode="json", exclude_none=True)
|
||||
return {
|
||||
path: field
|
||||
for path, field in _iter_state_field_declarations(
|
||||
self.properties,
|
||||
root_schema,
|
||||
prefix=(),
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Make `field_map()` compatibility wrapper**
|
||||
|
||||
Change `field_map()` to:
|
||||
|
||||
```python
|
||||
def field_map(self) -> dict[str, StateFieldDecl]:
|
||||
"""Return reducer-aware declarations keyed by rootless display path."""
|
||||
return {".".join(path.parts): field for path, field in self.field_index().items()}
|
||||
```
|
||||
|
||||
Note: this display map is ambiguous for literal dotted segments, but values are correct. New runtime code should move to `field_index()`.
|
||||
|
||||
- [ ] **Step 4: Update `_iter_state_field_declarations` signature**
|
||||
|
||||
Change from string prefix:
|
||||
|
||||
```python
|
||||
prefix: str
|
||||
) -> Iterator[tuple[str, StateFieldDecl]]:
|
||||
```
|
||||
|
||||
to typed prefix:
|
||||
|
||||
```python
|
||||
prefix: tuple[str, ...]
|
||||
) -> Iterator[tuple[StatePath, StateFieldDecl]]:
|
||||
```
|
||||
|
||||
Inside loop:
|
||||
|
||||
```python
|
||||
path_parts = _append_state_part(prefix, name)
|
||||
path = StatePath(path_parts)
|
||||
display_path = ".".join(path.parts)
|
||||
```
|
||||
|
||||
Use `display_path` only in error messages and reducer validation labels.
|
||||
|
||||
- [ ] **Step 5: Update yielded `StateFieldDecl` construction**
|
||||
|
||||
Change:
|
||||
|
||||
```python
|
||||
"path": StatePath.of(path),
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
"path": path,
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Update recursive calls**
|
||||
|
||||
Pass:
|
||||
|
||||
```python
|
||||
prefix=path.parts
|
||||
```
|
||||
|
||||
not a dotted string.
|
||||
|
||||
- [ ] **Step 7: Run focused tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_nested_state_paths.py::test_state_schema_preserves_literal_dotted_property_names tests/core/test_nested_state_paths.py::test_state_schema_field_map_keeps_display_key_for_literal_dotted_property -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Move Runtime Lookup to Typed State Paths
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||
- Test: `tests/core/test_nested_state_paths.py`
|
||||
- Test: `tests/core/test_atomic_state_patches.py`
|
||||
|
||||
- [ ] **Step 1: Inspect current runtime lookup**
|
||||
|
||||
Current likely shape:
|
||||
|
||||
```python
|
||||
state_fields = workflow.state_schema.field_map()
|
||||
field = state_fields.get(".".join(path.parts))
|
||||
```
|
||||
|
||||
This should move to `field_index()` where available.
|
||||
|
||||
- [ ] **Step 2: Update runtime type hints**
|
||||
|
||||
Change helpers from:
|
||||
|
||||
```python
|
||||
state_fields: Mapping[str, StateFieldDecl]
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
state_fields: Mapping[StatePath, StateFieldDecl]
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Use typed lookup**
|
||||
|
||||
When resolving reducer for a write target:
|
||||
|
||||
```python
|
||||
declared_field = state_fields.get(target)
|
||||
```
|
||||
|
||||
where `target` is already a `StatePath`.
|
||||
|
||||
If code currently has only path parts, construct:
|
||||
|
||||
```python
|
||||
target = StatePath(tuple(path_parts))
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update affected-field overlap helper**
|
||||
|
||||
If `_affected_state_fields(...)` compares string prefixes, make it compare tuple parts:
|
||||
|
||||
```python
|
||||
def _is_prefix(prefix: tuple[str, ...], parts: tuple[str, ...]) -> bool:
|
||||
return parts[: len(prefix)] == prefix
|
||||
```
|
||||
|
||||
This preserves exact path semantics without reparsing dotted display text.
|
||||
|
||||
- [ ] **Step 5: Run runtime-focused tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_atomic_state_patches.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Structural `StateFieldDecl.path` Dump
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/models/schemas.py`
|
||||
- Test: `tests/core/test_nested_state_paths.py`
|
||||
- Test: `tests/core/test_schema_validation.py`
|
||||
|
||||
- [ ] **Step 1: Check current expectations**
|
||||
|
||||
Existing tests may expect:
|
||||
|
||||
```python
|
||||
{"path": "state.person.name"}
|
||||
```
|
||||
|
||||
Decide based on current path model direction. Since `StatePath` now serializes structurally elsewhere, prefer:
|
||||
|
||||
```json
|
||||
{ "path": { "root": "state", "parts": ["person.name"] } }
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Change serializer**
|
||||
|
||||
Remove this serializer:
|
||||
|
||||
```python
|
||||
@field_serializer("path")
|
||||
def _serialize_path(self, path: StatePath) -> str:
|
||||
return str(path)
|
||||
```
|
||||
|
||||
or change it to:
|
||||
|
||||
```python
|
||||
@field_serializer("path")
|
||||
def _serialize_path(self, path: StatePath) -> dict[str, str | list[str]]:
|
||||
return StatePath._serialize(path)
|
||||
```
|
||||
|
||||
Prefer removal if Pydantic uses the existing `StatePath` serializer correctly.
|
||||
|
||||
- [ ] **Step 3: Update tests**
|
||||
|
||||
Update or add:
|
||||
|
||||
```python
|
||||
def test_state_field_decl_model_dump_serializes_path_structurally() -> None:
|
||||
field = StateFieldDecl(path=StatePath(("person.name",)), schema={"type": "string"})
|
||||
|
||||
dumped = field.model_dump(mode="json")
|
||||
|
||||
assert dumped["path"] == {"root": "state", "parts": ["person.name"]}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused schema tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py -q
|
||||
```
|
||||
|
||||
Expected: pass after expectation updates.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Authoring State Metadata Path Sweep
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_authoring/schemas.py`
|
||||
- Test: `tests/authoring/test_schemas.py`
|
||||
|
||||
- [ ] **Step 1: Add failing authoring test for literal dotted Pydantic field alias if possible**
|
||||
|
||||
If Pydantic field aliases are already used in this project, add:
|
||||
|
||||
```python
|
||||
class DotAliasState(BaseModel):
|
||||
person_name: Annotated[
|
||||
str,
|
||||
Field(alias="person.name"),
|
||||
state_field(reducer="wf.std.replace"),
|
||||
]
|
||||
|
||||
|
||||
def test_state_schema_from_preserves_literal_dotted_alias_paths() -> None:
|
||||
schema = state_schema_from(DotAliasState)
|
||||
|
||||
fields = schema.field_index()
|
||||
|
||||
assert StatePath(("person.name",)) in fields
|
||||
```
|
||||
|
||||
If field aliases are not supported by the current authoring schema flow, document that Python model field names remain Python identifiers and alias path support is out of scope.
|
||||
|
||||
- [ ] **Step 2: Replace string path traversal with tuple parts**
|
||||
|
||||
In `src/wf_authoring/schemas.py`, update metadata collection helpers:
|
||||
|
||||
```python
|
||||
def _iter_model_metadata(
|
||||
model_type: type[BaseModel],
|
||||
*,
|
||||
prefix: tuple[str, ...] = (),
|
||||
) -> Iterator[tuple[tuple[str, ...], StateFieldMetadata]]:
|
||||
```
|
||||
|
||||
Use one literal segment per field name or alias:
|
||||
|
||||
```python
|
||||
field_name = field_info.alias or name
|
||||
path = (*prefix, field_name)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update lookup helpers to accept tuple parts**
|
||||
|
||||
Change:
|
||||
|
||||
```python
|
||||
_lookup_mutable_property_schema(schema_payload, path)
|
||||
_state_field_default(value, path, property_schema)
|
||||
```
|
||||
|
||||
to tuple-based forms:
|
||||
|
||||
```python
|
||||
_lookup_mutable_property_schema(schema_payload, path_parts)
|
||||
_state_field_default(value, path_parts, property_schema)
|
||||
```
|
||||
|
||||
Use field name lookup carefully; default lookup for nested aliases may need to remain conservative.
|
||||
|
||||
- [ ] **Step 4: Run authoring schema tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_schemas.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: ReducerRef Capability Ref Plan Stub
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/structural_refs.md`
|
||||
- Create: `docs/superpowers/plans/YYYY-MM-DD-reducer-ref-structural-capability.md`
|
||||
|
||||
- [ ] **Step 1: Document reducer refs are capability refs**
|
||||
|
||||
In `docs/structural_refs.md`, add:
|
||||
|
||||
```text
|
||||
Reducer refs are capability refs, not graph paths. `wf.std.add` is shorthand
|
||||
for source `wf.std`, capability key `add`. The reducer cleanup should move
|
||||
ReducerRef toward structural CapabilityRef while keeping string reducer names
|
||||
as parse-only shorthand.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Create follow-up plan stub**
|
||||
|
||||
Create a separate plan with only the intended boundary:
|
||||
|
||||
- `ReducerRef.name: str` remains compatibility display/shorthand for now.
|
||||
- Add `ReducerRef.ref: CapabilityRef` or replace `name` with a `CapabilityRef` after artifact/source dependency code is ready.
|
||||
- Update artifact dependency extraction from reducer refs.
|
||||
- Keep configured reducers as `{ref/name, config}`.
|
||||
|
||||
Do not implement reducer structural refs in the state-schema path sweep unless the user explicitly asks to combine them.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused core tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_atomic_state_patches.py tests/core/test_schema_validation.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run authoring schema tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/authoring/test_schemas.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run full tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 4: Run lint/type checks**
|
||||
|
||||
```bash
|
||||
uvx ruff check src/wf_core src/wf_authoring tests/core tests/authoring
|
||||
uvx ruff format --check src/wf_core src/wf_authoring tests/core tests/authoring
|
||||
uv run basedpyright --level error src/wf_core src/wf_authoring tests/core tests/authoring
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- ruff check passes
|
||||
- format check passes
|
||||
- basedpyright reports `0 errors`
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- JSON Schema property names containing dots stay one `StatePath` segment.
|
||||
- Runtime reducer lookup uses `StatePath`, not rootless dotted strings.
|
||||
- `field_map()` remains available for compatibility but is not the preferred internal API.
|
||||
- `StateFieldDecl.path` no longer forces string serialization if the project has moved to structural path JSON.
|
||||
- Reducer refs are documented as capability refs, not graph paths.
|
||||
- Reducer structural ref implementation is not accidentally mixed into the state schema path sweep.
|
||||
@@ -0,0 +1,644 @@
|
||||
# Structural Capability Refs Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Stop treating dotted capability names as authoritative data by making source/capability/workflow refs structural in saved artifacts and internal runtime paths.
|
||||
|
||||
**Architecture:** Public MCP/tool arguments may keep accepting old string names for compatibility, but model validation normalizes them into structural refs immediately. Saved artifacts, deployments, and generated JSON should write structured fields; display strings are derived only for UI/tool readability. Runtime resolution binds by explicit source fields, not by guessing where a dot-separated string should split.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic models, `wf_platform` refs, `wf_artifacts` models, `wf_mcp` workflow surface and broker service tests.
|
||||
|
||||
---
|
||||
|
||||
## Current Problem
|
||||
|
||||
Dotted names currently carry multiple meanings:
|
||||
|
||||
- `context7.default.query-docs` means provider/profile/capability.
|
||||
- `wf.std.replace` means system source/capability.
|
||||
- `workflow.echo_wrapper.v1` means workflow artifact/version.
|
||||
- `demo.foo.bar` may mean source `demo` with capability key `foo.bar`, or source `demo.foo` with capability key `bar`.
|
||||
|
||||
No parser can infer the right boundary from the string alone. Dotted strings are still useful as display names and human-entered compatibility inputs, but they must not be the stored source of truth.
|
||||
|
||||
## Canonical Shapes
|
||||
|
||||
### Source Identity
|
||||
|
||||
Use `SourceRef` for concrete or logical source identity.
|
||||
|
||||
```json
|
||||
{
|
||||
"provider": "context7",
|
||||
"profile": "default"
|
||||
}
|
||||
```
|
||||
|
||||
System sources may omit profile:
|
||||
|
||||
```json
|
||||
{
|
||||
"provider": "wf.std",
|
||||
"profile": null
|
||||
}
|
||||
```
|
||||
|
||||
For the first implementation pass, this can still serialize through the current `SourceRef` string internally, but the plan should leave room for `profile: null`.
|
||||
|
||||
### Capability Ref
|
||||
|
||||
Use an object with a source and a local key:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": "demo",
|
||||
"capability_key": "foo.bar"
|
||||
}
|
||||
```
|
||||
|
||||
`capability_key` is the name inside a known source. It may contain dots. It is not parsed for source meaning.
|
||||
|
||||
### Workflow Artifact Capability Ref
|
||||
|
||||
Workflow artifacts are not external source capabilities. Store them separately:
|
||||
|
||||
```json
|
||||
{
|
||||
"artifact_id": "echo_wrapper",
|
||||
"version": 1
|
||||
}
|
||||
```
|
||||
|
||||
The display string `workflow.echo_wrapper.v1` remains computable for list output and old input parsing.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_platform/refs.py`
|
||||
- Add structural capability ref input/output support.
|
||||
- Keep compact-string parsing for old inputs.
|
||||
- Make JSON serialization emit object shape for canonical saves.
|
||||
|
||||
- Modify `src/wf_artifacts/refs.py`
|
||||
- Add Pydantic validation/serialization for `WorkflowCapabilityRef`.
|
||||
- Accept old `workflow.<artifact_id>.v<version>` strings as parse-only input.
|
||||
- Serialize new saves as `{"artifact_id": "...", "version": 1}`.
|
||||
|
||||
- Modify `src/wf_artifacts/models.py`
|
||||
- Make `RequiredCapability.ref` canonical structural JSON.
|
||||
- Keep `logical_source` and `capability_name` as compatibility accessors only.
|
||||
- Keep accepting old dict/map/string shapes.
|
||||
- Make `WorkflowDeployment.bindings` save as structural list, not dict.
|
||||
|
||||
- Modify `src/wf_artifacts/references.py`
|
||||
- Replace string-concatenated `logical_ref` construction with structural `CapabilityRef`.
|
||||
- Return display strings only where legacy maps still require them.
|
||||
|
||||
- Modify `src/wf_mcp/workflow_surface/refs.py`
|
||||
- Parse workflow-surface ids into a union of structural refs.
|
||||
- Keep old string input compatibility for MCP tool calls.
|
||||
|
||||
- Modify `src/wf_mcp/workflow_surface/runtime_dependencies.py`
|
||||
- Keep source-prefix binding logic for legacy plan node strings.
|
||||
- Add a path for structural plan node refs when the plan model supports them.
|
||||
- Keep the current dotted-local-name regression test.
|
||||
|
||||
- Modify `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- When saving artifacts from plans/drafts/workspaces, emit structural required capability refs.
|
||||
- Keep response display names for humans/LLMs.
|
||||
|
||||
- Add or modify tests:
|
||||
- `tests/wf_platform/test_refs.py`
|
||||
- `tests/wf_artifacts/test_refs.py`
|
||||
- `tests/wf_mcp/test_workflow_surface_refs.py`
|
||||
- `tests/wf_mcp/test_service.py`
|
||||
- `tests/wf_mcp/test_workflow_surface.py`
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Make `CapabilityRef` Serialize Structurally
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_platform/refs.py`
|
||||
- Test: `tests/wf_platform/test_refs.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests**
|
||||
|
||||
Add tests that prove three things:
|
||||
|
||||
```python
|
||||
from pydantic import BaseModel
|
||||
|
||||
from wf_platform import CapabilityRef, SourceRef
|
||||
|
||||
|
||||
class RefHolder(BaseModel):
|
||||
ref: CapabilityRef
|
||||
|
||||
|
||||
def test_capability_ref_accepts_legacy_string_input() -> None:
|
||||
holder = RefHolder.model_validate({"ref": "demo.foo.bar"})
|
||||
|
||||
# Legacy parsing is best-effort only. It stays for old input compatibility.
|
||||
assert str(holder.ref) == "demo.foo.bar"
|
||||
|
||||
|
||||
def test_capability_ref_accepts_structural_input() -> None:
|
||||
holder = RefHolder.model_validate(
|
||||
{"ref": {"source": "demo", "capability_key": "foo.bar"}}
|
||||
)
|
||||
|
||||
assert holder.ref.source == SourceRef.parse("demo")
|
||||
assert holder.ref.name == "foo.bar"
|
||||
|
||||
|
||||
def test_capability_ref_serializes_structurally() -> None:
|
||||
holder = RefHolder(
|
||||
ref=CapabilityRef(source=SourceRef.parse("demo"), name="foo.bar")
|
||||
)
|
||||
|
||||
assert holder.model_dump(mode="json")["ref"] == {
|
||||
"source": "demo",
|
||||
"capability_key": "foo.bar",
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify red**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_platform/test_refs.py -q
|
||||
```
|
||||
|
||||
Expected: structural input or structural serialization fails because `CapabilityRef` currently serializes as a string.
|
||||
|
||||
- [ ] **Step 3: Implement structural validation/serialization**
|
||||
|
||||
Update `CapabilityRef.__get_pydantic_core_schema__` to:
|
||||
|
||||
- accept existing `CapabilityRef`
|
||||
- accept legacy string via `CapabilityRef.parse`
|
||||
- accept dict `{"source": str, "capability_key": str}`
|
||||
- serialize as dict `{"source": str(self.source), "capability_key": self.name}`
|
||||
|
||||
Keep `__str__` unchanged because display strings are still useful.
|
||||
|
||||
- [ ] **Step 4: Run tests to verify green**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_platform/test_refs.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Make `WorkflowCapabilityRef` Structural
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/refs.py`
|
||||
- Test: `tests/wf_artifacts/test_refs.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests**
|
||||
|
||||
```python
|
||||
from pydantic import BaseModel
|
||||
|
||||
from wf_artifacts import WorkflowCapabilityRef
|
||||
|
||||
|
||||
class WorkflowRefHolder(BaseModel):
|
||||
ref: WorkflowCapabilityRef
|
||||
|
||||
|
||||
def test_workflow_capability_ref_accepts_legacy_string_input() -> None:
|
||||
holder = WorkflowRefHolder.model_validate({"ref": "workflow.echo_wrapper.v1"})
|
||||
|
||||
assert holder.ref.artifact_id == "echo_wrapper"
|
||||
assert holder.ref.version == 1
|
||||
|
||||
|
||||
def test_workflow_capability_ref_accepts_structural_input() -> None:
|
||||
holder = WorkflowRefHolder.model_validate(
|
||||
{"ref": {"artifact_id": "echo_wrapper", "version": 1}}
|
||||
)
|
||||
|
||||
assert holder.ref.artifact_id == "echo_wrapper"
|
||||
assert holder.ref.version == 1
|
||||
|
||||
|
||||
def test_workflow_capability_ref_serializes_structurally() -> None:
|
||||
holder = WorkflowRefHolder(ref=WorkflowCapabilityRef("echo_wrapper", 1))
|
||||
|
||||
assert holder.model_dump(mode="json")["ref"] == {
|
||||
"artifact_id": "echo_wrapper",
|
||||
"version": 1,
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify red**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_artifacts/test_refs.py -q
|
||||
```
|
||||
|
||||
Expected: Pydantic validation fails because `WorkflowCapabilityRef` has no schema hook.
|
||||
|
||||
- [ ] **Step 3: Implement Pydantic schema hook**
|
||||
|
||||
Add a Pydantic core schema method to `WorkflowCapabilityRef` that accepts legacy string and structural dict input, then serializes structurally.
|
||||
|
||||
- [ ] **Step 4: Run tests to verify green**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_artifacts/test_refs.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Save Required Capabilities in New Shape
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/models.py`
|
||||
- Modify: `src/wf_artifacts/references.py`
|
||||
- Test: `tests/wf_artifacts/test_models.py`
|
||||
- Test: existing workflow-surface artifact creation tests
|
||||
|
||||
- [ ] **Step 1: Write failing model serialization tests**
|
||||
|
||||
Add tests proving old inputs parse and new outputs dump structurally:
|
||||
|
||||
```python
|
||||
from wf_artifacts import RequiredCapability, WorkflowArtifact
|
||||
|
||||
|
||||
def test_required_capability_accepts_legacy_logical_fields_but_dumps_ref_object() -> None:
|
||||
capability = RequiredCapability.model_validate(
|
||||
{
|
||||
"logical_source": "demo",
|
||||
"capability_name": "foo.bar",
|
||||
"kind": "node_spec",
|
||||
}
|
||||
)
|
||||
|
||||
assert capability.logical_source == "demo"
|
||||
assert capability.capability_name == "foo.bar"
|
||||
assert capability.model_dump(mode="json")["ref"] == {
|
||||
"source": "demo",
|
||||
"capability_key": "foo.bar",
|
||||
}
|
||||
|
||||
|
||||
def test_workflow_artifact_accepts_legacy_required_capability_map_but_dumps_list() -> None:
|
||||
artifact = WorkflowArtifact.model_validate(
|
||||
{
|
||||
"id": "echo",
|
||||
"version": 1,
|
||||
"title": "Echo",
|
||||
"input_schema": {"type": "object", "properties": {}},
|
||||
"output_schema": {"type": "object", "properties": {}},
|
||||
"outcomes": ["completed"],
|
||||
"plan": {"name": "echo", "nodes": [], "edges": []},
|
||||
"required_capabilities": {
|
||||
"demo.foo.bar": {"kind": "node_spec"},
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
dumped = artifact.model_dump(mode="json")
|
||||
assert dumped["required_capabilities"][0]["ref"] == {
|
||||
"source": "demo.foo",
|
||||
"capability_key": "bar",
|
||||
}
|
||||
```
|
||||
|
||||
This test documents legacy string parsing as best-effort. New artifact creation should avoid this path when source bindings are known.
|
||||
|
||||
- [ ] **Step 2: Run tests to verify red**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_artifacts/test_models.py -q
|
||||
```
|
||||
|
||||
Expected: dumps still contain old string refs.
|
||||
|
||||
- [ ] **Step 3: Update model serialization**
|
||||
|
||||
After Task 1, `RequiredCapability.ref` should dump structurally automatically. Ensure `WorkflowArtifact._reject_duplicate_required_capabilities` still works by using `str(capability.capability_ref())` only for internal duplicate checking.
|
||||
|
||||
- [ ] **Step 4: Update reference creation**
|
||||
|
||||
In `src/wf_artifacts/references.py`, replace this conceptual behavior:
|
||||
|
||||
```python
|
||||
logical_ref = "demo.foo.bar"
|
||||
RequiredCapability(ref=CapabilityRef.parse(logical_ref), ...)
|
||||
```
|
||||
|
||||
with structural construction:
|
||||
|
||||
```python
|
||||
capability_ref = CapabilityRef(
|
||||
source=SourceRef.parse(logical_source),
|
||||
name=capability_name,
|
||||
)
|
||||
RequiredCapability(ref=capability_ref, ...)
|
||||
```
|
||||
|
||||
Only derive `str(capability_ref)` for compatibility map keys.
|
||||
|
||||
- [ ] **Step 5: Run tests to verify green**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_artifacts/test_models.py tests/wf_mcp/test_workflow_surface.py -q
|
||||
```
|
||||
|
||||
Expected: artifact creation still works; saved dumps use structural refs.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Make Deployment Bindings Structural on Save
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/models.py`
|
||||
- Test: `tests/wf_artifacts/test_models.py`
|
||||
- Test: `tests/wf_mcp/test_workflow_surface.py`
|
||||
|
||||
- [ ] **Step 1: Write failing binding serialization test**
|
||||
|
||||
```python
|
||||
from wf_artifacts import WorkflowDeployment
|
||||
|
||||
|
||||
def test_workflow_deployment_accepts_legacy_binding_map_but_dumps_structural_list() -> None:
|
||||
deployment = WorkflowDeployment.model_validate(
|
||||
{
|
||||
"id": "echo.personal",
|
||||
"artifact_id": "echo",
|
||||
"artifact_version": 1,
|
||||
"bindings": {"demo": "demo.personal", "wf.std": "wf.std"},
|
||||
}
|
||||
)
|
||||
|
||||
dumped = deployment.model_dump(mode="json")
|
||||
assert dumped["bindings"] == [
|
||||
{"logical_source": "demo", "concrete_source": "demo.personal"},
|
||||
{"logical_source": "wf.std", "concrete_source": "wf.std"},
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify red**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_artifacts/test_models.py -q
|
||||
```
|
||||
|
||||
Expected: output may already be a list, but confirm `SourceRef` serialization remains stable.
|
||||
|
||||
- [ ] **Step 3: Keep current binding shape but document it as structural**
|
||||
|
||||
`SourceBinding` already separates logical and concrete source fields. Add docstrings explaining:
|
||||
|
||||
- `logical_source` is an artifact-local alias.
|
||||
- `concrete_source` is the deployment-selected source id.
|
||||
- neither field is a capability name.
|
||||
|
||||
- [ ] **Step 4: Run tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_artifacts/test_models.py tests/wf_mcp/test_workflow_surface.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Stop Parsing Workflow Capability Strings as Generic Capabilities
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/refs.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Test: `tests/wf_mcp/test_workflow_surface_refs.py`
|
||||
- Test: `tests/wf_mcp/test_workflow_surface.py`
|
||||
|
||||
- [ ] **Step 1: Write tests for separate ref domains**
|
||||
|
||||
```python
|
||||
from wf_artifacts import WorkflowCapabilityRef
|
||||
from wf_mcp.workflow_surface.refs import parse_workflow_surface_capability_id
|
||||
from wf_platform import CapabilityRef
|
||||
|
||||
|
||||
def test_workflow_surface_ref_parser_keeps_workflow_artifacts_separate() -> None:
|
||||
parsed = parse_workflow_surface_capability_id("workflow.echo_wrapper.v1")
|
||||
|
||||
assert isinstance(parsed, WorkflowCapabilityRef)
|
||||
assert parsed.artifact_id == "echo_wrapper"
|
||||
assert parsed.version == 1
|
||||
|
||||
|
||||
def test_workflow_surface_ref_parser_keeps_source_capabilities_structural() -> None:
|
||||
parsed = parse_workflow_surface_capability_id(
|
||||
{"source": "demo", "capability_key": "foo.bar"}
|
||||
)
|
||||
|
||||
assert isinstance(parsed, CapabilityRef)
|
||||
assert str(parsed.source) == "demo"
|
||||
assert parsed.name == "foo.bar"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify red**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface_refs.py -q
|
||||
```
|
||||
|
||||
Expected: dict input fails because parser currently accepts strings only.
|
||||
|
||||
- [ ] **Step 3: Update parser input type**
|
||||
|
||||
Allow parser input as:
|
||||
|
||||
```python
|
||||
str | dict[str, object]
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- if string starts with `workflow.`, try `WorkflowCapabilityRef.parse`
|
||||
- if dict has `artifact_id` and `version`, parse as workflow artifact ref
|
||||
- otherwise parse as `CapabilityRef`
|
||||
|
||||
- [ ] **Step 4: Run workflow surface tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_workflow_surface_refs.py tests/wf_mcp/test_workflow_surface.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Keep Runtime Binding Source-Aware
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/runtime_dependencies.py`
|
||||
- Test: `tests/wf_mcp/test_service.py`
|
||||
|
||||
- [ ] **Step 1: Keep the dotted-local-name regression**
|
||||
|
||||
Ensure this test remains:
|
||||
|
||||
```python
|
||||
def test_service_runs_logical_source_plan_with_dotted_local_name() -> None:
|
||||
...
|
||||
plan = _single_echo_plan("logical_dotted_local_name_plan", "demo.foo.bar")
|
||||
...
|
||||
bindings=[{"logical_source": "demo", "concrete_source": "demo.personal"}]
|
||||
...
|
||||
assert run.output["echoed"] == "hello"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add a unit test for longest binding prefix**
|
||||
|
||||
Add a focused test through public runtime behavior where both `demo` and `demo.pro` are bound, and `demo.pro.foo.bar` resolves through `demo.pro`.
|
||||
|
||||
- [ ] **Step 3: Keep `_bound_node_names` or move it to a shared helper**
|
||||
|
||||
If more than one module needs source-prefix binding, move the helper to a small module such as:
|
||||
|
||||
```text
|
||||
src/wf_artifacts/bindings.py
|
||||
```
|
||||
|
||||
Do not move it preemptively if runtime remains the only caller.
|
||||
|
||||
- [ ] **Step 4: Run tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_mcp/test_service.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Update Docs to State the Rule
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/workflow_capabilities.md`
|
||||
- Create or modify: `docs/structural_refs.md`
|
||||
|
||||
- [ ] **Step 1: Add the core rule**
|
||||
|
||||
Add this rule prominently:
|
||||
|
||||
```text
|
||||
Qualified names are display strings. They are not authoritative identifiers.
|
||||
Saved workflow artifacts and deployments should store structural refs.
|
||||
Old strings are accepted at API boundaries only for compatibility.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add examples**
|
||||
|
||||
Include examples for:
|
||||
|
||||
```json
|
||||
{ "source": "demo", "capability_key": "foo.bar" }
|
||||
```
|
||||
|
||||
```json
|
||||
{ "artifact_id": "echo_wrapper", "version": 1 }
|
||||
```
|
||||
|
||||
```json
|
||||
{ "logical_source": "demo", "concrete_source": "demo.personal" }
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Mention path refs are separate**
|
||||
|
||||
Add:
|
||||
|
||||
```text
|
||||
Capability refs and graph paths are different domains. Path strings such as
|
||||
state.person.name should migrate separately to path models.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/wf_platform tests/wf_artifacts tests/wf_mcp/test_service.py tests/wf_mcp/test_workflow_surface.py -q
|
||||
```
|
||||
|
||||
Expected: all selected tests pass.
|
||||
|
||||
- [ ] **Step 2: Run full tests when workspace is stable**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite passes, except any explicitly user-owned temporary rewrite tests if the user says to ignore them.
|
||||
|
||||
- [ ] **Step 3: Run linters/type checker**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uvx ruff check
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: no new errors.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Notes
|
||||
|
||||
- This plan does not require changing all workflow plan node refs in one pass. Runtime accepts legacy plan strings while artifact metadata becomes structural first.
|
||||
- This plan does not solve graph path ambiguity. Graph paths need their own migration to `GraphPath`/`LocalPath`.
|
||||
- This plan does not force `profile` into every model immediately. It keeps `SourceRef` compatible and documents `profile` as future concrete source structure.
|
||||
- This plan keeps MCP/client compatibility by accepting old string inputs and continuing to display derived names.
|
||||
@@ -0,0 +1,219 @@
|
||||
# Structural Graph Paths Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Save canonical graph paths as structural objects while keeping old dotted strings as parse-only compatibility input.
|
||||
|
||||
**Architecture:** The core already has first-class path types: `GraphSourcePath`, `StatePath`, and `LocalPath`. Update those Pydantic hooks to accept structural dict input and serialize structurally in JSON mode. Keep `str(path)` for display and legacy fields. Do not redesign `NodeUse.input` / `output`; those structs already replaced deprecated `in_map` / `out_map`.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic core schema hooks, `wf_core.paths`, `wf_core.models.steps`, pytest.
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
Canonical node bindings already exist:
|
||||
|
||||
```json
|
||||
{
|
||||
"input": [{ "path": "input.message", "target": "message" }],
|
||||
"output": [{ "source": "echoed", "target": "state.echoed" }]
|
||||
}
|
||||
```
|
||||
|
||||
Internally these parse to:
|
||||
|
||||
- `GraphSourcePath`
|
||||
- `LocalPath`
|
||||
- `StatePath`
|
||||
|
||||
The remaining problem is serialization. These path objects currently dump as strings, so saved JSON still relies on dot-separated path grammar.
|
||||
|
||||
## Canonical Shape
|
||||
|
||||
Graph source paths:
|
||||
|
||||
```json
|
||||
{ "root": "state", "parts": ["person", "name"] }
|
||||
```
|
||||
|
||||
State write paths:
|
||||
|
||||
```json
|
||||
{ "root": "state", "parts": ["person", "name"] }
|
||||
```
|
||||
|
||||
Local node paths:
|
||||
|
||||
```json
|
||||
{ "root": "local", "parts": ["payload", "text"] }
|
||||
```
|
||||
|
||||
Local root remains explicit:
|
||||
|
||||
```json
|
||||
{ "root": "local", "parts": [] }
|
||||
```
|
||||
|
||||
Old strings such as `"state.person.name"` and `"."` remain accepted input.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add Structural Serialization for Path Types
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/paths.py`
|
||||
- Test: `tests/core/test_path_values.py`
|
||||
|
||||
- [ ] **Step 1: Update tests first**
|
||||
|
||||
Change `test_pydantic_accepts_path_strings_and_serializes_strings` into structural JSON expectations:
|
||||
|
||||
```python
|
||||
dumped = payload.model_dump(mode="json")
|
||||
assert dumped["source"] == {"root": "input", "parts": ["user"]}
|
||||
assert dumped["target"] == {"root": "state", "parts": ["person"]}
|
||||
assert dumped["local"] == {"root": "local", "parts": ["user"]}
|
||||
```
|
||||
|
||||
Keep `model_dump()` expectations if useful for Python-mode compatibility only if the implementation intentionally keeps Python mode as strings. Otherwise assert structural dumps in both modes.
|
||||
|
||||
- [ ] **Step 2: Add structural input tests**
|
||||
|
||||
Add a test:
|
||||
|
||||
```python
|
||||
payload = Payload.model_validate({
|
||||
"source": {"root": "input", "parts": ["user.name"]},
|
||||
"target": {"root": "state", "parts": ["person.name"]},
|
||||
"local": {"root": "local", "parts": ["payload.text"]},
|
||||
})
|
||||
|
||||
assert payload.source == GraphSourcePath.input("user.name")
|
||||
assert payload.target == StatePath.of("person.name")
|
||||
assert payload.local == LocalPath.of("payload.text")
|
||||
```
|
||||
|
||||
This documents that structural `parts` are literal field names. Old string inputs still split on dots for compatibility, but structural parts such as `"user.name"` are not split again.
|
||||
|
||||
- [ ] **Step 3: Implement path serializers**
|
||||
|
||||
In `src/wf_core/paths.py`, update each path type:
|
||||
|
||||
- `LocalPath` accepts string, object instance, and dict `{"root": "local", "parts": list[str]}`
|
||||
- `GraphSourcePath` accepts string, object instance, and dict `{"root": "input"|"state"|"context", "parts": list[str]}`
|
||||
- `StatePath` accepts string, object instance, and dict `{"root": "state", "parts": list[str]}`
|
||||
|
||||
Serialize as dicts in JSON mode:
|
||||
|
||||
```python
|
||||
{"root": "local", "parts": list(value.parts)}
|
||||
{"root": value.root, "parts": list(value.parts)}
|
||||
{"root": "state", "parts": list(value.parts)}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused path tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_path_values.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Update Canonical Node Binding Dumps
|
||||
|
||||
**Files:**
|
||||
|
||||
- Test: `tests/core/test_canonical_node_bindings.py`
|
||||
- Test: `tests/authoring/test_builder.py`
|
||||
|
||||
- [ ] **Step 1: Update canonical node dump expectations**
|
||||
|
||||
In `tests/core/test_canonical_node_bindings.py`, update JSON-mode expectations:
|
||||
|
||||
```python
|
||||
assert dumped["input"][1]["path"] == {"root": "input", "parts": ["message"]}
|
||||
assert dumped["input"][1]["target"] == {"root": "local", "parts": ["message"]}
|
||||
assert dumped["output"][0]["source"] == {"root": "local", "parts": ["echoed"]}
|
||||
assert dumped["output"][0]["target"] == {"root": "state", "parts": ["echoed"]}
|
||||
```
|
||||
|
||||
Deprecated `in_map` / `out_map` inputs should continue parsing, but dumps must omit those old fields and emit structural paths.
|
||||
|
||||
- [ ] **Step 2: Update authoring serialization expectations**
|
||||
|
||||
In `tests/authoring/test_builder.py`, update any `model_dump(mode="json")` expectations that currently assert path strings.
|
||||
|
||||
- [ ] **Step 3: Run focused binding/authoring tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_canonical_node_bindings.py tests/authoring/test_builder.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Update Docs
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/structural_refs.md`
|
||||
- Modify: any path/core docs if directly relevant.
|
||||
|
||||
- [ ] **Step 1: Add graph path note**
|
||||
|
||||
Extend the path note in `docs/structural_refs.md`:
|
||||
|
||||
```text
|
||||
New canonical graph path JSON uses root/parts objects. Old strings are accepted
|
||||
at parse boundaries for compatibility.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add examples**
|
||||
|
||||
Include examples:
|
||||
|
||||
```json
|
||||
{"root": "input", "parts": ["message"]}
|
||||
{"root": "state", "parts": ["echoed"]}
|
||||
{"root": "local", "parts": []}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Verification
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest tests/core/test_path_values.py tests/core/test_canonical_node_bindings.py tests/authoring/test_builder.py -q
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run full tests**
|
||||
|
||||
```bash
|
||||
uv run --with pytest pytest -q
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run checks**
|
||||
|
||||
```bash
|
||||
uvx ruff check src/wf_core/paths.py tests/core/test_path_values.py tests/core/test_canonical_node_bindings.py tests/authoring/test_builder.py
|
||||
uv run basedpyright --level error src/wf_core/paths.py tests/core/test_path_values.py tests/core/test_canonical_node_bindings.py tests/authoring/test_builder.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Notes
|
||||
|
||||
- This plan does not revive `in_map` / `out_map`; those remain deprecated parse-only fields.
|
||||
- This plan relaxes path segment validation. Structural `parts` preserve literal field names, including dots and spaces. Old dotted string inputs still split on dots for compatibility.
|
||||
- This plan changes saved JSON shape for canonical path fields, so broad tests are required.
|
||||
+604
@@ -0,0 +1,604 @@
|
||||
# Concurrent Foreach Barrier Write Semantics Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Enforce deterministic and explicit write semantics when concurrent foreach sibling item lineages commit at the barrier.
|
||||
|
||||
**Architecture:** Keep per-node patch building unchanged. Add a barrier-only validation step before replaying item patches: inspect all item patch destination paths, reject ambiguous sibling writes, and allow multi-writer paths only when the exact declared state path has a reducer whose metadata is `mergeable`. Barrier replay still happens in item-index order through existing reducer logic.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, Pydantic v2 models, pytest, `StatePath`, `StateSchema.field_index()`, `StatePatch`, `ForeachBarrierState`, and existing reducer definitions.
|
||||
|
||||
---
|
||||
|
||||
## Semantics
|
||||
|
||||
This slice owns sibling write policy at a foreach barrier.
|
||||
|
||||
Allowed:
|
||||
|
||||
- A destination path written by exactly one item lineage uses normal state rules.
|
||||
- A destination path written by multiple item lineages is allowed only if that exact declared state path has a `mergeable` reducer.
|
||||
- Multi-writer reducer replay is deterministic item-index order.
|
||||
- Multiple nodes inside the same item lineage may write multiple paths; that is item-local overlay behavior from Slice 2.
|
||||
|
||||
Rejected:
|
||||
|
||||
- Multiple sibling item lineages writing the same destination path with missing reducer/default replace.
|
||||
- Multiple sibling item lineages writing the same destination path with an `exclusive` reducer such as `wf.std.replace`.
|
||||
- Sibling item lineages writing ancestor/descendant paths such as `state.person` and `state.person.name`.
|
||||
|
||||
Important distinction:
|
||||
|
||||
- This is **not** normal node output validation. Node-level `build_output_patch(...)` still rejects overlapping output paths inside one node.
|
||||
- This is **not** reducer implementation work. Existing reducers stay pure and domain-agnostic.
|
||||
- This is **not** deep merge policy. Ancestor/descendant sibling writes are rejected for now.
|
||||
|
||||
---
|
||||
|
||||
## Files
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||
- Add barrier write validation helpers.
|
||||
- Call them from `build_barrier_patch(...)` before replay.
|
||||
- Test: `tests/core/test_concurrent_foreach.py`
|
||||
- Add end-to-end concurrent foreach conflict tests.
|
||||
- Test: `tests/core/test_atomic_state_patches.py`
|
||||
- Add focused `build_barrier_patch(...)` unit tests if end-to-end setup becomes too noisy.
|
||||
- Modify: `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`
|
||||
- Mark write semantics as implemented.
|
||||
- Modify: `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`
|
||||
- Link this plan under Slice 3.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Focused Barrier Same-Path Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/core/test_atomic_state_patches.py`
|
||||
|
||||
- [ ] **Step 1: Add imports**
|
||||
|
||||
Ensure `tests/core/test_atomic_state_patches.py` imports:
|
||||
|
||||
```python
|
||||
from wf_core.runtime.ops.state import StatePatch, build_barrier_patch
|
||||
```
|
||||
|
||||
If `StatePatch` is already imported, only add `build_barrier_patch`.
|
||||
|
||||
- [ ] **Step 2: Add test for same-path writes without reducer**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_barrier_rejects_sibling_same_path_writes_without_reducer() -> None:
|
||||
workflow = Workflow(
|
||||
name="barrier_conflict",
|
||||
input_schema=SchemaRef(properties={}),
|
||||
state_schema=StateSchema.from_field_map(
|
||||
{"value": StateField(type="string")}
|
||||
),
|
||||
output_schema=SchemaRef(properties={}),
|
||||
node_defs=[],
|
||||
start="unused",
|
||||
nodes=[],
|
||||
edges=[],
|
||||
)
|
||||
|
||||
with pytest.raises(WorkflowExecutionError, match="multiple sibling writes"):
|
||||
build_barrier_patch(
|
||||
workflow,
|
||||
[
|
||||
StatePatch(changes={"state.value": "a"}),
|
||||
StatePatch(changes={"state.value": "b"}),
|
||||
],
|
||||
{},
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add test for explicit replace still rejected**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_barrier_rejects_sibling_same_path_writes_with_explicit_replace() -> None:
|
||||
workflow = Workflow(
|
||||
name="barrier_replace_conflict",
|
||||
input_schema=SchemaRef(properties={}),
|
||||
state_schema=StateSchema.from_field_map(
|
||||
{
|
||||
"value": StateField(
|
||||
type="string",
|
||||
reducer=ReducerRef(name="wf.std.replace"),
|
||||
)
|
||||
}
|
||||
),
|
||||
output_schema=SchemaRef(properties={}),
|
||||
node_defs=[],
|
||||
start="unused",
|
||||
nodes=[],
|
||||
edges=[],
|
||||
)
|
||||
|
||||
with pytest.raises(WorkflowExecutionError, match="mergeable reducer"):
|
||||
build_barrier_patch(
|
||||
workflow,
|
||||
[
|
||||
StatePatch(changes={"state.value": "a"}),
|
||||
StatePatch(changes={"state.value": "b"}),
|
||||
],
|
||||
{},
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add test for mergeable reducer allowing same-path writes**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_barrier_allows_sibling_same_path_writes_with_non_replace_reducer() -> None:
|
||||
workflow = Workflow(
|
||||
name="barrier_reducer",
|
||||
input_schema=SchemaRef(properties={}),
|
||||
state_schema=StateSchema.from_field_map(
|
||||
{
|
||||
"seen": StateField(
|
||||
type="array",
|
||||
reducer=ReducerRef(name="wf.std.append"),
|
||||
)
|
||||
}
|
||||
),
|
||||
output_schema=SchemaRef(properties={}),
|
||||
node_defs=[],
|
||||
start="unused",
|
||||
nodes=[],
|
||||
edges=[],
|
||||
)
|
||||
|
||||
patch = build_barrier_patch(
|
||||
workflow,
|
||||
[
|
||||
StatePatch(changes={"state.seen": "a"}),
|
||||
StatePatch(changes={"state.seen": "b"}),
|
||||
],
|
||||
{},
|
||||
)
|
||||
|
||||
assert patch.changes["state.seen"] == ["a", "b"]
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run tests and verify failures**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_same_path_writes_without_reducer tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_same_path_writes_with_explicit_replace tests/core/test_atomic_state_patches.py::test_barrier_allows_sibling_same_path_writes_with_non_replace_reducer -q
|
||||
```
|
||||
|
||||
Expected before implementation:
|
||||
|
||||
```text
|
||||
two reject tests fail because current barrier accepts replace semantics
|
||||
```
|
||||
|
||||
The reducer test may already pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Add Ancestor/Descendant Conflict Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/core/test_atomic_state_patches.py`
|
||||
|
||||
- [ ] **Step 1: Add ancestor/descendant conflict test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_barrier_rejects_sibling_ancestor_descendant_writes() -> None:
|
||||
workflow = Workflow(
|
||||
name="barrier_ancestor_conflict",
|
||||
input_schema=SchemaRef(properties={}),
|
||||
state_schema=StateSchema.from_field_map(
|
||||
{
|
||||
"person": StateField(
|
||||
type="object",
|
||||
properties={"name": {"type": "string"}},
|
||||
),
|
||||
"person.name": StateField(type="string"),
|
||||
}
|
||||
),
|
||||
output_schema=SchemaRef(properties={}),
|
||||
node_defs=[],
|
||||
start="unused",
|
||||
nodes=[],
|
||||
edges=[],
|
||||
)
|
||||
|
||||
with pytest.raises(WorkflowExecutionError, match="overlapping sibling writes"):
|
||||
build_barrier_patch(
|
||||
workflow,
|
||||
[
|
||||
StatePatch(changes={"state.person": {"name": "Ada"}}),
|
||||
StatePatch(changes={"state.person.name": "Grace"}),
|
||||
],
|
||||
{},
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add same-item ancestor/descendant note test only if needed**
|
||||
|
||||
Do **not** add a same-item ancestor/descendant test unless current behavior changes unexpectedly. Same-node output overlap is already rejected by `build_output_patch(...)`, and multi-node same-item writes are item-local overlay behavior. This slice only owns sibling conflicts.
|
||||
|
||||
- [ ] **Step 3: Run focused test and verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_ancestor_descendant_writes -q
|
||||
```
|
||||
|
||||
Expected before implementation:
|
||||
|
||||
```text
|
||||
FAILED because current barrier replays both writes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Implement Barrier Write Analysis
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||
|
||||
- [ ] **Step 1: Add helper dataclass**
|
||||
|
||||
Near `StatePatch`, add:
|
||||
|
||||
```python
|
||||
@dataclass(slots=True, frozen=True)
|
||||
class _BarrierWrite:
|
||||
"""One item-lineage write observed before a barrier commit."""
|
||||
|
||||
item_index: int
|
||||
path: StatePath
|
||||
source_key: str
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add reducer policy predicate**
|
||||
|
||||
Add below `build_barrier_patch(...)` or near private helpers:
|
||||
|
||||
```python
|
||||
def _allows_sibling_writes(
|
||||
path: StatePath,
|
||||
state_fields: Mapping[StatePath, StateFieldDecl],
|
||||
reducers: Mapping[str, ReducerDefinition] | None,
|
||||
) -> bool:
|
||||
field = state_fields.get(path)
|
||||
if field is None or field.reducer is None:
|
||||
return False
|
||||
return reducer_allows_sibling_writes(field.reducer, reducers)
|
||||
```
|
||||
|
||||
If `StateFieldDecl.reducer` is never `None` for undeclared/default fields, inspect the actual model and adjust:
|
||||
|
||||
```python
|
||||
return field.reducer.name != "wf.std.replace"
|
||||
```
|
||||
|
||||
but preserve the rule: only a mergeable reducer allows sibling same-path writes.
|
||||
|
||||
- [ ] **Step 3: Add overlap predicate for barrier paths**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def _state_paths_overlap(left: StatePath, right: StatePath) -> bool:
|
||||
left_parts = left.parts
|
||||
right_parts = right.parts
|
||||
return left_parts == right_parts or _is_prefix(left_parts, right_parts) or _is_prefix(
|
||||
right_parts,
|
||||
left_parts,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add write collection helper**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def _barrier_writes(item_patches: Sequence[StatePatch]) -> list[_BarrierWrite]:
|
||||
writes: list[_BarrierWrite] = []
|
||||
for item_index, item_patch in enumerate(item_patches):
|
||||
for destination in item_patch.changes:
|
||||
path = StatePath.parse(destination)
|
||||
writes.append(
|
||||
_BarrierWrite(
|
||||
item_index=item_index,
|
||||
path=path,
|
||||
source_key=destination,
|
||||
)
|
||||
)
|
||||
return writes
|
||||
```
|
||||
|
||||
Important: `item_index` here is the order in `item_patches`, which `foreach.py` already passes sorted by real item index. Do not infer item ids from path strings.
|
||||
|
||||
- [ ] **Step 5: Add validation helper**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def validate_barrier_writes(
|
||||
item_patches: Sequence[StatePatch],
|
||||
state_fields: Mapping[StatePath, StateFieldDecl],
|
||||
*,
|
||||
reducers: Mapping[str, ReducerDefinition] | None = None,
|
||||
) -> None:
|
||||
"""Reject ambiguous sibling writes before replaying a foreach barrier.
|
||||
|
||||
Normal node patch validation handles one node output. This helper handles
|
||||
writes from different foreach item lineages that will commit together.
|
||||
"""
|
||||
writes = _barrier_writes(item_patches)
|
||||
for index, left in enumerate(writes):
|
||||
for right in writes[index + 1 :]:
|
||||
if left.item_index == right.item_index:
|
||||
continue
|
||||
if left.path == right.path:
|
||||
if _allows_sibling_writes(left.path, state_fields, reducers):
|
||||
continue
|
||||
raise WorkflowExecutionError(
|
||||
"multiple sibling writes to "
|
||||
f"{left.source_key!r} require a mergeable reducer"
|
||||
)
|
||||
if _state_paths_overlap(left.path, right.path):
|
||||
raise WorkflowExecutionError(
|
||||
"overlapping sibling writes are not supported at a foreach "
|
||||
f"barrier: {left.source_key!r} and {right.source_key!r}"
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Call validation from `build_barrier_patch(...)`**
|
||||
|
||||
In `build_barrier_patch(...)`, after:
|
||||
|
||||
```python
|
||||
state_fields = workflow.state_schema.field_index()
|
||||
```
|
||||
|
||||
add:
|
||||
|
||||
```python
|
||||
validate_barrier_writes(item_patches, state_fields, reducers=reducers)
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Verify focused unit tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_same_path_writes_without_reducer tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_same_path_writes_with_explicit_replace tests/core/test_atomic_state_patches.py::test_barrier_allows_sibling_same_path_writes_with_non_replace_reducer tests/core/test_atomic_state_patches.py::test_barrier_rejects_sibling_ancestor_descendant_writes -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add End-To-End Concurrent Foreach Coverage
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/core/test_concurrent_foreach.py`
|
||||
|
||||
- [ ] **Step 1: Add same-path no-reducer workflow helper**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def _same_path_replace_workflow() -> Workflow:
|
||||
foreach = ForeachNode.model_validate(
|
||||
{
|
||||
"id": "each",
|
||||
"type": "foreach",
|
||||
"over": "state.items",
|
||||
"as": "item",
|
||||
"mode": "concurrent",
|
||||
"concurrent": {"max_active": 2, "max_outstanding": 2},
|
||||
}
|
||||
)
|
||||
return Workflow(
|
||||
name="concurrent_foreach_replace_conflict",
|
||||
input_schema=SchemaRef(
|
||||
type="object",
|
||||
properties={"items": {"type": "array"}},
|
||||
),
|
||||
state_schema=StateSchema.from_field_map(
|
||||
{
|
||||
"items": StateField(type="array"),
|
||||
"winner": StateField(type="string"),
|
||||
}
|
||||
),
|
||||
output_schema=SchemaRef(type="object", properties={}),
|
||||
node_defs=[
|
||||
NodeDef(
|
||||
name="write_winner",
|
||||
input_schema=SchemaRef(
|
||||
type="object",
|
||||
properties={"value": {}},
|
||||
required=["value"],
|
||||
),
|
||||
output_schema=SchemaRef(
|
||||
type="object",
|
||||
properties={"winner": {}},
|
||||
required=["winner"],
|
||||
),
|
||||
outcomes=["ok"],
|
||||
)
|
||||
],
|
||||
start="each",
|
||||
nodes=[
|
||||
foreach,
|
||||
NodeUse.model_validate(
|
||||
{
|
||||
"id": "write_winner",
|
||||
"type": "node",
|
||||
"node": "write_winner",
|
||||
"input": [{"target": "value", "path": "context.item"}],
|
||||
"output": [{"source": "winner", "target": "state.winner"}],
|
||||
}
|
||||
),
|
||||
],
|
||||
edges=[
|
||||
Edge.model_validate(
|
||||
{"from": "each", "outcome": "loop", "to": "write_winner"}
|
||||
),
|
||||
Edge.model_validate({"from": "write_winner", "outcome": "ok", "to": END}),
|
||||
Edge.model_validate({"from": "each", "outcome": "done", "to": END}),
|
||||
],
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add end-to-end rejection test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_sync_concurrent_foreach_rejects_sibling_replace_writes() -> None:
|
||||
workflow = _same_path_replace_workflow()
|
||||
|
||||
with pytest.raises(WorkflowExecutionError, match="mergeable reducer"):
|
||||
execute_workflow(
|
||||
workflow,
|
||||
{"items": ["a", "b"]},
|
||||
{
|
||||
"write_winner": lambda payload, _ctx: {
|
||||
"outcome": "ok",
|
||||
"output": {"winner": payload["value"]},
|
||||
}
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Strengthen existing happy path**
|
||||
|
||||
The existing `test_sync_concurrent_foreach_interleaves_items_and_commits_at_barrier`
|
||||
already proves explicit `wf.std.append` allows sibling writes to `state.seen`.
|
||||
Do not duplicate it.
|
||||
|
||||
- [ ] **Step 4: Run focused end-to-end tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_rejects_sibling_replace_writes tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_interleaves_items_and_commits_at_barrier -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Update Docs
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`
|
||||
- Modify: `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Verify ADR merge rules current state**
|
||||
|
||||
In `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`, under
|
||||
`## Merge and Reducer Rules`, verify that the implementation status is
|
||||
documented:
|
||||
|
||||
```markdown
|
||||
Current barrier validation enforces this policy for sibling foreach item
|
||||
lineages. Same-path sibling writes require a `mergeable` reducer on the exact
|
||||
destination state path. Ancestor/descendant sibling writes are rejected until a
|
||||
future explicit deep merge policy exists.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify roadmap Slice 3**
|
||||
|
||||
In `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`,
|
||||
under Slice 3, verify the plan points to this slice:
|
||||
|
||||
```markdown
|
||||
Plan:
|
||||
|
||||
- See [`2026-05-22-concurrent-foreach-barrier-write-semantics.md`](2026-05-22-concurrent-foreach-barrier-write-semantics.md).
|
||||
```
|
||||
|
||||
If implementing immediately, also mark it as implemented in Current State after tests pass.
|
||||
|
||||
- [ ] **Step 3: Verify docs mentions**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "sibling writes|ancestor/descendant|mergeable reducer|barrier write" docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md
|
||||
```
|
||||
|
||||
Expected: the ADR and roadmap both mention the semantics.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- No new source files.
|
||||
|
||||
- [ ] **Step 1: Run focused core tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_atomic_state_patches.py tests/core/test_concurrent_foreach.py tests/core/test_foreach_barrier_state.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run authoring smoke tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/authoring/test_demo_workflow.py tests/authoring/test_builder.py tests/authoring/test_ops.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run full suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: pass, allowing known intentional environment-only skips.
|
||||
|
||||
- [ ] **Step 4: Run lint/type/format checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uvx ruff check src tests
|
||||
uvx ruff format --check src tests docs
|
||||
uv run basedpyright --level error src tests
|
||||
```
|
||||
|
||||
Expected: all pass with 0 type errors.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: the plan covers same-path sibling writes, mergeable reducer requirements, replace rejection, ancestor/descendant rejection, deterministic reducer order, end-to-end foreach behavior, and docs.
|
||||
- Placeholder scan: all tasks include concrete code or exact commands; no TBD placeholders.
|
||||
- Type consistency: the plan uses existing `StatePatch`, `StatePath`, `StateFieldDecl`, `ReducerRef`, `StateSchema.from_field_map`, and `WorkflowExecutionError`.
|
||||
@@ -0,0 +1,648 @@
|
||||
# Concurrent Foreach Item Overlays Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make concurrent foreach item frames read their own buffered state writes so multi-step item bodies are safe.
|
||||
|
||||
**Architecture:** Keep `RunState.state` as committed parent state. Store item-local overlay patches in the foreach parent barrier, keyed by item index/frame id. `state_view_for_frame(...)` should return parent state plus that item's staged overlay; sibling overlays remain invisible. This plan does not implement sibling write conflict policy; Slice 3 owns write semantics across different item lineages.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, pytest, existing `StatePatch`, `ForeachBarrierState`, `safe_resolve_path`, and runtime scheduler modules.
|
||||
|
||||
---
|
||||
|
||||
## Boundary With Slice 3
|
||||
|
||||
This slice answers:
|
||||
|
||||
- Can node B in the same concurrent item read node A's buffered state write?
|
||||
- Can multi-step item bodies run without reading stale parent state?
|
||||
- Are sibling item overlays isolated from each other?
|
||||
|
||||
This slice does **not** answer:
|
||||
|
||||
- Should two sibling items be allowed to write the same state path without a reducer?
|
||||
- Should ancestor/descendant sibling writes conflict?
|
||||
- Should barrier trace `state_changes` show raw per-item inputs or final aggregate values?
|
||||
|
||||
Those are Slice 3 write semantics. Do not add broad write-conflict policy here except what is already enforced by `build_output_patch(...)` for a single node output.
|
||||
|
||||
---
|
||||
|
||||
## Files
|
||||
|
||||
- Modify: `src/wf_core/runtime/foreach_state.py`
|
||||
- Accumulate successful item patches per item instead of storing only one patch.
|
||||
- Expose helpers to get item overlay patches by frame/index.
|
||||
- Modify: `src/wf_core/runtime/ops/overlays.py`
|
||||
- Replace the current no-op seam with parent-state plus item-local staged writes.
|
||||
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||
- Build output patches against the frame-visible state view, not always `run.state`.
|
||||
- Append item-local patches for concurrent item frames.
|
||||
- Modify: `src/wf_core/runtime/ops/foreach.py`
|
||||
- Remove the single-node item-body guard.
|
||||
- Keep `item_error.action != "fail"` unsupported.
|
||||
- Test: `tests/core/test_concurrent_foreach.py`
|
||||
- Add multi-step item-body tests.
|
||||
- Test: `tests/core/test_foreach_barrier_state.py`
|
||||
- Add item patch accumulation tests.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Failing Multi-Step Overlay Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/core/test_concurrent_foreach.py`
|
||||
|
||||
- [ ] **Step 1: Add a two-node item body test**
|
||||
|
||||
Append this test:
|
||||
|
||||
```python
|
||||
def test_sync_concurrent_foreach_item_reads_own_buffered_write() -> None:
|
||||
workflow = _workflow(
|
||||
state_schema=StateSchema.from_field_map(
|
||||
{
|
||||
"items": StateField(type="array"),
|
||||
"scratch": StateField(type="string"),
|
||||
"seen": StateField(
|
||||
type="array",
|
||||
reducer=ReducerRef(name="wf.std.append"),
|
||||
),
|
||||
}
|
||||
),
|
||||
foreach=ForeachNode.model_validate(
|
||||
{
|
||||
"id": "each",
|
||||
"type": "foreach",
|
||||
"over": "state.items",
|
||||
"as": "item",
|
||||
"mode": "concurrent",
|
||||
"concurrent": {"max_active": 2, "max_outstanding": 2},
|
||||
}
|
||||
),
|
||||
)
|
||||
workflow.node_defs.extend(
|
||||
[
|
||||
NodeDef(
|
||||
name="read_scratch",
|
||||
input_schema=SchemaRef(
|
||||
type="object",
|
||||
properties={"scratch": {}},
|
||||
required=["scratch"],
|
||||
),
|
||||
output_schema=SchemaRef(
|
||||
type="object",
|
||||
properties={"seen": {}},
|
||||
required=["seen"],
|
||||
),
|
||||
outcomes=["ok"],
|
||||
)
|
||||
]
|
||||
)
|
||||
workflow.nodes.append(
|
||||
NodeUse.model_validate(
|
||||
{
|
||||
"id": "read_scratch",
|
||||
"type": "node",
|
||||
"node": "read_scratch",
|
||||
"input": [{"target": "scratch", "path": "state.scratch"}],
|
||||
"output": [{"source": "seen", "target": "state.seen"}],
|
||||
}
|
||||
)
|
||||
)
|
||||
workflow.edges = [
|
||||
Edge.model_validate({"from": "each", "outcome": "loop", "to": "record"}),
|
||||
Edge.model_validate({"from": "record", "outcome": "ok", "to": "read_scratch"}),
|
||||
Edge.model_validate({"from": "read_scratch", "outcome": "ok", "to": END}),
|
||||
Edge.model_validate({"from": "each", "outcome": "done", "to": END}),
|
||||
]
|
||||
|
||||
run = execute_workflow(
|
||||
workflow,
|
||||
{"items": ["a", "b", "c"]},
|
||||
{
|
||||
"record": lambda payload, _ctx: {
|
||||
"outcome": "ok",
|
||||
"output": {"scratch": f"scratch:{payload['value']}"},
|
||||
},
|
||||
"read_scratch": lambda payload, _ctx: {
|
||||
"outcome": "ok",
|
||||
"output": {"seen": payload["scratch"]},
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
assert run.state["seen"] == ["scratch:a", "scratch:b", "scratch:c"]
|
||||
```
|
||||
|
||||
Important detail: do not mutate the generic `_workflow(...)` helper for this test.
|
||||
Create a dedicated helper such as `_multi_step_overlay_workflow()` whose `record`
|
||||
`NodeUse` writes `scratch` to `state.scratch`. The test is specifically about an
|
||||
item-local write followed by an item-local read, so the workflow shape should be
|
||||
self-contained and obvious.
|
||||
|
||||
- [ ] **Step 2: Add a sibling isolation test**
|
||||
|
||||
Append this test:
|
||||
|
||||
```python
|
||||
def test_sync_concurrent_foreach_sibling_overlays_do_not_leak() -> None:
|
||||
workflow = _workflow(
|
||||
state_schema=StateSchema.from_field_map(
|
||||
{
|
||||
"items": StateField(type="array"),
|
||||
"scratch": StateField(type="string"),
|
||||
"seen": StateField(
|
||||
type="array",
|
||||
reducer=ReducerRef(name="wf.std.append"),
|
||||
),
|
||||
}
|
||||
),
|
||||
foreach=ForeachNode.model_validate(
|
||||
{
|
||||
"id": "each",
|
||||
"type": "foreach",
|
||||
"over": "state.items",
|
||||
"as": "item",
|
||||
"mode": "concurrent",
|
||||
"concurrent": {"max_active": 2, "max_outstanding": 2},
|
||||
}
|
||||
),
|
||||
)
|
||||
workflow.node_defs.extend(
|
||||
[
|
||||
NodeDef(
|
||||
name="read_scratch",
|
||||
input_schema=SchemaRef(
|
||||
type="object",
|
||||
properties={"scratch": {}},
|
||||
required=["scratch"],
|
||||
),
|
||||
output_schema=SchemaRef(
|
||||
type="object",
|
||||
properties={"seen": {}},
|
||||
required=["seen"],
|
||||
),
|
||||
outcomes=["ok"],
|
||||
)
|
||||
]
|
||||
)
|
||||
workflow.nodes.append(
|
||||
NodeUse.model_validate(
|
||||
{
|
||||
"id": "read_scratch",
|
||||
"type": "node",
|
||||
"node": "read_scratch",
|
||||
"input": [{"target": "scratch", "path": "state.scratch"}],
|
||||
"output": [{"source": "seen", "target": "state.seen"}],
|
||||
}
|
||||
)
|
||||
)
|
||||
workflow.edges = [
|
||||
Edge.model_validate({"from": "each", "outcome": "loop", "to": "record"}),
|
||||
Edge.model_validate({"from": "record", "outcome": "ok", "to": "read_scratch"}),
|
||||
Edge.model_validate({"from": "read_scratch", "outcome": "ok", "to": END}),
|
||||
Edge.model_validate({"from": "each", "outcome": "done", "to": END}),
|
||||
]
|
||||
|
||||
run = execute_workflow(
|
||||
workflow,
|
||||
{"items": ["a", "b"]},
|
||||
{
|
||||
"record": lambda payload, _ctx: {
|
||||
"outcome": "ok",
|
||||
"output": {"scratch": payload["value"]},
|
||||
},
|
||||
"read_scratch": lambda payload, _ctx: {
|
||||
"outcome": "ok",
|
||||
"output": {"seen": payload["scratch"]},
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
assert run.state["seen"] == ["a", "b"]
|
||||
```
|
||||
|
||||
This catches the bad implementation where item `b` sees item `a`'s staged write or vice versa.
|
||||
|
||||
- [ ] **Step 3: Run tests and verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_item_reads_own_buffered_write tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_sibling_overlays_do_not_leak -q
|
||||
```
|
||||
|
||||
Expected before implementation:
|
||||
|
||||
```text
|
||||
FAILED with "concurrent foreach v1 only supports loop bodies with one node"
|
||||
```
|
||||
|
||||
or, if the guard was already removed by another worker:
|
||||
|
||||
```text
|
||||
FAILED because state.scratch is missing/stale
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Accumulate Per-Item Patches
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/foreach_state.py`
|
||||
- Modify: `tests/core/test_foreach_barrier_state.py`
|
||||
|
||||
- [ ] **Step 1: Add patch accumulation tests**
|
||||
|
||||
Append to `tests/core/test_foreach_barrier_state.py`:
|
||||
|
||||
```python
|
||||
def test_foreach_barrier_accumulates_multiple_patches_for_one_item() -> None:
|
||||
barrier = ForeachBarrierState(mode="concurrent")
|
||||
|
||||
barrier.add_success_patch(
|
||||
index=0,
|
||||
frame_id="child-0",
|
||||
patch=StatePatch(changes={"state.scratch": "a"}),
|
||||
)
|
||||
barrier.add_success_patch(
|
||||
index=0,
|
||||
frame_id="child-0",
|
||||
patch=StatePatch(changes={"state.seen": "a"}),
|
||||
)
|
||||
|
||||
result = barrier.pending_results[0]
|
||||
assert result.patch.changes["state.scratch"] == "a"
|
||||
assert result.patch.changes["state.seen"] == "a"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test and verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_foreach_barrier_state.py::test_foreach_barrier_accumulates_multiple_patches_for_one_item -q
|
||||
```
|
||||
|
||||
Expected before implementation:
|
||||
|
||||
```text
|
||||
FAILED with "already recorded"
|
||||
```
|
||||
|
||||
After this plan is implemented, both tests should pass.
|
||||
|
||||
- [ ] **Step 3: Replace duplicate rejection with patch merge**
|
||||
|
||||
In `src/wf_core/runtime/foreach_state.py`, change `add_success_patch(...)` to merge changes for the same item:
|
||||
|
||||
```python
|
||||
def add_success_patch(
|
||||
self, *, index: int, frame_id: str, patch: StatePatch
|
||||
) -> None:
|
||||
"""Buffer or extend successful item patches by item index.
|
||||
|
||||
A multi-step item body may produce multiple node patches. They are
|
||||
accumulated for the same item lineage and replayed by the barrier in
|
||||
item index order. Overlap inside one item remains governed by the normal
|
||||
node output patch rules for each node; Slice 3 owns sibling conflict
|
||||
policy at the barrier.
|
||||
"""
|
||||
existing = self.pending_results.get(index)
|
||||
if existing is None:
|
||||
self.pending_results[index] = PendingItemResult(
|
||||
index=index,
|
||||
frame_id=frame_id,
|
||||
status="succeeded",
|
||||
patch=patch,
|
||||
)
|
||||
return
|
||||
if existing.frame_id != frame_id:
|
||||
raise WorkflowExecutionError(
|
||||
f"foreach item result for index {index!r} belongs to frame "
|
||||
f"{existing.frame_id!r}, got {frame_id!r}"
|
||||
)
|
||||
existing.patch.changes.update(patch.changes)
|
||||
```
|
||||
|
||||
This intentionally updates only `changes`; the barrier replays changes into a fresh staged state later. Do not try to merge `_prepared_writes` here.
|
||||
|
||||
- [ ] **Step 4: Update duplicate test**
|
||||
|
||||
Replace the prior duplicate-item-result test with a frame-mismatch test:
|
||||
|
||||
```python
|
||||
def test_foreach_barrier_rejects_item_result_frame_mismatch() -> None:
|
||||
barrier = ForeachBarrierState(mode="concurrent")
|
||||
patch = StatePatch(changes={"state.count": 1})
|
||||
|
||||
barrier.add_success_patch(index=0, frame_id="child-0", patch=patch)
|
||||
|
||||
with pytest.raises(WorkflowExecutionError, match="belongs to frame"):
|
||||
barrier.add_success_patch(index=0, frame_id="child-1", patch=patch)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Verify barrier tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_foreach_barrier_state.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Build Item-Local State Views
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/overlays.py`
|
||||
- Test: `tests/core/test_concurrent_foreach.py`
|
||||
|
||||
- [ ] **Step 1: Implement overlay state view**
|
||||
|
||||
Replace `state_view_for_frame(...)` in `src/wf_core/runtime/ops/overlays.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from copy import deepcopy
|
||||
from typing import Any
|
||||
|
||||
from wf_core.run_state import ExecutionFrame, RunState
|
||||
from wf_core.runtime.foreach_state import ForeachBarrierState, item_frame_owner
|
||||
from wf_core.runtime.ops.state import safe_set_nested_value
|
||||
from wf_core.paths import StatePath
|
||||
|
||||
|
||||
def state_view_for_frame(run: RunState, frame: ExecutionFrame) -> dict[str, Any]:
|
||||
"""Return committed parent state plus this frame's item-local overlay.
|
||||
|
||||
Concurrent foreach item frames buffer writes in the parent barrier until the
|
||||
foreach barrier commits. Later nodes in the same item must still read those
|
||||
earlier writes, while sibling item frames must not see them.
|
||||
"""
|
||||
owner = item_frame_owner(frame)
|
||||
if owner is None:
|
||||
return run.state
|
||||
|
||||
parent_frame_id, foreach_node_id, item_index = owner
|
||||
parent_frame = run.frames[parent_frame_id]
|
||||
barrier = ForeachBarrierState.from_frame(parent_frame, foreach_node_id)
|
||||
if barrier is None or barrier.mode != "concurrent":
|
||||
return run.state
|
||||
|
||||
pending = barrier.pending_results.get(item_index)
|
||||
if pending is None:
|
||||
return run.state
|
||||
|
||||
state_view = deepcopy(run.state)
|
||||
for destination, value in pending.patch.changes.items():
|
||||
path = StatePath.parse(destination)
|
||||
safe_set_nested_value(state_view, list(path.parts), value)
|
||||
return state_view
|
||||
```
|
||||
|
||||
Do not apply reducers here. The overlay view is an item-local read model, not the final parent commit. Reducers are applied at patch build time for each node and again at the barrier for aggregate commit.
|
||||
|
||||
- [ ] **Step 2: Verify overlay tests still fail on guard**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_item_reads_own_buffered_write -q
|
||||
```
|
||||
|
||||
Expected: if the single-node guard is still present, failure remains the guard. If guard was removed by another worker, this may already pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Build Output Patches Against Frame State View
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||
- Test: `tests/core/test_concurrent_foreach.py`
|
||||
|
||||
- [ ] **Step 1: Reuse the resolved state view for output patching**
|
||||
|
||||
Currently `_resolve_node_execution(...)` computes `state_view` but returns only input/context. Change it to return the state view too:
|
||||
|
||||
```python
|
||||
) -> tuple[dict[str, Any], RuntimeContext, dict[str, Any]]:
|
||||
```
|
||||
|
||||
Return:
|
||||
|
||||
```python
|
||||
return resolved_input, context, state_view
|
||||
```
|
||||
|
||||
Update both callers:
|
||||
|
||||
```python
|
||||
resolved_input, context, state_view = _resolve_node_execution(...)
|
||||
```
|
||||
|
||||
Then pass `state_view` into `_finalize_node_execution(...)`:
|
||||
|
||||
```python
|
||||
state=state_view,
|
||||
```
|
||||
|
||||
Add a parameter to `_finalize_node_execution(...)`:
|
||||
|
||||
```python
|
||||
state_view: dict[str, Any],
|
||||
```
|
||||
|
||||
And change `build_output_patch(...)` call from:
|
||||
|
||||
```python
|
||||
run.state,
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
state_view,
|
||||
```
|
||||
|
||||
This is required for node B in one item to build a patch using node A's staged value.
|
||||
|
||||
- [ ] **Step 2: Run focused overlay test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_item_reads_own_buffered_write -q
|
||||
```
|
||||
|
||||
Expected: still fails until the single-node guard is removed.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Lift The Single-Node Concurrent Body Restriction
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/foreach.py`
|
||||
- Modify: `tests/core/test_concurrent_foreach.py`
|
||||
|
||||
- [ ] **Step 1: Remove the old rejection test**
|
||||
|
||||
Delete or rewrite `test_sync_concurrent_foreach_rejects_multi_step_item_body_for_now`.
|
||||
|
||||
If preserving regression coverage is preferred, replace it with:
|
||||
|
||||
```python
|
||||
def test_sync_concurrent_foreach_allows_multi_step_item_body_with_overlay() -> None:
|
||||
# Use the same workflow shape as
|
||||
# test_sync_concurrent_foreach_item_reads_own_buffered_write.
|
||||
# Assert the workflow completes and output contains all expected values.
|
||||
```
|
||||
|
||||
Prefer not duplicating the full workflow; reuse the dedicated helper introduced
|
||||
for the overlay read/write tests:
|
||||
|
||||
```python
|
||||
def _multi_step_overlay_workflow() -> Workflow:
|
||||
...
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Remove validation call and helper**
|
||||
|
||||
In `src/wf_core/runtime/ops/foreach.py`, remove:
|
||||
|
||||
```python
|
||||
_validate_single_node_loop_body(index, step)
|
||||
```
|
||||
|
||||
Delete `_validate_single_node_loop_body(...)`.
|
||||
|
||||
Remove unused imports:
|
||||
|
||||
```python
|
||||
from wf_core.models.steps import NodeUse
|
||||
from wf_core.tokens import END
|
||||
```
|
||||
|
||||
Keep graph traversal validation out of this slice. Normal workflow validation and runtime edge lookup still define whether graph topology is routable.
|
||||
|
||||
- [ ] **Step 3: Verify multi-step overlay tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_item_reads_own_buffered_write tests/core/test_concurrent_foreach.py::test_sync_concurrent_foreach_sibling_overlays_do_not_leak -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Document Overlay Semantics
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`
|
||||
- Modify: `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Update ADR current-state note**
|
||||
|
||||
In `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`, verify the
|
||||
barrier commit section contains:
|
||||
|
||||
```markdown
|
||||
Current sync execution supports item-local read overlays for concurrent foreach
|
||||
item frames. `RunState.state` remains committed parent state, while
|
||||
`state_view_for_frame` overlays the current item's buffered writes for reads by
|
||||
later nodes in the same item lineage. Sibling overlays remain invisible until
|
||||
the foreach barrier commits.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify roadmap slice statuses**
|
||||
|
||||
In `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`,
|
||||
verify Slice 1 is marked implemented and this plan is linked under Slice 2.
|
||||
|
||||
Use:
|
||||
|
||||
```markdown
|
||||
Plan:
|
||||
|
||||
- See [`2026-05-22-concurrent-foreach-item-overlays.md`](2026-05-22-concurrent-foreach-item-overlays.md).
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Verify docs reference no stale limitation**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "one node|no-op overlay|multi-step concurrent item bodies are rejected" docs src tests
|
||||
```
|
||||
|
||||
Expected: no stale claims except historical plan text in the already-completed V1 plan.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- No new files unless tests require helper extraction.
|
||||
|
||||
- [ ] **Step 1: Run focused core tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_concurrent_foreach.py tests/core/test_foreach_barrier_state.py tests/core/test_scheduler.py tests/core/test_atomic_state_patches.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run authoring smoke tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/authoring/test_demo_workflow.py tests/authoring/test_builder.py tests/authoring/test_ops.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run full suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: pass, allowing known intentional environment-only skips.
|
||||
|
||||
- [ ] **Step 4: Run lint/type/format checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uvx ruff check src tests
|
||||
uvx ruff format --check src tests docs
|
||||
uv run basedpyright --level error src tests
|
||||
```
|
||||
|
||||
Expected: all pass with 0 type errors.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: the plan makes item-local overlays real, supports multi-step concurrent item bodies, keeps sibling overlays isolated, and explicitly defers sibling write conflict policy.
|
||||
- Placeholder scan: no task uses “TBD” or “add tests” without concrete test content.
|
||||
- Type consistency: the plan uses current names: `ForeachBarrierState`, `PendingItemResult`, `StatePatch`, `state_view_for_frame`, `item_frame_owner`, and `build_output_patch`.
|
||||
@@ -0,0 +1,211 @@
|
||||
# Concurrent Foreach Phase 4 Roadmap
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Split concurrent foreach execution into safe, independently testable runtime slices.
|
||||
|
||||
**Architecture:** `foreach(mode="concurrent")` is the workflow mode. Sync runtime should interleave admitted item frames one node call at a time; async runtime may later run admitted async node handlers simultaneously. All item writes must flow through `StatePatch` and commit at a foreach barrier, not directly from child frames into shared state.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, Pydantic v2, pytest, `wf_core.runtime.scheduler`, `wf_core.runtime.foreach_state`, `wf_core.runtime.ops.state`.
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
Already implemented:
|
||||
|
||||
- `ForeachNode.mode` accepts canonical `"concurrent"`.
|
||||
- `ForeachConcurrentPolicy` exists with `max_active`, `max_outstanding`, and `interrupt="quiesce"`.
|
||||
- Legacy `mode="parallel"` / `parallel={...}` parse into canonical concurrent shape.
|
||||
- `ForeachItemErrorPolicy` exists with `fail`, `skip`, and `collect`.
|
||||
- `completed_with_errors` is derived for `skip` and `collect`.
|
||||
- `collect_to` is validated as a declared array state field.
|
||||
- `StatePatch`, `build_output_patch(...)`, and `commit_state_patch(...)` exist.
|
||||
- `ForeachBarrierState`, `PendingItemResult`, and `ItemErrorRecord` exist.
|
||||
- Serial foreach progress now uses `ForeachBarrierState`.
|
||||
- Sync `foreach(mode="concurrent")` runs with fail-only item policy, bounded
|
||||
admission, deterministic interleaving, item-local overlays, and barrier
|
||||
commits.
|
||||
- Multi-step concurrent item bodies are supported for fail-only item policy.
|
||||
- Barrier write validation rejects ambiguous sibling writes: same-path sibling
|
||||
writes require a `mergeable` reducer, and ancestor/descendant
|
||||
sibling writes are rejected.
|
||||
- Concurrent `item_error.action="skip"` and `"collect"` are supported.
|
||||
- `collect` writes ordered item error records to `collect_to`, writes an empty
|
||||
list on clean success, and emits `completed_with_errors` only when failures
|
||||
were collected.
|
||||
- Async runtime batches ready concurrent-foreach item node handlers so handler
|
||||
awaits may overlap up to admitted `max_active` work. State finalization and
|
||||
traces still happen sequentially after handler results return.
|
||||
|
||||
## Non-Goals For Phase 4
|
||||
|
||||
- Do not implement Fork/Gather graph nodes.
|
||||
- Do not turn `JoinNode` into a real barrier.
|
||||
- Do not run sync node handlers in threads or processes.
|
||||
- Do not add OpenTelemetry.
|
||||
- Do not add platform-level source/tool semaphores.
|
||||
- Do not add persistent run storage.
|
||||
|
||||
## Slice 1: Sync Concurrent Foreach, Fail-Only
|
||||
|
||||
Implement first because it proves the scheduler and frame admission model without async task orchestration or handled item failures.
|
||||
|
||||
Scope:
|
||||
|
||||
- `foreach(mode="concurrent", item_error.action="fail")` runs in sync runtime.
|
||||
- Parent foreach admits up to `concurrent.max_active` item frames.
|
||||
- Scheduler interleaves item frames one step at a time.
|
||||
- Child output writes are buffered as per-item `StatePatch` objects.
|
||||
- Barrier commits all successful item patches only when every item succeeds.
|
||||
- Any item runtime failure fails the whole run.
|
||||
- `skip` and `collect` remain runtime-unsupported for concurrent mode.
|
||||
|
||||
Plan:
|
||||
|
||||
- See [`2026-05-22-concurrent-foreach-v1-sync-fail-only.md`](2026-05-22-concurrent-foreach-v1-sync-fail-only.md).
|
||||
|
||||
## Slice 2: Item-Local Overlays
|
||||
|
||||
Implemented after Slice 1 because fail-only concurrent foreach needed
|
||||
lineage-local reads before multi-step item bodies could be supported. Overlays
|
||||
let later nodes in one item read earlier buffered writes from the same item
|
||||
without exposing those writes to siblings.
|
||||
|
||||
Scope:
|
||||
|
||||
- `state_view_for_frame(...)` returns committed parent state plus current item
|
||||
overlay for concurrent foreach item frames.
|
||||
- Parent `RunState.state` remains unchanged until the barrier commits.
|
||||
- Item patches accumulate across multiple nodes in the same item lineage.
|
||||
- Multi-step concurrent item bodies are supported.
|
||||
- Sibling item overlays remain isolated.
|
||||
- Sibling write conflict policy remains deferred to Slice 3.
|
||||
|
||||
Plan:
|
||||
|
||||
- See [`2026-05-22-concurrent-foreach-item-overlays.md`](2026-05-22-concurrent-foreach-item-overlays.md).
|
||||
|
||||
## Slice 3: Barrier Commit Conflict Semantics
|
||||
|
||||
Implement after Slice 2 so conflict checks operate on real item-local overlays
|
||||
and multi-step item patches.
|
||||
|
||||
Scope:
|
||||
|
||||
- Detect sibling lineage writes to the same state path.
|
||||
- If exactly one lineage writes a destination path, default replace is allowed.
|
||||
- If multiple sibling lineages write the same destination path, a declared
|
||||
`mergeable` reducer is required.
|
||||
- Ancestor/descendant writes across sibling lineages are conflicts unless an explicit future merge strategy covers them.
|
||||
- Commit order is item index order, never completion order.
|
||||
|
||||
Files likely touched:
|
||||
|
||||
- `src/wf_core/runtime/foreach_state.py`
|
||||
- `src/wf_core/runtime/ops/state.py`
|
||||
- `src/wf_core/runtime/ops/foreach.py`
|
||||
- `tests/core/test_concurrent_foreach.py`
|
||||
|
||||
Key tests:
|
||||
|
||||
- `test_concurrent_foreach_rejects_sibling_writes_without_reducer`
|
||||
- `test_concurrent_foreach_applies_reducer_in_item_index_order`
|
||||
- `test_concurrent_foreach_rejects_ancestor_descendant_write_conflict`
|
||||
|
||||
Plan:
|
||||
|
||||
- See [`2026-05-22-concurrent-foreach-barrier-write-semantics.md`](2026-05-22-concurrent-foreach-barrier-write-semantics.md).
|
||||
|
||||
## Slice 4: Item Error Policies
|
||||
|
||||
Implemented after barrier success commits became correct.
|
||||
|
||||
Scope:
|
||||
|
||||
- `item_error.action="skip"` continues after item runtime failures.
|
||||
- `item_error.action="collect"` continues and writes structured errors to `collect_to`.
|
||||
- Both emit `completed_with_errors` if at least one item failed.
|
||||
- `collect` writes an empty list and emits `done` if all items succeed.
|
||||
- Failed item frames remain `FAILED`; parent foreach decides whether the failure is handled.
|
||||
|
||||
Files likely touched:
|
||||
|
||||
- `src/wf_core/runtime/foreach_state.py`
|
||||
- `src/wf_core/runtime/ops/foreach.py`
|
||||
- `tests/core/test_concurrent_foreach_errors.py`
|
||||
|
||||
Key tests:
|
||||
|
||||
- `test_concurrent_foreach_skip_emits_completed_with_errors`
|
||||
- `test_concurrent_foreach_collect_writes_ordered_error_records`
|
||||
- `test_concurrent_foreach_collect_writes_empty_list_on_clean_success`
|
||||
|
||||
## Slice 5: Async Concurrent Foreach
|
||||
|
||||
Implemented after sync semantics stabilized.
|
||||
|
||||
Scope:
|
||||
|
||||
- Async runtime may have multiple async node handler calls in flight.
|
||||
- `concurrent.max_active` caps admitted/running item work.
|
||||
- Sync handlers are still called normally; no thread/process executor.
|
||||
- Trace remains append-only chronological execution history.
|
||||
- Barrier commit order remains item index order.
|
||||
|
||||
Files likely touched:
|
||||
|
||||
- `src/wf_core/runtime/engine.py`
|
||||
- `src/wf_core/runtime/step.py`
|
||||
- `src/wf_core/runtime/ops/nodes.py`
|
||||
- `src/wf_core/runtime/ops/foreach.py`
|
||||
- `tests/core/test_concurrent_foreach_async.py`
|
||||
|
||||
Key tests:
|
||||
|
||||
- `test_async_concurrent_foreach_respects_max_active`
|
||||
- `test_async_concurrent_foreach_commits_in_item_index_order`
|
||||
|
||||
## Slice 6: Interrupt Quiescence
|
||||
|
||||
Implemented after async execution exists.
|
||||
|
||||
Scope:
|
||||
|
||||
- If any concurrent item interrupts, the whole run pauses.
|
||||
- No new item frames are admitted after the interrupt.
|
||||
- Already-started async node calls drain to pending results.
|
||||
- The caller gets control only at a quiescent point.
|
||||
- Pending results do not commit until resume/commit policy allows it.
|
||||
- Item frames that route into an `InterruptNode` are prioritized before the
|
||||
parent foreach can refill capacity.
|
||||
- Already-started async handler calls drain at the batch boundary; state
|
||||
finalization remains sequential.
|
||||
|
||||
Files likely touched:
|
||||
|
||||
- `src/wf_core/runtime/engine.py`
|
||||
- `src/wf_core/runtime/preparation.py`
|
||||
- `src/wf_core/runtime/ops/interrupts.py`
|
||||
- `src/wf_core/runtime/ops/foreach.py`
|
||||
- `tests/core/test_concurrent_foreach_interrupts.py`
|
||||
|
||||
Key tests:
|
||||
|
||||
- `test_concurrent_foreach_interrupt_returns_before_refill`
|
||||
- `test_resume_prioritizes_interrupted_item_frame_before_siblings`
|
||||
|
||||
## Execution Order
|
||||
|
||||
1. Sync concurrent foreach, fail-only.
|
||||
2. Item-local overlays for multi-step item bodies.
|
||||
3. Barrier conflict semantics.
|
||||
4. `skip` / `collect` item error policies.
|
||||
5. Async concurrent foreach.
|
||||
6. Interrupt quiescence.
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: the roadmap covers scheduler admission, barrier commits, reducer conflicts, handled item failures, async handler execution, and interrupt quiescence.
|
||||
- Placeholder scan: each slice has scope, likely files, and named tests; concrete code lives in the slice-specific plan.
|
||||
- Type consistency: the roadmap uses canonical `concurrent`, `ForeachConcurrentPolicy`, `ForeachBarrierState`, `PendingItemResult`, and `StatePatch`.
|
||||
@@ -0,0 +1,591 @@
|
||||
# Concurrent Foreach Roadmap Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Implement concurrent foreach incrementally without breaking serial workflows or duplicating state-write logic.
|
||||
|
||||
**Architecture:** The work was split into four independently shippable layers: policy models, state patch extraction, barrier runtime state, and concurrent execution. Each layer preserves current serial behavior and adds tests before implementation. Sync runtime supports deterministic interleaving for concurrent foreach; async runtime can additionally run admitted async node handlers simultaneously.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, dataclasses, pytest, basedpyright, ruff, existing `wf_core` scheduler/runtime modules.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
- Phase 1 is implemented: foreach policy models, legacy `parallel` parse-only
|
||||
compatibility, derived `completed_with_errors` outcomes, and collect
|
||||
destination validation exist.
|
||||
- Phase 2 is implemented: node output writes can be split into
|
||||
`build_output_patch(...)` and `commit_state_patch(...)`; the old
|
||||
`apply_output_bindings(...)` helper remains as the compatibility wrapper.
|
||||
- Phase 3 is implemented: `wf_core.runtime.foreach_state` owns typed barrier
|
||||
metadata and serial foreach progress now uses that metadata instead of ad hoc
|
||||
`foreach_progress`.
|
||||
- Phase 4 is implemented: `foreach(mode="concurrent")` supports sync
|
||||
interleaving, async item-node batching, barrier commits, item error policies,
|
||||
and quiescent interrupt handling.
|
||||
- Phase 4 details live in the dedicated roadmap:
|
||||
[`2026-05-22-concurrent-foreach-phase4-roadmap.md`](2026-05-22-concurrent-foreach-phase4-roadmap.md).
|
||||
Start with
|
||||
[`2026-05-22-concurrent-foreach-v1-sync-fail-only.md`](2026-05-22-concurrent-foreach-v1-sync-fail-only.md).
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Foreach Policy Models
|
||||
|
||||
**Goal:** Add the future policy shape while keeping runtime behavior serial-only.
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/models/steps.py`
|
||||
- Modify: `src/wf_core/validation/outcomes.py`
|
||||
- Modify: `src/wf_core/validation/steps.py`
|
||||
- Modify: `src/wf_authoring/builder/core.py`
|
||||
- Test: `tests/core/test_foreach_policy.py`
|
||||
- Test: `tests/authoring/test_builder.py`
|
||||
|
||||
- [ ] **Step 1: Add failing model tests**
|
||||
|
||||
Create `tests/core/test_foreach_policy.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from wf_core.models.steps import ForeachNode
|
||||
|
||||
|
||||
def test_serial_foreach_defaults_to_fail_item_policy() -> None:
|
||||
node = ForeachNode.model_validate(
|
||||
{
|
||||
"id": "each",
|
||||
"type": "foreach",
|
||||
"over": {"root": "state", "parts": ["items"]},
|
||||
"as": "item",
|
||||
}
|
||||
)
|
||||
|
||||
assert node.mode == "serial"
|
||||
assert node.item_error.action == "fail"
|
||||
assert node.item_error.collect_to is None
|
||||
assert node.concurrent is None
|
||||
|
||||
|
||||
def test_collect_item_policy_requires_collect_to() -> None:
|
||||
with pytest.raises(ValidationError, match="collect_to"):
|
||||
ForeachNode.model_validate(
|
||||
{
|
||||
"id": "each",
|
||||
"type": "foreach",
|
||||
"over": {"root": "state", "parts": ["items"]},
|
||||
"as": "item",
|
||||
"item_error": {"action": "collect"},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_concurrent_policy_requires_concurrent_mode() -> None:
|
||||
with pytest.raises(ValidationError, match="concurrent policy"):
|
||||
ForeachNode.model_validate(
|
||||
{
|
||||
"id": "each",
|
||||
"type": "foreach",
|
||||
"over": {"root": "state", "parts": ["items"]},
|
||||
"as": "item",
|
||||
"concurrent": {"max_active": 4, "max_outstanding": 20},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_concurrent_policy_validates_capacity_order() -> None:
|
||||
with pytest.raises(ValidationError, match="max_outstanding"):
|
||||
ForeachNode.model_validate(
|
||||
{
|
||||
"id": "each",
|
||||
"type": "foreach",
|
||||
"over": {"root": "state", "parts": ["items"]},
|
||||
"as": "item",
|
||||
"mode": "concurrent",
|
||||
"concurrent": {"max_active": 10, "max_outstanding": 4},
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add policy models**
|
||||
|
||||
In `src/wf_core/models/steps.py`, add:
|
||||
|
||||
```python
|
||||
from typing import Self
|
||||
|
||||
|
||||
class ForeachItemErrorPolicy(BaseModel):
|
||||
"""Policy for runtime failures inside one foreach item lineage."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
action: Literal["fail", "skip", "collect"] = "fail"
|
||||
collect_to: StatePath | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_collect_to(self) -> Self:
|
||||
if self.action == "collect" and self.collect_to is None:
|
||||
raise ValueError("collect item error policy requires collect_to")
|
||||
if self.action != "collect" and self.collect_to is not None:
|
||||
raise ValueError("collect_to is only valid when action='collect'")
|
||||
return self
|
||||
|
||||
|
||||
class ForeachConcurrentPolicy(BaseModel):
|
||||
"""Concurrency policy for foreach frame admission."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
max_active: int = Field(default=4, ge=1)
|
||||
max_outstanding: int = Field(default=20, ge=1)
|
||||
interrupt: Literal["quiesce"] = "quiesce"
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_capacity(self) -> Self:
|
||||
if self.max_outstanding < self.max_active:
|
||||
raise ValueError("max_outstanding must be >= max_active")
|
||||
return self
|
||||
```
|
||||
|
||||
Update `ForeachNode`:
|
||||
|
||||
```python
|
||||
item_error: ForeachItemErrorPolicy = Field(default_factory=ForeachItemErrorPolicy)
|
||||
concurrent: ForeachConcurrentPolicy | None = None
|
||||
on_item_error: Literal["fail", "collect", "skip"] | None = Field(
|
||||
default=None,
|
||||
exclude=True,
|
||||
description="Deprecated parse-only shorthand; use item_error.action.",
|
||||
)
|
||||
```
|
||||
|
||||
Add a `model_validator(mode="before")` that converts old `on_item_error` into `item_error.action`.
|
||||
|
||||
Add a `model_validator(mode="after")` that enforces:
|
||||
|
||||
```python
|
||||
if self.mode == "concurrent" and self.concurrent is None:
|
||||
raise ValueError("concurrent foreach requires concurrent policy")
|
||||
if self.mode == "serial" and self.concurrent is not None:
|
||||
raise ValueError("concurrent policy is only valid when mode='concurrent'")
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update derived outcomes**
|
||||
|
||||
In `src/wf_core/validation/outcomes.py`, update foreach outcome derivation:
|
||||
|
||||
```python
|
||||
if step.type == "foreach":
|
||||
outcomes = {"loop", "done"}
|
||||
if step.item_error.action in {"skip", "collect"}:
|
||||
outcomes.add("completed_with_errors")
|
||||
return outcomes
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Validate collect destination schema**
|
||||
|
||||
In `src/wf_core/validation/steps.py`, when `node.item_error.action == "collect"`:
|
||||
|
||||
```python
|
||||
destination_root = _state_destination_root(node.item_error.collect_to)
|
||||
if destination_root is None or destination_root not in state_root_fields:
|
||||
report.add(...)
|
||||
```
|
||||
|
||||
Add a follow-up test that collect-to unknown state root reports a validation issue.
|
||||
|
||||
- [ ] **Step 5: Keep runtime unsupported**
|
||||
|
||||
In `src/wf_core/runtime/ops/foreach.py`, keep:
|
||||
|
||||
```python
|
||||
if step.mode != "serial":
|
||||
raise WorkflowExecutionError("concurrent foreach execution is not implemented yet")
|
||||
```
|
||||
|
||||
Add a comment:
|
||||
|
||||
```python
|
||||
# Policy models are accepted before execution support so saved workflows can
|
||||
# validate shape, but runtime must reject concurrent until barrier commits exist.
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Verify phase**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_foreach_policy.py tests/authoring/test_builder.py -q
|
||||
uvx ruff check src tests
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: State Patch Extraction
|
||||
|
||||
**Goal:** Split current node output writes into reusable “build patch” and “commit patch” operations without changing current serial behavior.
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||
- Test: `tests/core/test_atomic_state_patches.py`
|
||||
- Test: `tests/core/test_nested_state_paths.py`
|
||||
|
||||
- [ ] **Step 1: Add state patch model**
|
||||
|
||||
In `src/wf_core/runtime/ops/state.py`, add:
|
||||
|
||||
```python
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class StatePatch:
|
||||
"""Validated state writes produced by one step before commit."""
|
||||
|
||||
changes: dict[str, Any] = field(default_factory=dict)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Extract patch builder**
|
||||
|
||||
Refactor existing `apply_output_bindings(...)` into:
|
||||
|
||||
```python
|
||||
def build_output_patch(
|
||||
workflow: Workflow,
|
||||
bindings: Sequence[OutputBinding],
|
||||
output: Mapping[str, Any],
|
||||
state: MutableMapping[str, Any],
|
||||
*,
|
||||
reducers: Mapping[str, ReducerDefinition] | None = None,
|
||||
missing_field_message: str = "node output is missing required field {field}",
|
||||
) -> StatePatch:
|
||||
...
|
||||
```
|
||||
|
||||
This function should:
|
||||
|
||||
- validate source paths
|
||||
- validate destination paths
|
||||
- calculate reducer-aware changes
|
||||
- not mutate `state`
|
||||
|
||||
- [ ] **Step 3: Extract patch committer**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def commit_state_patch(
|
||||
state: MutableMapping[str, Any],
|
||||
patch: StatePatch,
|
||||
) -> dict[str, Any]:
|
||||
"""Commit a validated patch to state and return committed changes."""
|
||||
for path, value in patch.changes.items():
|
||||
set_nested_value(state, split_state_path(path), value)
|
||||
return dict(patch.changes)
|
||||
```
|
||||
|
||||
Use the existing typed path helpers; do not reintroduce dotted-string parsing if a typed path helper exists.
|
||||
|
||||
- [ ] **Step 4: Preserve old API**
|
||||
|
||||
Keep `apply_output_bindings(...)` as a wrapper:
|
||||
|
||||
```python
|
||||
patch = build_output_patch(...)
|
||||
return commit_state_patch(state, patch)
|
||||
```
|
||||
|
||||
Existing callers should keep working.
|
||||
|
||||
- [ ] **Step 5: Add equivalence tests**
|
||||
|
||||
Add tests that compare:
|
||||
|
||||
```python
|
||||
old_changes = apply_output_bindings(...)
|
||||
patch = build_output_patch(...)
|
||||
new_changes = commit_state_patch(state2, patch)
|
||||
assert old_changes["state.some_path"] == new_changes["state.some_path"]
|
||||
assert state1["some_path"] == state2["some_path"]
|
||||
```
|
||||
|
||||
Do not assert whole dict equality unless the test intentionally owns the full structure.
|
||||
|
||||
- [ ] **Step 6: Verify phase**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_atomic_state_patches.py tests/core/test_nested_state_paths.py tests/authoring/test_demo_workflow.py -q
|
||||
uvx ruff check src tests
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: all pass; full suite should still pass before moving on.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Foreach Barrier Runtime State
|
||||
|
||||
**Goal:** Add resumable barrier metadata and pending result structures without enabling concurrent execution.
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/scheduler.py`
|
||||
- Create: `src/wf_core/runtime/foreach_state.py`
|
||||
- Modify: `src/wf_core/runtime/ops/foreach.py`
|
||||
- Test: `tests/core/test_foreach_barrier_state.py`
|
||||
|
||||
- [ ] **Step 1: Add pending result dataclasses**
|
||||
|
||||
Create `src/wf_core/runtime/foreach_state.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Literal
|
||||
|
||||
from wf_core.runtime.ops.state import StatePatch
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class ItemErrorRecord:
|
||||
"""Structured runtime failure record for one foreach item."""
|
||||
|
||||
index: int
|
||||
frame_id: str
|
||||
node_id: str
|
||||
error_type: str
|
||||
message: str
|
||||
item: Any = None
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class PendingItemResult:
|
||||
"""Buffered item result waiting for foreach barrier commit."""
|
||||
|
||||
index: int
|
||||
frame_id: str
|
||||
status: Literal["succeeded", "failed"]
|
||||
patch: StatePatch = field(default_factory=StatePatch)
|
||||
error: ItemErrorRecord | None = None
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add foreach barrier state**
|
||||
|
||||
In the same file:
|
||||
|
||||
```python
|
||||
@dataclass(slots=True)
|
||||
class ForeachBarrierState:
|
||||
"""Resumable state owned by one foreach parent frame."""
|
||||
|
||||
next_index: int = 0
|
||||
active_frame_ids: tuple[str, ...] = ()
|
||||
outstanding_frame_ids: tuple[str, ...] = ()
|
||||
pending_results: dict[int, PendingItemResult] = field(default_factory=dict)
|
||||
```
|
||||
|
||||
Add `to_metadata()` / `from_frame()` helpers. Wrong frame kind returns `None`; malformed metadata for a foreach parent raises `WorkflowExecutionError`.
|
||||
|
||||
- [ ] **Step 3: Move serial progress into typed state**
|
||||
|
||||
Current serial foreach uses:
|
||||
|
||||
```python
|
||||
progress_map = frame.metadata.setdefault("foreach_progress", {})
|
||||
```
|
||||
|
||||
Replace with typed barrier state, but keep behavior equivalent:
|
||||
|
||||
```python
|
||||
barrier = ForeachBarrierState.from_frame(frame) or ForeachBarrierState()
|
||||
loop_index = barrier.next_index
|
||||
barrier.next_index += 1
|
||||
frame.metadata["foreach_barrier"] = barrier.to_metadata()
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add serialization tests**
|
||||
|
||||
Test:
|
||||
|
||||
```python
|
||||
def test_foreach_barrier_state_round_trips_through_frame_metadata() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
Assert specific fields:
|
||||
|
||||
```python
|
||||
assert loaded.next_index == 2
|
||||
assert loaded.outstanding_frame_ids == ("child-1",)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Keep serial behavior passing**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_foreach_barrier_state.py tests/authoring/test_demo_workflow.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Concurrent Foreach Execution
|
||||
|
||||
**Goal:** Enable `foreach(mode="concurrent")` using policy limits, pending results, and barrier commits. Sync runtime interleaves admitted item frames one node call at a time; async runtime may run admitted async node handler calls simultaneously.
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/foreach.py`
|
||||
- Modify: `src/wf_core/runtime/step.py`
|
||||
- Modify: `src/wf_core/runtime/engine.py`
|
||||
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||
- Modify: `src/wf_core/runtime/foreach_state.py`
|
||||
- Test: `tests/core/test_concurrent_foreach.py`
|
||||
|
||||
- [ ] **Step 1: Add sync interleaving and async execution tests**
|
||||
|
||||
Create `tests/core/test_concurrent_foreach.py` with:
|
||||
|
||||
```python
|
||||
def test_sync_runtime_interleaves_concurrent_foreach() -> None:
|
||||
...
|
||||
|
||||
|
||||
async def test_async_runtime_accepts_concurrent_foreach() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
Historical expectation before Phase 4: both tests failed because runtime rejected
|
||||
concurrent mode. Current implementation status: these tests should pass.
|
||||
|
||||
- [ ] **Step 2: Add capacity tests**
|
||||
|
||||
Use async node handlers that record start/completion order and block on `asyncio.Event`.
|
||||
|
||||
Test:
|
||||
|
||||
```python
|
||||
async def test_concurrent_foreach_respects_max_active() -> None:
|
||||
...
|
||||
assert max_seen_active == 2
|
||||
```
|
||||
|
||||
Use `max_active=2`.
|
||||
|
||||
- [ ] **Step 3: Add outstanding tests**
|
||||
|
||||
Use a node that blocks internally through a future block helper or controlled async wait.
|
||||
|
||||
Test:
|
||||
|
||||
```python
|
||||
async def test_blocked_items_count_against_max_outstanding_not_active() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
This may require a small test-only node that blocks through the runtime-supported internal wait. If internal blocking is not implemented yet, defer this test to subgraph/internal-wait work and keep `max_outstanding` tested through queued children.
|
||||
|
||||
- [ ] **Step 4: Add collect/skip tests**
|
||||
|
||||
Tests:
|
||||
|
||||
```python
|
||||
async def test_concurrent_collect_writes_ordered_errors_and_emits_completed_with_errors() -> None:
|
||||
...
|
||||
|
||||
|
||||
async def test_concurrent_skip_emits_completed_with_errors_without_hidden_state() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
Assert:
|
||||
|
||||
```python
|
||||
assert run.state["document_errors"][0]["index"] == 1
|
||||
assert run.trace[-1].outcome == "completed_with_errors"
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Add barrier commit ordering test**
|
||||
|
||||
Use nodes that complete out of order but write list-like results.
|
||||
|
||||
Assert committed state is ordered by item index, not completion order.
|
||||
|
||||
- [ ] **Step 6: Implement concurrent child scheduling**
|
||||
|
||||
In `step_foreach`, branch by mode:
|
||||
|
||||
```python
|
||||
if step.mode == "serial":
|
||||
return step_foreach_serial(...)
|
||||
return step_foreach_concurrent(...)
|
||||
```
|
||||
|
||||
`step_foreach_concurrent` should:
|
||||
|
||||
- inspect `ForeachBarrierState`
|
||||
- start children while `active < max_active` and `outstanding < max_outstanding`
|
||||
- block parent when waiting for children
|
||||
- finish when all items terminal
|
||||
- commit barrier patches in item index order
|
||||
- emit `done` or `completed_with_errors`
|
||||
|
||||
- [ ] **Step 7: Add async node-call budget seam**
|
||||
|
||||
Add execution option shape only if needed by implementation:
|
||||
|
||||
```python
|
||||
@dataclass(slots=True)
|
||||
class RuntimeLimits:
|
||||
max_active_node_calls: int = 16
|
||||
```
|
||||
|
||||
If this is too large for the first concurrent pass, leave global node-call budget as follow-up and rely on foreach `max_active`.
|
||||
|
||||
- [ ] **Step 8: Verify phase**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_concurrent_foreach.py tests/authoring/test_demo_workflow.py -q
|
||||
uv run pytest -q
|
||||
uvx ruff check src tests
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order Recommendation
|
||||
|
||||
Ship these as separate commits/PRs:
|
||||
|
||||
1. Phase 1: policy shape and validation
|
||||
2. Phase 2: patch extraction with no behavior change
|
||||
3. Phase 3: barrier metadata with serial behavior unchanged
|
||||
4. Phase 4: concurrent execution
|
||||
|
||||
Phase 4 depended on Phase 2 and Phase 3 because concurrent foreach needs patch extraction and resumable barrier state. Keep future fork/gather or native subgraph work layered on top of those runtime primitives instead of replacing them.
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: ADR 0002 decisions are represented across the four phases.
|
||||
- Intentional gaps: explicit Fork/Gather, lineage-token graph nodes, OpenTelemetry, platform source/tool caps, and full run persistence are not included.
|
||||
- Risk control: phases 1-3 preserved serial behavior until phase 4 enabled `mode="concurrent"`.
|
||||
+1120
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,756 @@
|
||||
# Scheduler Foundation Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add the internal scheduler foundation needed for future concurrent foreach and native subgraphs while preserving current serial workflow behavior.
|
||||
|
||||
**Architecture:** `RunState.frames` remains the source of frame lifecycle state, and a new serialized `ready_frame_ids` queue defines deterministic scheduling order. A new internal `wf_core.runtime.scheduler` module owns frame creation, enqueue/select, block/wake, and no-ready-frame resolution. Existing sync and async engines both use scheduler selection before `prepare_step`; node execution stays split between sync and async paths.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, pytest, Pydantic workflow models, existing `wf_core` runtime modules.
|
||||
|
||||
---
|
||||
|
||||
## Files
|
||||
|
||||
- Modify: `src/wf_core/run_state.py`
|
||||
- Add `FrameStatus.BLOCKED`.
|
||||
- Add `RunState.ready_frame_ids: list[str]`.
|
||||
- Create: `src/wf_core/runtime/scheduler.py`
|
||||
- Internal scheduler helpers and typed block/foreach metadata helpers.
|
||||
- Modify: `src/wf_core/runtime/ops/runs.py`
|
||||
- Initialize root frame and ready queue.
|
||||
- Modify: `src/wf_core/runtime/ops/flow.py`
|
||||
- Re-enqueue normal frame advances and complete terminal frames through scheduler helpers.
|
||||
- Modify: `src/wf_core/runtime/ops/foreach.py`
|
||||
- Use typed foreach metadata, explicit child frame creation, block parent, enqueue child.
|
||||
- Modify: `src/wf_core/runtime/ops/frames.py`
|
||||
- Keep context helpers; demote stack-collapse usage or leave compatibility wrappers that call scheduler helpers.
|
||||
- Modify: `src/wf_core/runtime/preparation.py`
|
||||
- Stop relying on stack-style frame collapse for selection.
|
||||
- Resume interrupt by waking/enqueueing the resumed frame.
|
||||
- Modify: `src/wf_core/runtime/engine.py`
|
||||
- Use scheduler loop for sync and async resume.
|
||||
- Modify: `src/wf_core/runtime/step.py`
|
||||
- Keep `prepare_step()` boundary; ensure selected frame is already chosen by scheduler.
|
||||
- Test: `tests/core/test_scheduler.py`
|
||||
- Unit tests for scheduler helpers.
|
||||
- Test: `tests/core/test_run_state.py`
|
||||
- Add serialization/additive field checks if needed.
|
||||
- Test: existing foreach/interrupt tests under `tests/authoring/` and `tests/core/`
|
||||
- Add focused regressions where the behavior is not already covered.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add RunState Scheduler Fields
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/run_state.py`
|
||||
- Test: `tests/core/test_scheduler.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for ready queue serialization and frame status**
|
||||
|
||||
Create `tests/core/test_scheduler.py` with:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_core.run_state import FrameStatus, RunState, RunStatus
|
||||
|
||||
|
||||
def test_run_state_serializes_ready_frame_ids() -> None:
|
||||
run = RunState(
|
||||
workflow_name="demo",
|
||||
status=RunStatus.PENDING,
|
||||
workflow_input={},
|
||||
state={},
|
||||
ready_frame_ids=["root"],
|
||||
)
|
||||
|
||||
dumped = run.to_dict()
|
||||
|
||||
assert dumped["ready_frame_ids"] == ["root"]
|
||||
|
||||
|
||||
def test_frame_status_has_blocked() -> None:
|
||||
assert FrameStatus.BLOCKED == "blocked"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run failing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_scheduler.py -q
|
||||
```
|
||||
|
||||
Expected: fails because `ready_frame_ids` and `FrameStatus.BLOCKED` do not exist.
|
||||
|
||||
- [ ] **Step 3: Implement minimal state fields**
|
||||
|
||||
In `src/wf_core/run_state.py`:
|
||||
|
||||
```python
|
||||
class FrameStatus(StrEnum):
|
||||
PENDING = "pending"
|
||||
RUNNING = "running"
|
||||
BLOCKED = "blocked"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
INTERRUPTED = "interrupted"
|
||||
```
|
||||
|
||||
Add to `RunState`:
|
||||
|
||||
```python
|
||||
ready_frame_ids: list[str] = field(default_factory=list)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Verify**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_scheduler.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Create Internal Scheduler Helpers
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_core/runtime/scheduler.py`
|
||||
- Test: `tests/core/test_scheduler.py`
|
||||
|
||||
- [ ] **Step 1: Add failing scheduler helper tests**
|
||||
|
||||
Append tests:
|
||||
|
||||
```python
|
||||
import pytest
|
||||
|
||||
from wf_core.errors import WorkflowExecutionError
|
||||
from wf_core.run_state import ExecutionFrame
|
||||
from wf_core.runtime.scheduler import (
|
||||
add_frame,
|
||||
block_frame_on_children,
|
||||
enqueue_frame,
|
||||
select_next_frame,
|
||||
wake_frame,
|
||||
)
|
||||
|
||||
|
||||
def _run() -> RunState:
|
||||
return RunState(
|
||||
workflow_name="demo",
|
||||
status=RunStatus.RUNNING,
|
||||
workflow_input={},
|
||||
state={},
|
||||
)
|
||||
|
||||
|
||||
def test_add_frame_rejects_duplicate_frame_ids() -> None:
|
||||
run = _run()
|
||||
add_frame(run, ExecutionFrame(id="root", kind="root", node_id="a"))
|
||||
|
||||
with pytest.raises(WorkflowExecutionError, match="duplicate frame id"):
|
||||
add_frame(run, ExecutionFrame(id="root", kind="root", node_id="a"))
|
||||
|
||||
|
||||
def test_enqueue_is_unique_and_priority_moves_to_front() -> None:
|
||||
run = _run()
|
||||
add_frame(run, ExecutionFrame(id="a", kind="root", node_id="a"))
|
||||
add_frame(run, ExecutionFrame(id="b", kind="root", node_id="b"))
|
||||
|
||||
enqueue_frame(run, "a")
|
||||
enqueue_frame(run, "b")
|
||||
enqueue_frame(run, "a")
|
||||
enqueue_frame(run, "a", front=True)
|
||||
|
||||
assert run.ready_frame_ids == ["a", "b"]
|
||||
|
||||
|
||||
def test_select_next_frame_pops_and_marks_running() -> None:
|
||||
run = _run()
|
||||
add_frame(run, ExecutionFrame(id="root", kind="root", node_id="a"))
|
||||
enqueue_frame(run, "root")
|
||||
|
||||
frame = select_next_frame(run)
|
||||
|
||||
assert frame is not None
|
||||
assert frame.id == "root"
|
||||
assert frame.status == FrameStatus.RUNNING
|
||||
assert run.ready_frame_ids == []
|
||||
assert run.current_frame_id == "root"
|
||||
assert run.current_node_id == "a"
|
||||
|
||||
|
||||
def test_blocked_frame_is_not_selectable_until_woken() -> None:
|
||||
run = _run()
|
||||
add_frame(run, ExecutionFrame(id="parent", kind="root", node_id="foreach"))
|
||||
block_frame_on_children(run, "parent", ("child",))
|
||||
|
||||
assert select_next_frame(run) is None
|
||||
|
||||
wake_frame(run, "parent")
|
||||
selected = select_next_frame(run)
|
||||
|
||||
assert selected is not None
|
||||
assert selected.id == "parent"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run failing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_scheduler.py -q
|
||||
```
|
||||
|
||||
Expected: fails because `wf_core.runtime.scheduler` does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement scheduler helpers**
|
||||
|
||||
Create `src/wf_core/runtime/scheduler.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
from dataclasses import dataclass
|
||||
|
||||
from wf_core.errors import WorkflowExecutionError
|
||||
from wf_core.run_state import ExecutionFrame, FrameStatus, RunState, RunStatus
|
||||
|
||||
|
||||
@dataclass(slots=True, frozen=True)
|
||||
class BlockedOnChildren:
|
||||
"""Typed block reason for frames waiting on child frame completion."""
|
||||
|
||||
child_frame_ids: tuple[str, ...]
|
||||
|
||||
@classmethod
|
||||
def from_frame(cls, frame: ExecutionFrame) -> "BlockedOnChildren | None":
|
||||
raw = frame.metadata.get("blocked_on")
|
||||
if raw is None:
|
||||
return None
|
||||
if not isinstance(raw, dict) or raw.get("type") != "child_frames":
|
||||
raise WorkflowExecutionError(
|
||||
f"malformed block reason for frame {frame.id!r}"
|
||||
)
|
||||
raw_ids = raw.get("frame_ids")
|
||||
if not isinstance(raw_ids, list) or not all(
|
||||
isinstance(item, str) for item in raw_ids
|
||||
):
|
||||
raise WorkflowExecutionError(
|
||||
f"malformed child frame ids for frame {frame.id!r}"
|
||||
)
|
||||
return cls(tuple(raw_ids))
|
||||
|
||||
def to_metadata(self) -> dict[str, object]:
|
||||
return {"type": "child_frames", "frame_ids": list(self.child_frame_ids)}
|
||||
|
||||
|
||||
def add_frame(run: RunState, frame: ExecutionFrame, *, ready: bool = False) -> None:
|
||||
"""Add a frame once; frame id reuse is always a runtime invariant error."""
|
||||
if frame.id in run.frames:
|
||||
raise WorkflowExecutionError(f"duplicate frame id {frame.id!r}")
|
||||
run.frames[frame.id] = frame
|
||||
if ready:
|
||||
enqueue_frame(run, frame.id)
|
||||
|
||||
|
||||
def enqueue_frame(run: RunState, frame_id: str, *, front: bool = False) -> None:
|
||||
"""Put a pending frame in the ready queue without creating duplicates."""
|
||||
frame = _frame(run, frame_id)
|
||||
if frame.status != FrameStatus.PENDING:
|
||||
raise WorkflowExecutionError(
|
||||
f"cannot enqueue frame {frame_id!r} with status {frame.status!s}"
|
||||
)
|
||||
if frame_id in run.ready_frame_ids:
|
||||
run.ready_frame_ids.remove(frame_id)
|
||||
if front:
|
||||
run.ready_frame_ids.insert(0, frame_id)
|
||||
else:
|
||||
run.ready_frame_ids.append(frame_id)
|
||||
|
||||
|
||||
def select_next_frame(run: RunState) -> ExecutionFrame | None:
|
||||
"""Select the next ready frame and expose it through compatibility cursor fields."""
|
||||
while run.ready_frame_ids:
|
||||
frame_id = run.ready_frame_ids.pop(0)
|
||||
frame = _frame(run, frame_id)
|
||||
if frame.status != FrameStatus.PENDING:
|
||||
raise WorkflowExecutionError(
|
||||
f"ready frame {frame_id!r} has status {frame.status!s}"
|
||||
)
|
||||
frame.status = FrameStatus.RUNNING
|
||||
run.current_frame_id = frame.id
|
||||
run.sync_from_current_frame()
|
||||
return frame
|
||||
return None
|
||||
|
||||
|
||||
def mark_frame_pending(run: RunState, frame_id: str, *, front: bool = False) -> None:
|
||||
"""Mark a live frame pending and enqueue it for future execution."""
|
||||
frame = _frame(run, frame_id)
|
||||
frame.status = FrameStatus.PENDING
|
||||
enqueue_frame(run, frame_id, front=front)
|
||||
|
||||
|
||||
def block_frame_on_children(
|
||||
run: RunState, frame_id: str, child_frame_ids: Sequence[str]
|
||||
) -> None:
|
||||
"""Mark a frame blocked on child completion and remove it from the ready queue."""
|
||||
frame = _frame(run, frame_id)
|
||||
run.ready_frame_ids = [item for item in run.ready_frame_ids if item != frame_id]
|
||||
frame.status = FrameStatus.BLOCKED
|
||||
frame.metadata["blocked_on"] = BlockedOnChildren(
|
||||
tuple(child_frame_ids)
|
||||
).to_metadata()
|
||||
|
||||
|
||||
def wake_frame(run: RunState, frame_id: str, *, front: bool = False) -> None:
|
||||
"""Wake a blocked or interrupted frame and enqueue it as pending."""
|
||||
frame = _frame(run, frame_id)
|
||||
if frame.status not in {FrameStatus.BLOCKED, FrameStatus.INTERRUPTED}:
|
||||
raise WorkflowExecutionError(
|
||||
f"cannot wake frame {frame_id!r} with status {frame.status!s}"
|
||||
)
|
||||
frame.status = FrameStatus.PENDING
|
||||
frame.metadata.pop("blocked_on", None)
|
||||
enqueue_frame(run, frame_id, front=front)
|
||||
|
||||
|
||||
def resolve_no_ready_frames(run: RunState) -> RunStatus:
|
||||
"""Classify an empty ready queue into a terminal or blocked run state."""
|
||||
if run.status == RunStatus.INTERRUPTED:
|
||||
return RunStatus.INTERRUPTED
|
||||
if any(frame.status == FrameStatus.FAILED for frame in run.frames.values()):
|
||||
return RunStatus.FAILED
|
||||
if run.frames and all(
|
||||
frame.status == FrameStatus.COMPLETED for frame in run.frames.values()
|
||||
):
|
||||
return RunStatus.COMPLETED
|
||||
if any(frame.status == FrameStatus.BLOCKED for frame in run.frames.values()):
|
||||
raise WorkflowExecutionError("run has no ready frames and is deadlocked")
|
||||
raise WorkflowExecutionError("run has no ready frames")
|
||||
|
||||
|
||||
def _frame(run: RunState, frame_id: str) -> ExecutionFrame:
|
||||
frame = run.frames.get(frame_id)
|
||||
if frame is None:
|
||||
raise WorkflowExecutionError(f"unknown frame id {frame_id!r}")
|
||||
return frame
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Verify helper tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_scheduler.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Initialize Root Through Scheduler
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/runs.py`
|
||||
- Test: `tests/core/test_scheduler.py`
|
||||
|
||||
- [ ] **Step 1: Add root initialization test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
from wf_core import SchemaRef, StateSchema, Workflow
|
||||
from wf_core.runtime.ops.runs import create_run_state
|
||||
|
||||
|
||||
def test_create_run_state_queues_root_frame() -> None:
|
||||
workflow = Workflow(
|
||||
name="demo",
|
||||
input_schema=SchemaRef(properties={}),
|
||||
state_schema=StateSchema(fields={}),
|
||||
output_schema=SchemaRef(properties={}),
|
||||
node_defs=[],
|
||||
start="first",
|
||||
nodes=[],
|
||||
edges=[],
|
||||
)
|
||||
|
||||
run = create_run_state(workflow, {})
|
||||
|
||||
assert run.current_frame_id == "root"
|
||||
assert run.current_node_id == "first"
|
||||
assert run.ready_frame_ids == ["root"]
|
||||
assert run.frames["root"].status == FrameStatus.PENDING
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run failing test if needed**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_scheduler.py::test_create_run_state_queues_root_frame -q
|
||||
```
|
||||
|
||||
Expected: fails until root ready queue initialization exists.
|
||||
|
||||
- [ ] **Step 3: Update run creation**
|
||||
|
||||
In `src/wf_core/runtime/ops/runs.py`, create the root frame through `add_frame(..., ready=True)` and set compatibility cursor fields.
|
||||
|
||||
- [ ] **Step 4: Verify**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_scheduler.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Re-Enqueue Normal Frame Advances
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/flow.py`
|
||||
- Modify: `src/wf_core/runtime/step.py`
|
||||
- Test: `tests/core/test_scheduler.py`
|
||||
|
||||
- [ ] **Step 1: Add normal advance test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
from wf_core.runtime.ops.flow import advance_frame
|
||||
|
||||
|
||||
def test_advance_frame_requeues_non_terminal_frame() -> None:
|
||||
run = _run()
|
||||
add_frame(run, ExecutionFrame(id="root", kind="root", node_id="a"))
|
||||
run.current_frame_id = "root"
|
||||
run.sync_from_current_frame()
|
||||
frame = run.current_frame()
|
||||
frame.status = FrameStatus.RUNNING
|
||||
|
||||
advance_frame(run, frame, outcome="ok", next_node_id="b")
|
||||
|
||||
assert frame.status == FrameStatus.PENDING
|
||||
assert run.ready_frame_ids == ["root"]
|
||||
assert run.current_node_id == "b"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run failing test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_scheduler.py::test_advance_frame_requeues_non_terminal_frame -q
|
||||
```
|
||||
|
||||
Expected: fails because `advance_frame` currently leaves non-terminal status as running and does not enqueue.
|
||||
|
||||
- [ ] **Step 3: Update `advance_frame`**
|
||||
|
||||
In `src/wf_core/runtime/ops/flow.py`, after setting node state:
|
||||
|
||||
```python
|
||||
from wf_core.runtime.scheduler import mark_frame_pending
|
||||
```
|
||||
|
||||
Use:
|
||||
|
||||
```python
|
||||
if next_node_id == END:
|
||||
frame.status = FrameStatus.COMPLETED
|
||||
frame.finished_at_node_id = END
|
||||
else:
|
||||
frame.finished_at_node_id = None
|
||||
mark_frame_pending(run, frame.id)
|
||||
run.sync_from_current_frame()
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Verify focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_scheduler.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Migrate Engine Loops To Scheduler Selection
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/engine.py`
|
||||
- Modify: `src/wf_core/runtime/preparation.py`
|
||||
- Modify: `src/wf_core/runtime/step.py`
|
||||
- Test: existing workflow tests
|
||||
|
||||
- [ ] **Step 1: Add regression tests if existing coverage is insufficient**
|
||||
|
||||
Before adding new tests, run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/authoring/test_demo_workflow.py tests/core/test_scheduler.py -q
|
||||
```
|
||||
|
||||
Expected before implementation: likely failures after Task 4 until engine selection is updated.
|
||||
|
||||
- [ ] **Step 2: Update engine loops**
|
||||
|
||||
Change `resume_workflow` and `resume_workflow_async` to:
|
||||
|
||||
```python
|
||||
while True:
|
||||
frame = select_next_frame(run)
|
||||
if frame is None:
|
||||
status = resolve_no_ready_frames(run)
|
||||
if status == RunStatus.COMPLETED:
|
||||
break
|
||||
return run
|
||||
step_workflow(...)
|
||||
if run.status == RunStatus.INTERRUPTED:
|
||||
return run
|
||||
```
|
||||
|
||||
Use the same selection flow in the async loop before `await step_workflow_async(...)`.
|
||||
|
||||
- [ ] **Step 3: Remove stack-style frame collapse from step preparation**
|
||||
|
||||
In `prepare_step`, remove old stack-style frame collapse and keep it focused on resolving the selected frame’s node. It should still return `None` for interrupted/end states.
|
||||
|
||||
- [ ] **Step 4: Verify serial workflows**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/authoring/test_demo_workflow.py tests/core/test_scheduler.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Make Serial Foreach Use Block/Wake
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/foreach.py`
|
||||
- Modify: `src/wf_core/runtime/ops/frames.py`
|
||||
- Create or extend: `tests/core/test_scheduler.py`
|
||||
- Test: `tests/authoring/test_demo_workflow.py`
|
||||
|
||||
- [ ] **Step 1: Add foreach block/wake regression test**
|
||||
|
||||
Add a focused test using an existing demo workflow or a small builder workflow that asserts:
|
||||
|
||||
```python
|
||||
foreach_entries = [entry for entry in run.trace if entry.step_type == "foreach"]
|
||||
assert foreach_entries
|
||||
assert all(frame.status != FrameStatus.BLOCKED for frame in run.frames.values())
|
||||
assert any(frame.kind == "foreach_iteration" for frame in run.frames.values())
|
||||
```
|
||||
|
||||
Then add a lower-level test if needed for child completion waking parent:
|
||||
|
||||
```python
|
||||
def test_child_completion_wakes_blocked_parent() -> None:
|
||||
run = _run()
|
||||
add_frame(run, ExecutionFrame(id="parent", kind="root", node_id="foreach"))
|
||||
add_frame(
|
||||
run,
|
||||
ExecutionFrame(
|
||||
id="child",
|
||||
kind="foreach_iteration",
|
||||
node_id="__end__",
|
||||
parent_frame_id="parent",
|
||||
),
|
||||
)
|
||||
block_frame_on_children(run, "parent", ("child",))
|
||||
run.frames["child"].status = FrameStatus.COMPLETED
|
||||
|
||||
wake_parent_if_children_complete(run, "child")
|
||||
|
||||
assert run.frames["parent"].status == FrameStatus.PENDING
|
||||
assert run.ready_frame_ids == ["parent"]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement typed foreach metadata**
|
||||
|
||||
In `src/wf_core/runtime/scheduler.py` or a small sibling module if the file grows too large:
|
||||
|
||||
```python
|
||||
@dataclass(slots=True, frozen=True)
|
||||
class ForeachIterationMetadata:
|
||||
foreach_node_id: str
|
||||
loop_index: int
|
||||
loop_item: object
|
||||
loop_alias: str
|
||||
|
||||
@classmethod
|
||||
def from_frame(cls, frame: ExecutionFrame) -> "ForeachIterationMetadata | None":
|
||||
if frame.kind != "foreach_iteration":
|
||||
return None
|
||||
...
|
||||
|
||||
def to_metadata(self) -> dict[str, object]:
|
||||
...
|
||||
```
|
||||
|
||||
Wrong frame kind returns `None`; malformed foreach iteration metadata raises `WorkflowExecutionError`.
|
||||
|
||||
- [ ] **Step 3: Implement parent wake helper**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def wake_parent_if_children_complete(run: RunState, child_frame_id: str) -> None:
|
||||
child = _frame(run, child_frame_id)
|
||||
parent_id = child.parent_frame_id
|
||||
if parent_id is None:
|
||||
return
|
||||
parent = _frame(run, parent_id)
|
||||
block = BlockedOnChildren.from_frame(parent)
|
||||
if block is None:
|
||||
return
|
||||
if all(run.frames[item].status == FrameStatus.COMPLETED for item in block.child_frame_ids):
|
||||
wake_frame(run, parent_id)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update foreach child creation**
|
||||
|
||||
In `step_foreach`, replace direct `run.frames[child_id] = ...` with `add_frame(..., ready=True)`, then call `block_frame_on_children(run, frame.id, (child_id,))`. Parent should not stay in ready queue while child runs.
|
||||
|
||||
- [ ] **Step 5: Verify foreach behavior**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/authoring/test_demo_workflow.py tests/core/test_scheduler.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Resume Interrupt Through Ready Queue
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/preparation.py`
|
||||
- Modify: `src/wf_core/runtime/ops/interrupts.py`
|
||||
- Test: existing interrupt tests or new focused tests
|
||||
|
||||
- [ ] **Step 1: Find existing interrupt tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "interrupt|resume_payload|resume_outcome" tests src -g '*.py'
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add/adjust test for resume priority**
|
||||
|
||||
Add a focused test that creates an interrupted frame plus another ready frame and verifies resume places interrupted frame first:
|
||||
|
||||
```python
|
||||
def test_resume_wakes_interrupted_frame_at_front() -> None:
|
||||
run = _run()
|
||||
add_frame(run, ExecutionFrame(id="waiting", kind="root", node_id="ask"))
|
||||
add_frame(run, ExecutionFrame(id="sibling", kind="root", node_id="work"))
|
||||
run.frames["waiting"].status = FrameStatus.INTERRUPTED
|
||||
run.frames["sibling"].status = FrameStatus.PENDING
|
||||
run.ready_frame_ids = ["sibling"]
|
||||
|
||||
wake_frame(run, "waiting", front=True)
|
||||
|
||||
assert run.ready_frame_ids == ["waiting", "sibling"]
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update resume code**
|
||||
|
||||
After `resume_interrupt(...)`, wake/enqueue the resumed frame at the front. Keep one outstanding `run.interrupt`.
|
||||
|
||||
- [ ] **Step 4: Verify interrupt tests**
|
||||
|
||||
Run focused interrupt tests found in Step 1 plus:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_scheduler.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Full Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- Potentially update docs if implementation differs from ADR.
|
||||
|
||||
- [ ] **Step 1: Run focused workflow tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_scheduler.py tests/authoring/test_demo_workflow.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run full test suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: pass, except known environment-only skips.
|
||||
|
||||
- [ ] **Step 3: Run lint/type checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uvx ruff check src tests
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: ruff passes and basedpyright reports 0 errors.
|
||||
|
||||
- [ ] **Step 4: Review docs**
|
||||
|
||||
Check:
|
||||
|
||||
```bash
|
||||
git diff -- CONTEXT.md docs/adr/0001-scheduler-foundation-before-concurrent-foreach.md docs/current_roadmap.md
|
||||
```
|
||||
|
||||
Expected: docs remain aligned with implemented first-pass behavior.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: ADR decisions are covered by tasks for `BLOCKED`, ready queue, scheduler helpers, sync/async engine migration, serial foreach block/wake, interrupt resume priority, and verification.
|
||||
- Intentional gaps: no `foreach(mode="concurrent")`, no `ForeachConcurrentPolicy`, no lineage patches, no BarrierNode, and no public scheduler exports.
|
||||
- Type consistency: helper names are stable across tasks: `add_frame`, `enqueue_frame`, `select_next_frame`, `mark_frame_pending`, `block_frame_on_children`, `wake_frame`, `wake_parent_if_children_complete`, and `resolve_no_ready_frames`.
|
||||
@@ -0,0 +1,748 @@
|
||||
# Lineage State Runtime Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add runtime scopes and lineages before native subgraphs so sibling branches/items can own isolated replayable state writes and later merge through barriers.
|
||||
|
||||
**Architecture:** Implements [`../specs/2026-05-24-lineage-state-runtime-design.md`](../specs/2026-05-24-lineage-state-runtime-design.md). Keep `RunState.state` as the committed root-scope compatibility state, add `RuntimeScope`, `LineageState`, and `StateWrite`, and migrate concurrent foreach from foreach-specific overlays to lineage-backed state views. A frame says where execution is; a scope says which workflow state root execution belongs to; a lineage says which pending writes that execution can see.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, existing `StatePatch`, `StatePath`, `ReducerRef`, `ForeachBarrierState`, pytest, basedpyright, ruff.
|
||||
|
||||
---
|
||||
|
||||
## Current Implementation Status
|
||||
|
||||
This plan is being implemented incrementally. The full `RuntimeScope` /
|
||||
`LineageState` storage model below is still future work, but the runtime now has
|
||||
the compatibility subset needed before native subgraphs:
|
||||
|
||||
- `StateWrite` exists and records `incoming_value` for replay plus
|
||||
`visible_value` for same-lineage reads.
|
||||
- `StatePatch` stores ordered `writes` while preserving `changes` as the
|
||||
trace/compatibility view.
|
||||
- `LineageStateView` materializes committed state plus visible lineage writes.
|
||||
- Concurrent foreach item overlays read `StateWrite.visible_value`.
|
||||
- Foreach pending result metadata persists write records and `lineage_id`.
|
||||
- `ExecutionFrame` and `RuntimeContext` carry `scope_id`, `lineage_id`, and
|
||||
`parent_lineage_id`.
|
||||
- Concurrent foreach child frames receive deterministic, opaque lineage ids,
|
||||
including nested foreach frames.
|
||||
- `RunState` has root scope/lineage storage, scope-aware state views, and
|
||||
generic non-root node writes buffer into `RunState.lineages`.
|
||||
- New concurrent foreach item writes are stored in `RunState.lineages`.
|
||||
`ForeachBarrierState` now keeps scheduling/result metadata plus compatibility
|
||||
patches for old serialized barrier data.
|
||||
|
||||
Direct commits now go through a scope-root commit decision: top-level frames
|
||||
commit to root state, and prepared native-child root frames commit to their
|
||||
child scope state. Descendant item/branch lineages still buffer writes until a
|
||||
barrier or future gather commits them.
|
||||
|
||||
Remaining work should avoid jumping straight into a broad rewrite. Native
|
||||
subgraph scaffolding and non-interrupting prepared-child execution are now
|
||||
present (`SubgraphNode`, structural `WorkflowRef`, terminal workflow outcomes,
|
||||
authoring helpers, and `PreparedSubgraph`). Child graphs execute through their
|
||||
own scope/lineage and map output back at completion. Interrupt bubbling and
|
||||
saved/deployed workflow resolution remain later work.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify: `src/wf_core/run_state.py`
|
||||
- Add `RuntimeScope`, `StateWrite`, `LineageState`.
|
||||
- Add `ExecutionFrame.scope_id` and `ExecutionFrame.lineage_id`.
|
||||
- Add `RunState.scopes` and `RunState.lineages`.
|
||||
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||
- Change `StatePatch` from only path-value maps to ordered `StateWrite` records while preserving `changes` as a compatibility/trace view.
|
||||
- Create: `src/wf_core/runtime/lineage.py`
|
||||
- Own scope/lineage lookup, state view materialization, append writes, and conversion of completed lineage writes into barrier patches.
|
||||
- Modify: `src/wf_core/runtime/ops/runs.py`
|
||||
- Initialize root scope and root lineage.
|
||||
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||
- Resolve node input from frame scope/lineage view and buffer non-root writes into lineage records.
|
||||
- Modify: `src/wf_core/runtime/ops/foreach.py`
|
||||
- Create concurrent item lineages and commit completed lineage writes through the barrier.
|
||||
- Modify: `src/wf_core/runtime/foreach_state.py`
|
||||
- Store completed lineage ids in pending item results; keep old patch metadata parse-compatible.
|
||||
- Modify: `src/wf_core/runtime/ops/overlays.py`
|
||||
- Reduce to a compatibility facade over lineage state views.
|
||||
- Test: `tests/core/test_lineage_state.py`
|
||||
- Unit tests for root scope/lineage, state views, write records, and non-root write buffering.
|
||||
- Test: `tests/core/test_atomic_state_patches.py`
|
||||
- Tests for ordered `StateWrite` records and compatibility `changes`.
|
||||
- Test: `tests/core/test_concurrent_foreach.py`
|
||||
- Regression tests for sibling isolation, same-item visibility, and deterministic barrier commits.
|
||||
- Docs: `docs/wf_core_architecture.md`, `docs/current_roadmap.md`, `docs/superpowers/specs/2026-05-24-native-subgraphs-design.md`
|
||||
- Document scope/lineage as the prerequisite for native subgraphs.
|
||||
|
||||
---
|
||||
|
||||
## Core Shape
|
||||
|
||||
Target runtime state types:
|
||||
|
||||
```python
|
||||
@dataclass(slots=True)
|
||||
class StateWrite:
|
||||
"""One reducer-aware write record owned by a lineage or patch."""
|
||||
|
||||
path: StatePath
|
||||
incoming_value: Any
|
||||
visible_value: Any
|
||||
reducer: ReducerRef
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class RuntimeScope:
|
||||
"""Committed state root for one workflow activation."""
|
||||
|
||||
id: str
|
||||
workflow_name: str
|
||||
committed_state: dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class LineageState:
|
||||
"""Pending ordered writes visible to frames in one lineage."""
|
||||
|
||||
id: str
|
||||
scope_id: str
|
||||
parent_id: str | None = None
|
||||
writes: list[StateWrite] = field(default_factory=list)
|
||||
```
|
||||
|
||||
Target patch shape:
|
||||
|
||||
```python
|
||||
@dataclass(slots=True)
|
||||
class StatePatch:
|
||||
"""Validated state writes produced by one step before commit."""
|
||||
|
||||
writes: list[StateWrite] = dataclass_field(default_factory=list)
|
||||
_staged_state: dict[str, Any] = dataclass_field(default_factory=dict, repr=False)
|
||||
|
||||
@property
|
||||
def changes(self) -> dict[str, Any]:
|
||||
return {str(write.path): write.incoming_value for write in self.writes}
|
||||
|
||||
@property
|
||||
def visible_values(self) -> dict[str, Any]:
|
||||
return {str(write.path): write.visible_value for write in self.writes}
|
||||
```
|
||||
|
||||
Compatibility requirement: existing tests and callers that read `patch.changes`
|
||||
should continue to work. New lineage code must use ordered `writes`, not
|
||||
flattened final values.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add Ordered StateWrite Records to StatePatch
|
||||
|
||||
Status: implemented as the compatibility shape. `StatePatch.changes` remains a
|
||||
stored compatibility dict rather than a derived-only property for now.
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||
- Test: `tests/core/test_atomic_state_patches.py`
|
||||
|
||||
- [ ] **Step 1: Add failing tests**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_output_patch_records_incoming_and_visible_values() -> None:
|
||||
workflow = _workflow_with_state_field(
|
||||
path="state.count",
|
||||
schema={"type": "integer"},
|
||||
reducer="wf.std.add",
|
||||
)
|
||||
state = {"count": 2}
|
||||
|
||||
patch = build_output_patch(
|
||||
workflow,
|
||||
[OutputBinding.model_validate({"source": "delta", "target": "state.count"})],
|
||||
{"delta": 3},
|
||||
state,
|
||||
)
|
||||
|
||||
assert patch.changes["state.count"] == 3
|
||||
assert patch.visible_values["state.count"] == 5
|
||||
assert patch.writes[0].incoming_value == 3
|
||||
assert patch.writes[0].visible_value == 5
|
||||
|
||||
|
||||
def test_barrier_replays_incoming_values_not_lineage_visible_values() -> None:
|
||||
workflow = _workflow_with_state_field(
|
||||
path="state.number",
|
||||
schema={"type": "integer"},
|
||||
reducer="wf.std.add",
|
||||
)
|
||||
patch = build_barrier_patch(
|
||||
workflow,
|
||||
[
|
||||
StatePatch(
|
||||
writes=[
|
||||
StateWrite(
|
||||
path=StatePath(("number",)),
|
||||
incoming_value=3,
|
||||
visible_value=5,
|
||||
reducer=ReducerRef(name="wf.std.add"),
|
||||
)
|
||||
]
|
||||
),
|
||||
StatePatch(
|
||||
writes=[
|
||||
StateWrite(
|
||||
path=StatePath(("number",)),
|
||||
incoming_value=1,
|
||||
visible_value=3,
|
||||
reducer=ReducerRef(name="wf.std.add"),
|
||||
)
|
||||
]
|
||||
),
|
||||
],
|
||||
{"number": 2},
|
||||
)
|
||||
|
||||
assert patch.changes["state.number"] == 6
|
||||
assert patch.visible_values["state.number"] == 6
|
||||
```
|
||||
|
||||
Add imports:
|
||||
|
||||
```python
|
||||
from wf_core.models.reducers import ReducerRef
|
||||
from wf_core.paths import StatePath
|
||||
from wf_core.run_state import StateWrite
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run expected failing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_atomic_state_patches.py::test_output_patch_records_incoming_and_visible_values tests/core/test_atomic_state_patches.py::test_barrier_replays_incoming_values_not_lineage_visible_values -q
|
||||
```
|
||||
|
||||
Expected: failure because `StateWrite`, `StatePatch.writes`, and `visible_values`
|
||||
do not exist.
|
||||
|
||||
- [ ] **Step 3: Implement `StateWrite` and patch views**
|
||||
|
||||
In `src/wf_core/run_state.py`, add `StateWrite` near runtime dataclasses:
|
||||
|
||||
```python
|
||||
@dataclass(slots=True)
|
||||
class StateWrite:
|
||||
path: StatePath
|
||||
incoming_value: Any
|
||||
visible_value: Any
|
||||
reducer: ReducerRef
|
||||
```
|
||||
|
||||
Import `ReducerRef` and `StatePath`.
|
||||
|
||||
In `src/wf_core/runtime/ops/state.py`, change `StatePatch` to store ordered
|
||||
writes and expose `changes` / `visible_values` properties. Keep `_staged_state`.
|
||||
|
||||
- [ ] **Step 4: Build write records in `build_output_patch`**
|
||||
|
||||
When `prepare_state_value(...)` returns a merged value, also capture the reducer
|
||||
used for the destination. If needed, extract reducer lookup from
|
||||
`prepare_state_value(...)` into a helper:
|
||||
|
||||
```python
|
||||
def reducer_for_state_path(
|
||||
path: StatePath,
|
||||
state_fields: Mapping[StatePath, StateFieldDecl],
|
||||
) -> ReducerRef:
|
||||
field = state_fields.get(path)
|
||||
return field.reducer if field and field.reducer else ReducerRef(name="wf.std.replace")
|
||||
```
|
||||
|
||||
Create:
|
||||
|
||||
```python
|
||||
StateWrite(
|
||||
path=destination_path,
|
||||
incoming_value=value,
|
||||
visible_value=merged_value,
|
||||
reducer=reducer,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Replay incoming values in `build_barrier_patch`**
|
||||
|
||||
Update `build_barrier_patch(...)` to iterate over `item_patch.writes`, not over
|
||||
`item_patch.changes.items()`. Replay `write.incoming_value` against the staged
|
||||
state. The resulting barrier patch should contain one `StateWrite` per final
|
||||
destination with both incoming and visible values set to the final committed
|
||||
aggregate value, because the barrier is the public commit point.
|
||||
|
||||
- [ ] **Step 6: Run atomic patch tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_atomic_state_patches.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Add Runtime Scopes and Root Lineage
|
||||
|
||||
Status: partially implemented. Frames and runtime context carry `scope_id`,
|
||||
`lineage_id`, and `parent_lineage_id`, but `RunState.scopes`,
|
||||
`RunState.lineages`, `RuntimeScope`, and `LineageState` are not implemented yet.
|
||||
This is deliberate; foreach still stores pending writes in barrier metadata.
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/run_state.py`
|
||||
- Modify: `src/wf_core/runtime/ops/runs.py`
|
||||
- Test: `tests/core/test_lineage_state.py`
|
||||
|
||||
- [ ] **Step 1: Add root initialization test**
|
||||
|
||||
Create `tests/core/test_lineage_state.py` with a local minimal workflow helper
|
||||
that builds a tiny core `Workflow`. Then add:
|
||||
|
||||
```python
|
||||
def test_create_run_state_initializes_root_scope_and_lineage() -> None:
|
||||
workflow = minimal_workflow()
|
||||
|
||||
run = create_run_state(workflow, {"value": "seed"})
|
||||
|
||||
assert run.scopes["root"].id == "root"
|
||||
assert run.scopes["root"].workflow_name == workflow.name
|
||||
assert run.scopes["root"].committed_state["value"] == "seed"
|
||||
assert run.lineages["root"].id == "root"
|
||||
assert run.lineages["root"].scope_id == "root"
|
||||
assert run.lineages["root"].parent_id is None
|
||||
assert run.lineages["root"].writes == []
|
||||
assert run.frames["root"].scope_id == "root"
|
||||
assert run.frames["root"].lineage_id == "root"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run expected failing test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_lineage_state.py::test_create_run_state_initializes_root_scope_and_lineage -q
|
||||
```
|
||||
|
||||
Expected: failure because `scopes`, `lineages`, `scope_id`, and `lineage_id`
|
||||
do not exist.
|
||||
|
||||
- [ ] **Step 3: Add dataclasses and fields**
|
||||
|
||||
In `src/wf_core/run_state.py`, add `RuntimeScope` and `LineageState`. Add
|
||||
`scope_id: str = "root"` and `lineage_id: str = "root"` to `ExecutionFrame`.
|
||||
Add `scopes` and `lineages` to `RunState`.
|
||||
|
||||
- [ ] **Step 4: Initialize root scope and lineage**
|
||||
|
||||
In `src/wf_core/runtime/ops/runs.py`, initialize:
|
||||
|
||||
```python
|
||||
run = RunState(
|
||||
workflow_name=workflow.name,
|
||||
status=RunStatus.PENDING,
|
||||
workflow_input=dict(workflow_input),
|
||||
state=state,
|
||||
scopes={
|
||||
"root": RuntimeScope(
|
||||
id="root",
|
||||
workflow_name=workflow.name,
|
||||
committed_state=state,
|
||||
)
|
||||
},
|
||||
lineages={"root": LineageState(id="root", scope_id="root")},
|
||||
current_frame_id="root",
|
||||
current_node_id=workflow.start,
|
||||
)
|
||||
```
|
||||
|
||||
The root scope may share the same dict object as `RunState.state` during this
|
||||
migration.
|
||||
|
||||
- [ ] **Step 5: Run focused test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_lineage_state.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Add Lineage Runtime Helpers
|
||||
|
||||
Status: partially implemented. `LineageStateView` and
|
||||
`lineage_writes_for_frame(run, frame)` exist in `src/wf_core/runtime/lineage.py`,
|
||||
backed by current foreach metadata.
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_core/runtime/lineage.py`
|
||||
- Test: `tests/core/test_lineage_state.py`
|
||||
|
||||
- [ ] **Step 1: Add helper tests**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_lineage_state_view_applies_visible_values_only_for_reads() -> None:
|
||||
workflow = minimal_workflow()
|
||||
run = create_run_state(workflow, {"number": 2})
|
||||
add_lineage(run, scope_id="root", lineage_id="branch", parent_id="root")
|
||||
append_lineage_writes(
|
||||
run,
|
||||
scope_id="root",
|
||||
lineage_id="branch",
|
||||
writes=[
|
||||
StateWrite(
|
||||
path=StatePath(("number",)),
|
||||
incoming_value=3,
|
||||
visible_value=5,
|
||||
reducer=ReducerRef(name="wf.std.add"),
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
view = lineage_state_view(run, scope_id="root", lineage_id="branch")
|
||||
|
||||
assert view["number"] == 5
|
||||
assert run.state["number"] == 2
|
||||
|
||||
|
||||
def test_lineage_write_patch_preserves_incoming_values_for_barrier_replay() -> None:
|
||||
workflow = minimal_workflow()
|
||||
run = create_run_state(workflow, {"number": 2})
|
||||
add_lineage(run, scope_id="root", lineage_id="branch", parent_id="root")
|
||||
append_lineage_writes(
|
||||
run,
|
||||
scope_id="root",
|
||||
lineage_id="branch",
|
||||
writes=[
|
||||
StateWrite(
|
||||
path=StatePath(("number",)),
|
||||
incoming_value=3,
|
||||
visible_value=5,
|
||||
reducer=ReducerRef(name="wf.std.add"),
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
patch = lineage_patch(run, scope_id="root", lineage_id="branch")
|
||||
|
||||
assert patch.writes[0].incoming_value == 3
|
||||
assert patch.writes[0].visible_value == 5
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run expected failing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_lineage_state.py -q
|
||||
```
|
||||
|
||||
Expected: import failure for `wf_core.runtime.lineage`.
|
||||
|
||||
- [ ] **Step 3: Implement `runtime.lineage`**
|
||||
|
||||
Create helpers:
|
||||
|
||||
```python
|
||||
def add_lineage(
|
||||
run: RunState, *, scope_id: str, lineage_id: str, parent_id: str
|
||||
) -> None: ...
|
||||
|
||||
def append_lineage_writes(
|
||||
run: RunState,
|
||||
*,
|
||||
scope_id: str,
|
||||
lineage_id: str,
|
||||
writes: Sequence[StateWrite],
|
||||
) -> None: ...
|
||||
|
||||
def lineage_patch(run: RunState, *, scope_id: str, lineage_id: str) -> StatePatch: ...
|
||||
|
||||
def lineage_state_view(
|
||||
run: RunState, *, scope_id: str, lineage_id: str
|
||||
) -> dict[str, Any]: ...
|
||||
```
|
||||
|
||||
`lineage_state_view(...)` should deep-copy `run.scopes[scope_id].committed_state`
|
||||
and apply `write.visible_value` from ancestor/current lineage writes in order.
|
||||
`lineage_patch(...)` should return ordered writes with incoming values intact.
|
||||
|
||||
- [ ] **Step 4: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_lineage_state.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Route Node Reads and Non-Root Writes Through Lineage
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/overlays.py`
|
||||
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||
- Test: `tests/core/test_lineage_state.py`
|
||||
|
||||
- [ ] **Step 1: Add non-root write buffering test**
|
||||
|
||||
Add a test with a one-node workflow that reads `state.value`, writes
|
||||
`state.value`, and runs the frame with `lineage_id="child"`. Assert:
|
||||
|
||||
```python
|
||||
assert result.state_changes == {}
|
||||
assert run.state["value"] == "root"
|
||||
assert run.lineages["child"].writes[0].incoming_value == "root-child"
|
||||
assert lineage_state_view(run, scope_id="root", lineage_id="child")["value"] == "root-child"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run expected failing test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_lineage_state.py::test_non_root_lineage_node_writes_are_buffered_not_committed -q
|
||||
```
|
||||
|
||||
Expected: failure because node execution still commits or cannot read through
|
||||
lineage.
|
||||
|
||||
- [ ] **Step 3: Update overlay facade**
|
||||
|
||||
`state_view_for_frame(run, frame)` should call:
|
||||
|
||||
```python
|
||||
lineage_state_view(run, scope_id=frame.scope_id, lineage_id=frame.lineage_id)
|
||||
```
|
||||
|
||||
For root scope/root lineage it may return `run.state` directly as an optimization.
|
||||
|
||||
- [ ] **Step 4: Update node finalization**
|
||||
|
||||
In `_finalize_node_execution(...)`:
|
||||
|
||||
- if `is_root_lineage_frame(frame)`, commit patch to `run.state`
|
||||
- otherwise append `patch.writes` to the frame lineage and return empty committed
|
||||
`state_changes`
|
||||
|
||||
- [ ] **Step 5: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_lineage_state.py tests/core/test_atomic_state_patches.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Migrate Concurrent Foreach to Lineages
|
||||
|
||||
Status: implemented for new concurrent foreach results. Concurrent foreach
|
||||
child frames have lineage ids, nested item lineages are tested, item writes are
|
||||
stored in `RunState.lineages`, and pending item results persist `lineage_id`.
|
||||
`ForeachBarrierState.patch` remains as a compatibility fallback.
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/foreach.py`
|
||||
- Modify: `src/wf_core/runtime/foreach_state.py`
|
||||
- Test: `tests/core/test_concurrent_foreach.py`
|
||||
- Test: `tests/core/test_foreach_barrier_state.py`
|
||||
|
||||
- [ ] **Step 1: Add item lineage regression**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def test_concurrent_foreach_item_frames_use_distinct_lineages() -> None:
|
||||
workflow = _workflow(mode="concurrent", concurrent={"max_active": 2})
|
||||
run = execute_workflow(workflow, {"items": ["a", "b"]}, {"record": _record_handler})
|
||||
|
||||
item_frames = [frame for frame in run.frames.values() if frame.kind == "foreach_iteration"]
|
||||
|
||||
assert len(item_frames) == 2
|
||||
assert item_frames[0].lineage_id != "root"
|
||||
assert item_frames[1].lineage_id != "root"
|
||||
assert item_frames[0].lineage_id != item_frames[1].lineage_id
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add same-item read regression**
|
||||
|
||||
Add or keep a multi-step concurrent foreach test where item node 1 writes
|
||||
`state.scratch`, item node 2 reads `state.scratch`, and siblings do not see each
|
||||
other's scratch.
|
||||
|
||||
- [ ] **Step 3: Store lineage id on pending item result**
|
||||
|
||||
Add `lineage_id: str | None = None` to `PendingItemResult`, parse it from
|
||||
metadata, and serialize it back. Keep old `patch` parse compatibility.
|
||||
|
||||
- [ ] **Step 4: Create item lineages on admission**
|
||||
|
||||
In `_admit_concurrent_children(...)`, before adding the child frame:
|
||||
|
||||
```python
|
||||
child_lineage_id = child_id
|
||||
add_lineage(
|
||||
run,
|
||||
scope_id=frame.scope_id,
|
||||
lineage_id=child_lineage_id,
|
||||
parent_id=frame.lineage_id,
|
||||
)
|
||||
```
|
||||
|
||||
Pass `scope_id=frame.scope_id` and `lineage_id=child_lineage_id` to the child
|
||||
`ExecutionFrame`.
|
||||
|
||||
- [ ] **Step 5: Record completed lineage ids**
|
||||
|
||||
When a child completes, record `child.lineage_id` on the barrier pending result.
|
||||
Do not copy flattened visible values into the barrier.
|
||||
|
||||
- [ ] **Step 6: Build barrier from lineage patches**
|
||||
|
||||
In `_finish_concurrent_foreach(...)`, construct `success_patches` from
|
||||
`lineage_patch(run, scope_id=frame.scope_id, lineage_id=result.lineage_id)` for
|
||||
new results. Keep existing `result.patch` fallback for old metadata.
|
||||
|
||||
- [ ] **Step 7: Run foreach tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_concurrent_foreach.py tests/core/test_concurrent_foreach_async.py tests/core/test_foreach_barrier_state.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Remove Foreach-Specific Overlay Coupling
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/overlays.py`
|
||||
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||
- Test: `tests/core`
|
||||
|
||||
- [ ] **Step 1: Remove foreach imports from node/overlay state path**
|
||||
|
||||
Ensure `ops/nodes.py` and `ops/overlays.py` do not import
|
||||
`ForeachBarrierState` or `item_frame_owner`.
|
||||
|
||||
- [ ] **Step 2: Run core tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Update Docs
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/wf_core_architecture.md`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
- Modify: `docs/superpowers/specs/2026-05-24-native-subgraphs-design.md`
|
||||
|
||||
- [ ] **Step 1: Document scope/lineage**
|
||||
|
||||
Add a section explaining:
|
||||
|
||||
- scope is workflow state root
|
||||
- frame is scheduler position
|
||||
- lineage is pending write ownership
|
||||
- concurrent foreach uses child lineages
|
||||
- native subgraphs require child scopes
|
||||
|
||||
- [ ] **Step 2: Update native subgraph spec**
|
||||
|
||||
Ensure it says native subgraphs depend on child runtime scopes plus lineages,
|
||||
not lineage alone.
|
||||
|
||||
- [ ] **Step 3: Run doc red-flag scan**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "U[N]RESOLVED|I[N]COMPLETE|F[I]LL_ME|D[E]CIDE_ME" docs/wf_core_architecture.md docs/current_roadmap.md docs/superpowers/specs/2026-05-24-native-subgraphs-design.md
|
||||
```
|
||||
|
||||
Expected: no output.
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Full Verification
|
||||
|
||||
- [ ] **Step 1: Run tests**
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run type check**
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error src tests examples
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run lint**
|
||||
|
||||
```bash
|
||||
uvx ruff check src tests examples
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run format check**
|
||||
|
||||
```bash
|
||||
uvx ruff format --check src tests examples
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
Spec coverage:
|
||||
|
||||
- Scope is represented explicitly and is available for native subgraph state
|
||||
roots.
|
||||
- Lineage stores ordered replayable writes, not full state and not only visible
|
||||
values.
|
||||
- `StatePatch` preserves incoming values for gather/barrier replay.
|
||||
- Same-lineage reads use visible values.
|
||||
- Concurrent foreach is the first migration target.
|
||||
|
||||
Type consistency:
|
||||
|
||||
- `StateWrite.incoming_value` is replay/trace input.
|
||||
- `StateWrite.visible_value` is same-lineage read value.
|
||||
- `RuntimeScope.committed_state` is scope-local committed state.
|
||||
- `ExecutionFrame.scope_id` and `lineage_id` select visibility.
|
||||
@@ -0,0 +1,108 @@
|
||||
# Native Subgraph Interrupt Resume Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Let a prepared native child workflow interrupt its parent run and later resume inside the original child scope.
|
||||
|
||||
**Architecture:** Add a typed internal interrupt route that distinguishes the public parent-subgraph identity from the actual interrupted child frame. Make interrupt request/resume operations scope-aware, then have engine resume select the prepared child workflow and reducers before applying child resume bindings. The parent `SubgraphNode` remains blocked until the child reaches its ordinary terminal outcome.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, Pydantic workflow models, pytest, Ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Child Interrupt Contract
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/run_state.py`
|
||||
- Modify: `tests/core/test_subgraph_step.py`
|
||||
|
||||
- [x] **Step 1: Write failing child interrupt/resume tests**
|
||||
|
||||
Add a child workflow containing an `InterruptNode` followed by a node or terminal step. Execute it through a parent `SubgraphNode` and assert:
|
||||
|
||||
```python
|
||||
assert run.status == RunStatus.INTERRUPTED
|
||||
assert run.interrupt is not None
|
||||
assert run.interrupt.node_id == "child"
|
||||
assert run.interrupt.payload["question"] == "confirm?"
|
||||
assert run.frames["root"].status == FrameStatus.BLOCKED
|
||||
|
||||
resumed = resume_workflow(
|
||||
parent,
|
||||
run,
|
||||
{},
|
||||
resume_payload={"answer": "yes"},
|
||||
subgraphs={"child.workflow": prepared_child},
|
||||
)
|
||||
assert resumed.status == RunStatus.COMPLETED
|
||||
assert resumed.output["answer"] == "yes"
|
||||
```
|
||||
|
||||
- [x] **Step 2: Run tests and observe the existing explicit rejection**
|
||||
|
||||
Run: `uv run pytest -q tests/core/test_subgraph_step.py`
|
||||
|
||||
Expected: FAIL because child `InterruptNode` execution currently raises that child interrupts are unsupported.
|
||||
|
||||
- [x] **Step 3: Add structural route state**
|
||||
|
||||
Add `InterruptRoute` to `run_state.py` containing the interrupted child
|
||||
`frame_id`, `node_id`, `scope_id`, `lineage_id`, and `workflow_ref`. Add an
|
||||
optional `route` field to `InterruptRequest`; root interrupt requests continue
|
||||
to use `route=None`.
|
||||
|
||||
### Task 2: Scope-Aware Interrupt Creation and Resume
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/ops/handlers.py`
|
||||
- Modify: `src/wf_core/runtime/ops/interrupts.py`
|
||||
- Modify: `src/wf_core/runtime/preparation.py`
|
||||
- Modify: `src/wf_core/runtime/step.py`
|
||||
- Modify: `src/wf_core/runtime/engine.py`
|
||||
|
||||
- [x] **Step 1: Permit child interrupts and build their payload from child scope**
|
||||
|
||||
Remove the explicit child rejection. Build child interrupt request bindings
|
||||
from `state_view_for_frame(...)` and `scope_input_for_frame(...)`, not from the
|
||||
root compatibility dictionaries. For a non-root scope, find the owning parent
|
||||
subgraph frame for public identity and attach `InterruptRoute` for resume.
|
||||
|
||||
- [x] **Step 2: Resume through the routed child workflow**
|
||||
|
||||
When an interrupted request has `route`, restore the routed child as the
|
||||
current frame, resolve its `PreparedSubgraph`, build the child workflow index,
|
||||
and apply `resume` output bindings into the child scope using normal
|
||||
scope-aware patch commit logic. Root interrupts retain the existing path.
|
||||
|
||||
- [x] **Step 3: Verify parent completion behavior**
|
||||
|
||||
After child resume, scheduling must continue child execution first. Only after
|
||||
the child finishes may the blocked parent subgraph frame wake and map child
|
||||
output into parent state.
|
||||
|
||||
### Task 3: Verification and Documentation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/wf_core_architecture.md`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
|
||||
- [x] **Step 1: Update documented limitations**
|
||||
|
||||
Document that prepared child interrupts now bubble and resume locally, while
|
||||
artifact/deployment resolution for nested children remains outside core.
|
||||
|
||||
- [x] **Step 2: Run focused verification**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest -q tests/core/test_subgraph_step.py tests/core/test_concurrent_foreach_interrupts.py
|
||||
uvx ruff check src/wf_core tests/core/test_subgraph_step.py tests/core/test_concurrent_foreach_interrupts.py
|
||||
uvx ruff format --check src/wf_core tests/core/test_subgraph_step.py tests/core/test_concurrent_foreach_interrupts.py
|
||||
uv run basedpyright --level error src/wf_core tests/core/test_subgraph_step.py tests/core/test_concurrent_foreach_interrupts.py
|
||||
```
|
||||
|
||||
Expected: all commands exit successfully.
|
||||
@@ -0,0 +1,397 @@
|
||||
# Saved Subgraph Platform Resolution Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Execute non-interrupting saved child workflow artifacts natively from a parent deployment while resolving all descendant dependencies through the parent deployment binding environment.
|
||||
|
||||
**Architecture:** `wf_core` remains unaware of artifact storage: it only looks up caller-prepared `PreparedSubgraph` dependencies by structural workflow-ref display key. A new focused workflow-surface resolver loads exact child artifact versions, traverses descendants with cycle detection, validates descendant capabilities and existing interrupt limitations, and prepares child workflows for `WfMcpService` execution. Future child deployment overrides remain outside this slice and must be keyed by subgraph use site, not artifact identity.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, `wf_core` native subgraphs, `wf_artifacts` stores/deployment diagnostics, `wf_mcp` workflow surface, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_core/runtime/subgraphs.py`: accept a caller-prepared structural saved `WorkflowRef`; do not load artifacts.
|
||||
- Create `src/wf_mcp/workflow_surface/saved_subgraphs.py`: own saved-child traversal, diagnostic production, and preparation of executable child dependencies.
|
||||
- Modify `src/wf_mcp/workflow_surface/handlers.py`: include descendant dependency/interrupt diagnostics in deployment validation.
|
||||
- Modify `src/wf_mcp/broker/service/core.py`: supply prepared saved children to `execute_workflow_async`.
|
||||
- Modify `tests/core/test_subgraph_step.py`: cover core execution of an already-prepared saved ref.
|
||||
- Create `tests/wf_mcp/test_saved_subgraphs.py`: cover deployment-bound saved child execution and unrunnable descendant cases.
|
||||
- Modify `docs/current_roadmap.md` and `docs/workflow_artifacts.md`: record the new runnable saved-child path and remaining persisted-resume limitation.
|
||||
|
||||
### Task 1: Core Accepts Prepared Saved References
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_core/runtime/subgraphs.py`
|
||||
- Test: `tests/core/test_subgraph_step.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing core test**
|
||||
|
||||
Add a test that constructs a parent `SubgraphNode` with:
|
||||
|
||||
```python
|
||||
workflow=WorkflowRef(artifact_id="child", version=1)
|
||||
```
|
||||
|
||||
and supplies:
|
||||
|
||||
```python
|
||||
subgraphs={
|
||||
"workflow.child.v1": PreparedSubgraph(
|
||||
workflow=child,
|
||||
registry={"echo": echo_handler},
|
||||
reducers={},
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
Assert the parent run completes and maps the child output into parent state.
|
||||
|
||||
- [ ] **Step 2: Run the core test to verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/core/test_subgraph_step.py
|
||||
```
|
||||
|
||||
Expected: FAIL because `resolve_prepared_subgraph()` currently rejects a saved structural ref before checking supplied prepared dependencies.
|
||||
|
||||
- [ ] **Step 3: Make prepared dependency lookup structural**
|
||||
|
||||
Update `resolve_prepared_subgraph()`:
|
||||
|
||||
```python
|
||||
def resolve_prepared_subgraph(
|
||||
ref: WorkflowRef,
|
||||
subgraphs: Mapping[str, PreparedSubgraph[HandlerT]] | None,
|
||||
) -> PreparedSubgraph[HandlerT]:
|
||||
"""Resolve a caller-prepared child; artifact loading is not a core concern."""
|
||||
key = ref.name if ref.name is not None else ref.display
|
||||
prepared = None if subgraphs is None else subgraphs.get(key)
|
||||
if prepared is None:
|
||||
raise WorkflowExecutionError(
|
||||
f"no prepared child workflow registered for {ref.display!r}"
|
||||
)
|
||||
return prepared
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the core test to verify it passes**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/core/test_subgraph_step.py
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 2: Traverse and Prepare Saved Child Artifacts
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/workflow_surface/saved_subgraphs.py`
|
||||
- Test: `tests/wf_mcp/test_saved_subgraphs.py`
|
||||
|
||||
- [ ] **Step 1: Write failing traversal tests**
|
||||
|
||||
Add focused tests for a helper that receives a root artifact plan containing a
|
||||
structural `SubgraphNode` ref and a `FileWorkflowArtifactStore`:
|
||||
|
||||
```python
|
||||
resolution = resolve_saved_subgraph_tree(
|
||||
root_artifact=parent,
|
||||
artifact_store=artifact_store,
|
||||
)
|
||||
assert resolution.artifacts_by_ref["workflow.child.v1"].id == "child"
|
||||
assert resolution.diagnostics == []
|
||||
```
|
||||
|
||||
Add tests asserting:
|
||||
|
||||
```python
|
||||
assert resolution.diagnostics[0].code == "workflow_dependency_missing"
|
||||
assert resolution.diagnostics[0].code == "workflow_dependency_cycle"
|
||||
```
|
||||
|
||||
for a missing child and a parent/child cycle respectively.
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
|
||||
```
|
||||
|
||||
Expected: FAIL because `saved_subgraphs.py` and its resolver do not exist.
|
||||
|
||||
- [ ] **Step 3: Implement saved-child tree discovery**
|
||||
|
||||
Create a typed resolution object:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SavedSubgraphTree:
|
||||
"""Saved descendant artifacts keyed by structural workflow-ref display."""
|
||||
|
||||
artifacts_by_ref: dict[str, WorkflowArtifact]
|
||||
diagnostics: list[DependencyDiagnostic]
|
||||
```
|
||||
|
||||
Implement:
|
||||
|
||||
```python
|
||||
def resolve_saved_subgraph_tree(
|
||||
*,
|
||||
root_artifact: WorkflowArtifact,
|
||||
artifact_store: WorkflowArtifactStore,
|
||||
) -> SavedSubgraphTree:
|
||||
"""Load exact saved descendants and report missing refs or cycles."""
|
||||
```
|
||||
|
||||
Parse each artifact plan as `RawWorkflowPlan`, visit only `SubgraphNode`
|
||||
instances whose `workflow.artifact_id` and `.version` are present, load the
|
||||
exact artifact, and recurse. Keep an active artifact stack of
|
||||
`(artifact_id, version)` values so only recursion cycles fail; repeated reuse
|
||||
of the same child in separate branches is allowed.
|
||||
|
||||
Construct direct diagnostics without inventing a fake capability:
|
||||
|
||||
```python
|
||||
DependencyDiagnostic(
|
||||
severity=DiagnosticSeverity.ERROR,
|
||||
code="workflow_dependency_missing",
|
||||
logical_ref=ref.display,
|
||||
message=f"Saved child workflow {ref.display!r} is unavailable.",
|
||||
repair_hint="Save the referenced artifact version or update the parent graph.",
|
||||
)
|
||||
```
|
||||
|
||||
Use analogous text for `workflow_dependency_cycle`.
|
||||
|
||||
- [ ] **Step 4: Run traversal tests to verify they pass**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
|
||||
```
|
||||
|
||||
Expected: traversal tests PASS.
|
||||
|
||||
### Task 3: Validate Descendants Under One Deployment Environment
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/saved_subgraphs.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Test: `tests/wf_mcp/test_saved_subgraphs.py`
|
||||
|
||||
- [ ] **Step 1: Write failing public-validation tests**
|
||||
|
||||
Add tests that save a parent artifact referencing a child artifact whose plan
|
||||
uses logical node `demo.echo_tool`. Save one parent deployment:
|
||||
|
||||
```python
|
||||
WorkflowDeployment(
|
||||
id="parent.personal",
|
||||
artifact_id="parent",
|
||||
artifact_version=1,
|
||||
bindings={"demo": "demo.personal"},
|
||||
)
|
||||
```
|
||||
|
||||
Assert:
|
||||
|
||||
```python
|
||||
result = asyncio.run(handlers.validate_deployment(deployment_id="parent.personal"))
|
||||
assert result["status"] == "runnable"
|
||||
```
|
||||
|
||||
Add descendant failure assertions:
|
||||
|
||||
```python
|
||||
assert result["diagnostics"][0]["code"] == "binding_missing"
|
||||
assert result["diagnostics"][0]["logical_ref"] == "demo.echo_tool"
|
||||
```
|
||||
|
||||
and for an interrupting saved child:
|
||||
|
||||
```python
|
||||
assert result["status"] == "unrunnable"
|
||||
assert result["diagnostics"][0]["code"] == "unsupported_interrupt"
|
||||
```
|
||||
|
||||
The interrupt diagnostic must be reported before execution because
|
||||
`run_deployment` is still one-shot.
|
||||
|
||||
- [ ] **Step 2: Run validation tests to verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
|
||||
```
|
||||
|
||||
Expected: FAIL because `_deployment_validation()` validates only the root artifact.
|
||||
|
||||
- [ ] **Step 3: Add descendant validation composition**
|
||||
|
||||
Add a helper in `saved_subgraphs.py`:
|
||||
|
||||
```python
|
||||
def validate_saved_subgraph_tree(
|
||||
*,
|
||||
tree: SavedSubgraphTree,
|
||||
deployment: WorkflowDeployment,
|
||||
sources: list[AvailableSource],
|
||||
unsupported_interrupt: Callable[[WorkflowArtifact], DependencyDiagnostic | None],
|
||||
) -> list[DependencyDiagnostic]:
|
||||
"""Validate descendants in the root deployment environment."""
|
||||
```
|
||||
|
||||
It should begin with tree discovery diagnostics, then for each loaded child
|
||||
call `validate_deployment_dependencies(...)`, and finally append the existing
|
||||
unsupported-interrupt diagnostic for that child when present.
|
||||
|
||||
Update `WorkflowSurfaceHandlers._deployment_validation()` to discover the
|
||||
saved tree and extend the root diagnostic list with descendant diagnostics.
|
||||
Preserve the root artifact interrupt check in `run_deployment()`; it remains
|
||||
the existing surface behavior.
|
||||
|
||||
- [ ] **Step 4: Run validation tests to verify they pass**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
|
||||
```
|
||||
|
||||
Expected: descendant validation tests PASS.
|
||||
|
||||
### Task 4: Execute Prepared Saved Children
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/saved_subgraphs.py`
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Test: `tests/wf_mcp/test_saved_subgraphs.py`
|
||||
|
||||
- [ ] **Step 1: Write failing end-to-end execution tests**
|
||||
|
||||
Use the parent deployment and child artifact from Task 3. Assert:
|
||||
|
||||
```python
|
||||
payload = asyncio.run(
|
||||
handlers.run_deployment(
|
||||
deployment_id="parent.personal",
|
||||
workflow_input={"text": "hello"},
|
||||
)
|
||||
)
|
||||
assert payload["status"] == "completed"
|
||||
assert payload["output"]["echoed"] == "hello"
|
||||
```
|
||||
|
||||
Add a nested parent -> middle -> child test where only the parent deployment
|
||||
contains `{"demo": "demo.personal"}`, and assert the grandchild node executes
|
||||
through the inherited binding.
|
||||
|
||||
- [ ] **Step 2: Run execution tests to verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
|
||||
```
|
||||
|
||||
Expected: FAIL because the service does not provide prepared saved children to core.
|
||||
|
||||
- [ ] **Step 3: Prepare executable children and supply them to core**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def prepare_saved_subgraphs(
|
||||
*,
|
||||
tree: SavedSubgraphTree,
|
||||
deployment: WorkflowDeployment | None,
|
||||
sources: dict[str, CapabilitySource],
|
||||
compile_plan: Callable[[RawWorkflowPlan, dict[str, str] | None], Workflow],
|
||||
) -> dict[str, PreparedSubgraph[AsyncRegistryHandler]]:
|
||||
"""Compile loaded descendants with the parent deployment bindings."""
|
||||
```
|
||||
|
||||
For each child artifact, parse its plan, resolve its node/reducer runtime
|
||||
dependencies with `resolve_runtime_dependencies(...)`, compile it, and return
|
||||
the dependency under its structural ref display key:
|
||||
|
||||
```python
|
||||
prepared[workflow_ref_display] = PreparedSubgraph(
|
||||
workflow=compile_plan(plan, dependencies.node_name_bindings),
|
||||
registry=dependencies.node_registry,
|
||||
reducers=dependencies.reducers,
|
||||
)
|
||||
```
|
||||
|
||||
Update `WfMcpService.run_workflow_from_plan()` to resolve the saved tree for
|
||||
`runtime_artifact` when an artifact store exists, prepare loaded children, and
|
||||
pass:
|
||||
|
||||
```python
|
||||
subgraphs=prepared_subgraphs
|
||||
```
|
||||
|
||||
to `execute_workflow_async(...)`.
|
||||
|
||||
- [ ] **Step 4: Run saved-child tests to verify they pass**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/wf_mcp/test_saved_subgraphs.py
|
||||
```
|
||||
|
||||
Expected: all saved-subgraph tests PASS.
|
||||
|
||||
### Task 5: Documentation and Full Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
- Modify: `docs/workflow_artifacts.md`
|
||||
|
||||
- [ ] **Step 1: Document current support**
|
||||
|
||||
Record:
|
||||
|
||||
- Non-interrupting saved child artifacts now run natively through deployments.
|
||||
- Descendant logical dependencies inherit the root deployment binding environment.
|
||||
- Missing/cyclic/interrupting saved descendants are reported as unrunnable.
|
||||
- Explicit per-child deployment overrides and persisted saved-interrupt resume remain future work.
|
||||
|
||||
- [ ] **Step 2: Run focused and full verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/core/test_subgraph_step.py tests/wf_mcp/test_saved_subgraphs.py
|
||||
uv run pytest -q
|
||||
uvx ruff check src/wf_core src/wf_mcp tests/core/test_subgraph_step.py tests/wf_mcp/test_saved_subgraphs.py
|
||||
uvx ruff format --check src/wf_core/runtime/subgraphs.py src/wf_mcp/workflow_surface/saved_subgraphs.py src/wf_mcp/workflow_surface/handlers.py src/wf_mcp/broker/service/core.py tests/core/test_subgraph_step.py tests/wf_mcp/test_saved_subgraphs.py
|
||||
uv run basedpyright --level error src/wf_core src/wf_mcp tests/core/test_subgraph_step.py tests/wf_mcp/test_saved_subgraphs.py
|
||||
```
|
||||
|
||||
Expected: all commands pass, with the repository's intentionally skipped live
|
||||
integration test remaining skipped unless its environment is provided.
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: the plan covers exact artifact loading, inherited bindings,
|
||||
cycle/missing diagnostics, preserved interrupt rejection, and native execution.
|
||||
- Boundary check: artifact traversal and dependency preparation stay in
|
||||
`wf_mcp`; `wf_core` accepts only caller-prepared structural refs.
|
||||
- Future compatibility: no child deployment field is added; per-use-site
|
||||
override remains an additive future platform feature.
|
||||
@@ -0,0 +1,52 @@
|
||||
# Stateful Transparent Proxy Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Preserve one upstream MCP session for visible proxy operations made by one connected downstream client, while documenting that generic upstream list/resource notifications are still not relayed.
|
||||
|
||||
**Architecture:** The workflow execution pool already owns background/offline sessions. The transparent proxy should use FastMCP's `StatefulProxyClient`, which is designed for Playwright-like upstreams and scopes reuse to one downstream MCP session. This slice does not unify interactive proxy sessions with deployment runtimes and does not implement arbitrary notification rebroadcast.
|
||||
|
||||
**Tech Stack:** Python 3.14, FastMCP `StatefulProxyClient` and `FastMCPProxy`, pytest fixture MCP server, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Prove Stateful Proxy Behavior
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/fixtures/mcp_echo_server.py`
|
||||
- Modify: `tests/wf_mcp/test_proxy.py`
|
||||
- Modify: `tests/wf_mcp/test_protocol_relay.py`
|
||||
|
||||
- [ ] Add fixture tools that store and read a value in the upstream server process.
|
||||
- [ ] Add a proxy test that writes the value through one proxied request and reads it through another request in the same downstream client session.
|
||||
- [ ] Change protocol-relay coverage to assert that `tools/list_changed`, `resources/list_changed`, `prompts/list_changed`, and `resources/updated` are not yet relayed. Keep string-valued logging forwarding as a strict expected-failure tripwire because the installed FastMCP `StatefulProxyClient` handler currently assumes mapping-valued MCP log data.
|
||||
- [ ] Run the focused tests and confirm they fail before proxy construction changes.
|
||||
|
||||
### Task 2: Use FastMCP Stateful Proxy Sessions
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/proxy/mounts.py`
|
||||
|
||||
- [ ] Replace `create_proxy(Client(...))` with `StatefulProxyClient(...)` and `FastMCPProxy(client_factory=client.new_stateful, ...)`.
|
||||
- [ ] Preserve the existing `ProxyNamespace` and `ResourceLinkNamespace` transforms exactly as mounted-provider output transforms.
|
||||
- [ ] Add a docstring/comment stating that FastMCP owns the interactive session lifecycle and this is intentionally separate from offline workflow execution sessions.
|
||||
|
||||
### Task 3: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- Test: `tests/wf_mcp/test_proxy.py`
|
||||
- Test: `tests/wf_mcp/test_protocol_relay.py`
|
||||
|
||||
- [ ] Run focused proxy/protocol tests.
|
||||
- [ ] Run `uv run pytest -q`.
|
||||
- [ ] Run `uvx ruff check`.
|
||||
- [ ] Run `uv run basedpyright --level error`.
|
||||
|
||||
## Scope Boundary
|
||||
|
||||
- Interactive visible proxy calls share state within one downstream MCP client session.
|
||||
- Deployment execution continues to use the owned runtime pool because scheduled/background runs may exist without a downstream client session.
|
||||
- Generic upstream notification relay remains separate work. FastMCP's stateful path is intended to forward logs/progress/elicitation, but string-valued MCP log data currently exposes an upstream FastMCP handler bug and remains documented with an expected-failure test.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,25 @@
|
||||
# Superseded OpenAPI Generated-Client Plan
|
||||
|
||||
This plan is intentionally retired.
|
||||
|
||||
The generated-client approach made runtime execution depend on parsing
|
||||
`openapi-python-client` generated Python functions to recover parameter-name
|
||||
mappings such as:
|
||||
|
||||
```text
|
||||
OpenAPI public name: petId
|
||||
generated Python kwarg: pet_id
|
||||
```
|
||||
|
||||
That dependency direction is too fragile. Do not continue the generated-client
|
||||
tasks from this file's old history.
|
||||
|
||||
Use the replacement plan instead:
|
||||
|
||||
```text
|
||||
docs/superpowers/plans/2026-05-27-openapi-core-capability-source.md
|
||||
```
|
||||
|
||||
The replacement plan uses the OpenAPI document as source of truth,
|
||||
`openapi-core` for validation/unmarshalling, and a small generic `httpx`
|
||||
request builder for execution.
|
||||
@@ -0,0 +1,638 @@
|
||||
# OpenAPI Core Capability Source Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Let a saved OpenAPI document become a workflow capability source whose operations appear as workflow-facing `NodeSpec`s and execute through spec-driven HTTP requests, without parsing generated Python client code.
|
||||
|
||||
**Architecture:** The OpenAPI document is the source of truth for inventory, JSON Schemas, operation paths, parameters, request bodies, and response validation. `openapi-core` validates/unmarshals OpenAPI requests and responses; a small local `httpx` adapter builds and sends HTTP requests from public OpenAPI-shaped payloads. No generated Python client runtime, no AST parsing, and no case-conversion dependency.
|
||||
|
||||
**Tech Stack:** Python 3.14, `openapi-core`, `httpx`, existing `jsonschema`, `wf_authoring.NodeSpec`, `wf_platform.CapabilitySource`, `wf_mcp` service registration.
|
||||
|
||||
---
|
||||
|
||||
## Why This Replaces The Generated-Client Plan
|
||||
|
||||
The first plan made the ugly part parameter-name recovery:
|
||||
|
||||
```text
|
||||
OpenAPI/public input: path.petId, header.X-Trace-ID
|
||||
generated Python fn: pet_id, x_trace_id
|
||||
```
|
||||
|
||||
Because `openapi-python-client` did not expose a stable operation manifest for those mappings, the implementation parsed generated endpoint functions. That is the wrong dependency direction. We should not inspect generated Python to recover metadata already present in the OpenAPI document.
|
||||
|
||||
The revised runtime shape is:
|
||||
|
||||
```text
|
||||
workflow input
|
||||
-> OpenAPI-shaped request parts
|
||||
-> openapi-core validates/unmarshals request
|
||||
-> local httpx request builder sends request
|
||||
-> openapi-core validates/unmarshals response
|
||||
-> generic workflow outcome
|
||||
```
|
||||
|
||||
This makes validation and execution boring. Outcome mapping remains intentionally generic until saved wrappers add business semantics.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not generate Python clients for v1 runtime execution.
|
||||
- Do not parse generated Python, generated docstrings, or generated function signatures.
|
||||
- Do not invent a full OpenAPI validator. Use `openapi-core` for request/response validation where possible and `jsonschema` for existing schema-boundary checks.
|
||||
- Do not make every HTTP status a business outcome. Raw OpenAPI operations expose generic transport outcomes.
|
||||
- Do not implement custom auth UX in this slice. Leave auth as explicit configuration fields and later integrate with the existing auth/store layer.
|
||||
- Do not expose every OpenAPI operation as a top-level MCP tool. Expose operations as workflow capabilities first.
|
||||
|
||||
## V1 Public Payload Shape
|
||||
|
||||
Workflow inputs stay OpenAPI-shaped:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": {"petId": "pet-1"},
|
||||
"query": {"includeOwner": true},
|
||||
"header": {"X-Trace-ID": "abc"},
|
||||
"cookie": {},
|
||||
"body": {"name": "Ada"}
|
||||
}
|
||||
```
|
||||
|
||||
No `petId -> pet_id` translation exists because there is no generated Python function.
|
||||
|
||||
## V1 Outcome Semantics
|
||||
|
||||
Every raw OpenAPI operation node exposes:
|
||||
|
||||
```text
|
||||
ok
|
||||
http_error
|
||||
unexpected_status
|
||||
validation_error
|
||||
transport_error
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `ok`: response status is a declared 2xx response and response validation passes.
|
||||
- `http_error`: response status is a declared non-2xx response and response validation passes.
|
||||
- `unexpected_status`: response status is not declared and no `default` response covers it.
|
||||
- `validation_error`: request or response does not match the OpenAPI document.
|
||||
- `transport_error`: HTTP client raises before a response exists.
|
||||
|
||||
Output shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"status_code": 200,
|
||||
"headers": {},
|
||||
"body": {},
|
||||
"validation_errors": []
|
||||
}
|
||||
```
|
||||
|
||||
`body` is JSON when the response is JSON, text for text responses, bytes/base64 later if needed. Keep binary response support out of v1 unless a test fixture forces it.
|
||||
|
||||
## Planned File Structure
|
||||
|
||||
- Keep: `src/wf_openapi/__init__.py`
|
||||
- Public exports for the optional OpenAPI capability-source package.
|
||||
- Keep/modify: `src/wf_openapi/models.py`
|
||||
- Operation/source/execution models. Remove generated-client metadata.
|
||||
- Keep/modify: `src/wf_openapi/spec.py`
|
||||
- Load OpenAPI documents, normalize operations, merge inherited path-item parameters with operation-local overrides.
|
||||
- Keep/modify: `src/wf_openapi/schemas.py`
|
||||
- Produce JSON Schema contracts from effective OpenAPI operation inputs/outputs.
|
||||
- Replace: `src/wf_openapi/executor.py`
|
||||
- Generic `httpx` + `openapi-core` operation executor.
|
||||
- Remove or repurpose: `src/wf_openapi/codegen.py`
|
||||
- Delete generated-client runtime helpers. If kept temporarily, it must not be used by runtime/source tests.
|
||||
- Create: `src/wf_openapi/request.py`
|
||||
- Build method, URL, headers, cookies, query params, and JSON body from OpenAPI-shaped payload.
|
||||
- Create: `src/wf_openapi/validation.py`
|
||||
- Thin adapter between local request/response objects and `openapi-core` protocols.
|
||||
- Keep/modify: `src/wf_openapi/source.py`
|
||||
- Build `CapabilitySource` and `NodeSpec`s using generic execution config.
|
||||
- Tests:
|
||||
- `tests/openapi/test_spec_inventory.py`
|
||||
- `tests/openapi/test_schemas.py` if split becomes useful.
|
||||
- `tests/openapi/test_request_builder.py`
|
||||
- `tests/openapi/test_executor.py`
|
||||
- `tests/openapi/test_source.py`
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Dependency And Plan Reset
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `pyproject.toml`
|
||||
- Modify: `uv.lock`
|
||||
- Modify: `docs/superpowers/plans/2026-05-27-openapi-capability-source.md`
|
||||
- Test: `tests/openapi/test_codegen_executor.py` may be removed or replaced later.
|
||||
|
||||
- [ ] **Step 1: Replace runtime dependency**
|
||||
|
||||
In `pyproject.toml`, remove `openapi-python-client` unless another committed package already uses it. Add:
|
||||
|
||||
```toml
|
||||
"openapi-core>=0.19",
|
||||
```
|
||||
|
||||
Keep `httpx` if already present transitively or directly; add it directly if `wf_openapi` imports it.
|
||||
|
||||
- [ ] **Step 2: Refresh lockfile**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv lock
|
||||
```
|
||||
|
||||
Expected: lockfile updates successfully.
|
||||
|
||||
- [ ] **Step 3: Mark generated-client plan superseded**
|
||||
|
||||
Keep the superseded note at the top of:
|
||||
|
||||
```text
|
||||
docs/superpowers/plans/2026-05-27-openapi-capability-source.md
|
||||
```
|
||||
|
||||
Expected: future agents do not continue Task 5/6 AST parsing work.
|
||||
|
||||
- [ ] **Step 4: Verify import availability**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run python -c "import openapi_core, httpx; print(openapi_core.__name__, httpx.__name__)"
|
||||
```
|
||||
|
||||
Expected: prints `openapi_core httpx`.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Remove Generated-Client Runtime Coupling
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_openapi/codegen.py`
|
||||
- Modify: `src/wf_openapi/executor.py`
|
||||
- Modify: `tests/openapi/test_codegen_executor.py`
|
||||
|
||||
- [ ] **Step 1: Write failing guard test**
|
||||
|
||||
Add a test that proves runtime no longer imports generated-client metadata:
|
||||
|
||||
```python
|
||||
def test_openapi_runtime_does_not_require_generated_manifest() -> None:
|
||||
from wf_openapi.executor import OpenApiExecutionConfig
|
||||
|
||||
config = OpenApiExecutionConfig(base_url="https://api.example.test")
|
||||
|
||||
assert config.base_url == "https://api.example.test"
|
||||
assert not hasattr(config, "generated_package")
|
||||
assert not hasattr(config, "operation_modules")
|
||||
assert not hasattr(config, "parameter_arguments")
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/openapi/test_codegen_executor.py::test_openapi_runtime_does_not_require_generated_manifest
|
||||
```
|
||||
|
||||
Expected before implementation: FAIL because generated-client fields still exist.
|
||||
|
||||
- [ ] **Step 2: Simplify execution config**
|
||||
|
||||
Replace generated-client config with:
|
||||
|
||||
```python
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class OpenApiExecutionConfig:
|
||||
"""Runtime config for spec-driven OpenAPI HTTP execution."""
|
||||
|
||||
base_url: str
|
||||
timeout_seconds: float = 30.0
|
||||
```
|
||||
|
||||
Do not include generated package/module/parameter mapping fields.
|
||||
|
||||
- [ ] **Step 3: Remove generated manifest helpers from runtime path**
|
||||
|
||||
Delete or quarantine:
|
||||
|
||||
```python
|
||||
GeneratedOperationMetadata
|
||||
load_generated_operation_manifest
|
||||
generate_openapi_client
|
||||
```
|
||||
|
||||
If `codegen.py` remains, its module docstring must say it is experimental/offline tooling and not used by source/executor runtime.
|
||||
|
||||
- [ ] **Step 4: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/openapi
|
||||
uv run ruff check src/wf_openapi tests/openapi
|
||||
uv run basedpyright --level error src/wf_openapi tests/openapi
|
||||
```
|
||||
|
||||
Expected: generated-client tests that no longer match are removed/replaced; remaining tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Generic Request Builder
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_openapi/request.py`
|
||||
- Test: `tests/openapi/test_request_builder.py`
|
||||
|
||||
- [ ] **Step 1: Write request builder tests**
|
||||
|
||||
Create `tests/openapi/test_request_builder.py`:
|
||||
|
||||
```python
|
||||
from wf_openapi.request import build_http_request_parts
|
||||
from wf_openapi.spec import load_openapi_operations
|
||||
|
||||
FIXTURE = "tests/openapi/fixtures/petstore_minimal.openapi.json"
|
||||
|
||||
|
||||
def test_build_http_request_parts_uses_public_openapi_names() -> None:
|
||||
operation = next(op for op in load_openapi_operations(FIXTURE) if op.name == "get_pet")
|
||||
|
||||
parts = build_http_request_parts(
|
||||
operation,
|
||||
base_url="https://api.example.test/v1",
|
||||
payload={
|
||||
"path": {"petId": "pet-1"},
|
||||
"query": {"includeOwner": True},
|
||||
"header": {"X-Trace-ID": "trace-1"},
|
||||
},
|
||||
)
|
||||
|
||||
assert parts.method == "GET"
|
||||
assert parts.url == "https://api.example.test/v1/pets/pet-1"
|
||||
assert parts.params["includeOwner"] is True
|
||||
assert parts.headers["X-Trace-ID"] == "trace-1"
|
||||
```
|
||||
|
||||
Expected before implementation: FAIL because `wf_openapi.request` does not exist.
|
||||
|
||||
- [ ] **Step 2: Implement request parts**
|
||||
|
||||
Create:
|
||||
|
||||
```python
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Mapping
|
||||
from urllib.parse import quote
|
||||
|
||||
from wf_openapi.models import OpenApiOperation
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class HttpRequestParts:
|
||||
"""OpenAPI-shaped request parts ready for httpx."""
|
||||
|
||||
method: str
|
||||
url: str
|
||||
params: dict[str, Any] = field(default_factory=dict)
|
||||
headers: dict[str, str] = field(default_factory=dict)
|
||||
cookies: dict[str, str] = field(default_factory=dict)
|
||||
json: Any | None = None
|
||||
|
||||
|
||||
def build_http_request_parts(
|
||||
operation: OpenApiOperation,
|
||||
*,
|
||||
base_url: str,
|
||||
payload: Mapping[str, Any],
|
||||
) -> HttpRequestParts:
|
||||
"""Build an HTTP request without renaming public OpenAPI fields."""
|
||||
path_values = _mapping(payload, "path")
|
||||
path = operation.path
|
||||
for parameter in operation.effective_parameters:
|
||||
if parameter.get("in") != "path":
|
||||
continue
|
||||
name = parameter["name"]
|
||||
if name not in path_values:
|
||||
raise ValueError(f"missing path parameter {name!r}")
|
||||
path = path.replace("{" + name + "}", quote(str(path_values[name]), safe=""))
|
||||
|
||||
return HttpRequestParts(
|
||||
method=operation.method.upper(),
|
||||
url=base_url.rstrip("/") + path,
|
||||
params=dict(_mapping(payload, "query")),
|
||||
headers={str(k): str(v) for k, v in _mapping(payload, "header").items()},
|
||||
cookies={str(k): str(v) for k, v in _mapping(payload, "cookie").items()},
|
||||
json=payload.get("body"),
|
||||
)
|
||||
|
||||
|
||||
def _mapping(payload: Mapping[str, Any], key: str) -> Mapping[str, Any]:
|
||||
value = payload.get(key, {})
|
||||
if not isinstance(value, Mapping):
|
||||
raise ValueError(f"{key} must be an object")
|
||||
return value
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add edge tests**
|
||||
|
||||
Add tests for:
|
||||
|
||||
```text
|
||||
missing path parameter -> ValueError
|
||||
non-object query/header/cookie/path -> ValueError
|
||||
body passes through as json payload
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Verify**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/openapi/test_request_builder.py
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: openapi-core Validation Adapter
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_openapi/validation.py`
|
||||
- Test: `tests/openapi/test_validation.py`
|
||||
|
||||
- [ ] **Step 1: Write validation tests**
|
||||
|
||||
Create tests that load the fixture and validate a request built from public payload:
|
||||
|
||||
```python
|
||||
from wf_openapi.request import build_http_request_parts
|
||||
from wf_openapi.spec import load_openapi, load_openapi_operations
|
||||
from wf_openapi.validation import validate_openapi_request
|
||||
|
||||
FIXTURE = "tests/openapi/fixtures/petstore_minimal.openapi.json"
|
||||
|
||||
|
||||
def test_validate_openapi_request_accepts_public_payload() -> None:
|
||||
document = load_openapi(FIXTURE)
|
||||
operation = next(op for op in load_openapi_operations(FIXTURE) if op.name == "get_pet")
|
||||
parts = build_http_request_parts(
|
||||
operation,
|
||||
base_url="https://api.example.test",
|
||||
payload={"path": {"petId": "pet-1"}},
|
||||
)
|
||||
|
||||
result = validate_openapi_request(document, parts)
|
||||
|
||||
assert result.valid is True
|
||||
assert result.errors == []
|
||||
```
|
||||
|
||||
Expected before implementation: FAIL because validation adapter does not exist.
|
||||
|
||||
- [ ] **Step 2: Implement minimal protocol objects**
|
||||
|
||||
Implement local request/response protocol adapters required by `openapi-core`. Keep them in `validation.py` and document that they are intentionally thin protocol shims.
|
||||
|
||||
The adapter must carry:
|
||||
|
||||
```text
|
||||
method
|
||||
full_url_pattern or path pattern if required by openapi-core
|
||||
parameters/path/query/header/cookie
|
||||
body
|
||||
mimetype
|
||||
```
|
||||
|
||||
If `openapi-core` requires a different protocol shape, adapt only this file.
|
||||
|
||||
- [ ] **Step 3: Validate response path**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def validate_openapi_response(document, request_parts, response_parts) -> ValidationResult:
|
||||
...
|
||||
```
|
||||
|
||||
Test declared `200` response and undeclared status behavior.
|
||||
|
||||
- [ ] **Step 4: Verify**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/openapi/test_validation.py
|
||||
uv run basedpyright --level error src/wf_openapi/validation.py tests/openapi/test_validation.py
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Generic HTTP Executor
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_openapi/executor.py`
|
||||
- Test: `tests/openapi/test_executor.py`
|
||||
|
||||
- [ ] **Step 1: Write executor tests with mocked transport**
|
||||
|
||||
Use `httpx.MockTransport`:
|
||||
|
||||
```python
|
||||
import httpx
|
||||
|
||||
from wf_openapi.executor import OpenApiExecutionConfig, call_openapi_operation
|
||||
from wf_openapi.spec import load_openapi, load_openapi_operations
|
||||
|
||||
FIXTURE = "tests/openapi/fixtures/petstore_minimal.openapi.json"
|
||||
|
||||
|
||||
async def test_call_openapi_operation_maps_success() -> None:
|
||||
document = load_openapi(FIXTURE)
|
||||
operation = next(op for op in load_openapi_operations(FIXTURE) if op.name == "get_pet")
|
||||
|
||||
async def handler(request: httpx.Request) -> httpx.Response:
|
||||
assert request.url.path == "/pets/pet-1"
|
||||
return httpx.Response(200, json={"id": "pet-1"})
|
||||
|
||||
result = await call_openapi_operation(
|
||||
document,
|
||||
operation,
|
||||
OpenApiExecutionConfig(base_url="https://api.example.test"),
|
||||
{"path": {"petId": "pet-1"}},
|
||||
client=httpx.AsyncClient(transport=httpx.MockTransport(handler)),
|
||||
)
|
||||
|
||||
assert result.outcome == "ok"
|
||||
assert result.value.status_code == 200
|
||||
assert result.value.body["id"] == "pet-1"
|
||||
```
|
||||
|
||||
Expected before implementation: FAIL because executor still uses generated client or wrong signature.
|
||||
|
||||
- [ ] **Step 2: Implement executor**
|
||||
|
||||
`call_openapi_operation(...)` should:
|
||||
|
||||
```text
|
||||
build request parts
|
||||
validate/unmarshal request
|
||||
send with httpx.AsyncClient
|
||||
parse response body by content-type
|
||||
validate/unmarshal response
|
||||
return NodeReturn with generic outcome
|
||||
```
|
||||
|
||||
Transport exceptions become `transport_error`. Validation failures become `validation_error`.
|
||||
|
||||
- [ ] **Step 3: Add outcome tests**
|
||||
|
||||
Add tests for:
|
||||
|
||||
```text
|
||||
declared non-2xx -> http_error
|
||||
undeclared status -> unexpected_status
|
||||
invalid request -> validation_error
|
||||
httpx transport exception -> transport_error
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Verify**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/openapi/test_executor.py
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Source Integration
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_openapi/source.py`
|
||||
- Test: `tests/openapi/test_source.py`
|
||||
|
||||
- [ ] **Step 1: Write source execution test**
|
||||
|
||||
Build a `CapabilitySource`, get `source.capabilities.node_specs["petstore.default.get_pet"]`, call its async handler with public payload, and use `httpx.MockTransport` through runtime/config injection.
|
||||
|
||||
Expected before implementation: FAIL because source still uses generated metadata or does not pass executor dependencies.
|
||||
|
||||
- [ ] **Step 2: Update source builder**
|
||||
|
||||
`build_openapi_capability_source(...)` should accept:
|
||||
|
||||
```python
|
||||
document_path: Path
|
||||
source_id: str
|
||||
base_url: str
|
||||
```
|
||||
|
||||
It should not accept:
|
||||
|
||||
```text
|
||||
generated_package
|
||||
operation_modules
|
||||
parameter_arguments
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Preserve schema contracts**
|
||||
|
||||
Ensure each `NodeSpec` still exposes:
|
||||
|
||||
```text
|
||||
input_schema_contract from operation input schema
|
||||
output_schema_contract from operation output schema
|
||||
outcomes = ("ok", "http_error", "unexpected_status", "validation_error", "transport_error")
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Verify**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/openapi/test_source.py
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Docs And Final Cleanup
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `docs/openapi_capability_source.md`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
- Delete or rewrite: generated-client-only tests/files if no longer used.
|
||||
|
||||
- [ ] **Step 1: Document the boundary**
|
||||
|
||||
Create `docs/openapi_capability_source.md` with:
|
||||
|
||||
```markdown
|
||||
# OpenAPI Capability Sources
|
||||
|
||||
OpenAPI sources expose raw API operations as workflow capabilities.
|
||||
|
||||
The OpenAPI document is the source of truth. Runtime execution uses a generic
|
||||
httpx request builder and openapi-core validation. The runtime does not parse
|
||||
generated Python clients and does not rename public OpenAPI fields.
|
||||
|
||||
Raw OpenAPI nodes expose generic transport outcomes. Saved wrappers should add
|
||||
business-specific outcomes such as `not_found`, `rate_limited`, or
|
||||
`needs_input`.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Note deferred auth/body/binary support**
|
||||
|
||||
Document:
|
||||
|
||||
```text
|
||||
auth integration: future
|
||||
binary/multipart request bodies: future
|
||||
rich outcome mapping: wrappers, not raw operations
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Final verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q tests/openapi
|
||||
uv run ruff check src/wf_openapi tests/openapi
|
||||
uv run ruff format --check src/wf_openapi tests/openapi
|
||||
uv run basedpyright --level error src/wf_openapi tests/openapi
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- The plan no longer requires generated Python client parsing.
|
||||
- Public OpenAPI names remain public workflow names.
|
||||
- Validation is library-backed through `openapi-core`, not hand-rolled.
|
||||
- HTTP execution is locally owned but small and testable with `httpx.MockTransport`.
|
||||
- Outcome mapping stays generic and wrapper-friendly.
|
||||
- Auth, multipart/binary, and business outcomes are deferred explicitly.
|
||||
@@ -0,0 +1,306 @@
|
||||
# Draft Output Binding Docs Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Clarify the two different `output` binding shapes in workflow drafts so MCP/LLM clients stop applying step-level `source`/`target` bindings to top-level workflow output projection.
|
||||
|
||||
**Architecture:** This is docs-only for now. The core model is internally coherent: step-level `output` uses `OutputBinding` (`source` local -> `target` state), while top-level workflow `output` uses input-binding shape (`path` graph -> `target` local output payload). Update the main draft docs, runbook, and schema-facing descriptions to teach this explicitly without changing runtime behavior.
|
||||
|
||||
**Tech Stack:** Markdown docs, Pydantic field descriptions, pytest schema/docs tests, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
Do:
|
||||
|
||||
- Add a clear “Two Outputs, Different Shapes” docs section.
|
||||
- Show exact JSON for both step-level output and top-level workflow output.
|
||||
- Explain the legacy fallback: empty top-level `output` projects same-name top-level state fields.
|
||||
- Update MCP model field descriptions so schema viewers see “uses `path`, not `source`” for top-level output.
|
||||
- Add docs/test assertions that this guidance is exported.
|
||||
|
||||
Do not:
|
||||
|
||||
- Rename fields to `writes` / `returns`.
|
||||
- Change validation behavior.
|
||||
- Remove legacy same-name output fallback.
|
||||
- Add automatic MCP content block extraction.
|
||||
|
||||
## Files
|
||||
|
||||
- Modify: `docs/workflow_drafts.md`
|
||||
- Add the primary explanation and examples.
|
||||
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
|
||||
- Add a short warning in the draft patching section.
|
||||
- Modify: `docs/workflow_capabilities.md`
|
||||
- Mention that `next_actions.patch_examples` may include top-level output projection examples.
|
||||
- Modify: `src/wf_artifacts/drafts/models.py`
|
||||
- Improve `WorkflowDraft.output` field description.
|
||||
- Modify: `src/wf_core/models/workflow.py`
|
||||
- Improve `Workflow.output` field description.
|
||||
- Modify: `tests/wf_mcp/server/test_docs.py`
|
||||
- Assert exported docs include the new guidance.
|
||||
- Modify if needed: `tests/wf_mcp/server/test_config.py`
|
||||
- Assert schema descriptions include “path, not source” if exposed in tool schema.
|
||||
|
||||
## Task 1: Document The Two Output Shapes
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/workflow_drafts.md`
|
||||
|
||||
- [ ] **Step 1: Add docs section**
|
||||
|
||||
After the “Important details” list or before “Explicit Outputs And Error Outcomes”, add:
|
||||
|
||||
```markdown
|
||||
## Two Outputs, Different Shapes
|
||||
|
||||
Drafts have two fields named `output`, but they do different jobs.
|
||||
|
||||
### Step-Level `steps.<id>.output`
|
||||
|
||||
Step output writes a node's local return payload into workflow state. It uses
|
||||
`source` / `target`:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": { "root": "local", "parts": ["text"] },
|
||||
"target": { "root": "state", "parts": ["result_text"] }
|
||||
}
|
||||
```
|
||||
|
||||
Read this as:
|
||||
|
||||
```text
|
||||
node output.text -> state.result_text
|
||||
```
|
||||
|
||||
### Top-Level `output`
|
||||
|
||||
Top-level workflow output projects graph values into the final public workflow
|
||||
output payload. It uses input-binding shape: `path` / `target`, not
|
||||
`source` / `target`.
|
||||
|
||||
```json
|
||||
{
|
||||
"path": { "root": "state", "parts": ["result_text"] },
|
||||
"target": { "root": "local", "parts": ["result_text"] }
|
||||
}
|
||||
```
|
||||
|
||||
Read this as:
|
||||
|
||||
```text
|
||||
state.result_text -> workflow output.result_text
|
||||
```
|
||||
|
||||
If top-level `output` is empty, the runtime keeps the legacy same-name fallback:
|
||||
for every field in `output_schema`, it copies the top-level state field with the
|
||||
same name when present. That fallback is convenient, but explicit output
|
||||
projection is clearer for new workflows.
|
||||
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run grep check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "Two Outputs, Different Shapes|path.*not.*source|state.result_text -> workflow output.result_text" docs/workflow_drafts.md
|
||||
```
|
||||
|
||||
Expected: all terms appear.
|
||||
|
||||
## Task 2: Update Runbook Warning And Example
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
|
||||
|
||||
- [ ] **Step 1: Add warning near draft patching section**
|
||||
|
||||
Near “Patch Or Validate The Workspace”, add:
|
||||
|
||||
```markdown
|
||||
When patching output bindings, keep the two levels separate:
|
||||
|
||||
- Step-level `steps.<id>.output` uses `source` local -> `target` state.
|
||||
- Top-level `output` uses `path` graph -> `target` local output payload.
|
||||
|
||||
For explicit final output projection from state, use:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": { "root": "state", "parts": ["result_text"] },
|
||||
"target": { "root": "local", "parts": ["result_text"] }
|
||||
}
|
||||
```
|
||||
|
||||
Do not use `source` at top level. `source` belongs to step output bindings.
|
||||
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run grep check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "Do not use `source` at top level|steps.<id>.output|result_text" docs/wf_mcp_end_to_end_runbook.md
|
||||
```
|
||||
|
||||
Expected: all terms appear.
|
||||
|
||||
## Task 3: Update Field Descriptions
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_artifacts/drafts/models.py`
|
||||
- Modify: `src/wf_core/models/workflow.py`
|
||||
|
||||
- [ ] **Step 1: Update `WorkflowDraft.output` field description**
|
||||
|
||||
Change:
|
||||
|
||||
```python
|
||||
output: list[InputBinding] = Field(default_factory=list)
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
output: list[InputBinding] = Field(
|
||||
default_factory=list,
|
||||
description=(
|
||||
"Top-level workflow output projection. Uses input-binding shape: "
|
||||
"`path` reads from input/state/context and `target` writes to the "
|
||||
"local public output payload. Do not use step output `source` here."
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `Workflow.output` field description**
|
||||
|
||||
In `src/wf_core/models/workflow.py`, extend the `output` description to include:
|
||||
|
||||
```python
|
||||
"Use `path`, not `source`; `source` belongs to step-level node output bindings."
|
||||
```
|
||||
|
||||
Keep the existing legacy fallback explanation.
|
||||
|
||||
- [ ] **Step 3: Run focused schema/type checks**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py
|
||||
uv run ruff check src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py
|
||||
uv run ruff format --check src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 4: Add Exported Docs Test
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/server/test_docs.py`
|
||||
|
||||
- [ ] **Step 1: Add docs resource assertion**
|
||||
|
||||
In the docs resource test that reads workflow authoring/draft docs, assert:
|
||||
|
||||
```python
|
||||
assert "Two Outputs, Different Shapes" in text
|
||||
assert "Do not use step output `source` here" in text or "Do not use `source` at top level" in text
|
||||
```
|
||||
|
||||
Use the existing variable names in the file. Do not assert whole payload dict equality.
|
||||
|
||||
- [ ] **Step 2: Run focused docs test**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/server/test_docs.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 5: Optional Schema Description Test
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify if needed: `tests/wf_mcp/server/test_config.py`
|
||||
|
||||
- [ ] **Step 1: Inspect whether draft output field description is exposed**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
If this test already inspects `create_draft_workspace` request schemas, add:
|
||||
|
||||
```python
|
||||
output_description = minimal_request["properties"]["output"]["description"]
|
||||
assert "path" in output_description
|
||||
assert "source" in output_description
|
||||
```
|
||||
|
||||
If the schema nests the description differently, skip this test change and rely
|
||||
on `test_docs.py`.
|
||||
|
||||
## Task 6: Final Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched docs and Python files.
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/server/test_docs.py tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run touched-file lint/type checks**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run ruff check src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py tests/wf_mcp/server/test_docs.py tests/wf_mcp/server/test_config.py
|
||||
uv run ruff format --check src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py tests/wf_mcp/server/test_docs.py tests/wf_mcp/server/test_config.py
|
||||
uv run basedpyright --level error src/wf_artifacts/drafts/models.py src/wf_core/models/workflow.py tests/wf_mcp/server/test_docs.py tests/wf_mcp/server/test_config.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Optional full suite**
|
||||
|
||||
Run when time allows:
|
||||
|
||||
```powershell
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected current baseline: full suite passes with the existing skip/xfail count.
|
||||
|
||||
## Notes For Opencode
|
||||
|
||||
- This is docs/description work only.
|
||||
- Do not rename model fields.
|
||||
- Do not remove fallback same-name projection.
|
||||
- Do not auto-extract MCP content blocks.
|
||||
- The exact mental model to teach is:
|
||||
|
||||
```text
|
||||
steps.call.output: local node output -> workflow state
|
||||
workflow output: input/state/context graph path -> public output payload
|
||||
```
|
||||
@@ -0,0 +1,518 @@
|
||||
# Next Actions Model Refactor Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Move wrapper-draft `next_actions` from handler-local dict helpers into reusable typed workflow-surface models and constructors.
|
||||
|
||||
**Architecture:** Create `src/wf_mcp/workflow_surface/next_actions.py` as the single home for advisory guidance models. Keep the current `create_draft_workspace_from_capability` JSON output stable while adding generic `NextActions` / `NextActionPatchExample` types and `NextActions.from_wrapper_hints(...)`. Do not add deployment/run guidance in this pass.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
Do:
|
||||
|
||||
- Create `workflow_surface/next_actions.py`.
|
||||
- Move `WrapperDraftPatchExample` and `WrapperDraftNextActions` into generic models.
|
||||
- Replace handler-local `_wrapper_draft_next_actions` and `_wrapper_draft_patch_examples` with `NextActions.from_wrapper_hints(...)`.
|
||||
- Keep existing output fields stable:
|
||||
- `can_save_now`
|
||||
- `recommended_next_tool`
|
||||
- `reason`
|
||||
- `patch_examples`
|
||||
- `warnings`
|
||||
- Add `can_continue` as an additive field.
|
||||
- Preserve all existing tests and add targeted serialization/model tests.
|
||||
|
||||
Do not:
|
||||
|
||||
- Add `next_actions` to `validate_deployment`, `run_deployment`, or `resume_run`.
|
||||
- Enforce `can_save_now`.
|
||||
- Change MCP tool names.
|
||||
- Add automatic semantic mapping or MCP content extraction.
|
||||
|
||||
## Files
|
||||
|
||||
- Create: `src/wf_mcp/workflow_surface/next_actions.py`
|
||||
- Generic guidance models and wrapper-hints constructor.
|
||||
- Modify: `src/wf_mcp/workflow_surface/models.py`
|
||||
- Import/reuse `NextActions` for `CreateDraftWorkspaceFromCapabilityResult`.
|
||||
- Remove local wrapper-specific next-action models.
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Import `NextActions`.
|
||||
- Replace private dict helpers with `NextActions.from_wrapper_hints(...).model_dump(mode="json")`.
|
||||
- Test: `tests/wf_mcp/workflow_surface/test_next_actions.py`
|
||||
- Direct model/constructor tests.
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_drafts.py`
|
||||
- Add assertions for `can_continue`.
|
||||
- Modify if needed: `tests/wf_mcp/server/test_config.py`
|
||||
- Keep schema assertions passing with generic model names.
|
||||
|
||||
## Task 1: Add Direct NextActions Model Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_mcp/workflow_surface/test_next_actions.py`
|
||||
|
||||
- [ ] **Step 1: Create failing tests**
|
||||
|
||||
Create `tests/wf_mcp/workflow_surface/test_next_actions.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_mcp.workflow_surface.next_actions import NextActions, NextActionTool
|
||||
|
||||
|
||||
def test_next_actions_from_high_confidence_wrapper_hints() -> None:
|
||||
hints = {
|
||||
"confidence": "high",
|
||||
"missing_decisions": [],
|
||||
"notes": ["Hints are scaffolding, not semantic guarantees."],
|
||||
}
|
||||
|
||||
actions = NextActions.from_wrapper_hints(
|
||||
workspace_id="echo_wrapper",
|
||||
revision=3,
|
||||
hints=hints,
|
||||
)
|
||||
dumped = actions.model_dump(mode="json")
|
||||
|
||||
assert dumped["can_continue"] is True
|
||||
assert dumped["can_save_now"] is True
|
||||
assert dumped["recommended_next_tool"] == (
|
||||
NextActionTool.VALIDATE_DRAFT_WORKSPACE.value
|
||||
)
|
||||
assert "high confidence" in dumped["reason"]
|
||||
assert dumped["patch_examples"] == []
|
||||
assert dumped["warnings"] == []
|
||||
|
||||
|
||||
def test_next_actions_from_low_confidence_wrapper_hints() -> None:
|
||||
hints = {
|
||||
"confidence": "low",
|
||||
"missing_decisions": [{"kind": "review_nested_output"}],
|
||||
"notes": ["Raw MCP content blocks are not workflow-shaped."],
|
||||
}
|
||||
|
||||
actions = NextActions.from_wrapper_hints(
|
||||
workspace_id="content_wrapper",
|
||||
revision=5,
|
||||
hints=hints,
|
||||
)
|
||||
dumped = actions.model_dump(mode="json")
|
||||
|
||||
assert dumped["can_continue"] is True
|
||||
assert dumped["can_save_now"] is False
|
||||
assert dumped["recommended_next_tool"] == (
|
||||
NextActionTool.PATCH_DRAFT_WORKSPACE.value
|
||||
)
|
||||
assert "missing wrapper decisions" in dumped["reason"]
|
||||
assert dumped["patch_examples"][0]["request"]["workspace_id"] == "content_wrapper"
|
||||
assert dumped["patch_examples"][0]["request"]["revision"] == 5
|
||||
assert dumped["warnings"] == ["Raw MCP content blocks are not workflow-shaped."]
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
|
||||
```
|
||||
|
||||
Expected: fail because `wf_mcp.workflow_surface.next_actions` does not exist.
|
||||
|
||||
## Task 2: Create `next_actions.py`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/workflow_surface/next_actions.py`
|
||||
|
||||
- [ ] **Step 1: Add generic models and constructor**
|
||||
|
||||
Create `src/wf_mcp/workflow_surface/next_actions.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import StrEnum
|
||||
from typing import Any, Self
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from .wrapper_hints import WrapperAuthoringHints
|
||||
|
||||
|
||||
class NextActionTool(StrEnum):
|
||||
"""Stable MCP workflow tools that guidance may recommend."""
|
||||
|
||||
PATCH_DRAFT_WORKSPACE = "wf.workflow.patch_draft_workspace"
|
||||
VALIDATE_DRAFT_WORKSPACE = "wf.workflow.validate_draft_workspace"
|
||||
VALIDATE_DEPLOYMENT = "wf.workflow.validate_deployment"
|
||||
RUN_DEPLOYMENT = "wf.workflow.run_deployment"
|
||||
RESUME_RUN = "wf.workflow.resume_run"
|
||||
READ_RUN_TRACE = "wf.workflow.read_run_trace"
|
||||
|
||||
|
||||
class NextActionPatchExample(BaseModel):
|
||||
"""Concrete example request for a recommended workflow tool."""
|
||||
|
||||
description: str = Field(description="Human-readable reason for this example.")
|
||||
tool: NextActionTool = Field(description="MCP workflow tool to call.")
|
||||
request: dict[str, Any] = Field(
|
||||
description="JSON request payload to pass to the tool."
|
||||
)
|
||||
|
||||
|
||||
class NextActions(BaseModel):
|
||||
"""Advisory continuation hints for MCP workflow clients."""
|
||||
|
||||
can_continue: bool = Field(
|
||||
description=(
|
||||
"Whether there is an obvious next workflow-surface tool call. "
|
||||
"Advisory only."
|
||||
)
|
||||
)
|
||||
can_save_now: bool | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Advisory wrapper-authoring signal. False means review is "
|
||||
"recommended before saving; the server does not enforce it."
|
||||
),
|
||||
)
|
||||
recommended_next_tool: NextActionTool | None = Field(
|
||||
default=None,
|
||||
description="Suggested next MCP workflow tool, if one is obvious.",
|
||||
)
|
||||
reason: str = Field(description="Short explanation for the recommendation.")
|
||||
patch_examples: list[NextActionPatchExample] = Field(
|
||||
default_factory=list,
|
||||
description="Concrete JSON Patch examples for common missing decisions.",
|
||||
)
|
||||
warnings: list[str] = Field(
|
||||
default_factory=list,
|
||||
description="Non-blocking warnings copied from low-confidence hints.",
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_wrapper_hints(
|
||||
cls,
|
||||
*,
|
||||
workspace_id: str,
|
||||
revision: int,
|
||||
hints: WrapperAuthoringHints | dict[str, Any],
|
||||
) -> Self:
|
||||
"""Create guidance after create_draft_workspace_from_capability."""
|
||||
hint_payload = _hint_payload(hints)
|
||||
confidence = str(hint_payload.get("confidence", "low"))
|
||||
missing_decisions = hint_payload.get("missing_decisions")
|
||||
notes = [
|
||||
str(note) for note in hint_payload.get("notes", []) if isinstance(note, str)
|
||||
]
|
||||
has_missing = (
|
||||
isinstance(missing_decisions, list) and len(missing_decisions) > 0
|
||||
)
|
||||
can_save_now = confidence == "high" and not has_missing
|
||||
if can_save_now:
|
||||
return cls(
|
||||
can_continue=True,
|
||||
can_save_now=True,
|
||||
recommended_next_tool=NextActionTool.VALIDATE_DRAFT_WORKSPACE,
|
||||
reason=(
|
||||
"Wrapper hints are high confidence and have no missing decisions."
|
||||
),
|
||||
patch_examples=[],
|
||||
warnings=[],
|
||||
)
|
||||
|
||||
return cls(
|
||||
can_continue=True,
|
||||
can_save_now=False,
|
||||
recommended_next_tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
|
||||
reason="Review missing wrapper decisions before saving.",
|
||||
patch_examples=_wrapper_draft_patch_examples(
|
||||
workspace_id=workspace_id,
|
||||
revision=revision,
|
||||
hints=hint_payload,
|
||||
),
|
||||
warnings=notes,
|
||||
)
|
||||
```
|
||||
|
||||
Then add below:
|
||||
|
||||
```python
|
||||
def _hint_payload(hints: WrapperAuthoringHints | dict[str, Any]) -> dict[str, Any]:
|
||||
"""Return a JSON-compatible wrapper hint payload."""
|
||||
if isinstance(hints, WrapperAuthoringHints):
|
||||
return hints.model_dump(mode="json")
|
||||
return dict(hints)
|
||||
|
||||
|
||||
def _wrapper_draft_patch_examples(
|
||||
*,
|
||||
workspace_id: str,
|
||||
revision: int,
|
||||
hints: dict[str, Any],
|
||||
) -> list[NextActionPatchExample]:
|
||||
"""Return conservative JSON Patch examples without guessing semantics."""
|
||||
examples: list[NextActionPatchExample] = []
|
||||
missing_decisions = hints.get("missing_decisions")
|
||||
if not isinstance(missing_decisions, list):
|
||||
return examples
|
||||
decision_kinds = {
|
||||
str(decision.get("kind"))
|
||||
for decision in missing_decisions
|
||||
if isinstance(decision, dict)
|
||||
}
|
||||
if {"choose_output_fields", "review_nested_output"} & decision_kinds:
|
||||
examples.append(
|
||||
NextActionPatchExample(
|
||||
description=(
|
||||
"Replace output bindings after choosing which capability "
|
||||
"outputs should be written to workflow state."
|
||||
),
|
||||
tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
|
||||
request={
|
||||
"workspace_id": workspace_id,
|
||||
"revision": revision,
|
||||
"patch": [
|
||||
{
|
||||
"op": "replace",
|
||||
"path": "/draft/steps/call/output",
|
||||
"value": [],
|
||||
}
|
||||
],
|
||||
},
|
||||
)
|
||||
)
|
||||
if "confirm_boolean_outcomes" in decision_kinds:
|
||||
examples.append(
|
||||
NextActionPatchExample(
|
||||
description=(
|
||||
"Review boolean output candidates before adding routing; "
|
||||
"do not route on boolean fields automatically."
|
||||
),
|
||||
tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
|
||||
request={
|
||||
"workspace_id": workspace_id,
|
||||
"revision": revision,
|
||||
"patch": [],
|
||||
},
|
||||
)
|
||||
)
|
||||
return examples
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run direct tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 3: Use Generic Models In MCP Result Schema
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/models.py`
|
||||
- Modify: `tests/wf_mcp/server/test_config.py`
|
||||
|
||||
- [ ] **Step 1: Replace local models**
|
||||
|
||||
In `src/wf_mcp/workflow_surface/models.py`:
|
||||
|
||||
1. Import:
|
||||
|
||||
```python
|
||||
from .next_actions import NextActions
|
||||
```
|
||||
|
||||
2. Delete local classes:
|
||||
|
||||
```python
|
||||
class WrapperDraftPatchExample(...)
|
||||
class WrapperDraftNextActions(...)
|
||||
```
|
||||
|
||||
3. Change:
|
||||
|
||||
```python
|
||||
next_actions: WrapperDraftNextActions = Field(...)
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
next_actions: NextActions = Field(
|
||||
description=(
|
||||
"Advisory next step guidance derived from wrapper_hints. "
|
||||
"The server does not enforce can_save_now."
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update schema test for additive field**
|
||||
|
||||
In `tests/wf_mcp/server/test_config.py`, keep existing assertions and add:
|
||||
|
||||
```python
|
||||
assert "can_continue" in next_actions_schema["properties"]
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
Expected: pass after handler is updated in Task 4. If this fails only because handler runtime does not return `can_continue`, proceed to Task 4.
|
||||
|
||||
## Task 4: Replace Handler Helpers
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_drafts.py`
|
||||
|
||||
- [ ] **Step 1: Add output assertions for additive field**
|
||||
|
||||
In `tests/wf_mcp/workflow_surface/test_drafts.py`, in both next action assertions, add:
|
||||
|
||||
```python
|
||||
assert next_actions["can_continue"] is True
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py::test_workflow_surface_creates_draft_workspace_from_capability_hints tests/wf_mcp/workflow_surface/test_drafts.py::test_workflow_surface_low_confidence_draft_returns_patch_guidance -q
|
||||
```
|
||||
|
||||
Expected: fail because current dict helper does not emit `can_continue`.
|
||||
|
||||
- [ ] **Step 2: Import and use `NextActions`**
|
||||
|
||||
In `src/wf_mcp/workflow_surface/handlers.py`, add:
|
||||
|
||||
```python
|
||||
from .next_actions import NextActions
|
||||
```
|
||||
|
||||
Change the return in `create_draft_workspace_from_capability` to:
|
||||
|
||||
```python
|
||||
"next_actions": NextActions.from_wrapper_hints(
|
||||
workspace_id=workspace_id,
|
||||
revision=int(result["revision"]),
|
||||
hints=hints,
|
||||
).model_dump(mode="json"),
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Delete private helpers**
|
||||
|
||||
Remove from `handlers.py`:
|
||||
|
||||
```python
|
||||
_wrapper_draft_next_actions
|
||||
_wrapper_draft_patch_examples
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "_wrapper_draft_next_actions|_wrapper_draft_patch_examples" src/wf_mcp/workflow_surface/handlers.py
|
||||
```
|
||||
|
||||
Expected: no matches.
|
||||
|
||||
- [ ] **Step 4: Run focused behavior tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 5: Update Docs If Needed
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify if needed: `docs/workflow_capabilities.md`
|
||||
- Modify if needed: `docs/superpowers/specs/2026-05-31-workflow-surface-next-actions-design.md`
|
||||
|
||||
- [ ] **Step 1: Check docs already match implementation**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "can_continue|NextActions|next_actions" docs/workflow_capabilities.md docs/superpowers/specs/2026-05-31-workflow-surface-next-actions-design.md
|
||||
```
|
||||
|
||||
If `workflow_capabilities.md` does not mention `can_continue`, add:
|
||||
|
||||
```markdown
|
||||
`next_actions.can_continue` is advisory. It says whether the response has an
|
||||
obvious workflow-surface tool to call next.
|
||||
```
|
||||
|
||||
## Task 6: Final Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run lint/format checks**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run ruff check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
|
||||
uv run ruff format --check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run touched-file type check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 4: Optional full suite**
|
||||
|
||||
Run when time allows:
|
||||
|
||||
```powershell
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected current baseline: full suite passes with the existing skip/xfail count.
|
||||
|
||||
## Notes For Opencode
|
||||
|
||||
- Keep this as a refactor plus additive `can_continue`.
|
||||
- Do not add deployment/run guidance yet.
|
||||
- Do not enforce `can_save_now`.
|
||||
- Keep JSON output stable for existing fields.
|
||||
- `next_actions` is UX guidance. Diagnostics and validation remain source of truth.
|
||||
@@ -0,0 +1,629 @@
|
||||
# Next Actions Model Refactor Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Move MCP workflow guidance UX into reusable `NextActions` models so wrapper draft guidance stops living as ad-hoc dict helpers in `handlers.py`.
|
||||
|
||||
**Architecture:** Add `src/wf_mcp/workflow_surface/next_actions.py` as the single owner of next-action enums, response models, and the first constructor: `NextActions.from_wrapper_hints(...)`. Keep the existing `create_draft_workspace_from_capability` JSON stable, with `can_continue` as the only additive field. Do not add deployment/run guidance in this pass; this is the foundation for those later constructors.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, FastMCP schema generation, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create `src/wf_mcp/workflow_surface/next_actions.py`
|
||||
- Owns `NextActionTool`, `NextActionPatchExample`, and `NextActions`.
|
||||
- Owns wrapper-hint guidance policy currently in `handlers.py`.
|
||||
- Contains docstrings explaining that `next_actions` is advisory, not authority.
|
||||
|
||||
- Modify `src/wf_mcp/workflow_surface/models.py`
|
||||
- Reuse the generic models for MCP response schemas.
|
||||
- Keep compatibility aliases for `WrapperDraftPatchExample` and `WrapperDraftNextActions` if local imports or generated schemas still reference the old names.
|
||||
|
||||
- Modify `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Remove `_wrapper_draft_next_actions(...)` and `_wrapper_draft_patch_examples(...)`.
|
||||
- Call `NextActions.from_wrapper_hints(...).model_dump(mode="json")`.
|
||||
|
||||
- Create `tests/wf_mcp/workflow_surface/test_next_actions.py`
|
||||
- Unit-test the generic model without the full draft store.
|
||||
|
||||
- Modify `tests/wf_mcp/workflow_surface/test_drafts.py`
|
||||
- Keep integration coverage through `create_draft_workspace_from_capability`.
|
||||
- Assert old fields still exist and `can_continue` is additive.
|
||||
|
||||
- Modify `tests/wf_mcp/server/test_config.py`
|
||||
- Assert the MCP output schema documents `can_continue` and still documents `can_save_now`.
|
||||
|
||||
- Modify `docs/workflow_capabilities.md`
|
||||
- Add one short note that `next_actions` is advisory guidance and not validation authority.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Do not add `NextActions.from_deployment_validation(...)` in this pass.
|
||||
- Do not add `NextActions.from_run_result(...)` in this pass.
|
||||
- Do not change current wrapper patch example semantics.
|
||||
- Do not infer MCP content block extraction or boolean routing.
|
||||
- Do not make `can_save_now` authoritative.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Generic Next-Actions Unit Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_mcp/workflow_surface/test_next_actions.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for high-confidence wrapper hints**
|
||||
|
||||
Create `tests/wf_mcp/workflow_surface/test_next_actions.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_mcp.workflow_surface.next_actions import NextActionTool, NextActions
|
||||
|
||||
|
||||
def test_next_actions_from_high_confidence_wrapper_hints_can_validate() -> None:
|
||||
actions = NextActions.from_wrapper_hints(
|
||||
workspace_id="echo_workspace",
|
||||
revision=3,
|
||||
hints={
|
||||
"confidence": "high",
|
||||
"missing_decisions": [],
|
||||
"notes": [],
|
||||
},
|
||||
)
|
||||
|
||||
dumped = actions.model_dump(mode="json")
|
||||
assert dumped["can_continue"] is True
|
||||
assert dumped["can_save_now"] is True
|
||||
assert dumped["recommended_next_tool"] == (
|
||||
NextActionTool.VALIDATE_DRAFT_WORKSPACE.value
|
||||
)
|
||||
assert "high confidence" in dumped["reason"]
|
||||
assert dumped["patch_examples"] == []
|
||||
assert dumped["warnings"] == []
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write failing tests for low-confidence wrapper hints**
|
||||
|
||||
Append to `tests/wf_mcp/workflow_surface/test_next_actions.py`:
|
||||
|
||||
```python
|
||||
def test_next_actions_from_low_confidence_wrapper_hints_can_patch() -> None:
|
||||
actions = NextActions.from_wrapper_hints(
|
||||
workspace_id="echo_workspace",
|
||||
revision=4,
|
||||
hints={
|
||||
"confidence": "low",
|
||||
"missing_decisions": [
|
||||
{
|
||||
"kind": "review_nested_output",
|
||||
"message": "Review nested output fields before mapping.",
|
||||
},
|
||||
{
|
||||
"kind": "confirm_boolean_outcomes",
|
||||
"message": "Boolean fields may be data, not outcomes.",
|
||||
},
|
||||
],
|
||||
"notes": ["Raw MCP tool output is not workflow-shaped."],
|
||||
},
|
||||
)
|
||||
|
||||
dumped = actions.model_dump(mode="json")
|
||||
assert dumped["can_continue"] is True
|
||||
assert dumped["can_save_now"] is False
|
||||
assert dumped["recommended_next_tool"] == NextActionTool.PATCH_DRAFT_WORKSPACE.value
|
||||
assert "missing wrapper decisions" in dumped["reason"]
|
||||
assert dumped["warnings"][0] == "Raw MCP tool output is not workflow-shaped."
|
||||
assert len(dumped["patch_examples"]) == 2
|
||||
assert dumped["patch_examples"][0]["tool"] == (
|
||||
NextActionTool.PATCH_DRAFT_WORKSPACE.value
|
||||
)
|
||||
assert dumped["patch_examples"][0]["request"]["workspace_id"] == "echo_workspace"
|
||||
assert dumped["patch_examples"][0]["request"]["revision"] == 4
|
||||
assert dumped["patch_examples"][0]["request"]["patch"][0]["path"] == (
|
||||
"/draft/steps/call/output"
|
||||
)
|
||||
assert dumped["patch_examples"][1]["request"]["patch"] == []
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the focused test to verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
|
||||
```
|
||||
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'wf_mcp.workflow_surface.next_actions'`.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Implement `next_actions.py`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/workflow_surface/next_actions.py`
|
||||
|
||||
- [ ] **Step 1: Create the generic models and wrapper-hints constructor**
|
||||
|
||||
Create `src/wf_mcp/workflow_surface/next_actions.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import StrEnum
|
||||
from typing import Any, Self
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from .wrapper_hints import WrapperAuthoringHints
|
||||
|
||||
|
||||
class NextActionTool(StrEnum):
|
||||
"""Stable MCP workflow tools that guidance may recommend."""
|
||||
|
||||
PATCH_DRAFT_WORKSPACE = "wf.workflow.patch_draft_workspace"
|
||||
VALIDATE_DRAFT_WORKSPACE = "wf.workflow.validate_draft_workspace"
|
||||
VALIDATE_DEPLOYMENT = "wf.workflow.validate_deployment"
|
||||
RUN_DEPLOYMENT = "wf.workflow.run_deployment"
|
||||
RESUME_RUN = "wf.workflow.resume_run"
|
||||
READ_RUN_TRACE = "wf.workflow.read_run_trace"
|
||||
|
||||
|
||||
class NextActionPatchExample(BaseModel):
|
||||
"""Concrete example request for a recommended MCP workflow tool."""
|
||||
|
||||
description: str = Field(description="Human-readable reason for this example.")
|
||||
tool: NextActionTool = Field(description="MCP workflow tool to call.")
|
||||
request: dict[str, Any] = Field(
|
||||
description="JSON request payload to pass to the tool."
|
||||
)
|
||||
|
||||
|
||||
class NextActions(BaseModel):
|
||||
"""Advisory continuation hints for MCP workflow clients.
|
||||
|
||||
This object is guidance, not authority. Validation diagnostics and runtime
|
||||
status remain the source of truth; clients should treat this as a compact
|
||||
answer to "what tool should I call next?"
|
||||
"""
|
||||
|
||||
can_continue: bool = Field(
|
||||
description=(
|
||||
"Whether there is an obvious next workflow-surface tool call. "
|
||||
"Advisory only."
|
||||
)
|
||||
)
|
||||
can_save_now: bool | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Advisory wrapper-authoring signal. False means review is "
|
||||
"recommended before saving; the server does not enforce this."
|
||||
),
|
||||
)
|
||||
recommended_next_tool: NextActionTool | None = Field(
|
||||
default=None,
|
||||
description="Suggested next MCP workflow tool, if one is obvious.",
|
||||
)
|
||||
reason: str = Field(description="Short explanation for the recommendation.")
|
||||
patch_examples: list[NextActionPatchExample] = Field(
|
||||
default_factory=list,
|
||||
description="Concrete JSON Patch examples for common missing decisions.",
|
||||
)
|
||||
warnings: list[str] = Field(
|
||||
default_factory=list,
|
||||
description="Non-blocking warnings copied from low-confidence hints.",
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_wrapper_hints(
|
||||
cls,
|
||||
*,
|
||||
workspace_id: str,
|
||||
revision: int,
|
||||
hints: WrapperAuthoringHints | dict[str, Any],
|
||||
) -> Self:
|
||||
"""Create guidance after bootstrapping a wrapper draft workspace."""
|
||||
payload = (
|
||||
hints.model_dump(mode="json") if isinstance(hints, WrapperAuthoringHints)
|
||||
else hints
|
||||
)
|
||||
confidence = str(payload.get("confidence", "low"))
|
||||
missing_decisions = payload.get("missing_decisions")
|
||||
notes = [
|
||||
str(note) for note in payload.get("notes", []) if isinstance(note, str)
|
||||
]
|
||||
has_missing = (
|
||||
isinstance(missing_decisions, list) and len(missing_decisions) > 0
|
||||
)
|
||||
can_save_now = confidence == "high" and not has_missing
|
||||
if can_save_now:
|
||||
return cls(
|
||||
can_continue=True,
|
||||
can_save_now=True,
|
||||
recommended_next_tool=NextActionTool.VALIDATE_DRAFT_WORKSPACE,
|
||||
reason="Wrapper hints are high confidence and have no missing decisions.",
|
||||
patch_examples=[],
|
||||
warnings=[],
|
||||
)
|
||||
|
||||
return cls(
|
||||
can_continue=True,
|
||||
can_save_now=False,
|
||||
recommended_next_tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
|
||||
reason="Review missing wrapper decisions before saving.",
|
||||
patch_examples=_wrapper_draft_patch_examples(
|
||||
workspace_id=workspace_id,
|
||||
revision=revision,
|
||||
hints=payload,
|
||||
),
|
||||
warnings=notes,
|
||||
)
|
||||
|
||||
|
||||
def _wrapper_draft_patch_examples(
|
||||
*,
|
||||
workspace_id: str,
|
||||
revision: int,
|
||||
hints: dict[str, Any],
|
||||
) -> list[NextActionPatchExample]:
|
||||
"""Return conservative JSON Patch examples without guessing semantics."""
|
||||
examples: list[NextActionPatchExample] = []
|
||||
missing_decisions = hints.get("missing_decisions")
|
||||
if not isinstance(missing_decisions, list):
|
||||
return examples
|
||||
decision_kinds = {
|
||||
str(decision.get("kind"))
|
||||
for decision in missing_decisions
|
||||
if isinstance(decision, dict)
|
||||
}
|
||||
if {"choose_output_fields", "review_nested_output"} & decision_kinds:
|
||||
examples.append(
|
||||
NextActionPatchExample(
|
||||
description=(
|
||||
"Replace output bindings after choosing which capability "
|
||||
"outputs should be written to workflow state."
|
||||
),
|
||||
tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
|
||||
request={
|
||||
"workspace_id": workspace_id,
|
||||
"revision": revision,
|
||||
"patch": [
|
||||
{
|
||||
"op": "replace",
|
||||
"path": "/draft/steps/call/output",
|
||||
"value": [],
|
||||
}
|
||||
],
|
||||
},
|
||||
)
|
||||
)
|
||||
if "confirm_boolean_outcomes" in decision_kinds:
|
||||
examples.append(
|
||||
NextActionPatchExample(
|
||||
description=(
|
||||
"Review boolean output candidates before adding routing; "
|
||||
"do not route on boolean fields automatically."
|
||||
),
|
||||
tool=NextActionTool.PATCH_DRAFT_WORKSPACE,
|
||||
request={
|
||||
"workspace_id": workspace_id,
|
||||
"revision": revision,
|
||||
"patch": [],
|
||||
},
|
||||
)
|
||||
)
|
||||
return examples
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the focused unit test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Replace Wrapper-Specific MCP Models with Generic Models
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/models.py`
|
||||
|
||||
- [ ] **Step 1: Import generic next-action models**
|
||||
|
||||
Near the other local imports in `src/wf_mcp/workflow_surface/models.py`, add:
|
||||
|
||||
```python
|
||||
from .next_actions import NextActionPatchExample, NextActions
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Remove wrapper-specific model class bodies**
|
||||
|
||||
Delete the current `WrapperDraftPatchExample` and `WrapperDraftNextActions` class definitions.
|
||||
|
||||
Replace them with compatibility aliases immediately before `CreateDraftWorkspaceFromCapabilityResult`:
|
||||
|
||||
```python
|
||||
# Compatibility aliases for older imports. The JSON fields are now generic
|
||||
# workflow-surface guidance, not wrapper-only policy.
|
||||
WrapperDraftPatchExample = NextActionPatchExample
|
||||
WrapperDraftNextActions = NextActions
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Keep `CreateDraftWorkspaceFromCapabilityResult` typed with the alias**
|
||||
|
||||
Leave the field shape unchanged except that the alias now points at the generic type:
|
||||
|
||||
```python
|
||||
class CreateDraftWorkspaceFromCapabilityResult(DraftWorkspaceResult):
|
||||
"""Draft workspace result plus wrapper hints and advisory next actions."""
|
||||
|
||||
wrapper_hints: dict[str, Any] = Field(
|
||||
description=(
|
||||
"The wrapper_hints payload used before applying request overrides. "
|
||||
"Use this to patch uncertain maps or schemas by revision."
|
||||
)
|
||||
)
|
||||
next_actions: WrapperDraftNextActions = Field(
|
||||
description=(
|
||||
"Advisory next step guidance derived from wrapper_hints. "
|
||||
"The server does not enforce can_save_now."
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run model import check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run python -c "from wf_mcp.workflow_surface.models import WrapperDraftNextActions, CreateDraftWorkspaceFromCapabilityResult; print(WrapperDraftNextActions.__name__)"
|
||||
```
|
||||
|
||||
Expected output includes:
|
||||
|
||||
```text
|
||||
NextActions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Replace Handler Dict Helpers with `NextActions`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
|
||||
- [ ] **Step 1: Import `NextActions`**
|
||||
|
||||
Add to the local imports in `src/wf_mcp/workflow_surface/handlers.py`:
|
||||
|
||||
```python
|
||||
from .next_actions import NextActions
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `create_draft_workspace_from_capability`**
|
||||
|
||||
Replace the `next_actions` part of the return value with:
|
||||
|
||||
```python
|
||||
return {
|
||||
**result,
|
||||
"wrapper_hints": hints,
|
||||
"next_actions": NextActions.from_wrapper_hints(
|
||||
workspace_id=workspace_id,
|
||||
revision=int(result["revision"]),
|
||||
hints=hints,
|
||||
).model_dump(mode="json"),
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Delete handler-local guidance helpers**
|
||||
|
||||
Delete these functions from `src/wf_mcp/workflow_surface/handlers.py`:
|
||||
|
||||
```python
|
||||
def _wrapper_draft_next_actions(...)
|
||||
def _wrapper_draft_patch_examples(...)
|
||||
```
|
||||
|
||||
Do not delete `_source_id_for_capability(...)`, which follows those helpers.
|
||||
|
||||
- [ ] **Step 4: Run focused draft tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py -q
|
||||
```
|
||||
|
||||
Expected: PASS except for assertions that still need additive `can_continue` checks in Task 5.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Update Integration and Schema Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_drafts.py`
|
||||
- Modify: `tests/wf_mcp/server/test_config.py`
|
||||
|
||||
- [ ] **Step 1: Add `can_continue` assertions to draft integration tests**
|
||||
|
||||
In `tests/wf_mcp/workflow_surface/test_drafts.py`, find the test that asserts high-confidence `next_actions`.
|
||||
|
||||
Add this assertion near the other `next_actions` assertions:
|
||||
|
||||
```python
|
||||
assert next_actions["can_continue"] is True
|
||||
```
|
||||
|
||||
Keep these existing assertions:
|
||||
|
||||
```python
|
||||
assert next_actions["can_save_now"] is True
|
||||
assert next_actions["recommended_next_tool"] == (
|
||||
"wf.workflow.validate_draft_workspace"
|
||||
)
|
||||
assert "high confidence" in next_actions["reason"]
|
||||
assert next_actions["patch_examples"] == []
|
||||
assert next_actions["warnings"] == []
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add `can_continue` assertion to low-confidence integration test**
|
||||
|
||||
In the low-confidence/content-block test in `tests/wf_mcp/workflow_surface/test_drafts.py`, add:
|
||||
|
||||
```python
|
||||
assert next_actions["can_continue"] is True
|
||||
```
|
||||
|
||||
Keep field-level assertions. Do not assert whole dict equality.
|
||||
|
||||
- [ ] **Step 3: Update MCP schema test**
|
||||
|
||||
In `tests/wf_mcp/server/test_config.py`, find the `create_draft_workspace_from_capability` output schema test.
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
next_actions_schema = result_schema["properties"]["next_actions"]
|
||||
next_action_properties = next_actions_schema["properties"]
|
||||
assert "can_continue" in next_action_properties
|
||||
assert "Advisory" in next_action_properties["can_continue"]["description"]
|
||||
assert "can_save_now" in next_action_properties
|
||||
assert "Advisory" in next_action_properties["can_save_now"]["description"]
|
||||
```
|
||||
|
||||
If the existing test already has `next_actions_schema`, reuse it. Do not assert the entire generated schema.
|
||||
|
||||
- [ ] **Step 4: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Add a Small Documentation Note
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/workflow_capabilities.md`
|
||||
|
||||
- [ ] **Step 1: Add an advisory-guidance note**
|
||||
|
||||
Find the section that mentions `next_actions` or wrapper draft guidance. Add this paragraph:
|
||||
|
||||
```markdown
|
||||
`next_actions` is advisory guidance, not validation authority. It gives MCP
|
||||
clients a compact "what should I call next?" pointer, while diagnostics,
|
||||
artifact validation, deployment validation, and runtime status remain the
|
||||
source of truth.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify docs reference still works**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/server/test_docs.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Full Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py tests/wf_mcp/server/test_docs.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 2: Run formatting check on touched files**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff format --check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 3: Run lint on touched files**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 4: Run type check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 5: Run the full test suite if time allows**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: current suite status should remain unchanged from the baseline, currently about `723 passed, 1 skipped, 1 xfailed`.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- Spec coverage:
|
||||
- `next_actions.py` is created.
|
||||
- Generic models exist.
|
||||
- Wrapper-hint constructor exists.
|
||||
- Handler-local dict helpers are removed.
|
||||
- Existing JSON fields remain stable.
|
||||
- Deployment/run guidance is intentionally deferred.
|
||||
|
||||
- Placeholder scan:
|
||||
- No `TBD`.
|
||||
- No open-ended "add appropriate validation".
|
||||
- Each code step includes exact snippets.
|
||||
|
||||
- Type consistency:
|
||||
- `NextActionTool.PATCH_DRAFT_WORKSPACE.value` serializes to the existing string.
|
||||
- `WrapperDraftNextActions` remains importable as an alias.
|
||||
- `CreateDraftWorkspaceFromCapabilityResult.next_actions` keeps the same public response location.
|
||||
|
||||
## Handoff Notes
|
||||
|
||||
This plan is intentionally small. It should not change workflow behavior or draft semantics. If any test fails outside `next_actions` schema/serialization, stop and inspect before broadening the change.
|
||||
@@ -0,0 +1,481 @@
|
||||
# Workflow Lifecycle Operator Docs Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make the MCP-facing workflow lifecycle understandable end-to-end for LLM clients and human operators.
|
||||
|
||||
**Architecture:** This is a documentation and schema-test pass, not a runtime redesign. The docs should describe the current stable workflow control surface: discover capabilities, create a draft workspace, save an artifact, save/validate/delete a deployment, run, inspect trace slices, and resume interrupted durable runs. Keep examples aligned with the actual MCP tool names and current compact-response philosophy.
|
||||
|
||||
**Tech Stack:** Markdown docs, Python tests, FastMCP tool metadata, `uv run pytest`, `uv run ruff`, `uv run basedpyright --level error`.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
Implement documentation for the current workflow lifecycle. Do not add new MCP tools unless a test proves a documented flow cannot be expressed with existing tools.
|
||||
|
||||
Current tools to document as primary path:
|
||||
|
||||
- `wf.workflow.list_capabilities`
|
||||
- `wf.workflow.inspect_capability`
|
||||
- `wf.workflow.create_draft_workspace_from_capability`
|
||||
- `wf.workflow.patch_draft_workspace`
|
||||
- `wf.workflow.validate_draft_workspace`
|
||||
- `wf.workflow.create_artifact_from_workspace`
|
||||
- `wf.workflow.save_deployment`
|
||||
- `wf.workflow.validate_deployment`
|
||||
- `wf.workflow.run_deployment`
|
||||
- `wf.workflow.inspect_run`
|
||||
- `wf.workflow.read_run_trace`
|
||||
- `wf.workflow.resume_run`
|
||||
- `wf.workflow.delete_deployment`
|
||||
|
||||
Important behavior to make explicit:
|
||||
|
||||
- `validate_deployment` is stored-catalog validation by default.
|
||||
- `validate_deployment(live_check=true)` contacts required upstream sources and can spawn stdio/network work.
|
||||
- `run_deployment` returns compact status and `trace_count`; do not request trace by default.
|
||||
- `read_run_trace` is the explicit bounded debug path.
|
||||
- `resume_run` works for durable stopped runs when the run is interrupted and dependencies remain compatible.
|
||||
- `delete_deployment` removes only mutable deployment bindings. It does not delete immutable workflow artifacts or durable run records.
|
||||
|
||||
## Files
|
||||
|
||||
- Modify: `docs/wf_mcp_operator_manual.md`
|
||||
- Primary mental model and short command flow.
|
||||
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
|
||||
- Full happy path with concrete MCP tool payloads.
|
||||
- Modify: `docs/wf_mcp_troubleshooting.md`
|
||||
- Failure-mode lookup for validation, live checks, run, resume, and cleanup.
|
||||
- Modify: `docs/durable_run_operations.md`
|
||||
- Clarify run/resume/trace lifecycle and relationship to deployment deletion.
|
||||
- Modify: `docs/README.md`
|
||||
- Ensure the docs index points readers to the operator manual/runbook/troubleshooting in the right order.
|
||||
- Modify: `tests/wf_mcp/server/test_docs.py`
|
||||
- Add low-cost assertions that the exported docs resources include the new lifecycle terms.
|
||||
- Modify only if needed: `src/wf_mcp/documentation.py`
|
||||
- Do not add new docs resources unless existing resources do not expose the updated docs.
|
||||
|
||||
## Task 1: Update The Operator Manual Primary Path
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/wf_mcp_operator_manual.md`
|
||||
|
||||
- [ ] **Step 1: Find the current workflow tool family section**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "Workflow Tools|workflow tools|validate_deployment|run_deployment|delete_deployment" docs/wf_mcp_operator_manual.md
|
||||
```
|
||||
|
||||
Expected: existing sections mention workflow discovery, deployment validation, run, and resume.
|
||||
|
||||
- [ ] **Step 2: Add a concise primary-path checklist**
|
||||
|
||||
Add or update a section named exactly:
|
||||
|
||||
```markdown
|
||||
## Primary Workflow Lifecycle
|
||||
```
|
||||
|
||||
Use this content, adjusting surrounding prose only for fit:
|
||||
|
||||
```markdown
|
||||
## Primary Workflow Lifecycle
|
||||
|
||||
Use this path when an LLM client needs to build, test, and run a saved workflow.
|
||||
|
||||
1. Discover workflow-ready capabilities with `wf.workflow.list_capabilities`.
|
||||
2. Inspect the selected capability with `wf.workflow.inspect_capability`.
|
||||
3. Create a patchable draft with `wf.workflow.create_draft_workspace_from_capability`.
|
||||
4. Patch the draft with `wf.workflow.patch_draft_workspace`.
|
||||
5. Validate the draft with `wf.workflow.validate_draft_workspace`.
|
||||
6. Save an immutable workflow or wrapper artifact with `wf.workflow.create_artifact_from_workspace` or `wf.workflow.create_wrapper_from_workspace`.
|
||||
7. Save a mutable deployment with `wf.workflow.save_deployment`.
|
||||
8. Validate the deployment with `wf.workflow.validate_deployment`.
|
||||
9. Optionally call `wf.workflow.validate_deployment` with `live_check=true` before a real run.
|
||||
10. Run with `wf.workflow.run_deployment`.
|
||||
11. If the run returns `interrupted`, resume with `wf.workflow.resume_run`.
|
||||
12. Inspect stopped runs with `wf.workflow.inspect_run`; read bounded trace slices with `wf.workflow.read_run_trace`.
|
||||
13. Delete temporary deployments with `wf.workflow.delete_deployment`.
|
||||
|
||||
Artifacts are immutable saved definitions. Deployments are mutable environment bindings. Runs are durable stopped execution records. Deleting a deployment does not delete artifacts or existing run records.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Document `live_check` next to deployment validation**
|
||||
|
||||
Find the `validate_deployment` section and add:
|
||||
|
||||
```markdown
|
||||
By default, `validate_deployment` validates against the broker's current source inventory and saved catalog snapshots. This is cheap and side-effect-light.
|
||||
|
||||
Pass `live_check=true` only when you explicitly want to contact each required upstream source. A live check may spawn stdio MCP servers or perform network I/O. Live-check failures are returned as `source_unreachable` diagnostics.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Document `delete_deployment` in the tool table**
|
||||
|
||||
Add a table row near deployment tools:
|
||||
|
||||
```markdown
|
||||
| Delete a temporary deployment binding | `wf.workflow.delete_deployment` |
|
||||
```
|
||||
|
||||
Also add:
|
||||
|
||||
```markdown
|
||||
`delete_deployment` removes the saved deployment binding only. It does not delete workflow artifacts, wrapper artifacts, or run checkpoints.
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run a docs grep sanity check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "Primary Workflow Lifecycle|live_check|source_unreachable|delete_deployment" docs/wf_mcp_operator_manual.md
|
||||
```
|
||||
|
||||
Expected: all four terms appear.
|
||||
|
||||
## Task 2: Update The End-To-End Runbook With Concrete Payloads
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
|
||||
|
||||
- [ ] **Step 1: Locate the happy-path flow**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "create_draft_workspace_from_capability|save_deployment|validate_deployment|run_deployment|resume_run" docs/wf_mcp_end_to_end_runbook.md
|
||||
```
|
||||
|
||||
Expected: existing examples for draft creation, deployment validation, and run execution.
|
||||
|
||||
- [ ] **Step 2: Add a compact lifecycle summary near the top**
|
||||
|
||||
Add:
|
||||
|
||||
```markdown
|
||||
## Minimal Lifecycle Summary
|
||||
|
||||
The shortest dependable lifecycle is:
|
||||
|
||||
```text
|
||||
list_capabilities
|
||||
inspect_capability
|
||||
create_draft_workspace_from_capability
|
||||
patch_draft_workspace
|
||||
validate_draft_workspace
|
||||
create_artifact_from_workspace
|
||||
save_deployment
|
||||
validate_deployment
|
||||
run_deployment
|
||||
inspect_run or read_run_trace only when needed
|
||||
resume_run only when status is interrupted
|
||||
delete_deployment for temporary deployments
|
||||
```
|
||||
|
||||
Do not expect a newly saved workflow to appear as a new MCP tool in an existing client session. Use `run_deployment` and `call_capability` as stable front doors.
|
||||
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add an explicit live validation example**
|
||||
|
||||
Near the deployment validation example, add:
|
||||
|
||||
```markdown
|
||||
### Optional Live Source Check
|
||||
|
||||
Use this before a real run when you need to know whether the bound upstream source can currently answer.
|
||||
|
||||
```yaml
|
||||
tool: wf.workflow.validate_deployment
|
||||
arguments:
|
||||
deployment_id: "example.personal"
|
||||
live_check: true
|
||||
```
|
||||
|
||||
Expected successful shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"deployment_id": "example.personal",
|
||||
"status": "runnable",
|
||||
"diagnostics": []
|
||||
}
|
||||
```
|
||||
|
||||
If a bound upstream source is down, expect `status="unrunnable"` and a diagnostic with `code="source_unreachable"`.
|
||||
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add a cleanup example**
|
||||
|
||||
Near the end of the runbook, add:
|
||||
|
||||
```markdown
|
||||
### Cleanup Temporary Deployments
|
||||
|
||||
Temporary test deployments can be removed without touching immutable artifacts.
|
||||
|
||||
```yaml
|
||||
tool: wf.workflow.delete_deployment
|
||||
arguments:
|
||||
deployment_id: "example.personal"
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```json
|
||||
{
|
||||
"deployment_id": "example.personal",
|
||||
"deleted": true
|
||||
}
|
||||
```
|
||||
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run grep sanity check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "Minimal Lifecycle Summary|Optional Live Source Check|delete_deployment|source_unreachable" docs/wf_mcp_end_to_end_runbook.md
|
||||
```
|
||||
|
||||
Expected: all four terms appear.
|
||||
|
||||
## Task 3: Update Troubleshooting For Live Checks And Cleanup
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/wf_mcp_troubleshooting.md`
|
||||
|
||||
- [ ] **Step 1: Locate validation diagnostics**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "binding_missing|source_missing|source_disabled|capability_missing|schema_changed|source_unreachable|delete_deployment" docs/wf_mcp_troubleshooting.md
|
||||
```
|
||||
|
||||
Expected: existing sections for static diagnostics; `source_unreachable` may be missing.
|
||||
|
||||
- [ ] **Step 2: Add `source_unreachable` section**
|
||||
|
||||
Add this section after `source_disabled` or near other deployment validation diagnostics:
|
||||
|
||||
```markdown
|
||||
## `validate_deployment(live_check=true)` Says `source_unreachable`
|
||||
|
||||
Meaning: static deployment validation found a matching saved source/catalog, but the live upstream source could not answer when contacted.
|
||||
|
||||
Common causes:
|
||||
|
||||
- stdio MCP server command is missing or exits during startup
|
||||
- network MCP server is offline
|
||||
- auth/config changed outside the broker
|
||||
- source process starts too slowly and hits the live-check timeout
|
||||
|
||||
What to do:
|
||||
|
||||
1. Check the connection with `wf.admin.get_connection_statuses`.
|
||||
2. Refresh or reload the config if the source was recently enabled.
|
||||
3. Fix the source command/auth/network outside the workflow artifact.
|
||||
4. Run `wf.workflow.validate_deployment` again with `live_check=true`.
|
||||
|
||||
Do not fix this by editing the workflow artifact unless the source capability itself changed. This is an environment problem, not workflow business logic.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add deployment cleanup troubleshooting**
|
||||
|
||||
Add:
|
||||
|
||||
```markdown
|
||||
## Test Deployment Clutter
|
||||
|
||||
Symptom: `wf.workflow.list_deployments` shows temporary deployments from earlier tests or LLM attempts.
|
||||
|
||||
Use:
|
||||
|
||||
```yaml
|
||||
tool: wf.workflow.delete_deployment
|
||||
arguments:
|
||||
deployment_id: "test_alias_check"
|
||||
```
|
||||
|
||||
This deletes only the mutable deployment binding. Saved artifacts and durable run records remain.
|
||||
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run grep sanity check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "source_unreachable|Test Deployment Clutter|delete_deployment|test_alias_check" docs/wf_mcp_troubleshooting.md
|
||||
```
|
||||
|
||||
Expected: all four terms appear.
|
||||
|
||||
## Task 4: Update Durable Run Docs For Deployment Deletion Boundaries
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/durable_run_operations.md`
|
||||
|
||||
- [ ] **Step 1: Locate run/deployment lifecycle text**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "deployment|run_deployment|resume_run|inspect_run|read_run_trace|delete_deployment" docs/durable_run_operations.md
|
||||
```
|
||||
|
||||
Expected: current run/resume docs; `delete_deployment` may be missing.
|
||||
|
||||
- [ ] **Step 2: Add deployment deletion boundary note**
|
||||
|
||||
Add under the run lifecycle or deployment section:
|
||||
|
||||
```markdown
|
||||
## Deployment Deletion Boundary
|
||||
|
||||
`wf.workflow.delete_deployment` removes the mutable deployment binding. It does not delete:
|
||||
|
||||
- immutable workflow artifacts
|
||||
- wrapper artifacts
|
||||
- stored run records
|
||||
- run checkpoints
|
||||
|
||||
Existing run records keep the pinned deployment and artifact environment captured at run time. Deleting a deployment prevents future runs through that deployment id, but it does not erase historical stopped-run inspection data.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Reconfirm trace guidance**
|
||||
|
||||
Ensure the doc says:
|
||||
|
||||
```markdown
|
||||
Use `inspect_run` for compact stopped-run summaries. Use `read_run_trace` only when trace entries are needed, and always request a bounded `trace_range`.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run grep sanity check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "Deployment Deletion Boundary|delete_deployment|read_run_trace|trace_range" docs/durable_run_operations.md
|
||||
```
|
||||
|
||||
Expected: all four terms appear.
|
||||
|
||||
## Task 5: Update Docs Index And Exported Docs Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/README.md`
|
||||
- Modify: `tests/wf_mcp/server/test_docs.py`
|
||||
|
||||
- [ ] **Step 1: Update docs index ordering**
|
||||
|
||||
In `docs/README.md`, ensure these entries exist and read clearly:
|
||||
|
||||
```markdown
|
||||
- [`wf_mcp_operator_manual.md`](wf_mcp_operator_manual.md): start here for the MCP-facing workflow lifecycle and tool families.
|
||||
- [`wf_mcp_end_to_end_runbook.md`](wf_mcp_end_to_end_runbook.md): concrete tool-call runbook from capability discovery through deployment, run, resume, and cleanup.
|
||||
- [`wf_mcp_troubleshooting.md`](wf_mcp_troubleshooting.md): diagnostics and repair steps for source, deployment, run, and resume failures.
|
||||
- [`durable_run_operations.md`](durable_run_operations.md): durable run records, compact inspection, bounded traces, and resume semantics.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add docs resource assertions**
|
||||
|
||||
Open `tests/wf_mcp/server/test_docs.py` and find the test that reads `wf://docs/operator-manual` or docs resources.
|
||||
|
||||
Add field-level assertions instead of whole-dict assertions:
|
||||
|
||||
```python
|
||||
assert "Primary Workflow Lifecycle" in manual_text
|
||||
assert "live_check" in manual_text
|
||||
assert "delete_deployment" in manual_text
|
||||
```
|
||||
|
||||
If the test currently uses a different variable name than `manual_text`, use the existing variable. Do not assert whole payload dict equality.
|
||||
|
||||
- [ ] **Step 3: Run focused docs tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/server/test_docs.py -q
|
||||
```
|
||||
|
||||
Expected: all tests in that file pass.
|
||||
|
||||
## Task 6: Final Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All modified docs and tests.
|
||||
|
||||
- [ ] **Step 1: Run docs grep checks**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "Primary Workflow Lifecycle|live_check|source_unreachable|delete_deployment|Deployment Deletion Boundary" docs
|
||||
```
|
||||
|
||||
Expected: terms appear in the intended docs, not only in historical plans.
|
||||
|
||||
- [ ] **Step 2: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/server/test_docs.py tests/wf_mcp/server/test_tools.py tests/wf_mcp/workflow_surface/test_deployments.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run lint/format checks for touched files**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run ruff check tests/wf_mcp/server/test_docs.py
|
||||
uv run ruff format --check tests/wf_mcp/server/test_docs.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 4: Run type check for touched Python files**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error tests/wf_mcp/server/test_docs.py
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 5: Optional full test suite**
|
||||
|
||||
Run when time allows:
|
||||
|
||||
```powershell
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected current baseline: full suite passes with the existing skip/xfail count.
|
||||
|
||||
## Notes For Opencode
|
||||
|
||||
- Keep this docs-first. Do not add new runtime behavior unless a doc assertion proves the current docs cannot represent the actual tool surface.
|
||||
- Prefer concise examples over giant JSON payloads.
|
||||
- Use exact current tool names with the `wf.workflow.*` namespace.
|
||||
- Do not document raw MCP proxy tools as the workflow authoring path.
|
||||
- Do not claim `delete_deployment` deletes artifacts or runs.
|
||||
- Do not tell users to expect saved workflows to appear as new MCP tools in existing client sessions.
|
||||
@@ -0,0 +1,485 @@
|
||||
# Wrapper Draft Next Actions Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add advisory `next_actions` to `wf.workflow.create_draft_workspace_from_capability` so LLM clients can continue safely even when they cannot easily read the docs.
|
||||
|
||||
**Architecture:** Keep `wrapper_hints` as the source of truth for scaffold confidence, missing decisions, and mapping warnings. Add a small typed result object that converts those hints into concrete next-tool guidance. `can_save_now` is advisory only; do not block saving or enforce policy in this slice.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, FastMCP tool schemas, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
Add `next_actions` to `CreateDraftWorkspaceFromCapabilityResult` and handler payloads.
|
||||
|
||||
Do:
|
||||
|
||||
- Return machine-readable guidance from `create_draft_workspace_from_capability`.
|
||||
- Make `can_save_now` advisory only.
|
||||
- Recommend `wf.workflow.patch_draft_workspace` when `wrapper_hints.missing_decisions` is non-empty or confidence is low.
|
||||
- Recommend `wf.workflow.validate_draft_workspace` when the scaffold looks safe enough to validate.
|
||||
- Include patch examples for common missing decisions.
|
||||
- Document the field and add schema tests.
|
||||
|
||||
Do not:
|
||||
|
||||
- Block `create_artifact_from_workspace`.
|
||||
- Add a new tool.
|
||||
- Infer raw MCP `content[0].text` automatically.
|
||||
- Treat boolean output candidates as real routing semantics.
|
||||
- Replace `wrapper_hints`; `next_actions` should summarize and guide, not duplicate every hint.
|
||||
|
||||
## Files
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/models.py`
|
||||
- Add typed `WrapperDraftNextActions` and `WrapperDraftPatchExample` Pydantic models.
|
||||
- Add `next_actions` field to `CreateDraftWorkspaceFromCapabilityResult`.
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Add helper that derives `next_actions` from `wrapper_hints` and workspace id/revision.
|
||||
- Include `next_actions` in `create_draft_workspace_from_capability` result.
|
||||
- Test: `tests/wf_mcp/workflow_surface/test_drafts.py`
|
||||
- Assert high-confidence draft returns validate/save guidance.
|
||||
- Assert low-confidence content-block draft returns patch guidance and advisory `can_save_now=false`.
|
||||
- Test: `tests/wf_mcp/server/test_config.py`
|
||||
- Assert output schema exposes `next_actions` with field descriptions.
|
||||
- Modify: `docs/workflow_capabilities.md`
|
||||
- Explain `next_actions` as advisory continuation hints.
|
||||
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
|
||||
- Mention that clients can follow `next_actions` after draft creation.
|
||||
|
||||
## Data Shape
|
||||
|
||||
Add this output shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"next_actions": {
|
||||
"can_save_now": true,
|
||||
"recommended_next_tool": "wf.workflow.validate_draft_workspace",
|
||||
"reason": "Wrapper hints are high confidence and have no missing decisions.",
|
||||
"patch_examples": [],
|
||||
"warnings": []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Low-confidence example:
|
||||
|
||||
```json
|
||||
{
|
||||
"next_actions": {
|
||||
"can_save_now": false,
|
||||
"recommended_next_tool": "wf.workflow.patch_draft_workspace",
|
||||
"reason": "Review missing wrapper decisions before saving.",
|
||||
"patch_examples": [
|
||||
{
|
||||
"description": "Replace the output bindings after choosing workflow state fields.",
|
||||
"tool": "wf.workflow.patch_draft_workspace",
|
||||
"request": {
|
||||
"workspace_id": "echo_wrapper",
|
||||
"revision": 1,
|
||||
"patch": [
|
||||
{
|
||||
"op": "replace",
|
||||
"path": "/draft/steps/call/output",
|
||||
"value": []
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"warnings": [
|
||||
"Raw MCP content blocks are not workflow-shaped. Use an explicit wrapper or extraction node."
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Task 1: Add Result Models And Schema Test
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/models.py`
|
||||
- Modify: `tests/wf_mcp/server/test_config.py`
|
||||
|
||||
- [ ] **Step 1: Add failing schema assertions**
|
||||
|
||||
In `tests/wf_mcp/server/test_config.py`, inside the test that already inspects `create_draft_workspace_from_capability` output schema, add:
|
||||
|
||||
```python
|
||||
assert "next_actions" in from_capability_output["properties"]
|
||||
next_actions_schema = from_capability_output["properties"]["next_actions"]
|
||||
assert "recommended_next_tool" in next_actions_schema["properties"]
|
||||
assert "patch_examples" in next_actions_schema["properties"]
|
||||
assert "advisory" in next_actions_schema["properties"]["can_save_now"]["description"]
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
Expected: fail because `next_actions` is not in the output schema.
|
||||
|
||||
- [ ] **Step 2: Add Pydantic result models**
|
||||
|
||||
In `src/wf_mcp/workflow_surface/models.py`, add near `CreateDraftWorkspaceFromCapabilityResult`:
|
||||
|
||||
```python
|
||||
class WrapperDraftPatchExample(BaseModel):
|
||||
"""Concrete patch-workspace example for a likely next authoring edit."""
|
||||
|
||||
description: str = Field(description="Human-readable reason for this patch.")
|
||||
tool: str = Field(description="MCP tool to call for this example.")
|
||||
request: dict[str, Any] = Field(
|
||||
description="JSON request payload to pass to the tool."
|
||||
)
|
||||
|
||||
|
||||
class WrapperDraftNextActions(BaseModel):
|
||||
"""Advisory continuation hints after bootstrapping a wrapper draft."""
|
||||
|
||||
can_save_now: bool = Field(
|
||||
description=(
|
||||
"Advisory only. False means the scaffold likely needs review before "
|
||||
"saving, but the server does not enforce this."
|
||||
)
|
||||
)
|
||||
recommended_next_tool: str = Field(
|
||||
description=(
|
||||
"Suggested next MCP tool, usually wf.workflow.validate_draft_workspace "
|
||||
"or wf.workflow.patch_draft_workspace."
|
||||
)
|
||||
)
|
||||
reason: str = Field(description="Short explanation for the recommendation.")
|
||||
patch_examples: list[WrapperDraftPatchExample] = Field(
|
||||
default_factory=list,
|
||||
description="Concrete JSON Patch examples for common missing decisions.",
|
||||
)
|
||||
warnings: list[str] = Field(
|
||||
default_factory=list,
|
||||
description="Non-blocking warnings copied from low-confidence wrapper hints.",
|
||||
)
|
||||
```
|
||||
|
||||
Then update:
|
||||
|
||||
```python
|
||||
class CreateDraftWorkspaceFromCapabilityResult(DraftWorkspaceResult):
|
||||
"""Draft workspace result plus wrapper hints and advisory next actions."""
|
||||
|
||||
wrapper_hints: dict[str, Any] = Field(...)
|
||||
next_actions: WrapperDraftNextActions = Field(
|
||||
description=(
|
||||
"Advisory next step guidance derived from wrapper_hints. "
|
||||
"The server does not enforce can_save_now."
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run schema test**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
Expected: schema assertion passes, but runtime tests may still fail later until handler returns `next_actions`.
|
||||
|
||||
## Task 2: Derive Next Actions In The Handler
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_drafts.py`
|
||||
|
||||
- [ ] **Step 1: Add high-confidence behavior test**
|
||||
|
||||
In `tests/wf_mcp/workflow_surface/test_drafts.py`, extend `test_workflow_surface_creates_draft_workspace_from_capability_hints`:
|
||||
|
||||
```python
|
||||
next_actions = result["next_actions"]
|
||||
assert next_actions["can_save_now"] is True
|
||||
assert next_actions["recommended_next_tool"] == "wf.workflow.validate_draft_workspace"
|
||||
assert "high confidence" in next_actions["reason"]
|
||||
assert next_actions["patch_examples"] == []
|
||||
assert next_actions["warnings"] == []
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py::test_workflow_surface_creates_draft_workspace_from_capability_hints -q
|
||||
```
|
||||
|
||||
Expected: fail because handler does not return `next_actions`.
|
||||
|
||||
- [ ] **Step 2: Add low-confidence behavior test**
|
||||
|
||||
Find or create a content-block capability test near existing wrapper hint tests. Use an output schema shaped like:
|
||||
|
||||
```python
|
||||
content_output_schema = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"content": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"type": {"type": "string"},
|
||||
"text": {"type": "string"},
|
||||
},
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Register a NodeSpec/capability with that output schema, call `create_draft_workspace_from_capability`, then assert:
|
||||
|
||||
```python
|
||||
next_actions = result["next_actions"]
|
||||
assert next_actions["can_save_now"] is False
|
||||
assert next_actions["recommended_next_tool"] == "wf.workflow.patch_draft_workspace"
|
||||
assert "missing wrapper decisions" in next_actions["reason"]
|
||||
assert next_actions["patch_examples"][0]["tool"] == "wf.workflow.patch_draft_workspace"
|
||||
assert next_actions["patch_examples"][0]["request"]["workspace_id"] == "content_wrapper"
|
||||
assert next_actions["patch_examples"][0]["request"]["revision"] == result["revision"]
|
||||
assert next_actions["warnings"]
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py -q
|
||||
```
|
||||
|
||||
Expected: fail until implementation exists.
|
||||
|
||||
- [ ] **Step 3: Implement helper in handler**
|
||||
|
||||
In `src/wf_mcp/workflow_surface/handlers.py`, add a helper near other draft helpers:
|
||||
|
||||
```python
|
||||
def _wrapper_draft_next_actions(
|
||||
*,
|
||||
workspace_id: str,
|
||||
revision: int,
|
||||
hints: dict[str, Any],
|
||||
) -> dict[str, Any]:
|
||||
"""Convert wrapper_hints into advisory next-tool guidance for MCP clients."""
|
||||
confidence = str(hints.get("confidence", "low"))
|
||||
missing_decisions = hints.get("missing_decisions")
|
||||
notes = [str(note) for note in hints.get("notes", []) if isinstance(note, str)]
|
||||
has_missing = isinstance(missing_decisions, list) and len(missing_decisions) > 0
|
||||
can_save_now = confidence == "high" and not has_missing
|
||||
if can_save_now:
|
||||
return {
|
||||
"can_save_now": True,
|
||||
"recommended_next_tool": "wf.workflow.validate_draft_workspace",
|
||||
"reason": "Wrapper hints are high confidence and have no missing decisions.",
|
||||
"patch_examples": [],
|
||||
"warnings": [],
|
||||
}
|
||||
|
||||
return {
|
||||
"can_save_now": False,
|
||||
"recommended_next_tool": "wf.workflow.patch_draft_workspace",
|
||||
"reason": "Review missing wrapper decisions before saving.",
|
||||
"patch_examples": _wrapper_draft_patch_examples(
|
||||
workspace_id=workspace_id,
|
||||
revision=revision,
|
||||
hints=hints,
|
||||
),
|
||||
"warnings": notes,
|
||||
}
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def _wrapper_draft_patch_examples(
|
||||
*,
|
||||
workspace_id: str,
|
||||
revision: int,
|
||||
hints: dict[str, Any],
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Return conservative JSON Patch examples without guessing semantics."""
|
||||
examples: list[dict[str, Any]] = []
|
||||
missing_decisions = hints.get("missing_decisions")
|
||||
if not isinstance(missing_decisions, list):
|
||||
return examples
|
||||
decision_kinds = {
|
||||
str(decision.get("kind"))
|
||||
for decision in missing_decisions
|
||||
if isinstance(decision, dict)
|
||||
}
|
||||
if {
|
||||
"choose_output_fields",
|
||||
"review_nested_output",
|
||||
} & decision_kinds:
|
||||
examples.append(
|
||||
{
|
||||
"description": (
|
||||
"Replace output bindings after choosing which capability "
|
||||
"outputs should be written to workflow state."
|
||||
),
|
||||
"tool": "wf.workflow.patch_draft_workspace",
|
||||
"request": {
|
||||
"workspace_id": workspace_id,
|
||||
"revision": revision,
|
||||
"patch": [
|
||||
{
|
||||
"op": "replace",
|
||||
"path": "/draft/steps/call/output",
|
||||
"value": [],
|
||||
}
|
||||
],
|
||||
},
|
||||
}
|
||||
)
|
||||
if "confirm_boolean_outcomes" in decision_kinds:
|
||||
examples.append(
|
||||
{
|
||||
"description": (
|
||||
"Review boolean output candidates before adding routing; "
|
||||
"do not route on boolean fields automatically."
|
||||
),
|
||||
"tool": "wf.workflow.patch_draft_workspace",
|
||||
"request": {
|
||||
"workspace_id": workspace_id,
|
||||
"revision": revision,
|
||||
"patch": [],
|
||||
},
|
||||
}
|
||||
)
|
||||
return examples
|
||||
```
|
||||
|
||||
Then change the return in `create_draft_workspace_from_capability`:
|
||||
|
||||
```python
|
||||
return {
|
||||
**result,
|
||||
"wrapper_hints": hints,
|
||||
"next_actions": _wrapper_draft_next_actions(
|
||||
workspace_id=workspace_id,
|
||||
revision=int(result["revision"]),
|
||||
hints=hints,
|
||||
),
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run behavior tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 3: Document Next Actions
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/workflow_capabilities.md`
|
||||
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
|
||||
|
||||
- [ ] **Step 1: Update workflow capability docs**
|
||||
|
||||
In `docs/workflow_capabilities.md`, near the `wrapper_hints` section, add:
|
||||
|
||||
```markdown
|
||||
`create_draft_workspace_from_capability` also returns `next_actions`.
|
||||
This is advisory guidance for clients that cannot easily read the full docs.
|
||||
It summarizes whether the scaffold is safe-looking enough to validate, which
|
||||
tool to call next, and concrete patch examples for common missing decisions.
|
||||
|
||||
`next_actions.can_save_now` is not enforced. A caller can still save a low
|
||||
confidence draft, but the field exists to make that risk explicit.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update runbook**
|
||||
|
||||
In `docs/wf_mcp_end_to_end_runbook.md`, near the draft-from-capability example, add:
|
||||
|
||||
```markdown
|
||||
After `create_draft_workspace_from_capability`, inspect `next_actions`.
|
||||
If `recommended_next_tool` is `wf.workflow.patch_draft_workspace`, apply or
|
||||
adapt the returned `patch_examples` before saving. If it recommends
|
||||
`wf.workflow.validate_draft_workspace`, validate the draft before creating an
|
||||
artifact.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Grep docs**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "next_actions|can_save_now|patch_examples" docs/workflow_capabilities.md docs/wf_mcp_end_to_end_runbook.md
|
||||
```
|
||||
|
||||
Expected: all three terms appear.
|
||||
|
||||
## Task 4: Final Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py tests/wf_mcp/test_workflow_wrapper_hints.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run touched-file lint and format check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run ruff check src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
|
||||
uv run ruff format --check src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run touched-file type check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error src/wf_mcp/workflow_surface/models.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_drafts.py tests/wf_mcp/server/test_config.py
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 4: Optional full suite**
|
||||
|
||||
Run when time allows:
|
||||
|
||||
```powershell
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected current baseline: full suite passes with the existing skip/xfail count.
|
||||
|
||||
## Notes For Opencode
|
||||
|
||||
- `can_save_now` is advisory. Do not enforce it.
|
||||
- Do not add a new save gate.
|
||||
- Do not generate semantic routes from boolean fields.
|
||||
- Keep patch examples conservative; empty patch examples are acceptable when the missing decision cannot be safely represented.
|
||||
- Avoid whole-dict assertions. Assert individual fields.
|
||||
- Existing docs may be long; keep additions short.
|
||||
@@ -0,0 +1,707 @@
|
||||
# Deployment And Run Next Actions Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add advisory `next_actions` guidance to deployment validation and run lifecycle responses so MCP clients know the safest next workflow tool to call.
|
||||
|
||||
**Architecture:** Extend the existing `src/wf_mcp/workflow_surface/next_actions.py` module with deployment/run constructors instead of putting more UX policy in `handlers.py`. Thread the resulting `NextActions` object into `validate_deployment(...)` and `_run_payload(...)`, which covers `run_deployment`, `resume_run`, `inspect_run`, and `read_run_trace`. Keep diagnostics and runtime status authoritative; `next_actions` remains advisory.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_mcp/workflow_surface/next_actions.py`
|
||||
- Add `NextActions.from_deployment_validation(...)`.
|
||||
- Add `NextActions.from_run_result(...)`.
|
||||
- Add tiny private helpers for diagnostic code inspection and bounded trace examples.
|
||||
|
||||
- Modify `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Add `next_actions` to `validate_deployment(...)`.
|
||||
- Add optional `next_actions` construction inside `_run_payload(...)`.
|
||||
- Do not duplicate branching logic in individual handler methods.
|
||||
|
||||
- Modify `tests/wf_mcp/workflow_surface/test_next_actions.py`
|
||||
- Unit-test deployment/run constructors directly.
|
||||
|
||||
- Modify `tests/wf_mcp/workflow_surface/test_deployments.py`
|
||||
- Assert `validate_deployment` returns useful next actions for runnable and unrunnable deployments.
|
||||
|
||||
- Modify `tests/wf_mcp/workflow_surface/test_runs.py`
|
||||
- Assert run lifecycle responses return next actions for completed and failed runs.
|
||||
- Add interrupt/resume tests only if an existing fixture/helper makes this small; otherwise leave interrupt-specific coverage to constructor unit tests.
|
||||
|
||||
- Modify `tests/wf_mcp/server/test_config.py`
|
||||
- Assert output schemas for `validate_deployment` and `run_deployment` include `next_actions`.
|
||||
|
||||
- Modify `docs/workflow_capabilities.md`
|
||||
- Add a short note that deployment/run responses now include advisory `next_actions`.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Do not create new workflow tools.
|
||||
- Do not make `next_actions` required for correctness.
|
||||
- Do not read or return full traces automatically.
|
||||
- Do not add auto-repair behavior.
|
||||
- Do not change existing status strings, diagnostics, or run payload fields.
|
||||
- Do not implement persisted resume changes in this pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Constructor Unit Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_next_actions.py`
|
||||
|
||||
- [ ] **Step 1: Add imports**
|
||||
|
||||
At the top of `tests/wf_mcp/workflow_surface/test_next_actions.py`, add:
|
||||
|
||||
```python
|
||||
from wf_artifacts import DependencyDiagnostic, DiagnosticSeverity
|
||||
```
|
||||
|
||||
Keep the existing `NextActionTool` / `NextActions` import.
|
||||
|
||||
- [ ] **Step 2: Add deployment validation constructor tests**
|
||||
|
||||
Append to `tests/wf_mcp/workflow_surface/test_next_actions.py`:
|
||||
|
||||
```python
|
||||
def test_next_actions_from_runnable_deployment_recommends_run() -> None:
|
||||
actions = NextActions.from_deployment_validation(
|
||||
deployment_id="echo.personal",
|
||||
diagnostics=[],
|
||||
)
|
||||
|
||||
dumped = actions.model_dump(mode="json")
|
||||
assert dumped["can_continue"] is True
|
||||
assert dumped["recommended_next_tool"] == NextActionTool.RUN_DEPLOYMENT.value
|
||||
assert "run_deployment" in dumped["reason"]
|
||||
assert dumped["warnings"] == []
|
||||
|
||||
|
||||
def test_next_actions_from_unrunnable_deployment_recommends_validation_retry() -> None:
|
||||
diagnostic = DependencyDiagnostic(
|
||||
severity=DiagnosticSeverity.ERROR,
|
||||
code="source_unreachable",
|
||||
logical_ref="demo.echo_tool",
|
||||
bound_source="demo.personal",
|
||||
message="Live check for upstream source 'demo.personal' failed.",
|
||||
repair_hint="Start or reconnect the source.",
|
||||
)
|
||||
|
||||
actions = NextActions.from_deployment_validation(
|
||||
deployment_id="echo.personal",
|
||||
diagnostics=[diagnostic],
|
||||
)
|
||||
|
||||
dumped = actions.model_dump(mode="json")
|
||||
assert dumped["can_continue"] is True
|
||||
assert dumped["recommended_next_tool"] == NextActionTool.VALIDATE_DEPLOYMENT.value
|
||||
assert "fix or reconnect" in dumped["reason"]
|
||||
assert dumped["warnings"][0] == "source_unreachable: demo.personal"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add run result constructor tests**
|
||||
|
||||
Append to `tests/wf_mcp/workflow_surface/test_next_actions.py`:
|
||||
|
||||
```python
|
||||
def test_next_actions_from_completed_run_has_no_required_next_tool() -> None:
|
||||
actions = NextActions.from_run_result(
|
||||
run_id="run_123",
|
||||
status="completed",
|
||||
trace_count=2,
|
||||
diagnostics=[],
|
||||
)
|
||||
|
||||
dumped = actions.model_dump(mode="json")
|
||||
assert dumped["can_continue"] is False
|
||||
assert dumped["recommended_next_tool"] is None
|
||||
assert "completed" in dumped["reason"]
|
||||
assert dumped["patch_examples"] == []
|
||||
|
||||
|
||||
def test_next_actions_from_failed_run_recommends_bounded_trace() -> None:
|
||||
actions = NextActions.from_run_result(
|
||||
run_id="run_123",
|
||||
status="failed",
|
||||
trace_count=12,
|
||||
diagnostics=[],
|
||||
)
|
||||
|
||||
dumped = actions.model_dump(mode="json")
|
||||
assert dumped["can_continue"] is True
|
||||
assert dumped["recommended_next_tool"] == NextActionTool.READ_RUN_TRACE.value
|
||||
assert "bounded trace" in dumped["reason"]
|
||||
assert dumped["patch_examples"][0]["tool"] == NextActionTool.READ_RUN_TRACE.value
|
||||
assert dumped["patch_examples"][0]["request"]["run_id"] == "run_123"
|
||||
assert dumped["patch_examples"][0]["request"]["trace_range"]["start"] == 0
|
||||
assert dumped["patch_examples"][0]["request"]["trace_range"]["limit"] == 25
|
||||
|
||||
|
||||
def test_next_actions_from_interrupted_run_recommends_resume() -> None:
|
||||
actions = NextActions.from_run_result(
|
||||
run_id="run_123",
|
||||
status="interrupted",
|
||||
trace_count=3,
|
||||
diagnostics=[],
|
||||
)
|
||||
|
||||
dumped = actions.model_dump(mode="json")
|
||||
assert dumped["can_continue"] is True
|
||||
assert dumped["recommended_next_tool"] == NextActionTool.RESUME_RUN.value
|
||||
assert "resume_run" in dumped["reason"]
|
||||
assert dumped["patch_examples"] == []
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the new tests to verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
|
||||
```
|
||||
|
||||
Expected: FAIL with `AttributeError` for missing `from_deployment_validation` and `from_run_result`.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Implement Deployment/Run Constructors
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/next_actions.py`
|
||||
|
||||
- [ ] **Step 1: Add type-only imports**
|
||||
|
||||
In `src/wf_mcp/workflow_surface/next_actions.py`, add:
|
||||
|
||||
```python
|
||||
from collections.abc import Sequence
|
||||
```
|
||||
|
||||
Do not import `DependencyDiagnostic` directly unless needed at runtime. The constructors can accept diagnostics as objects or dicts to keep coupling low.
|
||||
|
||||
- [ ] **Step 2: Add `from_deployment_validation`**
|
||||
|
||||
Inside `class NextActions`, after `from_wrapper_hints(...)`, add:
|
||||
|
||||
```python
|
||||
@classmethod
|
||||
def from_deployment_validation(
|
||||
cls,
|
||||
*,
|
||||
deployment_id: str,
|
||||
diagnostics: Sequence[object],
|
||||
) -> Self:
|
||||
"""Create guidance after validate_deployment."""
|
||||
if not diagnostics:
|
||||
return cls(
|
||||
can_continue=True,
|
||||
can_save_now=None,
|
||||
recommended_next_tool=NextActionTool.RUN_DEPLOYMENT,
|
||||
reason=(
|
||||
f"Deployment {deployment_id!r} is runnable; call "
|
||||
"wf.workflow.run_deployment with workflow_input."
|
||||
),
|
||||
patch_examples=[],
|
||||
warnings=[],
|
||||
)
|
||||
|
||||
codes = {_diagnostic_field(diagnostic, "code") for diagnostic in diagnostics}
|
||||
warnings = [_diagnostic_warning(diagnostic) for diagnostic in diagnostics]
|
||||
if "source_unreachable" in codes:
|
||||
reason = (
|
||||
"One or more live sources are unreachable; fix or reconnect the "
|
||||
"source, then rerun wf.workflow.validate_deployment with live_check=true."
|
||||
)
|
||||
elif "source_missing" in codes or "binding_missing" in codes:
|
||||
reason = (
|
||||
"Deployment bindings or sources are missing; inspect the deployment "
|
||||
"and save corrected bindings before running."
|
||||
)
|
||||
elif "capability_missing" in codes or "schema_changed" in codes:
|
||||
reason = (
|
||||
"A required capability is missing or drifted; inspect capabilities "
|
||||
"or refresh sources, then validate again."
|
||||
)
|
||||
else:
|
||||
reason = (
|
||||
"Deployment is not runnable; inspect diagnostics, repair the "
|
||||
"deployment or sources, then validate again."
|
||||
)
|
||||
return cls(
|
||||
can_continue=True,
|
||||
can_save_now=None,
|
||||
recommended_next_tool=NextActionTool.VALIDATE_DEPLOYMENT,
|
||||
reason=reason,
|
||||
patch_examples=[],
|
||||
warnings=warnings,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add `from_run_result`**
|
||||
|
||||
Inside `class NextActions`, after `from_deployment_validation(...)`, add:
|
||||
|
||||
```python
|
||||
@classmethod
|
||||
def from_run_result(
|
||||
cls,
|
||||
*,
|
||||
run_id: str | None,
|
||||
status: str,
|
||||
trace_count: int,
|
||||
diagnostics: Sequence[object],
|
||||
) -> Self:
|
||||
"""Create guidance after run_deployment, inspect_run, resume_run, or read_run_trace."""
|
||||
warnings = [_diagnostic_warning(diagnostic) for diagnostic in diagnostics]
|
||||
if status == "interrupted" and run_id is not None:
|
||||
return cls(
|
||||
can_continue=True,
|
||||
can_save_now=None,
|
||||
recommended_next_tool=NextActionTool.RESUME_RUN,
|
||||
reason=(
|
||||
"Run is interrupted; call wf.workflow.resume_run with this "
|
||||
"run_id and the interrupt response payload."
|
||||
),
|
||||
patch_examples=[],
|
||||
warnings=warnings,
|
||||
)
|
||||
if status in {"failed", "unrunnable"}:
|
||||
examples = (
|
||||
[_bounded_trace_example(run_id=run_id, trace_count=trace_count)]
|
||||
if run_id is not None and trace_count > 0
|
||||
else []
|
||||
)
|
||||
return cls(
|
||||
can_continue=bool(examples),
|
||||
can_save_now=None,
|
||||
recommended_next_tool=(
|
||||
NextActionTool.READ_RUN_TRACE if examples else None
|
||||
),
|
||||
reason=(
|
||||
"Run failed; read a bounded trace slice for debugging."
|
||||
if examples
|
||||
else "Run failed before producing trace entries; inspect diagnostics and error."
|
||||
),
|
||||
patch_examples=examples,
|
||||
warnings=warnings,
|
||||
)
|
||||
if status == "completed":
|
||||
examples = (
|
||||
[_bounded_trace_example(run_id=run_id, trace_count=trace_count)]
|
||||
if run_id is not None and trace_count > 0
|
||||
else []
|
||||
)
|
||||
return cls(
|
||||
can_continue=False,
|
||||
can_save_now=None,
|
||||
recommended_next_tool=None,
|
||||
reason=(
|
||||
"Run completed. No required next workflow tool; use read_run_trace "
|
||||
"with a bounded trace_range only if debugging."
|
||||
),
|
||||
patch_examples=examples,
|
||||
warnings=warnings,
|
||||
)
|
||||
return cls(
|
||||
can_continue=False,
|
||||
can_save_now=None,
|
||||
recommended_next_tool=None,
|
||||
reason=f"Run status {status!r} has no obvious next workflow tool.",
|
||||
patch_examples=[],
|
||||
warnings=warnings,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add private diagnostic helpers**
|
||||
|
||||
Below `_wrapper_draft_patch_examples(...)`, add:
|
||||
|
||||
```python
|
||||
def _diagnostic_field(diagnostic: object, field: str) -> str | None:
|
||||
"""Read a diagnostic field from either a Pydantic model or a JSON dict."""
|
||||
if isinstance(diagnostic, dict):
|
||||
value = diagnostic.get(field)
|
||||
else:
|
||||
value = getattr(diagnostic, field, None)
|
||||
return value if isinstance(value, str) else None
|
||||
|
||||
|
||||
def _diagnostic_warning(diagnostic: object) -> str:
|
||||
"""Format one compact diagnostic warning for next_actions."""
|
||||
code = _diagnostic_field(diagnostic, "code") or "diagnostic"
|
||||
bound_source = _diagnostic_field(diagnostic, "bound_source")
|
||||
logical_ref = _diagnostic_field(diagnostic, "logical_ref")
|
||||
if bound_source:
|
||||
return f"{code}: {bound_source}"
|
||||
if logical_ref:
|
||||
return f"{code}: {logical_ref}"
|
||||
return code
|
||||
|
||||
|
||||
def _bounded_trace_example(
|
||||
*,
|
||||
run_id: str,
|
||||
trace_count: int,
|
||||
) -> NextActionPatchExample:
|
||||
"""Return a safe read_run_trace request; never suggest full trace reads."""
|
||||
return NextActionPatchExample(
|
||||
description=(
|
||||
"Read a bounded debug trace slice. Increase start/limit only when needed."
|
||||
),
|
||||
tool=NextActionTool.READ_RUN_TRACE,
|
||||
request={
|
||||
"run_id": run_id,
|
||||
"trace_range": {
|
||||
"start": 0,
|
||||
"limit": min(trace_count, 25),
|
||||
},
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run constructor tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Thread NextActions Through Deployment Validation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_deployments.py`
|
||||
|
||||
- [ ] **Step 1: Add deployment assertions**
|
||||
|
||||
In `test_workflow_surface_validate_deployment_live_check_is_opt_in`, after current payload assertions, add:
|
||||
|
||||
```python
|
||||
assert payload["next_actions"]["can_continue"] is True
|
||||
assert payload["next_actions"]["recommended_next_tool"] == (
|
||||
"wf.workflow.run_deployment"
|
||||
)
|
||||
```
|
||||
|
||||
In `test_workflow_surface_validates_deployment_dependencies`, after diagnostic assertions, add:
|
||||
|
||||
```python
|
||||
assert payload["next_actions"]["can_continue"] is True
|
||||
assert payload["next_actions"]["recommended_next_tool"] == (
|
||||
"wf.workflow.validate_deployment"
|
||||
)
|
||||
assert payload["next_actions"]["warnings"][0] == "source_missing: context7.personal"
|
||||
```
|
||||
|
||||
If the exact warning uses `context7` instead of `context7.personal`, keep the assertion stable by checking fields separately:
|
||||
|
||||
```python
|
||||
assert payload["next_actions"]["warnings"]
|
||||
assert "source_missing" in payload["next_actions"]["warnings"][0]
|
||||
```
|
||||
|
||||
Prefer the exact assertion only if the implementation returns `bound_source`.
|
||||
|
||||
- [ ] **Step 2: Update `validate_deployment` return payload**
|
||||
|
||||
In `src/wf_mcp/workflow_surface/handlers.py`, replace the return body in `validate_deployment(...)` with:
|
||||
|
||||
```python
|
||||
status = "unrunnable" if diagnostics else "runnable"
|
||||
diagnostic_payloads = [
|
||||
diagnostic.model_dump(mode="json") for diagnostic in diagnostics
|
||||
]
|
||||
return {
|
||||
"deployment_id": deployment.id,
|
||||
"artifact_id": artifact.id,
|
||||
"artifact_version": artifact.version,
|
||||
"status": status,
|
||||
"diagnostics": diagnostic_payloads,
|
||||
"next_actions": NextActions.from_deployment_validation(
|
||||
deployment_id=deployment.id,
|
||||
diagnostics=diagnostics,
|
||||
).model_dump(mode="json"),
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run deployment tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_deployments.py tests/wf_mcp/workflow_surface/test_next_actions.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Thread NextActions Through Run Payloads
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_runs.py`
|
||||
|
||||
- [ ] **Step 1: Add completed-run assertions**
|
||||
|
||||
In `test_workflow_surface_runs_non_interrupting_deployment`, after current run payload assertions, add:
|
||||
|
||||
```python
|
||||
assert payload["next_actions"]["can_continue"] is False
|
||||
assert payload["next_actions"]["recommended_next_tool"] is None
|
||||
assert "completed" in payload["next_actions"]["reason"]
|
||||
```
|
||||
|
||||
After the `inspected` assertions in the same test, add:
|
||||
|
||||
```python
|
||||
assert inspected["next_actions"]["can_continue"] is False
|
||||
assert inspected["next_actions"]["recommended_next_tool"] is None
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add failed-run assertions**
|
||||
|
||||
In `test_workflow_surface_failed_deployment_exposes_error_on_run_and_inspect`, after current assertions, add:
|
||||
|
||||
```python
|
||||
assert payload["next_actions"]["recommended_next_tool"] is None
|
||||
assert "before producing trace" in payload["next_actions"]["reason"]
|
||||
assert inspected["next_actions"]["recommended_next_tool"] is None
|
||||
```
|
||||
|
||||
This failing artifact currently has `trace_count == 0`, so the safe guidance is diagnostics/error, not `read_run_trace`.
|
||||
|
||||
- [ ] **Step 3: Add trace-detail assertion**
|
||||
|
||||
In `test_workflow_surface_run_deployment_can_include_trace_detail`, after the current trace assertions, add:
|
||||
|
||||
```python
|
||||
assert payload["next_actions"]["patch_examples"][0]["request"]["trace_range"][
|
||||
"limit"
|
||||
] == 1
|
||||
```
|
||||
|
||||
This confirms completed runs may include bounded trace guidance without making it required.
|
||||
|
||||
- [ ] **Step 4: Update `_run_payload`**
|
||||
|
||||
In `src/wf_mcp/workflow_surface/handlers.py`, inside `_run_payload(...)`, add `next_actions` to the base `payload` dict:
|
||||
|
||||
```python
|
||||
"next_actions": NextActions.from_run_result(
|
||||
run_id=run_id,
|
||||
status=status,
|
||||
trace_count=trace_count,
|
||||
diagnostics=diagnostics or [],
|
||||
).model_dump(mode="json"),
|
||||
```
|
||||
|
||||
The resulting base payload should include:
|
||||
|
||||
```python
|
||||
payload = {
|
||||
"deployment_id": deployment.id,
|
||||
"artifact_id": artifact.id,
|
||||
"artifact_version": artifact.version,
|
||||
"status": status,
|
||||
"run_id": run_id,
|
||||
"resume_readiness": resume_readiness,
|
||||
"interrupt": interrupt,
|
||||
"outcome": outcome,
|
||||
"error": error,
|
||||
"output": output,
|
||||
"diagnostics": [
|
||||
diagnostic.model_dump(mode="json") for diagnostic in diagnostics or []
|
||||
],
|
||||
"trace_count": trace_count,
|
||||
"next_actions": NextActions.from_run_result(
|
||||
run_id=run_id,
|
||||
status=status,
|
||||
trace_count=trace_count,
|
||||
diagnostics=diagnostics or [],
|
||||
).model_dump(mode="json"),
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run run tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_runs.py tests/wf_mcp/workflow_surface/test_next_actions.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Update MCP Output Schema Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/server/test_config.py`
|
||||
- Modify: `tests/wf_mcp/server/test_tools.py`
|
||||
|
||||
- [ ] **Step 1: Add schema assertions for deployment/run outputs**
|
||||
|
||||
In `tests/wf_mcp/server/test_config.py`, find the existing workflow-surface schema test that checks `create_draft_workspace_from_capability` output contains `next_actions`.
|
||||
|
||||
Add assertions for `wf.workflow.validate_deployment` and `wf.workflow.run_deployment` output schemas. Use the same local `by_name` / tool lookup style already in that test file:
|
||||
|
||||
```python
|
||||
validate_deployment = by_name["wf.workflow.validate_deployment"]
|
||||
run_deployment = by_name["wf.workflow.run_deployment"]
|
||||
|
||||
validate_output = validate_deployment.outputSchema
|
||||
run_output = run_deployment.outputSchema
|
||||
|
||||
assert "next_actions" in validate_output["properties"]
|
||||
assert "recommended_next_tool" in validate_output["properties"]["next_actions"][
|
||||
"properties"
|
||||
]
|
||||
assert "next_actions" in run_output["properties"]
|
||||
assert "recommended_next_tool" in run_output["properties"]["next_actions"][
|
||||
"properties"
|
||||
]
|
||||
```
|
||||
|
||||
If this test file uses `tool.outputSchema` through dict access instead of attributes, follow the existing style in the file. Do not assert the whole schema.
|
||||
|
||||
- [ ] **Step 2: Add tool description assertion only if output schema exists**
|
||||
|
||||
In `tests/wf_mcp/server/test_tools.py`, add a small assertion that `run_deployment.description` or the output schema describes `next_actions` only if that file already inspects output schemas. Do not add fragile full-schema assertions.
|
||||
|
||||
If `test_tools.py` only checks input schemas and titles, skip this step.
|
||||
|
||||
- [ ] **Step 3: Run server tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/server/test_config.py tests/wf_mcp/server/test_tools.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Update Docs
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/workflow_capabilities.md`
|
||||
|
||||
- [ ] **Step 1: Add deployment/run guidance note**
|
||||
|
||||
Add this paragraph near the existing `next_actions` explanation:
|
||||
|
||||
```markdown
|
||||
Deployment validation and run lifecycle responses also expose `next_actions`.
|
||||
For runnable deployments this points to `wf.workflow.run_deployment`; for
|
||||
unrunnable deployments it points back to validation after the caller repairs
|
||||
bindings, sources, or schema drift. Failed runs never suggest reading an
|
||||
unbounded trace; trace guidance always uses a bounded `trace_range`.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run docs tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/server/test_docs.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused test set**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_deployments.py tests/wf_mcp/workflow_surface/test_runs.py tests/wf_mcp/server/test_config.py tests/wf_mcp/server/test_tools.py tests/wf_mcp/server/test_docs.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 2: Run formatting check on touched files**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff format --check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_deployments.py tests/wf_mcp/workflow_surface/test_runs.py tests/wf_mcp/server/test_config.py tests/wf_mcp/server/test_tools.py
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 3: Run lint on touched files**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface/test_deployments.py tests/wf_mcp/workflow_surface/test_runs.py tests/wf_mcp/server/test_config.py tests/wf_mcp/server/test_tools.py
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 4: Run type check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 5: Optional full suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite remains green with the existing skip/xfail count.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `next_actions` is present on `validate_deployment` responses.
|
||||
- `next_actions` is present on all `_run_payload(...)` responses.
|
||||
- Completed runs do not require another tool.
|
||||
- Failed runs recommend bounded trace only when a trace exists.
|
||||
- Interrupted runs recommend `wf.workflow.resume_run`.
|
||||
- Diagnostics remain authoritative; guidance only summarizes.
|
||||
- No unbounded trace read is suggested.
|
||||
- Existing fields remain unchanged.
|
||||
|
||||
## Notes For Opencode
|
||||
|
||||
- Keep this as an additive UX change.
|
||||
- Do not make any runtime/deployment behavior depend on `next_actions`.
|
||||
- Do not add a new tool.
|
||||
- Do not broaden trace payloads.
|
||||
- If server schema tests are awkward, prefer field-level assertions over whole-schema equality.
|
||||
@@ -0,0 +1,215 @@
|
||||
# Move RawWorkflowPlan to wf_api.models
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Extract `RawWorkflowPlan` from `wf_mcp.models` to canonical `wf_api.models`, keeping a compatibility shim in `wf_mcp.models`.
|
||||
|
||||
**Architecture:** `RawWorkflowPlan` is a standalone Pydantic model with no dependencies on other `wf_mcp.models` definitions. It depends only on `pydantic` and `wf_core` (Edge, InputBinding, Step). This makes it safe to move without entanglement. The `wf_mcp.models` module will re-export from `wf_api.models` as a shim.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, pytest, ruff, basedpyright
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Create `src/wf_api/models.py` with RawWorkflowPlan
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/models.py`
|
||||
|
||||
- [ ] **Step 1: Create the file with the model**
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
from wf_core import Edge
|
||||
from wf_core.models.steps import InputBinding, Step
|
||||
|
||||
|
||||
class RawWorkflowPlan(BaseModel):
|
||||
"""Raw authoring plan using the same graph step and edge models as core."""
|
||||
|
||||
name: str
|
||||
input_schema: dict[str, Any]
|
||||
state_schema: dict[str, Any]
|
||||
output_schema: dict[str, Any]
|
||||
outcomes: list[str] = Field(
|
||||
default_factory=lambda: ["ok"],
|
||||
description=(
|
||||
"Declared public workflow outcomes. Legacy plans without this field "
|
||||
"default to ok."
|
||||
),
|
||||
)
|
||||
output: list[InputBinding] = Field(
|
||||
default_factory=list,
|
||||
description=(
|
||||
"Optional root workflow output bindings. Sources read graph paths "
|
||||
"such as state.result and targets write the public output payload."
|
||||
),
|
||||
)
|
||||
start: str
|
||||
nodes: list[Step]
|
||||
edges: list[Edge]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify the file was created correctly**
|
||||
|
||||
Run: `python -c "from wf_api.models import RawWorkflowPlan; print(RawWorkflowPlan.__name__)"`
|
||||
Expected: `RawWorkflowPlan`
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Replace `wf_mcp.models.RawWorkflowPlan` definition with shim
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/models.py`
|
||||
|
||||
- [ ] **Step 1: Replace the RawWorkflowPlan class definition with a re-export**
|
||||
|
||||
Replace the `RawWorkflowPlan` class block (lines 45-68) with:
|
||||
|
||||
```python
|
||||
# RawWorkflowPlan moved to wf_api.models; re-exported here for backward compat.
|
||||
from wf_api.models import RawWorkflowPlan # noqa: F401
|
||||
```
|
||||
|
||||
Keep the original import block for pydantic, Edge, InputBinding, Step — they are still used indirectly via the re-export. Actually, after removing the class definition, `BaseModel`, `Field`, `Edge`, `InputBinding`, `Step` are no longer needed by this file. Remove those imports if no other class in the file uses them.
|
||||
|
||||
Check: The remaining classes in `wf_mcp/models.py` are `ConnectionConfig`, `AuthRecord`, `CatalogSnapshot`, `BrokerConfig`, `dump_catalog_snapshot`. These use `dataclass`, `field`, `Path`, `Any`, `CatalogNodeEntry`, `CatalogPromptEntry`, `CatalogResourceEntry`. They do NOT use `BaseModel`, `Field`, `Edge`, `InputBinding`, `Step`.
|
||||
|
||||
So remove: `from pydantic import BaseModel, Field`, `from wf_core import Edge`, `from wf_core.models.steps import InputBinding, Step`.
|
||||
|
||||
- [ ] **Step 2: Run ruff on the file**
|
||||
|
||||
Run: `uv run ruff check src/wf_mcp/models.py`
|
||||
Expected: no errors
|
||||
|
||||
- [ ] **Step 3: Run basedpyright on the file**
|
||||
|
||||
Run: `uv run basedpyright --level error src/wf_mcp/models.py`
|
||||
Expected: no errors
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Update `wf_mcp/__init__.py` to import from shim
|
||||
|
||||
**Files:**
|
||||
|
||||
- Verify: `src/wf_mcp/__init__.py`
|
||||
|
||||
No change needed — `wf_mcp/__init__.py` already imports `RawWorkflowPlan` from `.models`, and the shim re-exports it. Verify this still works.
|
||||
|
||||
- [ ] **Step 1: Verify the import chain works**
|
||||
|
||||
Run: `python -c "from wf_mcp import RawWorkflowPlan; print(RawWorkflowPlan.__name__)"`
|
||||
Expected: `RawWorkflowPlan`
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add focused tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_api/test_raw_workflow_plan_extraction.py`
|
||||
|
||||
- [ ] **Step 1: Write the tests**
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
def test_canonical_import_from_wf_api_models() -> None:
|
||||
from wf_api.models import RawWorkflowPlan
|
||||
|
||||
assert RawWorkflowPlan.__name__ == "RawWorkflowPlan"
|
||||
|
||||
|
||||
def test_compat_import_from_wf_mcp_models() -> None:
|
||||
from wf_mcp.models import RawWorkflowPlan as CompatPlan
|
||||
|
||||
assert CompatPlan.__name__ == "RawWorkflowPlan"
|
||||
|
||||
|
||||
def test_canonical_and_compat_are_identical() -> None:
|
||||
from wf_api.models import RawWorkflowPlan as Canonical
|
||||
from wf_mcp.models import RawWorkflowPlan as Compat
|
||||
|
||||
assert Canonical is Compat
|
||||
```
|
||||
|
||||
Note: The import direction rule is already covered by `tests/wf_api/test_import_direction.py::test_wf_api_has_no_wf_mcp_imports`. No need to duplicate.
|
||||
|
||||
- [ ] **Step 2: Run the new tests**
|
||||
|
||||
Run: `uv run pytest tests/wf_api/test_raw_workflow_plan_extraction.py -v`
|
||||
Expected: all 3 PASS
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Update test imports to use canonical path
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/service/conftest.py`
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_runs.py`
|
||||
|
||||
- [ ] **Step 1: Update `tests/wf_mcp/service/conftest.py`**
|
||||
|
||||
Change line 8 from:
|
||||
|
||||
```python
|
||||
from wf_mcp.models import AuthRecord, ConnectionConfig, RawWorkflowPlan
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
from wf_api.models import RawWorkflowPlan
|
||||
from wf_mcp.models import AuthRecord, ConnectionConfig
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `tests/wf_mcp/workflow_surface/test_runs.py`**
|
||||
|
||||
Change line 32 from:
|
||||
|
||||
```python
|
||||
from wf_mcp.models import RawWorkflowPlan
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
from wf_api.models import RawWorkflowPlan
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run ruff on touched files**
|
||||
|
||||
Run: `uv run ruff check tests/wf_mcp/service/conftest.py tests/wf_mcp/workflow_surface/test_runs.py`
|
||||
Expected: no errors
|
||||
|
||||
- [ ] **Step 4: Run basedpyright on touched files**
|
||||
|
||||
Run: `uv run basedpyright --level error tests/wf_mcp/service/conftest.py tests/wf_mcp/workflow_surface/test_runs.py`
|
||||
Expected: no errors
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Run full test suite and verify
|
||||
|
||||
- [ ] **Step 1: Run pytest**
|
||||
|
||||
Run: `uv run pytest -q`
|
||||
Expected: all tests pass
|
||||
|
||||
- [ ] **Step 2: Run ruff on all touched files**
|
||||
|
||||
Run: `uv run ruff check src/wf_api/models.py src/wf_mcp/models.py src/wf_mcp/__init__.py tests/wf_api/test_raw_workflow_plan_extraction.py tests/wf_mcp/service/conftest.py tests/wf_mcp/workflow_surface/test_runs.py`
|
||||
Expected: no errors
|
||||
|
||||
- [ ] **Step 3: Run basedpyright on touched files**
|
||||
|
||||
Run: `uv run basedpyright --level error src/wf_api/models.py src/wf_mcp/models.py`
|
||||
Expected: no errors
|
||||
@@ -0,0 +1,585 @@
|
||||
# wf_api Extraction Roadmap
|
||||
|
||||
> **For agentic workers:** This is an architecture roadmap, not a deterministic implementation checklist. Use it to choose and scope future focused implementation plans. Do not execute multiple slices at once.
|
||||
|
||||
**Goal:** Extract a protocol-neutral workflow application API from `wf_mcp` while preserving current process-local behavior and avoiding a large semantic rewrite.
|
||||
|
||||
> Current update: the original `WorkflowApiBackend` seam was useful for proving
|
||||
> dependency direction, but has been collapsed. `WorkflowApi` now composes
|
||||
> domain services directly from `WorkflowOperationContext`; MCP owns only
|
||||
> context construction and tool schemas.
|
||||
|
||||
**Architecture:** `wf_api` becomes the long-lived in-process application service layer. `wf_mcp`, `wf_cli`, and future HTTP/UI adapters call `wf_api`; `wf_api` must not import `wf_mcp`.
|
||||
|
||||
**Current State:** Slice 1 originally introduced `wf_api.WorkflowApi`, a
|
||||
`WorkflowApiBackend` protocol, and an MCP adapter backend. Later slices removed
|
||||
that double-delegation seam: `WorkflowApi` now composes domain services directly
|
||||
from `WorkflowOperationContext`. Both CLI and MCP workflow tools call
|
||||
`WorkflowApi`; `wf_api` imports no `wf_mcp` modules. `WorkflowSurfaceHandlers`
|
||||
is now a thin MCP compatibility subclass rather than the operation
|
||||
implementation.
|
||||
|
||||
Slice 3 moved the protocol-neutral workflow helpers into `wf_api`: constants,
|
||||
capability refs, wrapper hints, next actions, raw workflow plan model, runtime
|
||||
dependency resolution, saved subgraph preparation, and durable run lifecycle
|
||||
helpers. The old `wf_mcp.workflow_surface.*` module paths remain compatibility
|
||||
shims for those helpers.
|
||||
|
||||
**Current Constraint:** `WfMcpService` still acts as a compatibility facade over
|
||||
focused broker services. Recent slices have extracted source/catalog, runtime,
|
||||
upstream transport, events, connection sync, and content access; the remaining
|
||||
work is to keep shrinking facade responsibilities while preserving process-local
|
||||
behavior.
|
||||
|
||||
---
|
||||
|
||||
## Target Shape
|
||||
|
||||
```text
|
||||
wf_core = workflow execution kernel
|
||||
wf_authoring = Python authoring sugar and NodeSpec construction
|
||||
wf_artifacts = saved workflow/deployment/run models and stores
|
||||
wf_platform = source/capability/event platform primitives
|
||||
wf_api = process-local workflow application service/use cases
|
||||
wf_mcp = MCP adapter and MCP runtime/proxy/admin surface
|
||||
wf_cli = CLI adapter
|
||||
```
|
||||
|
||||
Desired dependency direction:
|
||||
|
||||
```text
|
||||
wf_cli ─┐
|
||||
├──> wf_api ───> wf_artifacts / wf_platform / wf_core / wf_authoring
|
||||
wf_mcp ─┘
|
||||
```
|
||||
|
||||
Forbidden dependency direction:
|
||||
|
||||
```text
|
||||
wf_api -> wf_mcp
|
||||
```
|
||||
|
||||
Process-local behavior remains the default:
|
||||
|
||||
```python
|
||||
config = load_broker_config(path)
|
||||
service = build_service_from_config(config)
|
||||
api = WorkflowApi(WfMcpWorkflowApiBackend(service))
|
||||
```
|
||||
|
||||
No FastAPI, daemon, socket, auth redesign, or network boundary is required for this extraction.
|
||||
|
||||
## Slice 1: Dependency Direction Cleanup
|
||||
|
||||
### Goal
|
||||
|
||||
Make both CLI and MCP call a protocol-neutral `WorkflowApi`, while `WorkflowApi`
|
||||
does not import `wf_mcp`.
|
||||
|
||||
### Allowed
|
||||
|
||||
- Create `src/wf_api/`.
|
||||
- Add `WorkflowApiBackend` protocol.
|
||||
- Add `WorkflowApi`.
|
||||
- Add `WfMcpWorkflowApiBackend` adapter around `WfMcpService`.
|
||||
- Update `wf_cli.context` to construct `WorkflowApi`.
|
||||
- Update MCP workflow tool registration to use `WorkflowApi`.
|
||||
- Keep compatibility aliases if existing imports need a transition.
|
||||
- Keep payloads and behavior unchanged.
|
||||
|
||||
### Not Allowed
|
||||
|
||||
- Do not split `WorkflowApi` by domain yet.
|
||||
- Do not move every helper module yet.
|
||||
- Do not rename events yet.
|
||||
- Do not redesign stores.
|
||||
- Do not add FastAPI.
|
||||
- Do not change response payloads.
|
||||
- Do not change command/tool names.
|
||||
|
||||
### Implemented Shape
|
||||
|
||||
```text
|
||||
src/wf_api/
|
||||
__init__.py
|
||||
backend.py # WorkflowApiBackend high-level operation protocol
|
||||
service.py # WorkflowApi thin delegating facade
|
||||
|
||||
src/wf_mcp/workflow_surface/
|
||||
handlers.py # existing implementation; now backend plumbing
|
||||
tools.py # MCP adapter; calls WorkflowApi
|
||||
|
||||
src/wf_mcp/broker/service/
|
||||
workflow_api_backend.py # WfMcpWorkflowApiBackend
|
||||
```
|
||||
|
||||
`WorkflowApiBackend` currently exposes high-level workflow operations that mirror
|
||||
the old workflow surface (`list_capabilities`, `create_draft_workspace`, `run_deployment`,
|
||||
and so on). That is intentionally not the final clean domain API. It keeps
|
||||
behavior and payloads stable while introducing the dependency seam. Later slices
|
||||
can replace selected `dict[str, Any]` method boundaries with stronger domain
|
||||
models after callers are routed through `WorkflowApi`.
|
||||
|
||||
Live source validation remains behind the MCP backend adapter because it touches
|
||||
MCP connections, adapters, and auth. `wf_api` owns the operation name, but the
|
||||
current backend owns the live-check implementation.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- `wf_api` imports no `wf_mcp` modules. **Done.**
|
||||
- `wf_cli` uses `WorkflowApi`. **Done.**
|
||||
- `wf_mcp.workflow_surface.tools` uses `WorkflowApi`. **Done.**
|
||||
- Existing MCP workflow-surface tests pass. **Done at implementation time.**
|
||||
- Existing CLI tests pass. **Done at implementation time.**
|
||||
- Behavior and payloads are unchanged. **Intended and guarded by tests.**
|
||||
|
||||
## Slice 2: Stabilize API Names And Compatibility Shims
|
||||
|
||||
### Goal
|
||||
|
||||
Make naming honest without breaking callers.
|
||||
|
||||
### Docs-First Current Slice
|
||||
|
||||
Before renames, document the new ownership:
|
||||
|
||||
- `wf_api.WorkflowApi` is the application-facing process-local API.
|
||||
- `wf_api.WorkflowApiBackend` is the high-level backend protocol.
|
||||
- `wf_mcp.broker.service.WfMcpWorkflowApiBackend` adapts the current MCP service
|
||||
stack into the backend protocol.
|
||||
- `wf_mcp.workflow_surface.WorkflowSurfaceHandlers` is legacy/internal
|
||||
implementation plumbing. New adapter code should not treat it as the canonical
|
||||
API.
|
||||
|
||||
### Later Likely Work
|
||||
|
||||
- Rename `WorkflowSurfaceHandlers` usage to `WorkflowApi` in tests and CLI code.
|
||||
- If a class rename is chosen, keep a temporary import shim:
|
||||
|
||||
```python
|
||||
from wf_api import WorkflowApi as WorkflowSurfaceHandlers
|
||||
```
|
||||
|
||||
- Update docs to say:
|
||||
|
||||
```text
|
||||
MCP tools and CLI commands are adapters over wf_api.
|
||||
```
|
||||
|
||||
- Rename test fixture helpers from `handlers(...)` to `api(...)` if useful.
|
||||
|
||||
### If/Then
|
||||
|
||||
- If shims create confusion, remove them quickly after imports are migrated.
|
||||
- If too many downstream imports still expect `WorkflowSurfaceHandlers`, keep the shim for one release/work session and document it as deprecated.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- New code imports `WorkflowApi`.
|
||||
- Old name remains only in compatibility modules or is gone.
|
||||
- Docs describe `wf_api` as the application service layer.
|
||||
|
||||
## Slice 3: Move Protocol-Neutral Workflow Surface Modules
|
||||
|
||||
### Goal
|
||||
|
||||
Move helper modules that are not MCP-specific out of `wf_mcp.workflow_surface`.
|
||||
|
||||
### Completed Moves
|
||||
|
||||
```text
|
||||
wf_mcp.workflow_surface.constants -> wf_api.constants
|
||||
wf_mcp.workflow_surface.refs -> wf_api.refs
|
||||
wf_mcp.workflow_surface.wrapper_hints -> wf_api.wrapper_hints
|
||||
wf_mcp.workflow_surface.next_actions -> wf_api.next_actions
|
||||
wf_mcp.models.RawWorkflowPlan -> wf_api.models.RawWorkflowPlan
|
||||
wf_mcp.workflow_surface.runtime_dependencies -> wf_api.runtime_dependencies
|
||||
wf_mcp.workflow_surface.saved_subgraphs -> wf_api.saved_subgraphs
|
||||
wf_mcp.workflow_surface.run_lifecycle -> wf_api.run_lifecycle
|
||||
```
|
||||
|
||||
The full `wf_mcp.workflow_surface.models` module did not move. It still holds
|
||||
MCP tool request/response schemas such as `TraceRange` and workflow tool result
|
||||
models. Move or split those only when the MCP schema boundary is clearer.
|
||||
|
||||
### If/Then
|
||||
|
||||
- If a module imports MCP connection/adapters/auth, do not move it in this slice.
|
||||
- If imports become circular, leave a shim in the old location and move one module at a time.
|
||||
- If a helper is really platform vocabulary, consider `wf_platform` instead of `wf_api`.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- `wf_api` owns protocol-neutral workflow API helpers. **Done.**
|
||||
- `wf_mcp.workflow_surface` keeps MCP adapter/schema code plus compatibility
|
||||
shims. **Mostly done.**
|
||||
- Tests still pass with import-only or near-import-only changes. **Done at
|
||||
implementation time.**
|
||||
|
||||
## Slice 4: Split The Big API By Domain
|
||||
|
||||
### Goal
|
||||
|
||||
Reduce the large `WorkflowApi` class after the package boundary is correct.
|
||||
|
||||
### Slice 4A: Operation Context Scaffolding
|
||||
|
||||
Do not move handler methods first. `WorkflowSurfaceHandlers` methods currently
|
||||
reach through `self.service` for stores, capability sources, events, live source
|
||||
calls, adapter lookup, and catalog helpers. Moving method bodies before defining
|
||||
that dependency surface would either make `wf_api` import `wf_mcp` or produce a
|
||||
fake split where every domain service still depends on the whole MCP service.
|
||||
|
||||
First introduce a small protocol-neutral operation context in `wf_api`:
|
||||
|
||||
```text
|
||||
src/wf_api/
|
||||
operation_context.py # protocols/dataclass for stores, sources, events, live calls
|
||||
```
|
||||
|
||||
The exact names may change, but the context should answer these questions:
|
||||
|
||||
- How does workflow API code access artifact, draft workspace, deployment, and run stores?
|
||||
- How does it read planner-visible `CapabilitySource` objects?
|
||||
- How does it record artifact/deployment/run lifecycle events without importing MCP event types?
|
||||
- How does it perform live source validation or live capability calls without importing MCP adapters/auth?
|
||||
- Which existing `WfMcpService` helpers are still required by moved domain methods?
|
||||
|
||||
`WfMcpWorkflowApiBackend` or another MCP-owned adapter can build this context
|
||||
from `WfMcpService`. The context is scaffolding only: Slice 4A should not move
|
||||
capability/draft/artifact/deployment/run method bodies yet and should not change
|
||||
public payloads.
|
||||
|
||||
### Candidate Shape
|
||||
|
||||
```text
|
||||
src/wf_api/
|
||||
service.py # facade that composes domain services
|
||||
operation_context.py
|
||||
capabilities.py
|
||||
drafts.py
|
||||
artifacts.py
|
||||
deployments.py
|
||||
runs.py
|
||||
```
|
||||
|
||||
Possible facade:
|
||||
|
||||
```python
|
||||
class WorkflowApi:
|
||||
capabilities: CapabilityApi
|
||||
drafts: DraftApi
|
||||
artifacts: ArtifactApi
|
||||
deployments: DeploymentApi
|
||||
runs: RunApi
|
||||
```
|
||||
|
||||
Compatibility can keep flat methods:
|
||||
|
||||
```python
|
||||
async def list_capabilities(...):
|
||||
return await self.capabilities.list_capabilities(...)
|
||||
```
|
||||
|
||||
### Planned Domain Split Order
|
||||
|
||||
After Slice 4A proves the operation-context seam, split method groups in small
|
||||
behavior-preserving slices:
|
||||
|
||||
#### Slice 4B: Drafts First
|
||||
|
||||
Move stateless draft and draft workspace operations first:
|
||||
|
||||
```text
|
||||
validate_draft
|
||||
compile_draft
|
||||
patch_draft
|
||||
list_draft_workspaces
|
||||
create_draft_workspace
|
||||
get_draft_workspace
|
||||
delete_draft_workspace
|
||||
validate_draft_workspace
|
||||
patch_draft_workspace
|
||||
set_draft_name
|
||||
set_draft_route
|
||||
set_step_input_map
|
||||
set_step_output_map
|
||||
create_minimal_draft_workspace
|
||||
```
|
||||
|
||||
Reason: drafts mostly use the draft workspace store, workflow draft compiler,
|
||||
wrapper hints, and deterministic patch helpers. They have the lowest live-source
|
||||
and durable-runtime coupling.
|
||||
|
||||
Leave `create_draft_workspace_from_capability` in the MCP-backed handler during
|
||||
4B. It depends on `inspect_capability`, wrapper hints, and capability source
|
||||
inspection, so it should move with either a small follow-up capability bootstrap
|
||||
slice or Slice 4E.
|
||||
|
||||
#### Slice 4C: Artifacts And Deployments
|
||||
|
||||
Move saved artifact and deployment operations next:
|
||||
|
||||
```text
|
||||
list_artifacts
|
||||
save_artifact
|
||||
create_artifact_from_plan
|
||||
create_artifact_from_draft
|
||||
create_artifact_from_workspace
|
||||
create_wrapper_from_workspace
|
||||
inspect_artifact
|
||||
list_deployments
|
||||
inspect_deployment
|
||||
save_deployment
|
||||
delete_deployment
|
||||
validate_deployment
|
||||
```
|
||||
|
||||
Reason: this group is store-heavy and introduces dependency validation, saved
|
||||
subgraph tree resolution, and event recording. It should move only after drafts
|
||||
prove the context seam.
|
||||
|
||||
#### Slice 4D: Runs
|
||||
|
||||
Move run lifecycle operations after artifacts/deployments:
|
||||
|
||||
```text
|
||||
run_deployment
|
||||
resume_run
|
||||
inspect_run
|
||||
read_run_trace
|
||||
```
|
||||
|
||||
Reason: runs are runtime-sensitive. They touch durable checkpoints, pinned
|
||||
dependency environments, resume readiness, prepared saved subgraphs, trace
|
||||
slicing, and compact next-action guidance. This should not be the first method
|
||||
move.
|
||||
|
||||
#### Slice 4E: Capabilities Last
|
||||
|
||||
Move workflow capability operations last:
|
||||
|
||||
```text
|
||||
list_capabilities
|
||||
inspect_capability
|
||||
call_capability
|
||||
create_draft_workspace_from_capability
|
||||
```
|
||||
|
||||
Reason: capabilities look simple but are the messiest boundary. They combine
|
||||
planner-visible source inventory, wrapper artifacts, direct wrapper calls,
|
||||
external live source calls, source visibility, and schema/wrapper hints.
|
||||
`create_draft_workspace_from_capability` also belongs here because it is driven
|
||||
by `inspect_capability` wrapper hints. Keep them in the MCP-backed
|
||||
implementation until the other domain services are stable.
|
||||
|
||||
#### After Slice 4E: Helper Promotion Cleanup
|
||||
|
||||
Once the handler is mostly a compatibility adapter, promote duplicated helper
|
||||
symbols into stable homes instead of leaving long-term cross-domain private
|
||||
imports:
|
||||
|
||||
```text
|
||||
raw_plan_from_artifact -> wf_api.artifact_plans or wf_artifacts
|
||||
artifact_capability_id -> wf_api artifact/capability refs helper
|
||||
available_sources_from_capability_sources -> wf_api source snapshot helper
|
||||
```
|
||||
|
||||
This should be a cleanup slice, not part of 4E unless required to avoid circular
|
||||
imports or behavior drift.
|
||||
|
||||
### If/Then
|
||||
|
||||
- If the context starts mirroring all of `WfMcpService`, stop and split it into
|
||||
smaller protocols rather than creating a new god object.
|
||||
- If callers benefit from flat methods, keep the facade flat and split internals only.
|
||||
- If domain APIs are clean enough, expose nested services later.
|
||||
- If a method spans domains, keep it in the facade until a better boundary appears.
|
||||
- If live source calls cannot be abstracted cleanly yet, leave capability
|
||||
calling in the MCP backend and move drafts/artifacts first.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- `wf_api` has an explicit operation context/protocol seam that imports no
|
||||
`wf_mcp` modules.
|
||||
- `WfMcpWorkflowApiBackend` can adapt `WfMcpService` into that seam.
|
||||
- No workflow method behavior changes in Slice 4A.
|
||||
- Each domain file is readable on its own.
|
||||
- Public payloads remain unchanged.
|
||||
- `wf_cli` and `wf_mcp` do not care about the internal split.
|
||||
|
||||
## Slice 5: Move Listing/Event Primitives To Better Homes
|
||||
|
||||
### Goal
|
||||
|
||||
Remove remaining protocol-neutral utilities from MCP-named packages.
|
||||
|
||||
### Current Recommendation
|
||||
|
||||
Split this into two different concerns. Listing/helper consolidation is
|
||||
behavior-preserving cleanup and should happen first. Event migration changes
|
||||
domain vocabulary and should remain separate until lifecycle event semantics are
|
||||
clearer.
|
||||
|
||||
### Slice 5A/5B: Listing And Workflow Helper Consolidation
|
||||
|
||||
Concrete plan:
|
||||
|
||||
```text
|
||||
docs/superpowers/plans/2026-06-02-wf-api-slice-5a-5b-helper-consolidation.md
|
||||
```
|
||||
|
||||
Planned moves:
|
||||
|
||||
```text
|
||||
wf_api.capabilities._matches_query -> wf_api.listing.matches_query
|
||||
wf_api.artifacts._matches_query -> wf_api.listing.matches_query
|
||||
wf_api.capabilities._paged_list_payload -> wf_api.listing.paged_list_payload
|
||||
wf_api.artifacts._paged_list_payload -> wf_api.listing.paged_list_payload
|
||||
wf_mcp.workflow_surface.handlers fallback -> wf_api.listing.paged_list_payload
|
||||
|
||||
wf_api.runs._raw_plan_from_artifact -> wf_api.artifact_plans.raw_plan_from_artifact
|
||||
wf_api.capabilities._raw_plan_from_artifact -> wf_api.artifact_plans.raw_plan_from_artifact
|
||||
wf_api.capabilities._artifact_capability_id -> wf_api.artifact_refs.artifact_capability_id
|
||||
wf_api.artifacts._artifact_capability_id -> wf_api.artifact_refs.artifact_capability_id
|
||||
wf_api.{drafts,artifacts,capabilities} requirement helpers
|
||||
-> wf_api.capability_requirements
|
||||
```
|
||||
|
||||
Do not move `wf_mcp.shared.pagination` in this slice. It is still used by proxy
|
||||
tool search/listing code, so treating it as dead workflow-surface debt would be
|
||||
incorrect.
|
||||
|
||||
### Post-5 Helper Cleanup: Workflow Surface Test Thinning
|
||||
|
||||
Concrete plan:
|
||||
|
||||
```text
|
||||
docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning.md
|
||||
```
|
||||
|
||||
Intent:
|
||||
|
||||
```text
|
||||
wf_api tests = canonical application behavior tests
|
||||
wf_mcp.workflow_surface tests = adapter/schema/live-source/integration smoke tests
|
||||
```
|
||||
|
||||
Do not replace stronger workflow-surface integration tests with weaker unit
|
||||
tests. Only remove a handler test when an equal-or-stronger `wf_api` test exists
|
||||
and at least one handler-level smoke/delegation test still protects the adapter
|
||||
path.
|
||||
|
||||
### Candidate Moves
|
||||
|
||||
```text
|
||||
wf_mcp.shared.listing.matches_query -> wf_platform.listing or wf_api.listing
|
||||
wf_mcp.shared.listing.paged_list_payload -> wf_platform.listing or wf_api.listing
|
||||
wf_mcp.events.McpEvent -> wf_platform.events.DomainEvent
|
||||
wf_mcp.events.EventBus -> wf_platform.events.EventBus
|
||||
```
|
||||
|
||||
### If/Then
|
||||
|
||||
- If events are only used by MCP admin/proxy code, leave them in `wf_mcp`.
|
||||
- If events describe artifact/deployment/run lifecycle, move or fork them into `wf_platform`.
|
||||
- If renaming `McpEvent` causes churn, introduce `DomainEvent` first and keep `McpEvent` as an alias temporarily.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- Workflow API lifecycle events no longer require MCP naming.
|
||||
- Listing helpers used by CLI/API are not imported from `wf_mcp`.
|
||||
|
||||
## Slice 6: Store Construction And Config Boundary
|
||||
|
||||
### Goal
|
||||
|
||||
Make store/runtime construction less MCP-owned.
|
||||
|
||||
### Current Problem
|
||||
|
||||
`build_service_from_config` and `WfMcpService.__post_init__` currently build or own protocol-neutral stores. CLI uses this path because it is pragmatic, but long term config/store construction should be reusable without MCP server assumptions.
|
||||
|
||||
### Candidate Work
|
||||
|
||||
- Extract config-to-store construction into a neutral builder.
|
||||
- Keep MCP connection config in `wf_mcp`.
|
||||
- Let `wf_cli` and future API/server adapters reuse the neutral store builder.
|
||||
|
||||
### If/Then
|
||||
|
||||
- If extraction creates too many config model moves, defer it.
|
||||
- If CLI needs only current config behavior, keep using `wf_mcp` builder until FastAPI/UI pressure appears.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- Artifact/draft/run stores can be constructed without starting MCP concepts.
|
||||
- Current `wf_mcp.config.json` remains supported.
|
||||
|
||||
## Slice 7: Future HTTP/FastAPI Adapter
|
||||
|
||||
### Goal
|
||||
|
||||
Expose `WorkflowApi` over HTTP only after the in-process API is stable.
|
||||
|
||||
### Not Yet
|
||||
|
||||
Do not start this until:
|
||||
|
||||
- `wf_api` exists.
|
||||
- CLI and MCP both call `wf_api`.
|
||||
- Run/draft/deployment payloads are stable enough.
|
||||
- Auth and multi-client lifecycle questions are explicit.
|
||||
|
||||
### Future Shape
|
||||
|
||||
```text
|
||||
wf_http or wf_server
|
||||
routes/
|
||||
capabilities.py
|
||||
drafts.py
|
||||
artifacts.py
|
||||
deployments.py
|
||||
runs.py
|
||||
```
|
||||
|
||||
Routes should be thin:
|
||||
|
||||
```python
|
||||
@router.post("/runs")
|
||||
async def start_run(...):
|
||||
return await api.run_deployment(...)
|
||||
```
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- HTTP is an adapter, not a new source of workflow logic.
|
||||
- Process-local API remains usable without HTTP.
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Should `WorkflowApiBackend` be one protocol or several domain protocols?
|
||||
- Recommendation for Slice 1: one protocol. Split later only if it hurts.
|
||||
|
||||
2. Should live source validation live in `wf_api`?
|
||||
- Recommendation: `wf_api` owns the operation, backend owns the live-check implementation.
|
||||
|
||||
3. Should `wf_api` expose flat methods or nested services?
|
||||
- Recommendation for Slice 1: flat methods for compatibility. Consider nested internals later.
|
||||
|
||||
4. Should `WorkflowSurfaceHandlers` disappear immediately?
|
||||
- Recommendation: no. Keep a short-lived shim if it reduces churn.
|
||||
|
||||
5. Should `wf_cli` stop importing `wf_mcp` after Slice 1?
|
||||
- Not fully. It may still use `wf_mcp` config/service construction until store/config extraction happens.
|
||||
|
||||
## Immediate Next Plan Status
|
||||
|
||||
Slice 1 implementation plan exists and was executed:
|
||||
|
||||
```text
|
||||
docs/superpowers/plans/2026-06-01-wf-api-slice-1-dependency-direction.md
|
||||
```
|
||||
|
||||
The next implementation plan should cover Slice 2 only. Prefer docs and
|
||||
compatibility naming first; do not move helper modules until Slice 3.
|
||||
+1500
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,503 @@
|
||||
# wf_api Slice 3A: Refs And Constants Move Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Move the protocol-neutral workflow API refs and constants helpers from `wf_mcp.workflow_surface` into `wf_api`, while preserving old imports as compatibility shims.
|
||||
|
||||
**Architecture:** `wf_api` is now the canonical home for process-local workflow API helpers that are not MCP-specific. This slice moves only `constants.py` and `refs.py` because both are small and import only `wf_artifacts` / `wf_platform`. `wf_mcp.workflow_surface.constants` and `wf_mcp.workflow_surface.refs` remain thin re-export shims so existing imports keep working.
|
||||
|
||||
**Tech Stack:** Python 3.14+, Pydantic-backed refs from `wf_artifacts` and `wf_platform`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
- Create `src/wf_api/constants.py`.
|
||||
- Create `src/wf_api/refs.py`.
|
||||
- Re-export the new helpers from `src/wf_api/__init__.py`.
|
||||
- Replace `wf_mcp.workflow_surface.constants` with a compatibility shim.
|
||||
- Replace `wf_mcp.workflow_surface.refs` with a compatibility shim.
|
||||
- Update `src/wf_mcp/workflow_surface/handlers.py` to import canonical helpers from `wf_api`.
|
||||
- Add/adjust tests for canonical imports and shim compatibility.
|
||||
- Keep `wf_api` free of `wf_mcp` imports.
|
||||
|
||||
### Out Of Scope
|
||||
|
||||
- Do not move `models.py`.
|
||||
- Do not move `next_actions.py`.
|
||||
- Do not move `wrapper_hints.py`.
|
||||
- Do not move `run_lifecycle.py`.
|
||||
- Do not move `runtime_dependencies.py`.
|
||||
- Do not move `saved_subgraphs.py`.
|
||||
- Do not rename `WorkflowSurfaceHandlers`.
|
||||
- Do not change public payloads, tool names, command names, or parsing behavior.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
### New Canonical Files
|
||||
|
||||
| File | Responsibility |
|
||||
| --- | --- |
|
||||
| `src/wf_api/constants.py` | Canonical workflow API literals used by draft/helper code. |
|
||||
| `src/wf_api/refs.py` | Canonical parser for workflow-surface capability IDs. |
|
||||
|
||||
### Compatibility Shims
|
||||
|
||||
| File | Responsibility |
|
||||
| --- | --- |
|
||||
| `src/wf_mcp/workflow_surface/constants.py` | Re-export constants from `wf_api.constants`; no local logic. |
|
||||
| `src/wf_mcp/workflow_surface/refs.py` | Re-export refs from `wf_api.refs`; no local logic. |
|
||||
|
||||
### Modified Consumers
|
||||
|
||||
| File | Change |
|
||||
| --- | --- |
|
||||
| `src/wf_api/__init__.py` | Re-export moved helpers. |
|
||||
| `src/wf_mcp/workflow_surface/handlers.py` | Import constants and parser from `wf_api`. |
|
||||
| `tests/wf_mcp/test_workflow_surface_refs.py` | Keep shim compatibility tests and add canonical import tests. |
|
||||
| `tests/wf_api/test_import_direction.py` | Existing guard should continue to pass. |
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add Canonical `wf_api.constants`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/constants.py`
|
||||
|
||||
- [ ] **Step 1: Create `src/wf_api/constants.py`**
|
||||
|
||||
Write this exact file:
|
||||
|
||||
```python
|
||||
"""Protocol-neutral workflow API literals used by draft/helper code."""
|
||||
|
||||
DEFAULT_CALL_STEP_ID = "call"
|
||||
DEFAULT_ERROR_STEP_ID = "tool_error"
|
||||
DEFAULT_OK_OUTCOME = "ok"
|
||||
DEFAULT_ERROR_OUTCOME = "error"
|
||||
RUNTIME_ERROR_CAPABILITY = "wf.std.runtime_error"
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_CALL_STEP_ID",
|
||||
"DEFAULT_ERROR_OUTCOME",
|
||||
"DEFAULT_ERROR_STEP_ID",
|
||||
"DEFAULT_OK_OUTCOME",
|
||||
"RUNTIME_ERROR_CAPABILITY",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run import smoke check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_api.constants import DEFAULT_CALL_STEP_ID, RUNTIME_ERROR_CAPABILITY; print(DEFAULT_CALL_STEP_ID, RUNTIME_ERROR_CAPABILITY)"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
call wf.std.runtime_error
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Add Canonical `wf_api.refs`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/refs.py`
|
||||
|
||||
- [ ] **Step 1: Create `src/wf_api/refs.py`**
|
||||
|
||||
Write this exact file:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, TypeAlias
|
||||
|
||||
from wf_artifacts import WorkflowCapabilityRef
|
||||
from wf_platform import CapabilityRef
|
||||
|
||||
WorkflowSurfaceCapabilityId: TypeAlias = CapabilityRef | WorkflowCapabilityRef
|
||||
|
||||
|
||||
def parse_workflow_surface_capability_id(
|
||||
value: str | dict[str, Any],
|
||||
) -> WorkflowSurfaceCapabilityId:
|
||||
"""Parse a workflow API capability id into its real domain ref.
|
||||
|
||||
API callers still pass strings at protocol boundaries. Internally,
|
||||
workflow-facing capability ids are either live source capabilities or saved
|
||||
wrapper artifacts, so this parser avoids inventing a third identifier model.
|
||||
"""
|
||||
if isinstance(value, dict):
|
||||
if "artifact_id" in value and "version" in value:
|
||||
return WorkflowCapabilityRef._validate(value)
|
||||
return CapabilityRef._validate(value)
|
||||
|
||||
try:
|
||||
return WorkflowCapabilityRef.parse(value)
|
||||
except ValueError:
|
||||
return CapabilityRef.parse(value)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run import smoke check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_api.refs import parse_workflow_surface_capability_id; print(parse_workflow_surface_capability_id('workflow.echo_wrapper.v2'))"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
workflow.echo_wrapper.v2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Re-export Helpers From `wf_api`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
|
||||
- [ ] **Step 1: Update imports and `__all__`**
|
||||
|
||||
Change `src/wf_api/__init__.py` to include these imports:
|
||||
|
||||
```python
|
||||
from .constants import (
|
||||
DEFAULT_CALL_STEP_ID,
|
||||
DEFAULT_ERROR_OUTCOME,
|
||||
DEFAULT_ERROR_STEP_ID,
|
||||
DEFAULT_OK_OUTCOME,
|
||||
RUNTIME_ERROR_CAPABILITY,
|
||||
)
|
||||
from .refs import WorkflowSurfaceCapabilityId, parse_workflow_surface_capability_id
|
||||
```
|
||||
|
||||
Ensure `__all__` includes:
|
||||
|
||||
```python
|
||||
__all__ = [
|
||||
"DEFAULT_CALL_STEP_ID",
|
||||
"DEFAULT_ERROR_OUTCOME",
|
||||
"DEFAULT_ERROR_STEP_ID",
|
||||
"DEFAULT_OK_OUTCOME",
|
||||
"RUNTIME_ERROR_CAPABILITY",
|
||||
"TraceRange",
|
||||
"WorkflowApi",
|
||||
"WorkflowApiBackend",
|
||||
"WorkflowSurfaceCapabilityId",
|
||||
"parse_workflow_surface_capability_id",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run import smoke check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_api import DEFAULT_OK_OUTCOME, parse_workflow_surface_capability_id; print(DEFAULT_OK_OUTCOME, parse_workflow_surface_capability_id('demo.personal.echo_tool'))"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
ok demo.personal.echo_tool
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Convert Old Workflow-Surface Modules To Shims
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/constants.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/refs.py`
|
||||
|
||||
- [ ] **Step 1: Replace `src/wf_mcp/workflow_surface/constants.py`**
|
||||
|
||||
Replace the file with this shim:
|
||||
|
||||
```python
|
||||
"""Compatibility shim for workflow API constants.
|
||||
|
||||
New code should import these literals from `wf_api.constants`. This module stays
|
||||
so older MCP workflow-surface imports keep working during extraction.
|
||||
"""
|
||||
|
||||
from wf_api.constants import (
|
||||
DEFAULT_CALL_STEP_ID,
|
||||
DEFAULT_ERROR_OUTCOME,
|
||||
DEFAULT_ERROR_STEP_ID,
|
||||
DEFAULT_OK_OUTCOME,
|
||||
RUNTIME_ERROR_CAPABILITY,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_CALL_STEP_ID",
|
||||
"DEFAULT_ERROR_OUTCOME",
|
||||
"DEFAULT_ERROR_STEP_ID",
|
||||
"DEFAULT_OK_OUTCOME",
|
||||
"RUNTIME_ERROR_CAPABILITY",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Replace `src/wf_mcp/workflow_surface/refs.py`**
|
||||
|
||||
Replace the file with this shim:
|
||||
|
||||
```python
|
||||
"""Compatibility shim for workflow API capability refs.
|
||||
|
||||
New code should import from `wf_api.refs`. This module stays so older MCP
|
||||
workflow-surface imports keep working during extraction.
|
||||
"""
|
||||
|
||||
from wf_api.refs import (
|
||||
WorkflowSurfaceCapabilityId,
|
||||
parse_workflow_surface_capability_id,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"WorkflowSurfaceCapabilityId",
|
||||
"parse_workflow_surface_capability_id",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run shim import smoke check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_mcp.workflow_surface.constants import DEFAULT_CALL_STEP_ID; from wf_mcp.workflow_surface.refs import parse_workflow_surface_capability_id; print(DEFAULT_CALL_STEP_ID, parse_workflow_surface_capability_id('workflow.echo_wrapper.v2'))"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
call workflow.echo_wrapper.v2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Update Canonical Imports In `WorkflowSurfaceHandlers`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
|
||||
- [ ] **Step 1: Change constants import**
|
||||
|
||||
Replace:
|
||||
|
||||
```python
|
||||
from .constants import (
|
||||
DEFAULT_CALL_STEP_ID,
|
||||
DEFAULT_ERROR_OUTCOME,
|
||||
DEFAULT_ERROR_STEP_ID,
|
||||
DEFAULT_OK_OUTCOME,
|
||||
RUNTIME_ERROR_CAPABILITY,
|
||||
)
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
from wf_api.constants import (
|
||||
DEFAULT_CALL_STEP_ID,
|
||||
DEFAULT_ERROR_OUTCOME,
|
||||
DEFAULT_ERROR_STEP_ID,
|
||||
DEFAULT_OK_OUTCOME,
|
||||
RUNTIME_ERROR_CAPABILITY,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Change refs import**
|
||||
|
||||
Replace:
|
||||
|
||||
```python
|
||||
from .refs import parse_workflow_surface_capability_id
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
from wf_api.refs import parse_workflow_surface_capability_id
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run import smoke check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_mcp.workflow_surface.handlers import WorkflowSurfaceHandlers; print(WorkflowSurfaceHandlers.__name__)"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
WorkflowSurfaceHandlers
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Update Ref Tests For Canonical And Shim Imports
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/test_workflow_surface_refs.py`
|
||||
|
||||
- [ ] **Step 1: Update imports**
|
||||
|
||||
Change the top imports to:
|
||||
|
||||
```python
|
||||
from wf_api.refs import parse_workflow_surface_capability_id
|
||||
from wf_artifacts import WorkflowCapabilityRef
|
||||
from wf_mcp.workflow_surface.refs import (
|
||||
parse_workflow_surface_capability_id as parse_workflow_surface_capability_id_shim,
|
||||
)
|
||||
from wf_platform import CapabilityRef
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add shim compatibility test**
|
||||
|
||||
Append this test to the file:
|
||||
|
||||
```python
|
||||
def test_workflow_surface_refs_shim_reexports_canonical_parser() -> None:
|
||||
assert parse_workflow_surface_capability_id_shim is parse_workflow_surface_capability_id
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add constants shim compatibility test**
|
||||
|
||||
Append this test to the file:
|
||||
|
||||
```python
|
||||
def test_workflow_surface_constants_shim_reexports_canonical_literals() -> None:
|
||||
from wf_api.constants import DEFAULT_CALL_STEP_ID
|
||||
from wf_mcp.workflow_surface.constants import (
|
||||
DEFAULT_CALL_STEP_ID as DEFAULT_CALL_STEP_ID_SHIM,
|
||||
)
|
||||
|
||||
assert DEFAULT_CALL_STEP_ID_SHIM == DEFAULT_CALL_STEP_ID
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/test_workflow_surface_refs.py tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Search For Remaining Canonical Import Opportunities
|
||||
|
||||
**Files:**
|
||||
|
||||
- Inspect only unless the search finds new low-risk direct consumers.
|
||||
|
||||
- [ ] **Step 1: Search old imports**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "from \\.constants|from \\.refs|from wf_mcp\\.workflow_surface\\.(constants|refs)" src tests
|
||||
```
|
||||
|
||||
Expected remaining matches:
|
||||
|
||||
```text
|
||||
src/wf_mcp/workflow_surface/constants.py
|
||||
src/wf_mcp/workflow_surface/refs.py
|
||||
tests/wf_mcp/test_workflow_surface_refs.py
|
||||
```
|
||||
|
||||
If any other production module imports the old paths, update it to import from
|
||||
`wf_api.constants` or `wf_api.refs`.
|
||||
|
||||
- [ ] **Step 2: Search for accidental `wf_api -> wf_mcp` imports**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "wf_mcp" src/wf_api tests/wf_api
|
||||
```
|
||||
|
||||
Expected matches only in test text/docstrings for the import-direction guard,
|
||||
not in `src/wf_api/*.py`.
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Verification
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api tests/wf_mcp/test_workflow_surface_refs.py tests/wf_mcp/workflow_surface -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 2: Run CLI context smoke tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_cli/test_context.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run ruff on touched files**
|
||||
|
||||
```powershell
|
||||
uv run ruff check src/wf_api src/wf_mcp/workflow_surface/constants.py src/wf_mcp/workflow_surface/refs.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/test_workflow_surface_refs.py tests/wf_api
|
||||
```
|
||||
|
||||
Expected: all checks pass.
|
||||
|
||||
- [ ] **Step 4: Run basedpyright on touched files**
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error src/wf_api src/wf_mcp/workflow_surface/constants.py src/wf_mcp/workflow_surface/refs.py src/wf_mcp/workflow_surface/handlers.py tests/wf_mcp/test_workflow_surface_refs.py tests/wf_api
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 5: Optional full suite**
|
||||
|
||||
Run this if time allows:
|
||||
|
||||
```powershell
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite passes with the project’s existing skipped/xfailed counts.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `wf_api.constants` imports no `wf_mcp`.
|
||||
- `wf_api.refs` imports no `wf_mcp`.
|
||||
- Old `wf_mcp.workflow_surface.constants` import path still works.
|
||||
- Old `wf_mcp.workflow_surface.refs` import path still works.
|
||||
- `WorkflowSurfaceHandlers` imports the canonical `wf_api` helpers.
|
||||
- No public payload shape changed.
|
||||
- No behavior changed.
|
||||
- No other workflow-surface helper moved in this slice.
|
||||
@@ -0,0 +1,548 @@
|
||||
# wf_api Slice 3B: Guidance Helpers Move Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Move protocol-neutral wrapper guidance helpers from `wf_mcp.workflow_surface` into `wf_api`, while preserving old imports as compatibility shims.
|
||||
|
||||
**Architecture:** `wrapper_hints.py` and `next_actions.py` are coupled guidance helpers: `next_actions` imports `WrapperAuthoringHints`, and both describe workflow authoring UX rather than MCP transport behavior. This slice moves them together to avoid a half-moved dependency. The old `wf_mcp.workflow_surface` modules remain thin re-export shims so MCP schemas/tests and older imports keep working.
|
||||
|
||||
**Tech Stack:** Python 3.14+, Pydantic v2 models, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
- Create `src/wf_api/wrapper_hints.py`.
|
||||
- Create `src/wf_api/next_actions.py`.
|
||||
- Re-export selected public guidance types/functions from `src/wf_api/__init__.py`.
|
||||
- Replace `wf_mcp.workflow_surface.wrapper_hints` with a compatibility shim.
|
||||
- Replace `wf_mcp.workflow_surface.next_actions` with a compatibility shim.
|
||||
- Update `src/wf_mcp/workflow_surface/handlers.py` to import canonical helpers from `wf_api`.
|
||||
- Update `src/wf_mcp/workflow_surface/models.py` to import canonical next-action models from `wf_api`.
|
||||
- Update direct tests to use canonical imports while preserving shim compatibility tests.
|
||||
- Keep `wf_api` free of `wf_mcp` imports.
|
||||
|
||||
### Out Of Scope
|
||||
|
||||
- Do not move `models.py`.
|
||||
- Do not move `run_lifecycle.py`.
|
||||
- Do not move `runtime_dependencies.py`.
|
||||
- Do not move `saved_subgraphs.py`.
|
||||
- Do not rename `WorkflowSurfaceHandlers`.
|
||||
- Do not change wrapper hint behavior.
|
||||
- Do not change next action behavior.
|
||||
- Do not change public payloads, MCP tool names, CLI command names, or JSON schema field names.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
### New Canonical Files
|
||||
|
||||
| File | Responsibility |
|
||||
| --- | --- |
|
||||
| `src/wf_api/wrapper_hints.py` | Wrapper scaffolding hints, confidence, missing-decision models, and conservative schema mapping helper. |
|
||||
| `src/wf_api/next_actions.py` | Advisory next-action models and factory helpers for wrapper/deployment/run responses. |
|
||||
|
||||
### Compatibility Shims
|
||||
|
||||
| File | Responsibility |
|
||||
| --- | --- |
|
||||
| `src/wf_mcp/workflow_surface/wrapper_hints.py` | Re-export wrapper hint helpers from `wf_api.wrapper_hints`. |
|
||||
| `src/wf_mcp/workflow_surface/next_actions.py` | Re-export next action helpers from `wf_api.next_actions`. |
|
||||
|
||||
### Modified Consumers
|
||||
|
||||
| File | Change |
|
||||
| --- | --- |
|
||||
| `src/wf_api/__init__.py` | Re-export public guidance helpers. |
|
||||
| `src/wf_mcp/workflow_surface/handlers.py` | Import `NextActions` and wrapper hint helpers from `wf_api`. |
|
||||
| `src/wf_mcp/workflow_surface/models.py` | Import `NextActionPatchExample` and `NextActions` from `wf_api.next_actions`. |
|
||||
| `tests/wf_mcp/test_workflow_wrapper_hints.py` | Use canonical `wf_api.wrapper_hints`; add shim identity test. |
|
||||
| `tests/wf_mcp/workflow_surface/test_next_actions.py` | Use canonical `wf_api.next_actions`; add shim identity test. |
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Create Canonical `wf_api.wrapper_hints`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/wrapper_hints.py`
|
||||
|
||||
- [ ] **Step 1: Copy existing implementation**
|
||||
|
||||
Create `src/wf_api/wrapper_hints.py` by copying the complete current contents of:
|
||||
|
||||
```text
|
||||
src/wf_mcp/workflow_surface/wrapper_hints.py
|
||||
```
|
||||
|
||||
Do not change behavior. The copied module must not import `wf_mcp`.
|
||||
|
||||
- [ ] **Step 2: Add `__all__` at the end**
|
||||
|
||||
Append this block to the copied file:
|
||||
|
||||
```python
|
||||
__all__ = [
|
||||
"MissingDecision",
|
||||
"MissingDecisionKind",
|
||||
"OutcomeCandidate",
|
||||
"OutcomeCandidateKind",
|
||||
"WrapperAuthoringHints",
|
||||
"WrapperHintConfidence",
|
||||
"WrapperOutcomePolicy",
|
||||
"workflow_output_schema_for_authoring",
|
||||
"wrapper_hints_for_capability",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run import smoke check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_api.wrapper_hints import WrapperAuthoringHints, wrapper_hints_for_capability; print(WrapperAuthoringHints.__name__, wrapper_hints_for_capability.__name__)"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
WrapperAuthoringHints wrapper_hints_for_capability
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Create Canonical `wf_api.next_actions`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/next_actions.py`
|
||||
|
||||
- [ ] **Step 1: Copy existing implementation**
|
||||
|
||||
Create `src/wf_api/next_actions.py` by copying the complete current contents of:
|
||||
|
||||
```text
|
||||
src/wf_mcp/workflow_surface/next_actions.py
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Keep local wrapper hint import canonical**
|
||||
|
||||
Ensure the copied file imports wrapper hints from the new `wf_api` package via:
|
||||
|
||||
```python
|
||||
from .wrapper_hints import WrapperAuthoringHints
|
||||
```
|
||||
|
||||
The copied module must not import `wf_mcp`.
|
||||
|
||||
- [ ] **Step 3: Add `__all__` at the end**
|
||||
|
||||
Append this block to the copied file:
|
||||
|
||||
```python
|
||||
__all__ = [
|
||||
"NextActionPatchExample",
|
||||
"NextActionTool",
|
||||
"NextActions",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run import smoke check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_api.next_actions import NextActionTool, NextActions; print(NextActionTool.RUN_DEPLOYMENT, NextActions.__name__)"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
wf.workflow.run_deployment NextActions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Re-export Guidance Helpers From `wf_api`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
|
||||
- [ ] **Step 1: Add imports**
|
||||
|
||||
Add these imports:
|
||||
|
||||
```python
|
||||
from .next_actions import NextActionPatchExample, NextActionTool, NextActions
|
||||
from .wrapper_hints import (
|
||||
MissingDecision,
|
||||
MissingDecisionKind,
|
||||
OutcomeCandidate,
|
||||
OutcomeCandidateKind,
|
||||
WrapperAuthoringHints,
|
||||
WrapperHintConfidence,
|
||||
WrapperOutcomePolicy,
|
||||
workflow_output_schema_for_authoring,
|
||||
wrapper_hints_for_capability,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add names to `__all__`**
|
||||
|
||||
Ensure `__all__` includes these names:
|
||||
|
||||
```python
|
||||
"MissingDecision",
|
||||
"MissingDecisionKind",
|
||||
"NextActionPatchExample",
|
||||
"NextActionTool",
|
||||
"NextActions",
|
||||
"OutcomeCandidate",
|
||||
"OutcomeCandidateKind",
|
||||
"WrapperAuthoringHints",
|
||||
"WrapperHintConfidence",
|
||||
"WrapperOutcomePolicy",
|
||||
"workflow_output_schema_for_authoring",
|
||||
"wrapper_hints_for_capability",
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run top-level import smoke check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_api import NextActions, WrapperAuthoringHints; print(NextActions.__name__, WrapperAuthoringHints.__name__)"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
NextActions WrapperAuthoringHints
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Convert Old Workflow-Surface Guidance Modules To Shims
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/wrapper_hints.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/next_actions.py`
|
||||
|
||||
- [ ] **Step 1: Replace `src/wf_mcp/workflow_surface/wrapper_hints.py`**
|
||||
|
||||
Replace the file with this shim:
|
||||
|
||||
```python
|
||||
"""Compatibility shim for workflow API wrapper authoring hints.
|
||||
|
||||
New code should import from `wf_api.wrapper_hints`. This module stays so older
|
||||
MCP workflow-surface imports keep working during extraction.
|
||||
"""
|
||||
|
||||
from wf_api.wrapper_hints import (
|
||||
MissingDecision,
|
||||
MissingDecisionKind,
|
||||
OutcomeCandidate,
|
||||
OutcomeCandidateKind,
|
||||
WrapperAuthoringHints,
|
||||
WrapperHintConfidence,
|
||||
WrapperOutcomePolicy,
|
||||
workflow_output_schema_for_authoring,
|
||||
wrapper_hints_for_capability,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"MissingDecision",
|
||||
"MissingDecisionKind",
|
||||
"OutcomeCandidate",
|
||||
"OutcomeCandidateKind",
|
||||
"WrapperAuthoringHints",
|
||||
"WrapperHintConfidence",
|
||||
"WrapperOutcomePolicy",
|
||||
"workflow_output_schema_for_authoring",
|
||||
"wrapper_hints_for_capability",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Replace `src/wf_mcp/workflow_surface/next_actions.py`**
|
||||
|
||||
Replace the file with this shim:
|
||||
|
||||
```python
|
||||
"""Compatibility shim for workflow API next-action guidance.
|
||||
|
||||
New code should import from `wf_api.next_actions`. This module stays so older
|
||||
MCP workflow-surface imports keep working during extraction.
|
||||
"""
|
||||
|
||||
from wf_api.next_actions import NextActionPatchExample, NextActionTool, NextActions
|
||||
|
||||
__all__ = [
|
||||
"NextActionPatchExample",
|
||||
"NextActionTool",
|
||||
"NextActions",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run shim import smoke check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_mcp.workflow_surface.wrapper_hints import WrapperAuthoringHints; from wf_mcp.workflow_surface.next_actions import NextActions; print(WrapperAuthoringHints.__name__, NextActions.__name__)"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
WrapperAuthoringHints NextActions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Update Production Imports To Canonical Paths
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/models.py`
|
||||
|
||||
- [ ] **Step 1: Update `handlers.py` next action import**
|
||||
|
||||
Replace:
|
||||
|
||||
```python
|
||||
from .next_actions import NextActions
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
from wf_api.next_actions import NextActions
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `handlers.py` wrapper hint import**
|
||||
|
||||
Replace:
|
||||
|
||||
```python
|
||||
from .wrapper_hints import (
|
||||
workflow_output_schema_for_authoring,
|
||||
wrapper_hints_for_capability,
|
||||
)
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
from wf_api.wrapper_hints import (
|
||||
workflow_output_schema_for_authoring,
|
||||
wrapper_hints_for_capability,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update `models.py` next action import**
|
||||
|
||||
Replace:
|
||||
|
||||
```python
|
||||
from .next_actions import NextActionPatchExample, NextActions
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
from wf_api.next_actions import NextActionPatchExample, NextActions
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run production import smoke check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_mcp.workflow_surface.handlers import WorkflowSurfaceHandlers; from wf_mcp.workflow_surface.models import WrapperDraftNextActions; print(WorkflowSurfaceHandlers.__name__, WrapperDraftNextActions.__name__)"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
WorkflowSurfaceHandlers NextActions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Update Direct Tests And Add Shim Compatibility Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/test_workflow_wrapper_hints.py`
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_next_actions.py`
|
||||
|
||||
- [ ] **Step 1: Update wrapper hints test imports**
|
||||
|
||||
In `tests/wf_mcp/test_workflow_wrapper_hints.py`, replace imports from:
|
||||
|
||||
```python
|
||||
from wf_mcp.workflow_surface.wrapper_hints import (
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
from wf_api.wrapper_hints import (
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add wrapper hints shim identity test**
|
||||
|
||||
Append this test to `tests/wf_mcp/test_workflow_wrapper_hints.py`:
|
||||
|
||||
```python
|
||||
def test_workflow_surface_wrapper_hints_shim_reexports_canonical_helper() -> None:
|
||||
from wf_api.wrapper_hints import wrapper_hints_for_capability
|
||||
from wf_mcp.workflow_surface.wrapper_hints import (
|
||||
wrapper_hints_for_capability as wrapper_hints_for_capability_shim,
|
||||
)
|
||||
|
||||
assert wrapper_hints_for_capability_shim is wrapper_hints_for_capability
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update next actions test imports**
|
||||
|
||||
In `tests/wf_mcp/workflow_surface/test_next_actions.py`, replace:
|
||||
|
||||
```python
|
||||
from wf_mcp.workflow_surface.next_actions import NextActionTool, NextActions
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
from wf_api.next_actions import NextActionTool, NextActions
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add next actions shim identity test**
|
||||
|
||||
Append this test to `tests/wf_mcp/workflow_surface/test_next_actions.py`:
|
||||
|
||||
```python
|
||||
def test_workflow_surface_next_actions_shim_reexports_canonical_model() -> None:
|
||||
from wf_api.next_actions import NextActions
|
||||
from wf_mcp.workflow_surface.next_actions import NextActions as NextActionsShim
|
||||
|
||||
assert NextActionsShim is NextActions
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Search For Remaining Canonical Import Opportunities
|
||||
|
||||
**Files:**
|
||||
|
||||
- Inspect only unless the search finds new low-risk direct consumers.
|
||||
|
||||
- [ ] **Step 1: Search old imports**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "from \\.next_actions|from \\.wrapper_hints|from wf_mcp\\.workflow_surface\\.(next_actions|wrapper_hints)" src tests
|
||||
```
|
||||
|
||||
Expected remaining matches:
|
||||
|
||||
```text
|
||||
src/wf_mcp/workflow_surface/next_actions.py
|
||||
src/wf_mcp/workflow_surface/wrapper_hints.py
|
||||
tests/wf_mcp/test_workflow_wrapper_hints.py
|
||||
tests/wf_mcp/workflow_surface/test_next_actions.py
|
||||
```
|
||||
|
||||
If any other production module imports the old paths, update it to import from
|
||||
`wf_api.next_actions` or `wf_api.wrapper_hints`.
|
||||
|
||||
- [ ] **Step 2: Search for accidental `wf_api -> wf_mcp` imports**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "from wf_mcp|import wf_mcp|wf_mcp\\." src/wf_api
|
||||
```
|
||||
|
||||
Expected: no matches.
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Verification
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_mcp/workflow_surface tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 2: Run CLI tests that assert next_actions/wrapper_hints payloads**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_cli/test_discovery_lifecycle.py tests/wf_cli/test_run_deploy.py -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 3: Run ruff on touched files**
|
||||
|
||||
```powershell
|
||||
uv run ruff check src/wf_api src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/wrapper_hints.py src/wf_mcp/workflow_surface/handlers.py src/wf_mcp/workflow_surface/models.py tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_api
|
||||
```
|
||||
|
||||
Expected: all checks pass.
|
||||
|
||||
- [ ] **Step 4: Run basedpyright on touched files**
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error src/wf_api src/wf_mcp/workflow_surface/next_actions.py src/wf_mcp/workflow_surface/wrapper_hints.py src/wf_mcp/workflow_surface/handlers.py src/wf_mcp/workflow_surface/models.py tests/wf_mcp/test_workflow_wrapper_hints.py tests/wf_mcp/workflow_surface/test_next_actions.py tests/wf_api
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 5: Optional full suite**
|
||||
|
||||
Run this if time allows:
|
||||
|
||||
```powershell
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite passes with the project’s existing skipped/xfailed counts.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `wf_api.wrapper_hints` imports no `wf_mcp`.
|
||||
- `wf_api.next_actions` imports no `wf_mcp`.
|
||||
- Old `wf_mcp.workflow_surface.wrapper_hints` import path still works.
|
||||
- Old `wf_mcp.workflow_surface.next_actions` import path still works.
|
||||
- `WorkflowSurfaceHandlers` imports canonical guidance helpers from `wf_api`.
|
||||
- `wf_mcp.workflow_surface.models` imports canonical next-action models from `wf_api`.
|
||||
- Wrapper hint behavior is unchanged.
|
||||
- Next action behavior is unchanged.
|
||||
- No public payload shape changed.
|
||||
- No other workflow-surface helper moved in this slice.
|
||||
@@ -0,0 +1,535 @@
|
||||
# wf_api Slice 4A: Operation Context Scaffolding Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add a protocol-neutral workflow operation context seam so future domain services can be extracted from `WorkflowSurfaceHandlers` without depending on `WfMcpService`.
|
||||
|
||||
**Architecture:** This is scaffolding only. `wf_api.operation_context` defines small protocols and a `WorkflowOperationContext` dataclass for stores, capability sources, event recording, workflow runtime operations, and optional live source checking. `wf_mcp.broker.service.workflow_operation_context` adapts the current `WfMcpService` into that context. No workflow handler methods move in this slice and no public payloads change.
|
||||
|
||||
**Tech Stack:** Python 3.14+, `typing.Protocol`, dataclasses, existing `wf_artifacts`, `wf_platform`, `wf_authoring`, `wf_core`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
- Create `src/wf_api/operation_context.py`.
|
||||
- Define focused protocols instead of a new god object.
|
||||
- Create `src/wf_mcp/broker/service/workflow_operation_context.py` to adapt `WfMcpService`.
|
||||
- Add tests proving:
|
||||
- `wf_api.operation_context` imports no `wf_mcp`.
|
||||
- a `WfMcpService` can be adapted into `WorkflowOperationContext`.
|
||||
- context stores/sources point to the existing service objects.
|
||||
- event recording and runtime methods delegate to the service.
|
||||
- Add docstrings explaining this is scaffolding for later domain splits.
|
||||
|
||||
### Out Of Scope
|
||||
|
||||
- Do not move methods out of `WorkflowSurfaceHandlers`.
|
||||
- Do not change `WorkflowApiBackend`.
|
||||
- Do not change MCP tool request/response models.
|
||||
- Do not change public payloads.
|
||||
- Do not rename `WorkflowSurfaceHandlers`.
|
||||
- Do not add FastAPI/HTTP.
|
||||
- Do not hide or remove compatibility shims.
|
||||
|
||||
---
|
||||
|
||||
## Design
|
||||
|
||||
### New `wf_api.operation_context`
|
||||
|
||||
The context is not a service locator. It is the smallest explicit set of
|
||||
capabilities that extracted domain services will need.
|
||||
|
||||
Protocol groups:
|
||||
|
||||
- `WorkflowEventRecorder`: record one lifecycle event object.
|
||||
- `WorkflowSpecProvider`: look up a qualified node spec and expose capability sources.
|
||||
- `WorkflowArtifactCataloger`: produce saved artifact catalog entries.
|
||||
- `WorkflowRuntimeRunner`: run and resume compiled workflow plans.
|
||||
- `WorkflowLiveSourceChecker`: optional live-source validation hook.
|
||||
|
||||
Dataclass:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class WorkflowOperationContext:
|
||||
artifact_store: WorkflowArtifactStore | None
|
||||
draft_workspace_store: DraftWorkspaceStore | None
|
||||
run_store: RunStore | None
|
||||
capability_sources: Mapping[str, CapabilitySource]
|
||||
events: WorkflowEventRecorder
|
||||
specs: WorkflowSpecProvider
|
||||
artifacts: WorkflowArtifactCataloger
|
||||
runtime: WorkflowRuntimeRunner
|
||||
live_sources: WorkflowLiveSourceChecker | None = None
|
||||
```
|
||||
|
||||
This will look somewhat broad, but each field is a small protocol or simple
|
||||
store reference. If implementation pressure makes this mirror all of
|
||||
`WfMcpService`, stop and split protocols further.
|
||||
|
||||
### MCP Adapter
|
||||
|
||||
`wf_mcp.broker.service.workflow_operation_context.context_from_service(service)`
|
||||
builds the context from the current `WfMcpService`.
|
||||
|
||||
This adapter may import `wf_mcp`; `wf_api` must not.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add `wf_api.operation_context`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/operation_context.py`
|
||||
|
||||
- [ ] **Step 1: Create `src/wf_api/operation_context.py`**
|
||||
|
||||
Write this file:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Protocol
|
||||
|
||||
from wf_artifacts import (
|
||||
DraftWorkspaceStore,
|
||||
RunStore,
|
||||
WorkflowArtifact,
|
||||
WorkflowArtifactCatalogEntry,
|
||||
WorkflowArtifactStore,
|
||||
WorkflowDeployment,
|
||||
)
|
||||
from wf_authoring import AsyncRegistryHandler
|
||||
from wf_core import AsyncNodeHandler, RunState, Workflow
|
||||
from wf_core.runtime.ops.merges import ReducerDefinition
|
||||
from wf_platform import CapabilitySource
|
||||
|
||||
from .models import RawWorkflowPlan
|
||||
from .saved_subgraphs import SavedSubgraphTree
|
||||
|
||||
|
||||
class WorkflowEventRecorder(Protocol):
|
||||
"""Records workflow lifecycle events without exposing MCP event types."""
|
||||
|
||||
def record_event(self, event: object) -> None:
|
||||
"""Record one event object supplied by an adapter-owned event factory."""
|
||||
|
||||
|
||||
class WorkflowSpecProvider(Protocol):
|
||||
"""Provides planner-visible capability sources and qualified node specs."""
|
||||
|
||||
@property
|
||||
def capability_sources(self) -> Mapping[str, CapabilitySource]:
|
||||
"""Planner-visible capability sources keyed by source id."""
|
||||
|
||||
def get_qualified_spec(self, qualified_name: str) -> object:
|
||||
"""Return the node spec for one fully qualified capability name."""
|
||||
|
||||
|
||||
class WorkflowArtifactCataloger(Protocol):
|
||||
"""Formats saved workflow artifacts for list/detail surfaces."""
|
||||
|
||||
def workflow_artifact_catalog_entry(
|
||||
self, artifact: WorkflowArtifact
|
||||
) -> WorkflowArtifactCatalogEntry:
|
||||
"""Return the catalog entry representation for one saved artifact."""
|
||||
|
||||
|
||||
class WorkflowRuntimeRunner(Protocol):
|
||||
"""Runs and resumes workflow plans using an adapter-owned runtime backend."""
|
||||
|
||||
async def run_workflow_from_plan(
|
||||
self,
|
||||
plan: RawWorkflowPlan,
|
||||
*,
|
||||
workflow_input: dict[str, Any],
|
||||
node_name_bindings: dict[str, str] | None = None,
|
||||
registry: dict[str, AsyncRegistryHandler] | None = None,
|
||||
reducers: dict[str, ReducerDefinition] | None = None,
|
||||
prepared_subgraphs: dict[str, object] | None = None,
|
||||
) -> RunState:
|
||||
"""Execute one raw workflow plan and return its run state."""
|
||||
|
||||
async def resume_workflow_from_plan(
|
||||
self,
|
||||
plan: RawWorkflowPlan,
|
||||
*,
|
||||
run: RunState,
|
||||
resume_payload: dict[str, Any] | None,
|
||||
resume_outcome: str,
|
||||
node_name_bindings: dict[str, str] | None = None,
|
||||
registry: dict[str, AsyncRegistryHandler] | None = None,
|
||||
reducers: dict[str, ReducerDefinition] | None = None,
|
||||
prepared_subgraphs: dict[str, object] | None = None,
|
||||
) -> RunState:
|
||||
"""Resume one interrupted raw workflow plan and return its run state."""
|
||||
|
||||
|
||||
class WorkflowLiveSourceChecker(Protocol):
|
||||
"""Optional hook for validating live external source availability."""
|
||||
|
||||
async def available_sources(self) -> list[object]:
|
||||
"""Return source availability records understood by the caller."""
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class WorkflowOperationContext:
|
||||
"""Protocol-neutral dependencies needed by workflow API operations.
|
||||
|
||||
This is scaffolding for splitting the large MCP-backed handler into domain
|
||||
services. Keep this shape explicit; do not add arbitrary access to the whole
|
||||
MCP service.
|
||||
"""
|
||||
|
||||
artifact_store: WorkflowArtifactStore | None
|
||||
draft_workspace_store: DraftWorkspaceStore | None
|
||||
run_store: RunStore | None
|
||||
capability_sources: Mapping[str, CapabilitySource]
|
||||
events: WorkflowEventRecorder
|
||||
specs: WorkflowSpecProvider
|
||||
artifacts: WorkflowArtifactCataloger
|
||||
runtime: WorkflowRuntimeRunner
|
||||
live_sources: WorkflowLiveSourceChecker | None = None
|
||||
|
||||
|
||||
__all__ = [
|
||||
"WorkflowArtifactCataloger",
|
||||
"WorkflowEventRecorder",
|
||||
"WorkflowLiveSourceChecker",
|
||||
"WorkflowOperationContext",
|
||||
"WorkflowRuntimeRunner",
|
||||
"WorkflowSpecProvider",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run import smoke check**
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_api.operation_context import WorkflowOperationContext; print(WorkflowOperationContext.__name__)"
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
WorkflowOperationContext
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Re-export Operation Context Types
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
|
||||
- [ ] **Step 1: Add imports**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from .operation_context import (
|
||||
WorkflowArtifactCataloger,
|
||||
WorkflowEventRecorder,
|
||||
WorkflowLiveSourceChecker,
|
||||
WorkflowOperationContext,
|
||||
WorkflowRuntimeRunner,
|
||||
WorkflowSpecProvider,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add names to `__all__`**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
"WorkflowArtifactCataloger",
|
||||
"WorkflowEventRecorder",
|
||||
"WorkflowLiveSourceChecker",
|
||||
"WorkflowOperationContext",
|
||||
"WorkflowRuntimeRunner",
|
||||
"WorkflowSpecProvider",
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run top-level import smoke check**
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_api import WorkflowOperationContext, WorkflowRuntimeRunner; print(WorkflowOperationContext.__name__, WorkflowRuntimeRunner.__name__)"
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
WorkflowOperationContext WorkflowRuntimeRunner
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Add MCP Adapter For Operation Context
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/broker/service/workflow_operation_context.py`
|
||||
|
||||
- [ ] **Step 1: Create adapter file**
|
||||
|
||||
Write this file:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from wf_api.operation_context import (
|
||||
WorkflowArtifactCataloger,
|
||||
WorkflowEventRecorder,
|
||||
WorkflowLiveSourceChecker,
|
||||
WorkflowOperationContext,
|
||||
WorkflowRuntimeRunner,
|
||||
WorkflowSpecProvider,
|
||||
)
|
||||
|
||||
from .core import WfMcpService
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class WfMcpWorkflowEventRecorder(WorkflowEventRecorder):
|
||||
"""Adapter-owned event recorder backed by WfMcpService."""
|
||||
|
||||
service: WfMcpService
|
||||
|
||||
def record_event(self, event: object) -> None:
|
||||
self.service._record_event(event) # noqa: SLF001
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class WfMcpWorkflowSpecProvider(WorkflowSpecProvider):
|
||||
"""Adapter-owned spec provider backed by WfMcpService."""
|
||||
|
||||
service: WfMcpService
|
||||
|
||||
@property
|
||||
def capability_sources(self):
|
||||
return self.service.capability_sources
|
||||
|
||||
def get_qualified_spec(self, qualified_name: str) -> object:
|
||||
return self.service._get_qualified_spec(qualified_name) # noqa: SLF001
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class WfMcpWorkflowArtifactCataloger(WorkflowArtifactCataloger):
|
||||
"""Adapter-owned artifact catalog formatter backed by WfMcpService."""
|
||||
|
||||
service: WfMcpService
|
||||
|
||||
def workflow_artifact_catalog_entry(self, artifact):
|
||||
return self.service.workflow_artifact_catalog_entry(artifact)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class WfMcpWorkflowRuntimeRunner(WorkflowRuntimeRunner):
|
||||
"""Adapter-owned runtime runner backed by WfMcpService."""
|
||||
|
||||
service: WfMcpService
|
||||
|
||||
async def run_workflow_from_plan(self, plan, **kwargs):
|
||||
return await self.service.run_workflow_from_plan(plan, **kwargs)
|
||||
|
||||
async def resume_workflow_from_plan(self, plan, **kwargs):
|
||||
return await self.service.resume_workflow_from_plan(plan, **kwargs)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class WfMcpWorkflowLiveSourceChecker(WorkflowLiveSourceChecker):
|
||||
"""Placeholder live source checker; real live checks remain in handlers today."""
|
||||
|
||||
service: WfMcpService
|
||||
|
||||
async def available_sources(self) -> list[object]:
|
||||
# Existing live source availability logic still lives near handlers.
|
||||
# Slice 4A only creates the seam; it does not move live-check behavior.
|
||||
return []
|
||||
|
||||
|
||||
def context_from_service(service: WfMcpService) -> WorkflowOperationContext:
|
||||
"""Adapt the current MCP service stack into a protocol-neutral context."""
|
||||
specs = WfMcpWorkflowSpecProvider(service)
|
||||
return WorkflowOperationContext(
|
||||
artifact_store=service.artifact_store,
|
||||
draft_workspace_store=service.draft_workspace_store,
|
||||
run_store=service.run_store,
|
||||
capability_sources=specs.capability_sources,
|
||||
events=WfMcpWorkflowEventRecorder(service),
|
||||
specs=specs,
|
||||
artifacts=WfMcpWorkflowArtifactCataloger(service),
|
||||
runtime=WfMcpWorkflowRuntimeRunner(service),
|
||||
live_sources=WfMcpWorkflowLiveSourceChecker(service),
|
||||
)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"WfMcpWorkflowArtifactCataloger",
|
||||
"WfMcpWorkflowEventRecorder",
|
||||
"WfMcpWorkflowLiveSourceChecker",
|
||||
"WfMcpWorkflowRuntimeRunner",
|
||||
"WfMcpWorkflowSpecProvider",
|
||||
"context_from_service",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run adapter import smoke check**
|
||||
|
||||
```powershell
|
||||
uv run python -c "from wf_mcp.broker.service.workflow_operation_context import context_from_service; print(context_from_service.__name__)"
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
context_from_service
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Add Focused Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_api/test_operation_context.py`
|
||||
|
||||
- [ ] **Step 1: Write tests**
|
||||
|
||||
Write this file:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from wf_api.operation_context import WorkflowOperationContext
|
||||
from wf_cli.context import load_cli_context
|
||||
from wf_mcp.broker.service.workflow_operation_context import context_from_service
|
||||
|
||||
|
||||
def test_wf_api_operation_context_imports_no_wf_mcp() -> None:
|
||||
path = Path(__file__).resolve().parents[2] / "src" / "wf_api" / "operation_context.py"
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
|
||||
|
||||
violations: list[str] = []
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.ImportFrom) and node.module is not None:
|
||||
if node.module.startswith("wf_mcp"):
|
||||
violations.append(f"{node.lineno}: from {node.module} import ...")
|
||||
elif isinstance(node, ast.Import):
|
||||
for alias in node.names:
|
||||
if alias.name.startswith("wf_mcp"):
|
||||
violations.append(f"{node.lineno}: import {alias.name}")
|
||||
|
||||
assert violations == []
|
||||
|
||||
|
||||
def test_context_from_service_exposes_existing_store_objects(tmp_path: Path) -> None:
|
||||
config_path = tmp_path / "wf_mcp.config.json"
|
||||
config_path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"store_root": ".wf_mcp_store",
|
||||
"connections": [
|
||||
{
|
||||
"id": "demo.personal",
|
||||
"server": "demo",
|
||||
"account": "personal",
|
||||
}
|
||||
],
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
cli_context = load_cli_context(config_path)
|
||||
|
||||
operation_context = context_from_service(cli_context.service)
|
||||
|
||||
assert isinstance(operation_context, WorkflowOperationContext)
|
||||
assert operation_context.artifact_store is cli_context.service.artifact_store
|
||||
assert operation_context.draft_workspace_store is cli_context.service.draft_workspace_store
|
||||
assert operation_context.run_store is cli_context.service.run_store
|
||||
assert operation_context.capability_sources is cli_context.service.capability_sources
|
||||
|
||||
|
||||
def test_context_from_service_delegates_specs_and_events(tmp_path: Path) -> None:
|
||||
config_path = tmp_path / "wf_mcp.config.json"
|
||||
config_path.write_text(
|
||||
json.dumps({"store_root": ".wf_mcp_store", "connections": []}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
cli_context = load_cli_context(config_path)
|
||||
operation_context = context_from_service(cli_context.service)
|
||||
|
||||
event = object()
|
||||
operation_context.events.record_event(event)
|
||||
|
||||
assert cli_context.service.events[-1] is event
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run focused tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_operation_context.py tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Verification
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_operation_context.py tests/wf_api/test_import_direction.py tests/wf_cli/test_context.py -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 2: Run ruff on touched files**
|
||||
|
||||
```powershell
|
||||
uv run ruff check src/wf_api/operation_context.py src/wf_api/__init__.py src/wf_mcp/broker/service/workflow_operation_context.py tests/wf_api/test_operation_context.py
|
||||
```
|
||||
|
||||
Expected: all checks pass.
|
||||
|
||||
- [ ] **Step 3: Run basedpyright on touched files**
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error src/wf_api/operation_context.py src/wf_api/__init__.py src/wf_mcp/broker/service/workflow_operation_context.py tests/wf_api/test_operation_context.py
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 4: Optional full suite**
|
||||
|
||||
```powershell
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite passes with the project’s existing skipped/xfailed counts.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `wf_api.operation_context` imports no `wf_mcp`.
|
||||
- The context is not a broad wrapper around all of `WfMcpService`.
|
||||
- No `WorkflowSurfaceHandlers` method body moved.
|
||||
- No public payload changed.
|
||||
- MCP-owned adapter code is the only new code that imports `WfMcpService`.
|
||||
- Live source behavior remains unchanged; the live-source protocol is scaffolding only.
|
||||
@@ -0,0 +1,509 @@
|
||||
# wf_api Slice 4B: Draft Service Extraction Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Move draft validation, draft workspace editing, and minimal draft bootstrapping out of `WorkflowSurfaceHandlers` into a protocol-neutral `wf_api.drafts.WorkflowDraftApi`.
|
||||
|
||||
**Architecture:** `WorkflowDraftApi` depends on `WorkflowOperationContext`, not `WfMcpService`. `WorkflowSurfaceHandlers` keeps the public method names and delegates the draft subset to `WorkflowDraftApi`. This is the first real method-body extraction, so the slice intentionally excludes methods that save artifacts, record events, or call `inspect_capability`.
|
||||
|
||||
**Tech Stack:** Python 3.14+, `wf_api.operation_context`, `wf_artifacts` draft helpers, `wf_core` path/binding models, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### Move In This Slice
|
||||
|
||||
Move these methods from `WorkflowSurfaceHandlers` to `wf_api.drafts.WorkflowDraftApi`:
|
||||
|
||||
```text
|
||||
validate_draft
|
||||
compile_draft
|
||||
patch_draft
|
||||
list_draft_workspaces
|
||||
create_draft_workspace
|
||||
get_draft_workspace
|
||||
delete_draft_workspace
|
||||
validate_draft_workspace
|
||||
patch_draft_workspace
|
||||
set_draft_name
|
||||
set_draft_route
|
||||
set_step_input_map
|
||||
set_step_output_map
|
||||
create_minimal_draft_workspace
|
||||
```
|
||||
|
||||
Move these draft-only helper functions to `wf_api.drafts`:
|
||||
|
||||
```text
|
||||
_required_capabilities_for_plan
|
||||
_required_capability_payloads
|
||||
_observed_node_specs
|
||||
_draft_input_maps
|
||||
_draft_output_map
|
||||
_draft_input_bindings_payload
|
||||
_draft_output_bindings_payload
|
||||
_graph_path_payload
|
||||
_local_path_payload
|
||||
_state_path_payload
|
||||
_escape_json_pointer
|
||||
```
|
||||
|
||||
### Do Not Move In This Slice
|
||||
|
||||
Do not move:
|
||||
|
||||
```text
|
||||
create_draft_workspace_from_capability
|
||||
create_artifact_from_draft
|
||||
create_artifact_from_workspace
|
||||
create_wrapper_from_workspace
|
||||
```
|
||||
|
||||
Reasons:
|
||||
|
||||
- `create_draft_workspace_from_capability` depends on `inspect_capability`, which belongs to the capability domain. Move it later after the capability-inspection seam is explicit.
|
||||
- artifact-from-draft/workspace methods save artifacts and record events. Move them with artifacts/deployments in Slice 4C.
|
||||
|
||||
Temporary duplication is allowed for private helper functions used by both
|
||||
draft preview and artifact creation. If a helper still has live callers in
|
||||
`WorkflowSurfaceHandlers` after draft delegation, keep the old copy until Slice
|
||||
4C moves the artifact/deployment methods. Do not make `wf_api` import
|
||||
`wf_mcp` just to avoid duplication.
|
||||
|
||||
### Invariants
|
||||
|
||||
- No public payload changes.
|
||||
- No MCP tool schema changes.
|
||||
- `WorkflowSurfaceHandlers` still exposes the same methods.
|
||||
- `wf_api` imports no `wf_mcp`.
|
||||
- Draft methods delegate through `WorkflowDraftApi`.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Create `wf_api.drafts`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/drafts.py`
|
||||
|
||||
- [ ] **Step 1: Create `WorkflowDraftApi` skeleton**
|
||||
|
||||
Create `src/wf_api/drafts.py` with imports and class skeleton:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
from typing import Any
|
||||
|
||||
from wf_artifacts import (
|
||||
DraftWorkspaceStore,
|
||||
RequiredCapability,
|
||||
build_workflow_artifact_from_plan,
|
||||
compile_workflow_draft,
|
||||
create_draft_workspace as create_draft_workspace_record,
|
||||
get_draft_workspace as get_draft_workspace_record,
|
||||
patch_draft_workspace as patch_draft_workspace_record,
|
||||
patch_workflow_draft,
|
||||
validate_workflow_draft,
|
||||
)
|
||||
from wf_core.models.steps import (
|
||||
InputBinding,
|
||||
InputPathBinding,
|
||||
InputValueBinding,
|
||||
OutputBinding,
|
||||
)
|
||||
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
|
||||
from wf_platform import CapabilityRef, NodeSpecInventory
|
||||
|
||||
from .constants import (
|
||||
DEFAULT_CALL_STEP_ID,
|
||||
DEFAULT_ERROR_OUTCOME,
|
||||
DEFAULT_ERROR_STEP_ID,
|
||||
DEFAULT_OK_OUTCOME,
|
||||
RUNTIME_ERROR_CAPABILITY,
|
||||
)
|
||||
from .operation_context import WorkflowOperationContext
|
||||
|
||||
|
||||
class WorkflowDraftApi:
|
||||
"""Draft validation and workspace editing operations.
|
||||
|
||||
This service deliberately excludes artifact persistence and capability
|
||||
inspection. Those domains still live in the MCP-backed handler until later
|
||||
extraction slices.
|
||||
"""
|
||||
|
||||
def __init__(self, context: WorkflowOperationContext) -> None:
|
||||
self.context = context
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Export the draft service**
|
||||
|
||||
Add `WorkflowDraftApi` to `src/wf_api/__init__.py` so future adapters can use
|
||||
the canonical import path:
|
||||
|
||||
```python
|
||||
from .drafts import WorkflowDraftApi
|
||||
```
|
||||
|
||||
Also add `"WorkflowDraftApi"` to `__all__`.
|
||||
|
||||
- [ ] **Step 3: Add store helper**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def _draft_store(self) -> DraftWorkspaceStore:
|
||||
if self.context.draft_workspace_store is None:
|
||||
raise KeyError("draft workspace store is not configured")
|
||||
return self.context.draft_workspace_store
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add outcome lookup helper**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def _outcomes_for_capability(self, qualified_name: str) -> tuple[str, ...] | None:
|
||||
try:
|
||||
spec = self.context.specs.get_qualified_spec(qualified_name)
|
||||
except KeyError:
|
||||
return None
|
||||
outcomes = getattr(spec, "outcomes", None)
|
||||
return tuple(outcomes) if outcomes is not None else None
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Move Stateless Draft Methods
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/drafts.py`
|
||||
|
||||
- [ ] **Step 1: Add `validate_draft`**
|
||||
|
||||
```python
|
||||
async def validate_draft(self, *, draft: dict[str, Any]) -> dict[str, Any]:
|
||||
return validate_workflow_draft(
|
||||
draft,
|
||||
outcome_lookup=self._outcomes_for_capability,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add `compile_draft`**
|
||||
|
||||
```python
|
||||
async def compile_draft(self, *, draft: dict[str, Any]) -> dict[str, Any]:
|
||||
plan = compile_workflow_draft(draft)
|
||||
return {
|
||||
"compiled_plan": plan,
|
||||
"required_capabilities": _required_capability_payloads(
|
||||
_required_capabilities_for_plan(
|
||||
plan,
|
||||
source_bindings=None,
|
||||
context=self.context,
|
||||
)
|
||||
),
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add `patch_draft`**
|
||||
|
||||
```python
|
||||
async def patch_draft(
|
||||
self,
|
||||
*,
|
||||
draft: dict[str, Any],
|
||||
patch: list[dict[str, Any]],
|
||||
) -> dict[str, Any]:
|
||||
return patch_workflow_draft(draft, patch)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Move Draft Workspace Methods
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/drafts.py`
|
||||
|
||||
- [ ] **Step 1: Add workspace CRUD and validation methods**
|
||||
|
||||
Move these method bodies exactly from `WorkflowSurfaceHandlers`, replacing
|
||||
`self._draft_store()` with the new `WorkflowDraftApi._draft_store()`:
|
||||
|
||||
```text
|
||||
list_draft_workspaces
|
||||
create_draft_workspace
|
||||
get_draft_workspace
|
||||
delete_draft_workspace
|
||||
validate_draft_workspace
|
||||
patch_draft_workspace
|
||||
```
|
||||
|
||||
Keep behavior and return payloads identical.
|
||||
|
||||
- [ ] **Step 2: Add patch convenience methods**
|
||||
|
||||
Move these method bodies exactly:
|
||||
|
||||
```text
|
||||
set_draft_name
|
||||
set_draft_route
|
||||
set_step_input_map
|
||||
set_step_output_map
|
||||
```
|
||||
|
||||
They should call `self.patch_draft_workspace(...)` inside `WorkflowDraftApi`.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Move Minimal Draft Bootstrap
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/drafts.py`
|
||||
|
||||
- [ ] **Step 1: Add `create_minimal_draft_workspace`**
|
||||
|
||||
Move `WorkflowSurfaceHandlers.create_minimal_draft_workspace` into
|
||||
`WorkflowDraftApi` unchanged except:
|
||||
|
||||
- use `self._outcomes_for_capability(...)`
|
||||
- use `self.create_draft_workspace(...)`
|
||||
- keep the existing comments about provider-specific error envelopes
|
||||
|
||||
- [ ] **Step 2: Add helper functions**
|
||||
|
||||
Move these helper functions from `handlers.py` to the bottom of `wf_api.drafts`:
|
||||
|
||||
```text
|
||||
_draft_input_maps
|
||||
_draft_output_map
|
||||
_draft_input_bindings_payload
|
||||
_draft_output_bindings_payload
|
||||
_graph_path_payload
|
||||
_local_path_payload
|
||||
_state_path_payload
|
||||
_escape_json_pointer
|
||||
```
|
||||
|
||||
Do not change their behavior.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Move Required-Capability Draft Helpers
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/drafts.py`
|
||||
|
||||
- [ ] **Step 1: Move `_required_capabilities_for_plan`**
|
||||
|
||||
Move the helper from `handlers.py` and change its signature from:
|
||||
|
||||
```python
|
||||
def _required_capabilities_for_plan(
|
||||
plan: dict[str, Any],
|
||||
*,
|
||||
source_bindings: dict[str, str] | None,
|
||||
service: WfMcpService,
|
||||
) -> dict[str, RequiredCapability]:
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
def _required_capabilities_for_plan(
|
||||
plan: dict[str, Any],
|
||||
*,
|
||||
source_bindings: dict[str, str] | None,
|
||||
context: WorkflowOperationContext,
|
||||
) -> dict[str, RequiredCapability]:
|
||||
```
|
||||
|
||||
Inside it, call `_observed_node_specs(context)` instead of
|
||||
`_observed_node_specs(service)`.
|
||||
|
||||
- [ ] **Step 2: Move `_observed_node_specs`**
|
||||
|
||||
Change its signature from:
|
||||
|
||||
```python
|
||||
def _observed_node_specs(service: WfMcpService) -> dict[str, NodeSpecInventory]:
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
def _observed_node_specs(
|
||||
context: WorkflowOperationContext,
|
||||
) -> dict[str, NodeSpecInventory]:
|
||||
```
|
||||
|
||||
Loop over `context.capability_sources.values()`.
|
||||
|
||||
- [ ] **Step 3: Move `_required_capability_payloads`**
|
||||
|
||||
Move it unchanged.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Wire `WorkflowSurfaceHandlers` To Delegate Draft Methods
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
|
||||
- [ ] **Step 1: Add imports**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from wf_api.drafts import WorkflowDraftApi
|
||||
from wf_mcp.broker.service.workflow_operation_context import context_from_service
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Instantiate draft service**
|
||||
|
||||
In `WorkflowSurfaceHandlers.__init__`, add:
|
||||
|
||||
```python
|
||||
self._drafts = WorkflowDraftApi(context_from_service(service))
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Replace moved method bodies with delegates**
|
||||
|
||||
For each moved method, keep the same signature and replace the body with a call
|
||||
to `self._drafts`.
|
||||
|
||||
Example:
|
||||
|
||||
```python
|
||||
async def validate_draft(self, *, draft: dict[str, Any]) -> dict[str, Any]:
|
||||
return await self._drafts.validate_draft(draft=draft)
|
||||
```
|
||||
|
||||
Apply this pattern to every method in the Slice 4B move list.
|
||||
|
||||
- [ ] **Step 4: Remove moved helper functions from `handlers.py`**
|
||||
|
||||
Delete only helper functions that are no longer used in `handlers.py`:
|
||||
|
||||
```text
|
||||
_draft_input_maps
|
||||
_draft_output_map
|
||||
_draft_input_bindings_payload
|
||||
_draft_output_bindings_payload
|
||||
_graph_path_payload
|
||||
_local_path_payload
|
||||
_state_path_payload
|
||||
```
|
||||
|
||||
Remove `_escape_json_pointer` only if no remaining references exist in `handlers.py`.
|
||||
|
||||
Do not remove from `handlers.py` yet unless `rg` proves there are no remaining
|
||||
callers:
|
||||
|
||||
```text
|
||||
_required_capabilities_for_plan
|
||||
_required_capability_payloads
|
||||
_observed_node_specs
|
||||
```
|
||||
|
||||
Artifact creation currently still uses these helpers. It is acceptable for
|
||||
`wf_api.drafts` and `handlers.py` to each have a copy until Slice 4C moves the
|
||||
artifact/deployment methods. If the implementor can safely share them from
|
||||
`wf_api` without creating an MCP import or changing behavior, that is allowed,
|
||||
but not required for this slice.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Add Focused Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_api/test_drafts_service.py`
|
||||
|
||||
- [ ] **Step 1: Write direct service tests**
|
||||
|
||||
Create tests that build a `WfMcpService` through existing test helpers or
|
||||
`load_cli_context`, adapt it with `context_from_service`, then instantiate
|
||||
`WorkflowDraftApi`.
|
||||
|
||||
Cover:
|
||||
|
||||
- `patch_draft` applies a JSON patch.
|
||||
- `create_draft_workspace` creates a workspace.
|
||||
- `patch_draft_workspace` updates revision.
|
||||
- `validate_draft_workspace` refreshes status.
|
||||
- `create_minimal_draft_workspace` returns the same shape as before for a simple `wf.std` capability or a registered test spec.
|
||||
|
||||
- [ ] **Step 2: Add delegation smoke test**
|
||||
|
||||
In an existing workflow-surface draft test file or a new focused test, assert
|
||||
that `WorkflowSurfaceHandlers.validate_draft(...)` and
|
||||
`WorkflowDraftApi.validate_draft(...)` return equivalent status/diagnostics for
|
||||
the same draft.
|
||||
|
||||
Do not assert entire dict equality; compare stable fields individually.
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Verification
|
||||
|
||||
- [ ] **Step 1: Run draft tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_drafts_service.py tests/wf_mcp/workflow_surface/test_drafts.py -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 2: Run MCP schema/config tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/server/test_config.py tests/wf_mcp/workflow_surface -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 3: Run ruff on touched files**
|
||||
|
||||
```powershell
|
||||
uv run ruff check src/wf_api/drafts.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_drafts_service.py
|
||||
```
|
||||
|
||||
Expected: all checks pass.
|
||||
|
||||
- [ ] **Step 4: Run basedpyright on touched files**
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error src/wf_api/drafts.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_drafts_service.py
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 5: Optional full suite**
|
||||
|
||||
```powershell
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite passes with the project’s existing skipped/xfailed counts.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `wf_api.drafts` imports no `wf_mcp`.
|
||||
- `WorkflowSurfaceHandlers` public draft method signatures are unchanged.
|
||||
- Moved methods delegate through `WorkflowDraftApi`.
|
||||
- `create_draft_workspace_from_capability` remains in handlers.
|
||||
- artifact-saving methods remain in handlers.
|
||||
- No public payload shape changed.
|
||||
- No MCP schema changed.
|
||||
- No new dependency on the whole `WfMcpService` inside `wf_api`.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,542 @@
|
||||
# wf CLI Docs And Skill Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Create real user-facing `wf` CLI documentation and a small agent skill, then update `wf explain` cards to reference those docs instead of planning specs.
|
||||
|
||||
**Architecture:** Treat `docs/superpowers/*` as planning history only, not runtime/user guidance. The canonical CLI reference should live at `docs/wf_cli.md`; the optional repo-local skill should live at `skills/wf-cli/SKILL.md` and point agents to the real doc plus the safest command flow.
|
||||
|
||||
**Tech Stack:** Markdown docs, existing `wf_cli.explain` registry, pytest, ruff.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
Create:
|
||||
|
||||
```text
|
||||
docs/wf_cli.md
|
||||
skills/wf-cli/SKILL.md
|
||||
```
|
||||
|
||||
Modify:
|
||||
|
||||
```text
|
||||
docs/README.md
|
||||
src/wf_cli/explain/entries.py
|
||||
tests/wf_cli/test_explain.py
|
||||
```
|
||||
|
||||
Do not add command aliases in this slice. `wf cap` and `wf deploy` are the real registered command names.
|
||||
|
||||
Do not link `wf explain` runtime guidance to:
|
||||
|
||||
```text
|
||||
docs/superpowers/specs/*
|
||||
docs/superpowers/plans/*
|
||||
```
|
||||
|
||||
Those are planning artifacts, not user/operator docs.
|
||||
|
||||
## Task 1: Add Real CLI User Documentation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `docs/wf_cli.md`
|
||||
- Modify: `docs/README.md`
|
||||
|
||||
- [ ] **Step 1: Create `docs/wf_cli.md`**
|
||||
|
||||
Create `docs/wf_cli.md`:
|
||||
|
||||
```markdown
|
||||
# wf CLI
|
||||
|
||||
`wf` is the workflow platform command-line interface. It is a second front door
|
||||
beside MCP: useful for shell-driven authoring, local validation, file-based
|
||||
patches, and agent workflows that do better with commands than giant MCP
|
||||
schemas.
|
||||
|
||||
`wf` uses the same config/store stack as the MCP server in v1:
|
||||
|
||||
```bash
|
||||
wf --config wf_mcp.config.json <command>
|
||||
```
|
||||
|
||||
If `--config` is omitted, `wf_mcp.config.json` is used.
|
||||
|
||||
## Output Policy
|
||||
|
||||
JSON is the default output format for every command.
|
||||
|
||||
List/discovery commands may support:
|
||||
|
||||
```text
|
||||
--format json # complete machine-readable payload
|
||||
--format ids # one identifier per line
|
||||
--format compact # one concise line per item
|
||||
```
|
||||
|
||||
Detail and mutation commands are JSON-only unless documented otherwise.
|
||||
|
||||
There is no `table` format in v1.
|
||||
|
||||
## Lifecycle
|
||||
|
||||
The normal CLI workflow is:
|
||||
|
||||
1. Inspect capabilities.
|
||||
2. Create a draft workspace from a capability.
|
||||
3. Inspect or patch the draft.
|
||||
4. Validate the draft.
|
||||
5. Save an artifact.
|
||||
6. Save a deployment with source bindings.
|
||||
7. Validate the deployment.
|
||||
8. Run the deployment.
|
||||
9. Read bounded trace detail only when debugging.
|
||||
|
||||
## Capability Discovery
|
||||
|
||||
List capabilities:
|
||||
|
||||
```bash
|
||||
wf cap list
|
||||
wf cap list --source wf.std --format ids
|
||||
wf cap list --query echo --format compact
|
||||
```
|
||||
|
||||
Inspect one capability:
|
||||
|
||||
```bash
|
||||
wf cap inspect wf.std.concat
|
||||
```
|
||||
|
||||
`inspect` returns the full contract, including `wrapper_hints` when available.
|
||||
Hints are scaffolding, not semantic guarantees.
|
||||
|
||||
## Draft Workspaces
|
||||
|
||||
Create a draft from a capability:
|
||||
|
||||
```bash
|
||||
wf draft create-from-capability concat_ws wf.std.concat --name concat_ws
|
||||
```
|
||||
|
||||
List and inspect drafts:
|
||||
|
||||
```bash
|
||||
wf draft list --format compact
|
||||
wf draft inspect concat_ws
|
||||
wf draft inspect concat_ws --include-draft
|
||||
```
|
||||
|
||||
Patch a draft with RFC 6902 JSON Patch:
|
||||
|
||||
```bash
|
||||
wf draft patch concat_ws \
|
||||
--revision 1 \
|
||||
--input '[{"op":"replace","path":"/name","value":"concat_ws_v2"}]'
|
||||
```
|
||||
|
||||
Validate:
|
||||
|
||||
```bash
|
||||
wf draft validate concat_ws
|
||||
```
|
||||
|
||||
Save as an artifact:
|
||||
|
||||
```bash
|
||||
wf draft save concat_ws \
|
||||
--artifact concat_ws \
|
||||
--version 1 \
|
||||
--title "Concat Workflow" \
|
||||
--outcome ok \
|
||||
--binding wf.std=wf.std
|
||||
```
|
||||
|
||||
Use `--kind wrapper` when saving a callable wrapper artifact:
|
||||
|
||||
```bash
|
||||
wf draft save concat_ws \
|
||||
--artifact concat_wrapper \
|
||||
--version 1 \
|
||||
--title "Concat Wrapper" \
|
||||
--kind wrapper \
|
||||
--outcome ok \
|
||||
--binding wf.std=wf.std
|
||||
```
|
||||
|
||||
## Artifacts
|
||||
|
||||
List and inspect artifacts:
|
||||
|
||||
```bash
|
||||
wf artifact list --format ids
|
||||
wf artifact list --kind wrapper --format compact
|
||||
wf artifact inspect concat_ws 1
|
||||
```
|
||||
|
||||
Artifacts are immutable saved workflow definitions. List output is compact by
|
||||
design; use `inspect` for full details.
|
||||
|
||||
## Deployments
|
||||
|
||||
Save a deployment from flags:
|
||||
|
||||
```bash
|
||||
wf deploy save concat_ws.default \
|
||||
--artifact concat_ws \
|
||||
--version 1 \
|
||||
--binding wf.std=wf.std
|
||||
```
|
||||
|
||||
Save a deployment from JSON:
|
||||
|
||||
```bash
|
||||
wf deploy save --input-file deployment.json
|
||||
```
|
||||
|
||||
List, inspect, validate, and delete:
|
||||
|
||||
```bash
|
||||
wf deploy list --format compact
|
||||
wf deploy inspect concat_ws.default
|
||||
wf deploy validate concat_ws.default
|
||||
wf deploy validate concat_ws.default --live
|
||||
wf deploy delete concat_ws.default
|
||||
```
|
||||
|
||||
`--live` performs opt-in upstream liveness checks. Static validation can pass
|
||||
even when a live external source is temporarily unreachable.
|
||||
|
||||
## Runs And Traces
|
||||
|
||||
Start a deployment:
|
||||
|
||||
```bash
|
||||
wf run start concat_ws.default \
|
||||
--input '{"items":["red","blue"],"separator":" + "}'
|
||||
```
|
||||
|
||||
Inspect a run without trace detail:
|
||||
|
||||
```bash
|
||||
wf run inspect run_123
|
||||
```
|
||||
|
||||
Read a bounded trace slice:
|
||||
|
||||
```bash
|
||||
wf run trace run_123 --from 0 --limit 25
|
||||
```
|
||||
|
||||
Trace output can be large. Always request a bounded range.
|
||||
|
||||
## Explain
|
||||
|
||||
Explain stable diagnostic/error codes:
|
||||
|
||||
```bash
|
||||
wf explain source_missing
|
||||
wf explain deployment_unrunnable --format markdown
|
||||
wf explain --input-file validation-output.json
|
||||
wf explain --list --format compact
|
||||
```
|
||||
|
||||
`wf explain` is exact-match and docs-backed. It is not fuzzy search and does not
|
||||
generate prose.
|
||||
|
||||
## Common Diagnostics
|
||||
|
||||
### `source_missing`
|
||||
|
||||
A required logical source is not available or not bound.
|
||||
|
||||
Check:
|
||||
|
||||
```bash
|
||||
wf deploy inspect <deployment_id>
|
||||
wf cap list
|
||||
wf deploy validate <deployment_id> --live
|
||||
```
|
||||
|
||||
### `binding_missing`
|
||||
|
||||
A deployment is missing a required logical-to-concrete source binding.
|
||||
|
||||
Check artifact requirements, then save the deployment with all required
|
||||
bindings:
|
||||
|
||||
```bash
|
||||
wf deploy save <deployment_id> \
|
||||
--artifact <artifact_id> \
|
||||
--version <version> \
|
||||
--binding <logical>=<concrete>
|
||||
```
|
||||
|
||||
### `capability_missing`
|
||||
|
||||
The bound source does not expose a required capability.
|
||||
|
||||
Check:
|
||||
|
||||
```bash
|
||||
wf cap list --source <source_id>
|
||||
wf deploy inspect <deployment_id>
|
||||
```
|
||||
|
||||
### `schema_changed`
|
||||
|
||||
A saved dependency schema no longer matches the live capability. Inspect the
|
||||
live capability, patch the draft or wrapper, and save a new artifact version.
|
||||
|
||||
### `deployment_unrunnable`
|
||||
|
||||
The deployment failed validation and should not be run yet.
|
||||
|
||||
Check:
|
||||
|
||||
```bash
|
||||
wf deploy validate <deployment_id>
|
||||
wf explain --input-file validation-output.json
|
||||
```
|
||||
|
||||
## Known Limits
|
||||
|
||||
- The CLI reuses `wf_mcp` service/config/store wiring in v1.
|
||||
- Config loading registers stores and connections, but not arbitrary in-memory
|
||||
test `NodeSpec` functions.
|
||||
- Targeted draft editing helpers such as `wf draft step add` are not in v1.
|
||||
- `wf` does not replace MCP resources/prompts or interactive MCP clients.
|
||||
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add CLI doc to docs index**
|
||||
|
||||
Modify `docs/README.md` under `## Current Overview` or a new `## CLI` section:
|
||||
|
||||
```markdown
|
||||
- [`wf_cli.md`](wf_cli.md): workflow platform CLI commands, output formats,
|
||||
lifecycle flow, and common diagnostics.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Verify doc links manually**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
Test-Path docs/wf_cli.md
|
||||
Select-String -Path docs/README.md -Pattern 'wf_cli.md'
|
||||
```
|
||||
|
||||
Expected: both show the new doc exists and is indexed.
|
||||
|
||||
## Task 2: Add Repo-Local Agent Skill
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `skills/wf-cli/SKILL.md`
|
||||
|
||||
- [ ] **Step 1: Create `skills/wf-cli/SKILL.md`**
|
||||
|
||||
Create `skills/wf-cli/SKILL.md`:
|
||||
|
||||
```markdown
|
||||
---
|
||||
name: wf-cli
|
||||
description: Use when authoring, validating, deploying, running, or debugging workflows through the repo-local `wf` CLI.
|
||||
---
|
||||
|
||||
# wf CLI
|
||||
|
||||
Use the `wf` CLI when an agent needs a shell-friendly workflow lifecycle:
|
||||
|
||||
1. Discover capabilities.
|
||||
2. Create or patch a draft workspace.
|
||||
3. Validate the draft.
|
||||
4. Save an artifact.
|
||||
5. Save and validate a deployment.
|
||||
6. Run the deployment.
|
||||
7. Read bounded trace slices only when debugging.
|
||||
|
||||
Canonical docs:
|
||||
|
||||
- `docs/wf_cli.md`
|
||||
- `docs/workflow_capabilities.md`
|
||||
- `docs/workflow_drafts.md`
|
||||
- `docs/workflow_artifacts.md`
|
||||
- `docs/durable_run_operations.md`
|
||||
|
||||
## Core Commands
|
||||
|
||||
```bash
|
||||
wf cap list --format ids
|
||||
wf cap inspect <capability>
|
||||
|
||||
wf draft create-from-capability <workspace_id> <capability>
|
||||
wf draft inspect <workspace_id> --include-draft
|
||||
wf draft patch <workspace_id> --revision <n> --input-file patch.json
|
||||
wf draft validate <workspace_id>
|
||||
wf draft save <workspace_id> --artifact <artifact_id> --version <n> --title <title>
|
||||
|
||||
wf deploy save <deployment_id> --artifact <artifact_id> --version <n> --binding <logical>=<concrete>
|
||||
wf deploy validate <deployment_id>
|
||||
wf run start <deployment_id> --input-file input.json
|
||||
wf run trace <run_id> --from 0 --limit 25
|
||||
```
|
||||
|
||||
## Rules
|
||||
|
||||
- Prefer `--input-file` for large JSON.
|
||||
- Prefer `--format ids` or `--format compact` for discovery.
|
||||
- Do not request unbounded traces.
|
||||
- Do not treat wrapper hints as semantic guarantees.
|
||||
- If validation fails, run `wf explain <code>` or `wf explain --input-file <validation-output.json>`.
|
||||
- Do not use docs under `docs/superpowers/` as user-facing runtime guidance.
|
||||
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Do not wire skill installation**
|
||||
|
||||
Do not add marketplace/plugin installation logic in this slice. This repo-local
|
||||
skill is a source document for future packaging; it is not automatically active
|
||||
until a user installs or copies it into their agent environment.
|
||||
|
||||
## Task 3: Update `wf explain` Doc References
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_cli/explain/entries.py`
|
||||
- Test: `tests/wf_cli/test_explain.py`
|
||||
|
||||
- [ ] **Step 1: Add tests that all explain doc refs are user-facing and existing**
|
||||
|
||||
Append to `tests/wf_cli/test_explain.py`:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def test_explain_related_docs_do_not_point_to_planning_artifacts() -> None:
|
||||
for entry in DEFAULT_EXPLAIN_REGISTRY.list_full_entries():
|
||||
for related_doc in entry.related_docs:
|
||||
assert "docs/superpowers/" not in related_doc
|
||||
|
||||
|
||||
def test_explain_related_doc_files_exist() -> None:
|
||||
repo_root = Path(__file__).resolve().parents[2]
|
||||
for entry in DEFAULT_EXPLAIN_REGISTRY.list_full_entries():
|
||||
for related_doc in entry.related_docs:
|
||||
path_text = related_doc.split("#", 1)[0]
|
||||
if path_text.startswith("docs/"):
|
||||
assert (repo_root / path_text).exists(), path_text
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add registry full-entry accessor**
|
||||
|
||||
Modify `src/wf_cli/explain/registry.py`:
|
||||
|
||||
```python
|
||||
def list_full_entries(self) -> list[ExplainCard]:
|
||||
"""Return full cards for internal validation/tests."""
|
||||
return list(self._entries.values())
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update explain cards to use `docs/wf_cli.md`**
|
||||
|
||||
Modify `src/wf_cli/explain/entries.py`:
|
||||
|
||||
```python
|
||||
related_docs=[
|
||||
"docs/wf_cli.md#deployments",
|
||||
"docs/workflow_capabilities.md",
|
||||
],
|
||||
```
|
||||
|
||||
Use these mappings:
|
||||
|
||||
```text
|
||||
source_missing -> docs/wf_cli.md#common-diagnostics, docs/workflow_capabilities.md
|
||||
source_unreachable -> docs/wf_cli.md#deployments, docs/wf_mcp_troubleshooting.md
|
||||
binding_missing -> docs/wf_cli.md#deployments, docs/workflow_artifacts.md
|
||||
capability_missing -> docs/wf_cli.md#capability-discovery, docs/workflow_capabilities.md
|
||||
schema_changed -> docs/wf_cli.md#common-diagnostics, docs/schema_validation.md
|
||||
deployment_unrunnable -> docs/wf_cli.md#common-diagnostics, docs/current_roadmap.md
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run explain tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_explain.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 4: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- No new files unless lint/format requires cleanup.
|
||||
|
||||
- [ ] **Step 1: Run focused CLI tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli -q
|
||||
```
|
||||
|
||||
Expected: all CLI tests pass.
|
||||
|
||||
- [ ] **Step 2: Run focused lint**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_cli tests/wf_cli
|
||||
```
|
||||
|
||||
Expected: no lint errors.
|
||||
|
||||
- [ ] **Step 3: Run focused format check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff format --check src/wf_cli tests/wf_cli
|
||||
```
|
||||
|
||||
Expected: no formatting changes required. If this fails, run:
|
||||
|
||||
```bash
|
||||
uv run ruff format src/wf_cli tests/wf_cli
|
||||
```
|
||||
|
||||
Then rerun the format check.
|
||||
|
||||
- [ ] **Step 4: Run docs path check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
Test-Path docs/wf_cli.md
|
||||
Test-Path skills/wf-cli/SKILL.md
|
||||
rg -n "docs/superpowers/" src/wf_cli/explain docs/wf_cli.md skills/wf-cli/SKILL.md
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- `docs/wf_cli.md` exists.
|
||||
- `skills/wf-cli/SKILL.md` exists.
|
||||
- `rg` finds no `docs/superpowers/` runtime guidance references in those paths.
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- [ ] `wf explain` cards no longer link to `docs/superpowers/specs` or `docs/superpowers/plans`.
|
||||
- [ ] `docs/wf_cli.md` contains the lifecycle that the CLI actually supports today.
|
||||
- [ ] `docs/README.md` points to the new CLI doc.
|
||||
- [ ] The skill is repo-local documentation only; no install/packaging behavior was added.
|
||||
- [ ] Tests verify explain-card doc refs are not stale.
|
||||
@@ -0,0 +1,955 @@
|
||||
# wf CLI Explain Registry Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add a docs-backed `wf explain` command that explains stable workflow diagnostic/error codes from exact codes or CLI/MCP-style error JSON.
|
||||
|
||||
**Architecture:** Keep explanation data in a small protocol-neutral registry under `wf_cli.explain`, then make `src/wf_cli/commands/explain.py` a thin Typer boundary. This slice does not change run/deploy behavior and does not implement fuzzy search, generated prose, or a large FAQ.
|
||||
|
||||
**Tech Stack:** Python 3.14, Typer, Pydantic v2, pytest `CliRunner`, existing `wf_cli.io.emit_json`.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
Create:
|
||||
|
||||
```text
|
||||
src/wf_cli/explain/
|
||||
__init__.py # public exports for the explain registry package
|
||||
entries.py # curated explanation cards; no runtime workflow logic
|
||||
models.py # Pydantic models for cards and list summaries
|
||||
parser.py # extracts stable codes from direct strings and JSON payloads
|
||||
registry.py # exact lookup/list API over curated cards
|
||||
tests/wf_cli/test_explain.py
|
||||
```
|
||||
|
||||
Modify:
|
||||
|
||||
```text
|
||||
src/wf_cli/commands/explain.py
|
||||
```
|
||||
|
||||
Do not modify:
|
||||
|
||||
```text
|
||||
src/wf_cli/commands/runs.py
|
||||
src/wf_cli/commands/deployments.py
|
||||
src/wf_mcp/
|
||||
src/wf_core/
|
||||
```
|
||||
|
||||
## Behavior Contract
|
||||
|
||||
Supported commands:
|
||||
|
||||
```bash
|
||||
wf explain source_missing
|
||||
wf explain source_missing --format json
|
||||
wf explain source_missing --format markdown
|
||||
wf explain source_missing --format compact
|
||||
wf explain --input-file error.json
|
||||
wf explain --stdin
|
||||
wf explain --list
|
||||
```
|
||||
|
||||
Supported initial codes:
|
||||
|
||||
```text
|
||||
source_missing
|
||||
source_unreachable
|
||||
binding_missing
|
||||
capability_missing
|
||||
schema_changed
|
||||
deployment_unrunnable
|
||||
```
|
||||
|
||||
JSON output rules:
|
||||
|
||||
- `wf explain <code>` returns one full card object.
|
||||
- `wf explain --input-file error.json` returns `{"entries": [full_card, ...]}` because input JSON can contain multiple diagnostics.
|
||||
- `wf explain --stdin` follows the same shape as `--input-file`.
|
||||
- `wf explain --list` returns a lean index: `{"entries": [{"code": "...", "summary": "..."}]}`.
|
||||
|
||||
Markdown output rules:
|
||||
|
||||
- One card renders as a small markdown document.
|
||||
- Multiple cards render as multiple markdown documents separated by one blank line.
|
||||
- List output renders a bullet list.
|
||||
|
||||
Compact output rules:
|
||||
|
||||
- One line per card: `code: summary`.
|
||||
- List output uses the same one-line shape.
|
||||
|
||||
Input extraction rules:
|
||||
|
||||
- Direct string code: `source_missing`
|
||||
- JSON object with `code`: `{"code": "source_missing"}`
|
||||
- JSON object with `error.code`: `{"error": {"code": "deployment_unrunnable"}}`
|
||||
- JSON object with diagnostics: `{"diagnostics": [{"code": "source_missing"}, {"code": "schema_changed"}]}`
|
||||
- JSON list of diagnostics: `[{"code": "source_missing"}, {"code": "schema_changed"}]`
|
||||
- Preserve first-seen order and dedupe repeated codes.
|
||||
- Unknown codes should fail with a Typer error that includes the unknown code.
|
||||
|
||||
## Task 1: Add Registry Models And Entries
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_cli/explain/models.py`
|
||||
- Create: `src/wf_cli/explain/entries.py`
|
||||
- Create: `src/wf_cli/explain/__init__.py`
|
||||
- Test: `tests/wf_cli/test_explain.py`
|
||||
|
||||
- [ ] **Step 1: Write failing model/registry smoke tests**
|
||||
|
||||
Create `tests/wf_cli/test_explain.py` with these first tests:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_cli.explain import DEFAULT_EXPLAIN_REGISTRY
|
||||
|
||||
|
||||
def test_explain_registry_returns_full_card_for_known_code() -> None:
|
||||
card = DEFAULT_EXPLAIN_REGISTRY.get("source_missing")
|
||||
|
||||
assert card.code == "source_missing"
|
||||
assert "source" in card.summary.lower()
|
||||
assert card.why_it_happens
|
||||
assert card.how_to_fix
|
||||
assert card.related_docs
|
||||
|
||||
|
||||
def test_explain_registry_list_is_lean() -> None:
|
||||
entries = DEFAULT_EXPLAIN_REGISTRY.list_entries()
|
||||
|
||||
source_missing = next(entry for entry in entries if entry.code == "source_missing")
|
||||
assert source_missing.code == "source_missing"
|
||||
assert "source" in source_missing.summary.lower()
|
||||
assert not hasattr(source_missing, "how_to_fix")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the failing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_explain.py -q
|
||||
```
|
||||
|
||||
Expected: fail because `wf_cli.explain` does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement Pydantic models**
|
||||
|
||||
Create `src/wf_cli/explain/models.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class ExplainCard(BaseModel):
|
||||
"""Human-curated help for one stable workflow diagnostic/error code."""
|
||||
|
||||
code: str = Field(min_length=1, description="Stable diagnostic or CLI error code.")
|
||||
summary: str = Field(min_length=1, description="One-sentence explanation.")
|
||||
why_it_happens: list[str] = Field(
|
||||
description="Common causes, ordered from most likely to least likely."
|
||||
)
|
||||
how_to_fix: list[str] = Field(
|
||||
description="Concrete next steps an agent or user can try."
|
||||
)
|
||||
related_docs: list[str] = Field(
|
||||
default_factory=list,
|
||||
description="Documentation resource IDs or file references.",
|
||||
)
|
||||
|
||||
|
||||
class ExplainSummary(BaseModel):
|
||||
"""Lean index entry for `wf explain --list`."""
|
||||
|
||||
code: str = Field(min_length=1)
|
||||
summary: str = Field(min_length=1)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Implement curated entries**
|
||||
|
||||
Create `src/wf_cli/explain/entries.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from .models import ExplainCard
|
||||
|
||||
|
||||
EXPLAIN_CARDS: tuple[ExplainCard, ...] = (
|
||||
ExplainCard(
|
||||
code="source_missing",
|
||||
summary="A required logical source is not available or not bound.",
|
||||
why_it_happens=[
|
||||
"The artifact requires a logical source that the deployment did not bind.",
|
||||
"The concrete source was removed, renamed, disabled, or never registered.",
|
||||
"A saved wrapper or workflow depends on a source that is absent in this config.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Run `wf deploy inspect <deployment_id>` and check the bindings.",
|
||||
"Run `wf cap list` to confirm the concrete source is available.",
|
||||
"Save the deployment again with the missing logical source bound.",
|
||||
"Run `wf deploy validate <deployment_id> --live` after changing bindings.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli_usage.md#deployment-validation",
|
||||
"docs/workflow_capabilities.md",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code="source_unreachable",
|
||||
summary="A concrete source exists in config but could not be reached.",
|
||||
why_it_happens=[
|
||||
"The upstream MCP server or local process failed during liveness checks.",
|
||||
"The source command, URL, authentication, or environment is invalid.",
|
||||
"The source is slow or hung and exceeded the bounded liveness timeout.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Check the source command or URL in the active config.",
|
||||
"Start or restart the upstream server.",
|
||||
"Run validation without `--live` if you only need static deployment checks.",
|
||||
"Run `wf deploy validate <deployment_id> --live` again after fixing the source.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli_usage.md#deployment-validation",
|
||||
"docs/wf_mcp_unified_proxy_plan.md",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code="binding_missing",
|
||||
summary="A deployment is missing a required logical-to-concrete source binding.",
|
||||
why_it_happens=[
|
||||
"The artifact was saved with required capabilities under a logical source.",
|
||||
"The deployment was saved without a binding for that logical source.",
|
||||
"A binding field was misspelled or placed under the wrong payload key.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Inspect the artifact requirements.",
|
||||
"Inspect the deployment bindings.",
|
||||
"Save the deployment with `bindings` entries that map each logical source.",
|
||||
"Use `wf deploy validate <deployment_id>` to confirm the binding set.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli_usage.md#save-and-validate-a-deployment",
|
||||
"docs/workflow_capabilities.md#sources",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code="capability_missing",
|
||||
summary="A required capability is not present on the bound source.",
|
||||
why_it_happens=[
|
||||
"The upstream source no longer exposes the tool or node spec.",
|
||||
"The workflow was bound to the wrong account/profile/source.",
|
||||
"The capability was renamed after the artifact was saved.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Run `wf cap list` and search for the expected capability.",
|
||||
"Inspect the deployment bindings for the affected logical source.",
|
||||
"Rebind to a concrete source that exposes the capability.",
|
||||
"Rebuild or patch the artifact if the capability was intentionally renamed.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/workflow_capabilities.md",
|
||||
"docs/wf_cli_usage.md#capability-discovery",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code="schema_changed",
|
||||
summary="A saved dependency schema no longer matches the live capability.",
|
||||
why_it_happens=[
|
||||
"The upstream tool or node spec changed its input/output schema.",
|
||||
"The deployment is bound to a different source profile than the one used before.",
|
||||
"A wrapper assumes fields that the live capability no longer declares.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Inspect the live capability.",
|
||||
"Compare it with the saved artifact dependency summary.",
|
||||
"Patch the draft or wrapper to match the new schema.",
|
||||
"Save a new artifact version and deployment after validating the change.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/workflow_capabilities.md#dependency-validation",
|
||||
"docs/schema_validation.md",
|
||||
],
|
||||
),
|
||||
ExplainCard(
|
||||
code="deployment_unrunnable",
|
||||
summary="The deployment failed validation and should not be run yet.",
|
||||
why_it_happens=[
|
||||
"One or more required sources, capabilities, schemas, or bindings are invalid.",
|
||||
"The deployment points at an artifact version that cannot be resolved.",
|
||||
"Live validation found an upstream source or capability problem.",
|
||||
],
|
||||
how_to_fix=[
|
||||
"Run `wf deploy validate <deployment_id>` and read the diagnostics.",
|
||||
"Run `wf explain --input-file <validation-output.json>` for diagnostic details.",
|
||||
"Fix source bindings or rebuild the artifact version.",
|
||||
"Re-run validation before starting the deployment.",
|
||||
],
|
||||
related_docs=[
|
||||
"docs/wf_cli_usage.md#deployment-validation",
|
||||
"docs/current_roadmap.md",
|
||||
],
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Implement registry**
|
||||
|
||||
Create `src/wf_cli/explain/registry.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable
|
||||
|
||||
from .entries import EXPLAIN_CARDS
|
||||
from .models import ExplainCard, ExplainSummary
|
||||
|
||||
|
||||
class UnknownExplainCode(KeyError):
|
||||
"""Raised when a diagnostic code is not present in the curated registry."""
|
||||
|
||||
|
||||
class ExplainRegistry:
|
||||
"""Exact-match registry for docs-backed explanation cards."""
|
||||
|
||||
def __init__(self, entries: Iterable[ExplainCard] = EXPLAIN_CARDS) -> None:
|
||||
self._entries = {entry.code: entry for entry in entries}
|
||||
|
||||
def get(self, code: str) -> ExplainCard:
|
||||
"""Return a full explanation card for one stable code."""
|
||||
try:
|
||||
return self._entries[code]
|
||||
except KeyError as exc:
|
||||
raise UnknownExplainCode(code) from exc
|
||||
|
||||
def list_entries(self) -> list[ExplainSummary]:
|
||||
"""Return lean summaries for discovery output."""
|
||||
return [
|
||||
ExplainSummary(code=entry.code, summary=entry.summary)
|
||||
for entry in self._entries.values()
|
||||
]
|
||||
|
||||
|
||||
DEFAULT_EXPLAIN_REGISTRY = ExplainRegistry()
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Export public API**
|
||||
|
||||
Create `src/wf_cli/explain/__init__.py`:
|
||||
|
||||
```python
|
||||
"""Docs-backed explanation registry for workflow CLI diagnostics."""
|
||||
|
||||
from .models import ExplainCard, ExplainSummary
|
||||
from .registry import DEFAULT_EXPLAIN_REGISTRY, ExplainRegistry, UnknownExplainCode
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_EXPLAIN_REGISTRY",
|
||||
"ExplainCard",
|
||||
"ExplainRegistry",
|
||||
"ExplainSummary",
|
||||
"UnknownExplainCode",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Run tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_explain.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 2: Add JSON/String Code Parser
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_cli/explain/parser.py`
|
||||
- Modify: `src/wf_cli/explain/__init__.py`
|
||||
- Test: `tests/wf_cli/test_explain.py`
|
||||
|
||||
- [ ] **Step 1: Add parser tests**
|
||||
|
||||
Replace the import block at the top of `tests/wf_cli/test_explain.py` with:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from wf_cli.explain import (
|
||||
DEFAULT_EXPLAIN_REGISTRY,
|
||||
ExplainInputError,
|
||||
extract_explain_codes,
|
||||
parse_explain_input,
|
||||
)
|
||||
```
|
||||
|
||||
Then append these tests to `tests/wf_cli/test_explain.py`:
|
||||
|
||||
```python
|
||||
|
||||
|
||||
def test_parse_explain_input_accepts_direct_code() -> None:
|
||||
assert parse_explain_input("source_missing") == ["source_missing"]
|
||||
|
||||
|
||||
def test_parse_explain_input_extracts_error_code() -> None:
|
||||
raw = '{"error": {"code": "deployment_unrunnable"}}'
|
||||
|
||||
assert parse_explain_input(raw) == ["deployment_unrunnable"]
|
||||
|
||||
|
||||
def test_parse_explain_input_extracts_diagnostic_codes_in_order() -> None:
|
||||
raw = """
|
||||
{
|
||||
"diagnostics": [
|
||||
{"code": "source_missing"},
|
||||
{"code": "schema_changed"},
|
||||
{"code": "source_missing"}
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
assert parse_explain_input(raw) == ["source_missing", "schema_changed"]
|
||||
|
||||
|
||||
def test_extract_explain_codes_accepts_diagnostic_list() -> None:
|
||||
value = [{"code": "binding_missing"}, {"code": "capability_missing"}]
|
||||
|
||||
assert extract_explain_codes(value) == ["binding_missing", "capability_missing"]
|
||||
|
||||
|
||||
def test_parse_explain_input_rejects_json_without_codes() -> None:
|
||||
with pytest.raises(ExplainInputError, match="no explainable code"):
|
||||
parse_explain_input('{"status": "failed"}')
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run parser tests to verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_explain.py -q
|
||||
```
|
||||
|
||||
Expected: fail because `parser.py` and exports do not exist.
|
||||
|
||||
- [ ] **Step 3: Implement parser**
|
||||
|
||||
Create `src/wf_cli/explain/parser.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
|
||||
class ExplainInputError(ValueError):
|
||||
"""Raised when `wf explain` input cannot be reduced to stable codes."""
|
||||
|
||||
|
||||
def parse_explain_input(raw: str) -> list[str]:
|
||||
"""Parse a direct code or JSON payload into first-seen unique codes."""
|
||||
stripped = raw.strip()
|
||||
if not stripped:
|
||||
raise ExplainInputError("explain input is empty")
|
||||
if stripped.startswith("{") or stripped.startswith("["):
|
||||
try:
|
||||
value = json.loads(stripped)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise ExplainInputError(f"invalid JSON explain input: {exc.msg}") from exc
|
||||
return extract_explain_codes(value)
|
||||
return [stripped]
|
||||
|
||||
|
||||
def extract_explain_codes(value: Any) -> list[str]:
|
||||
"""Extract known diagnostic-code shapes without guessing or fuzzy matching."""
|
||||
codes: list[str] = []
|
||||
_collect_codes(value, codes)
|
||||
deduped = _dedupe(codes)
|
||||
if not deduped:
|
||||
raise ExplainInputError("no explainable code found in input")
|
||||
return deduped
|
||||
|
||||
|
||||
def _collect_codes(value: Any, codes: list[str]) -> None:
|
||||
if isinstance(value, str):
|
||||
codes.append(value)
|
||||
return
|
||||
if isinstance(value, list):
|
||||
for item in value:
|
||||
_collect_codes(item, codes)
|
||||
return
|
||||
if not isinstance(value, dict):
|
||||
return
|
||||
|
||||
code = value.get("code")
|
||||
if isinstance(code, str):
|
||||
codes.append(code)
|
||||
|
||||
error = value.get("error")
|
||||
if isinstance(error, dict):
|
||||
error_code = error.get("code")
|
||||
if isinstance(error_code, str):
|
||||
codes.append(error_code)
|
||||
|
||||
diagnostics = value.get("diagnostics")
|
||||
if isinstance(diagnostics, list):
|
||||
for diagnostic in diagnostics:
|
||||
_collect_codes(diagnostic, codes)
|
||||
|
||||
|
||||
def _dedupe(codes: list[str]) -> list[str]:
|
||||
seen: set[str] = set()
|
||||
result: list[str] = []
|
||||
for code in codes:
|
||||
if code in seen:
|
||||
continue
|
||||
seen.add(code)
|
||||
result.append(code)
|
||||
return result
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Export parser helpers**
|
||||
|
||||
Modify `src/wf_cli/explain/__init__.py`:
|
||||
|
||||
```python
|
||||
"""Docs-backed explanation registry for workflow CLI diagnostics."""
|
||||
|
||||
from .models import ExplainCard, ExplainSummary
|
||||
from .parser import ExplainInputError, extract_explain_codes, parse_explain_input
|
||||
from .registry import DEFAULT_EXPLAIN_REGISTRY, ExplainRegistry, UnknownExplainCode
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_EXPLAIN_REGISTRY",
|
||||
"ExplainCard",
|
||||
"ExplainInputError",
|
||||
"ExplainRegistry",
|
||||
"ExplainSummary",
|
||||
"UnknownExplainCode",
|
||||
"extract_explain_codes",
|
||||
"parse_explain_input",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run parser tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_explain.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 3: Implement `wf explain` Command And Rendering
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_cli/commands/explain.py`
|
||||
- Modify: `src/wf_cli/app.py`
|
||||
- Test: `tests/wf_cli/test_explain.py`
|
||||
- Test: `tests/wf_cli/test_app.py`
|
||||
|
||||
- [ ] **Step 1: Add CLI tests**
|
||||
|
||||
Replace the import block at the top of `tests/wf_cli/test_explain.py` with:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from wf_cli.app import app
|
||||
from wf_cli.explain import (
|
||||
DEFAULT_EXPLAIN_REGISTRY,
|
||||
ExplainInputError,
|
||||
extract_explain_codes,
|
||||
parse_explain_input,
|
||||
)
|
||||
```
|
||||
|
||||
Then append these tests to `tests/wf_cli/test_explain.py`:
|
||||
|
||||
```python
|
||||
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
def test_wf_explain_code_outputs_full_json_card() -> None:
|
||||
result = runner.invoke(app, ["explain", "source_missing"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
payload = json.loads(result.output)
|
||||
assert payload["code"] == "source_missing"
|
||||
assert payload["summary"]
|
||||
assert payload["why_it_happens"]
|
||||
assert payload["how_to_fix"]
|
||||
|
||||
|
||||
def test_wf_explain_list_outputs_lean_json_index() -> None:
|
||||
result = runner.invoke(app, ["explain", "--list"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
payload = json.loads(result.output)
|
||||
first = payload["entries"][0]
|
||||
assert first["code"]
|
||||
assert first["summary"]
|
||||
assert "how_to_fix" not in first
|
||||
|
||||
|
||||
def test_wf_explain_markdown_format() -> None:
|
||||
result = runner.invoke(app, ["explain", "source_missing", "--format", "markdown"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "# source_missing" in result.output
|
||||
assert "## How To Fix" in result.output
|
||||
|
||||
|
||||
def test_wf_explain_compact_format() -> None:
|
||||
result = runner.invoke(app, ["explain", "source_missing", "--format", "compact"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert result.output.strip().startswith("source_missing: ")
|
||||
|
||||
|
||||
def test_wf_explain_input_file_outputs_multiple_cards(tmp_path) -> None:
|
||||
error_file = tmp_path / "error.json"
|
||||
error_file.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"diagnostics": [
|
||||
{"code": "source_missing"},
|
||||
{"code": "schema_changed"},
|
||||
{"code": "source_missing"},
|
||||
]
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
result = runner.invoke(app, ["explain", "--input-file", str(error_file)])
|
||||
|
||||
assert result.exit_code == 0
|
||||
payload = json.loads(result.output)
|
||||
assert len(payload["entries"]) == 2
|
||||
assert payload["entries"][0]["code"] == "source_missing"
|
||||
assert payload["entries"][1]["code"] == "schema_changed"
|
||||
|
||||
|
||||
def test_wf_explain_stdin_outputs_multiple_cards() -> None:
|
||||
result = runner.invoke(
|
||||
app,
|
||||
["explain", "--stdin"],
|
||||
input='{"error": {"code": "deployment_unrunnable"}}',
|
||||
)
|
||||
|
||||
assert result.exit_code == 0
|
||||
payload = json.loads(result.output)
|
||||
assert payload["entries"][0]["code"] == "deployment_unrunnable"
|
||||
|
||||
|
||||
def test_wf_explain_unknown_code_fails_clearly() -> None:
|
||||
result = runner.invoke(app, ["explain", "not_a_real_code"])
|
||||
|
||||
assert result.exit_code != 0
|
||||
assert "not_a_real_code" in result.output
|
||||
|
||||
|
||||
def test_wf_explain_list_rejects_other_input_modes() -> None:
|
||||
result = runner.invoke(app, ["explain", "source_missing", "--list"])
|
||||
|
||||
assert result.exit_code != 0
|
||||
assert "--list cannot be combined" in result.output
|
||||
```
|
||||
|
||||
Append this test to `tests/wf_cli/test_app.py`:
|
||||
|
||||
```python
|
||||
def test_wf_explain_help_shows_input_modes() -> None:
|
||||
result = runner.invoke(app, ["explain", "--help"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "--input-file" in result.output
|
||||
assert "--stdin" in result.output
|
||||
assert "--list" in result.output
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run CLI tests to verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_explain.py tests/wf_cli/test_app.py -q
|
||||
```
|
||||
|
||||
Expected: fail because `wf explain` has no callback implementation.
|
||||
|
||||
- [ ] **Step 3: Implement command**
|
||||
|
||||
Replace `src/wf_cli/commands/explain.py` with:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import StrEnum
|
||||
from pathlib import Path
|
||||
from typing import Annotated
|
||||
|
||||
import typer
|
||||
|
||||
from wf_cli.explain import (
|
||||
DEFAULT_EXPLAIN_REGISTRY,
|
||||
ExplainCard,
|
||||
ExplainInputError,
|
||||
ExplainSummary,
|
||||
UnknownExplainCode,
|
||||
parse_explain_input,
|
||||
)
|
||||
from wf_cli.io import emit_json
|
||||
|
||||
|
||||
class ExplainFormat(StrEnum):
|
||||
"""Output formats supported by `wf explain`."""
|
||||
|
||||
JSON = "json"
|
||||
MARKDOWN = "markdown"
|
||||
COMPACT = "compact"
|
||||
|
||||
|
||||
def explain_command(
|
||||
code: Annotated[
|
||||
str | None,
|
||||
typer.Argument(help="Diagnostic/error code, or JSON payload containing codes."),
|
||||
] = None,
|
||||
input_file: Annotated[
|
||||
Path | None,
|
||||
typer.Option("--input-file", help="Read diagnostic/error JSON from a file."),
|
||||
] = None,
|
||||
read_stdin: Annotated[
|
||||
bool,
|
||||
typer.Option("--stdin", help="Read diagnostic/error JSON from standard input."),
|
||||
] = False,
|
||||
list_entries: Annotated[
|
||||
bool,
|
||||
typer.Option("--list", help="List known explanation codes."),
|
||||
] = False,
|
||||
output_format: Annotated[
|
||||
ExplainFormat,
|
||||
typer.Option("--format", help="Output format."),
|
||||
] = ExplainFormat.JSON,
|
||||
) -> None:
|
||||
"""Explain exact workflow diagnostic codes without generated prose."""
|
||||
try:
|
||||
if list_entries:
|
||||
if code is not None or input_file is not None or read_stdin:
|
||||
raise ExplainInputError(
|
||||
"--list cannot be combined with code, --input-file, or --stdin"
|
||||
)
|
||||
_emit_summaries(DEFAULT_EXPLAIN_REGISTRY.list_entries(), output_format)
|
||||
return
|
||||
codes = _read_codes(code=code, input_file=input_file, read_stdin=read_stdin)
|
||||
cards = [DEFAULT_EXPLAIN_REGISTRY.get(item) for item in codes]
|
||||
except (ExplainInputError, UnknownExplainCode) as exc:
|
||||
raise typer.BadParameter(_error_message(exc)) from exc
|
||||
|
||||
if len(cards) == 1 and input_file is None and not read_stdin:
|
||||
_emit_card(cards[0], output_format)
|
||||
else:
|
||||
_emit_cards(cards, output_format)
|
||||
|
||||
|
||||
def _read_codes(
|
||||
*,
|
||||
code: str | None,
|
||||
input_file: Path | None,
|
||||
read_stdin: bool,
|
||||
) -> list[str]:
|
||||
"""Resolve the mutually exclusive input modes supported by `wf explain`."""
|
||||
selected = sum(value is not None for value in (code, input_file)) + int(read_stdin)
|
||||
if selected == 0:
|
||||
raise ExplainInputError("provide a code, --input-file, --stdin, or --list")
|
||||
if selected > 1:
|
||||
raise ExplainInputError("code, --input-file, and --stdin are mutually exclusive")
|
||||
if code is not None:
|
||||
return parse_explain_input(code)
|
||||
if input_file is not None:
|
||||
try:
|
||||
return parse_explain_input(input_file.read_text(encoding="utf-8"))
|
||||
except OSError as exc:
|
||||
message = f"could not read input file {input_file!s}: {exc}"
|
||||
raise ExplainInputError(message) from exc
|
||||
return parse_explain_input(typer.get_text_stream("stdin").read())
|
||||
|
||||
|
||||
def _emit_card(card: ExplainCard, output_format: ExplainFormat) -> None:
|
||||
if output_format is ExplainFormat.JSON:
|
||||
emit_json(card.model_dump(mode="json"))
|
||||
return
|
||||
if output_format is ExplainFormat.MARKDOWN:
|
||||
print(_card_markdown(card))
|
||||
return
|
||||
print(_card_compact(card))
|
||||
|
||||
|
||||
def _emit_cards(cards: list[ExplainCard], output_format: ExplainFormat) -> None:
|
||||
if output_format is ExplainFormat.JSON:
|
||||
emit_json({"entries": [card.model_dump(mode="json") for card in cards]})
|
||||
return
|
||||
if output_format is ExplainFormat.MARKDOWN:
|
||||
print("\n\n".join(_card_markdown(card) for card in cards))
|
||||
return
|
||||
print("\n".join(_card_compact(card) for card in cards))
|
||||
|
||||
|
||||
def _emit_summaries(
|
||||
summaries: list[ExplainSummary],
|
||||
output_format: ExplainFormat,
|
||||
) -> None:
|
||||
if output_format is ExplainFormat.JSON:
|
||||
emit_json(
|
||||
{"entries": [summary.model_dump(mode="json") for summary in summaries]}
|
||||
)
|
||||
return
|
||||
if output_format is ExplainFormat.MARKDOWN:
|
||||
print("\n".join(f"- `{item.code}`: {item.summary}" for item in summaries))
|
||||
return
|
||||
print("\n".join(f"{item.code}: {item.summary}" for item in summaries))
|
||||
|
||||
|
||||
def _card_markdown(card: ExplainCard) -> str:
|
||||
lines = [
|
||||
f"# {card.code}",
|
||||
"",
|
||||
card.summary,
|
||||
"",
|
||||
"## Why It Happens",
|
||||
*[f"- {item}" for item in card.why_it_happens],
|
||||
"",
|
||||
"## How To Fix",
|
||||
*[f"- {item}" for item in card.how_to_fix],
|
||||
]
|
||||
if card.related_docs:
|
||||
lines.extend(["", "## Related Docs", *[f"- {item}" for item in card.related_docs]])
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _card_compact(card: ExplainCard) -> str:
|
||||
return f"{card.code}: {card.summary}"
|
||||
|
||||
|
||||
def _error_message(exc: Exception) -> str:
|
||||
if isinstance(exc, UnknownExplainCode):
|
||||
return f"unknown explain code: {exc.args[0]}"
|
||||
return str(exc)
|
||||
```
|
||||
|
||||
Then modify `src/wf_cli/app.py` so `explain` is registered as a direct command,
|
||||
not as a Typer sub-app:
|
||||
|
||||
```python
|
||||
app.add_typer(caps.app, name="cap")
|
||||
app.add_typer(drafts.app, name="draft")
|
||||
app.add_typer(artifacts.app, name="artifact")
|
||||
app.add_typer(deployments.app, name="deploy")
|
||||
app.add_typer(runs.app, name="run")
|
||||
app.add_typer(docs.app, name="docs")
|
||||
app.add_typer(schema.app, name="schema")
|
||||
app.command("explain")(explain.explain_command)
|
||||
```
|
||||
|
||||
Reason: `wf explain <code> --format markdown` is a single command with options,
|
||||
not a command group. The Typer sub-app callback pattern with
|
||||
`invoke_without_command=True` can misparse callback options as subcommand names
|
||||
when registered via `add_typer()`.
|
||||
|
||||
- [ ] **Step 4: Run CLI tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_explain.py tests/wf_cli/test_app.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 4: Verify Focused Slice
|
||||
|
||||
**Files:**
|
||||
|
||||
- No new files unless formatting changes are required.
|
||||
|
||||
- [ ] **Step 1: Run focused CLI tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli -q
|
||||
```
|
||||
|
||||
Expected: all `tests/wf_cli` tests pass.
|
||||
|
||||
- [ ] **Step 2: Run focused lint**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_cli tests/wf_cli
|
||||
```
|
||||
|
||||
Expected: no lint errors.
|
||||
|
||||
- [ ] **Step 3: Run focused format check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff format --check src/wf_cli tests/wf_cli
|
||||
```
|
||||
|
||||
Expected: no formatting changes required. If this fails, run:
|
||||
|
||||
```bash
|
||||
uv run ruff format src/wf_cli tests/wf_cli
|
||||
```
|
||||
|
||||
Then rerun the format check.
|
||||
|
||||
- [ ] **Step 4: Run type check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- [ ] `wf explain --list` returns lean summaries, not full cards.
|
||||
- [ ] `wf explain <code>` returns a full card.
|
||||
- [ ] `wf explain --input-file` and `wf explain --stdin` support multiple cards.
|
||||
- [ ] Unknown codes fail clearly and include the unknown code string.
|
||||
- [ ] No fuzzy matching or generated prose was added.
|
||||
- [ ] No run/deploy behavior changed.
|
||||
- [ ] The new package has docstrings around the registry/parser boundary.
|
||||
@@ -0,0 +1,715 @@
|
||||
# wf CLI Foundation Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add the `wf_cli` package, Typer entrypoint, shared CLI context/config loader, and JSON IO helpers without implementing workflow commands yet.
|
||||
|
||||
**Architecture:** Create a protocol-neutral `wf_cli` package as a second front door beside `wf_mcp`. The first slice wires Typer command groups and reusable helpers only; later plans will add `deploy/run`, `explain`, and draft authoring commands. The CLI may construct `wf_mcp` service/handler objects through `wf_cli.context` for v1, but command modules should not directly own MCP-specific setup.
|
||||
|
||||
**Tech Stack:** Python 3.14, Typer, Pydantic v2, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `pyproject.toml`
|
||||
- Add `typer>=0.16`.
|
||||
- Add script entrypoint `wf = "wf_cli.app:main"`.
|
||||
|
||||
- Create `src/wf_cli/__init__.py`
|
||||
- Minimal package marker and future exports.
|
||||
|
||||
- Create `src/wf_cli/app.py`
|
||||
- Owns the Typer root app.
|
||||
- Registers lifecycle command groups.
|
||||
- Exposes `main()`.
|
||||
|
||||
- Create `src/wf_cli/context.py`
|
||||
- Owns config loading and service/handler construction.
|
||||
- Reuses `wf_mcp.broker.load_broker_config` and `build_service_from_config` for v1.
|
||||
|
||||
- Create `src/wf_cli/io.py`
|
||||
- Owns JSON input parsing from inline JSON and files.
|
||||
- Owns JSON output formatting.
|
||||
- Provides a small `CliInputError` for bad CLI payloads.
|
||||
|
||||
- Create `src/wf_cli/commands/__init__.py`
|
||||
- Exports command group apps.
|
||||
|
||||
- Create command modules:
|
||||
- `src/wf_cli/commands/caps.py`
|
||||
- `src/wf_cli/commands/drafts.py`
|
||||
- `src/wf_cli/commands/artifacts.py`
|
||||
- `src/wf_cli/commands/deployments.py`
|
||||
- `src/wf_cli/commands/runs.py`
|
||||
- `src/wf_cli/commands/docs.py`
|
||||
- `src/wf_cli/commands/schema.py`
|
||||
- `src/wf_cli/commands/explain.py`
|
||||
|
||||
- Create tests:
|
||||
- `tests/wf_cli/test_app.py`
|
||||
- `tests/wf_cli/test_context.py`
|
||||
- `tests/wf_cli/test_io.py`
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Do not implement real workflow commands in this slice.
|
||||
- Do not duplicate MCP workflow logic.
|
||||
- Do not add draft mutation helpers yet.
|
||||
- Do not add `wf explain` registry entries yet.
|
||||
- Do not add subprocess CLI tests yet; use Typer `CliRunner` and direct function tests.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Typer Dependency And Script
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `pyproject.toml`
|
||||
|
||||
- [ ] **Step 1: Add dependency and entrypoint**
|
||||
|
||||
In `pyproject.toml`, add `typer>=0.16` to `[project].dependencies`:
|
||||
|
||||
```toml
|
||||
dependencies = [
|
||||
"fastmcp>=3.2.4",
|
||||
"httpx>=0.28",
|
||||
"jsonpatch>=1.33",
|
||||
"jsonschema>=4.26",
|
||||
"mcp[cli,rich]>=1",
|
||||
"openapi-core>=0.19",
|
||||
"pydantic>=2",
|
||||
"typer>=0.16",
|
||||
]
|
||||
```
|
||||
|
||||
In `[project.scripts]`, add `wf` while preserving `wf-mcp`:
|
||||
|
||||
```toml
|
||||
[project.scripts]
|
||||
wf = "wf_cli.app:main"
|
||||
wf-mcp = "wf_mcp.cli:main"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Sync dependencies if needed**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv lock
|
||||
```
|
||||
|
||||
Expected: `uv.lock` updates if Typer is not already present transitively.
|
||||
|
||||
If `uv lock` cannot access the network, stop and report the dependency-lock blocker. Do not manually edit `uv.lock`.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Add App Skeleton Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_cli/test_app.py`
|
||||
|
||||
- [ ] **Step 1: Write failing Typer app tests**
|
||||
|
||||
Create `tests/wf_cli/test_app.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from wf_cli.app import app
|
||||
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
def test_wf_help_lists_lifecycle_groups() -> None:
|
||||
result = runner.invoke(app, ["--help"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "cap" in result.output
|
||||
assert "draft" in result.output
|
||||
assert "artifact" in result.output
|
||||
assert "deploy" in result.output
|
||||
assert "run" in result.output
|
||||
assert "schema" in result.output
|
||||
assert "explain" in result.output
|
||||
|
||||
|
||||
def test_wf_run_group_help_exists() -> None:
|
||||
result = runner.invoke(app, ["run", "--help"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "Run workflow deployments" in result.output
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_app.py -q
|
||||
```
|
||||
|
||||
Expected: FAIL because `wf_cli` does not exist.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Create Typer App And Command Groups
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_cli/__init__.py`
|
||||
- Create: `src/wf_cli/app.py`
|
||||
- Create: `src/wf_cli/commands/__init__.py`
|
||||
- Create: `src/wf_cli/commands/caps.py`
|
||||
- Create: `src/wf_cli/commands/drafts.py`
|
||||
- Create: `src/wf_cli/commands/artifacts.py`
|
||||
- Create: `src/wf_cli/commands/deployments.py`
|
||||
- Create: `src/wf_cli/commands/runs.py`
|
||||
- Create: `src/wf_cli/commands/docs.py`
|
||||
- Create: `src/wf_cli/commands/schema.py`
|
||||
- Create: `src/wf_cli/commands/explain.py`
|
||||
|
||||
- [ ] **Step 1: Create package marker**
|
||||
|
||||
Create `src/wf_cli/__init__.py`:
|
||||
|
||||
```python
|
||||
"""Workflow platform command-line interface."""
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Create command group modules**
|
||||
|
||||
Create `src/wf_cli/commands/caps.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import typer
|
||||
|
||||
app = typer.Typer(
|
||||
name="cap",
|
||||
help="Inspect and call workflow capabilities.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
```
|
||||
|
||||
Create `src/wf_cli/commands/drafts.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import typer
|
||||
|
||||
app = typer.Typer(
|
||||
name="draft",
|
||||
help="Create, inspect, patch, validate, and save draft workflows.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
```
|
||||
|
||||
Create `src/wf_cli/commands/artifacts.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import typer
|
||||
|
||||
app = typer.Typer(
|
||||
name="artifact",
|
||||
help="List and inspect saved workflow artifacts.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
```
|
||||
|
||||
Create `src/wf_cli/commands/deployments.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import typer
|
||||
|
||||
app = typer.Typer(
|
||||
name="deploy",
|
||||
help="Save, inspect, validate, and delete workflow deployments.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
```
|
||||
|
||||
Create `src/wf_cli/commands/runs.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import typer
|
||||
|
||||
app = typer.Typer(
|
||||
name="run",
|
||||
help="Run workflow deployments and inspect durable runs.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
```
|
||||
|
||||
Create `src/wf_cli/commands/docs.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import typer
|
||||
|
||||
app = typer.Typer(
|
||||
name="docs",
|
||||
help="List and read workflow documentation resources.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
```
|
||||
|
||||
Create `src/wf_cli/commands/schema.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import typer
|
||||
|
||||
app = typer.Typer(
|
||||
name="schema",
|
||||
help="Print expected input shapes for wf commands.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
```
|
||||
|
||||
Create `src/wf_cli/commands/explain.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import typer
|
||||
|
||||
app = typer.Typer(
|
||||
name="explain",
|
||||
help="Explain workflow diagnostic and CLI error codes.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Create command package exports**
|
||||
|
||||
Create `src/wf_cli/commands/__init__.py`:
|
||||
|
||||
```python
|
||||
"""Typer command groups for the wf CLI."""
|
||||
|
||||
from . import artifacts, caps, deployments, docs, drafts, explain, runs, schema
|
||||
|
||||
__all__ = [
|
||||
"artifacts",
|
||||
"caps",
|
||||
"deployments",
|
||||
"docs",
|
||||
"drafts",
|
||||
"explain",
|
||||
"runs",
|
||||
"schema",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Create root app**
|
||||
|
||||
Create `src/wf_cli/app.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Annotated
|
||||
|
||||
import typer
|
||||
|
||||
from .commands import artifacts, caps, deployments, docs, drafts, explain, runs, schema
|
||||
|
||||
app = typer.Typer(
|
||||
name="wf",
|
||||
help="Workflow platform CLI.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
|
||||
|
||||
@app.callback()
|
||||
def root(
|
||||
config: Annotated[
|
||||
str,
|
||||
typer.Option(
|
||||
"--config",
|
||||
help="Path to workflow/MCP config JSON.",
|
||||
),
|
||||
] = "wf_mcp.config.json",
|
||||
) -> None:
|
||||
"""Run workflow platform commands."""
|
||||
# The root callback owns global options only. Command modules should load
|
||||
# context explicitly so tests can call command functions without Typer state.
|
||||
_ = config
|
||||
|
||||
|
||||
app.add_typer(caps.app, name="cap")
|
||||
app.add_typer(drafts.app, name="draft")
|
||||
app.add_typer(artifacts.app, name="artifact")
|
||||
app.add_typer(deployments.app, name="deploy")
|
||||
app.add_typer(runs.app, name="run")
|
||||
app.add_typer(docs.app, name="docs")
|
||||
app.add_typer(schema.app, name="schema")
|
||||
app.add_typer(explain.app, name="explain")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Console script entrypoint for `wf`."""
|
||||
app()
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run app tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_app.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add JSON IO Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_cli/test_io.py`
|
||||
|
||||
- [ ] **Step 1: Write failing IO tests**
|
||||
|
||||
Create `tests/wf_cli/test_io.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from wf_cli.io import CliInputError, emit_json, parse_json_input
|
||||
|
||||
|
||||
def test_parse_json_input_reads_inline_json() -> None:
|
||||
payload = parse_json_input(input_json='{"text": "hello"}', input_file=None)
|
||||
|
||||
assert payload["text"] == "hello"
|
||||
|
||||
|
||||
def test_parse_json_input_reads_file(tmp_path) -> None:
|
||||
path = tmp_path / "payload.json"
|
||||
path.write_text('{"text": "from file"}', encoding="utf-8")
|
||||
|
||||
payload = parse_json_input(input_json=None, input_file=path)
|
||||
|
||||
assert payload["text"] == "from file"
|
||||
|
||||
|
||||
def test_parse_json_input_rejects_both_inline_and_file(tmp_path) -> None:
|
||||
path = tmp_path / "payload.json"
|
||||
path.write_text("{}", encoding="utf-8")
|
||||
|
||||
with pytest.raises(CliInputError, match="mutually exclusive"):
|
||||
parse_json_input(input_json="{}", input_file=path)
|
||||
|
||||
|
||||
def test_parse_json_input_rejects_invalid_json() -> None:
|
||||
with pytest.raises(CliInputError, match="invalid JSON"):
|
||||
parse_json_input(input_json="{", input_file=None)
|
||||
|
||||
|
||||
def test_emit_json_writes_pretty_json(capsys) -> None:
|
||||
emit_json({"ok": True, "items": [1]})
|
||||
captured = capsys.readouterr()
|
||||
payload = json.loads(captured.out)
|
||||
|
||||
assert payload["ok"] is True
|
||||
assert payload["items"][0] == 1
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_io.py -q
|
||||
```
|
||||
|
||||
Expected: FAIL because `wf_cli.io` does not exist.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Implement JSON IO Helpers
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_cli/io.py`
|
||||
|
||||
- [ ] **Step 1: Create IO helpers**
|
||||
|
||||
Create `src/wf_cli/io.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
class CliInputError(ValueError):
|
||||
"""Raised when CLI JSON/file input cannot be parsed safely."""
|
||||
|
||||
|
||||
def parse_json_input(
|
||||
*,
|
||||
input_json: str | None,
|
||||
input_file: Path | None,
|
||||
) -> dict[str, Any]:
|
||||
"""Parse exactly one JSON object from inline JSON or a file path."""
|
||||
if input_json is not None and input_file is not None:
|
||||
raise CliInputError("--input and --input-file are mutually exclusive")
|
||||
if input_json is None and input_file is None:
|
||||
return {}
|
||||
raw = input_json if input_json is not None else _read_input_file(input_file)
|
||||
try:
|
||||
payload = json.loads(raw)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise CliInputError(f"invalid JSON input: {exc.msg}") from exc
|
||||
if not isinstance(payload, dict):
|
||||
raise CliInputError("JSON input must be an object")
|
||||
return payload
|
||||
|
||||
|
||||
def emit_json(payload: Any) -> None:
|
||||
"""Write JSON output in the CLI default machine-readable format."""
|
||||
print(json.dumps(payload, indent=2, sort_keys=True))
|
||||
|
||||
|
||||
def _read_input_file(path: Path | None) -> str:
|
||||
"""Read a required JSON input file."""
|
||||
if path is None:
|
||||
raise CliInputError("input file path is required")
|
||||
try:
|
||||
return path.read_text(encoding="utf-8")
|
||||
except OSError as exc:
|
||||
raise CliInputError(f"could not read input file {path!s}: {exc}") from exc
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run IO tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_io.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Add CLI Context Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_cli/test_context.py`
|
||||
|
||||
- [ ] **Step 1: Write failing context tests**
|
||||
|
||||
Create `tests/wf_cli/test_context.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
from wf_cli.context import load_cli_context
|
||||
|
||||
from tests.wf_mcp.test_support import local_temp_root
|
||||
|
||||
|
||||
def test_load_cli_context_builds_service_and_handlers() -> None:
|
||||
tmp_path = local_temp_root() / "wf_cli_context"
|
||||
tmp_path.mkdir(parents=True, exist_ok=True)
|
||||
config_path = tmp_path / "wf_mcp.config.json"
|
||||
config_path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"store_root": ".wf_mcp_store",
|
||||
"connections": [
|
||||
{
|
||||
"id": "demo.personal",
|
||||
"server": "demo",
|
||||
"account": "personal",
|
||||
}
|
||||
],
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
context = load_cli_context(config_path)
|
||||
|
||||
assert context.config_path == config_path
|
||||
assert context.service.connections.list_all()[0].id == "demo.personal"
|
||||
assert context.handlers.service is context.service
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_context.py -q
|
||||
```
|
||||
|
||||
Expected: FAIL because `wf_cli.context` does not exist.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Implement CLI Context Loader
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_cli/context.py`
|
||||
|
||||
- [ ] **Step 1: Create context loader**
|
||||
|
||||
Create `src/wf_cli/context.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from wf_mcp.broker import build_service_from_config, load_broker_config
|
||||
from wf_mcp.broker.service import WfMcpService
|
||||
from wf_mcp.workflow_surface import WorkflowSurfaceHandlers
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CliContext:
|
||||
"""Protocol-neutral CLI handle over the current workflow service stack.
|
||||
|
||||
V1 intentionally reuses wf_mcp service construction because that is where
|
||||
config, store, source, artifact, draft, and run wiring currently lives. Keep
|
||||
this dependency behind context.py so later extraction does not affect every
|
||||
command module.
|
||||
"""
|
||||
|
||||
config_path: Path
|
||||
service: WfMcpService
|
||||
handlers: WorkflowSurfaceHandlers
|
||||
|
||||
|
||||
def load_cli_context(config_path: str | Path) -> CliContext:
|
||||
"""Load config and build workflow-surface handlers for CLI commands."""
|
||||
resolved_config_path = Path(config_path)
|
||||
config = load_broker_config(resolved_config_path)
|
||||
service = build_service_from_config(config)
|
||||
return CliContext(
|
||||
config_path=resolved_config_path,
|
||||
service=service,
|
||||
handlers=WorkflowSurfaceHandlers(service),
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run context tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_context.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Run Foundation Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused CLI tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_app.py tests/wf_cli/test_io.py tests/wf_cli/test_context.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 2: Run Typer help manually through uv**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run wf --help
|
||||
```
|
||||
|
||||
Expected: exit 0 and output lists lifecycle groups including `deploy`, `run`, `draft`, and `explain`.
|
||||
|
||||
- [ ] **Step 3: Run lint on touched files**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_cli tests/wf_cli
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 4: Run format check on touched files**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff format --check src/wf_cli tests/wf_cli
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Run type check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `wf_cli` is a new package, not under `wf_mcp`.
|
||||
- `wf` script exists and `wf-mcp` still exists.
|
||||
- Typer is the only CLI framework used in `wf_cli`.
|
||||
- Command groups exist but do not pretend to implement workflow behavior.
|
||||
- Shared config/service construction lives in `wf_cli.context`.
|
||||
- JSON parsing/printing lives in `wf_cli.io`.
|
||||
- No workflow logic is duplicated from MCP handlers.
|
||||
- No app-domain command groups (`scenario`, `risk`, `decision`, etc.) were added.
|
||||
|
||||
## Notes For Opencode
|
||||
|
||||
- This is a foundation slice. Do not implement `deploy validate` or `run start` here.
|
||||
- If Typer dependency locking fails because of network access, stop and report it.
|
||||
- Keep command modules boring and small.
|
||||
- Do not move stores out of `wf_mcp` yet; only hide the dependency behind `wf_cli.context`.
|
||||
@@ -0,0 +1,678 @@
|
||||
# wf CLI Run And Deploy Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Implement the first useful `wf` CLI vertical slice: `wf deploy validate`, `wf run start`, `wf run inspect`, and `wf run trace`.
|
||||
|
||||
**Architecture:** Keep command modules thin Typer wrappers over `WorkflowSurfaceHandlers` loaded through `wf_cli.context`. Use real file-backed stores in tests so CLI behavior matches the MCP workflow surface. Keep JSON output as the default and reuse `wf_cli.io` for payload parsing/printing.
|
||||
|
||||
**Tech Stack:** Python 3.14, Typer, Pydantic v2, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_cli/app.py`
|
||||
- Store root `--config` in Typer context so subcommands can load the requested config.
|
||||
|
||||
- Modify `src/wf_cli/context.py`
|
||||
- Add `config_path_from_context(ctx)` helper.
|
||||
|
||||
- Modify `src/wf_cli/commands/deployments.py`
|
||||
- Add `validate` command.
|
||||
|
||||
- Modify `src/wf_cli/commands/runs.py`
|
||||
- Add `start`, `inspect`, and `trace` commands.
|
||||
|
||||
- Create `tests/wf_cli/test_run_deploy.py`
|
||||
- End-to-end CLI tests using `CliRunner`.
|
||||
- Seed artifact/deployment data into the configured store.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Do not implement `wf deploy save`, `wf deploy delete`, or `wf deploy inspect`.
|
||||
- Do not implement `wf run resume`.
|
||||
- Do not implement `wf cap`, `wf draft`, `wf explain`, or `wf schema`.
|
||||
- Do not add table/ids/compact formatting.
|
||||
- Do not add stdin input yet; use `--input` and `--input-file`.
|
||||
- Do not mock `WorkflowSurfaceHandlers`; use the real service/store path.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Make Global `--config` Available To Commands
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_cli/app.py`
|
||||
- Modify: `src/wf_cli/context.py`
|
||||
- Modify: `tests/wf_cli/test_app.py`
|
||||
|
||||
- [ ] **Step 1: Add a failing test for config propagation**
|
||||
|
||||
Append to `tests/wf_cli/test_app.py`:
|
||||
|
||||
```python
|
||||
def test_root_callback_stores_config_path() -> None:
|
||||
result = runner.invoke(app, ["--config", "custom.json", "run", "--help"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "Run workflow deployments" in result.output
|
||||
```
|
||||
|
||||
This mostly locks down that root `--config` remains accepted before subcommands.
|
||||
|
||||
- [ ] **Step 2: Update root callback to use Typer context**
|
||||
|
||||
In `src/wf_cli/app.py`, change the callback signature to:
|
||||
|
||||
```python
|
||||
@app.callback()
|
||||
def root(
|
||||
ctx: typer.Context,
|
||||
config: Annotated[
|
||||
str,
|
||||
typer.Option(
|
||||
"--config",
|
||||
help="Path to workflow/MCP config JSON.",
|
||||
),
|
||||
] = "wf_mcp.config.json",
|
||||
) -> None:
|
||||
"""Run workflow platform commands."""
|
||||
ctx.obj = {"config_path": config}
|
||||
```
|
||||
|
||||
Keep `import typer` already present.
|
||||
|
||||
- [ ] **Step 3: Add config helper**
|
||||
|
||||
In `src/wf_cli/context.py`, add:
|
||||
|
||||
```python
|
||||
import typer
|
||||
```
|
||||
|
||||
Then add below `CliContext`:
|
||||
|
||||
```python
|
||||
def config_path_from_context(ctx: typer.Context) -> str:
|
||||
"""Return the root --config path captured by the Typer callback."""
|
||||
obj = ctx.obj if isinstance(ctx.obj, dict) else {}
|
||||
value = obj.get("config_path", "wf_mcp.config.json")
|
||||
return value if isinstance(value, str) else "wf_mcp.config.json"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run app/context tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_app.py tests/wf_cli/test_context.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Add CLI Run/Deploy Test Fixture Helpers
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_cli/test_run_deploy.py`
|
||||
|
||||
- [ ] **Step 1: Create fixture helpers**
|
||||
|
||||
Create `tests/wf_cli/test_run_deploy.py` with this opening:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from wf_artifacts import FileWorkflowArtifactStore, WorkflowDeployment
|
||||
from wf_cli.app import app
|
||||
from wf_mcp.models import ConnectionConfig
|
||||
|
||||
from tests.wf_mcp.test_support import echo_tool, local_temp_root
|
||||
from tests.wf_mcp.workflow_surface.conftest import echo_artifact
|
||||
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
def _write_config(root: Path) -> Path:
|
||||
config_path = root / "wf_mcp.config.json"
|
||||
config_path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"store_root": ".wf_mcp_store",
|
||||
"connections": [
|
||||
{
|
||||
"id": "demo.personal",
|
||||
"server": "demo",
|
||||
"account": "personal",
|
||||
}
|
||||
],
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
return config_path
|
||||
|
||||
|
||||
def _seed_echo_deployment(root: Path) -> Path:
|
||||
config_path = _write_config(root)
|
||||
store_root = root / ".wf_mcp_store"
|
||||
artifact_store = FileWorkflowArtifactStore(store_root)
|
||||
artifact_store.save_artifact(echo_artifact())
|
||||
artifact_store.save_deployment(
|
||||
WorkflowDeployment(
|
||||
id="echo.personal",
|
||||
artifact_id="echo",
|
||||
artifact_version=1,
|
||||
bindings=[{"logical_source": "demo", "concrete_source": "demo.personal"}],
|
||||
)
|
||||
)
|
||||
return config_path
|
||||
```
|
||||
|
||||
This fixture writes the same config/store shape that `wf_cli.context` loads.
|
||||
|
||||
- [ ] **Step 2: Add failing deploy validate test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_wf_deploy_validate_outputs_json() -> None:
|
||||
root = local_temp_root() / "wf_cli_deploy_validate"
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
config_path = _seed_echo_deployment(root)
|
||||
|
||||
result = runner.invoke(
|
||||
app,
|
||||
["--config", str(config_path), "deploy", "validate", "echo.personal"],
|
||||
)
|
||||
|
||||
assert result.exit_code == 0
|
||||
payload = json.loads(result.output)
|
||||
assert payload["deployment_id"] == "echo.personal"
|
||||
assert payload["status"] == "runnable"
|
||||
assert payload["next_actions"]["recommended_next_tool"] == (
|
||||
"wf.workflow.run_deployment"
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run test to verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_run_deploy.py::test_wf_deploy_validate_outputs_json -q
|
||||
```
|
||||
|
||||
Expected: FAIL because `deploy validate` does not exist.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Implement `wf deploy validate`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_cli/commands/deployments.py`
|
||||
|
||||
- [ ] **Step 1: Add imports and command**
|
||||
|
||||
Replace `src/wf_cli/commands/deployments.py` with:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from typing import Annotated
|
||||
|
||||
import typer
|
||||
|
||||
from wf_cli.context import config_path_from_context, load_cli_context
|
||||
from wf_cli.io import emit_json
|
||||
|
||||
app = typer.Typer(
|
||||
name="deploy",
|
||||
help="Save, inspect, validate, and delete workflow deployments.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
|
||||
|
||||
@app.command("validate")
|
||||
def validate_deployment(
|
||||
ctx: typer.Context,
|
||||
deployment_id: Annotated[str, typer.Argument(help="Deployment id to validate.")],
|
||||
live: Annotated[
|
||||
bool,
|
||||
typer.Option(
|
||||
"--live",
|
||||
help="Also perform opt-in upstream liveness checks.",
|
||||
),
|
||||
] = False,
|
||||
) -> None:
|
||||
"""Validate one saved workflow deployment."""
|
||||
context = load_cli_context(config_path_from_context(ctx))
|
||||
payload = asyncio.run(
|
||||
context.handlers.validate_deployment(
|
||||
deployment_id=deployment_id,
|
||||
live_check=live,
|
||||
)
|
||||
)
|
||||
emit_json(payload)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run deploy test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_run_deploy.py::test_wf_deploy_validate_outputs_json -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add Run Command Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_cli/test_run_deploy.py`
|
||||
|
||||
- [ ] **Step 1: Add run start test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_wf_run_start_accepts_inline_json_input() -> None:
|
||||
root = local_temp_root() / "wf_cli_run_start"
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
config_path = _seed_echo_deployment(root)
|
||||
|
||||
result = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"--config",
|
||||
str(config_path),
|
||||
"run",
|
||||
"start",
|
||||
"echo.personal",
|
||||
"--input",
|
||||
'{"text": "hello"}',
|
||||
],
|
||||
)
|
||||
|
||||
assert result.exit_code == 0
|
||||
payload = json.loads(result.output)
|
||||
assert payload["status"] == "completed"
|
||||
assert payload["output"]["echoed"] == "hello"
|
||||
assert isinstance(payload["run_id"], str)
|
||||
assert payload["next_actions"]["can_continue"] is False
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add run start file input test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_wf_run_start_accepts_input_file() -> None:
|
||||
root = local_temp_root() / "wf_cli_run_start_file"
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
config_path = _seed_echo_deployment(root)
|
||||
input_path = root / "input.json"
|
||||
input_path.write_text('{"text": "from file"}', encoding="utf-8")
|
||||
|
||||
result = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"--config",
|
||||
str(config_path),
|
||||
"run",
|
||||
"start",
|
||||
"echo.personal",
|
||||
"--input-file",
|
||||
str(input_path),
|
||||
],
|
||||
)
|
||||
|
||||
assert result.exit_code == 0
|
||||
payload = json.loads(result.output)
|
||||
assert payload["status"] == "completed"
|
||||
assert payload["output"]["echoed"] == "from file"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add inspect and trace test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_wf_run_inspect_and_trace_existing_run() -> None:
|
||||
root = local_temp_root() / "wf_cli_run_inspect_trace"
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
config_path = _seed_echo_deployment(root)
|
||||
start = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"--config",
|
||||
str(config_path),
|
||||
"run",
|
||||
"start",
|
||||
"echo.personal",
|
||||
"--input",
|
||||
'{"text": "hello"}',
|
||||
],
|
||||
)
|
||||
run_id = json.loads(start.output)["run_id"]
|
||||
|
||||
inspected = runner.invoke(
|
||||
app,
|
||||
["--config", str(config_path), "run", "inspect", run_id],
|
||||
)
|
||||
traced = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"--config",
|
||||
str(config_path),
|
||||
"run",
|
||||
"trace",
|
||||
run_id,
|
||||
"--from",
|
||||
"0",
|
||||
"--limit",
|
||||
"1",
|
||||
],
|
||||
)
|
||||
|
||||
assert inspected.exit_code == 0
|
||||
inspected_payload = json.loads(inspected.output)
|
||||
assert inspected_payload["run_id"] == run_id
|
||||
assert inspected_payload["status"] == "completed"
|
||||
assert "trace" not in inspected_payload
|
||||
|
||||
assert traced.exit_code == 0
|
||||
traced_payload = json.loads(traced.output)
|
||||
assert traced_payload["run_id"] == run_id
|
||||
assert traced_payload["trace_start"] == 0
|
||||
assert traced_payload["trace_limit"] == 1
|
||||
assert traced_payload["trace"][0]["node_id"] == "echo"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_run_deploy.py -q
|
||||
```
|
||||
|
||||
Expected: deploy test passes, run tests fail because run commands do not exist.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Implement Run Commands
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_cli/commands/runs.py`
|
||||
|
||||
- [ ] **Step 1: Replace run command module**
|
||||
|
||||
Replace `src/wf_cli/commands/runs.py` with:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
from typing import Annotated
|
||||
|
||||
import typer
|
||||
|
||||
from wf_cli.context import config_path_from_context, load_cli_context
|
||||
from wf_cli.io import CliInputError, emit_json, parse_json_input
|
||||
from wf_mcp.workflow_surface import TraceRange
|
||||
|
||||
app = typer.Typer(
|
||||
name="run",
|
||||
help="Run workflow deployments and inspect durable runs.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
|
||||
|
||||
@app.command("start")
|
||||
def start_run(
|
||||
ctx: typer.Context,
|
||||
deployment_id: Annotated[str, typer.Argument(help="Deployment id to run.")],
|
||||
input_json: Annotated[
|
||||
str | None,
|
||||
typer.Option("--input", help="Workflow input JSON object."),
|
||||
] = None,
|
||||
input_file: Annotated[
|
||||
Path | None,
|
||||
typer.Option("--input-file", help="Path to workflow input JSON object."),
|
||||
] = None,
|
||||
trace_from: Annotated[
|
||||
int | None,
|
||||
typer.Option("--trace-from", min=0, help="Optional trace slice start."),
|
||||
] = None,
|
||||
trace_limit: Annotated[
|
||||
int | None,
|
||||
typer.Option("--trace-limit", min=1, max=100, help="Optional trace slice limit."),
|
||||
] = None,
|
||||
) -> None:
|
||||
"""Start one workflow deployment."""
|
||||
try:
|
||||
workflow_input = parse_json_input(input_json=input_json, input_file=input_file)
|
||||
except CliInputError as exc:
|
||||
raise typer.BadParameter(str(exc)) from exc
|
||||
context = load_cli_context(config_path_from_context(ctx))
|
||||
trace_range = _optional_trace_range(start=trace_from, limit=trace_limit)
|
||||
payload = asyncio.run(
|
||||
context.handlers.run_deployment(
|
||||
deployment_id=deployment_id,
|
||||
workflow_input=workflow_input,
|
||||
trace_range=trace_range,
|
||||
)
|
||||
)
|
||||
emit_json(payload)
|
||||
|
||||
|
||||
@app.command("inspect")
|
||||
def inspect_run(
|
||||
ctx: typer.Context,
|
||||
run_id: Annotated[str, typer.Argument(help="Durable run id to inspect.")],
|
||||
) -> None:
|
||||
"""Inspect a durable run without trace entries."""
|
||||
context = load_cli_context(config_path_from_context(ctx))
|
||||
emit_json(asyncio.run(context.handlers.inspect_run(run_id=run_id)))
|
||||
|
||||
|
||||
@app.command("trace")
|
||||
def trace_run(
|
||||
ctx: typer.Context,
|
||||
run_id: Annotated[str, typer.Argument(help="Durable run id to trace.")],
|
||||
trace_from: Annotated[
|
||||
int,
|
||||
typer.Option("--from", min=0, help="Zero-based trace start offset."),
|
||||
] = 0,
|
||||
limit: Annotated[
|
||||
int,
|
||||
typer.Option("--limit", min=1, max=100, help="Maximum trace entries."),
|
||||
] = 25,
|
||||
) -> None:
|
||||
"""Read a bounded debug trace slice."""
|
||||
context = load_cli_context(config_path_from_context(ctx))
|
||||
payload = asyncio.run(
|
||||
context.handlers.read_run_trace(
|
||||
run_id=run_id,
|
||||
trace_range=TraceRange(start=trace_from, limit=limit),
|
||||
)
|
||||
)
|
||||
emit_json(payload)
|
||||
|
||||
|
||||
def _optional_trace_range(*, start: int | None, limit: int | None) -> TraceRange | None:
|
||||
"""Build a trace range only when the caller requested trace detail."""
|
||||
if start is None and limit is None:
|
||||
return None
|
||||
return TraceRange(start=start or 0, limit=limit or 25)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run CLI run/deploy tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_run_deploy.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Verify Help And Input Error Behavior
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_cli/test_app.py`
|
||||
- Modify: `tests/wf_cli/test_run_deploy.py`
|
||||
|
||||
- [ ] **Step 1: Add help assertions**
|
||||
|
||||
Append to `tests/wf_cli/test_app.py`:
|
||||
|
||||
```python
|
||||
def test_wf_deploy_validate_help_exists() -> None:
|
||||
result = runner.invoke(app, ["deploy", "validate", "--help"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "Validate one saved workflow deployment" in result.output
|
||||
|
||||
|
||||
def test_wf_run_start_help_exists() -> None:
|
||||
result = runner.invoke(app, ["run", "start", "--help"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "--input-file" in result.output
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add bad JSON assertion**
|
||||
|
||||
Append to `tests/wf_cli/test_run_deploy.py`:
|
||||
|
||||
```python
|
||||
def test_wf_run_start_reports_bad_json() -> None:
|
||||
root = local_temp_root() / "wf_cli_run_bad_json"
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
config_path = _seed_echo_deployment(root)
|
||||
|
||||
result = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"--config",
|
||||
str(config_path),
|
||||
"run",
|
||||
"start",
|
||||
"echo.personal",
|
||||
"--input",
|
||||
"{",
|
||||
],
|
||||
)
|
||||
|
||||
assert result.exit_code != 0
|
||||
assert "invalid JSON" in result.output
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_app.py tests/wf_cli/test_run_deploy.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run all CLI tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 2: Run focused workflow surface tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface/test_deployments.py tests/wf_mcp/workflow_surface/test_runs.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 3: Run lint**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_cli tests/wf_cli
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 4: Run format check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff format --check src/wf_cli tests/wf_cli
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Run type check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `wf deploy validate` returns handler JSON with `next_actions`.
|
||||
- `wf run start` accepts `--input` and `--input-file`.
|
||||
- `wf run inspect` omits trace entries.
|
||||
- `wf run trace` requires bounded `--from` / `--limit`.
|
||||
- Commands load the root `--config` path.
|
||||
- Command modules stay thin and do not duplicate workflow logic.
|
||||
- No unrelated CLI groups or draft authoring commands were implemented.
|
||||
|
||||
## Notes For Opencode
|
||||
|
||||
- Keep this to run/deploy commands only.
|
||||
- Do not implement `wf run resume` in this slice.
|
||||
- Do not add `--format`; JSON stays default.
|
||||
- Use real file-backed stores in tests.
|
||||
- If Typer callback context is annoying, keep the smallest helper in `wf_cli.context` rather than passing global state.
|
||||
@@ -0,0 +1,761 @@
|
||||
# wf_api Remove Double Delegation Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Collapse workflow calls from `WorkflowApi -> WfMcpWorkflowApiBackend -> WorkflowSurfaceHandlers -> domain services` to `WorkflowApi -> domain services`.
|
||||
|
||||
**Architecture:** `WorkflowApi` becomes the protocol-neutral application facade that composes `WorkflowCapabilityApi`, `WorkflowDraftApi`, `WorkflowArtifactApi`, `WorkflowDeploymentApi`, and `WorkflowRunApi` from a `WorkflowOperationContext`. MCP and CLI construct `WorkflowApi(context_from_service(service))` directly. `WorkflowSurfaceHandlers` remains only as a temporary compatibility shim for legacy imports/tests, and `WfMcpWorkflowApiBackend` / `WorkflowApiBackend` are removed.
|
||||
|
||||
**Tech Stack:** Python 3.14, `wf_api`, `wf_mcp`, dataclasses, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Current Chain
|
||||
|
||||
```text
|
||||
wf_mcp tools / wf_cli
|
||||
-> WorkflowApi
|
||||
-> WfMcpWorkflowApiBackend
|
||||
-> WorkflowSurfaceHandlers
|
||||
-> WorkflowCapabilityApi / WorkflowDraftApi / WorkflowArtifactApi / WorkflowDeploymentApi / WorkflowRunApi
|
||||
```
|
||||
|
||||
This creates two mechanical delegation layers. Adding one workflow operation currently requires touching at least `WorkflowApi`, `WorkflowApiBackend`, `WfMcpWorkflowApiBackend`, and usually `WorkflowSurfaceHandlers`.
|
||||
|
||||
## Target Chain
|
||||
|
||||
```text
|
||||
wf_mcp tools / wf_cli
|
||||
-> WorkflowApi
|
||||
-> WorkflowCapabilityApi / WorkflowDraftApi / WorkflowArtifactApi / WorkflowDeploymentApi / WorkflowRunApi
|
||||
```
|
||||
|
||||
Legacy imports of `WorkflowSurfaceHandlers` may still work, but only as a thin wrapper around `WorkflowApi`.
|
||||
|
||||
## Files
|
||||
|
||||
- Modify: `src/wf_api/models.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
- Modify: `src/wf_api/service.py`
|
||||
- Delete: `src/wf_api/backend.py`
|
||||
- Delete: `src/wf_mcp/broker/service/workflow_api_backend.py`
|
||||
- Modify: `src/wf_cli/context.py`
|
||||
- Modify: `src/wf_cli/commands/runs.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/tools.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Modify: `tests/wf_cli/test_context.py`
|
||||
- Modify: `tests/wf_api/test_cli_context_uses_api.py`
|
||||
- Add: `tests/wf_api/test_direct_service.py`
|
||||
- Add: `tests/wf_api/test_no_double_delegation.py`
|
||||
- Modify docs: `docs/current_roadmap.md`, `docs/wf_mcp_architecture.md`, `docs/superpowers/plans/2026-06-01-wf-api-extraction-roadmap.md`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Move TraceRange Out of `backend.py`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/models.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
- Modify: `src/wf_cli/commands/runs.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/tools.py`
|
||||
|
||||
- [ ] **Step 1: Write import regression test**
|
||||
|
||||
Add to `tests/wf_api/test_raw_workflow_plan_extraction.py`:
|
||||
|
||||
```python
|
||||
def test_trace_range_exports_from_wf_api_models() -> None:
|
||||
from wf_api import TraceRange
|
||||
from wf_api.models import TraceRange as CanonicalTraceRange
|
||||
|
||||
assert TraceRange is CanonicalTraceRange
|
||||
assert TraceRange(start=1, limit=2).start == 1
|
||||
assert TraceRange(start=1, limit=2).limit == 2
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run failing test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api\test_raw_workflow_plan_extraction.py::test_trace_range_exports_from_wf_api_models -q
|
||||
```
|
||||
|
||||
Expected: fail because `wf_api.models.TraceRange` does not exist yet.
|
||||
|
||||
- [ ] **Step 3: Add `TraceRange` to `wf_api.models`**
|
||||
|
||||
In `src/wf_api/models.py`, add imports:
|
||||
|
||||
```python
|
||||
from dataclasses import dataclass
|
||||
```
|
||||
|
||||
Then add before `RawWorkflowPlan`:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class TraceRange:
|
||||
"""Caller-bounded debug trace slice for durable deployment runs."""
|
||||
|
||||
start: int = 0
|
||||
limit: int = 25
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update `wf_api.__init__` export**
|
||||
|
||||
Change:
|
||||
|
||||
```python
|
||||
from .backend import TraceRange, WorkflowApiBackend
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
from .models import RawWorkflowPlan, TraceRange
|
||||
```
|
||||
|
||||
Remove `"WorkflowApiBackend"` from `__all__`. Keep `"TraceRange"`.
|
||||
|
||||
If `RawWorkflowPlan` was not exported before, include it only if already expected by tests; do not add new public API unless the existing file already imports it elsewhere.
|
||||
|
||||
- [ ] **Step 5: Update imports that referenced `wf_api.backend.TraceRange`**
|
||||
|
||||
In `src/wf_cli/commands/runs.py`, replace:
|
||||
|
||||
```python
|
||||
from wf_api.backend import TraceRange
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
from wf_api import TraceRange
|
||||
```
|
||||
|
||||
In `src/wf_mcp/workflow_surface/tools.py`, remove:
|
||||
|
||||
```python
|
||||
from wf_api.backend import TraceRange as ApiTraceRange
|
||||
```
|
||||
|
||||
Also remove `_to_api_trace_range()`. Later tasks pass MCP `TraceRange` directly because `WorkflowRunApi` validates trace ranges structurally through `TraceRangeLike`.
|
||||
|
||||
- [ ] **Step 6: Verify Task 1**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api\test_raw_workflow_plan_extraction.py::test_trace_range_exports_from_wf_api_models tests\wf_cli\test_run_deploy.py tests\wf_mcp\server\test_tools.py -q
|
||||
uv run ruff check src\wf_api\models.py src\wf_api\__init__.py src\wf_cli\commands\runs.py src\wf_mcp\workflow_surface\tools.py tests\wf_api\test_raw_workflow_plan_extraction.py
|
||||
uv run ruff format --check src\wf_api\models.py src\wf_api\__init__.py src\wf_cli\commands\runs.py src\wf_mcp\workflow_surface\tools.py tests\wf_api\test_raw_workflow_plan_extraction.py
|
||||
```
|
||||
|
||||
Expected: tests pass, lint pass, format pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Make `WorkflowApi` Compose Domain Services Directly
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/service.py`
|
||||
- Add: `tests/wf_api/test_direct_service.py`
|
||||
|
||||
- [ ] **Step 1: Write direct-composition tests**
|
||||
|
||||
Create `tests/wf_api/test_direct_service.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
|
||||
from wf_artifacts import FileWorkflowArtifactStore
|
||||
from wf_api import WorkflowApi
|
||||
from wf_api.artifacts import WorkflowArtifactApi
|
||||
from wf_api.capabilities import WorkflowCapabilityApi
|
||||
from wf_api.deployments import WorkflowDeploymentApi
|
||||
from wf_api.drafts import WorkflowDraftApi
|
||||
from wf_api.runs import WorkflowRunApi
|
||||
from wf_mcp.broker import WfMcpService
|
||||
from wf_mcp.broker.service.workflow_operation_context import context_from_service
|
||||
from wf_mcp.models import ConnectionConfig
|
||||
from wf_mcp.storage import FileStore
|
||||
|
||||
from tests.wf_mcp.test_support import echo_tool, local_temp_root
|
||||
|
||||
|
||||
def _api() -> WorkflowApi:
|
||||
root = local_temp_root() / "wf_api_direct_composition"
|
||||
service = WfMcpService(
|
||||
store=FileStore(root / "mcp"),
|
||||
artifact_store=FileWorkflowArtifactStore(root),
|
||||
)
|
||||
service.register_connection(
|
||||
ConnectionConfig(id="demo.personal", server="demo", account="personal")
|
||||
)
|
||||
service.register_specs("demo.personal", echo_tool)
|
||||
return WorkflowApi(context_from_service(service))
|
||||
|
||||
|
||||
def test_workflow_api_composes_domain_services() -> None:
|
||||
api = _api()
|
||||
|
||||
assert isinstance(api.capabilities, WorkflowCapabilityApi)
|
||||
assert isinstance(api.drafts, WorkflowDraftApi)
|
||||
assert isinstance(api.artifacts, WorkflowArtifactApi)
|
||||
assert isinstance(api.deployments, WorkflowDeploymentApi)
|
||||
assert isinstance(api.runs, WorkflowRunApi)
|
||||
assert not hasattr(api, "backend")
|
||||
|
||||
|
||||
def test_workflow_api_direct_capability_call() -> None:
|
||||
api = _api()
|
||||
|
||||
result = asyncio.run(
|
||||
api.call_capability(
|
||||
qualified_name="demo.personal.echo_tool",
|
||||
payload={"text": "hello"},
|
||||
)
|
||||
)
|
||||
|
||||
assert result["kind"] == "node_spec"
|
||||
assert result["outcome"] == "ok"
|
||||
assert result["output"] == {"echoed": "hello"}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run failing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api\test_direct_service.py -q
|
||||
```
|
||||
|
||||
Expected: fail because `WorkflowApi` still expects a `WorkflowApiBackend`.
|
||||
|
||||
- [ ] **Step 3: Rewrite `WorkflowApi.__init__`**
|
||||
|
||||
In `src/wf_api/service.py`, replace:
|
||||
|
||||
```python
|
||||
from .backend import TraceRange, WorkflowApiBackend
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
from .artifacts import WorkflowArtifactApi
|
||||
from .capabilities import WorkflowCapabilityApi
|
||||
from .deployments import WorkflowDeploymentApi
|
||||
from .drafts import WorkflowDraftApi
|
||||
from .models import TraceRange
|
||||
from .operation_context import WorkflowOperationContext
|
||||
from .runs import TraceRangeLike, WorkflowRunApi
|
||||
```
|
||||
|
||||
Replace the class docstring and constructor:
|
||||
|
||||
```python
|
||||
class WorkflowApi:
|
||||
"""Protocol-neutral workflow application facade.
|
||||
|
||||
This facade owns the stable application entry point. It composes the
|
||||
domain APIs from a WorkflowOperationContext so MCP, CLI, and future HTTP
|
||||
callers share one operation surface without importing wf_mcp.
|
||||
"""
|
||||
|
||||
def __init__(self, context: WorkflowOperationContext) -> None:
|
||||
self.context = context
|
||||
self.capabilities = WorkflowCapabilityApi(context)
|
||||
self.drafts = WorkflowDraftApi(context)
|
||||
self.artifacts = WorkflowArtifactApi(context)
|
||||
self.deployments = WorkflowDeploymentApi(context)
|
||||
self.runs = WorkflowRunApi(context)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Replace backend delegations with domain service delegations**
|
||||
|
||||
In `src/wf_api/service.py`, replace these groups:
|
||||
|
||||
Capabilities:
|
||||
|
||||
```python
|
||||
self.backend.list_capabilities(...) -> self.capabilities.list_capabilities(...)
|
||||
self.backend.inspect_capability(...) -> self.capabilities.inspect_capability(...)
|
||||
self.backend.call_capability(...) -> self.capabilities.call_capability(...)
|
||||
self.backend.create_draft_workspace_from_capability(...) -> self.capabilities.create_draft_workspace_from_capability(...)
|
||||
```
|
||||
|
||||
Artifacts:
|
||||
|
||||
```python
|
||||
self.backend.list_artifacts(...) -> self.artifacts.list_artifacts(...)
|
||||
self.backend.inspect_artifact(...) -> self.artifacts.inspect_artifact(...)
|
||||
self.backend.save_artifact(...) -> self.artifacts.save_artifact(...)
|
||||
self.backend.create_artifact_from_plan(...) -> self.artifacts.create_artifact_from_plan(...)
|
||||
self.backend.create_artifact_from_draft(...) -> self.artifacts.create_artifact_from_draft(...)
|
||||
self.backend.create_artifact_from_workspace(...) -> self.artifacts.create_artifact_from_workspace(...)
|
||||
self.backend.create_wrapper_from_workspace(...) -> self.artifacts.create_wrapper_from_workspace(...)
|
||||
```
|
||||
|
||||
Drafts:
|
||||
|
||||
```python
|
||||
self.backend.validate_draft(...) -> self.drafts.validate_draft(...)
|
||||
self.backend.compile_draft(...) -> self.drafts.compile_draft(...)
|
||||
self.backend.patch_draft(...) -> self.drafts.patch_draft(...)
|
||||
self.backend.list_draft_workspaces() -> self.drafts.list_draft_workspaces()
|
||||
self.backend.create_draft_workspace(...) -> self.drafts.create_draft_workspace(...)
|
||||
self.backend.get_draft_workspace(...) -> self.drafts.get_draft_workspace(...)
|
||||
self.backend.delete_draft_workspace(...) -> self.drafts.delete_draft_workspace(...)
|
||||
self.backend.validate_draft_workspace(...) -> self.drafts.validate_draft_workspace(...)
|
||||
self.backend.patch_draft_workspace(...) -> self.drafts.patch_draft_workspace(...)
|
||||
self.backend.set_draft_name(...) -> self.drafts.set_draft_name(...)
|
||||
self.backend.set_draft_route(...) -> self.drafts.set_draft_route(...)
|
||||
self.backend.set_step_input_map(...) -> self.drafts.set_step_input_map(...)
|
||||
self.backend.set_step_output_map(...) -> self.drafts.set_step_output_map(...)
|
||||
self.backend.create_minimal_draft_workspace(...) -> self.drafts.create_minimal_draft_workspace(...)
|
||||
```
|
||||
|
||||
Deployments:
|
||||
|
||||
```python
|
||||
self.backend.list_deployments() -> self.deployments.list_deployments()
|
||||
self.backend.inspect_deployment(...) -> self.deployments.inspect_deployment(...)
|
||||
self.backend.save_deployment(...) -> self.deployments.save_deployment(...)
|
||||
self.backend.delete_deployment(...) -> self.deployments.delete_deployment(...)
|
||||
self.backend.validate_deployment(...) -> self.deployments.validate_deployment(...)
|
||||
```
|
||||
|
||||
Runs:
|
||||
|
||||
```python
|
||||
self.backend.run_deployment(...) -> self.runs.run_deployment(...)
|
||||
self.backend.resume_run(...) -> self.runs.resume_run(...)
|
||||
self.backend.inspect_run(...) -> self.runs.inspect_run(...)
|
||||
self.backend.read_run_trace(...) -> self.runs.read_run_trace(...)
|
||||
```
|
||||
|
||||
For run methods, change type hints from `TraceRange | None` to `TraceRangeLike | None` and `TraceRange` to `TraceRangeLike` so MCP Pydantic `TraceRange` and CLI dataclass `TraceRange` both remain accepted structurally.
|
||||
|
||||
- [ ] **Step 5: Verify Task 2**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api\test_direct_service.py tests\wf_api\test_capability_api.py tests\wf_api\test_drafts_service.py tests\wf_api\test_artifact_api.py tests\wf_api\test_deployment_api.py tests\wf_api\test_run_api.py -q
|
||||
uv run ruff check src\wf_api\service.py tests\wf_api\test_direct_service.py
|
||||
uv run ruff format --check src\wf_api\service.py tests\wf_api\test_direct_service.py
|
||||
```
|
||||
|
||||
Expected: direct service tests and domain API tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Update CLI and MCP Tool Construction
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_cli/context.py`
|
||||
- Modify: `tests/wf_cli/test_context.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/tools.py`
|
||||
- Add: `tests/wf_api/test_no_double_delegation.py`
|
||||
|
||||
- [ ] **Step 1: Write no-backend-chain test**
|
||||
|
||||
Create `tests/wf_api/test_no_double_delegation.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _imports_module(path: Path, module_name: str) -> bool:
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.ImportFrom) and node.module == module_name:
|
||||
return True
|
||||
if isinstance(node, ast.Import):
|
||||
if any(alias.name == module_name for alias in node.names):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def test_cli_and_mcp_tools_do_not_import_backend_adapter() -> None:
|
||||
root = Path(__file__).resolve().parents[2]
|
||||
|
||||
assert not _imports_module(
|
||||
root / "src" / "wf_cli" / "context.py",
|
||||
"wf_mcp.broker.service.workflow_api_backend",
|
||||
)
|
||||
assert not _imports_module(
|
||||
root / "src" / "wf_mcp" / "workflow_surface" / "tools.py",
|
||||
"wf_mcp.broker.service.workflow_api_backend",
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run failing no-backend-chain test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api\test_no_double_delegation.py -q
|
||||
```
|
||||
|
||||
Expected: fail because CLI context and MCP tools still import `WfMcpWorkflowApiBackend`.
|
||||
|
||||
- [ ] **Step 3: Update CLI context**
|
||||
|
||||
In `src/wf_cli/context.py`, remove:
|
||||
|
||||
```python
|
||||
from wf_mcp.broker.service.workflow_api_backend import WfMcpWorkflowApiBackend
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from wf_mcp.broker.service.workflow_operation_context import context_from_service
|
||||
```
|
||||
|
||||
Change `load_cli_context()`:
|
||||
|
||||
```python
|
||||
handlers=WorkflowApi(context_from_service(service)),
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update CLI context test**
|
||||
|
||||
In `tests/wf_cli/test_context.py`, replace the private backend-chain assertion:
|
||||
|
||||
```python
|
||||
assert context.handlers.backend._handlers.service is context.service # type: ignore[attr-defined]
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
assert context.handlers.context.artifact_store is context.service.artifact_store
|
||||
assert context.handlers.context.draft_workspace_store is context.service.draft_workspace_store
|
||||
assert context.handlers.context.run_store is context.service.run_store
|
||||
```
|
||||
|
||||
This tests the public context seam instead of the deleted backend chain.
|
||||
|
||||
- [ ] **Step 5: Update MCP workflow tools**
|
||||
|
||||
In `src/wf_mcp/workflow_surface/tools.py`, remove:
|
||||
|
||||
```python
|
||||
from wf_mcp.broker.service.workflow_api_backend import WfMcpWorkflowApiBackend
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from wf_mcp.broker.service.workflow_operation_context import context_from_service
|
||||
```
|
||||
|
||||
Change:
|
||||
|
||||
```python
|
||||
handlers = WorkflowApi(WfMcpWorkflowApiBackend(service))
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
handlers = WorkflowApi(context_from_service(service))
|
||||
```
|
||||
|
||||
For run tools, pass `request.trace_range` or `trace_range` directly to `handlers.*`. Remove conversions through `ApiTraceRange`.
|
||||
|
||||
- [ ] **Step 6: Verify Task 3**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api\test_no_double_delegation.py tests\wf_cli\test_context.py tests\wf_cli tests\wf_mcp\server\test_tools.py tests\wf_mcp\workflow_surface -q
|
||||
uv run ruff check src\wf_cli\context.py src\wf_mcp\workflow_surface\tools.py tests\wf_cli\test_context.py tests\wf_api\test_no_double_delegation.py
|
||||
uv run ruff format --check src\wf_cli\context.py src\wf_mcp\workflow_surface\tools.py tests\wf_cli\test_context.py tests\wf_api\test_no_double_delegation.py
|
||||
```
|
||||
|
||||
Expected: CLI and MCP workflow tool tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Shrink `WorkflowSurfaceHandlers` to Compatibility Shim
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Modify: `tests/wf_api/test_direct_service.py` or add a small handler shim test
|
||||
|
||||
- [ ] **Step 1: Add compatibility shim test**
|
||||
|
||||
Add to `tests/wf_api/test_direct_service.py`:
|
||||
|
||||
```python
|
||||
def test_workflow_surface_handlers_is_compatibility_shim() -> None:
|
||||
from wf_api import WorkflowApi
|
||||
from wf_mcp.workflow_surface import WorkflowSurfaceHandlers
|
||||
|
||||
root = local_temp_root() / "workflow_surface_handler_shim"
|
||||
service = WfMcpService(
|
||||
store=FileStore(root / "mcp"),
|
||||
artifact_store=FileWorkflowArtifactStore(root),
|
||||
)
|
||||
|
||||
handlers = WorkflowSurfaceHandlers(service)
|
||||
|
||||
assert isinstance(handlers, WorkflowApi)
|
||||
assert handlers.service is service
|
||||
assert handlers.context.artifact_store is service.artifact_store
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run failing compatibility test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api\test_direct_service.py::test_workflow_surface_handlers_is_compatibility_shim -q
|
||||
```
|
||||
|
||||
Expected: fail because `WorkflowSurfaceHandlers` is not a `WorkflowApi` subclass yet.
|
||||
|
||||
- [ ] **Step 3: Replace `WorkflowSurfaceHandlers` implementation**
|
||||
|
||||
Replace `src/wf_mcp/workflow_surface/handlers.py` with:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_api import WorkflowApi
|
||||
|
||||
from ..broker.service.workflow_operation_context import context_from_service
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ..broker.service import WfMcpService
|
||||
|
||||
|
||||
class WorkflowSurfaceHandlers(WorkflowApi):
|
||||
"""Compatibility wrapper for old wf_mcp.workflow_surface imports.
|
||||
|
||||
New code should construct `WorkflowApi(context_from_service(service))`
|
||||
directly. This shim keeps tests and legacy broker artifact tools working
|
||||
while the MCP surface is migrated.
|
||||
"""
|
||||
|
||||
def __init__(self, service: WfMcpService) -> None:
|
||||
self.service = service
|
||||
super().__init__(context_from_service(service))
|
||||
|
||||
|
||||
__all__ = ["WorkflowSurfaceHandlers"]
|
||||
```
|
||||
|
||||
This file should no longer import domain services directly.
|
||||
|
||||
- [ ] **Step 4: Verify handler compatibility**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api\test_direct_service.py::test_workflow_surface_handlers_is_compatibility_shim tests\wf_mcp\workflow_surface tests\wf_mcp\test_saved_subgraphs.py tests\wf_mcp\broker -q
|
||||
uv run ruff check src\wf_mcp\workflow_surface\handlers.py tests\wf_api\test_direct_service.py
|
||||
uv run ruff format --check src\wf_mcp\workflow_surface\handlers.py tests\wf_api\test_direct_service.py
|
||||
```
|
||||
|
||||
Expected: old handler tests pass through the shim.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Delete Backend Protocol and Adapter
|
||||
|
||||
**Files:**
|
||||
|
||||
- Delete: `src/wf_api/backend.py`
|
||||
- Delete: `src/wf_mcp/broker/service/workflow_api_backend.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
- Modify docs that describe the old backend chain
|
||||
|
||||
- [ ] **Step 1: Delete backend files**
|
||||
|
||||
Delete:
|
||||
|
||||
```text
|
||||
src/wf_api/backend.py
|
||||
src/wf_mcp/broker/service/workflow_api_backend.py
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Remove public backend export**
|
||||
|
||||
In `src/wf_api/__init__.py`, ensure there is no import or `__all__` entry for `WorkflowApiBackend`.
|
||||
|
||||
- [ ] **Step 3: Search for live backend references**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "WorkflowApiBackend|WfMcpWorkflowApiBackend|workflow_api_backend|\\.backend" src tests
|
||||
```
|
||||
|
||||
Expected: no live source/test references.
|
||||
|
||||
Historical docs under `docs/superpowers/plans/2026-06-01-*` may still mention the old slice. Do not rewrite historical plans except the active roadmap files named in Task 6.
|
||||
|
||||
- [ ] **Step 4: Verify deletion**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api\test_import_direction.py tests\wf_api\test_no_double_delegation.py tests\wf_api\test_cli_context_uses_api.py -q
|
||||
uv run ruff check src\wf_api src\wf_cli\context.py src\wf_mcp\workflow_surface src\wf_mcp\broker\service tests\wf_api
|
||||
uv run ruff format --check src\wf_api src\wf_cli\context.py src\wf_mcp\workflow_surface src\wf_mcp\broker\service tests\wf_api
|
||||
```
|
||||
|
||||
Expected: tests pass, lint pass, format pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Update Active Docs
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
- Modify: `docs/wf_mcp_architecture.md`
|
||||
- Modify: `docs/superpowers/plans/2026-06-01-wf-api-extraction-roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Update `docs/current_roadmap.md`**
|
||||
|
||||
Replace the bullet that says the next useful slice is removing double-delegation with:
|
||||
|
||||
```markdown
|
||||
- Double-delegation has been removed: CLI and MCP workflow tools construct
|
||||
`WorkflowApi(context_from_service(service))` directly. `WorkflowSurfaceHandlers`
|
||||
remains only as a temporary compatibility shim for older imports.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `docs/wf_mcp_architecture.md`**
|
||||
|
||||
Find the architecture text that contains:
|
||||
|
||||
```text
|
||||
wf_api.WorkflowApi ───> WorkflowApiBackend
|
||||
```
|
||||
|
||||
Replace that diagram/text with:
|
||||
|
||||
```text
|
||||
wf_mcp.workflow_surface.tools
|
||||
-> wf_api.WorkflowApi
|
||||
-> wf_api domain services
|
||||
-> WorkflowOperationContext
|
||||
-> WfMcpService adapters/stores/runtime
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```markdown
|
||||
`WorkflowSurfaceHandlers` is a compatibility shim only. New entrypoints should
|
||||
construct `WorkflowApi(context_from_service(service))` directly.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update active extraction roadmap**
|
||||
|
||||
In `docs/superpowers/plans/2026-06-01-wf-api-extraction-roadmap.md`, add a current-state note near the top:
|
||||
|
||||
```markdown
|
||||
> Current update: the original `WorkflowApiBackend` seam was useful for proving
|
||||
> dependency direction, but has been collapsed. `WorkflowApi` now composes
|
||||
> domain services directly from `WorkflowOperationContext`; MCP owns only
|
||||
> context construction and tool schemas.
|
||||
```
|
||||
|
||||
Do not rewrite the historical task bodies. They describe prior slices.
|
||||
|
||||
- [ ] **Step 4: Verify docs**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git diff --check -- docs\current_roadmap.md docs\wf_mcp_architecture.md docs\superpowers\plans\2026-06-01-wf-api-extraction-roadmap.md
|
||||
```
|
||||
|
||||
Expected: no whitespace errors.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Final Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused workflow API/MCP/CLI tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api tests\wf_cli tests\wf_mcp\workflow_surface tests\wf_mcp\server\test_tools.py tests\wf_mcp\test_saved_subgraphs.py -q
|
||||
```
|
||||
|
||||
Expected: selected tests pass.
|
||||
|
||||
- [ ] **Step 2: Run full suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite passes with known skip/xfail counts.
|
||||
|
||||
- [ ] **Step 3: Run lint and format checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src\wf_api src\wf_cli src\wf_mcp tests\wf_api tests\wf_cli tests\wf_mcp
|
||||
uv run ruff format --check src\wf_api src\wf_cli src\wf_mcp tests\wf_api tests\wf_cli tests\wf_mcp
|
||||
```
|
||||
|
||||
Expected: all checks pass.
|
||||
|
||||
- [ ] **Step 4: Run typecheck**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: `0 errors, 0 warnings, 0 notes`. If the command exits nonzero only because of the known workspace enumeration warning, report that exactly.
|
||||
|
||||
- [ ] **Step 5: Final reference check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "WorkflowApiBackend|WfMcpWorkflowApiBackend|workflow_api_backend|\\.backend" src tests docs\current_roadmap.md docs\wf_mcp_architecture.md
|
||||
```
|
||||
|
||||
Expected: no references in live source/tests/current docs.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: The plan removes `WorkflowApiBackend`, deletes `WfMcpWorkflowApiBackend`, updates CLI/MCP construction, keeps handler compatibility, moves `TraceRange`, and updates active docs.
|
||||
- Placeholder scan: No `TODO`/`TBD` placeholders remain.
|
||||
- Type consistency: `WorkflowApi` accepts `WorkflowOperationContext`; run trace methods accept `TraceRangeLike`; `TraceRange` is a convenience DTO exported from `wf_api.models`.
|
||||
- Scope check: This does not remove all `WorkflowSurfaceHandlers` tests or legacy imports. It reduces handlers to a shim; deleting the shim is a later cleanup once `wf_mcp.broker.artifact_tools` and legacy tests stop importing it.
|
||||
+682
@@ -0,0 +1,682 @@
|
||||
# wf_api Slice 4C: Artifacts And Deployments Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Move saved artifact and deployment operations out of `WorkflowSurfaceHandlers` into protocol-neutral `wf_api` domain services.
|
||||
|
||||
**Architecture:** Add `WorkflowArtifactApi` and `WorkflowDeploymentApi` that depend on `WorkflowOperationContext`, not `WfMcpService`. Keep `WorkflowSurfaceHandlers` public method signatures unchanged and delegate artifact/deployment methods to the new services. Extend operation-context protocols for event emission and live source checks so `wf_api` does not import MCP event factories, adapters, auth, or connections.
|
||||
|
||||
**Tech Stack:** Python 3.14+, `wf_api.operation_context`, `wf_api.drafts`, `wf_artifacts`, `wf_platform`, `wf_core`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### Move In This Slice
|
||||
|
||||
Move these methods from `WorkflowSurfaceHandlers`:
|
||||
|
||||
```text
|
||||
list_artifacts
|
||||
save_artifact
|
||||
create_artifact_from_plan
|
||||
create_artifact_from_draft
|
||||
create_artifact_from_workspace
|
||||
create_wrapper_from_workspace
|
||||
inspect_artifact
|
||||
list_deployments
|
||||
inspect_deployment
|
||||
save_deployment
|
||||
delete_deployment
|
||||
validate_deployment
|
||||
```
|
||||
|
||||
Move or duplicate only the helpers required by those methods:
|
||||
|
||||
```text
|
||||
_available_sources
|
||||
_suggested_self_bindings
|
||||
_observed_node_specs
|
||||
_capability_name
|
||||
_artifact_capability_id
|
||||
_deployment_summary
|
||||
```
|
||||
|
||||
### Do Not Move In This Slice
|
||||
|
||||
Do not move:
|
||||
|
||||
```text
|
||||
list_capabilities
|
||||
inspect_capability
|
||||
call_capability
|
||||
_wrapper_artifact_for_capability_name
|
||||
_wrapper_capability_summaries
|
||||
_wrapper_capability_detail
|
||||
_call_wrapper_artifact
|
||||
run_deployment
|
||||
resume_run
|
||||
inspect_run
|
||||
read_run_trace
|
||||
_raw_plan_from_artifact
|
||||
_run_payload
|
||||
_interrupt_payload
|
||||
```
|
||||
|
||||
Reasons:
|
||||
|
||||
- Capability methods still combine live source specs, saved wrappers, and direct test calls. Move them in Slice 4E.
|
||||
- Run methods depend on durable run checkpoints, runtime preparation, trace slicing, and saved subgraph execution. Move them in Slice 4D.
|
||||
- `_raw_plan_from_artifact` is still needed by wrapper capability calls and run methods. Leave it in `handlers.py` until those domains move or extract it separately.
|
||||
|
||||
### Invariants
|
||||
|
||||
- No public payload changes.
|
||||
- No MCP tool schema changes.
|
||||
- `WorkflowSurfaceHandlers` still exposes the same methods.
|
||||
- `wf_api` imports no `wf_mcp`.
|
||||
- Event construction stays adapter-owned.
|
||||
- Live upstream checks stay adapter-owned.
|
||||
- Temporary private helper duplication is allowed when capability/run methods still need a helper in `handlers.py`.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Extend Operation Context For Events And Live Checks
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/operation_context.py`
|
||||
- Create: `src/wf_mcp/broker/service/workflow_live_checks.py`
|
||||
- Modify: `src/wf_mcp/broker/service/workflow_operation_context.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Test: `tests/wf_api/test_operation_context.py`
|
||||
|
||||
- [ ] **Step 1: Add protocol methods**
|
||||
|
||||
In `src/wf_api/operation_context.py`, update `WorkflowEventRecorder`:
|
||||
|
||||
```python
|
||||
class WorkflowEventRecorder(Protocol):
|
||||
"""Records workflow lifecycle events without exposing MCP event types."""
|
||||
|
||||
def record_event(self, event: object) -> None:
|
||||
"""Record one adapter-native event object."""
|
||||
...
|
||||
|
||||
def record_workflow_event(
|
||||
self,
|
||||
event_type: str,
|
||||
*,
|
||||
capability_id: str,
|
||||
payload: dict[str, Any],
|
||||
) -> None:
|
||||
"""Record one workflow lifecycle event by protocol-neutral fields."""
|
||||
...
|
||||
```
|
||||
|
||||
Update `WorkflowLiveSourceChecker`:
|
||||
|
||||
```python
|
||||
class WorkflowLiveSourceChecker(Protocol):
|
||||
"""Optional hook for validating live external source availability."""
|
||||
|
||||
async def deployment_diagnostics(
|
||||
self,
|
||||
*,
|
||||
deployment: WorkflowDeployment,
|
||||
artifacts: Sequence[WorkflowArtifact],
|
||||
) -> list[DependencyDiagnostic]:
|
||||
"""Return opt-in live-source diagnostics for a deployment tree."""
|
||||
...
|
||||
```
|
||||
|
||||
Add imports:
|
||||
|
||||
```python
|
||||
from collections.abc import Mapping, Sequence
|
||||
from wf_artifacts import DependencyDiagnostic, WorkflowDeployment
|
||||
```
|
||||
|
||||
Remove or keep `available_sources()` only if still used by tests. New moved code should use `deployment_diagnostics(...)`.
|
||||
|
||||
- [ ] **Step 2: Move MCP live-check helper out of handlers**
|
||||
|
||||
Create `src/wf_mcp/broker/service/workflow_live_checks.py` and move these
|
||||
handler-level live-check pieces into it:
|
||||
|
||||
```text
|
||||
LIVE_SOURCE_CHECK_TIMEOUT_SECONDS
|
||||
_LIVE_SOURCE_CHECK_FAILURES
|
||||
live_source_diagnostics
|
||||
_required_live_sources
|
||||
```
|
||||
|
||||
Rename `_live_source_diagnostics(...)` to public module-private-adapter helper
|
||||
`live_source_diagnostics(...)`.
|
||||
|
||||
The new module should own the MCP-only imports:
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
|
||||
import anyio
|
||||
import httpx
|
||||
from mcp.client.streamable_http import StreamableHTTPError
|
||||
from mcp.shared.exceptions import McpError
|
||||
|
||||
from wf_artifacts import DependencyDiagnostic, DiagnosticSeverity, WorkflowArtifact, WorkflowDeployment
|
||||
from wf_mcp.broker.service.adapters import require_adapter
|
||||
from wf_mcp.broker.service.core import WfMcpService
|
||||
```
|
||||
|
||||
Keep the existing docstring explaining that live checks perform opt-in upstream
|
||||
I/O. This split is required to avoid a circular import:
|
||||
|
||||
```text
|
||||
handlers.py -> workflow_operation_context.py -> handlers.py
|
||||
```
|
||||
|
||||
After the move, update `handlers.py` to import `live_source_diagnostics` from
|
||||
the new module for as long as `validate_deployment` still lives in handlers.
|
||||
When Task 4 delegates `validate_deployment`, remove that handler import if it
|
||||
is unused.
|
||||
|
||||
- [ ] **Step 3: Implement MCP adapter methods**
|
||||
|
||||
In `src/wf_mcp/broker/service/workflow_operation_context.py`, import:
|
||||
|
||||
```python
|
||||
from collections.abc import Sequence
|
||||
|
||||
from wf_artifacts import DependencyDiagnostic, WorkflowArtifact, WorkflowDeployment
|
||||
from wf_mcp.events import make_event
|
||||
from wf_mcp.broker.service.workflow_live_checks import live_source_diagnostics
|
||||
```
|
||||
|
||||
Then update event recorder:
|
||||
|
||||
```python
|
||||
def record_workflow_event(
|
||||
self,
|
||||
event_type: str,
|
||||
*,
|
||||
capability_id: str,
|
||||
payload: dict[str, Any],
|
||||
) -> None:
|
||||
self.service._record_event( # noqa: SLF001
|
||||
make_event(event_type, capability_id=capability_id, payload=payload)
|
||||
)
|
||||
```
|
||||
|
||||
Update live source checker:
|
||||
|
||||
```python
|
||||
async def deployment_diagnostics(
|
||||
self,
|
||||
*,
|
||||
deployment: WorkflowDeployment,
|
||||
artifacts: Sequence[WorkflowArtifact],
|
||||
) -> list[DependencyDiagnostic]:
|
||||
return await live_source_diagnostics(
|
||||
self.service,
|
||||
deployment=deployment,
|
||||
artifacts=artifacts,
|
||||
)
|
||||
```
|
||||
|
||||
Do not import handler modules from `workflow_operation_context.py`.
|
||||
|
||||
- [ ] **Step 4: Update operation context tests**
|
||||
|
||||
In `tests/wf_api/test_operation_context.py`, add a test that calls:
|
||||
|
||||
```python
|
||||
operation_context.events.record_workflow_event(
|
||||
"workflow_artifact_saved",
|
||||
capability_id="workflow.demo.v1",
|
||||
payload={"artifact_id": "demo", "version": 1},
|
||||
)
|
||||
```
|
||||
|
||||
Then assert the service recorded an event with stable fields individually. Do not assert full dict equality.
|
||||
|
||||
- [ ] **Step 5: Run focused tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_operation_context.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Create `wf_api.artifacts`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/artifacts.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
- Test: `tests/wf_api/test_artifact_api.py`
|
||||
|
||||
- [ ] **Step 1: Create service skeleton**
|
||||
|
||||
Create `src/wf_api/artifacts.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
from typing import Any
|
||||
|
||||
from wf_artifacts import (
|
||||
ArtifactKind,
|
||||
RequiredCapability,
|
||||
WorkflowArtifact,
|
||||
create_workflow_artifact_from_plan as build_workflow_artifact_from_plan,
|
||||
)
|
||||
from wf_platform import CapabilityRef, NodeSpecInventory
|
||||
|
||||
from .drafts import WorkflowDraftApi
|
||||
from .models import RawWorkflowPlan
|
||||
from .operation_context import WorkflowOperationContext
|
||||
|
||||
|
||||
class WorkflowArtifactApi:
|
||||
"""Saved workflow artifact operations.
|
||||
|
||||
Event construction is intentionally delegated through
|
||||
WorkflowOperationContext so this module stays protocol-neutral.
|
||||
"""
|
||||
|
||||
def __init__(self, context: WorkflowOperationContext) -> None:
|
||||
self.context = context
|
||||
self.drafts = WorkflowDraftApi(context)
|
||||
|
||||
def _artifact_store(self):
|
||||
if self.context.artifact_store is None:
|
||||
raise KeyError("workflow artifact store is not configured")
|
||||
return self.context.artifact_store
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Move artifact methods**
|
||||
|
||||
Move these bodies from `WorkflowSurfaceHandlers`, replacing `self.service` access:
|
||||
|
||||
```text
|
||||
list_artifacts
|
||||
save_artifact
|
||||
create_artifact_from_plan
|
||||
create_artifact_from_draft
|
||||
inspect_artifact
|
||||
```
|
||||
|
||||
Required replacements:
|
||||
|
||||
```python
|
||||
self.service.artifact_store -> self._artifact_store()
|
||||
self.service.workflow_artifact_catalog_entry(artifact) -> self.context.artifacts.workflow_artifact_catalog_entry(artifact)
|
||||
self.service._record_event(make_event(...)) -> self.context.events.record_workflow_event(...)
|
||||
_observed_node_specs(self.service) -> _observed_node_specs(self.context)
|
||||
```
|
||||
|
||||
Keep return payloads byte-for-byte equivalent except for dictionary ordering.
|
||||
|
||||
- [ ] **Step 3: Move workspace artifact methods**
|
||||
|
||||
Move:
|
||||
|
||||
```text
|
||||
create_artifact_from_workspace
|
||||
create_wrapper_from_workspace
|
||||
```
|
||||
|
||||
Use `self.context.draft_workspace_store` through `self.drafts` or a local store helper. Preserve current behavior:
|
||||
|
||||
- validate workspace draft first
|
||||
- return `saved: False` with diagnostics when invalid
|
||||
- call `create_artifact_from_draft(...)` when valid
|
||||
- wrapper path passes `kind="wrapper"`
|
||||
|
||||
- [ ] **Step 4: Add helpers**
|
||||
|
||||
Add private helpers to `wf_api.artifacts`:
|
||||
|
||||
```text
|
||||
_required_capability_payloads
|
||||
_suggested_self_bindings
|
||||
_observed_node_specs
|
||||
_plan_nodes
|
||||
_artifact_capability_id
|
||||
```
|
||||
|
||||
Duplicate `_required_capability_payloads`, `_observed_node_specs`, and `_plan_nodes` from `wf_api.drafts` for now instead of importing private draft helpers. We can consolidate after Slice 4C if duplication becomes annoying.
|
||||
|
||||
Do not remove `_artifact_capability_id` from `handlers.py`; wrapper capability methods still need it until Slice 4E.
|
||||
|
||||
- [ ] **Step 5: Export artifact service**
|
||||
|
||||
In `src/wf_api/__init__.py`:
|
||||
|
||||
```python
|
||||
from .artifacts import WorkflowArtifactApi
|
||||
```
|
||||
|
||||
Add `"WorkflowArtifactApi"` to `__all__`.
|
||||
|
||||
- [ ] **Step 6: Add focused tests**
|
||||
|
||||
Create `tests/wf_api/test_artifact_api.py` with tests that instantiate `WorkflowArtifactApi(context_from_service(service))`:
|
||||
|
||||
- `save_artifact` stores a `WorkflowArtifact` and returns `saved: True`.
|
||||
- `create_artifact_from_plan` saves an artifact and includes observed node specs.
|
||||
- `create_artifact_from_workspace` returns `saved: False` when workspace validation fails.
|
||||
- `create_wrapper_from_workspace` saves `kind == "wrapper"`.
|
||||
- Handler delegation for `inspect_artifact` returns the same stable fields as direct `WorkflowArtifactApi.inspect_artifact`.
|
||||
|
||||
Use field-by-field assertions unless asserting a known closed model shape.
|
||||
|
||||
- [ ] **Step 7: Run artifact tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_artifact_api.py tests/wf_api/test_drafts_service.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Create `wf_api.deployments`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/deployments.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
- Test: `tests/wf_api/test_deployment_api.py`
|
||||
|
||||
- [ ] **Step 1: Create service skeleton**
|
||||
|
||||
Create `src/wf_api/deployments.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from wf_artifacts import (
|
||||
AvailableCapability,
|
||||
AvailableSource,
|
||||
DependencyDiagnostic,
|
||||
WorkflowArtifact,
|
||||
WorkflowDeployment,
|
||||
hash_json_schema,
|
||||
validate_deployment_dependencies,
|
||||
)
|
||||
from wf_platform import CapabilitySource
|
||||
|
||||
from .next_actions import NextActions
|
||||
from .operation_context import WorkflowOperationContext
|
||||
from .saved_subgraphs import resolve_saved_subgraph_tree, validate_saved_subgraph_tree
|
||||
|
||||
|
||||
class WorkflowDeploymentApi:
|
||||
"""Saved deployment operations and dependency validation."""
|
||||
|
||||
def __init__(self, context: WorkflowOperationContext) -> None:
|
||||
self.context = context
|
||||
|
||||
def _artifact_store(self):
|
||||
if self.context.artifact_store is None:
|
||||
raise KeyError("workflow artifact store is not configured")
|
||||
return self.context.artifact_store
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Move deployment methods**
|
||||
|
||||
Move these bodies from `WorkflowSurfaceHandlers`:
|
||||
|
||||
```text
|
||||
list_deployments
|
||||
inspect_deployment
|
||||
save_deployment
|
||||
delete_deployment
|
||||
validate_deployment
|
||||
```
|
||||
|
||||
Required replacements:
|
||||
|
||||
```python
|
||||
self.service.artifact_store -> self._artifact_store()
|
||||
self.service._record_event(make_event(...)) -> self.context.events.record_workflow_event(...)
|
||||
_available_sources(self.service) -> _available_sources(self.context.capability_sources)
|
||||
```
|
||||
|
||||
For `validate_deployment(live_check=True)`, use:
|
||||
|
||||
```python
|
||||
if live_check and self.context.live_sources is not None:
|
||||
diagnostics.extend(
|
||||
await self.context.live_sources.deployment_diagnostics(
|
||||
deployment=deployment,
|
||||
artifacts=[artifact, *tree.artifacts_by_ref.values()],
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
If `live_check=True` and `live_sources is None`, preserve static validation only. Do not invent a new warning payload in this slice.
|
||||
|
||||
- [ ] **Step 3: Move deployment validation helper**
|
||||
|
||||
Move `_deployment_validation` logic into `WorkflowDeploymentApi` as a private method:
|
||||
|
||||
```python
|
||||
def _deployment_validation(
|
||||
self,
|
||||
deployment_id: str,
|
||||
) -> tuple[WorkflowDeployment, WorkflowArtifact, list[DependencyDiagnostic], SavedSubgraphTree]:
|
||||
...
|
||||
```
|
||||
|
||||
Use `self._artifact_store()` and `_available_sources(self.context.capability_sources)`.
|
||||
|
||||
- [ ] **Step 4: Add helper functions**
|
||||
|
||||
Add private helpers:
|
||||
|
||||
```text
|
||||
_available_sources
|
||||
_capability_name
|
||||
_deployment_summary
|
||||
```
|
||||
|
||||
Adapt `_available_sources` to accept `Mapping[str, CapabilitySource]` instead of `WfMcpService`.
|
||||
|
||||
Do not remove `_available_sources` or `_capability_name` from `handlers.py` if run/capability methods still use them.
|
||||
|
||||
- [ ] **Step 5: Export deployment service**
|
||||
|
||||
In `src/wf_api/__init__.py`:
|
||||
|
||||
```python
|
||||
from .deployments import WorkflowDeploymentApi
|
||||
```
|
||||
|
||||
Add `"WorkflowDeploymentApi"` to `__all__`.
|
||||
|
||||
- [ ] **Step 6: Add focused tests**
|
||||
|
||||
Create `tests/wf_api/test_deployment_api.py` with tests that instantiate `WorkflowDeploymentApi(context_from_service(service))`:
|
||||
|
||||
- `save_deployment` stores and returns stable deployment fields.
|
||||
- `list_deployments` returns compact summaries.
|
||||
- `delete_deployment` removes one deployment.
|
||||
- `validate_deployment(live_check=False)` returns `runnable` for a valid binding.
|
||||
- `validate_deployment(live_check=True)` calls the operation-context live checker. A simple fake context may be easier than MCP service setup for this test.
|
||||
- Handler delegation for `validate_deployment` returns the same stable status/diagnostic fields as direct `WorkflowDeploymentApi.validate_deployment`.
|
||||
|
||||
- [ ] **Step 7: Run deployment tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_deployment_api.py tests/wf_mcp/workflow_surface/test_deployments.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Wire `WorkflowSurfaceHandlers`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
|
||||
- [ ] **Step 1: Add imports**
|
||||
|
||||
```python
|
||||
from wf_api.artifacts import WorkflowArtifactApi
|
||||
from wf_api.deployments import WorkflowDeploymentApi
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Instantiate services**
|
||||
|
||||
In `WorkflowSurfaceHandlers.__init__`, avoid building multiple independent contexts:
|
||||
|
||||
```python
|
||||
context = context_from_service(service)
|
||||
self._drafts = WorkflowDraftApi(context)
|
||||
self._artifacts = WorkflowArtifactApi(context)
|
||||
self._deployments = WorkflowDeploymentApi(context)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Replace moved artifact methods with delegates**
|
||||
|
||||
Replace bodies for:
|
||||
|
||||
```text
|
||||
list_artifacts
|
||||
save_artifact
|
||||
create_artifact_from_plan
|
||||
create_artifact_from_draft
|
||||
create_artifact_from_workspace
|
||||
create_wrapper_from_workspace
|
||||
inspect_artifact
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```python
|
||||
async def inspect_artifact(self, *, artifact_id: str, version: int) -> dict[str, Any]:
|
||||
return await self._artifacts.inspect_artifact(
|
||||
artifact_id=artifact_id,
|
||||
version=version,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Replace moved deployment methods with delegates**
|
||||
|
||||
Replace bodies for:
|
||||
|
||||
```text
|
||||
list_deployments
|
||||
inspect_deployment
|
||||
save_deployment
|
||||
delete_deployment
|
||||
validate_deployment
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```python
|
||||
async def validate_deployment(
|
||||
self,
|
||||
*,
|
||||
deployment_id: str,
|
||||
live_check: bool = False,
|
||||
) -> dict[str, Any]:
|
||||
return await self._deployments.validate_deployment(
|
||||
deployment_id=deployment_id,
|
||||
live_check=live_check,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Remove only unused imports/helpers**
|
||||
|
||||
After delegation, run:
|
||||
|
||||
```powershell
|
||||
rg -n "_available_sources|_suggested_self_bindings|_observed_node_specs|_deployment_summary|_artifact_capability_id|_capability_name" src/wf_mcp/workflow_surface/handlers.py
|
||||
```
|
||||
|
||||
Remove a helper from `handlers.py` only if it has no remaining caller there.
|
||||
|
||||
Expected likely result:
|
||||
|
||||
- `_suggested_self_bindings`, `_observed_node_specs`, `_deployment_summary` can probably be removed.
|
||||
- `_artifact_capability_id`, `_capability_name`, `_available_sources` may still be needed by capability/run methods. Keep them if referenced.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Verification
|
||||
|
||||
- [ ] **Step 1: Run focused wf_api tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py tests/wf_api/test_drafts_service.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run workflow surface tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_mcp/workflow_surface -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run import-direction test**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: pass; `wf_api` has no `wf_mcp` imports.
|
||||
|
||||
- [ ] **Step 4: Run ruff on touched files**
|
||||
|
||||
```powershell
|
||||
uv run ruff check src/wf_api/artifacts.py src/wf_api/deployments.py src/wf_api/operation_context.py src/wf_api/__init__.py src/wf_mcp/broker/service/workflow_operation_context.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py
|
||||
```
|
||||
|
||||
Expected: all checks pass.
|
||||
|
||||
- [ ] **Step 5: Run basedpyright on touched files**
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error src/wf_api/artifacts.py src/wf_api/deployments.py src/wf_api/operation_context.py src/wf_mcp/broker/service/workflow_operation_context.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 6: Optional full suite**
|
||||
|
||||
```powershell
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite passes with the project’s existing skipped/xfailed counts.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `wf_api.artifacts` imports no `wf_mcp`.
|
||||
- `wf_api.deployments` imports no `wf_mcp`.
|
||||
- Event construction remains in `wf_mcp.broker.service.workflow_operation_context`.
|
||||
- Live upstream adapter/auth probing remains in `wf_mcp`.
|
||||
- `WorkflowSurfaceHandlers` public artifact/deployment method signatures are unchanged.
|
||||
- `create_draft_workspace_from_capability` still lives in `WorkflowSurfaceHandlers`.
|
||||
- Capability methods still live in `WorkflowSurfaceHandlers`.
|
||||
- Run methods still live in `WorkflowSurfaceHandlers`.
|
||||
- No public payload shape changed.
|
||||
- No MCP schema changed.
|
||||
- Temporary helper duplication is documented and deliberate.
|
||||
@@ -0,0 +1,574 @@
|
||||
# wf_api Slice 4D: Run Lifecycle Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Move deployment run, resume, stopped-run inspection, and bounded trace reading out of `WorkflowSurfaceHandlers` into a protocol-neutral `wf_api.runs.WorkflowRunApi`.
|
||||
|
||||
**Architecture:** `WorkflowRunApi` depends on `WorkflowOperationContext` and `WorkflowDeploymentApi`, not `WfMcpService`. Runtime execution remains adapter-owned through `WorkflowRuntimeRunner`; run persistence and payload shaping move into `wf_api`. Keep MCP Pydantic request models at the MCP boundary and pass only a structural trace range into `wf_api`.
|
||||
|
||||
**Tech Stack:** Python 3.14+, `wf_api.operation_context`, `wf_api.deployments`, `wf_api.run_lifecycle`, `wf_api.saved_subgraphs`, `wf_artifacts` run store models, `wf_core.RunState`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### Move In This Slice
|
||||
|
||||
Move these methods from `WorkflowSurfaceHandlers` to `wf_api.runs.WorkflowRunApi`:
|
||||
|
||||
```text
|
||||
run_deployment
|
||||
resume_run
|
||||
inspect_run
|
||||
read_run_trace
|
||||
```
|
||||
|
||||
Move or duplicate only the helpers needed by those methods:
|
||||
|
||||
```text
|
||||
_run_store
|
||||
_raw_plan_from_artifact
|
||||
_plan_field
|
||||
_run_payload
|
||||
_interrupt_payload
|
||||
```
|
||||
|
||||
### Do Not Move In This Slice
|
||||
|
||||
Do not move:
|
||||
|
||||
```text
|
||||
list_capabilities
|
||||
inspect_capability
|
||||
call_capability
|
||||
_wrapper_artifact_for_capability_name
|
||||
_wrapper_capability_summaries
|
||||
_wrapper_capability_detail
|
||||
_call_wrapper_artifact
|
||||
```
|
||||
|
||||
Reasons:
|
||||
|
||||
- Capability methods still own wrapper discovery and direct test calls.
|
||||
- `_raw_plan_from_artifact` is still needed by wrapper direct calls in `handlers.py`; duplicate it temporarily in `wf_api.runs` or move it to a small shared `wf_api` helper only if that does not widen the slice.
|
||||
|
||||
### Invariants
|
||||
|
||||
- No public payload changes.
|
||||
- No MCP tool schema changes.
|
||||
- `WorkflowSurfaceHandlers` public run method signatures stay unchanged.
|
||||
- `wf_api` imports no `wf_mcp`.
|
||||
- Runtime event construction remains adapter-owned in `WfMcpService`.
|
||||
- `run_deployment` still persists stopped runs.
|
||||
- `resume_run` still revalidates pinned dependency environments before mutating state.
|
||||
- Trace payloads remain opt-in and bounded by `trace_range`.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Align Runtime Protocol With Actual Runtime Calls
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/operation_context.py`
|
||||
- Modify: `src/wf_mcp/broker/service/workflow_operation_context.py`
|
||||
- Test: `tests/wf_api/test_operation_context.py`
|
||||
|
||||
- [ ] **Step 1: Update `WorkflowRuntimeRunner` protocol**
|
||||
|
||||
In `src/wf_api/operation_context.py`, replace the older generic runtime kwargs with the current deployment-aware shape:
|
||||
|
||||
```python
|
||||
from wf_api.saved_subgraphs import SavedSubgraphTree
|
||||
```
|
||||
|
||||
```python
|
||||
class WorkflowRuntimeRunner(Protocol):
|
||||
"""Runs and resumes workflow plans using an adapter-owned runtime backend."""
|
||||
|
||||
async def run_workflow_from_plan(
|
||||
self,
|
||||
plan: RawWorkflowPlan,
|
||||
workflow_input: dict[str, Any],
|
||||
deployment: WorkflowDeployment | None = None,
|
||||
artifact: WorkflowArtifact | None = None,
|
||||
saved_subgraph_tree: SavedSubgraphTree | None = None,
|
||||
) -> RunState:
|
||||
"""Execute one raw workflow plan and return its run state."""
|
||||
...
|
||||
|
||||
async def resume_workflow_from_plan(
|
||||
self,
|
||||
plan: RawWorkflowPlan,
|
||||
run: RunState,
|
||||
*,
|
||||
resume_payload: dict[str, Any],
|
||||
resume_outcome: str,
|
||||
deployment: WorkflowDeployment | None = None,
|
||||
artifact: WorkflowArtifact | None = None,
|
||||
saved_subgraph_tree: SavedSubgraphTree | None = None,
|
||||
) -> RunState:
|
||||
"""Resume one interrupted raw workflow plan and return its run state."""
|
||||
...
|
||||
```
|
||||
|
||||
Remove unused imports from the protocol file if `AsyncRegistryHandler` or
|
||||
`ReducerDefinition` are no longer needed.
|
||||
|
||||
- [ ] **Step 2: Give adapter methods explicit signatures**
|
||||
|
||||
In `src/wf_mcp/broker/service/workflow_operation_context.py`, replace `**kwargs`
|
||||
runtime adapter methods with explicit signatures matching the protocol:
|
||||
|
||||
```python
|
||||
async def run_workflow_from_plan(
|
||||
self,
|
||||
plan,
|
||||
workflow_input,
|
||||
deployment=None,
|
||||
artifact=None,
|
||||
saved_subgraph_tree=None,
|
||||
):
|
||||
return await self.service.run_workflow_from_plan(
|
||||
plan,
|
||||
workflow_input,
|
||||
deployment=deployment,
|
||||
artifact=artifact,
|
||||
saved_subgraph_tree=saved_subgraph_tree,
|
||||
)
|
||||
```
|
||||
|
||||
Do the same for `resume_workflow_from_plan(...)`.
|
||||
|
||||
- [ ] **Step 3: Run operation-context tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_operation_context.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Create `wf_api.runs`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/runs.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
- Test: `tests/wf_api/test_run_api.py`
|
||||
|
||||
- [ ] **Step 1: Create service skeleton and trace range protocol**
|
||||
|
||||
Create `src/wf_api/runs.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import asdict
|
||||
from typing import Any, Protocol
|
||||
|
||||
from wf_artifacts import (
|
||||
DependencyDiagnostic,
|
||||
RunStore,
|
||||
WorkflowArtifact,
|
||||
WorkflowDeployment,
|
||||
)
|
||||
from wf_core import RunState
|
||||
|
||||
from .deployments import WorkflowDeploymentApi, _available_sources
|
||||
from .models import RawWorkflowPlan
|
||||
from .next_actions import NextActions
|
||||
from .run_lifecycle import (
|
||||
create_pinned_environment,
|
||||
has_blocking_diagnostics,
|
||||
load_stored_run,
|
||||
mark_resume_blocked,
|
||||
persist_stopped_run,
|
||||
restore_interrupted_run,
|
||||
validate_pinned_resume_environment,
|
||||
)
|
||||
from .saved_subgraphs import saved_subgraph_tree_from_snapshots
|
||||
from .operation_context import WorkflowOperationContext
|
||||
|
||||
|
||||
class TraceRangeLike(Protocol):
|
||||
"""Small structural trace range accepted from MCP, CLI, or HTTP adapters."""
|
||||
|
||||
start: int
|
||||
limit: int
|
||||
|
||||
|
||||
class WorkflowRunApi:
|
||||
"""Deployment run lifecycle operations.
|
||||
|
||||
Runtime execution stays behind WorkflowOperationContext.runtime so wf_api
|
||||
does not depend on MCP service internals.
|
||||
"""
|
||||
|
||||
def __init__(self, context: WorkflowOperationContext) -> None:
|
||||
self.context = context
|
||||
self.deployments = WorkflowDeploymentApi(context)
|
||||
|
||||
def _run_store(self) -> RunStore:
|
||||
if self.context.run_store is None:
|
||||
raise KeyError("workflow run store is not configured")
|
||||
return self.context.run_store
|
||||
```
|
||||
|
||||
Use `TraceRangeLike | None` for run methods. This lets handler methods pass
|
||||
their MCP Pydantic `TraceRange` without importing it into `wf_api`.
|
||||
|
||||
- [ ] **Step 2: Export run service**
|
||||
|
||||
In `src/wf_api/__init__.py`:
|
||||
|
||||
```python
|
||||
from .runs import WorkflowRunApi
|
||||
```
|
||||
|
||||
Add `"WorkflowRunApi"` to `__all__`.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Move Run Methods
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/runs.py`
|
||||
|
||||
- [ ] **Step 1: Move `run_deployment`**
|
||||
|
||||
Move the current handler body into `WorkflowRunApi.run_deployment(...)`.
|
||||
|
||||
Required replacements:
|
||||
|
||||
```python
|
||||
self._deployments.deployment_validation(...) -> self.deployments.deployment_validation(...)
|
||||
self.service.run_workflow_from_plan(...) -> self.context.runtime.run_workflow_from_plan(...)
|
||||
self._run_store() -> self._run_store()
|
||||
```
|
||||
|
||||
Call runtime with the same arguments:
|
||||
|
||||
```python
|
||||
run = await self.context.runtime.run_workflow_from_plan(
|
||||
plan,
|
||||
workflow_input,
|
||||
deployment=deployment,
|
||||
artifact=artifact,
|
||||
saved_subgraph_tree=tree,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Move `resume_run`**
|
||||
|
||||
Move the current handler body into `WorkflowRunApi.resume_run(...)`.
|
||||
|
||||
Required replacements:
|
||||
|
||||
```python
|
||||
validate_pinned_resume_environment(..., sources=_available_sources(self.service))
|
||||
```
|
||||
|
||||
becomes:
|
||||
|
||||
```python
|
||||
validate_pinned_resume_environment(
|
||||
record=record,
|
||||
sources=_available_sources(self.context.capability_sources),
|
||||
)
|
||||
```
|
||||
|
||||
Call runtime with:
|
||||
|
||||
```python
|
||||
run = await self.context.runtime.resume_workflow_from_plan(
|
||||
plan,
|
||||
stopped_run,
|
||||
resume_payload=resume_payload,
|
||||
resume_outcome=resume_outcome,
|
||||
deployment=environment.deployment,
|
||||
artifact=environment.root_artifact,
|
||||
saved_subgraph_tree=tree,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Move stopped-run readers**
|
||||
|
||||
Move:
|
||||
|
||||
```text
|
||||
inspect_run
|
||||
read_run_trace
|
||||
```
|
||||
|
||||
Preserve current payload shape:
|
||||
|
||||
- `inspect_run` returns no trace list.
|
||||
- `read_run_trace` returns only `trace_range.start : start + limit`.
|
||||
- both include `trace_count`.
|
||||
- both include `next_actions` via `_run_payload`.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Move Run Helpers
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/runs.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
|
||||
- [ ] **Step 1: Add private helpers to `wf_api.runs`**
|
||||
|
||||
Move or duplicate these helpers into `src/wf_api/runs.py`:
|
||||
|
||||
```text
|
||||
_raw_plan_from_artifact
|
||||
_plan_field
|
||||
_run_payload
|
||||
_interrupt_payload
|
||||
```
|
||||
|
||||
Keep the trace comment inside `_run_payload`:
|
||||
|
||||
```python
|
||||
# Trace entries can grow quickly, so the public run tool only includes
|
||||
# a bounded debug slice when the caller explicitly asks for a range.
|
||||
```
|
||||
|
||||
This comment is important because trace bloat is a public UX boundary.
|
||||
|
||||
- [ ] **Step 2: Keep handler copies only if needed**
|
||||
|
||||
After handler delegation, run:
|
||||
|
||||
```powershell
|
||||
rg -n "_raw_plan_from_artifact|_run_payload|_interrupt_payload|_plan_field" src/wf_mcp/workflow_surface/handlers.py
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- `_raw_plan_from_artifact` likely remains because `_call_wrapper_artifact` still uses it.
|
||||
- `_plan_field` remains if `_raw_plan_from_artifact` remains.
|
||||
- `_run_payload` and `_interrupt_payload` should be removable if no handler run methods remain.
|
||||
|
||||
Remove only helpers with no remaining handler callers.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Wire `WorkflowSurfaceHandlers`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
|
||||
- [ ] **Step 1: Add import**
|
||||
|
||||
```python
|
||||
from wf_api.runs import WorkflowRunApi
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Instantiate run service**
|
||||
|
||||
In `WorkflowSurfaceHandlers.__init__`, reuse the same context object:
|
||||
|
||||
```python
|
||||
context = context_from_service(service)
|
||||
self._drafts = WorkflowDraftApi(context)
|
||||
self._artifacts = WorkflowArtifactApi(context)
|
||||
self._deployments = WorkflowDeploymentApi(context)
|
||||
self._runs = WorkflowRunApi(context)
|
||||
```
|
||||
|
||||
Do not call `context_from_service(service)` separately for every domain service.
|
||||
|
||||
- [ ] **Step 3: Replace run method bodies with delegates**
|
||||
|
||||
Replace:
|
||||
|
||||
```text
|
||||
run_deployment
|
||||
resume_run
|
||||
inspect_run
|
||||
read_run_trace
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```python
|
||||
async def inspect_run(self, *, run_id: str) -> dict[str, Any]:
|
||||
"""Return one durable stopped-run summary without debug trace entries."""
|
||||
return await self._runs.inspect_run(run_id=run_id)
|
||||
```
|
||||
|
||||
For `trace_range`, pass the MCP model object through directly:
|
||||
|
||||
```python
|
||||
return await self._runs.run_deployment(
|
||||
deployment_id=deployment_id,
|
||||
workflow_input=workflow_input,
|
||||
trace_range=trace_range,
|
||||
)
|
||||
```
|
||||
|
||||
`WorkflowRunApi` accepts it structurally through `TraceRangeLike`.
|
||||
|
||||
- [ ] **Step 4: Remove now-unused imports**
|
||||
|
||||
After replacing run methods, remove imports from `handlers.py` only if `ruff`
|
||||
confirms they are unused. Likely candidates:
|
||||
|
||||
```text
|
||||
dataclasses.asdict
|
||||
RunStore
|
||||
run_lifecycle helpers
|
||||
saved_subgraph_tree_from_snapshots
|
||||
```
|
||||
|
||||
Do not remove `SavedSubgraphTree`, `direct_wrapper_interrupt_diagnostic`,
|
||||
`resolve_saved_subgraph_tree`, or `_raw_plan_from_artifact` if wrapper/capability
|
||||
methods still need them.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Add Focused Run API Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_api/test_run_api.py`
|
||||
|
||||
- [ ] **Step 1: Cover unrunnable deployment path**
|
||||
|
||||
Create a test that saves a deployment with missing/unbound requirements and
|
||||
asserts:
|
||||
|
||||
```python
|
||||
result = asyncio.run(api.run_deployment(...))
|
||||
assert result["status"] == "unrunnable"
|
||||
assert result["run_id"] is None
|
||||
assert result["trace_count"] == 0
|
||||
assert result["diagnostics"][0]["code"]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Cover completed run persistence**
|
||||
|
||||
Use existing test helpers (`echo_tool`, local temp store patterns) to register a
|
||||
valid source, save an artifact/deployment, run it, and assert:
|
||||
|
||||
```python
|
||||
assert result["status"] == "completed"
|
||||
assert isinstance(result["run_id"], str)
|
||||
assert result["resume_readiness"] == "not_applicable"
|
||||
assert result["trace_count"] >= 1
|
||||
```
|
||||
|
||||
Then load the run from the run store and assert it exists.
|
||||
|
||||
- [ ] **Step 3: Cover inspect and bounded trace**
|
||||
|
||||
After a completed run:
|
||||
|
||||
```python
|
||||
summary = asyncio.run(api.inspect_run(run_id=run_id))
|
||||
trace = asyncio.run(api.read_run_trace(run_id=run_id, trace_range=SimpleTraceRange(start=0, limit=1)))
|
||||
```
|
||||
|
||||
Assert:
|
||||
|
||||
```python
|
||||
assert "trace" not in summary
|
||||
assert trace["trace_start"] == 0
|
||||
assert trace["trace_limit"] == 1
|
||||
assert len(trace["trace"]) <= 1
|
||||
assert trace["trace_count"] == summary["trace_count"]
|
||||
```
|
||||
|
||||
Define local helper:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class SimpleTraceRange:
|
||||
start: int
|
||||
limit: int
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Cover handler delegation**
|
||||
|
||||
Add one smoke test comparing stable fields from:
|
||||
|
||||
```python
|
||||
handler_result = asyncio.run(WorkflowSurfaceHandlers(service).inspect_run(run_id=run_id))
|
||||
api_result = asyncio.run(WorkflowRunApi(context_from_service(service)).inspect_run(run_id=run_id))
|
||||
```
|
||||
|
||||
Compare `status`, `run_id`, `trace_count`, and `resume_readiness` individually.
|
||||
|
||||
Do not duplicate every old workflow-surface run test. `wf_api` should own run
|
||||
behavior; `wf_mcp` should keep only adapter/schema/delegation coverage.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Verification
|
||||
|
||||
- [ ] **Step 1: Run focused run tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_run_api.py tests/wf_mcp/workflow_surface/test_runs.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run deployment/artifact tests because runs reuse them**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py tests/wf_api/test_operation_context.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run import-direction test**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: pass; `wf_api` has no `wf_mcp` imports.
|
||||
|
||||
- [ ] **Step 4: Run ruff on touched files**
|
||||
|
||||
```powershell
|
||||
uv run ruff check src/wf_api/runs.py src/wf_api/operation_context.py src/wf_api/__init__.py src/wf_mcp/broker/service/workflow_operation_context.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_run_api.py
|
||||
```
|
||||
|
||||
Expected: all checks pass.
|
||||
|
||||
- [ ] **Step 5: Run basedpyright on touched files**
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error src/wf_api/runs.py src/wf_api/operation_context.py src/wf_mcp/broker/service/workflow_operation_context.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_run_api.py
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 6: Optional full suite**
|
||||
|
||||
```powershell
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite passes with the project’s existing skipped/xfailed counts.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `wf_api.runs` imports no `wf_mcp`.
|
||||
- Runtime execution goes through `WorkflowOperationContext.runtime`.
|
||||
- Run persistence uses `WorkflowOperationContext.run_store`.
|
||||
- `WorkflowSurfaceHandlers` public run signatures are unchanged.
|
||||
- `TraceRange` stays structural at the `wf_api` layer.
|
||||
- Trace list remains opt-in and bounded.
|
||||
- `resume_run` still blocks when pinned dependency validation fails.
|
||||
- Capability direct wrapper calls still work because handler keeps `_raw_plan_from_artifact` if needed.
|
||||
- No public payload shape changed.
|
||||
- No MCP schema changed.
|
||||
@@ -0,0 +1,663 @@
|
||||
# wf_api Slice 4E: Capabilities Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Move workflow capability discovery, inspection, direct capability calls, wrapper capability projection, and capability-backed draft bootstrap out of `WorkflowSurfaceHandlers` into `wf_api.capabilities.WorkflowCapabilityApi`.
|
||||
|
||||
**Architecture:** `WorkflowCapabilityApi` depends on `WorkflowOperationContext`, `WorkflowDraftApi`, and existing `wf_api` helper modules. It must not depend on `WfMcpService`, MCP events, MCP tools, or MCP request models. `WorkflowSurfaceHandlers` becomes a thin adapter/delegator for workflow operations.
|
||||
|
||||
**Tech Stack:** Python 3.14+, `wf_api.operation_context`, `wf_api.drafts`, `wf_api.runs`, `wf_api.wrapper_hints`, `wf_api.refs`, `wf_artifacts`, `wf_authoring`, `wf_core.RuntimeContext`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### Move In This Slice
|
||||
|
||||
Move these methods from `WorkflowSurfaceHandlers`:
|
||||
|
||||
```text
|
||||
list_capabilities
|
||||
inspect_capability
|
||||
call_capability
|
||||
create_draft_workspace_from_capability
|
||||
```
|
||||
|
||||
Move these wrapper/capability private methods:
|
||||
|
||||
```text
|
||||
_wrapper_artifact_for_capability_name
|
||||
_wrapper_capability_summaries
|
||||
_wrapper_capability_detail
|
||||
_call_wrapper_artifact
|
||||
```
|
||||
|
||||
Move or duplicate only the helpers needed by those methods:
|
||||
|
||||
```text
|
||||
_schema_field_names
|
||||
_source_id_for_capability
|
||||
_artifact_capability_id
|
||||
_raw_plan_from_artifact
|
||||
_required_capability_payloads
|
||||
_draft_name_from_capability
|
||||
```
|
||||
|
||||
### Do Not Move In This Slice
|
||||
|
||||
Do not move MCP tool registration, MCP request/response Pydantic models, proxy/admin/broker runtime code, or CLI code.
|
||||
|
||||
### Invariants
|
||||
|
||||
- No public payload changes.
|
||||
- No MCP tool schema changes.
|
||||
- `WorkflowSurfaceHandlers` public capability method signatures stay unchanged.
|
||||
- `wf_api` imports no `wf_mcp`.
|
||||
- Direct raw NodeSpec calls still use `build_async_registry`.
|
||||
- Direct wrapper calls still reject unsupported interrupting wrappers through `direct_wrapper_interrupt_diagnostic`.
|
||||
- Full saved workflows still run through deployments, not direct capability calls.
|
||||
- `create_draft_workspace_from_capability` keeps using inspect-capability wrapper hints.
|
||||
|
||||
---
|
||||
|
||||
## Design Notes
|
||||
|
||||
Capability extraction is the final domain split because it spans several concepts:
|
||||
|
||||
- live planner-visible source NodeSpecs
|
||||
- saved wrapper artifacts projected as workflow-facing capabilities
|
||||
- direct capability REPL calls
|
||||
- wrapper calls through workflow execution
|
||||
- wrapper-hint-driven draft bootstrap
|
||||
|
||||
Keep this in one `WorkflowCapabilityApi` for now. Do not create five tiny services unless tests prove the file is too large after extraction.
|
||||
|
||||
Temporary private helper duplication is allowed. A later cleanup can promote common helpers such as `artifact_capability_id`, `raw_plan_from_artifact`, and source snapshots into better shared modules. Do not widen this slice just to make helper names perfect.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Create `wf_api.capabilities`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/capabilities.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
- Test: `tests/wf_api/test_capability_api.py`
|
||||
|
||||
- [ ] **Step 1: Create service skeleton**
|
||||
|
||||
Create `src/wf_api/capabilities.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
from typing import Any
|
||||
|
||||
from wf_artifacts import (
|
||||
DependencyDiagnostic,
|
||||
DiagnosticSeverity,
|
||||
WorkflowArtifact,
|
||||
WorkflowCapabilityRef,
|
||||
)
|
||||
from wf_authoring import build_async_registry
|
||||
from wf_core import RuntimeContext
|
||||
from wf_core.models.steps import InputBinding, OutputBinding
|
||||
from wf_core.paths import GraphSourcePath
|
||||
from wf_platform import CapabilitySource, page_items
|
||||
|
||||
from .drafts import WorkflowDraftApi
|
||||
from .models import RawWorkflowPlan
|
||||
from .operation_context import WorkflowOperationContext
|
||||
from .refs import parse_workflow_surface_capability_id
|
||||
from .saved_subgraphs import direct_wrapper_interrupt_diagnostic
|
||||
from .wrapper_hints import (
|
||||
workflow_output_schema_for_authoring,
|
||||
wrapper_hints_for_capability,
|
||||
)
|
||||
|
||||
|
||||
class WorkflowCapabilityApi:
|
||||
"""Workflow-facing capability discovery, inspection, and REPL calls.
|
||||
|
||||
This service owns the source/wrapper projection, while adapter-specific MCP
|
||||
tool schemas stay outside wf_api.
|
||||
"""
|
||||
|
||||
def __init__(self, context: WorkflowOperationContext) -> None:
|
||||
self.context = context
|
||||
self.drafts = WorkflowDraftApi(context)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add local list helpers**
|
||||
|
||||
Add local helpers rather than importing `wf_mcp.shared`:
|
||||
|
||||
```python
|
||||
def _matches_query(*values: object, query: str | None) -> bool:
|
||||
if query is None:
|
||||
return True
|
||||
needle = query.strip().casefold()
|
||||
if not needle:
|
||||
return True
|
||||
return any(needle in str(value).casefold() for value in values if value is not None)
|
||||
|
||||
|
||||
def _paged_list_payload(
|
||||
key: str,
|
||||
items: Sequence[dict[str, Any]],
|
||||
*,
|
||||
cursor: str | None,
|
||||
limit: int,
|
||||
) -> dict[str, Any]:
|
||||
page = page_items(items, cursor=cursor, limit=limit)
|
||||
return {key: list(page.items), "next_cursor": page.next_cursor, "total": page.total}
|
||||
```
|
||||
|
||||
This duplicates current list behavior without importing MCP shared helpers into
|
||||
`wf_api`.
|
||||
|
||||
- [ ] **Step 3: Export capability service**
|
||||
|
||||
In `src/wf_api/__init__.py`:
|
||||
|
||||
```python
|
||||
from .capabilities import WorkflowCapabilityApi
|
||||
```
|
||||
|
||||
Add `"WorkflowCapabilityApi"` to `__all__`.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Move Discovery And Inspection
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/capabilities.py`
|
||||
|
||||
- [ ] **Step 1: Move `list_capabilities`**
|
||||
|
||||
Move the existing handler body into:
|
||||
|
||||
```python
|
||||
async def list_capabilities(
|
||||
self,
|
||||
*,
|
||||
query: str | None = None,
|
||||
source_id: str | None = None,
|
||||
cursor: str | None = None,
|
||||
limit: int = 50,
|
||||
) -> dict[str, Any]:
|
||||
...
|
||||
```
|
||||
|
||||
Required replacements:
|
||||
|
||||
```python
|
||||
self.service.capability_sources -> self.context.capability_sources
|
||||
self._wrapper_capability_summaries(...) -> self._wrapper_capability_summaries(...)
|
||||
matches_query(...) -> _matches_query(...)
|
||||
paged_list_payload(...) -> _paged_list_payload(...)
|
||||
```
|
||||
|
||||
Preserve sorting and response shape.
|
||||
|
||||
- [ ] **Step 2: Move `inspect_capability`**
|
||||
|
||||
Move the existing handler body into:
|
||||
|
||||
```python
|
||||
async def inspect_capability(self, *, qualified_name: str) -> dict[str, Any]:
|
||||
...
|
||||
```
|
||||
|
||||
Required replacements:
|
||||
|
||||
```python
|
||||
self.service.capability_sources -> self.context.capability_sources
|
||||
self._wrapper_capability_detail(...) -> self._wrapper_capability_detail(...)
|
||||
```
|
||||
|
||||
Preserve:
|
||||
|
||||
- enabled/planner visibility filtering
|
||||
- wrapper detail fallback
|
||||
- `KeyError(f"unknown workflow capability {qualified_name!r}")`
|
||||
- `wrapper_hints` payload
|
||||
|
||||
- [ ] **Step 3: Add helper functions**
|
||||
|
||||
Move or duplicate:
|
||||
|
||||
```text
|
||||
_schema_field_names
|
||||
_artifact_capability_id
|
||||
_required_capability_payloads
|
||||
```
|
||||
|
||||
Do not import private helpers from `wf_api.artifacts` or `wf_api.runs` in this slice unless the import is already public. Private duplication is acceptable here.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Move Wrapper Capability Projection
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/capabilities.py`
|
||||
|
||||
- [ ] **Step 1: Add artifact store helper**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def _artifact_store(self):
|
||||
return self.context.artifact_store
|
||||
```
|
||||
|
||||
Do not raise from this helper. Existing wrapper projection returns no wrapper
|
||||
rows/details when artifact store is absent.
|
||||
|
||||
- [ ] **Step 2: Move `_wrapper_artifact_for_capability_name`**
|
||||
|
||||
Move the existing method, replacing:
|
||||
|
||||
```python
|
||||
self.service.artifact_store -> self.context.artifact_store
|
||||
```
|
||||
|
||||
Preserve current behavior:
|
||||
|
||||
- invalid capability ids return `None`
|
||||
- non-wrapper artifacts return `None`
|
||||
- missing artifact store returns `None`
|
||||
- missing artifact id/version returns `None`
|
||||
|
||||
- [ ] **Step 3: Move `_wrapper_capability_summaries`**
|
||||
|
||||
Move the method and replace:
|
||||
|
||||
```python
|
||||
matches_query(...) -> _matches_query(...)
|
||||
```
|
||||
|
||||
Preserve `source_id not in {None, "workflow"}` filtering and the existing row shape.
|
||||
|
||||
- [ ] **Step 4: Move `_wrapper_capability_detail`**
|
||||
|
||||
Move the method unchanged except helper references now point to local functions.
|
||||
|
||||
Preserve:
|
||||
|
||||
- `kind == "wrapper_artifact"`
|
||||
- `required_capabilities`
|
||||
- `wrapper_hints`
|
||||
- output/input schema fields
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Move Direct Capability Calls
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/capabilities.py`
|
||||
|
||||
- [ ] **Step 1: Move `call_capability`**
|
||||
|
||||
Move the existing handler body into:
|
||||
|
||||
```python
|
||||
async def call_capability(
|
||||
self,
|
||||
*,
|
||||
qualified_name: str,
|
||||
payload: dict[str, Any],
|
||||
deployment_id: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
...
|
||||
```
|
||||
|
||||
Required replacements:
|
||||
|
||||
```python
|
||||
self.service._get_qualified_spec(qualified_name) -> self.context.specs.get_qualified_spec(qualified_name)
|
||||
self.service.capability_sources -> self.context.capability_sources
|
||||
self._call_wrapper_artifact(...) -> self._call_wrapper_artifact(...)
|
||||
```
|
||||
|
||||
Preserve direct NodeSpec call behavior:
|
||||
|
||||
- build handler with `build_async_registry(spec)[spec.name]`
|
||||
- pass `RuntimeContext(current_node_id=spec.name)`
|
||||
- catch `Exception` and return `capability_call_failed` diagnostic payload
|
||||
- successful response returns `kind: "node_spec"` and empty diagnostics
|
||||
|
||||
- [ ] **Step 2: Move `_call_wrapper_artifact`**
|
||||
|
||||
Move the wrapper call method into `WorkflowCapabilityApi`.
|
||||
|
||||
Required replacements:
|
||||
|
||||
```python
|
||||
self.service.artifact_store -> self.context.artifact_store
|
||||
self.service.run_workflow_from_plan(...) -> self.context.runtime.run_workflow_from_plan(...)
|
||||
```
|
||||
|
||||
Call runtime with the same deployment/artifact arguments:
|
||||
|
||||
```python
|
||||
run = await self.context.runtime.run_workflow_from_plan(
|
||||
plan,
|
||||
payload,
|
||||
deployment=deployment,
|
||||
artifact=artifact,
|
||||
)
|
||||
```
|
||||
|
||||
Preserve:
|
||||
|
||||
- `direct_wrapper_interrupt_diagnostic` rejection
|
||||
- deployment target validation
|
||||
- `kind: "wrapper_artifact"`
|
||||
- `outcome: run.status.value`
|
||||
- `output: run.output`
|
||||
|
||||
- [ ] **Step 3: Move raw plan helper**
|
||||
|
||||
Move or duplicate:
|
||||
|
||||
```text
|
||||
_raw_plan_from_artifact
|
||||
_plan_field
|
||||
```
|
||||
|
||||
Do not import private `_raw_plan_from_artifact` from `wf_api.runs` unless you
|
||||
first make it public. Keeping a local copy is acceptable for this slice.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Move Capability-Backed Draft Bootstrap
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/capabilities.py`
|
||||
|
||||
- [ ] **Step 1: Move `create_draft_workspace_from_capability`**
|
||||
|
||||
Move the existing handler body into `WorkflowCapabilityApi`.
|
||||
|
||||
Required replacements:
|
||||
|
||||
```python
|
||||
capability = await self.inspect_capability(...)
|
||||
result = await self.drafts.create_minimal_draft_workspace(...)
|
||||
```
|
||||
|
||||
Preserve:
|
||||
|
||||
- wrapper-hint defaults
|
||||
- explicit `input` overrides `input_map`
|
||||
- explicit `output` overrides `output_map`
|
||||
- returned `wrapper_hints`
|
||||
- returned `next_actions`
|
||||
|
||||
- [ ] **Step 2: Move `_draft_name_from_capability`**
|
||||
|
||||
Move or duplicate:
|
||||
|
||||
```python
|
||||
def _draft_name_from_capability(capability_name: str) -> str:
|
||||
"""Return a stable draft name when caller does not provide one."""
|
||||
return capability_name.replace(".", "_").replace("-", "_")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Wire `WorkflowSurfaceHandlers`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
|
||||
- [ ] **Step 1: Add import and instance**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from wf_api.capabilities import WorkflowCapabilityApi
|
||||
```
|
||||
|
||||
In `WorkflowSurfaceHandlers.__init__`, reuse the existing operation context:
|
||||
|
||||
```python
|
||||
context = context_from_service(service)
|
||||
self._capabilities = WorkflowCapabilityApi(context)
|
||||
self._drafts = WorkflowDraftApi(context)
|
||||
self._artifacts = WorkflowArtifactApi(context)
|
||||
self._deployments = WorkflowDeploymentApi(context)
|
||||
self._runs = WorkflowRunApi(context)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Delegate moved methods**
|
||||
|
||||
Replace method bodies for:
|
||||
|
||||
```text
|
||||
list_capabilities
|
||||
inspect_capability
|
||||
call_capability
|
||||
create_draft_workspace_from_capability
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```python
|
||||
async def inspect_capability(self, *, qualified_name: str) -> dict[str, Any]:
|
||||
"""Return one planner-visible workflow capability contract."""
|
||||
return await self._capabilities.inspect_capability(qualified_name=qualified_name)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Remove moved private methods**
|
||||
|
||||
Remove these from `handlers.py` after delegation:
|
||||
|
||||
```text
|
||||
_wrapper_artifact_for_capability_name
|
||||
_wrapper_capability_summaries
|
||||
_wrapper_capability_detail
|
||||
_call_wrapper_artifact
|
||||
```
|
||||
|
||||
Then run:
|
||||
|
||||
```powershell
|
||||
rg -n "_schema_field_names|_source_id_for_capability|_artifact_capability_id|_raw_plan_from_artifact|_plan_field|_draft_name_from_capability|_required_capability_payloads" src/wf_mcp/workflow_surface/handlers.py
|
||||
```
|
||||
|
||||
Remove each helper only if it has no remaining handler caller. The target after
|
||||
4E should be close to zero private workflow-domain helpers in `handlers.py`.
|
||||
|
||||
- [ ] **Step 4: Prune imports**
|
||||
|
||||
Use `ruff check` to remove unused imports. Likely candidates:
|
||||
|
||||
```text
|
||||
DependencyDiagnostic
|
||||
DiagnosticSeverity
|
||||
WorkflowArtifact
|
||||
WorkflowCapabilityRef
|
||||
CapabilitySource
|
||||
build_async_registry
|
||||
RuntimeContext
|
||||
direct_wrapper_interrupt_diagnostic
|
||||
workflow_output_schema_for_authoring
|
||||
wrapper_hints_for_capability
|
||||
parse_workflow_surface_capability_id
|
||||
matches_query
|
||||
paged_list_payload
|
||||
```
|
||||
|
||||
Do not remove imports still needed by method signatures such as `InputBinding`,
|
||||
`OutputBinding`, `GraphSourcePath`, `TraceRange`, or `RawWorkflowPlan`.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Add Focused Capability API Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_api/test_capability_api.py`
|
||||
|
||||
- [ ] **Step 1: Cover live source capability listing and inspection**
|
||||
|
||||
Build a service with `echo_tool`, adapt with `context_from_service`, instantiate
|
||||
`WorkflowCapabilityApi`, and assert:
|
||||
|
||||
```python
|
||||
listed = asyncio.run(api.list_capabilities())
|
||||
assert listed["total"] >= 1
|
||||
assert any(item["name"] == "demo.personal.echo_tool" for item in listed["capabilities"])
|
||||
|
||||
detail = asyncio.run(api.inspect_capability(qualified_name="demo.personal.echo_tool"))
|
||||
assert detail["name"] == "demo.personal.echo_tool"
|
||||
assert "wrapper_hints" in detail
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Cover direct NodeSpec call**
|
||||
|
||||
Call:
|
||||
|
||||
```python
|
||||
result = asyncio.run(
|
||||
api.call_capability(
|
||||
qualified_name="demo.personal.echo_tool",
|
||||
payload={"text": "hello"},
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
Assert stable fields:
|
||||
|
||||
```python
|
||||
assert result["kind"] == "node_spec"
|
||||
assert result["outcome"] == "ok"
|
||||
assert result["diagnostics"] == []
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Cover saved wrapper projection**
|
||||
|
||||
Save a wrapper artifact and assert:
|
||||
|
||||
- `list_capabilities(source_id="workflow")` includes `kind == "wrapper_artifact"`
|
||||
- `inspect_capability(qualified_name="workflow.<id>.v<version>")` returns wrapper detail
|
||||
- `call_capability(...)` executes the wrapper through runtime and returns `kind == "wrapper_artifact"`
|
||||
|
||||
Use existing artifact helpers where possible. Do not duplicate entire run tests.
|
||||
|
||||
- [ ] **Step 4: Cover capability-backed draft bootstrap**
|
||||
|
||||
Call `create_draft_workspace_from_capability(...)` and assert:
|
||||
|
||||
```python
|
||||
assert result["workspace_id"] == "echo_ws"
|
||||
assert result["revision"] == 1
|
||||
assert "wrapper_hints" in result
|
||||
assert "next_actions" in result
|
||||
```
|
||||
|
||||
Fetch the workspace through `WorkflowDraftApi` and assert the draft uses the
|
||||
expected capability name.
|
||||
|
||||
- [ ] **Step 5: Cover handler delegation smoke**
|
||||
|
||||
Compare stable fields from handler and direct API for one method:
|
||||
|
||||
```python
|
||||
handler_result = asyncio.run(handlers.inspect_capability(qualified_name=name))
|
||||
api_result = asyncio.run(api.inspect_capability(qualified_name=name))
|
||||
assert handler_result["name"] == api_result["name"]
|
||||
assert handler_result["kind"] == api_result["kind"]
|
||||
```
|
||||
|
||||
Do not duplicate every capability behavior test in both layers.
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Verification
|
||||
|
||||
- [ ] **Step 1: Run focused capability tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_capability_api.py tests/wf_mcp/workflow_surface/test_capabilities.py -q
|
||||
```
|
||||
|
||||
If `tests/wf_mcp/workflow_surface/test_capabilities.py` does not exist, run the
|
||||
closest existing workflow-surface capability tests discovered by `rg -n "call_capability|inspect_capability|list_capabilities" tests/wf_mcp`.
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run adjacent API tests**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_drafts_service.py tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py tests/wf_api/test_run_api.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run import-direction test**
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: pass; `wf_api` has no `wf_mcp` imports.
|
||||
|
||||
- [ ] **Step 4: Run ruff on touched files**
|
||||
|
||||
```powershell
|
||||
uv run ruff check src/wf_api/capabilities.py src/wf_api/__init__.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_capability_api.py
|
||||
```
|
||||
|
||||
Expected: all checks pass.
|
||||
|
||||
- [ ] **Step 5: Run basedpyright on touched files**
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error src/wf_api/capabilities.py src/wf_mcp/workflow_surface/handlers.py tests/wf_api/test_capability_api.py
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 6: Optional full suite**
|
||||
|
||||
```powershell
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite passes with the project’s existing skipped/xfailed counts.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- `wf_api.capabilities` imports no `wf_mcp`.
|
||||
- `WorkflowSurfaceHandlers` public capability signatures are unchanged.
|
||||
- `create_draft_workspace_from_capability` moved with capability inspection.
|
||||
- Direct NodeSpec calls still work.
|
||||
- Saved wrapper discovery and direct wrapper calls still work.
|
||||
- Full saved workflows still require deployments.
|
||||
- No public payload shape changed.
|
||||
- No MCP schema changed.
|
||||
- Handler is now mostly a thin compatibility adapter over `wf_api` domain services.
|
||||
|
||||
## Follow-Up Cleanup After 4E
|
||||
|
||||
After this slice lands, consider a cleanup plan to promote shared helpers:
|
||||
|
||||
```text
|
||||
wf_api.runs._raw_plan_from_artifact -> wf_api.artifact_plans.raw_plan_from_artifact
|
||||
wf_api.capabilities._artifact_capability_id -> wf_api.artifact_refs.artifact_capability_id
|
||||
wf_api.deployments._available_sources -> wf_api.source_snapshots.available_sources_from_capability_sources
|
||||
```
|
||||
|
||||
Do not do that cleanup inside 4E unless it is required to remove circular imports
|
||||
or duplicate behavior bugs.
|
||||
+836
@@ -0,0 +1,836 @@
|
||||
# wf_api Slice 5A/5B Helper Consolidation Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Consolidate duplicated workflow API listing, artifact-plan, source-snapshot, and dependency-summary helpers into protocol-neutral `wf_api` modules without changing public payloads.
|
||||
|
||||
**Architecture:** Slice 5A moves workflow list helpers out of MCP-named modules for workflow API callers. Slice 5B promotes duplicated private helpers from `wf_api.capabilities`, `wf_api.artifacts`, `wf_api.drafts`, `wf_api.runs`, and `wf_api.deployments` into focused `wf_api` helper modules. `wf_mcp.shared.pagination` stays untouched because proxy/admin code still uses it.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, `wf_api`, `wf_artifacts`, `wf_platform.page_items`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
- Create `src/wf_api/listing.py` for `matches_query` and `paged_list_payload`.
|
||||
- Create `src/wf_api/artifact_plans.py` for `raw_plan_from_artifact`, `plan_field`, and `plan_nodes`.
|
||||
- Create `src/wf_api/artifact_refs.py` for `artifact_capability_id`.
|
||||
- Create `src/wf_api/capability_requirements.py` for `required_capability_payloads`, `observed_node_specs`, and `required_capabilities_for_plan`.
|
||||
- Create `src/wf_api/source_snapshots.py` for deployment/run source snapshot helpers if currently duplicated in `deployments.py` and `runs.py`.
|
||||
- Update `src/wf_api/{capabilities,artifacts,drafts,runs,deployments}.py` to import the shared helpers.
|
||||
- Update `src/wf_mcp/workflow_surface/handlers.py` no-store `list_artifacts` fallback to use `wf_api.listing.paged_list_payload`.
|
||||
- Add focused `wf_api` tests for the promoted helpers.
|
||||
|
||||
### Out of Scope
|
||||
|
||||
- Do not move or delete `wf_mcp.shared.pagination`; `src/wf_mcp/proxy/tools.py` still uses it.
|
||||
- Do not move event primitives in this slice.
|
||||
- Do not delete `wf_mcp.workflow_surface.*` compatibility shims.
|
||||
- Do not change MCP tool names, response payload shapes, pagination semantics, artifact IDs, or capability IDs.
|
||||
- Do not move request/response Pydantic models out of `wf_mcp.workflow_surface.models`.
|
||||
|
||||
## File Map
|
||||
|
||||
| File | Responsibility |
|
||||
| --- | --- |
|
||||
| `src/wf_api/listing.py` | Workflow API list filtering and common paged response payloads. |
|
||||
| `src/wf_api/artifact_plans.py` | Safe extraction of `RawWorkflowPlan` and plan node dictionaries from saved artifacts. |
|
||||
| `src/wf_api/artifact_refs.py` | Stable workflow artifact capability IDs. |
|
||||
| `src/wf_api/capability_requirements.py` | Required-capability payloads and observed NodeSpec inventory projection. |
|
||||
| `src/wf_api/source_snapshots.py` | Serializable source snapshots used by deployment validation and run resume checks. |
|
||||
| `src/wf_api/__init__.py` | Optional re-exports for public helper symbols. |
|
||||
| `src/wf_api/capabilities.py` | Remove local duplicates; import shared helpers. |
|
||||
| `src/wf_api/artifacts.py` | Remove local duplicates; import shared helpers. |
|
||||
| `src/wf_api/drafts.py` | Remove local duplicates; import shared helpers. |
|
||||
| `src/wf_api/runs.py` | Remove local `raw_plan_from_artifact`; import shared helpers. |
|
||||
| `src/wf_api/deployments.py` | Import source snapshot helper if duplicated there. |
|
||||
| `src/wf_mcp/workflow_surface/handlers.py` | Stop importing workflow list helper from `wf_mcp.shared`. |
|
||||
| `tests/wf_api/test_listing.py` | Unit tests for query matching and paged payload shape. |
|
||||
| `tests/wf_api/test_artifact_helpers.py` | Unit tests for plan extraction, artifact refs, and dependency helper outputs. |
|
||||
| `tests/wf_api/test_import_direction.py` | Existing guard; must continue passing. |
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add `wf_api.listing`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/listing.py`
|
||||
- Test: `tests/wf_api/test_listing.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
|
||||
- [ ] **Step 1: Write focused listing tests**
|
||||
|
||||
Create `tests/wf_api/test_listing.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_api.listing import matches_query, paged_list_payload
|
||||
|
||||
|
||||
def test_matches_query_accepts_empty_or_missing_query() -> None:
|
||||
assert matches_query("Alpha", query=None) is True
|
||||
assert matches_query("Alpha", query=" ") is True
|
||||
|
||||
|
||||
def test_matches_query_searches_non_none_values_case_insensitively() -> None:
|
||||
assert matches_query(None, "Demo Echo", query="echo") is True
|
||||
assert matches_query(None, "Demo Echo", query="missing") is False
|
||||
|
||||
|
||||
def test_paged_list_payload_preserves_common_shape() -> None:
|
||||
payload = paged_list_payload(
|
||||
"nodes",
|
||||
[{"name": "a"}, {"name": "b"}, {"name": "c"}],
|
||||
cursor=None,
|
||||
limit=2,
|
||||
)
|
||||
|
||||
assert payload["nodes"] == [{"name": "a"}, {"name": "b"}]
|
||||
assert payload["total"] == 3
|
||||
assert payload["next_cursor"] is not None
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests and verify import failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_listing.py -q
|
||||
```
|
||||
|
||||
Expected: fail because `wf_api.listing` does not exist.
|
||||
|
||||
- [ ] **Step 3: Add `src/wf_api/listing.py`**
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
from typing import Any, TypeVar
|
||||
|
||||
from wf_platform import page_items
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
|
||||
def matches_query(*values: object, query: str | None) -> bool:
|
||||
"""Return whether a compact discovery row matches a human search query."""
|
||||
if query is None:
|
||||
return True
|
||||
needle = query.strip().casefold()
|
||||
if not needle:
|
||||
return True
|
||||
return any(needle in str(value).casefold() for value in values if value is not None)
|
||||
|
||||
|
||||
def paged_list_payload(
|
||||
key: str,
|
||||
items: Sequence[T],
|
||||
*,
|
||||
cursor: str | None,
|
||||
limit: int,
|
||||
) -> dict[str, Any]:
|
||||
"""Build the shared workflow API list response shape."""
|
||||
page = page_items(items, cursor=cursor, limit=limit)
|
||||
return {
|
||||
key: list(page.items),
|
||||
"next_cursor": page.next_cursor,
|
||||
"total": page.total,
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Export helpers from `src/wf_api/__init__.py`**
|
||||
|
||||
Add imports:
|
||||
|
||||
```python
|
||||
from .listing import matches_query, paged_list_payload
|
||||
```
|
||||
|
||||
Add to `__all__`:
|
||||
|
||||
```python
|
||||
"matches_query",
|
||||
"paged_list_payload",
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run listing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_listing.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Route Current Workflow API Listing Calls Through `wf_api.listing`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/capabilities.py`
|
||||
- Modify: `src/wf_api/artifacts.py`
|
||||
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
|
||||
- Test: existing `tests/wf_api/test_capability_api.py`, `tests/wf_api/test_artifact_api.py`
|
||||
|
||||
- [ ] **Step 1: Replace local listing helpers in `src/wf_api/capabilities.py`**
|
||||
|
||||
Remove local `_matches_query` and `_paged_list_payload`.
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from .listing import matches_query, paged_list_payload
|
||||
```
|
||||
|
||||
Replace calls:
|
||||
|
||||
```python
|
||||
_matches_query(...)
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
matches_query(...)
|
||||
```
|
||||
|
||||
Replace calls:
|
||||
|
||||
```python
|
||||
_paged_list_payload(...)
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
paged_list_payload(...)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Replace local listing helpers in `src/wf_api/artifacts.py`**
|
||||
|
||||
Remove local `_matches_query`, `_paged_list_payload`, the local `TypeVar`, and now-unused `wf_platform.page_items` import.
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from .listing import matches_query, paged_list_payload
|
||||
```
|
||||
|
||||
Replace local helper calls the same way as Task 2 Step 1.
|
||||
|
||||
- [ ] **Step 3: Stop workflow handler fallback from importing `wf_mcp.shared` listing**
|
||||
|
||||
In `src/wf_mcp/workflow_surface/handlers.py`, replace:
|
||||
|
||||
```python
|
||||
from ..shared import paged_list_payload
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
from wf_api.listing import paged_list_payload
|
||||
```
|
||||
|
||||
The no-store fallback must keep returning:
|
||||
|
||||
```python
|
||||
return paged_list_payload("nodes", [], cursor=cursor, limit=limit)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_listing.py tests/wf_api/test_capability_api.py tests/wf_api/test_artifact_api.py tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Add Artifact Plan And Artifact Ref Helpers
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/artifact_plans.py`
|
||||
- Create: `src/wf_api/artifact_refs.py`
|
||||
- Test: `tests/wf_api/test_artifact_helpers.py`
|
||||
|
||||
- [ ] **Step 1: Add failing tests for artifact helper behavior**
|
||||
|
||||
Create `tests/wf_api/test_artifact_helpers.py` with the imports below. If a helper fixture for artifacts already exists in nearby tests, use it; otherwise construct the minimal `WorkflowArtifact` inline with valid fields copied from existing `tests/wf_api/test_artifact_api.py`.
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from wf_api.artifact_plans import plan_field, plan_nodes, raw_plan_from_artifact
|
||||
from wf_api.artifact_refs import artifact_capability_id
|
||||
|
||||
|
||||
def test_artifact_capability_id_uses_workflow_ref_shape(echo_artifact) -> None:
|
||||
assert artifact_capability_id(echo_artifact) == (
|
||||
f"workflow.{echo_artifact.id}.v{echo_artifact.version}"
|
||||
)
|
||||
|
||||
|
||||
def test_raw_plan_from_artifact_preserves_required_plan_fields(echo_artifact) -> None:
|
||||
plan = raw_plan_from_artifact(echo_artifact)
|
||||
|
||||
assert plan.name == echo_artifact.plan["name"]
|
||||
assert plan.start == echo_artifact.plan["start"]
|
||||
assert len(plan.nodes) == len(echo_artifact.plan["nodes"])
|
||||
|
||||
|
||||
def test_plan_field_reports_missing_field(echo_artifact) -> None:
|
||||
broken = echo_artifact.model_copy(
|
||||
update={"plan": {key: value for key, value in echo_artifact.plan.items() if key != "start"}}
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="missing plan field 'start'"):
|
||||
plan_field(broken, "start")
|
||||
|
||||
|
||||
def test_plan_nodes_returns_only_dict_nodes(echo_artifact) -> None:
|
||||
artifact = echo_artifact.model_copy(
|
||||
update={"plan": {**echo_artifact.plan, "nodes": [{"id": "a"}, "bad"]}}
|
||||
)
|
||||
|
||||
assert plan_nodes(artifact) == [{"id": "a"}]
|
||||
```
|
||||
|
||||
If there is no reusable `echo_artifact` fixture, add a private helper in this test file instead of importing fixtures across packages.
|
||||
|
||||
- [ ] **Step 2: Run the tests and verify import failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_artifact_helpers.py -q
|
||||
```
|
||||
|
||||
Expected: fail because modules do not exist.
|
||||
|
||||
- [ ] **Step 3: Create `src/wf_api/artifact_refs.py`**
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_artifacts import WorkflowArtifact, WorkflowCapabilityRef
|
||||
|
||||
|
||||
def artifact_capability_id(artifact: WorkflowArtifact) -> str:
|
||||
"""Return the stable workflow capability name for a saved artifact."""
|
||||
return str(
|
||||
WorkflowCapabilityRef(
|
||||
artifact_id=artifact.id,
|
||||
version=artifact.version,
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Create `src/wf_api/artifact_plans.py`**
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from wf_artifacts import WorkflowArtifact
|
||||
|
||||
from .models import RawWorkflowPlan
|
||||
|
||||
|
||||
def raw_plan_from_artifact(artifact: WorkflowArtifact) -> RawWorkflowPlan:
|
||||
"""Validate the stored raw workflow plan shape expected by runtime calls."""
|
||||
return RawWorkflowPlan.model_validate(
|
||||
{
|
||||
"name": plan_field(artifact, "name"),
|
||||
"input_schema": plan_field(artifact, "input_schema"),
|
||||
"state_schema": plan_field(artifact, "state_schema"),
|
||||
"output_schema": plan_field(artifact, "output_schema"),
|
||||
"outcomes": artifact.plan.get("outcomes", ["ok"]),
|
||||
"output": artifact.plan.get("output", []),
|
||||
"start": plan_field(artifact, "start"),
|
||||
"nodes": plan_field(artifact, "nodes"),
|
||||
"edges": plan_field(artifact, "edges"),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def plan_field(artifact: WorkflowArtifact, field_name: str) -> Any:
|
||||
"""Return one required raw-plan field with an artifact-specific error."""
|
||||
try:
|
||||
return artifact.plan[field_name]
|
||||
except KeyError as exc:
|
||||
raise ValueError(
|
||||
f"workflow artifact {artifact.id}@{artifact.version} "
|
||||
f"is missing plan field {field_name!r}"
|
||||
) from exc
|
||||
|
||||
|
||||
def plan_nodes(artifact: WorkflowArtifact) -> list[dict[str, Any]]:
|
||||
"""Return only dict-shaped node entries from a saved raw plan."""
|
||||
nodes = artifact.plan.get("nodes", [])
|
||||
return [node for node in nodes if isinstance(node, dict)]
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Export helper modules if desired**
|
||||
|
||||
In `src/wf_api/__init__.py`, export only stable helper names if the package already re-exports helpers. If the file is intentionally selective, skip this step and keep imports module-qualified.
|
||||
|
||||
- [ ] **Step 6: Run helper tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_artifact_helpers.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Replace Duplicate Artifact Plan/Ref Helpers In Domain APIs
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/capabilities.py`
|
||||
- Modify: `src/wf_api/artifacts.py`
|
||||
- Modify: `src/wf_api/runs.py`
|
||||
- Test: existing capability/artifact/run API tests
|
||||
|
||||
- [ ] **Step 1: Update `src/wf_api/capabilities.py` imports**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from .artifact_plans import raw_plan_from_artifact
|
||||
from .artifact_refs import artifact_capability_id
|
||||
```
|
||||
|
||||
Remove local `_raw_plan_from_artifact`, `_plan_field`, and `_artifact_capability_id`.
|
||||
|
||||
Replace:
|
||||
|
||||
```python
|
||||
_raw_plan_from_artifact(...)
|
||||
_artifact_capability_id(...)
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
raw_plan_from_artifact(...)
|
||||
artifact_capability_id(...)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `src/wf_api/artifacts.py` imports**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from .artifact_plans import plan_nodes
|
||||
from .artifact_refs import artifact_capability_id
|
||||
```
|
||||
|
||||
Remove local `_plan_nodes` and `_artifact_capability_id`.
|
||||
|
||||
Replace calls with `plan_nodes(...)` and `artifact_capability_id(...)`.
|
||||
|
||||
- [ ] **Step 3: Update `src/wf_api/runs.py` imports**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from .artifact_plans import raw_plan_from_artifact
|
||||
```
|
||||
|
||||
Remove local `_raw_plan_from_artifact` and `_plan_field`.
|
||||
|
||||
Replace calls with `raw_plan_from_artifact(...)`.
|
||||
|
||||
- [ ] **Step 4: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_artifact_helpers.py tests/wf_api/test_capability_api.py tests/wf_api/test_artifact_api.py tests/wf_api/test_run_api.py tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Add Shared Capability Requirement Helpers
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/capability_requirements.py`
|
||||
- Modify: `src/wf_api/drafts.py`
|
||||
- Modify: `src/wf_api/artifacts.py`
|
||||
- Modify: `src/wf_api/capabilities.py`
|
||||
- Test: `tests/wf_api/test_artifact_helpers.py`
|
||||
|
||||
- [ ] **Step 1: Add tests for requirement payload and observed specs**
|
||||
|
||||
Append to `tests/wf_api/test_artifact_helpers.py`:
|
||||
|
||||
```python
|
||||
from wf_api.capability_requirements import (
|
||||
observed_node_specs,
|
||||
required_capability_payloads,
|
||||
)
|
||||
|
||||
|
||||
def test_required_capability_payloads_sorts_by_name(required_capabilities) -> None:
|
||||
payload = required_capability_payloads(required_capabilities)
|
||||
|
||||
assert list(payload) == sorted(required_capabilities)
|
||||
first = next(iter(payload.values()))
|
||||
assert "ref" in first
|
||||
assert "kind" in first
|
||||
|
||||
|
||||
def test_observed_node_specs_projects_enabled_context_specs(operation_context) -> None:
|
||||
observed = observed_node_specs(operation_context)
|
||||
|
||||
assert isinstance(observed, dict)
|
||||
assert all(hasattr(detail, "name") for detail in observed.values())
|
||||
```
|
||||
|
||||
If `required_capabilities` or `operation_context` fixtures do not exist, create explicit local helpers by copying the smallest valid setup from `tests/wf_api/test_artifact_api.py` or `tests/wf_api/test_capability_api.py`. Do not import private helpers from production modules.
|
||||
|
||||
- [ ] **Step 2: Create `src/wf_api/capability_requirements.py`**
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from wf_artifacts import (
|
||||
RequiredCapability,
|
||||
WorkflowArtifact,
|
||||
create_workflow_artifact_from_plan as build_workflow_artifact_from_plan,
|
||||
)
|
||||
from wf_platform import CapabilityRef, NodeSpecInventory
|
||||
|
||||
from .artifact_plans import plan_nodes
|
||||
from .operation_context import WorkflowOperationContext
|
||||
|
||||
|
||||
def required_capability_payloads(
|
||||
requirements: dict[str, RequiredCapability],
|
||||
) -> dict[str, dict[str, Any]]:
|
||||
"""Return deterministic JSON payloads for required capabilities."""
|
||||
return {
|
||||
name: capability.model_dump(mode="json")
|
||||
for name, capability in sorted(requirements.items())
|
||||
}
|
||||
|
||||
|
||||
def observed_node_specs(
|
||||
context: WorkflowOperationContext,
|
||||
) -> dict[str, NodeSpecInventory]:
|
||||
"""Project current executable specs into serializable observed contracts."""
|
||||
observed: dict[str, NodeSpecInventory] = {}
|
||||
for source in context.capability_sources.values():
|
||||
inventory = source.as_inventory()
|
||||
observed.update(
|
||||
{detail.name: detail for detail in inventory.capabilities.node_spec_details}
|
||||
)
|
||||
return observed
|
||||
|
||||
|
||||
def required_capabilities_for_plan(
|
||||
plan: dict[str, Any],
|
||||
*,
|
||||
source_bindings: dict[str, str] | None,
|
||||
context: WorkflowOperationContext,
|
||||
) -> dict[str, RequiredCapability]:
|
||||
"""Infer a draft dependency summary without persisting an artifact."""
|
||||
artifact = build_workflow_artifact_from_plan(
|
||||
artifact_id="draft_preview",
|
||||
version=1,
|
||||
title="Draft Preview",
|
||||
plan=plan,
|
||||
outcomes=("completed",),
|
||||
source_bindings=source_bindings,
|
||||
observed_node_specs=observed_node_specs(context),
|
||||
)
|
||||
requirements = artifact.required_capability_map()
|
||||
for node in plan_nodes(artifact):
|
||||
raw_ref = node.get("node")
|
||||
if not isinstance(raw_ref, str) or raw_ref in requirements:
|
||||
continue
|
||||
try:
|
||||
parsed = CapabilityRef.parse(raw_ref)
|
||||
except ValueError:
|
||||
continue
|
||||
requirements[raw_ref] = RequiredCapability(
|
||||
ref=parsed,
|
||||
kind="node_spec",
|
||||
)
|
||||
return requirements
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update `src/wf_api/drafts.py`**
|
||||
|
||||
Import:
|
||||
|
||||
```python
|
||||
from .capability_requirements import (
|
||||
observed_node_specs,
|
||||
required_capabilities_for_plan,
|
||||
required_capability_payloads,
|
||||
)
|
||||
```
|
||||
|
||||
Replace:
|
||||
|
||||
```python
|
||||
_required_capability_payloads(...)
|
||||
_required_capabilities_for_plan(...)
|
||||
_observed_node_specs(...)
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
required_capability_payloads(...)
|
||||
required_capabilities_for_plan(...)
|
||||
observed_node_specs(...)
|
||||
```
|
||||
|
||||
Remove local `_required_capabilities_for_plan`, `_required_capability_payloads`, `_observed_node_specs`, and `_plan_nodes` if no longer used.
|
||||
|
||||
- [ ] **Step 4: Update `src/wf_api/artifacts.py`**
|
||||
|
||||
Import:
|
||||
|
||||
```python
|
||||
from .capability_requirements import (
|
||||
observed_node_specs,
|
||||
required_capability_payloads,
|
||||
)
|
||||
```
|
||||
|
||||
Replace local helper calls and remove local duplicate helper definitions.
|
||||
|
||||
- [ ] **Step 5: Update `src/wf_api/capabilities.py`**
|
||||
|
||||
Import:
|
||||
|
||||
```python
|
||||
from .capability_requirements import required_capability_payloads
|
||||
```
|
||||
|
||||
Replace local helper calls and remove the local duplicate helper definition.
|
||||
|
||||
- [ ] **Step 6: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_artifact_helpers.py tests/wf_api/test_drafts_service.py tests/wf_api/test_artifact_api.py tests/wf_api/test_capability_api.py tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Add Source Snapshot Helper If Duplicated
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/source_snapshots.py`
|
||||
- Modify: `src/wf_api/deployments.py`
|
||||
- Modify: `src/wf_api/runs.py`
|
||||
- Test: existing deployment/run API tests
|
||||
|
||||
- [ ] **Step 1: Inspect current source snapshot helper names**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "_available_sources|AvailableSource|AvailableCapability|capability_name" src/wf_api src/wf_mcp/workflow_surface
|
||||
```
|
||||
|
||||
Expected: identify whether `_available_sources` still exists in `src/wf_api/deployments.py` and is imported by `src/wf_api/runs.py`.
|
||||
|
||||
- [ ] **Step 2: Create `src/wf_api/source_snapshots.py` only if a helper exists**
|
||||
|
||||
If `_available_sources` exists, move it as:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
|
||||
from wf_artifacts import AvailableCapability, AvailableSource
|
||||
from wf_platform import CapabilitySource
|
||||
|
||||
|
||||
def available_sources_from_capability_sources(
|
||||
sources: Mapping[str, CapabilitySource],
|
||||
) -> dict[str, AvailableSource]:
|
||||
"""Project live capability sources into pinned resume-validation snapshots."""
|
||||
return {
|
||||
source_id: AvailableSource(
|
||||
id=source.id,
|
||||
capabilities={
|
||||
name: AvailableCapability(name=name)
|
||||
for name in source.capabilities.node_specs
|
||||
},
|
||||
)
|
||||
for source_id, source in sources.items()
|
||||
}
|
||||
```
|
||||
|
||||
If the existing helper carries more fields than `name`, preserve those fields exactly. Do not simplify the payload.
|
||||
|
||||
- [ ] **Step 3: Update deployment/run imports**
|
||||
|
||||
Replace duplicated or cross-domain imports with:
|
||||
|
||||
```python
|
||||
from .source_snapshots import available_sources_from_capability_sources
|
||||
```
|
||||
|
||||
Use it wherever resume/deployment validation needs current source snapshots.
|
||||
|
||||
- [ ] **Step 4: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_deployment_api.py tests/wf_api/test_run_api.py tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Remove Duplicate Private Helpers And Guard Imports
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_api/capabilities.py`
|
||||
- Modify: `src/wf_api/artifacts.py`
|
||||
- Modify: `src/wf_api/drafts.py`
|
||||
- Modify: `src/wf_api/runs.py`
|
||||
- Modify: `src/wf_api/deployments.py`
|
||||
- Test: import-direction guard
|
||||
|
||||
- [ ] **Step 1: Search for leftover duplicated helpers**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "def _matches_query|def _paged_list_payload|def _raw_plan_from_artifact|def _plan_field|def _artifact_capability_id|def _required_capability_payloads|def _observed_node_specs|def _plan_nodes|def _available_sources" src/wf_api src/wf_mcp/workflow_surface
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- No duplicate helper definitions in domain API modules.
|
||||
- `src/wf_mcp/shared/listing.py` may still define `matches_query` and `paged_list_payload`; leave it alone unless no MCP code imports it.
|
||||
- `src/wf_mcp/shared/pagination.py` must remain.
|
||||
|
||||
- [ ] **Step 2: Search for forbidden workflow listing import**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "from \.\.shared import paged_list_payload|from wf_mcp.shared import paged_list_payload" src/wf_mcp/workflow_surface src/wf_api
|
||||
```
|
||||
|
||||
Expected: no matches.
|
||||
|
||||
- [ ] **Step 3: Verify `wf_api` still imports no `wf_mcp`**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Final Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused wf_api workflow tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_listing.py tests/wf_api/test_artifact_helpers.py tests/wf_api/test_drafts_service.py tests/wf_api/test_artifact_api.py tests/wf_api/test_deployment_api.py tests/wf_api/test_run_api.py tests/wf_api/test_capability_api.py tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run adapter-focused workflow surface tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/workflow_surface tests/wf_mcp/test_server.py -q
|
||||
```
|
||||
|
||||
Expected: pass. If `tests/wf_mcp/workflow_surface` does not exist in this checkout, run the nearest existing workflow-surface test files discovered with `rg -n "WorkflowSurfaceHandlers|register_workflow_tools" tests/wf_mcp`.
|
||||
|
||||
- [ ] **Step 3: Run lint and type checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_api src/wf_mcp/workflow_surface tests/wf_api
|
||||
uv run ruff format --check src/wf_api src/wf_mcp/workflow_surface tests/wf_api
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: all pass with zero new diagnostics.
|
||||
|
||||
- [ ] **Step 4: Optional full suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: existing suite status remains at least as good as before this slice.
|
||||
|
||||
---
|
||||
|
||||
## Handoff Report Requirements
|
||||
|
||||
When done, report:
|
||||
|
||||
- Files created.
|
||||
- Files modified.
|
||||
- Exact helpers moved and their new canonical module.
|
||||
- Any helpers intentionally left in place and why.
|
||||
- Verification commands and outputs.
|
||||
- Any deviations from this plan.
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: covers roadmap Slice 5 listing cleanup and post-Slice-4 helper promotion. Event primitives are explicitly deferred because their semantics are larger than this helper cleanup.
|
||||
- Placeholder scan: no `TBD`, no unspecified edge handling, no “write tests for above” without concrete examples.
|
||||
- Type consistency: helper names are stable and public names omit leading underscores; domain modules should import from `wf_api.*`, never `wf_mcp.*`.
|
||||
@@ -0,0 +1,511 @@
|
||||
# wf_api Store Ownership Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make workflow artifact, draft workspace, and run store ownership explicit so `wf_api`, CLI, MCP, and future HTTP entrypoints can share stores without relying on `WfMcpService.__post_init__`.
|
||||
|
||||
**Architecture:** Add a protocol-neutral `WorkflowStores` bundle in `wf_api` that groups the three workflow stores. MCP config construction remains responsible for creating file-backed stores from `BrokerConfig.store_root`; `WfMcpService` receives stores but no longer manufactures them from its MCP `Store`. Existing process-local behavior stays intact through `build_service_from_config`.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, `wf_api`, `wf_artifacts`, `wf_mcp`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Current Problem
|
||||
|
||||
`src/wf_mcp/broker/service/core.py` currently does this in `WfMcpService.__post_init__`:
|
||||
|
||||
```python
|
||||
if self.artifact_store is None:
|
||||
self.artifact_store = FileWorkflowArtifactStore(_store_root(self.store))
|
||||
if self.draft_workspace_store is None:
|
||||
self.draft_workspace_store = FileDraftWorkspaceStore(_store_root(self.store))
|
||||
if self.run_store is None:
|
||||
self.run_store = FileRunStore(_store_root(self.store))
|
||||
```
|
||||
|
||||
That makes a protocol-specific service decide protocol-neutral workflow persistence. It also hides missing-store tests because `WfMcpService(store=FileStore(...))` silently creates workflow stores.
|
||||
|
||||
`src/wf_mcp/broker/config.py::build_service_from_config` already does the right thing by passing all three stores explicitly. This slice preserves that behavior and removes the fallback.
|
||||
|
||||
## Target Ownership Rule
|
||||
|
||||
- `wf_artifacts` owns store protocols and file store implementations.
|
||||
- `wf_api` may group protocol-neutral workflow stores into a small DTO.
|
||||
- `wf_mcp` owns MCP config loading and calls the DTO factory for file-backed process-local stores.
|
||||
- `WfMcpService` owns broker state, connections, adapters, source catalogs, events, and execution wiring.
|
||||
- `WfMcpService` does not create workflow artifact/draft/run stores by guessing from the MCP catalog/auth store.
|
||||
|
||||
## Files
|
||||
|
||||
- Create: `src/wf_api/stores.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
- Modify: `src/wf_mcp/broker/config.py`
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Modify: direct `WfMcpService(...)` tests only where they rely on implicit workflow stores
|
||||
- Test: `tests/wf_api/test_stores.py`
|
||||
- Test: `tests/wf_mcp/service/test_catalog.py`
|
||||
- Test: `tests/wf_mcp/test_broker_server.py`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Protocol-Neutral Store Bundle
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/stores.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
- Test: `tests/wf_api/test_stores.py`
|
||||
|
||||
- [ ] **Step 1: Write failing store bundle tests**
|
||||
|
||||
Create `tests/wf_api/test_stores.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_api.stores import WorkflowStores, file_workflow_stores
|
||||
from wf_artifacts import (
|
||||
FileDraftWorkspaceStore,
|
||||
FileRunStore,
|
||||
FileWorkflowArtifactStore,
|
||||
)
|
||||
|
||||
from tests.wf_mcp.test_support import local_temp_root
|
||||
|
||||
|
||||
def test_file_workflow_stores_constructs_all_three_file_stores() -> None:
|
||||
root = local_temp_root() / "wf_api_file_workflow_stores"
|
||||
|
||||
stores = file_workflow_stores(root)
|
||||
|
||||
assert isinstance(stores, WorkflowStores)
|
||||
assert isinstance(stores.artifact_store, FileWorkflowArtifactStore)
|
||||
assert isinstance(stores.draft_workspace_store, FileDraftWorkspaceStore)
|
||||
assert isinstance(stores.run_store, FileRunStore)
|
||||
assert stores.artifact_store.root == root
|
||||
assert stores.draft_workspace_store.root == root
|
||||
assert stores.run_store.root == root
|
||||
|
||||
|
||||
def test_wf_api_exports_workflow_stores() -> None:
|
||||
from wf_api import WorkflowStores as ExportedWorkflowStores
|
||||
from wf_api import file_workflow_stores as exported_file_workflow_stores
|
||||
|
||||
assert ExportedWorkflowStores is WorkflowStores
|
||||
assert exported_file_workflow_stores is file_workflow_stores
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run failing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api\test_stores.py -q
|
||||
```
|
||||
|
||||
Expected: import failure for `wf_api.stores`.
|
||||
|
||||
- [ ] **Step 3: Implement `wf_api.stores`**
|
||||
|
||||
Create `src/wf_api/stores.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from wf_artifacts import (
|
||||
DraftWorkspaceStore,
|
||||
FileDraftWorkspaceStore,
|
||||
FileRunStore,
|
||||
FileWorkflowArtifactStore,
|
||||
RunStore,
|
||||
WorkflowArtifactStore,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class WorkflowStores:
|
||||
"""Protocol-neutral persistence dependencies for workflow APIs."""
|
||||
|
||||
artifact_store: WorkflowArtifactStore
|
||||
draft_workspace_store: DraftWorkspaceStore
|
||||
run_store: RunStore
|
||||
|
||||
|
||||
def file_workflow_stores(root: str | Path) -> WorkflowStores:
|
||||
"""Create process-local file-backed workflow stores under one root."""
|
||||
store_root = Path(root)
|
||||
return WorkflowStores(
|
||||
artifact_store=FileWorkflowArtifactStore(store_root),
|
||||
draft_workspace_store=FileDraftWorkspaceStore(store_root),
|
||||
run_store=FileRunStore(store_root),
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["WorkflowStores", "file_workflow_stores"]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Export from `wf_api`**
|
||||
|
||||
Update `src/wf_api/__init__.py`:
|
||||
|
||||
```python
|
||||
from .stores import WorkflowStores, file_workflow_stores
|
||||
```
|
||||
|
||||
Add both names to `__all__`.
|
||||
|
||||
- [ ] **Step 5: Verify Task 1**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api\test_stores.py -q
|
||||
uv run ruff check src\wf_api\stores.py tests\wf_api\test_stores.py
|
||||
uv run ruff format --check src\wf_api\stores.py tests\wf_api\test_stores.py
|
||||
```
|
||||
|
||||
Expected: tests pass, lint pass, format pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Move Config Store Construction Through the Bundle
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/config.py`
|
||||
- Test: `tests/wf_mcp/test_broker_server.py`
|
||||
|
||||
- [ ] **Step 1: Strengthen config construction test**
|
||||
|
||||
Find `tests/wf_mcp/test_broker_server.py::test_build_service_from_config_uses_store_root_for_artifacts`.
|
||||
|
||||
Update it to assert all three stores use the configured root:
|
||||
|
||||
```python
|
||||
def test_build_service_from_config_uses_store_root_for_workflow_stores() -> None:
|
||||
store_root = local_temp_root() / "broker_config_workflow_stores"
|
||||
config = BrokerConfig(store_root=store_root, connections=[])
|
||||
|
||||
service = build_service_from_config(config)
|
||||
|
||||
assert isinstance(service.artifact_store, FileWorkflowArtifactStore)
|
||||
assert isinstance(service.draft_workspace_store, FileDraftWorkspaceStore)
|
||||
assert isinstance(service.run_store, FileRunStore)
|
||||
assert service.artifact_store.root == store_root
|
||||
assert service.draft_workspace_store.root == store_root
|
||||
assert service.run_store.root == store_root
|
||||
```
|
||||
|
||||
Ensure the test imports:
|
||||
|
||||
```python
|
||||
from wf_artifacts import FileDraftWorkspaceStore, FileRunStore, FileWorkflowArtifactStore
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the focused test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_mcp\test_broker_server.py::test_build_service_from_config_uses_store_root_for_workflow_stores -q
|
||||
```
|
||||
|
||||
Expected: pass before the config refactor, proving current behavior is covered.
|
||||
|
||||
- [ ] **Step 3: Update `build_service_from_config` to use `file_workflow_stores`**
|
||||
|
||||
In `src/wf_mcp/broker/config.py`, replace direct file store imports:
|
||||
|
||||
```python
|
||||
from wf_api import file_workflow_stores
|
||||
```
|
||||
|
||||
Remove:
|
||||
|
||||
```python
|
||||
from wf_artifacts import (
|
||||
FileDraftWorkspaceStore,
|
||||
FileRunStore,
|
||||
FileWorkflowArtifactStore,
|
||||
)
|
||||
```
|
||||
|
||||
Then update `build_service_from_config`:
|
||||
|
||||
```python
|
||||
def build_service_from_config(config: BrokerConfig) -> WfMcpService:
|
||||
"""Create a broker service with SDK adapters for configured connections."""
|
||||
runtime_factory = PersistentSessionFactory()
|
||||
workflow_stores = file_workflow_stores(config.store_root)
|
||||
service = WfMcpService(
|
||||
store=FileStore(config.store_root),
|
||||
artifact_store=workflow_stores.artifact_store,
|
||||
draft_workspace_store=workflow_stores.draft_workspace_store,
|
||||
run_store=workflow_stores.run_store,
|
||||
# Discovery can use short-lived SDK sessions. Workflow execution needs
|
||||
# a persistent runtime so stateful MCP servers keep session/page state
|
||||
# across sequential workflow nodes.
|
||||
tool_executor=McpRuntimePool(runtime_factory.create),
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Verify Task 2**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_mcp\test_broker_server.py::test_build_service_from_config_uses_store_root_for_workflow_stores -q
|
||||
uv run ruff check src\wf_mcp\broker\config.py tests\wf_mcp\test_broker_server.py
|
||||
uv run ruff format --check src\wf_mcp\broker\config.py tests\wf_mcp\test_broker_server.py
|
||||
```
|
||||
|
||||
Expected: tests pass, lint pass, format pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Remove Implicit Workflow Store Creation from WfMcpService
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Modify: `tests/wf_mcp/service/test_catalog.py`
|
||||
|
||||
- [ ] **Step 1: Replace the old default-store test**
|
||||
|
||||
Find `tests/wf_mcp/service/test_catalog.py::test_service_installs_default_draft_workspace_store`.
|
||||
|
||||
Replace it with:
|
||||
|
||||
```python
|
||||
def test_service_does_not_install_workflow_stores_implicitly() -> None:
|
||||
root = local_temp_root() / "service_no_implicit_workflow_stores"
|
||||
service = WfMcpService(store=FileStore(root))
|
||||
|
||||
assert service.artifact_store is None
|
||||
assert service.draft_workspace_store is None
|
||||
assert service.run_store is None
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run failing test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_mcp\service\test_catalog.py::test_service_does_not_install_workflow_stores_implicitly -q
|
||||
```
|
||||
|
||||
Expected: fail because `WfMcpService.__post_init__` still creates stores.
|
||||
|
||||
- [ ] **Step 3: Remove implicit creation from `WfMcpService.__post_init__`**
|
||||
|
||||
In `src/wf_mcp/broker/service/core.py`, remove the imports:
|
||||
|
||||
```python
|
||||
FileDraftWorkspaceStore,
|
||||
FileRunStore,
|
||||
FileWorkflowArtifactStore,
|
||||
```
|
||||
|
||||
Remove the `_store_root` helper entirely:
|
||||
|
||||
```python
|
||||
def _store_root(store: Store) -> Path:
|
||||
"""Return the file root for stores that expose one, else use local default."""
|
||||
root = getattr(store, "root", None)
|
||||
return root if isinstance(root, Path) else Path(".wf_mcp_store")
|
||||
```
|
||||
|
||||
Update `WfMcpService.__post_init__` to:
|
||||
|
||||
```python
|
||||
def __post_init__(self) -> None:
|
||||
"""Install broker-local system specs when enabled.
|
||||
|
||||
Workflow stores are injected by entrypoint/config construction. This service
|
||||
must not guess workflow persistence from the MCP catalog/auth store because
|
||||
CLI, MCP, and future HTTP frontends may share or swap those stores.
|
||||
"""
|
||||
if self.include_builtin_specs:
|
||||
for source in builtin_sources().values():
|
||||
self.register_capability_source(source)
|
||||
self.register_capability_source(admin_source())
|
||||
```
|
||||
|
||||
If `Path` becomes unused in `core.py`, remove `from pathlib import Path`.
|
||||
|
||||
- [ ] **Step 4: Verify Task 3**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_mcp\service\test_catalog.py::test_service_does_not_install_workflow_stores_implicitly -q
|
||||
uv run ruff check src\wf_mcp\broker\service\core.py tests\wf_mcp\service\test_catalog.py
|
||||
uv run ruff format --check src\wf_mcp\broker\service\core.py tests\wf_mcp\service\test_catalog.py
|
||||
```
|
||||
|
||||
Expected: tests pass, lint pass, format pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Fix Direct Service Tests That Need Workflow Stores
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify only tests that fail after Task 3.
|
||||
|
||||
- [ ] **Step 1: Run targeted workflow API/service tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api tests\wf_mcp\workflow_surface tests\wf_mcp\test_broker_server.py tests\wf_mcp\service -q
|
||||
```
|
||||
|
||||
Expected: if failures appear, they should be tests that constructed `WfMcpService(store=...)` but then used workflow artifact/draft/run operations.
|
||||
|
||||
- [ ] **Step 2: Patch only failing tests by injecting stores explicitly**
|
||||
|
||||
For any failing direct `WfMcpService(...)` test that needs workflow stores, use this pattern:
|
||||
|
||||
```python
|
||||
from wf_api import file_workflow_stores
|
||||
|
||||
|
||||
root = local_temp_root() / "test_specific_name"
|
||||
workflow_stores = file_workflow_stores(root)
|
||||
service = WfMcpService(
|
||||
store=FileStore(root / "mcp"),
|
||||
artifact_store=workflow_stores.artifact_store,
|
||||
draft_workspace_store=workflow_stores.draft_workspace_store,
|
||||
run_store=workflow_stores.run_store,
|
||||
)
|
||||
```
|
||||
|
||||
Do not add stores to tests that only exercise broker catalog/admin/source behavior.
|
||||
|
||||
- [ ] **Step 3: Keep no-store behavior tests intact**
|
||||
|
||||
Tests like these should keep `artifact_store=None` through `WorkflowOperationContext` or direct service construction because they prove graceful no-store behavior:
|
||||
|
||||
```python
|
||||
assert result["nodes"] == []
|
||||
assert result["deployments"] == []
|
||||
```
|
||||
|
||||
Do not “fix” these by injecting stores unless the test is explicitly about stored workflow data.
|
||||
|
||||
- [ ] **Step 4: Verify targeted tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api tests\wf_mcp\workflow_surface tests\wf_mcp\test_broker_server.py tests\wf_mcp\service -q
|
||||
uv run ruff check tests\wf_api tests\wf_mcp\workflow_surface tests\wf_mcp\test_broker_server.py tests\wf_mcp\service
|
||||
uv run ruff format --check tests\wf_api tests\wf_mcp\workflow_surface tests\wf_mcp\test_broker_server.py tests\wf_mcp\service
|
||||
```
|
||||
|
||||
Expected: targeted tests pass and no broad fixture churn.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Document the Store Ownership Rule
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md`
|
||||
- Modify: `docs/current_roadmap.md` if it already has a `wf_api` section
|
||||
|
||||
- [ ] **Step 1: Update the extraction map**
|
||||
|
||||
In `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md`, under `### Store Ownership Ambiguity`, replace the section with:
|
||||
|
||||
```markdown
|
||||
### Store Ownership
|
||||
|
||||
- `wf_artifacts` owns workflow store protocols and file-backed implementations.
|
||||
- `wf_api.stores.WorkflowStores` groups the artifact, draft workspace, and run stores as protocol-neutral API dependencies.
|
||||
- MCP config construction creates file-backed workflow stores from `BrokerConfig.store_root` and injects them into `WfMcpService`.
|
||||
- `WfMcpService.__post_init__` no longer creates workflow stores from its MCP `Store`; direct service tests must inject stores when they exercise workflow persistence.
|
||||
- Future HTTP/API entrypoints should construct or receive the same `WorkflowStores` bundle instead of importing `wf_mcp`.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update roadmap only if there is a matching section**
|
||||
|
||||
If `docs/current_roadmap.md` has a `wf_api` or API extraction section, add:
|
||||
|
||||
```markdown
|
||||
- Workflow store ownership is explicit: entrypoints construct/inject `WorkflowStores`; `WfMcpService` no longer guesses stores from the MCP store root.
|
||||
```
|
||||
|
||||
If there is no matching section, skip this file.
|
||||
|
||||
- [ ] **Step 3: Verify docs are not stale**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "_store_root\\(|creates default `FileWorkflowArtifactStore`|installs default.*store" src docs tests
|
||||
```
|
||||
|
||||
Expected: no current docs/tests claim `WfMcpService` installs default workflow stores. Historical plans may still mention old implementation; leave historical plans alone unless they are current roadmap/research docs.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Final Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- All touched files.
|
||||
|
||||
- [ ] **Step 1: Run focused suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests\wf_api tests\wf_mcp\workflow_surface tests\wf_mcp\test_broker_server.py tests\wf_mcp\service -q
|
||||
```
|
||||
|
||||
Expected: all selected tests pass.
|
||||
|
||||
- [ ] **Step 2: Run full suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite passes with the repo’s known skip/xfail counts.
|
||||
|
||||
- [ ] **Step 3: Run lint and format checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src\wf_api src\wf_mcp tests\wf_api tests\wf_mcp
|
||||
uv run ruff format --check src\wf_api src\wf_mcp tests\wf_api tests\wf_mcp
|
||||
```
|
||||
|
||||
Expected: all checks pass.
|
||||
|
||||
- [ ] **Step 4: Run typecheck**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: `0 errors, 0 warnings, 0 notes`. If the command exits nonzero only because of the known workspace enumeration warning, report that exactly.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: This plan covers explicit store creation, `WfMcpService.__post_init__` cleanup, config behavior preservation, test migration, and docs.
|
||||
- Placeholder scan: No `TODO`/`TBD` placeholders remain. Historical-plan references are explicitly scoped.
|
||||
- Type consistency: `WorkflowStores` uses protocol types from `wf_artifacts`; `file_workflow_stores()` returns file-backed implementations; `WfMcpService` field types do not change.
|
||||
- Scope check: This does not implement FastAPI, persisted-run conflict handling, or store locking. Those are separate slices.
|
||||
+69
@@ -0,0 +1,69 @@
|
||||
# wf_mcp Workflow Surface Test Thinning Ledger
|
||||
|
||||
This ledger records every `wf_mcp.workflow_surface` test removed or kept during
|
||||
the thinning pass. Do not delete a test unless the `replacement` column points
|
||||
to equal-or-stronger coverage.
|
||||
|
||||
| Test | Decision | Replacement / Reason |
|
||||
| --- | --- | --- |
|
||||
| test_artifacts::test_workflow_surface_lists_artifact_catalog_entries | keep | Handler adapter smoke test for compact artifact listing; reduced to minimal assertions. |
|
||||
| test_artifacts::test_workflow_surface_pages_and_filters_artifact_catalog_entries | remove | Covered by WorkflowArtifactApi list tests plus wf_api.listing pagination tests; handler keeps one list_artifacts smoke test. |
|
||||
| test_capabilities::test_workflow_surface_lists_planner_visible_capabilities | keep | Handler list smoke test; reduced to minimal adapter-path assertions. |
|
||||
| test_capabilities::test_workflow_surface_filters_stdlib_capabilities_by_source | remove | Covered by WorkflowCapabilityApi source/query filtering; handler list smoke remains. |
|
||||
| test_capabilities::test_workflow_surface_call_capability_returns_structured_error | keep | Protected: MCP/service behavior — structured error on capability call failure. |
|
||||
| test_capabilities::test_workflow_surface_lists_saved_wrapper_capabilities | remove | Covered by WorkflowCapabilityApi saved wrapper list tests. |
|
||||
| test_capabilities::test_workflow_surface_inspects_one_capability | keep | Handler inspect smoke test; ensures adapter path works for single capability inspection. |
|
||||
| test_capabilities::test_workflow_surface_inspect_capability_includes_wrapper_hints | keep | Protected: MCP/service behavior — wrapper hints detail through handler. |
|
||||
| test_capabilities::test_workflow_surface_inspects_saved_wrapper_capability | remove | Covered by WorkflowCapabilityApi saved wrapper inspect tests. |
|
||||
| test_capabilities::test_workflow_surface_does_not_auto_map_raw_mcp_content_blocks | keep | Protected: MCP content-block mapping behavior. |
|
||||
| test_deployments::test_workflow_surface_validates_deployment_dependencies | keep | Handler-level dependency validation has stronger next-action assertions than wf_api. |
|
||||
| test_deployments::test_workflow_surface_validate_deployment_live_check_is_opt_in | keep | Protected: MCP service adapter — live check is opt-in. |
|
||||
| test_deployments::test_workflow_surface_validate_deployment_live_check_reports_unreachable_source | keep | Protected: MCP service adapter — unreachable source reporting. |
|
||||
| test_deployments::test_workflow_surface_validate_deployment_live_check_reports_missing_connection | keep | Protected: MCP service adapter — missing connection reporting. |
|
||||
| test_deployments::test_workflow_surface_records_artifact_and_deployment_save_events | keep | Protected: service event recording. |
|
||||
| test_deployments::test_workflow_surface_save_deployment_accepts_deployment_id_alias | keep | Protected: request alias normalization. |
|
||||
| test_deployments::test_workflow_surface_deletes_deployment | keep | Protected: delete event recording. |
|
||||
| test_deployments::test_workflow_surface_save_deployment_rejects_id_and_deployment_id | keep | Protected: XOR validation on id/deployment_id. |
|
||||
| test_deployments::test_workflow_surface_lists_compact_deployment_summaries_and_inspects_detail | keep | Protected: compact-vs-detail response shape. |
|
||||
| test_drafts::test_workflow_surface_validates_draft_without_saving | remove | Covered by WorkflowDraftApi.validate_draft via test_delegation_smoke_validate_draft_equivalence. |
|
||||
| test_drafts::test_workflow_surface_rejects_unknown_draft_route_outcome_when_spec_is_known | keep | Live outcome lookup through handler; not duplicated in wf_api. |
|
||||
| test_drafts::test_workflow_surface_creates_artifact_from_draft_with_binding_suggestions | keep | Binding suggestions and artifact persistence through handler. |
|
||||
| test_drafts::test_workflow_surface_draft_artifact_requires_std_self_binding | keep | binding_missing diagnostic through handler/deployment integration. |
|
||||
| test_drafts::test_workflow_surface_patches_draft_without_saving | remove | Covered by WorkflowDraftApi.patch_draft. |
|
||||
| test_drafts::test_workflow_surface_creates_and_gets_draft_workspace | remove | Covered by WorkflowDraftApi.create_draft_workspace and get_draft_workspace assertions in tests/wf_api/test_drafts_service.py. |
|
||||
| test_drafts::test_workflow_surface_lists_draft_workspaces | remove | Covered by tests/wf_api/test_drafts_service.py::test_list_draft_workspaces_returns_sorted_summaries_without_drafts. |
|
||||
| test_drafts::test_workflow_surface_deletes_draft_workspace | remove | Covered by tests/wf_api/test_drafts_service.py::test_delete_draft_workspace_is_idempotent. |
|
||||
| test_drafts::test_workflow_surface_patch_helpers_update_draft_workspace | remove | Covered by tests/wf_api/test_drafts_service.py::test_draft_workspace_patch_helpers_update_revision_and_bindings. |
|
||||
| test_drafts::test_workflow_surface_validates_draft_workspace_with_live_outcomes | keep | Live outcome lookup through handler/service stack. |
|
||||
| test_drafts::test_workflow_surface_patches_draft_workspace_by_revision | remove | Covered by WorkflowDraftApi.patch_draft_workspace. |
|
||||
| test_drafts::test_workflow_surface_creates_minimal_draft_workspace_with_error_route | keep | MCP request model parsing and error route generation. |
|
||||
| test_drafts::test_workflow_surface_minimal_draft_honors_explicit_error_message_source | keep | Explicit error_message_source handling. |
|
||||
| test_drafts::test_minimal_draft_request_accepts_structural_error_message_source | keep | MCP Pydantic model validation (CreateMinimalDraftWorkspaceRequest). |
|
||||
| test_drafts::test_workflow_surface_accepts_canonical_bindings_for_minimal_workspace | keep | Canonical InputPathBinding/OutputBinding through handler. |
|
||||
| test_drafts::test_workflow_surface_creates_draft_workspace_from_capability_hints | keep | Wrapper hints and next-actions through handler. |
|
||||
| test_drafts::test_workflow_surface_creates_artifact_from_workspace | keep | Artifact persistence with schema snapshots through handler. |
|
||||
| test_drafts::test_workflow_surface_workspace_artifact_infers_raw_concrete_dependency | keep | Source dependency inference through handler. |
|
||||
| test_drafts::test_workflow_surface_creates_wrapper_from_workspace | keep | Wrapper creation through handler. |
|
||||
| test_drafts::test_workflow_surface_low_confidence_draft_returns_patch_guidance | keep | Next-action guidance with patch examples through handler. |
|
||||
| test_runs::test_raw_workflow_plan_uses_core_step_and_edge_models | keep | DEVIATION: Plan said wf_api covers this, but test_raw_workflow_plan_extraction.py only tests imports (canonical, compat, identity). The surface test is the only one exercising actual RawWorkflowPlan step/edge model parsing. Kept to avoid coverage gap. |
|
||||
| test_runs::test_workflow_surface_runs_non_interrupting_deployment | keep | Persisted run records, trace slicing, response model, next-actions. |
|
||||
| test_runs::test_workflow_surface_failed_deployment_exposes_error_on_run_and_inspect | keep | Failed run error exposure and inspect_run. |
|
||||
| test_runs::test_workflow_surface_run_deployment_can_include_trace_detail | keep | Protected: MCP TraceRange, RunDeploymentResult model validation. |
|
||||
| test_runs::test_workflow_surface_run_deployment_can_read_empty_trace_range | keep | Protected: empty trace range behavior. |
|
||||
| test_runs::test_workflow_surface_runs_deployment_with_bound_node_spec_dependency | keep | Protected: logical source binding. |
|
||||
| test_runs::test_workflow_surface_runs_artifact_created_from_concrete_node_ref | keep | Protected: concrete node ref artifact creation and run. |
|
||||
| test_runs::test_workflow_surface_detects_drift_from_saved_node_spec_snapshot | keep | Protected: schema drift detection. |
|
||||
| test_runs::test_workflow_surface_runs_deployment_with_bound_reducer_dependency | keep | Protected: reducer dependency integration. |
|
||||
| test_wrappers::test_workflow_surface_creates_wrapper_artifact_from_plan | keep | Handler integration: wrapper artifact creation from plan. |
|
||||
| test_wrappers::test_workflow_surface_creates_artifact_with_logical_node_refs | keep | Handler integration: logical node ref resolution. |
|
||||
| test_wrappers::test_workflow_surface_calls_saved_wrapper_artifact | keep | Handler integration: saved wrapper direct call. |
|
||||
| test_wrappers::test_workflow_surface_calls_live_node_spec_with_self_describing_response | keep | Handler integration: live node spec call. |
|
||||
| test_wrappers::test_workflow_surface_calls_saved_wrapper_artifact_with_deployment_bindings | keep | Protected: saved wrapper call with deployment bindings. |
|
||||
| test_next_actions::test_next_actions_from_high_confidence_wrapper_hints_can_validate | keep | NextActions model unit test. |
|
||||
| test_next_actions::test_next_actions_from_low_confidence_wrapper_hints_can_patch | keep | NextActions model unit test. |
|
||||
| test_next_actions::test_next_actions_from_runnable_deployment_recommends_run | keep | NextActions model unit test. |
|
||||
| test_next_actions::test_next_actions_from_unrunnable_deployment_recommends_validation_retry | keep | NextActions model unit test. |
|
||||
| test_next_actions::test_next_actions_from_completed_run_has_no_required_next_tool | keep | NextActions model unit test. |
|
||||
| test_next_actions::test_next_actions_from_failed_run_recommends_bounded_trace | keep | NextActions model unit test. |
|
||||
| test_next_actions::test_next_actions_from_interrupted_run_recommends_resume | keep | NextActions model unit test. |
|
||||
| test_next_actions::test_workflow_surface_next_actions_shim_reexports_canonical_model | keep | Shim re-export verification. |
|
||||
+545
@@ -0,0 +1,545 @@
|
||||
# wf_mcp Workflow Surface Test Thinning Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Reduce duplicated `wf_mcp.workflow_surface` behavior tests now covered by `wf_api` tests while preserving adapter, schema, live-source, run, wrapper, and end-to-end coverage.
|
||||
|
||||
**Architecture:** `wf_api` tests are the canonical behavior tests for application-service logic. `wf_mcp.workflow_surface` tests should prove that `WorkflowSurfaceHandlers` exposes that behavior through the MCP-owned adapter boundary and should keep any test that exercises MCP-specific request models, live checks, service event recording, tool schema, or realistic integration paths.
|
||||
|
||||
**Tech Stack:** Python 3.14, pytest, `wf_api`, `wf_mcp.workflow_surface`, `WorkflowSurfaceHandlers`, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Coverage Policy
|
||||
|
||||
Use this rule for every deletion:
|
||||
|
||||
```text
|
||||
Only remove a wf_mcp.workflow_surface test when an equal-or-stronger wf_api test
|
||||
already covers the behavior and a smaller handler smoke/delegation test still
|
||||
proves the adapter path.
|
||||
```
|
||||
|
||||
Keep tests that cover:
|
||||
|
||||
- MCP request/response Pydantic models such as `TraceRange`, `RunDeploymentResult`, and `CreateMinimalDraftWorkspaceRequest`.
|
||||
- live source checks through `WfMcpService` adapters.
|
||||
- service event recording.
|
||||
- handler-to-service wiring.
|
||||
- saved wrapper calls with deployment bindings.
|
||||
- source binding, schema drift, reducer dependency, subgraph, or durable run integration.
|
||||
- next-action guidance generated through real handler operations.
|
||||
|
||||
Thin tests that only repeat:
|
||||
|
||||
- list filtering/pagination details already covered by `WorkflowCapabilityApi` or `WorkflowArtifactApi`.
|
||||
- basic inspect/list payload details already covered by `wf_api` domain tests.
|
||||
- basic draft workspace CRUD details already covered by `WorkflowDraftApi`.
|
||||
- basic artifact/deployment CRUD details already covered by `WorkflowArtifactApi` or `WorkflowDeploymentApi`.
|
||||
|
||||
## File Map
|
||||
|
||||
| File | Planned role |
|
||||
| --- | --- |
|
||||
| `tests/wf_mcp/workflow_surface/test_artifacts.py` | Thin to one handler adapter smoke test for compact artifact listing. |
|
||||
| `tests/wf_mcp/workflow_surface/test_capabilities.py` | Thin list/filter/inspect duplicates; keep wrapper-hints, MCP content-block, direct call error, and saved wrapper coverage if not stronger in `wf_api`. |
|
||||
| `tests/wf_mcp/workflow_surface/test_deployments.py` | Keep live-check, events, alias/XOR, delete event, and compact-vs-detail tests. Maybe remove only duplicate dependency validation if covered by `wf_api`. |
|
||||
| `tests/wf_mcp/workflow_surface/test_drafts.py` | Keep most tests; remove only the simplest validate/patch/list CRUD duplicates if `wf_api` has equal coverage. |
|
||||
| `tests/wf_mcp/workflow_surface/test_runs.py` | Keep run/trace/model tests. Do not thin in first pass except the raw plan model extraction test if duplicated in `tests/wf_api/test_raw_workflow_plan_extraction.py`. |
|
||||
| `tests/wf_mcp/workflow_surface/test_wrappers.py` | Keep all tests in first pass; these are handler integration paths and direct capability REPL behavior. |
|
||||
| `tests/wf_api/*` | Do not weaken. Add missing behavior tests here before removing handler duplicates. |
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Build A Deletion Ledger Before Editing
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
|
||||
|
||||
- [ ] **Step 1: Create the ledger file**
|
||||
|
||||
Create `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`:
|
||||
|
||||
```markdown
|
||||
# wf_mcp Workflow Surface Test Thinning Ledger
|
||||
|
||||
This ledger records every `wf_mcp.workflow_surface` test removed or kept during
|
||||
the thinning pass. Do not delete a test unless the `replacement` column points
|
||||
to equal-or-stronger coverage.
|
||||
|
||||
| Test | Decision | Replacement / Reason |
|
||||
| --- | --- | --- |
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Populate the initial ledger with all workflow-surface tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "^def test_" tests/wf_mcp/workflow_surface
|
||||
```
|
||||
|
||||
Append each test name to the ledger with `Decision` set to `unclassified`.
|
||||
|
||||
Expected: the ledger contains every test from:
|
||||
|
||||
```text
|
||||
tests/wf_mcp/workflow_surface/test_artifacts.py
|
||||
tests/wf_mcp/workflow_surface/test_capabilities.py
|
||||
tests/wf_mcp/workflow_surface/test_deployments.py
|
||||
tests/wf_mcp/workflow_surface/test_drafts.py
|
||||
tests/wf_mcp/workflow_surface/test_runs.py
|
||||
tests/wf_mcp/workflow_surface/test_wrappers.py
|
||||
tests/wf_mcp/workflow_surface/test_next_actions.py
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Mark protected tests**
|
||||
|
||||
Mark these as `keep` unless a later task explicitly adds stronger coverage:
|
||||
|
||||
```text
|
||||
tests/wf_mcp/workflow_surface/test_capabilities.py::test_workflow_surface_call_capability_returns_structured_error
|
||||
tests/wf_mcp/workflow_surface/test_capabilities.py::test_workflow_surface_inspect_capability_includes_wrapper_hints
|
||||
tests/wf_mcp/workflow_surface/test_capabilities.py::test_workflow_surface_does_not_auto_map_raw_mcp_content_blocks
|
||||
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_validate_deployment_live_check_is_opt_in
|
||||
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_validate_deployment_live_check_reports_unreachable_source
|
||||
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_validate_deployment_live_check_reports_missing_connection
|
||||
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_records_artifact_and_deployment_save_events
|
||||
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_save_deployment_accepts_deployment_id_alias
|
||||
tests/wf_mcp/workflow_surface/test_deployments.py::test_workflow_surface_save_deployment_rejects_id_and_deployment_id
|
||||
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_run_deployment_can_include_trace_detail
|
||||
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_run_deployment_can_read_empty_trace_range
|
||||
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_runs_deployment_with_bound_node_spec_dependency
|
||||
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_runs_artifact_created_from_concrete_node_ref
|
||||
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_detects_drift_from_saved_node_spec_snapshot
|
||||
tests/wf_mcp/workflow_surface/test_runs.py::test_workflow_surface_runs_deployment_with_bound_reducer_dependency
|
||||
tests/wf_mcp/workflow_surface/test_wrappers.py
|
||||
```
|
||||
|
||||
Reason: these cover adapter wiring, MCP/service behavior, direct wrapper calls,
|
||||
trace schema behavior, binding logic, reducer dependencies, or important
|
||||
integration seams.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Thin Artifact Listing Duplicates
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_artifacts.py`
|
||||
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
|
||||
- Test: `tests/wf_api/test_artifact_api.py`, `tests/wf_mcp/workflow_surface/test_artifacts.py`
|
||||
|
||||
- [ ] **Step 1: Confirm wf_api artifact coverage**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "list_artifacts|pagination|kind|query|plan\" not in" tests/wf_api/test_artifact_api.py tests/wf_api/test_listing.py
|
||||
```
|
||||
|
||||
Expected: `wf_api` coverage exists for empty list payload, compact artifact rows,
|
||||
query/kind behavior, and pagination helper shape. If it does not, add the
|
||||
missing assertion to `tests/wf_api/test_artifact_api.py` before deleting any
|
||||
handler test.
|
||||
|
||||
- [ ] **Step 2: Keep one handler smoke test**
|
||||
|
||||
Keep `test_workflow_surface_lists_artifact_catalog_entries` and reduce it only
|
||||
if needed to these adapter-boundary assertions:
|
||||
|
||||
```python
|
||||
def test_workflow_surface_lists_artifact_catalog_entries() -> None:
|
||||
artifact_store = FileWorkflowArtifactStore(local_temp_root() / "surface_artifacts")
|
||||
artifact_store.save_artifact(artifact())
|
||||
h = handlers(artifact_store)
|
||||
|
||||
payload = asyncio.run(h.list_artifacts())
|
||||
|
||||
nodes = payload["nodes"]
|
||||
assert payload["total"] == 1
|
||||
assert payload["next_cursor"] is None
|
||||
assert nodes[0]["name"] == "workflow.summarize_docs.v1"
|
||||
assert nodes[0]["artifact_id"] == "summarize_docs"
|
||||
assert "plan" not in nodes[0]
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Remove duplicate pagination/filter handler test**
|
||||
|
||||
Delete:
|
||||
|
||||
```text
|
||||
test_workflow_surface_pages_and_filters_artifact_catalog_entries
|
||||
```
|
||||
|
||||
Mark it in the ledger:
|
||||
|
||||
```text
|
||||
remove | Covered by WorkflowArtifactApi list tests plus wf_api.listing pagination tests; handler keeps one list_artifacts smoke test.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run artifact tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_artifact_api.py tests/wf_api/test_listing.py tests/wf_mcp/workflow_surface/test_artifacts.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Thin Capability List/Inspect Duplicates Conservatively
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_capabilities.py`
|
||||
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
|
||||
- Test: `tests/wf_api/test_capability_api.py`, `tests/wf_mcp/workflow_surface/test_capabilities.py`
|
||||
|
||||
- [ ] **Step 1: Confirm wf_api capability coverage**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "list_capabilities|inspect_capability|saved_wrapper|wrapper_hints|runtime_error|content blocks" tests/wf_api/test_capability_api.py tests/wf_mcp/workflow_surface/test_capabilities.py
|
||||
```
|
||||
|
||||
Expected: `wf_api` covers planner-visible listing, source filtering, unknown
|
||||
inspect, saved wrapper list/inspect, and direct handler delegation smoke.
|
||||
|
||||
- [ ] **Step 2: Keep one handler list smoke test**
|
||||
|
||||
Keep `test_workflow_surface_lists_planner_visible_capabilities`, but keep it
|
||||
compact. It should assert adapter path and summary shape only:
|
||||
|
||||
```python
|
||||
def test_workflow_surface_lists_planner_visible_capabilities() -> None:
|
||||
h = handlers(FileWorkflowArtifactStore(local_temp_root() / "surface_caps"))
|
||||
|
||||
payload = asyncio.run(h.list_capabilities(limit=2))
|
||||
first = payload["capabilities"][0]
|
||||
|
||||
assert len(payload["capabilities"]) == 2
|
||||
assert payload["total"] >= 2
|
||||
assert payload["next_cursor"] == "2"
|
||||
assert first["kind"] == "node_spec"
|
||||
assert "input_schema" not in first
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Remove duplicate source-filter test if wf_api has it**
|
||||
|
||||
Delete:
|
||||
|
||||
```text
|
||||
test_workflow_surface_filters_stdlib_capabilities_by_source
|
||||
```
|
||||
|
||||
Only delete it if `tests/wf_api/test_capability_api.py` contains:
|
||||
|
||||
```text
|
||||
test_list_capabilities_filters_by_source
|
||||
```
|
||||
|
||||
Mark the ledger:
|
||||
|
||||
```text
|
||||
remove | Covered by WorkflowCapabilityApi source/query filtering; handler list smoke remains.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Remove duplicate saved-wrapper list/inspect tests if wf_api has them**
|
||||
|
||||
Delete these only if `tests/wf_api/test_capability_api.py` has equivalent saved
|
||||
wrapper list and inspect tests:
|
||||
|
||||
```text
|
||||
test_workflow_surface_lists_saved_wrapper_capabilities
|
||||
test_workflow_surface_inspects_saved_wrapper_capability
|
||||
```
|
||||
|
||||
Mark the ledger:
|
||||
|
||||
```text
|
||||
remove | Covered by WorkflowCapabilityApi saved wrapper list/inspect tests.
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Keep MCP-specific and guidance-sensitive capability tests**
|
||||
|
||||
Keep these tests unchanged:
|
||||
|
||||
```text
|
||||
test_workflow_surface_call_capability_returns_structured_error
|
||||
test_workflow_surface_inspect_capability_includes_wrapper_hints
|
||||
test_workflow_surface_does_not_auto_map_raw_mcp_content_blocks
|
||||
```
|
||||
|
||||
Do not remove `test_workflow_surface_inspects_one_capability` unless there is
|
||||
still another handler-level inspect smoke test after this task.
|
||||
|
||||
- [ ] **Step 6: Run capability tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_capability_api.py tests/wf_mcp/workflow_surface/test_capabilities.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Thin Deployment Tests Only Where Purely Duplicated
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_deployments.py`
|
||||
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
|
||||
- Test: `tests/wf_api/test_deployment_api.py`, `tests/wf_mcp/workflow_surface/test_deployments.py`
|
||||
|
||||
- [ ] **Step 1: Keep MCP live-check and event tests**
|
||||
|
||||
Do not delete:
|
||||
|
||||
```text
|
||||
test_workflow_surface_validate_deployment_live_check_is_opt_in
|
||||
test_workflow_surface_validate_deployment_live_check_reports_unreachable_source
|
||||
test_workflow_surface_validate_deployment_live_check_reports_missing_connection
|
||||
test_workflow_surface_records_artifact_and_deployment_save_events
|
||||
test_workflow_surface_save_deployment_accepts_deployment_id_alias
|
||||
test_workflow_surface_deletes_deployment
|
||||
test_workflow_surface_save_deployment_rejects_id_and_deployment_id
|
||||
test_workflow_surface_lists_compact_deployment_summaries_and_inspects_detail
|
||||
```
|
||||
|
||||
Reason: these exercise MCP service adapters, event recording, request alias
|
||||
normalization, and compact-vs-detail response shape.
|
||||
|
||||
- [ ] **Step 2: Evaluate basic dependency validation duplicate**
|
||||
|
||||
Check whether `tests/wf_api/test_deployment_api.py` covers dependency validation
|
||||
next-actions for an unrunnable deployment:
|
||||
|
||||
```bash
|
||||
rg -n "source_missing|binding_missing|next_actions|unrunnable" tests/wf_api/test_deployment_api.py
|
||||
```
|
||||
|
||||
If the `wf_api` test has equal next-action assertions, delete:
|
||||
|
||||
```text
|
||||
test_workflow_surface_validates_deployment_dependencies
|
||||
```
|
||||
|
||||
If it does not, keep the handler test and mark it:
|
||||
|
||||
```text
|
||||
keep | Handler-level dependency validation still has stronger next-action assertions than wf_api.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run deployment tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_deployment_api.py tests/wf_mcp/workflow_surface/test_deployments.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Thin Draft Tests With High Caution
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_drafts.py`
|
||||
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
|
||||
- Test: `tests/wf_api/test_drafts_service.py`, `tests/wf_mcp/workflow_surface/test_drafts.py`
|
||||
|
||||
- [ ] **Step 1: Identify pure draft API duplicates**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "validate_draft|patch_draft|list_draft_workspaces|delete_draft_workspace|create_minimal_draft_workspace|create_draft_workspace_from_capability|create_artifact_from_workspace|create_wrapper_from_workspace" tests/wf_api/test_drafts_service.py tests/wf_api/test_artifact_api.py tests/wf_api/test_capability_api.py tests/wf_mcp/workflow_surface/test_drafts.py
|
||||
```
|
||||
|
||||
Expected: most simple draft workspace CRUD and patch helper behavior exists in
|
||||
`tests/wf_api/test_drafts_service.py`.
|
||||
|
||||
- [ ] **Step 2: Remove only pure CRUD duplicates**
|
||||
|
||||
Candidates for removal if `wf_api` tests cover equal behavior:
|
||||
|
||||
```text
|
||||
test_workflow_surface_validates_draft_without_saving
|
||||
test_workflow_surface_patches_draft_without_saving
|
||||
test_workflow_surface_creates_and_gets_draft_workspace
|
||||
test_workflow_surface_lists_draft_workspaces
|
||||
test_workflow_surface_deletes_draft_workspace
|
||||
test_workflow_surface_patch_helpers_update_draft_workspace
|
||||
test_workflow_surface_patches_draft_workspace_by_revision
|
||||
```
|
||||
|
||||
For each removed test, add a ledger row with the exact `wf_api` replacement test.
|
||||
|
||||
- [ ] **Step 3: Keep guidance, model, and artifact integration tests**
|
||||
|
||||
Do not delete these in this pass:
|
||||
|
||||
```text
|
||||
test_workflow_surface_rejects_unknown_draft_route_outcome_when_spec_is_known
|
||||
test_workflow_surface_creates_artifact_from_draft_with_binding_suggestions
|
||||
test_workflow_surface_draft_artifact_requires_std_self_binding
|
||||
test_workflow_surface_validates_draft_workspace_with_live_outcomes
|
||||
test_workflow_surface_creates_minimal_draft_workspace_with_error_route
|
||||
test_workflow_surface_minimal_draft_honors_explicit_error_message_source
|
||||
test_minimal_draft_request_accepts_structural_error_message_source
|
||||
test_workflow_surface_accepts_canonical_bindings_for_minimal_workspace
|
||||
test_workflow_surface_creates_draft_workspace_from_capability_hints
|
||||
test_workflow_surface_creates_artifact_from_workspace
|
||||
test_workflow_surface_workspace_artifact_infers_raw_concrete_dependency
|
||||
test_workflow_surface_creates_wrapper_from_workspace
|
||||
test_workflow_surface_low_confidence_draft_returns_patch_guidance
|
||||
```
|
||||
|
||||
Reason: these cover live outcome lookup, request model parsing, wrapper hints,
|
||||
next actions, artifact persistence, and source dependency inference.
|
||||
|
||||
- [ ] **Step 4: Run draft tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_drafts_service.py tests/wf_api/test_artifact_api.py tests/wf_api/test_capability_api.py tests/wf_mcp/workflow_surface/test_drafts.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Keep Run And Wrapper Tests Mostly Intact
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/workflow_surface/test_runs.py`
|
||||
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
|
||||
- Test: `tests/wf_api/test_run_api.py`, `tests/wf_mcp/workflow_surface/test_runs.py`, `tests/wf_mcp/workflow_surface/test_wrappers.py`
|
||||
|
||||
- [ ] **Step 1: Remove raw plan model duplicate only**
|
||||
|
||||
If `tests/wf_api/test_raw_workflow_plan_extraction.py` covers core step and edge
|
||||
model parsing, delete:
|
||||
|
||||
```text
|
||||
tests/wf_mcp/workflow_surface/test_runs.py::test_raw_workflow_plan_uses_core_step_and_edge_models
|
||||
```
|
||||
|
||||
Mark the ledger:
|
||||
|
||||
```text
|
||||
remove | RawWorkflowPlan extraction is canonical in wf_api model tests.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Keep run deployment behavior tests**
|
||||
|
||||
Keep all remaining tests in `tests/wf_mcp/workflow_surface/test_runs.py`.
|
||||
|
||||
Reason: they cover persisted run records, trace slicing with MCP `TraceRange`,
|
||||
response Pydantic model validation, logical source binding, schema drift, and
|
||||
reducer dependency integration through the handler/service stack.
|
||||
|
||||
- [ ] **Step 3: Keep wrapper tests**
|
||||
|
||||
Do not delete tests in `tests/wf_mcp/workflow_surface/test_wrappers.py` in this
|
||||
pass.
|
||||
|
||||
Reason: wrapper direct calls and deployment-bound wrapper calls are meaningful
|
||||
handler integration tests even if `WorkflowCapabilityApi` also has lower-level
|
||||
coverage.
|
||||
|
||||
- [ ] **Step 4: Run run/wrapper tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_run_api.py tests/wf_api/test_raw_workflow_plan_extraction.py tests/wf_mcp/workflow_surface/test_runs.py tests/wf_mcp/workflow_surface/test_wrappers.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Final Review And Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md`
|
||||
- Test: all workflow-surface and wf_api tests
|
||||
|
||||
- [ ] **Step 1: Ensure ledger has no unclassified rows**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "unclassified" docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning-ledger.md
|
||||
```
|
||||
|
||||
Expected: no matches.
|
||||
|
||||
- [ ] **Step 2: Run full focused API and adapter tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api tests/wf_mcp/workflow_surface tests/wf_mcp/server/test_tools.py tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Run lint and type checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check tests/wf_api tests/wf_mcp/workflow_surface
|
||||
uv run ruff format --check tests/wf_api tests/wf_mcp/workflow_surface
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- Ruff commands pass.
|
||||
- Basedpyright reports `0 errors`; if it exits nonzero only because workspace
|
||||
enumeration exceeds 10 seconds, report that exact output as an environment
|
||||
issue rather than a type failure.
|
||||
|
||||
- [ ] **Step 4: Optional full suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: existing suite status remains at least as good as before this pass.
|
||||
|
||||
---
|
||||
|
||||
## Handoff Report Requirements
|
||||
|
||||
When done, report:
|
||||
|
||||
- Tests removed, grouped by file.
|
||||
- Tests kept intentionally, with reasons for any controversial keeps.
|
||||
- Any new or strengthened `wf_api` tests.
|
||||
- Ledger path.
|
||||
- Verification commands and exact outputs.
|
||||
- Deviations from the plan.
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: the plan preserves “good tests” by requiring a ledger and exact replacement coverage before any deletion.
|
||||
- Placeholder scan: no deferred implementation slots; each deletion candidate has a guard and replacement rule.
|
||||
- Type consistency: all referenced test paths and test names were taken from the current tree inspection.
|
||||
+605
@@ -0,0 +1,605 @@
|
||||
# WfMcpService ConnectionService Extraction Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Move connection registration and config reconciliation out of `WfMcpService` into a focused `ConnectionService`.
|
||||
|
||||
**Architecture:** `ConnectionService` owns the broker-local `ConnectionRegistry` and emits connection lifecycle events. `SourceCatalogService` still owns capability sources, so `ConnectionService` binds to it after both services are constructed. `WfMcpService` remains a compatibility facade with `.connections`, `register_connection()`, and `sync_connections_from_config()` delegating to the new service.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, pytest, ruff, basedpyright, existing `wf_mcp` broker service modules.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create `src/wf_mcp/broker/service/connection_service.py`
|
||||
- Owns `ConnectionRegistry`.
|
||||
- Validates connection IDs and reserved IDs.
|
||||
- Registers connections and hydrates source catalog snapshots.
|
||||
- Reconciles config reload changes.
|
||||
- Modify `src/wf_mcp/broker/service/core.py`
|
||||
- Removes direct `ConnectionRegistry` field from `WfMcpService`.
|
||||
- Constructs `ConnectionService`, passes its lookup/list callbacks into `SourceCatalogService`, then binds the source catalog back to `ConnectionService`.
|
||||
- Keeps compatibility property/method delegates.
|
||||
- Create `tests/wf_mcp/service/test_connection_service.py`
|
||||
- Direct tests for `ConnectionService`.
|
||||
- Service facade smoke test for `.connections` compatibility.
|
||||
- Modify `docs/current_roadmap.md`
|
||||
- Mark the connection-service extraction as the current/complete slice after implementation.
|
||||
- Optionally modify `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md`
|
||||
- Add one ownership note if the file still tracks `WfMcpService` decomposition.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Direct ConnectionService Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tests/wf_mcp/service/test_connection_service.py`
|
||||
|
||||
- [ ] **Step 1: Create direct tests for the new service boundary**
|
||||
|
||||
Create `tests/wf_mcp/service/test_connection_service.py` with:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_mcp.broker.service.connection_service import ConnectionService
|
||||
from wf_mcp.broker.service.events import BrokerEventRecorder
|
||||
from wf_mcp.broker.service.source_catalog import SourceCatalogService
|
||||
from wf_mcp.events import EventBus
|
||||
from wf_mcp.models import BrokerConfig, ConnectionConfig
|
||||
from wf_mcp.runtime import ToolExecutor
|
||||
from wf_mcp.storage import FileStore
|
||||
|
||||
from ..test_support import local_temp_root
|
||||
|
||||
|
||||
def _source_catalog(service: ConnectionService) -> SourceCatalogService:
|
||||
store = FileStore(local_temp_root() / "connection_service_catalog")
|
||||
|
||||
def _tool_executor_for(_connection: ConnectionConfig) -> ToolExecutor:
|
||||
raise AssertionError("tool executor should not be needed in these tests")
|
||||
|
||||
catalog = SourceCatalogService(
|
||||
store=store,
|
||||
connection_lookup=service.get,
|
||||
connection_list_enabled=service.list_enabled,
|
||||
connection_list_all=service.list_all,
|
||||
tool_executor_for=_tool_executor_for,
|
||||
load_auth=lambda _connection_id: None,
|
||||
emit_event=service.events.record_event,
|
||||
)
|
||||
service.bind_source_catalog(catalog)
|
||||
return catalog
|
||||
|
||||
|
||||
def test_connection_service_rejects_reserved_connection_ids() -> None:
|
||||
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
|
||||
_source_catalog(service)
|
||||
|
||||
for connection_id in ("wf.admin", "wf.mcp"):
|
||||
try:
|
||||
service.register_connection(
|
||||
ConnectionConfig(id=connection_id, server="wf", account="reserved")
|
||||
)
|
||||
except ValueError as exc:
|
||||
assert connection_id in str(exc)
|
||||
assert "reserved by wf-mcp" in str(exc)
|
||||
else:
|
||||
raise AssertionError(f"expected {connection_id!r} to be rejected")
|
||||
|
||||
|
||||
def test_connection_service_registers_connection_and_empty_source() -> None:
|
||||
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
|
||||
catalog = _source_catalog(service)
|
||||
|
||||
service.register_connection(
|
||||
ConnectionConfig(id="demo.personal", server="demo", account="personal")
|
||||
)
|
||||
|
||||
assert service.get("demo.personal").server == "demo"
|
||||
assert [connection.id for connection in service.list_enabled()] == ["demo.personal"]
|
||||
source = catalog.capability_sources["demo.personal"]
|
||||
assert source.enabled is True
|
||||
assert source.description == "No catalog loaded for demo.personal."
|
||||
assert service.events.list_events()[0].kind == "connection_registered"
|
||||
assert service.events.list_events()[0].connection_id == "demo.personal"
|
||||
|
||||
|
||||
def test_connection_service_sync_removes_retired_connections_and_sources() -> None:
|
||||
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
|
||||
catalog = _source_catalog(service)
|
||||
service.register_connection(
|
||||
ConnectionConfig(id="demo.personal", server="demo", account="personal")
|
||||
)
|
||||
|
||||
service.sync_connections_from_config(
|
||||
BrokerConfig(store_root=local_temp_root(), connections=[])
|
||||
)
|
||||
|
||||
assert service.list_all() == []
|
||||
assert "demo.personal" not in catalog.capability_sources
|
||||
|
||||
|
||||
def test_connection_service_sync_updates_existing_source_enabled_flag() -> None:
|
||||
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
|
||||
catalog = _source_catalog(service)
|
||||
service.register_connection(
|
||||
ConnectionConfig(id="demo.personal", server="demo", account="personal")
|
||||
)
|
||||
|
||||
service.sync_connections_from_config(
|
||||
BrokerConfig(
|
||||
store_root=local_temp_root(),
|
||||
connections=[
|
||||
ConnectionConfig(
|
||||
id="demo.personal",
|
||||
server="demo",
|
||||
account="personal",
|
||||
enabled=False,
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
assert service.get("demo.personal").enabled is False
|
||||
assert catalog.capability_sources["demo.personal"].enabled is False
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the direct test and confirm it fails before implementation**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
|
||||
```
|
||||
|
||||
Expected: import failure for `wf_mcp.broker.service.connection_service`.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Implement ConnectionService
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/broker/service/connection_service.py`
|
||||
|
||||
- [ ] **Step 1: Add the service implementation**
|
||||
|
||||
Create `src/wf_mcp/broker/service/connection_service.py` with:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from ...connections import ConnectionRegistry, parse_connection_id
|
||||
from ...models import BrokerConfig, ConnectionConfig
|
||||
from ...shared.names import RESERVED_CONNECTION_IDS
|
||||
from .events import BrokerEventRecorder
|
||||
from .source_catalog import SourceCatalogService
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class ConnectionService:
|
||||
"""Own broker connection registration and config reconciliation.
|
||||
|
||||
SourceCatalogService needs connection lookup callbacks during construction,
|
||||
while registering a connection needs source-catalog hydration. The catalog is
|
||||
therefore bound after both services exist; `_source_catalog()` makes that
|
||||
construction cycle explicit and fail-fast.
|
||||
"""
|
||||
|
||||
events: BrokerEventRecorder
|
||||
connections: ConnectionRegistry = field(default_factory=ConnectionRegistry)
|
||||
source_catalog: SourceCatalogService | None = None
|
||||
|
||||
def bind_source_catalog(self, source_catalog: SourceCatalogService) -> None:
|
||||
self.source_catalog = source_catalog
|
||||
|
||||
def get(self, connection_id: str) -> ConnectionConfig:
|
||||
return self.connections.get(connection_id)
|
||||
|
||||
def list_all(self) -> list[ConnectionConfig]:
|
||||
return self.connections.list_all()
|
||||
|
||||
def list_enabled(self) -> list[ConnectionConfig]:
|
||||
return self.connections.list_enabled()
|
||||
|
||||
def register_connection(self, connection: ConnectionConfig) -> None:
|
||||
self._validate_connection_id(connection.id)
|
||||
self.connections.register(connection)
|
||||
self._source_catalog().hydrate_connection_source_from_snapshot(connection)
|
||||
self.events.record_kind(
|
||||
"connection_registered",
|
||||
connection_id=connection.id,
|
||||
payload={"server": connection.server, "account": connection.account},
|
||||
)
|
||||
|
||||
def sync_connections_from_config(self, config: BrokerConfig) -> None:
|
||||
"""Reconcile registry/source state after the public server reloads config."""
|
||||
source_catalog = self._source_catalog()
|
||||
next_ids = {connection.id for connection in config.connections}
|
||||
previous_ids = set(self.connections.connections)
|
||||
for connection_id in previous_ids - next_ids:
|
||||
del self.connections.connections[connection_id]
|
||||
source_catalog.capability_sources.pop(connection_id, None)
|
||||
|
||||
for connection in config.connections:
|
||||
self._validate_connection_id(connection.id)
|
||||
self.connections.register(connection)
|
||||
source = source_catalog.capability_sources.get(connection.id)
|
||||
if source is None:
|
||||
source_catalog.hydrate_connection_source_from_snapshot(connection)
|
||||
else:
|
||||
source.enabled = connection.enabled
|
||||
|
||||
def _source_catalog(self) -> SourceCatalogService:
|
||||
if self.source_catalog is None:
|
||||
raise RuntimeError("ConnectionService requires a bound SourceCatalogService")
|
||||
return self.source_catalog
|
||||
|
||||
@staticmethod
|
||||
def _validate_connection_id(connection_id: str) -> None:
|
||||
parse_connection_id(connection_id)
|
||||
if connection_id in RESERVED_CONNECTION_IDS:
|
||||
raise ValueError(f"connection id {connection_id!r} is reserved by wf-mcp")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the direct tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
- [ ] **Step 3: Run ruff on the new files**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/connection_service.py tests/wf_mcp/service/test_connection_service.py
|
||||
```
|
||||
|
||||
Expected: all checks pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Wire WfMcpService Through ConnectionService
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
|
||||
- [ ] **Step 1: Update imports and dataclass fields**
|
||||
|
||||
In `src/wf_mcp/broker/service/core.py`:
|
||||
|
||||
Remove:
|
||||
|
||||
```python
|
||||
from ...connections import ConnectionRegistry, parse_connection_id
|
||||
from ...shared.names import RESERVED_CONNECTION_IDS
|
||||
```
|
||||
|
||||
Replace with:
|
||||
|
||||
```python
|
||||
from ...connections import ConnectionRegistry
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
from .connection_service import ConnectionService
|
||||
```
|
||||
|
||||
In `WfMcpService`, remove the dataclass field:
|
||||
|
||||
```python
|
||||
connections: ConnectionRegistry = field(default_factory=ConnectionRegistry)
|
||||
```
|
||||
|
||||
Add this init-false field near the other service fields:
|
||||
|
||||
```python
|
||||
connection_service: ConnectionService = field(init=False)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Construct and bind the connection service**
|
||||
|
||||
In `__post_init__`, replace the source-catalog construction block with this shape:
|
||||
|
||||
```python
|
||||
self.events = BrokerEventRecorder(self.event_bus)
|
||||
self.connection_service = ConnectionService(events=self.events)
|
||||
self.upstream = UpstreamTransportService(
|
||||
store=self.store,
|
||||
event_sink=self.events.record_event,
|
||||
tool_executor=self.tool_executor,
|
||||
)
|
||||
self.source_catalog = SourceCatalogService(
|
||||
store=self.store,
|
||||
connection_lookup=self.connection_service.get,
|
||||
connection_list_enabled=self.connection_service.list_enabled,
|
||||
connection_list_all=self.connection_service.list_all,
|
||||
tool_executor_for=self.upstream.tool_executor_for,
|
||||
load_auth=self.upstream.load_auth,
|
||||
emit_event=self.events.record_event,
|
||||
default_catalog_max_age_seconds=self.default_catalog_max_age_seconds,
|
||||
)
|
||||
self.connection_service.bind_source_catalog(self.source_catalog)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Preserve `.connections` compatibility as a property**
|
||||
|
||||
Add this property below `capability_sources` or above it:
|
||||
|
||||
```python
|
||||
@property
|
||||
def connections(self) -> ConnectionRegistry:
|
||||
"""Compatibility view of the broker connection registry.
|
||||
|
||||
Connection lifecycle ownership has moved to ConnectionService. Keep this
|
||||
property because admin handlers, CLI helpers, and tests still inspect the
|
||||
registry through the service facade.
|
||||
"""
|
||||
return self.connection_service.connections
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Replace connection lifecycle method bodies with delegates**
|
||||
|
||||
Replace `register_connection` with:
|
||||
|
||||
```python
|
||||
def register_connection(self, connection: ConnectionConfig) -> None:
|
||||
self.connection_service.register_connection(connection)
|
||||
```
|
||||
|
||||
Replace `sync_connections_from_config` with:
|
||||
|
||||
```python
|
||||
def sync_connections_from_config(self, config: BrokerConfig) -> None:
|
||||
self.connection_service.sync_connections_from_config(config)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run focused service tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/service/test_catalog.py tests/wf_mcp/service/test_sources.py tests/wf_mcp/test_events.py -q
|
||||
```
|
||||
|
||||
Expected: all selected tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add Facade Compatibility Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/service/test_connection_service.py`
|
||||
|
||||
- [ ] **Step 1: Add WfMcpService compatibility coverage**
|
||||
|
||||
Append these imports:
|
||||
|
||||
```python
|
||||
from wf_mcp.broker import WfMcpService
|
||||
```
|
||||
|
||||
Append these tests:
|
||||
|
||||
```python
|
||||
def test_wfmcpservice_exposes_connection_registry_from_connection_service() -> None:
|
||||
service = WfMcpService(store=FileStore(local_temp_root() / "connection_facade"))
|
||||
|
||||
service.register_connection(
|
||||
ConnectionConfig(id="demo.personal", server="demo", account="personal")
|
||||
)
|
||||
|
||||
assert service.connections is service.connection_service.connections
|
||||
assert service.connections.get("demo.personal").account == "personal"
|
||||
assert "demo.personal" in service.capability_sources
|
||||
|
||||
|
||||
def test_wfmcpservice_sync_connections_delegates_to_connection_service() -> None:
|
||||
service = WfMcpService(store=FileStore(local_temp_root() / "connection_sync"))
|
||||
service.register_connection(
|
||||
ConnectionConfig(id="demo.personal", server="demo", account="personal")
|
||||
)
|
||||
|
||||
service.sync_connections_from_config(
|
||||
BrokerConfig(
|
||||
store_root=local_temp_root(),
|
||||
connections=[
|
||||
ConnectionConfig(
|
||||
id="demo.work",
|
||||
server="demo",
|
||||
account="work",
|
||||
enabled=True,
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
assert [connection.id for connection in service.connections.list_all()] == [
|
||||
"demo.work"
|
||||
]
|
||||
assert "demo.personal" not in service.capability_sources
|
||||
assert "demo.work" in service.capability_sources
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the compatibility tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Clean Imports and Verify Call Sites
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Possibly modify files only if ruff reports stale imports.
|
||||
|
||||
- [ ] **Step 1: Search for stale direct ownership assumptions**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n 'parse_connection_id|RESERVED_CONNECTION_IDS|connection_service|ConnectionRegistry|connections: ConnectionRegistry' src/wf_mcp/broker/service tests/wf_mcp/service
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- `parse_connection_id` and `RESERVED_CONNECTION_IDS` appear in `connection_service.py`, not `core.py`.
|
||||
- `connections: ConnectionRegistry` appears in `connection_service.py`, not `core.py`.
|
||||
- `connection_service` appears in `core.py` and direct tests.
|
||||
|
||||
- [ ] **Step 2: Run ruff on modified service files**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/connection_service.py tests/wf_mcp/service/test_connection_service.py
|
||||
```
|
||||
|
||||
Expected: all checks pass. If ruff reports unused imports in `core.py`, remove only those imports.
|
||||
|
||||
- [ ] **Step 3: Run basedpyright on modified source**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected: 0 errors.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Update Roadmap and Extraction Map
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
- Modify if present/relevant: `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md`
|
||||
|
||||
- [ ] **Step 1: Update `docs/current_roadmap.md`**
|
||||
|
||||
Find the bullet that says:
|
||||
|
||||
```markdown
|
||||
- Next planned service extraction: move connection registration/config reconciliation
|
||||
into a `ConnectionService`. That slice should own reserved connection-id
|
||||
rejection, `register_connection`, `sync_connections_from_config`, and source
|
||||
hydration coordination while leaving `WfMcpService` as a compatibility
|
||||
coordinator.
|
||||
```
|
||||
|
||||
Replace it with:
|
||||
|
||||
```markdown
|
||||
- Connection ownership now lives in `ConnectionService`: it owns the broker
|
||||
`ConnectionRegistry`, reserved connection-id rejection, `register_connection`,
|
||||
and `sync_connections_from_config`. `WfMcpService.connections` remains a
|
||||
compatibility property while source hydration still belongs to
|
||||
`SourceCatalogService`.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update the extraction map if it contains the WfMcpService split notes**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n 'ConnectionService|connection registration|sync_connections_from_config|WfMcpService' docs/superpowers/research/2026-06-01-wf-api-extraction-map.md
|
||||
```
|
||||
|
||||
If the file exists and contains the service split section, add this short note near the other extracted-service bullets:
|
||||
|
||||
```markdown
|
||||
- Connection registration/config reload reconciliation is now owned by
|
||||
`wf_mcp.broker.service.connection_service.ConnectionService`. The service owns
|
||||
the `ConnectionRegistry`; `WfMcpService.connections` is only a compatibility
|
||||
property.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run docs grep to verify roadmap wording**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n 'ConnectionService|Connection ownership|Next planned service extraction' docs/current_roadmap.md docs/superpowers/research/2026-06-01-wf-api-extraction-map.md
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- `docs/current_roadmap.md` mentions completed `ConnectionService` ownership.
|
||||
- No stale "Next planned service extraction" wording for this same slice remains.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Final Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- No new files.
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/service/test_catalog.py tests/wf_mcp/service/test_sources.py tests/wf_mcp/service/test_events.py tests/wf_mcp/test_broker_server.py tests/wf_mcp/test_admin_surface.py -q
|
||||
```
|
||||
|
||||
Expected: all selected tests pass.
|
||||
|
||||
- [ ] **Step 2: Run full test suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
Expected: full suite passes with the existing skipped/xfailed counts only.
|
||||
|
||||
- [ ] **Step 3: Run final static checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check
|
||||
uv run ruff format --check
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- ruff check passes.
|
||||
- ruff format check passes for Python files.
|
||||
- basedpyright reports 0 errors.
|
||||
|
||||
If markdown format checks complain about preview-only markdown behavior, do not rewrite unrelated markdown. Report it as a formatting-tool limitation and keep the code checks green.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: The plan moves reserved ID validation, `register_connection`, and `sync_connections_from_config` into `ConnectionService`; preserves `.connections` compatibility; keeps source hydration coordination explicit through a post-construction bind.
|
||||
- Placeholder scan: No `TBD`, generic "add tests", or unfilled implementation steps remain.
|
||||
- Type consistency: `ConnectionService` exposes `get`, `list_all`, and `list_enabled` so `SourceCatalogService` can use bound methods without depending on `WfMcpService`.
|
||||
- Risk: The construction cycle between connection lookup and source hydration is intentionally represented by `bind_source_catalog()`. The fail-fast `_source_catalog()` guard prevents silent use before binding.
|
||||
+691
@@ -0,0 +1,691 @@
|
||||
# WfMcpService Event Recorder Extraction Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Extract broker event recording and catalog-change event fanout from `WfMcpService` into a focused `BrokerEventRecorder` while preserving existing event history, subscribers, and public service methods.
|
||||
|
||||
**Architecture:** Add `BrokerEventRecorder` under `wf_mcp.broker.service`. It wraps the existing `EventBus`, owns `record_event`, `record_kind`, `record_catalog_change_events`, and `list_events`. `WfMcpService` keeps `event_bus` for compatibility with server/proxy wiring and keeps `_record_event` / `_record_catalog_change_events` as delegates until later cleanup.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, existing `wf_mcp.events.EventBus`/`McpEvent`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
Move now:
|
||||
|
||||
- Direct `event_bus.publish(...)` usage.
|
||||
- `list_events`.
|
||||
- `_record_event`, as `BrokerEventRecorder.record_event`.
|
||||
- `_record_catalog_change_events`, as `BrokerEventRecorder.record_catalog_change_events`.
|
||||
- Event construction helper for simple workflow/API events, as `BrokerEventRecorder.record_kind`.
|
||||
|
||||
Keep now:
|
||||
|
||||
- `WfMcpService.event_bus` dataclass field for server/proxy compatibility.
|
||||
- `WfMcpService.list_events`, `_record_event`, and `_record_catalog_change_events` as delegates.
|
||||
- Event kind strings and payload shapes.
|
||||
- Existing `wf_api.operation_context.WorkflowEventRecorder` protocol.
|
||||
|
||||
Do not do in this slice:
|
||||
|
||||
- Do not move `EventBus` itself.
|
||||
- Do not change notification projection.
|
||||
- Do not convert all event callers to a new domain event enum.
|
||||
- Do not remove private delegate methods from `WfMcpService`.
|
||||
|
||||
---
|
||||
|
||||
## Target File Structure
|
||||
|
||||
- Create `src/wf_mcp/broker/service/events.py`
|
||||
- Defines `BrokerEventRecorder`.
|
||||
- Owns catalog-change fanout logic.
|
||||
- Has docstrings stating this is broker-local event recording, not MCP notification delivery.
|
||||
|
||||
- Modify `src/wf_mcp/broker/service/core.py`
|
||||
- Add `events: BrokerEventRecorder = field(init=False)`.
|
||||
- Construct it from `event_bus` in `__post_init__`.
|
||||
- Pass `self.events.record_event` into `UpstreamTransportService`, `SourceCatalogService`, and `WorkflowRuntimeService`.
|
||||
- Delegate public/private event methods.
|
||||
|
||||
- Modify `src/wf_mcp/broker/service/workflow_operation_context.py`
|
||||
- `WfMcpWorkflowEventRecorder` should hold `BrokerEventRecorder`, not call `service._record_event`.
|
||||
|
||||
- Add tests in `tests/wf_mcp/service/test_event_recorder.py`.
|
||||
|
||||
- Update docs:
|
||||
- `docs/current_roadmap.md`.
|
||||
- `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` if stale.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add BrokerEventRecorder Skeleton
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/broker/service/events.py`
|
||||
- Create: `tests/wf_mcp/service/test_event_recorder.py`
|
||||
|
||||
- [ ] **Step 1: Write direct event recorder tests**
|
||||
|
||||
Create `tests/wf_mcp/service/test_event_recorder.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_mcp.broker.service.events import BrokerEventRecorder
|
||||
from wf_mcp.events import EventBus, make_event
|
||||
|
||||
|
||||
def test_broker_event_recorder_records_existing_event() -> None:
|
||||
bus = EventBus()
|
||||
recorder = BrokerEventRecorder(bus)
|
||||
event = make_event("connection_registered", connection_id="demo.personal")
|
||||
|
||||
recorder.record_event(event)
|
||||
|
||||
assert recorder.list_events()[0] is event
|
||||
assert bus.list_events()[0] is event
|
||||
|
||||
|
||||
def test_broker_event_recorder_builds_simple_event() -> None:
|
||||
recorder = BrokerEventRecorder(EventBus())
|
||||
|
||||
recorder.record_kind(
|
||||
"workflow_artifact_saved",
|
||||
capability_id="echo",
|
||||
payload={"version": 1},
|
||||
)
|
||||
|
||||
event = recorder.list_events()[0]
|
||||
assert event.kind == "workflow_artifact_saved"
|
||||
assert event.capability_id == "echo"
|
||||
assert event.payload["version"] == 1
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests and verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_event_recorder.py::test_broker_event_recorder_records_existing_event tests/wf_mcp/service/test_event_recorder.py::test_broker_event_recorder_builds_simple_event -q
|
||||
```
|
||||
|
||||
Expected: import failure because `wf_mcp.broker.service.events` does not exist.
|
||||
|
||||
- [ ] **Step 3: Create BrokerEventRecorder**
|
||||
|
||||
Create `src/wf_mcp/broker/service/events.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from wf_mcp.events import EventBus, McpEvent, make_event
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class BrokerEventRecorder:
|
||||
"""Broker-local event recorder backed by the existing EventBus.
|
||||
|
||||
This class records and fans out local service events. MCP notifications are
|
||||
still projected by subscribers/resources elsewhere; this is only the broker
|
||||
event emission boundary.
|
||||
"""
|
||||
|
||||
event_bus: EventBus
|
||||
|
||||
def record_event(self, event: McpEvent) -> None:
|
||||
self.event_bus.publish(event)
|
||||
|
||||
def record_kind(
|
||||
self,
|
||||
event_type: str,
|
||||
*,
|
||||
connection_id: str | None = None,
|
||||
capability_id: str | None = None,
|
||||
workflow_name: str | None = None,
|
||||
payload: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
self.record_event(
|
||||
make_event(
|
||||
event_type,
|
||||
connection_id=connection_id,
|
||||
capability_id=capability_id,
|
||||
workflow_name=workflow_name,
|
||||
payload=payload or {},
|
||||
)
|
||||
)
|
||||
|
||||
def list_events(self) -> list[McpEvent]:
|
||||
return self.event_bus.list_events()
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_event_recorder.py::test_broker_event_recorder_records_existing_event tests/wf_mcp/service/test_event_recorder.py::test_broker_event_recorder_builds_simple_event -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/events.py tests/wf_mcp/service/test_event_recorder.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Move Catalog-Change Fanout Into BrokerEventRecorder
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/events.py`
|
||||
- Test: `tests/wf_mcp/service/test_event_recorder.py`
|
||||
|
||||
- [ ] **Step 1: Write direct catalog fanout test**
|
||||
|
||||
Append to `tests/wf_mcp/service/test_event_recorder.py`:
|
||||
|
||||
```python
|
||||
from wf_mcp.models import CatalogSnapshot
|
||||
|
||||
|
||||
def test_broker_event_recorder_records_catalog_change_fanout() -> None:
|
||||
recorder = BrokerEventRecorder(EventBus())
|
||||
snapshot = CatalogSnapshot(
|
||||
connection_id="demo.personal",
|
||||
nodes=[
|
||||
{
|
||||
"qualified_name": "demo.personal.echo",
|
||||
"connection_id": "demo.personal",
|
||||
"local_name": "echo",
|
||||
"input_schema": {},
|
||||
"output_schema": {},
|
||||
"outcomes": ["ok"],
|
||||
}
|
||||
],
|
||||
resources=[
|
||||
{
|
||||
"qualified_name": "demo.personal.resource.welcome",
|
||||
"connection_id": "demo.personal",
|
||||
"local_name": "welcome",
|
||||
"uri": "demo://welcome",
|
||||
}
|
||||
],
|
||||
prompts=[
|
||||
{
|
||||
"qualified_name": "demo.personal.prompt.welcome",
|
||||
"connection_id": "demo.personal",
|
||||
"local_name": "welcome",
|
||||
}
|
||||
],
|
||||
fetched_at_epoch_ms=1,
|
||||
max_age_seconds=300,
|
||||
)
|
||||
|
||||
recorder.record_catalog_change_events(
|
||||
"demo.personal",
|
||||
snapshot,
|
||||
reason="catalog_refresh",
|
||||
)
|
||||
|
||||
events = recorder.list_events()
|
||||
event_kinds = [event.kind for event in events]
|
||||
assert event_kinds == [
|
||||
"tools_changed",
|
||||
"resources_changed",
|
||||
"prompts_changed",
|
||||
"catalog_changed",
|
||||
]
|
||||
assert events[-1].payload["reason"] == "catalog_refresh"
|
||||
assert events[-1].payload["node_count"] == 1
|
||||
assert events[-1].payload["resource_count"] == 1
|
||||
assert events[-1].payload["prompt_count"] == 1
|
||||
```
|
||||
|
||||
If `CatalogSnapshot` requires additional fields in this repo, inspect an existing fixture in `tests/wf_mcp/service/test_events.py` and use the same minimal shape. Do not weaken the assertion to only “some event exists.”
|
||||
|
||||
- [ ] **Step 2: Run the fanout test and verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_event_recorder.py::test_broker_event_recorder_records_catalog_change_fanout -q
|
||||
```
|
||||
|
||||
Expected: fail because `record_catalog_change_events` does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement catalog fanout**
|
||||
|
||||
In `src/wf_mcp/broker/service/events.py`, import:
|
||||
|
||||
```python
|
||||
from wf_mcp.models import CatalogSnapshot
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
def record_catalog_change_events(
|
||||
self,
|
||||
connection_id: str,
|
||||
snapshot: CatalogSnapshot,
|
||||
*,
|
||||
reason: str,
|
||||
) -> None:
|
||||
"""Emit local change events that future MCP notifications can project."""
|
||||
counts = {
|
||||
"node_count": len(snapshot.nodes),
|
||||
"resource_count": len(snapshot.resources),
|
||||
"prompt_count": len(snapshot.prompts),
|
||||
}
|
||||
if snapshot.nodes:
|
||||
self.record_kind(
|
||||
"tools_changed",
|
||||
connection_id=connection_id,
|
||||
payload={"reason": reason, "node_count": counts["node_count"]},
|
||||
)
|
||||
if snapshot.resources:
|
||||
self.record_kind(
|
||||
"resources_changed",
|
||||
connection_id=connection_id,
|
||||
payload={
|
||||
"reason": reason,
|
||||
"resource_count": counts["resource_count"],
|
||||
},
|
||||
)
|
||||
if snapshot.prompts:
|
||||
self.record_kind(
|
||||
"prompts_changed",
|
||||
connection_id=connection_id,
|
||||
payload={"reason": reason, "prompt_count": counts["prompt_count"]},
|
||||
)
|
||||
self.record_kind(
|
||||
"catalog_changed",
|
||||
connection_id=connection_id,
|
||||
payload={"reason": reason, **counts},
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run event recorder tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_event_recorder.py -q
|
||||
```
|
||||
|
||||
Expected: all tests in the file pass.
|
||||
|
||||
- [ ] **Step 5: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/events.py tests/wf_mcp/service/test_event_recorder.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Wire BrokerEventRecorder Into WfMcpService
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Test: `tests/wf_mcp/test_events.py`
|
||||
- Test: `tests/wf_mcp/service/test_event_recorder.py`
|
||||
|
||||
- [ ] **Step 1: Add service identity test**
|
||||
|
||||
Append to `tests/wf_mcp/service/test_event_recorder.py`:
|
||||
|
||||
```python
|
||||
from wf_mcp.broker import WfMcpService
|
||||
from wf_mcp.storage import FileStore
|
||||
|
||||
from ..test_support import local_temp_root
|
||||
|
||||
|
||||
def test_wfmcpservice_uses_broker_event_recorder() -> None:
|
||||
bus = EventBus()
|
||||
service = WfMcpService(
|
||||
store=FileStore(local_temp_root() / "service_event_recorder"),
|
||||
event_bus=bus,
|
||||
)
|
||||
|
||||
service._record_event( # noqa: SLF001
|
||||
make_event("connection_registered", connection_id="demo.personal")
|
||||
)
|
||||
|
||||
assert service.events.event_bus is bus
|
||||
assert service.list_events()[0].kind == "connection_registered"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the service identity test and verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_event_recorder.py::test_wfmcpservice_uses_broker_event_recorder -q
|
||||
```
|
||||
|
||||
Expected: fail because `service.events` does not exist.
|
||||
|
||||
- [ ] **Step 3: Add events field to WfMcpService**
|
||||
|
||||
In `src/wf_mcp/broker/service/core.py`, import:
|
||||
|
||||
```python
|
||||
from .events import BrokerEventRecorder
|
||||
```
|
||||
|
||||
Add dataclass field:
|
||||
|
||||
```python
|
||||
events: BrokerEventRecorder = field(init=False)
|
||||
```
|
||||
|
||||
At the top of `__post_init__`, before constructing `UpstreamTransportService`, add:
|
||||
|
||||
```python
|
||||
self.events = BrokerEventRecorder(self.event_bus)
|
||||
```
|
||||
|
||||
Update service construction callbacks:
|
||||
|
||||
```python
|
||||
self.upstream = UpstreamTransportService(
|
||||
store=self.store,
|
||||
event_sink=self.events.record_event,
|
||||
tool_executor=self.tool_executor,
|
||||
)
|
||||
```
|
||||
|
||||
```python
|
||||
emit_event=self.events.record_event,
|
||||
```
|
||||
|
||||
```python
|
||||
emit_event=self.events.record_event,
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Delegate event methods**
|
||||
|
||||
Replace:
|
||||
|
||||
```python
|
||||
def list_events(self) -> list[McpEvent]:
|
||||
return self.event_bus.list_events()
|
||||
|
||||
def _record_event(self, event: McpEvent) -> None:
|
||||
self.event_bus.publish(event)
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
def list_events(self) -> list[McpEvent]:
|
||||
return self.events.list_events()
|
||||
|
||||
def _record_event(self, event: McpEvent) -> None:
|
||||
self.events.record_event(event)
|
||||
```
|
||||
|
||||
Replace `_record_catalog_change_events` body with:
|
||||
|
||||
```python
|
||||
self.events.record_catalog_change_events(
|
||||
connection_id,
|
||||
snapshot,
|
||||
reason=reason,
|
||||
)
|
||||
```
|
||||
|
||||
Keep the method signature and docstring.
|
||||
|
||||
- [ ] **Step 5: Run service event tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_event_recorder.py::test_wfmcpservice_uses_broker_event_recorder tests/wf_mcp/test_events.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 6: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/events.py tests/wf_mcp/service/test_event_recorder.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Update WorkflowOperationContext Event Adapter
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/workflow_operation_context.py`
|
||||
- Test: `tests/wf_api/test_operation_context.py`
|
||||
|
||||
- [ ] **Step 1: Strengthen context event adapter test**
|
||||
|
||||
In `tests/wf_api/test_operation_context.py`, add or update a test:
|
||||
|
||||
```python
|
||||
def test_context_event_recorder_uses_broker_event_recorder(tmp_path: Path) -> None:
|
||||
service = WfMcpService(store=FileStore(tmp_path / "context_events"))
|
||||
context = context_from_service(service)
|
||||
|
||||
context.events.record_workflow_event(
|
||||
"workflow_artifact_saved",
|
||||
capability_id="echo",
|
||||
payload={"version": 1},
|
||||
)
|
||||
|
||||
assert service.events.list_events()[-1].kind == "workflow_artifact_saved"
|
||||
assert service.events.list_events()[-1].capability_id == "echo"
|
||||
```
|
||||
|
||||
If this file already has an equivalent `record_workflow_event` test, update its assertions to read through `service.events.list_events()` instead of only `service.list_events()`.
|
||||
|
||||
- [ ] **Step 2: Run the context event test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_operation_context.py::test_context_event_recorder_uses_broker_event_recorder -q
|
||||
```
|
||||
|
||||
If the test name was updated instead of added, run the actual updated test name.
|
||||
|
||||
Expected: fail until the adapter stops calling private service methods, or pass if Task 3 delegates already cover it.
|
||||
|
||||
- [ ] **Step 3: Update WfMcpWorkflowEventRecorder**
|
||||
|
||||
In `src/wf_mcp/broker/service/workflow_operation_context.py`, import:
|
||||
|
||||
```python
|
||||
from .events import BrokerEventRecorder
|
||||
```
|
||||
|
||||
Change:
|
||||
|
||||
```python
|
||||
class WfMcpWorkflowEventRecorder(WorkflowEventRecorder):
|
||||
"""Adapter-owned event recorder backed by WfMcpService."""
|
||||
|
||||
service: WfMcpService
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
class WfMcpWorkflowEventRecorder(WorkflowEventRecorder):
|
||||
"""Adapter-owned event recorder backed by BrokerEventRecorder."""
|
||||
|
||||
events: BrokerEventRecorder
|
||||
```
|
||||
|
||||
Replace method bodies:
|
||||
|
||||
```python
|
||||
def record_event(self, event: Any) -> None:
|
||||
self.events.record_event(event)
|
||||
|
||||
def record_workflow_event(
|
||||
self,
|
||||
event_type: str,
|
||||
*,
|
||||
capability_id: str,
|
||||
payload: dict[str, Any],
|
||||
) -> None:
|
||||
self.events.record_kind(
|
||||
event_type,
|
||||
capability_id=capability_id,
|
||||
payload=payload,
|
||||
)
|
||||
```
|
||||
|
||||
In `context_from_service`, change:
|
||||
|
||||
```python
|
||||
events=WfMcpWorkflowEventRecorder(service),
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
events=WfMcpWorkflowEventRecorder(service.events),
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run operation context tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_operation_context.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/workflow_operation_context.py tests/wf_api/test_operation_context.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Clean Imports, Docs, and Verify
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Modify: `src/wf_mcp/broker/service/workflow_operation_context.py`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
- Modify: `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` if stale.
|
||||
|
||||
- [ ] **Step 1: Remove stale imports**
|
||||
|
||||
After the move, `src/wf_mcp/broker/service/core.py` should no longer import `make_event` directly unless it is still used for local resource/local prompt/connection registration events.
|
||||
|
||||
If `make_event` is still used only in these service-local cases:
|
||||
|
||||
```python
|
||||
connection_registered
|
||||
resource_read_completed for local docs
|
||||
prompt_get_completed for local docs
|
||||
```
|
||||
|
||||
leave it for now. Those local cases can move in a later connection/resource admin extraction.
|
||||
|
||||
`src/wf_mcp/broker/service/workflow_operation_context.py` should no longer import `make_event`.
|
||||
|
||||
- [ ] **Step 2: Add roadmap note**
|
||||
|
||||
In `docs/current_roadmap.md`, under the service extraction bullets, add:
|
||||
|
||||
```markdown
|
||||
- Broker event recording is being separated from broker coordination.
|
||||
`BrokerEventRecorder` now owns EventBus publication, event history reads,
|
||||
simple event construction, and catalog-change fanout. `WfMcpService` keeps
|
||||
delegate methods for compatibility.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update extraction map if stale**
|
||||
|
||||
If `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` says `WfMcpService` directly owns event recording/fanout, add:
|
||||
|
||||
```markdown
|
||||
Event recording ownership is now split: `BrokerEventRecorder` owns EventBus
|
||||
publication, simple event construction, event history reads, and catalog-change
|
||||
fanout. `WfMcpService` remains the coordinator and compatibility façade.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_event_recorder.py tests/wf_mcp/test_events.py tests/wf_mcp/service/test_events.py tests/wf_api/test_operation_context.py tests/wf_mcp/workflow_surface/test_deployments.py -q
|
||||
```
|
||||
|
||||
Expected: all selected tests pass.
|
||||
|
||||
- [ ] **Step 5: Run full verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
uv run ruff check src/wf_mcp/broker/service src/wf_api tests/wf_mcp/service tests/wf_api
|
||||
uv run ruff format --check src/wf_mcp/broker/service src/wf_api tests/wf_mcp/service tests/wf_api docs/current_roadmap.md
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- pytest passes.
|
||||
- ruff check passes.
|
||||
- ruff format check passes.
|
||||
- basedpyright reports `0 errors`. If the known workspace enumeration warning causes a nonzero exit despite `0 errors`, record the exact output.
|
||||
|
||||
---
|
||||
|
||||
## Non-Goals and Follow-Up Slices
|
||||
|
||||
This plan intentionally leaves these for later:
|
||||
|
||||
1. **Connection service extraction:** move `register_connection`, reserved-id policy, and config reconciliation.
|
||||
2. **Local resource/prompt admin extraction:** move local docs event emission out of `WfMcpService.read_resource` and `render_prompt`.
|
||||
3. **Final service rename:** once `WfMcpService` mostly wires implementation services, consider a clearer broker coordinator name.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: The plan extracts event recording and catalog-change fanout while preserving existing event bus compatibility and event payload shapes.
|
||||
- Placeholder scan: No placeholder implementation steps are left. The one import cleanup step explicitly says when to keep `make_event` because local admin event emission remains in `WfMcpService`.
|
||||
- Type consistency: `BrokerEventRecorder` consistently wraps `EventBus`; `WfMcpService.events` holds the recorder; workflow operation context depends on the recorder instead of private service methods.
|
||||
+709
@@ -0,0 +1,709 @@
|
||||
# WfMcpService Resource/Prompt Access Extraction Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Extract the local-docs-aware resource read and prompt render logic from `WfMcpService` into a focused `ContentAccessService` while preserving existing MCP admin, CLI, and workflow behavior.
|
||||
|
||||
**Architecture:** Add `ContentAccessService` under `wf_mcp.broker.service`. It owns the "is this a local doc? if so return inline, else look up catalog entry and delegate upstream" branching that currently lives in `WfMcpService.read_resource` and `WfMcpService.render_prompt`. `WfMcpService` remains the compatibility coordinator; its `read_resource` and `render_prompt` become one-line delegates.
|
||||
|
||||
**Why a new service instead of folding into `UpstreamTransportService`:** The local-docs check depends on `SourceCatalogService`, not on upstream adapters. Putting it in `UpstreamTransportService` would add a cross-cutting catalog dependency to a transport-focused service. A thin content-access layer keeps the boundary clean.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, `SourceCatalogService`, `UpstreamTransportService`, `ConnectionService`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
Move now:
|
||||
|
||||
- `WfMcpService.read_resource` branching logic (local docs check + upstream delegation).
|
||||
- `WfMcpService.render_prompt` branching logic (local docs check + upstream delegation).
|
||||
|
||||
Keep now:
|
||||
|
||||
- `invoke_method`. Already a clean delegate to `upstream`; no local-docs branching. Does not belong in this service.
|
||||
- `send_notification`. Same reasoning.
|
||||
- `refresh_connection_catalog`. Different semantics; stays on `WfMcpService` / `UpstreamTransportService`.
|
||||
- All other `WfMcpService` delegates and compatibility properties.
|
||||
- Event recording internals on `BrokerEventRecorder`.
|
||||
|
||||
Do not do in this slice:
|
||||
|
||||
- Do not move `invoke_method` or `send_notification` into `ContentAccessService`.
|
||||
- Do not move `refresh_connection_catalog`.
|
||||
- Do not rename `WfMcpService`.
|
||||
- Do not change adapter implementations.
|
||||
- Do not change MCP tool schemas or CLI behavior.
|
||||
|
||||
---
|
||||
|
||||
## Target File Structure
|
||||
|
||||
- Create `src/wf_mcp/broker/service/content_access.py`
|
||||
- Owns `ContentAccessService`.
|
||||
- Depends on `SourceCatalogService` (for local docs lookup and catalog entry resolution), `UpstreamTransportService` (for upstream reads/renders), `ConnectionService` (for connection lookup), and `BrokerEventRecorder` (for local-docs event recording).
|
||||
- Contains docstrings stating this is a broker-internal service, not a protocol-neutral API.
|
||||
|
||||
- Modify `src/wf_mcp/broker/service/core.py`
|
||||
- Add `content_access: ContentAccessService = field(init=False)`.
|
||||
- Construct it after `source_catalog`, `upstream`, `connection_service`, and `events` in `__post_init__`.
|
||||
- Replace `read_resource` and `render_prompt` bodies with one-line delegates.
|
||||
|
||||
- Add tests in `tests/wf_mcp/service/test_content_access.py`.
|
||||
|
||||
- Update docs:
|
||||
- `docs/superpowers/research/2026-06-02-wfmcpservice-remaining-responsibilities.md` if stale.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add ContentAccessService Skeleton
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/broker/service/content_access.py`
|
||||
- Create: `tests/wf_mcp/service/test_content_access.py`
|
||||
|
||||
- [ ] **Step 1: Write a direct local-resource test**
|
||||
|
||||
Create `tests/wf_mcp/service/test_content_access.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from wf_mcp.broker import WfMcpService
|
||||
from wf_mcp.broker.service.content_access import ContentAccessService
|
||||
from wf_mcp.broker.service.connection_service import ConnectionService
|
||||
from wf_mcp.broker.service.events import BrokerEventRecorder
|
||||
from wf_mcp.broker.service.source_catalog import SourceCatalogService
|
||||
from wf_mcp.broker.service.upstream_transport import UpstreamTransportService
|
||||
from wf_mcp.events import EventBus
|
||||
from wf_mcp.models import ConnectionConfig
|
||||
from wf_mcp.storage import FileStore
|
||||
from wf_platform import (
|
||||
CapabilityBuckets,
|
||||
CapabilitySource,
|
||||
DocumentationPrompt,
|
||||
DocumentationResource,
|
||||
SourceVisibility,
|
||||
)
|
||||
|
||||
from ..test_support import FakeAdapter, local_temp_root
|
||||
|
||||
|
||||
def _make_content_access(
|
||||
*,
|
||||
store_root: str = "content_access_default",
|
||||
) -> tuple[ContentAccessService, BrokerEventRecorder]:
|
||||
store = FileStore(local_temp_root() / store_root)
|
||||
events = BrokerEventRecorder(EventBus())
|
||||
connection_service = ConnectionService(events=events)
|
||||
upstream = UpstreamTransportService(
|
||||
store=store,
|
||||
event_sink=events.record_event,
|
||||
)
|
||||
source_catalog = SourceCatalogService(
|
||||
store=store,
|
||||
connection_lookup=connection_service.get,
|
||||
connection_list_enabled=connection_service.list_enabled,
|
||||
connection_list_all=connection_service.list_all,
|
||||
tool_executor_for=upstream.tool_executor_for,
|
||||
load_auth=upstream.load_auth,
|
||||
emit_event=events.record_event,
|
||||
)
|
||||
connection_service.bind_source_catalog(source_catalog)
|
||||
_register_local_docs(source_catalog)
|
||||
content_access = ContentAccessService(
|
||||
source_catalog=source_catalog,
|
||||
upstream=upstream,
|
||||
connection_service=connection_service,
|
||||
event_sink=events.record_event,
|
||||
)
|
||||
return content_access, events
|
||||
|
||||
|
||||
def _register_local_docs(source_catalog: SourceCatalogService) -> None:
|
||||
"""Install deterministic local docs without depending on repo Markdown files."""
|
||||
source_catalog.register_capability_source(
|
||||
CapabilitySource(
|
||||
id="test.docs",
|
||||
kind="system",
|
||||
capabilities=CapabilityBuckets(
|
||||
resources={
|
||||
"test.docs.example": DocumentationResource(
|
||||
name="test.docs.example",
|
||||
uri="wf://docs/example",
|
||||
title="Example Doc",
|
||||
description="Test documentation resource.",
|
||||
mime_type="text/markdown",
|
||||
text="# Example",
|
||||
)
|
||||
},
|
||||
prompts={
|
||||
"test.docs.guide": DocumentationPrompt(
|
||||
name="test.docs.guide",
|
||||
title="Guide Prompt",
|
||||
description="Test documentation prompt.",
|
||||
text="Use the test docs.",
|
||||
)
|
||||
},
|
||||
),
|
||||
visibility=SourceVisibility(planner=True),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def test_content_access_reads_local_documentation_resource() -> None:
|
||||
content_access, events = _make_content_access()
|
||||
|
||||
result = asyncio.run(
|
||||
content_access.read_resource("test.docs.example")
|
||||
)
|
||||
|
||||
assert result["contents"][0]["uri"] == "wf://docs/example"
|
||||
assert result["contents"][0]["text"] == "# Example"
|
||||
assert "resource_read_completed" in [e.kind for e in events.list_events()]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the test and verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_reads_local_documentation_resource -q
|
||||
```
|
||||
|
||||
Expected: import failure because `content_access.py` does not exist.
|
||||
|
||||
- [ ] **Step 3: Create the skeleton service**
|
||||
|
||||
Create `src/wf_mcp/broker/service/content_access.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from wf_mcp.events import McpEvent, make_event
|
||||
|
||||
from .connection_service import ConnectionService
|
||||
from .source_catalog import SourceCatalogService
|
||||
from .upstream_transport import UpstreamTransportService
|
||||
|
||||
EventSink = Callable[[McpEvent], None]
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class ContentAccessService:
|
||||
"""Own local-docs-aware resource reads and prompt renders for the broker.
|
||||
|
||||
This service checks SourceCatalogService for local documentation entries
|
||||
before falling back to upstream transport. It is broker-internal, not a
|
||||
protocol-neutral content API.
|
||||
"""
|
||||
|
||||
source_catalog: SourceCatalogService
|
||||
upstream: UpstreamTransportService
|
||||
connection_service: ConnectionService
|
||||
event_sink: EventSink
|
||||
|
||||
async def read_resource(self, qualified_name: str) -> dict[str, Any]:
|
||||
local_resource = self.source_catalog.local_documentation_resource(
|
||||
qualified_name
|
||||
)
|
||||
if local_resource is not None:
|
||||
self.event_sink(
|
||||
make_event(
|
||||
"resource_read_completed",
|
||||
capability_id=qualified_name,
|
||||
payload={"uri": local_resource.uri, "source": "local"},
|
||||
)
|
||||
)
|
||||
return {
|
||||
"contents": [
|
||||
{
|
||||
"uri": local_resource.uri,
|
||||
"mimeType": local_resource.mime_type,
|
||||
"text": local_resource.text,
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
resource = self.source_catalog.get_resource(qualified_name)
|
||||
connection = self.connection_service.get(resource.connection_id)
|
||||
return await self.upstream.read_resource(
|
||||
connection,
|
||||
qualified_name,
|
||||
resource.uri,
|
||||
)
|
||||
|
||||
async def render_prompt(
|
||||
self,
|
||||
qualified_name: str,
|
||||
*,
|
||||
arguments: dict[str, str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
local_prompt = self.source_catalog.local_documentation_prompt(qualified_name)
|
||||
if local_prompt is not None:
|
||||
self.event_sink(
|
||||
make_event(
|
||||
"prompt_get_completed",
|
||||
capability_id=qualified_name,
|
||||
payload={
|
||||
"argument_keys": sorted((arguments or {}).keys()),
|
||||
"source": "local",
|
||||
},
|
||||
)
|
||||
)
|
||||
return {
|
||||
"description": local_prompt.description,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": {
|
||||
"type": "text",
|
||||
"text": local_prompt.text,
|
||||
},
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
prompt = self.source_catalog.get_prompt(qualified_name)
|
||||
connection = self.connection_service.get(prompt.connection_id)
|
||||
return await self.upstream.render_prompt(
|
||||
connection,
|
||||
qualified_name,
|
||||
prompt.local_name,
|
||||
arguments,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_reads_local_documentation_resource -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/content_access.py tests/wf_mcp/service/test_content_access.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Add Direct Upstream Resource and Prompt Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/service/test_content_access.py`
|
||||
|
||||
- [ ] **Step 1: Add upstream resource read test**
|
||||
|
||||
Append to `tests/wf_mcp/service/test_content_access.py`:
|
||||
|
||||
```python
|
||||
def test_content_access_reads_upstream_resource_with_events() -> None:
|
||||
service = WfMcpService(
|
||||
store=FileStore(local_temp_root() / "content_upstream_resource")
|
||||
)
|
||||
service.register_connection(
|
||||
ConnectionConfig(id="demo.personal", server="demo", account="personal")
|
||||
)
|
||||
service.register_adapter("demo", FakeAdapter())
|
||||
asyncio.run(service.refresh_connection_catalog("demo.personal"))
|
||||
|
||||
result = asyncio.run(
|
||||
service.content_access.read_resource("demo.personal.resource.welcome")
|
||||
)
|
||||
|
||||
assert result["contents"][0]["text"] == "Welcome from the fake adapter resource."
|
||||
event_kinds = [e.kind for e in service.list_events()]
|
||||
assert "resource_read_started" in event_kinds
|
||||
assert "resource_read_completed" in event_kinds
|
||||
|
||||
|
||||
def test_content_access_renders_upstream_prompt_with_events() -> None:
|
||||
service = WfMcpService(
|
||||
store=FileStore(local_temp_root() / "content_upstream_prompt")
|
||||
)
|
||||
service.register_connection(
|
||||
ConnectionConfig(id="demo.personal", server="demo", account="personal")
|
||||
)
|
||||
service.register_adapter("demo", FakeAdapter())
|
||||
asyncio.run(service.refresh_connection_catalog("demo.personal"))
|
||||
|
||||
result = asyncio.run(
|
||||
service.content_access.render_prompt(
|
||||
"demo.personal.prompt.summarize",
|
||||
arguments={"text": "hello world"},
|
||||
)
|
||||
)
|
||||
|
||||
assert "hello world" in result["messages"][0]["content"]["text"]
|
||||
event_kinds = [e.kind for e in service.list_events()]
|
||||
assert "prompt_get_started" in event_kinds
|
||||
assert "prompt_get_completed" in event_kinds
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests and verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_reads_upstream_resource_with_events tests/wf_mcp/service/test_content_access.py::test_content_access_renders_upstream_prompt_with_events -q
|
||||
```
|
||||
|
||||
Expected: fail because `service.content_access` does not exist on `WfMcpService`.
|
||||
|
||||
- [ ] **Step 3: Wire ContentAccessService into WfMcpService**
|
||||
|
||||
In `src/wf_mcp/broker/service/core.py`, import:
|
||||
|
||||
```python
|
||||
from .content_access import ContentAccessService
|
||||
```
|
||||
|
||||
Add field:
|
||||
|
||||
```python
|
||||
content_access: ContentAccessService = field(init=False)
|
||||
```
|
||||
|
||||
In `__post_init__`, after constructing `upstream`, `source_catalog`, `connection_service`, and `events`:
|
||||
|
||||
```python
|
||||
self.content_access = ContentAccessService(
|
||||
source_catalog=self.source_catalog,
|
||||
upstream=self.upstream,
|
||||
connection_service=self.connection_service,
|
||||
event_sink=self.events.record_event,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_reads_upstream_resource_with_events tests/wf_mcp/service/test_content_access.py::test_content_access_renders_upstream_prompt_with_events -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/content_access.py tests/wf_mcp/service/test_content_access.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Delegate WfMcpService.read_resource and render_prompt
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Test: `tests/wf_mcp/service/test_events.py`
|
||||
- Test: `tests/wf_mcp/service/test_content_access.py`
|
||||
|
||||
- [ ] **Step 1: Replace read_resource body with delegate**
|
||||
|
||||
In `src/wf_mcp/broker/service/core.py`, replace:
|
||||
|
||||
```python
|
||||
async def read_resource(self, qualified_name: str) -> dict[str, Any]:
|
||||
local_resource = self.source_catalog.local_documentation_resource(
|
||||
qualified_name
|
||||
)
|
||||
if local_resource is not None:
|
||||
self._record_event(
|
||||
make_event(
|
||||
"resource_read_completed",
|
||||
capability_id=qualified_name,
|
||||
payload={"uri": local_resource.uri, "source": "local"},
|
||||
)
|
||||
)
|
||||
return {
|
||||
"contents": [
|
||||
{
|
||||
"uri": local_resource.uri,
|
||||
"mimeType": local_resource.mime_type,
|
||||
"text": local_resource.text,
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
resource = self.get_resource(qualified_name)
|
||||
connection = self.connections.get(resource.connection_id)
|
||||
return await self.upstream.read_resource(
|
||||
connection,
|
||||
qualified_name,
|
||||
resource.uri,
|
||||
)
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
async def read_resource(self, qualified_name: str) -> dict[str, Any]:
|
||||
return await self.content_access.read_resource(qualified_name)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Replace render_prompt body with delegate**
|
||||
|
||||
Replace:
|
||||
|
||||
```python
|
||||
async def render_prompt(
|
||||
self,
|
||||
qualified_name: str,
|
||||
*,
|
||||
arguments: dict[str, str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
local_prompt = self.source_catalog.local_documentation_prompt(qualified_name)
|
||||
if local_prompt is not None:
|
||||
self._record_event(
|
||||
make_event(
|
||||
"prompt_get_completed",
|
||||
capability_id=qualified_name,
|
||||
payload={
|
||||
"argument_keys": sorted((arguments or {}).keys()),
|
||||
"source": "local",
|
||||
},
|
||||
)
|
||||
)
|
||||
return {
|
||||
"description": local_prompt.description,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": {
|
||||
"type": "text",
|
||||
"text": local_prompt.text,
|
||||
},
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
prompt = self.get_prompt(qualified_name)
|
||||
connection = self.connections.get(prompt.connection_id)
|
||||
return await self.upstream.render_prompt(
|
||||
connection,
|
||||
qualified_name,
|
||||
prompt.local_name,
|
||||
arguments,
|
||||
)
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
async def render_prompt(
|
||||
self,
|
||||
qualified_name: str,
|
||||
*,
|
||||
arguments: dict[str, str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
return await self.content_access.render_prompt(
|
||||
qualified_name,
|
||||
arguments=arguments,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the existing event/proxy tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_events.py::test_service_can_proxy_resource_reads_and_prompt_gets -q
|
||||
```
|
||||
|
||||
Expected: pass. This proves the delegate preserves existing behavior.
|
||||
|
||||
- [ ] **Step 4: Run full content_access tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_content_access.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/core.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Add Local Prompt Test and Edge-Case Coverage
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_mcp/service/test_content_access.py`
|
||||
|
||||
- [ ] **Step 1: Add local prompt test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_content_access_renders_local_documentation_prompt() -> None:
|
||||
content_access, events = _make_content_access(
|
||||
store_root="content_access_local_prompt"
|
||||
)
|
||||
|
||||
result = asyncio.run(
|
||||
content_access.render_prompt("test.docs.guide")
|
||||
)
|
||||
|
||||
assert result["description"] == "Test documentation prompt."
|
||||
assert result["messages"][0]["role"] == "user"
|
||||
assert result["messages"][0]["content"]["text"] == "Use the test docs."
|
||||
assert "prompt_get_completed" in [e.kind for e in events.list_events()]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the local prompt test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_renders_local_documentation_prompt -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 3: Add missing-resource error test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_content_access_raises_on_unknown_resource() -> None:
|
||||
content_access, _ = _make_content_access(
|
||||
store_root="content_access_missing_resource"
|
||||
)
|
||||
|
||||
with pytest.raises(KeyError):
|
||||
asyncio.run(content_access.read_resource("nonexistent.resource"))
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the error test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_content_access.py::test_content_access_raises_on_unknown_resource -q
|
||||
```
|
||||
|
||||
Expected: pass (KeyError from catalog lookup).
|
||||
|
||||
- [ ] **Step 5: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check tests/wf_mcp/service/test_content_access.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Clean Unused Imports and Verify
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Modify: `docs/superpowers/research/2026-06-02-wfmcpservice-remaining-responsibilities.md` if stale.
|
||||
|
||||
- [ ] **Step 1: Remove stale imports from core.py**
|
||||
|
||||
After the delegate change, `core.py` may no longer use `make_event` directly (verify with grep). If `make_event` is still used by `_record_catalog_change_events` or other methods, keep it. Only remove imports that are truly unused after the extraction.
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "make_event" src/wf_mcp/broker/service/core.py
|
||||
```
|
||||
|
||||
If only the removed `read_resource`/`render_prompt` bodies used `make_event`, remove the import. Otherwise keep it.
|
||||
|
||||
- [ ] **Step 2: Update research doc if stale**
|
||||
|
||||
If `docs/superpowers/research/2026-06-02-wfmcpservice-remaining-responsibilities.md` still lists `read_resource` and `render_prompt` as "Mixed real responsibility", update their classification:
|
||||
|
||||
```markdown
|
||||
| `read_resource` | Delegates to `ContentAccessService` | Compatibility facade | Keep for admin/MCP callers; new code should call `content_access`. |
|
||||
| `render_prompt` | Delegates to `ContentAccessService` | Compatibility facade | Keep for admin/MCP callers; new code should call `content_access`. |
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run focused verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_content_access.py tests/wf_mcp/service/test_events.py -q
|
||||
```
|
||||
|
||||
Expected: all selected tests pass.
|
||||
|
||||
- [ ] **Step 4: Run full verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
uv run ruff check src/wf_mcp/broker/service tests/wf_mcp/service
|
||||
uv run ruff format --check src/wf_mcp/broker/service tests/wf_mcp/service
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- pytest passes.
|
||||
- ruff check passes.
|
||||
- ruff format check passes.
|
||||
- basedpyright reports `0 errors`. If the known workspace enumeration warning causes a nonzero exit despite `0 errors`, record the exact output.
|
||||
|
||||
---
|
||||
|
||||
## Non-Goals and Follow-Up Slices
|
||||
|
||||
This plan intentionally leaves these for later:
|
||||
|
||||
1. **`invoke_method`/`send_notification`:** Already clean delegates to `upstream`. No local-docs branching. Moving them would add coupling without reducing complexity.
|
||||
2. **`refresh_connection_catalog`:** Different semantics (catalog refresh, not content access). Stays on `WfMcpService` / `UpstreamTransportService`.
|
||||
3. **Connection registry extraction:** Already done in `ConnectionService`; do not change it in this slice.
|
||||
4. **Event recorder extraction:** Already done in `BrokerEventRecorder`; do not change it in this slice.
|
||||
5. **`workflow_artifact_catalog_entry`:** Not broker content access; handle in a separate small cleanup.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: The plan extracts the local-docs-aware resource/prompt branching logic while preserving current public `WfMcpService` methods as one-line delegates. Upstream event semantics are unchanged.
|
||||
- Placeholder scan: No placeholder implementation tasks are left. Local docs tests use deterministic test-only resource and prompt keys.
|
||||
- Type consistency: `ContentAccessService` depends on `SourceCatalogService`, `UpstreamTransportService`, `ConnectionService`, and an `EventSink` callback. `WfMcpService.content_access` is exposed as a public field so callers (admin handlers, tests) can use it directly.
|
||||
@@ -0,0 +1,891 @@
|
||||
# WfMcpService Runtime Extraction Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Extract workflow compile/prepare/run/resume responsibilities from `WfMcpService` into a focused runtime implementation service while preserving existing public service methods and MCP/CLI behavior.
|
||||
|
||||
**Architecture:** Add `WorkflowRuntimeService` under `wf_mcp.broker.service`. It depends on `SourceCatalogService`, an optional artifact store, and an event emitter. `WfMcpService` remains the broker coordinator and compatibility façade; its runtime methods become thin delegates. This follows the previous `SourceCatalogService` extraction and keeps transport/auth/catalog refresh responsibilities out of this slice.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, `wf_core` runtime APIs, `wf_api.runtime_dependencies`, `wf_api.saved_subgraphs`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
Move now:
|
||||
|
||||
- `compile_plan`.
|
||||
- `_prepare_workflow_runtime`, renamed to `prepare_workflow_runtime` on the new service.
|
||||
- `run_workflow_from_plan`.
|
||||
- `resume_workflow_from_plan`.
|
||||
- Runtime event emission for `workflow_run_started`, `workflow_run_completed`, and `workflow_run_resumed`.
|
||||
|
||||
Keep now:
|
||||
|
||||
- Existing `WfMcpService.compile_plan`, `run_workflow_from_plan`, and `resume_workflow_from_plan` public method names as delegates.
|
||||
- Existing source/catalog behavior in `SourceCatalogService`.
|
||||
- Connection/adapters/auth/upstream I/O on `WfMcpService`.
|
||||
- Catalog refresh on `WfMcpService`.
|
||||
- Resource/prompt/raw method calls on `WfMcpService`.
|
||||
- Event bus implementation on `WfMcpService`.
|
||||
|
||||
Do not do in this slice:
|
||||
|
||||
- Do not introduce a protocol-neutral runtime service in `wf_api`.
|
||||
- Do not move `WorkflowOperationContext` itself.
|
||||
- Do not change MCP tool schemas, CLI commands, run payload shape, or saved-run lifecycle models.
|
||||
- Do not rename `WfMcpService`.
|
||||
|
||||
---
|
||||
|
||||
## Target File Structure
|
||||
|
||||
- Create `src/wf_mcp/broker/service/workflow_runtime.py`
|
||||
- Owns runtime compile/prepare/run/resume.
|
||||
- Has docstrings explaining that durable resume currently rebuilds dependencies from current in-memory service state.
|
||||
- Depends on `SourceCatalogService`, optional `WorkflowArtifactStore`, and event emitter callback.
|
||||
|
||||
- Modify `src/wf_mcp/broker/service/core.py`
|
||||
- Add `workflow_runtime: WorkflowRuntimeService = field(init=False)`.
|
||||
- Construct it in `__post_init__` after `source_catalog`.
|
||||
- Keep existing runtime methods as delegates.
|
||||
- Remove runtime-only imports after the move.
|
||||
|
||||
- Modify `src/wf_mcp/broker/service/workflow_operation_context.py`
|
||||
- `WfMcpWorkflowRuntimeRunner` should call `service.workflow_runtime` directly.
|
||||
|
||||
- Add direct runtime tests in `tests/wf_mcp/service/test_workflow_runtime.py`
|
||||
- Component-level compile/run tests.
|
||||
- Compatibility tests that `WfMcpService` still delegates and emits the same events.
|
||||
|
||||
- Update docs:
|
||||
- `docs/current_roadmap.md`
|
||||
- `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` if stale.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add WorkflowRuntimeService Skeleton and Compile Test
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_mcp/broker/service/workflow_runtime.py`
|
||||
- Create: `tests/wf_mcp/service/test_workflow_runtime.py`
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
|
||||
- [ ] **Step 1: Write the direct compile test**
|
||||
|
||||
Create `tests/wf_mcp/service/test_workflow_runtime.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_core import NodeUse
|
||||
from wf_mcp.broker.service.source_catalog import SourceCatalogService
|
||||
from wf_mcp.broker.service.workflow_runtime import WorkflowRuntimeService
|
||||
from wf_mcp.models import ConnectionConfig
|
||||
from wf_mcp.storage import FileStore
|
||||
from wf_platform import CapabilityBuckets, CapabilitySource, SourceVisibility
|
||||
|
||||
from ..test_support import echo_tool, local_temp_root
|
||||
from .conftest import single_echo_plan
|
||||
|
||||
|
||||
def _unused_tool_executor(connection: ConnectionConfig):
|
||||
raise AssertionError("tool executor should not be used by direct compile tests")
|
||||
|
||||
|
||||
def _source_catalog() -> SourceCatalogService:
|
||||
connection = ConnectionConfig(
|
||||
id="demo.personal",
|
||||
server="demo",
|
||||
account="personal",
|
||||
)
|
||||
catalog = SourceCatalogService(
|
||||
store=FileStore(local_temp_root() / "runtime_source_catalog"),
|
||||
connection_lookup=lambda connection_id: connection,
|
||||
connection_list_enabled=lambda: [connection],
|
||||
connection_list_all=lambda: [connection],
|
||||
tool_executor_for=_unused_tool_executor,
|
||||
load_auth=lambda connection_id: None,
|
||||
emit_event=lambda event: None,
|
||||
)
|
||||
catalog.register_capability_source(
|
||||
CapabilitySource(
|
||||
id="demo.personal",
|
||||
kind="connection",
|
||||
capabilities=CapabilityBuckets(
|
||||
node_specs={"demo.personal.echo_tool": echo_tool}
|
||||
),
|
||||
visibility=SourceVisibility(planner=True),
|
||||
)
|
||||
)
|
||||
return catalog
|
||||
|
||||
|
||||
def test_workflow_runtime_service_compiles_plan_directly() -> None:
|
||||
runtime = WorkflowRuntimeService(
|
||||
source_catalog=_source_catalog(),
|
||||
artifact_store=None,
|
||||
emit_event=lambda event: None,
|
||||
)
|
||||
|
||||
workflow = runtime.compile_plan(
|
||||
single_echo_plan("runtime_compile", "demo.echo_tool"),
|
||||
{"demo.echo_tool": "demo.personal.echo_tool"},
|
||||
)
|
||||
|
||||
node = workflow.nodes[0]
|
||||
assert isinstance(node, NodeUse)
|
||||
assert node.node == "demo.personal.echo_tool"
|
||||
assert "demo.personal.echo_tool" in workflow.node_defs
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the compile test and verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_compiles_plan_directly -q
|
||||
```
|
||||
|
||||
Expected: import failure because `wf_mcp.broker.service.workflow_runtime` does not exist.
|
||||
|
||||
- [ ] **Step 3: Create WorkflowRuntimeService with compile_plan**
|
||||
|
||||
Create `src/wf_mcp/broker/service/workflow_runtime.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from wf_artifacts import WorkflowArtifactStore
|
||||
from wf_authoring import NodeSpec
|
||||
from wf_core import NodeUse, Workflow
|
||||
from wf_api.models import RawWorkflowPlan
|
||||
|
||||
from ...events import McpEvent
|
||||
from .source_catalog import SourceCatalogService
|
||||
|
||||
EventEmitter = Callable[[McpEvent], None]
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class WorkflowRuntimeService:
|
||||
"""Compile and execute workflow plans against broker-owned runtime deps.
|
||||
|
||||
This service is still an MCP broker implementation detail. It receives
|
||||
source/catalog state from `SourceCatalogService`, but it does not own
|
||||
connections, adapters, auth, or upstream discovery.
|
||||
"""
|
||||
|
||||
source_catalog: SourceCatalogService
|
||||
artifact_store: WorkflowArtifactStore | None
|
||||
emit_event: EventEmitter
|
||||
|
||||
def compile_plan(
|
||||
self,
|
||||
plan: RawWorkflowPlan,
|
||||
node_name_bindings: dict[str, str] | None = None,
|
||||
) -> Workflow:
|
||||
node_defs: dict[str, Any] = {}
|
||||
bindings = node_name_bindings or {}
|
||||
for step in plan.nodes:
|
||||
if not isinstance(step, NodeUse):
|
||||
continue
|
||||
qualified_name = bindings.get(step.node, step.node)
|
||||
spec: NodeSpec[Any, Any] = self.source_catalog.get_qualified_spec(
|
||||
qualified_name
|
||||
)
|
||||
node_defs[qualified_name] = spec.to_node_def()
|
||||
|
||||
nodes = []
|
||||
for node in plan.nodes:
|
||||
payload = node.model_dump(by_alias=True)
|
||||
if isinstance(node, NodeUse):
|
||||
payload["node"] = bindings.get(node.node, node.node)
|
||||
nodes.append(payload)
|
||||
|
||||
payload = {
|
||||
"name": plan.name,
|
||||
"input_schema": plan.input_schema,
|
||||
"state_schema": plan.state_schema,
|
||||
"output_schema": plan.output_schema,
|
||||
"output": [binding.model_dump(mode="json") for binding in plan.output],
|
||||
"outcomes": plan.outcomes,
|
||||
"start": plan.start,
|
||||
"node_defs": [node.model_dump() for node in node_defs.values()],
|
||||
"nodes": nodes,
|
||||
"edges": [edge.model_dump(by_alias=True) for edge in plan.edges],
|
||||
}
|
||||
return Workflow.model_validate(payload)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the compile test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_compiles_plan_directly -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/workflow_runtime.py tests/wf_mcp/service/test_workflow_runtime.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Wire Runtime Service Into WfMcpService as a Delegate
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Test: `tests/wf_mcp/service/test_workflow_runtime.py`
|
||||
|
||||
- [ ] **Step 1: Add compatibility identity and delegate tests**
|
||||
|
||||
Append to `tests/wf_mcp/service/test_workflow_runtime.py`:
|
||||
|
||||
```python
|
||||
from wf_mcp.broker import WfMcpService
|
||||
|
||||
|
||||
def test_wfmcpservice_constructs_workflow_runtime_with_source_catalog() -> None:
|
||||
service = WfMcpService(store=FileStore(local_temp_root() / "runtime_delegate"))
|
||||
|
||||
assert service.workflow_runtime.source_catalog is service.source_catalog
|
||||
assert service.workflow_runtime.artifact_store is service.artifact_store
|
||||
|
||||
|
||||
def test_wfmcpservice_compile_plan_delegates_to_workflow_runtime() -> None:
|
||||
service = WfMcpService(store=FileStore(local_temp_root() / "runtime_compile_delegate"))
|
||||
service.register_connection(
|
||||
ConnectionConfig(id="demo.personal", server="demo", account="personal")
|
||||
)
|
||||
service.register_specs("demo.personal", echo_tool)
|
||||
|
||||
workflow = service.compile_plan(
|
||||
single_echo_plan("runtime_delegate_compile", "demo.echo_tool"),
|
||||
{"demo.echo_tool": "demo.personal.echo_tool"},
|
||||
)
|
||||
|
||||
assert "demo.personal.echo_tool" in workflow.node_defs
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the new tests and verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_wfmcpservice_constructs_workflow_runtime_with_source_catalog tests/wf_mcp/service/test_workflow_runtime.py::test_wfmcpservice_compile_plan_delegates_to_workflow_runtime -q
|
||||
```
|
||||
|
||||
Expected: first test fails because `workflow_runtime` does not exist.
|
||||
|
||||
- [ ] **Step 3: Construct workflow_runtime in WfMcpService**
|
||||
|
||||
In `src/wf_mcp/broker/service/core.py`, import:
|
||||
|
||||
```python
|
||||
from .workflow_runtime import WorkflowRuntimeService
|
||||
```
|
||||
|
||||
Add the dataclass field:
|
||||
|
||||
```python
|
||||
workflow_runtime: WorkflowRuntimeService = field(init=False)
|
||||
```
|
||||
|
||||
In `__post_init__`, after `self.source_catalog = SourceCatalogService(...)`, add:
|
||||
|
||||
```python
|
||||
self.workflow_runtime = WorkflowRuntimeService(
|
||||
source_catalog=self.source_catalog,
|
||||
artifact_store=self.artifact_store,
|
||||
emit_event=self._record_event,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Delegate compile_plan**
|
||||
|
||||
Replace `WfMcpService.compile_plan` body with:
|
||||
|
||||
```python
|
||||
return self.workflow_runtime.compile_plan(plan, node_name_bindings)
|
||||
```
|
||||
|
||||
Keep the method signature unchanged.
|
||||
|
||||
- [ ] **Step 5: Run delegate tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_wfmcpservice_constructs_workflow_runtime_with_source_catalog tests/wf_mcp/service/test_workflow_runtime.py::test_wfmcpservice_compile_plan_delegates_to_workflow_runtime -q
|
||||
```
|
||||
|
||||
Expected: both pass.
|
||||
|
||||
- [ ] **Step 6: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/workflow_runtime.py tests/wf_mcp/service/test_workflow_runtime.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Move Runtime Preparation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/workflow_runtime.py`
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Test: `tests/wf_mcp/service/test_workflow_runtime.py`
|
||||
|
||||
- [ ] **Step 1: Add a direct preparation test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
def test_workflow_runtime_service_prepares_node_registry_and_reducers() -> None:
|
||||
runtime = WorkflowRuntimeService(
|
||||
source_catalog=_source_catalog(),
|
||||
artifact_store=None,
|
||||
emit_event=lambda event: None,
|
||||
)
|
||||
|
||||
workflow, registry, reducers, prepared_subgraphs = runtime.prepare_workflow_runtime(
|
||||
single_echo_plan("runtime_prepare", "demo.echo_tool"),
|
||||
deployment=None,
|
||||
artifact=None,
|
||||
)
|
||||
|
||||
assert "demo.personal.echo_tool" in workflow.node_defs
|
||||
assert "demo.personal.echo_tool" in registry
|
||||
assert isinstance(reducers, dict)
|
||||
assert prepared_subgraphs == {}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the preparation test and verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_prepares_node_registry_and_reducers -q
|
||||
```
|
||||
|
||||
Expected: fail because `prepare_workflow_runtime` does not exist.
|
||||
|
||||
- [ ] **Step 3: Move _prepare_workflow_runtime into WorkflowRuntimeService**
|
||||
|
||||
In `src/wf_mcp/broker/service/workflow_runtime.py`, add imports:
|
||||
|
||||
```python
|
||||
from wf_artifacts import WorkflowArtifact, WorkflowDeployment
|
||||
from wf_api.runtime_dependencies import resolve_runtime_dependencies
|
||||
from wf_api.saved_subgraphs import (
|
||||
SavedSubgraphTree,
|
||||
prepare_saved_subgraphs,
|
||||
resolve_saved_subgraph_tree,
|
||||
)
|
||||
```
|
||||
|
||||
Add the method:
|
||||
|
||||
```python
|
||||
def prepare_workflow_runtime(
|
||||
self,
|
||||
plan: RawWorkflowPlan,
|
||||
*,
|
||||
deployment: WorkflowDeployment | None,
|
||||
artifact: WorkflowArtifact | None,
|
||||
saved_subgraph_tree: SavedSubgraphTree | None = None,
|
||||
) -> tuple[Workflow, dict[str, Any], dict[str, Any], dict[str, Any]]:
|
||||
"""Resolve bindings once into the executable pieces core expects.
|
||||
|
||||
Saved-run resume still rebuilds prepared dependencies from the current
|
||||
in-memory broker state. Durable resume will need a stricter snapshot,
|
||||
but this keeps the current platform boundary explicit.
|
||||
"""
|
||||
plan_node_names = [
|
||||
node.node for node in plan.nodes if isinstance(node, NodeUse)
|
||||
]
|
||||
runtime_artifact = artifact or WorkflowArtifact(
|
||||
id=plan.name,
|
||||
version=1,
|
||||
title=plan.name,
|
||||
input_schema=plan.input_schema,
|
||||
output_schema=plan.output_schema,
|
||||
outcomes=("completed",),
|
||||
plan=plan.model_dump(mode="json", by_alias=True),
|
||||
)
|
||||
dependencies = resolve_runtime_dependencies(
|
||||
artifact=runtime_artifact,
|
||||
deployment=deployment,
|
||||
sources=self.source_catalog.capability_sources,
|
||||
plan_node_names=plan_node_names,
|
||||
)
|
||||
prepared_subgraphs = {}
|
||||
if saved_subgraph_tree is not None:
|
||||
prepared_subgraphs = prepare_saved_subgraphs(
|
||||
tree=saved_subgraph_tree,
|
||||
deployment=deployment,
|
||||
sources=self.source_catalog.capability_sources,
|
||||
compile_plan=self.compile_plan,
|
||||
)
|
||||
elif artifact is not None and self.artifact_store is not None:
|
||||
tree = resolve_saved_subgraph_tree(
|
||||
root_artifact=artifact,
|
||||
artifact_store=self.artifact_store,
|
||||
)
|
||||
prepared_subgraphs = prepare_saved_subgraphs(
|
||||
tree=tree,
|
||||
deployment=deployment,
|
||||
sources=self.source_catalog.capability_sources,
|
||||
compile_plan=self.compile_plan,
|
||||
)
|
||||
workflow = self.compile_plan(plan, dependencies.node_name_bindings)
|
||||
return (
|
||||
workflow,
|
||||
dependencies.node_registry,
|
||||
dependencies.reducers,
|
||||
prepared_subgraphs,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Delegate _prepare_workflow_runtime**
|
||||
|
||||
In `src/wf_mcp/broker/service/core.py`, replace `_prepare_workflow_runtime` body with:
|
||||
|
||||
```python
|
||||
return self.workflow_runtime.prepare_workflow_runtime(
|
||||
plan,
|
||||
deployment=deployment,
|
||||
artifact=artifact,
|
||||
saved_subgraph_tree=saved_subgraph_tree,
|
||||
)
|
||||
```
|
||||
|
||||
Keep the private method signature unchanged for compatibility with any tests or internal callers.
|
||||
|
||||
- [ ] **Step 5: Run preparation and hydrated runtime regression tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_prepares_node_registry_and_reducers tests/wf_mcp/service/test_catalog.py::test_service_hydrates_planner_specs_from_stored_catalog -q
|
||||
```
|
||||
|
||||
Expected: both pass.
|
||||
|
||||
- [ ] **Step 6: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/workflow_runtime.py tests/wf_mcp/service/test_workflow_runtime.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Move Run and Resume Execution
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/workflow_runtime.py`
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Test: `tests/wf_mcp/service/test_workflow_runtime.py`
|
||||
- Test: `tests/wf_api/test_run_api.py`
|
||||
|
||||
- [ ] **Step 1: Add a direct run test with event assertions**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
|
||||
|
||||
def test_workflow_runtime_service_runs_plan_and_emits_events() -> None:
|
||||
events = []
|
||||
runtime = WorkflowRuntimeService(
|
||||
source_catalog=_source_catalog(),
|
||||
artifact_store=None,
|
||||
emit_event=events.append,
|
||||
)
|
||||
|
||||
run = asyncio.run(
|
||||
runtime.run_workflow_from_plan(
|
||||
single_echo_plan("runtime_run", "demo.echo_tool"),
|
||||
{"text": "hello"},
|
||||
)
|
||||
)
|
||||
|
||||
assert run.output["echoed"] == "hello"
|
||||
assert [event.type for event in events] == [
|
||||
"workflow_run_started",
|
||||
"workflow_run_completed",
|
||||
]
|
||||
assert events[1].payload["status"] == "completed"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the direct run test and verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_runs_plan_and_emits_events -q
|
||||
```
|
||||
|
||||
Expected: fail because `WorkflowRuntimeService.run_workflow_from_plan` does not exist.
|
||||
|
||||
- [ ] **Step 3: Add run and resume methods to WorkflowRuntimeService**
|
||||
|
||||
In `src/wf_mcp/broker/service/workflow_runtime.py`, add imports:
|
||||
|
||||
```python
|
||||
from wf_core import (
|
||||
RunState,
|
||||
execute_workflow_result_async,
|
||||
resume_workflow_result_async,
|
||||
)
|
||||
from ...events import make_event
|
||||
```
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
async def run_workflow_from_plan(
|
||||
self,
|
||||
plan: RawWorkflowPlan,
|
||||
workflow_input: dict[str, Any],
|
||||
deployment: WorkflowDeployment | None = None,
|
||||
artifact: WorkflowArtifact | None = None,
|
||||
saved_subgraph_tree: SavedSubgraphTree | None = None,
|
||||
) -> RunState:
|
||||
self.emit_event(
|
||||
make_event(
|
||||
"workflow_run_started",
|
||||
workflow_name=plan.name,
|
||||
payload={"input_keys": sorted(workflow_input.keys())},
|
||||
)
|
||||
)
|
||||
workflow, registry, reducers, prepared_subgraphs = (
|
||||
self.prepare_workflow_runtime(
|
||||
plan,
|
||||
deployment=deployment,
|
||||
artifact=artifact,
|
||||
saved_subgraph_tree=saved_subgraph_tree,
|
||||
)
|
||||
)
|
||||
run = await execute_workflow_result_async(
|
||||
workflow,
|
||||
workflow_input,
|
||||
registry,
|
||||
reducers=reducers,
|
||||
subgraphs=prepared_subgraphs,
|
||||
)
|
||||
self.emit_event(
|
||||
make_event(
|
||||
"workflow_run_completed",
|
||||
workflow_name=plan.name,
|
||||
payload={"status": run.status.value},
|
||||
)
|
||||
)
|
||||
return run
|
||||
|
||||
async def resume_workflow_from_plan(
|
||||
self,
|
||||
plan: RawWorkflowPlan,
|
||||
run: RunState,
|
||||
*,
|
||||
resume_payload: dict[str, Any],
|
||||
resume_outcome: str = "submitted",
|
||||
deployment: WorkflowDeployment | None = None,
|
||||
artifact: WorkflowArtifact | None = None,
|
||||
saved_subgraph_tree: SavedSubgraphTree | None = None,
|
||||
) -> RunState:
|
||||
"""Resume one stopped run using its prepared runtime dependency boundary."""
|
||||
workflow, registry, reducers, prepared_subgraphs = (
|
||||
self.prepare_workflow_runtime(
|
||||
plan,
|
||||
deployment=deployment,
|
||||
artifact=artifact,
|
||||
saved_subgraph_tree=saved_subgraph_tree,
|
||||
)
|
||||
)
|
||||
resumed = await resume_workflow_result_async(
|
||||
workflow,
|
||||
run,
|
||||
registry,
|
||||
resume_payload=resume_payload,
|
||||
resume_outcome=resume_outcome,
|
||||
reducers=reducers,
|
||||
subgraphs=prepared_subgraphs,
|
||||
)
|
||||
self.emit_event(
|
||||
make_event(
|
||||
"workflow_run_resumed",
|
||||
workflow_name=plan.name,
|
||||
payload={"status": resumed.status.value},
|
||||
)
|
||||
)
|
||||
return resumed
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Delegate WfMcpService run/resume**
|
||||
|
||||
Replace `WfMcpService.run_workflow_from_plan` body with:
|
||||
|
||||
```python
|
||||
return await self.workflow_runtime.run_workflow_from_plan(
|
||||
plan,
|
||||
workflow_input,
|
||||
deployment=deployment,
|
||||
artifact=artifact,
|
||||
saved_subgraph_tree=saved_subgraph_tree,
|
||||
)
|
||||
```
|
||||
|
||||
Replace `WfMcpService.resume_workflow_from_plan` body with:
|
||||
|
||||
```python
|
||||
return await self.workflow_runtime.resume_workflow_from_plan(
|
||||
plan,
|
||||
run,
|
||||
resume_payload=resume_payload,
|
||||
resume_outcome=resume_outcome,
|
||||
deployment=deployment,
|
||||
artifact=artifact,
|
||||
saved_subgraph_tree=saved_subgraph_tree,
|
||||
)
|
||||
```
|
||||
|
||||
Keep public signatures unchanged.
|
||||
|
||||
- [ ] **Step 5: Run direct and API run tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py::test_workflow_runtime_service_runs_plan_and_emits_events tests/wf_api/test_run_api.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 6: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/core.py src/wf_mcp/broker/service/workflow_runtime.py tests/wf_mcp/service/test_workflow_runtime.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Point WorkflowOperationContext Runtime Adapter at workflow_runtime
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/workflow_operation_context.py`
|
||||
- Test: `tests/wf_api/test_operation_context.py`
|
||||
- Test: `tests/wf_api/test_run_api.py`
|
||||
|
||||
- [ ] **Step 1: Add an adapter identity test**
|
||||
|
||||
In `tests/wf_api/test_operation_context.py`, add:
|
||||
|
||||
```python
|
||||
def test_context_runtime_runner_uses_workflow_runtime_service() -> None:
|
||||
service = WfMcpService(store=FileStore(local_temp_root() / "context_runtime"))
|
||||
context = context_from_service(service)
|
||||
|
||||
assert getattr(context.runtime, "runtime") is service.workflow_runtime
|
||||
```
|
||||
|
||||
If this file does not have `local_temp_root`, use the same temp-store helper style already used by its neighboring tests.
|
||||
|
||||
- [ ] **Step 2: Run the adapter test and verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_operation_context.py::test_context_runtime_runner_uses_workflow_runtime_service -q
|
||||
```
|
||||
|
||||
Expected: fail because `WfMcpWorkflowRuntimeRunner` stores `service`, not `runtime`.
|
||||
|
||||
- [ ] **Step 3: Update runtime adapter**
|
||||
|
||||
In `src/wf_mcp/broker/service/workflow_operation_context.py`, import:
|
||||
|
||||
```python
|
||||
from .workflow_runtime import WorkflowRuntimeService
|
||||
```
|
||||
|
||||
Change:
|
||||
|
||||
```python
|
||||
class WfMcpWorkflowRuntimeRunner(WorkflowRuntimeRunner):
|
||||
"""Adapter-owned runtime runner backed by WfMcpService."""
|
||||
|
||||
service: WfMcpService
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
class WfMcpWorkflowRuntimeRunner(WorkflowRuntimeRunner):
|
||||
"""Adapter-owned runtime runner backed by WorkflowRuntimeService."""
|
||||
|
||||
runtime: WorkflowRuntimeService
|
||||
```
|
||||
|
||||
Replace calls from `self.service.run_workflow_from_plan(...)` and `self.service.resume_workflow_from_plan(...)` to `self.runtime.run_workflow_from_plan(...)` and `self.runtime.resume_workflow_from_plan(...)`.
|
||||
|
||||
In `context_from_service`, change:
|
||||
|
||||
```python
|
||||
runtime=WfMcpWorkflowRuntimeRunner(service),
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
runtime=WfMcpWorkflowRuntimeRunner(service.workflow_runtime),
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run context and run API tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_operation_context.py::test_context_runtime_runner_uses_workflow_runtime_service tests/wf_api/test_run_api.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 5: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/broker/service/workflow_operation_context.py tests/wf_api/test_operation_context.py
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Clean Imports, Docs, and Verify
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_mcp/broker/service/core.py`
|
||||
- Modify: `src/wf_mcp/broker/service/workflow_runtime.py`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
- Modify: `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` if stale.
|
||||
|
||||
- [ ] **Step 1: Remove stale runtime imports from core.py**
|
||||
|
||||
After the move, `src/wf_mcp/broker/service/core.py` should no longer import runtime-only names such as:
|
||||
|
||||
```python
|
||||
from wf_core import NodeUse, Workflow, execute_workflow_result_async, resume_workflow_result_async
|
||||
from wf_api.runtime_dependencies import resolve_runtime_dependencies
|
||||
from wf_api.saved_subgraphs import prepare_saved_subgraphs, resolve_saved_subgraph_tree
|
||||
```
|
||||
|
||||
Keep names still needed for type annotations, public signatures, or non-runtime service methods:
|
||||
|
||||
```python
|
||||
from wf_core import RunState
|
||||
from wf_api.models import RawWorkflowPlan
|
||||
from wf_api.saved_subgraphs import SavedSubgraphTree
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add roadmap note**
|
||||
|
||||
In `docs/current_roadmap.md`, under the wf_api/service extraction bullets, add:
|
||||
|
||||
```markdown
|
||||
- Workflow runtime execution is being separated from broker coordination.
|
||||
`WorkflowRuntimeService` now owns plan compilation, dependency preparation,
|
||||
run, and resume; `WfMcpService` keeps delegate methods for compatibility.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update extraction map if stale**
|
||||
|
||||
If `docs/superpowers/research/2026-06-01-wf-api-extraction-map.md` says `WfMcpService` directly owns workflow runtime execution, add or update a note:
|
||||
|
||||
```markdown
|
||||
Workflow runtime ownership is now split: `WorkflowRuntimeService` owns plan
|
||||
compilation, dependency preparation, run, and resume. `WfMcpService` remains the
|
||||
broker coordinator and compatibility façade.
|
||||
```
|
||||
|
||||
Do not edit the file if it already describes this state.
|
||||
|
||||
- [ ] **Step 4: Run focused verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_workflow_runtime.py tests/wf_mcp/service/test_catalog.py::test_service_hydrates_planner_specs_from_stored_catalog tests/wf_api/test_operation_context.py tests/wf_api/test_run_api.py tests/wf_mcp/workflow_surface/test_runs.py -q
|
||||
```
|
||||
|
||||
Expected: all selected tests pass.
|
||||
|
||||
- [ ] **Step 5: Run full verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
uv run ruff check src/wf_mcp/broker/service src/wf_api tests/wf_mcp/service tests/wf_api
|
||||
uv run ruff format --check src/wf_mcp/broker/service src/wf_api tests/wf_mcp/service tests/wf_api docs/current_roadmap.md
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- pytest passes.
|
||||
- ruff check passes.
|
||||
- ruff format check passes.
|
||||
- basedpyright reports `0 errors`. If the known workspace enumeration warning causes a nonzero exit despite `0 errors`, record the exact output.
|
||||
|
||||
---
|
||||
|
||||
## Non-Goals and Follow-Up Slices
|
||||
|
||||
This plan intentionally leaves these slices for later:
|
||||
|
||||
1. **Transport/upstream service extraction:** move connection lookup, adapter lookup, auth loading, resource reads, prompt rendering, raw method calls, and notifications.
|
||||
2. **Event recorder extraction:** turn `_record_event` and catalog change event emission into an injected event recorder implementation.
|
||||
3. **WfMcpService rename:** once most implementations are extracted, rename the remaining coordinator to a clearer broker runtime name if the public import impact is acceptable.
|
||||
4. **Protocol-neutral API expansion:** decide whether runtime execution belongs behind a `wf_api` implementation protocol once CLI/HTTP need a shared process boundary.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: The plan extracts compile/prepare/run/resume and preserves current public service methods, context adaptation, saved subgraph preparation, and run payload behavior.
|
||||
- Placeholder scan: No placeholders or vague “write tests” steps remain; each task has explicit code snippets and commands.
|
||||
- Type consistency: `WorkflowRuntimeService` receives `SourceCatalogService`, `WorkflowArtifactStore | None`, and `EventEmitter`; later tasks use the same names and signatures.
|
||||
+1146
File diff suppressed because it is too large
Load Diff
+1079
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,29 @@
|
||||
# Workflow Skill Docs Mirror Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add a repo-local `wf-workflow` skill with agent-facing references and expose those same skill files through the MCP documentation source.
|
||||
|
||||
**Architecture:** `docs/` remains project-facing documentation. `skills/wf-workflow/` becomes a self-contained agent-facing operating manual with distilled references. `src/wf_mcp/documentation.py` exposes the skill files as documentation resources under `wf://skills/...` without changing existing `wf://docs/...` resources.
|
||||
|
||||
**Tech Stack:** Markdown skills, `wf_mcp.documentation`, `wf_platform.DocumentationResource`, pytest.
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
- [ ] Add a failing docs resource test for `wf://skills/wf-workflow/SKILL.md` and one reference.
|
||||
- [ ] Create `skills/wf-workflow/SKILL.md`.
|
||||
- [ ] Create `skills/wf-workflow/references/workflow-lifecycle.md`.
|
||||
- [ ] Create `skills/wf-workflow/references/capabilities-and-wrappers.md`.
|
||||
- [ ] Create `skills/wf-workflow/references/draft-workspaces.md`.
|
||||
- [ ] Create `skills/wf-workflow/references/troubleshooting.md`.
|
||||
- [ ] Update `src/wf_mcp/documentation.py` to load the skill files as resources.
|
||||
- [ ] Verify focused docs tests and lint.
|
||||
|
||||
## Guardrails
|
||||
|
||||
- Do not replace the existing `skills/wf-cli` skill.
|
||||
- Do not make `SKILL.md` a link-only file.
|
||||
- Do not symlink skill references to `docs/`; checked-in distilled files are more portable.
|
||||
- Do not remove existing `wf://docs/...` resources.
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,519 @@
|
||||
# Persisted Run/Resume Hardening Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Lock down the persisted run/resume contract with missing regression tests and add a fail-fast required-store factory for durable API frontends.
|
||||
|
||||
**Architecture:** The current V1 run/resume behavior mostly exists in `wf_api.runs` and `wf_api.run_lifecycle`. This plan avoids redesigning `RunState`, retry, checkpoint cadence, or transactional storage. It adds tests for weakly covered contract rules, then introduces a protocol-neutral durable-context helper that future HTTP/API frontends can use before constructing `WorkflowApi`.
|
||||
|
||||
**Tech Stack:** Python 3.14, pytest, Pydantic v2, `wf_api`, `wf_artifacts`, `wf_core`, existing MCP-backed test fixtures.
|
||||
|
||||
---
|
||||
|
||||
## File Map
|
||||
|
||||
- Modify `tests/wf_api/test_run_api.py`
|
||||
- Add contract tests for rejected resume of non-interrupted stored runs.
|
||||
- Add contract test proving deleted deployments do not break stored run inspection.
|
||||
- Strengthen trace-range validation test to prove store lookup is not touched.
|
||||
- Create `src/wf_api/durable_context.py`
|
||||
- Add required-store validation for durable frontends.
|
||||
- Add `durable_workflow_api(context)` helper returning `WorkflowApi`.
|
||||
- Modify `src/wf_api/__init__.py`
|
||||
- Export the durable-context helpers.
|
||||
- Create `tests/wf_api/test_durable_context.py`
|
||||
- Cover success and missing-store failures.
|
||||
- Modify `docs/superpowers/specs/2026-06-03-persisted-run-resume-contract.md`
|
||||
- Mark these V1 hardening items as implemented after tests/helper land.
|
||||
- Modify `docs/current_roadmap.md`
|
||||
- Add a short note that durable required-store factory exists; transactional backend remains future work.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- No transactional run store.
|
||||
- No concurrent resume compare-and-swap.
|
||||
- No paged run listing/checkpoint listing.
|
||||
- No automatic retry/timeout policy.
|
||||
- No HTTP server routes.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add Missing Run Contract Tests
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/wf_api/test_run_api.py`
|
||||
|
||||
- [ ] **Step 1: Add imports for stored-run helpers**
|
||||
|
||||
At the top of `tests/wf_api/test_run_api.py`, extend imports:
|
||||
|
||||
```python
|
||||
from wf_artifacts import (
|
||||
FileWorkflowArtifactStore,
|
||||
FileRunStore,
|
||||
ResumeReadiness,
|
||||
WorkflowDeployment,
|
||||
)
|
||||
```
|
||||
|
||||
If `ResumeReadiness` is already imported later by another task, keep one import only.
|
||||
|
||||
- [ ] **Step 2: Add test for non-interrupted resume rejection**
|
||||
|
||||
Append this test after `test_run_api_completed_run_persists`:
|
||||
|
||||
```python
|
||||
def test_run_api_rejects_resume_for_completed_run() -> None:
|
||||
root = local_temp_root() / "run_api_resume_completed_rejected"
|
||||
service, _ = _service_with_echo(root)
|
||||
context = context_from_service(service)
|
||||
api = WorkflowRunApi(context)
|
||||
|
||||
result = asyncio.run(
|
||||
api.run_deployment(
|
||||
deployment_id="echo.personal",
|
||||
workflow_input={"text": "hello"},
|
||||
)
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="is not interrupted"):
|
||||
asyncio.run(
|
||||
api.resume_run(
|
||||
run_id=result["run_id"],
|
||||
resume_payload={"answer": "ignored"},
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
This locks down: `resume_run` only resumes stored runs whose latest status is `interrupted`.
|
||||
|
||||
- [ ] **Step 3: Add test that deleting deployment does not erase stored inspection**
|
||||
|
||||
Append this test after the non-interrupted resume test:
|
||||
|
||||
```python
|
||||
def test_run_api_inspect_uses_pinned_environment_after_deployment_deleted() -> None:
|
||||
root = local_temp_root() / "run_api_inspect_after_deployment_deleted"
|
||||
service, artifact_store = _service_with_echo(root)
|
||||
context = context_from_service(service)
|
||||
api = WorkflowRunApi(context)
|
||||
|
||||
result = asyncio.run(
|
||||
api.run_deployment(
|
||||
deployment_id="echo.personal",
|
||||
workflow_input={"text": "hello"},
|
||||
)
|
||||
)
|
||||
artifact_store.delete_deployment("echo.personal")
|
||||
|
||||
summary = asyncio.run(api.inspect_run(run_id=result["run_id"]))
|
||||
|
||||
assert summary["status"] == "completed"
|
||||
assert summary["run_id"] == result["run_id"]
|
||||
assert summary["deployment_id"] == "echo.personal"
|
||||
assert summary["artifact_id"] == "echo"
|
||||
assert summary["output"]["echoed"] == "hello"
|
||||
```
|
||||
|
||||
This locks down: stored run inspection reads pinned environment from `WorkflowRunRecord`, not mutable deployment storage.
|
||||
|
||||
- [ ] **Step 4: Strengthen trace-range-before-store validation**
|
||||
|
||||
In `test_run_api_rejects_invalid_trace_range_before_store_lookup`, replace the setup with a service whose run store raises if accessed:
|
||||
|
||||
```python
|
||||
class ExplodingRunStore(FileRunStore):
|
||||
def get_run(self, run_id: str):
|
||||
raise AssertionError("run store must not be read before trace_range validation")
|
||||
```
|
||||
|
||||
Then build the service explicitly:
|
||||
|
||||
```python
|
||||
root = local_temp_root() / "run_api_invalid_trace_range"
|
||||
service, _ = _service_with_echo(root)
|
||||
service.run_store = ExplodingRunStore(root / "exploding_runs")
|
||||
context = context_from_service(service)
|
||||
api = WorkflowRunApi(context)
|
||||
```
|
||||
|
||||
Keep the two existing `pytest.raises(ValueError, ...)` assertions unchanged. The test must still pass, proving invalid trace ranges fail before store lookup.
|
||||
|
||||
- [ ] **Step 5: Run focused run API tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_run_api.py -q
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
all tests in tests/wf_api/test_run_api.py pass
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit Task 1**
|
||||
|
||||
```bash
|
||||
git add tests/wf_api/test_run_api.py
|
||||
git commit -m "test: lock down durable run resume contract"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Add Required-Store Durable Context Helper
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/wf_api/durable_context.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
- Test: `tests/wf_api/test_durable_context.py`
|
||||
|
||||
- [ ] **Step 1: Write failing durable-context tests**
|
||||
|
||||
Create `tests/wf_api/test_durable_context.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from wf_api import WorkflowApi
|
||||
from wf_api.durable_context import durable_workflow_api, require_workflow_stores
|
||||
from wf_api.operation_context import WorkflowOperationContext
|
||||
from wf_api.stores import file_workflow_stores
|
||||
from wf_mcp.broker import WfMcpService
|
||||
from wf_mcp.broker.service.workflow_operation_context import context_from_service
|
||||
from wf_mcp.storage import FileStore
|
||||
|
||||
|
||||
def test_require_workflow_stores_returns_existing_store_bundle(tmp_path) -> None:
|
||||
stores = file_workflow_stores(tmp_path / "workflow_stores")
|
||||
service = WfMcpService(
|
||||
store=FileStore(tmp_path / "mcp"),
|
||||
artifact_store=stores.artifact_store,
|
||||
draft_workspace_store=stores.draft_workspace_store,
|
||||
run_store=stores.run_store,
|
||||
)
|
||||
context = context_from_service(service)
|
||||
|
||||
required = require_workflow_stores(context)
|
||||
|
||||
assert required.artifact_store is stores.artifact_store
|
||||
assert required.draft_workspace_store is stores.draft_workspace_store
|
||||
assert required.run_store is stores.run_store
|
||||
|
||||
|
||||
def test_require_workflow_stores_rejects_missing_store(tmp_path) -> None:
|
||||
service = WfMcpService(
|
||||
store=FileStore(tmp_path / "mcp"),
|
||||
artifact_store=None,
|
||||
draft_workspace_store=None,
|
||||
run_store=None,
|
||||
)
|
||||
context = context_from_service(service)
|
||||
|
||||
with pytest.raises(ValueError, match="durable workflow API requires stores"):
|
||||
require_workflow_stores(context)
|
||||
|
||||
|
||||
def test_durable_workflow_api_returns_workflow_api_with_same_context(tmp_path) -> None:
|
||||
stores = file_workflow_stores(tmp_path / "workflow_stores")
|
||||
service = WfMcpService(
|
||||
store=FileStore(tmp_path / "mcp"),
|
||||
artifact_store=stores.artifact_store,
|
||||
draft_workspace_store=stores.draft_workspace_store,
|
||||
run_store=stores.run_store,
|
||||
)
|
||||
context = context_from_service(service)
|
||||
|
||||
api = durable_workflow_api(context)
|
||||
|
||||
assert isinstance(api, WorkflowApi)
|
||||
assert api.context is context
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests and verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_durable_context.py -q
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
FAIL with ModuleNotFoundError: No module named 'wf_api.durable_context'
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Implement durable context helper**
|
||||
|
||||
Create `src/wf_api/durable_context.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from .operation_context import WorkflowOperationContext
|
||||
from .service import WorkflowApi
|
||||
from .stores import WorkflowStores
|
||||
|
||||
|
||||
def require_workflow_stores(context: WorkflowOperationContext) -> WorkflowStores:
|
||||
"""Return required stores or fail before constructing durable frontends.
|
||||
|
||||
`WorkflowOperationContext` keeps stores optional for compatibility tests and
|
||||
lightweight MCP surfaces. Durable API surfaces need all stores up front so a
|
||||
run cannot start without somewhere to persist artifacts, drafts, and stopped
|
||||
execution state.
|
||||
"""
|
||||
missing = []
|
||||
if context.artifact_store is None:
|
||||
missing.append("artifact_store")
|
||||
if context.draft_workspace_store is None:
|
||||
missing.append("draft_workspace_store")
|
||||
if context.run_store is None:
|
||||
missing.append("run_store")
|
||||
if missing:
|
||||
raise ValueError(
|
||||
"durable workflow API requires stores: " + ", ".join(missing)
|
||||
)
|
||||
return WorkflowStores(
|
||||
artifact_store=context.artifact_store,
|
||||
draft_workspace_store=context.draft_workspace_store,
|
||||
run_store=context.run_store,
|
||||
)
|
||||
|
||||
|
||||
def durable_workflow_api(context: WorkflowOperationContext) -> WorkflowApi:
|
||||
"""Construct a WorkflowApi only after durable store dependencies exist."""
|
||||
require_workflow_stores(context)
|
||||
return WorkflowApi(context)
|
||||
|
||||
|
||||
__all__ = ["durable_workflow_api", "require_workflow_stores"]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Export helpers from wf_api**
|
||||
|
||||
Modify `src/wf_api/__init__.py` to import and export:
|
||||
|
||||
```python
|
||||
from .durable_context import durable_workflow_api, require_workflow_stores
|
||||
```
|
||||
|
||||
Add these names to `__all__`:
|
||||
|
||||
```python
|
||||
"durable_workflow_api",
|
||||
"require_workflow_stores",
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run durable-context tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_durable_context.py -q
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
3 passed
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run wf_api import-direction test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_import_direction.py -q
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
1 passed
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Commit Task 2**
|
||||
|
||||
```bash
|
||||
git add src/wf_api/durable_context.py src/wf_api/__init__.py tests/wf_api/test_durable_context.py
|
||||
git commit -m "feat: add durable workflow API store guard"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Document Implemented Hardening
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/superpowers/specs/2026-06-03-persisted-run-resume-contract.md`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Update spec current gaps**
|
||||
|
||||
In `docs/superpowers/specs/2026-06-03-persisted-run-resume-contract.md`, under `## Current Gaps / Next Implementation Work`, replace item 1 with:
|
||||
|
||||
```markdown
|
||||
1. **Required stores for durable API**
|
||||
- Implemented for process-local frontends through
|
||||
`wf_api.durable_context.require_workflow_stores()` and
|
||||
`wf_api.durable_context.durable_workflow_api()`.
|
||||
- `WorkflowOperationContext` still allows optional stores for MCP test and
|
||||
compatibility paths.
|
||||
```
|
||||
|
||||
Under `## Implementation Order`, replace item 1 with:
|
||||
|
||||
```markdown
|
||||
1. Contract regression tests now cover:
|
||||
- non-interrupted `resume_run` rejection
|
||||
- deleted deployment does not erase existing run inspection
|
||||
- trace range validates before store lookup
|
||||
- blocked resume writes no checkpoint (`tests/wf_mcp/test_saved_subgraphs.py`)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update current roadmap**
|
||||
|
||||
In `docs/current_roadmap.md`, find the durable/persisted run section and add:
|
||||
|
||||
```markdown
|
||||
- `wf_api.durable_context` now provides a required-store guard for future
|
||||
durable HTTP/API frontends. It preserves the current process-local behavior
|
||||
while failing fast if artifact, draft, or run stores are missing.
|
||||
```
|
||||
|
||||
If no durable/persisted run section exists, add it under the current `wf_api` or runtime roadmap notes.
|
||||
|
||||
- [ ] **Step 3: Run docs grep sanity check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "durable_workflow_api|require_workflow_stores|non-interrupted|blocked resume writes no checkpoint" docs
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
matches in persisted-run spec and/or current roadmap
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit Task 3**
|
||||
|
||||
```bash
|
||||
git add docs/superpowers/specs/2026-06-03-persisted-run-resume-contract.md docs/current_roadmap.md
|
||||
git commit -m "docs: record durable run hardening status"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Final Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- Verify all touched code/tests/docs.
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_run_api.py tests/wf_api/test_durable_context.py tests/wf_api/test_import_direction.py tests/wf_mcp/test_saved_subgraphs.py -q
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
all selected tests pass
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run broader wf_api suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api -q
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
all wf_api tests pass
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run ruff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_api tests/wf_api tests/wf_mcp/test_saved_subgraphs.py
|
||||
uv run ruff format --check src/wf_api tests/wf_api tests/wf_mcp/test_saved_subgraphs.py
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
All checks passed
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run basedpyright**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
0 errors, 0 warnings, 0 notes
|
||||
```
|
||||
|
||||
Known caveat: this repo may still exit nonzero with the workspace enumeration warning even when it reports `0 errors`.
|
||||
|
||||
- [ ] **Step 5: Final report**
|
||||
|
||||
Report:
|
||||
|
||||
```text
|
||||
Implemented persisted run/resume hardening:
|
||||
- added missing contract regression tests
|
||||
- added wf_api durable required-store guard
|
||||
- updated persisted-run spec and roadmap
|
||||
|
||||
Verification:
|
||||
- focused tests: ...
|
||||
- wf_api tests: ...
|
||||
- ruff: ...
|
||||
- basedpyright: ...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
Spec coverage:
|
||||
|
||||
- `resume_run` only resumes interrupted runs: Task 1.
|
||||
- Pinned environment survives deployment deletion: Task 1.
|
||||
- Trace range validates before store lookup: Task 1.
|
||||
- Blocked resume writes no checkpoint: already covered by `tests/wf_mcp/test_saved_subgraphs.py`; referenced in Task 3 and final focused verification.
|
||||
- Required-store context/factory for durable API surfaces: Task 2.
|
||||
- Transactional backend, CAS resume guard, run listing, progress, retry/timeout: explicitly out of scope.
|
||||
|
||||
Placeholder scan:
|
||||
|
||||
- No placeholder implementation steps.
|
||||
- Code snippets include exact functions and expected assertions.
|
||||
|
||||
Type consistency:
|
||||
|
||||
- `WorkflowStores`, `WorkflowOperationContext`, and `WorkflowApi` names match current `src/wf_api`.
|
||||
- `TraceRangeLike` remains unchanged; this plan does not alter MCP/CLI schemas.
|
||||
@@ -0,0 +1,565 @@
|
||||
# Remote CLI Lifecycle RPC Methods Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make the `wf` draft/artifact/deploy commands work against `client.target.kind = "rpc_http"` instead of failing fast as local-only commands.
|
||||
|
||||
**Architecture:** Extend the fixed JSON-RPC method set, then extend `RpcWorkflowApiClient` to structurally support every `WorkflowApi` method used by the CLI. Only after the client and server both support the methods should the CLI command modules switch from local-only context to target-aware context. Keep this as remote workflow lifecycle support, not source registry/auth/MCP hosting.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, fastapi-jsonrpc, httpx, Typer, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- Add missing JSON-RPC methods for draft workspace, artifact, and deployment operations used by CLI.
|
||||
- Add matching `RpcWorkflowApiClient` methods.
|
||||
- Route `wf draft`, `wf artifact`, and `wf deploy` through `load_cli_context_from_typer`.
|
||||
- Add remote CLI integration tests for create draft → validate → save artifact → save deployment → validate deployment.
|
||||
- Keep existing local CLI behavior and old `wf_mcp.config.json` compatibility.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- Source registry.
|
||||
- MCP/OpenAPI source config.
|
||||
- `/mcp` hosting.
|
||||
- Auth.
|
||||
- SQL stores.
|
||||
- Remote docs/schema/explain commands.
|
||||
- Streaming/progress.
|
||||
|
||||
---
|
||||
|
||||
## Required Method Coverage
|
||||
|
||||
The client must implement every method used by these command modules:
|
||||
|
||||
```text
|
||||
src/wf_cli/commands/drafts.py
|
||||
list_draft_workspaces
|
||||
get_draft_workspace
|
||||
create_draft_workspace_from_capability
|
||||
patch_draft_workspace
|
||||
validate_draft_workspace
|
||||
create_wrapper_from_workspace
|
||||
create_artifact_from_workspace
|
||||
|
||||
src/wf_cli/commands/artifacts.py
|
||||
list_artifacts
|
||||
inspect_artifact
|
||||
|
||||
src/wf_cli/commands/deployments.py
|
||||
validate_deployment
|
||||
list_deployments
|
||||
inspect_deployment
|
||||
save_deployment
|
||||
delete_deployment
|
||||
```
|
||||
|
||||
Do not route a command module to target-aware context until the client has the
|
||||
methods that module calls.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add Missing RPC DTOs and Server Methods for Artifacts/Deployments
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_transport_rpc_http/models.py`
|
||||
- Modify: `src/wf_transport_rpc_http/app.py`
|
||||
- Modify: `src/wf_transport_rpc_http/__init__.py`
|
||||
- Modify: `tests/wf_transport_rpc_http/test_app.py`
|
||||
|
||||
- [ ] **Step 1: Add focused RPC app tests**
|
||||
|
||||
Append a test that seeds an artifact/deployment through `server.api`, then calls:
|
||||
|
||||
```text
|
||||
workflow.artifacts.list
|
||||
workflow.artifacts.inspect
|
||||
workflow.deployments.list
|
||||
workflow.deployments.inspect
|
||||
workflow.deployments.delete
|
||||
```
|
||||
|
||||
Assertions:
|
||||
|
||||
```python
|
||||
assert listed_artifacts["result"]["nodes"]
|
||||
assert inspected_artifact["result"]["artifact_id"] == "rpc_lifecycle"
|
||||
assert listed_deployments["result"]["deployments"]
|
||||
assert inspected_deployment["result"]["deployment_id"] == "rpc_lifecycle.default"
|
||||
assert deleted["result"]["deployment_id"] == "rpc_lifecycle.default"
|
||||
```
|
||||
|
||||
Use the existing `_constant_plan()` helper in `tests/wf_transport_rpc_http/test_app.py` if possible. Do not bypass the RPC app for the operations under test.
|
||||
|
||||
- [ ] **Step 2: Run the new test and verify method-not-found failures**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_app.py::test_rpc_artifact_and_deployment_catalog_methods -q
|
||||
```
|
||||
|
||||
Expected: fail with JSON-RPC method-not-found for the newly required methods.
|
||||
|
||||
- [ ] **Step 3: Add DTOs**
|
||||
|
||||
In `src/wf_transport_rpc_http/models.py`, add:
|
||||
|
||||
```python
|
||||
class ListArtifactsParams(RpcParamsModel):
|
||||
query: str | None = None
|
||||
kind: str | None = None
|
||||
cursor: str | None = None
|
||||
limit: int = Field(default=50, ge=1, le=100)
|
||||
|
||||
|
||||
class InspectArtifactParams(RpcParamsModel):
|
||||
artifact_id: str = Field(min_length=1)
|
||||
version: int = Field(ge=1)
|
||||
|
||||
|
||||
class ListDeploymentsParams(RpcParamsModel):
|
||||
pass
|
||||
|
||||
|
||||
class InspectDeploymentParams(RpcParamsModel):
|
||||
deployment_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
class DeleteDeploymentParams(RpcParamsModel):
|
||||
deployment_id: str = Field(min_length=1)
|
||||
```
|
||||
|
||||
Export these from `src/wf_transport_rpc_http/__init__.py`.
|
||||
|
||||
- [ ] **Step 4: Register artifact/deployment methods**
|
||||
|
||||
In `src/wf_transport_rpc_http/app.py`, import the new DTOs and register:
|
||||
|
||||
```text
|
||||
workflow.artifacts.list -> server.api.list_artifacts
|
||||
workflow.artifacts.inspect -> server.api.inspect_artifact
|
||||
workflow.deployments.list -> server.api.list_deployments
|
||||
workflow.deployments.inspect -> server.api.inspect_deployment
|
||||
workflow.deployments.delete -> server.api.delete_deployment
|
||||
```
|
||||
|
||||
Use the same expected-error handling as existing methods:
|
||||
|
||||
```python
|
||||
except (ValueError, KeyError, LookupError, FileNotFoundError) as exc:
|
||||
raise_workflow_rpc_error(exc)
|
||||
```
|
||||
|
||||
For `ListArtifactsParams.kind`, pass the string through as `kind=params.kind`.
|
||||
If basedpyright complains because `WorkflowApi.list_artifacts` expects
|
||||
`ArtifactKind | None`, narrow with:
|
||||
|
||||
```python
|
||||
kind = params.kind if params.kind in {"workflow", "wrapper"} else None
|
||||
```
|
||||
|
||||
and reject invalid values in the DTO if needed.
|
||||
|
||||
- [ ] **Step 5: Run focused transport tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_app.py -q
|
||||
uv run ruff check src/wf_transport_rpc_http tests/wf_transport_rpc_http
|
||||
uv run basedpyright --level error src/wf_transport_rpc_http tests/wf_transport_rpc_http
|
||||
```
|
||||
|
||||
Expected: tests pass, ruff clean, basedpyright 0 errors.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Add Draft Workspace RPC Methods
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_transport_rpc_http/models.py`
|
||||
- Modify: `src/wf_transport_rpc_http/app.py`
|
||||
- Modify: `src/wf_transport_rpc_http/__init__.py`
|
||||
- Modify: `tests/wf_transport_rpc_http/test_app.py`
|
||||
|
||||
- [ ] **Step 1: Add focused draft workspace RPC test**
|
||||
|
||||
Append a test that calls:
|
||||
|
||||
```text
|
||||
workflow.draft_workspaces.create_from_capability
|
||||
workflow.draft_workspaces.list
|
||||
workflow.draft_workspaces.get
|
||||
workflow.draft_workspaces.validate
|
||||
workflow.draft_workspaces.patch
|
||||
workflow.draft_workspaces.create_artifact
|
||||
workflow.draft_workspaces.create_wrapper
|
||||
```
|
||||
|
||||
Use a small patch such as changing the draft name or title through the existing
|
||||
workspace patch format. Assert:
|
||||
|
||||
```python
|
||||
assert created["result"]["workspace_id"] == "remote_ws"
|
||||
assert listed["result"]["workspaces"]
|
||||
assert fetched["result"]["workspace_id"] == "remote_ws"
|
||||
assert validated["result"]["status"] in {"valid", "invalid"}
|
||||
assert patched["result"]["revision"] == created["result"]["revision"] + 1
|
||||
assert artifact["result"]["artifact_id"] == "remote_artifact"
|
||||
assert wrapper["result"]["artifact_id"] == "remote_wrapper"
|
||||
```
|
||||
|
||||
If wrapper creation requires an output-capable draft and the simple capability
|
||||
draft cannot satisfy it, keep `create_wrapper` covered by client method tests
|
||||
and document why the RPC app integration test only covers `create_artifact`.
|
||||
|
||||
- [ ] **Step 2: Run the new test and verify method-not-found failures**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_app.py::test_rpc_draft_workspace_methods -q
|
||||
```
|
||||
|
||||
Expected: fail with method-not-found for missing workspace methods.
|
||||
|
||||
- [ ] **Step 3: Add DTOs**
|
||||
|
||||
Add to `src/wf_transport_rpc_http/models.py`:
|
||||
|
||||
```python
|
||||
class ListDraftWorkspacesParams(RpcParamsModel):
|
||||
pass
|
||||
|
||||
|
||||
class GetDraftWorkspaceParams(RpcParamsModel):
|
||||
workspace_id: str = Field(min_length=1)
|
||||
include_draft: bool = False
|
||||
|
||||
|
||||
class PatchDraftWorkspaceParams(RpcParamsModel):
|
||||
workspace_id: str = Field(min_length=1)
|
||||
revision: int = Field(ge=1)
|
||||
patch: list[dict[str, Any]]
|
||||
|
||||
|
||||
class ValidateDraftWorkspaceParams(RpcParamsModel):
|
||||
workspace_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
class CreateArtifactFromWorkspaceParams(RpcParamsModel):
|
||||
workspace_id: str = Field(min_length=1)
|
||||
artifact_id: str = Field(min_length=1)
|
||||
version: int = Field(ge=1)
|
||||
title: str = Field(min_length=1)
|
||||
outcomes: list[str]
|
||||
kind: str = "workflow"
|
||||
description: str | None = None
|
||||
required_capabilities: dict[str, dict[str, Any]] | None = None
|
||||
source_bindings: dict[str, str] | None = None
|
||||
created_from_catalog_version: str | None = None
|
||||
|
||||
|
||||
class CreateWrapperFromWorkspaceParams(RpcParamsModel):
|
||||
workspace_id: str = Field(min_length=1)
|
||||
artifact_id: str = Field(min_length=1)
|
||||
version: int = Field(ge=1)
|
||||
title: str = Field(min_length=1)
|
||||
outcomes: list[str]
|
||||
description: str | None = None
|
||||
required_capabilities: dict[str, dict[str, Any]] | None = None
|
||||
source_bindings: dict[str, str] | None = None
|
||||
created_from_catalog_version: str | None = None
|
||||
```
|
||||
|
||||
Export these from `src/wf_transport_rpc_http/__init__.py`.
|
||||
|
||||
- [ ] **Step 4: Register draft workspace methods**
|
||||
|
||||
In `src/wf_transport_rpc_http/app.py`, register:
|
||||
|
||||
```text
|
||||
workflow.draft_workspaces.list
|
||||
workflow.draft_workspaces.get
|
||||
workflow.draft_workspaces.create_from_capability
|
||||
workflow.draft_workspaces.patch
|
||||
workflow.draft_workspaces.validate
|
||||
workflow.draft_workspaces.create_artifact
|
||||
workflow.draft_workspaces.create_wrapper
|
||||
```
|
||||
|
||||
Map them to the matching `server.api` methods:
|
||||
|
||||
```python
|
||||
server.api.list_draft_workspaces()
|
||||
server.api.get_draft_workspace(...)
|
||||
server.api.create_draft_workspace_from_capability(...)
|
||||
server.api.patch_draft_workspace(...)
|
||||
server.api.validate_draft_workspace(...)
|
||||
server.api.create_artifact_from_workspace(...)
|
||||
server.api.create_wrapper_from_workspace(...)
|
||||
```
|
||||
|
||||
The existing `workflow.drafts.create_from_capability` method may stay for
|
||||
backward compatibility. It can call the same API method as
|
||||
`workflow.draft_workspaces.create_from_capability`.
|
||||
|
||||
- [ ] **Step 5: Run focused transport tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_app.py -q
|
||||
uv run ruff check src/wf_transport_rpc_http tests/wf_transport_rpc_http
|
||||
uv run basedpyright --level error src/wf_transport_rpc_http tests/wf_transport_rpc_http
|
||||
```
|
||||
|
||||
Expected: tests pass, ruff clean, basedpyright 0 errors.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Extend RpcWorkflowApiClient
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_transport_rpc_http/client.py`
|
||||
- Modify: `tests/wf_transport_rpc_http/test_client.py`
|
||||
|
||||
- [ ] **Step 1: Add client tests for the newly exposed methods**
|
||||
|
||||
Add tests covering:
|
||||
|
||||
```text
|
||||
list_artifacts / inspect_artifact
|
||||
list_deployments / inspect_deployment / validate_deployment / delete_deployment
|
||||
list_draft_workspaces / get_draft_workspace / validate_draft_workspace
|
||||
create_draft_workspace_from_capability
|
||||
patch_draft_workspace
|
||||
create_artifact_from_workspace
|
||||
create_wrapper_from_workspace if feasible
|
||||
```
|
||||
|
||||
Use `httpx.ASGITransport(app=create_rpc_app(server))` like the existing client
|
||||
tests. Seed artifacts/deployments through `server.api` where the method under
|
||||
test is only read/list/inspect. For create/patch workspace tests, call through
|
||||
the client.
|
||||
|
||||
- [ ] **Step 2: Run client tests and verify failures**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_client.py -q
|
||||
```
|
||||
|
||||
Expected: fail because `RpcWorkflowApiClient` lacks the new methods.
|
||||
|
||||
- [ ] **Step 3: Implement client methods**
|
||||
|
||||
Add one client method for each CLI-used `WorkflowApi` method listed in
|
||||
"Required Method Coverage".
|
||||
|
||||
Wire methods to the JSON-RPC names from Tasks 1-2. Examples:
|
||||
|
||||
```python
|
||||
async def list_artifacts(...):
|
||||
return await self._call("workflow.artifacts.list", {...})
|
||||
|
||||
async def get_draft_workspace(...):
|
||||
return await self._call("workflow.draft_workspaces.get", {...})
|
||||
|
||||
async def create_artifact_from_workspace(...):
|
||||
return await self._call("workflow.draft_workspaces.create_artifact", {...})
|
||||
```
|
||||
|
||||
Keep signatures close to `WorkflowApi` so basedpyright accepts CLI command
|
||||
calls without casts.
|
||||
|
||||
- [ ] **Step 4: Run client and type checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_client.py -q
|
||||
uv run basedpyright --level error src/wf_transport_rpc_http src/wf_cli
|
||||
```
|
||||
|
||||
Expected: client tests pass and basedpyright reports 0 errors.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Route Draft/Artifact/Deploy CLI Through Target-Aware Context
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/wf_cli/commands/drafts.py`
|
||||
- Modify: `src/wf_cli/commands/artifacts.py`
|
||||
- Modify: `src/wf_cli/commands/deployments.py`
|
||||
- Modify: `tests/wf_cli/test_remote_target.py`
|
||||
|
||||
- [ ] **Step 1: Write remote CLI lifecycle test**
|
||||
|
||||
Add a test that starts an in-process JSON-RPC app and invokes Typer commands
|
||||
with:
|
||||
|
||||
```text
|
||||
wf --config wf.json --url http://test/rpc draft create-from-capability ...
|
||||
wf --config wf.json --url http://test/rpc draft validate ...
|
||||
wf --config wf.json --url http://test/rpc draft save ...
|
||||
wf --config wf.json --url http://test/rpc artifact inspect ...
|
||||
wf --config wf.json --url http://test/rpc deploy save ...
|
||||
wf --config wf.json --url http://test/rpc deploy validate ...
|
||||
```
|
||||
|
||||
Use the same `httpx.AsyncClient` monkeypatch pattern already present in
|
||||
`tests/wf_cli/test_remote_target.py`.
|
||||
|
||||
Assertions:
|
||||
|
||||
```python
|
||||
assert created.exit_code == 0
|
||||
assert validated.exit_code == 0
|
||||
assert saved_artifact.exit_code == 0
|
||||
assert inspected_artifact.exit_code == 0
|
||||
assert saved_deployment.exit_code == 0
|
||||
assert validated_deployment.exit_code == 0
|
||||
```
|
||||
|
||||
Also assert key output fragments:
|
||||
|
||||
```python
|
||||
assert '"workspace_id": "remote_ws"' in created.output
|
||||
assert '"status": "valid"' in validated.output
|
||||
assert '"artifact_id": "remote_artifact"' in saved_artifact.output
|
||||
assert '"deployment_id": "remote_artifact.default"' in saved_deployment.output
|
||||
assert '"status": "runnable"' in validated_deployment.output
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test and verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_remote_target.py::test_wf_remote_draft_artifact_deploy_lifecycle -q
|
||||
```
|
||||
|
||||
Expected: fail because command modules still use local-only context.
|
||||
|
||||
- [ ] **Step 3: Switch command imports**
|
||||
|
||||
In `src/wf_cli/commands/drafts.py`, `artifacts.py`, and `deployments.py`,
|
||||
replace:
|
||||
|
||||
```python
|
||||
from wf_cli.context import load_local_cli_context_from_typer as load_cli_context
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```python
|
||||
from wf_cli.context import load_cli_context_from_typer as load_cli_context
|
||||
```
|
||||
|
||||
Keep the local `load_cli_context` alias name so existing tests that monkeypatch
|
||||
`wf_cli.commands.<module>.load_cli_context` keep working.
|
||||
|
||||
- [ ] **Step 4: Run CLI tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_cli/test_remote_target.py tests/wf_cli/test_run_deploy.py tests/wf_cli/test_discovery_lifecycle.py -q
|
||||
uv run basedpyright --level error src/wf_cli src/wf_transport_rpc_http
|
||||
```
|
||||
|
||||
Expected: tests pass and basedpyright reports 0 errors.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Documentation and Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/superpowers/specs/2026-06-03-workflow-config-targets-and-sources.md`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Update spec status**
|
||||
|
||||
Update the implementation status to say:
|
||||
|
||||
```markdown
|
||||
- remote JSON-RPC client support now covers capability, draft workspace,
|
||||
artifact, deployment, and run CLI commands
|
||||
- draft/artifact/deploy commands no longer fail fast for `rpc_http` targets
|
||||
```
|
||||
|
||||
Remove or update the older local-only/fail-fast status line.
|
||||
|
||||
- [ ] **Step 2: Update roadmap**
|
||||
|
||||
Update the roadmap note to say the basic remote CLI lifecycle is wired:
|
||||
|
||||
```markdown
|
||||
selected `wf` commands can target JSON-RPC HTTP
|
||||
```
|
||||
|
||||
should become:
|
||||
|
||||
```markdown
|
||||
the basic `wf` lifecycle can target JSON-RPC HTTP: capability discovery,
|
||||
draft workspace authoring, artifact/deployment operations, run, inspect, and
|
||||
bounded trace.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run focused verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http tests/wf_cli/test_remote_target.py tests/wf_cli/test_run_deploy.py tests/wf_cli/test_discovery_lifecycle.py -q
|
||||
uv run ruff check src/wf_transport_rpc_http src/wf_cli tests/wf_transport_rpc_http tests/wf_cli
|
||||
uv run ruff format --check src/wf_transport_rpc_http src/wf_cli tests/wf_transport_rpc_http tests/wf_cli
|
||||
uv run basedpyright --level error src/wf_transport_rpc_http src/wf_cli tests/wf_transport_rpc_http tests/wf_cli
|
||||
```
|
||||
|
||||
Expected: pass, except basedpyright may still exit non-zero for the known
|
||||
workspace enumeration warning only when run at full workspace scope. For the
|
||||
focused command above, expect 0 errors.
|
||||
|
||||
- [ ] **Step 4: Run full verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
uv run ruff check
|
||||
uv run ruff format --check
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- pytest passes with current skip/xfail count
|
||||
- ruff passes
|
||||
- basedpyright reports `0 errors`; if it exits 1 only due to workspace
|
||||
enumeration warning, report that exactly
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Notes
|
||||
|
||||
This plan intentionally adds server RPC methods before client methods and client
|
||||
methods before CLI routing. That sequence prevents the previous half-migration
|
||||
problem where commands could receive a partial client.
|
||||
|
||||
Do not remove old local tests or monkeypatch seams. The command modules can keep
|
||||
a local import alias named `load_cli_context` for compatibility with existing
|
||||
tests, but the imported helper should become target-aware after Task 4.
|
||||
|
||||
@@ -0,0 +1,577 @@
|
||||
# Source Admin API Surface Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add a protocol-neutral read-only source/admin surface in `wf_api` and make MCP admin source tools delegate through it.
|
||||
|
||||
**Architecture:** `WorkflowApiSurface` stays workflow-lifecycle-only. Source/admin operations get a sibling `WorkflowSourceAdminSurface` plus `WorkflowSourceAdminApi` implementation over `WorkflowOperationContext.specs.capability_sources`. The MCP admin handler becomes an adapter over this neutral API while connection/raw MCP methods remain MCP-broker-owned.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, Protocols, Pydantic-backed platform models, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Current Findings
|
||||
|
||||
- Source catalog internals already moved out of the old god service into `src/wf_mcp/broker/service/source_catalog.py`.
|
||||
- `wf_api` does not currently have a source/admin API module.
|
||||
- MCP admin source tools still flow through:
|
||||
|
||||
```text
|
||||
wf_mcp.admin_surface.tools
|
||||
-> BrokerAdminHandlers
|
||||
-> WfMcpService.list_source_summaries / inspect_source
|
||||
-> SourceCatalogService
|
||||
```
|
||||
|
||||
- `WorkflowOperationContext.specs.capability_sources` already exposes the source inventory `wf_api` needs for read-only source listing and inspection.
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- `list_sources(cursor=None, limit=50) -> dict`
|
||||
- `inspect_source(source_id: str) -> dict`
|
||||
- Protocol-neutral `WorkflowSourceAdminSurface`
|
||||
- Local implementation `WorkflowSourceAdminApi`
|
||||
- MCP `BrokerAdminHandlers.list_sources()` and `.inspect_source()` delegate through `WorkflowSourceAdminApi`
|
||||
- Focused tests proving payload compatibility with existing MCP source output
|
||||
|
||||
Out of scope for this slice:
|
||||
|
||||
- Adding/removing/updating sources
|
||||
- Store-backed source registry
|
||||
- Connection status, catalog refresh, raw method invocation
|
||||
- JSON-RPC transport methods for source admin
|
||||
- CLI `wf source ...` commands
|
||||
|
||||
Those are follow-up slices once this neutral seam exists.
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create `src/wf_api/source_admin.py`
|
||||
- Owns `WorkflowSourceAdminApi`.
|
||||
- Uses only `WorkflowOperationContext` and platform source models.
|
||||
- Imports no `wf_mcp`.
|
||||
|
||||
- Modify `src/wf_api/surface.py`
|
||||
- Adds sibling protocol `WorkflowSourceAdminSurface`.
|
||||
- Does not make `WorkflowApiSurface` inherit it.
|
||||
|
||||
- Modify `src/wf_api/__init__.py`
|
||||
- Exports `WorkflowSourceAdminApi` and `WorkflowSourceAdminSurface`.
|
||||
|
||||
- Modify `src/wf_mcp/admin_surface/handlers/broker.py`
|
||||
- Construct `WorkflowSourceAdminApi(context_from_service(service))`.
|
||||
- Delegate `list_sources` and `inspect_source` to it.
|
||||
- Keep connection/catalog/resource/raw methods unchanged.
|
||||
- Make those two source methods async, matching the async MCP tool boundary.
|
||||
|
||||
- Modify `src/wf_mcp/admin_surface/tools.py`
|
||||
- Await async source handler methods.
|
||||
|
||||
- Create `tests/wf_api/test_source_admin_api.py`
|
||||
- Direct neutral API tests.
|
||||
|
||||
- Modify `tests/wf_mcp/test_admin_surface.py`
|
||||
- Add adapter smoke coverage for the neutral source admin delegation.
|
||||
|
||||
- Modify `docs/current_roadmap.md`
|
||||
- Mark the read-only neutral source/admin seam as completed.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add failing `wf_api` source admin tests
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/wf_api/test_source_admin_api.py`
|
||||
|
||||
- [ ] **Step 1: Write direct tests**
|
||||
|
||||
Create `tests/wf_api/test_source_admin_api.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from wf_api import WorkflowSourceAdminApi
|
||||
from wf_api.operation_context import WorkflowOperationContext
|
||||
from wf_authoring import NodeSpec
|
||||
from wf_platform import (
|
||||
CapabilityBuckets,
|
||||
CapabilitySource,
|
||||
SourcePermissions,
|
||||
SourceVisibility,
|
||||
)
|
||||
|
||||
|
||||
class DummyEvents:
|
||||
def record_event(self, event: object) -> None:
|
||||
pass
|
||||
|
||||
def record_workflow_event(
|
||||
self,
|
||||
event_type: str,
|
||||
*,
|
||||
capability_id: str,
|
||||
payload: dict[str, Any],
|
||||
) -> None:
|
||||
pass
|
||||
|
||||
|
||||
class DummyRuntime:
|
||||
async def run_workflow_from_plan(self, *args: Any, **kwargs: Any) -> object:
|
||||
raise AssertionError("source admin tests must not run workflows")
|
||||
|
||||
async def resume_workflow_from_plan(
|
||||
self,
|
||||
*args: Any,
|
||||
**kwargs: Any,
|
||||
) -> object:
|
||||
raise AssertionError("source admin tests must not resume workflows")
|
||||
|
||||
|
||||
class StaticSpecProvider:
|
||||
def __init__(self, sources: dict[str, CapabilitySource]) -> None:
|
||||
self._sources = sources
|
||||
|
||||
@property
|
||||
def capability_sources(self) -> dict[str, CapabilitySource]:
|
||||
return self._sources
|
||||
|
||||
def get_qualified_spec(self, qualified_name: str) -> NodeSpec[Any, Any]:
|
||||
raise KeyError(f"unknown capability {qualified_name!r}")
|
||||
|
||||
|
||||
def _api(*sources: CapabilitySource) -> WorkflowSourceAdminApi:
|
||||
provider = StaticSpecProvider({source.id: source for source in sources})
|
||||
return WorkflowSourceAdminApi(
|
||||
WorkflowOperationContext(
|
||||
artifact_store=None,
|
||||
draft_workspace_store=None,
|
||||
run_store=None,
|
||||
events=DummyEvents(),
|
||||
specs=provider,
|
||||
runtime=DummyRuntime(),
|
||||
live_sources=None,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _source(source_id: str, *, enabled: bool = True) -> CapabilitySource:
|
||||
return CapabilitySource(
|
||||
id=source_id,
|
||||
kind="connection",
|
||||
enabled=enabled,
|
||||
capabilities=CapabilityBuckets(),
|
||||
visibility=SourceVisibility(
|
||||
planner=True,
|
||||
mcp_client=True,
|
||||
admin_dashboard=True,
|
||||
),
|
||||
permissions=SourcePermissions(calls_upstream=True),
|
||||
description=f"{source_id} source",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_source_admin_lists_compact_sources_in_id_order() -> None:
|
||||
api = _api(_source("zeta.personal"), _source("alpha.personal", enabled=False))
|
||||
|
||||
payload = await api.list_sources(limit=10)
|
||||
|
||||
assert payload["total"] == 2
|
||||
assert payload["next_cursor"] is None
|
||||
assert [source["id"] for source in payload["sources"]] == [
|
||||
"alpha.personal",
|
||||
"zeta.personal",
|
||||
]
|
||||
assert payload["sources"][0]["enabled"] is False
|
||||
assert payload["sources"][1]["description"] == "zeta.personal source"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_source_admin_pages_sources() -> None:
|
||||
api = _api(_source("a"), _source("b"), _source("c"))
|
||||
|
||||
first = await api.list_sources(limit=2)
|
||||
second = await api.list_sources(cursor=first["next_cursor"], limit=2)
|
||||
|
||||
assert [source["id"] for source in first["sources"]] == ["a", "b"]
|
||||
assert first["next_cursor"] == "2"
|
||||
assert [source["id"] for source in second["sources"]] == ["c"]
|
||||
assert second["next_cursor"] is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_source_admin_inspects_full_source_inventory() -> None:
|
||||
api = _api(_source("demo.personal"))
|
||||
|
||||
payload = await api.inspect_source(source_id="demo.personal")
|
||||
|
||||
assert payload["id"] == "demo.personal"
|
||||
assert payload["kind"] == "connection"
|
||||
assert payload["description"] == "demo.personal source"
|
||||
assert payload["visibility"]["planner"] is True
|
||||
assert payload["permissions"]["calls_upstream"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_source_admin_inspect_unknown_source_raises_clear_key_error() -> None:
|
||||
api = _api(_source("demo.personal"))
|
||||
|
||||
with pytest.raises(KeyError, match="unknown source 'missing.source'"):
|
||||
await api.inspect_source(source_id="missing.source")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests and verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_source_admin_api.py -q
|
||||
```
|
||||
|
||||
Expected: FAIL because `WorkflowSourceAdminApi` is not exported yet.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Implement `WorkflowSourceAdminApi`
|
||||
|
||||
**Files:**
|
||||
- Create: `src/wf_api/source_admin.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
|
||||
- [ ] **Step 1: Add source admin API**
|
||||
|
||||
Create `src/wf_api/source_admin.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from wf_platform import page_items
|
||||
|
||||
from .operation_context import WorkflowOperationContext
|
||||
|
||||
|
||||
class WorkflowSourceAdminApi:
|
||||
"""Read-only protocol-neutral source inventory operations.
|
||||
|
||||
This is a sibling to WorkflowApi, not part of WorkflowApiSurface, because
|
||||
source administration is server/platform management rather than workflow
|
||||
lifecycle execution.
|
||||
"""
|
||||
|
||||
def __init__(self, context: WorkflowOperationContext) -> None:
|
||||
self.context = context
|
||||
|
||||
async def list_sources(
|
||||
self,
|
||||
*,
|
||||
cursor: str | None = None,
|
||||
limit: int = 50,
|
||||
) -> dict[str, Any]:
|
||||
summaries = [
|
||||
source.as_status().model_dump(mode="json")
|
||||
for source in sorted(
|
||||
self.context.specs.capability_sources.values(),
|
||||
key=lambda source: source.id,
|
||||
)
|
||||
]
|
||||
page = page_items(summaries, cursor=cursor, limit=limit)
|
||||
return {
|
||||
"sources": list(page.items),
|
||||
"next_cursor": page.next_cursor,
|
||||
"total": page.total,
|
||||
}
|
||||
|
||||
async def inspect_source(self, *, source_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
source = self.context.specs.capability_sources[source_id]
|
||||
except KeyError as exc:
|
||||
raise KeyError(f"unknown source {source_id!r}") from exc
|
||||
return source.as_inventory().model_dump(mode="json")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Export from `wf_api`**
|
||||
|
||||
Modify `src/wf_api/__init__.py`:
|
||||
|
||||
```python
|
||||
from .source_admin import WorkflowSourceAdminApi
|
||||
```
|
||||
|
||||
Add `"WorkflowSourceAdminApi"` to `__all__`.
|
||||
|
||||
- [ ] **Step 3: Run direct tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_source_admin_api.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Add sibling surface protocol
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_api/surface.py`
|
||||
- Modify: `src/wf_api/__init__.py`
|
||||
- Test: `tests/wf_api/test_source_admin_api.py`
|
||||
|
||||
- [ ] **Step 1: Add protocol conformance test**
|
||||
|
||||
Append to `tests/wf_api/test_source_admin_api.py`:
|
||||
|
||||
```python
|
||||
from wf_api import WorkflowSourceAdminSurface
|
||||
|
||||
|
||||
def test_source_admin_api_satisfies_surface_protocol() -> None:
|
||||
api: WorkflowSourceAdminSurface = _api(_source("demo.personal"))
|
||||
|
||||
assert api is not None
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add protocol**
|
||||
|
||||
In `src/wf_api/surface.py`, add this class near the other surface protocols:
|
||||
|
||||
```python
|
||||
class WorkflowSourceAdminSurface(Protocol):
|
||||
"""Read-only source/admin methods exposed by platform frontends."""
|
||||
|
||||
async def list_sources(
|
||||
self,
|
||||
*,
|
||||
cursor: str | None = None,
|
||||
limit: int = 50,
|
||||
) -> dict[str, Any]: ...
|
||||
|
||||
async def inspect_source(
|
||||
self,
|
||||
*,
|
||||
source_id: str,
|
||||
) -> dict[str, Any]: ...
|
||||
```
|
||||
|
||||
Add `"WorkflowSourceAdminSurface"` to `__all__`.
|
||||
|
||||
Do not add it as a base class of `WorkflowApiSurface`.
|
||||
|
||||
- [ ] **Step 3: Export protocol**
|
||||
|
||||
Modify `src/wf_api/__init__.py`:
|
||||
|
||||
```python
|
||||
from .surface import WorkflowSourceAdminSurface
|
||||
```
|
||||
|
||||
Add `"WorkflowSourceAdminSurface"` to `__all__`.
|
||||
|
||||
- [ ] **Step 4: Run tests and type check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_source_admin_api.py tests/wf_api/test_import_direction.py -q
|
||||
uv run basedpyright --level error src/wf_api tests/wf_api/test_source_admin_api.py
|
||||
```
|
||||
|
||||
Expected: tests PASS, basedpyright reports 0 errors.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Delegate MCP admin source tools through `wf_api`
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_mcp/admin_surface/handlers/broker.py`
|
||||
- Modify: `src/wf_mcp/admin_surface/tools.py`
|
||||
- Test: `tests/wf_mcp/test_admin_surface.py`
|
||||
|
||||
- [ ] **Step 1: Add adapter smoke assertion**
|
||||
|
||||
In `tests/wf_mcp/test_admin_surface.py`, inside
|
||||
`test_broker_admin_handlers_list_connections_and_events`, add:
|
||||
|
||||
```python
|
||||
sources = _run(handlers.list_sources(limit=100))
|
||||
|
||||
source_ids = {source["id"] for source in sources["sources"]}
|
||||
assert "wf.std" in source_ids
|
||||
assert "wf.docs" in source_ids
|
||||
assert sources["total"] >= 2
|
||||
```
|
||||
|
||||
This test uses the existing `_run()` helper.
|
||||
|
||||
- [ ] **Step 2: Run the updated test and verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_admin_surface.py::test_broker_admin_handlers_list_connections_and_events -q
|
||||
```
|
||||
|
||||
Expected before delegation: FAIL because `BrokerAdminHandlers.list_sources()`
|
||||
is still sync and returns a dict, not an awaitable. This failure drives the async
|
||||
source-handler cleanup.
|
||||
|
||||
- [ ] **Step 3: Change handler implementation**
|
||||
|
||||
Modify `src/wf_mcp/admin_surface/handlers/broker.py`:
|
||||
|
||||
```python
|
||||
from wf_api import WorkflowSourceAdminApi, WorkflowSourceAdminSurface
|
||||
from wf_mcp.broker.service.workflow_operation_context import context_from_service
|
||||
```
|
||||
|
||||
Update `__init__`:
|
||||
|
||||
```python
|
||||
def __init__(self, service: WfMcpService) -> None:
|
||||
self.service = service
|
||||
self.sources: WorkflowSourceAdminSurface = WorkflowSourceAdminApi(
|
||||
context_from_service(service)
|
||||
)
|
||||
```
|
||||
|
||||
Update source methods:
|
||||
|
||||
```python
|
||||
async def list_sources(
|
||||
self,
|
||||
*,
|
||||
cursor: str | None = None,
|
||||
limit: int = 50,
|
||||
) -> dict[str, Any]:
|
||||
return await self.sources.list_sources(cursor=cursor, limit=limit)
|
||||
|
||||
async def inspect_source(self, source_id: str) -> dict[str, Any]:
|
||||
return await self.sources.inspect_source(source_id=source_id)
|
||||
```
|
||||
|
||||
Do not add an `asyncio.run()` bridge. The handler is called from async MCP tools,
|
||||
so source methods should be async at this boundary.
|
||||
|
||||
- [ ] **Step 4: Await source handler calls in MCP tools**
|
||||
|
||||
In `src/wf_mcp/admin_surface/tools.py`, update:
|
||||
|
||||
```python
|
||||
return await handlers.list_sources(cursor=cursor, limit=limit)
|
||||
```
|
||||
|
||||
and:
|
||||
|
||||
```python
|
||||
return await handlers.inspect_source(source_id)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run MCP admin tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_admin_surface.py tests/wf_mcp/server/test_config.py tests/wf_mcp/test_broker_server.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Documentation and verification
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
- Modify: `docs/superpowers/specs/2026-06-03-cli-api-alignment-notes.md`
|
||||
|
||||
- [ ] **Step 1: Update roadmap**
|
||||
|
||||
In `docs/current_roadmap.md`, under **Durable API service shape** or
|
||||
**CLI/API alignment**, add:
|
||||
|
||||
```markdown
|
||||
- Completed: read-only source inventory now has a protocol-neutral
|
||||
`WorkflowSourceAdminApi` / `WorkflowSourceAdminSurface`; MCP admin source
|
||||
tools delegate through it while connection/raw MCP operations remain
|
||||
broker-owned.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update CLI/API notes**
|
||||
|
||||
In `docs/superpowers/specs/2026-06-03-cli-api-alignment-notes.md`, under
|
||||
**Next Slices**, replace the source/admin item with:
|
||||
|
||||
```markdown
|
||||
1. **Source/admin transport and CLI commands**
|
||||
- Build JSON-RPC methods and `wf source ...` commands over
|
||||
`WorkflowSourceAdminSurface`.
|
||||
- Keep mutation out until the store-backed source registry is designed.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_source_admin_api.py tests/wf_mcp/test_admin_surface.py tests/wf_mcp/server/test_config.py tests/wf_mcp/test_broker_server.py -q
|
||||
uv run ruff check src/wf_api src/wf_mcp/admin_surface tests/wf_api/test_source_admin_api.py tests/wf_mcp/test_admin_surface.py tests/wf_mcp/server/test_config.py tests/wf_mcp/test_broker_server.py
|
||||
uv run ruff format --check src/wf_api src/wf_mcp/admin_surface tests/wf_api/test_source_admin_api.py tests/wf_mcp/test_admin_surface.py tests/wf_mcp/server/test_config.py tests/wf_mcp/test_broker_server.py
|
||||
uv run basedpyright --level error src/wf_api src/wf_mcp/admin_surface tests/wf_api/test_source_admin_api.py tests/wf_mcp/test_admin_surface.py
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- pytest PASS
|
||||
- ruff check PASS
|
||||
- ruff format PASS
|
||||
- basedpyright 0 errors
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add src/wf_api/source_admin.py src/wf_api/surface.py src/wf_api/__init__.py src/wf_mcp/admin_surface/handlers/broker.py src/wf_mcp/admin_surface/tools.py tests/wf_api/test_source_admin_api.py tests/wf_mcp/test_admin_surface.py docs/current_roadmap.md docs/superpowers/specs/2026-06-03-cli-api-alignment-notes.md
|
||||
git commit -m "feat: add source admin api surface"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Follow-Up Slices
|
||||
|
||||
1. **Source/admin JSON-RPC transport**
|
||||
- Add fixed methods such as `workflow.sources.list` and
|
||||
`workflow.sources.inspect`.
|
||||
- Add a dedicated RPC source-admin client or mixin, but keep the lifecycle
|
||||
`RpcWorkflowApiClient` contract clear.
|
||||
|
||||
2. **CLI `wf source` commands**
|
||||
- Add `wf source list` and `wf source inspect`.
|
||||
- Use the same target-aware context pattern as workflow lifecycle commands,
|
||||
but the CLI context may need a second handler field for source admin.
|
||||
|
||||
3. **Store-backed source registry**
|
||||
- Config remains bootstrap.
|
||||
- Server-owned dynamic source changes persist through a source registry store.
|
||||
- Source identity stays structural: source id, provider/account/profile, and
|
||||
concrete transport details are not inferred from dotted display names.
|
||||
|
||||
4. **Mutable source admin**
|
||||
- Add source create/update/delete only after store persistence and validation
|
||||
rules exist.
|
||||
- Enforce duplicate id behavior and liveness/validation diagnostics before
|
||||
a source can become runnable.
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: read-only source list/inspect, neutral surface, MCP adapter, and docs are covered.
|
||||
- Placeholder scan: no TBD/TODO placeholders.
|
||||
- Type consistency: `WorkflowSourceAdminApi`, `WorkflowSourceAdminSurface`, `source_id`, `cursor`, and `limit` names are consistent across tasks.
|
||||
- Risk: `BrokerAdminHandlers.list_sources` / `inspect_source` become async. The plan updates the direct handler test and the MCP tool wrappers that call them.
|
||||
@@ -0,0 +1,421 @@
|
||||
# Source Registry Next Slices Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Move the source registry toward a generic platform/API boundary, then wire startup merge and finally mutation without locking MCP-specific assumptions into `wf_api`.
|
||||
|
||||
**Architecture:** Split generic registry mechanics from MCP-specific source entries. `wf_api` should own the generic registry file/store concepts once they no longer import MCP validators. `wf_mcp` should own MCP source entry models and conversion into `ConnectionConfig` / broker services. Startup merge happens after the split so config-vs-store precedence is implemented against the right abstractions.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, existing `wf_mcp.source_registry`, `wf_api`, `wf_config`, `WfMcpService`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
Slice 1 created `src/wf_mcp/source_registry.py` with:
|
||||
|
||||
- `SourceRegistryModel`
|
||||
- `StdioSourceTransport`
|
||||
- `HttpSourceTransport`
|
||||
- `McpSourceRegistryEntry`
|
||||
- `SourceRegistryFile`
|
||||
- `SourceRegistryStore`
|
||||
- `FileSourceRegistryStore`
|
||||
|
||||
This is useful and tested, but still MCP-shaped:
|
||||
|
||||
- id validation uses `wf_mcp.connections.parse_connection_id`
|
||||
- reserved ids come from `wf_mcp.shared.names`
|
||||
- source entry type is `McpSourceRegistryEntry`
|
||||
- transport definitions are MCP transports
|
||||
|
||||
Slice 2A then moved generic registry mechanics to `wf_api.source_registry`,
|
||||
while `wf_mcp.source_registry` kept MCP-specific entries and transports. Slice
|
||||
2B added `registry_entry_to_connection_config()`.
|
||||
|
||||
The next executable slice is startup merge:
|
||||
`docs/superpowers/plans/2026-06-03-source-registry-startup-merge.md`.
|
||||
|
||||
## Slice Order
|
||||
|
||||
1. **Slice 2A: Generic Registry Mechanics**
|
||||
- **Status: complete.**
|
||||
- Move generic validation/store/file mechanics to `wf_api`.
|
||||
- Keep MCP entry/transport validation in `wf_mcp`.
|
||||
- Prefer boring helpers over deep Pydantic generics where that keeps the
|
||||
boundary easier to type-check.
|
||||
- Do not change runtime behavior.
|
||||
|
||||
2. **Slice 2B: MCP Entry Conversion**
|
||||
- **Status: complete.**
|
||||
- Add conversion helpers between `McpSourceRegistryEntry` and
|
||||
`ConnectionConfig`.
|
||||
- Keep config merge out of scope.
|
||||
|
||||
3. **Slice 3: Startup Merge**
|
||||
- **Status: complete.**
|
||||
- Load registry at broker/server startup.
|
||||
- Merge config and registry with config precedence.
|
||||
- Emit events/diagnostics for shadowed registry entries.
|
||||
|
||||
4. **Slice 4: Read Desired Registry Through Admin**
|
||||
- **Status: complete.**
|
||||
- Expose desired registry entries separately from observed source inventory.
|
||||
- `WorkflowSourceRegistryApi` provides neutral read-only access.
|
||||
- JSON-RPC methods `workflow.admin.source_registry.list` / `.inspect`.
|
||||
- CLI commands `wf admin registry list` / `wf admin registry inspect`.
|
||||
- Local/static servers report unavailable instead of empty.
|
||||
- Concrete MCP-backed `WorkflowServer` construction remains future work.
|
||||
|
||||
5. **Slice 5: Mutation Commands**
|
||||
- **Status: complete.**
|
||||
- Add add/update/enable/disable/remove operations.
|
||||
- Use registry store, validation, and atomic writes.
|
||||
- Keep auth/catalog cleanup deferred.
|
||||
- JSON-RPC/CLI calls work for targets that expose the registry-admin surface;
|
||||
local/static servers report unavailable and concrete MCP-backed
|
||||
`WorkflowServer` construction remains future work.
|
||||
|
||||
6. **Slice 6: Config Ownership Policy**
|
||||
- **Status: complete.**
|
||||
- Replace implicit config-shadowing with explicit `locked` / `seed`
|
||||
ownership policy.
|
||||
- `locked` config entries remain operator-owned and shadow/reject registry
|
||||
mutation for the same id.
|
||||
- `seed` config entries bootstrap missing store entries, then the store owns
|
||||
later admin changes.
|
||||
- Update startup merge diagnostics and registry admin payloads so users can
|
||||
see why a source is mutable or shadowed.
|
||||
|
||||
---
|
||||
|
||||
## Slice 2A: Generic Registry Mechanics
|
||||
|
||||
### Goal
|
||||
|
||||
Move generic registry store mechanics out of `wf_mcp` without pretending MCP
|
||||
source entries are generic.
|
||||
|
||||
### Target Shape
|
||||
|
||||
Create `src/wf_api/source_registry.py` with protocol-neutral mechanics only:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
from typing import Generic, Protocol, TypeVar
|
||||
from uuid import uuid4
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
|
||||
class SourceRegistryBaseModel(BaseModel):
|
||||
"""Base model for persisted source registry state; reject misspelled fields."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
|
||||
SOURCE_REGISTRY_ID_PATTERN = r"^[A-Za-z0-9_][A-Za-z0-9_.-]*$"
|
||||
|
||||
|
||||
def validate_source_registry_id(value: str) -> str:
|
||||
"""Validate ids that are safe as registry keys and filesystem path segments.
|
||||
|
||||
This helper intentionally does not parse provider/account meaning. MCP can
|
||||
layer stricter `parse_connection_id` validation on top while other future
|
||||
source families can reuse the safe-id rule.
|
||||
"""
|
||||
|
||||
if not re.fullmatch(SOURCE_REGISTRY_ID_PATTERN, value):
|
||||
raise ValueError(
|
||||
"source id must start with alphanumeric or underscore and contain "
|
||||
"only [A-Za-z0-9_.-]"
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def validate_unique_source_ids(entries: list[object]) -> None:
|
||||
"""Reject duplicate `id` fields without owning the entry model shape."""
|
||||
|
||||
seen: set[str] = set()
|
||||
for entry in entries:
|
||||
source_id = getattr(entry, "id", None)
|
||||
if not isinstance(source_id, str):
|
||||
raise ValueError("source registry entries must expose string id")
|
||||
if source_id in seen:
|
||||
raise ValueError(f"duplicate source id {source_id!r}")
|
||||
seen.add(source_id)
|
||||
|
||||
|
||||
RegistryT = TypeVar("RegistryT", bound=BaseModel)
|
||||
|
||||
|
||||
class SourceRegistryStore(Protocol[RegistryT]):
|
||||
def load_registry(self) -> RegistryT: ...
|
||||
|
||||
def save_registry(self, registry: RegistryT) -> None: ...
|
||||
|
||||
|
||||
class AtomicJsonRegistryStore(Generic[RegistryT]):
|
||||
"""Filesystem implementation for small desired-registry documents."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
root: Path,
|
||||
*,
|
||||
filename: str,
|
||||
registry_type: type[RegistryT],
|
||||
empty_factory: Callable[[], RegistryT],
|
||||
corrupt_label: str,
|
||||
) -> None:
|
||||
self.root = root
|
||||
self.filename = filename
|
||||
self.registry_type = registry_type
|
||||
self.empty_factory = empty_factory
|
||||
self.corrupt_label = corrupt_label
|
||||
self.root.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@property
|
||||
def path(self) -> Path:
|
||||
return self.root / self.filename
|
||||
|
||||
def load_registry(self) -> RegistryT:
|
||||
if not self.path.exists():
|
||||
return self.empty_factory()
|
||||
try:
|
||||
data = json.loads(self.path.read_text(encoding="utf-8"))
|
||||
except json.JSONDecodeError as exc:
|
||||
raise ValueError(f"{self.corrupt_label} is corrupted: {self.path}") from exc
|
||||
return self.registry_type.model_validate(data)
|
||||
|
||||
def save_registry(self, registry: RegistryT) -> None:
|
||||
validated = self.registry_type.model_validate(registry.model_dump(mode="json"))
|
||||
payload = json.dumps(validated.model_dump(mode="json"), indent=2)
|
||||
tmp_path = self.path.with_name(f"{self.path.name}.{uuid4().hex}.tmp")
|
||||
tmp_path.write_text(payload, encoding="utf-8")
|
||||
tmp_path.replace(self.path)
|
||||
```
|
||||
|
||||
Then `src/wf_mcp/source_registry.py` should import these generic pieces and keep:
|
||||
|
||||
- `StdioSourceTransport`
|
||||
- `HttpSourceTransport`
|
||||
- `SourceTransport`
|
||||
- `McpSourceRegistryEntry`
|
||||
- `SourceRegistryFile` or `McpSourceRegistryFile` as the MCP registry document
|
||||
model, using `validate_unique_source_ids(self.sources)`
|
||||
- `FileSourceRegistryStore` as the MCP concrete store wrapper around
|
||||
`AtomicJsonRegistryStore[SourceRegistryFile]`
|
||||
|
||||
### Tests
|
||||
|
||||
Move generic tests to `tests/wf_api/test_source_registry.py`:
|
||||
|
||||
- safe id validation accepts normal registry ids and rejects unsafe ids
|
||||
- duplicate id helper rejects repeated ids using a fake entry object
|
||||
- missing file returns empty generic registry
|
||||
- save/load round trip
|
||||
- corrupted JSON wraps as `ValueError`
|
||||
|
||||
Keep MCP tests in `tests/wf_mcp/test_source_registry.py`:
|
||||
|
||||
- MCP id validation
|
||||
- reserved id rejection
|
||||
- stdio/http transport models
|
||||
- `McpFileSourceRegistryStore` round trip
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- `wf_api.source_registry` imports no `wf_mcp`.
|
||||
- Existing `wf_mcp.source_registry` public behavior remains compatible.
|
||||
- Tests pass.
|
||||
- No startup/runtime behavior changes.
|
||||
|
||||
---
|
||||
|
||||
## Slice 2B: MCP Entry Conversion
|
||||
|
||||
### Goal
|
||||
|
||||
Add explicit conversion helpers so startup merge can convert registry entries
|
||||
into broker connection configs without duplicating field logic.
|
||||
|
||||
Status: complete. The implemented helper preserves entry metadata, `auth_ref`,
|
||||
profile, transport details, and a `source_registry` marker.
|
||||
|
||||
### New Helpers
|
||||
|
||||
In `src/wf_mcp/source_registry.py`:
|
||||
|
||||
```python
|
||||
def registry_entry_to_connection_config(
|
||||
entry: McpSourceRegistryEntry,
|
||||
) -> ConnectionConfig:
|
||||
return ConnectionConfig(
|
||||
id=entry.id,
|
||||
server=entry.provider,
|
||||
account=entry.account,
|
||||
enabled=entry.enabled,
|
||||
metadata={
|
||||
**entry.metadata,
|
||||
"auth_ref": entry.auth_ref,
|
||||
"profile": entry.profile,
|
||||
"transport": entry.transport.model_dump(mode="json"),
|
||||
"source_registry": True,
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
And optionally:
|
||||
|
||||
```python
|
||||
def connection_config_to_registry_entry(
|
||||
connection: ConnectionConfig,
|
||||
) -> McpSourceRegistryEntry | None:
|
||||
...
|
||||
```
|
||||
|
||||
Only add reverse conversion if an implementation needs it. Do not guess unknown
|
||||
transport metadata.
|
||||
|
||||
### Tests
|
||||
|
||||
- registry entry converts to `ConnectionConfig`
|
||||
- provider/account/profile/transport metadata are preserved
|
||||
- disabled entry creates disabled connection config
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- Conversion is explicit and tested.
|
||||
- No startup/runtime behavior changes yet.
|
||||
|
||||
---
|
||||
|
||||
## Slice 3: Startup Merge
|
||||
|
||||
### Goal
|
||||
|
||||
Load desired dynamic registry state during service construction and merge it
|
||||
with config-defined connections/sources.
|
||||
|
||||
### Merge Rules
|
||||
|
||||
1. Built-in reserved ids always win.
|
||||
2. Config-defined entries win over registry entries with the same id.
|
||||
3. Registry entries fill ids not present in config.
|
||||
|
||||
### Implementation Direction
|
||||
|
||||
- Broker config construction should create a `McpFileSourceRegistryStore`.
|
||||
- `WfMcpService` or `ConnectionService` should accept an optional registry store.
|
||||
- On startup/reload:
|
||||
- load config connections
|
||||
- load registry entries
|
||||
- convert registry entries to `ConnectionConfig`
|
||||
- merge with config precedence
|
||||
- register merged connections
|
||||
- emit an event for registry entries shadowed by config
|
||||
|
||||
### Tests
|
||||
|
||||
- absent registry preserves current config-only behavior
|
||||
- registry-only connection appears after service construction
|
||||
- config shadows same-id registry entry
|
||||
- invalid registry fails startup clearly
|
||||
- disabled registry source hydrates disabled
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- Existing legacy config behavior is unchanged when no registry file exists.
|
||||
- Dynamic registry entries persist across service recreation.
|
||||
- Shadowed entries do not silently override config.
|
||||
|
||||
---
|
||||
|
||||
## Slice 4: Read Desired Registry Through Admin
|
||||
|
||||
### Goal
|
||||
|
||||
Expose desired registry entries distinctly from observed source inventory.
|
||||
|
||||
Status: complete for API/transport/CLI plumbing. `WorkflowSourceRegistryApi`
|
||||
provides neutral read-only access. JSON-RPC methods
|
||||
`workflow.admin.source_registry.list` / `.inspect` are registered. CLI commands
|
||||
`wf admin registry list` / `wf admin registry inspect` are available for targets
|
||||
that expose the surface. Local/static servers report
|
||||
`source_registry_unavailable`. Concrete MCP-backed `WorkflowServer` construction
|
||||
remains future work.
|
||||
|
||||
### Why
|
||||
|
||||
`wf source list` currently reports runtime/observed source inventory. Registry
|
||||
entries are desired server-owned configuration. Users need both views when a
|
||||
source exists in registry but is disabled, shadowed, or not hydrated.
|
||||
|
||||
### Candidate Commands
|
||||
|
||||
- `wf admin registry list`
|
||||
- `wf admin registry inspect SOURCE_ID`
|
||||
|
||||
or:
|
||||
|
||||
- `wf source registry list`
|
||||
- `wf source registry inspect SOURCE_ID`
|
||||
|
||||
Pick one naming shape in the implementation plan.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- Desired registry view is not confused with observed source inventory.
|
||||
- Shadowed/disabled state is visible.
|
||||
- No mutation yet.
|
||||
|
||||
---
|
||||
|
||||
## Slice 5: Mutation Commands
|
||||
|
||||
### Goal
|
||||
|
||||
Add safe registry mutation.
|
||||
|
||||
Status: complete. Implementation:
|
||||
[2026-06-04 source registry mutations](../plans/2026-06-04-source-registry-mutations.md).
|
||||
|
||||
### Operations
|
||||
|
||||
- add source
|
||||
- update source
|
||||
- enable source
|
||||
- disable source
|
||||
- remove source
|
||||
|
||||
### Rules
|
||||
|
||||
- Validate full registry before commit.
|
||||
- Write atomically.
|
||||
- Do not mutate config files.
|
||||
- Do not delete auth/catalog files in v1.
|
||||
- Prefer disable over remove for sources referenced by deployments.
|
||||
- Optional live validation can be a flag, not required for save.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- RPC methods exist.
|
||||
- CLI commands exist.
|
||||
- Mutations persist across process restart for targets backed by a registry
|
||||
store; local/static servers report unavailable.
|
||||
- Validation errors are actionable.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- This is a multi-slice roadmap, not a single execution plan for all mutation work.
|
||||
- Slice 2A is the immediate next implementation target and resolves the location problem.
|
||||
- Startup merge is intentionally after generic/MCP split and conversion helpers.
|
||||
- Config ownership policy is intentionally after mutation commands, because it
|
||||
changes precedence semantics rather than introducing persistence.
|
||||
@@ -0,0 +1,351 @@
|
||||
# Source Registry Startup Merge Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Load persisted dynamic MCP source registry entries during broker/server construction and merge them with config-defined connections without changing behavior when the registry file is absent.
|
||||
|
||||
**Architecture:** Keep parsing/loading config separate from runtime hydration. `wf_mcp.source_registry` owns MCP registry entries and conversion to `ConnectionConfig`; `ConnectionService` owns connection reconciliation and source-catalog hydration; `build_service_from_config()` wires the default file store from `BrokerConfig.store_root`.
|
||||
|
||||
**Tech Stack:** Python 3.14, dataclasses, Pydantic v2, `wf_mcp.source_registry`, `ConnectionService`, `WfMcpService`, `build_service_from_config`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Preconditions
|
||||
|
||||
Completed before this plan:
|
||||
|
||||
- Generic registry mechanics exist in `wf_api.source_registry`.
|
||||
- MCP-specific registry models remain in `wf_mcp.source_registry`.
|
||||
- `registry_entry_to_connection_config()` exists and is tested.
|
||||
- No startup/runtime merge uses the registry yet.
|
||||
|
||||
## Merge Rules
|
||||
|
||||
1. Config-defined connections win over registry entries with the same id.
|
||||
2. Registry entries fill ids not present in config.
|
||||
3. Reserved ids remain rejected by existing `ConnectionService` validation.
|
||||
4. Shadowed registry entries must be visible through an event/diagnostic, not silently ignored.
|
||||
5. Missing `source_registry.json` must preserve current config-only behavior.
|
||||
|
||||
Do not mutate config files. Do not delete auth/catalog files.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add Merge Helper Tests First
|
||||
|
||||
- [ ] Add focused tests in `tests/wf_mcp/service/test_connection_service.py`.
|
||||
- [ ] Import:
|
||||
|
||||
```python
|
||||
from wf_mcp.source_registry import (
|
||||
FileSourceRegistryStore,
|
||||
McpSourceRegistryEntry,
|
||||
SourceRegistryFile,
|
||||
StdioSourceTransport,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] Add helper:
|
||||
|
||||
```python
|
||||
def _registry_entry(
|
||||
source_id: str = "demo.registry",
|
||||
*,
|
||||
enabled: bool = True,
|
||||
) -> McpSourceRegistryEntry:
|
||||
return McpSourceRegistryEntry(
|
||||
id=source_id,
|
||||
kind="mcp",
|
||||
enabled=enabled,
|
||||
provider="demo",
|
||||
account=source_id.rsplit(".", 1)[-1],
|
||||
transport=StdioSourceTransport(command="demo-server"),
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] Add `test_connection_service_sync_merges_registry_entries`.
|
||||
|
||||
Expected behavior:
|
||||
|
||||
```python
|
||||
store = FileSourceRegistryStore(tmp_path)
|
||||
store.save_registry(SourceRegistryFile(sources=[_registry_entry()]))
|
||||
|
||||
service.sync_connections_from_config(
|
||||
BrokerConfig(store_root=tmp_path, connections=[]),
|
||||
source_registry_store=store,
|
||||
)
|
||||
|
||||
assert [connection.id for connection in service.list_all()] == ["demo.registry"]
|
||||
assert "demo.registry" in catalog.capability_sources
|
||||
```
|
||||
|
||||
- [ ] Add `test_connection_service_sync_config_shadows_registry_entry`.
|
||||
|
||||
Expected behavior:
|
||||
|
||||
```python
|
||||
store.save_registry(SourceRegistryFile(sources=[_registry_entry("demo.same")]))
|
||||
service.sync_connections_from_config(
|
||||
BrokerConfig(
|
||||
store_root=tmp_path,
|
||||
connections=[
|
||||
ConnectionConfig(id="demo.same", server="demo", account="config"),
|
||||
],
|
||||
),
|
||||
source_registry_store=store,
|
||||
)
|
||||
|
||||
assert service.get("demo.same").account == "config"
|
||||
assert service.events.list_events()[-1].kind == "source_registry_ignored_config_shadow"
|
||||
assert service.events.list_events()[-1].connection_id == "demo.same"
|
||||
```
|
||||
|
||||
- [ ] Add `test_connection_service_sync_registry_disabled_entry_hydrates_disabled_source`.
|
||||
|
||||
Expected behavior:
|
||||
|
||||
```python
|
||||
store.save_registry(SourceRegistryFile(sources=[_registry_entry(enabled=False)]))
|
||||
service.sync_connections_from_config(
|
||||
BrokerConfig(store_root=tmp_path, connections=[]),
|
||||
source_registry_store=store,
|
||||
)
|
||||
|
||||
assert service.get("demo.registry").enabled is False
|
||||
assert catalog.capability_sources["demo.registry"].enabled is False
|
||||
```
|
||||
|
||||
- [ ] Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
|
||||
```
|
||||
|
||||
Expected: new tests fail until Task 2.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Implement Registry-Aware Connection Reconciliation
|
||||
|
||||
- [ ] Update `src/wf_mcp/broker/service/connection_service.py`.
|
||||
- [ ] Import:
|
||||
|
||||
```python
|
||||
from ...source_registry import (
|
||||
SourceRegistryStore,
|
||||
registry_entry_to_connection_config,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] Change `sync_connections_from_config` signature:
|
||||
|
||||
```python
|
||||
def sync_connections_from_config(
|
||||
self,
|
||||
config: BrokerConfig,
|
||||
*,
|
||||
source_registry_store: SourceRegistryStore | None = None,
|
||||
) -> None:
|
||||
```
|
||||
|
||||
- [ ] Build a merged connection list before existing remove/update/register logic:
|
||||
|
||||
```python
|
||||
connections = list(config.connections)
|
||||
config_ids = {connection.id for connection in connections}
|
||||
if source_registry_store is not None:
|
||||
registry = source_registry_store.load_registry()
|
||||
for entry in registry.sources:
|
||||
if entry.id in config_ids:
|
||||
self.events.record_kind(
|
||||
"source_registry_ignored_config_shadow",
|
||||
connection_id=entry.id,
|
||||
payload={
|
||||
"server": entry.provider,
|
||||
"account": entry.account,
|
||||
"reason": "config_connection_takes_precedence",
|
||||
},
|
||||
)
|
||||
continue
|
||||
connections.append(registry_entry_to_connection_config(entry))
|
||||
```
|
||||
|
||||
- [ ] Reuse the existing reconciliation logic against the merged `connections` list.
|
||||
- [ ] Keep `BrokerConfig` immutable; do not mutate `config.connections`.
|
||||
- [ ] Keep existing event behavior for registered/updated/removed connections.
|
||||
- [ ] Add a short comment near the merge explaining config precedence.
|
||||
|
||||
- [ ] Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
|
||||
```
|
||||
|
||||
Expected: all connection service tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Wire WfMcpService Facade
|
||||
|
||||
- [ ] Update `src/wf_mcp/broker/service/core.py`.
|
||||
- [ ] Import `SourceRegistryStore` from `wf_mcp.source_registry`.
|
||||
- [ ] Change facade method signature:
|
||||
|
||||
```python
|
||||
def sync_connections_from_config(
|
||||
self,
|
||||
config: BrokerConfig,
|
||||
*,
|
||||
source_registry_store: SourceRegistryStore | None = None,
|
||||
) -> None:
|
||||
self.connection_service.sync_connections_from_config(
|
||||
config,
|
||||
source_registry_store=source_registry_store,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] Add/adjust a facade test in `tests/wf_mcp/service/test_connection_service.py` proving `WfMcpService.sync_connections_from_config(..., source_registry_store=store)` delegates and hydrates a registry connection.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Wire Build From Config
|
||||
|
||||
- [ ] Update `src/wf_mcp/broker/config.py`.
|
||||
- [ ] Import `FileSourceRegistryStore`.
|
||||
- [ ] Construct the store in `build_service_from_config(config)`:
|
||||
|
||||
```python
|
||||
source_registry_store = FileSourceRegistryStore(config.store_root)
|
||||
```
|
||||
|
||||
- [ ] Replace the manual connection loop with one registry-aware sync call:
|
||||
|
||||
```python
|
||||
service.sync_connections_from_config(
|
||||
config,
|
||||
source_registry_store=source_registry_store,
|
||||
)
|
||||
for connection in service.connections.list_all():
|
||||
if connection.server not in service.adapters:
|
||||
service.register_adapter(connection.server, McpSdkAdapter())
|
||||
```
|
||||
|
||||
Important:
|
||||
|
||||
- Preserve adapter registration for config-defined and registry-defined connections.
|
||||
- Do not load the registry in `load_broker_config`; it should only parse config files.
|
||||
- Do not save the registry during startup.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Add Build-Service Integration Tests
|
||||
|
||||
- [ ] Update `tests/wf_mcp/test_broker_server.py` or `tests/wf_mcp/server/test_config.py`.
|
||||
- [ ] Add `test_build_service_from_config_loads_source_registry_entries`.
|
||||
|
||||
Setup:
|
||||
|
||||
```python
|
||||
config = BrokerConfig(store_root=tmp_path, connections=[])
|
||||
FileSourceRegistryStore(tmp_path).save_registry(
|
||||
SourceRegistryFile(sources=[_registry_entry("fixture.registry")])
|
||||
)
|
||||
service = build_service_from_config(config)
|
||||
```
|
||||
|
||||
Assertions:
|
||||
|
||||
```python
|
||||
assert service.connections.get("fixture.registry").server == "fixture"
|
||||
assert "fixture" in service.adapters
|
||||
assert "fixture.registry" in service.capability_sources
|
||||
```
|
||||
|
||||
- [ ] Add `test_build_service_from_config_config_shadows_registry`.
|
||||
|
||||
Assertions:
|
||||
|
||||
```python
|
||||
assert service.connections.get("fixture.same").account == "config"
|
||||
assert any(
|
||||
event.kind == "source_registry_ignored_config_shadow"
|
||||
and event.connection_id == "fixture.same"
|
||||
for event in service.list_events()
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] Add `test_build_service_from_config_absent_registry_preserves_existing_behavior` if no existing test already covers this. It can extend the current `test_build_service_from_config_registers_connections`.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Update Docs
|
||||
|
||||
- [ ] Update `docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md`.
|
||||
|
||||
Mark startup merge as complete or in-progress, depending on final status:
|
||||
|
||||
```md
|
||||
### Slice 3: Startup Merge
|
||||
|
||||
Status: complete. Broker/service construction now loads `source_registry.json`,
|
||||
merges config-defined connections with dynamic registry entries, preserves config
|
||||
precedence, and emits `source_registry_ignored_config_shadow` for shadowed
|
||||
registry entries.
|
||||
```
|
||||
|
||||
- [ ] Update `docs/current_roadmap.md`.
|
||||
|
||||
Replace the source-registry note with:
|
||||
|
||||
```md
|
||||
- Source registry startup merge is implemented: absent registry preserves
|
||||
config-only behavior, registry-only entries hydrate as dynamic connections,
|
||||
and config entries shadow same-id registry entries with an event.
|
||||
```
|
||||
|
||||
- [ ] Update `docs/superpowers/plans/2026-06-03-source-registry-next-slices.md`.
|
||||
|
||||
Mark Slice 2A and Slice 2B complete if not already done, and mark Slice 3 complete after implementation.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Verify
|
||||
|
||||
- [ ] Run focused tests:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/test_broker_server.py tests/wf_mcp/server/test_config.py -q
|
||||
```
|
||||
|
||||
- [ ] Run source registry tests:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_source_registry.py tests/wf_mcp/test_source_registry.py -q
|
||||
```
|
||||
|
||||
- [ ] Run quality checks:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp/source_registry.py src/wf_mcp/broker/config.py src/wf_mcp/broker/service/connection_service.py src/wf_mcp/broker/service/core.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/test_broker_server.py tests/wf_mcp/server/test_config.py
|
||||
uv run basedpyright --level error src/wf_mcp/source_registry.py src/wf_mcp/broker/config.py src/wf_mcp/broker/service/connection_service.py src/wf_mcp/broker/service/core.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/test_broker_server.py tests/wf_mcp/server/test_config.py
|
||||
```
|
||||
|
||||
- [ ] If focused tests pass, run the full suite if time permits:
|
||||
|
||||
```bash
|
||||
uv run pytest -q
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Missing registry file keeps existing config-only behavior.
|
||||
- Registry-only entries become registered connections and hydrated source catalog entries.
|
||||
- Config entries shadow same-id registry entries.
|
||||
- Shadowing emits `source_registry_ignored_config_shadow`.
|
||||
- Disabled registry entries hydrate disabled connection/source state.
|
||||
- `load_broker_config()` remains config-only parsing.
|
||||
- No registry mutation commands are added in this slice.
|
||||
- `wf_api` still imports no `wf_mcp`.
|
||||
@@ -0,0 +1,433 @@
|
||||
# Source Registry Store Slice 1 Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add validated source registry models and a filesystem store for desired server-owned source configuration, without wiring it into startup or mutation commands yet.
|
||||
|
||||
**Architecture:** The new registry is desired configuration state and stays separate from existing auth/catalog storage. Models live in `wf_mcp` for this first slice because current connection/source registry semantics are still MCP-provider-specific; the file-store interface is small enough to move later. Runtime merge, RPC/CLI mutation, and config reconciliation are explicitly deferred.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, existing `wf_mcp.connections.parse_connection_id`, existing `RESERVED_CONNECTION_IDS`, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- Typed `SourceRegistryFile` model.
|
||||
- Typed `McpSourceRegistryEntry` model.
|
||||
- Typed `StdioSourceTransport` and `HttpSourceTransport` models.
|
||||
- Duplicate id validation.
|
||||
- Reserved id validation.
|
||||
- ID validation using existing connection id rules.
|
||||
- `SourceRegistryStore` protocol.
|
||||
- `FileSourceRegistryStore` using `<store_root>/source_registry.json`.
|
||||
- Atomic filesystem writes.
|
||||
- Tests for load missing file, save/load round trip, validation errors, and path.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- Startup merge with config.
|
||||
- Runtime hydration from registry.
|
||||
- Mutating API/CLI/RPC commands.
|
||||
- Auth record changes.
|
||||
- Catalog deletion or cleanup.
|
||||
- SQL/remote stores.
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create `src/wf_mcp/source_registry.py`
|
||||
- Pydantic models and validation.
|
||||
- Protocol and file store.
|
||||
- No dependency on `WfMcpService`.
|
||||
|
||||
- Modify `src/wf_mcp/storage/__init__.py`
|
||||
- No change in this slice unless the implementor decides a store export is needed.
|
||||
- Prefer exporting from `wf_mcp.source_registry`, not overloading `wf_mcp.storage`.
|
||||
|
||||
- Create `tests/wf_mcp/test_source_registry.py`
|
||||
- Direct model/store tests.
|
||||
|
||||
- Modify `docs/current_roadmap.md`
|
||||
- Mark Slice 1 complete after implementation.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add failing source registry model tests
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/wf_mcp/test_source_registry.py`
|
||||
|
||||
- [ ] **Step 1: Write model validation tests**
|
||||
|
||||
Create `tests/wf_mcp/test_source_registry.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from wf_mcp.source_registry import (
|
||||
HttpSourceTransport,
|
||||
McpSourceRegistryEntry,
|
||||
SourceRegistryFile,
|
||||
StdioSourceTransport,
|
||||
)
|
||||
|
||||
|
||||
def _entry(source_id: str = "github.work") -> McpSourceRegistryEntry:
|
||||
return McpSourceRegistryEntry(
|
||||
id=source_id,
|
||||
provider="github",
|
||||
account="work",
|
||||
transport=StdioSourceTransport(
|
||||
command="npx",
|
||||
args=("-y", "@modelcontextprotocol/server-github"),
|
||||
env={"GITHUB_TOKEN": "${GITHUB_TOKEN}"},
|
||||
),
|
||||
auth_ref=source_id,
|
||||
metadata={"purpose": "tests"},
|
||||
)
|
||||
|
||||
|
||||
def test_source_registry_entry_keeps_identity_and_transport_structural() -> None:
|
||||
entry = _entry()
|
||||
|
||||
assert entry.id == "github.work"
|
||||
assert entry.provider == "github"
|
||||
assert entry.account == "work"
|
||||
assert entry.profile is None
|
||||
assert entry.transport.kind == "stdio"
|
||||
assert entry.transport.command == "npx"
|
||||
assert entry.auth_ref == "github.work"
|
||||
|
||||
|
||||
def test_source_registry_accepts_http_transport() -> None:
|
||||
entry = McpSourceRegistryEntry(
|
||||
id="github.http",
|
||||
provider="github",
|
||||
account="work",
|
||||
transport=HttpSourceTransport(url="https://example.test/mcp"),
|
||||
)
|
||||
|
||||
assert entry.transport.kind == "http"
|
||||
assert str(entry.transport.url) == "https://example.test/mcp"
|
||||
|
||||
|
||||
def test_source_registry_rejects_duplicate_ids() -> None:
|
||||
with pytest.raises(ValueError, match="duplicate source id 'github.work'"):
|
||||
SourceRegistryFile(sources=[_entry("github.work"), _entry("github.work")])
|
||||
|
||||
|
||||
def test_source_registry_rejects_reserved_ids() -> None:
|
||||
with pytest.raises(ValueError, match="reserved"):
|
||||
_entry("wf.admin")
|
||||
|
||||
|
||||
def test_source_registry_rejects_unsafe_ids() -> None:
|
||||
with pytest.raises(ValueError, match="connection id"):
|
||||
_entry("../bad")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests and verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_source_registry.py -q
|
||||
```
|
||||
|
||||
Expected: FAIL because `wf_mcp.source_registry` does not exist yet.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Implement source registry models
|
||||
|
||||
**Files:**
|
||||
- Create: `src/wf_mcp/source_registry.py`
|
||||
- Test: `tests/wf_mcp/test_source_registry.py`
|
||||
|
||||
- [ ] **Step 1: Add models and validators**
|
||||
|
||||
Create `src/wf_mcp/source_registry.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import Mapping, Sequence
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Literal, Protocol
|
||||
|
||||
from pydantic import AnyHttpUrl, BaseModel, ConfigDict, Field, field_validator, model_validator
|
||||
|
||||
from .connections import parse_connection_id
|
||||
from .shared.names import RESERVED_CONNECTION_IDS
|
||||
|
||||
|
||||
class SourceRegistryModel(BaseModel):
|
||||
"""Base model for persisted source registry state; reject misspelled fields."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
|
||||
class StdioSourceTransport(SourceRegistryModel):
|
||||
kind: Literal["stdio"] = "stdio"
|
||||
command: str = Field(min_length=1)
|
||||
args: tuple[str, ...] = ()
|
||||
env: dict[str, str] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class HttpSourceTransport(SourceRegistryModel):
|
||||
kind: Literal["http"] = "http"
|
||||
url: AnyHttpUrl
|
||||
headers: dict[str, str] = Field(default_factory=dict)
|
||||
|
||||
|
||||
SourceTransport = Annotated[
|
||||
StdioSourceTransport | HttpSourceTransport,
|
||||
Field(discriminator="kind"),
|
||||
]
|
||||
|
||||
|
||||
class McpSourceRegistryEntry(SourceRegistryModel):
|
||||
"""Desired MCP source configuration persisted by server-owned mutation."""
|
||||
|
||||
id: str
|
||||
kind: Literal["mcp"] = "mcp"
|
||||
enabled: bool = True
|
||||
provider: str = Field(min_length=1)
|
||||
account: str = Field(min_length=1)
|
||||
profile: str | None = None
|
||||
transport: SourceTransport
|
||||
auth_ref: str | None = None
|
||||
metadata: dict[str, object] = Field(default_factory=dict)
|
||||
|
||||
@field_validator("id")
|
||||
@classmethod
|
||||
def validate_id(cls, value: str) -> str:
|
||||
parse_connection_id(value)
|
||||
if value in RESERVED_CONNECTION_IDS:
|
||||
raise ValueError(f"source id {value!r} is reserved")
|
||||
return value
|
||||
|
||||
|
||||
class SourceRegistryFile(SourceRegistryModel):
|
||||
version: Literal[1] = 1
|
||||
sources: list[McpSourceRegistryEntry] = Field(default_factory=list)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_unique_source_ids(self) -> SourceRegistryFile:
|
||||
seen: set[str] = set()
|
||||
for source in self.sources:
|
||||
if source.id in seen:
|
||||
raise ValueError(f"duplicate source id {source.id!r}")
|
||||
seen.add(source.id)
|
||||
return self
|
||||
|
||||
def source_map(self) -> dict[str, McpSourceRegistryEntry]:
|
||||
return {source.id: source for source in self.sources}
|
||||
|
||||
|
||||
class SourceRegistryStore(Protocol):
|
||||
"""Persistence boundary for desired server-owned source configuration."""
|
||||
|
||||
def load_registry(self) -> SourceRegistryFile:
|
||||
"""Return the stored registry, or an empty registry when absent."""
|
||||
...
|
||||
|
||||
def save_registry(self, registry: SourceRegistryFile) -> None:
|
||||
"""Persist one validated registry atomically."""
|
||||
...
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run model tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_source_registry.py -q
|
||||
```
|
||||
|
||||
Expected: model tests PASS except store tests are not added yet.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Add failing file store tests
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/wf_mcp/test_source_registry.py`
|
||||
|
||||
- [ ] **Step 1: Append file store tests**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
from wf_mcp.source_registry import FileSourceRegistryStore
|
||||
|
||||
|
||||
def test_file_source_registry_store_loads_empty_registry_when_missing(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
store = FileSourceRegistryStore(tmp_path)
|
||||
|
||||
registry = store.load_registry()
|
||||
|
||||
assert registry.version == 1
|
||||
assert registry.sources == []
|
||||
assert store.path == tmp_path / "source_registry.json"
|
||||
|
||||
|
||||
def test_file_source_registry_store_round_trips_registry(tmp_path: Path) -> None:
|
||||
store = FileSourceRegistryStore(tmp_path)
|
||||
registry = SourceRegistryFile(sources=[_entry("github.work")])
|
||||
|
||||
store.save_registry(registry)
|
||||
loaded = store.load_registry()
|
||||
|
||||
assert loaded.source_map()["github.work"].provider == "github"
|
||||
assert loaded.source_map()["github.work"].transport.kind == "stdio"
|
||||
|
||||
|
||||
def test_file_source_registry_store_validates_loaded_registry(tmp_path: Path) -> None:
|
||||
store = FileSourceRegistryStore(tmp_path)
|
||||
store.path.write_text(
|
||||
'{"version": 1, "sources": [{"id": "wf.admin", "provider": "wf", '
|
||||
'"account": "admin", "transport": {"kind": "stdio", "command": "x"}}]}',
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="reserved"):
|
||||
store.load_registry()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests and verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_source_registry.py -q
|
||||
```
|
||||
|
||||
Expected: FAIL because `FileSourceRegistryStore` does not exist yet.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Implement file store
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_mcp/source_registry.py`
|
||||
- Test: `tests/wf_mcp/test_source_registry.py`
|
||||
|
||||
- [ ] **Step 1: Add file store implementation**
|
||||
|
||||
Append to `src/wf_mcp/source_registry.py`:
|
||||
|
||||
```python
|
||||
class FileSourceRegistryStore:
|
||||
"""Filesystem implementation for desired source registry state."""
|
||||
|
||||
def __init__(self, root: Path) -> None:
|
||||
self.root = root
|
||||
self.root.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@property
|
||||
def path(self) -> Path:
|
||||
return self.root / "source_registry.json"
|
||||
|
||||
def load_registry(self) -> SourceRegistryFile:
|
||||
if not self.path.exists():
|
||||
return SourceRegistryFile()
|
||||
data = json.loads(self.path.read_text(encoding="utf-8"))
|
||||
return SourceRegistryFile.model_validate(data)
|
||||
|
||||
def save_registry(self, registry: SourceRegistryFile) -> None:
|
||||
# Validate again at the store boundary so callers cannot persist stale or
|
||||
# partially constructed model-like objects after mutation.
|
||||
validated = SourceRegistryFile.model_validate(
|
||||
registry.model_dump(mode="json")
|
||||
)
|
||||
payload = json.dumps(validated.model_dump(mode="json"), indent=2)
|
||||
tmp_path = self.path.with_name(f"{self.path.name}.tmp")
|
||||
tmp_path.write_text(payload, encoding="utf-8")
|
||||
tmp_path.replace(self.path)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add `__all__`**
|
||||
|
||||
At the bottom of `src/wf_mcp/source_registry.py`, add:
|
||||
|
||||
```python
|
||||
__all__ = [
|
||||
"FileSourceRegistryStore",
|
||||
"HttpSourceTransport",
|
||||
"McpSourceRegistryEntry",
|
||||
"SourceRegistryFile",
|
||||
"SourceRegistryStore",
|
||||
"SourceTransport",
|
||||
"StdioSourceTransport",
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_source_registry.py -q
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Documentation and verification
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Update roadmap**
|
||||
|
||||
In `docs/current_roadmap.md`, near the store-backed source registry note, add:
|
||||
|
||||
```markdown
|
||||
- First source registry implementation slice complete: validated registry
|
||||
models plus `FileSourceRegistryStore` exist, but startup merge and mutation
|
||||
commands are still deferred.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_source_registry.py -q
|
||||
uv run ruff check src/wf_mcp/source_registry.py tests/wf_mcp/test_source_registry.py
|
||||
uv run ruff format --check src/wf_mcp/source_registry.py tests/wf_mcp/test_source_registry.py
|
||||
uv run basedpyright --level error src/wf_mcp/source_registry.py tests/wf_mcp/test_source_registry.py
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- pytest PASS
|
||||
- ruff check PASS
|
||||
- ruff format PASS
|
||||
- basedpyright 0 errors
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add src/wf_mcp/source_registry.py tests/wf_mcp/test_source_registry.py docs/current_roadmap.md
|
||||
git commit -m "feat: add source registry file store"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: Slice 1 only is covered: models, validation, file store, tests, docs.
|
||||
- Placeholder scan: no TBD/TODO placeholders.
|
||||
- Type consistency: `SourceRegistryFile`, `McpSourceRegistryEntry`, `SourceRegistryStore`, and `FileSourceRegistryStore` names are consistent.
|
||||
- Deferred work is explicit: startup merge, runtime hydration, RPC/CLI mutation, and auth/catalog cleanup are not part of this slice.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,220 @@
|
||||
# CodeRabbit Cleanup Follow-Ups Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Resolve non-blocking cleanup findings from the June 4 CodeRabbit review without mixing broad refactors into the focused correctness patch.
|
||||
|
||||
**Architecture:** Keep behavior unchanged unless a task explicitly says otherwise. Prefer documentation, small helpers, and narrowly scoped tests over cross-package refactors. Do not move modules or rename public APIs in this follow-up.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, Typer, FastAPI JSON-RPC, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
This plan intentionally excludes the already-fixed review items:
|
||||
|
||||
- CLI config exception tuple syntax in `src/wf_cli/context.py`
|
||||
- CLI source-registry file JSON object validation and exception chaining
|
||||
- RPC method module structure-test coverage
|
||||
- stale long-lived API and CLI/API alignment doc statuses
|
||||
- typed `tmp_path` annotations in `tests/wf_config/test_config_models.py`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Document Source-Registry Mutation Asymmetry
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/superpowers/plans/2026-06-04-source-registry-mutations.md`
|
||||
|
||||
- [ ] **Step 1: Add rationale under shadow handling**
|
||||
|
||||
Find the section mentioning:
|
||||
|
||||
```markdown
|
||||
A future `allow_shadow` flag can relax add; do not add it in this slice.
|
||||
```
|
||||
|
||||
Append:
|
||||
|
||||
```markdown
|
||||
Rationale: `add` rejects config-shadowed ids to prevent silent no-ops: adding a
|
||||
registry entry that cannot activate while config owns the same id. Existing
|
||||
shadowed registry entries can still be updated, enabled, disabled, or removed so
|
||||
operators can prepare store state for config removal or future `seed` ownership
|
||||
policy.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify doc diff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git diff -- docs/superpowers/plans/2026-06-04-source-registry-mutations.md
|
||||
```
|
||||
|
||||
Expected: only the rationale paragraph changed.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Clarify RPC Target Selection Precedence
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/superpowers/plans/2026-06-03-workflow-config-and-rpc-cli-target.md`
|
||||
|
||||
- [ ] **Step 1: Add precedence table near the plan introduction**
|
||||
|
||||
Add:
|
||||
|
||||
```markdown
|
||||
### Target Selection Precedence
|
||||
|
||||
1. `--url` CLI override selects an RPC HTTP target.
|
||||
2. `--local` CLI override selects the in-process local target.
|
||||
3. Config file `client.target` selects the configured target.
|
||||
4. Missing target config defaults to local.
|
||||
|
||||
CLI overrides intentionally win over config so one-off diagnostics can point at
|
||||
a different server without editing the config file.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify no code references are changed**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git diff --stat
|
||||
```
|
||||
|
||||
Expected: only the plan document is changed by this task.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Extract Source-Registry RPC Availability Helper
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_transport_rpc_http/methods_source_registry.py`
|
||||
- Test: `tests/wf_transport_rpc_http/test_source_registry_rpc.py`
|
||||
|
||||
- [ ] **Step 1: Add a helper**
|
||||
|
||||
In `src/wf_transport_rpc_http/methods_source_registry.py`, add:
|
||||
|
||||
```python
|
||||
def _require_source_registry_admin(
|
||||
server: WorkflowServer,
|
||||
*,
|
||||
operation: str,
|
||||
) -> WorkflowSourceRegistrySurface:
|
||||
admin = server.source_registry_admin
|
||||
if admin is None:
|
||||
raise WorkflowRpcError(
|
||||
data={
|
||||
"code": "source_registry_unavailable",
|
||||
"message": (
|
||||
f"source registry admin {operation} are not available "
|
||||
"for this server"
|
||||
),
|
||||
}
|
||||
)
|
||||
return admin
|
||||
```
|
||||
|
||||
Import `WorkflowSourceRegistrySurface` from `wf_api` if needed.
|
||||
|
||||
- [ ] **Step 2: Replace repeated `None` checks**
|
||||
|
||||
Use:
|
||||
|
||||
```python
|
||||
admin = _require_source_registry_admin(server, operation="reads")
|
||||
```
|
||||
|
||||
for list/inspect, and:
|
||||
|
||||
```python
|
||||
admin = _require_source_registry_admin(server, operation="mutations")
|
||||
```
|
||||
|
||||
for add/update/enable/disable/remove.
|
||||
|
||||
- [ ] **Step 3: Run source-registry RPC tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_source_registry_rpc.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Clarify Chronological Event Ordering
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_api/admin.py`
|
||||
- Test: existing admin API tests if present
|
||||
|
||||
- [ ] **Step 1: Add a comment above `list_events`**
|
||||
|
||||
Add a short comment/docstring note near `WorkflowAdminApi.list_events`:
|
||||
|
||||
```python
|
||||
# Preserve provider order for events; event providers are expected to return
|
||||
# chronological order and callers may rely on that ordering for diagnostics.
|
||||
```
|
||||
|
||||
Do not sort event payloads in this task.
|
||||
|
||||
- [ ] **Step 2: Run admin API tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api -q
|
||||
```
|
||||
|
||||
Expected: all `wf_api` tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Record Shared ID Pattern Follow-Up
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Add a small platform cleanup bullet**
|
||||
|
||||
Under the platform cleanup/architecture section, add:
|
||||
|
||||
```markdown
|
||||
- Cleanup candidate: consolidate store/source registry id validation patterns
|
||||
(`SOURCE_REGISTRY_ID_PATTERN`, `STORE_ID_PATTERN`) only after another package
|
||||
needs the same rule. Today they intentionally stay close to their stores.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify docs only**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git diff -- docs/current_roadmap.md
|
||||
```
|
||||
|
||||
Expected: only the cleanup bullet changed.
|
||||
|
||||
---
|
||||
|
||||
## Final Verification
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_source_registry_rpc.py tests/wf_api -q
|
||||
uv run ruff check src/wf_transport_rpc_http/methods_source_registry.py src/wf_api/admin.py docs/superpowers/plans/2026-06-04-source-registry-mutations.md docs/superpowers/plans/2026-06-03-workflow-config-and-rpc-cli-target.md docs/current_roadmap.md
|
||||
uv run basedpyright --level error src/wf_transport_rpc_http/methods_source_registry.py src/wf_api/admin.py
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: pytest exits 0, ruff exits 0, basedpyright exits 0, and `git diff --check` reports no whitespace errors.
|
||||
@@ -0,0 +1,722 @@
|
||||
# Source Config Ownership Policy Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace implicit "config always shadows registry" behavior with explicit config ownership policy for MCP source entries: `locked` vs `seed`.
|
||||
|
||||
**Architecture:** Keep v1 safety by making `locked` the default for existing config connections. Add `seed` as an explicit opt-in that materializes a missing store entry, then lets the store-backed registry own future admin changes. Do this in the MCP broker config path first; neutral `wf_config.server.sources` only models built-in stdlib sources today and should not grow MCP transport config in this slice.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, wf_mcp broker config/service, wf_mcp source registry, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Current Context
|
||||
|
||||
Relevant current behavior:
|
||||
|
||||
- `src/wf_mcp/broker/service/connection_service.py::ConnectionService.sync_connections_from_config()` loads config connections first, then ignores same-id registry entries.
|
||||
- Same-id registry entries emit `source_registry_ignored_config_shadow`.
|
||||
- `src/wf_mcp/source_registry.py::McpSourceRegistryEntry` is the persisted desired source entry.
|
||||
- `src/wf_mcp/models.py::ConnectionConfig` is the broker runtime config model used by legacy MCP config.
|
||||
- `src/wf_mcp/control.py::BrokerConfigFile.to_runtime()` converts config-file connection declarations into `ConnectionConfig`.
|
||||
|
||||
Intended new policy:
|
||||
|
||||
- `locked`: config owns the source id. Same-id registry entries remain shadowed.
|
||||
- `seed`: config bootstraps a missing registry entry. Once the registry entry exists, registry owns later runtime state for that id.
|
||||
- Backward compatibility: existing config with no policy behaves as `locked`.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- Do not add MCP source transports to neutral `wf_config.server.sources`.
|
||||
- Do not add auth/catalog cleanup.
|
||||
- Do not implement live remount without reload.
|
||||
- Do not alter built-in reserved ids (`wf.std`, `wf.recipes`, `wf.admin`).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Policy Field to Broker Runtime and Config Models
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_mcp/models.py`
|
||||
- Modify: `src/wf_mcp/control.py`
|
||||
- Test: `tests/wf_mcp/test_broker_config.py` or nearest existing broker config test file
|
||||
|
||||
- [ ] **Step 1: Inspect existing model shape**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
rg -n "class ConnectionConfig|class .*Connection" src/wf_mcp/models.py src/wf_mcp/control.py tests/wf_mcp -g '*.py'
|
||||
```
|
||||
|
||||
Expected: locate `ConnectionConfig` and the Pydantic config-file model that constructs it.
|
||||
|
||||
- [ ] **Step 2: Add a policy type and field to runtime config**
|
||||
|
||||
In `src/wf_mcp/models.py`, add a type alias near `ConnectionConfig`:
|
||||
|
||||
```python
|
||||
SourceConfigOwnership = Literal["locked", "seed"]
|
||||
```
|
||||
|
||||
Add to `ConnectionConfig`:
|
||||
|
||||
```python
|
||||
source_config_ownership: SourceConfigOwnership = "locked"
|
||||
```
|
||||
|
||||
If `Literal` is not imported, import it from `typing`.
|
||||
|
||||
- [ ] **Step 3: Add field to config-file connection model**
|
||||
|
||||
In `src/wf_mcp/control.py`, add the same field to the config-file connection model:
|
||||
|
||||
```python
|
||||
source_config_ownership: SourceConfigOwnership = "locked"
|
||||
```
|
||||
|
||||
When converting to `ConnectionConfig`, pass:
|
||||
|
||||
```python
|
||||
source_config_ownership=self.source_config_ownership
|
||||
```
|
||||
|
||||
If the config-file model is named differently, update the exact class that owns `id`, `server`, and `account`.
|
||||
|
||||
- [ ] **Step 4: Add config parsing tests**
|
||||
|
||||
In the existing broker config test file, add:
|
||||
|
||||
```python
|
||||
def test_broker_config_connection_defaults_to_locked(tmp_path: Path) -> None:
|
||||
config_path = tmp_path / "wf_mcp.config.json"
|
||||
config_path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"store_root": str(tmp_path / "store"),
|
||||
"connections": [
|
||||
{"id": "demo.default", "server": "demo", "account": "default"}
|
||||
],
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
config = load_broker_config(config_path)
|
||||
|
||||
assert config.connections[0].source_config_ownership == "locked"
|
||||
|
||||
|
||||
def test_broker_config_connection_accepts_seed_policy(tmp_path: Path) -> None:
|
||||
config_path = tmp_path / "wf_mcp.config.json"
|
||||
config_path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"store_root": str(tmp_path / "store"),
|
||||
"connections": [
|
||||
{
|
||||
"id": "demo.default",
|
||||
"server": "demo",
|
||||
"account": "default",
|
||||
"source_config_ownership": "seed",
|
||||
}
|
||||
],
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
config = load_broker_config(config_path)
|
||||
|
||||
assert config.connections[0].source_config_ownership == "seed"
|
||||
```
|
||||
|
||||
Import `json`, `Path`, and `load_broker_config` as needed.
|
||||
|
||||
- [ ] **Step 5: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_broker_config.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Convert Seed Config Connections to Registry Entries
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_mcp/source_registry.py`
|
||||
- Test: `tests/wf_mcp/test_source_registry.py`
|
||||
|
||||
- [ ] **Step 1: Add conversion helper**
|
||||
|
||||
In `src/wf_mcp/source_registry.py`, add:
|
||||
|
||||
```python
|
||||
def connection_config_to_registry_entry(
|
||||
connection: ConnectionConfig,
|
||||
) -> McpSourceRegistryEntry:
|
||||
"""Materialize a seed config connection into persisted registry state.
|
||||
|
||||
Seed config is bootstrap-only. The registry entry must carry enough source
|
||||
identity to become the future desired-state owner after first startup.
|
||||
"""
|
||||
transport = connection.metadata.get("transport")
|
||||
if not isinstance(transport, dict):
|
||||
raise ValueError(
|
||||
f"seed connection {connection.id!r} requires metadata.transport"
|
||||
)
|
||||
profile = connection.metadata.get("profile")
|
||||
auth_ref = connection.metadata.get("auth_ref")
|
||||
return McpSourceRegistryEntry.model_validate(
|
||||
{
|
||||
"id": connection.id,
|
||||
"enabled": connection.enabled,
|
||||
"provider": connection.server,
|
||||
"account": connection.account,
|
||||
"profile": profile if isinstance(profile, str) else None,
|
||||
"transport": transport,
|
||||
"auth_ref": auth_ref if isinstance(auth_ref, str) else None,
|
||||
"metadata": {
|
||||
key: value
|
||||
for key, value in connection.metadata.items()
|
||||
if key not in {"transport", "profile", "auth_ref", "source_registry"}
|
||||
},
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
Also export it in `__all__`.
|
||||
|
||||
- [ ] **Step 2: Add conversion test**
|
||||
|
||||
In `tests/wf_mcp/test_source_registry.py`, add:
|
||||
|
||||
```python
|
||||
def test_connection_config_to_registry_entry_preserves_transport_metadata() -> None:
|
||||
connection = ConnectionConfig(
|
||||
id="github.work",
|
||||
server="github",
|
||||
account="work",
|
||||
enabled=False,
|
||||
metadata={
|
||||
"transport": {"kind": "stdio", "command": "npx", "args": ["server"]},
|
||||
"profile": "corp",
|
||||
"auth_ref": "secret://github/work",
|
||||
"region": "us",
|
||||
},
|
||||
)
|
||||
|
||||
entry = connection_config_to_registry_entry(connection)
|
||||
|
||||
assert entry.id == "github.work"
|
||||
assert entry.provider == "github"
|
||||
assert entry.account == "work"
|
||||
assert entry.enabled is False
|
||||
assert entry.profile == "corp"
|
||||
assert entry.auth_ref == "secret://github/work"
|
||||
assert entry.transport.kind == "stdio"
|
||||
assert entry.metadata["region"] == "us"
|
||||
```
|
||||
|
||||
Add imports for `ConnectionConfig` and `connection_config_to_registry_entry`.
|
||||
|
||||
- [ ] **Step 3: Add missing transport failure test**
|
||||
|
||||
```python
|
||||
def test_connection_config_to_registry_entry_requires_transport_metadata() -> None:
|
||||
connection = ConnectionConfig(id="github.work", server="github", account="work")
|
||||
|
||||
with pytest.raises(ValueError, match="requires metadata.transport"):
|
||||
connection_config_to_registry_entry(connection)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run source registry tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_source_registry.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Implement Locked vs Seed Startup Merge
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_mcp/broker/service/connection_service.py`
|
||||
- Test: `tests/wf_mcp/service/test_connection_service.py`
|
||||
|
||||
- [ ] **Step 1: Update imports**
|
||||
|
||||
In `src/wf_mcp/broker/service/connection_service.py`, import:
|
||||
|
||||
```python
|
||||
from ...source_registry import (
|
||||
SourceRegistryFile,
|
||||
SourceRegistryStore,
|
||||
connection_config_to_registry_entry,
|
||||
registry_entry_to_connection_config,
|
||||
)
|
||||
```
|
||||
|
||||
If `SourceRegistryStore` and `registry_entry_to_connection_config` are already imported, extend the existing import.
|
||||
|
||||
- [ ] **Step 2: Rewrite registry merge block**
|
||||
|
||||
Inside `sync_connections_from_config`, replace the current registry loop with this logic:
|
||||
|
||||
```python
|
||||
connections = list(config.connections)
|
||||
config_by_id = {connection.id: connection for connection in connections}
|
||||
registry_entries = {}
|
||||
registry_changed = False
|
||||
|
||||
if source_registry_store is not None:
|
||||
registry = source_registry_store.load_registry()
|
||||
registry_entries = registry.source_map()
|
||||
|
||||
for connection in connections:
|
||||
if connection.source_config_ownership != "seed":
|
||||
continue
|
||||
if connection.id in registry_entries:
|
||||
continue
|
||||
seeded = connection_config_to_registry_entry(connection)
|
||||
registry_entries[seeded.id] = seeded
|
||||
registry_changed = True
|
||||
self.events.record_kind(
|
||||
"source_registry_seeded_from_config",
|
||||
connection_id=seeded.id,
|
||||
payload={"server": seeded.provider, "account": seeded.account},
|
||||
)
|
||||
|
||||
if registry_changed:
|
||||
source_registry_store.save_registry(
|
||||
SourceRegistryFile(sources=list(registry_entries.values()))
|
||||
)
|
||||
|
||||
merged_connections: list[ConnectionConfig] = []
|
||||
for connection in connections:
|
||||
registry_entry = registry_entries.get(connection.id)
|
||||
if connection.source_config_ownership == "seed" and registry_entry is not None:
|
||||
merged_connections.append(registry_entry_to_connection_config(registry_entry))
|
||||
continue
|
||||
merged_connections.append(connection)
|
||||
|
||||
merged_ids = {connection.id for connection in merged_connections}
|
||||
for entry in registry_entries.values():
|
||||
config_connection = config_by_id.get(entry.id)
|
||||
if config_connection is not None:
|
||||
if config_connection.source_config_ownership == "locked":
|
||||
self.events.record_kind(
|
||||
"source_registry_ignored_config_shadow",
|
||||
connection_id=entry.id,
|
||||
payload={
|
||||
"server": entry.provider,
|
||||
"account": entry.account,
|
||||
"reason": "locked_config_connection_takes_precedence",
|
||||
},
|
||||
)
|
||||
continue
|
||||
if entry.id not in merged_ids:
|
||||
merged_connections.append(registry_entry_to_connection_config(entry))
|
||||
|
||||
connections = merged_connections
|
||||
```
|
||||
|
||||
Important notes:
|
||||
|
||||
- `locked` keeps current behavior.
|
||||
- `seed` with no store entry writes a store entry, then uses that store entry.
|
||||
- `seed` with an existing store entry uses the store entry, not config.
|
||||
- Registry-only entries still hydrate as before.
|
||||
|
||||
- [ ] **Step 3: Add locked behavior regression test**
|
||||
|
||||
In `tests/wf_mcp/service/test_connection_service.py`, keep or add:
|
||||
|
||||
```python
|
||||
def test_connection_service_sync_locked_config_shadows_registry_entry() -> None:
|
||||
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
|
||||
catalog = _source_catalog(service)
|
||||
store = FileSourceRegistryStore(local_temp_root() / "locked_shadow")
|
||||
store.save_registry(
|
||||
SourceRegistryFile(
|
||||
sources=[
|
||||
_registry_entry(
|
||||
"demo.default",
|
||||
provider="registry",
|
||||
account="stored",
|
||||
)
|
||||
]
|
||||
)
|
||||
)
|
||||
config = BrokerConfig(
|
||||
store_root=local_temp_root(),
|
||||
connections=[
|
||||
ConnectionConfig(
|
||||
id="demo.default",
|
||||
server="config",
|
||||
account="locked",
|
||||
source_config_ownership="locked",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
service.bind_source_catalog(catalog)
|
||||
service.sync_connections_from_config(config, source_registry_store=store)
|
||||
|
||||
connection = service.get("demo.default")
|
||||
assert connection.server == "config"
|
||||
assert connection.account == "locked"
|
||||
assert any(
|
||||
event.kind == "source_registry_ignored_config_shadow"
|
||||
for event in service.events.list_events()
|
||||
)
|
||||
```
|
||||
|
||||
Use existing helpers if names differ.
|
||||
|
||||
- [ ] **Step 4: Add seed materialization test**
|
||||
|
||||
```python
|
||||
def test_connection_service_sync_seed_config_materializes_registry_entry() -> None:
|
||||
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
|
||||
catalog = _source_catalog(service)
|
||||
store_root = local_temp_root() / "seed_materialized"
|
||||
store = FileSourceRegistryStore(store_root)
|
||||
config = BrokerConfig(
|
||||
store_root=local_temp_root(),
|
||||
connections=[
|
||||
ConnectionConfig(
|
||||
id="demo.default",
|
||||
server="demo",
|
||||
account="default",
|
||||
metadata={"transport": {"kind": "stdio", "command": "demo-server"}},
|
||||
source_config_ownership="seed",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
service.bind_source_catalog(catalog)
|
||||
service.sync_connections_from_config(config, source_registry_store=store)
|
||||
|
||||
registry = store.load_registry()
|
||||
assert registry.sources[0].id == "demo.default"
|
||||
assert registry.sources[0].provider == "demo"
|
||||
assert service.get("demo.default").metadata["source_registry"] is True
|
||||
assert any(
|
||||
event.kind == "source_registry_seeded_from_config"
|
||||
for event in service.events.list_events()
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Add seed existing-store-wins test**
|
||||
|
||||
```python
|
||||
def test_connection_service_sync_seed_existing_registry_entry_wins() -> None:
|
||||
service = ConnectionService(events=BrokerEventRecorder(EventBus()))
|
||||
catalog = _source_catalog(service)
|
||||
store = FileSourceRegistryStore(local_temp_root() / "seed_existing")
|
||||
store.save_registry(
|
||||
SourceRegistryFile(
|
||||
sources=[
|
||||
_registry_entry(
|
||||
"demo.default",
|
||||
provider="registry",
|
||||
account="stored",
|
||||
)
|
||||
]
|
||||
)
|
||||
)
|
||||
config = BrokerConfig(
|
||||
store_root=local_temp_root(),
|
||||
connections=[
|
||||
ConnectionConfig(
|
||||
id="demo.default",
|
||||
server="config",
|
||||
account="seed",
|
||||
metadata={"transport": {"kind": "stdio", "command": "config-server"}},
|
||||
source_config_ownership="seed",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
service.bind_source_catalog(catalog)
|
||||
service.sync_connections_from_config(config, source_registry_store=store)
|
||||
|
||||
connection = service.get("demo.default")
|
||||
assert connection.server == "registry"
|
||||
assert connection.account == "stored"
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run connection-service tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_connection_service.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Update Registry Mutation Shadow Checks
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_mcp/broker/service/source_registry_admin.py`
|
||||
- Test: `tests/wf_mcp/service/test_source_registry_admin.py`
|
||||
|
||||
- [ ] **Step 1: Add config ownership lookup helper**
|
||||
|
||||
In `SourceRegistryAdminProvider`, add:
|
||||
|
||||
```python
|
||||
def _config_connection(self, source_id: str) -> ConnectionConfig | None:
|
||||
for connection in self.config_connections:
|
||||
if connection.id == source_id:
|
||||
return connection
|
||||
return None
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Change `config_source_ids` if needed**
|
||||
|
||||
Keep `config_source_ids()` unchanged for read payload compatibility:
|
||||
|
||||
```python
|
||||
def config_source_ids(self) -> set[str]:
|
||||
return {connection.id for connection in self.config_connections}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update `add_registry_entry` shadow rejection**
|
||||
|
||||
Replace the current config id check with:
|
||||
|
||||
```python
|
||||
config_connection = self._config_connection(source_id)
|
||||
if (
|
||||
config_connection is not None
|
||||
and config_connection.source_config_ownership == "locked"
|
||||
):
|
||||
raise ValueError(
|
||||
f"cannot add {source_id!r}: id is locked by a config connection"
|
||||
)
|
||||
```
|
||||
|
||||
For `seed`, adding remains allowed only if no registry entry already exists.
|
||||
|
||||
- [ ] **Step 4: Add locked add rejection test**
|
||||
|
||||
In `tests/wf_mcp/service/test_source_registry_admin.py`, add:
|
||||
|
||||
```python
|
||||
def test_add_rejects_locked_config_shadow(tmp_path: Path) -> None:
|
||||
provider = _provider(tmp_path, config_ids=frozenset({"github.work"}))
|
||||
|
||||
with pytest.raises(ValueError, match="locked by a config connection"):
|
||||
provider.add_registry_entry(_entry_dict("github.work"))
|
||||
```
|
||||
|
||||
If `_provider` cannot pass policy, update it to build `ConnectionConfig(..., source_config_ownership="locked")`.
|
||||
|
||||
- [ ] **Step 5: Add seed add allowed test**
|
||||
|
||||
Update `_provider` to accept config connections or config policy, then add:
|
||||
|
||||
```python
|
||||
def test_add_allows_seed_config_shadow_when_registry_missing(tmp_path: Path) -> None:
|
||||
provider = _provider(
|
||||
tmp_path,
|
||||
config_connections=[
|
||||
ConnectionConfig(
|
||||
id="github.work",
|
||||
server="github",
|
||||
account="work",
|
||||
source_config_ownership="seed",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
result = provider.add_registry_entry(_entry_dict("github.work"))
|
||||
|
||||
assert result.id == "github.work"
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run source registry admin tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/service/test_source_registry_admin.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Expose Ownership in Admin Registry Payloads
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_api/source_registry_admin.py`
|
||||
- Modify: `src/wf_mcp/broker/service/source_registry_admin.py`
|
||||
- Test: `tests/wf_api/test_source_registry_admin_api.py`
|
||||
- Test: `tests/wf_mcp/service/test_source_registry_admin.py`
|
||||
|
||||
- [ ] **Step 1: Extend provider protocol**
|
||||
|
||||
In `WorkflowSourceRegistryProvider`, add:
|
||||
|
||||
```python
|
||||
def config_source_ownership(self) -> Mapping[str, str]: ...
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement provider method**
|
||||
|
||||
In `SourceRegistryAdminProvider`, add:
|
||||
|
||||
```python
|
||||
def config_source_ownership(self) -> dict[str, str]:
|
||||
return {
|
||||
connection.id: connection.source_config_ownership
|
||||
for connection in self.config_connections
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Include ownership in summary and inspect payloads**
|
||||
|
||||
In `WorkflowSourceRegistryApi`, compute:
|
||||
|
||||
```python
|
||||
ownership = self._provider.config_source_ownership()
|
||||
```
|
||||
|
||||
Update `_entry_summary` signature to accept `ownership: Mapping[str, str]`.
|
||||
|
||||
Include:
|
||||
|
||||
```python
|
||||
"config_ownership": ownership.get(entry_id),
|
||||
"mutable": ownership.get(entry_id) != "locked",
|
||||
```
|
||||
|
||||
For inspect payloads, include the same fields at top level:
|
||||
|
||||
```python
|
||||
"config_ownership": ownership.get(source_id),
|
||||
"mutable": ownership.get(source_id) != "locked",
|
||||
```
|
||||
|
||||
Keep existing `shadowed_by_config` for compatibility.
|
||||
|
||||
- [ ] **Step 4: Update fake providers in tests**
|
||||
|
||||
In `tests/wf_api/test_source_registry_admin_api.py`, add:
|
||||
|
||||
```python
|
||||
def config_source_ownership(self) -> dict[str, str]:
|
||||
return {source_id: "locked" for source_id in self._config_ids}
|
||||
```
|
||||
|
||||
or allow the fake to accept an ownership mapping.
|
||||
|
||||
- [ ] **Step 5: Add API payload test**
|
||||
|
||||
```python
|
||||
def test_list_registry_entries_reports_config_ownership_and_mutability() -> None:
|
||||
api = WorkflowSourceRegistryApi(
|
||||
provider=FakeRegistryProvider(
|
||||
[FakeRegistryEntry(id="github.work")],
|
||||
config_ids={"github.work"},
|
||||
)
|
||||
)
|
||||
|
||||
payload = asyncio.run(api.list_registry_entries())
|
||||
|
||||
entry = payload["entries"][0]
|
||||
assert entry["shadowed_by_config"] is True
|
||||
assert entry["config_ownership"] == "locked"
|
||||
assert entry["mutable"] is False
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run API/provider tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_source_registry_admin_api.py tests/wf_mcp/service/test_source_registry_admin.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Update Docs and Roadmap Status
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md`
|
||||
- Modify: `docs/superpowers/plans/2026-06-03-source-registry-next-slices.md`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Update spec Slice 6 status**
|
||||
|
||||
In `docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md`, change Slice 6 status from planned to complete and summarize:
|
||||
|
||||
```markdown
|
||||
Status: complete. MCP broker config connections now support
|
||||
`source_config_ownership="locked" | "seed"`. `locked` preserves v1 shadowing.
|
||||
`seed` materializes missing store entries and lets existing registry entries
|
||||
own future runtime state.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update next-slices plan**
|
||||
|
||||
In `docs/superpowers/plans/2026-06-03-source-registry-next-slices.md`, mark Slice 6 complete.
|
||||
|
||||
- [ ] **Step 3: Update current roadmap**
|
||||
|
||||
In `docs/current_roadmap.md`, replace the planned wording with implemented wording:
|
||||
|
||||
```markdown
|
||||
Config ownership policy is implemented for MCP broker config connections:
|
||||
`locked` entries stay operator-owned, while `seed` entries bootstrap missing
|
||||
store entries and then let the store own later admin changes.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run doc diff check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git diff -- docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md docs/superpowers/plans/2026-06-03-source-registry-next-slices.md docs/current_roadmap.md
|
||||
```
|
||||
|
||||
Expected: docs match the implemented slice and do not claim neutral `wf_config.server.sources` supports MCP source ownership yet.
|
||||
|
||||
---
|
||||
|
||||
## Final Verification
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_source_registry.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/service/test_source_registry_admin.py tests/wf_api/test_source_registry_admin_api.py -q
|
||||
uv run ruff check src/wf_mcp/models.py src/wf_mcp/control.py src/wf_mcp/source_registry.py src/wf_mcp/broker/service/connection_service.py src/wf_mcp/broker/service/source_registry_admin.py src/wf_api/source_registry_admin.py tests/wf_mcp/test_source_registry.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/service/test_source_registry_admin.py tests/wf_api/test_source_registry_admin_api.py
|
||||
uv run basedpyright --level error src/wf_mcp/models.py src/wf_mcp/control.py src/wf_mcp/source_registry.py src/wf_mcp/broker/service/connection_service.py src/wf_mcp/broker/service/source_registry_admin.py src/wf_api/source_registry_admin.py tests/wf_mcp/test_source_registry.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/service/test_source_registry_admin.py tests/wf_api/test_source_registry_admin_api.py
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: pytest exits 0, ruff exits 0, basedpyright exits 0, and `git diff --check` reports no whitespace errors.
|
||||
|
||||
## Self-Review
|
||||
|
||||
- This plan intentionally keeps `locked` as the default for backward compatibility.
|
||||
- This plan does not add MCP source entries to neutral `wf_config` because current neutral source config only supports built-in sources.
|
||||
- This plan requires `seed` config connections to carry `metadata.transport`; without transport, the config cannot be materialized into a durable registry entry.
|
||||
- The admin payload adds `config_ownership` and `mutable` without removing `shadowed_by_config`, preserving compatibility.
|
||||
@@ -0,0 +1,347 @@
|
||||
# Source Registry Admin Reads Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Expose the persisted desired source registry through read-only admin APIs, JSON-RPC, and CLI without confusing it with observed runtime source inventory.
|
||||
|
||||
**Architecture:** Desired registry state is server/platform configuration. It is not workflow lifecycle state and not observed catalog/source inventory. Add a neutral read-only `WorkflowSourceRegistryApi` over a provider protocol in `wf_api`; implement the provider in `wf_mcp` using `FileSourceRegistryStore` and optional config connection ids for shadow information. Keep mutations out of scope.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, `wf_api`, `wf_mcp.source_registry`, `wf_transport_rpc_http`, `wf_cli`, Typer, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Naming Decision
|
||||
|
||||
Use admin/config naming:
|
||||
|
||||
- JSON-RPC:
|
||||
- `workflow.admin.source_registry.list`
|
||||
- `workflow.admin.source_registry.inspect`
|
||||
- CLI:
|
||||
- `wf admin registry list`
|
||||
- `wf admin registry inspect SOURCE_ID`
|
||||
|
||||
Do **not** use `wf source registry ...` in this slice. `wf source list` already means observed/hydrated source inventory. Registry reads are desired server-owned configuration state.
|
||||
|
||||
---
|
||||
|
||||
## Payload Shape
|
||||
|
||||
List payload:
|
||||
|
||||
```json
|
||||
{
|
||||
"entries": [
|
||||
{
|
||||
"id": "github.work",
|
||||
"kind": "mcp",
|
||||
"enabled": true,
|
||||
"provider": "github",
|
||||
"account": "work",
|
||||
"profile": null,
|
||||
"transport_kind": "stdio",
|
||||
"auth_ref": "github.work",
|
||||
"shadowed_by_config": false
|
||||
}
|
||||
],
|
||||
"next_cursor": null,
|
||||
"total": 1
|
||||
}
|
||||
```
|
||||
|
||||
Inspect payload:
|
||||
|
||||
```json
|
||||
{
|
||||
"entry": {
|
||||
"id": "github.work",
|
||||
"kind": "mcp",
|
||||
"enabled": true,
|
||||
"provider": "github",
|
||||
"account": "work",
|
||||
"profile": null,
|
||||
"transport": {"kind": "stdio", "command": "npx", "args": [], "env": {}},
|
||||
"auth_ref": "github.work",
|
||||
"metadata": {}
|
||||
},
|
||||
"shadowed_by_config": false
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- List returns summaries; inspect returns full entry detail.
|
||||
- `shadowed_by_config` is advisory. If a caller has no config provider, return
|
||||
`false` rather than guessing.
|
||||
- Do not include auth secret payloads. `auth_ref` is only an id/reference.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Add Neutral Source Registry Admin API
|
||||
|
||||
- [ ] Create `src/wf_api/source_registry_admin.py`.
|
||||
- [ ] Define:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping, Sequence, Set
|
||||
from dataclasses import asdict, is_dataclass
|
||||
from typing import Any, Protocol
|
||||
|
||||
from wf_platform import page_items
|
||||
|
||||
|
||||
class WorkflowSourceRegistryProvider(Protocol):
|
||||
"""Provides desired source registry state for read-only admin frontends."""
|
||||
|
||||
def list_registry_entries(self) -> Sequence[Mapping[str, Any] | object]: ...
|
||||
|
||||
def config_source_ids(self) -> Set[str]: ...
|
||||
```
|
||||
|
||||
- [ ] Define `WorkflowSourceRegistryApi` with:
|
||||
|
||||
```python
|
||||
async def list_registry_entries(
|
||||
self,
|
||||
*,
|
||||
cursor: str | None = None,
|
||||
limit: int = 50,
|
||||
) -> dict[str, Any]: ...
|
||||
|
||||
async def inspect_registry_entry(self, *, source_id: str) -> dict[str, Any]: ...
|
||||
```
|
||||
|
||||
- [ ] Normalize provider objects using the same style as `wf_api.admin._payload`:
|
||||
mapping, dataclass, or Pydantic `model_dump(mode="json")`.
|
||||
- [ ] Add helpers:
|
||||
|
||||
```python
|
||||
def _entry_summary(entry: dict[str, Any], shadowed_ids: set[str]) -> dict[str, Any]:
|
||||
transport = entry.get("transport")
|
||||
transport_kind = transport.get("kind") if isinstance(transport, Mapping) else None
|
||||
return {
|
||||
"id": entry["id"],
|
||||
"kind": entry["kind"],
|
||||
"enabled": entry["enabled"],
|
||||
"provider": entry.get("provider"),
|
||||
"account": entry.get("account"),
|
||||
"profile": entry.get("profile"),
|
||||
"transport_kind": transport_kind,
|
||||
"auth_ref": entry.get("auth_ref"),
|
||||
"shadowed_by_config": entry["id"] in shadowed_ids,
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] `inspect_registry_entry()` should raise `KeyError(f"unknown registry source {source_id!r}")` when missing.
|
||||
- [ ] Export `WorkflowSourceRegistryApi` and `WorkflowSourceRegistryProvider` from `src/wf_api/__init__.py`.
|
||||
- [ ] Add `WorkflowSourceRegistrySurface` to `src/wf_api/surface.py` and `__all__`.
|
||||
|
||||
### Tests
|
||||
|
||||
- [ ] Create `tests/wf_api/test_source_registry_admin_api.py`.
|
||||
- [ ] Test list returns compact summaries in id order.
|
||||
- [ ] Test pagination.
|
||||
- [ ] Test inspect returns full entry and shadow flag.
|
||||
- [ ] Test unknown inspect raises clear `KeyError`.
|
||||
- [ ] Test the concrete API satisfies `WorkflowSourceRegistrySurface`.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Add MCP Provider for Registry Reads
|
||||
|
||||
- [ ] Create `src/wf_mcp/broker/service/source_registry_admin.py`.
|
||||
- [ ] Define:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from ...models import ConnectionConfig
|
||||
from ...source_registry import SourceRegistryStore
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class SourceRegistryAdminProvider:
|
||||
"""Read desired MCP source registry state without mutating it."""
|
||||
|
||||
source_registry_store: SourceRegistryStore
|
||||
config_connections: Sequence[ConnectionConfig] = field(default_factory=tuple)
|
||||
|
||||
def list_registry_entries(self) -> list[object]:
|
||||
return list(self.source_registry_store.load_registry().sources)
|
||||
|
||||
def config_source_ids(self) -> set[str]:
|
||||
return {connection.id for connection in self.config_connections}
|
||||
```
|
||||
|
||||
- [ ] Keep this provider read-only.
|
||||
- [ ] Do not load auth records or catalog snapshots here.
|
||||
|
||||
### Tests
|
||||
|
||||
- [ ] Create `tests/wf_mcp/service/test_source_registry_admin.py`.
|
||||
- [ ] Test provider lists entries from `FileSourceRegistryStore`.
|
||||
- [ ] Test provider reports config-shadowed ids.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Wire Server Context
|
||||
|
||||
- [ ] Update `src/wf_server/context.py`.
|
||||
- [ ] Add a nullable `source_registry_admin` field to `WorkflowServer`:
|
||||
|
||||
```python
|
||||
source_registry_admin: WorkflowSourceRegistryApi | None = None
|
||||
```
|
||||
|
||||
- [ ] For the local/static server builder, leave it as `None`. Local/static server has no file-backed MCP source registry.
|
||||
- [ ] This field is used by JSON-RPC registration; missing support should return a structured RPC error.
|
||||
|
||||
### Tests
|
||||
|
||||
- [ ] Update `tests/wf_server/test_local_static_server.py` if needed to assert local/static construction still works.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Wire MCP/CLI Server Construction
|
||||
|
||||
- [ ] Identify the current `WorkflowServer` construction path for JSON-RPC HTTP target servers.
|
||||
- [ ] When constructing a server from broker config, create:
|
||||
|
||||
```python
|
||||
source_registry_admin = WorkflowSourceRegistryApi(
|
||||
SourceRegistryAdminProvider(
|
||||
source_registry_store=FileSourceRegistryStore(config.store_root),
|
||||
config_connections=config.connections,
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] Attach it to `WorkflowServer`.
|
||||
- [ ] Do not alter source registry startup merge behavior in this slice.
|
||||
|
||||
### Tests
|
||||
|
||||
- [ ] Add or update a JSON-RPC server construction test that seeds `source_registry.json` and asserts `server.source_registry_admin` is not `None`.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Add JSON-RPC Methods and Client Mixin
|
||||
|
||||
- [ ] Add `src/wf_transport_rpc_http/methods_source_registry.py`.
|
||||
- [ ] Register:
|
||||
|
||||
```python
|
||||
workflow.admin.source_registry.list
|
||||
workflow.admin.source_registry.inspect
|
||||
```
|
||||
|
||||
- [ ] If `server.source_registry_admin is None`, raise `WorkflowRpcError` with:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": "source_registry_unavailable",
|
||||
"message": "source registry admin reads are not available for this server"
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] Add `src/wf_transport_rpc_http/client_source_registry.py`.
|
||||
- [ ] Add `RpcSourceRegistryClientMixin` with:
|
||||
|
||||
```python
|
||||
async def list_registry_entries(self, *, cursor: str | None = None, limit: int = 50) -> dict[str, Any]: ...
|
||||
|
||||
async def inspect_registry_entry(self, *, source_id: str) -> dict[str, Any]: ...
|
||||
```
|
||||
|
||||
- [ ] Include the mixin in `src/wf_transport_rpc_http/client.py`.
|
||||
- [ ] Register methods in `src/wf_transport_rpc_http/app.py`.
|
||||
|
||||
### Tests
|
||||
|
||||
- [ ] Add focused JSON-RPC method tests.
|
||||
- [ ] Add client tests proving the mixin calls the correct method names.
|
||||
- [ ] Add a local/static unavailable test.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Add CLI Commands
|
||||
|
||||
- [ ] Add `src/wf_cli/commands/source_registry.py`.
|
||||
- [ ] Register it under the existing `wf admin` app:
|
||||
|
||||
```bash
|
||||
wf admin registry list
|
||||
wf admin registry inspect SOURCE_ID
|
||||
```
|
||||
|
||||
- [ ] Use the target-aware CLI context, not local-only context.
|
||||
- [ ] Output JSON by default, following existing CLI command conventions.
|
||||
- [ ] Do not add mutation flags.
|
||||
|
||||
### Tests
|
||||
|
||||
- [ ] Add CLI tests:
|
||||
- local/server target lists entries
|
||||
- inspect returns full entry
|
||||
- missing entry exits nonzero or returns structured error, matching current CLI patterns
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Docs
|
||||
|
||||
- [ ] Update `docs/current_roadmap.md`.
|
||||
- [ ] Update `docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md`.
|
||||
- [ ] Update `docs/superpowers/plans/2026-06-03-source-registry-next-slices.md`.
|
||||
- [ ] Add a short note to CLI docs if there is a current CLI command reference:
|
||||
|
||||
```md
|
||||
`wf admin registry list` shows desired persisted registry entries. `wf source list`
|
||||
shows observed/hydrated source inventory. Use both when debugging disabled,
|
||||
shadowed, or not-yet-hydrated sources.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Verify
|
||||
|
||||
- [ ] Focused tests:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_source_registry_admin_api.py tests/wf_mcp/service/test_source_registry_admin.py -q
|
||||
```
|
||||
|
||||
- [ ] RPC/CLI tests:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http tests/wf_cli -q
|
||||
```
|
||||
|
||||
- [ ] Existing source registry tests:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_source_registry.py tests/wf_mcp/test_source_registry.py -q
|
||||
```
|
||||
|
||||
- [ ] Quality:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_api src/wf_mcp src/wf_transport_rpc_http src/wf_cli tests/wf_api tests/wf_mcp tests/wf_transport_rpc_http tests/wf_cli
|
||||
uv run basedpyright --level error src/wf_api src/wf_mcp src/wf_transport_rpc_http src/wf_cli tests/wf_api tests/wf_mcp tests/wf_transport_rpc_http tests/wf_cli
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Desired registry reads are available separately from observed source inventory.
|
||||
- List is compact and paginated.
|
||||
- Inspect returns full persisted entry details.
|
||||
- Shadowed-by-config status is visible.
|
||||
- Local/static servers report registry reads unavailable instead of pretending to have an empty registry.
|
||||
- No mutation commands are added.
|
||||
- `wf source list` behavior is unchanged.
|
||||
@@ -0,0 +1,365 @@
|
||||
# Source Registry Mutations Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add safe source registry mutation operations so server-owned MCP sources can be added, updated, enabled, disabled, and removed without editing config files by hand.
|
||||
|
||||
**Architecture:** Mutations target only the persisted desired registry, never config files. `wf_api` stays protocol-neutral by accepting/returning registry entry dictionaries. The MCP provider owns validation through `McpSourceRegistryEntry` / `SourceRegistryFile` and persistence through `SourceRegistryStore`. Runtime hydration still happens through the existing startup/reload merge path.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, `wf_api.source_registry_admin`, `wf_mcp.source_registry`, `wf_transport_rpc_http`, `wf_cli`, Typer, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
Add these operations:
|
||||
|
||||
- add source registry entry
|
||||
- update source registry entry
|
||||
- enable source registry entry
|
||||
- disable source registry entry
|
||||
- remove source registry entry
|
||||
|
||||
Out of scope:
|
||||
|
||||
- no auth record creation/deletion
|
||||
- no catalog snapshot deletion
|
||||
- no config file mutation
|
||||
- no live transport validation by default
|
||||
- no raw proxy remount behavior changes
|
||||
|
||||
---
|
||||
|
||||
## Semantics
|
||||
|
||||
### Config Shadowing
|
||||
|
||||
Config-defined connections still win over registry entries during startup/reload.
|
||||
|
||||
For mutation v1:
|
||||
|
||||
- `add` rejects a source id that already exists in config to avoid silently adding a registry entry with no runtime effect.
|
||||
- `update`, `enable`, `disable`, and `remove` may operate on existing registry entries even if they are currently shadowed by config.
|
||||
- A future `allow_shadow` flag can relax `add`; do not add it in this slice.
|
||||
|
||||
Rationale: `add` rejects config-shadowed ids to prevent silent no-ops: adding a
|
||||
registry entry that cannot activate while config owns the same id. Existing
|
||||
shadowed registry entries can still be updated, enabled, disabled, or removed so
|
||||
operators can prepare store state for config removal or future `seed` ownership
|
||||
policy.
|
||||
|
||||
### Full-Registry Validation
|
||||
|
||||
Every mutation must:
|
||||
|
||||
1. load the current registry
|
||||
2. build a new `SourceRegistryFile`
|
||||
3. validate the full file
|
||||
4. save one atomic replacement
|
||||
5. return the updated entry or removal payload
|
||||
|
||||
Never mutate the loaded registry object in-place and then save after partial validation. Build a new list first.
|
||||
|
||||
### Events
|
||||
|
||||
Mutations should emit events if an event recorder is available:
|
||||
|
||||
- `source_registry_entry_added`
|
||||
- `source_registry_entry_updated`
|
||||
- `source_registry_entry_enabled`
|
||||
- `source_registry_entry_disabled`
|
||||
- `source_registry_entry_removed`
|
||||
|
||||
Payload should include at least:
|
||||
|
||||
```json
|
||||
{"source_id": "github.work", "provider": "github", "account": "work"}
|
||||
```
|
||||
|
||||
If adding event wiring is awkward for this slice, keep mutation payloads correct and document event wiring as future work. Do not block mutation safety on event polish.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Extend Neutral API Provider Protocol
|
||||
|
||||
- [ ] Update `src/wf_api/source_registry_admin.py`.
|
||||
- [ ] Add mutation methods to `WorkflowSourceRegistryProvider`:
|
||||
|
||||
```python
|
||||
def add_registry_entry(self, entry: Mapping[str, Any]) -> Mapping[str, Any] | object: ...
|
||||
|
||||
def update_registry_entry(
|
||||
self,
|
||||
source_id: str,
|
||||
patch: Mapping[str, Any],
|
||||
) -> Mapping[str, Any] | object: ...
|
||||
|
||||
def set_registry_entry_enabled(
|
||||
self,
|
||||
source_id: str,
|
||||
enabled: bool,
|
||||
) -> Mapping[str, Any] | object: ...
|
||||
|
||||
def remove_registry_entry(self, source_id: str) -> Mapping[str, Any] | object: ...
|
||||
```
|
||||
|
||||
- [ ] Add async API methods:
|
||||
|
||||
```python
|
||||
async def add_registry_entry(self, *, entry: dict[str, Any]) -> dict[str, Any]: ...
|
||||
|
||||
async def update_registry_entry(
|
||||
self,
|
||||
*,
|
||||
source_id: str,
|
||||
patch: dict[str, Any],
|
||||
) -> dict[str, Any]: ...
|
||||
|
||||
async def enable_registry_entry(self, *, source_id: str) -> dict[str, Any]: ...
|
||||
|
||||
async def disable_registry_entry(self, *, source_id: str) -> dict[str, Any]: ...
|
||||
|
||||
async def remove_registry_entry(self, *, source_id: str) -> dict[str, Any]: ...
|
||||
```
|
||||
|
||||
- [ ] Return shapes:
|
||||
|
||||
```python
|
||||
{"entry": <full entry>, "shadowed_by_config": bool}
|
||||
{"removed": true, "source_id": "..."}
|
||||
```
|
||||
|
||||
- [ ] Reuse existing `_payload()` and shadow helper logic.
|
||||
- [ ] Update `WorkflowSourceRegistrySurface` in `src/wf_api/surface.py`.
|
||||
|
||||
### Tests
|
||||
|
||||
- [ ] Extend `tests/wf_api/test_source_registry_admin_api.py`.
|
||||
- [ ] Add fake mutable provider.
|
||||
- [ ] Test each API method delegates and returns normalized payloads.
|
||||
- [ ] Test remove payload.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Implement MCP Mutation Provider
|
||||
|
||||
- [ ] Update `src/wf_mcp/broker/service/source_registry_admin.py`.
|
||||
- [ ] Keep `SourceRegistryAdminProvider` as the read/write provider.
|
||||
- [ ] Add optional event sink if practical:
|
||||
|
||||
```python
|
||||
from collections.abc import Callable
|
||||
from ...events import McpEvent
|
||||
|
||||
event_sink: Callable[[McpEvent], None] | None = None
|
||||
```
|
||||
|
||||
If this creates too much coupling, skip event sink and document it.
|
||||
|
||||
- [ ] Add helpers:
|
||||
|
||||
```python
|
||||
def _load(self) -> SourceRegistryFile: ...
|
||||
def _save(self, sources: list[McpSourceRegistryEntry]) -> SourceRegistryFile: ...
|
||||
def _entry_map(self, registry: SourceRegistryFile) -> dict[str, McpSourceRegistryEntry]: ...
|
||||
def _require_entry(self, source_id: str) -> McpSourceRegistryEntry: ...
|
||||
```
|
||||
|
||||
- [ ] `add_registry_entry(entry)`:
|
||||
- reject if `entry["id"]` is in `config_source_ids()`
|
||||
- validate with `McpSourceRegistryEntry.model_validate(entry)`
|
||||
- reject duplicate existing registry id
|
||||
- save full `SourceRegistryFile`
|
||||
- return added entry
|
||||
|
||||
- [ ] `update_registry_entry(source_id, patch)`:
|
||||
- require existing registry entry
|
||||
- reject changing `id` in v1 unless it equals `source_id`
|
||||
- merge existing full JSON entry with patch
|
||||
- validate with `McpSourceRegistryEntry`
|
||||
- save full file
|
||||
- return updated entry
|
||||
|
||||
- [ ] `set_registry_entry_enabled(source_id, enabled)`:
|
||||
- require existing registry entry
|
||||
- update `enabled`
|
||||
- save full file
|
||||
- return updated entry
|
||||
|
||||
- [ ] `remove_registry_entry(source_id)`:
|
||||
- require existing registry entry
|
||||
- save full file without the entry
|
||||
- return `{"removed": True, "source_id": source_id}`
|
||||
- do not delete auth/catalog
|
||||
|
||||
### Tests
|
||||
|
||||
- [ ] Extend or create tests in `tests/wf_mcp/service/test_source_registry_admin.py`.
|
||||
- [ ] Test add persists and round-trips through store.
|
||||
- [ ] Test add rejects config-shadowed id.
|
||||
- [ ] Test add rejects duplicate registry id.
|
||||
- [ ] Test update persists provider/account/transport changes.
|
||||
- [ ] Test update rejects id change.
|
||||
- [ ] Test enable/disable persist.
|
||||
- [ ] Test remove persists absence and does not touch unrelated entries.
|
||||
- [ ] Test missing source raises clear `KeyError`.
|
||||
- [ ] Test malformed payload raises validation error with actionable message.
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Add RPC Mutation Methods
|
||||
|
||||
- [ ] Update `src/wf_transport_rpc_http/models.py`.
|
||||
- [ ] Add params:
|
||||
|
||||
```python
|
||||
class AddRegistryEntryParams(RpcParamsModel):
|
||||
entry: dict[str, Any]
|
||||
|
||||
|
||||
class UpdateRegistryEntryParams(RpcParamsModel):
|
||||
source_id: str = Field(min_length=1)
|
||||
patch: dict[str, Any]
|
||||
|
||||
|
||||
class RegistryEntryIdParams(RpcParamsModel):
|
||||
source_id: str = Field(min_length=1)
|
||||
```
|
||||
|
||||
- [ ] Update `src/wf_transport_rpc_http/methods_source_registry.py`.
|
||||
- [ ] Register:
|
||||
|
||||
```text
|
||||
workflow.admin.source_registry.add
|
||||
workflow.admin.source_registry.update
|
||||
workflow.admin.source_registry.enable
|
||||
workflow.admin.source_registry.disable
|
||||
workflow.admin.source_registry.remove
|
||||
```
|
||||
|
||||
- [ ] Keep unavailable behavior identical to read methods:
|
||||
|
||||
```json
|
||||
{"code": "source_registry_unavailable", ...}
|
||||
```
|
||||
|
||||
- [ ] Use `Params(...)` for required params, matching inspect methods.
|
||||
- [ ] Update `src/wf_transport_rpc_http/client_source_registry.py`.
|
||||
- [ ] Add matching client methods.
|
||||
|
||||
### Tests
|
||||
|
||||
- [ ] Extend `tests/wf_transport_rpc_http/test_source_registry_rpc.py`.
|
||||
- [ ] Add positive method tests using a fake `WorkflowServer` with `WorkflowSourceRegistryApi`.
|
||||
- [ ] Add unavailable mutation test on local/static server.
|
||||
- [ ] Add client method tests for method names and payloads.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Add CLI Mutation Commands
|
||||
|
||||
- [ ] Update `src/wf_cli/commands/source_registry.py`.
|
||||
- [ ] Add commands:
|
||||
|
||||
```bash
|
||||
wf admin registry add --input '{"id":"github.work",...}'
|
||||
wf admin registry update SOURCE_ID --patch '{"enabled":false}'
|
||||
wf admin registry enable SOURCE_ID
|
||||
wf admin registry disable SOURCE_ID
|
||||
wf admin registry remove SOURCE_ID
|
||||
```
|
||||
|
||||
- [ ] Support `--input-file` for add and `--patch-file` for update if existing CLI helpers make it easy.
|
||||
- [ ] JSON inline input is required for v1. Do not design a large flag matrix for every MCP field yet.
|
||||
- [ ] For remove, require `--confirm` to avoid accidental deletion:
|
||||
|
||||
```bash
|
||||
wf admin registry remove github.work --confirm
|
||||
```
|
||||
|
||||
- [ ] If `source_registry_admin is None`, keep the existing "not available" behavior.
|
||||
|
||||
### Tests
|
||||
|
||||
- [ ] Extend `tests/wf_cli/test_source_registry.py`.
|
||||
- [ ] Test help includes add/update/enable/disable/remove.
|
||||
- [ ] Test local/static unavailable for one mutation command.
|
||||
- [ ] Test remove without `--confirm` fails.
|
||||
- [ ] Test commands delegate to a fake context where possible, or use RPC client monkeypatch patterns already used in CLI tests.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Wire Real MCP Server Provider If Available
|
||||
|
||||
Current state: `WorkflowServer.source_registry_admin` exists, but local/static servers set it to `None`. There may still be no concrete MCP-backed `WorkflowServer` construction path.
|
||||
|
||||
- [ ] If a concrete MCP-backed `WorkflowServer` construction path exists, wire:
|
||||
|
||||
```python
|
||||
WorkflowSourceRegistryApi(
|
||||
provider=SourceRegistryAdminProvider(
|
||||
source_registry_store=FileSourceRegistryStore(config.store_root),
|
||||
config_connections=config.connections,
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] If no such path exists, do not invent it in this slice. Keep docs explicit that RPC/CLI mutation methods require a target exposing `source_registry_admin`.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Docs
|
||||
|
||||
- [ ] Update `docs/current_roadmap.md`.
|
||||
- [ ] Update `docs/superpowers/specs/2026-06-03-store-backed-source-registry-design.md`.
|
||||
- [ ] Update `docs/superpowers/plans/2026-06-03-source-registry-next-slices.md`.
|
||||
- [ ] Document:
|
||||
- registry mutations change desired state only
|
||||
- config files are not mutated
|
||||
- auth/catalog are not deleted on remove
|
||||
- startup/reload is still how runtime hydration sees changes
|
||||
- config-shadowed add is rejected in v1
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Verify
|
||||
|
||||
- [ ] Focused API/provider tests:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_source_registry_admin_api.py tests/wf_mcp/service/test_source_registry_admin.py -q
|
||||
```
|
||||
|
||||
- [ ] RPC/CLI tests:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_source_registry_rpc.py tests/wf_cli/test_source_registry.py -q
|
||||
```
|
||||
|
||||
- [ ] Existing registry/startup tests:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_api/test_source_registry.py tests/wf_mcp/test_source_registry.py tests/wf_mcp/service/test_connection_service.py tests/wf_mcp/test_broker_server.py -q
|
||||
```
|
||||
|
||||
- [ ] Quality:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_api/source_registry_admin.py src/wf_mcp/broker/service/source_registry_admin.py src/wf_transport_rpc_http src/wf_cli tests/wf_api/test_source_registry_admin_api.py tests/wf_mcp/service/test_source_registry_admin.py tests/wf_transport_rpc_http/test_source_registry_rpc.py tests/wf_cli/test_source_registry.py
|
||||
uv run basedpyright --level error src/wf_api/source_registry_admin.py src/wf_mcp/broker/service/source_registry_admin.py src/wf_transport_rpc_http src/wf_cli tests/wf_api/test_source_registry_admin_api.py tests/wf_mcp/service/test_source_registry_admin.py tests/wf_transport_rpc_http/test_source_registry_rpc.py tests/wf_cli/test_source_registry.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Registry add/update/enable/disable/remove work through neutral API methods.
|
||||
- MCP provider validates all writes through `McpSourceRegistryEntry` / `SourceRegistryFile`.
|
||||
- Whole registry is validated before every save.
|
||||
- Config files are not mutated.
|
||||
- Auth/catalog files are not deleted.
|
||||
- Adding an id owned by config is rejected in v1.
|
||||
- Existing shadowed registry entries can still be updated or removed.
|
||||
- RPC methods and CLI commands exist.
|
||||
- Remove requires explicit CLI confirmation.
|
||||
- Observed source inventory behavior is unchanged until reload/startup rehydrates runtime state.
|
||||
Reference in New Issue
Block a user