docs: plan wider workflow config sources
This commit is contained in:
@@ -162,6 +162,27 @@ implementation state.
|
||||
JSON-RPC transport over an MCP-backed `WorkflowServer`, making the remote
|
||||
CLI path usable with MCP sources and desired source registry operations.
|
||||
- Next concrete platform slices:
|
||||
- Wider `wf_config` source model: migrate MCP broker config concepts into the
|
||||
neutral server config instead of preserving `wf_mcp.config.json` as a
|
||||
peer forever. `wf_config.server.sources[]` already exists as a
|
||||
discriminated union, but only `stdlib` is implemented today. Add MCP source
|
||||
variants that carry source id, provider/account/profile, ownership policy,
|
||||
transport shape, auth reference, and metadata. The old MCP broker config
|
||||
becomes a compatibility input that normalizes into the wider config model.
|
||||
After that, `wf-rpc-server --config ...` can build MCP-backed sources from
|
||||
neutral config and `--mcp-config` can be deprecated or treated as a legacy
|
||||
alias.
|
||||
`McpSourceRegistryEntry` already has most of the target shape; the one
|
||||
ownership field comes from legacy `ConnectionConfig.source_config_ownership`.
|
||||
When migrating, carry that policy into the neutral MCP source variant with a
|
||||
clearer name such as `config_ownership` or `ownership`, rather than leaking
|
||||
the old connection-centric field name.
|
||||
- Transport package boundary cleanup follows the config migration. The current
|
||||
`wf-rpc-server --mcp-config` hookup proves the product path but makes
|
||||
`wf_transport_rpc_http.cli` import `wf_mcp.broker`, tripping the existing
|
||||
import-direction guard. The durable fix is not a permanent split launcher;
|
||||
it is making `wf_config` wide enough that the RPC server can compose from
|
||||
neutral config while MCP-specific adapters stay selected by source kind.
|
||||
- Manual product smoke: run `wf-rpc-server --mcp-config ...`, point
|
||||
`wf --url ...` at it, and capture real CLI/server UX gaps before adding
|
||||
more architecture.
|
||||
@@ -287,6 +308,12 @@ implementation state.
|
||||
and `sync_connections_from_config`. `WfMcpService.connections` remains a
|
||||
compatibility property while source hydration still belongs to
|
||||
`SourceCatalogService`.
|
||||
- Several reusable implementation pieces still live in `wf_mcp` because they
|
||||
are MCP-shaped today (`source_registry.py`, `broker/server.py`, and focused
|
||||
`broker/service/*` services). The next config migration should make the
|
||||
split explicit: neutral config/registry mechanics belong in `wf_config`,
|
||||
`wf_api`, `wf_server`, or another platform package; MCP-specific transport,
|
||||
adapter, and upstream session behavior stays in `wf_mcp`.
|
||||
|
||||
Frame stress points remaining for native subgraphs and future fork/gather:
|
||||
|
||||
|
||||
@@ -0,0 +1,528 @@
|
||||
# Neutral Config MCP Server Composition Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Let `wf-rpc-server --config <workflow-config.json>` build an MCP-backed `WorkflowServer` when neutral `wf_config.server.sources[]` contains `kind: "mcp"` entries.
|
||||
|
||||
**Architecture:** Keep JSON-RPC app/client/method modules transport-only. The CLI may load neutral `wf_config`; MCP-specific conversion stays in `wf_mcp`. This plan assumes the prior plan has added `wf_config.McpSourceConfig`, `StdioSourceTransportConfig`, and `HttpSourceTransportConfig`.
|
||||
|
||||
**Tech Stack:** Pydantic config models, `wf_mcp.broker.config`, `wf_mcp.source_registry`, Typer CLI tests, ASGITransport JSON-RPC tests.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_mcp/broker/config.py`: add conversion from neutral `WorkflowConfigFile` into `BrokerConfig`.
|
||||
- Modify `src/wf_mcp/source_registry.py`: add conversion from neutral `McpSourceConfig` to `McpSourceRegistryEntry` or `ConnectionConfig`.
|
||||
- Modify `src/wf_transport_rpc_http/cli.py`: when `--config` has MCP sources, build MCP-backed server from neutral config instead of requiring `--mcp-config`.
|
||||
- Modify `tests/wf_mcp/server/test_config.py` or create `tests/wf_mcp/test_workflow_config_bridge.py`: test neutral config conversion into broker runtime.
|
||||
- Modify `tests/wf_transport_rpc_http/test_cli.py`: test `wf-rpc-server --config` selects MCP-backed server for neutral MCP source config.
|
||||
- Modify docs to mark this slice complete.
|
||||
|
||||
## Preconditions
|
||||
|
||||
This plan assumes these names exist from the prior plan:
|
||||
|
||||
```python
|
||||
from wf_config import (
|
||||
HttpSourceTransportConfig,
|
||||
McpSourceConfig,
|
||||
StdioSourceTransportConfig,
|
||||
WorkflowConfigFile,
|
||||
)
|
||||
```
|
||||
|
||||
Do not start this plan until `uv run pytest tests/wf_config/test_config_models.py -q` passes.
|
||||
|
||||
## Task 1: Test Neutral Config to BrokerConfig Conversion
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/wf_mcp/test_workflow_config_bridge.py`
|
||||
|
||||
- [ ] **Step 1: Create test file**
|
||||
|
||||
Create `tests/wf_mcp/test_workflow_config_bridge.py`:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_config import WorkflowConfigFile
|
||||
from wf_mcp.broker.config import broker_config_from_workflow_config
|
||||
|
||||
|
||||
def test_broker_config_from_workflow_config_converts_mcp_sources(tmp_path) -> None:
|
||||
workflow_config = WorkflowConfigFile.model_validate(
|
||||
{
|
||||
"version": 1,
|
||||
"server": {
|
||||
"store": {"kind": "filesystem", "root": str(tmp_path / "store")},
|
||||
"sources": [
|
||||
{
|
||||
"kind": "mcp",
|
||||
"id": "everything.default",
|
||||
"enabled": True,
|
||||
"provider": "everything",
|
||||
"account": "default",
|
||||
"profile": "dev",
|
||||
"ownership": "seed",
|
||||
"transport": {
|
||||
"kind": "stdio",
|
||||
"command": "uvx",
|
||||
"args": ["mcp-server-everything"],
|
||||
"env": {"DEBUG": "1"},
|
||||
},
|
||||
"auth_ref": "auth.everything.default",
|
||||
"metadata": {"description": "Everything test server"},
|
||||
}
|
||||
],
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
broker_config = broker_config_from_workflow_config(workflow_config)
|
||||
|
||||
assert broker_config.store_root == tmp_path / "store"
|
||||
assert len(broker_config.connections) == 1
|
||||
connection = broker_config.connections[0]
|
||||
assert connection.id == "everything.default"
|
||||
assert connection.server == "everything"
|
||||
assert connection.account == "default"
|
||||
assert connection.enabled is True
|
||||
assert connection.source_config_ownership == "seed"
|
||||
assert connection.metadata["profile"] == "dev"
|
||||
assert connection.metadata["auth_ref"] == "auth.everything.default"
|
||||
assert connection.metadata["transport"] == {
|
||||
"kind": "stdio",
|
||||
"command": "uvx",
|
||||
"args": ["mcp-server-everything"],
|
||||
"env": {"DEBUG": "1"},
|
||||
}
|
||||
assert connection.metadata["description"] == "Everything test server"
|
||||
|
||||
|
||||
def test_broker_config_from_workflow_config_ignores_non_mcp_sources(tmp_path) -> None:
|
||||
workflow_config = WorkflowConfigFile.model_validate(
|
||||
{
|
||||
"version": 1,
|
||||
"server": {
|
||||
"store": {"kind": "filesystem", "root": str(tmp_path / "store")},
|
||||
"sources": [{"kind": "stdlib", "id": "wf.std"}],
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
broker_config = broker_config_from_workflow_config(workflow_config)
|
||||
|
||||
assert broker_config.store_root == tmp_path / "store"
|
||||
assert broker_config.connections == []
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test and verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_workflow_config_bridge.py -q
|
||||
```
|
||||
|
||||
Expected: fail because `broker_config_from_workflow_config` does not exist.
|
||||
|
||||
## Task 2: Implement Conversion From Neutral Config
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_mcp/source_registry.py`
|
||||
- Modify: `src/wf_mcp/broker/config.py`
|
||||
|
||||
- [ ] **Step 1: Add neutral source conversion helper**
|
||||
|
||||
In `src/wf_mcp/source_registry.py`, add under `connection_config_to_registry_entry`:
|
||||
|
||||
```python
|
||||
def workflow_mcp_source_to_connection_config(source: object) -> ConnectionConfig:
|
||||
"""Convert neutral wf_config MCP source config into a broker connection.
|
||||
|
||||
Keep this adapter in wf_mcp because the output is MCP broker runtime state.
|
||||
The input is intentionally typed as object to avoid making wf_mcp's public
|
||||
registry module part of wf_config's import graph.
|
||||
"""
|
||||
from .models import ConnectionConfig
|
||||
|
||||
if getattr(source, "kind", None) != "mcp":
|
||||
raise ValueError("expected wf_config MCP source")
|
||||
transport = getattr(source, "transport")
|
||||
metadata = dict(getattr(source, "metadata", {}))
|
||||
metadata.update(
|
||||
{
|
||||
"transport": transport.model_dump(mode="json"),
|
||||
"source_registry": False,
|
||||
}
|
||||
)
|
||||
profile = getattr(source, "profile", None)
|
||||
if profile is not None:
|
||||
metadata["profile"] = profile
|
||||
auth_ref = getattr(source, "auth_ref", None)
|
||||
if auth_ref is not None:
|
||||
metadata["auth_ref"] = auth_ref
|
||||
return ConnectionConfig(
|
||||
id=getattr(source, "id"),
|
||||
server=getattr(source, "provider"),
|
||||
account=getattr(source, "account"),
|
||||
enabled=getattr(source, "enabled"),
|
||||
metadata=metadata,
|
||||
source_config_ownership=getattr(source, "ownership"),
|
||||
)
|
||||
```
|
||||
|
||||
Add `"workflow_mcp_source_to_connection_config"` to `__all__`.
|
||||
|
||||
- [ ] **Step 2: Add broker config bridge**
|
||||
|
||||
In `src/wf_mcp/broker/config.py`, add imports:
|
||||
|
||||
```python
|
||||
from wf_config import WorkflowConfigFile
|
||||
from ..source_registry import FileSourceRegistryStore, workflow_mcp_source_to_connection_config
|
||||
```
|
||||
|
||||
Replace the existing `FileSourceRegistryStore` import line accordingly.
|
||||
|
||||
Add after `load_broker_config`:
|
||||
|
||||
```python
|
||||
def broker_config_from_workflow_config(config: WorkflowConfigFile) -> BrokerConfig:
|
||||
"""Create MCP broker runtime config from neutral workflow server config."""
|
||||
return BrokerConfig(
|
||||
store_root=config.server.store.root,
|
||||
connections=[
|
||||
workflow_mcp_source_to_connection_config(source)
|
||||
for source in config.server.sources
|
||||
if getattr(source, "kind", None) == "mcp"
|
||||
],
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run conversion test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_workflow_config_bridge.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 3: Build MCP-Backed WorkflowServer From Neutral Config
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_mcp/broker/server.py`
|
||||
- Modify: `tests/wf_transport_rpc_http/test_cli.py`
|
||||
|
||||
- [ ] **Step 1: Add broker server helper**
|
||||
|
||||
In `src/wf_mcp/broker/server.py`, import:
|
||||
|
||||
```python
|
||||
from wf_config import WorkflowConfigFile
|
||||
from .config import broker_config_from_workflow_config
|
||||
```
|
||||
|
||||
Add below `build_workflow_server_from_config`:
|
||||
|
||||
```python
|
||||
def build_workflow_server_from_workflow_config(
|
||||
config: WorkflowConfigFile,
|
||||
) -> WorkflowServer:
|
||||
"""Build an MCP-backed WorkflowServer from neutral workflow config sources."""
|
||||
return build_workflow_server_from_config(
|
||||
broker_config_from_workflow_config(config)
|
||||
)
|
||||
```
|
||||
|
||||
Add it to `__all__` in this file and `src/wf_mcp/broker/__init__.py`.
|
||||
|
||||
- [ ] **Step 2: Add CLI selection test**
|
||||
|
||||
In `tests/wf_transport_rpc_http/test_cli.py`, add:
|
||||
|
||||
```python
|
||||
def test_rpc_server_cli_config_with_mcp_source_uses_mcp_builder(
|
||||
monkeypatch, tmp_path
|
||||
) -> None:
|
||||
captured = {}
|
||||
|
||||
def fake_build_from_workflow_config(config):
|
||||
captured["source_kinds"] = [source.kind for source in config.server.sources]
|
||||
return object()
|
||||
|
||||
def fake_create_rpc_app(server, *, rpc_path="/rpc"):
|
||||
captured["server"] = server
|
||||
captured["rpc_path"] = rpc_path
|
||||
return "app"
|
||||
|
||||
def fake_run(app, *, host, port, access_log):
|
||||
captured["run"] = {
|
||||
"app": app,
|
||||
"host": host,
|
||||
"port": port,
|
||||
"access_log": access_log,
|
||||
}
|
||||
|
||||
monkeypatch.setattr(
|
||||
"wf_transport_rpc_http.cli.build_workflow_server_from_workflow_config",
|
||||
fake_build_from_workflow_config,
|
||||
)
|
||||
monkeypatch.setattr("wf_transport_rpc_http.cli.create_rpc_app", fake_create_rpc_app)
|
||||
monkeypatch.setattr("wf_transport_rpc_http.cli.uvicorn.run", fake_run)
|
||||
|
||||
config_path = tmp_path / "wf.json"
|
||||
config_path.write_text(
|
||||
"""
|
||||
{
|
||||
"version": 1,
|
||||
"server": {
|
||||
"store": {"kind": "filesystem", "root": ".wf_store"},
|
||||
"transports": [{"kind": "rpc_http", "host": "127.0.0.1", "port": 8765}],
|
||||
"sources": [
|
||||
{
|
||||
"kind": "mcp",
|
||||
"id": "everything.default",
|
||||
"provider": "everything",
|
||||
"account": "default",
|
||||
"transport": {"kind": "stdio", "command": "uvx"}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
""",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
from wf_transport_rpc_http.cli import app
|
||||
from typer.testing import CliRunner
|
||||
|
||||
result = CliRunner().invoke(app, ["--config", str(config_path)])
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert captured["source_kinds"] == ["mcp"]
|
||||
assert captured["run"]["app"] == "app"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run test and verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_cli.py::test_rpc_server_cli_config_with_mcp_source_uses_mcp_builder -q
|
||||
```
|
||||
|
||||
Expected: fail because `wf_transport_rpc_http.cli` does not import or use `build_workflow_server_from_workflow_config`.
|
||||
|
||||
## Task 4: Wire RPC Server CLI to Neutral MCP Sources
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_transport_rpc_http/cli.py`
|
||||
|
||||
- [ ] **Step 1: Import the new builder**
|
||||
|
||||
Update the MCP import line:
|
||||
|
||||
```python
|
||||
from wf_mcp.broker import (
|
||||
build_workflow_server_from_config,
|
||||
build_workflow_server_from_workflow_config,
|
||||
load_broker_config,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Select MCP builder when neutral config has MCP sources**
|
||||
|
||||
Inside the `if config is not None:` block, after `workflow_config = load_workflow_config(config)`, add:
|
||||
|
||||
```python
|
||||
has_mcp_sources = any(
|
||||
getattr(source, "kind", None) == "mcp"
|
||||
for source in workflow_config.server.sources
|
||||
)
|
||||
if server is None and has_mcp_sources:
|
||||
server = build_workflow_server_from_workflow_config(workflow_config)
|
||||
```
|
||||
|
||||
Keep the existing filesystem-store validation guarded by `server is None`.
|
||||
|
||||
- [ ] **Step 3: Run CLI test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_cli.py::test_rpc_server_cli_config_with_mcp_source_uses_mcp_builder -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 5: End-to-End RPC Composition Test
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/wf_transport_rpc_http/test_mcp_backed_server_rpc.py`
|
||||
|
||||
- [ ] **Step 1: Add direct neutral config server test**
|
||||
|
||||
Append:
|
||||
|
||||
```python
|
||||
from wf_config import WorkflowConfigFile
|
||||
from wf_mcp.broker.server import build_workflow_server_from_workflow_config
|
||||
|
||||
|
||||
async def test_mcp_backed_rpc_can_be_built_from_neutral_workflow_config(
|
||||
tmp_path,
|
||||
) -> None:
|
||||
workflow_config = WorkflowConfigFile.model_validate(
|
||||
{
|
||||
"version": 1,
|
||||
"server": {
|
||||
"store": {"kind": "filesystem", "root": str(tmp_path / "store")},
|
||||
"sources": [
|
||||
{
|
||||
"kind": "mcp",
|
||||
"id": "demo.default",
|
||||
"provider": "demo",
|
||||
"account": "default",
|
||||
"transport": {"kind": "stdio", "command": "demo-server"},
|
||||
}
|
||||
],
|
||||
},
|
||||
}
|
||||
)
|
||||
server = build_workflow_server_from_workflow_config(workflow_config)
|
||||
app = create_rpc_app(server)
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
|
||||
async with httpx.AsyncClient(
|
||||
transport=transport, base_url="http://test"
|
||||
) as http_client:
|
||||
connections = await _rpc(
|
||||
http_client, "workflow.admin.connections.list", {}
|
||||
)
|
||||
|
||||
assert connections["result"]["connections"][0]["id"] == "demo.default"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_transport_rpc_http/test_mcp_backed_server_rpc.py::test_mcp_backed_rpc_can_be_built_from_neutral_workflow_config -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 6: Document Completion and Legacy Status
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
- Modify: `docs/superpowers/specs/2026-06-03-long-lived-workflow-api-boundary.md`
|
||||
- Modify: `docs/wf_cli.md`
|
||||
|
||||
- [ ] **Step 1: Update roadmap**
|
||||
|
||||
In `docs/current_roadmap.md`, under the wider config bullet, append:
|
||||
|
||||
```markdown
|
||||
Runtime bridge complete: neutral `kind: "mcp"` source entries can now build
|
||||
the MCP-backed `WorkflowServer`. `--mcp-config` remains supported as a
|
||||
legacy compatibility path while new configs should prefer
|
||||
`server.sources[]`.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update long-lived API spec**
|
||||
|
||||
In the Slice 1/2 status area, append:
|
||||
|
||||
```markdown
|
||||
Runtime bridge complete when `wf-rpc-server --config <path>` can compose an
|
||||
MCP-backed server from neutral `server.sources[]` entries. `--mcp-config`
|
||||
remains a compatibility alias until existing users migrate.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update CLI docs**
|
||||
|
||||
In `docs/wf_cli.md`, under "Remote Server", add a neutral config example:
|
||||
|
||||
```markdown
|
||||
Prefer neutral workflow config for new MCP-backed servers:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"server": {
|
||||
"store": {"kind": "filesystem", "root": ".wf_store"},
|
||||
"transports": [{"kind": "rpc_http", "host": "127.0.0.1", "port": 8765}],
|
||||
"sources": [
|
||||
{
|
||||
"kind": "mcp",
|
||||
"id": "everything.default",
|
||||
"provider": "everything",
|
||||
"account": "default",
|
||||
"transport": {"kind": "stdio", "command": "uvx", "args": ["mcp-server-everything"]}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`--mcp-config` is still accepted for legacy broker config files.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run docs diff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git diff -- docs/current_roadmap.md docs/superpowers/specs/2026-06-03-long-lived-workflow-api-boundary.md docs/wf_cli.md
|
||||
```
|
||||
|
||||
Expected: docs describe neutral config as preferred and `--mcp-config` as legacy.
|
||||
|
||||
## Task 7: Final Verification and Commit
|
||||
|
||||
**Files:**
|
||||
- All touched files from prior tasks.
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_mcp/test_workflow_config_bridge.py tests/wf_transport_rpc_http/test_cli.py tests/wf_transport_rpc_http/test_mcp_backed_server_rpc.py -q
|
||||
```
|
||||
|
||||
Expected: pass. If the existing import-direction guard still fails in unrelated runs, mention it in the report; this slice reduces but may not fully remove the current `wf_transport_rpc_http.cli -> wf_mcp` dependency.
|
||||
|
||||
- [ ] **Step 2: Run lint/type checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_mcp src/wf_transport_rpc_http tests/wf_mcp tests/wf_transport_rpc_http
|
||||
uv run basedpyright --level error src/wf_mcp src/wf_transport_rpc_http tests/wf_mcp tests/wf_transport_rpc_http
|
||||
```
|
||||
|
||||
Expected: both pass with 0 errors.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git add src/wf_mcp src/wf_transport_rpc_http tests/wf_mcp tests/wf_transport_rpc_http docs/current_roadmap.md docs/superpowers/specs/2026-06-03-long-lived-workflow-api-boundary.md docs/wf_cli.md
|
||||
git commit -m "feat: build mcp server from workflow config"
|
||||
```
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- This plan does not move MCP runtime/session logic into `wf_config`.
|
||||
- Neutral config is the preferred new user-facing shape.
|
||||
- Legacy `--mcp-config` remains supported.
|
||||
- `server.sources[]` is the source of truth for new MCP-backed server config.
|
||||
- The JSON-RPC method/app/client modules stay transport-only; only launcher/composition code touches MCP.
|
||||
@@ -0,0 +1,456 @@
|
||||
# wf_config MCP Source Model Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add a neutral `kind: "mcp"` source variant to `wf_config.server.sources[]` so MCP source definitions can live in the wider workflow config model.
|
||||
|
||||
**Architecture:** Keep this slice pure `wf_config`: no imports from `wf_mcp`, no server construction changes, no runtime behavior changes. The new config shape should intentionally mirror `wf_mcp.source_registry.McpSourceRegistryEntry`, with the legacy `ConnectionConfig.source_config_ownership` policy renamed to a neutral config/source field.
|
||||
|
||||
**Tech Stack:** Pydantic v2 discriminated unions, `AnyHttpUrl`, existing `tests/wf_config/test_config_models.py`.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `src/wf_config/models.py`: add transport config models, `McpSourceConfig`, and include it in the existing `SourceConfig` union.
|
||||
- Modify `src/wf_config/__init__.py`: export the new config types.
|
||||
- Modify `tests/wf_config/test_config_models.py`: add model parsing/validation tests for MCP sources.
|
||||
- Modify `docs/current_roadmap.md` and `docs/superpowers/specs/2026-06-03-long-lived-workflow-api-boundary.md`: mark this slice complete after implementation.
|
||||
|
||||
## Current Context
|
||||
|
||||
Current neutral config has:
|
||||
|
||||
```python
|
||||
class StdlibSourceConfig(WorkflowConfigModel):
|
||||
kind: Literal["stdlib"]
|
||||
id: Literal["wf.std", "wf.recipes"]
|
||||
|
||||
|
||||
SourceConfig = Annotated[
|
||||
StdlibSourceConfig,
|
||||
Field(discriminator="kind"),
|
||||
]
|
||||
```
|
||||
|
||||
MCP’s desired source shape already exists at `src/wf_mcp/source_registry.py` as `McpSourceRegistryEntry`, but that module imports MCP-specific validators and models. Do not import it into `wf_config`.
|
||||
|
||||
The source id rule to mirror is currently in `src/wf_mcp/connections.py`:
|
||||
|
||||
```python
|
||||
CONNECTION_ID_PATTERN = r"^[A-Za-z0-9_][A-Za-z0-9_.-]*$"
|
||||
```
|
||||
|
||||
and MCP ids must look like `<provider>.<account>`.
|
||||
|
||||
## Task 1: Add MCP Source Config Tests
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/wf_config/test_config_models.py`
|
||||
|
||||
- [ ] **Step 1: Add imports**
|
||||
|
||||
Update the import block from `wf_config` to include the new classes that will be implemented:
|
||||
|
||||
```python
|
||||
from wf_config import (
|
||||
FilesystemStoreConfig,
|
||||
HttpSourceTransportConfig,
|
||||
LocalTargetConfig,
|
||||
McpSourceConfig,
|
||||
RpcHttpTargetConfig,
|
||||
RpcHttpTransportConfig,
|
||||
StdioSourceTransportConfig,
|
||||
StdlibSourceConfig,
|
||||
WorkflowConfigFile,
|
||||
load_workflow_config,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add stdio MCP source parsing test**
|
||||
|
||||
Append this test:
|
||||
|
||||
```python
|
||||
def test_workflow_config_parses_mcp_stdio_source() -> None:
|
||||
config = WorkflowConfigFile.model_validate(
|
||||
{
|
||||
"version": 1,
|
||||
"server": {
|
||||
"sources": [
|
||||
{
|
||||
"kind": "mcp",
|
||||
"id": "everything.default",
|
||||
"enabled": True,
|
||||
"provider": "everything",
|
||||
"account": "default",
|
||||
"profile": "dev",
|
||||
"ownership": "seed",
|
||||
"transport": {
|
||||
"kind": "stdio",
|
||||
"command": "uvx",
|
||||
"args": ["mcp-server-everything"],
|
||||
"env": {"DEBUG": "1"},
|
||||
},
|
||||
"auth_ref": "auth.everything.default",
|
||||
"metadata": {"description": "Everything test server"},
|
||||
}
|
||||
]
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
source = config.server.sources[0]
|
||||
assert isinstance(source, McpSourceConfig)
|
||||
assert source.id == "everything.default"
|
||||
assert source.enabled is True
|
||||
assert source.provider == "everything"
|
||||
assert source.account == "default"
|
||||
assert source.profile == "dev"
|
||||
assert source.ownership == "seed"
|
||||
assert isinstance(source.transport, StdioSourceTransportConfig)
|
||||
assert source.transport.command == "uvx"
|
||||
assert source.transport.args == ("mcp-server-everything",)
|
||||
assert source.transport.env == {"DEBUG": "1"}
|
||||
assert source.auth_ref == "auth.everything.default"
|
||||
assert source.metadata["description"] == "Everything test server"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add HTTP MCP source parsing test**
|
||||
|
||||
Append this test:
|
||||
|
||||
```python
|
||||
def test_workflow_config_parses_mcp_http_source() -> None:
|
||||
config = WorkflowConfigFile.model_validate(
|
||||
{
|
||||
"version": 1,
|
||||
"server": {
|
||||
"sources": [
|
||||
{
|
||||
"kind": "mcp",
|
||||
"id": "context7.default",
|
||||
"provider": "context7",
|
||||
"account": "default",
|
||||
"transport": {
|
||||
"kind": "http",
|
||||
"url": "http://127.0.0.1:3000/mcp",
|
||||
"headers": {"X-Test": "yes"},
|
||||
},
|
||||
}
|
||||
]
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
source = config.server.sources[0]
|
||||
assert isinstance(source, McpSourceConfig)
|
||||
assert source.enabled is True
|
||||
assert source.ownership == "locked"
|
||||
assert isinstance(source.transport, HttpSourceTransportConfig)
|
||||
assert str(source.transport.url) == "http://127.0.0.1:3000/mcp"
|
||||
assert source.transport.headers == {"X-Test": "yes"}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add validation tests**
|
||||
|
||||
Append these tests:
|
||||
|
||||
```python
|
||||
def test_workflow_config_rejects_mcp_source_without_provider_account_shape() -> None:
|
||||
with pytest.raises(ValidationError, match="source id must look like"):
|
||||
WorkflowConfigFile.model_validate(
|
||||
{
|
||||
"version": 1,
|
||||
"server": {
|
||||
"sources": [
|
||||
{
|
||||
"kind": "mcp",
|
||||
"id": "everything",
|
||||
"provider": "everything",
|
||||
"account": "default",
|
||||
"transport": {"kind": "stdio", "command": "uvx"},
|
||||
}
|
||||
]
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_workflow_config_rejects_unsafe_mcp_source_id() -> None:
|
||||
with pytest.raises(ValidationError, match="source id must start"):
|
||||
WorkflowConfigFile.model_validate(
|
||||
{
|
||||
"version": 1,
|
||||
"server": {
|
||||
"sources": [
|
||||
{
|
||||
"kind": "mcp",
|
||||
"id": ".hidden.default",
|
||||
"provider": "hidden",
|
||||
"account": "default",
|
||||
"transport": {"kind": "stdio", "command": "uvx"},
|
||||
}
|
||||
]
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_workflow_config_rejects_duplicate_source_ids_across_kinds() -> None:
|
||||
with pytest.raises(ValidationError, match="duplicate source id"):
|
||||
WorkflowConfigFile.model_validate(
|
||||
{
|
||||
"version": 1,
|
||||
"server": {
|
||||
"sources": [
|
||||
{"kind": "stdlib", "id": "wf.std"},
|
||||
{
|
||||
"kind": "mcp",
|
||||
"id": "wf.std",
|
||||
"provider": "wf",
|
||||
"account": "std",
|
||||
"transport": {"kind": "stdio", "command": "uvx"},
|
||||
},
|
||||
]
|
||||
},
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run tests and verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_config/test_config_models.py -q
|
||||
```
|
||||
|
||||
Expected: fail with import errors for `McpSourceConfig`, `StdioSourceTransportConfig`, and `HttpSourceTransportConfig`.
|
||||
|
||||
## Task 2: Implement Neutral MCP Source Config Models
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_config/models.py`
|
||||
|
||||
- [ ] **Step 1: Add imports**
|
||||
|
||||
Update imports:
|
||||
|
||||
```python
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Literal
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add ownership and source id constants**
|
||||
|
||||
Place after `ServerTransportConfig`:
|
||||
|
||||
```python
|
||||
SourceConfigOwnership = Literal["locked", "seed"]
|
||||
SOURCE_ID_PATTERN = r"^[A-Za-z0-9_][A-Za-z0-9_.-]*$"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add transport models**
|
||||
|
||||
Place before `StdlibSourceConfig`:
|
||||
|
||||
```python
|
||||
class StdioSourceTransportConfig(WorkflowConfigModel):
|
||||
kind: Literal["stdio"] = "stdio"
|
||||
command: str = Field(min_length=1)
|
||||
args: tuple[str, ...] = ()
|
||||
env: dict[str, str] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class HttpSourceTransportConfig(WorkflowConfigModel):
|
||||
kind: Literal["http"] = "http"
|
||||
url: AnyHttpUrl
|
||||
headers: dict[str, str] = Field(default_factory=dict)
|
||||
|
||||
|
||||
SourceTransportConfig = Annotated[
|
||||
StdioSourceTransportConfig | HttpSourceTransportConfig,
|
||||
Field(discriminator="kind"),
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add MCP source model**
|
||||
|
||||
Place after `StdlibSourceConfig`:
|
||||
|
||||
```python
|
||||
class McpSourceConfig(WorkflowConfigModel):
|
||||
"""Neutral config shape for MCP-backed workflow capability sources.
|
||||
|
||||
This intentionally mirrors `wf_mcp.source_registry.McpSourceRegistryEntry`
|
||||
without importing MCP modules. `ownership` carries the old
|
||||
`ConnectionConfig.source_config_ownership` policy with neutral terminology.
|
||||
"""
|
||||
|
||||
kind: Literal["mcp"] = "mcp"
|
||||
id: str
|
||||
enabled: bool = True
|
||||
provider: str = Field(min_length=1)
|
||||
account: str = Field(min_length=1)
|
||||
profile: str | None = None
|
||||
ownership: SourceConfigOwnership = "locked"
|
||||
transport: SourceTransportConfig
|
||||
auth_ref: str | None = None
|
||||
metadata: dict[str, object] = Field(default_factory=dict)
|
||||
|
||||
@field_validator("id")
|
||||
@classmethod
|
||||
def validate_source_id(cls, value: str) -> str:
|
||||
if not re.fullmatch(SOURCE_ID_PATTERN, value):
|
||||
raise ValueError(
|
||||
"source id must start with alphanumeric or underscore and contain "
|
||||
"only [A-Za-z0-9_.-]"
|
||||
)
|
||||
if "." not in value:
|
||||
raise ValueError("source id must look like '<provider>.<account>'")
|
||||
provider, account = value.split(".", 1)
|
||||
if not provider or not account:
|
||||
raise ValueError("source id must look like '<provider>.<account>'")
|
||||
return value
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Extend SourceConfig union**
|
||||
|
||||
Replace the existing union with:
|
||||
|
||||
```python
|
||||
SourceConfig = Annotated[
|
||||
StdlibSourceConfig | McpSourceConfig,
|
||||
Field(discriminator="kind"),
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_config/test_config_models.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 3: Export New Config Types
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_config/__init__.py`
|
||||
|
||||
- [ ] **Step 1: Add imports**
|
||||
|
||||
Update the import block to include:
|
||||
|
||||
```python
|
||||
HttpSourceTransportConfig,
|
||||
McpSourceConfig,
|
||||
SourceConfigOwnership,
|
||||
SourceTransportConfig,
|
||||
StdioSourceTransportConfig,
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add `__all__` entries**
|
||||
|
||||
Add:
|
||||
|
||||
```python
|
||||
"HttpSourceTransportConfig",
|
||||
"McpSourceConfig",
|
||||
"SourceConfigOwnership",
|
||||
"SourceTransportConfig",
|
||||
"StdioSourceTransportConfig",
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_config/test_config_models.py -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
## Task 4: Document Slice Completion
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
- Modify: `docs/superpowers/specs/2026-06-03-long-lived-workflow-api-boundary.md`
|
||||
|
||||
- [ ] **Step 1: Update roadmap**
|
||||
|
||||
In `docs/current_roadmap.md`, under the "Wider `wf_config` source model" bullet, append:
|
||||
|
||||
```markdown
|
||||
First slice complete: `wf_config.server.sources[]` now accepts
|
||||
`kind: "mcp"` entries with stdio/http transport, auth reference, metadata,
|
||||
enabled flag, and `locked` / `seed` ownership policy. Runtime composition
|
||||
from these entries is the next slice.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update spec**
|
||||
|
||||
In `docs/superpowers/specs/2026-06-03-long-lived-workflow-api-boundary.md`, under the Slice 1 text, append:
|
||||
|
||||
```markdown
|
||||
Model slice complete when `wf_config.server.sources[]` accepts `kind: "mcp"`
|
||||
entries. The next slice converts those neutral source entries into MCP
|
||||
broker runtime connections and server composition.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run docs diff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git diff -- docs/current_roadmap.md docs/superpowers/specs/2026-06-03-long-lived-workflow-api-boundary.md
|
||||
```
|
||||
|
||||
Expected: only the completion notes above.
|
||||
|
||||
## Task 5: Final Verification and Commit
|
||||
|
||||
**Files:**
|
||||
- All touched files from prior tasks.
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/wf_config/test_config_models.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
- [ ] **Step 2: Run lint/type checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/wf_config tests/wf_config
|
||||
uv run basedpyright --level error src/wf_config tests/wf_config
|
||||
```
|
||||
|
||||
Expected: both pass with 0 errors.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git add src/wf_config tests/wf_config docs/current_roadmap.md docs/superpowers/specs/2026-06-03-long-lived-workflow-api-boundary.md
|
||||
git commit -m "feat: add mcp source config model"
|
||||
```
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- This plan does not import `wf_mcp` into `wf_config`.
|
||||
- This plan does not change runtime behavior.
|
||||
- The ownership field is neutral (`ownership`), while docs explain its legacy origin.
|
||||
- Duplicate source id validation continues to work across all source kinds.
|
||||
@@ -59,7 +59,9 @@ composition. It should not call `WfMcpService`.
|
||||
|
||||
HTTP is one transport adapter, not "the API." JSON-RPC over HTTP, JSON-RPC over
|
||||
WebSocket, and possibly MCP can become sibling transports around the same
|
||||
server/application boundary.
|
||||
server/application boundary. Server composition should be driven by the wider
|
||||
neutral `wf_config` model, including source definitions; MCP broker config is a
|
||||
legacy/source-specific input to normalize, not a permanent peer config family.
|
||||
|
||||
`wf_server` may initially be small. Its role is to prove that a non-MCP process
|
||||
can construct the same application boundary with required stores and an explicit
|
||||
@@ -364,16 +366,37 @@ Current MCP-backed server status:
|
||||
- `wf-rpc-server --mcp-config <path>` can serve JSON-RPC over that server.
|
||||
- Source registry read/mutation APIs are reachable remotely when the target
|
||||
exposes `source_registry_admin`.
|
||||
- Boundary caveat: the first `--mcp-config` hook intentionally proved the
|
||||
product path quickly, but it currently makes the transport CLI import
|
||||
`wf_mcp.broker`. That violates the original transport-package boundary and
|
||||
the existing import-direction guard. Treat this as a cleanup slice, not the
|
||||
desired final shape. The intended cleanup is to widen `wf_config` so MCP
|
||||
sources are configured through `server.sources[]`, with `wf_mcp.config.json`
|
||||
handled as compatibility input.
|
||||
|
||||
Next implementation slices should be:
|
||||
|
||||
1. Manual product smoke with the real CLI/server commands. Record UX/runtime
|
||||
1. Wider `wf_config` source model. Add an MCP source config variant under
|
||||
`server.sources[]` that can express the current broker connection shape:
|
||||
source id, provider/account/profile, `locked` / `seed` ownership, stdio/http
|
||||
transport, auth reference, enabled flag, and metadata. Keep legacy
|
||||
`wf_mcp.config.json` parsing as a compatibility adapter into the wider
|
||||
config, not as the future primary shape.
|
||||
`McpSourceRegistryEntry` already expresses most of this shape. The
|
||||
`locked` / `seed` policy currently lives on legacy
|
||||
`ConnectionConfig.source_config_ownership`; migrate that as a neutral source
|
||||
ownership/config policy field, not as a connection-specific name.
|
||||
2. Transport package boundary cleanup. Keep JSON-RPC method/app/client modules
|
||||
transport-only. After `wf_config` can describe MCP sources, `wf-rpc-server
|
||||
--config ...` should compose MCP-backed sources from neutral config and the
|
||||
`--mcp-config` path can become deprecated/legacy.
|
||||
3. Manual product smoke with the real CLI/server commands. Record UX/runtime
|
||||
gaps before broadening architecture.
|
||||
2. Source registry apply/reload semantics. Registry mutation currently updates
|
||||
4. Source registry apply/reload semantics. Registry mutation currently updates
|
||||
desired persisted state; the next explicit decision is whether changes apply
|
||||
only after restart, through an explicit reload/apply operation, or through
|
||||
automatic live reconciliation. Prefer explicit reload/apply for v1.
|
||||
3. Persisted resume across server restart. Rebuild the MCP-backed RPC server
|
||||
5. Persisted resume across server restart. Rebuild the MCP-backed RPC server
|
||||
from the same stores and prove interrupted runs resume from the stored
|
||||
checkpoint and pinned dependency environment.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user