wf-mcp reorg 3 the generated docs

docs/wf_mcp_architecture.md

@@ -0,0 +1,53 @@
+# wf_mcp Architecture Boundaries
+
+`wf_mcp` is one distribution for now, but it is organized as separable concerns.
+The goal is to keep future package extraction cheap without adding packaging
+overhead before the APIs settle.
+
+## Packages
+
+| Package | Responsibility |
+| --- | --- |
+| `wf_mcp.transparent_proxy` | Expose configured upstream MCP servers as a transparent MCP proxy. Owns proxy runtime, admin tools, and proxy tool listing helpers. |
+| `wf_mcp.broker` | Coordinate remembered connections, catalog snapshots, discovery, events, and workflow execution through broker services. |
+| `wf_mcp.workflow` | Convert discovered MCP tools into `wf_authoring` / `wf_core` node specs. |
+| `wf_mcp.sdk` | Speak to upstream MCP servers through the MCP Python SDK. Owns adapter protocols, SDK transport/session calls, and SDK object converters. |
+| `wf_mcp.control` | Parse and mutate file-backed proxy/broker configuration. |
+| `wf_mcp.storage` | Persist auth records and catalog snapshots. |
+| `wf_mcp.shared` | Pure helpers used across concerns, such as names, pagination, and error payloads. |
+
+Root modules such as `wf_mcp.store`, `wf_mcp.service`, and
+`wf_mcp.mcp_sdk_adapter` are compatibility shims. New internal imports should
+prefer the concern package directly.
+
+## Dependency Rules
+
+- `wf_mcp.sdk` should not import `wf_core` or `wf_authoring`.
+- `wf_mcp.transparent_proxy` should not import `wf_mcp.workflow`.
+- `wf_mcp.workflow` is the only layer that converts MCP capabilities into node specs.
+- `wf_mcp.broker` may coordinate `sdk`, `storage`, `control`, and `workflow`.
+- `wf_mcp.control` should not know about live MCP clients or workflow execution.
+- `wf_mcp.shared` should stay pure and should not import other `wf_mcp` concern packages.
+- Root compatibility shims should stay thin: import and re-export only.
+
+## Hot Reload
+
+Transparent proxy reload is intentionally isolated in
+`wf_mcp.transparent_proxy.runtime`. FastMCP does not currently expose a complete
+provider/proxy unmount lifecycle that we can rely on for safe per-connection
+teardown. Until that exists, reload should be treated as best-effort remounting,
+not a fully safe session/subscription lifecycle.
+
+Do not add notification proxying or long-lived subscription handling across
+reloads without first introducing an explicit mount lifecycle boundary.
+
+## Future Extraction
+
+If this becomes multiple distributions, likely split points are:
+
+- `wf-mcp-proxy`: `transparent_proxy`, `control`, `shared`
+- `wf-mcp-broker`: `broker`, `storage`, `workflow`, `shared`
+- `wf-mcp-sdk`: `sdk`, `capabilities`, `models`, `shared`
+
+For now, keep one distribution and use import discipline to preserve those
+boundaries.

pyproject.toml

@@ -2,7 +2,7 @@
 name = "lda-wf"
 version = "0.0.1"
 description = "Add your description here"
-readme = "README.md"
+readme = "readme.md"
 authors = [{ name = "lda", email = "[email protected]" }]
 requires-python = ">=3.14"
 dependencies = [

readme.md

@@ -2,9 +2,10 @@
 
 ## Related docs
 
-- [authoring_sketch.md](authoring_sketch.md): `wf_authoring` direction, including `@node`, `NodeSpec`, builder ergonomics, async registry seams, and subgraph-as-node.
+- [docs/authoring_sketch.md](docs/authoring_sketch.md): `wf_authoring` direction, including `@node`, `NodeSpec`, builder ergonomics, async registry seams, and subgraph-as-node.
-- [wf_mcp_plan.md](wf_mcp_plan.md): `wf_mcp` direction as a namespaced MCP capability broker plus workflow build/run layer.
+- [docs/wf_mcp_plan.md](docs/wf_mcp_plan.md): `wf_mcp` direction as a namespaced MCP capability broker plus workflow build/run layer.
-- [scratchpad.md](scratchpad.md): rougher design history and intermediate spec notes that fed the current model.
+- [docs/wf_mcp_architecture.md](docs/wf_mcp_architecture.md): current `wf_mcp` package boundaries and dependency rules.
+- [docs/scratchpad.md](docs/scratchpad.md): rougher design history and intermediate spec notes that fed the current model.
 
 ## What is this
 

src/wf_mcp/sdk/adapter.py

@@ -8,7 +8,6 @@ from mcp import ClientResult
 from mcp.client.session import ClientSession
 from mcp.client.stdio import StdioServerParameters, stdio_client
 from mcp.client.streamable_http import streamable_http_client
-from mcp.types import CallToolResult as McpCallToolResult
 from mcp.types import (
     ClientNotification,
     ClientRequest,
@@ -16,14 +15,17 @@ from mcp.types import (
     ListResourcesResult,
     ListToolsResult,
 )
-from mcp.types import Prompt as McpPrompt
-from mcp.types import Resource as McpResource
-from mcp.types import Tool as McpTool
 from pydantic import AnyUrl
 
 from ..capabilities import DiscoveredPrompt, DiscoveredResource, DiscoveredTool
 from ..models import AuthRecord, ConnectionConfig
 from .base import BackendAdapter, ToolCallResult
+from .converters import (
+    prompt_to_discovered,
+    resource_to_discovered,
+    tool_result_to_call_result,
+    tool_to_discovered,
+)
 
 
 def _auth_headers(auth: AuthRecord | None) -> dict[str, str]:
@@ -36,67 +38,6 @@ def _auth_headers(auth: AuthRecord | None) -> dict[str, str]:
     return headers
 
 
-def _tool_to_discovered(tool: McpTool) -> DiscoveredTool:
-    output_schema = tool.outputSchema or {
-        "type": "object",
-        "properties": {"content": {"type": "array"}},
-    }
-    display_name = (
-        tool.annotations.title
-        if tool.annotations is not None and tool.annotations.title
-        else tool.title
-    )
-    return DiscoveredTool(
-        name=tool.name,
-        title=display_name,
-        description=tool.description,
-        input_schema=tool.inputSchema,
-        output_schema=output_schema,
-        outcomes=("ok", "error"),
-        metadata=tool.model_dump(by_alias=True, mode="json"),
-    )
-
-
-def _resource_to_discovered(resource: McpResource) -> DiscoveredResource:
-    local_name = resource.name or str(resource.uri)
-    return DiscoveredResource(
-        uri=str(resource.uri),
-        name=local_name,
-        title=resource.title,
-        description=resource.description,
-        mime_type=resource.mimeType,
-        metadata=resource.model_dump(by_alias=True, mode="json"),
-    )
-
-
-def _prompt_to_discovered(prompt: McpPrompt) -> DiscoveredPrompt:
-    arguments = [
-        argument.model_dump(by_alias=True, mode="json")
-        for argument in prompt.arguments or []
-    ]
-    return DiscoveredPrompt(
-        name=prompt.name,
-        title=prompt.title,
-        description=prompt.description,
-        arguments=arguments,
-        metadata=prompt.model_dump(by_alias=True, mode="json"),
-    )
-
-
-def _tool_result_to_call_result(result: McpCallToolResult) -> ToolCallResult:
-    if result.structuredContent is not None:
-        output = result.structuredContent
-    else:
-        output = {
-            "content": [item.model_dump(by_alias=True) for item in result.content]
-        }
-    return ToolCallResult(
-        outcome="error" if result.isError else "ok",
-        output=output,
-        meta=result.meta or {},
-    )
-
-
 class McpSdkAdapter(BackendAdapter):
     @asynccontextmanager
     async def _session(
@@ -149,7 +90,7 @@ class McpSdkAdapter(BackendAdapter):
     ) -> list[DiscoveredTool]:
         async with self._session(connection, auth) as session:
             result: ListToolsResult = await session.list_tools()
-            return [_tool_to_discovered(tool) for tool in result.tools]
+            return [tool_to_discovered(tool) for tool in result.tools]
 
     async def list_resources(
         self,
@@ -158,7 +99,7 @@ class McpSdkAdapter(BackendAdapter):
     ) -> list[DiscoveredResource]:
         async with self._session(connection, auth) as session:
             result: ListResourcesResult = await session.list_resources()
-            return [_resource_to_discovered(resource) for resource in result.resources]
+            return [resource_to_discovered(resource) for resource in result.resources]
 
     async def list_prompts(
         self,
@@ -167,7 +108,7 @@ class McpSdkAdapter(BackendAdapter):
     ) -> list[DiscoveredPrompt]:
         async with self._session(connection, auth) as session:
             result: ListPromptsResult = await session.list_prompts()
-            return [_prompt_to_discovered(prompt) for prompt in result.prompts]
+            return [prompt_to_discovered(prompt) for prompt in result.prompts]
 
     async def get_connection_metadata(
         self,
@@ -235,4 +176,4 @@ class McpSdkAdapter(BackendAdapter):
     ) -> ToolCallResult:
         async with self._session(connection, auth) as session:
             result = await session.call_tool(tool_name, payload)
-            return _tool_result_to_call_result(result)
+            return tool_result_to_call_result(result)

src/wf_mcp/sdk/converters.py

@@ -0,0 +1,74 @@
+from __future__ import annotations
+
+from mcp.types import CallToolResult as McpCallToolResult
+from mcp.types import Prompt as McpPrompt
+from mcp.types import Resource as McpResource
+from mcp.types import Tool as McpTool
+
+from ..capabilities import DiscoveredPrompt, DiscoveredResource, DiscoveredTool
+from .base import ToolCallResult
+
+
+def tool_to_discovered(tool: McpTool) -> DiscoveredTool:
+    """Convert an MCP SDK tool into the broker discovery model."""
+    output_schema = tool.outputSchema or {
+        "type": "object",
+        "properties": {"content": {"type": "array"}},
+    }
+    display_name = (
+        tool.annotations.title
+        if tool.annotations is not None and tool.annotations.title
+        else tool.title
+    )
+    return DiscoveredTool(
+        name=tool.name,
+        title=display_name,
+        description=tool.description,
+        input_schema=tool.inputSchema,
+        output_schema=output_schema,
+        outcomes=("ok", "error"),
+        metadata=tool.model_dump(by_alias=True, mode="json"),
+    )
+
+
+def resource_to_discovered(resource: McpResource) -> DiscoveredResource:
+    """Convert an MCP SDK resource into the broker discovery model."""
+    local_name = resource.name or str(resource.uri)
+    return DiscoveredResource(
+        uri=str(resource.uri),
+        name=local_name,
+        title=resource.title,
+        description=resource.description,
+        mime_type=resource.mimeType,
+        metadata=resource.model_dump(by_alias=True, mode="json"),
+    )
+
+
+def prompt_to_discovered(prompt: McpPrompt) -> DiscoveredPrompt:
+    """Convert an MCP SDK prompt into the broker discovery model."""
+    arguments = [
+        argument.model_dump(by_alias=True, mode="json")
+        for argument in prompt.arguments or []
+    ]
+    return DiscoveredPrompt(
+        name=prompt.name,
+        title=prompt.title,
+        description=prompt.description,
+        arguments=arguments,
+        metadata=prompt.model_dump(by_alias=True, mode="json"),
+    )
+
+
+def tool_result_to_call_result(result: McpCallToolResult) -> ToolCallResult:
+    """Convert an MCP SDK tool call result into the adapter result model."""
+    if result.structuredContent is not None:
+        output = result.structuredContent
+    else:
+        output = {
+            "content": [item.model_dump(by_alias=True) for item in result.content]
+        }
+    return ToolCallResult(
+        outcome="error" if result.isError else "ok",
+        output=output,
+        meta=result.meta or {},
+    )