lda-wf/docs/wf_mcp_architecture.md

wf_mcp Architecture Boundaries

wf_mcp is one distribution for now, but it is organized as separable concerns. The goal is to keep future package extraction cheap without adding packaging overhead before the APIs settle.

For the short operator-facing map of the current nouns and tool families, start with wf_mcp_operator_manual.md.

Packages

Package	Responsibility
wf_mcp.proxy	Expose configured upstream MCP servers through mounted FastMCP proxy providers. Owns proxy runtime, admin tools, and proxy tool listing helpers.
wf_mcp.broker	Coordinate remembered connections, catalog snapshots, discovery, events, and workflow execution through broker services.
wf_mcp.workflow	Convert discovered MCP tools into wf_authoring / wf_core node specs.
wf_mcp.sdk	Speak to upstream MCP servers through the MCP Python SDK. Owns adapter protocols, SDK transport/session calls, and SDK object converters.
wf_mcp.control	Parse and mutate file-backed proxy/broker configuration.
wf_mcp.storage	Persist auth records and catalog snapshots.
wf_mcp.shared	Pure helpers used across concerns, such as names, pagination, and error payloads.

The root wf_mcp package is a small public facade for common user entrypoints, not a dump of every internal helper. If a caller needs broker internals, SDK adapter protocols, proxy admin pieces, or shared name parsing, import the relevant concern package directly.

Dependency Rules

• wf_api defines the workflow application surface and process-local implementation. wf_mcp may import and adapt it; wf_api must not import wf_mcp.
• wf_mcp.sdk should not import wf_core or wf_authoring.
• wf_mcp.proxy should not import wf_mcp.workflow.
• wf_mcp.workflow is the only layer that converts MCP capabilities into node specs.
• wf_mcp.broker may coordinate sdk, storage, control, and workflow.
• wf_mcp.control should not know about live MCP clients or workflow execution.
• wf_mcp.shared should stay pure and should not import other wf_mcp concern packages.
• Root compatibility shims should stay thin: import and re-export only.

Workflow API Boundary

Workflow lifecycle operations now have a protocol-neutral front door:

wf_mcp.workflow_surface.tools
  -> wf_api.WorkflowApiSurface implementation
  -> wf_api.WorkflowApi
  -> wf_api domain services
  -> WorkflowOperationContext
  -> WfMcpService adapters/stores/runtime

WorkflowSurfaceHandlers is a compatibility shim only. New entrypoints should construct WorkflowApi(context_from_service(service)) directly.

The old backend-adapter/protocol layer has been removed. WorkflowApiSurface is now the structural workflow operation contract. The local WorkflowApi implementation composes domain services (WorkflowCapabilityApi, WorkflowDraftApi, WorkflowArtifactApi, WorkflowDeploymentApi, WorkflowRunApi) from a WorkflowOperationContext.

New code should type consumers against wf_api.WorkflowApiSurface when they can work with either local or remote implementations. Use concrete WorkflowApi only when same-process stores/runtime are required. Do not add new callers that import WorkflowSurfaceHandlers directly unless they are compatibility tests.

The broader application-service boundary is documented in wf_api_architecture.md.

This is a dependency-direction cleanup, not a full domain split. Most API methods still mirror the old workflow-surface payloads and return dict[str, Any].

Protocol-neutral workflow helpers have moved to wf_api: constants, capability refs, wrapper hints, next actions, raw workflow plans, runtime dependency resolution, saved subgraph preparation, and durable run lifecycle helpers. The old wf_mcp.workflow_surface.* helper modules are compatibility shims. MCP tool schemas and registration still live in wf_mcp.workflow_surface.models and wf_mcp.workflow_surface.tools.

Broker Catalogs

The broker keeps two related catalog views:

• get_catalog() is the backend MCP catalog. It only includes enabled upstream connection snapshots loaded from storage.
• get_planner_catalog() is the workflow-planning catalog. It includes backend connection snapshots plus broker-local system sources such as wf.std.

Broker-local sources are not fake MCP backend connections. They are registered as service spec sources so raw workflow plans can address nodes like wf.std.runtime_error without polluting connection status, auth, adapter lookup, or persisted backend catalog snapshots.

The longer-term source model is described in wf_mcp_capability_sources.md. In that model, sources own tools, workflow node specs, prompts, and resources, while broker, proxy, planner, and admin UI surfaces project different capability kinds.

Hot Reload

Proxy reload is intentionally isolated in wf_mcp.proxy.runtime. FastMCP does not currently expose a complete provider/proxy unmount lifecycle that we can rely on for safe per-connection teardown. Until that exists, reload should be treated as best-effort remounting, not a fully safe session/subscription lifecycle.

Unified mode currently reuses ProxyRuntime as its proxy mounting engine. The old TransparentProxyRuntime compatibility alias has been removed; use ProxyRuntime directly.

ProxyRuntime now owns a small ProxyMountRegistry. Proxy/admin tools are registered once on the top-level local provider; reload only clears the visible mounted upstream provider list and rebuilds it from current config. Unchanged enabled connections reuse their cached proxy mount instead of recreating a new client/proxy pair every time. Disabled or removed connections are no longer mounted after reload, but their cached mounts are only retired internally for now; they are not safely closed or unmounted because FastMCP does not yet expose the lifecycle hook we need.

The reused FastMCP proxies are not holding one forever-open upstream connection. create_proxy_mount() gives create_proxy(...) a disconnected client, and FastMCP creates fresh request clients from it. What persists across reload is the proxy/provider object and its component-list caches. FastMCP refreshes those caches on explicit list_* calls and otherwise expires them after its TTL.

After a successful reload, the runtime publishes local tools_changed, resources_changed, prompts_changed, and catalog_changed events when an event bus is supplied. The admin MCP tool projects the same event kinds into MCP list-changed notifications for the current client session. Config mutation tools still only stage changes and return requires_reload; they do not emit list-changed notifications until reload remounts the visible capability set. Internally, reload metadata uses ProxyReloadResult; MCP tools serialize that typed result to a plain payload at the boundary. Proxy tool listing similarly uses ProxyToolPayload / ProxyToolsPage internally and serializes to admin MCP payloads at the boundary.

Do not add more lifecycle behavior outside ProxyMountRegistry. Cached clients still need clear close/reconnect/error semantics, and the registry is the single place where that future behavior should land. Prefer FastMCP's official unmount/provider lifecycle when it becomes available.

Do not add notification proxying or long-lived subscription handling across reloads without first introducing an explicit mount lifecycle boundary.

The current practical proxy roadmap, including which FastMCP gaps are worth working around locally and which should stay upstream-dependent for now, lives in wf_mcp_proxy_reality_and_roadmap.md.

The public MCP mode split has been retired. The execution plan is historical/superpowers/plans/2026-05-16-retire-legacy-mcp-modes.md: broker and proxy were legacy public launch surfaces, while the ordinary server now exposes both local capabilities and proxied upstream capabilities. Internal concern packages remain useful even though the public mode choices are gone.

Future Extraction

If this becomes multiple distributions, likely split points are:

• wf-transport-mcp: MCP-facing transport that exposes neutral WorkflowServer / wf_api workflow, source-admin, and platform-admin surfaces to MCP clients.
• wf-sources-mcp: upstream MCP source provider that owns external MCP server discovery, sessions, tool invocation, resources/prompts, FastMCP integration, and conversion of discovered MCP tools into workflow node specs.
• wf-mcp-proxy: compatibility/proxy runtime for mounting upstream MCP servers through FastMCP, if that remains useful after wf-sources-mcp exists.

The key boundary is direction, not naming: "MCP as a client transport" and "MCP as an upstream workflow source" are separate roles. The current wf_mcp package still contains both roles plus compatibility entrypoints. New server/transport code should depend on wf_server / wf_api surfaces and keep MCP-specific upstream behavior behind a source-provider adapter.

For now, keep one distribution and use import discipline to preserve these future split points.