# Source Architecture

This map explains how workflow capability sources fit together. Use it when
deciding where to add a new source family or when untangling `wf_sources_mcp`
from the older `wf_mcp` compatibility package.

## Short Version

```text
wf_authoring
  creates NodeSpec, reducers, and authored workflows

wf_platform
  defines neutral source DTOs:
    CapabilitySource
    CapabilityBuckets
    SourceVisibility
    SourcePermissions

wf_api
  consumes source DTOs through WorkflowSpecProvider
  owns built-in local sources:
    wf.std
    wf.recipes

wf_sources_mcp
  implements MCP as an upstream source provider

wf_sources_python
  implements trusted in-process Python source loading

wf_server
  composes configured providers into WorkflowServer

wf_transport_*
  exposes WorkflowServer over protocols
```

The workflow API should not care whether a capability came from MCP, Python,
OpenAPI, or a built-in source. It should see `CapabilitySource` and executable
`NodeSpec` objects.

## Package Responsibilities

| Package | Owns | Does not own |
| --- | --- | --- |
| `wf_authoring` | `@node`, `NodeSpec`, builder DSL, reducers, reusable authored ops. | Server config, source loading, MCP sessions. |
| `wf_platform` | Neutral source DTOs and source visibility/permission metadata. | Provider-specific loading or execution. |
| `wf_api` | Application operations over capabilities, drafts, artifacts, deployments, and runs. Built-in `wf.std` / `wf.recipes` sources. | MCP/Python/OpenAPI source-specific behavior. |
| `wf_sources_mcp` | MCP source ids, connection models, auth/catalog stores, discovery, SDK facade, persistent runtime pool, converters, wrappers. | MCP frontend/proxy compatibility, durable server composition. |
| `wf_sources_python` | Trusted Python module registry loading and projection to `CapabilitySource`. | Authoring primitives, registry mutation/apply, sandboxing. |
| `wf_server` | `WorkflowServer` composition from config/store/source providers. | JSON-RPC method definitions, MCP protocol frontend. |
| `wf_transport_rpc_http` | JSON-RPC HTTP app/client around an existing `WorkflowServer`. | Server startup policy, source-provider composition. |
| `wf_client` | Async Python facade over a narrow client port: capability discovery, local authoring, artifact/deployment snapshots, and durable runs. | Draft workspace authoring, server composition, transport registration. |
| `wf_mcp` | Legacy/special-purpose MCP frontend, broker glue, proxy, compatibility shims. | New durable product behavior unless explicitly retiring old callers. |

## Data Flow

```text
wf_config.server.sources[]
  -> wf_server.config selects source providers
  -> provider loads live source inventory
  -> CapabilitySource map
  -> WorkflowSpecProvider
  -> WorkflowApi / WorkflowServer
  -> transport or CLI
```

Python applications consume the same server through `wf_client.App`; the
client's local builder and immutable snapshots sit above the transport:

```text
wf_client.App
  -> WorkflowClientPort
  -> RpcWorkflowApiClient
  -> JSON-RPC HTTP
  -> WorkflowServer / WorkflowApi
```

The first shared provider seam is intentionally static:

```python
class WorkflowSourceProvider(Protocol):
    def load_sources(self) -> Mapping[str, CapabilitySource]: ...
```

This covers source families that can project configured inventory into
workflow-facing `CapabilitySource` objects. Provider-specific runtime pools,
admin/apply hooks, auth, catalog caches, and live health checks stay outside
this narrow seam until a source family needs them.

## Capability, Tool, Resource, And Prompt

Use these terms precisely:

- A **source** is the owner and namespace. Examples: `everything.default`,
  `wf.std`, `wf.source`, or a future OpenAPI/Python source.
- A **tool** is provider-native. For MCP, it is an MCP tool discovered from an
  upstream server.
- A **workflow capability** is workflow-native. It is the `NodeSpec` shape a
  graph can call with typed input, typed output, outcomes, validation, and trace
  behavior. Tools can be projected into workflow capabilities, but the two are
  not identical concepts.
- A **resource** is source-owned addressable content. The URI is not globally
  meaningful by itself; it must be interpreted with the source that owns it.
- A **prompt** is source-owned prompt/template inventory. Listing prompts is
  inventory; rendering a prompt is an upstream operation and may be stateful.

Saved workflow data should prefer logical source references over concrete
source ids. For example, a resource ref should store:

```json
{"logical_source": "drive", "uri": "gdrive://file/abc"}
```

The deployment binding decides whether `drive` means `drive.personal`,
`drive.work`, or another concrete source. Platform sources such as `wf.std` and
`wf.source` are special because their logical source id is also their concrete
source id, so they do not require deployment bindings. Legacy explicit
self-bindings such as `wf.std=wf.std` are accepted as no-op compatibility, but
new deployments should omit them. Validation rejects non-self platform bindings
such as `wf.std=custom.std` as stale or misleading configuration.

Runtime dereference is explicit. Passing a resource ref by value does not fetch
content. A helper capability such as `wf.source.read_resource` receives the ref,
uses runtime/platform context to resolve the logical source, and applies bounded
output policy before returning text into workflow state/output.

Prompt rendering uses the same source ownership rules, but it is not yet exposed
as a workflow helper source. Keep prompt support at inventory/inspection level
until there is a concrete graph use case, argument schema contract, and bounded
output policy equivalent to `wf.source.read_resource`.

For MCP, the provider also owns stateful upstream sessions:

```text
McpSourceConnection
  -> McpRuntimePool
  -> McpSourceClient
  -> MCP ClientSession
```

For Python, the provider is simpler:

```text
PythonSourceConfig(path, module, registry)
  -> PythonSourceProvider
  -> import module
  -> load NodeSpec registry
  -> qualify specs under source id
  -> CapabilitySource(kind="python")
```

For a concrete operator flow, see the
[`Python source runbook`](runbooks/python-source.md). It covers writing `ops.py`,
configuring `path`/`module`/`registry`, validating the config, calling a
capability, and running a saved workflow deployment.

## Built-Ins Versus Configured Sources

`wf.std` and `wf.recipes` are built-in local sources owned by `wf_api.local_sources`.
They are always platform-versioned content.

Configured sources are explicit server/operator choices:

- `kind: "mcp"`: upstream MCP server capabilities.
- `kind: "python"`: trusted in-process project capabilities.
- future `kind: "openapi"`: HTTP/OpenAPI operations.

Do not move `wf.std` or `wf.recipes` into `wf_sources_python`. They are not
operator-configured project sources.

Generated draft workflows may still use built-in helper sources such as
`wf.std`. Platform sources do not need deployment bindings, so deployment
examples should bind configured sources only. If validation reports a platform
source binding, remove it unless it is an old self-binding kept only for
compatibility.

## `wf_sources_mcp` Internal Layers

`wf_sources_mcp` is clearer if read from bottom to top:

```text
ids / transports / connections
  source identity and MCP connection description

auth / storage
  source auth records and catalog cache files

client
  one live MCP ClientSession facade

runtime
  persistent session pool for stateful upstream operations

sdk
  one-shot adapter, operation protocols, and MCP-to-workflow converters

catalog / discovery / tool_wrappers
  turn MCP tools/resources/prompts into workflow-facing source inventories
```

`wf_mcp` may still re-export or adapt some of this while old callers exist. New
durable server/source work should prefer `wf_sources_mcp` directly.

## Adding A New Source Family

Start with a provider package:

```text
src/wf_sources_<kind>/
  __init__.py
  loader.py or provider.py
  tests/
```

Then add:

1. A `wf_config` discriminated-union source model.
2. A provider loader that returns `CapabilitySource`.
3. `wf_server.config` composition for that source kind.
4. Tests proving `cap list`, `cap call`, and one workflow run path.
5. Docs stating which parts are static, mutable, reloadable, or deferred.

Do not add source-family branches inside `wf_api` run execution. Source-specific
logic belongs in the provider package or server composition layer.

## Python client boundary

`wf_client` is a consumer-facing layer, not another server API.
`App.from_http_jsonrpc()` creates a lazy HTTP transport; the first
capability/artifact/deployment/run operation performs I/O. `EditableWorkflow`
keeps graph construction local and uses the port only for remote validation and
artifact persistence. The server remains responsible for durable artifact,
deployment, and run stores and for resolving concrete source bindings.

The client intentionally omits draft workspace operations. Drafts are a
server-side authoring/admin surface, and a server composition must opt in to
registering their JSON-RPC methods. This keeps a normal Python application
focused on the stable artifact -> deployment -> run lifecycle while retaining
the underlying draft implementation for explicit server users.