lda-wf/docs/wf_api_architecture.md

# wf_api Architecture Boundary

`wf_api` is the process-local workflow application service layer. It is not just
a DTO wrapper around `wf_core`, and it is not an MCP transport package.

The package owns workflow-facing application operations that combine execution,
saved artifacts, deployment validation, authoring guidance, progressive payloads,
and run lifecycle policy into a stable API that CLI, MCP, and future HTTP
frontends can share.

## Responsibility Map

| Package | Responsibility |
| --- | --- |
| `wf_core` | Workflow execution semantics: graph models, runtime state, scheduler, path/state mapping, interrupts, subgraphs, foreach, and run-state codec. |
| `wf_artifacts` | Saved definitions and persistence contracts: workflow artifacts, deployments, draft workspaces, run records, checkpoints, artifact/deployment validation models. |
| `wf_platform` | Shared capability/source concepts and platform-facing contracts such as capability refs, source inventory models, documentation source models, and JSON schema helpers. |
| `wf_api` | Application workflow contract and process-local implementation over core/artifacts/platform: capability discovery, wrapper hints, draft editing, artifact/deployment operations, run/resume operations, next actions, and progressive response shaping. |
| `wf_server` | Durable server composition boundary that hosts a `WorkflowApi` plus optional admin/source-registry surfaces. |
| `wf_sources_mcp` | MCP-as-upstream-source implementation: source ids, source registry DTOs, auth/catalog stores, discovery, SDK client/facade, persistent runtime pool, and tool-wrapper helpers. |
| `wf_mcp` | MCP frontend/compatibility package: old `wf-mcp` server entry points, broker glue around MCP-hosted services, proxy/admin tools, and compatibility shims while callers migrate. |
| `wf_transport_rpc_http` | JSON-RPC-over-HTTP transport adapter and remote client over `WorkflowApiSurface`, not a reimplementation of workflow business logic. |
| `wf_client` | Async Python consumer facade over a narrow capability/artifact/deployment/run port. It reconstructs immutable snapshots and keeps representations bounded and inert. |
| future `wf_http` / WebSocket / MCP server transports | Additional transports over `WorkflowApiSurface`, not new workflow application APIs. |
| `wf_cli` | CLI frontend over `WorkflowApiSurface`; it may run locally against process-local stores or target a remote JSON-RPC backend. |

## What Belongs In wf_api

`wf_api` should contain code that is:

- protocol-neutral between MCP, CLI, and future HTTP
- workflow-application policy rather than low-level graph execution
- response shaping for human/LLM clients, such as compact list payloads and
  bounded trace slices
- authoring guidance, such as wrapper hints and next actions
- orchestration across `wf_core`, `wf_artifacts`, and `wf_platform`
- durable run lifecycle policy over `RunStore` and `WorkflowRuntimeRunner`

Examples that belong in `wf_api`:

- `WorkflowApi`
- `WorkflowApiSurface`
- domain surface protocols such as `WorkflowDraftSurface` and
  `WorkflowRunSurface`
- `WorkflowCapabilityApi`
- `WorkflowDraftApi`
- `WorkflowDraftAuthoringApi`
- `WorkflowArtifactApi`
- `WorkflowDeploymentApi`
- `WorkflowRunApi`
- wrapper hints
- next actions
- runtime dependency resolution
- saved subgraph preparation helpers
- run lifecycle helpers such as `persist_stopped_run()` and
  `validate_pinned_resume_environment()`

## What Does Not Belong In wf_api

`wf_api` must not contain:

- MCP SDK calls
- FastMCP tool/resource/prompt registration
- MCP content block models or MCP schema workarounds
- broker config file mutation
- upstream MCP session/runtime management
- proxy mounting or reload logic
- local process service construction that assumes `WfMcpService`
- scheduler/execution semantics that belong in `wf_core`
- persistence model definitions that belong in `wf_artifacts`

The hard import rule remains:

```text
wf_mcp -> wf_api is allowed
wf_api -> wf_mcp is forbidden
```

## WorkflowOperationContext

`WorkflowOperationContext` is the adapter seam that lets `wf_api` stay
transport-neutral.

Current shape:

```python
WorkflowOperationContext(
    artifact_store=...,
    draft_workspace_store=...,
    run_store=...,
    events=...,
    specs=...,
    runtime=...,
    live_sources=...,
)
```

Important rules:

- Source inventory goes through `context.specs.capability_sources`.
- Qualified node lookup goes through `context.specs.get_qualified_spec()`.
- Runtime execution goes through `context.runtime`.
- Workflow events go through `context.events.record_workflow_event()`.
- Optional live source checks go through `context.live_sources`.
- Stores are still optional for MCP/test compatibility, but durable API
  frontends should construct stricter contexts with required stores.

Do not add a catch-all `service` field to the context. If a domain API needs a
new dependency, add a narrow protocol or explicit field.

## Python client lifecycle

Hypothetically, an application that wants to turn a discovered capability into
a durable run would use the following complete flow:

```python
from pydantic import BaseModel

from wf_client import App
from wf_authoring import input_from, input_value, output_to, state_path


class Input(BaseModel):
    request_id: str


class State(BaseModel):
    value: str | None = None


class Output(BaseModel):
    value: str

app = App.from_http_jsonrpc("http://localhost:8765/rpc")
capability = await app.capability("wf.std.constant")
graph = app.new_workflow(
    "example",
    input_schema=Input,
    state_schema=State,
    output_schema=Output,
)
step = graph.use(
    capability,
    id="constant",
    input=[input_value("value", "hello")],
    output=[output_to("value", state_path("value"))],
)
end = graph.end("ok", id="end_ok")
graph.set_entry_point(step)
graph.connect(step, "ok", end)
graph.set_output([input_from(state_path("value"), "value")])
validation = await graph.validate()
validation.raise_for_errors()
artifact = await graph.save(version=1)
run = await artifact.run({"request_id": "request-1"})
```

Pydantic models, typed mappings, and raw JSON Schema are accepted, but Python
applications should normally keep their contracts as Python types. If a
contract changes while the graph is being authored, call
`graph.set_contract(state_schema=..., output_schema=..., outcomes=...)`.
Supplied fields replace their whole contract atomically; omitted fields and
existing graph bindings remain, so validate after the replacement.

The graph is an in-process builder. Validation is local structural checking
plus a server plan check. Saving creates an immutable, versioned artifact; it
does not execute anything. A deployment is the server's runnable configuration
for one exact artifact version, including logical-to-concrete source bindings
and drift policy. A run is a durable execution record for that deployment;
inspection and bounded trace reads return snapshots, while resume is an
explicit operation for interrupted runs.

`wf_client` does not expose draft workspaces. Draft API classes remain useful
to server/admin and console callers, but normal server composition keeps draft
JSON-RPC registration opt-in so artifact, deployment, and run durability do not
depend on a draft store.

Existing remote objects can be discovered without eagerly loading their full
plans, bindings, or traces. `app.artifacts(...)` and `app.runs(...)` return
paged immutable summary rows; `app.deployments()` returns an immutable tuple.
Call `app.workflow(id, version=...)`, `app.deployment(id)`, or `app.run(id)` to
reconstruct the selected rich object.

## WorkflowApiSurface And Domain Services

`WorkflowApiSurface` is the public application contract shared by local and
remote frontends. It is a structural protocol, not a base class. Implementations
can satisfy it by method shape:

- `WorkflowApi` implements the surface in-process by composing domain services
  over `WorkflowOperationContext`.
- `RpcWorkflowApiClient` implements the same flat surface by serializing calls
  to fixed JSON-RPC method names.
- Future auth, cache, recording, WebSocket, or MCP-server adapters should also
  target the same surface instead of importing concrete implementation classes.

Current remote CLI flow:

```text
wf_cli
  -> wf_transport_rpc_http.RpcWorkflowApiClient
  -> wf_server.WorkflowServer
  -> wf_api.WorkflowApi / admin surfaces
  -> wf_sources_mcp or other source implementations
```

`RpcWorkflowApiClient` is intentionally composed from domain mixins over one
transport primitive, `RpcCaller._call(method, params)`. Domain mixins must not
own HTTP state or duplicate `_call` stubs; they are method bundles over the
shared JSON-RPC request primitive.

The surface is split into domain protocols:

```text
WorkflowApiSurface
  WorkflowCapabilitySurface
  WorkflowDraftSurface
  WorkflowArtifactSurface
  WorkflowDeploymentSurface
  WorkflowRunSurface
```

The domain services below are the process-local implementation pieces, not the
contract itself.

`WorkflowApi` composes focused domain APIs:

```text
WorkflowApi
  capabilities: WorkflowCapabilityApi
  drafts: WorkflowDraftApi | None  # only when drafts=True
  draft_authoring: WorkflowDraftAuthoringApi | None  # only when drafts=True
  artifacts: WorkflowArtifactApi
  deployments: WorkflowDeploymentApi
  runs: WorkflowRunApi
```

These domain services are allowed to return `dict[str, Any]` payloads because
they define application-facing response contracts consumed by multiple
frontends. Internally, they should prefer typed models from `wf_core`,
`wf_artifacts`, and `wf_platform`, then serialize at the boundary.

`WorkflowDraftApi` owns draft workspace lifecycle, validation, compilation,
JSON Patch application, and focused low-level map edits. `WorkflowDraftAuthoringApi`
is the semantic authoring layer above it: capability-aware bootstrap, bind,
typed step insertion, branch, handle, and remove helpers lower intent into
ordinary draft workspace patches while preserving revision checks.

`workflow.draft_workspaces.add_step` accepts a discriminated `DraftStep` value,
while `step_id` remains the separate map key chosen by the caller. The operation
atomically inserts that step plus an optional incoming `RouteSource` and any
top-level outcome routes, so a failed structural check does not partially wire
the graph. Decision targets for `when`, `choose`, and `match` remain embedded in
their typed payloads. The composed `add_step_from_capability` operation remains
separate because it also resolves capability metadata, projects schemas, and
requires complete declared-outcome coverage.

## Relationship To wf_core

`wf_core` is lower-level than `wf_api`.

`wf_api` may:

- compile or validate workflow-facing requests into core/artifact models
- call runtime runners that execute core workflows
- shape run traces and diagnostics for clients

`wf_api` must not:

- decide scheduler semantics
- mutate `RunState` internals directly
- add graph node semantics
- encode MCP/client-specific behavior into core execution

If a behavior changes how workflows execute, it probably belongs in `wf_core`
or in an explicit runtime dependency injected into `wf_core`, not in `wf_api`.

## Relationship To wf_artifacts

`wf_artifacts` owns durable definitions and storage contracts. `wf_api` owns
operations over them.

Examples:

- `WorkflowArtifact`, `WorkflowDeployment`, `WorkflowRunRecord`, and
  `RunCheckpoint` belong in `wf_artifacts`.
- `create_artifact_from_plan`, `save_deployment`, `run_deployment`, and
  `resume_run` application flows belong in `wf_api`.

`wf_api` should not define parallel persistence models for the same durable
concepts. If a response needs a different shape, create response payload helpers
or next actions rather than duplicating the storage model.

## Relationship To wf_mcp

`wf_mcp` adapts MCP into `wf_api`.

Current MCP path:

```text
wf_mcp.workflow_surface.tools
  -> WorkflowApi(context_from_service(service))
  -> wf_api domain service
  -> WorkflowOperationContext protocol
  -> focused broker service / store / runtime implementation
```

MCP-specific concerns stay outside `wf_api`:

- tool schema names and safe tool names
- MCP resources/prompts-as-tools registration
- upstream MCP source liveness checks
- FastMCP notifications
- MCP content block normalization
- broker connection config/reload

## Future HTTP/API Boundary

Future HTTP/WebSocket/MCP server transports should consume
`WorkflowApiSurface`, usually backed by a process-local `WorkflowApi` instance.
They must not copy MCP handlers or workflow business logic.

Expected shape:

```text
transport route/controller
  -> WorkflowApiSurface implementation
  -> WorkflowApi(required_store_context) when running process-locally
  -> wf_api domain services
```

The HTTP layer should own:

- request/response framework models
- auth/session policy
- API routing
- streaming/progress transport if needed
- construction of a durable `WorkflowOperationContext`

The HTTP layer should not own:

- workflow validation semantics
- run resume semantics
- wrapper hint policy
- deployment dependency validation

## Current Roadmap Implication

The next implementation work should follow this order:

1. Continue hardening persisted run/resume behavior behind `WorkflowRunApi`.
2. Keep CLI and transport entrypoints typed against `WorkflowApiSurface`.
3. Add missing server/source/auth operations behind the same surface or explicit
   sibling admin surfaces.
4. Add future WebSocket/MCP-server transports as adapters, not as new workflow
   application layers.

Workflow primitives such as fork/gather and additional authoring sugar should
resume after the durability/platform boundary is stable.