lda-wf/docs/historical/superpowers/plans/2026-06-01-wf-api-extraction-roadmap.md

wf_api Extraction Roadmap

For agentic workers: This is an architecture roadmap, not a deterministic implementation checklist. Use it to choose and scope future focused implementation plans. Do not execute multiple slices at once.

Goal: Extract a protocol-neutral workflow application API from wf_mcp while preserving current process-local behavior and avoiding a large semantic rewrite.

Current update: the original WorkflowApiBackend seam was useful for proving dependency direction, but has been collapsed. WorkflowApi now composes domain services directly from WorkflowOperationContext; MCP owns only context construction and tool schemas.

Architecture: wf_api becomes the long-lived in-process application service layer. wf_mcp, wf_cli, and future HTTP/UI adapters call wf_api; wf_api must not import wf_mcp.

Current State: Slice 1 originally introduced wf_api.WorkflowApi, a WorkflowApiBackend protocol, and an MCP adapter backend. Later slices removed that double-delegation seam: WorkflowApi now composes domain services directly from WorkflowOperationContext. Both CLI and MCP workflow tools call WorkflowApi; wf_api imports no wf_mcp modules. WorkflowSurfaceHandlers is now a thin MCP compatibility subclass rather than the operation implementation.

Slice 3 moved the protocol-neutral workflow helpers into wf_api: constants, capability refs, wrapper hints, next actions, raw workflow plan model, runtime dependency resolution, saved subgraph preparation, and durable run lifecycle helpers. The old wf_mcp.workflow_surface.* module paths remain compatibility shims for those helpers.

Current Constraint: WfMcpService still acts as a compatibility facade over focused broker services. Recent slices have extracted source/catalog, runtime, upstream transport, events, connection sync, and content access; the remaining work is to keep shrinking facade responsibilities while preserving process-local behavior.

---

Target Shape

wf_core       = workflow execution kernel
wf_authoring = Python authoring sugar and NodeSpec construction
wf_artifacts = saved workflow/deployment/run models and stores
wf_platform  = source/capability/event platform primitives
wf_api       = process-local workflow application service/use cases
wf_mcp       = MCP adapter and MCP runtime/proxy/admin surface
wf_cli       = CLI adapter

Desired dependency direction:

wf_cli ─┐
        ├──> wf_api ───> wf_artifacts / wf_platform / wf_core / wf_authoring
wf_mcp ─┘

Forbidden dependency direction:

wf_api -> wf_mcp

Process-local behavior remains the default:

config = load_broker_config(path)
service = build_service_from_config(config)
api = WorkflowApi(WfMcpWorkflowApiBackend(service))

No FastAPI, daemon, socket, auth redesign, or network boundary is required for this extraction.

Slice 1: Dependency Direction Cleanup

Goal

Make both CLI and MCP call a protocol-neutral WorkflowApi, while WorkflowApi does not import wf_mcp.

Allowed

• Create src/wf_api/.
• Add WorkflowApiBackend protocol.
• Add WorkflowApi.
• Add WfMcpWorkflowApiBackend adapter around WfMcpService.
• Update wf_cli.context to construct WorkflowApi.
• Update MCP workflow tool registration to use WorkflowApi.
• Keep compatibility aliases if existing imports need a transition.
• Keep payloads and behavior unchanged.

Not Allowed

• Do not split WorkflowApi by domain yet.
• Do not move every helper module yet.
• Do not rename events yet.
• Do not redesign stores.
• Do not add FastAPI.
• Do not change response payloads.
• Do not change command/tool names.

Implemented Shape

src/wf_api/
  __init__.py
  backend.py       # WorkflowApiBackend high-level operation protocol
  service.py       # WorkflowApi thin delegating facade

src/wf_mcp/workflow_surface/
  handlers.py      # existing implementation; now backend plumbing
  tools.py         # MCP adapter; calls WorkflowApi

src/wf_mcp/broker/service/
  workflow_api_backend.py  # WfMcpWorkflowApiBackend

WorkflowApiBackend currently exposes high-level workflow operations that mirror the old workflow surface (list_capabilities, create_draft_workspace, run_deployment, and so on). That is intentionally not the final clean domain API. It keeps behavior and payloads stable while introducing the dependency seam. Later slices can replace selected dict[str, Any] method boundaries with stronger domain models after callers are routed through WorkflowApi.

Live source validation remains behind the MCP backend adapter because it touches MCP connections, adapters, and auth. wf_api owns the operation name, but the current backend owns the live-check implementation.

Success Criteria

• wf_api imports no wf_mcp modules. Done.
• wf_cli uses WorkflowApi. Done.
• wf_mcp.workflow_surface.tools uses WorkflowApi. Done.
• Existing MCP workflow-surface tests pass. Done at implementation time.
• Existing CLI tests pass. Done at implementation time.
• Behavior and payloads are unchanged. Intended and guarded by tests.

Slice 2: Stabilize API Names And Compatibility Shims

Goal

Make naming honest without breaking callers.

Docs-First Current Slice

Before renames, document the new ownership:

• wf_api.WorkflowApi is the application-facing process-local API.
• wf_api.WorkflowApiBackend is the high-level backend protocol.
• wf_mcp.broker.service.WfMcpWorkflowApiBackend adapts the current MCP service stack into the backend protocol.
• wf_mcp.workflow_surface.WorkflowSurfaceHandlers is legacy/internal implementation plumbing. New adapter code should not treat it as the canonical API.

Later Likely Work

• Rename WorkflowSurfaceHandlers usage to WorkflowApi in tests and CLI code.
• If a class rename is chosen, keep a temporary import shim:

from wf_api import WorkflowApi as WorkflowSurfaceHandlers

• Update docs to say:

MCP tools and CLI commands are adapters over wf_api.

• Rename test fixture helpers from handlers(...) to api(...) if useful.

If/Then

• If shims create confusion, remove them quickly after imports are migrated.
• If too many downstream imports still expect WorkflowSurfaceHandlers, keep the shim for one release/work session and document it as deprecated.

Success Criteria

• New code imports WorkflowApi.
• Old name remains only in compatibility modules or is gone.
• Docs describe wf_api as the application service layer.

Slice 3: Move Protocol-Neutral Workflow Surface Modules

Goal

Move helper modules that are not MCP-specific out of wf_mcp.workflow_surface.

Completed Moves

wf_mcp.workflow_surface.constants             -> wf_api.constants
wf_mcp.workflow_surface.refs                  -> wf_api.refs
wf_mcp.workflow_surface.wrapper_hints         -> wf_api.wrapper_hints
wf_mcp.workflow_surface.next_actions          -> wf_api.next_actions
wf_mcp.models.RawWorkflowPlan                 -> wf_api.models.RawWorkflowPlan
wf_mcp.workflow_surface.runtime_dependencies  -> wf_api.runtime_dependencies
wf_mcp.workflow_surface.saved_subgraphs       -> wf_api.saved_subgraphs
wf_mcp.workflow_surface.run_lifecycle         -> wf_api.run_lifecycle

The full wf_mcp.workflow_surface.models module did not move. It still holds MCP tool request/response schemas such as TraceRange and workflow tool result models. Move or split those only when the MCP schema boundary is clearer.

If/Then

• If a module imports MCP connection/adapters/auth, do not move it in this slice.
• If imports become circular, leave a shim in the old location and move one module at a time.
• If a helper is really platform vocabulary, consider wf_platform instead of wf_api.

Success Criteria

• wf_api owns protocol-neutral workflow API helpers. Done.
• wf_mcp.workflow_surface keeps MCP adapter/schema code plus compatibility shims. Mostly done.
• Tests still pass with import-only or near-import-only changes. Done at implementation time.

Slice 4: Split The Big API By Domain

Goal

Reduce the large WorkflowApi class after the package boundary is correct.

Slice 4A: Operation Context Scaffolding

Do not move handler methods first. WorkflowSurfaceHandlers methods currently reach through self.service for stores, capability sources, events, live source calls, adapter lookup, and catalog helpers. Moving method bodies before defining that dependency surface would either make wf_api import wf_mcp or produce a fake split where every domain service still depends on the whole MCP service.

First introduce a small protocol-neutral operation context in wf_api:

src/wf_api/
  operation_context.py  # protocols/dataclass for stores, sources, events, live calls

The exact names may change, but the context should answer these questions:

• How does workflow API code access artifact, draft workspace, deployment, and run stores?
• How does it read planner-visible CapabilitySource objects?
• How does it record artifact/deployment/run lifecycle events without importing MCP event types?
• How does it perform live source validation or live capability calls without importing MCP adapters/auth?
• Which existing WfMcpService helpers are still required by moved domain methods?

WfMcpWorkflowApiBackend or another MCP-owned adapter can build this context from WfMcpService. The context is scaffolding only: Slice 4A should not move capability/draft/artifact/deployment/run method bodies yet and should not change public payloads.

Candidate Shape

src/wf_api/
  service.py        # facade that composes domain services
  operation_context.py
  capabilities.py
  drafts.py
  artifacts.py
  deployments.py
  runs.py

Possible facade:

class WorkflowApi:
    capabilities: CapabilityApi
    drafts: DraftApi
    artifacts: ArtifactApi
    deployments: DeploymentApi
    runs: RunApi

Compatibility can keep flat methods:

async def list_capabilities(...):
    return await self.capabilities.list_capabilities(...)

Planned Domain Split Order

After Slice 4A proves the operation-context seam, split method groups in small behavior-preserving slices:

Slice 4B: Drafts First

Move stateless draft and draft workspace operations first:

validate_draft
compile_draft
patch_draft
list_draft_workspaces
create_draft_workspace
get_draft_workspace
delete_draft_workspace
validate_draft_workspace
patch_draft_workspace
set_draft_name
set_draft_route
set_step_input_map
set_step_output_map
create_minimal_draft_workspace

Reason: drafts mostly use the draft workspace store, workflow draft compiler, wrapper hints, and deterministic patch helpers. They have the lowest live-source and durable-runtime coupling.

Leave create_draft_workspace_from_capability in the MCP-backed handler during 4B. It depends on inspect_capability, wrapper hints, and capability source inspection, so it should move with either a small follow-up capability bootstrap slice or Slice 4E.

Slice 4C: Artifacts And Deployments

Move saved artifact and deployment operations next:

list_artifacts
save_artifact
create_artifact_from_plan
create_artifact_from_draft
create_artifact_from_workspace
create_wrapper_from_workspace
inspect_artifact
list_deployments
inspect_deployment
save_deployment
delete_deployment
validate_deployment

Reason: this group is store-heavy and introduces dependency validation, saved subgraph tree resolution, and event recording. It should move only after drafts prove the context seam.

Slice 4D: Runs

Move run lifecycle operations after artifacts/deployments:

run_deployment
resume_run
inspect_run
read_run_trace

Reason: runs are runtime-sensitive. They touch durable checkpoints, pinned dependency environments, resume readiness, prepared saved subgraphs, trace slicing, and compact next-action guidance. This should not be the first method move.

Slice 4E: Capabilities Last

Move workflow capability operations last:

list_capabilities
inspect_capability
call_capability
create_draft_workspace_from_capability

Reason: capabilities look simple but are the messiest boundary. They combine planner-visible source inventory, wrapper artifacts, direct wrapper calls, external live source calls, source visibility, and schema/wrapper hints. create_draft_workspace_from_capability also belongs here because it is driven by inspect_capability wrapper hints. Keep them in the MCP-backed implementation until the other domain services are stable.

After Slice 4E: Helper Promotion Cleanup

Once the handler is mostly a compatibility adapter, promote duplicated helper symbols into stable homes instead of leaving long-term cross-domain private imports:

raw_plan_from_artifact                 -> wf_api.artifact_plans or wf_artifacts
artifact_capability_id                 -> wf_api artifact/capability refs helper
available_sources_from_capability_sources -> wf_api source snapshot helper

This should be a cleanup slice, not part of 4E unless required to avoid circular imports or behavior drift.

If/Then

• If the context starts mirroring all of WfMcpService, stop and split it into smaller protocols rather than creating a new god object.
• If callers benefit from flat methods, keep the facade flat and split internals only.
• If domain APIs are clean enough, expose nested services later.
• If a method spans domains, keep it in the facade until a better boundary appears.
• If live source calls cannot be abstracted cleanly yet, leave capability calling in the MCP backend and move drafts/artifacts first.

Success Criteria

• wf_api has an explicit operation context/protocol seam that imports no wf_mcp modules.
• WfMcpWorkflowApiBackend can adapt WfMcpService into that seam.
• No workflow method behavior changes in Slice 4A.
• Each domain file is readable on its own.
• Public payloads remain unchanged.
• wf_cli and wf_mcp do not care about the internal split.

Slice 5: Move Listing/Event Primitives To Better Homes

Goal

Remove remaining protocol-neutral utilities from MCP-named packages.

Current Recommendation

Split this into two different concerns. Listing/helper consolidation is behavior-preserving cleanup and should happen first. Event migration changes domain vocabulary and should remain separate until lifecycle event semantics are clearer.

Slice 5A/5B: Listing And Workflow Helper Consolidation

Concrete plan:

docs/superpowers/plans/2026-06-02-wf-api-slice-5a-5b-helper-consolidation.md

Planned moves:

wf_api.capabilities._matches_query          -> wf_api.listing.matches_query
wf_api.artifacts._matches_query             -> wf_api.listing.matches_query
wf_api.capabilities._paged_list_payload     -> wf_api.listing.paged_list_payload
wf_api.artifacts._paged_list_payload        -> wf_api.listing.paged_list_payload
wf_mcp.workflow_surface.handlers fallback   -> wf_api.listing.paged_list_payload

wf_api.runs._raw_plan_from_artifact         -> wf_api.artifact_plans.raw_plan_from_artifact
wf_api.capabilities._raw_plan_from_artifact -> wf_api.artifact_plans.raw_plan_from_artifact
wf_api.capabilities._artifact_capability_id -> wf_api.artifact_refs.artifact_capability_id
wf_api.artifacts._artifact_capability_id    -> wf_api.artifact_refs.artifact_capability_id
wf_api.{drafts,artifacts,capabilities} requirement helpers
                                             -> wf_api.capability_requirements

Do not move wf_mcp.shared.pagination in this slice. It is still used by proxy tool search/listing code, so treating it as dead workflow-surface debt would be incorrect.

Post-5 Helper Cleanup: Workflow Surface Test Thinning

Concrete plan:

docs/superpowers/plans/2026-06-02-wf-mcp-workflow-surface-test-thinning.md

Intent:

wf_api tests                 = canonical application behavior tests
wf_mcp.workflow_surface tests = adapter/schema/live-source/integration smoke tests

Do not replace stronger workflow-surface integration tests with weaker unit tests. Only remove a handler test when an equal-or-stronger wf_api test exists and at least one handler-level smoke/delegation test still protects the adapter path.

Candidate Moves

wf_mcp.shared.listing.matches_query       -> wf_platform.listing or wf_api.listing
wf_mcp.shared.listing.paged_list_payload  -> wf_platform.listing or wf_api.listing
wf_mcp.events.McpEvent                    -> wf_platform.events.DomainEvent
wf_mcp.events.EventBus                    -> wf_platform.events.EventBus

If/Then

• If events are only used by MCP admin/proxy code, leave them in wf_mcp.
• If events describe artifact/deployment/run lifecycle, move or fork them into wf_platform.
• If renaming McpEvent causes churn, introduce DomainEvent first and keep McpEvent as an alias temporarily.

Success Criteria

• Workflow API lifecycle events no longer require MCP naming.
• Listing helpers used by CLI/API are not imported from wf_mcp.

Slice 6: Store Construction And Config Boundary

Goal

Make store/runtime construction less MCP-owned.

Current Problem

build_service_from_config and WfMcpService.__post_init__ currently build or own protocol-neutral stores. CLI uses this path because it is pragmatic, but long term config/store construction should be reusable without MCP server assumptions.

Candidate Work

• Extract config-to-store construction into a neutral builder.
• Keep MCP connection config in wf_mcp.
• Let wf_cli and future API/server adapters reuse the neutral store builder.

If/Then

• If extraction creates too many config model moves, defer it.
• If CLI needs only current config behavior, keep using wf_mcp builder until FastAPI/UI pressure appears.

Success Criteria

• Artifact/draft/run stores can be constructed without starting MCP concepts.
• Current wf_mcp.config.json remains supported.

Slice 7: Future HTTP/FastAPI Adapter

Goal

Expose WorkflowApi over HTTP only after the in-process API is stable.

Not Yet

Do not start this until:

• wf_api exists.
• CLI and MCP both call wf_api.
• Run/draft/deployment payloads are stable enough.
• Auth and multi-client lifecycle questions are explicit.

Future Shape

wf_http or wf_server
  routes/
    capabilities.py
    drafts.py
    artifacts.py
    deployments.py
    runs.py

Routes should be thin:

@router.post("/runs")
async def start_run(...):
    return await api.run_deployment(...)

Success Criteria

• HTTP is an adapter, not a new source of workflow logic.
• Process-local API remains usable without HTTP.

Open Questions

1. Should WorkflowApiBackend be one protocol or several domain protocols?

   • Recommendation for Slice 1: one protocol. Split later only if it hurts.

2. Should live source validation live in wf_api?

   • Recommendation: wf_api owns the operation, backend owns the live-check implementation.

3. Should wf_api expose flat methods or nested services?

   • Recommendation for Slice 1: flat methods for compatibility. Consider nested internals later.

4. Should WorkflowSurfaceHandlers disappear immediately?

   • Recommendation: no. Keep a short-lived shim if it reduces churn.

5. Should wf_cli stop importing wf_mcp after Slice 1?

   • Not fully. It may still use wf_mcp config/service construction until store/config extraction happens.

Immediate Next Plan Status

Slice 1 implementation plan exists and was executed:

docs/superpowers/plans/2026-06-01-wf-api-slice-1-dependency-direction.md

The next implementation plan should cover Slice 2 only. Prefer docs and compatibility naming first; do not move helper modules until Slice 3.