lda.chat/lda-wf
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: refresh roadmap and status plan

Browse Source
This commit is contained in:
lda
2026-06-08 23:16:17 +07:00 Verified
parent 5b81380f2a
commit 04122e12ac
2 changed files with 604 additions and 509 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current_roadmap.md
+102 -509
View File
@@ -1,531 +1,124 @@
# Current Roadmap
This is the short active roadmap after the core type-shape cleanup and MCP
workflow authoring cleanup pass. It is based on both the current docs and the
implementation state.
This is the live, short roadmap. Completed implementation plans and long slice
history live under [`historical/`](historical/). Current architecture references:
## Completed Cleanup Pass
- [`wf_api_architecture.md`](wf_api_architecture.md): workflow API, server,
transport, and source package boundaries.
- [`project_map.md`](project_map.md): package map and entrypoints.
- [`wf_cli.md`](wf_cli.md): current CLI usage.
1. **Docs index and prune**
- Current architecture docs are separated from historical plans and scratch
notes.
- The active roadmap now lives here instead of being scattered through older
planning files.
## Current Product Shape
2. **MCP workflow authoring UX**
- The operator manual categorizes workflow tools into discovery, draft
workspace, stateless draft, artifact/deployment, run/debug, and raw escape
hatch groups.
- List-style tools are more compact, while inspect/run tools carry the
detailed payloads.
```text
wf_cli
-> local WorkflowApi or wf_transport_rpc_http.RpcWorkflowApiClient
-> wf_server.WorkflowServer
-> wf_api.WorkflowApi / admin surfaces
-> wf_core / wf_artifacts / wf_sources_mcp
```
3. **Wrapper creation ergonomics**
- Wrapper draft helpers can suggest state schema, input bindings, output
bindings, default `ok` / `error` handling, and missing decisions.
- The end-to-end runbook documents the wrapper path from capability
discovery through deployment/run.
The durable product path is now `wf-rpc-server` plus neutral `wf_config` /
`wf_server` composition. The old `wf-mcp` script remains a legacy/special-purpose
MCP entrypoint and compatibility surface.
4. **Run and deployment story**
- Deployment listing is summary-first, with dedicated inspection for detail.
- `run_deployment` returns compact status by default and exposes trace slices
through an explicit `trace_range`.
- Dependency validation and error output remain part of the run path.
## Priority 1: Product Smoke And Status UX
5. **Source inventory polish**
- `list_sources` / `inspect_source` now present source-owned capabilities
progressively.
- Source inventory distinguishes external sources, local workflow-facing
sources, docs/resources, and admin-only control surfaces.
The platform is usable enough to test as a product. Next work should focus on
clear operator feedback before adding more architecture.
6. **Workflow API seam**
- `wf_api.WorkflowApiSurface` is now the protocol-neutral workflow operation
contract consumed by CLI and transport adapters.
- `wf_api.WorkflowApi` is the process-local implementation used by MCP
workflow tools and local CLI/server composition.
- `wf_api` imports no `wf_mcp` modules. `WorkflowApi` composes domain
services directly from `WorkflowOperationContext`; MCP owns only context
construction and tool schemas.
- Protocol-neutral helpers now live in `wf_api`: refs/constants, wrapper
hints, next actions, raw workflow plans, runtime dependencies, saved
subgraph preparation, and durable run lifecycle helpers. Old
`wf_mcp.workflow_surface` helper paths remain compatibility shims.
- The boundary is documented in
[wf_api architecture](./wf_api_architecture.md).
- Add `wf status` as a compact target/server status command.
- Run a real CLI smoke script against `wf-rpc-server --config wf.config.json`.
- Capture UX gaps as small follow-up items: confusing errors, missing examples,
poor command help, and target/config ambiguity.
- Keep status read-only; do not mutate registry, auth, config, or stores.
## Active Next Roadmap
## Priority 2: Durable Run/Resume Hardening
1. **WorkflowOperationContext simplification**
- Completed: the duplicated top-level
`WorkflowOperationContext.capability_sources` field was removed.
- Keep `WorkflowSpecProvider.capability_sources` as the single source inventory
path for `wf_api` consumers.
- The audit is in
[2026-06-03 WorkflowOperationContext shape audit](./superpowers/research/2026-06-03-workflow-operation-context-audit.md).
The v1 durable run and resume path exists, including persisted interrupted runs,
bounded trace reads, dependency revalidation, and process-rebuild resume tests.
Remaining hardening should focus on correctness under real server use.
2. **Persisted run/resume spec**
- Completed: the process-restart resume contract is defined in
[2026-06-03 persisted run/resume contract](./superpowers/specs/2026-06-03-persisted-run-resume-contract.md).
- It covers run records, pinned deployment/artifact/subgraph environment,
source/capability validation, trace paging, and interrupt-only pause
semantics.
- Keep ordinary dead tools/sources as diagnostics or failed runs, not implicit
pauses.
- Add same-run concurrency protection around `resume_run`.
- Clarify store-level locking/transaction expectations for filesystem stores.
- Preserve existing semantics: broken pinned dependencies return blocked
readiness and diagnostics; ordinary live tool/source failures are failed runs,
not implicit pauses.
- Active specs:
- [`persisted run/resume contract`](superpowers/specs/2026-06-03-persisted-run-resume-contract.md)
- [`durable workflow runs and resume`](superpowers/specs/2026-05-26-durable-workflow-runs-and-resume-design.md)
3. **Persisted run/resume implementation**
- Implement the load/validate/resume flow behind `WorkflowRunApi`, `RunStore`,
and `WorkflowRuntimeRunner`.
- Do not reintroduce direct `WfMcpService` coupling into the workflow API.
- `wf_api.durable_context` now provides a required-store guard for future
durable frontends. It preserves the current process-local behavior while
failing fast if artifact, draft, or run stores are missing.
## Priority 3: Source/Auth/Config Polish
4. **Durable API service shape**
- Decide the non-MCP frontend boundary for a long-lived API process.
- Reuse `WorkflowApi` and the focused broker services where possible.
- Keep config/store construction and auth explicit.
- Current design direction is recorded in
[2026-06-03 long-lived workflow API boundary](./superpowers/specs/2026-06-03-long-lived-workflow-api-boundary.md):
initial slices proved a lightweight local/static server and JSON-RPC
transport; later slices add transport siblings, source providers, auth,
streaming/progress, transactional storage, and live upstream MCP sources.
- First slice implemented: `wf_server` can construct a local/static durable
`WorkflowApi` without `WfMcpService`.
- Completed: the first JSON-RPC-over-HTTP transport can expose the
local/static `WorkflowServer` through fixed dotted methods.
- Completed: workflow config now distinguishes client targets from server
hosting config, the basic `wf` lifecycle can target JSON-RPC HTTP:
capability discovery, draft workspace authoring, artifact/deployment
operations, run, inspect, and bounded trace.
- Completed: `wf_transport_rpc_http` is split by workflow domain. The
public `RpcWorkflowApiClient` still satisfies `WorkflowApiSurface`, while
client methods and server JSON-RPC registrations live in focused
capability, draft, artifact, deployment, and run modules.
- Completed: JSON-RPC and CLI now expose `call_capability` through
`workflow.capabilities.call` and `wf cap call`, so local and remote users
can smoke-test a capability before creating draft workspaces.
- Completed: RPC client domain mixins now share a single typed
`RpcCaller._call(method, params)` transport primitive instead of
repeating `_call` stubs in every mixin.
- Completed: read-only source inventory now has a protocol-neutral
`WorkflowSourceAdminApi` / `WorkflowSourceAdminSurface`; MCP admin source
tools delegate through it while connection/raw MCP operations remain
broker-owned.
- Completed: read-only source inventory is exposed through JSON-RPC HTTP and
`wf source list` / `wf source inspect`.
- Completed: read-only admin/config sibling surface covers connection
inventory, connection status, and broker/server events. Keep it separate
from `WorkflowApiSurface`; this is platform management, not workflow
lifecycle.
- Completed: read-only admin/config now has a neutral `WorkflowAdminApi` /
`WorkflowAdminSurface`, is exposed through JSON-RPC HTTP, and is available
through `wf admin connections`, `wf admin statuses`, and
`wf admin events`.
- Mutating source/connection config commands now target the store-backed
source registry plan instead of config files. Config can bootstrap sources,
while server-owned dynamic source changes need validated registry writes.
- The store-backed source registry design is recorded in
[2026-06-03 store-backed source registry](./superpowers/specs/2026-06-03-store-backed-source-registry-design.md).
- First source registry implementation slices complete: validated registry
models, `FileSourceRegistryStore`, generic `wf_api` registry mechanics,
MCP entry conversion, and startup merge are implemented.
- Source registry startup merge is implemented: absent registry preserves
config-only behavior, registry-only entries hydrate as dynamic connections,
and config entries shadow same-id registry entries with an event.
- Config shadowing is the v1 conservative behavior, not the final ownership
model. The planned follow-up is explicit config ownership policy:
`locked` config entries stay operator-owned, while `seed` entries
bootstrap missing store entries and then let the store own later admin
changes.
- Completed: config ownership policy is implemented for MCP broker config
connections: `locked` entries stay operator-owned, while `seed` entries
bootstrap missing store entries and then let the store own later admin
changes.
- Historical source registry slice planning is archived in
[2026-06-03 source registry next slices](./historical/superpowers/plans/2026-06-03-source-registry-next-slices.md):
desired-registry admin reads and safe mutation commands are complete.
- Completed: MCP-backed `WorkflowServer` construction is available through
`wf_mcp.broker.server.build_workflow_server_from_config`. JSON-RPC can now
expose real MCP-backed workflow, source-admin, admin, and desired source
registry surfaces without making `wf_server` import `wf_mcp`.
- Completed: desired-registry admin read plumbing is available through
`WorkflowSourceRegistryApi`, JSON-RPC methods
(`workflow.admin.source_registry.list` / `.inspect`), and CLI commands
(`wf admin registry list` / `wf admin registry inspect`). Local/static
servers report unavailable instead of empty; concrete MCP-backed
`WorkflowServer` construction is now available through
`wf_mcp.broker.server`.
- Completed: desired-registry mutation operations are available through
`WorkflowSourceRegistryApi` (add/update/enable/disable/remove),
JSON-RPC methods (`workflow.admin.source_registry.add` / `.update` /
`.enable` / `.disable` / `.remove`), and CLI commands
(`wf admin registry add` / `update` / `enable` / `disable` / `remove`)
for targets that expose the registry-admin surface. Mutations target
persisted desired registry state only; config files, auth records, and
catalog snapshots are not mutated. Config-shadowed add is rejected in v1.
Remove requires `--confirm` in CLI; local/static servers report
unavailable.
- Completed: `wf-rpc-server --mcp-config wf_mcp.config.json` starts the
JSON-RPC transport over an MCP-backed `WorkflowServer`, making the remote
CLI path usable with MCP sources and desired source registry operations.
- Next concrete platform slices:
- Wider `wf_config` source model: migrate MCP broker config concepts into the
neutral server config instead of preserving `wf_mcp.config.json` as a
peer forever. `wf_config.server.sources[]` already exists as a
discriminated union, but only `stdlib` is implemented today. Add MCP source
variants that carry source id, provider/account/profile, ownership policy,
transport shape, auth reference, and metadata. The old MCP broker config
becomes a compatibility input that normalizes into the wider config model.
After that, `wf-rpc-server --config ...` can build MCP-backed sources from
neutral config and `--mcp-config` can be deprecated or treated as a legacy
alias.
`McpSourceRegistryEntry` already has most of the target shape; the one
ownership field comes from legacy `ConnectionConfig.source_config_ownership`.
When migrating, carry that policy into the neutral MCP source variant with a
clearer name such as `config_ownership` or `ownership`, rather than leaking
the old connection-centric field name.
First slice complete: `wf_config.server.sources[]` now accepts
`kind: "mcp"` entries with stdio/http transport, auth reference, metadata,
enabled flag, and `locked` / `seed` ownership policy. Runtime composition
from these entries is the next slice.
Runtime bridge complete: neutral `kind: "mcp"` source entries can now build
the MCP-backed `WorkflowServer`. `--mcp-config` remains supported as a
legacy compatibility path while new configs should prefer
`server.sources[]`.
- Transport package boundary cleanup follows the config migration. The current
`wf-rpc-server --mcp-config` hookup proves the product path but makes
`wf_transport_rpc_http.cli` import `wf_mcp.broker`, tripping the existing
import-direction guard. The durable fix is not a permanent split launcher;
it is making `wf_config` wide enough that the RPC server can compose from
neutral config while MCP-specific adapters stay selected by source kind.
Completed: `wf_transport_rpc_http` no longer imports `wf_mcp`; server
composition from neutral or legacy MCP config lives behind `wf_server.config`.
- Legacy config migration: add a converter from old `wf_mcp.config.json`
(`store_root`, `connections[]`) into the wider `wf_config` shape
(`server.store`, `server.sources[]`). `server.store` is already a
discriminated union (`kind: "filesystem"` today, future SQL/store backends
later), so the conversion should map old `store_root` to
`server.store.root` without inventing a parallel store field. Old MCP HTTP
metadata values such as `http`, `streamable-http`, `streamable_http`, and
`sse` should normalize into the neutral HTTP source transport while
preserving enough metadata for MCP/FastMCP compatibility. `sse` is legacy
protocol shape, but keep conversion support because FastMCP deployments may
still use it.
Completed: `wf config migrate-mcp` converts legacy broker config files into
neutral workflow config files without mutating the original.
- Manual product smoke: run `wf-rpc-server --mcp-config ...`, point
`wf --url ...` at it, and capture real CLI/server UX gaps before adding
more architecture.
- Source registry apply/reload: decide and implement how persisted registry
mutations affect the running source catalog. Prefer an explicit
apply/reload operation before automatic live remount.
- Completed: desired source registry mutations can now be applied explicitly
through `wf admin registry apply` / `workflow.admin.source_registry.apply`.
V1 apply reconciles registry state into the current server connection/source
graph; it does not auto-apply mutations, mutate config files, or remount
MCP proxy providers.
- Completed: persisted resume across server rebuild is covered through the
MCP-backed JSON-RPC path. A neutral-config `WorkflowServer` can start an
interrupting run, be rebuilt from the same filesystem stores, inspect the
interrupted run, and resume it to completion through `RpcWorkflowApiClient`.
- Completed: MCP upstream source runtime cleanup now starts with a typed
`McpSourceConnection` seam in `wf_sources_mcp`, not by moving
`runtime/factory.py` as-is. The active plan was
[2026-06-07 MCP source connection seam](./historical/superpowers/plans/2026-06-07-mcp-source-connection-seam.md).
- Completed: shared MCP session opener exists in `wf_sources_mcp.client`.
One-shot adapter (`McpSdkAdapter`) and persistent runtime
(`PersistentSessionFactory`) both use it.
- Completed: persistent MCP runtime moved to `wf_sources_mcp.runtime`.
`PersistentMcpSession`, `PersistentSessionFactory`, `McpRuntimePool`,
and `connection_runtime_fingerprint` are now canonical in
`wf_sources_mcp.runtime`; `wf_mcp.runtime.*` are compatibility shims.
Runtime remains tool-call-only. The completed plan was
[2026-06-07 MCP runtime package move](./historical/superpowers/plans/2026-06-07-mcp-runtime-package-move.md).
- Completed: one-shot MCP SDK adapter moved to `wf_sources_mcp.sdk.adapter`.
`McpSdkAdapter` is now canonical in `wf_sources_mcp`; `wf_mcp.sdk.*`
remains a compatibility shim for old imports. Persistent runtime is still
tool-call-only.
- Completed: shared `McpSourceClient` facade introduced in
`wf_sources_mcp.client`. The one-shot SDK adapter delegates MCP operation
calls and conversion through the facade; persistent runtime remains
tool-call-only until a separate owner-task routing slice.
- Completed: persistent MCP runtime owner now uses a generic explicit
operation queue with request metadata and `McpSourceClient` execution.
Public runtime remains tool-call-only; `operation` strings are diagnostics
labels, not dispatch.
- Completed: persistent MCP runtime can now route `read_resource` through
the owner-task queue and `McpSourceClient`. This is intentionally a thin
wrapper over the existing source-client facade; prompt/raw method runtime
operations remain separate future slices.
- Completed: persistent MCP runtime can now route `get_prompt` through
the owner-task queue and `McpSourceClient`. This keeps prompt reads
stateful without adding raw method invocation or notification support.
- Completed: broker content access now prefers a configured stateful MCP
runtime for `read_resource` and `get_prompt`, with one-shot adapter fallback.
Catalog refresh/discovery remains one-shot by policy.
- Completed: stateful MCP runtime now has protocol slices for tools,
resources, and prompts, and can route session-scoped `list_resources` and
`list_prompts` through the owner task. Catalog refresh still uses one-shot
adapter policy.
- Completed: persistent MCP runtime now implements the full upstream MCP
operation surface used by the one-shot adapter. Broker upstream operations
prefer the shared runtime pool when configured, with one-shot adapters
retained as fallback.
- Completed: MCP source catalog aggregation helpers (`CombinedCatalog` and
`snapshot_from_specs`) now live in `wf_sources_mcp.catalog`; the old
`wf_mcp.broker.catalog` path is a compatibility shim.
- Completed: JSON-RPC MCP-backed workflow runs now have deterministic
session-reuse coverage: repeated runs against one source use one
`McpRuntimePool` session and preserve session-local state.
- Auth/source secrets boundary: keep registry desired state separate from
upstream credentials, and surface missing auth as validation diagnostics.
The contract is now specified in
[2026-06-06 auth/source secrets boundary](./superpowers/specs/2026-06-06-auth-source-secrets-boundary.md):
sources carry `auth_ref`, runtime resolves through an auth store interface,
and the current filesystem auth files are only one adapter.
First implementation slice complete: neutral auth records/store protocol
exist in `wf_api`, MCP runtime auth resolution prefers explicit `auth_ref`
with legacy connection-id fallback, and MCP payload interpretation is
isolated in provider-specific adapter helpers.
Second implementation slice complete: missing explicit auth refs now surface
as `auth_not_found` diagnostics in live source checks and source registry
apply summaries.
Third implementation slice complete: read-only auth admin summaries are
available through MCP-backed server admin, JSON-RPC, and CLI. Summaries show
ids, schemes, metadata, and payload keys only; secret payload values remain
hidden.
Fourth implementation slice complete: local/dev auth records can be saved and
deleted through neutral admin, JSON-RPC, and `wf admin auth`. This is still not
a production secret manager or OAuth flow; payload values are accepted only as
write inputs and never returned.
Not done: auth is still compatibility-grade. There is no OAuth flow,
production secret manager, provider-specific display model, or
full removal of the legacy MCP auth record shape yet.
- Role-specific server stores: the current neutral config has one
`server.store` root that backs workflow records, desired source registry,
catalog/cache snapshots, and local/dev auth records. The next config slice
should add optional `server.stores.*` overrides while preserving
`server.store` as the fallback for every missing role. First implementation
should stay filesystem-only; secret managers, SQL, and object stores are
later backend implementations.
First role-specific store slice complete: neutral config now accepts optional
`server.stores.workflow`, `server.stores.auth`,
`server.stores.source_registry`, and `server.stores.catalog_cache`
filesystem overrides. Missing roles still fall back to `server.store`.
Follow-up complete: MCP compatibility auth and catalog/cache stores are now
split at the service boundary. `FileStore` remains as a compatibility
wrapper, while neutral config role roots can drive `FileAuthStore` and
`FileCatalogStore` separately.
- Completed: `wf run watch` starts run progress UX with polling over existing
`inspect_run` and optional bounded `read_run_trace`. SSE/WebSocket/MCP
progress remains deferred until polling UX proves insufficient.
- Completed: CLI remote error formatting now routes expected operation and
HTTP transport failures through compact Typer/Click errors by default.
`wf --verbose ...` preserves raw exception behavior for debugging.
- MCP package split direction: keep separating "MCP as a client transport"
from "MCP as an upstream workflow source provider." The future shape is
likely `wf_transport_mcp` for exposing workflow/admin surfaces to MCP
clients, and `wf_sources_mcp` for discovering/invoking upstream MCP servers
as workflow capabilities. The current `wf_mcp` package still contains both
roles plus compatibility entrypoints; new server/transport work should avoid
depending on that combined facade.
First `wf_sources_mcp` slice complete: MCP auth helpers and focused
auth/catalog stores now live in `wf_sources_mcp`, with `wf_mcp` compatibility
shims preserved. Runtime/session/source-registry moves remain future slices.
Keep `wf_mcp` re-export shims for compatibility and add import-direction
tests so `wf_sources_mcp` does not depend on workflow/admin surface,
frontend server, or proxy modules.
Second `wf_sources_mcp` slice complete: MCP desired source registry
models, file store, and conversion helpers now live in
`wf_sources_mcp.source_registry`, with `wf_mcp.source_registry` retained
as a compatibility shim.
Broker DTO construction moved out of `wf_sources_mcp`: source-provider
modules use structural legacy inputs only, while `wf_mcp.source_registry`
owns helpers that construct `ConnectionConfig`.
Third `wf_sources_mcp` slice complete: upstream MCP catalog/discovery DTOs
and catalog snapshot dumping now live in `wf_sources_mcp.catalog`, with
`wf_mcp.capabilities` and `wf_mcp.catalog.models` retained as shims.
Fourth `wf_sources_mcp` slice complete: upstream SDK protocol/result
types (`BackendAdapter`, `ToolExecutor`, `ToolCallResult`) now live in
`wf_sources_mcp.sdk`, with `wf_mcp.sdk` and `wf_mcp.runtime.protocols`
retained as compatibility shims.
Fifth `wf_sources_mcp` slice complete: MCP SDK conversion helpers now live
in `wf_sources_mcp.sdk.converters`, with `wf_mcp.sdk.converters` retained
as a compatibility shim.
- Completed: MCP upstream capability discovery (`discover_connection_capabilities`
and `DiscoveredConnectionCapabilities`) now lives in `wf_sources_mcp.discovery`.
Tool-to-NodeSpec wrapping remains in `wf_mcp` until the event/wrapper seam
is neutralized.
- Completed: MCP JSON-schema-to-Pydantic model helper now lives in
`wf_sources_mcp.schema_models`. `wf_mcp.workflow.wrappers` still owns
tool wrapper event emission, but no longer owns the shared schema compiler.
- Completed: MCP tool wrapper event emission now uses neutral
`wf_sources_mcp.tool_events` DTOs. Broker discovery projects those events
into `McpEvent`, preparing `wrap_discovered_tool` for a package move.
- Completed: MCP discovered-tool wrapper generation (`wrap_discovered_tool`)
now lives in `wf_sources_mcp.tool_wrappers`. `wf_mcp.workflow` remains a
compatibility shim; broker discovery imports the canonical wrapper.
- Completed: neutral `specs_from_discovered_tools` now lives in
`wf_sources_mcp.discovery`. `wf_mcp.broker.discovery` remains as the
broker adapter for `ConnectionConfig` and `McpEvent` projection.
- Completed: upstream MCP adapter lookup (`require_adapter`) now lives in
`wf_sources_mcp.adapters`, with `wf_mcp.broker.service.adapters` retained
as a compatibility shim.
- Completed: MCP source ID validation is canonical in `wf_sources_mcp.ids`.
`wf_sources_mcp` no longer imports legacy `wf_mcp.connections` or
`wf_mcp.shared.names` for source ID/path-safety checks.
The `wf-mcp` script is now a legacy/special-purpose MCP entrypoint, not the
preferred durable workflow server. New product paths should target
`wf-rpc-server` plus neutral `wf_config`/`wf_server` composition, then keep
shrinking `wf_mcp` toward upstream MCP source utilities, MCP transport
adapters, proxy/debug compatibility, and old entrypoint shims.
MCP UI/App metadata is source metadata only for now. Do not advertise widget
or MCP Apps support through workflow transports until a dedicated MCP
frontend transport owns iframe hosting, `ui://` resources, app-only tool
calls, and bridge semantics. Raw proxy/debug paths may expose upstream MCP
behavior explicitly, but that is not the durable workflow surface.
- Cleanup candidate: consolidate store/source registry id validation patterns
(`SOURCE_REGISTRY_ID_PATTERN`, `STORE_ID_PATTERN`) only after another package
needs the same rule. Today they intentionally stay close to their stores.
- Longer term: make the MCP frontend an adapter over these neutral workflow,
source-admin, and config-admin surfaces so the old `wf_mcp` server entry
point can shrink or retire.
Source registry, neutral MCP source config, role-specific stores, and local/dev
auth admin are implemented. The next work is polish, not new broad surfaces.
5. **CLI/API alignment**
- Completed for the basic lifecycle: selected `wf` commands can target local
process-backed stores/runtime or JSON-RPC HTTP through the same
`WorkflowApiSurface`.
- Current alignment notes are recorded in
[2026-06-03 CLI/API alignment notes](./superpowers/specs/2026-06-03-cli-api-alignment-notes.md).
- Completed: no workflow lifecycle command imports
`load_local_cli_context_from_typer`; `wf docs`, `wf schema`, and
`wf explain` remain static/local utilities for now.
- Preserve the current local CLI path until server source registry/auth/admin
operations are proven remotely.
- Keep config bootstrap separate from mutable store-backed source registry state.
- Keep auth payload values write-only; display summaries must show metadata and
payload keys only.
- Keep role-specific stores filesystem-only until a real SQL/secret-manager slice
is planned.
- Deferred auth work: OAuth/OIDC, production secret manager integration,
encrypted-at-rest file format, and provider-specific display models.
- Active specs:
- [`workflow config targets and sources`](superpowers/specs/2026-06-03-workflow-config-targets-and-sources.md)
- [`store-backed source registry`](superpowers/specs/2026-06-03-store-backed-source-registry-design.md)
- [`auth/source secrets boundary`](superpowers/specs/2026-06-06-auth-source-secrets-boundary.md)
6. **Workflow primitive polish**
- Return to native subgraph polish, fork/gather, foreach follow-ups, and graph
authoring UX after the durability/platform path is stable.
## Priority 4: MCP Package Split Finish Line
## Runtime and Platform Roadmap
`wf_sources_mcp` now owns upstream MCP source implementation pieces: ids,
registry DTOs, auth/catalog stores, discovery/catalog DTOs, SDK adapter/facade,
runtime pool, schema helpers, tool events, wrappers, and adapter lookup.
- Scheduler foundation decision record:
[ADR 0001](./adr/0001-scheduler-foundation-before-concurrent-foreach.md).
- Concurrent foreach policy decision record:
[ADR 0002](./adr/0002-concurrent-foreach-policy-and-barrier-commits.md).
- Native subgraph design spec:
[2026-05-24 native subgraphs](./superpowers/specs/2026-05-24-native-subgraphs-design.md).
- **Native subgraphs / graph-as-node**: core has `SubgraphNode`, structural
`WorkflowRef`, workflow-level outcomes plus explicit `EndNode` termination,
authoring helpers (`subgraph_ref` / `WorkflowBuilder.subgraph`), and artifact
reference conversion helpers. Core can now execute a prepared local child
workflow through an isolated child scope/lineage, preserve its trace entries,
map child output through the boundary, and route by the child's terminal
outcome. Prepared child interrupts now bubble through a typed internal route
and resume inside child scope while the public request identifies the parent
subgraph boundary. The workflow platform now resolves non-interrupting saved
child artifact refs into native prepared dependencies; descendant logical
capabilities inherit the root deployment binding environment, and missing or
cyclic saved children fail validation before a run starts. Wrapper helpers
currently run child workflows as ordinary nodes; native
`SubgraphNode` is now the graph-as-node path for prepared children.
`WorkflowBuilder.prepare_subgraph()` and `WorkflowBuilder.resume()` make the
local runnable/resumable path available without core-runtime plumbing.
Saved interrupting artifacts can now pause and resume through
`run_deployment`/`resume_run` across handler/server recreation by restoring
stopped checkpoints and pinned root/child artifact definitions.
- **Concurrent foreach**: implemented in core with explicit scheduling,
reducer/merge semantics, item error policy, async handler batching, and
quiescent interrupt behavior. Remaining work is polish and future reuse of
its barrier/lineage machinery by native subgraphs and fork/gather. Current
lineage progress includes ordered `StateWrite` records, `LineageStateView`,
foreach item `lineage_id`s, nested foreach lineage identity, root
`RuntimeScope` / `LineageState` storage, scope-aware reads, and non-root write
buffering. New concurrent foreach item writes are stored in
`RunState.lineages`, while `ForeachBarrierState` keeps scheduling/result
metadata and compatibility patches. Scope-root commits now apply to both the
root workflow and prepared native child scopes through the explicit
scope/lineage commit helper.
- **Durable run history and resume**: the design is recorded in
[2026-05-26 durable workflow runs](./superpowers/specs/2026-05-26-durable-workflow-runs-and-resume-design.md).
A validated `RunState` codec and dedicated run/checkpoint store now persist
interrupted, completed, and failed stopped snapshots. Stable `run_id` values
support compact `inspect_run` and bounded `read_run_trace` reads. Resume
revalidates its pinned dependency environment and reports `blocked` without
consuming input when a required source is unavailable. Ordinary live
tool/source failures remain failed runs, not implicit pauses.
- **OpenAPI capability sources**: raw OpenAPI operations can be represented as
workflow-facing capabilities using the OpenAPI document as the source of
truth. Runtime execution now follows the `openapi-core` plan: public payloads
keep OpenAPI names, generic `httpx` builds requests, and `openapi-core`
validates/unmarshals requests and responses. Generated Python client parsing
is explicitly retired. See
[OpenAPI capability sources](./openapi_capability_source.md).
- **Protocol-native long-running runs**: investigate MCP tasks/progress
notifications for long-running workflow execution. Avoid inventing a custom
"start" convention unless protocol-native behavior is insufficient.
- **Dynamic saved workflows as tools**: defer until the stable run/inspect
surface is strong. Many MCP clients do not refresh tool lists reliably, so
`wf.workflow.run_deployment` remains the dependable front door.
- **Dashboard/source controls**: future UI should consume the same source
inventory and deployment metadata instead of reverse-engineering MCP tools.
- **Workflow API extraction**: completed staged extraction context is archived in
[wf_api extraction roadmap](./historical/superpowers/plans/2026-06-01-wf-api-extraction-roadmap.md).
Protocol-neutral operation context and domain services now exist behind
`wf_api`; MCP tool schemas and tool registration stay in `wf_mcp`.
- Workflow store ownership is explicit: entrypoints construct/inject `WorkflowStores`; `WfMcpService` no longer guesses stores from the MCP store root.
- Double-delegation has been removed: CLI and MCP workflow tools construct
`WorkflowApi(context_from_service(service))` directly. `WorkflowSurfaceHandlers`
remains only as a temporary compatibility shim for older imports.
- `WfMcpService` is being reduced into injected implementation services. Source
registry and catalog projection now live in `SourceCatalogService`; the old
service methods remain as compatibility delegates for MCP broker callers.
- Workflow runtime execution is being separated from broker coordination.
`WorkflowRuntimeService` now owns plan compilation, dependency preparation,
run, and resume; `WfMcpService` keeps delegate methods for compatibility.
- Upstream MCP transport is being separated from broker coordination.
`UpstreamTransportService` now owns adapter registration, auth persistence,
catalog refresh I/O, resource/prompt reads, raw method/notification calls,
generated-tool executor selection, and live source diagnostics.
- Broker event recording is being separated from broker coordination.
`BrokerEventRecorder` now owns EventBus publication, event history reads,
simple event construction, and catalog-change fanout. `WfMcpService` keeps
delegate methods for compatibility.
- Connection ownership now lives in `ConnectionService`: it owns the broker
`ConnectionRegistry`, reserved connection-id rejection, `register_connection`,
and `sync_connections_from_config`. `WfMcpService.connections` remains a
compatibility property while source hydration still belongs to
`SourceCatalogService`.
- MCP model ownership is being compartmentalized. `wf_mcp.broker.models` owns
broker/connection config dataclasses, `wf_mcp.catalog.models` owns catalog
snapshots, and `wf_mcp.auth` owns the legacy MCP auth record plus adapter
helpers. `wf_mcp.models` remains a compatibility facade for older imports.
- Several reusable implementation pieces still live in `wf_mcp` because they
are MCP-shaped today (`source_registry.py`, `broker/server.py`, and focused
`broker/service/*` services). The next config migration should make the
split explicit: neutral config/registry mechanics belong in `wf_config`,
`wf_api`, `wf_server`, or another platform package; MCP-specific transport,
adapter, and upstream session behavior stays in `wf_mcp`.
Next split work should be selective:
Frame stress points remaining for native subgraphs and future fork/gather:
- Avoid new dependencies on the combined `wf_mcp` facade from durable server or
transport packages.
- Keep `wf_mcp` compatibility shims until callers are retired deliberately.
- Move only pieces with clear package ownership. Do not move proxy/UI/App
metadata support into workflow transports by accident.
- MCP UI/App metadata remains source/proxy metadata only; do not advertise MCP
Apps/widget support through durable workflow transports yet.
- `RunState.current_frame_id` remains the selected execution cursor even though
concurrent foreach now schedules multiple child frames. Native subgraphs
must preserve that cursor model while owning a nested child execution scope.
- `ExecutionFrame.metadata` has typed foreach access paths, but subgraphs still
need typed child-workflow ownership and completion metadata rather than new
ad hoc dictionary fields.
- Subgraph frames need child workflow identity/version/deployment binding, not
just a generic metadata dictionary.
- `RunState.current_node_id` duplicates the current frame's node id for
convenience. Any multi-frame scheduler must either keep that as a selected
cursor or replace it with an explicit scheduling view.
## Priority 5: Runtime/Core Polish Later
## Why This Order
Core runtime foundations for native subgraphs, concurrent foreach, lineage state,
and durable stopped-run resume exist. Return here after product/server UX is
stable.
The MCP workflow authoring path is now usable enough for real testing. The next
bottleneck is runtime/platform correctness: optional per-use-site child
deployment overrides, protocol-native progress reporting, and stronger durable
run operations beyond stopped checkpoints. Concurrent foreach, native saved
child execution, and durable interrupt resume now supply scheduler/lineage
precedent. Those remaining pieces should come before adding more high-level
authoring sugar.
- Native subgraph polish: optional per-use-site child deployment overrides and
clearer child trace inspection.
- Concurrent foreach polish: reuse barrier/lineage machinery for future
fork/gather.
- Protocol-native progress: investigate MCP tasks/progress or WebSocket/SSE only
after polling `wf run watch` proves insufficient.
- OpenAPI sources: continue from [`openapi capability sources`](openapi_capability_source.md)
when a real non-MCP source is needed.
## Recently Completed Platform Milestones
- `WorkflowApiSurface` is the protocol-neutral workflow operation contract.
- `wf_transport_rpc_http` exposes local/static and MCP-backed `WorkflowServer`
over JSON-RPC HTTP.
- `wf` can target local or remote workflow APIs for capability discovery, draft
authoring, artifact/deployment operations, run, inspect, bounded trace,
resume, and `cap call`.
- Desired source registry reads, mutations, and explicit apply/reload are exposed
through JSON-RPC and CLI.
- Neutral `wf_config` can express MCP sources, role-specific filesystem stores,
and client/server target separation.
- `wf config migrate-mcp` converts legacy broker configs to neutral workflow
config without mutating the original.
- `McpRuntimePool` is shared for stateful upstream MCP operations and has
JSON-RPC E2E coverage proving session reuse across workflow runs.
- `wf run watch` provides polling-based progress UX.
- CLI expected errors are compact by default; `wf --verbose ...` preserves raw
tracebacks for debugging.
## Historical References
- [`wf_api extraction roadmap`](historical/superpowers/plans/2026-06-01-wf-api-extraction-roadmap.md)
- [`source registry next slices`](historical/superpowers/plans/2026-06-03-source-registry-next-slices.md)
- [`MCP source connection seam`](historical/superpowers/plans/2026-06-07-mcp-source-connection-seam.md)
- [`MCP runtime RPC session reuse E2E`](historical/superpowers/plans/2026-06-08-mcp-runtime-rpc-session-reuse-e2e.md)
docs/superpowers/plans/2026-06-08-wf-status-and-product-smoke.md
+502
View File
@@ -0,0 +1,502 @@
# wf Status And Product Smoke Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add a read-only `wf status` command that tells users what target they are connected to and summarizes server/source/admin state, then add a small product smoke test path for the remote server workflow.
**Architecture:** Do not add a new backend status endpoint in this slice. The CLI should compose status from existing surfaces already available on `CliContext`: workflow capability list, source inventory, admin connection/status/events/auth summaries, and source registry summaries when available. Local/static servers and remote RPC servers should both work; unavailable optional admin surfaces should report `available: false` instead of failing the whole command.
**Tech Stack:** Python 3.14, Typer CLI, existing `WorkflowApiSurface` / admin surface protocols, JSON output via `wf_cli.io.emit_json`, pytest, basedpyright, ruff.
---
## File Structure
- Create: `src/wf_cli/commands/status.py`
- Owns `wf status`.
- Builds one compact JSON payload from existing CLI context surfaces.
- Contains small private helpers for safe optional calls.
- Modify: `src/wf_cli/app.py`
- Register `status` as a top-level command.
- Modify: `tests/wf_cli/test_remote_target.py`
- Add remote status test using the existing ASGI-backed RPC client patch seam.
- Create: `tests/wf_cli/test_status.py`
- Add local/static and partial-unavailable behavior tests.
- Modify: `docs/wf_cli.md`
- Document `wf status`.
- Modify: `docs/current_roadmap.md`
- Mark this slice complete after implementation.
Do not modify `wf_server`, `wf_api`, or `wf_transport_rpc_http` unless tests prove an existing surface is missing.
---
## Payload Contract
`wf status` must emit JSON:
```json
{
"target": {
"mode": "local|remote",
"config_path": "wf.config.json",
"url": "http://127.0.0.1:8765/rpc|null"
},
"workflow": {
"capability_count": 88,
"sample_capabilities": ["wf.std.constant"]
},
"sources": {
"available": true,
"source_count": 7,
"sample_sources": ["wf.std"]
},
"admin": {
"available": true,
"connection_count": 4,
"status_count": 4,
"event_count": 12,
"auth_count": 1
},
"registry": {
"available": true,
"entry_count": 0
}
}
```
Rules:
- Counts may be `0`.
- Optional sections must keep `available: false` when unavailable.
- Do not include auth payload values.
- Do not include full capability/source/connection lists; this command is a quick orientation summary.
- Do not mutate config, stores, registry, auth, or catalog.
---
### Task 1: Add Local Status Command Tests
**Files:**
- Create: `tests/wf_cli/test_status.py`
- Create later: `src/wf_cli/commands/status.py`
- Modify later: `src/wf_cli/app.py`
- [ ] **Step 1: Write failing local status test**
Create `tests/wf_cli/test_status.py`:
```python
from __future__ import annotations
import json
from typer.testing import CliRunner
from wf_cli.app import app
def test_wf_status_local_static_target(tmp_path) -> None:
config_path = tmp_path / "wf.json"
config_path.write_text(
json.dumps(
{
"version": 1,
"client": {"target": {"kind": "local"}},
"server": {
"store": {
"kind": "filesystem",
"root": str(tmp_path / "store"),
}
},
}
),
encoding="utf-8",
)
result = CliRunner().invoke(
app,
[
"--config",
str(config_path),
"--local",
"status",
],
)
assert result.exit_code == 0, result.output
payload = json.loads(result.output)
assert payload["target"]["mode"] == "local"
assert payload["target"]["config_path"] == str(config_path)
assert payload["target"]["url"] is None
assert payload["workflow"]["capability_count"] >= 1
assert "wf.std.constant" in payload["workflow"]["sample_capabilities"]
assert payload["sources"]["available"] is True
assert payload["sources"]["source_count"] >= 1
assert payload["admin"]["available"] is True
assert payload["registry"]["available"] is False
```
- [ ] **Step 2: Run the failing test**
Run:
```powershell
uv run pytest tests/wf_cli/test_status.py::test_wf_status_local_static_target -q
```
Expected: FAIL because `wf status` is not registered.
---
### Task 2: Implement `wf status`
**Files:**
- Create: `src/wf_cli/commands/status.py`
- Modify: `src/wf_cli/app.py`
- [ ] **Step 1: Create status command module**
Create `src/wf_cli/commands/status.py`:
```python
from __future__ import annotations
from typing import Any
import typer
from wf_cli.context import CliTyperState, load_cli_context_from_typer
from wf_cli.io import emit_json
from wf_cli.remote_errors import run_cli_operation
def status_command(ctx: typer.Context) -> None:
"""Print a compact read-only summary of the selected workflow target."""
state = CliTyperState.from_context(ctx)
context = load_cli_context_from_typer(ctx)
payload: dict[str, Any] = {
"target": {
"mode": "remote" if state.rpc_url is not None else "local",
"config_path": str(context.config_path),
"url": state.rpc_url,
},
"workflow": _workflow_status(context),
"sources": _sources_status(context),
"admin": _admin_status(context),
"registry": _registry_status(context),
}
emit_json(payload)
def _workflow_status(context) -> dict[str, Any]:
capabilities = run_cli_operation(
context,
context.handlers.list_capabilities(limit=20),
)
items = capabilities.get("capabilities", [])
names = [
item.get("name")
for item in items
if isinstance(item, dict) and isinstance(item.get("name"), str)
]
return {
"capability_count": len(items),
"sample_capabilities": names[:5],
}
def _sources_status(context) -> dict[str, Any]:
try:
payload = run_cli_operation(context, context.source_admin.list_sources(limit=20))
except Exception as exc:
return _unavailable(exc)
sources = payload.get("sources", [])
source_ids = [
item.get("id")
for item in sources
if isinstance(item, dict) and isinstance(item.get("id"), str)
]
return {
"available": True,
"source_count": len(sources),
"sample_sources": source_ids[:5],
}
def _admin_status(context) -> dict[str, Any]:
try:
connections = run_cli_operation(context, context.admin.list_connections())
statuses = run_cli_operation(context, context.admin.get_connection_statuses())
events = run_cli_operation(context, context.admin.list_events())
auth = run_cli_operation(context, context.admin.list_auth_records())
except Exception as exc:
return _unavailable(exc)
return {
"available": True,
"connection_count": len(connections.get("connections", [])),
"status_count": len(statuses.get("statuses", [])),
"event_count": len(events.get("events", [])),
"auth_count": len(auth.get("auth_records", [])),
}
def _registry_status(context) -> dict[str, Any]:
admin = context.source_registry_admin
if admin is None:
return {"available": False, "reason": "source registry admin is not configured"}
try:
payload = run_cli_operation(context, admin.list_registry_entries(limit=20))
except Exception as exc:
return _unavailable(exc)
return {
"available": True,
"entry_count": len(payload.get("entries", [])),
}
def _unavailable(exc: Exception) -> dict[str, Any]:
return {
"available": False,
"reason": str(exc),
}
```
- [ ] **Step 2: Register top-level command**
Modify `src/wf_cli/app.py`:
```python
from .commands import (
admin,
artifacts,
caps,
deployments,
docs,
drafts,
explain,
runs,
schema,
sources,
status,
)
```
Add registration near the other top-level commands:
```python
app.command("status")(status.status_command)
```
- [ ] **Step 3: Run local status test**
Run:
```powershell
uv run pytest tests/wf_cli/test_status.py::test_wf_status_local_static_target -q
```
Expected: PASS.
- [ ] **Step 4: Typecheck the new command**
If basedpyright reports unknown types in helper functions, add local annotations by importing `CliContext`:
```python
from wf_cli.context import CliContext, CliTyperState, load_cli_context_from_typer
```
Then annotate helpers:
```python
def _workflow_status(context: CliContext) -> dict[str, Any]:
```
Run:
```powershell
uv run basedpyright --level error src\wf_cli\commands\status.py tests\wf_cli\test_status.py
```
Expected: `0 errors`.
---
### Task 3: Add Remote Status Test
**Files:**
- Modify: `tests/wf_cli/test_remote_target.py`
- [ ] **Step 1: Add remote test using existing RPC patch helper**
Append this test near the existing remote target CLI tests:
```python
def test_wf_status_uses_rpc_url_override(monkeypatch, tmp_path) -> None:
server = build_local_static_workflow_server(tmp_path / "store")
_patch_rpc_client_to_server(monkeypatch, server)
config_path = tmp_path / "wf.json"
config_path.write_text('{"version": 1}', encoding="utf-8")
result = CliRunner().invoke(
app,
[
"--config",
str(config_path),
"--url",
"http://test/rpc",
"status",
],
)
assert result.exit_code == 0, result.output
payload = json.loads(result.output)
assert payload["target"]["mode"] == "remote"
assert payload["target"]["url"] == "http://test/rpc"
assert payload["workflow"]["capability_count"] >= 1
assert payload["sources"]["available"] is True
assert payload["admin"]["available"] is True
assert payload["registry"]["available"] is False
```
- [ ] **Step 2: Run remote test**
Run:
```powershell
uv run pytest tests/wf_cli/test_remote_target.py::test_wf_status_uses_rpc_url_override -q
```
Expected: PASS.
---
### Task 4: Add Product Smoke Script Documentation
**Files:**
- Modify: `docs/wf_cli.md`
- Modify: `docs/current_roadmap.md`
- [ ] **Step 1: Document `wf status`**
In `docs/wf_cli.md`, under the remote server section or before capability
discovery, add:
````markdown
Check the selected target:
```bash
wf status
wf --url http://127.0.0.1:8765/rpc status
```
`status` is read-only. It reports the selected target, capability/source
availability, admin counts, auth record count, and desired registry count when
the target exposes those admin surfaces. It does not return auth payload values.
```
````
- [ ] **Step 2: Mark roadmap item complete**
In `docs/current_roadmap.md`, under `Priority 1: Product Smoke And Status UX`,
change:
```markdown
- Add `wf status` as a compact target/server status command.
```
to:
```markdown
- Completed: `wf status` is a compact read-only target/server status command.
```
Leave the manual product smoke item open.
- [ ] **Step 3: Run docs lint**
Run:
```powershell
uv run ruff check docs\wf_cli.md docs\current_roadmap.md
```
Expected: no lint errors. Ruff may report "No Python files found"; that is acceptable for markdown-only paths.
---
### Task 5: Final Verification
**Files:** all touched files.
- [ ] **Step 1: Run focused tests**
Run:
```powershell
uv run pytest tests/wf_cli/test_status.py tests/wf_cli/test_remote_target.py -q
```
Expected: all tests pass.
- [ ] **Step 2: Run broader CLI/RPC smoke tests**
Run:
```powershell
uv run pytest tests/wf_transport_rpc_http tests/wf_cli/test_remote_target.py tests/wf_cli/test_status.py -q
```
Expected: all tests pass.
- [ ] **Step 3: Run lint**
Run:
```powershell
uv run ruff check src\wf_cli tests\wf_cli tests\wf_transport_rpc_http
```
Expected: `All checks passed!`
- [ ] **Step 4: Run typecheck**
Run:
```powershell
uv run basedpyright --level error src\wf_cli tests\wf_cli\test_status.py tests\wf_cli\test_remote_target.py
```
Expected: `0 errors, 0 warnings, 0 notes`.
- [ ] **Step 5: Manual product smoke**
If a server is running:
```powershell
uv run wf --url http://127.0.0.1:8765/rpc status
uv run wf --url http://127.0.0.1:8765/rpc cap call wf.std.constant --input '{"value":"status smoke"}'
```
Expected:
- `status` returns JSON with `target.mode == "remote"`.
- `cap call` returns `outcome == "ok"`.
- [ ] **Step 6: Commit**
```powershell
git add src\wf_cli\commands\status.py src\wf_cli\app.py tests\wf_cli\test_status.py tests\wf_cli\test_remote_target.py docs\wf_cli.md docs\current_roadmap.md
git commit -m "feat: add workflow status command"
```
---
## Self-Review Checklist
- Spec coverage: this plan implements the roadmap's first Priority 1 item (`wf status`) and leaves manual product smoke as a visible follow-up.
- Placeholder scan: no TODO/TBD placeholders remain.
- Scope check: no new backend endpoint, no store mutation, no new server state.
- Type consistency: `CliContext`, `CliTyperState`, and existing admin/source methods match current code.
- Security: auth status returns counts only; no payload values.