lda-wf/docs/current_roadmap.md

Current roadmap

This roadmap records the product shape, active implementation order, and constraints that must survive future work. Completed narratives and executable plans live under historical/.

Use these references for orientation:

• project_map.md: package map and entry points
• wf_api_architecture.md: application programming interface (API), server, transport, and source boundaries
• wf_core_architecture.md: workflow model and runtime architecture
• wf_cli.md: current command-line interface (CLI) usage

Current product shape

The durable product path uses a neutral server composition behind local or remote clients:

wf_client / wf_cli / web console
  -> local WorkflowApi or JSON-RPC client
  -> wf_server.WorkflowServer
  -> wf_api application and administration surfaces
  -> wf_core / wf_artifacts / source providers

wf-rpc-server is the durable remote entry point. The old wf-mcp entry point remains a legacy or special-purpose Model Context Protocol (MCP) surface.

The Python client covers capability discovery, local graph authoring, remote validation, immutable artifact save, deployment selection, and durable runs. Draft workspaces remain a separate administration surface. The primary saved workflow lifecycle stays:

author -> validate -> save artifact -> deploy -> run -> inspect or resume

Active runtime sequence

The next two slices build on the foreach control-region, structured-context, scheduler, lineage, barrier, and persisted run step budget foundations in this order.

1. Consolidate runtime identity resolution

Introduce one internal resolver for a frame, lineage, runtime scope, and foreach activation environment. The resolver should validate the canonical identity chain once so fork/gather code does not pass related identifiers independently or repeat ownership walks.

2. Implement explicit fork and gather

Reuse the scheduler, activation, lineage, and reducer-aware barrier machinery:

• ADR-0006: explicit fork and topology-driven gather
• Draft contract and verification gates

Production planning remains gated on correlation analysis and merge-order semantics. The draft records two reproduced reference-model defects and points to archived runnable research; passing that prototype's tests is not approval to copy it into the runtime.

The isolated reference verification plan closes occurrence regressions and gathers executable decision evidence first; it does not authorize production fork/gather changes.

Outcomes continue to choose one transition. Forks create concurrent branch activations. Gathers wait on declared incoming topology, merge compatible lineages according to policy, and emit one continuation.

Runtime work after fork and gather

Defer these slices until the active sequence exposes a concrete need:

• Add optional per-use child deployment overrides and clearer child trace inspection for native subgraphs
• Investigate protocol-native progress or streaming only if polling through wf run watch proves inadequate
• Continue OpenAPI capability sources when a real non-MCP source requires them

Durable platform constraints

These rules describe current boundaries. New work should preserve them.

Run and resume correctness

Persisted interrupted runs, bounded trace reads, dependency revalidation, process-rebuild resume, and same-process resume serialization exist.

• Broken pinned dependencies produce blocked readiness and diagnostics
• Live tool or source failures produce failed runs, not implicit pauses
• Store-level concurrency must follow the store transaction boundary
• The current contracts are persisted run and resume and durable workflow runs

Source, authentication, and configuration boundaries

Source registry state, static configuration, runtime source sessions, and authentication records remain separate concerns.

• Keep configuration bootstrap separate from mutable source registry state
• Keep secret payload values write-only; inspection may expose metadata and payload keys
• Keep role-specific stores filesystem-backed until a real database or secret manager slice is planned
• Add new source families through the generic runtime source lifecycle instead of forcing them through MCP connection configuration
• Preserve the server and transport boundary

Current source contracts:

• workflow configuration and sources
• store-backed source registry
• authentication and source secrets

MCP package ownership

wf_sources_mcp owns upstream MCP source implementation. Keep durable server and transport packages independent of the combined wf_mcp facade. Retain compatibility shims only for real callers, move code only when ownership is clear, and keep MCP application or widget metadata out of durable workflow transports.

Established runtime baseline

The active sequence can assume these foundations:

• Native subgraph scopes and durable return to the parent node
• Concurrent foreach with activation barriers and reducer-aware lineage merges
• Validated foreach back-edges with one static control region per node use
• Structured runtime context shared by execution, expressions, validation, and authoring references
• Removal of the pass-through JoinNode; future GatherNode starts with its actual synchronization contract and no placeholder compatibility
• Durable stopped-run inspection and resume
• Persisted run-wide step budget (RunLimits, steps_executed, computed steps_remaining) covering every frame and subgraph scope, surviving checkpoint and resume, with exhaustion as a failed run
• Python client reconstruction of capabilities, artifacts, deployments, and runs through the API

The current foreach return contract is foreach back-edge design. The current context contract is structured runtime context.

Historical entry points

Use these completed roadmaps when implementation history matters:

• API extraction roadmap
• source registry slices
• MCP source connection seam
• MCP runtime session reuse