lda-wf/docs/project_map.md

Project Map

This repository has workflow kernel, API/server, transport, source, CLI, examples, and tests packages. The older MCP package still exists, but new durable client paths should go through wf_server plus transport/source packages.

For the source-provider-specific map and source/tool/capability terminology, see source_architecture.md. For source provider setup examples, see source_provider_guide.md.

For a presentation-oriented summary of the current product path and demo flow, see workflow platform presentation. For running and auditing external-agent workflow challenges, see agent challenge evaluation. For verified Python 3.14 dependency constraints and their removal criteria, see dependency compatibility.

Packages

Package	Purpose	Usual callers
wf_core	Deterministic workflow kernel: models, validation, runtime, run state, traces, interrupts, foreach, and path/state operations.	Runtime users, wf_authoring, workflow adapters.
wf_authoring	Ergonomic workflow construction: @node, NodeSpec, builder DSL, conditions, path helpers, reusable ops, subgraph nodes.	Humans, tests, future LLM workflow builders.
wf_api	Workflow application surface over core/artifacts/platform: capabilities, drafts, artifacts, deployments, runs, and source/admin surfaces.	wf_cli, wf_server, JSON-RPC clients, future transports.
wf_server	Durable server composition boundary around WorkflowApi plus optional admin/source-registry surfaces. Owns the wf-rpc-server startup CLI/policy.	Transport packages and server startup code.
wf_transport_rpc_http	JSON-RPC-over-HTTP app/client and compatibility CLI shim.	Remote wf clients and local server smoke tests.
wf_client	Async-native Python client for capability discovery, local authoring, immutable artifacts, deployments, and durable runs.	Python applications and notebooks using a workflow server.
wf_sources_mcp	MCP-as-upstream-source implementation: ids, registry DTOs, auth/catalog stores, discovery, SDK client/facade, runtime pool, wrappers.	wf_server, broker glue, MCP source tests.
wf_mcp	MCP frontend/compatibility package: legacy wf-mcp entrypoints, broker glue, proxy/admin tools, and shims while extraction continues.	Compatibility callers and MCP transport work.
wf_cli	Command-line frontend over local or remote workflow APIs.	Humans, scripts, agent skills.
wf_contract_manifest	Tooling that normalizes the composed workflow OpenRPC document into the checked transport-neutral contract manifest and detects drift.	Python and TypeScript contract generation and tests.
@lda/workflow-rpc	Effect RPC client boundary plus generated compile-time inventory and raw wire types for all manifest operations. It includes a fail-closed representative JSON Schema-to-Effect translator; runtime decoders and supported operations remain authored subsets.	Web console, Hono server, future TypeScript workflow clients.
@lda/presentation-sync	Shared, bounded wire contract for ephemeral LAN presentation rooms.	Browser @lda/console and Hono @lda/web-server.
@lda/console	React console with a persistent routed shell, loopback connection flow, capability discovery and schema-generated direct-call playground, bounded capability-backed draft authoring with canonical mutation handling, and routed artifact/deployment/run exploration.	Browser users and the built @lda/web-server static host.
@lda/web-server	Hono API/static server that enforces the browser operation policy, proxies typed workflow reads, serves console/dist, and owns presentation room transport.	Browser @lda/console and local workflow operators.

The TypeScript presentation synchronization boundary is deliberately narrow. @lda/web-server owns room creation, membership, revision ordering, expiry, presence, and termination. @lda/console owns storyboard and navigation semantics and publishes only canonical hashes through @lda/presentation-sync; no storyboard data or workflow operation enters the room service. Browser workflow operations remain behind the Hono server and can continue to reach a loopback-only workflow RPC server.

The console workspace boundary is web/apps/console/src/workspace. Its domain/ clients own capability, draft-workspace, and lifecycle read contracts; route modules consume those clients through the evidence-aware read executor instead of calling transport operation strings directly. ConsoleShell owns the persistent connection header, lifecycle navigation, and operation-evidence ledger while /console/* route modules own their read-only page projections. The Discover route's capability playground uses the same write executor for explicit, acknowledged direct calls. It resolves supported local JSON Schema references for generated forms and retains bounded, redacted target-bearing evidence; a direct call does not create workflow lifecycle records. The selected-step authoring boundary keeps canonical bindings in src/wf_core/models/input_bindings.py; the console's recursive expression projection and controls live in web/apps/console/src/workspace/authoring/input-expression-editor.ts and InputExpressionControl.tsx. These editors emit one expression binding for a constructed array or object rather than synthetic indexed targets. authoring-contract-models.ts and authoring-contract-client.ts carry the revision-scoped, backend-owned inventory used by the graph's derived Input, State, Output, and Outcomes projections. WorkflowContractInspector.tsx edits those workflow-level contracts through the existing focused draft mutations; AuthoringPathPicker.tsx offers normal grouped choices first and an explicit Advanced custom-path fallback. The inventory may advertise node-scoped runtime context only for a selected step where conservative execution-scope analysis proves it applicable. It does not make context a permanent graph node or expose it as a final workflow-output source.

Important Entry Points

• wf_core: public kernel facade for common runtime/model imports.
• wf_core.runtime: execute_workflow, resume_workflow, step_workflow, and async variants.
• wf_core.models: concrete Pydantic workflow model package.
• wf_core.validation: structural workflow validation.
• wf_authoring: public authoring facade.
• wf_authoring.WorkflowBuilder: graph construction.
• wf_authoring.node: typed Python function to NodeSpec.
• docs/wf_authoring_control_flow.md: when to use branch, handle, match, when, and choose.
• wf_api.WorkflowApi: process-local workflow application facade.
• wf_api.models: canonical transport-neutral request and result models shared by process-local surfaces and remote transports. Legacy wf_mcp DTOs are compatibility contracts, not a second source of truth.
• wf_server.WorkflowServer: durable workflow server composition object.
• wf_transport_rpc_http.RpcWorkflowApiClient: JSON-RPC client implementing the workflow/admin surfaces over HTTP.
• wf_client.App: transport-independent async Python facade over capabilities, authored workflows, saved artifacts, deployments, and durable runs.
• wf_transport_rpc_http.create_rpc_app: JSON-RPC HTTP adapter over an existing WorkflowServer.
• wf_sources_mcp.McpRuntimePool: persistent MCP source runtime for stateful upstream tools/resources/prompts.
• wf_mcp: MCP-specific frontend and compatibility package.
• wf-mcp: legacy/special-purpose MCP script from pyproject.toml.
• wf-rpc-server: preferred durable workflow server script for CLI/API clients, implemented by wf_server.cli.
• python -m wf_contract_manifest write|check: regenerate or verify contracts/workflow-api.manifest.json from the real composed workflow server.
• pnpm --dir web --filter @lda/workflow-rpc contract:write|contract:check: regenerate or verify the checked TypeScript wire inventory and raw types.
• wf_mcp.broker.WfMcpService.get_catalog(): backend MCP catalog snapshots.
• wf_mcp.broker.WfMcpService.get_planner_catalog(): backend snapshots plus broker-local workflow sources such as wf.std and wf.mcp.

Examples

• examples/demo_workflow.py contains the declared demo workflow and demo node registry used by main.py and workflow tests. It is intentionally outside wf_core so the kernel package does not carry fixture/demo code.
• examples/authoring_control_flow.py demonstrates WorkflowBuilder.branch, handle, match, when, choose, and use_ref with executable examples.
• examples/wrapper_status_route.py and examples/wrapper_normalization.py show two wrapper styles: routing on provider status fields, and converting provider status fields into workflow outcomes.
• examples/mcp_workflow_surface.py shows the fixture-style MCP workflow path: discover a backend tool, create a draft artifact, save a deployment, and run it while wiring the generated ok and error outcomes.
• examples/rpc_cli_smoke.py spawns wf-rpc-server, runs the bounded CLI lifecycle from the RPC CLI smoke runbook, and cleans up. Use --keep-temp to preserve the generated config/store on failure.
• examples/browser_click_workflow/ is a serial browser-click workflow example with bounded before/after snapshots and full lifecycle tests.

Python client walkthrough

The Python client is intended for an application that already has a running workflow server. This is the complete shape of a real client call; the schema arguments may be JSON Schema dictionaries or the application's schema model values:

from wf_client import App

app = App.from_http_jsonrpc("http://localhost:8765/rpc")
capability = await app.capability("wf.std.constant")
graph = app.new_workflow(
    "example",
    input_schema=InputModel,
    state_schema=StateModel,
    output_schema=OutputModel,
)
step = graph.use(capability)
graph.set_entry_point(step)
validation = await graph.validate()
validation.raise_for_errors()
artifact = await graph.save(version=1)
run = await artifact.run({})

The graph is a local, mutable builder. validate() checks its structure locally and then asks the server to validate the serialized plan. save() persists an immutable artifact version; it does not deploy or execute the graph. artifact.run() selects or creates a deployment, validates its source bindings, and starts a durable run. The returned run is a loaded snapshot; call refresh(), resume(), or bounded trace(start=..., limit=...) when more server state is needed.

Draft workspaces are intentionally not part of wf_client. They are a separate server/admin surface and must be explicitly enabled when composing a server.

• examples/agent_challenges/ contains reusable opencode challenge harnesses for evaluating whether agents can use the public workflow CLI/server path.

Documentation

• docs/thesis/system-design-implementation.md — formal thesis/system-design draft.
• docs/thesis/evidence-index.md — claim-to-evidence map for the thesis draft.
• docs/runbooks/agent-challenge-evaluation.md — operator runbook for challenge trials, manual audits, and report interpretation.

Tests

• tests/authoring: builder, node decorator, ops, async runtime, subgraph, and demo workflow comparisons.
• tests/wf_mcp: MCP SDK adapter, broker, proxy, storage, CLI, and naming behavior.
• tests/rewrite: local rewrite/port experiments that should keep exercising real user ergonomics.
• tests/fixtures: test-only helper servers and fixtures.

Verification Commands

uv run --with pytest pytest -q
uv run ruff check src tests main.py examples
uv run basedpyright src\wf_core tests\authoring tests\rewrite examples main.py --level error

Use uv run --env-file .env --with pytest pytest -q when live MCP-backed tests need local environment configuration.

Where To Add Things

• Add new executable workflow semantics in wf_core.runtime / wf_core.runtime.ops.
• Add new graph/model syntax in wf_core.models, then validate it in wf_core.validation.
• Add author convenience helpers in wf_authoring, not wf_core.
• Add upstream MCP source/provider behavior in wf_sources_mcp.
• Keep wf_mcp changes limited to MCP frontend/broker/proxy compatibility unless the work is explicitly retiring old callers.
• Add durable workflow server behavior in wf_server or transport packages, not the legacy wf-mcp entrypoint.
• Add broker-local workflow utilities as WfMcpService spec sources, not as fake MCP connections.
• Add runnable examples in examples.
• Add test-only servers or helpers in tests/fixtures.