update/fmt docs, lineage_writes_for_frame

This commit is contained in:
lda
2026-05-24 23:22:58 +07:00 Verified
parent e48656c165
commit 2214746a54
50 changed files with 555 additions and 221 deletions
+5 -1
View File
@@ -3,17 +3,20 @@
`lda-workflow-as-struct` is a Python 3.14 prototype for `lda.chat`: an AI-assisted workflow system where an LLM plans structured workflows and a deterministic executor validates/runs them.
Main packages live under `src/`:
- `wf_core`: workflow model, validation, runtime semantics, frames, trace, interrupts, foreach, async execution.
- `wf_authoring`: ergonomic authoring layer including `@node`, `NodeSpec`, `WorkflowBuilder`, conditions, paths, and subgraph wrapping.
- `wf_mcp`: MCP broker/proxy layer for managing multiple backend MCP connections, transparent FastMCP proxying, live config/admin tools, hot reload, tool introspection, discovery/catalog snapshots, and eventual workflow build/run integration.
Important docs:
- `readme.md`: running design notes and architecture.
- `authoring_sketch.md`: authoring API direction.
- `wf_mcp_plan.md`: MCP proxy/broker/workflow integration plan.
- `scratchpad.md`: rough design history.
Current MCP direction:
- Transparent proxy mode is the main product path.
- Old broker mode remains useful for debugging/admin/catalog operations.
- Protocol-native FastMCP proxying exposes upstream tools/resources/prompts as first-class MCP capabilities.
@@ -21,6 +24,7 @@ Current MCP direction:
- Direct Serena is configured outside `wf_mcp` and should be preferred for code navigation/editing because it does not reset when `wf-mcp` hot-reloads.
Recent `wf_mcp` capabilities:
- Pydantic config boundary in `config_models.py`.
- Config mutation boundary in `config_manager.py`.
- Proxy validation in `proxy_validation.py`.
@@ -29,4 +33,4 @@ Recent `wf_mcp` capabilities:
- Opaque cursor pagination helpers in `pagination.py`.
- Admin tools under `wf.mcp_*`: list/get config, add/update/enable/disable/remove connection, reload config, list/get proxy tools.
- `wf.mcp_list_proxy_tools` supports `connection_id`, `query`, `limit`, and `cursor`; returns `{tools, nextCursor, total}`.
- `wf.mcp_get_proxy_tool` returns one detailed proxied tool row with schema where available.
- `wf.mcp_get_proxy_tool` returns one detailed proxied tool row with schema where available.
+5 -1
View File
@@ -1,6 +1,7 @@
# Style And Conventions
General:
- Python 3.14, `src/` layout, Pydantic v2 where external/boundary validation is useful.
- Prefer explicit dataclasses for runtime/internal models and Pydantic for config/wire-ish boundary validation.
- Async-first for MCP calls and workflow runtime interactions.
@@ -8,6 +9,7 @@ General:
- Do not leak workflow-only fields into MCP `tools/list`.
MCP/proxy conventions:
- Transparent proxy mode is the product path; old broker mode is secondary/debug/admin-oriented.
- Admin tools live under the reserved namespace `wf.mcp_*`.
- Upstream FastMCP names use Namespace behavior: `<connection_id>_<local_tool_name>`, e.g. `everything.default_echo`.
@@ -17,6 +19,7 @@ MCP/proxy conventions:
- `wf_mcp.config.json` is user-owned live config. Do not edit/revert it unless explicitly asked.
Code style:
- Use precise type hints and modern Python collection syntax (`list[str]`, `dict[str, Any]`).
- Keep modules layered by responsibility; avoid stuffing everything into service/runtime files.
- Prefer small helper modules when behavior becomes a boundary (`config_models.py`, `config_manager.py`, `proxy_validation.py`, `names.py`, `pagination.py`).
@@ -25,6 +28,7 @@ Code style:
- For MCP client `CallToolResult.structured_content`, guard for `None` in tests before subscripting; use helper assertions where useful.
Editing rules from repo collaboration:
- Use `apply_patch` for manual code edits.
- Do not revert user-owned changes.
- Direct Serena MCP is available and can be used for semantic navigation/editing; onboarding is already complete.
- Direct Serena MCP is available and can be used for semantic navigation/editing; onboarding is already complete.
+7 -1
View File
@@ -3,24 +3,29 @@
Use PowerShell on Windows from the repo root.
Testing:
- `uv run --with pytest pytest -q`
- Focused MCP proxy tests: `uv run --with pytest pytest tests/test_wf_mcp_transparent_proxy.py -q`
- Focused names/pagination examples: `uv run --with pytest pytest tests/test_wf_mcp_names.py tests/test_wf_mcp_transparent_proxy.py -q`
Lint/type checks:
- `uv run ruff check src/wf_mcp tests`
- Focused basedpyright example: `uv run basedpyright src/wf_mcp/transparent_proxy.py src/wf_mcp/pagination.py --level error`
Formatting:
- `uv run ruff format`
CLI / MCP server:
- `uv run wf-mcp --config wf_mcp.config.json serve`
- Transparent proxy mode is default.
- Old broker mode: `uv run wf-mcp --config wf_mcp.config.json serve --mode broker`
- Optional compatibility/search flags: `--resources-as-tools`, `--prompts-as-tools`, `--search-tools`
Useful live MCP admin tools exposed by `wf-mcp`:
- `wf.mcp_list_connections`
- `wf.mcp_get_config`
- `wf.mcp_add_connection`
@@ -33,8 +38,9 @@ Useful live MCP admin tools exposed by `wf-mcp`:
- `wf.mcp_get_proxy_tool`
Useful Windows shell commands:
- Fast search: `rg "pattern" path`
- List files: `Get-ChildItem -Force`
- Read file: `Get-Content -Path path`
- Git status: `git status --short`
- Diff: `git diff -- path`
- Diff: `git diff -- path`
@@ -1,6 +1,7 @@
# Task Completion Checklist
Before considering a code task done:
- Run focused tests for the touched area.
- Run full tests when the change affects shared behavior: `uv run --with pytest pytest -q`.
- Run ruff on touched source/tests, usually `uv run ruff check src/wf_mcp tests` for MCP work.
@@ -10,8 +11,10 @@ Before considering a code task done:
- Summarize functional changes and verification results concisely.
Recent known-good full-suite count after proxy tool pagination/detail work:
- `48 passed, 1 skipped`
Known environment notes:
- Windows sandbox may block commands with `CreateProcessAsUserW failed: 5`; retry important commands with escalation rather than working around via unsafe shell tricks.
- Codex/native MCP tool schemas may not refresh dynamically after `wf-mcp` hot reload. A fresh client/session may be needed to see newly added MCP tools.
- Codex/native MCP tool schemas may not refresh dynamically after `wf-mcp` hot reload. A fresh client/session may be needed to see newly added MCP tools.
+1 -2
View File
@@ -1,7 +1,6 @@
# the name by which the project can be referenced within Serena
project_name: "lda-workflow-as-struct"
# list of languages for which language servers are started; choose from:
# al angular ansible bash clojure
# cpp cpp_ccls crystal csharp csharp_omnisharp
@@ -31,7 +30,7 @@ project_name: "lda-workflow-as-struct"
# The first language is the default language and the respective language server will be used as a fallback.
# Note that when using the JetBrains backend, language servers are not used and this list is correspondingly ignored.
languages:
- python
- python
# the encoding used by text files in the project
# For a list of possible encodings, see https://docs.python.org/3.11/library/codecs.html#standard-encodings
+1 -1
View File
@@ -23,7 +23,7 @@ same thing
```python
class OutputMap(BaseModel, Generic[TypeT]):
output: Pathexpr
key (give me a better key name): SupportToValue[TypeT]
key (give me a better key name): SupportToValue[TypeT]
```
look at this support to value, or use another fitting name.
+12 -2
View File
@@ -63,8 +63,18 @@ Typical entry points:
```json
{
"input": [{"target": {"root": "local", "parts": ["text"]}, "path": {"root": "input", "parts": ["text"]}}],
"output": [{"source": {"root": "local", "parts": ["echoed"]}, "target": {"root": "state", "parts": ["echoed"]}}]
"input": [
{
"target": { "root": "local", "parts": ["text"] },
"path": { "root": "input", "parts": ["text"] }
}
],
"output": [
{
"source": { "root": "local", "parts": ["echoed"] },
"target": { "root": "state", "parts": ["echoed"] }
}
]
}
```
+28 -28
View File
@@ -43,18 +43,18 @@ Examples:
{
"input": [
{
"target": {"root": "local", "parts": ["user", "name"]},
"path": {"root": "state", "parts": ["person", "name"]}
"target": { "root": "local", "parts": ["user", "name"] },
"path": { "root": "state", "parts": ["person", "name"] }
},
{
"target": {"root": "local", "parts": ["mode"]},
"target": { "root": "local", "parts": ["mode"] },
"value": "fast"
}
],
"output": [
{
"source": {"root": "local", "parts": ["job", "wage"]},
"target": {"root": "state", "parts": ["job", "wage"]}
"source": { "root": "local", "parts": ["job", "wage"] },
"target": { "root": "state", "parts": ["job", "wage"] }
}
]
}
@@ -66,14 +66,14 @@ Whole-object mapping remains valid:
{
"input": [
{
"target": {"root": "local", "parts": ["user"]},
"path": {"root": "state", "parts": ["person"]}
"target": { "root": "local", "parts": ["user"] },
"path": { "root": "state", "parts": ["person"] }
}
],
"output": [
{
"source": {"root": "local", "parts": ["user"]},
"target": {"root": "state", "parts": ["person"]}
"source": { "root": "local", "parts": ["user"] },
"target": { "root": "state", "parts": ["person"] }
}
]
}
@@ -85,14 +85,14 @@ Whole-payload mapping uses the local root path `"."`:
{
"input": [
{
"target": {"root": "local", "parts": []},
"path": {"root": "state", "parts": ["rates"]}
"target": { "root": "local", "parts": [] },
"path": { "root": "state", "parts": ["rates"] }
}
],
"output": [
{
"source": {"root": "local", "parts": []},
"target": {"root": "state", "parts": ["rates"]}
"source": { "root": "local", "parts": [] },
"target": { "root": "state", "parts": ["rates"] }
}
]
}
@@ -122,12 +122,12 @@ Valid:
```json
[
{
"target": {"root": "local", "parts": ["user", "name"]},
"path": {"root": "state", "parts": ["person", "name"]}
"target": { "root": "local", "parts": ["user", "name"] },
"path": { "root": "state", "parts": ["person", "name"] }
},
{
"target": {"root": "local", "parts": ["user", "email"]},
"path": {"root": "state", "parts": ["person", "email"]}
"target": { "root": "local", "parts": ["user", "email"] },
"path": { "root": "state", "parts": ["person", "email"] }
}
]
```
@@ -137,12 +137,12 @@ Invalid:
```json
[
{
"target": {"root": "local", "parts": ["user"]},
"path": {"root": "state", "parts": ["person"]}
"target": { "root": "local", "parts": ["user"] },
"path": { "root": "state", "parts": ["person"] }
},
{
"target": {"root": "local", "parts": ["user", "name"]},
"path": {"root": "state", "parts": ["person", "name"]}
"target": { "root": "local", "parts": ["user", "name"] },
"path": { "root": "state", "parts": ["person", "name"] }
}
]
```
@@ -160,12 +160,12 @@ Invalid:
```json
[
{
"source": {"root": "local", "parts": ["user"]},
"target": {"root": "state", "parts": ["person"]}
"source": { "root": "local", "parts": ["user"] },
"target": { "root": "state", "parts": ["person"] }
},
{
"source": {"root": "local", "parts": ["user", "name"]},
"target": {"root": "state", "parts": ["person", "name"]}
"source": { "root": "local", "parts": ["user", "name"] },
"target": { "root": "state", "parts": ["person", "name"] }
}
]
```
@@ -228,8 +228,8 @@ and uses it for state writes.
"person": {
"type": "object",
"properties": {
"name": {"type": "string", "reducer": "wf.std.replace"},
"tags": {"type": "array", "reducer": "wf.std.append"}
"name": { "type": "string", "reducer": "wf.std.replace" },
"tags": { "type": "array", "reducer": "wf.std.append" }
}
},
"profile": {
@@ -245,7 +245,7 @@ Deprecated `fields` state declarations are still accepted at parse boundaries:
```json
{
"fields": {
"person.name": {"type": "string", "reducer": "wf.std.replace"}
"person.name": { "type": "string", "reducer": "wf.std.replace" }
}
}
```
+4 -1
View File
@@ -53,7 +53,10 @@ implementation state.
- **Concurrent foreach**: implemented in core with explicit scheduling,
reducer/merge semantics, item error policy, async handler batching, and
quiescent interrupt behavior. Remaining work is polish and future reuse of
its barrier/lineage machinery by native subgraphs and fork/gather.
its barrier/lineage machinery by native subgraphs and fork/gather. Current
lineage progress includes ordered `StateWrite` records, `LineageStateView`,
foreach item `lineage_id`s, and nested foreach lineage identity. Full
`RuntimeScope` / `LineageState` storage is still future work.
- **Persistent run history**: add a run store before adding stable `run_id`,
`inspect_run`, or `read_run_trace(run_id, range)` APIs. Current traces are
returned directly from immediate run responses.
+17 -17
View File
@@ -55,23 +55,23 @@ proxied: demo://everything/default/resource/static/document/instructions.md
- Resource links embedded inside tool results are not rewritten by the proxy.
For example, `get-resource-links` returns raw upstream URIs such as:
```text
demo://resource/dynamic/text/2
```
```text
demo://resource/dynamic/text/2
```
`wf-mcp` cannot read that raw URI. The manually namespaced URI works for
normal dynamic resources:
```text
demo://everything/default/resource/dynamic/text/2
```
```text
demo://everything/default/resource/dynamic/text/2
```
- Session resource links from `gzip-file-as-resource` are not currently usable
through `wf-mcp`. The tool returned:
```text
demo://resource/session/probe.txt
```
```text
demo://resource/session/probe.txt
```
Neither the raw URI nor the manually namespaced URI was readable through
`wf-mcp` during the live probe. Direct `everything` read also failed in this
@@ -80,9 +80,9 @@ demo://resource/session/probe.txt
- `simulate-research-query` is discovered, but calling it fails cleanly:
```text
Tool simulate-research-query requires task augmentation (taskSupport: 'required')
```
```text
Tool simulate-research-query requires task augmentation (taskSupport: 'required')
```
This is a task/protocol support gap, not an ordinary tool-call failure.
@@ -334,13 +334,13 @@ prompts_changed -> PromptListChangedNotification
This layer should not know about FastMCP sessions. It should be easy to test
with plain `mcp.types` objects.
2. Done: add a fake/test notification sink.
1. Done: add a fake/test notification sink.
The first sink should only record which MCP notification objects would be sent.
This proves the event-to-notification mapping without depending on Codex,
Inspector, stdio behavior, or Streamable HTTP behavior.
3. Done: add a FastMCP `Context` notification sink.
1. Done: add a FastMCP `Context` notification sink.
This sink can call:
@@ -352,7 +352,7 @@ It is only valid while handling a request that has an active FastMCP context.
This should be treated as a session-scoped projection, not a global broadcast
system.
4. Partly done: wire local admin operations first.
1. Partly done: wire local admin operations first.
Best first live target:
@@ -375,7 +375,7 @@ Remaining cleanup:
This is intentionally local. It does not require solving upstream notification
forwarding.
5. Verify with Inspector/Codex.
1. Verify with Inspector/Codex.
Expected outcomes:
@@ -384,7 +384,7 @@ Expected outcomes:
- clients that ignore notifications can still manually call list/search tools
- no workflow or broker correctness depends on notification delivery
6. Done: investigate baseline upstream notification forwarding.
1. Done: investigate baseline upstream notification forwarding.
Direct fixture-server tests show that list-changed, resource-updated, and
logging notifications are emitted upstream but are not forwarded automatically
+2 -2
View File
@@ -165,8 +165,8 @@ stays part of the reducer reference payload:
```json
{
"ref": {"source": "wf.std", "capability_key": "modulo_add"},
"config": {"modulus": 10}
"ref": { "source": "wf.std", "capability_key": "modulo_add" },
"config": { "modulus": 10 }
}
```
@@ -28,6 +28,7 @@
### Task 1: Add Canonical Capability Source Model
**Files:**
- Create: `src/wf_mcp/broker/service/capability_sources.py`
- Modify: `src/wf_mcp/broker/service/sources.py`
- Test: `tests/wf_mcp/test_service.py`
@@ -201,6 +202,7 @@ Expected: fail until service stores `capability_sources`; pass after Task 2.
### Task 2: Make `WfMcpService` Store Capability Sources
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- Modify: `src/wf_mcp/broker/service/specs.py`
- Test: `tests/wf_mcp/test_service.py`
@@ -310,6 +312,7 @@ Expected: pass.
### Task 3: Move All Authoring Ops Into `wf.std`
**Files:**
- Modify: `src/wf_mcp/broker/service/builtins.py`
- Test: `tests/wf_mcp/test_service.py`
@@ -426,6 +429,7 @@ Expected: pass.
### Task 4: Add `wf.admin` Source Without MCP Exposure
**Files:**
- Create: `src/wf_mcp/broker/admin_capabilities.py`
- Modify: `src/wf_mcp/broker/service/core.py`
- Test: `tests/wf_mcp/test_service.py`
@@ -554,6 +558,7 @@ Expected: pass.
### Task 5: Project Broker Admin Tools From `wf.admin`
**Files:**
- Modify: `src/wf_mcp/broker/tools.py`
- Test: `tests/wf_mcp/test_broker_server.py`
@@ -627,6 +632,7 @@ Expected: pass.
### Task 6: Normalize Transparent Proxy Admin Naming Strategy
**Files:**
- Modify: `src/wf_mcp/shared/names.py`
- Modify: `src/wf_mcp/transparent_proxy/runtime.py`
- Test: `tests/wf_mcp/test_names.py`
@@ -710,6 +716,7 @@ Expected: pass if `LdaNamespace` preserves dotted names. If FastMCP still emits
### Task 7: Update Docs And Full Verification
**Files:**
- Modify: `docs/wf_mcp_capability_sources.md`
- Modify: `docs/wf_mcp_architecture.md`
- Test: full verification commands
@@ -21,6 +21,7 @@
## Task 1: Artifact Models
**Files:**
- Create: `src/wf_artifacts/models.py`
- Create: `src/wf_artifacts/__init__.py`
- Test: `tests/artifacts/test_models.py`
@@ -214,6 +215,7 @@ Expected: pass.
## Task 2: File Artifact Store
**Files:**
- Modify: `src/wf_artifacts/store.py`
- Modify: `src/wf_artifacts/__init__.py`
- Test: `tests/artifacts/test_store.py`
@@ -401,6 +403,7 @@ Expected: pass.
## Task 3: Verification
**Files:**
- No production changes unless verification exposes issues.
- [ ] **Step 1: Run focused artifact tests**
@@ -212,6 +212,7 @@ config_reloaded
## Phase 1: Inventory And Adapter Reality Check
**Files:**
- Create: `docs/mcp_protocol_proxy_inventory.md`
- Inspect: `src/wf_mcp/sdk/adapter.py`
- Inspect: `src/wf_mcp/transparent_proxy/runtime.py`
@@ -228,12 +229,13 @@ config_reloaded
- sampling
- tasks
- [ ] Identify SDK gaps before implementation. If an MCP feature is not
accessible through FastMCP/MCP SDK at our current version, document it instead
of inventing a fake abstraction.
accessible through FastMCP/MCP SDK at our current version, document it instead
of inventing a fake abstraction.
## Phase 2: Extract Shared Workflow/Admin Handlers
**Files:**
- Create: `src/wf_mcp/workflow_surface/handlers.py`
- Create: `src/wf_mcp/admin_surface/handlers.py`
- Modify: `src/wf_mcp/broker/artifact_tools.py`
@@ -243,17 +245,18 @@ config_reloaded
- Test: `tests/wf_mcp/test_transparent_proxy.py`
- [ ] Move workflow artifact list/save/inspect/validate/run logic into shared
handler functions/classes.
handler functions/classes.
- [x] Move admin list/refresh/config/reload logic into shared handler
functions/classes.
functions/classes.
- [x] Keep broker compatibility tool names working.
- [x] Keep transparent proxy admin tool names working.
- [ ] Do not change behavior in this phase; only remove duplicated logic and
create a single implementation path.
create a single implementation path.
## Phase 3: Unified Server Factory
**Files:**
- Create: `src/wf_mcp/server/unified.py`
- Modify: `src/wf_mcp/cli.py`
- Modify: `src/wf_mcp/broker/server.py`
@@ -277,6 +280,7 @@ config_reloaded
## Phase 4: Namespacing And Collision Policy
**Files:**
- Modify: `src/wf_mcp/shared/names.py`
- Test: `tests/wf_mcp/test_names.py`
- Test: `tests/wf_mcp/test_unified_server.py`
@@ -286,28 +290,30 @@ config_reloaded
- [ ] Keep `wf.mcp.*` for workflow runtime helpers, not admin.
- [ ] Keep upstream proxy names collision-safe.
- [ ] Reject configured connection ids that collide with reserved local
namespaces.
namespaces.
- [ ] Decide whether compatibility broker names remain visible by default in
unified mode. Recommended: yes during migration, no after migration.
unified mode. Recommended: yes during migration, no after migration.
## Phase 5: Tool/Resource/Prompt Projection Parity
**Files:**
- Modify: `src/wf_mcp/transparent_proxy/runtime.py`
- Modify: unified server files from Phase 3.
- Test: `tests/wf_mcp/test_unified_server.py`
- [ ] Ensure upstream tools and stable workflow/admin tools both appear in
`tools/list`.
`tools/list`.
- [ ] Ensure upstream resources appear in `resources/list` and can be read.
- [ ] Ensure upstream prompts appear in `prompts/list` and can be rendered.
- [ ] Ensure resources-as-tools and prompts-as-tools remain optional projection
modes, not the only way to access resources/prompts.
modes, not the only way to access resources/prompts.
- [ ] Ensure search/pagination includes stable local tools and upstream tools.
## Phase 6: Local Notification Bus
**Files:**
- Create: `src/wf_mcp/events/bus.py`
- Modify: `src/wf_mcp/broker/events.py`
- Modify: `src/wf_mcp/broker/service/core.py`
@@ -330,17 +336,18 @@ config_reloaded
- [x] `resources_changed`
- [x] `prompts_changed`
- [ ] Do not emit MCP notifications yet unless the server/session API is
clearly available. This phase creates the source of truth.
clearly available. This phase creates the source of truth.
## Phase 7: MCP Notifications
**Files:**
- Modify: unified server files from Phase 3.
- Modify: `src/wf_mcp/events/bus.py`
- Test: `tests/wf_mcp/test_unified_server.py`
- [ ] Emit MCP list-changed notifications when local or upstream catalogs
change, if supported:
change, if supported:
- `notifications/tools/list_changed`
- `notifications/resources/list_changed`
- `notifications/prompts/list_changed`
@@ -348,61 +355,64 @@ config_reloaded
- [ ] Proxy upstream logging notifications where supported.
- [ ] Ensure clients that ignore notifications can still poll/list manually.
- [ ] Add tests that assert notifications are requested/emitted through whatever
FastMCP/MCP SDK surface is available. If no testable surface exists, document
the limitation in `docs/mcp_protocol_proxy_inventory.md`.
FastMCP/MCP SDK surface is available. If no testable surface exists, document
the limitation in `docs/mcp_protocol_proxy_inventory.md`.
## Phase 8: Elicitation And Sampling Routing
**Files:**
- Create: `src/wf_mcp/protocol/elicitation.py`
- Create: `src/wf_mcp/protocol/sampling.py`
- Modify: SDK adapter/session layer if supported.
- Test: `tests/wf_mcp/test_protocol_proxy.py`
- [ ] Determine how upstream MCP SDK exposes server-to-client elicitation
requests.
requests.
- [ ] Determine how FastMCP exposes downstream client elicitation responses.
- [ ] Route upstream elicitation requests to the downstream client only when the
downstream client advertised support.
downstream client advertised support.
- [ ] Route upstream sampling requests to the downstream client only when the
downstream client advertised support.
downstream client advertised support.
- [ ] Preserve request ids/correlation ids so responses return to the correct
upstream session.
upstream session.
- [ ] Return structured unsupported diagnostics when routing is impossible.
- [ ] Do not convert elicitation/sampling into normal tools as the primary
behavior.
behavior.
## Phase 9: Tasks And Long-Running Workflow Runs
**Files:**
- Create: `src/wf_mcp/workflow_surface/runs.py`
- Modify: unified server files from Phase 3.
- Test: `tests/wf_mcp/test_workflow_tasks.py`
- [ ] Prefer MCP Tasks for long-running `wf.workflow.run_deployment` when the
client/server support task execution.
client/server support task execution.
- [ ] Keep synchronous run behavior for short/manual local tests.
- [ ] Add a compatibility run store only if MCP Tasks are unavailable or
insufficient for Codex/Inspector.
insufficient for Codex/Inspector.
- [ ] Map workflow interrupts to task status such as `input_required` only after
the runtime supports the needed resume model.
the runtime supports the needed resume model.
- [ ] Do not implement durable scheduling/cron here.
## Phase 10: Mode Migration
**Files:**
- Modify: `docs/wf_mcp_architecture.md`
- Modify: `docs/wf_mcp_capability_sources.md`
- Modify: `src/wf_mcp/cli.py`
- Test: `tests/wf_mcp/test_cli.py`
- [ ] Document unified mode as the recommended local mode once it passes manual
Codex and Inspector checks.
Codex and Inspector checks.
- [ ] Keep broker/proxy modes as compatibility modes.
- [ ] Mark compatibility broker tool names as legacy once namespaced local tools
work in unified mode.
work in unified mode.
- [ ] Do not delete compatibility modes until tests cover every important
surface.
surface.
> **Superseded on 2026-05-16:** the compatibility period is now considered long
> enough. The current plan is to retire the public broker/proxy mode split in
@@ -420,9 +430,9 @@ config_reloaded
- [ ] Inspector shows upstream resources and prompts.
- [ ] Inspector shows local workflow tools with useful names/descriptions.
- [ ] If everything-server elicitation is triggered, the proxy either routes it
correctly or returns a clear unsupported diagnostic.
correctly or returns a clear unsupported diagnostic.
- [ ] If everything-server progress/logging is triggered, the proxy either
forwards it correctly or documents why not.
forwards it correctly or documents why not.
## Non-Goals
@@ -30,37 +30,40 @@ merge MCP server modes and does not add native subgraphs.
### Task 1: Validate Plan Shape During Artifact Creation
**Files:**
- Modify: `src/wf_artifacts/factory.py`
- Test: `tests/artifacts/test_factory.py`
- [ ] Add a failing test proving `create_workflow_artifact_from_plan` rejects a
plan that cannot become a `wf_core.Workflow`.
plan that cannot become a `wf_core.Workflow`.
- [ ] Implement validation by constructing `wf_core.Workflow.model_validate`
from the plan fields.
from the plan fields.
- [ ] Keep the validation dependency one-way: `wf_artifacts` may import
`wf_core`, but `wf_core` must not import `wf_artifacts`.
`wf_core`, but `wf_core` must not import `wf_artifacts`.
- [ ] Return `ValueError` with a concise message containing the failing field
path or Pydantic error message.
path or Pydantic error message.
- [ ] Run `uv run --with pytest pytest tests\artifacts\test_factory.py -q`.
### Task 2: Add Artifact Creation Diagnostics
**Files:**
- Modify: `src/wf_artifacts/models.py`
- Modify: `src/wf_artifacts/factory.py`
- Test: `tests/artifacts/test_factory.py`
- [ ] Decide whether creation failures should raise exceptions only or also
expose a `validate_workflow_artifact_plan(...) -> list[DependencyDiagnostic]`
style function.
expose a `validate_workflow_artifact_plan(...) -> list[DependencyDiagnostic]`
style function.
- [ ] Recommended v1: add a separate `validate_workflow_artifact_plan(plan)`
that returns structured diagnostics, while the factory still raises on errors.
that returns structured diagnostics, while the factory still raises on errors.
- [ ] Add tests for missing `input_schema`, missing `output_schema`, invalid
state schema, and missing start node.
state schema, and missing start node.
### Task 3: Validate Direct Workflow Dependencies
**Files:**
- Modify: `src/wf_artifacts/validation.py`
- Test: `tests/artifacts/test_validation.py`
@@ -68,35 +71,37 @@ merge MCP server modes and does not add native subgraphs.
- [ ] Validate exact artifact-version pins.
- [ ] Return `dependency_missing` when a child artifact version does not exist.
- [ ] Return `dependency_cycle` when direct/transitive workflow artifact
dependencies cycle.
dependencies cycle.
- [ ] Do not copy child dependency snapshots into the parent.
### Task 4: Improve Capability Contract Checking
**Files:**
- Modify: `src/wf_artifacts/validation.py`
- Test: `tests/artifacts/test_validation.py`
- Optional: `src/wf_mcp/broker/artifact_tools.py`
- [ ] Preserve current hash comparison behavior.
- [ ] Add tests for kind mismatch, for example required `tool` but available
`node_spec`.
`node_spec`.
- [ ] Decide whether `node_spec` can satisfy `tool` when the provider is an MCP
wrapper. Recommended: no implicit kind coercion in `wf_artifacts`; adapters
should present the available kind they mean to expose.
wrapper. Recommended: no implicit kind coercion in `wf_artifacts`; adapters
should present the available kind they mean to expose.
- [ ] Add diagnostics with `code="capability_kind_mismatch"`.
### Task 5: Document Current Runtime Limitations In Tool Responses
**Files:**
- Modify: `src/wf_mcp/broker/artifact_tools.py`
- Test: `tests/wf_mcp/test_broker_server.py`
- [ ] Keep interrupting artifacts rejected by `run_workflow_deployment`.
- [ ] Add `repair_hint` text explaining native subgraphs/nested resume are not
implemented yet.
implemented yet.
- [ ] Add a test proving unsupported interrupt artifacts return a diagnostic
instead of raising.
instead of raising.
## Verification
@@ -13,6 +13,7 @@
### Task 1: Pin Wrapper-Artifact Call Semantics
**Files:**
- Modify: `tests/wf_mcp/test_service.py`
- [ ] **Step 1: Write the failing test**
@@ -28,11 +29,13 @@ Expected: FAIL because `call_capability()` only resolves live specs today.
### Task 2: Resolve Wrapper Artifacts in the Workflow Surface
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- [ ] **Step 1: Implement minimal wrapper-artifact resolution**
Add a small helper that:
- recognizes stable artifact node names
- loads the artifact from the store
- rejects non-wrapper artifacts
@@ -49,6 +52,7 @@ Expected: PASS.
### Task 3: Verify the Whole Project
**Files:**
- No additional files.
- [ ] **Step 1: Run focused workflow-surface tests**
@@ -44,6 +44,7 @@
### Task 1: Pin Node-Local Path Behavior
**Files:**
- Modify: `tests/core/test_validation.py`
- Modify: `tests/core/test_runtime.py`
@@ -100,6 +101,7 @@ Expected: FAIL because node-local map sides are top-level-only today.
### Task 2: Add Node-Local Path Helpers
**Files:**
- Create: `src/wf_core/local_paths.py`
- Modify: `src/wf_core/validation/steps.py`
@@ -148,6 +150,7 @@ Expected: PASS for validation-specific cases.
### Task 3: Execute Nested Local Mappings
**Files:**
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Modify: `src/wf_core/runtime/ops/state.py`
@@ -184,6 +187,7 @@ Expected: PASS for nested mapping behavior.
### Task 4: Introduce Prepared Patch Commits and Extract Merge Dispatch
**Files:**
- Create: `src/wf_core/runtime/ops/merges.py`
- Modify: `src/wf_core/runtime/ops/state.py`
- Modify: `tests/core/test_state_ops.py`
@@ -248,6 +252,7 @@ Expected: PASS.
### Task 5: Keep Authoring and Docs Aligned
**Files:**
- Modify: `src/wf_authoring/builder/mapping.py`
- Modify: `docs/core_state_mapping_and_merge.md` if implementation details differ
- Modify: `docs/scratchpad.md` only if wording drift appears
@@ -272,6 +277,7 @@ declarations yet.
### Task 6: Verify the Whole Project
**Files:**
- No additional files.
- [ ] **Step 1: Run focused suites**
@@ -55,6 +55,7 @@
## Task 1: Split Draft Code Into Focused Modules
**Files:**
- Create: `src/wf_artifacts/drafts/models.py`
- Create: `src/wf_artifacts/drafts/api.py`
- Create: `src/wf_artifacts/drafts/adapter.py`
@@ -232,6 +233,7 @@ git commit -m "refactor: add keyed workflow draft models"
## Task 2: Add `use_ref` And Thin Adapter Over `WorkflowBuilder`
**Files:**
- Create: `src/wf_artifacts/drafts/adapter.py`
- Modify only if needed: `src/wf_authoring/builder/core.py`
- Modify only if needed: `src/wf_authoring/ops/*`
@@ -349,6 +351,7 @@ git commit -m "feat: adapt workflow drafts through workflow builder"
## Task 3: Replace Prototype Public API
**Files:**
- Create: `src/wf_artifacts/drafts/api.py`
- Modify: `src/wf_artifacts/drafts.py`
- Modify: `src/wf_artifacts/__init__.py`
@@ -441,6 +444,7 @@ git commit -m "feat: replace draft prototype with keyed public api"
## Task 4: Update MCP Workflow Surface
**Files:**
- Modify: `tests/wf_mcp/test_workflow_surface.py`
- Modify: `tests/wf_mcp/test_server.py`
- Modify only if needed: `src/wf_mcp/workflow_surface/handlers.py`
@@ -507,6 +511,7 @@ git commit -m "feat: accept keyed workflow drafts over mcp"
## Task 5: Add Outcome Validation When Capability Contracts Are Available
**Files:**
- Modify: `src/wf_artifacts/drafts/api.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
@@ -569,6 +574,7 @@ git commit -m "feat: validate draft routes against known outcomes"
## Task 6: Update Documentation
**Files:**
- Modify: `docs/workflow_drafts.md`
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
- Modify: `docs/wf_mcp_operator_manual.md`
@@ -624,6 +630,7 @@ git commit -m "docs: describe keyed workflow draft surface"
## Task 7: Full Verification
**Files:**
- No new files
- [ ] **Step 1: Run focused verification**
@@ -156,6 +156,7 @@ Default response stays compact. If `include_draft=True`, include the full draft.
## Task 1: Add Draft Workspace Domain Models
**Files:**
- Create: `src/wf_artifacts/draft_workspaces/models.py`
- Create: `src/wf_artifacts/draft_workspaces/__init__.py`
- Modify: `src/wf_artifacts/__init__.py`
@@ -333,6 +334,7 @@ git commit -m "feat: add draft workspace models"
## Task 2: Add File-Backed Draft Workspace Store
**Files:**
- Create: `src/wf_artifacts/draft_workspaces/store.py`
- Modify: `src/wf_artifacts/draft_workspaces/__init__.py`
- Modify: `src/wf_artifacts/__init__.py`
@@ -499,6 +501,7 @@ git commit -m "feat: persist draft workspaces"
## Task 3: Add Draft Workspace API Functions
**Files:**
- Create: `src/wf_artifacts/draft_workspaces/api.py`
- Modify: `src/wf_artifacts/draft_workspaces/__init__.py`
- Modify: `src/wf_artifacts/__init__.py`
@@ -712,6 +715,7 @@ git commit -m "feat: patch draft workspaces by revision"
## Task 4: Wire Draft Workspace Store Into WfMcpService
**Files:**
- Modify: `src/wf_mcp/broker/service/core.py`
- Modify: `src/wf_mcp/broker/config.py`
- Test: `tests/wf_mcp/test_service.py`
@@ -800,6 +804,7 @@ git commit -m "feat: attach draft workspace store to service"
## Task 5: Add Workflow Surface Workspace Handlers
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
@@ -964,6 +969,7 @@ git commit -m "feat: expose draft workspace handlers"
## Task 6: Add Minimal Draft Workspace Bootstrapper
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
@@ -1100,6 +1106,7 @@ git commit -m "feat: bootstrap minimal draft workspaces"
## Task 7: Add Artifact Creation From Workspace
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
@@ -1219,6 +1226,7 @@ git commit -m "feat: save artifacts from draft workspaces"
## Task 8: Register MCP Workspace Tools
**Files:**
- Modify: `src/wf_mcp/workflow_surface/tools.py`
- Modify: `src/wf_mcp/workflow_surface/models.py`
- Modify: `src/wf_mcp/transparent_proxy/runtime.py`
@@ -1335,6 +1343,7 @@ git commit -m "feat: expose draft workspace mcp tools"
## Task 9: Documentation
**Files:**
- Modify: `docs/workflow_drafts.md`
- Modify: `docs/wf_mcp_operator_manual.md`
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
@@ -1413,6 +1422,7 @@ git commit -m "docs: describe draft workspace authoring loop"
## Task 10: Final Verification
**Files:**
- No code changes unless failures require fixes.
- [ ] **Step 1: Run focused test suite**
@@ -41,6 +41,7 @@ rather than reimplementing the earlier tasks.
## Task 1: Add Typed Path Values
**Files:**
- Modify: `src/wf_core/paths.py`
- Modify: `src/wf_core/local_paths.py`
- Create: `tests/core/test_path_values.py`
@@ -218,6 +219,7 @@ Expected: all tests in `test_path_values.py` pass.
## Task 2: Add Canonical Node Binding Models
**Files:**
- Modify: `src/wf_core/models/steps.py`
- Test: `tests/core/test_canonical_node_bindings.py`
@@ -380,6 +382,7 @@ Expected: all tests in `test_canonical_node_bindings.py` pass.
## Task 3: Move Runtime Node Input Resolution To Canonical Bindings
**Files:**
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_nested_mappings.py`
- Test: `tests/core/test_canonical_node_bindings.py`
@@ -474,6 +477,7 @@ Expected: pass.
## Task 4: Move Runtime Output Writes To Canonical Bindings And Atomic Patches
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_atomic_state_patches.py`
@@ -592,6 +596,7 @@ Expected: pass.
## Task 5: Update Validation For Canonical Bindings
**Files:**
- Modify: `src/wf_core/validation/steps.py`
- Test: `tests/core/test_mapping_validation.py`
- Test: `tests/core/test_canonical_node_bindings.py`
@@ -651,6 +656,7 @@ Expected: pass.
## Task 6: Add Canonical State Schema Fields
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Modify: `src/wf_core/runtime/ops/state.py`
- Modify: `src/wf_core/validation/steps.py`
@@ -747,6 +753,7 @@ Expected: pass.
## Task 7: Harden SchemaRef With JSON Schema Validation
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Modify: `src/wf_core/runtime/ops/schemas.py`
- Test: `tests/core/test_schema_validation.py`
@@ -819,6 +826,7 @@ Expected: pass.
## Task 8: Update Authoring Helpers To Emit Canonical Bindings
**Files:**
- Modify: `src/wf_authoring/dsl/paths.py`
- Modify: `src/wf_authoring/dsl/conditions.py`
- Modify: `src/wf_authoring/builder/core.py`
@@ -886,6 +894,7 @@ Expected: authoring tests pass.
## Task 9: Full Compatibility And Regression Pass
**Files:**
- Modify docs/examples only if tests show stale serialized shapes.
- Test: full repo.
@@ -13,6 +13,7 @@
### Task 1: Add Canonical State Schema Tests
**Files:**
- Modify: `tests/core/test_nested_state_paths.py`
- Modify: `tests/core/test_schema_validation.py`
@@ -100,6 +101,7 @@ Expected: new tests fail because `StateSchema` still serializes as `fields: [...
### Task 2: Implement JSON-Schema-Native `StateSchema`
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- [ ] **Step 1: Make `StateSchema` inherit JSON Schema fields directly**
@@ -118,19 +120,26 @@ required: list[str] = Field(default_factory=list)
Keep accepting:
```json
{"fields": [{"path": "state.count", "type": "integer", "reducer": "wf.std.add"}]}
{
"fields": [
{ "path": "state.count", "type": "integer", "reducer": "wf.std.add" }
]
}
```
and:
```json
{"fields": {"count": {"type": "integer", "reducer": "wf.std.add"}}}
{ "fields": { "count": { "type": "integer", "reducer": "wf.std.add" } } }
```
by converting both into:
```json
{"type": "object", "properties": {"count": {"type": "integer", "reducer": "wf.std.add"}}}
{
"type": "object",
"properties": { "count": { "type": "integer", "reducer": "wf.std.add" } }
}
```
- [ ] **Step 3: Add `field_map()` as an internal compiled index**
@@ -154,6 +163,7 @@ Use `SchemaRef`/`jsonschema` validation for the complete state schema. Add expli
### Task 3: Update Artifact Reducer Extraction
**Files:**
- Modify: `src/wf_artifacts/factory.py`
- [ ] **Step 1: Extract reducer dependencies from `state_schema.properties`**
@@ -177,6 +187,7 @@ is included in required capabilities.
### Task 4: Update Authoring Conversion
**Files:**
- Modify: `src/wf_authoring/schemas.py`
- Modify: `tests/authoring/test_schemas.py`
@@ -191,6 +202,7 @@ Return `StateSchema.model_validate(schema_with_reducer_keywords)` so generated s
### Task 5: Update Docs and Examples
**Files:**
- Modify: `docs/core_state_mapping_and_merge.md`
- Modify: `docs/workflow_drafts.md`
- Modify: `docs/wf_mcp_operator_manual.md`
@@ -221,6 +233,7 @@ State clearly that `reducer` is not standard JSON Schema behavior. JSON Schema v
### Task 6: Verification
**Files:**
- All touched files
- [ ] **Step 1: Run focused tests**
@@ -13,6 +13,7 @@
### Task 1: Make Platform Refs Pydantic Boundary Types
**Files:**
- Modify: `src/wf_platform/refs.py`
- Modify: `tests/refs/test_platform_refs.py`
@@ -48,6 +49,7 @@ Reject whitespace-only refs and empty segments. Do not over-restrict valid MCP/s
### Task 2: Convert Deployment Bindings to List-of-Struct
**Files:**
- Modify: `src/wf_artifacts/models.py`
- Modify: `tests/artifacts/test_models.py`
- Modify: `tests/artifacts/test_store.py`
@@ -71,13 +73,15 @@ bindings: list[SourceBinding] = Field(default_factory=list)
Parse-only compatibility:
```json
{"bindings": {"demo": "demo.personal"}}
{ "bindings": { "demo": "demo.personal" } }
```
should normalize to:
```json
{"bindings": [{"logical_source": "demo", "concrete_source": "demo.personal"}]}
{
"bindings": [{ "logical_source": "demo", "concrete_source": "demo.personal" }]
}
```
- [ ] **Step 3: Add `binding_map()`**
@@ -95,6 +99,7 @@ Reject duplicate `logical_source` values during validation.
### Task 3: Convert Required Capabilities to List-of-Struct
**Files:**
- Modify: `src/wf_artifacts/models.py`
- Modify: `src/wf_artifacts/factory.py`
- Modify: `src/wf_artifacts/references.py`
@@ -168,6 +173,7 @@ Reject duplicate `ref` values during validation.
### Task 4: Update Call Sites to Use Helper Maps
**Files:**
- Modify: `src/wf_artifacts/validation.py`
- Modify: `src/wf_artifacts/catalog.py`
- Modify: `src/wf_mcp/workflow_surface/runtime_dependencies.py`
@@ -201,6 +207,7 @@ MCP tools that accept `required_capabilities` from callers may still accept dict
### Task 5: Update Docs
**Files:**
- Modify: `docs/workflow_artifacts.md`
- [ ] **Step 1: Replace dict binding examples**
@@ -210,7 +217,7 @@ Use:
```json
{
"bindings": [
{"logical_source": "context7", "concrete_source": "context7.default"}
{ "logical_source": "context7", "concrete_source": "context7.default" }
]
}
```
@@ -236,6 +243,7 @@ State that old dict shapes are accepted at parse boundaries but not emitted by m
### Task 6: Verification
**Files:**
- All touched files
- [ ] **Step 1: Run focused tests**
@@ -64,6 +64,7 @@ MCP JSON still serializes those enum values as strings.
## Task 1: Add Enum-Backed Hint Models
**Files:**
- Create: `src/wf_mcp/workflow_surface/wrapper_hints.py`
- Test: `tests/wf_mcp/test_workflow_wrapper_hints.py`
@@ -229,6 +230,7 @@ Expected: pass.
## Task 2: Derive Simple Wrapper Hints From Capability Schemas
**Files:**
- Modify: `src/wf_mcp/workflow_surface/wrapper_hints.py`
- Test: `tests/wf_mcp/test_workflow_wrapper_hints.py`
@@ -466,6 +468,7 @@ Expected: pass.
## Task 3: Add Boolean Outcome Candidate Tests
**Files:**
- Modify: `tests/wf_mcp/test_workflow_wrapper_hints.py`
- Modify: `src/wf_mcp/workflow_surface/wrapper_hints.py` only if tests reveal gaps.
@@ -535,6 +538,7 @@ Expected: pass. If arbitrary boolean fields produce candidates, fix `CONTROL_BOO
## Task 4: Add Complex Output Missing Decision Tests
**Files:**
- Modify: `tests/wf_mcp/test_workflow_wrapper_hints.py`
- Modify: `src/wf_mcp/workflow_surface/wrapper_hints.py` only if tests reveal gaps.
@@ -598,6 +602,7 @@ Expected: pass.
## Task 5: Wire Hints Into `inspect_capability`
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
@@ -675,6 +680,7 @@ Expected: pass.
## Task 6: Add Hints For Saved Wrapper Artifact Inspection
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
@@ -727,6 +733,7 @@ Expected: pass.
## Task 7: Document Hint Semantics
**Files:**
- Modify: `docs/workflow_capabilities.md`
- [ ] **Step 1: Add documentation section**
@@ -771,6 +778,7 @@ Expected: pass.
## Task 8: Final Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused tests**
@@ -811,4 +819,3 @@ Expected: ruff passes, basedpyright has 0 errors, and formatting is clean.
- Placeholder scan: No `TBD`, `TODO`, or unspecified implementation steps remain.
- Type consistency: `WrapperHintConfidence`, `WrapperOutcomePolicy`, `OutcomeCandidateKind`, and `MissingDecisionKind` are defined before use and serialize through Pydantic models.
- Scope check: This plan does not create or save wrappers automatically; it only adds hint payloads for authoring.
@@ -15,7 +15,7 @@
The core now supports structural path objects:
```json
{"root": "state", "parts": ["person.name", "three and four"]}
{ "root": "state", "parts": ["person.name", "three and four"] }
```
But `wf_authoring` still stores paths as strings:
@@ -137,6 +137,7 @@ into display strings just to pass through `wf_authoring`.
## Task 1: Add Path Input Coercion Module
**Files:**
- Create: `src/wf_authoring/dsl/path_inputs.py`
- Test: `tests/authoring/test_path_inputs.py`
@@ -241,6 +242,7 @@ Expected: all tests pass.
## Task 2: Make DSL Path Helpers Typed
**Files:**
- Modify: `src/wf_authoring/dsl/paths.py`
- Modify: `src/wf_authoring/dsl/conditions.py`
- Test: `tests/authoring/test_path_inputs.py`
@@ -333,6 +335,7 @@ Expected: all pass.
## Task 3: Make Builder Maps Accept Typed Path Inputs
**Files:**
- Modify: `src/wf_authoring/builder/mapping.py`
- Modify: `src/wf_authoring/builder/core.py`
- Modify: `src/wf_authoring/dsl/mapping.py`
@@ -434,6 +437,7 @@ Expected: all pass.
## Task 3.5: Foreach Boundary Check
**Files:**
- Inspect: `src/wf_authoring/builder/core.py`
- Inspect: `src/wf_core/models/steps.py` or current foreach model location
@@ -463,6 +467,7 @@ Do not document foreach as fully structural until the core field is structural.
## Task 4: Docs and Examples
**Files:**
- Modify: `docs/structural_refs.md`
- Modify or create an authoring docs/example if one already exists.
@@ -39,14 +39,14 @@ That matters because structural path dicts cannot be Python dict keys. The JSON/
{
"input": [
{
"target": {"root": "local", "parts": ["payload.email"]},
"path": {"root": "input", "parts": ["email.address"]}
"target": { "root": "local", "parts": ["payload.email"] },
"path": { "root": "input", "parts": ["email.address"] }
}
],
"output": [
{
"source": {"root": "local", "parts": ["result.score"]},
"target": {"root": "state", "parts": ["score"]}
"source": { "root": "local", "parts": ["result.score"] },
"target": { "root": "state", "parts": ["score"] }
}
]
}
@@ -211,6 +211,7 @@ Map sugar is for hashable Python authoring values only.
## Task 1: Add Canonical Binding Normalizers
**Files:**
- Modify: `src/wf_authoring/builder/mapping.py`
- Test: `tests/authoring/test_builder.py`
@@ -336,6 +337,7 @@ Expected: still fails until builder signatures are updated.
## Task 2: Add `input` / `output` to `use()`
**Files:**
- Modify: `src/wf_authoring/builder/core.py`
- Test: `tests/authoring/test_builder.py`
@@ -457,6 +459,7 @@ Expected: pass.
## Task 3: Add `input` / `output` to `use_ref()`
**Files:**
- Modify: `src/wf_authoring/builder/core.py`
- Test: `tests/authoring/test_builder.py`
@@ -542,6 +545,7 @@ Expected: pass.
## Task 4: Deprecate Map Sugar Explicitly
**Files:**
- Modify: `src/wf_authoring/builder/core.py`
- Test: `tests/authoring/test_builder.py`
@@ -642,6 +646,7 @@ Expected: both pass.
## Task 5: Reject Mixed Styles and Dict Keys Clearly
**Files:**
- Modify: `src/wf_authoring/builder/core.py`
- Modify: `src/wf_authoring/builder/mapping.py`
- Test: `tests/authoring/test_builder.py`
@@ -742,6 +747,7 @@ Expected: all pass.
## Task 6: Docs
**Files:**
- Modify: `docs/structural_refs.md`
- Modify: `docs/authoring_sketch.md`
- Modify: `docs/core_state_mapping_and_merge.md`
@@ -812,6 +818,7 @@ level.
## Task 7: Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused authoring builder tests**
@@ -46,18 +46,18 @@ Recommended node use shape:
"node": "demo.echo",
"input": [
{
"target": {"root": "local", "parts": ["text"]},
"path": {"root": "input", "parts": ["text"]}
"target": { "root": "local", "parts": ["text"] },
"path": { "root": "input", "parts": ["text"] }
},
{
"target": {"root": "local", "parts": ["limit"]},
"target": { "root": "local", "parts": ["limit"] },
"value": 3
}
],
"output": [
{
"source": {"root": "local", "parts": ["echoed"]},
"target": {"root": "state", "parts": ["echoed"]}
"source": { "root": "local", "parts": ["echoed"] },
"target": { "root": "state", "parts": ["echoed"] }
}
]
}
@@ -67,8 +67,8 @@ Recommended configured reducer shape:
```json
{
"ref": {"source": "wf.std", "capability_key": "modulo_add"},
"config": {"modulus": 10}
"ref": { "source": "wf.std", "capability_key": "modulo_add" },
"config": { "modulus": 10 }
}
```
@@ -116,6 +116,7 @@ Compact unconfigured reducer shorthand remains accepted:
## Task 1: Inventory MCP/Draft Surfaces That Emit Map Sugar
**Files:**
- Read-only first:
- `src/wf_mcp/workflow_surface/models.py`
- `src/wf_mcp/workflow_surface/handlers.py`
@@ -164,6 +165,7 @@ Findings from the first inventory pass:
## Task 2: Draft Adapter Emits Canonical Builder Bindings
**Files:**
- Modify: `src/wf_artifacts/drafts/adapter.py`
- Modify: `tests/artifacts/test_draft_adapter.py`
@@ -212,6 +214,7 @@ Expected: pass and no new deprecation warnings from the adapter.
## Task 3: Workflow Surface Requests Prefer Canonical Shapes
**Files:**
- Modify: `src/wf_mcp/workflow_surface/models.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `tests/wf_mcp/test_workflow_surface.py`
@@ -262,6 +265,7 @@ Expected: pass.
## Task 4: Inspect/List Outputs Show Canonical Refs and Display Strings Separately
**Files:**
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Modify: `src/wf_platform/sources.py` if inventory models need fields
- Modify: tests in `tests/wf_mcp`
@@ -273,7 +277,7 @@ For source/capability inspection responses, assert reducers include enough info:
```json
{
"name": "wf.std.add",
"ref": {"source": "wf.std", "capability_key": "add"},
"ref": { "source": "wf.std", "capability_key": "add" },
"description": "..."
}
```
@@ -305,6 +309,7 @@ Expected: pass.
## Task 5: Docs and MCP Tool Descriptions
**Files:**
- Modify: `docs/wf_mcp_operator_manual.md`
- Modify: `docs/workflow_drafts.md`
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
@@ -37,7 +37,7 @@ New canonical reducer ref:
```json
{
"ref": {"source": "wf.std", "capability_key": "add"},
"ref": { "source": "wf.std", "capability_key": "add" },
"config": {}
}
```
@@ -49,7 +49,7 @@ Compatibility inputs:
```
```json
{"name": "wf.std.add", "config": {"modulus": 10}}
{ "name": "wf.std.add", "config": { "modulus": 10 } }
```
For now, `ReducerRef.name` remains available as a display/registry key compatibility property. Runtime reducer registries are still keyed by strings such as `wf.std.add`.
@@ -83,6 +83,7 @@ For now, `ReducerRef.name` remains available as a display/registry key compatibi
## Task 1: Pin ReducerRef Compatibility and Canonical Dump
**Files:**
- Modify: `tests/core/test_nested_state_paths.py`
- [ ] **Step 1: Add reducer ref tests**
@@ -136,6 +137,7 @@ Expected: fail because `ReducerRef` does not parse strings and has no `ref`.
## Task 2: Implement Structural ReducerRef
**Files:**
- Modify: `src/wf_core/models/reducers.py`
- [ ] **Step 1: Update imports**
@@ -207,6 +209,7 @@ Expected: pass.
## Task 3: Update Reducer Field Serializers and Existing Expectations
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Modify tests that assert reducer dumps
@@ -228,8 +231,8 @@ Decide canonical output:
```json
{
"ref": {"source": "wf.std", "capability_key": "modulo_add"},
"config": {"modulus": 10}
"ref": { "source": "wf.std", "capability_key": "modulo_add" },
"config": { "modulus": 10 }
}
```
@@ -261,6 +264,7 @@ Expected: pass after updating expectations for configured reducer dumps if neede
## Task 4: Update Artifact Reducer Dependency Extraction
**Files:**
- Modify: `src/wf_artifacts/factory.py`
- Modify: `tests/artifacts/test_factory.py`
@@ -330,6 +334,7 @@ Expected: pass.
## Task 5: Runtime Compatibility Check
**Files:**
- Tests only unless failures require runtime changes
- [ ] **Step 1: Run reducer runtime tests**
@@ -349,6 +354,7 @@ Only if needed, update lookup code to use `reducer.name` as the compatibility st
## Task 6: Docs
**Files:**
- Modify: `docs/structural_refs.md`
- Modify: `docs/core_state_mapping_and_merge.md`
@@ -15,7 +15,7 @@
We just moved graph/node bindings toward structural paths:
```json
{"root": "state", "parts": ["person.name"]}
{ "root": "state", "parts": ["person.name"] }
```
But state schema indexing still builds rootless dotted strings in places:
@@ -31,7 +31,7 @@ That can corrupt JSON Schema property names containing dots:
{
"type": "object",
"properties": {
"person.name": {"type": "string", "reducer": "wf.std.replace"}
"person.name": { "type": "string", "reducer": "wf.std.replace" }
}
}
```
@@ -106,6 +106,7 @@ Reducer name: source capability ref, should use CapabilityRef later
## Task 1: Pin Literal Dotted State Property Behavior
**Files:**
- Test: `tests/core/test_nested_state_paths.py`
- Test: `tests/core/test_schema_validation.py`
@@ -164,6 +165,7 @@ Expected: fail before implementation.
## Task 2: Add Typed State Field Index
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Test: `tests/core/test_nested_state_paths.py`
@@ -272,6 +274,7 @@ Expected: pass.
## Task 3: Move Runtime Lookup to Typed State Paths
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- Test: `tests/core/test_nested_state_paths.py`
- Test: `tests/core/test_atomic_state_patches.py`
@@ -341,6 +344,7 @@ Expected: pass.
## Task 4: Structural `StateFieldDecl.path` Dump
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Test: `tests/core/test_nested_state_paths.py`
- Test: `tests/core/test_schema_validation.py`
@@ -356,7 +360,7 @@ Existing tests may expect:
Decide based on current path model direction. Since `StatePath` now serializes structurally elsewhere, prefer:
```json
{"path": {"root": "state", "parts": ["person.name"]}}
{ "path": { "root": "state", "parts": ["person.name"] } }
```
- [ ] **Step 2: Change serializer**
@@ -405,6 +409,7 @@ Expected: pass after expectation updates.
## Task 5: Authoring State Metadata Path Sweep
**Files:**
- Modify: `src/wf_authoring/schemas.py`
- Test: `tests/authoring/test_schemas.py`
@@ -481,6 +486,7 @@ Expected: pass.
## Task 6: ReducerRef Capability Ref Plan Stub
**Files:**
- Modify: `docs/structural_refs.md`
- Create: `docs/superpowers/plans/YYYY-MM-DD-reducer-ref-structural-capability.md`
@@ -511,6 +517,7 @@ Do not implement reducer structural refs in the state-schema path sweep unless t
## Task 7: Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused core tests**
@@ -120,6 +120,7 @@ The display string `workflow.echo_wrapper.v1` remains computable for list output
## Task 1: Make `CapabilityRef` Serialize Structurally
**Files:**
- Modify: `src/wf_platform/refs.py`
- Test: `tests/wf_platform/test_refs.py`
@@ -200,6 +201,7 @@ Expected: all tests pass.
## Task 2: Make `WorkflowCapabilityRef` Structural
**Files:**
- Modify: `src/wf_artifacts/refs.py`
- Test: `tests/wf_artifacts/test_refs.py`
@@ -269,6 +271,7 @@ Expected: all tests pass.
## Task 3: Save Required Capabilities in New Shape
**Files:**
- Modify: `src/wf_artifacts/models.py`
- Modify: `src/wf_artifacts/references.py`
- Test: `tests/wf_artifacts/test_models.py`
@@ -374,6 +377,7 @@ Expected: artifact creation still works; saved dumps use structural refs.
## Task 4: Make Deployment Bindings Structural on Save
**Files:**
- Modify: `src/wf_artifacts/models.py`
- Test: `tests/wf_artifacts/test_models.py`
- Test: `tests/wf_mcp/test_workflow_surface.py`
@@ -434,6 +438,7 @@ Expected: all tests pass.
## Task 5: Stop Parsing Workflow Capability Strings as Generic Capabilities
**Files:**
- Modify: `src/wf_mcp/workflow_surface/refs.py`
- Modify: `src/wf_mcp/workflow_surface/handlers.py`
- Test: `tests/wf_mcp/test_workflow_surface_refs.py`
@@ -504,6 +509,7 @@ Expected: all tests pass.
## Task 6: Keep Runtime Binding Source-Aware
**Files:**
- Modify: `src/wf_mcp/workflow_surface/runtime_dependencies.py`
- Test: `tests/wf_mcp/test_service.py`
@@ -550,6 +556,7 @@ Expected: all tests pass.
## Task 7: Update Docs to State the Rule
**Files:**
- Modify: `docs/workflow_capabilities.md`
- Create or modify: `docs/structural_refs.md`
@@ -568,15 +575,15 @@ Old strings are accepted at API boundaries only for compatibility.
Include examples for:
```json
{"source": "demo", "capability_key": "foo.bar"}
{ "source": "demo", "capability_key": "foo.bar" }
```
```json
{"artifact_id": "echo_wrapper", "version": 1}
{ "artifact_id": "echo_wrapper", "version": 1 }
```
```json
{"logical_source": "demo", "concrete_source": "demo.personal"}
{ "logical_source": "demo", "concrete_source": "demo.personal" }
```
- [ ] **Step 3: Mention path refs are separate**
@@ -593,6 +600,7 @@ state.person.name should migrate separately to path models.
## Task 8: Verification
**Files:**
- All touched files.
- [ ] **Step 1: Run focused tests**
@@ -16,12 +16,8 @@ Canonical node bindings already exist:
```json
{
"input": [
{"path": "input.message", "target": "message"}
],
"output": [
{"source": "echoed", "target": "state.echoed"}
]
"input": [{ "path": "input.message", "target": "message" }],
"output": [{ "source": "echoed", "target": "state.echoed" }]
}
```
@@ -38,25 +34,25 @@ The remaining problem is serialization. These path objects currently dump as str
Graph source paths:
```json
{"root": "state", "parts": ["person", "name"]}
{ "root": "state", "parts": ["person", "name"] }
```
State write paths:
```json
{"root": "state", "parts": ["person", "name"]}
{ "root": "state", "parts": ["person", "name"] }
```
Local node paths:
```json
{"root": "local", "parts": ["payload", "text"]}
{ "root": "local", "parts": ["payload", "text"] }
```
Local root remains explicit:
```json
{"root": "local", "parts": []}
{ "root": "local", "parts": [] }
```
Old strings such as `"state.person.name"` and `"."` remain accepted input.
@@ -66,6 +62,7 @@ Old strings such as `"state.person.name"` and `"."` remain accepted input.
## Task 1: Add Structural Serialization for Path Types
**Files:**
- Modify: `src/wf_core/paths.py`
- Test: `tests/core/test_path_values.py`
@@ -131,6 +128,7 @@ Expected: all tests pass.
## Task 2: Update Canonical Node Binding Dumps
**Files:**
- Test: `tests/core/test_canonical_node_bindings.py`
- Test: `tests/authoring/test_builder.py`
@@ -166,6 +164,7 @@ Expected: all tests pass.
## Task 3: Update Docs
**Files:**
- Modify: `docs/structural_refs.md`
- Modify: any path/core docs if directly relevant.
@@ -54,6 +54,7 @@ Important distinction:
### Task 1: Add Focused Barrier Same-Path Tests
**Files:**
- Modify: `tests/core/test_atomic_state_patches.py`
- [ ] **Step 1: Add imports**
@@ -188,6 +189,7 @@ The reducer test may already pass.
### Task 2: Add Ancestor/Descendant Conflict Tests
**Files:**
- Modify: `tests/core/test_atomic_state_patches.py`
- [ ] **Step 1: Add ancestor/descendant conflict test**
@@ -249,6 +251,7 @@ FAILED because current barrier replays both writes
### Task 3: Implement Barrier Write Analysis
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- [ ] **Step 1: Add helper dataclass**
@@ -389,6 +392,7 @@ Expected: pass.
### Task 4: Add End-To-End Concurrent Foreach Coverage
**Files:**
- Modify: `tests/core/test_concurrent_foreach.py`
- [ ] **Step 1: Add same-path no-reducer workflow helper**
@@ -501,6 +505,7 @@ Expected: pass.
### Task 5: Update Docs
**Files:**
- Modify: `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`
- Modify: `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`
@@ -545,6 +550,7 @@ Expected: the ADR and roadmap both mention the semantics.
### Task 6: Verification
**Files:**
- No new source files.
- [ ] **Step 1: Run focused core tests**
@@ -51,6 +51,7 @@ Those are Slice 3 write semantics. Do not add broad write-conflict policy here e
### Task 1: Add Failing Multi-Step Overlay Tests
**Files:**
- Modify: `tests/core/test_concurrent_foreach.py`
- [ ] **Step 1: Add a two-node item body test**
@@ -250,6 +251,7 @@ FAILED because state.scratch is missing/stale
### Task 2: Accumulate Per-Item Patches
**Files:**
- Modify: `src/wf_core/runtime/foreach_state.py`
- Modify: `tests/core/test_foreach_barrier_state.py`
@@ -358,6 +360,7 @@ Expected: pass.
### Task 3: Build Item-Local State Views
**Files:**
- Modify: `src/wf_core/runtime/ops/overlays.py`
- Test: `tests/core/test_concurrent_foreach.py`
@@ -422,6 +425,7 @@ Expected: if the single-node guard is still present, failure remains the guard.
### Task 4: Build Output Patches Against Frame State View
**Files:**
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_concurrent_foreach.py`
@@ -486,6 +490,7 @@ Expected: still fails until the single-node guard is removed.
### Task 5: Lift The Single-Node Concurrent Body Restriction
**Files:**
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Modify: `tests/core/test_concurrent_foreach.py`
@@ -544,6 +549,7 @@ Expected: pass.
### Task 6: Document Overlay Semantics
**Files:**
- Modify: `docs/adr/0002-concurrent-foreach-policy-and-barrier-commits.md`
- Modify: `docs/superpowers/plans/2026-05-22-concurrent-foreach-phase4-roadmap.md`
@@ -588,6 +594,7 @@ Expected: no stale claims except historical plan text in the already-completed V
### Task 7: Verification
**Files:**
- No new files unless tests require helper extraction.
- [ ] **Step 1: Run focused core tests**
@@ -34,6 +34,7 @@
**Goal:** Add the future policy shape while keeping runtime behavior serial-only.
**Files:**
- Modify: `src/wf_core/models/steps.py`
- Modify: `src/wf_core/validation/outcomes.py`
- Modify: `src/wf_core/validation/steps.py`
@@ -233,6 +234,7 @@ Expected: all pass.
**Goal:** Split current node output writes into reusable “build patch” and “commit patch” operations without changing current serial behavior.
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_atomic_state_patches.py`
@@ -272,6 +274,7 @@ def build_output_patch(
```
This function should:
- validate source paths
- validate destination paths
- calculate reducer-aware changes
@@ -338,6 +341,7 @@ Expected: all pass; full suite should still pass before moving on.
**Goal:** Add resumable barrier metadata and pending result structures without enabling concurrent execution.
**Files:**
- Modify: `src/wf_core/runtime/scheduler.py`
- Create: `src/wf_core/runtime/foreach_state.py`
- Modify: `src/wf_core/runtime/ops/foreach.py`
@@ -446,6 +450,7 @@ Expected: pass.
**Goal:** Enable `foreach(mode="concurrent")` using policy limits, pending results, and barrier commits. Sync runtime interleaves admitted item frames one node call at a time; async runtime may run admitted async node handler calls simultaneously.
**Files:**
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Modify: `src/wf_core/runtime/step.py`
- Modify: `src/wf_core/runtime/engine.py`
@@ -533,6 +538,7 @@ return step_foreach_concurrent(...)
```
`step_foreach_concurrent` should:
- inspect `ForeachBarrierState`
- start children while `active < max_active` and `outstanding < max_outstanding`
- block parent when waiting for children
@@ -38,6 +38,7 @@
### Task 1: Add Failing Sync Concurrent Foreach Tests
**Files:**
- Create: `tests/core/test_concurrent_foreach.py`
- [ ] **Step 1: Add the test file**
@@ -267,6 +268,7 @@ FAILED with message containing "concurrent foreach execution is not implemented
### Task 2: Add Barrier Admission Helpers
**Files:**
- Modify: `src/wf_core/runtime/foreach_state.py`
- Test: `tests/core/test_foreach_barrier_state.py`
@@ -345,6 +347,7 @@ Expected: pass.
### Task 3: Wake Foreach Parent After Each Child Completion
**Files:**
- Modify: `src/wf_core/runtime/scheduler.py`
- Test: `tests/core/test_scheduler.py`
@@ -442,6 +445,7 @@ Expected: pass.
### Task 4: Buffer Node Writes Inside Concurrent Foreach Item Frames
**Files:**
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Modify: `src/wf_core/runtime/foreach_state.py`
- Test: `tests/core/test_concurrent_foreach.py`
@@ -540,6 +544,7 @@ Expected:
### Task 5: Split Serial and Concurrent Foreach Runtime
**Files:**
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Modify: `src/wf_core/runtime/step.py`
- Test: `tests/core/test_concurrent_foreach.py`
@@ -696,6 +701,7 @@ Expected: pass.
### Task 6: Admit Concurrent Child Frames
**Files:**
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Test: `tests/core/test_concurrent_foreach.py`
@@ -837,6 +843,7 @@ Expected: failure after first admitted children complete, likely deadlock or no
### Task 7: Refill and Finish Concurrent Foreach
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Modify: `src/wf_core/runtime/ops/flow.py` or current child-completion caller if needed
@@ -1004,6 +1011,7 @@ Expected: the first two tests pass; runtime-error test may still need failure pr
### Task 8: Preserve Runtime Failure Semantics
**Files:**
- Modify: `src/wf_core/runtime/engine.py`
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Test: `tests/core/test_concurrent_foreach.py`
@@ -1059,6 +1067,7 @@ Expected: pass.
### Task 9: Regression and Verification
**Files:**
- Modify: docs only if implementation differs from plan.
- [ ] **Step 1: Run focused core tests**
@@ -44,6 +44,7 @@
### Task 1: Add RunState Scheduler Fields
**Files:**
- Modify: `src/wf_core/run_state.py`
- Test: `tests/core/test_scheduler.py`
@@ -120,6 +121,7 @@ Expected: pass.
### Task 2: Create Internal Scheduler Helpers
**Files:**
- Create: `src/wf_core/runtime/scheduler.py`
- Test: `tests/core/test_scheduler.py`
@@ -360,6 +362,7 @@ Expected: pass.
### Task 3: Initialize Root Through Scheduler
**Files:**
- Modify: `src/wf_core/runtime/ops/runs.py`
- Test: `tests/core/test_scheduler.py`
@@ -421,6 +424,7 @@ Expected: pass.
### Task 4: Re-Enqueue Normal Frame Advances
**Files:**
- Modify: `src/wf_core/runtime/ops/flow.py`
- Modify: `src/wf_core/runtime/step.py`
- Test: `tests/core/test_scheduler.py`
@@ -493,6 +497,7 @@ Expected: pass.
### Task 5: Migrate Engine Loops To Scheduler Selection
**Files:**
- Modify: `src/wf_core/runtime/engine.py`
- Modify: `src/wf_core/runtime/preparation.py`
- Modify: `src/wf_core/runtime/step.py`
@@ -546,6 +551,7 @@ Expected: pass.
### Task 6: Make Serial Foreach Use Block/Wake
**Files:**
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Modify: `src/wf_core/runtime/ops/frames.py`
- Create or extend: `tests/core/test_scheduler.py`
@@ -647,6 +653,7 @@ Expected: pass.
### Task 7: Resume Interrupt Through Ready Queue
**Files:**
- Modify: `src/wf_core/runtime/preparation.py`
- Modify: `src/wf_core/runtime/ops/interrupts.py`
- Test: existing interrupt tests or new focused tests
@@ -696,6 +703,7 @@ Expected: pass.
### Task 8: Full Verification
**Files:**
- Potentially update docs if implementation differs from ADR.
- [ ] **Step 1: Run focused workflow tests**
@@ -10,6 +10,31 @@
---
## Current Implementation Status
This plan is being implemented incrementally. The full `RuntimeScope` /
`LineageState` storage model below is still future work, but the runtime now has
the compatibility subset needed before native subgraphs:
- `StateWrite` exists and records `incoming_value` for replay plus
`visible_value` for same-lineage reads.
- `StatePatch` stores ordered `writes` while preserving `changes` as the
trace/compatibility view.
- `LineageStateView` materializes committed state plus visible lineage writes.
- Concurrent foreach item overlays read `StateWrite.visible_value`.
- Foreach pending result metadata persists write records and `lineage_id`.
- `ExecutionFrame` and `RuntimeContext` carry `scope_id`, `lineage_id`, and
`parent_lineage_id`.
- Concurrent foreach child frames receive deterministic, opaque lineage ids,
including nested foreach frames.
Remaining work should avoid jumping straight to native subgraphs. The next
small slice is to centralize "which writes are visible to this frame" behind a
helper, then later decide whether to add full `RunState.scopes` /
`RunState.lineages`.
---
## File Structure
- Modify: `src/wf_core/run_state.py`
@@ -102,7 +127,11 @@ flattened final values.
## Task 1: Add Ordered StateWrite Records to StatePatch
Status: implemented as the compatibility shape. `StatePatch.changes` remains a
stored compatibility dict rather than a derived-only property for now.
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- Test: `tests/core/test_atomic_state_patches.py`
@@ -254,7 +283,13 @@ Expected: pass.
## Task 2: Add Runtime Scopes and Root Lineage
Status: partially implemented. Frames and runtime context carry `scope_id`,
`lineage_id`, and `parent_lineage_id`, but `RunState.scopes`,
`RunState.lineages`, `RuntimeScope`, and `LineageState` are not implemented yet.
This is deliberate; foreach still stores pending writes in barrier metadata.
**Files:**
- Modify: `src/wf_core/run_state.py`
- Modify: `src/wf_core/runtime/ops/runs.py`
- Test: `tests/core/test_lineage_state.py`
@@ -338,7 +373,12 @@ Expected: pass.
## Task 3: Add Lineage Runtime Helpers
Status: partially implemented. `LineageStateView` exists in
`src/wf_core/runtime/ops/overlays.py`. The next incremental helper should be
`lineage_writes_for_frame(run, frame)`, backed by current foreach metadata.
**Files:**
- Create: `src/wf_core/runtime/lineage.py`
- Test: `tests/core/test_lineage_state.py`
@@ -448,6 +488,7 @@ Expected: pass.
## Task 4: Route Node Reads and Non-Root Writes Through Lineage
**Files:**
- Modify: `src/wf_core/runtime/ops/overlays.py`
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_lineage_state.py`
@@ -508,7 +549,13 @@ Expected: pass.
## Task 5: Migrate Concurrent Foreach to Lineages
Status: partially implemented. Concurrent foreach child frames now have lineage
ids, nested item lineages are tested, and pending item results persist
`lineage_id`. Patch ownership still lives in `ForeachBarrierState`, not in a
global lineage store.
**Files:**
- Modify: `src/wf_core/runtime/ops/foreach.py`
- Modify: `src/wf_core/runtime/foreach_state.py`
- Test: `tests/core/test_concurrent_foreach.py`
@@ -585,6 +632,7 @@ Expected: pass.
## Task 6: Remove Foreach-Specific Overlay Coupling
**Files:**
- Modify: `src/wf_core/runtime/ops/overlays.py`
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core`
@@ -609,6 +657,7 @@ Expected: pass.
## Task 7: Update Docs
**Files:**
- Modify: `docs/wf_core_architecture.md`
- Modify: `docs/current_roadmap.md`
- Modify: `docs/superpowers/specs/2026-05-24-native-subgraphs-design.md`
@@ -114,12 +114,10 @@ Examples:
```json
{
"input": [
{"target": "user.email", "path": "state.person.email"},
{"target": "mode", "value": "fast"}
{ "target": "user.email", "path": "state.person.email" },
{ "target": "mode", "value": "fast" }
],
"output": [
{"source": "result", "target": "state.result"}
]
"output": [{ "source": "result", "target": "state.result" }]
}
```
@@ -127,9 +125,7 @@ Whole payload input:
```json
{
"input": [
{"target": ".", "path": "state.rates"}
]
"input": [{ "target": ".", "path": "state.rates" }]
}
```
@@ -137,9 +133,7 @@ Whole payload literal input:
```json
{
"input": [
{"target": ".", "value": {"mode": "fast"}}
]
"input": [{ "target": ".", "value": { "mode": "fast" } }]
}
```
@@ -177,7 +171,7 @@ Old dict-shaped fields can be accepted at parse time and normalized:
```json
{
"fields": {
"person.tags": {"type": "array", "reducer": "wf.std.append"}
"person.tags": { "type": "array", "reducer": "wf.std.append" }
}
}
```
@@ -1,6 +1,6 @@
# Lineage State Runtime Design
Status: proposed
Status: partially implemented
Lineage is the missing primitive between the scheduler frame model and future
native subgraphs / fork-gather. A frame says where execution is. A scope says
@@ -12,6 +12,25 @@ foreach-specific barrier metadata to emulate item-local overlays. That worked
for concurrent foreach, but native subgraphs and future fork/gather need the
same state-visibility rule in a reusable core concept.
## Current Implementation Status
The first compatibility slices are implemented:
- `StateWrite` records reducer-aware `incoming_value` and `visible_value`.
- `StatePatch` preserves ordered writes while keeping `changes` for trace and
compatibility.
- `LineageStateView` materializes committed state plus lineage-visible writes.
- Concurrent foreach item reads use `visible_value`, while barriers replay
`incoming_value`.
- Foreach pending result metadata persists write records and `lineage_id`.
- Frames and runtime context carry `scope_id`, `lineage_id`, and
`parent_lineage_id`.
The full `RuntimeScope` / `LineageState` store is not implemented yet.
Currently, foreach still owns pending write storage through
`ForeachBarrierState`; the lineage ids are identity and diagnostics, not yet the
primary storage key.
## Problem
`RunState` currently owns too many meanings:
+5 -1
View File
@@ -311,7 +311,11 @@ contracts:
"node_spec_count": 12,
"reducer_count": 6,
"preview": {
"node_specs": ["wf.std.coalesce", "wf.std.constant", "wf.std.default_if_none"],
"node_specs": [
"wf.std.coalesce",
"wf.std.constant",
"wf.std.default_if_none"
],
"reducers": ["wf.std.add", "wf.std.append", "wf.std.max"]
},
"has_more": {
+4 -4
View File
@@ -515,14 +515,14 @@ Minimal example:
},
"input": [
{
"target": {"root": "local", "parts": ["text"]},
"path": {"root": "input", "parts": ["text"]}
"target": { "root": "local", "parts": ["text"] },
"path": { "root": "input", "parts": ["text"] }
}
],
"output": [
{
"source": {"root": "local", "parts": ["echoed"]},
"target": {"root": "state", "parts": ["echoed"]}
"source": { "root": "local", "parts": ["echoed"] },
"target": { "root": "state", "parts": ["echoed"] }
}
]
}
+37 -37
View File
@@ -71,14 +71,14 @@ A minimal draft looks like this:
"use": "demo.personal.echo_tool",
"input": [
{
"target": {"root": "local", "parts": ["text"]},
"path": {"root": "input", "parts": ["text"]}
"target": { "root": "local", "parts": ["text"] },
"path": { "root": "input", "parts": ["text"] }
}
],
"output": [
{
"source": {"root": "local", "parts": ["echoed"]},
"target": {"root": "state", "parts": ["echoed"]}
"source": { "root": "local", "parts": ["echoed"] },
"target": { "root": "state", "parts": ["echoed"] }
}
]
}
@@ -109,18 +109,18 @@ Draft `use` steps use the same canonical binding structs as core `NodeUse`:
{
"input": [
{
"target": {"root": "local", "parts": ["message"]},
"path": {"root": "input", "parts": ["text"]}
"target": { "root": "local", "parts": ["message"] },
"path": { "root": "input", "parts": ["text"] }
},
{
"target": {"root": "local", "parts": ["limit"]},
"target": { "root": "local", "parts": ["limit"] },
"value": 3
}
],
"output": [
{
"source": {"root": "local", "parts": ["echoed"]},
"target": {"root": "state", "parts": ["echoed"]}
"source": { "root": "local", "parts": ["echoed"] },
"target": { "root": "state", "parts": ["echoed"] }
}
]
}
@@ -139,13 +139,13 @@ not put `"user.name"` in one segment unless the actual JSON property name is
literally `user.name`. For normal nested objects, write:
```json
{"root": "input", "parts": ["user", "name"]}
{ "root": "input", "parts": ["user", "name"] }
```
not:
```json
{"root": "input", "parts": ["user.name"]}
{ "root": "input", "parts": ["user.name"] }
```
For example, this canonical input/output pair:
@@ -154,22 +154,22 @@ For example, this canonical input/output pair:
{
"input": [
{
"target": {"root": "local", "parts": ["user", "name"]},
"path": {"root": "input", "parts": ["user", "name"]}
"target": { "root": "local", "parts": ["user", "name"] },
"path": { "root": "input", "parts": ["user", "name"] }
},
{
"target": {"root": "local", "parts": ["job", "title"]},
"path": {"root": "state", "parts": ["job", "title"]}
"target": { "root": "local", "parts": ["job", "title"] },
"path": { "root": "state", "parts": ["job", "title"] }
}
],
"output": [
{
"source": {"root": "local", "parts": ["user", "age"]},
"target": {"root": "state", "parts": ["person", "age"]}
"source": { "root": "local", "parts": ["user", "age"] },
"target": { "root": "state", "parts": ["person", "age"] }
},
{
"source": {"root": "local", "parts": ["job", "years"]},
"target": {"root": "state", "parts": ["experience", "years"]}
"source": { "root": "local", "parts": ["job", "years"] },
"target": { "root": "state", "parts": ["experience", "years"] }
}
]
}
@@ -188,8 +188,8 @@ Do not reverse the direction. This is wrong:
{
"input": [
{
"target": {"root": "input", "parts": ["text"]},
"path": {"root": "local", "parts": ["message"]}
"target": { "root": "input", "parts": ["text"] },
"path": { "root": "local", "parts": ["message"] }
}
]
}
@@ -204,8 +204,8 @@ Do not put constants in path bindings. This is wrong:
{
"input": [
{
"target": {"root": "local", "parts": ["value"]},
"path": {"root": "input", "parts": ["CLICKED"]}
"target": { "root": "local", "parts": ["value"] },
"path": { "root": "input", "parts": ["CLICKED"] }
}
]
}
@@ -224,14 +224,14 @@ Calls a workflow capability.
"use": "demo.personal.echo_tool",
"input": [
{
"target": {"root": "local", "parts": ["text"]},
"path": {"root": "input", "parts": ["text"]}
"target": { "root": "local", "parts": ["text"] },
"path": { "root": "input", "parts": ["text"] }
}
],
"output": [
{
"source": {"root": "local", "parts": ["echoed"]},
"target": {"root": "state", "parts": ["echoed"]}
"source": { "root": "local", "parts": ["echoed"] },
"target": { "root": "state", "parts": ["echoed"] }
}
]
}
@@ -249,14 +249,14 @@ are part of the graph definition:
"use": "wf.std.constant",
"input": [
{
"target": {"root": "local", "parts": ["value"]},
"target": { "root": "local", "parts": ["value"] },
"value": "CLICKED"
}
],
"output": [
{
"source": {"root": "local", "parts": ["value"]},
"target": {"root": "state", "parts": ["wait_text"]}
"source": { "root": "local", "parts": ["value"] },
"target": { "root": "state", "parts": ["wait_text"] }
}
]
}
@@ -292,7 +292,7 @@ model: use `item_error` and `concurrent`, not draft-only field names.
```json
{
"foreach": {
"over": {"root": "state", "parts": ["items"]},
"over": { "root": "state", "parts": ["items"] },
"as": "item",
"mode": "serial",
"item_error": "fail"
@@ -305,7 +305,7 @@ Concurrent foreach uses the same canonical policy shape as core:
```json
{
"foreach": {
"over": {"root": "state", "parts": ["items"]},
"over": { "root": "state", "parts": ["items"] },
"as": "item",
"mode": "concurrent",
"concurrent": {
@@ -314,7 +314,7 @@ Concurrent foreach uses the same canonical policy shape as core:
},
"item_error": {
"action": "collect",
"collect_to": {"root": "state", "parts": ["item_errors"]}
"collect_to": { "root": "state", "parts": ["item_errors"] }
}
}
}
@@ -335,14 +335,14 @@ Declares an interrupting step.
"kind": "input",
"request": [
{
"target": {"root": "local", "parts": ["question"]},
"path": {"root": "state", "parts": ["question"]}
"target": { "root": "local", "parts": ["question"] },
"path": { "root": "state", "parts": ["question"] }
}
],
"resume": [
{
"source": {"root": "local", "parts": ["answer"]},
"target": {"root": "state", "parts": ["answer"]}
"source": { "root": "local", "parts": ["answer"] },
"target": { "root": "state", "parts": ["answer"] }
}
],
"outcomes": ["resumed", "cancelled"]
+4 -1
View File
@@ -98,7 +98,10 @@ def build_parent_workflow() -> WorkflowBuilder:
def run_parent_workflow() -> RunState:
"""Run the parent workflow around the wrapped child workflow."""
return build_parent_workflow().execute(
{"folder_id": "demo-folder", "should_email": False}
{
"folder_id": "demo-folder",
"should_email": False,
}
)
+2 -9
View File
@@ -51,15 +51,8 @@
],
"tools_generated": true,
"prompts_generated": true,
"keywords": [
"mcp",
"workflow",
"proxy",
"local-dev"
],
"keywords": ["mcp", "workflow", "proxy", "local-dev"],
"compatibility": {
"platforms": [
"win32"
]
"platforms": ["win32"]
}
}
+3
View File
@@ -30,3 +30,6 @@ package = true
[tool.basedpyright]
typeCheckingMode = "basic"
[tool.ruff.format]
preview = false
+4 -4
View File
@@ -133,14 +133,14 @@ Path bindings use structural paths in saved JSON:
{
"input": [
{
"target": {"root": "local", "parts": ["text"]},
"path": {"root": "input", "parts": ["text"]}
"target": { "root": "local", "parts": ["text"] },
"path": { "root": "input", "parts": ["text"] }
}
],
"output": [
{
"source": {"root": "local", "parts": ["echoed"]},
"target": {"root": "state", "parts": ["echoed"]}
"source": { "root": "local", "parts": ["echoed"] },
"target": { "root": "state", "parts": ["echoed"] }
}
]
}
+20 -6
View File
@@ -43,18 +43,32 @@ def state_view_for_frame(run: RunState, frame: ExecutionFrame) -> dict[str, Any]
foreach barrier commits. Later nodes in the same item must read those
earlier writes, while sibling item frames must not see them.
"""
owner = item_frame_owner(frame)
if owner is None:
writes = lineage_writes_for_frame(run, frame)
if not writes:
return run.state
return LineageStateView(run.state, writes).to_state_dict()
def lineage_writes_for_frame(
run: RunState, frame: ExecutionFrame
) -> Sequence[StateWrite]:
"""Return writes visible to this frame's current lineage.
This is still backed by concurrent foreach barrier metadata. Keeping the
lookup here gives future `RunState.lineages` or subgraph scopes one place to
plug in without making node execution understand foreach internals.
"""
owner = item_frame_owner(frame)
if owner is None:
return ()
parent_frame_id, foreach_node_id, item_index = owner
parent_frame = run.frames[parent_frame_id]
barrier = ForeachBarrierState.from_frame(parent_frame, foreach_node_id)
if barrier is None or barrier.mode != "concurrent":
return run.state
return ()
pending = barrier.pending_results.get(item_index)
if pending is None:
return run.state
return LineageStateView(run.state, pending.patch.writes).to_state_dict()
return ()
return pending.patch.writes
+15 -3
View File
@@ -482,7 +482,11 @@ def _same_item_reducer_visibility_workflow() -> Workflow:
edges=[
Edge.model_validate({"from": "each", "outcome": "loop", "to": "add_item"}),
Edge.model_validate(
{"from": "add_item", "outcome": "ok", "to": "read_number"}
{
"from": "add_item",
"outcome": "ok",
"to": "read_number",
}
),
Edge.model_validate({"from": "read_number", "outcome": "ok", "to": END}),
Edge.model_validate({"from": "each", "outcome": "done", "to": END}),
@@ -563,10 +567,18 @@ def _nested_foreach_lineage_workflow() -> Workflow:
],
edges=[
Edge.model_validate(
{"from": "outer_each", "outcome": "loop", "to": "inner_each"}
{
"from": "outer_each",
"outcome": "loop",
"to": "inner_each",
}
),
Edge.model_validate(
{"from": "inner_each", "outcome": "loop", "to": "record"}
{
"from": "inner_each",
"outcome": "loop",
"to": "record",
}
),
Edge.model_validate({"from": "record", "outcome": "ok", "to": END}),
Edge.model_validate({"from": "inner_each", "outcome": "done", "to": END}),
+56 -2
View File
@@ -5,13 +5,13 @@ import pytest
from wf_core.errors import WorkflowExecutionError
from wf_core.models.reducers import ReducerRef
from wf_core.paths import StatePath
from wf_core.run_state import ExecutionFrame, StateWrite
from wf_core.run_state import ExecutionFrame, RunState, RunStatus, StateWrite
from wf_core.runtime.foreach_state import (
ForeachBarrierState,
ItemErrorRecord,
PendingItemResult,
)
from wf_core.runtime.ops.overlays import LineageStateView
from wf_core.runtime.ops.overlays import LineageStateView, lineage_writes_for_frame
from wf_core.runtime.ops.state import StatePatch
@@ -116,6 +116,60 @@ def test_lineage_state_view_materializes_visible_values_without_mutating_base()
assert base_state["nested"]["value"] == "old"
def test_lineage_writes_for_frame_reads_current_foreach_pending_result() -> None:
parent = ExecutionFrame(id="root", kind="workflow", node_id="each")
child = ExecutionFrame(
id="root:each:0",
kind="foreach_iteration",
node_id="work",
parent_frame_id="root",
lineage_id="root/each[0]",
parent_lineage_id="root",
metadata={
"foreach_node_id": "each",
"loop_index": 0,
"loop_item": "a",
"loop_alias": "item",
},
)
patch = StatePatch(
writes=[
StateWrite(
path=StatePath(("count",)),
incoming_value=3,
visible_value=5,
reducer=ReducerRef(name="wf.std.add"),
)
]
)
barrier = ForeachBarrierState(
mode="concurrent",
pending_results={
0: PendingItemResult(
index=0,
frame_id=child.id,
status="succeeded",
lineage_id=child.lineage_id,
patch=patch,
)
},
)
barrier.save_to_frame(parent, "each")
run = RunState(
workflow_name="lineage",
status=RunStatus.PENDING,
workflow_input={},
state={"count": 2},
frames={parent.id: parent, child.id: child},
)
writes = lineage_writes_for_frame(run, child)
assert len(writes) == 1
assert writes[0].incoming_value == 3
assert writes[0].visible_value == 5
def test_foreach_barrier_state_returns_none_when_missing() -> None:
frame = ExecutionFrame(id="root", kind="root", node_id="each")
+1 -1
View File
@@ -2,7 +2,7 @@
so you know im using ts and influencing its design. im not just an user (lowk i am)
[llgd]: <https://git.ldlda.com/lda/langgraph-demo>
[llgd]: https://git.ldlda.com/lda/langgraph-demo
## to agents
+1 -3
View File
@@ -9,9 +9,7 @@
"metadata": {
"transport": "stdio",
"command": "pnpx",
"args": [
"@modelcontextprotocol/server-everything"
]
"args": ["@modelcontextprotocol/server-everything"]
}
}
]