This commit is contained in:
lda
2026-05-20 14:35:17 +07:00 Verified
parent 4b9e3cb26b
commit a74e016f7b
3 changed files with 1827 additions and 0 deletions
+571
View File
@@ -0,0 +1,571 @@
# Path And Mapping Scratch
This is a focused scratchpad for the `wf_core` path/mapping design thread.
Clean this into real docs later.
## Current Mapping Roles
`in_map` maps graph/runtime paths into node-local input paths:
```text
input.text -> text
state.person.name -> user.name
context.item -> document
```
`out_map` maps node-local output paths into workflow state destinations:
```text
echoed -> state.echoed
user.age -> state.person.age
. -> state.rates
```
`input_values` maps node-local input paths to literal values:
```text
mode -> "fast"
retry.count -> 3
```
## Path Kinds
Graph paths live in workflow/run space:
- `input.*`: original workflow invocation input.
- `state.*`: mutable workflow state.
- `context.*`: runtime frame context.
Node-local paths live inside one node input/output payload:
- `user.name`
- `job.years`
- `.` for the whole local payload.
## Context Paths
`context.*` currently exists. It is generated from the current execution frame,
not stored in workflow state.
Current context fields include:
- `context.prior_outcome`
- `context.activated_incoming_edge`
- foreach frames: `context.loop_item`
- foreach frames: `context.loop_index`
- foreach frames: `context.<loop_alias>` when the foreach node declares `as`.
Context paths are valid graph source paths for `in_map` when validation allows
context. They are not valid `out_map` destinations.
## Typed Internal Model Idea
Keep serialized maps as strings for JSON compatibility, but parse them at
runtime/validation boundaries:
```text
GraphPath(root="input", parts=("text",)) <-> "input.text"
GraphPath(root="state", parts=("person", "name")) <-> "state.person.name"
GraphPath(root="context", parts=("item",)) <-> "context.item"
LocalPath(parts=("user", "name")) <-> "user.name"
LocalPath.root() <-> "."
```
This lets validation reason about roots and parts without repeated string
prefix checks.
## Decisions So Far
Canonical path parsing/validation should live in `wf_core`.
`wf_authoring` should keep ergonomic constructors and condition wrappers, but
those should be thin wrappers around core path types.
Condition path fields should use typed graph source paths, not arbitrary
strings:
```text
PathOperand.path: GraphSourcePath
ExistsCondition.path: GraphSourcePath
```
Node input mapping should merge the old `in_map` and `input_values` concepts
into one list of binding structs:
```text
InputPathBinding:
target: LocalPath
path: GraphSourcePath
InputValueBinding:
target: LocalPath
value: JsonValue
```
Do not add a `kind` discriminator if the shape can be distinguished by `path`
vs `value`. Use strict models so `{path, value}` and `{}` fail.
Node output mapping should become a list of binding structs:
```text
OutputBinding:
source: LocalPath
target: StatePath
```
The same field name can mean different path kinds by position:
- input binding `target` is a node-local input path
- output binding `target` is a workflow state path
Root local path `"."` remains valid:
- input target `"."` means the whole node input payload is the mapped path/value
- output source `"."` means the whole node output payload is written
- literal input value binding to `"."` is valid and means the literal value is
the entire node input payload
- `"."` input binding must still be the only input binding on that node use
Overlap rules:
- if an input binding targets `"."`, it must be the only input binding
- input targets must not overlap, e.g. `user` and `user.name`
- output targets must not overlap, e.g. `state.person` and
`state.person.name`
- exact duplicate input targets are invalid
- exact duplicate output targets are invalid
- if output writes a parent, it cannot also write a child
- read paths may overlap; write paths may not overlap
State path validation and write behavior:
- writable `StatePath` must have its root declared in `state_schema.fields`
- whole-state write targets such as bare `state` stay out of scope for now
- nested state subpaths are allowed once the root exists in the schema
- exact nested state declarations are reducer/schema hints, not root ownership
- reducers apply only to the exact declared destination path
- missing object parents are created during runtime writes
- descending through an existing non-object parent fails at runtime
Runtime traversal helpers:
- all get/set traversal should go through focused helpers
- do not scatter `dict`/`getattr` logic through runtime code
- helpers may support mappings, Pydantic models, and dataclasses where safe
- exact supported object kinds can be implementation-defined, but the behavior
should be centralized and tested
Read vs write traversal:
- reads may support richer object access through centralized helpers
- writes are stricter
- nested writes require mutable mapping parents
- typed model/dataclass parents are not patched field-by-field
- strict typed values should be replaced as a whole, not partially mutated
- full replacements must adhere to the declared schema when validation exists
State write validation:
- validate writes against the exact declared state field schema when one exists
- replacement writes must satisfy the full target schema
- replacement writes that contain fields not allowed by the target schema fail
- partial object patches require an explicit merge-style reducer such as
`merge_object`; do not silently treat replace as merge
- do not validate the whole workflow state on every write
- undeclared nested subpaths under declared object roots remain flexible
- this keeps validation focused and avoids heavy whole-state checks
Workflow input/output validation:
- validate full workflow input against `workflow.input_schema` at run start
- keep workflow output as top-level projection from state for now
- `workflow.output_schema.properties` decides which top-level state fields are
exposed as final output
- do not add workflow-level output bindings in this path refactor
- if shaped output is needed, use a final node to shape state before END
Execution validation order:
1. validate workflow input at run start
2. resolve node input bindings into node payload
3. validate node payload against `node_def.input_schema`
4. execute node
5. coerce node result
6. validate node result output against `node_def.output_schema`
7. apply output bindings to state with focused state field validation
8. project final workflow output from state at END
Principle:
- validate node output before mutating state
- strange node results should fail fast and not pollute workflow state
State patch atomicity:
- output binding application is one atomic patch to workflow state
- no gradual state mutation while processing individual bindings
- resolve all sources first
- check overlaps first
- compute reducer/merged values first
- validate patch values first
- commit the patch only after preparation succeeds
- failed output binding application leaves prior state unchanged
Reducers:
- reducers run during patch preparation, not during commit
- reducers receive current value, incoming value, and optional config
- reducers return the merged value for the patch
- reducers must not mutate workflow state directly
- reducer failure aborts the whole patch before commit
Patch representation:
- public mappings are list-of-structs
- internal prepared patches can be flat path-keyed maps
- key type should be `StatePath`, not raw string
- flat patches make overlap checks, reducer lookup, validation, tracing, and
commit performance simpler
- do not expose flat path-keyed patch maps as the public authoring shape
Trace state changes:
- use typed `StatePath` internally
- prefer list-of-structs for public trace serialization/schema
- target shape:
```text
StateChange:
path: StatePath
value: JsonValue
```
- this avoids custom JSON object key serialization problems
- it gives cleaner schema and clearer MCP/LLM output
Runtime state value domain:
- workflow definitions and static binding values should be JSON-compatible
- in-memory runtime state may remain `Any`-ish for now
- persisted/checkpointed run state should require JSON-compatible values
- checkpoint serialization should be the strict boundary for non-serializable
runtime objects
Current initialization behavior:
- workflow input is currently copied into initial state by `init_run_state`
- this makes `input.foo` and `state.foo` both available at run start when the
input has `foo`
- target direction is explicit initialization, closer to LangGraph: input stays
input, state is mutated only by explicit graph behavior
- keep implicit seeding for now as compatibility unless/until there is a
dedicated migration
Canonical node binding shape:
```text
NodeUse.input: list[InputBinding]
NodeUse.output: list[OutputBinding]
```
Old `in_map`, `input_values`, and `out_map` can be accepted as parse-only
compatibility inputs, but the canonical model should store and serialize the new
list-of-structs shape.
Naming caveat:
- `NodeUse.input` means "bindings that build this node's input payload"
- graph path root `input.*` means "the workflow run input"
Docs must make this distinction explicit. The repeated word is acceptable only
if examples clearly show `input` as the binding list and `input.foo` as a graph
source path.
Static value bindings:
- `InputValueBinding.value` should be JSON-compatible
- workflow models should stay serializable/storable
- non-serializable runtime objects such as callbacks should not be embedded in
workflow definitions
- future LangGraph-store-like behavior should use explicit store/reference
mechanisms, not arbitrary Python objects inside the model
Context paths:
- `context.*` stays a first-class graph source path
- it is read-only and source-only
- root-only graph source paths are allowed for whole-container reads:
`input`, `state`, and `context`
- root-only graph source paths are useful for explicitly passing whole workflow
input/state/context into a node
- conditions can use `context.*`
- node input path bindings can use `context.*`
- output/write bindings cannot target `context.*`
- workflow validation does not need to statically prove every context key exists
- `exists(context.x)` returns false when missing
- comparisons against missing context paths fail clearly
Validation responsibility split:
- path types validate that a value is a well-formed path of the right kind
- workflow validation checks whether that path is legal in a specific workflow
- examples of workflow-specific checks:
- `input.foo` root exists in `input_schema`
- `state.foo` root exists in `state_schema`
- write destinations are state paths
- write destinations do not overlap
- path types should not need access to workflow schemas
Path segment syntax:
- use strict identifier-like segments:
`[A-Za-z_][A-Za-z0-9_]*`
- dots separate path segments
- empty segments are invalid
- arbitrary JSON keys containing dots or punctuation are not supported yet
- bracket/index syntax is out of scope until deliberately designed
- this tightens current behavior, which mostly split strings without much
segment validation
List indexing:
- no list indexing in core paths
- paths address object/dict/model fields, not list positions
- numeric/positional list segments are rejected
- encoding workflow meaning by array position is discouraged
- use foreach for item-wise behavior
- future continue/break-style foreach behavior may cover many selection cases
- explicit helper nodes such as an authoring `index` node can handle positional
lookup without complicating core path syntax
Local path syntax:
- `LocalPath` uses the same strict segment rules as graph paths
- valid examples: `user.name`, `job.years`, `.`
- invalid examples: `user..name`, `user-name`, `items.0`, `user["name"]`
- no external path library is planned; a small parser with a compiled segment
regex is enough because these paths have workflow-specific roots and rules
Hashability:
- path objects should be immutable/hashable
- use frozen dataclasses or equivalent immutable models
- public node bindings should still be list-of-structs, not dicts keyed by paths
- hashable paths are useful internally for lookups, sets, duplicate detection,
and overlap validation
Implementation choice:
- use frozen dataclass value objects with Pydantic core-schema hooks
- avoid Pydantic `BaseModel` for tiny path values unless hooks become too costly
- Pydantic should accept strings or existing path objects and store path objects
- JSON serialization should emit strings
Planned core path value objects:
```text
LocalPath(parts)
GraphSourcePath(root, parts) # root is input | state | context
StatePath(parts) # serializes with state. prefix
```
Shared internals:
- use shared parsing/segment/overlap helpers for all path kinds
- do not expose one generic public `Path` for every position
- distinct public path types preserve semantics:
- `LocalPath` for node-local payloads
- `GraphSourcePath` for readable graph sources
- `StatePath` for writable state destinations
- a small shared `PathParts` value or shared helper functions are both fine;
choose whichever keeps implementation simplest
Constructors and authoring sugar:
- core path types should have boring constructors/parsers for tests and runtime
- `wf_authoring` keeps pretty helpers such as `state_path`, `input_path`,
`context_path`, and condition expression sugar
- `Expr` and `PathExpr` stay in `wf_authoring`
- core owns `Condition` models and path value types
- authoring wrappers can behave like a trait/mixin over comparable path-like
values, adding `eq`, `ne`, `lt`, `le`, `gt`, `ge`, operator overloads, and
`exists`
Core constructor ergonomics:
- constructors may accept dotted strings and/or multiple fragments
- fragments are flattened by splitting on `.`
- examples:
- `GraphSourcePath.state("person.name")`
- `GraphSourcePath.state("person", "name")`
- `GraphSourcePath.state("foo", "bar.baz")`
- all final segments still pass strict segment validation
- empty fragments/segments are rejected
- `LocalPath.root()` / serialized `"."` is the only root marker exception
Parse vs construction:
- `parse()` reads the full serialized JSON form
- `of()` / root-specific constructors are ergonomic Python construction
- both use the same segment parsing/validation internals
- examples:
- `StatePath.parse("state.person.name")`
- `StatePath.of("person.name")`
- `GraphSourcePath.state("person.name")`
- `LocalPath.of("user.name")`
String form:
- `str(path)` returns the serialized JSON form
- examples:
- `str(StatePath.of("person.name")) == "state.person.name"`
- `str(GraphSourcePath.state("person.name")) == "state.person.name"`
- `str(LocalPath.of("user.name")) == "user.name"`
- `str(LocalPath.root()) == "."`
JSON Schema:
- path fields should expose as strings, not `{root, parts}` objects
- use pattern and description metadata where possible
- examples:
- `StatePath`: string matching `state.<segment>(.<segment>)*`
- `GraphSourcePath`: string matching `(input|state|context).<segment>(.<segment>)*`
- `LocalPath`: string matching `.` or `<segment>(.<segment>)*`
- use Pydantic hook / annotation metadata magic to keep external schemas clear
while storing rich path value objects internally
Error messages:
- path parsing errors should name the expected path kind
- include examples in errors where practical
- expected examples:
- graph source path: `input.foo`, `state.foo`, or `context.foo`
- state path: `state.foo`
- local path: `user.name` or `.`
- this should improve current error + hint behavior for MCP/LLM users
Missing paths:
- binding path reads should fail when missing
- do not add `on_missing` to core bindings now
- optional/default/missing behavior belongs in explicit nodes or conditions
- this keeps bindings as data movement, not hidden behavior
Binding descriptions:
- no `description` / `desc` fields on `wf_core` binding structs for now
- descriptions are useful in higher layers such as drafts, artifacts, authoring
helpers, or MCP-facing planning surfaces
- core bindings should stay structural/runtime-focused
Binding order:
- binding order is preserved
- runtime may apply bindings in order for deterministic traces/debugging
- order must not resolve conflicts
- overlapping write targets are validation errors, not "last write wins"
- future priority/override behavior must be explicit, not implicit list order
Core explicitness:
- empty `NodeUse.input` means empty node payload
- empty `NodeUse.output` means no state writes
- core performs no auto-mapping
- authoring/builder layers may infer mappings, but must emit explicit canonical
bindings into core models
State schema fields:
- move toward list-of-structs instead of dict keys
- canonical shape:
```text
StateSchema.fields: list[StateFieldDecl]
StateFieldDecl:
path: StatePath
type: string
reducer: ReducerRef
```
- serialized field paths include `state.` prefix, e.g. `state.person.tags`
- accept old dict shape at parse time for compatibility:
```text
fields = {
"person.tags": {"type": "array", "reducer": "wf.std.append"}
}
```
- normalize old shape to canonical list internally
- canonical serialization emits list shape
- duplicate field paths are validation errors
- exact reducer matching uses exact `StatePath`
Input/output JSON Schemas:
- do not replace JSON Schema with a custom schema language
- introduce a named JSON Schema boundary type or model
- validate schemas with the standard `jsonschema` library
- validation cost is acceptable at model/config boundaries
- this applies to workflow input/output schemas and node input/output schemas
- JSON Schema object keys are standard and should not be treated like workflow
path-map dict keys
- respect `$schema` when present using `jsonschema.validators.validator_for`
- default to Draft 2020-12 when `$schema` is absent
- keep `SchemaRef` as the core schema type for now
- make `SchemaRef` honest: it represents a JSON Schema object boundary
- strengthen `SchemaRef` with standard JSON Schema validation instead of
introducing a parallel `JsonSchema` type
Placement:
- canonical path value objects live in `wf_core.paths`
- `wf_core.models.*`, validation, runtime, and authoring import those types
- split `wf_core.paths` into a package later only if it grows too large
Compatibility fields:
- old `in_map`, `input_values`, and `out_map` are parse-only compatibility
inputs
- they should be treated as deprecated
- validated `NodeUse` stores only canonical `input` and `output` bindings
- canonical serialization emits only the new fields
- avoid dual state inside the model
- reject payloads that mix canonical fields with deprecated compatibility fields
- do not merge old and new syntax
- compatibility conversion preserves dict insertion order when converting old
maps into binding lists
- JSON Schema should advertise only canonical `input` / `output` fields
- deprecated compatibility fields should not be shown to new callers
Null and missing semantics:
- explicit null is a real value, not an omitted binding/value
- `exists(path)` means the path can be resolved, even when its value is null
- missing path and present-null path are different states
- compare against null explicitly when needed, e.g. `state.foo == null` or
`state.foo != null`
- null comparisons are valid anywhere normal conditions work
- missing paths still fail clearly for comparisons; only `exists(path)` treats
missing as false instead of an error
- input value bindings may intentionally bind null; this must not be treated as
"no value provided"
- input path bindings fail before node execution when the source path is
missing
- missing source paths must not silently bind null
- softer/defaulting behavior belongs in authoring helpers or explicit shaping
nodes, not implicit core behavior
Schema openness:
- JSON Schema / Pydantic-style `extra` controls what object keys are allowed
inside a value
- schema openness does not change path existence semantics
- core should not use "allow extra" as permission for speculative deep path
traversal
- if a workflow needs dynamic traversal through arbitrary/extra object shape,
use an explicit node that receives the relevant state value and decides what
to extract
- this avoids pretending static path validation can prove paths such as
`state.person.occupations.1.title` exist inside open-ended objects
@@ -0,0 +1,925 @@
# Core Path Bindings Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace loose core path/map strings with typed path objects and canonical list-of-struct node bindings while keeping deprecated shapes parse-compatible.
**Architecture:** Add immutable path value objects in `wf_core.paths`, then introduce canonical binding models in `wf_core.models.steps`. Runtime and validation move to the canonical bindings, while old `in_map`, `input_values`, `out_map`, and dict-shaped state fields are accepted only by model validators.
**Tech Stack:** Python 3.14, Pydantic v2, pytest, jsonschema, basedpyright, ruff.
---
## File Structure
- Modify `src/wf_core/paths.py`: own typed graph/state/local path objects and graph path resolution helpers.
- Modify `src/wf_core/local_paths.py`: keep compatibility wrappers over `LocalPath` plus local get/set helpers.
- Modify `src/wf_core/models/steps.py`: add `InputPathBinding`, `InputValueBinding`, `OutputBinding`, and canonical `NodeUse.input` / `NodeUse.output`.
- Modify `src/wf_core/models/conditions.py`: type condition path operands with `GraphSourcePath`.
- Modify `src/wf_core/models/schemas.py`: harden `SchemaRef` and add canonical state field declarations.
- Modify `src/wf_core/runtime/ops/nodes.py`: resolve canonical node input bindings.
- Modify `src/wf_core/runtime/ops/state.py`: apply canonical output bindings through an atomic state patch.
- Modify `src/wf_core/runtime/ops/schemas.py`: expose focused JSON Schema validation helpers.
- Modify `src/wf_core/validation/steps.py`: validate canonical bindings and typed paths.
- Modify `src/wf_authoring/dsl/paths.py`: emit core path objects while preserving ergonomic helpers.
- Modify `src/wf_authoring/dsl/conditions.py`: compile authoring expressions to core typed condition models.
- Add `tests/core/test_path_values.py`: path parsing, serialization, JSON Schema, and error tests.
- Add `tests/core/test_canonical_node_bindings.py`: canonical model parsing and deprecated compatibility tests.
- Add `tests/core/test_atomic_state_patches.py`: output binding, reducer, overlap, and atomicity tests.
- Update existing `tests/core/test_mapping_validation.py`, `tests/core/test_nested_mappings.py`, `tests/core/test_nested_state_paths.py`, and authoring tests as needed.
## Task 1: Add Typed Path Values
**Files:**
- Modify: `src/wf_core/paths.py`
- Modify: `src/wf_core/local_paths.py`
- Create: `tests/core/test_path_values.py`
- [ ] **Step 1: Write path value tests**
Add tests for parsing, string serialization, equality/hashability, invalid segments, root-only graph source reads, and no bare write state:
```python
import pytest
from pydantic import BaseModel, ValidationError
from wf_core.paths import GraphSourcePath, LocalPath, PathResolutionError, StatePath
def test_graph_source_path_accepts_root_and_nested_paths():
assert str(GraphSourcePath.parse("state")) == "state"
assert str(GraphSourcePath.parse("input.user")) == "input.user"
assert str(GraphSourcePath.context("loop_item")) == "context.loop_item"
def test_state_path_rejects_bare_state_write_target():
with pytest.raises(PathResolutionError, match="state path"):
StatePath.parse("state")
def test_local_path_supports_root_marker():
assert str(LocalPath.root()) == "."
assert str(LocalPath.of("user.name")) == "user.name"
@pytest.mark.parametrize("raw", ["", "state.", "state.items.0", "state.user-name"])
def test_paths_reject_invalid_segments(raw: str):
with pytest.raises(PathResolutionError):
GraphSourcePath.parse(raw)
def test_path_objects_are_hashable():
paths = {StatePath.of("person.name"), StatePath.of("person.name")}
assert len(paths) == 1
def test_pydantic_accepts_path_strings_and_serializes_strings():
class Payload(BaseModel):
source: GraphSourcePath
target: StatePath
local: LocalPath
payload = Payload.model_validate(
{"source": "input.user", "target": "state.person", "local": "user"}
)
assert payload.source == GraphSourcePath.input("user")
assert payload.model_dump(mode="json")["target"] == "state.person"
def test_pydantic_rejects_bad_path_string():
class Payload(BaseModel):
source: GraphSourcePath
with pytest.raises(ValidationError):
Payload.model_validate({"source": "output.foo"})
```
- [ ] **Step 2: Run path tests to verify they fail**
Run: `uv run --with pytest pytest tests/core/test_path_values.py -q`
Expected: failures because `GraphSourcePath`, `StatePath`, and `LocalPath` classes do not exist or do not validate strictly.
- [ ] **Step 3: Implement path value classes**
In `src/wf_core/paths.py`, add frozen dataclasses and shared parsing helpers. Keep existing helper function names as compatibility wrappers where practical.
Implementation shape:
```python
from dataclasses import dataclass
import re
from typing import Any, ClassVar, Literal
SEGMENT_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
@dataclass(frozen=True)
class LocalPath:
"""Node-local payload path. `.` means the whole local payload."""
parts: tuple[str, ...]
@classmethod
def root(cls) -> "LocalPath":
return cls(())
@classmethod
def of(cls, *fragments: str) -> "LocalPath":
return cls(_parse_fragments(*fragments, allow_empty=False))
@classmethod
def parse(cls, raw: str) -> "LocalPath":
if raw == ".":
return cls.root()
return cls.of(raw)
def __str__(self) -> str:
return "." if not self.parts else ".".join(self.parts)
```
Also add:
```python
GraphRoot = Literal["input", "state", "context"]
@dataclass(frozen=True)
class GraphSourcePath:
"""Readable workflow graph path rooted at input, state, or context."""
root: GraphRoot
parts: tuple[str, ...] = ()
@classmethod
def parse(cls, raw: str) -> "GraphSourcePath": ...
@classmethod
def input(cls, *fragments: str) -> "GraphSourcePath": ...
@classmethod
def state(cls, *fragments: str) -> "GraphSourcePath": ...
@classmethod
def context(cls, *fragments: str) -> "GraphSourcePath": ...
```
And:
```python
@dataclass(frozen=True)
class StatePath:
"""Writable workflow state path. Bare `state` is intentionally invalid."""
parts: tuple[str, ...]
@classmethod
def parse(cls, raw: str) -> "StatePath":
parsed = GraphSourcePath.parse(raw)
if parsed.root != "state" or not parsed.parts:
raise PathResolutionError("expected state path such as state.foo")
return cls(parsed.parts)
@classmethod
def of(cls, *fragments: str) -> "StatePath": ...
```
Add Pydantic `__get_pydantic_core_schema__` and `__get_pydantic_json_schema__` hooks for each class so strings validate into objects and serialize back to strings.
- [ ] **Step 4: Update local path wrappers**
In `src/wf_core/local_paths.py`, keep public functions but delegate parsing to `LocalPath.parse`:
```python
def split_local_path(path: str | LocalPath) -> list[str]:
"""Split one node-local path, accepting the new typed path object."""
parsed = path if isinstance(path, LocalPath) else LocalPath.parse(path)
return list(parsed.parts)
```
Update `paths_overlap` and `has_overlapping_paths` to accept `str | LocalPath`.
- [ ] **Step 5: Run path tests**
Run: `uv run --with pytest pytest tests/core/test_path_values.py -q`
Expected: all tests in `test_path_values.py` pass.
## Task 2: Add Canonical Node Binding Models
**Files:**
- Modify: `src/wf_core/models/steps.py`
- Test: `tests/core/test_canonical_node_bindings.py`
- [ ] **Step 1: Write canonical binding tests**
Create `tests/core/test_canonical_node_bindings.py`:
```python
import pytest
from pydantic import ValidationError
from wf_core.models.steps import NodeUse
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
def test_node_use_accepts_canonical_input_and_output_bindings():
node = NodeUse.model_validate(
{
"id": "echo",
"type": "node",
"node": "echo",
"input": [
{"target": "message", "path": "input.message"},
{"target": "mode", "value": None},
],
"output": [{"source": "echoed", "target": "state.echoed"}],
}
)
assert node.input[0].target == LocalPath.of("message")
assert node.input[0].path == GraphSourcePath.input("message")
assert node.input[1].value is None
assert node.output[0].target == StatePath.of("echoed")
def test_node_use_converts_old_maps_to_canonical_bindings():
node = NodeUse.model_validate(
{
"id": "echo",
"type": "node",
"node": "echo",
"in_map": {"input.message": "message"},
"input_values": {"mode": "fast"},
"out_map": {"echoed": "state.echoed"},
}
)
dumped = node.model_dump(mode="json")
assert "in_map" not in dumped
assert "input_values" not in dumped
assert "out_map" not in dumped
assert dumped["input"][0]["path"] == "input.message"
assert dumped["input"][1]["value"] == "fast"
assert dumped["output"][0]["target"] == "state.echoed"
def test_node_use_rejects_mixed_old_and_new_binding_styles():
with pytest.raises(ValidationError):
NodeUse.model_validate(
{
"id": "echo",
"type": "node",
"node": "echo",
"input": [{"target": "message", "path": "input.message"}],
"in_map": {"input.other": "other"},
}
)
def test_input_binding_rejects_path_and_value_together():
with pytest.raises(ValidationError):
NodeUse.model_validate(
{
"id": "bad",
"type": "node",
"node": "bad",
"input": [
{"target": "message", "path": "input.message", "value": "x"}
],
}
)
```
- [ ] **Step 2: Run binding tests to verify they fail**
Run: `uv run --with pytest pytest tests/core/test_canonical_node_bindings.py -q`
Expected: failures because canonical binding fields do not exist yet.
- [ ] **Step 3: Implement binding models**
In `src/wf_core/models/steps.py`, add:
```python
from pydantic import BaseModel, ConfigDict, Field, model_validator
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
class InputPathBinding(BaseModel):
"""Map one graph source path into one node-local input path."""
model_config = ConfigDict(extra="forbid")
target: LocalPath
path: GraphSourcePath
class InputValueBinding(BaseModel):
"""Map one static JSON-compatible value into one node-local input path."""
model_config = ConfigDict(extra="forbid")
target: LocalPath
value: object
InputBinding = Annotated[
InputPathBinding | InputValueBinding,
Field(union_mode="left_to_right"),
]
class OutputBinding(BaseModel):
"""Map one node-local output path into one workflow state path."""
model_config = ConfigDict(extra="forbid")
source: LocalPath
target: StatePath
```
Update `NodeUse`:
```python
class NodeUse(BaseModel):
...
input: list[InputBinding] = Field(default_factory=list)
output: list[OutputBinding] = Field(default_factory=list)
@model_validator(mode="before")
@classmethod
def _coerce_deprecated_maps(cls, data: object) -> object:
...
```
The validator should:
- If `input` or `output` is present, reject any of `in_map`, `input_values`, `out_map`.
- Convert `input_values` entries to `{"target": key, "value": value}` preserving order.
- Convert `in_map` entries to `{"target": destination, "path": source}` preserving order.
- Convert `out_map` entries to `{"source": source, "target": destination}` preserving order.
- Remove old keys from the normalized data.
- [ ] **Step 4: Run binding tests**
Run: `uv run --with pytest pytest tests/core/test_canonical_node_bindings.py -q`
Expected: all tests in `test_canonical_node_bindings.py` pass.
## Task 3: Move Runtime Node Input Resolution To Canonical Bindings
**Files:**
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_nested_mappings.py`
- Test: `tests/core/test_canonical_node_bindings.py`
- [ ] **Step 1: Add runtime tests for canonical input binding behavior**
In `tests/core/test_nested_mappings.py`, add a test that builds the existing minimal workflow style but uses `input` / `output` instead of old maps:
```python
def test_canonical_bindings_resolve_input_values_and_paths():
workflow = Workflow.model_validate(
{
"name": "canonical",
"input_schema": {"type": "object", "properties": {"message": {"type": "string"}}},
"state_schema": {"fields": {"echoed": {"type": "string"}}},
"output_schema": {"type": "object", "properties": {"echoed": {"type": "string"}}},
"start": "echo",
"node_defs": [
{
"name": "echo",
"input_schema": {
"type": "object",
"properties": {"message": {"type": "string"}, "mode": {"type": "string"}},
"required": ["message", "mode"],
},
"output_schema": {"type": "object", "properties": {"echoed": {"type": "string"}}},
"outcomes": ["ok"],
}
],
"nodes": [
{
"id": "echo",
"type": "node",
"node": "echo",
"input": [
{"target": "message", "path": "input.message"},
{"target": "mode", "value": "fast"},
],
"output": [{"source": "echoed", "target": "state.echoed"}],
}
],
"edges": [{"from": "echo", "outcome": "ok", "to": "__end__"}],
}
)
result = execute_workflow(
workflow,
{"message": "hi"},
registry={"echo": lambda payload, _ctx: {"echoed": f"{payload['mode']}:{payload['message']}"}},
)
assert result.output["echoed"] == "fast:hi"
```
- [ ] **Step 2: Run the focused test to verify failure**
Run: `uv run --with pytest pytest tests/core/test_nested_mappings.py::test_canonical_bindings_resolve_input_values_and_paths -q`
Expected: failure because runtime still reads `node.input_values`, `node.in_map`, and `node.out_map`.
- [ ] **Step 3: Update `_resolve_node_execution`**
In `src/wf_core/runtime/ops/nodes.py`, import binding classes and use `node.input`.
Implementation shape:
```python
from wf_core.models.steps import InputPathBinding, InputValueBinding
for binding in node.input:
if isinstance(binding, InputValueBinding):
value = binding.value
else:
value = safe_resolve_path(
str(binding.path),
state=run.state,
workflow_input=run.workflow_input,
context=context_values,
)
set_local_value(resolved_input, binding.target, value)
```
`set_local_value` should accept `LocalPath` after Task 1.
- [ ] **Step 4: Run canonical runtime test**
Run: `uv run --with pytest pytest tests/core/test_nested_mappings.py::test_canonical_bindings_resolve_input_values_and_paths -q`
Expected: pass.
## Task 4: Move Runtime Output Writes To Canonical Bindings And Atomic Patches
**Files:**
- Modify: `src/wf_core/runtime/ops/state.py`
- Modify: `src/wf_core/runtime/ops/nodes.py`
- Test: `tests/core/test_atomic_state_patches.py`
- [ ] **Step 1: Write atomic patch tests**
Create `tests/core/test_atomic_state_patches.py`:
```python
import pytest
from wf_core.errors import WorkflowExecutionError
from wf_core.models.workflow import Workflow
from wf_core.runtime.ops.state import apply_output_bindings
def _workflow() -> Workflow:
return Workflow.model_validate(
{
"name": "patch",
"input_schema": {"type": "object", "properties": {}},
"state_schema": {
"fields": {
"person": {"type": "object"},
"person.name": {"type": "string"},
}
},
"output_schema": {"type": "object", "properties": {}},
"start": "n",
"nodes": [],
"edges": [],
}
)
def test_output_bindings_commit_patch_atomically():
workflow = _workflow()
state = {"person": {"name": "old"}}
with pytest.raises(WorkflowExecutionError):
apply_output_bindings(
workflow,
[
{"source": "person.name", "target": "state.person.name"},
{"source": "missing", "target": "state.person.extra"},
],
{"person": {"name": "new"}},
state,
)
assert state["person"]["name"] == "old"
def test_output_bindings_reject_overlapping_write_targets():
workflow = _workflow()
state = {}
with pytest.raises(WorkflowExecutionError, match="overlapping"):
apply_output_bindings(
workflow,
[
{"source": "person", "target": "state.person"},
{"source": "person.name", "target": "state.person.name"},
],
{"person": {"name": "Ada"}},
state,
)
```
- [ ] **Step 2: Run atomic patch tests to verify failure**
Run: `uv run --with pytest pytest tests/core/test_atomic_state_patches.py -q`
Expected: failure because `apply_output_bindings` does not exist.
- [ ] **Step 3: Implement `apply_output_bindings`**
In `src/wf_core/runtime/ops/state.py`, add a canonical function:
```python
from wf_core.models.steps import OutputBinding
from wf_core.paths import StatePath
def apply_output_bindings(
workflow: Workflow,
bindings: Sequence[OutputBinding],
node_output: dict[str, Any],
state: dict[str, Any],
reducers: Mapping[str, ReducerDefinition] | None = None,
) -> dict[str, Any]:
"""Prepare and commit one atomic state patch from canonical output bindings."""
```
Function behavior:
- Validate no overlapping `binding.target`.
- Resolve every `binding.source` from `node_output` first.
- Build a prepared patch keyed by `StatePath`.
- Compute reducers into prepared merged values without mutating `state`.
- Commit all prepared values only after all prior steps succeed.
- Return JSON-friendly `dict[str, Any]` state changes using `str(path)` keys for now, until trace is separately migrated.
Keep `apply_output_map` as a compatibility wrapper that converts old map entries into `OutputBinding` and calls `apply_output_bindings`.
- [ ] **Step 4: Update node finalization**
In `src/wf_core/runtime/ops/nodes.py`, call `apply_output_bindings(workflow, node.output, result.output, run.state, reducers=reducers)` instead of `apply_output_map(...)`.
- [ ] **Step 5: Run state patch tests**
Run: `uv run --with pytest pytest tests/core/test_atomic_state_patches.py tests/core/test_nested_mappings.py -q`
Expected: pass.
## Task 5: Update Validation For Canonical Bindings
**Files:**
- Modify: `src/wf_core/validation/steps.py`
- Test: `tests/core/test_mapping_validation.py`
- Test: `tests/core/test_canonical_node_bindings.py`
- [ ] **Step 1: Add validation tests for canonical fields**
In `tests/core/test_mapping_validation.py`, add tests for invalid source paths, invalid destination paths, overlapping local input targets, and overlapping state output targets using canonical `input` / `output`.
Example:
```python
def test_validate_workflow_reports_overlapping_canonical_output_targets():
workflow = workflow_with_node(
node_use={
"id": "n",
"type": "node",
"node": "n",
"output": [
{"source": "person", "target": "state.person"},
{"source": "person.name", "target": "state.person.name"},
],
}
)
report = workflow.validate_structure()
assert any(issue.code == ValidationIssueCode.INVALID_DESTINATION_PATH for issue in report.issues)
```
Use the existing helper style in `tests/core/test_mapping_validation.py` rather than inventing a second full workflow factory if one already exists.
- [ ] **Step 2: Run mapping validation tests**
Run: `uv run --with pytest pytest tests/core/test_mapping_validation.py -q`
Expected: new canonical validation tests fail until validation reads `node.input` / `node.output`.
- [ ] **Step 3: Update `validate_node_use`**
In `src/wf_core/validation/steps.py`:
- Iterate `node.input`.
- For `InputValueBinding`, validate target local root against node input schema.
- For `InputPathBinding`, validate target and source graph path.
- Iterate `node.output`.
- Validate output source local root against node output schema.
- Validate destination `StatePath`.
- Use typed overlap helpers instead of raw map values.
- Keep issue paths readable, e.g. `nodes[0].input[1].target`.
- [ ] **Step 4: Run validation tests**
Run: `uv run --with pytest pytest tests/core/test_mapping_validation.py tests/core/test_canonical_node_bindings.py -q`
Expected: pass.
## Task 6: Add Canonical State Schema Fields
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Modify: `src/wf_core/runtime/ops/state.py`
- Modify: `src/wf_core/validation/steps.py`
- Test: `tests/core/test_nested_state_paths.py`
- Test: `tests/core/test_schema_validation.py`
- [ ] **Step 1: Write state schema canonical shape tests**
In `tests/core/test_nested_state_paths.py`, add:
```python
from wf_core.models.schemas import StateSchema
from wf_core.paths import StatePath
def test_state_schema_accepts_canonical_field_list():
schema = StateSchema.model_validate(
{
"fields": [
{"path": "state.person", "type": "object"},
{"path": "state.person.name", "type": "string", "reducer": "wf.std.replace"},
]
}
)
assert schema.fields[0].path == StatePath.of("person")
assert schema.field_map()["person.name"].type == "string"
def test_state_schema_accepts_deprecated_dict_shape():
schema = StateSchema.model_validate(
{"fields": {"person.name": {"type": "string"}}}
)
assert schema.model_dump(mode="json")["fields"][0]["path"] == "state.person.name"
```
- [ ] **Step 2: Run state schema tests to verify failure**
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py -q`
Expected: failure because `StateSchema.fields` is still a dict.
- [ ] **Step 3: Implement canonical `StateFieldDecl`**
In `src/wf_core/models/schemas.py`:
```python
class StateFieldDecl(BaseModel):
"""One declared state path plus validation and reducer metadata."""
path: StatePath
schema: SchemaRef = Field(default_factory=lambda: SchemaRef(type="object"))
reducer: ReducerRef = Field(default_factory=lambda: ReducerRef(name="wf.std.replace"))
trace: bool = True
default: Any = None
```
Preserve compatibility for old `type` directly on the field:
- For old dict values like `{"type": "string"}`, convert to `{"schema": {"type": "string"}}`.
- For canonical values, allow either `schema` or simple `type` as input if that keeps existing tests stable.
Update `StateSchema`:
```python
class StateSchema(BaseModel):
fields: list[StateFieldDecl] = Field(default_factory=list)
def field_map(self) -> dict[str, StateFieldDecl]:
return {".".join(field.path.parts): field for field in self.fields}
```
Add a model validator to accept old dict shape and normalize to list.
- [ ] **Step 4: Update callers of `workflow.state_schema.fields`**
Search: `rg 'state_schema\\.fields|\\.fields\\.get|set\\(workflow\\.state_schema\\.fields\\)' src tests`
Update code to use `workflow.state_schema.field_map()` when it needs lookup by rootless path.
Important updates:
- `src/wf_core/runtime/ops/state.py`
- `src/wf_core/validation/steps.py`
- any authoring or artifact code constructing state field maps.
- [ ] **Step 5: Run state schema tests**
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py -q`
Expected: pass.
## Task 7: Harden SchemaRef With JSON Schema Validation
**Files:**
- Modify: `src/wf_core/models/schemas.py`
- Modify: `src/wf_core/runtime/ops/schemas.py`
- Test: `tests/core/test_schema_validation.py`
- [ ] **Step 1: Add schema validation tests**
In `tests/core/test_schema_validation.py`, add tests:
```python
import pytest
from pydantic import ValidationError
from wf_core.models.schemas import SchemaRef
def test_schema_ref_accepts_valid_json_schema_with_defs():
schema = SchemaRef.model_validate(
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"$defs": {"Name": {"type": "string"}},
"properties": {"name": {"$ref": "#/$defs/Name"}},
}
)
assert schema.model_extra["$defs"]["Name"]["type"] == "string"
def test_schema_ref_rejects_invalid_json_schema():
with pytest.raises(ValidationError):
SchemaRef.model_validate({"type": 123})
```
- [ ] **Step 2: Run schema tests to verify failure**
Run: `uv run --with pytest pytest tests/core/test_schema_validation.py -q`
Expected: invalid schema is currently accepted.
- [ ] **Step 3: Add `jsonschema` validation**
In `src/wf_core/models/schemas.py`, import:
```python
from jsonschema import SchemaError
from jsonschema.validators import Draft202012Validator, validator_for
from pydantic import model_validator
```
Add an after validator to `SchemaRef`:
```python
@model_validator(mode="after")
def _validate_json_schema(self) -> "SchemaRef":
raw = self.model_dump(mode="python", exclude_none=True)
validator_cls = validator_for(raw, default=Draft202012Validator)
try:
validator_cls.check_schema(raw)
except SchemaError as exc:
raise ValueError(f"invalid JSON Schema: {exc.message}") from exc
return self
```
- [ ] **Step 4: Run schema tests**
Run: `uv run --with pytest pytest tests/core/test_schema_validation.py -q`
Expected: pass.
## Task 8: Update Authoring Helpers To Emit Canonical Bindings
**Files:**
- Modify: `src/wf_authoring/dsl/paths.py`
- Modify: `src/wf_authoring/dsl/conditions.py`
- Modify: `src/wf_authoring/builder/core.py`
- Test: `tests/authoring/test_builder.py`
- Test: `tests/authoring/test_conditions.py`
- Test: `tests/authoring/test_control_flow_examples.py`
- [ ] **Step 1: Add authoring tests for canonical dumps**
In `tests/authoring/test_builder.py`, add a test that builds a workflow and asserts the dumped node uses canonical `input` / `output`, not old maps:
```python
def test_builder_emits_canonical_node_bindings():
workflow = (
WorkflowBuilder("canonical")
.schemas(
input_schema={"type": "object", "properties": {"message": {"type": "string"}}},
state_schema={"fields": {"echoed": {"type": "string"}}},
output_schema={"type": "object", "properties": {"echoed": {"type": "string"}}},
)
.use(echo_node, id="echo", in_map={"input.message": "message"}, out_map={"echoed": "state.echoed"})
.start_at("echo")
.end("echo", "ok")
.build()
)
dumped_node = workflow.model_dump(mode="json")["nodes"][0]
assert "input" in dumped_node
assert "output" in dumped_node
assert "in_map" not in dumped_node
assert "out_map" not in dumped_node
```
Adapt helper names to the current builder API in the file.
- [ ] **Step 2: Run authoring builder tests**
Run: `uv run --with pytest pytest tests/authoring/test_builder.py tests/authoring/test_conditions.py -q`
Expected: new canonical dump test may fail until builder emits or model normalizes canonical shapes.
- [ ] **Step 3: Update path/condition authoring wrappers**
In `src/wf_authoring/dsl/paths.py`, make ergonomic helpers return wrappers around core path values or values accepted by core models. Preserve existing public behavior where possible:
```python
def state_path(*parts: str) -> GraphPath:
return GraphPath(str(GraphSourcePath.state(*parts)))
```
In `src/wf_authoring/dsl/conditions.py`, make `PathExpr` compile using `GraphSourcePath.parse` for `PathOperand`.
- [ ] **Step 4: Update builder to rely on canonical model normalization**
In `src/wf_authoring/builder/core.py`, either emit canonical binding dicts directly or keep passing old maps into `NodeUse.model_validate`. Prefer direct canonical emission where the builder already has enough structure.
Do not remove user-facing `in_map` / `out_map` builder parameters in this pass.
- [ ] **Step 5: Run authoring tests**
Run: `uv run --with pytest pytest tests/authoring -q`
Expected: authoring tests pass.
## Task 9: Full Compatibility And Regression Pass
**Files:**
- Modify docs/examples only if tests show stale serialized shapes.
- Test: full repo.
- [ ] **Step 1: Run core tests**
Run: `uv run --with pytest pytest tests/core tests/authoring tests/rewrite -q`
Expected: pass.
- [ ] **Step 2: Run artifact and MCP workflow-surface tests**
Run: `uv run --with pytest pytest tests/artifacts tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_workflow_wrappers.py tests/wf_mcp/test_mcp_workflow_surface_example.py -q`
Expected: pass.
- [ ] **Step 3: Run full test suite**
Run: `uv run --with pytest pytest -q`
Expected: pass, allowing any existing intentionally skipped environment-dependent tests.
- [ ] **Step 4: Run static checks**
Run:
```bash
uvx ruff check
uv run basedpyright --level error
```
Expected: ruff passes and basedpyright reports 0 errors.
- [ ] **Step 5: Format touched files**
Run:
```bash
uvx ruff format src/wf_core src/wf_authoring tests/core tests/authoring
```
Expected: files format cleanly.
## Self-Review Notes
- Spec coverage: typed paths, canonical bindings, parse-only compatibility, null/missing semantics, dynamic traversal deferral, state patch atomicity, reducer behavior, JSON Schema validation, authoring updates, and tracing shape are covered. Full trace migration is intentionally not implemented beyond returning string-keyed `state_changes` for compatibility.
- Placeholder scan: this plan avoids `TBD` and names concrete files, tests, commands, and behavior.
- Type consistency: `LocalPath`, `GraphSourcePath`, `StatePath`, `InputPathBinding`, `InputValueBinding`, `OutputBinding`, and `StateFieldDecl` are introduced before later tasks use them.
@@ -0,0 +1,331 @@
# Core Path Bindings Design
## Purpose
`wf_core` currently uses plain strings for graph paths, local node paths, input
maps, output maps, and state field keys. That made the early system simple, but
it also pushes too much meaning into ad hoc string parsing. This design makes
paths and bindings first-class core concepts while keeping JSON serialization
simple.
The goal is not to make every dynamic JSON traversal statically provable. The
goal is to make normal workflow data movement explicit, validated, serializable,
and easy for authoring/MCP layers to generate.
## Goals
- Replace loose map fields with canonical list-of-struct binding models.
- Store typed path value objects internally while serializing them as strings.
- Keep missing values distinct from explicit `null`.
- Make state writes atomic and reducer-aware.
- Keep compatibility with old workflow shapes through parse-only deprecated
fields.
- Keep dynamic/open-object traversal out of core and in explicit nodes.
## Non-Goals
- No list/index path syntax such as `items.0.name` or `items[0].name`.
- No arbitrary JSON-pointer support.
- No whole-state replacement writes in this pass.
- No implicit defaults for missing paths.
- No business logic in reducers or bindings.
- No custom schema language replacing JSON Schema.
## Path Types
Core introduces distinct path value objects:
```text
LocalPath(parts)
GraphSourcePath(root, parts) # root: input | state | context
StatePath(parts) # serializes as state.<parts>
```
These objects are immutable/hashable and are accepted by Pydantic from either
strings or existing instances. JSON serialization emits strings.
Examples:
```text
LocalPath.of("user.name") -> "user.name"
LocalPath.root() -> "."
GraphSourcePath.state("person.name") -> "state.person.name"
GraphSourcePath.parse("input") -> "input"
StatePath.of("person.name") -> "state.person.name"
```
Path parsing rules:
- Segments use `[A-Za-z_][A-Za-z0-9_]*`.
- Dots separate segments.
- Empty segments are invalid.
- Numeric/positional list segments are rejected.
- `LocalPath.root()` / `"."` is the only local root marker.
- Root-only graph source paths `input`, `state`, and `context` are valid reads.
`StatePath` write targets should not accept bare `state` in this pass. Whole
state replacement is too broad because it interacts with reducers, validation,
trace, and accidental deletion.
There is no reducer for bare `state`. Reducers attach only to declared non-root
`StatePath` fields.
## Canonical Node Bindings
`NodeUse` gets canonical binding fields:
```text
NodeUse.input: list[InputBinding]
NodeUse.output: list[OutputBinding]
```
Input bindings are distinguished by shape, not by an extra `kind` field:
```text
InputPathBinding:
target: LocalPath
path: GraphSourcePath
InputValueBinding:
target: LocalPath
value: JsonValue
```
Output bindings are:
```text
OutputBinding:
source: LocalPath
target: StatePath
```
The same field name can mean different path kinds by position. For example,
input binding `target` is node-local, while output binding `target` is workflow
state. Documentation and model field descriptions should make this explicit.
Root local path `"."` is valid:
- input target `"."` means the whole node input payload.
- output source `"."` means the whole node output payload.
- a `"."` input binding must be the only input binding.
Examples:
```json
{
"input": [
{"target": "user.email", "path": "state.person.email"},
{"target": "mode", "value": "fast"}
],
"output": [
{"source": "result", "target": "state.result"}
]
}
```
Whole payload input:
```json
{
"input": [
{"target": ".", "path": "state.rates"}
]
}
```
Whole payload literal input:
```json
{
"input": [
{"target": ".", "value": {"mode": "fast"}}
]
}
```
## Compatibility
Old fields are accepted only as deprecated parse inputs:
```text
in_map
input_values
out_map
```
After validation, `NodeUse` stores only canonical `input` and `output`
bindings. Canonical serialization emits only the new fields. Payloads that mix
canonical fields with deprecated fields should fail instead of merging two
styles.
This shape makes it easy to remove compatibility later: delete the parser
adapters without changing runtime internals.
State schema gets the same compatibility shape. The canonical form is:
```text
StateSchema.fields: list[StateFieldDecl]
StateFieldDecl:
path: StatePath
schema: SchemaRef
reducer: ReducerRef
```
Old dict-shaped fields can be accepted at parse time and normalized:
```json
{
"fields": {
"person.tags": {"type": "array", "reducer": "wf.std.append"}
}
}
```
Canonical serialization should emit list-of-structs with serialized state paths.
## Runtime Semantics
Node execution flow:
1. Validate workflow input against `workflow.input_schema`.
2. Resolve node input bindings into the node payload.
3. Validate node payload against `node_def.input_schema`.
4. Execute the node.
5. Coerce the node result.
6. Validate node output against `node_def.output_schema`.
7. Prepare an atomic state patch from output bindings.
8. Validate focused state patch values.
9. Commit the patch.
10. At `END`, project final output from top-level state fields using
`workflow.output_schema.properties`.
State writes are atomic. Runtime should resolve all output sources, detect
overlap conflicts, compute reducer results, validate patch values, then commit.
If any step fails, prior state is unchanged.
Reducers run during patch preparation. They receive current value, incoming
value, and optional config, then return the merged value. Reducers must not
mutate workflow state directly.
## Missing, Null, And Dynamic Data
Explicit `null` is a real value. It is not the same as a missing path.
Rules:
- `exists(path)` returns true when the path resolves, even if the value is null.
- `exists(path)` returns false when the path is missing.
- Comparisons such as `eq(null)` and `ne(null)` are valid.
- Comparisons against missing paths fail clearly.
- Input path bindings fail before node execution when the source path is
missing.
- Missing source paths must not silently bind null.
- Input value bindings may intentionally bind null.
`allow extra` / open object schemas do not authorize speculative deep traversal.
If a workflow needs dynamic object traversal, it should use an explicit node
that receives the relevant value and decides what to extract. For example, use
an `extract_title` node instead of trying to make core prove
`state.person.occupations.1.title`.
## Validation Rules
Path value objects validate syntax and path kind only. Workflow validation checks
whether a path is legal in a particular workflow.
Examples of workflow-specific checks:
- `input.foo` exists in `workflow.input_schema` when statically knowable.
- `state.foo` has a declared state root.
- write destinations are `StatePath`.
- output write targets do not overlap.
Overlap rules:
- read paths may overlap.
- input targets must not overlap.
- output targets must not overlap.
- exact duplicate targets are invalid.
- parent/child write targets such as `state.person` and `state.person.name`
are invalid together.
State write validation:
- Validate against the exact declared state field schema when one exists.
- Do not validate the whole workflow state on every write.
- Replacement writes must satisfy the full target schema.
- Replacement writes with forbidden extra fields fail.
- Partial object patches require an explicit merge-style reducer such as
`merge_object`.
- Do not silently treat `replace` as merge.
## JSON Schema Boundary
Core should not invent a schema language. `SchemaRef` should represent a JSON
Schema object boundary and be validated with the standard `jsonschema` library.
Rules:
- Respect `$schema` when present with `jsonschema.validators.validator_for`.
- Default to Draft 2020-12 when `$schema` is absent.
- Validate workflow input/output schemas and node input/output schemas at model
boundaries.
- Keep JSON Schema object keys separate from workflow path syntax.
Path fields should expose clear JSON Schema as strings with pattern and
description metadata, not `{root, parts}` objects.
## Authoring Layer
`wf_authoring` keeps ergonomic helpers:
- `state_path(...)`
- `input_path(...)`
- `context_path(...)`
- expression helpers such as `eq`, `ne`, `lt`, `le`, `gt`, `ge`, and `exists`
Those helpers should compile to core path objects and core `Condition` models.
Authoring may infer mappings for convenience, but it must emit explicit
canonical bindings into core models.
## Tracing
Runtime may use flat `dict[StatePath, value]` patches internally. Public trace
serialization should prefer list-of-structs:
```text
StateChange:
path: StatePath
value: JsonValue
```
This avoids custom JSON object-key serialization and gives MCP/LLM clients
clearer output.
## Implementation Phases
1. Add core path value objects and parser helpers.
2. Add canonical binding models with parse-only compatibility for old fields.
3. Update validation to reason over path objects and binding structs.
4. Update runtime input resolution and output patching to use canonical
bindings.
5. Add focused state patch preparation with atomic commit semantics.
6. Add JSON Schema validation hardening for `SchemaRef`.
7. Update `wf_authoring` builders/helpers to emit canonical bindings.
8. Update docs/examples and mark old fields as deprecated.
Each phase should keep existing tests green, with compatibility tests proving
old shapes still parse until support is intentionally removed.
## Open Risks
- Pydantic core-schema hooks for frozen path value objects may need careful
implementation to keep JSON Schema clean.
- Existing examples and MCP-facing draft tools may rely on old dict-shaped maps.
Compatibility adapters should isolate that churn.
- Focused schema validation for nested state writes depends on how much schema
information is available for exact declared state paths.
- Workflow input currently seeds initial state. This design keeps that
compatibility for now, but explicit initialization remains the target
direction.