plan
This commit is contained in:
@@ -0,0 +1,571 @@
|
|||||||
|
# Path And Mapping Scratch
|
||||||
|
|
||||||
|
This is a focused scratchpad for the `wf_core` path/mapping design thread.
|
||||||
|
Clean this into real docs later.
|
||||||
|
|
||||||
|
## Current Mapping Roles
|
||||||
|
|
||||||
|
`in_map` maps graph/runtime paths into node-local input paths:
|
||||||
|
|
||||||
|
```text
|
||||||
|
input.text -> text
|
||||||
|
state.person.name -> user.name
|
||||||
|
context.item -> document
|
||||||
|
```
|
||||||
|
|
||||||
|
`out_map` maps node-local output paths into workflow state destinations:
|
||||||
|
|
||||||
|
```text
|
||||||
|
echoed -> state.echoed
|
||||||
|
user.age -> state.person.age
|
||||||
|
. -> state.rates
|
||||||
|
```
|
||||||
|
|
||||||
|
`input_values` maps node-local input paths to literal values:
|
||||||
|
|
||||||
|
```text
|
||||||
|
mode -> "fast"
|
||||||
|
retry.count -> 3
|
||||||
|
```
|
||||||
|
|
||||||
|
## Path Kinds
|
||||||
|
|
||||||
|
Graph paths live in workflow/run space:
|
||||||
|
|
||||||
|
- `input.*`: original workflow invocation input.
|
||||||
|
- `state.*`: mutable workflow state.
|
||||||
|
- `context.*`: runtime frame context.
|
||||||
|
|
||||||
|
Node-local paths live inside one node input/output payload:
|
||||||
|
|
||||||
|
- `user.name`
|
||||||
|
- `job.years`
|
||||||
|
- `.` for the whole local payload.
|
||||||
|
|
||||||
|
## Context Paths
|
||||||
|
|
||||||
|
`context.*` currently exists. It is generated from the current execution frame,
|
||||||
|
not stored in workflow state.
|
||||||
|
|
||||||
|
Current context fields include:
|
||||||
|
|
||||||
|
- `context.prior_outcome`
|
||||||
|
- `context.activated_incoming_edge`
|
||||||
|
- foreach frames: `context.loop_item`
|
||||||
|
- foreach frames: `context.loop_index`
|
||||||
|
- foreach frames: `context.<loop_alias>` when the foreach node declares `as`.
|
||||||
|
|
||||||
|
Context paths are valid graph source paths for `in_map` when validation allows
|
||||||
|
context. They are not valid `out_map` destinations.
|
||||||
|
|
||||||
|
## Typed Internal Model Idea
|
||||||
|
|
||||||
|
Keep serialized maps as strings for JSON compatibility, but parse them at
|
||||||
|
runtime/validation boundaries:
|
||||||
|
|
||||||
|
```text
|
||||||
|
GraphPath(root="input", parts=("text",)) <-> "input.text"
|
||||||
|
GraphPath(root="state", parts=("person", "name")) <-> "state.person.name"
|
||||||
|
GraphPath(root="context", parts=("item",)) <-> "context.item"
|
||||||
|
LocalPath(parts=("user", "name")) <-> "user.name"
|
||||||
|
LocalPath.root() <-> "."
|
||||||
|
```
|
||||||
|
|
||||||
|
This lets validation reason about roots and parts without repeated string
|
||||||
|
prefix checks.
|
||||||
|
|
||||||
|
## Decisions So Far
|
||||||
|
|
||||||
|
Canonical path parsing/validation should live in `wf_core`.
|
||||||
|
|
||||||
|
`wf_authoring` should keep ergonomic constructors and condition wrappers, but
|
||||||
|
those should be thin wrappers around core path types.
|
||||||
|
|
||||||
|
Condition path fields should use typed graph source paths, not arbitrary
|
||||||
|
strings:
|
||||||
|
|
||||||
|
```text
|
||||||
|
PathOperand.path: GraphSourcePath
|
||||||
|
ExistsCondition.path: GraphSourcePath
|
||||||
|
```
|
||||||
|
|
||||||
|
Node input mapping should merge the old `in_map` and `input_values` concepts
|
||||||
|
into one list of binding structs:
|
||||||
|
|
||||||
|
```text
|
||||||
|
InputPathBinding:
|
||||||
|
target: LocalPath
|
||||||
|
path: GraphSourcePath
|
||||||
|
|
||||||
|
InputValueBinding:
|
||||||
|
target: LocalPath
|
||||||
|
value: JsonValue
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not add a `kind` discriminator if the shape can be distinguished by `path`
|
||||||
|
vs `value`. Use strict models so `{path, value}` and `{}` fail.
|
||||||
|
|
||||||
|
Node output mapping should become a list of binding structs:
|
||||||
|
|
||||||
|
```text
|
||||||
|
OutputBinding:
|
||||||
|
source: LocalPath
|
||||||
|
target: StatePath
|
||||||
|
```
|
||||||
|
|
||||||
|
The same field name can mean different path kinds by position:
|
||||||
|
|
||||||
|
- input binding `target` is a node-local input path
|
||||||
|
- output binding `target` is a workflow state path
|
||||||
|
|
||||||
|
Root local path `"."` remains valid:
|
||||||
|
|
||||||
|
- input target `"."` means the whole node input payload is the mapped path/value
|
||||||
|
- output source `"."` means the whole node output payload is written
|
||||||
|
- literal input value binding to `"."` is valid and means the literal value is
|
||||||
|
the entire node input payload
|
||||||
|
- `"."` input binding must still be the only input binding on that node use
|
||||||
|
|
||||||
|
Overlap rules:
|
||||||
|
|
||||||
|
- if an input binding targets `"."`, it must be the only input binding
|
||||||
|
- input targets must not overlap, e.g. `user` and `user.name`
|
||||||
|
- output targets must not overlap, e.g. `state.person` and
|
||||||
|
`state.person.name`
|
||||||
|
- exact duplicate input targets are invalid
|
||||||
|
- exact duplicate output targets are invalid
|
||||||
|
- if output writes a parent, it cannot also write a child
|
||||||
|
- read paths may overlap; write paths may not overlap
|
||||||
|
|
||||||
|
State path validation and write behavior:
|
||||||
|
|
||||||
|
- writable `StatePath` must have its root declared in `state_schema.fields`
|
||||||
|
- whole-state write targets such as bare `state` stay out of scope for now
|
||||||
|
- nested state subpaths are allowed once the root exists in the schema
|
||||||
|
- exact nested state declarations are reducer/schema hints, not root ownership
|
||||||
|
- reducers apply only to the exact declared destination path
|
||||||
|
- missing object parents are created during runtime writes
|
||||||
|
- descending through an existing non-object parent fails at runtime
|
||||||
|
|
||||||
|
Runtime traversal helpers:
|
||||||
|
|
||||||
|
- all get/set traversal should go through focused helpers
|
||||||
|
- do not scatter `dict`/`getattr` logic through runtime code
|
||||||
|
- helpers may support mappings, Pydantic models, and dataclasses where safe
|
||||||
|
- exact supported object kinds can be implementation-defined, but the behavior
|
||||||
|
should be centralized and tested
|
||||||
|
|
||||||
|
Read vs write traversal:
|
||||||
|
|
||||||
|
- reads may support richer object access through centralized helpers
|
||||||
|
- writes are stricter
|
||||||
|
- nested writes require mutable mapping parents
|
||||||
|
- typed model/dataclass parents are not patched field-by-field
|
||||||
|
- strict typed values should be replaced as a whole, not partially mutated
|
||||||
|
- full replacements must adhere to the declared schema when validation exists
|
||||||
|
|
||||||
|
State write validation:
|
||||||
|
|
||||||
|
- validate writes against the exact declared state field schema when one exists
|
||||||
|
- replacement writes must satisfy the full target schema
|
||||||
|
- replacement writes that contain fields not allowed by the target schema fail
|
||||||
|
- partial object patches require an explicit merge-style reducer such as
|
||||||
|
`merge_object`; do not silently treat replace as merge
|
||||||
|
- do not validate the whole workflow state on every write
|
||||||
|
- undeclared nested subpaths under declared object roots remain flexible
|
||||||
|
- this keeps validation focused and avoids heavy whole-state checks
|
||||||
|
|
||||||
|
Workflow input/output validation:
|
||||||
|
|
||||||
|
- validate full workflow input against `workflow.input_schema` at run start
|
||||||
|
- keep workflow output as top-level projection from state for now
|
||||||
|
- `workflow.output_schema.properties` decides which top-level state fields are
|
||||||
|
exposed as final output
|
||||||
|
- do not add workflow-level output bindings in this path refactor
|
||||||
|
- if shaped output is needed, use a final node to shape state before END
|
||||||
|
|
||||||
|
Execution validation order:
|
||||||
|
|
||||||
|
1. validate workflow input at run start
|
||||||
|
2. resolve node input bindings into node payload
|
||||||
|
3. validate node payload against `node_def.input_schema`
|
||||||
|
4. execute node
|
||||||
|
5. coerce node result
|
||||||
|
6. validate node result output against `node_def.output_schema`
|
||||||
|
7. apply output bindings to state with focused state field validation
|
||||||
|
8. project final workflow output from state at END
|
||||||
|
|
||||||
|
Principle:
|
||||||
|
|
||||||
|
- validate node output before mutating state
|
||||||
|
- strange node results should fail fast and not pollute workflow state
|
||||||
|
|
||||||
|
State patch atomicity:
|
||||||
|
|
||||||
|
- output binding application is one atomic patch to workflow state
|
||||||
|
- no gradual state mutation while processing individual bindings
|
||||||
|
- resolve all sources first
|
||||||
|
- check overlaps first
|
||||||
|
- compute reducer/merged values first
|
||||||
|
- validate patch values first
|
||||||
|
- commit the patch only after preparation succeeds
|
||||||
|
- failed output binding application leaves prior state unchanged
|
||||||
|
|
||||||
|
Reducers:
|
||||||
|
|
||||||
|
- reducers run during patch preparation, not during commit
|
||||||
|
- reducers receive current value, incoming value, and optional config
|
||||||
|
- reducers return the merged value for the patch
|
||||||
|
- reducers must not mutate workflow state directly
|
||||||
|
- reducer failure aborts the whole patch before commit
|
||||||
|
|
||||||
|
Patch representation:
|
||||||
|
|
||||||
|
- public mappings are list-of-structs
|
||||||
|
- internal prepared patches can be flat path-keyed maps
|
||||||
|
- key type should be `StatePath`, not raw string
|
||||||
|
- flat patches make overlap checks, reducer lookup, validation, tracing, and
|
||||||
|
commit performance simpler
|
||||||
|
- do not expose flat path-keyed patch maps as the public authoring shape
|
||||||
|
|
||||||
|
Trace state changes:
|
||||||
|
|
||||||
|
- use typed `StatePath` internally
|
||||||
|
- prefer list-of-structs for public trace serialization/schema
|
||||||
|
- target shape:
|
||||||
|
|
||||||
|
```text
|
||||||
|
StateChange:
|
||||||
|
path: StatePath
|
||||||
|
value: JsonValue
|
||||||
|
```
|
||||||
|
|
||||||
|
- this avoids custom JSON object key serialization problems
|
||||||
|
- it gives cleaner schema and clearer MCP/LLM output
|
||||||
|
|
||||||
|
Runtime state value domain:
|
||||||
|
|
||||||
|
- workflow definitions and static binding values should be JSON-compatible
|
||||||
|
- in-memory runtime state may remain `Any`-ish for now
|
||||||
|
- persisted/checkpointed run state should require JSON-compatible values
|
||||||
|
- checkpoint serialization should be the strict boundary for non-serializable
|
||||||
|
runtime objects
|
||||||
|
|
||||||
|
Current initialization behavior:
|
||||||
|
|
||||||
|
- workflow input is currently copied into initial state by `init_run_state`
|
||||||
|
- this makes `input.foo` and `state.foo` both available at run start when the
|
||||||
|
input has `foo`
|
||||||
|
- target direction is explicit initialization, closer to LangGraph: input stays
|
||||||
|
input, state is mutated only by explicit graph behavior
|
||||||
|
- keep implicit seeding for now as compatibility unless/until there is a
|
||||||
|
dedicated migration
|
||||||
|
|
||||||
|
Canonical node binding shape:
|
||||||
|
|
||||||
|
```text
|
||||||
|
NodeUse.input: list[InputBinding]
|
||||||
|
NodeUse.output: list[OutputBinding]
|
||||||
|
```
|
||||||
|
|
||||||
|
Old `in_map`, `input_values`, and `out_map` can be accepted as parse-only
|
||||||
|
compatibility inputs, but the canonical model should store and serialize the new
|
||||||
|
list-of-structs shape.
|
||||||
|
|
||||||
|
Naming caveat:
|
||||||
|
|
||||||
|
- `NodeUse.input` means "bindings that build this node's input payload"
|
||||||
|
- graph path root `input.*` means "the workflow run input"
|
||||||
|
|
||||||
|
Docs must make this distinction explicit. The repeated word is acceptable only
|
||||||
|
if examples clearly show `input` as the binding list and `input.foo` as a graph
|
||||||
|
source path.
|
||||||
|
|
||||||
|
Static value bindings:
|
||||||
|
|
||||||
|
- `InputValueBinding.value` should be JSON-compatible
|
||||||
|
- workflow models should stay serializable/storable
|
||||||
|
- non-serializable runtime objects such as callbacks should not be embedded in
|
||||||
|
workflow definitions
|
||||||
|
- future LangGraph-store-like behavior should use explicit store/reference
|
||||||
|
mechanisms, not arbitrary Python objects inside the model
|
||||||
|
|
||||||
|
Context paths:
|
||||||
|
|
||||||
|
- `context.*` stays a first-class graph source path
|
||||||
|
- it is read-only and source-only
|
||||||
|
- root-only graph source paths are allowed for whole-container reads:
|
||||||
|
`input`, `state`, and `context`
|
||||||
|
- root-only graph source paths are useful for explicitly passing whole workflow
|
||||||
|
input/state/context into a node
|
||||||
|
- conditions can use `context.*`
|
||||||
|
- node input path bindings can use `context.*`
|
||||||
|
- output/write bindings cannot target `context.*`
|
||||||
|
- workflow validation does not need to statically prove every context key exists
|
||||||
|
- `exists(context.x)` returns false when missing
|
||||||
|
- comparisons against missing context paths fail clearly
|
||||||
|
|
||||||
|
Validation responsibility split:
|
||||||
|
|
||||||
|
- path types validate that a value is a well-formed path of the right kind
|
||||||
|
- workflow validation checks whether that path is legal in a specific workflow
|
||||||
|
- examples of workflow-specific checks:
|
||||||
|
- `input.foo` root exists in `input_schema`
|
||||||
|
- `state.foo` root exists in `state_schema`
|
||||||
|
- write destinations are state paths
|
||||||
|
- write destinations do not overlap
|
||||||
|
- path types should not need access to workflow schemas
|
||||||
|
|
||||||
|
Path segment syntax:
|
||||||
|
|
||||||
|
- use strict identifier-like segments:
|
||||||
|
`[A-Za-z_][A-Za-z0-9_]*`
|
||||||
|
- dots separate path segments
|
||||||
|
- empty segments are invalid
|
||||||
|
- arbitrary JSON keys containing dots or punctuation are not supported yet
|
||||||
|
- bracket/index syntax is out of scope until deliberately designed
|
||||||
|
- this tightens current behavior, which mostly split strings without much
|
||||||
|
segment validation
|
||||||
|
|
||||||
|
List indexing:
|
||||||
|
|
||||||
|
- no list indexing in core paths
|
||||||
|
- paths address object/dict/model fields, not list positions
|
||||||
|
- numeric/positional list segments are rejected
|
||||||
|
- encoding workflow meaning by array position is discouraged
|
||||||
|
- use foreach for item-wise behavior
|
||||||
|
- future continue/break-style foreach behavior may cover many selection cases
|
||||||
|
- explicit helper nodes such as an authoring `index` node can handle positional
|
||||||
|
lookup without complicating core path syntax
|
||||||
|
|
||||||
|
Local path syntax:
|
||||||
|
|
||||||
|
- `LocalPath` uses the same strict segment rules as graph paths
|
||||||
|
- valid examples: `user.name`, `job.years`, `.`
|
||||||
|
- invalid examples: `user..name`, `user-name`, `items.0`, `user["name"]`
|
||||||
|
- no external path library is planned; a small parser with a compiled segment
|
||||||
|
regex is enough because these paths have workflow-specific roots and rules
|
||||||
|
|
||||||
|
Hashability:
|
||||||
|
|
||||||
|
- path objects should be immutable/hashable
|
||||||
|
- use frozen dataclasses or equivalent immutable models
|
||||||
|
- public node bindings should still be list-of-structs, not dicts keyed by paths
|
||||||
|
- hashable paths are useful internally for lookups, sets, duplicate detection,
|
||||||
|
and overlap validation
|
||||||
|
|
||||||
|
Implementation choice:
|
||||||
|
|
||||||
|
- use frozen dataclass value objects with Pydantic core-schema hooks
|
||||||
|
- avoid Pydantic `BaseModel` for tiny path values unless hooks become too costly
|
||||||
|
- Pydantic should accept strings or existing path objects and store path objects
|
||||||
|
- JSON serialization should emit strings
|
||||||
|
|
||||||
|
Planned core path value objects:
|
||||||
|
|
||||||
|
```text
|
||||||
|
LocalPath(parts)
|
||||||
|
GraphSourcePath(root, parts) # root is input | state | context
|
||||||
|
StatePath(parts) # serializes with state. prefix
|
||||||
|
```
|
||||||
|
|
||||||
|
Shared internals:
|
||||||
|
|
||||||
|
- use shared parsing/segment/overlap helpers for all path kinds
|
||||||
|
- do not expose one generic public `Path` for every position
|
||||||
|
- distinct public path types preserve semantics:
|
||||||
|
- `LocalPath` for node-local payloads
|
||||||
|
- `GraphSourcePath` for readable graph sources
|
||||||
|
- `StatePath` for writable state destinations
|
||||||
|
- a small shared `PathParts` value or shared helper functions are both fine;
|
||||||
|
choose whichever keeps implementation simplest
|
||||||
|
|
||||||
|
Constructors and authoring sugar:
|
||||||
|
|
||||||
|
- core path types should have boring constructors/parsers for tests and runtime
|
||||||
|
- `wf_authoring` keeps pretty helpers such as `state_path`, `input_path`,
|
||||||
|
`context_path`, and condition expression sugar
|
||||||
|
- `Expr` and `PathExpr` stay in `wf_authoring`
|
||||||
|
- core owns `Condition` models and path value types
|
||||||
|
- authoring wrappers can behave like a trait/mixin over comparable path-like
|
||||||
|
values, adding `eq`, `ne`, `lt`, `le`, `gt`, `ge`, operator overloads, and
|
||||||
|
`exists`
|
||||||
|
|
||||||
|
Core constructor ergonomics:
|
||||||
|
|
||||||
|
- constructors may accept dotted strings and/or multiple fragments
|
||||||
|
- fragments are flattened by splitting on `.`
|
||||||
|
- examples:
|
||||||
|
- `GraphSourcePath.state("person.name")`
|
||||||
|
- `GraphSourcePath.state("person", "name")`
|
||||||
|
- `GraphSourcePath.state("foo", "bar.baz")`
|
||||||
|
- all final segments still pass strict segment validation
|
||||||
|
- empty fragments/segments are rejected
|
||||||
|
- `LocalPath.root()` / serialized `"."` is the only root marker exception
|
||||||
|
|
||||||
|
Parse vs construction:
|
||||||
|
|
||||||
|
- `parse()` reads the full serialized JSON form
|
||||||
|
- `of()` / root-specific constructors are ergonomic Python construction
|
||||||
|
- both use the same segment parsing/validation internals
|
||||||
|
- examples:
|
||||||
|
- `StatePath.parse("state.person.name")`
|
||||||
|
- `StatePath.of("person.name")`
|
||||||
|
- `GraphSourcePath.state("person.name")`
|
||||||
|
- `LocalPath.of("user.name")`
|
||||||
|
|
||||||
|
String form:
|
||||||
|
|
||||||
|
- `str(path)` returns the serialized JSON form
|
||||||
|
- examples:
|
||||||
|
- `str(StatePath.of("person.name")) == "state.person.name"`
|
||||||
|
- `str(GraphSourcePath.state("person.name")) == "state.person.name"`
|
||||||
|
- `str(LocalPath.of("user.name")) == "user.name"`
|
||||||
|
- `str(LocalPath.root()) == "."`
|
||||||
|
|
||||||
|
JSON Schema:
|
||||||
|
|
||||||
|
- path fields should expose as strings, not `{root, parts}` objects
|
||||||
|
- use pattern and description metadata where possible
|
||||||
|
- examples:
|
||||||
|
- `StatePath`: string matching `state.<segment>(.<segment>)*`
|
||||||
|
- `GraphSourcePath`: string matching `(input|state|context).<segment>(.<segment>)*`
|
||||||
|
- `LocalPath`: string matching `.` or `<segment>(.<segment>)*`
|
||||||
|
- use Pydantic hook / annotation metadata magic to keep external schemas clear
|
||||||
|
while storing rich path value objects internally
|
||||||
|
|
||||||
|
Error messages:
|
||||||
|
|
||||||
|
- path parsing errors should name the expected path kind
|
||||||
|
- include examples in errors where practical
|
||||||
|
- expected examples:
|
||||||
|
- graph source path: `input.foo`, `state.foo`, or `context.foo`
|
||||||
|
- state path: `state.foo`
|
||||||
|
- local path: `user.name` or `.`
|
||||||
|
- this should improve current error + hint behavior for MCP/LLM users
|
||||||
|
|
||||||
|
Missing paths:
|
||||||
|
|
||||||
|
- binding path reads should fail when missing
|
||||||
|
- do not add `on_missing` to core bindings now
|
||||||
|
- optional/default/missing behavior belongs in explicit nodes or conditions
|
||||||
|
- this keeps bindings as data movement, not hidden behavior
|
||||||
|
|
||||||
|
Binding descriptions:
|
||||||
|
|
||||||
|
- no `description` / `desc` fields on `wf_core` binding structs for now
|
||||||
|
- descriptions are useful in higher layers such as drafts, artifacts, authoring
|
||||||
|
helpers, or MCP-facing planning surfaces
|
||||||
|
- core bindings should stay structural/runtime-focused
|
||||||
|
|
||||||
|
Binding order:
|
||||||
|
|
||||||
|
- binding order is preserved
|
||||||
|
- runtime may apply bindings in order for deterministic traces/debugging
|
||||||
|
- order must not resolve conflicts
|
||||||
|
- overlapping write targets are validation errors, not "last write wins"
|
||||||
|
- future priority/override behavior must be explicit, not implicit list order
|
||||||
|
|
||||||
|
Core explicitness:
|
||||||
|
|
||||||
|
- empty `NodeUse.input` means empty node payload
|
||||||
|
- empty `NodeUse.output` means no state writes
|
||||||
|
- core performs no auto-mapping
|
||||||
|
- authoring/builder layers may infer mappings, but must emit explicit canonical
|
||||||
|
bindings into core models
|
||||||
|
|
||||||
|
State schema fields:
|
||||||
|
|
||||||
|
- move toward list-of-structs instead of dict keys
|
||||||
|
- canonical shape:
|
||||||
|
|
||||||
|
```text
|
||||||
|
StateSchema.fields: list[StateFieldDecl]
|
||||||
|
|
||||||
|
StateFieldDecl:
|
||||||
|
path: StatePath
|
||||||
|
type: string
|
||||||
|
reducer: ReducerRef
|
||||||
|
```
|
||||||
|
|
||||||
|
- serialized field paths include `state.` prefix, e.g. `state.person.tags`
|
||||||
|
- accept old dict shape at parse time for compatibility:
|
||||||
|
|
||||||
|
```text
|
||||||
|
fields = {
|
||||||
|
"person.tags": {"type": "array", "reducer": "wf.std.append"}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- normalize old shape to canonical list internally
|
||||||
|
- canonical serialization emits list shape
|
||||||
|
- duplicate field paths are validation errors
|
||||||
|
- exact reducer matching uses exact `StatePath`
|
||||||
|
|
||||||
|
Input/output JSON Schemas:
|
||||||
|
|
||||||
|
- do not replace JSON Schema with a custom schema language
|
||||||
|
- introduce a named JSON Schema boundary type or model
|
||||||
|
- validate schemas with the standard `jsonschema` library
|
||||||
|
- validation cost is acceptable at model/config boundaries
|
||||||
|
- this applies to workflow input/output schemas and node input/output schemas
|
||||||
|
- JSON Schema object keys are standard and should not be treated like workflow
|
||||||
|
path-map dict keys
|
||||||
|
- respect `$schema` when present using `jsonschema.validators.validator_for`
|
||||||
|
- default to Draft 2020-12 when `$schema` is absent
|
||||||
|
- keep `SchemaRef` as the core schema type for now
|
||||||
|
- make `SchemaRef` honest: it represents a JSON Schema object boundary
|
||||||
|
- strengthen `SchemaRef` with standard JSON Schema validation instead of
|
||||||
|
introducing a parallel `JsonSchema` type
|
||||||
|
|
||||||
|
Placement:
|
||||||
|
|
||||||
|
- canonical path value objects live in `wf_core.paths`
|
||||||
|
- `wf_core.models.*`, validation, runtime, and authoring import those types
|
||||||
|
- split `wf_core.paths` into a package later only if it grows too large
|
||||||
|
|
||||||
|
Compatibility fields:
|
||||||
|
|
||||||
|
- old `in_map`, `input_values`, and `out_map` are parse-only compatibility
|
||||||
|
inputs
|
||||||
|
- they should be treated as deprecated
|
||||||
|
- validated `NodeUse` stores only canonical `input` and `output` bindings
|
||||||
|
- canonical serialization emits only the new fields
|
||||||
|
- avoid dual state inside the model
|
||||||
|
- reject payloads that mix canonical fields with deprecated compatibility fields
|
||||||
|
- do not merge old and new syntax
|
||||||
|
- compatibility conversion preserves dict insertion order when converting old
|
||||||
|
maps into binding lists
|
||||||
|
- JSON Schema should advertise only canonical `input` / `output` fields
|
||||||
|
- deprecated compatibility fields should not be shown to new callers
|
||||||
|
|
||||||
|
Null and missing semantics:
|
||||||
|
|
||||||
|
- explicit null is a real value, not an omitted binding/value
|
||||||
|
- `exists(path)` means the path can be resolved, even when its value is null
|
||||||
|
- missing path and present-null path are different states
|
||||||
|
- compare against null explicitly when needed, e.g. `state.foo == null` or
|
||||||
|
`state.foo != null`
|
||||||
|
- null comparisons are valid anywhere normal conditions work
|
||||||
|
- missing paths still fail clearly for comparisons; only `exists(path)` treats
|
||||||
|
missing as false instead of an error
|
||||||
|
- input value bindings may intentionally bind null; this must not be treated as
|
||||||
|
"no value provided"
|
||||||
|
- input path bindings fail before node execution when the source path is
|
||||||
|
missing
|
||||||
|
- missing source paths must not silently bind null
|
||||||
|
- softer/defaulting behavior belongs in authoring helpers or explicit shaping
|
||||||
|
nodes, not implicit core behavior
|
||||||
|
|
||||||
|
Schema openness:
|
||||||
|
|
||||||
|
- JSON Schema / Pydantic-style `extra` controls what object keys are allowed
|
||||||
|
inside a value
|
||||||
|
- schema openness does not change path existence semantics
|
||||||
|
- core should not use "allow extra" as permission for speculative deep path
|
||||||
|
traversal
|
||||||
|
- if a workflow needs dynamic traversal through arbitrary/extra object shape,
|
||||||
|
use an explicit node that receives the relevant state value and decides what
|
||||||
|
to extract
|
||||||
|
- this avoids pretending static path validation can prove paths such as
|
||||||
|
`state.person.occupations.1.title` exist inside open-ended objects
|
||||||
@@ -0,0 +1,925 @@
|
|||||||
|
# Core Path Bindings Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Replace loose core path/map strings with typed path objects and canonical list-of-struct node bindings while keeping deprecated shapes parse-compatible.
|
||||||
|
|
||||||
|
**Architecture:** Add immutable path value objects in `wf_core.paths`, then introduce canonical binding models in `wf_core.models.steps`. Runtime and validation move to the canonical bindings, while old `in_map`, `input_values`, `out_map`, and dict-shaped state fields are accepted only by model validators.
|
||||||
|
|
||||||
|
**Tech Stack:** Python 3.14, Pydantic v2, pytest, jsonschema, basedpyright, ruff.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
|
||||||
|
- Modify `src/wf_core/paths.py`: own typed graph/state/local path objects and graph path resolution helpers.
|
||||||
|
- Modify `src/wf_core/local_paths.py`: keep compatibility wrappers over `LocalPath` plus local get/set helpers.
|
||||||
|
- Modify `src/wf_core/models/steps.py`: add `InputPathBinding`, `InputValueBinding`, `OutputBinding`, and canonical `NodeUse.input` / `NodeUse.output`.
|
||||||
|
- Modify `src/wf_core/models/conditions.py`: type condition path operands with `GraphSourcePath`.
|
||||||
|
- Modify `src/wf_core/models/schemas.py`: harden `SchemaRef` and add canonical state field declarations.
|
||||||
|
- Modify `src/wf_core/runtime/ops/nodes.py`: resolve canonical node input bindings.
|
||||||
|
- Modify `src/wf_core/runtime/ops/state.py`: apply canonical output bindings through an atomic state patch.
|
||||||
|
- Modify `src/wf_core/runtime/ops/schemas.py`: expose focused JSON Schema validation helpers.
|
||||||
|
- Modify `src/wf_core/validation/steps.py`: validate canonical bindings and typed paths.
|
||||||
|
- Modify `src/wf_authoring/dsl/paths.py`: emit core path objects while preserving ergonomic helpers.
|
||||||
|
- Modify `src/wf_authoring/dsl/conditions.py`: compile authoring expressions to core typed condition models.
|
||||||
|
- Add `tests/core/test_path_values.py`: path parsing, serialization, JSON Schema, and error tests.
|
||||||
|
- Add `tests/core/test_canonical_node_bindings.py`: canonical model parsing and deprecated compatibility tests.
|
||||||
|
- Add `tests/core/test_atomic_state_patches.py`: output binding, reducer, overlap, and atomicity tests.
|
||||||
|
- Update existing `tests/core/test_mapping_validation.py`, `tests/core/test_nested_mappings.py`, `tests/core/test_nested_state_paths.py`, and authoring tests as needed.
|
||||||
|
|
||||||
|
## Task 1: Add Typed Path Values
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/wf_core/paths.py`
|
||||||
|
- Modify: `src/wf_core/local_paths.py`
|
||||||
|
- Create: `tests/core/test_path_values.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write path value tests**
|
||||||
|
|
||||||
|
Add tests for parsing, string serialization, equality/hashability, invalid segments, root-only graph source reads, and no bare write state:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import pytest
|
||||||
|
from pydantic import BaseModel, ValidationError
|
||||||
|
|
||||||
|
from wf_core.paths import GraphSourcePath, LocalPath, PathResolutionError, StatePath
|
||||||
|
|
||||||
|
|
||||||
|
def test_graph_source_path_accepts_root_and_nested_paths():
|
||||||
|
assert str(GraphSourcePath.parse("state")) == "state"
|
||||||
|
assert str(GraphSourcePath.parse("input.user")) == "input.user"
|
||||||
|
assert str(GraphSourcePath.context("loop_item")) == "context.loop_item"
|
||||||
|
|
||||||
|
|
||||||
|
def test_state_path_rejects_bare_state_write_target():
|
||||||
|
with pytest.raises(PathResolutionError, match="state path"):
|
||||||
|
StatePath.parse("state")
|
||||||
|
|
||||||
|
|
||||||
|
def test_local_path_supports_root_marker():
|
||||||
|
assert str(LocalPath.root()) == "."
|
||||||
|
assert str(LocalPath.of("user.name")) == "user.name"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("raw", ["", "state.", "state.items.0", "state.user-name"])
|
||||||
|
def test_paths_reject_invalid_segments(raw: str):
|
||||||
|
with pytest.raises(PathResolutionError):
|
||||||
|
GraphSourcePath.parse(raw)
|
||||||
|
|
||||||
|
|
||||||
|
def test_path_objects_are_hashable():
|
||||||
|
paths = {StatePath.of("person.name"), StatePath.of("person.name")}
|
||||||
|
assert len(paths) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_pydantic_accepts_path_strings_and_serializes_strings():
|
||||||
|
class Payload(BaseModel):
|
||||||
|
source: GraphSourcePath
|
||||||
|
target: StatePath
|
||||||
|
local: LocalPath
|
||||||
|
|
||||||
|
payload = Payload.model_validate(
|
||||||
|
{"source": "input.user", "target": "state.person", "local": "user"}
|
||||||
|
)
|
||||||
|
assert payload.source == GraphSourcePath.input("user")
|
||||||
|
assert payload.model_dump(mode="json")["target"] == "state.person"
|
||||||
|
|
||||||
|
|
||||||
|
def test_pydantic_rejects_bad_path_string():
|
||||||
|
class Payload(BaseModel):
|
||||||
|
source: GraphSourcePath
|
||||||
|
|
||||||
|
with pytest.raises(ValidationError):
|
||||||
|
Payload.model_validate({"source": "output.foo"})
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run path tests to verify they fail**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_path_values.py -q`
|
||||||
|
|
||||||
|
Expected: failures because `GraphSourcePath`, `StatePath`, and `LocalPath` classes do not exist or do not validate strictly.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement path value classes**
|
||||||
|
|
||||||
|
In `src/wf_core/paths.py`, add frozen dataclasses and shared parsing helpers. Keep existing helper function names as compatibility wrappers where practical.
|
||||||
|
|
||||||
|
Implementation shape:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from dataclasses import dataclass
|
||||||
|
import re
|
||||||
|
from typing import Any, ClassVar, Literal
|
||||||
|
|
||||||
|
SEGMENT_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class LocalPath:
|
||||||
|
"""Node-local payload path. `.` means the whole local payload."""
|
||||||
|
|
||||||
|
parts: tuple[str, ...]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def root(cls) -> "LocalPath":
|
||||||
|
return cls(())
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def of(cls, *fragments: str) -> "LocalPath":
|
||||||
|
return cls(_parse_fragments(*fragments, allow_empty=False))
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def parse(cls, raw: str) -> "LocalPath":
|
||||||
|
if raw == ".":
|
||||||
|
return cls.root()
|
||||||
|
return cls.of(raw)
|
||||||
|
|
||||||
|
def __str__(self) -> str:
|
||||||
|
return "." if not self.parts else ".".join(self.parts)
|
||||||
|
```
|
||||||
|
|
||||||
|
Also add:
|
||||||
|
|
||||||
|
```python
|
||||||
|
GraphRoot = Literal["input", "state", "context"]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GraphSourcePath:
|
||||||
|
"""Readable workflow graph path rooted at input, state, or context."""
|
||||||
|
|
||||||
|
root: GraphRoot
|
||||||
|
parts: tuple[str, ...] = ()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def parse(cls, raw: str) -> "GraphSourcePath": ...
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def input(cls, *fragments: str) -> "GraphSourcePath": ...
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def state(cls, *fragments: str) -> "GraphSourcePath": ...
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def context(cls, *fragments: str) -> "GraphSourcePath": ...
|
||||||
|
```
|
||||||
|
|
||||||
|
And:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class StatePath:
|
||||||
|
"""Writable workflow state path. Bare `state` is intentionally invalid."""
|
||||||
|
|
||||||
|
parts: tuple[str, ...]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def parse(cls, raw: str) -> "StatePath":
|
||||||
|
parsed = GraphSourcePath.parse(raw)
|
||||||
|
if parsed.root != "state" or not parsed.parts:
|
||||||
|
raise PathResolutionError("expected state path such as state.foo")
|
||||||
|
return cls(parsed.parts)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def of(cls, *fragments: str) -> "StatePath": ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Add Pydantic `__get_pydantic_core_schema__` and `__get_pydantic_json_schema__` hooks for each class so strings validate into objects and serialize back to strings.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Update local path wrappers**
|
||||||
|
|
||||||
|
In `src/wf_core/local_paths.py`, keep public functions but delegate parsing to `LocalPath.parse`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def split_local_path(path: str | LocalPath) -> list[str]:
|
||||||
|
"""Split one node-local path, accepting the new typed path object."""
|
||||||
|
parsed = path if isinstance(path, LocalPath) else LocalPath.parse(path)
|
||||||
|
return list(parsed.parts)
|
||||||
|
```
|
||||||
|
|
||||||
|
Update `paths_overlap` and `has_overlapping_paths` to accept `str | LocalPath`.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Run path tests**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_path_values.py -q`
|
||||||
|
|
||||||
|
Expected: all tests in `test_path_values.py` pass.
|
||||||
|
|
||||||
|
## Task 2: Add Canonical Node Binding Models
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/wf_core/models/steps.py`
|
||||||
|
- Test: `tests/core/test_canonical_node_bindings.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write canonical binding tests**
|
||||||
|
|
||||||
|
Create `tests/core/test_canonical_node_bindings.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import pytest
|
||||||
|
from pydantic import ValidationError
|
||||||
|
|
||||||
|
from wf_core.models.steps import NodeUse
|
||||||
|
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
|
||||||
|
|
||||||
|
|
||||||
|
def test_node_use_accepts_canonical_input_and_output_bindings():
|
||||||
|
node = NodeUse.model_validate(
|
||||||
|
{
|
||||||
|
"id": "echo",
|
||||||
|
"type": "node",
|
||||||
|
"node": "echo",
|
||||||
|
"input": [
|
||||||
|
{"target": "message", "path": "input.message"},
|
||||||
|
{"target": "mode", "value": None},
|
||||||
|
],
|
||||||
|
"output": [{"source": "echoed", "target": "state.echoed"}],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert node.input[0].target == LocalPath.of("message")
|
||||||
|
assert node.input[0].path == GraphSourcePath.input("message")
|
||||||
|
assert node.input[1].value is None
|
||||||
|
assert node.output[0].target == StatePath.of("echoed")
|
||||||
|
|
||||||
|
|
||||||
|
def test_node_use_converts_old_maps_to_canonical_bindings():
|
||||||
|
node = NodeUse.model_validate(
|
||||||
|
{
|
||||||
|
"id": "echo",
|
||||||
|
"type": "node",
|
||||||
|
"node": "echo",
|
||||||
|
"in_map": {"input.message": "message"},
|
||||||
|
"input_values": {"mode": "fast"},
|
||||||
|
"out_map": {"echoed": "state.echoed"},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
dumped = node.model_dump(mode="json")
|
||||||
|
assert "in_map" not in dumped
|
||||||
|
assert "input_values" not in dumped
|
||||||
|
assert "out_map" not in dumped
|
||||||
|
assert dumped["input"][0]["path"] == "input.message"
|
||||||
|
assert dumped["input"][1]["value"] == "fast"
|
||||||
|
assert dumped["output"][0]["target"] == "state.echoed"
|
||||||
|
|
||||||
|
|
||||||
|
def test_node_use_rejects_mixed_old_and_new_binding_styles():
|
||||||
|
with pytest.raises(ValidationError):
|
||||||
|
NodeUse.model_validate(
|
||||||
|
{
|
||||||
|
"id": "echo",
|
||||||
|
"type": "node",
|
||||||
|
"node": "echo",
|
||||||
|
"input": [{"target": "message", "path": "input.message"}],
|
||||||
|
"in_map": {"input.other": "other"},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_input_binding_rejects_path_and_value_together():
|
||||||
|
with pytest.raises(ValidationError):
|
||||||
|
NodeUse.model_validate(
|
||||||
|
{
|
||||||
|
"id": "bad",
|
||||||
|
"type": "node",
|
||||||
|
"node": "bad",
|
||||||
|
"input": [
|
||||||
|
{"target": "message", "path": "input.message", "value": "x"}
|
||||||
|
],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run binding tests to verify they fail**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_canonical_node_bindings.py -q`
|
||||||
|
|
||||||
|
Expected: failures because canonical binding fields do not exist yet.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement binding models**
|
||||||
|
|
||||||
|
In `src/wf_core/models/steps.py`, add:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||||
|
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
|
||||||
|
|
||||||
|
|
||||||
|
class InputPathBinding(BaseModel):
|
||||||
|
"""Map one graph source path into one node-local input path."""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
target: LocalPath
|
||||||
|
path: GraphSourcePath
|
||||||
|
|
||||||
|
|
||||||
|
class InputValueBinding(BaseModel):
|
||||||
|
"""Map one static JSON-compatible value into one node-local input path."""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
target: LocalPath
|
||||||
|
value: object
|
||||||
|
|
||||||
|
|
||||||
|
InputBinding = Annotated[
|
||||||
|
InputPathBinding | InputValueBinding,
|
||||||
|
Field(union_mode="left_to_right"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class OutputBinding(BaseModel):
|
||||||
|
"""Map one node-local output path into one workflow state path."""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
source: LocalPath
|
||||||
|
target: StatePath
|
||||||
|
```
|
||||||
|
|
||||||
|
Update `NodeUse`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class NodeUse(BaseModel):
|
||||||
|
...
|
||||||
|
input: list[InputBinding] = Field(default_factory=list)
|
||||||
|
output: list[OutputBinding] = Field(default_factory=list)
|
||||||
|
|
||||||
|
@model_validator(mode="before")
|
||||||
|
@classmethod
|
||||||
|
def _coerce_deprecated_maps(cls, data: object) -> object:
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
The validator should:
|
||||||
|
|
||||||
|
- If `input` or `output` is present, reject any of `in_map`, `input_values`, `out_map`.
|
||||||
|
- Convert `input_values` entries to `{"target": key, "value": value}` preserving order.
|
||||||
|
- Convert `in_map` entries to `{"target": destination, "path": source}` preserving order.
|
||||||
|
- Convert `out_map` entries to `{"source": source, "target": destination}` preserving order.
|
||||||
|
- Remove old keys from the normalized data.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run binding tests**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_canonical_node_bindings.py -q`
|
||||||
|
|
||||||
|
Expected: all tests in `test_canonical_node_bindings.py` pass.
|
||||||
|
|
||||||
|
## Task 3: Move Runtime Node Input Resolution To Canonical Bindings
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||||
|
- Test: `tests/core/test_nested_mappings.py`
|
||||||
|
- Test: `tests/core/test_canonical_node_bindings.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add runtime tests for canonical input binding behavior**
|
||||||
|
|
||||||
|
In `tests/core/test_nested_mappings.py`, add a test that builds the existing minimal workflow style but uses `input` / `output` instead of old maps:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def test_canonical_bindings_resolve_input_values_and_paths():
|
||||||
|
workflow = Workflow.model_validate(
|
||||||
|
{
|
||||||
|
"name": "canonical",
|
||||||
|
"input_schema": {"type": "object", "properties": {"message": {"type": "string"}}},
|
||||||
|
"state_schema": {"fields": {"echoed": {"type": "string"}}},
|
||||||
|
"output_schema": {"type": "object", "properties": {"echoed": {"type": "string"}}},
|
||||||
|
"start": "echo",
|
||||||
|
"node_defs": [
|
||||||
|
{
|
||||||
|
"name": "echo",
|
||||||
|
"input_schema": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"message": {"type": "string"}, "mode": {"type": "string"}},
|
||||||
|
"required": ["message", "mode"],
|
||||||
|
},
|
||||||
|
"output_schema": {"type": "object", "properties": {"echoed": {"type": "string"}}},
|
||||||
|
"outcomes": ["ok"],
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"nodes": [
|
||||||
|
{
|
||||||
|
"id": "echo",
|
||||||
|
"type": "node",
|
||||||
|
"node": "echo",
|
||||||
|
"input": [
|
||||||
|
{"target": "message", "path": "input.message"},
|
||||||
|
{"target": "mode", "value": "fast"},
|
||||||
|
],
|
||||||
|
"output": [{"source": "echoed", "target": "state.echoed"}],
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"edges": [{"from": "echo", "outcome": "ok", "to": "__end__"}],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
result = execute_workflow(
|
||||||
|
workflow,
|
||||||
|
{"message": "hi"},
|
||||||
|
registry={"echo": lambda payload, _ctx: {"echoed": f"{payload['mode']}:{payload['message']}"}},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.output["echoed"] == "fast:hi"
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run the focused test to verify failure**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_nested_mappings.py::test_canonical_bindings_resolve_input_values_and_paths -q`
|
||||||
|
|
||||||
|
Expected: failure because runtime still reads `node.input_values`, `node.in_map`, and `node.out_map`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Update `_resolve_node_execution`**
|
||||||
|
|
||||||
|
In `src/wf_core/runtime/ops/nodes.py`, import binding classes and use `node.input`.
|
||||||
|
|
||||||
|
Implementation shape:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from wf_core.models.steps import InputPathBinding, InputValueBinding
|
||||||
|
|
||||||
|
|
||||||
|
for binding in node.input:
|
||||||
|
if isinstance(binding, InputValueBinding):
|
||||||
|
value = binding.value
|
||||||
|
else:
|
||||||
|
value = safe_resolve_path(
|
||||||
|
str(binding.path),
|
||||||
|
state=run.state,
|
||||||
|
workflow_input=run.workflow_input,
|
||||||
|
context=context_values,
|
||||||
|
)
|
||||||
|
set_local_value(resolved_input, binding.target, value)
|
||||||
|
```
|
||||||
|
|
||||||
|
`set_local_value` should accept `LocalPath` after Task 1.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run canonical runtime test**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_nested_mappings.py::test_canonical_bindings_resolve_input_values_and_paths -q`
|
||||||
|
|
||||||
|
Expected: pass.
|
||||||
|
|
||||||
|
## Task 4: Move Runtime Output Writes To Canonical Bindings And Atomic Patches
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||||
|
- Modify: `src/wf_core/runtime/ops/nodes.py`
|
||||||
|
- Test: `tests/core/test_atomic_state_patches.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write atomic patch tests**
|
||||||
|
|
||||||
|
Create `tests/core/test_atomic_state_patches.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from wf_core.errors import WorkflowExecutionError
|
||||||
|
from wf_core.models.workflow import Workflow
|
||||||
|
from wf_core.runtime.ops.state import apply_output_bindings
|
||||||
|
|
||||||
|
|
||||||
|
def _workflow() -> Workflow:
|
||||||
|
return Workflow.model_validate(
|
||||||
|
{
|
||||||
|
"name": "patch",
|
||||||
|
"input_schema": {"type": "object", "properties": {}},
|
||||||
|
"state_schema": {
|
||||||
|
"fields": {
|
||||||
|
"person": {"type": "object"},
|
||||||
|
"person.name": {"type": "string"},
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output_schema": {"type": "object", "properties": {}},
|
||||||
|
"start": "n",
|
||||||
|
"nodes": [],
|
||||||
|
"edges": [],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_output_bindings_commit_patch_atomically():
|
||||||
|
workflow = _workflow()
|
||||||
|
state = {"person": {"name": "old"}}
|
||||||
|
|
||||||
|
with pytest.raises(WorkflowExecutionError):
|
||||||
|
apply_output_bindings(
|
||||||
|
workflow,
|
||||||
|
[
|
||||||
|
{"source": "person.name", "target": "state.person.name"},
|
||||||
|
{"source": "missing", "target": "state.person.extra"},
|
||||||
|
],
|
||||||
|
{"person": {"name": "new"}},
|
||||||
|
state,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert state["person"]["name"] == "old"
|
||||||
|
|
||||||
|
|
||||||
|
def test_output_bindings_reject_overlapping_write_targets():
|
||||||
|
workflow = _workflow()
|
||||||
|
state = {}
|
||||||
|
|
||||||
|
with pytest.raises(WorkflowExecutionError, match="overlapping"):
|
||||||
|
apply_output_bindings(
|
||||||
|
workflow,
|
||||||
|
[
|
||||||
|
{"source": "person", "target": "state.person"},
|
||||||
|
{"source": "person.name", "target": "state.person.name"},
|
||||||
|
],
|
||||||
|
{"person": {"name": "Ada"}},
|
||||||
|
state,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run atomic patch tests to verify failure**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_atomic_state_patches.py -q`
|
||||||
|
|
||||||
|
Expected: failure because `apply_output_bindings` does not exist.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement `apply_output_bindings`**
|
||||||
|
|
||||||
|
In `src/wf_core/runtime/ops/state.py`, add a canonical function:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from wf_core.models.steps import OutputBinding
|
||||||
|
from wf_core.paths import StatePath
|
||||||
|
|
||||||
|
|
||||||
|
def apply_output_bindings(
|
||||||
|
workflow: Workflow,
|
||||||
|
bindings: Sequence[OutputBinding],
|
||||||
|
node_output: dict[str, Any],
|
||||||
|
state: dict[str, Any],
|
||||||
|
reducers: Mapping[str, ReducerDefinition] | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Prepare and commit one atomic state patch from canonical output bindings."""
|
||||||
|
```
|
||||||
|
|
||||||
|
Function behavior:
|
||||||
|
|
||||||
|
- Validate no overlapping `binding.target`.
|
||||||
|
- Resolve every `binding.source` from `node_output` first.
|
||||||
|
- Build a prepared patch keyed by `StatePath`.
|
||||||
|
- Compute reducers into prepared merged values without mutating `state`.
|
||||||
|
- Commit all prepared values only after all prior steps succeed.
|
||||||
|
- Return JSON-friendly `dict[str, Any]` state changes using `str(path)` keys for now, until trace is separately migrated.
|
||||||
|
|
||||||
|
Keep `apply_output_map` as a compatibility wrapper that converts old map entries into `OutputBinding` and calls `apply_output_bindings`.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Update node finalization**
|
||||||
|
|
||||||
|
In `src/wf_core/runtime/ops/nodes.py`, call `apply_output_bindings(workflow, node.output, result.output, run.state, reducers=reducers)` instead of `apply_output_map(...)`.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Run state patch tests**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_atomic_state_patches.py tests/core/test_nested_mappings.py -q`
|
||||||
|
|
||||||
|
Expected: pass.
|
||||||
|
|
||||||
|
## Task 5: Update Validation For Canonical Bindings
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/wf_core/validation/steps.py`
|
||||||
|
- Test: `tests/core/test_mapping_validation.py`
|
||||||
|
- Test: `tests/core/test_canonical_node_bindings.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add validation tests for canonical fields**
|
||||||
|
|
||||||
|
In `tests/core/test_mapping_validation.py`, add tests for invalid source paths, invalid destination paths, overlapping local input targets, and overlapping state output targets using canonical `input` / `output`.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def test_validate_workflow_reports_overlapping_canonical_output_targets():
|
||||||
|
workflow = workflow_with_node(
|
||||||
|
node_use={
|
||||||
|
"id": "n",
|
||||||
|
"type": "node",
|
||||||
|
"node": "n",
|
||||||
|
"output": [
|
||||||
|
{"source": "person", "target": "state.person"},
|
||||||
|
{"source": "person.name", "target": "state.person.name"},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
report = workflow.validate_structure()
|
||||||
|
|
||||||
|
assert any(issue.code == ValidationIssueCode.INVALID_DESTINATION_PATH for issue in report.issues)
|
||||||
|
```
|
||||||
|
|
||||||
|
Use the existing helper style in `tests/core/test_mapping_validation.py` rather than inventing a second full workflow factory if one already exists.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run mapping validation tests**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_mapping_validation.py -q`
|
||||||
|
|
||||||
|
Expected: new canonical validation tests fail until validation reads `node.input` / `node.output`.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Update `validate_node_use`**
|
||||||
|
|
||||||
|
In `src/wf_core/validation/steps.py`:
|
||||||
|
|
||||||
|
- Iterate `node.input`.
|
||||||
|
- For `InputValueBinding`, validate target local root against node input schema.
|
||||||
|
- For `InputPathBinding`, validate target and source graph path.
|
||||||
|
- Iterate `node.output`.
|
||||||
|
- Validate output source local root against node output schema.
|
||||||
|
- Validate destination `StatePath`.
|
||||||
|
- Use typed overlap helpers instead of raw map values.
|
||||||
|
- Keep issue paths readable, e.g. `nodes[0].input[1].target`.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run validation tests**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_mapping_validation.py tests/core/test_canonical_node_bindings.py -q`
|
||||||
|
|
||||||
|
Expected: pass.
|
||||||
|
|
||||||
|
## Task 6: Add Canonical State Schema Fields
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/wf_core/models/schemas.py`
|
||||||
|
- Modify: `src/wf_core/runtime/ops/state.py`
|
||||||
|
- Modify: `src/wf_core/validation/steps.py`
|
||||||
|
- Test: `tests/core/test_nested_state_paths.py`
|
||||||
|
- Test: `tests/core/test_schema_validation.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write state schema canonical shape tests**
|
||||||
|
|
||||||
|
In `tests/core/test_nested_state_paths.py`, add:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from wf_core.models.schemas import StateSchema
|
||||||
|
from wf_core.paths import StatePath
|
||||||
|
|
||||||
|
|
||||||
|
def test_state_schema_accepts_canonical_field_list():
|
||||||
|
schema = StateSchema.model_validate(
|
||||||
|
{
|
||||||
|
"fields": [
|
||||||
|
{"path": "state.person", "type": "object"},
|
||||||
|
{"path": "state.person.name", "type": "string", "reducer": "wf.std.replace"},
|
||||||
|
]
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert schema.fields[0].path == StatePath.of("person")
|
||||||
|
assert schema.field_map()["person.name"].type == "string"
|
||||||
|
|
||||||
|
|
||||||
|
def test_state_schema_accepts_deprecated_dict_shape():
|
||||||
|
schema = StateSchema.model_validate(
|
||||||
|
{"fields": {"person.name": {"type": "string"}}}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert schema.model_dump(mode="json")["fields"][0]["path"] == "state.person.name"
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run state schema tests to verify failure**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py -q`
|
||||||
|
|
||||||
|
Expected: failure because `StateSchema.fields` is still a dict.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement canonical `StateFieldDecl`**
|
||||||
|
|
||||||
|
In `src/wf_core/models/schemas.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class StateFieldDecl(BaseModel):
|
||||||
|
"""One declared state path plus validation and reducer metadata."""
|
||||||
|
|
||||||
|
path: StatePath
|
||||||
|
schema: SchemaRef = Field(default_factory=lambda: SchemaRef(type="object"))
|
||||||
|
reducer: ReducerRef = Field(default_factory=lambda: ReducerRef(name="wf.std.replace"))
|
||||||
|
trace: bool = True
|
||||||
|
default: Any = None
|
||||||
|
```
|
||||||
|
|
||||||
|
Preserve compatibility for old `type` directly on the field:
|
||||||
|
|
||||||
|
- For old dict values like `{"type": "string"}`, convert to `{"schema": {"type": "string"}}`.
|
||||||
|
- For canonical values, allow either `schema` or simple `type` as input if that keeps existing tests stable.
|
||||||
|
|
||||||
|
Update `StateSchema`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class StateSchema(BaseModel):
|
||||||
|
fields: list[StateFieldDecl] = Field(default_factory=list)
|
||||||
|
|
||||||
|
def field_map(self) -> dict[str, StateFieldDecl]:
|
||||||
|
return {".".join(field.path.parts): field for field in self.fields}
|
||||||
|
```
|
||||||
|
|
||||||
|
Add a model validator to accept old dict shape and normalize to list.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Update callers of `workflow.state_schema.fields`**
|
||||||
|
|
||||||
|
Search: `rg 'state_schema\\.fields|\\.fields\\.get|set\\(workflow\\.state_schema\\.fields\\)' src tests`
|
||||||
|
|
||||||
|
Update code to use `workflow.state_schema.field_map()` when it needs lookup by rootless path.
|
||||||
|
|
||||||
|
Important updates:
|
||||||
|
|
||||||
|
- `src/wf_core/runtime/ops/state.py`
|
||||||
|
- `src/wf_core/validation/steps.py`
|
||||||
|
- any authoring or artifact code constructing state field maps.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Run state schema tests**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py -q`
|
||||||
|
|
||||||
|
Expected: pass.
|
||||||
|
|
||||||
|
## Task 7: Harden SchemaRef With JSON Schema Validation
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/wf_core/models/schemas.py`
|
||||||
|
- Modify: `src/wf_core/runtime/ops/schemas.py`
|
||||||
|
- Test: `tests/core/test_schema_validation.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add schema validation tests**
|
||||||
|
|
||||||
|
In `tests/core/test_schema_validation.py`, add tests:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import pytest
|
||||||
|
from pydantic import ValidationError
|
||||||
|
|
||||||
|
from wf_core.models.schemas import SchemaRef
|
||||||
|
|
||||||
|
|
||||||
|
def test_schema_ref_accepts_valid_json_schema_with_defs():
|
||||||
|
schema = SchemaRef.model_validate(
|
||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"type": "object",
|
||||||
|
"$defs": {"Name": {"type": "string"}},
|
||||||
|
"properties": {"name": {"$ref": "#/$defs/Name"}},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert schema.model_extra["$defs"]["Name"]["type"] == "string"
|
||||||
|
|
||||||
|
|
||||||
|
def test_schema_ref_rejects_invalid_json_schema():
|
||||||
|
with pytest.raises(ValidationError):
|
||||||
|
SchemaRef.model_validate({"type": 123})
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run schema tests to verify failure**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_schema_validation.py -q`
|
||||||
|
|
||||||
|
Expected: invalid schema is currently accepted.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Add `jsonschema` validation**
|
||||||
|
|
||||||
|
In `src/wf_core/models/schemas.py`, import:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from jsonschema import SchemaError
|
||||||
|
from jsonschema.validators import Draft202012Validator, validator_for
|
||||||
|
from pydantic import model_validator
|
||||||
|
```
|
||||||
|
|
||||||
|
Add an after validator to `SchemaRef`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def _validate_json_schema(self) -> "SchemaRef":
|
||||||
|
raw = self.model_dump(mode="python", exclude_none=True)
|
||||||
|
validator_cls = validator_for(raw, default=Draft202012Validator)
|
||||||
|
try:
|
||||||
|
validator_cls.check_schema(raw)
|
||||||
|
except SchemaError as exc:
|
||||||
|
raise ValueError(f"invalid JSON Schema: {exc.message}") from exc
|
||||||
|
return self
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run schema tests**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core/test_schema_validation.py -q`
|
||||||
|
|
||||||
|
Expected: pass.
|
||||||
|
|
||||||
|
## Task 8: Update Authoring Helpers To Emit Canonical Bindings
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/wf_authoring/dsl/paths.py`
|
||||||
|
- Modify: `src/wf_authoring/dsl/conditions.py`
|
||||||
|
- Modify: `src/wf_authoring/builder/core.py`
|
||||||
|
- Test: `tests/authoring/test_builder.py`
|
||||||
|
- Test: `tests/authoring/test_conditions.py`
|
||||||
|
- Test: `tests/authoring/test_control_flow_examples.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add authoring tests for canonical dumps**
|
||||||
|
|
||||||
|
In `tests/authoring/test_builder.py`, add a test that builds a workflow and asserts the dumped node uses canonical `input` / `output`, not old maps:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def test_builder_emits_canonical_node_bindings():
|
||||||
|
workflow = (
|
||||||
|
WorkflowBuilder("canonical")
|
||||||
|
.schemas(
|
||||||
|
input_schema={"type": "object", "properties": {"message": {"type": "string"}}},
|
||||||
|
state_schema={"fields": {"echoed": {"type": "string"}}},
|
||||||
|
output_schema={"type": "object", "properties": {"echoed": {"type": "string"}}},
|
||||||
|
)
|
||||||
|
.use(echo_node, id="echo", in_map={"input.message": "message"}, out_map={"echoed": "state.echoed"})
|
||||||
|
.start_at("echo")
|
||||||
|
.end("echo", "ok")
|
||||||
|
.build()
|
||||||
|
)
|
||||||
|
|
||||||
|
dumped_node = workflow.model_dump(mode="json")["nodes"][0]
|
||||||
|
assert "input" in dumped_node
|
||||||
|
assert "output" in dumped_node
|
||||||
|
assert "in_map" not in dumped_node
|
||||||
|
assert "out_map" not in dumped_node
|
||||||
|
```
|
||||||
|
|
||||||
|
Adapt helper names to the current builder API in the file.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run authoring builder tests**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/authoring/test_builder.py tests/authoring/test_conditions.py -q`
|
||||||
|
|
||||||
|
Expected: new canonical dump test may fail until builder emits or model normalizes canonical shapes.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Update path/condition authoring wrappers**
|
||||||
|
|
||||||
|
In `src/wf_authoring/dsl/paths.py`, make ergonomic helpers return wrappers around core path values or values accepted by core models. Preserve existing public behavior where possible:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def state_path(*parts: str) -> GraphPath:
|
||||||
|
return GraphPath(str(GraphSourcePath.state(*parts)))
|
||||||
|
```
|
||||||
|
|
||||||
|
In `src/wf_authoring/dsl/conditions.py`, make `PathExpr` compile using `GraphSourcePath.parse` for `PathOperand`.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Update builder to rely on canonical model normalization**
|
||||||
|
|
||||||
|
In `src/wf_authoring/builder/core.py`, either emit canonical binding dicts directly or keep passing old maps into `NodeUse.model_validate`. Prefer direct canonical emission where the builder already has enough structure.
|
||||||
|
|
||||||
|
Do not remove user-facing `in_map` / `out_map` builder parameters in this pass.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Run authoring tests**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/authoring -q`
|
||||||
|
|
||||||
|
Expected: authoring tests pass.
|
||||||
|
|
||||||
|
## Task 9: Full Compatibility And Regression Pass
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify docs/examples only if tests show stale serialized shapes.
|
||||||
|
- Test: full repo.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Run core tests**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/core tests/authoring tests/rewrite -q`
|
||||||
|
|
||||||
|
Expected: pass.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run artifact and MCP workflow-surface tests**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest tests/artifacts tests/wf_mcp/test_workflow_surface.py tests/wf_mcp/test_workflow_wrappers.py tests/wf_mcp/test_mcp_workflow_surface_example.py -q`
|
||||||
|
|
||||||
|
Expected: pass.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Run full test suite**
|
||||||
|
|
||||||
|
Run: `uv run --with pytest pytest -q`
|
||||||
|
|
||||||
|
Expected: pass, allowing any existing intentionally skipped environment-dependent tests.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run static checks**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uvx ruff check
|
||||||
|
uv run basedpyright --level error
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: ruff passes and basedpyright reports 0 errors.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Format touched files**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uvx ruff format src/wf_core src/wf_authoring tests/core tests/authoring
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: files format cleanly.
|
||||||
|
|
||||||
|
## Self-Review Notes
|
||||||
|
|
||||||
|
- Spec coverage: typed paths, canonical bindings, parse-only compatibility, null/missing semantics, dynamic traversal deferral, state patch atomicity, reducer behavior, JSON Schema validation, authoring updates, and tracing shape are covered. Full trace migration is intentionally not implemented beyond returning string-keyed `state_changes` for compatibility.
|
||||||
|
- Placeholder scan: this plan avoids `TBD` and names concrete files, tests, commands, and behavior.
|
||||||
|
- Type consistency: `LocalPath`, `GraphSourcePath`, `StatePath`, `InputPathBinding`, `InputValueBinding`, `OutputBinding`, and `StateFieldDecl` are introduced before later tasks use them.
|
||||||
@@ -0,0 +1,331 @@
|
|||||||
|
# Core Path Bindings Design
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
`wf_core` currently uses plain strings for graph paths, local node paths, input
|
||||||
|
maps, output maps, and state field keys. That made the early system simple, but
|
||||||
|
it also pushes too much meaning into ad hoc string parsing. This design makes
|
||||||
|
paths and bindings first-class core concepts while keeping JSON serialization
|
||||||
|
simple.
|
||||||
|
|
||||||
|
The goal is not to make every dynamic JSON traversal statically provable. The
|
||||||
|
goal is to make normal workflow data movement explicit, validated, serializable,
|
||||||
|
and easy for authoring/MCP layers to generate.
|
||||||
|
|
||||||
|
## Goals
|
||||||
|
|
||||||
|
- Replace loose map fields with canonical list-of-struct binding models.
|
||||||
|
- Store typed path value objects internally while serializing them as strings.
|
||||||
|
- Keep missing values distinct from explicit `null`.
|
||||||
|
- Make state writes atomic and reducer-aware.
|
||||||
|
- Keep compatibility with old workflow shapes through parse-only deprecated
|
||||||
|
fields.
|
||||||
|
- Keep dynamic/open-object traversal out of core and in explicit nodes.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- No list/index path syntax such as `items.0.name` or `items[0].name`.
|
||||||
|
- No arbitrary JSON-pointer support.
|
||||||
|
- No whole-state replacement writes in this pass.
|
||||||
|
- No implicit defaults for missing paths.
|
||||||
|
- No business logic in reducers or bindings.
|
||||||
|
- No custom schema language replacing JSON Schema.
|
||||||
|
|
||||||
|
## Path Types
|
||||||
|
|
||||||
|
Core introduces distinct path value objects:
|
||||||
|
|
||||||
|
```text
|
||||||
|
LocalPath(parts)
|
||||||
|
GraphSourcePath(root, parts) # root: input | state | context
|
||||||
|
StatePath(parts) # serializes as state.<parts>
|
||||||
|
```
|
||||||
|
|
||||||
|
These objects are immutable/hashable and are accepted by Pydantic from either
|
||||||
|
strings or existing instances. JSON serialization emits strings.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
```text
|
||||||
|
LocalPath.of("user.name") -> "user.name"
|
||||||
|
LocalPath.root() -> "."
|
||||||
|
GraphSourcePath.state("person.name") -> "state.person.name"
|
||||||
|
GraphSourcePath.parse("input") -> "input"
|
||||||
|
StatePath.of("person.name") -> "state.person.name"
|
||||||
|
```
|
||||||
|
|
||||||
|
Path parsing rules:
|
||||||
|
|
||||||
|
- Segments use `[A-Za-z_][A-Za-z0-9_]*`.
|
||||||
|
- Dots separate segments.
|
||||||
|
- Empty segments are invalid.
|
||||||
|
- Numeric/positional list segments are rejected.
|
||||||
|
- `LocalPath.root()` / `"."` is the only local root marker.
|
||||||
|
- Root-only graph source paths `input`, `state`, and `context` are valid reads.
|
||||||
|
|
||||||
|
`StatePath` write targets should not accept bare `state` in this pass. Whole
|
||||||
|
state replacement is too broad because it interacts with reducers, validation,
|
||||||
|
trace, and accidental deletion.
|
||||||
|
|
||||||
|
There is no reducer for bare `state`. Reducers attach only to declared non-root
|
||||||
|
`StatePath` fields.
|
||||||
|
|
||||||
|
## Canonical Node Bindings
|
||||||
|
|
||||||
|
`NodeUse` gets canonical binding fields:
|
||||||
|
|
||||||
|
```text
|
||||||
|
NodeUse.input: list[InputBinding]
|
||||||
|
NodeUse.output: list[OutputBinding]
|
||||||
|
```
|
||||||
|
|
||||||
|
Input bindings are distinguished by shape, not by an extra `kind` field:
|
||||||
|
|
||||||
|
```text
|
||||||
|
InputPathBinding:
|
||||||
|
target: LocalPath
|
||||||
|
path: GraphSourcePath
|
||||||
|
|
||||||
|
InputValueBinding:
|
||||||
|
target: LocalPath
|
||||||
|
value: JsonValue
|
||||||
|
```
|
||||||
|
|
||||||
|
Output bindings are:
|
||||||
|
|
||||||
|
```text
|
||||||
|
OutputBinding:
|
||||||
|
source: LocalPath
|
||||||
|
target: StatePath
|
||||||
|
```
|
||||||
|
|
||||||
|
The same field name can mean different path kinds by position. For example,
|
||||||
|
input binding `target` is node-local, while output binding `target` is workflow
|
||||||
|
state. Documentation and model field descriptions should make this explicit.
|
||||||
|
|
||||||
|
Root local path `"."` is valid:
|
||||||
|
|
||||||
|
- input target `"."` means the whole node input payload.
|
||||||
|
- output source `"."` means the whole node output payload.
|
||||||
|
- a `"."` input binding must be the only input binding.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"input": [
|
||||||
|
{"target": "user.email", "path": "state.person.email"},
|
||||||
|
{"target": "mode", "value": "fast"}
|
||||||
|
],
|
||||||
|
"output": [
|
||||||
|
{"source": "result", "target": "state.result"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Whole payload input:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"input": [
|
||||||
|
{"target": ".", "path": "state.rates"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Whole payload literal input:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"input": [
|
||||||
|
{"target": ".", "value": {"mode": "fast"}}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Compatibility
|
||||||
|
|
||||||
|
Old fields are accepted only as deprecated parse inputs:
|
||||||
|
|
||||||
|
```text
|
||||||
|
in_map
|
||||||
|
input_values
|
||||||
|
out_map
|
||||||
|
```
|
||||||
|
|
||||||
|
After validation, `NodeUse` stores only canonical `input` and `output`
|
||||||
|
bindings. Canonical serialization emits only the new fields. Payloads that mix
|
||||||
|
canonical fields with deprecated fields should fail instead of merging two
|
||||||
|
styles.
|
||||||
|
|
||||||
|
This shape makes it easy to remove compatibility later: delete the parser
|
||||||
|
adapters without changing runtime internals.
|
||||||
|
|
||||||
|
State schema gets the same compatibility shape. The canonical form is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
StateSchema.fields: list[StateFieldDecl]
|
||||||
|
|
||||||
|
StateFieldDecl:
|
||||||
|
path: StatePath
|
||||||
|
schema: SchemaRef
|
||||||
|
reducer: ReducerRef
|
||||||
|
```
|
||||||
|
|
||||||
|
Old dict-shaped fields can be accepted at parse time and normalized:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"fields": {
|
||||||
|
"person.tags": {"type": "array", "reducer": "wf.std.append"}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Canonical serialization should emit list-of-structs with serialized state paths.
|
||||||
|
|
||||||
|
## Runtime Semantics
|
||||||
|
|
||||||
|
Node execution flow:
|
||||||
|
|
||||||
|
1. Validate workflow input against `workflow.input_schema`.
|
||||||
|
2. Resolve node input bindings into the node payload.
|
||||||
|
3. Validate node payload against `node_def.input_schema`.
|
||||||
|
4. Execute the node.
|
||||||
|
5. Coerce the node result.
|
||||||
|
6. Validate node output against `node_def.output_schema`.
|
||||||
|
7. Prepare an atomic state patch from output bindings.
|
||||||
|
8. Validate focused state patch values.
|
||||||
|
9. Commit the patch.
|
||||||
|
10. At `END`, project final output from top-level state fields using
|
||||||
|
`workflow.output_schema.properties`.
|
||||||
|
|
||||||
|
State writes are atomic. Runtime should resolve all output sources, detect
|
||||||
|
overlap conflicts, compute reducer results, validate patch values, then commit.
|
||||||
|
If any step fails, prior state is unchanged.
|
||||||
|
|
||||||
|
Reducers run during patch preparation. They receive current value, incoming
|
||||||
|
value, and optional config, then return the merged value. Reducers must not
|
||||||
|
mutate workflow state directly.
|
||||||
|
|
||||||
|
## Missing, Null, And Dynamic Data
|
||||||
|
|
||||||
|
Explicit `null` is a real value. It is not the same as a missing path.
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- `exists(path)` returns true when the path resolves, even if the value is null.
|
||||||
|
- `exists(path)` returns false when the path is missing.
|
||||||
|
- Comparisons such as `eq(null)` and `ne(null)` are valid.
|
||||||
|
- Comparisons against missing paths fail clearly.
|
||||||
|
- Input path bindings fail before node execution when the source path is
|
||||||
|
missing.
|
||||||
|
- Missing source paths must not silently bind null.
|
||||||
|
- Input value bindings may intentionally bind null.
|
||||||
|
|
||||||
|
`allow extra` / open object schemas do not authorize speculative deep traversal.
|
||||||
|
If a workflow needs dynamic object traversal, it should use an explicit node
|
||||||
|
that receives the relevant value and decides what to extract. For example, use
|
||||||
|
an `extract_title` node instead of trying to make core prove
|
||||||
|
`state.person.occupations.1.title`.
|
||||||
|
|
||||||
|
## Validation Rules
|
||||||
|
|
||||||
|
Path value objects validate syntax and path kind only. Workflow validation checks
|
||||||
|
whether a path is legal in a particular workflow.
|
||||||
|
|
||||||
|
Examples of workflow-specific checks:
|
||||||
|
|
||||||
|
- `input.foo` exists in `workflow.input_schema` when statically knowable.
|
||||||
|
- `state.foo` has a declared state root.
|
||||||
|
- write destinations are `StatePath`.
|
||||||
|
- output write targets do not overlap.
|
||||||
|
|
||||||
|
Overlap rules:
|
||||||
|
|
||||||
|
- read paths may overlap.
|
||||||
|
- input targets must not overlap.
|
||||||
|
- output targets must not overlap.
|
||||||
|
- exact duplicate targets are invalid.
|
||||||
|
- parent/child write targets such as `state.person` and `state.person.name`
|
||||||
|
are invalid together.
|
||||||
|
|
||||||
|
State write validation:
|
||||||
|
|
||||||
|
- Validate against the exact declared state field schema when one exists.
|
||||||
|
- Do not validate the whole workflow state on every write.
|
||||||
|
- Replacement writes must satisfy the full target schema.
|
||||||
|
- Replacement writes with forbidden extra fields fail.
|
||||||
|
- Partial object patches require an explicit merge-style reducer such as
|
||||||
|
`merge_object`.
|
||||||
|
- Do not silently treat `replace` as merge.
|
||||||
|
|
||||||
|
## JSON Schema Boundary
|
||||||
|
|
||||||
|
Core should not invent a schema language. `SchemaRef` should represent a JSON
|
||||||
|
Schema object boundary and be validated with the standard `jsonschema` library.
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- Respect `$schema` when present with `jsonschema.validators.validator_for`.
|
||||||
|
- Default to Draft 2020-12 when `$schema` is absent.
|
||||||
|
- Validate workflow input/output schemas and node input/output schemas at model
|
||||||
|
boundaries.
|
||||||
|
- Keep JSON Schema object keys separate from workflow path syntax.
|
||||||
|
|
||||||
|
Path fields should expose clear JSON Schema as strings with pattern and
|
||||||
|
description metadata, not `{root, parts}` objects.
|
||||||
|
|
||||||
|
## Authoring Layer
|
||||||
|
|
||||||
|
`wf_authoring` keeps ergonomic helpers:
|
||||||
|
|
||||||
|
- `state_path(...)`
|
||||||
|
- `input_path(...)`
|
||||||
|
- `context_path(...)`
|
||||||
|
- expression helpers such as `eq`, `ne`, `lt`, `le`, `gt`, `ge`, and `exists`
|
||||||
|
|
||||||
|
Those helpers should compile to core path objects and core `Condition` models.
|
||||||
|
Authoring may infer mappings for convenience, but it must emit explicit
|
||||||
|
canonical bindings into core models.
|
||||||
|
|
||||||
|
## Tracing
|
||||||
|
|
||||||
|
Runtime may use flat `dict[StatePath, value]` patches internally. Public trace
|
||||||
|
serialization should prefer list-of-structs:
|
||||||
|
|
||||||
|
```text
|
||||||
|
StateChange:
|
||||||
|
path: StatePath
|
||||||
|
value: JsonValue
|
||||||
|
```
|
||||||
|
|
||||||
|
This avoids custom JSON object-key serialization and gives MCP/LLM clients
|
||||||
|
clearer output.
|
||||||
|
|
||||||
|
## Implementation Phases
|
||||||
|
|
||||||
|
1. Add core path value objects and parser helpers.
|
||||||
|
2. Add canonical binding models with parse-only compatibility for old fields.
|
||||||
|
3. Update validation to reason over path objects and binding structs.
|
||||||
|
4. Update runtime input resolution and output patching to use canonical
|
||||||
|
bindings.
|
||||||
|
5. Add focused state patch preparation with atomic commit semantics.
|
||||||
|
6. Add JSON Schema validation hardening for `SchemaRef`.
|
||||||
|
7. Update `wf_authoring` builders/helpers to emit canonical bindings.
|
||||||
|
8. Update docs/examples and mark old fields as deprecated.
|
||||||
|
|
||||||
|
Each phase should keep existing tests green, with compatibility tests proving
|
||||||
|
old shapes still parse until support is intentionally removed.
|
||||||
|
|
||||||
|
## Open Risks
|
||||||
|
|
||||||
|
- Pydantic core-schema hooks for frozen path value objects may need careful
|
||||||
|
implementation to keep JSON Schema clean.
|
||||||
|
- Existing examples and MCP-facing draft tools may rely on old dict-shaped maps.
|
||||||
|
Compatibility adapters should isolate that churn.
|
||||||
|
- Focused schema validation for nested state writes depends on how much schema
|
||||||
|
information is available for exact declared state paths.
|
||||||
|
- Workflow input currently seeds initial state. This design keeps that
|
||||||
|
compatibility for now, but explicit initialization remains the target
|
||||||
|
direction.
|
||||||
Reference in New Issue
Block a user