lda-wf/docs/historical/path_mapping_scratch.md

Path And Mapping Scratch

Historical note: this scratchpad records how the mapping model evolved. Current core behavior uses canonical input/output and interrupt request/resume binding lists. See core_state_mapping_and_merge.md.

This is a focused scratchpad for the wf_core path/mapping design thread. Clean this into real docs later.

Current Mapping Roles

in_map maps graph/runtime paths into node-local input paths:

input.text -> text
state.person.name -> user.name
context.item -> document

out_map maps node-local output paths into workflow state destinations:

echoed -> state.echoed
user.age -> state.person.age
. -> state.rates

input_values maps node-local input paths to literal values:

mode -> "fast"
retry.count -> 3

Path Kinds

Graph paths live in workflow/run space:

• input.*: original workflow invocation input.
• state.*: mutable workflow state.
• context.*: runtime frame context.

Node-local paths live inside one node input/output payload:

• user.name
• job.years
• . for the whole local payload.

Context Paths

context.* currently exists. It is generated from the current execution frame, not stored in workflow state.

Current context fields include:

• context.prior_outcome
• context.activated_incoming_edge
• foreach frames: context.loop_item
• foreach frames: context.loop_index
• foreach frames: context.<loop_alias> when the foreach node declares as.

Context paths are valid graph source paths for in_map when validation allows context. They are not valid out_map destinations.

Typed Internal Model Idea

Keep serialized maps as strings for JSON compatibility, but parse them at runtime/validation boundaries:

GraphPath(root="input", parts=("text",))          <-> "input.text"
GraphPath(root="state", parts=("person", "name")) <-> "state.person.name"
GraphPath(root="context", parts=("item",))        <-> "context.item"
LocalPath(parts=("user", "name"))                 <-> "user.name"
LocalPath.root()                                  <-> "."

This lets validation reason about roots and parts without repeated string prefix checks.

Decisions So Far

Canonical path parsing/validation should live in wf_core.

wf_authoring should keep ergonomic constructors and condition wrappers, but those should be thin wrappers around core path types.

Condition path fields should use typed graph source paths, not arbitrary strings:

PathOperand.path: GraphSourcePath
ExistsCondition.path: GraphSourcePath

Node input mapping should merge the old in_map and input_values concepts into one list of binding structs:

InputPathBinding:
  target: LocalPath
  path: GraphSourcePath

InputValueBinding:
  target: LocalPath
  value: JsonValue

Do not add a kind discriminator if the shape can be distinguished by path vs value. Use strict models so {path, value} and {} fail.

Node output mapping should become a list of binding structs:

OutputBinding:
  source: LocalPath
  target: StatePath

The same field name can mean different path kinds by position:

• input binding target is a node-local input path
• output binding target is a workflow state path

Root local path "." remains valid:

• input target "." means the whole node input payload is the mapped path/value
• output source "." means the whole node output payload is written
• literal input value binding to "." is valid and means the literal value is the entire node input payload
• "." input binding must still be the only input binding on that node use

Overlap rules:

• if an input binding targets ".", it must be the only input binding
• input targets must not overlap, e.g. user and user.name
• output targets must not overlap, e.g. state.person and state.person.name
• exact duplicate input targets are invalid
• exact duplicate output targets are invalid
• if output writes a parent, it cannot also write a child
• read paths may overlap; write paths may not overlap

State path validation and write behavior:

• writable StatePath must have its root declared in state_schema.properties
• whole-state write targets such as bare state stay out of scope for now
• nested state subpaths are allowed once the root exists in the schema
• exact nested state declarations are reducer/schema hints, not root ownership
• reducers apply only to the exact declared destination path
• missing object parents are created during runtime writes
• descending through an existing non-object parent fails at runtime

Runtime traversal helpers:

• all get/set traversal should go through focused helpers
• do not scatter dict/getattr logic through runtime code
• helpers may support mappings, Pydantic models, and dataclasses where safe
• exact supported object kinds can be implementation-defined, but the behavior should be centralized and tested

Read vs write traversal:

• reads may support richer object access through centralized helpers
• writes are stricter
• nested writes require mutable mapping parents
• typed model/dataclass parents are not patched field-by-field
• strict typed values should be replaced as a whole, not partially mutated
• full replacements must adhere to the declared schema when validation exists

State write validation:

• validate writes against the exact declared state field schema when one exists
• replacement writes must satisfy the full target schema
• replacement writes that contain fields not allowed by the target schema fail
• partial object patches require an explicit merge-style reducer such as merge_object; do not silently treat replace as merge
• do not validate the whole workflow state on every write
• undeclared nested subpaths under declared object roots remain flexible
• this keeps validation focused and avoids heavy whole-state checks

Workflow input/output validation:

• validate full workflow input against workflow.input_schema at run start
• keep workflow output as top-level projection from state for now
• workflow.output_schema.properties decides which top-level state fields are exposed as final output
• do not add workflow-level output bindings in this path refactor
• if shaped output is needed, use a final node to shape state before END

Execution validation order:

1. validate workflow input at run start
2. resolve node input bindings into node payload
3. validate node payload against node_def.input_schema
4. execute node
5. coerce node result
6. validate node result output against node_def.output_schema
7. apply output bindings to state with focused state field validation
8. project final workflow output from state at END

Principle:

• validate node output before mutating state
• strange node results should fail fast and not pollute workflow state

State patch atomicity:

• output binding application is one atomic patch to workflow state
• no gradual state mutation while processing individual bindings
• resolve all sources first
• check overlaps first
• compute reducer/merged values first
• validate patch values first
• commit the patch only after preparation succeeds
• failed output binding application leaves prior state unchanged

Reducers:

• reducers run during patch preparation, not during commit
• reducers receive current value, incoming value, and optional config
• reducers return the merged value for the patch
• reducers must not mutate workflow state directly
• reducer failure aborts the whole patch before commit

Patch representation:

• public mappings are list-of-structs
• internal prepared patches can be flat path-keyed maps
• key type should be StatePath, not raw string
• flat patches make overlap checks, reducer lookup, validation, tracing, and commit performance simpler
• do not expose flat path-keyed patch maps as the public authoring shape

Trace state changes:

• use typed StatePath internally
• prefer list-of-structs for public trace serialization/schema
• target shape:

StateChange:
  path: StatePath
  value: JsonValue

• this avoids custom JSON object key serialization problems
• it gives cleaner schema and clearer MCP/LLM output

Runtime state value domain:

• workflow definitions and static binding values should be JSON-compatible
• in-memory runtime state may remain Any-ish for now
• persisted/checkpointed run state should require JSON-compatible values
• checkpoint serialization should be the strict boundary for non-serializable runtime objects

Current initialization behavior:

• workflow input is currently copied into initial state by init_run_state
• this makes input.foo and state.foo both available at run start when the input has foo
• target direction is explicit initialization, closer to LangGraph: input stays input, state is mutated only by explicit graph behavior
• keep implicit seeding for now as compatibility unless/until there is a dedicated migration

Canonical node binding shape:

NodeUse.input: list[InputBinding]
NodeUse.output: list[OutputBinding]

Old in_map, input_values, and out_map can be accepted as parse-only compatibility inputs, but the canonical model should store and serialize the new list-of-structs shape.

Naming caveat:

• NodeUse.input means "bindings that build this node's input payload"
• graph path root input.* means "the workflow run input"

Docs must make this distinction explicit. The repeated word is acceptable only if examples clearly show input as the binding list and input.foo as a graph source path.

Static value bindings:

• InputValueBinding.value should be JSON-compatible
• workflow models should stay serializable/storable
• non-serializable runtime objects such as callbacks should not be embedded in workflow definitions
• future LangGraph-store-like behavior should use explicit store/reference mechanisms, not arbitrary Python objects inside the model

Context paths:

• context.* stays a first-class graph source path
• it is read-only and source-only
• root-only graph source paths are allowed for whole-container reads: input, state, and context
• root-only graph source paths are useful for explicitly passing whole workflow input/state/context into a node
• conditions can use context.*
• node input path bindings can use context.*
• output/write bindings cannot target context.*
• workflow validation does not need to statically prove every context key exists
• exists(context.x) returns false when missing
• comparisons against missing context paths fail clearly

Validation responsibility split:

• path types validate that a value is a well-formed path of the right kind
• workflow validation checks whether that path is legal in a specific workflow
• examples of workflow-specific checks:
  • input.foo root exists in input_schema
  • state.foo root exists in state_schema
  • write destinations are state paths
  • write destinations do not overlap
• path types should not need access to workflow schemas

Path segment syntax:

• use strict identifier-like segments: [A-Za-z_][A-Za-z0-9_]*
• dots separate path segments
• empty segments are invalid
• arbitrary JSON keys containing dots or punctuation are not supported yet
• bracket/index syntax is out of scope until deliberately designed
• this tightens current behavior, which mostly split strings without much segment validation

List indexing:

• no list indexing in core paths
• paths address object/dict/model fields, not list positions
• numeric/positional list segments are rejected
• encoding workflow meaning by array position is discouraged
• use foreach for item-wise behavior
• future continue/break-style foreach behavior may cover many selection cases
• explicit helper nodes such as an authoring index node can handle positional lookup without complicating core path syntax

Local path syntax:

• LocalPath uses the same strict segment rules as graph paths
• valid examples: user.name, job.years, .
• invalid examples: user..name, user-name, items.0, user["name"]
• no external path library is planned; a small parser with a compiled segment regex is enough because these paths have workflow-specific roots and rules

Hashability:

• path objects should be immutable/hashable
• use frozen dataclasses or equivalent immutable models
• public node bindings should still be list-of-structs, not dicts keyed by paths
• hashable paths are useful internally for lookups, sets, duplicate detection, and overlap validation

Implementation choice:

• use frozen dataclass value objects with Pydantic core-schema hooks
• avoid Pydantic BaseModel for tiny path values unless hooks become too costly
• Pydantic should accept strings or existing path objects and store path objects
• JSON serialization should emit strings

Planned core path value objects:

LocalPath(parts)
GraphSourcePath(root, parts)  # root is input | state | context
StatePath(parts)              # serializes with state. prefix

Shared internals:

• use shared parsing/segment/overlap helpers for all path kinds
• do not expose one generic public Path for every position
• distinct public path types preserve semantics:
  • LocalPath for node-local payloads
  • GraphSourcePath for readable graph sources
  • StatePath for writable state destinations
• a small shared PathParts value or shared helper functions are both fine; choose whichever keeps implementation simplest

Constructors and authoring sugar:

• core path types should have boring constructors/parsers for tests and runtime
• wf_authoring keeps pretty helpers such as state_path, input_path, context_path, and condition expression sugar
• Expr and PathExpr stay in wf_authoring
• core owns Condition models and path value types
• authoring wrappers can behave like a trait/mixin over comparable path-like values, adding eq, ne, lt, le, gt, ge, operator overloads, and exists

Core constructor ergonomics:

• constructors may accept dotted strings and/or multiple fragments
• fragments are flattened by splitting on .
• examples:
  • GraphSourcePath.state("person.name")
  • GraphSourcePath.state("person", "name")
  • GraphSourcePath.state("foo", "bar.baz")
• all final segments still pass strict segment validation
• empty fragments/segments are rejected
• LocalPath.root() / serialized "." is the only root marker exception

Parse vs construction:

• parse() reads the full serialized JSON form
• of() / root-specific constructors are ergonomic Python construction
• both use the same segment parsing/validation internals
• examples:
  • StatePath.parse("state.person.name")
  • StatePath.of("person.name")
  • GraphSourcePath.state("person.name")
  • LocalPath.of("user.name")

String form:

• str(path) returns the serialized JSON form
• examples:
  • str(StatePath.of("person.name")) == "state.person.name"
  • str(GraphSourcePath.state("person.name")) == "state.person.name"
  • str(LocalPath.of("user.name")) == "user.name"
  • str(LocalPath.root()) == "."

JSON Schema:

• path fields should expose as strings, not {root, parts} objects
• use pattern and description metadata where possible
• examples:
  • StatePath: string matching state.<segment>(.<segment>)*
  • GraphSourcePath: string matching (input|state|context).<segment>(.<segment>)*
  • LocalPath: string matching . or <segment>(.<segment>)*
• use Pydantic hook / annotation metadata magic to keep external schemas clear while storing rich path value objects internally

Error messages:

• path parsing errors should name the expected path kind
• include examples in errors where practical
• expected examples:
  • graph source path: input.foo, state.foo, or context.foo
  • state path: state.foo
  • local path: user.name or .
• this should improve current error + hint behavior for MCP/LLM users

Missing paths:

• binding path reads should fail when missing
• do not add on_missing to core bindings now
• optional/default/missing behavior belongs in explicit nodes or conditions
• this keeps bindings as data movement, not hidden behavior

Binding descriptions:

• no description / desc fields on wf_core binding structs for now
• descriptions are useful in higher layers such as drafts, artifacts, authoring helpers, or MCP-facing planning surfaces
• core bindings should stay structural/runtime-focused

Binding order:

• binding order is preserved
• runtime may apply bindings in order for deterministic traces/debugging
• order must not resolve conflicts
• overlapping write targets are validation errors, not "last write wins"
• future priority/override behavior must be explicit, not implicit list order

Core explicitness:

• empty NodeUse.input means empty node payload
• empty NodeUse.output means no state writes
• core performs no auto-mapping
• authoring/builder layers may infer mappings, but must emit explicit canonical bindings into core models

State schema fields:

• canonical shape is normal JSON Schema
• state field metadata such as reducer lives as a wf_core extension keyword on each property schema
• JSON Schema validators ignore reducer; wf_core validates it separately and compiles it into an exact-path runtime index
• accept old fields shapes at parse time for compatibility:

state_schema = {
  "type": "object",
  "properties": {
    "person": {
      "type": "object",
      "properties": {
        "tags": {"type": "array", "reducer": "wf.std.append"}
      }
    }
  }
}

• canonical serialization emits JSON Schema shape
• duplicate field paths are validation errors
• exact reducer matching uses exact StatePath

Input/output JSON Schemas:

• do not replace JSON Schema with a custom schema language
• introduce a named JSON Schema boundary type or model
• validate schemas with the standard jsonschema library
• validation cost is acceptable at model/config boundaries
• this applies to workflow input/output schemas and node input/output schemas
• JSON Schema object keys are standard and should not be treated like workflow path-map dict keys
• respect $schema when present using jsonschema.validators.validator_for
• default to Draft 2020-12 when $schema is absent
• keep SchemaRef as the core schema type for now
• make SchemaRef honest: it represents a JSON Schema object boundary
• strengthen SchemaRef with standard JSON Schema validation instead of introducing a parallel JsonSchema type

Placement:

• canonical path value objects live in wf_core.paths
• wf_core.models.*, validation, runtime, and authoring import those types
• split wf_core.paths into a package later only if it grows too large

Compatibility fields:

• old in_map, input_values, and out_map are parse-only compatibility inputs
• they should be treated as deprecated
• validated NodeUse stores only canonical input and output bindings
• canonical serialization emits only the new fields
• avoid dual state inside the model
• reject payloads that mix canonical fields with deprecated compatibility fields
• do not merge old and new syntax
• compatibility conversion preserves dict insertion order when converting old maps into binding lists
• JSON Schema should advertise only canonical input / output fields
• deprecated compatibility fields should not be shown to new callers

Null and missing semantics:

• explicit null is a real value, not an omitted binding/value
• exists(path) means the path can be resolved, even when its value is null
• missing path and present-null path are different states
• compare against null explicitly when needed, e.g. state.foo == null or state.foo != null
• null comparisons are valid anywhere normal conditions work
• missing paths still fail clearly for comparisons; only exists(path) treats missing as false instead of an error
• input value bindings may intentionally bind null; this must not be treated as "no value provided"
• input path bindings fail before node execution when the source path is missing
• missing source paths must not silently bind null
• softer/defaulting behavior belongs in authoring helpers or explicit shaping nodes, not implicit core behavior

Schema openness:

• JSON Schema / Pydantic-style extra controls what object keys are allowed inside a value
• schema openness does not change path existence semantics
• core should not use "allow extra" as permission for speculative deep path traversal
• if a workflow needs dynamic traversal through arbitrary/extra object shape, use an explicit node that receives the relevant state value and decides what to extract
• this avoids pretending static path validation can prove paths such as state.person.occupations.1.title exist inside open-ended objects