docs of dipping my hand on wf_core

docs/core_state_mapping_and_merge.md

@@ -0,0 +1,253 @@
+# Core State Mapping and Merge Semantics
+
+This document records the intended direction for `wf_core` mapping and state
+updates before the next round of core implementation work.
+
+## Why This Exists
+
+The current core model is intentionally explicit:
+
+- graph data moves between steps through workflow state
+- nodes declare their own input and output contracts
+- graph use-sites map between graph data and node-local data
+- merge behavior belongs to workflow state, not to node implementations
+
+That model is still right. The next pressure comes from structured tools and
+future nested execution:
+
+- real tools often accept deep payloads such as `user.name` and `job.title`
+- real tools often return deep payloads such as `job.wage`
+- wrappers should not need extra runtime nodes for pure boundary wiring
+- future native subgraphs and parallel foreach both need a stronger commit model
+- nested state reducers can become a differentiator over systems that only merge
+  whole top-level values
+
+## Canonical Mapping Rule
+
+The graph-facing side of a map is a graph path. The node-facing side is a
+node-local path.
+
+```text
+in_map:
+  graph source path -> node-local input path
+
+out_map:
+  node-local output path -> graph state destination path
+```
+
+Examples:
+
+```python
+in_map = {
+    "state.person.name": "user.name",
+    "state.digital.email": "user.email",
+    "state.job.title": "job.title",
+}
+
+out_map = {
+    "job.wage": "state.job.wage",
+    "job.years": "state.experience.years",
+    "user.age": "state.person.age",
+}
+```
+
+Whole-object mapping remains valid:
+
+```python
+in_map = {"state.person": "user"}
+out_map = {"user": "state.person"}
+```
+
+The change from the current implementation is only on the node-local sides:
+today they are top-level fields; in the target model they may be nested paths.
+
+## Explicitness Rules
+
+### Node-local writes must not overlap
+
+`in_map` constructs node input payloads. Its destination node-local paths must
+be pairwise non-overlapping.
+
+Valid:
+
+```python
+{
+    "state.person.name": "user.name",
+    "state.person.email": "user.email",
+}
+```
+
+Invalid:
+
+```python
+{
+    "state.person": "user",
+    "state.person.name": "user.name",
+}
+```
+
+The invalid form would require implicit object patch precedence. Authors must
+choose either whole-object mapping or explicit child mapping.
+
+### State writes must not overlap in one commit
+
+`out_map` mutates workflow state. Its destination graph paths must be pairwise
+non-overlapping inside one logical commit.
+
+Invalid:
+
+```python
+{
+    "user": "state.person",
+    "user.name": "state.person.name",
+}
+```
+
+The target model rejects these writes before mutating state.
+
+### Read overlap is allowed
+
+Overlap is forbidden on write targets, not read sources. It is valid to read
+both a whole object and one child into separate destinations when no constructed
+target overlaps.
+
+## Required Mapped Paths
+
+Mapped paths are assertions by the workflow author.
+
+- a missing graph source path in `in_map` is a runtime error
+- a missing node-local output path in `out_map` is a runtime error
+- optional/default behavior must be modeled explicitly later, not inferred from
+  a missing path
+
+## State Patch Commit
+
+A successful step should produce a logical state patch before state is mutated:
+
+1. resolve all mapped output paths
+2. ensure every required output path exists
+3. ensure destination state paths do not overlap
+4. prepare the complete write set
+5. commit the write set according to state merge rules
+
+This preserves the existing “no partial state commit before success” rule and
+creates a reusable boundary for:
+
+- ordinary node completion
+- serial foreach iteration completion
+- future parallel foreach result combination
+- future subgraph completion
+
+## State Declarations and Merge Rules
+
+The current implementation attaches merge behavior to declared top-level state
+fields. The target model should later allow nested declared state paths while
+keeping the internal representation flat:
+
+```python
+fields = {
+    "person.name": StateField(type="string", merge_strategy="replace"),
+    "person.tags": StateField(type="array", merge_strategy="append"),
+    "profile": StateField(type="object", merge_strategy="merge_object"),
+}
+```
+
+Presentation layers may rebuild a tree for humans. Core should keep the simpler
+path-keyed representation.
+
+### Exact-path ownership
+
+Merge behavior belongs only to the exact declared state path being written.
+
+- no ancestor inheritance
+- no descendant declarations altering parent writes
+- writing `state.profile` uses only the rule for `profile`
+- writing `state.profile.avatar` uses only the rule for `profile.avatar`
+
+If a path is undeclared, it defaults to `replace`, as undeclared state does
+today.
+
+### Built-in strategies
+
+Existing built-ins remain distinct:
+
+- `replace`
+- `append`
+- `merge_object`
+
+`merge_object` means shallow object merge at the exact destination path, similar
+to `dict.update` or `operator.or_`. It is not a recursive deep merge.
+
+If recursive merge is ever needed, it should be explicit rather than hidden
+inside `merge_object`.
+
+## Future Reducers
+
+Custom reducers should become a future capability family, similar to reusable
+node specs:
+
+- named
+- source-owned
+- inspectable
+- dependency-trackable
+
+State fields should reference reducers declaratively. Workflow artifacts should
+not embed arbitrary Python callables.
+
+Reducers should be pure:
+
+```text
+current_value, incoming_value -> merged_value
+```
+
+They should not receive node ids, frame ids, loop indexes, timestamps, or other
+runtime context. If behavior depends on workflow context, that is business logic
+and belongs in nodes or graph structure.
+
+Examples a future reducer library could support:
+
+- `max`
+- `set_union`
+- `modulo_add` with configuration such as modulus `10`
+
+## Implementation Phases
+
+### Phase 1: Nested node-local mappings
+
+- allow nested node-local paths on the destination side of `in_map`
+- allow nested node-local paths on the source side of `out_map`
+- reject overlapping write targets
+- commit node output through a validated state patch
+- validate top-level node-local roots statically; validate deeper shape when the
+  existing node schema makes that practical
+
+This phase immediately helps wrappers and structured tools while preserving
+current state merge behavior.
+
+### Phase 2: Nested declared state paths
+
+- allow exact nested state path declarations
+- resolve merge strategies by exact destination path only
+- keep undeclared paths as `replace`
+- keep `merge_object` shallow
+
+### Phase 3: Reducer capabilities
+
+- design source-owned reducer specs
+- add reducer dependency references to state metadata
+- resolve pure reducers through runtime/deployment registries
+
+### Phase 4: Core features that depend on this foundation
+
+- native subgraphs should reuse the same mapping and patch semantics at graph
+  boundaries
+- parallel foreach should combine validated patches and reject conflicts unless
+  exact-path merge rules permit combination
+
+## Non-Goals for the First Implementation
+
+- direct node-to-node data wires outside workflow state
+- implicit object patching from overlapping mapped paths
+- recursive deep merge hidden inside `merge_object`
+- arbitrary Python reducer callables stored in workflow models
+- shipping native subgraphs or parallel foreach as part of nested mapping work

docs/schema_validation.md

@@ -23,6 +23,7 @@ It still does not solve:
   external JSON Schema dialect
 - typed Python object creation from arbitrary JSON Schema
 - workflow state merge behavior
+- deep node-local map-path validation beyond statically knowable schema roots
 - better domain-specific error payloads beyond `WorkflowExecutionError`
 
 This means schema fields are mostly contracts for authoring, planning,

docs/scratchpad.md

@@ -394,13 +394,13 @@ Practical notes:
 `in_map`:
 
 - source is graph `input`, `state`, or future `context` path
-- destination is a declared node input field
+- destination is a node-local input path
 - missing required source path is a graph/runtime error
 - optional destination fields may be omitted
 
 `out_map`:
 
-- source is a declared node output field
+- source is a node-local output path
 - destination is a graph state path
 - mapping to undeclared state keys is allowed, but loses typed merge behavior
 - mapping a required output field that does not exist at runtime is node failure
@@ -452,8 +452,8 @@ Before execution, validator should be able to check:
 - every edge destination exists or is `__end__`
 - every edge outcome is declared by its source node type
 - every reachable declared outcome is wired
-- every `in_map` destination exists in node input schema
+- every `in_map` destination has a valid declared node-input root
-- every `out_map` source exists in node output schema
+- every `out_map` source has a valid declared node-output root
 - every condition node uses valid operators and operand shapes
 
 At runtime, executor should still check:

docs/superpowers/plans/2026-05-17-nested-node-local-mappings.md

@@ -0,0 +1,319 @@
+# Nested Node-Local Mappings Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Let workflow maps address nested node-local input/output paths while preserving explicit state-mediated data flow and preparing state writes for future reducer libraries.
+
+**Architecture:** Keep graph-facing paths unchanged. Add a small node-local path helper layer, validate non-overlapping write targets, construct nested node inputs from `in_map`, resolve nested node outputs from `out_map`, and refactor state writes into a prepared patch commit. Extract built-in merge-rule application behind a focused reducer-like module so future named reducer libraries can replace the dispatch seam without changing patch semantics.
+
+**Tech Stack:** Python, Pydantic, pytest, existing `wf_core` path/state runtime.
+
+---
+
+## File Structure
+
+- Modify `src/wf_core/paths.py`
+  - keep graph-path helpers
+  - add reusable path-overlap utility if it belongs at the generic path layer
+- Create `src/wf_core/local_paths.py`
+  - node-local dotted path parsing
+  - nested get/set for node-local payloads
+  - overlap checks for write targets
+- Create `src/wf_core/runtime/ops/merges.py`
+  - built-in exact-path merge-rule implementations
+  - future seam for named reducer registry
+- Modify `src/wf_core/runtime/ops/state.py`
+  - replace per-write mutation loop with prepared patch commit
+  - delegate merge-rule application to `merges.py`
+- Modify `src/wf_core/runtime/ops/nodes.py`
+  - build nested node inputs from `in_map`
+  - resolve nested node outputs into state patch writes
+- Modify `src/wf_core/validation/steps.py`
+  - validate node-local top-level roots
+  - reject overlapping write targets
+- Add/modify tests under `tests/core/`
+  - nested `in_map`
+  - nested `out_map`
+  - whole-object mapping still works
+  - overlapping write targets rejected
+  - missing nested output path fails
+  - merge dispatch remains behaviorally unchanged
+
+---
+
+### Task 1: Pin Node-Local Path Behavior
+
+**Files:**
+- Modify: `tests/core/test_validation.py`
+- Modify: `tests/core/test_runtime.py`
+
+- [ ] **Step 1: Add failing validation tests**
+
+Cover:
+
+```python
+def test_validation_allows_nested_node_local_paths() -> None:
+    ...
+
+
+def test_validation_rejects_overlapping_node_input_destinations() -> None:
+    ...
+
+
+def test_validation_rejects_overlapping_state_write_destinations() -> None:
+    ...
+```
+
+Expected rules:
+
+- `state.person.name -> user.name` is valid when `user` exists in the node input schema
+- `user -> state.person` and `user.name -> state.person.name` in one `out_map` is invalid because destination state paths overlap
+- `state.person -> user` plus `state.person.name -> user.name` in one `in_map` is invalid because destination node-local paths overlap
+
+- [ ] **Step 2: Add failing runtime tests**
+
+Cover:
+
+```python
+def test_runtime_builds_nested_node_input_from_in_map() -> None:
+    ...
+
+
+def test_runtime_reads_nested_node_output_from_out_map() -> None:
+    ...
+
+
+def test_runtime_missing_nested_node_output_path_fails() -> None:
+    ...
+```
+
+- [ ] **Step 3: Run focused tests and confirm failure**
+
+Run:
+
+```bash
+uv run --with pytest pytest tests/core/test_validation.py tests/core/test_runtime.py -q
+```
+
+Expected: FAIL because node-local map sides are top-level-only today.
+
+### Task 2: Add Node-Local Path Helpers
+
+**Files:**
+- Create: `src/wf_core/local_paths.py`
+- Modify: `src/wf_core/validation/steps.py`
+
+- [ ] **Step 1: Add minimal helper API**
+
+Implement:
+
+```python
+def split_local_path(path: str) -> list[str]: ...
+def get_local_value(payload: Mapping[str, Any], path: str) -> Any: ...
+def set_local_value(payload: dict[str, Any], path: str, value: Any) -> None: ...
+def paths_overlap(left: str, right: str) -> bool: ...
+def has_overlapping_paths(paths: Iterable[str]) -> bool: ...
+```
+
+Rules:
+
+- dotted local paths only
+- no empty segments
+- overlap means same path or ancestor/descendant path
+
+- [ ] **Step 2: Update validation**
+
+Use node-local path roots for schema checks:
+
+```python
+input_root = split_local_path(destination_path)[0]
+output_root = split_local_path(source_path)[0]
+```
+
+Reject:
+
+- overlapping `in_map` destination local paths
+- overlapping `out_map` destination state paths
+
+- [ ] **Step 3: Run focused validation tests**
+
+Run:
+
+```bash
+uv run --with pytest pytest tests/core/test_validation.py -q
+```
+
+Expected: PASS for validation-specific cases.
+
+### Task 3: Execute Nested Local Mappings
+
+**Files:**
+- Modify: `src/wf_core/runtime/ops/nodes.py`
+- Modify: `src/wf_core/runtime/ops/state.py`
+
+- [ ] **Step 1: Build nested node inputs**
+
+Replace flat input construction with:
+
+```python
+resolved_input: dict[str, Any] = {}
+for source_path, destination_path in node.in_map.items():
+    value = safe_resolve_path(...)
+    set_local_value(resolved_input, destination_path, value)
+```
+
+- [ ] **Step 2: Resolve nested node outputs**
+
+When preparing mapped output writes, use `get_local_value()` for each `out_map`
+source path instead of indexing only top-level output keys.
+
+- [ ] **Step 3: Preserve missing-path failures**
+
+Raise `WorkflowExecutionError` when a mapped nested output path is missing.
+
+- [ ] **Step 4: Run focused runtime tests**
+
+Run:
+
+```bash
+uv run --with pytest pytest tests/core/test_runtime.py -q
+```
+
+Expected: PASS for nested mapping behavior.
+
+### Task 4: Introduce Prepared Patch Commits and Extract Merge Dispatch
+
+**Files:**
+- Create: `src/wf_core/runtime/ops/merges.py`
+- Modify: `src/wf_core/runtime/ops/state.py`
+- Modify: `tests/core/test_state_ops.py`
+
+- [ ] **Step 1: Add failing patch-level tests**
+
+Cover:
+
+```python
+def test_state_patch_rejects_overlapping_destinations_before_mutation() -> None:
+    ...
+
+
+def test_builtin_merge_rules_preserve_existing_behavior() -> None:
+    ...
+```
+
+- [ ] **Step 2: Extract built-in merge implementations**
+
+Move the current strategy body out of `write_state_value()` into focused helpers:
+
+```python
+def apply_builtin_merge(
+    *,
+    strategy: str,
+    current_value: Any,
+    incoming_value: Any,
+    destination_path: str,
+) -> Any: ...
+```
+
+Keep current semantics:
+
+- `replace`
+- `append`
+- shallow `merge_object`
+
+Add a docstring that this is the future seam for source-owned named reducers,
+not custom reducer support yet.
+
+- [ ] **Step 3: Prepare full write sets before mutation**
+
+Refactor output mapping so it:
+
+1. resolves all mapped output values
+2. validates destination overlap
+3. prepares the patch
+4. applies merge behavior
+
+No state changes should occur before all mapped output paths are known-good.
+
+- [ ] **Step 4: Run state/runtime tests**
+
+Run:
+
+```bash
+uv run --with pytest pytest tests/core/test_state_ops.py tests/core/test_runtime.py -q
+```
+
+Expected: PASS.
+
+### Task 5: Keep Authoring and Docs Aligned
+
+**Files:**
+- Modify: `src/wf_authoring/builder/mapping.py`
+- Modify: `docs/core_state_mapping_and_merge.md` if implementation details differ
+- Modify: `docs/scratchpad.md` only if wording drift appears
+- Modify/Add: authoring tests as needed
+
+- [ ] **Step 1: Confirm authoring auto-maps remain top-level**
+
+Automatic maps should stay conservative unless there is an explicit reason to
+infer nested paths. The new feature is for explicit maps first.
+
+- [ ] **Step 2: Add one authoring regression**
+
+Prove that a builder can compile a workflow using explicit nested local map
+paths without extra helper nodes.
+
+- [ ] **Step 3: Update docs only for implementation drift**
+
+The design doc already states the target behavior. Keep docs in sync with final
+names and module boundaries, but do not broaden scope into nested state
+declarations yet.
+
+### Task 6: Verify the Whole Project
+
+**Files:**
+- No additional files.
+
+- [ ] **Step 1: Run focused suites**
+
+```bash
+uv run --with pytest pytest tests/core tests/authoring -q
+```
+
+- [ ] **Step 2: Run the full suite**
+
+```bash
+uv run --with pytest pytest -q
+```
+
+- [ ] **Step 3: Run type checking**
+
+```bash
+uv run basedpyright --level error
+```
+
+Expected:
+
+- tests pass
+- any remaining basedpyright failures are called out explicitly if they come
+  from existing generated/build/doc-fixture noise rather than this work
+
+---
+
+## Deliberate Non-Goals
+
+- nested declared state merge metadata
+- reducer capability registry
+- deep merge behavior
+- native subgraphs
+- parallel foreach
+- automatic inference of nested maps from schemas
+
+## Follow-On Plans
+
+After this lands:
+
+1. nested declared state paths with exact-path merge lookup
+2. reducer capability model / registry seam
+3. native subgraph design on the same map + patch boundary
+4. async-only parallel foreach using patch combination rules

docs/wf_core_architecture.md

@@ -71,6 +71,10 @@ limits and intended adapter seam.
 
 ## What This Cleanup Does Not Solve Yet
 
+- Node-local mapping paths are still top-level only in the current runtime.
+  The intended direction for nested node-local paths, patch commits, and future
+  nested merge metadata is documented in
+  [`core_state_mapping_and_merge.md`](core_state_mapping_and_merge.md).
 - Foreach is still serial-only. Parallel foreach needs an explicit scheduling
   model, not just `asyncio.gather`.
 - Interrupt lifecycle is still node-level and run-state-level. Long-lived