nested state path
This commit is contained in:
@@ -140,9 +140,8 @@ creates a reusable boundary for:
|
|||||||
|
|
||||||
## State Declarations and Merge Rules
|
## State Declarations and Merge Rules
|
||||||
|
|
||||||
The current implementation attaches merge behavior to declared top-level state
|
State merge behavior is attached to declared exact state paths while keeping the
|
||||||
fields. The target model should later allow nested declared state paths while
|
internal representation flat:
|
||||||
keeping the internal representation flat:
|
|
||||||
|
|
||||||
```python
|
```python
|
||||||
fields = {
|
fields = {
|
||||||
@@ -226,10 +225,12 @@ current state merge behavior.
|
|||||||
|
|
||||||
### Phase 2: Nested declared state paths
|
### Phase 2: Nested declared state paths
|
||||||
|
|
||||||
- allow exact nested state path declarations
|
Implemented in core:
|
||||||
- resolve merge strategies by exact destination path only
|
|
||||||
- keep undeclared paths as `replace`
|
- exact nested state path declarations
|
||||||
- keep `merge_object` shallow
|
- merge strategies resolved by exact destination path only
|
||||||
|
- undeclared paths remain `replace`
|
||||||
|
- `merge_object` remains shallow
|
||||||
|
|
||||||
### Phase 3: Reducer capabilities
|
### Phase 3: Reducer capabilities
|
||||||
|
|
||||||
|
|||||||
@@ -22,13 +22,13 @@ It still does not solve:
|
|||||||
- semantic compatibility between Pydantic-generated schemas and every possible
|
- semantic compatibility between Pydantic-generated schemas and every possible
|
||||||
external JSON Schema dialect
|
external JSON Schema dialect
|
||||||
- typed Python object creation from arbitrary JSON Schema
|
- typed Python object creation from arbitrary JSON Schema
|
||||||
- workflow state merge behavior
|
- workflow state merge behavior beyond declared exact-path metadata
|
||||||
- deep node-local map-path validation beyond statically knowable schema roots
|
- deep node-local map-path validation beyond statically knowable schema roots
|
||||||
- better domain-specific error payloads beyond `WorkflowExecutionError`
|
- better domain-specific error payloads beyond `WorkflowExecutionError`
|
||||||
|
|
||||||
This means schema fields are mostly contracts for authoring, planning,
|
This means schema fields are contracts for authoring, planning, documentation,
|
||||||
documentation, and mapping validation today. They are not yet strong runtime
|
and mapping validation. Runtime state merge behavior is separate metadata on
|
||||||
guards.
|
declared exact state paths; undeclared paths still use `replace`.
|
||||||
|
|
||||||
## Why This Matters
|
## Why This Matters
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,61 @@
|
|||||||
|
# Nested State Paths Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Let `wf_core` declare nested workflow state paths and apply merge behavior by exact destination path.
|
||||||
|
|
||||||
|
**Architecture:** Keep state declarations internally flat and path-keyed. Continue allowing undeclared state paths, but only exact declared paths receive typed merge behavior; parent declarations do not implicitly govern descendants. Reuse the state patch boundary added in Phase 1 and change only schema wording, validation coverage, runtime lookup, and docs.
|
||||||
|
|
||||||
|
**Tech Stack:** Python, Pydantic, pytest, existing `wf_core` state runtime.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
|
||||||
|
- Modify `src/wf_core/models/schemas.py`
|
||||||
|
- clarify that `StateField` / `StateSchema` are path-keyed, not root-only
|
||||||
|
- Modify `src/wf_core/runtime/ops/state.py`
|
||||||
|
- resolve merge metadata by exact written state path
|
||||||
|
- Add `tests/core/test_nested_state_paths.py`
|
||||||
|
- exact nested path merge behavior
|
||||||
|
- ancestor declarations do not govern descendants
|
||||||
|
- undeclared nested paths still replace
|
||||||
|
- Update `docs/core_state_mapping_and_merge.md`
|
||||||
|
- mark nested declared state paths as implemented
|
||||||
|
- Update `docs/schema_validation.md`
|
||||||
|
- clarify that runtime state merge metadata is now exact-path capable
|
||||||
|
|
||||||
|
### Task 1: Pin Exact-Path State Behavior
|
||||||
|
|
||||||
|
- [ ] Add failing tests proving:
|
||||||
|
- `state.person.tags` uses a declaration for `"person.tags"` with `append`
|
||||||
|
- a declaration for `"person"` does not cause `state.person.tags` to inherit `merge_object`
|
||||||
|
- undeclared `state.person.tags` defaults to `replace`
|
||||||
|
- [ ] Run the focused tests and confirm the nested exact-path cases fail under the current root-only lookup.
|
||||||
|
|
||||||
|
### Task 2: Implement Exact-Path Lookup
|
||||||
|
|
||||||
|
- [ ] Update state model docstrings to describe path-keyed declarations.
|
||||||
|
- [ ] In `write_state_value()`, look up `workflow.state_schema.fields[".".join(parts)]` instead of only the first path segment.
|
||||||
|
- [ ] Keep undeclared paths as `replace`.
|
||||||
|
- [ ] Run the focused tests and confirm they pass.
|
||||||
|
|
||||||
|
### Task 3: Keep Docs Honest
|
||||||
|
|
||||||
|
- [ ] Update the core mapping design doc so Phase 2 is recorded as implemented, not future work.
|
||||||
|
- [ ] Update schema-validation docs to note exact-path state metadata without broadening payload validation claims.
|
||||||
|
|
||||||
|
### Task 4: Verify
|
||||||
|
|
||||||
|
- [ ] Run `uv run --with pytest pytest tests/core -q`
|
||||||
|
- [ ] Run `uv run --with pytest pytest -q`
|
||||||
|
- [ ] Run `uv run basedpyright --level error`
|
||||||
|
- [ ] Call out any residual type-check failures that are unrelated to this work.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- flatten nested authoring schemas from `wf_authoring`
|
||||||
|
- reducer registries or custom reducer capabilities
|
||||||
|
- changing `merge_object` from shallow to deep
|
||||||
|
- making parent declarations inherit into child paths
|
||||||
|
- `START` token model changes
|
||||||
@@ -141,7 +141,11 @@ def pick_path(input: PickPathInput) -> ValueOutput:
|
|||||||
def project_fields(input: ProjectFieldsInput) -> MappingOutput:
|
def project_fields(input: ProjectFieldsInput) -> MappingOutput:
|
||||||
"""Return only the requested existing fields from a mapping."""
|
"""Return only the requested existing fields from a mapping."""
|
||||||
return MappingOutput(
|
return MappingOutput(
|
||||||
mapping={field: input.mapping[field] for field in input.fields if field in input.mapping}
|
mapping={
|
||||||
|
field: input.mapping[field]
|
||||||
|
for field in input.fields
|
||||||
|
if field in input.mapping
|
||||||
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ class SchemaRef(BaseModel):
|
|||||||
|
|
||||||
|
|
||||||
class StateField(BaseModel):
|
class StateField(BaseModel):
|
||||||
"""Declared root state field plus its runtime merge behavior."""
|
"""Declared state path plus its runtime merge behavior."""
|
||||||
|
|
||||||
type: str
|
type: str
|
||||||
merge_strategy: Literal["replace", "append", "merge_object"] = "replace"
|
merge_strategy: Literal["replace", "append", "merge_object"] = "replace"
|
||||||
@@ -26,7 +26,7 @@ class StateField(BaseModel):
|
|||||||
|
|
||||||
|
|
||||||
class StateSchema(BaseModel):
|
class StateSchema(BaseModel):
|
||||||
"""Workflow state schema keyed by declared root field name."""
|
"""Workflow state schema keyed by declared exact state path."""
|
||||||
|
|
||||||
model_config = ConfigDict(extra="allow")
|
model_config = ConfigDict(extra="allow")
|
||||||
|
|
||||||
|
|||||||
@@ -23,15 +23,23 @@ def apply_builtin_merge(
|
|||||||
|
|
||||||
if strategy == "append":
|
if strategy == "append":
|
||||||
if current_value is None:
|
if current_value is None:
|
||||||
return [incoming_value] if not isinstance(incoming_value, list) else incoming_value
|
return (
|
||||||
|
[incoming_value]
|
||||||
|
if not isinstance(incoming_value, list)
|
||||||
|
else incoming_value
|
||||||
|
)
|
||||||
if not isinstance(current_value, list):
|
if not isinstance(current_value, list):
|
||||||
raise WorkflowExecutionError(
|
raise WorkflowExecutionError(
|
||||||
f"cannot append into non-list state path {destination_path!r}"
|
f"cannot append into non-list state path {destination_path!r}"
|
||||||
)
|
)
|
||||||
return [
|
return (
|
||||||
|
[
|
||||||
*current_value,
|
*current_value,
|
||||||
*incoming_value,
|
*incoming_value,
|
||||||
] if isinstance(incoming_value, list) else [*current_value, incoming_value]
|
]
|
||||||
|
if isinstance(incoming_value, list)
|
||||||
|
else [*current_value, incoming_value]
|
||||||
|
)
|
||||||
|
|
||||||
if strategy == "merge_object":
|
if strategy == "merge_object":
|
||||||
if current_value is None:
|
if current_value is None:
|
||||||
|
|||||||
@@ -39,7 +39,9 @@ def apply_mapped_state(
|
|||||||
missing_field_message: str,
|
missing_field_message: str,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
if has_overlapping_paths(mapping.values()):
|
if has_overlapping_paths(mapping.values()):
|
||||||
raise WorkflowExecutionError("mapped state patch has overlapping destination paths")
|
raise WorkflowExecutionError(
|
||||||
|
"mapped state patch has overlapping destination paths"
|
||||||
|
)
|
||||||
|
|
||||||
patch: dict[str, Any] = {}
|
patch: dict[str, Any] = {}
|
||||||
for source_field, destination_path in mapping.items():
|
for source_field, destination_path in mapping.items():
|
||||||
@@ -69,8 +71,8 @@ def write_state_value(
|
|||||||
f"executor only supports writes into state.*, got {destination_path!r}"
|
f"executor only supports writes into state.*, got {destination_path!r}"
|
||||||
)
|
)
|
||||||
|
|
||||||
field_name = parts[0]
|
declared_path = ".".join(parts)
|
||||||
declared_field = workflow.state_schema.fields.get(field_name)
|
declared_field = workflow.state_schema.fields.get(declared_path)
|
||||||
merge_strategy = declared_field.merge_strategy if declared_field else "replace"
|
merge_strategy = declared_field.merge_strategy if declared_field else "replace"
|
||||||
key_path = parts
|
key_path = parts
|
||||||
current_value = get_nested_value(state, key_path)
|
current_value = get_nested_value(state, key_path)
|
||||||
|
|||||||
@@ -268,7 +268,10 @@ def test_project_fields_selects_named_keys() -> None:
|
|||||||
registry = build_registry(project_fields)
|
registry = build_registry(project_fields)
|
||||||
|
|
||||||
result = registry["authoring.project_fields"](
|
result = registry["authoring.project_fields"](
|
||||||
{"mapping": {"status": "done", "message": "ok", "debug": True}, "fields": ["status", "message"]},
|
{
|
||||||
|
"mapping": {"status": "done", "message": "ok", "debug": True},
|
||||||
|
"fields": ["status", "message"],
|
||||||
|
},
|
||||||
RuntimeContext(current_node_id="project_fields"),
|
RuntimeContext(current_node_id="project_fields"),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -282,7 +285,10 @@ def test_rename_fields_remaps_existing_keys() -> None:
|
|||||||
registry = build_registry(rename_fields)
|
registry = build_registry(rename_fields)
|
||||||
|
|
||||||
result = registry["authoring.rename_fields"](
|
result = registry["authoring.rename_fields"](
|
||||||
{"mapping": {"provider_status": "done", "provider_message": "ok"}, "renames": {"provider_status": "status", "provider_message": "message"}},
|
{
|
||||||
|
"mapping": {"provider_status": "done", "provider_message": "ok"},
|
||||||
|
"renames": {"provider_status": "status", "provider_message": "message"},
|
||||||
|
},
|
||||||
RuntimeContext(current_node_id="rename_fields"),
|
RuntimeContext(current_node_id="rename_fields"),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -21,7 +21,9 @@ def test_validation_rejects_overlapping_node_input_destinations() -> None:
|
|||||||
out_map={},
|
out_map={},
|
||||||
).validate_structure()
|
).validate_structure()
|
||||||
|
|
||||||
assert any("overlapping node-local input paths" in issue.message for issue in report.errors)
|
assert any(
|
||||||
|
"overlapping node-local input paths" in issue.message for issue in report.errors
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_validation_rejects_overlapping_state_write_destinations() -> None:
|
def test_validation_rejects_overlapping_state_write_destinations() -> None:
|
||||||
@@ -33,7 +35,10 @@ def test_validation_rejects_overlapping_state_write_destinations() -> None:
|
|||||||
},
|
},
|
||||||
).validate_structure()
|
).validate_structure()
|
||||||
|
|
||||||
assert any("overlapping state destination paths" in issue.message for issue in report.errors)
|
assert any(
|
||||||
|
"overlapping state destination paths" in issue.message
|
||||||
|
for issue in report.errors
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _workflow(*, in_map: dict[str, str], out_map: dict[str, str]) -> Workflow:
|
def _workflow(*, in_map: dict[str, str], out_map: dict[str, str]) -> Workflow:
|
||||||
|
|||||||
@@ -0,0 +1,48 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from wf_core import SchemaRef, StateField, StateSchema, Workflow
|
||||||
|
from wf_core.runtime.ops.state import write_state_value
|
||||||
|
|
||||||
|
|
||||||
|
def test_exact_nested_state_path_uses_declared_merge_strategy() -> None:
|
||||||
|
workflow = _workflow(
|
||||||
|
fields={"person.tags": StateField(type="array", merge_strategy="append")}
|
||||||
|
)
|
||||||
|
state = {"person": {"tags": ["seed"]}}
|
||||||
|
|
||||||
|
write_state_value(workflow, state, "state.person.tags", ["next"])
|
||||||
|
|
||||||
|
assert state["person"]["tags"] == ["seed", "next"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_parent_state_declaration_does_not_apply_to_nested_write() -> None:
|
||||||
|
workflow = _workflow(
|
||||||
|
fields={"person": StateField(type="object", merge_strategy="merge_object")}
|
||||||
|
)
|
||||||
|
state = {"person": {"tags": ["seed"]}}
|
||||||
|
|
||||||
|
write_state_value(workflow, state, "state.person.tags", ["next"])
|
||||||
|
|
||||||
|
assert state["person"]["tags"] == ["next"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_undeclared_nested_state_path_defaults_to_replace() -> None:
|
||||||
|
workflow = _workflow(fields={})
|
||||||
|
state = {"person": {"tags": ["seed"]}}
|
||||||
|
|
||||||
|
write_state_value(workflow, state, "state.person.tags", ["next"])
|
||||||
|
|
||||||
|
assert state["person"]["tags"] == ["next"]
|
||||||
|
|
||||||
|
|
||||||
|
def _workflow(*, fields: dict[str, StateField]) -> Workflow:
|
||||||
|
return Workflow(
|
||||||
|
name="nested_state_paths",
|
||||||
|
input_schema=SchemaRef(type="object", properties={}),
|
||||||
|
state_schema=StateSchema(fields=fields),
|
||||||
|
output_schema=SchemaRef(type="object", properties={}),
|
||||||
|
node_defs=[],
|
||||||
|
start="unused",
|
||||||
|
nodes=[],
|
||||||
|
edges=[],
|
||||||
|
)
|
||||||
Reference in New Issue
Block a user