reducer replaces merge_strategy.

This commit is contained in:
lda
2026-05-17 16:23:51 +07:00 Verified
parent c5b43c7dbe
commit 86ffb17ec6
14 changed files with 348 additions and 91 deletions
+22 -15
View File
@@ -145,9 +145,9 @@ internal representation flat:
```python
fields = {
"person.name": StateField(type="string", merge_strategy="replace"),
"person.tags": StateField(type="array", merge_strategy="append"),
"profile": StateField(type="object", merge_strategy="merge_object"),
"person.name": StateField(type="string", reducer="wf.std.replace"),
"person.tags": StateField(type="array", reducer="wf.std.append"),
"profile": StateField(type="object", reducer="wf.std.merge_object"),
}
```
@@ -174,13 +174,13 @@ today.
### Built-in strategies
Existing built-ins remain distinct:
Existing built-in reducers remain distinct:
- `replace`
- `append`
- `merge_object`
- `wf.std.replace`
- `wf.std.append`
- `wf.std.merge_object`
`merge_object` means shallow object merge at the exact destination path, similar
`wf.std.merge_object` means shallow object merge at the exact destination path, similar
to `dict.update` or `operator.or_`. It is not a recursive deep merge.
If recursive merge is ever needed, it should be explicit rather than hidden
@@ -188,16 +188,15 @@ inside `merge_object`.
## Future Reducers
Custom reducers should become a future capability family, similar to reusable
node specs:
Reducers are a capability family, similar to reusable node specs:
- named
- source-owned
- inspectable
- dependency-trackable
State fields should reference reducers declaratively. Workflow artifacts should
not embed arbitrary Python callables.
State fields reference reducers declaratively. Workflow artifacts do not embed
arbitrary Python callables.
Reducers should be pure:
@@ -240,9 +239,17 @@ Implemented in core:
### Phase 3: Reducer capabilities
- design source-owned reducer specs
- add reducer dependency references to state metadata
- resolve pure reducers through runtime/deployment registries
Implemented in core:
- state metadata references named reducers instead of merge strategies
- built-ins are registered as `wf.std.replace`, `wf.std.append`, and
`wf.std.merge_object`
- runtime resolves reducer names before state writes
Still future:
- source-owned reducer specs beyond the built-ins
- reducer dependency references at deployment/platform level
### Phase 4: Core features that depend on this foundation
+42
View File
@@ -36,6 +36,48 @@ Node and workflow boundaries can now reject wrong primitive/container types when
the schema declares them. This matters before workflows are generated by an LLM
or backed by arbitrary MCP tools.
## Authoring Footguns
### Prefer explicit entry arrays at LLM-facing boundaries
`dict[str, SomeModel]` is often pleasant in Python but weak as a human- or
LLM-facing schema. It commonly appears as a generic object with arbitrary keys,
and many clients communicate or render it much less clearly than an explicit
list shape.
Prefer:
```python
class Entry(BaseModel):
key: str
value: SomeModel
entries: list[Entry]
```
over:
```python
entries: dict[str, SomeModel]
```
when the schema is meant for MCP tools, LLM planning, or durable workflow
authoring contracts. Internal Python state can still use dictionaries when that
is the right runtime shape.
### Do not assume all JSON Schema consumers handle references equally
Pydantic-generated schemas may use `$defs` and local `$ref` references for
nested models. The runtime delegates validation to `jsonschema`, which supports
that structure, but display layers and downstream consumers may vary in how well
they present or reason about referenced shapes.
`wf_authoring` resolves the local Pydantic `$ref -> $defs` pattern only for its
own state-field projection. That helper is not a general-purpose JSON Schema
flattener, and arbitrary external schemas should not be assumed to share the
same shape.
## Intended Seam
The schema adapter lives behind:
+1 -1
View File
@@ -416,7 +416,7 @@ At minimum each declared field may carry:
```text
StateField
.type
.merge_strategy // replace | append | merge_object
.reducer // wf.std.replace | wf.std.append | wf.std.merge_object | ...
.trace? // whether to include in trace by default
```
@@ -0,0 +1,78 @@
# Reducer Capabilities Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace `merge_strategy` with named pure reducer references across core and authoring, using `wf.std.replace` as the default reducer.
**Architecture:** Introduce a small reducer registry in `wf_core`, register the current three built-ins as reducers, and have state writes resolve every declared reducer name through that registry. Keep undeclared state paths using default replace semantics. Migrate `wf_authoring.state_field()` and all state metadata/tests/docs to reducer names in the same pass so there is one merge concept in the codebase.
**Tech Stack:** Python, Pydantic, pytest, existing `wf_core` runtime and `wf_authoring` schema projection.
---
## File Structure
- Modify `src/wf_core/models/schemas.py`
- replace `merge_strategy` with `reducer`
- Replace/refactor `src/wf_core/runtime/ops/merges.py`
- reducer callable type
- built-in reducer functions
- default reducer registry
- reducer application helper
- Modify `src/wf_core/runtime/ops/state.py`
- resolve reducer names from state fields
- use default replace reducer for undeclared paths
- Modify `src/wf_authoring/schemas.py`
- expose `state_field(reducer=...)`
- project reducer metadata through flattened state paths
- Modify tests under `tests/core/`, `tests/authoring/`, and `tests/rewrite/`
- migrate old metadata
- add unknown reducer coverage
- Update docs mentioning `merge_strategy`
## Tasks
### Task 1: Pin Reducer Semantics
- [ ] Add tests proving:
- `StateField(type="string")` defaults to `wf.std.replace`
- `wf.std.append` preserves append behavior
- `wf.std.merge_object` preserves shallow object merge behavior
- unknown reducer names fail clearly
- exact nested state paths still use their own reducer
- [ ] Run focused core tests and confirm failure before implementation.
### Task 2: Replace Core Merge Strategy With Reducers
- [ ] Replace `merge_strategy` on `StateField` with `reducer`.
- [ ] Add reducer functions for `wf.std.replace`, `wf.std.append`, and `wf.std.merge_object`.
- [ ] Add a registry lookup path that raises for unknown reducer names.
- [ ] Update `write_state_value()` to resolve declared reducers and use `wf.std.replace` for undeclared paths.
- [ ] Run focused core tests and confirm reducer behavior is green.
### Task 3: Migrate Authoring
- [ ] Change `StateFieldMetadata` and `state_field()` to use `reducer`.
- [ ] Preserve nested metadata projection under reducer names.
- [ ] Update authoring/rewrite fixtures from `merge_strategy=` to `reducer=`.
- [ ] Run focused authoring tests and confirm they pass.
### Task 4: Update Docs
- [ ] Replace docs that describe `merge_strategy` with reducer terminology.
- [ ] Update examples to show reducer names, including the default replace reducer.
- [ ] Keep the design point that reducers are pure and source-owned.
### Task 5: Verify
- [ ] Run `uv run --with pytest pytest tests/core tests/authoring tests/rewrite -q`
- [ ] Run `uv run --with pytest pytest -q`
- [ ] Run `uv run basedpyright --level error`
## Non-Goals
- custom user-authored reducer registration through MCP/platform sources
- reducer parameters/configuration
- async reducers
- parallel foreach
- compatibility shims for `merge_strategy`
@@ -0,0 +1,109 @@
# Reducer Capabilities Design
## Goal
Make state merging a single capability system instead of keeping a built-in
`merge_strategy` path beside future custom reducers.
## Decision
`StateField` should reference exactly one reducer:
```python
class StateField(BaseModel):
type: str
reducer: str = "wf.std.replace"
trace: bool = True
default: Any = None
```
The current built-ins become the first reducer library:
- `wf.std.replace`
- `wf.std.append`
- `wf.std.merge_object`
There is no separate `merge_strategy` field after this migration.
## Reducer Contract
Reducers are pure merge functions:
```text
current_value, incoming_value -> merged_value
```
They do not receive node ids, frame ids, paths, timestamps, or other execution
context. If behavior needs workflow context, it belongs in nodes or graph
structure instead.
Reducers are named and resolved at runtime from a registry. Workflow artifacts
store the reducer name, not a Python callable.
## Runtime Model
`wf_core` owns:
- a reducer callable protocol/type
- a reducer registry
- default registration of the three built-ins
- lookup and execution during state writes
Missing reducer names are execution errors. Reducer failures are wrapped with
the destination path so the failing state write is obvious.
## Authoring Model
`wf_authoring.state_field()` changes from:
```python
state_field(merge_strategy="append")
```
to:
```python
state_field(reducer="wf.std.append")
```
Nested authored state projection continues to flatten exact state paths and now
copies reducer references onto those flattened fields.
## Why Reducer-Only
Keeping both `merge_strategy` and `reducer` would create two concepts for the
same job. Turning the current built-ins into reducers gives us:
- one merge abstraction
- source-owned reusable behavior
- inspectable future reducer libraries
- a direct path to custom reducers such as `wf.std.max`,
`wf.std.set_union`, or user-authored reducers
## Error Handling
- unknown reducer name: execution error before the state write commits
- reducer rejects a value shape: execution error from that reducer
- reducers remain pure, so there is no side-effect rollback problem
## Compatibility
This is an intentional model migration:
- core `StateField.merge_strategy` is removed
- authoring `state_field(merge_strategy=...)` is removed
- docs and tests migrate to reducer references
The project is still early enough that keeping both public shapes would create
more confusion than value.
## Testing
Tests should prove:
- `wf.std.replace` preserves current replace behavior
- `wf.std.append` preserves current append behavior
- `wf.std.merge_object` preserves current shallow object merge behavior
- exact nested state paths still select their own reducer
- unknown reducers fail clearly
- authoring metadata projects reducer names through nested state schemas
+6 -6
View File
@@ -24,15 +24,15 @@ def build_demo_workflow() -> Workflow:
"fields": {
"folder_id": {"type": "string"},
"should_email": {"type": "boolean"},
"documents": {"type": "array", "merge_strategy": "replace"},
"item_summaries": {"type": "array", "merge_strategy": "append"},
"summary": {"type": "string", "merge_strategy": "replace"},
"approved": {"type": "boolean", "merge_strategy": "replace"},
"documents": {"type": "array", "reducer": "wf.std.replace"},
"item_summaries": {"type": "array", "reducer": "wf.std.append"},
"summary": {"type": "string", "reducer": "wf.std.replace"},
"approved": {"type": "boolean", "reducer": "wf.std.replace"},
"approval_comment": {
"type": "string",
"merge_strategy": "replace",
"reducer": "wf.std.replace",
},
"email_status": {"type": "string", "merge_strategy": "replace"},
"email_status": {"type": "string", "reducer": "wf.std.replace"},
}
},
"output_schema": {
+5 -7
View File
@@ -1,7 +1,7 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Iterator, Literal
from typing import Any, Iterator
from pydantic import BaseModel, TypeAdapter
@@ -15,17 +15,17 @@ StateSchemaLike = StateSchema | type[BaseModel] | type[Any] | dict[str, Any]
class StateFieldMetadata:
"""Authoring metadata attached to BaseModel state fields."""
merge_strategy: Literal["replace", "append", "merge_object"] = "replace"
reducer: str = "wf.std.replace"
trace: bool = True
def state_field(
*,
merge_strategy: Literal["replace", "append", "merge_object"] = "replace",
reducer: str = "wf.std.replace",
trace: bool = True,
) -> StateFieldMetadata:
"""Declare workflow state behavior for an Annotated BaseModel field."""
return StateFieldMetadata(merge_strategy=merge_strategy, trace=trace)
return StateFieldMetadata(reducer=reducer, trace=trace)
def schema_ref_from(value: SchemaLike) -> SchemaRef:
@@ -51,9 +51,7 @@ def state_schema_from(value: StateSchemaLike) -> StateSchema:
fields = {
path: StateField(
type=_state_field_type(property_schema),
merge_strategy=metadata_by_name.get(
path, StateFieldMetadata()
).merge_strategy,
reducer=metadata_by_name.get(path, StateFieldMetadata()).reducer,
trace=metadata_by_name.get(path, StateFieldMetadata()).trace,
default=_state_field_default(value, path, property_schema),
)
+2 -2
View File
@@ -1,6 +1,6 @@
from __future__ import annotations
from typing import Any, Literal
from typing import Any
from pydantic import BaseModel, ConfigDict, Field
@@ -20,7 +20,7 @@ class StateField(BaseModel):
"""Declared state path plus its runtime merge behavior."""
type: str
merge_strategy: Literal["replace", "append", "merge_object"] = "replace"
reducer: str = "wf.std.replace"
trace: bool = True
default: Any = None
+39 -34
View File
@@ -1,57 +1,62 @@
from __future__ import annotations
from collections.abc import Callable, Mapping
from typing import Any
from wf_core.errors import WorkflowExecutionError
Reducer = Callable[[Any, Any], Any]
def apply_builtin_merge(
*,
strategy: str,
current_value: Any,
incoming_value: Any,
destination_path: str,
) -> Any:
"""Apply one built-in merge rule.
This is the future seam for source-owned reducer libraries. The current core
still supports only built-in rules and keeps them pure over current and
incoming values.
"""
if strategy == "replace":
def replace_reducer(_current_value: Any, incoming_value: Any) -> Any:
"""Replace the current state value with the incoming value."""
return incoming_value
if strategy == "append":
def append_reducer(current_value: Any, incoming_value: Any) -> Any:
"""Append one value or many values into a list-valued state path."""
if current_value is None:
return (
[incoming_value]
if not isinstance(incoming_value, list)
else incoming_value
)
return [incoming_value] if not isinstance(incoming_value, list) else incoming_value
if not isinstance(current_value, list):
raise WorkflowExecutionError(
f"cannot append into non-list state path {destination_path!r}"
)
raise TypeError("cannot append into non-list state value")
return (
[
*current_value,
*incoming_value,
]
[*current_value, *incoming_value]
if isinstance(incoming_value, list)
else [*current_value, incoming_value]
)
if strategy == "merge_object":
def merge_object_reducer(current_value: Any, incoming_value: Any) -> Any:
"""Shallow-merge object values at one exact state path."""
if current_value is None:
if not isinstance(incoming_value, dict):
raise WorkflowExecutionError(
f"cannot merge non-object value into {destination_path!r}"
)
raise TypeError("cannot merge non-object value")
return dict(incoming_value)
if not isinstance(current_value, dict) or not isinstance(incoming_value, dict):
raise WorkflowExecutionError(
f"merge_object requires dict values at {destination_path!r}"
)
raise TypeError("merge_object requires dict values")
return current_value | incoming_value
raise WorkflowExecutionError(f"unknown merge strategy {strategy!r}")
DEFAULT_REDUCERS: Mapping[str, Reducer] = {
"wf.std.replace": replace_reducer,
"wf.std.append": append_reducer,
"wf.std.merge_object": merge_object_reducer,
}
def apply_reducer(
*,
reducer_name: str,
current_value: Any,
incoming_value: Any,
destination_path: str,
reducers: Mapping[str, Reducer] = DEFAULT_REDUCERS,
) -> Any:
"""Apply one named pure reducer to a state write."""
reducer = reducers.get(reducer_name)
if reducer is None:
raise WorkflowExecutionError(f"unknown reducer {reducer_name!r}")
try:
return reducer(current_value, incoming_value)
except TypeError as exc:
raise WorkflowExecutionError(f"{exc} at {destination_path!r}") from exc
+4 -4
View File
@@ -12,7 +12,7 @@ from wf_core.paths import (
set_nested_value,
split_graph_path,
)
from wf_core.runtime.ops.merges import apply_builtin_merge
from wf_core.runtime.ops.merges import apply_reducer
def apply_output_map(
@@ -73,11 +73,11 @@ def write_state_value(
declared_path = ".".join(parts)
declared_field = workflow.state_schema.fields.get(declared_path)
merge_strategy = declared_field.merge_strategy if declared_field else "replace"
reducer_name = declared_field.reducer if declared_field else "wf.std.replace"
key_path = parts
current_value = get_nested_value(state, key_path)
merged_value = apply_builtin_merge(
strategy=merge_strategy,
merged_value = apply_reducer(
reducer_name=reducer_name,
current_value=current_value,
incoming_value=value,
destination_path=destination_path,
+2 -2
View File
@@ -41,7 +41,7 @@ class AutoBindState(BaseModel):
class AppendState(BaseModel):
items: Annotated[list[str], state_field(merge_strategy="append")] = Field(
items: Annotated[list[str], state_field(reducer="wf.std.append")] = Field(
default_factory=list
)
@@ -54,7 +54,7 @@ class DefaultedState(BaseModel):
class NestedPersonState(BaseModel):
name: str
tags: Annotated[list[str], state_field(merge_strategy="append")] = Field(
tags: Annotated[list[str], state_field(reducer="wf.std.append")] = Field(
default_factory=list
)
+4 -4
View File
@@ -46,7 +46,7 @@ def test_builder_accepts_typeddict_for_json_schema_refs() -> None:
assert workflow.input_schema.properties["text"]["type"] == "string"
def test_state_basemodel_can_declare_merge_strategy_with_annotated_metadata() -> None:
def test_state_basemodel_can_declare_reducer_with_annotated_metadata() -> None:
builder = WorkflowBuilder(
name="state_metadata_demo",
input_schema=WorkflowInput,
@@ -58,7 +58,7 @@ def test_state_basemodel_can_declare_merge_strategy_with_annotated_metadata() ->
workflow = builder.compile()
assert workflow.state_schema.fields["items"].type == "array"
assert workflow.state_schema.fields["items"].merge_strategy == "append"
assert workflow.state_schema.fields["items"].reducer == "wf.std.append"
def test_state_basemodel_seeds_safe_initial_defaults() -> None:
@@ -96,5 +96,5 @@ def test_nested_state_basemodel_projects_parent_and_child_paths() -> None:
assert workflow.state_schema.fields["person"].type == "object"
assert workflow.state_schema.fields["person.name"].type == "string"
assert workflow.state_schema.fields["person.tags"].type == "array"
assert workflow.state_schema.fields["person"].merge_strategy == "replace"
assert workflow.state_schema.fields["person.tags"].merge_strategy == "append"
assert workflow.state_schema.fields["person"].reducer == "wf.std.replace"
assert workflow.state_schema.fields["person.tags"].reducer == "wf.std.append"
+19 -3
View File
@@ -4,9 +4,9 @@ from wf_core import SchemaRef, StateField, StateSchema, Workflow
from wf_core.runtime.ops.state import write_state_value
def test_exact_nested_state_path_uses_declared_merge_strategy() -> None:
def test_exact_nested_state_path_uses_declared_reducer() -> None:
workflow = _workflow(
fields={"person.tags": StateField(type="array", merge_strategy="append")}
fields={"person.tags": StateField(type="array", reducer="wf.std.append")}
)
state = {"person": {"tags": ["seed"]}}
@@ -17,7 +17,7 @@ def test_exact_nested_state_path_uses_declared_merge_strategy() -> None:
def test_parent_state_declaration_does_not_apply_to_nested_write() -> None:
workflow = _workflow(
fields={"person": StateField(type="object", merge_strategy="merge_object")}
fields={"person": StateField(type="object", reducer="wf.std.merge_object")}
)
state = {"person": {"tags": ["seed"]}}
@@ -35,6 +35,22 @@ def test_undeclared_nested_state_path_defaults_to_replace() -> None:
assert state["person"]["tags"] == ["next"]
def test_state_field_defaults_to_replace_reducer() -> None:
assert StateField(type="string").reducer == "wf.std.replace"
def test_unknown_state_reducer_fails_clearly() -> None:
workflow = _workflow(fields={"person.tags": StateField(type="array", reducer="x.nope")})
state = {"person": {"tags": ["seed"]}}
try:
write_state_value(workflow, state, "state.person.tags", ["next"])
except Exception as exc:
assert "unknown reducer 'x.nope'" in str(exc)
else:
raise AssertionError("expected unknown reducer to fail")
def _workflow(*, fields: dict[str, StateField]) -> Workflow:
return Workflow(
name="nested_state_paths",
+4 -2
View File
@@ -111,7 +111,7 @@ class UnsophisticatedPool(TypedDict):
class Storage(BaseModel):
"i NEED to do this?"
storage: Annotated[list[Entity], state_field(merge_strategy="append")] = Field(
storage: Annotated[list[Entity], state_field(reducer="wf.std.append")] = Field(
default_factory=list
) # add!
@@ -125,7 +125,9 @@ class PartialRates(SophisticatedRates, total=False):
class Rates(BaseModel):
rates: Annotated[PartialRates, state_field(merge_strategy="merge_object")] # or_!
rates: Annotated[
PartialRates, state_field(reducer="wf.std.merge_object")
] # or_!
class CurrentPools(BaseModel):