concurrent foreach work

This commit is contained in:
lda
2026-05-22 13:27:05 +07:00 Verified
parent 4a7f7a5bec
commit 0a6500daf2
15 changed files with 1991 additions and 30 deletions
@@ -95,6 +95,12 @@ Patch creation and commit must extract/reuse the existing node output
validation, output binding, and reducer logic. Concurrent foreach must not validation, output binding, and reducer logic. Concurrent foreach must not
create a second write system. create a second write system.
Current sync V1 implements the barrier commit path only for `loop -> one node ->
END` item bodies. The runtime includes an explicit no-op overlay seam
(`state_view_for_frame`) so the next slice can add lineage-local reads without
rewiring node execution. Until that seam becomes real, multi-step concurrent
item bodies are rejected instead of reading stale parent state.
## Merge and Reducer Rules ## Merge and Reducer Rules
At a barrier, missing reducer means default replace only for single-writer At a barrier, missing reducer means default replace only for single-writer
@@ -0,0 +1,166 @@
# Concurrent Foreach Phase 4 Roadmap
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Split concurrent foreach execution into safe, independently testable runtime slices.
**Architecture:** `foreach(mode="concurrent")` is the workflow mode. Sync runtime should interleave admitted item frames one node call at a time; async runtime may later run admitted async node handlers simultaneously. All item writes must flow through `StatePatch` and commit at a foreach barrier, not directly from child frames into shared state.
**Tech Stack:** Python 3.14, dataclasses, Pydantic v2, pytest, `wf_core.runtime.scheduler`, `wf_core.runtime.foreach_state`, `wf_core.runtime.ops.state`.
---
## Current State
Already implemented:
- `ForeachNode.mode` accepts canonical `"concurrent"`.
- `ForeachConcurrentPolicy` exists with `max_active`, `max_outstanding`, and `interrupt="quiesce"`.
- Legacy `mode="parallel"` / `parallel={...}` parse into canonical concurrent shape.
- `ForeachItemErrorPolicy` exists with `fail`, `skip`, and `collect`.
- `completed_with_errors` is derived for `skip` and `collect`.
- `collect_to` is validated as a declared array state field.
- `StatePatch`, `build_output_patch(...)`, and `commit_state_patch(...)` exist.
- `ForeachBarrierState`, `PendingItemResult`, and `ItemErrorRecord` exist.
- Serial foreach progress now uses `ForeachBarrierState`.
- Runtime still rejects `foreach(mode="concurrent")`.
## Non-Goals For Phase 4
- Do not implement Fork/Gather graph nodes.
- Do not turn `JoinNode` into a real barrier.
- Do not run sync node handlers in threads or processes.
- Do not add OpenTelemetry.
- Do not add platform-level source/tool semaphores.
- Do not add persistent run storage.
## Slice 1: Sync Concurrent Foreach, Fail-Only
Implement first because it proves the scheduler and frame admission model without async task orchestration or handled item failures.
Scope:
- `foreach(mode="concurrent", item_error.action="fail")` runs in sync runtime.
- Parent foreach admits up to `concurrent.max_active` item frames.
- Scheduler interleaves item frames one step at a time.
- Child output writes are buffered as per-item `StatePatch` objects.
- Barrier commits all successful item patches only when every item succeeds.
- Any item runtime failure fails the whole run.
- `skip` and `collect` remain runtime-unsupported for concurrent mode.
Plan:
- See [`2026-05-22-concurrent-foreach-v1-sync-fail-only.md`](2026-05-22-concurrent-foreach-v1-sync-fail-only.md).
## Slice 2: Barrier Commit Conflict Semantics
Implement after Slice 1 if V1 keeps conflict behavior too conservative.
Scope:
- Detect sibling lineage writes to the same state path.
- If exactly one lineage writes a destination path, default replace is allowed.
- If multiple sibling lineages write the same destination path, a declared reducer is required.
- Ancestor/descendant writes across sibling lineages are conflicts unless an explicit future merge strategy covers them.
- Commit order is item index order, never completion order.
Files likely touched:
- `src/wf_core/runtime/foreach_state.py`
- `src/wf_core/runtime/ops/state.py`
- `src/wf_core/runtime/ops/foreach.py`
- `tests/core/test_concurrent_foreach.py`
Key tests:
- `test_concurrent_foreach_rejects_sibling_writes_without_reducer`
- `test_concurrent_foreach_applies_reducer_in_item_index_order`
- `test_concurrent_foreach_rejects_ancestor_descendant_write_conflict`
## Slice 3: Item Error Policies
Implement after barrier success commits are correct.
Scope:
- `item_error.action="skip"` continues after item runtime failures.
- `item_error.action="collect"` continues and writes structured errors to `collect_to`.
- Both emit `completed_with_errors` if at least one item failed.
- `collect` writes an empty list and emits `done` if all items succeed.
- Failed item frames remain `FAILED`; parent foreach decides whether the failure is handled.
Files likely touched:
- `src/wf_core/runtime/foreach_state.py`
- `src/wf_core/runtime/ops/foreach.py`
- `tests/core/test_concurrent_foreach_errors.py`
Key tests:
- `test_concurrent_foreach_skip_emits_completed_with_errors`
- `test_concurrent_foreach_collect_writes_ordered_error_records`
- `test_concurrent_foreach_collect_writes_empty_list_on_clean_success`
## Slice 4: Async Concurrent Foreach
Implement only after sync semantics are stable.
Scope:
- Async runtime may have multiple async node handler calls in flight.
- `concurrent.max_active` caps admitted/running item work.
- Sync handlers are still called normally; no thread/process executor.
- Trace remains append-only chronological execution history.
- Barrier commit order remains item index order.
Files likely touched:
- `src/wf_core/runtime/engine.py`
- `src/wf_core/runtime/step.py`
- `src/wf_core/runtime/ops/nodes.py`
- `src/wf_core/runtime/ops/foreach.py`
- `tests/core/test_concurrent_foreach_async.py`
Key tests:
- `test_async_concurrent_foreach_respects_max_active`
- `test_async_concurrent_foreach_commits_in_item_index_order`
## Slice 5: Interrupt Quiescence
Implement after async execution exists.
Scope:
- If any concurrent item interrupts, the whole run pauses.
- No new item frames are admitted after the interrupt.
- Already-started async node calls drain to pending results.
- The caller gets control only at a quiescent point.
- Pending results do not commit until resume/commit policy allows it.
Files likely touched:
- `src/wf_core/runtime/engine.py`
- `src/wf_core/runtime/preparation.py`
- `src/wf_core/runtime/ops/interrupts.py`
- `src/wf_core/runtime/ops/foreach.py`
- `tests/core/test_concurrent_foreach_interrupts.py`
Key tests:
- `test_concurrent_foreach_interrupt_returns_after_quiescence`
- `test_resume_prioritizes_interrupted_item_frame_before_siblings`
## Execution Order
1. Sync concurrent foreach, fail-only.
2. Barrier conflict semantics, if not fully covered by slice 1.
3. `skip` / `collect` item error policies.
4. Async concurrent foreach.
5. Interrupt quiescence.
## Self-Review
- Spec coverage: the roadmap covers scheduler admission, barrier commits, reducer conflicts, handled item failures, async handler execution, and interrupt quiescence.
- Placeholder scan: each slice has scope, likely files, and named tests; concrete code lives in the slice-specific plan.
- Type consistency: the roadmap uses canonical `concurrent`, `ForeachConcurrentPolicy`, `ForeachBarrierState`, `PendingItemResult`, and `StatePatch`.
@@ -22,6 +22,10 @@
- Phase 4 is not implemented: `foreach(mode="concurrent")` still validates as a - Phase 4 is not implemented: `foreach(mode="concurrent")` still validates as a
model shape but runtime execution rejects it until concurrent scheduling, model shape but runtime execution rejects it until concurrent scheduling,
barrier commits, and item failure handling are implemented. barrier commits, and item failure handling are implemented.
- Phase 4 is expanded into a dedicated roadmap:
[`2026-05-22-concurrent-foreach-phase4-roadmap.md`](2026-05-22-concurrent-foreach-phase4-roadmap.md).
Start with
[`2026-05-22-concurrent-foreach-v1-sync-fail-only.md`](2026-05-22-concurrent-foreach-v1-sync-fail-only.md).
--- ---
File diff suppressed because it is too large Load Diff
+46
View File
@@ -6,6 +6,7 @@ from typing import Any, Literal
from wf_core.errors import WorkflowExecutionError from wf_core.errors import WorkflowExecutionError
from wf_core.run_state import ExecutionFrame from wf_core.run_state import ExecutionFrame
from wf_core.runtime.ops.state import StatePatch from wf_core.runtime.ops.state import StatePatch
from wf_core.runtime.scheduler import ForeachIterationMetadata
_BARRIER_METADATA_KEY = "foreach_barriers" _BARRIER_METADATA_KEY = "foreach_barriers"
@@ -120,6 +121,7 @@ class ForeachBarrierState:
"""Resumable state owned by one foreach parent frame.""" """Resumable state owned by one foreach parent frame."""
next_index: int = 0 next_index: int = 0
mode: Literal["serial", "concurrent"] = "serial"
active_frame_ids: tuple[str, ...] = () active_frame_ids: tuple[str, ...] = ()
outstanding_frame_ids: tuple[str, ...] = () outstanding_frame_ids: tuple[str, ...] = ()
pending_results: dict[int, PendingItemResult] = field(default_factory=dict) pending_results: dict[int, PendingItemResult] = field(default_factory=dict)
@@ -156,11 +158,14 @@ class ForeachBarrierState:
if not isinstance(raw, dict): if not isinstance(raw, dict):
raise WorkflowExecutionError("malformed foreach barrier state") raise WorkflowExecutionError("malformed foreach barrier state")
next_index = raw.get("next_index") next_index = raw.get("next_index")
mode = raw.get("mode", "serial")
active_frame_ids = _string_tuple(raw.get("active_frame_ids", ())) active_frame_ids = _string_tuple(raw.get("active_frame_ids", ()))
outstanding_frame_ids = _string_tuple(raw.get("outstanding_frame_ids", ())) outstanding_frame_ids = _string_tuple(raw.get("outstanding_frame_ids", ()))
pending_results = raw.get("pending_results", {}) pending_results = raw.get("pending_results", {})
if not isinstance(next_index, int): if not isinstance(next_index, int):
raise WorkflowExecutionError("malformed foreach barrier next_index") raise WorkflowExecutionError("malformed foreach barrier next_index")
if mode not in {"serial", "concurrent"}:
raise WorkflowExecutionError("malformed foreach barrier mode")
if not isinstance(pending_results, dict): if not isinstance(pending_results, dict):
raise WorkflowExecutionError("malformed foreach barrier pending results") raise WorkflowExecutionError("malformed foreach barrier pending results")
parsed_results: dict[int, PendingItemResult] = {} parsed_results: dict[int, PendingItemResult] = {}
@@ -174,6 +179,7 @@ class ForeachBarrierState:
parsed_results[index] = PendingItemResult.from_metadata(raw_result) parsed_results[index] = PendingItemResult.from_metadata(raw_result)
return cls( return cls(
next_index=next_index, next_index=next_index,
mode=mode,
active_frame_ids=active_frame_ids, active_frame_ids=active_frame_ids,
outstanding_frame_ids=outstanding_frame_ids, outstanding_frame_ids=outstanding_frame_ids,
pending_results=parsed_results, pending_results=parsed_results,
@@ -196,6 +202,7 @@ class ForeachBarrierState:
def to_metadata(self) -> dict[str, Any]: def to_metadata(self) -> dict[str, Any]:
return { return {
"next_index": self.next_index, "next_index": self.next_index,
"mode": self.mode,
"active_frame_ids": list(self.active_frame_ids), "active_frame_ids": list(self.active_frame_ids),
"outstanding_frame_ids": list(self.outstanding_frame_ids), "outstanding_frame_ids": list(self.outstanding_frame_ids),
"pending_results": { "pending_results": {
@@ -204,6 +211,45 @@ class ForeachBarrierState:
}, },
} }
def start_child(self, frame_id: str) -> None:
"""Record one admitted child frame as active and outstanding."""
if frame_id in self.active_frame_ids or frame_id in self.outstanding_frame_ids:
raise WorkflowExecutionError(
f"foreach child frame {frame_id!r} already active"
)
self.active_frame_ids = (*self.active_frame_ids, frame_id)
self.outstanding_frame_ids = (*self.outstanding_frame_ids, frame_id)
def finish_child(self, frame_id: str) -> None:
"""Record one child frame as no longer active or outstanding."""
self.active_frame_ids = tuple(
item for item in self.active_frame_ids if item != frame_id
)
self.outstanding_frame_ids = tuple(
item for item in self.outstanding_frame_ids if item != frame_id
)
def add_success_patch(
self, *, index: int, frame_id: str, patch: StatePatch
) -> None:
"""Buffer one successful item patch by item index."""
self.pending_results[index] = PendingItemResult(
index=index,
frame_id=frame_id,
status="succeeded",
patch=patch,
)
def item_frame_owner(frame: ExecutionFrame) -> tuple[str, str, int] | None:
"""Return parent frame id, foreach node id, and item index for item frames."""
if frame.kind != "foreach_iteration" or frame.parent_frame_id is None:
return None
metadata = ForeachIterationMetadata.from_frame(frame)
if metadata is None:
return None
return frame.parent_frame_id, metadata.foreach_node_id, metadata.loop_index
def _string_tuple(raw: object) -> tuple[str, ...]: def _string_tuple(raw: object) -> tuple[str, ...]:
if isinstance(raw, tuple) and all(isinstance(item, str) for item in raw): if isinstance(raw, tuple) and all(isinstance(item, str) for item in raw):
+2 -2
View File
@@ -15,7 +15,7 @@ from wf_core.runtime.ops.schemas import validate_payload_against_schema
from wf_core.runtime.ops.state import project_output from wf_core.runtime.ops.state import project_output
from wf_core.runtime.scheduler import ( from wf_core.runtime.scheduler import (
mark_frame_pending, mark_frame_pending,
wake_parent_if_children_complete, wake_parent_for_child_progress,
) )
from wf_core.tokens import END from wf_core.tokens import END
@@ -81,7 +81,7 @@ def advance_frame(
if next_node_id == END: if next_node_id == END:
frame.status = FrameStatus.COMPLETED frame.status = FrameStatus.COMPLETED
frame.finished_at_node_id = END frame.finished_at_node_id = END
wake_parent_if_children_complete(run, frame.id) wake_parent_for_child_progress(run, frame.id)
else: else:
frame.finished_at_node_id = None frame.finished_at_node_id = None
mark_frame_pending(run, frame.id) mark_frame_pending(run, frame.id)
+240 -23
View File
@@ -1,19 +1,24 @@
from __future__ import annotations from __future__ import annotations
from collections.abc import Mapping
from wf_core.conditions import safe_resolve_path from wf_core.conditions import safe_resolve_path
from wf_core.errors import WorkflowExecutionError from wf_core.errors import WorkflowExecutionError
from wf_core.models.steps import ForeachNode from wf_core.models.steps import ForeachNode, NodeUse
from wf_core.models.workflow import Workflow from wf_core.models.workflow import Workflow
from wf_core.run_state import ExecutionFrame, FrameStatus, RunState, StepExecutionResult from wf_core.run_state import ExecutionFrame, FrameStatus, RunState, StepExecutionResult
from wf_core.runtime.foreach_state import ForeachBarrierState from wf_core.runtime.foreach_state import ForeachBarrierState
from wf_core.runtime.ops.flow import advance_frame, append_step_result_trace from wf_core.runtime.ops.flow import advance_frame, append_step_result_trace
from wf_core.runtime.ops.frames import frame_context_values from wf_core.runtime.ops.frames import frame_context_values
from wf_core.runtime.ops.index import WorkflowIndex from wf_core.runtime.ops.index import WorkflowIndex
from wf_core.runtime.ops.merges import ReducerDefinition
from wf_core.runtime.ops.state import build_barrier_patch, commit_state_patch
from wf_core.runtime.scheduler import ( from wf_core.runtime.scheduler import (
ForeachIterationMetadata, ForeachIterationMetadata,
add_frame, add_frame,
block_frame_on_children, block_frame_on_children,
) )
from wf_core.tokens import END
def step_foreach( def step_foreach(
@@ -21,25 +26,32 @@ def step_foreach(
run: RunState, run: RunState,
step: ForeachNode, step: ForeachNode,
index: WorkflowIndex, index: WorkflowIndex,
*,
reducers: Mapping[str, ReducerDefinition] | None = None,
) -> RunState:
if step.mode == "serial":
return _step_foreach_serial(workflow, run, step, index)
return _step_foreach_concurrent(
workflow,
run,
step,
index,
reducers=reducers,
)
def _step_foreach_serial(
workflow: Workflow,
run: RunState,
step: ForeachNode,
index: WorkflowIndex,
) -> RunState: ) -> RunState:
if step.mode != "serial": if step.mode != "serial":
raise WorkflowExecutionError( raise WorkflowExecutionError("serial foreach helper received non-serial mode")
"concurrent foreach execution is not implemented yet"
)
frame = run.current_frame() frame = run.current_frame()
barrier = ForeachBarrierState.from_frame(frame, step.id) or ForeachBarrierState() barrier = ForeachBarrierState.from_frame(frame, step.id) or ForeachBarrierState()
iterable = _resolve_foreach_iterable(run, frame, step)
iterable = safe_resolve_path(
str(step.over),
state=run.state,
workflow_input=run.workflow_input,
context=frame_context_values(frame),
)
if not isinstance(iterable, list):
raise WorkflowExecutionError(
f"foreach source {str(step.over)!r} must resolve to a list"
)
loop_index = barrier.next_index loop_index = barrier.next_index
if loop_index >= len(iterable): if loop_index >= len(iterable):
@@ -62,17 +74,10 @@ def step_foreach(
return run return run
loop_start = index.next_node_id(frame.node_id, "loop") loop_start = index.next_node_id(frame.node_id, "loop")
item = iterable[loop_index] item = iterable[loop_index]
barrier.next_index = loop_index + 1 barrier.next_index = loop_index + 1
barrier.save_to_frame(frame, step.id) barrier.save_to_frame(frame, step.id)
child_id = f"{frame.id}:{step.id}:{loop_index}" child_id = f"{frame.id}:{step.id}:{loop_index}"
child_metadata = ForeachIterationMetadata(
foreach_node_id=step.id,
loop_index=loop_index,
loop_item=item,
loop_alias=step.as_,
)
add_frame( add_frame(
run, run,
ExecutionFrame( ExecutionFrame(
@@ -81,7 +86,12 @@ def step_foreach(
node_id=loop_start, node_id=loop_start,
status=FrameStatus.PENDING, status=FrameStatus.PENDING,
parent_frame_id=frame.id, parent_frame_id=frame.id,
metadata=child_metadata.to_metadata(), metadata=ForeachIterationMetadata(
foreach_node_id=step.id,
loop_index=loop_index,
loop_item=item,
loop_alias=step.as_,
).to_metadata(),
), ),
ready=True, ready=True,
) )
@@ -101,3 +111,210 @@ def step_foreach(
) )
run.sync_from_current_frame() run.sync_from_current_frame()
return run return run
def _step_foreach_concurrent(
workflow: Workflow,
run: RunState,
step: ForeachNode,
index: WorkflowIndex,
*,
reducers: Mapping[str, ReducerDefinition] | None = None,
) -> RunState:
if step.item_error.action != "fail":
raise WorkflowExecutionError(
"concurrent foreach v1 only supports item_error.action='fail'"
)
if step.concurrent is None:
raise WorkflowExecutionError("concurrent foreach requires concurrent policy")
_validate_single_node_loop_body(index, step)
frame = run.current_frame()
barrier = ForeachBarrierState.from_frame(frame, step.id)
if barrier is None:
barrier = ForeachBarrierState(mode="concurrent")
elif barrier.mode != "concurrent":
raise WorkflowExecutionError("malformed concurrent foreach barrier mode")
_finish_completed_children(run, barrier)
iterable = _resolve_foreach_iterable(run, frame, step)
_admit_concurrent_children(
run=run,
frame=frame,
step=step,
index=index,
barrier=barrier,
iterable=iterable,
)
if barrier.next_index >= len(iterable) and not barrier.outstanding_frame_ids:
return _finish_concurrent_foreach(
workflow=workflow,
run=run,
frame=frame,
step=step,
index=index,
barrier=barrier,
reducers=reducers,
)
barrier.save_to_frame(frame, step.id)
block_frame_on_children(run, frame.id, barrier.outstanding_frame_ids)
run.sync_from_current_frame()
return run
def _resolve_foreach_iterable(
run: RunState,
frame: ExecutionFrame,
step: ForeachNode,
) -> list[object]:
iterable = safe_resolve_path(
str(step.over),
state=run.state,
workflow_input=run.workflow_input,
context=frame_context_values(frame),
)
if not isinstance(iterable, list):
raise WorkflowExecutionError(
f"foreach source {str(step.over)!r} must resolve to a list"
)
return iterable
def _validate_single_node_loop_body(index: WorkflowIndex, step: ForeachNode) -> None:
"""Reject multi-step concurrent item bodies until item overlays are real.
The current slice has a no-op item-state overlay seam. Without a real overlay,
multi-node item bodies would read stale parent state after earlier item-local
writes, so V1 only allows loop -> one node -> END.
"""
loop_start = index.next_node_id(step.id, "loop")
loop_step = index.nodes_by_id.get(loop_start)
if not isinstance(loop_step, NodeUse):
raise WorkflowExecutionError(
"concurrent foreach v1 only supports loop bodies with one node"
)
node_def = index.node_defs[loop_step.node]
for outcome in node_def.outcomes:
if index.next_node_id(loop_step.id, outcome) != END:
raise WorkflowExecutionError(
"concurrent foreach v1 only supports loop bodies with one node"
)
def _finish_completed_children(run: RunState, barrier: ForeachBarrierState) -> None:
for child_id in tuple(barrier.outstanding_frame_ids):
child = run.frames[child_id]
if child.status == FrameStatus.COMPLETED:
barrier.finish_child(child_id)
elif child.status == FrameStatus.FAILED:
message = child.metadata.get("error", "unknown item failure")
raise WorkflowExecutionError(
f"concurrent foreach item frame {child_id!r} failed: {message}"
)
def _admit_concurrent_children(
*,
run: RunState,
frame: ExecutionFrame,
step: ForeachNode,
index: WorkflowIndex,
barrier: ForeachBarrierState,
iterable: list[object],
) -> int:
if step.concurrent is None:
raise WorkflowExecutionError("concurrent foreach requires concurrent policy")
admitted = 0
loop_start = index.next_node_id(frame.node_id, "loop")
while (
barrier.next_index < len(iterable)
and len(barrier.active_frame_ids) < step.concurrent.max_active
and len(barrier.outstanding_frame_ids) < step.concurrent.max_outstanding
):
loop_index = barrier.next_index
item = iterable[loop_index]
child_id = f"{frame.id}:{step.id}:{loop_index}"
active_count = len(barrier.active_frame_ids)
barrier.next_index = loop_index + 1
barrier.start_child(child_id)
add_frame(
run,
ExecutionFrame(
id=child_id,
kind="foreach_iteration",
node_id=loop_start,
status=FrameStatus.PENDING,
parent_frame_id=frame.id,
metadata=ForeachIterationMetadata(
foreach_node_id=step.id,
loop_index=loop_index,
loop_item=item,
loop_alias=step.as_,
).to_metadata(),
),
ready=True,
)
append_step_result_trace(
run,
frame_id=frame.id,
node_id=frame.node_id,
step_type=step.type,
next_node_id=loop_start,
result=StepExecutionResult(
outcome="loop",
resolved_input={
"item": item,
"index": loop_index,
"active_count": active_count,
},
output={},
state_changes={},
),
)
admitted += 1
return admitted
def _finish_concurrent_foreach(
*,
workflow: Workflow,
run: RunState,
frame: ExecutionFrame,
step: ForeachNode,
index: WorkflowIndex,
barrier: ForeachBarrierState,
reducers: Mapping[str, ReducerDefinition] | None = None,
) -> RunState:
next_node_id = index.next_node_id(frame.node_id, "done")
combined = build_barrier_patch(
workflow,
[
barrier.pending_results[item_index].patch
for item_index in sorted(barrier.pending_results)
],
run.state,
reducers=reducers,
)
state_changes = commit_state_patch(run.state, combined)
append_step_result_trace(
run,
frame_id=frame.id,
node_id=frame.node_id,
step_type=step.type,
next_node_id=next_node_id,
result=StepExecutionResult(
outcome="done",
resolved_input={
"count": barrier.next_index,
"index": barrier.next_index,
"committed_items": len(barrier.pending_results),
},
output={},
state_changes=state_changes,
),
)
advance_frame(run, frame, outcome="done", next_node_id=next_node_id)
return run
+23 -3
View File
@@ -11,10 +11,12 @@ from wf_core.models.schemas import NodeDef
from wf_core.models.steps import InputPathBinding, InputValueBinding, NodeUse from wf_core.models.steps import InputPathBinding, InputValueBinding, NodeUse
from wf_core.models.workflow import Workflow from wf_core.models.workflow import Workflow
from wf_core.run_state import RunState, RuntimeContext, StepExecutionResult from wf_core.run_state import RunState, RuntimeContext, StepExecutionResult
from wf_core.runtime.foreach_state import ForeachBarrierState, item_frame_owner
from wf_core.runtime.ops.frames import frame_context_values from wf_core.runtime.ops.frames import frame_context_values
from wf_core.runtime.ops.merges import ReducerDefinition from wf_core.runtime.ops.merges import ReducerDefinition
from wf_core.runtime.ops.overlays import state_view_for_frame
from wf_core.runtime.ops.schemas import validate_payload_against_schema from wf_core.runtime.ops.schemas import validate_payload_against_schema
from wf_core.runtime.ops.state import apply_output_bindings from wf_core.runtime.ops.state import build_output_patch, commit_state_patch
NodeHandler = Callable[[dict[str, Any], RuntimeContext], NodeResult | dict[str, Any]] NodeHandler = Callable[[dict[str, Any], RuntimeContext], NodeResult | dict[str, Any]]
AsyncNodeHandler = Callable[ AsyncNodeHandler = Callable[
@@ -32,6 +34,7 @@ def _resolve_node_execution(
) -> tuple[dict[str, Any], RuntimeContext]: ) -> tuple[dict[str, Any], RuntimeContext]:
frame = run.current_frame() frame = run.current_frame()
context_values = frame_context_values(frame) context_values = frame_context_values(frame)
state_view = state_view_for_frame(run, frame)
resolved_input: dict[str, Any] = {} resolved_input: dict[str, Any] = {}
for binding in node.input: for binding in node.input:
if isinstance(binding, InputValueBinding): if isinstance(binding, InputValueBinding):
@@ -39,7 +42,7 @@ def _resolve_node_execution(
elif isinstance(binding, InputPathBinding): elif isinstance(binding, InputPathBinding):
value = safe_resolve_path( value = safe_resolve_path(
str(binding.path), str(binding.path),
state=run.state, state=state_view,
workflow_input=run.workflow_input, workflow_input=run.workflow_input,
context=context_values, context=context_values,
) )
@@ -85,13 +88,30 @@ def _finalize_node_execution(
validate_payload_against_schema( validate_payload_against_schema(
node_def.output_schema, result.output, f"node output for {node.id}" node_def.output_schema, result.output, f"node output for {node.id}"
) )
state_changes = apply_output_bindings( patch = build_output_patch(
workflow, workflow,
node.output, node.output,
result.output, result.output,
run.state, run.state,
reducers=reducers, reducers=reducers,
) )
owner = item_frame_owner(run.current_frame())
if owner is None:
state_changes = commit_state_patch(run.state, patch)
else:
parent_frame_id, foreach_node_id, item_index = owner
parent_frame = run.frames[parent_frame_id]
barrier = ForeachBarrierState.from_frame(parent_frame, foreach_node_id)
if barrier is not None and barrier.mode == "concurrent":
barrier.add_success_patch(
index=item_index,
frame_id=run.current_frame().id,
patch=patch,
)
barrier.save_to_frame(parent_frame, foreach_node_id)
state_changes = {}
else:
state_changes = commit_state_patch(run.state, patch)
return StepExecutionResult( return StepExecutionResult(
outcome=result.outcome, outcome=result.outcome,
resolved_input=resolved_input, resolved_input=resolved_input,
+16
View File
@@ -0,0 +1,16 @@
from __future__ import annotations
from typing import Any
from wf_core.run_state import ExecutionFrame, RunState
def state_view_for_frame(run: RunState, frame: ExecutionFrame) -> dict[str, Any]:
"""Return the state view visible to one execution frame.
This is intentionally a no-op seam for concurrent foreach V1. The first
sync-concurrent slice only supports single-node item bodies, so item frames
do not need to read their own prior buffered writes yet. The overlay slice
should replace this with parent-state plus item-local staged writes.
"""
return run.state
+40
View File
@@ -145,6 +145,46 @@ def commit_state_patch(state: dict[str, Any], patch: StatePatch) -> dict[str, An
return dict(patch.changes) return dict(patch.changes)
def build_barrier_patch(
workflow: Workflow,
item_patches: Sequence[StatePatch],
state: dict[str, Any],
*,
reducers: Mapping[str, ReducerDefinition] | None = None,
) -> StatePatch:
"""Build one committed barrier patch by replaying item writes in order.
Item patches are built against the parent-visible state. Their prepared
writes cannot be blindly merged because reducers must see the value produced
by earlier item patches. The barrier therefore replays trace-facing incoming
changes against a single staged state in deterministic item order.
"""
state_fields = workflow.state_schema.field_index()
staged_state = deepcopy(state)
prepared_patch: dict[StatePath, tuple[list[str], Any]] = {}
committed_changes: dict[str, Any] = {}
for item_patch in item_patches:
for destination, incoming_value in item_patch.changes.items():
destination_path = StatePath.parse(destination)
key_path, merged_value = prepare_state_value(
workflow,
staged_state,
destination_path,
incoming_value,
reducers=reducers,
state_fields=state_fields,
)
safe_set_nested_value(staged_state, key_path, merged_value)
prepared_patch[destination_path] = (key_path, merged_value)
committed_changes[destination] = merged_value
validate_staged_state_patch(staged_state, prepared_patch, state_fields)
return StatePatch(
changes=committed_changes,
_prepared_writes=prepared_patch,
_staged_state=staged_state,
)
def apply_mapped_state( def apply_mapped_state(
workflow: Workflow, workflow: Workflow,
source_data: dict[str, Any], source_data: dict[str, Any],
+15
View File
@@ -177,6 +177,21 @@ def wake_parent_if_children_complete(run: RunState, child_frame_id: str) -> None
wake_frame(run, parent_id) wake_frame(run, parent_id)
def wake_parent_for_child_progress(run: RunState, child_frame_id: str) -> None:
"""Wake a blocked parent after one child finishes so it can refill slots."""
child = _frame(run, child_frame_id)
parent_id = child.parent_frame_id
if parent_id is None:
return
parent = _frame(run, parent_id)
if parent.status != FrameStatus.BLOCKED:
return
block = BlockedOnChildren.from_frame(parent)
if block is None or child_frame_id not in block.child_frame_ids:
return
wake_frame(run, parent_id)
def resolve_no_ready_frames(run: RunState) -> RunStatus: def resolve_no_ready_frames(run: RunState) -> RunStatus:
"""Classify an empty ready queue into terminal, paused, or deadlocked state.""" """Classify an empty ready queue into terminal, paused, or deadlocked state."""
if run.status == RunStatus.INTERRUPTED: if run.status == RunStatus.INTERRUPTED:
+2 -2
View File
@@ -99,7 +99,7 @@ def step_workflow(
elif isinstance(step, InterruptNode): elif isinstance(step, InterruptNode):
return handle_interrupt_step(run, step) return handle_interrupt_step(run, step)
elif isinstance(step, ForeachNode): elif isinstance(step, ForeachNode):
return step_foreach(workflow, run, step, index) return step_foreach(workflow, run, step, index, reducers=reducers)
else: else:
raise WorkflowExecutionError( raise WorkflowExecutionError(
f"unsupported step type {getattr(step, 'type', type(step).__name__)!r}" f"unsupported step type {getattr(step, 'type', type(step).__name__)!r}"
@@ -152,7 +152,7 @@ async def step_workflow_async(
elif isinstance(step, InterruptNode): elif isinstance(step, InterruptNode):
return handle_interrupt_step(run, step) return handle_interrupt_step(run, step)
elif isinstance(step, ForeachNode): elif isinstance(step, ForeachNode):
return step_foreach(workflow, run, step, index) return step_foreach(workflow, run, step, index, reducers=reducers)
else: else:
raise WorkflowExecutionError( raise WorkflowExecutionError(
f"unsupported step type {getattr(step, 'type', type(step).__name__)!r}" f"unsupported step type {getattr(step, 'type', type(step).__name__)!r}"
+279
View File
@@ -0,0 +1,279 @@
from __future__ import annotations
from typing import Any
import pytest
from wf_core import (
END,
Edge,
ForeachNode,
NodeDef,
NodeUse,
ReducerRef,
SchemaRef,
StateField,
StateSchema,
Workflow,
WorkflowExecutionError,
execute_workflow,
)
def test_sync_concurrent_foreach_interleaves_items_and_commits_at_barrier() -> None:
workflow = _workflow(
state_schema=StateSchema.from_field_map(
{
"items": StateField(type="array"),
"seen": StateField(
type="array",
reducer=ReducerRef(name="wf.std.append"),
),
}
),
foreach=ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": "state.items",
"as": "item",
"mode": "concurrent",
"concurrent": {"max_active": 2, "max_outstanding": 2},
}
),
)
run = execute_workflow(
workflow,
{"items": ["a", "b", "c"]},
{"record": lambda payload, _ctx: {"outcome": "ok", "output": payload}},
)
assert run.output["seen"] == ["a", "b", "c"]
assert run.state["seen"] == ["a", "b", "c"]
foreach_entries = [entry for entry in run.trace if entry.step_type == "foreach"]
assert foreach_entries[-1].outcome == "done"
assert foreach_entries[-1].state_changes["state.seen"] == ["a", "b", "c"]
def test_sync_concurrent_foreach_respects_max_active_by_refill_trace() -> None:
workflow = _workflow(
state_schema=StateSchema.from_field_map(
{
"items": StateField(type="array"),
"seen": StateField(
type="array",
reducer=ReducerRef(name="wf.std.append"),
),
}
),
foreach=ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": "state.items",
"as": "item",
"mode": "concurrent",
"concurrent": {"max_active": 2, "max_outstanding": 2},
}
),
)
run = execute_workflow(
workflow,
{"items": ["a", "b", "c", "d"]},
{"record": lambda payload, _ctx: {"outcome": "ok", "output": payload}},
)
loop_entries = [
entry
for entry in run.trace
if entry.step_type == "foreach" and entry.outcome == "loop"
]
assert loop_entries[0].resolved_input["active_count"] == 0
assert loop_entries[1].resolved_input["active_count"] == 1
assert all(entry.resolved_input["active_count"] < 2 for entry in loop_entries)
def test_sync_concurrent_foreach_rejects_non_fail_item_policy_for_now() -> None:
workflow = _workflow(
state_schema=StateSchema.from_field_map(
{
"items": StateField(type="array"),
"seen": StateField(
type="array",
reducer=ReducerRef(name="wf.std.append"),
),
"errors": StateField(type="array"),
}
),
foreach=ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": "state.items",
"as": "item",
"mode": "concurrent",
"concurrent": {"max_active": 2, "max_outstanding": 2},
"item_error": {"action": "collect", "collect_to": "state.errors"},
}
),
include_completed_with_errors=True,
)
with pytest.raises(
WorkflowExecutionError,
match="only supports item_error.action='fail'",
):
execute_workflow(
workflow,
{"items": ["a"]},
{"record": lambda payload, _ctx: {"outcome": "ok", "output": payload}},
)
def test_sync_concurrent_foreach_fails_run_on_item_runtime_error() -> None:
workflow = _workflow(
state_schema=StateSchema.from_field_map(
{
"items": StateField(type="array"),
"seen": StateField(
type="array",
reducer=ReducerRef(name="wf.std.append"),
),
}
),
foreach=ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": "state.items",
"as": "item",
"mode": "concurrent",
"concurrent": {"max_active": 2, "max_outstanding": 2},
}
),
)
def fail_on_b(payload: dict[str, Any], _ctx: object) -> dict[str, Any]:
if payload["value"] == "b":
raise ValueError("bad item")
return {"outcome": "ok", "output": payload}
with pytest.raises(ValueError, match="bad item"):
execute_workflow(workflow, {"items": ["a", "b", "c"]}, {"record": fail_on_b})
def test_sync_concurrent_foreach_rejects_multi_step_item_body_for_now() -> None:
workflow = _workflow(
state_schema=StateSchema.from_field_map(
{
"items": StateField(type="array"),
"seen": StateField(
type="array",
reducer=ReducerRef(name="wf.std.append"),
),
}
),
foreach=ForeachNode.model_validate(
{
"id": "each",
"type": "foreach",
"over": "state.items",
"as": "item",
"mode": "concurrent",
"concurrent": {"max_active": 2, "max_outstanding": 2},
}
),
)
workflow.nodes.append(
NodeUse.model_validate(
{
"id": "after_record",
"type": "node",
"node": "record",
"input": [{"target": "value", "path": "state.seen"}],
"output": [{"source": "seen", "target": "state.seen"}],
}
)
)
workflow.edges = [
Edge.model_validate({"from": "each", "outcome": "loop", "to": "record"}),
Edge.model_validate({"from": "record", "outcome": "ok", "to": "after_record"}),
Edge.model_validate({"from": "after_record", "outcome": "ok", "to": END}),
Edge.model_validate({"from": "each", "outcome": "done", "to": END}),
]
with pytest.raises(
WorkflowExecutionError,
match="only supports loop bodies with one node",
):
execute_workflow(
workflow,
{"items": ["a"]},
{"record": lambda payload, _ctx: {"outcome": "ok", "output": payload}},
)
def _workflow(
*,
state_schema: StateSchema,
foreach: ForeachNode,
include_completed_with_errors: bool = False,
) -> Workflow:
edges = [
Edge.model_validate({"from": "each", "outcome": "loop", "to": "record"}),
Edge.model_validate({"from": "record", "outcome": "ok", "to": END}),
Edge.model_validate({"from": "each", "outcome": "done", "to": END}),
]
if include_completed_with_errors:
edges.append(
Edge.model_validate(
{"from": "each", "outcome": "completed_with_errors", "to": END}
)
)
return Workflow(
name="concurrent_foreach_v1",
input_schema=SchemaRef(
type="object",
properties={"items": {"type": "array"}},
),
state_schema=state_schema,
output_schema=SchemaRef(
type="object",
properties={"seen": {"type": "array"}},
),
node_defs=[
NodeDef(
name="record",
input_schema=SchemaRef(
type="object",
properties={"value": {}, "seen": {}},
required=["value", "seen"],
),
output_schema=SchemaRef(
type="object",
properties={"value": {}, "seen": {}},
required=["seen"],
),
outcomes=["ok"],
)
],
start="each",
nodes=[
foreach,
NodeUse.model_validate(
{
"id": "record",
"type": "node",
"node": "record",
"input": [
{"target": "value", "path": "context.item"},
{"target": "seen", "path": "context.item"},
],
"output": [{"source": "seen", "target": "state.seen"}],
}
),
],
edges=edges,
)
+19
View File
@@ -66,6 +66,25 @@ def test_foreach_barrier_state_rejects_malformed_metadata() -> None:
ForeachBarrierState.from_frame(frame, "each") ForeachBarrierState.from_frame(frame, "each")
def test_foreach_barrier_tracks_active_and_outstanding_children() -> None:
barrier = ForeachBarrierState()
barrier.start_child("child-0")
barrier.start_child("child-1")
barrier.finish_child("child-0")
assert barrier.active_frame_ids == ("child-1",)
assert barrier.outstanding_frame_ids == ("child-1",)
def test_foreach_barrier_rejects_duplicate_child_start() -> None:
barrier = ForeachBarrierState()
barrier.start_child("child-0")
with pytest.raises(WorkflowExecutionError, match="already active"):
barrier.start_child("child-0")
def test_item_error_record_rejects_negative_index() -> None: def test_item_error_record_rejects_negative_index() -> None:
with pytest.raises(WorkflowExecutionError, match="index"): with pytest.raises(WorkflowExecutionError, match="index"):
ItemErrorRecord.from_metadata( ItemErrorRecord.from_metadata(
+22
View File
@@ -15,6 +15,7 @@ from wf_core.runtime.scheduler import (
select_next_frame, select_next_frame,
resolve_no_ready_frames, resolve_no_ready_frames,
wake_frame, wake_frame,
wake_parent_for_child_progress,
wake_parent_if_children_complete, wake_parent_if_children_complete,
) )
@@ -160,6 +161,27 @@ def test_child_completion_wakes_blocked_parent() -> None:
assert run.ready_frame_ids == ["parent"] assert run.ready_frame_ids == ["parent"]
def test_wake_parent_when_child_finishes_for_refill() -> None:
run = _run()
add_frame(run, ExecutionFrame(id="parent", kind="root", node_id="foreach"))
add_frame(
run,
ExecutionFrame(
id="child",
kind="foreach_iteration",
node_id="__end__",
parent_frame_id="parent",
),
)
block_frame_on_children(run, "parent", ("child", "other"))
run.frames["child"].status = FrameStatus.COMPLETED
wake_parent_for_child_progress(run, "child")
assert run.frames["parent"].status == FrameStatus.PENDING
assert run.ready_frame_ids == ["parent"]
def test_resume_wakes_interrupted_frame_at_front() -> None: def test_resume_wakes_interrupted_frame_at_front() -> None:
run = _run() run = _run()
add_frame(run, ExecutionFrame(id="waiting", kind="root", node_id="ask")) add_frame(run, ExecutionFrame(id="waiting", kind="root", node_id="ask"))