docs: specify foreach back edge semantics

This commit is contained in:
lda
2026-09-04 01:44:30 +07:00 Verified
parent ca6425af32
commit 9aaf1b4d78
6 changed files with 240 additions and 11 deletions
+8
View File
@@ -281,6 +281,12 @@ The end of one execution frame. Its owner determines whether this completes a
workflow invocation, returns from a subgraph, or completes one foreach item.
_Avoid_: Always completing the run, domain outcome, implicit break
**Foreach Return**:
Completion of one foreach item through a graph back-edge targeting the item's
owning foreach. The owner resumes its serial controller or concurrent barrier;
the item does not execute the foreach node again.
_Avoid_: Workflow end, generic graph cycle, implicit break
**Fork**:
An explicit workflow control step that creates several concurrent branch
activations. A fork is distinct from an ordinary outcome, which selects exactly
@@ -567,6 +573,8 @@ _Avoid_: Job, invocation
its runtime scope through that lineage.
- Frame ancestry expresses scheduling ownership; lineage ancestry expresses
state visibility.
- A **Foreach Return** completes an item frame and returns control to its owning
foreach without ending the workflow scope.
- A **Frame Set** is the source of truth for runtime cursors.
- The **Scheduler Foundation** selects one **Runnable Frame** at a time in the
first pass.
+2
View File
@@ -114,6 +114,8 @@ docs as the active references:
concurrent foreach policy and barrier commit semantics.
- [`adr/0006-explicit-fork-and-topology-driven-gather.md`](adr/0006-explicit-fork-and-topology-driven-gather.md):
proposed explicit fork, gather-slot, and activation-token semantics.
- [`superpowers/specs/2026-09-04-foreach-back-edge-design.md`](superpowers/specs/2026-09-04-foreach-back-edge-design.md):
approved canonical foreach body-return and validation semantics.
- [`superpowers/specs/2026-05-24-native-subgraphs-design.md`](superpowers/specs/2026-05-24-native-subgraphs-design.md):
native subgraph design.
- [`superpowers/specs/2026-05-26-durable-workflow-runs-and-resume-design.md`](superpowers/specs/2026-05-26-durable-workflow-runs-and-resume-design.md):
@@ -185,6 +185,29 @@ Future foreach metadata should evolve into inherited structured lineage context:
- normal node authors should receive foreach values through mapped input;
inspecting runtime context is an advanced escape hatch
## Iteration Body Return
An iteration body returns through an ordinary edge targeting its owning
`ForeachNode`. The runtime recognizes that target using the item frame's typed
owner metadata, completes the child frame, and wakes the parent barrier. It does
not execute the foreach controller inside the child frame.
`END` and explicit `EndNode` are workflow/subgraph terminals, not foreach item
returns. Validation rejects either terminal while traversing an active foreach
item context. This replaces the earlier implementation convention in which an
item reached `END` to wake its parent.
The canonical graph therefore uses:
```text
foreach.loop -> body
body -> foreach
foreach.done -> continuation
```
The exact runtime, validation, migration, and nested-context rules are specified
in the [foreach back-edge design](../superpowers/specs/2026-09-04-foreach-back-edge-design.md).
## Deferred Work
Explicit Fork/Gather is deferred. A future `GatherNode` should expose explicit
@@ -97,12 +97,12 @@ overlap in the asynchronous runtime. Both modes must produce equivalent graph
semantics. Scheduler order decides when compatible work progresses, never which
arrivals belong together.
`END` means completion of the current execution frame, not necessarily
completion of the whole run. The frame owner determines the consequence: a
root frame completes its workflow invocation, a subgraph root returns to its
parent boundary, and a foreach item reports completion to its parent barrier.
For a foreach, normal item-frame completion therefore allows the controller to
admit or resume the next item without introducing a separate continue node.
`END` and explicit `EndNode` represent workflow/subgraph termination, not a
generic way to complete any child frame. A foreach item returns through a
back-edge targeting its owning `ForeachNode`; the runtime completes that item
frame and wakes its parent barrier without executing the controller inside the
child. Future fork branches likewise converge through explicit gathers rather
than independently terminating the containing workflow scope.
General break and race behavior are not part of the first fork/gather design.
The existing foreach `fail`, `skip`, and `collect` policies describe the
@@ -141,10 +141,10 @@ those relationships are canonical on the lineage and duplicated frame fields
can disagree with them. Runtime operations should resolve and validate a
frame's lineage and scope together.
**Model normal foreach completion as a continue node.** Rejected because
completion of an item frame already returns control to the owning foreach.
Break and race semantics remain separate future policies rather than additional
meanings assigned to ordinary outcomes.
**Model normal foreach completion as a continue node.** Rejected because an
ordinary back-edge to the owning foreach already expresses item return in the
canonical graph. Break and race semantics remain separate future policies
rather than additional meanings assigned to ordinary outcomes.
**Reuse or rename `JoinNode`.** Rejected as the default because the existing
node is a pass-through marker with no barrier contract.
+4 -1
View File
@@ -795,7 +795,10 @@ stable.
- Native subgraph polish: optional per-use-site child deployment overrides and
clearer child trace inspection.
- Concurrent foreach polish: reuse barrier/lineage machinery for future
- Active concurrent foreach correction: replace item-body `END` routes with
canonical back-edges to the owning foreach. The approved semantics are in the
[`foreach back-edge design`](superpowers/specs/2026-09-04-foreach-back-edge-design.md).
- After that correction, reuse the foreach barrier/lineage machinery for
fork/gather. The proposed control semantics are recorded in
[`ADR-0006`](adr/0006-explicit-fork-and-topology-driven-gather.md).
- Protocol-native progress: investigate MCP tasks/progress or WebSocket/SSE only
@@ -0,0 +1,193 @@
# Foreach Back-Edge Design
## Status
Approved in conversation on 2026-09-04. This document specifies canonical
foreach body-return semantics. It does not include the separately planned
ergonomic Python DSL or authorize fork/gather implementation.
## Purpose
Make the persisted workflow graph tell the truth about foreach control flow.
An iteration body returns to its owning `ForeachNode` through an ordinary
back-edge. It no longer routes to the workflow terminal `END` merely to make an
internal child frame finish.
The canonical shape is:
```text
a -> foreach
foreach.loop -> body
body.ok -> foreach
foreach.done -> c
c -> END
```
This preserves a flat, inspectable control-flow graph. The runtime may continue
to implement each item with a child execution frame and a parent barrier, but
that scheduler detail no longer changes the visible meaning of `END`.
## Decisions
- `ForeachNode.loop` enters one iteration body.
- An edge from an iteration body to that iteration frame's owning
`ForeachNode` returns the item to the controller.
- Returning to the owner completes the item frame and wakes the blocked parent
foreach frame. The child does not execute the `ForeachNode` again.
- The parent foreach frame remains positioned at the controller. It admits the
next serial or concurrent item, or emits `done`/`completed_with_errors` after
its barrier finishes.
- A foreach item path may not target `END` or an explicit `EndNode`. Workflow
terminal routes are outside an iteration body.
- `END` remains the workflow/subgraph terminal shorthand for workflow outcome
`ok`; explicit `EndNode` remains the terminal for other workflow outcomes.
- Existing `fail`, `skip`, and `collect` item-error policies are unchanged.
- The core graph stays flat. Foreach does not gain a nested serialized body.
- The existing manual `WorkflowBuilder.foreach()` and `connect()` interface
remains the only authoring surface in this slice; no new authoring interface
is added.
## Runtime Semantics
The parent frame owns the foreach controller and resumable barrier. Taking its
`loop` outcome creates an item frame whose metadata names that parent foreach.
The item frame begins at the `loop` edge target and follows ordinary outcome
edges.
Before ordinary frame advancement, runtime flow checks whether an item frame's
next target is its recorded owning foreach. If so, it records a foreach return:
```python
owner = foreach_item_owner(frame)
if owner is not None and next_node_id == owner.foreach_node_id:
complete_item_frame(run, frame)
wake_parent_for_child_progress(run, frame.id)
return
```
This is an ownership check, not generic cycle detection. A root or unrelated
frame targeting the same foreach node enters it normally. An item frame
targeting a different foreach node also enters that node normally. Only a
return to the frame's recorded owner completes the current item.
The completed child records the owning foreach as its terminal graph location
for trace and checkpoint inspection. Existing barrier code remains responsible
for consuming its lineage-local writes, applying item-error policy, and waking
or completing the parent.
Serial and concurrent modes share the same return meaning:
- serial mode has at most one live item frame and can admit the next item after
its return;
- concurrent mode may receive several independently identified item returns,
buffers them by item identity, and completes only after its existing barrier
policy is satisfied.
## Nested Foreach
Nested foreach remains structured by frame ownership:
```text
outer.loop -> inner
inner.loop -> work
work -> inner
inner.done -> after_inner
after_inner -> outer
outer.done -> after_outer
```
The `work` frame returns only to `inner`, its immediate owner. Once the inner
controller emits `done`, execution resumes in the outer item frame. Returning
`after_inner -> outer` then completes the outer item.
Runtime metadata already carries the immediate foreach owner. Static context
analysis must evolve from a single active foreach identifier to an ownership
stack so inner completion restores the outer item context.
## Validation and Analysis
Validation performs structured abstract traversal over `(node_id,
foreach_owner_stack)` rather than merely looking for graph cycles:
- a `loop` edge from a foreach pushes that foreach onto the stack;
- an edge targeting the top owner is an item return and ends that child
traversal;
- a `done` or `completed_with_errors` edge stays in the outer context;
- `END` or `EndNode` reached with a non-empty foreach stack is invalid;
- nested returns must target the immediate owner before an outer owner;
- traversal memoizes node plus owner stack so valid cycles terminate analysis.
A node may remain reachable in more than one execution context. Existing
context-contract analysis can report fields as conditional in that case. The
runtime must never infer edge meaning from graph history alone: frame ownership
always decides whether targeting a foreach is entry or return.
General unreachable-node detection is useful but is not part of this change.
It cannot establish foreach ownership because any node reached from a body is
connected by definition. General termination is also out of scope; finite
structural checks cannot prove that data-dependent cycles eventually exit.
## State and Failure Behavior
Back-edge return changes control representation, not state semantics.
Iteration writes remain buffered in the item lineage. Serial behavior and the
concurrent barrier continue to commit or merge those writes according to the
accepted concurrent-foreach ADR and declared reducers.
An ordinary node outcome named `error` remains domain control. An exception
remains a runtime item failure handled by `fail`, `skip`, or `collect`. Neither
kind of failure is encoded by the foreach back-edge itself.
## Migration
This is a clean canonical migration. Repository tests, examples, user-facing
docs, and generated fixtures change from:
```text
foreach.loop -> body
body -> END
```
to:
```text
foreach.loop -> body
body -> foreach
```
Configured local artifact stores contain no persisted foreach workflows at the
time of design. There is therefore no demonstrated persisted-data requirement
for keeping `body -> END` as a compatibility behavior. Validation rejects that
old shape rather than silently preserving two canonical return forms. If real
external persisted data is identified before implementation lands, it requires
an explicit migration decision rather than a permanent implicit shim.
Checkpoints from an in-progress old foreach execution are likewise not given a
speculative compatibility path without real data. The implementation must not
add parse-old behavior solely because repository fixtures previously used the
old topology.
## Testing
Focused tests must prove:
- serial item return wakes the parent and admits the next item;
- concurrent item returns preserve item identity and deterministic barrier
commits;
- a child targeting its owner returns instead of executing the foreach node;
- a root frame targeting the same foreach enters it normally;
- an item frame can enter a different nested foreach;
- nested foreach restores the outer item context after inner completion;
- item paths to `END` and explicit `EndNode` fail validation;
- traces and serialized checkpoints retain an inspectable return location;
- existing `fail`, `skip`, `collect`, interrupt, reducer-conflict, sync, and
async behavior remains intact after fixture migration.
## Deferred Work
- Ergonomic/context-manager Python authoring syntax.
- `break`, `continue` nodes, target ports, or iteration return dispositions.
- First-completed, first-error, first-success, cancellation, or race policies.
- General unreachable-node and non-terminating-cycle diagnostics.
- Fork/gather nodes and activation-token persistence.
- Nested serialized workflow blocks.