docs: specify foreach back edge semantics
This commit is contained in:
@@ -281,6 +281,12 @@ The end of one execution frame. Its owner determines whether this completes a
|
||||
workflow invocation, returns from a subgraph, or completes one foreach item.
|
||||
_Avoid_: Always completing the run, domain outcome, implicit break
|
||||
|
||||
**Foreach Return**:
|
||||
Completion of one foreach item through a graph back-edge targeting the item's
|
||||
owning foreach. The owner resumes its serial controller or concurrent barrier;
|
||||
the item does not execute the foreach node again.
|
||||
_Avoid_: Workflow end, generic graph cycle, implicit break
|
||||
|
||||
**Fork**:
|
||||
An explicit workflow control step that creates several concurrent branch
|
||||
activations. A fork is distinct from an ordinary outcome, which selects exactly
|
||||
@@ -567,6 +573,8 @@ _Avoid_: Job, invocation
|
||||
its runtime scope through that lineage.
|
||||
- Frame ancestry expresses scheduling ownership; lineage ancestry expresses
|
||||
state visibility.
|
||||
- A **Foreach Return** completes an item frame and returns control to its owning
|
||||
foreach without ending the workflow scope.
|
||||
- A **Frame Set** is the source of truth for runtime cursors.
|
||||
- The **Scheduler Foundation** selects one **Runnable Frame** at a time in the
|
||||
first pass.
|
||||
|
||||
@@ -114,6 +114,8 @@ docs as the active references:
|
||||
concurrent foreach policy and barrier commit semantics.
|
||||
- [`adr/0006-explicit-fork-and-topology-driven-gather.md`](adr/0006-explicit-fork-and-topology-driven-gather.md):
|
||||
proposed explicit fork, gather-slot, and activation-token semantics.
|
||||
- [`superpowers/specs/2026-09-04-foreach-back-edge-design.md`](superpowers/specs/2026-09-04-foreach-back-edge-design.md):
|
||||
approved canonical foreach body-return and validation semantics.
|
||||
- [`superpowers/specs/2026-05-24-native-subgraphs-design.md`](superpowers/specs/2026-05-24-native-subgraphs-design.md):
|
||||
native subgraph design.
|
||||
- [`superpowers/specs/2026-05-26-durable-workflow-runs-and-resume-design.md`](superpowers/specs/2026-05-26-durable-workflow-runs-and-resume-design.md):
|
||||
|
||||
@@ -185,6 +185,29 @@ Future foreach metadata should evolve into inherited structured lineage context:
|
||||
- normal node authors should receive foreach values through mapped input;
|
||||
inspecting runtime context is an advanced escape hatch
|
||||
|
||||
## Iteration Body Return
|
||||
|
||||
An iteration body returns through an ordinary edge targeting its owning
|
||||
`ForeachNode`. The runtime recognizes that target using the item frame's typed
|
||||
owner metadata, completes the child frame, and wakes the parent barrier. It does
|
||||
not execute the foreach controller inside the child frame.
|
||||
|
||||
`END` and explicit `EndNode` are workflow/subgraph terminals, not foreach item
|
||||
returns. Validation rejects either terminal while traversing an active foreach
|
||||
item context. This replaces the earlier implementation convention in which an
|
||||
item reached `END` to wake its parent.
|
||||
|
||||
The canonical graph therefore uses:
|
||||
|
||||
```text
|
||||
foreach.loop -> body
|
||||
body -> foreach
|
||||
foreach.done -> continuation
|
||||
```
|
||||
|
||||
The exact runtime, validation, migration, and nested-context rules are specified
|
||||
in the [foreach back-edge design](../superpowers/specs/2026-09-04-foreach-back-edge-design.md).
|
||||
|
||||
## Deferred Work
|
||||
|
||||
Explicit Fork/Gather is deferred. A future `GatherNode` should expose explicit
|
||||
|
||||
@@ -97,12 +97,12 @@ overlap in the asynchronous runtime. Both modes must produce equivalent graph
|
||||
semantics. Scheduler order decides when compatible work progresses, never which
|
||||
arrivals belong together.
|
||||
|
||||
`END` means completion of the current execution frame, not necessarily
|
||||
completion of the whole run. The frame owner determines the consequence: a
|
||||
root frame completes its workflow invocation, a subgraph root returns to its
|
||||
parent boundary, and a foreach item reports completion to its parent barrier.
|
||||
For a foreach, normal item-frame completion therefore allows the controller to
|
||||
admit or resume the next item without introducing a separate continue node.
|
||||
`END` and explicit `EndNode` represent workflow/subgraph termination, not a
|
||||
generic way to complete any child frame. A foreach item returns through a
|
||||
back-edge targeting its owning `ForeachNode`; the runtime completes that item
|
||||
frame and wakes its parent barrier without executing the controller inside the
|
||||
child. Future fork branches likewise converge through explicit gathers rather
|
||||
than independently terminating the containing workflow scope.
|
||||
|
||||
General break and race behavior are not part of the first fork/gather design.
|
||||
The existing foreach `fail`, `skip`, and `collect` policies describe the
|
||||
@@ -141,10 +141,10 @@ those relationships are canonical on the lineage and duplicated frame fields
|
||||
can disagree with them. Runtime operations should resolve and validate a
|
||||
frame's lineage and scope together.
|
||||
|
||||
**Model normal foreach completion as a continue node.** Rejected because
|
||||
completion of an item frame already returns control to the owning foreach.
|
||||
Break and race semantics remain separate future policies rather than additional
|
||||
meanings assigned to ordinary outcomes.
|
||||
**Model normal foreach completion as a continue node.** Rejected because an
|
||||
ordinary back-edge to the owning foreach already expresses item return in the
|
||||
canonical graph. Break and race semantics remain separate future policies
|
||||
rather than additional meanings assigned to ordinary outcomes.
|
||||
|
||||
**Reuse or rename `JoinNode`.** Rejected as the default because the existing
|
||||
node is a pass-through marker with no barrier contract.
|
||||
|
||||
@@ -795,7 +795,10 @@ stable.
|
||||
|
||||
- Native subgraph polish: optional per-use-site child deployment overrides and
|
||||
clearer child trace inspection.
|
||||
- Concurrent foreach polish: reuse barrier/lineage machinery for future
|
||||
- Active concurrent foreach correction: replace item-body `END` routes with
|
||||
canonical back-edges to the owning foreach. The approved semantics are in the
|
||||
[`foreach back-edge design`](superpowers/specs/2026-09-04-foreach-back-edge-design.md).
|
||||
- After that correction, reuse the foreach barrier/lineage machinery for
|
||||
fork/gather. The proposed control semantics are recorded in
|
||||
[`ADR-0006`](adr/0006-explicit-fork-and-topology-driven-gather.md).
|
||||
- Protocol-native progress: investigate MCP tasks/progress or WebSocket/SSE only
|
||||
|
||||
@@ -0,0 +1,193 @@
|
||||
# Foreach Back-Edge Design
|
||||
|
||||
## Status
|
||||
|
||||
Approved in conversation on 2026-09-04. This document specifies canonical
|
||||
foreach body-return semantics. It does not include the separately planned
|
||||
ergonomic Python DSL or authorize fork/gather implementation.
|
||||
|
||||
## Purpose
|
||||
|
||||
Make the persisted workflow graph tell the truth about foreach control flow.
|
||||
An iteration body returns to its owning `ForeachNode` through an ordinary
|
||||
back-edge. It no longer routes to the workflow terminal `END` merely to make an
|
||||
internal child frame finish.
|
||||
|
||||
The canonical shape is:
|
||||
|
||||
```text
|
||||
a -> foreach
|
||||
foreach.loop -> body
|
||||
body.ok -> foreach
|
||||
foreach.done -> c
|
||||
c -> END
|
||||
```
|
||||
|
||||
This preserves a flat, inspectable control-flow graph. The runtime may continue
|
||||
to implement each item with a child execution frame and a parent barrier, but
|
||||
that scheduler detail no longer changes the visible meaning of `END`.
|
||||
|
||||
## Decisions
|
||||
|
||||
- `ForeachNode.loop` enters one iteration body.
|
||||
- An edge from an iteration body to that iteration frame's owning
|
||||
`ForeachNode` returns the item to the controller.
|
||||
- Returning to the owner completes the item frame and wakes the blocked parent
|
||||
foreach frame. The child does not execute the `ForeachNode` again.
|
||||
- The parent foreach frame remains positioned at the controller. It admits the
|
||||
next serial or concurrent item, or emits `done`/`completed_with_errors` after
|
||||
its barrier finishes.
|
||||
- A foreach item path may not target `END` or an explicit `EndNode`. Workflow
|
||||
terminal routes are outside an iteration body.
|
||||
- `END` remains the workflow/subgraph terminal shorthand for workflow outcome
|
||||
`ok`; explicit `EndNode` remains the terminal for other workflow outcomes.
|
||||
- Existing `fail`, `skip`, and `collect` item-error policies are unchanged.
|
||||
- The core graph stays flat. Foreach does not gain a nested serialized body.
|
||||
- The existing manual `WorkflowBuilder.foreach()` and `connect()` interface
|
||||
remains the only authoring surface in this slice; no new authoring interface
|
||||
is added.
|
||||
|
||||
## Runtime Semantics
|
||||
|
||||
The parent frame owns the foreach controller and resumable barrier. Taking its
|
||||
`loop` outcome creates an item frame whose metadata names that parent foreach.
|
||||
The item frame begins at the `loop` edge target and follows ordinary outcome
|
||||
edges.
|
||||
|
||||
Before ordinary frame advancement, runtime flow checks whether an item frame's
|
||||
next target is its recorded owning foreach. If so, it records a foreach return:
|
||||
|
||||
```python
|
||||
owner = foreach_item_owner(frame)
|
||||
if owner is not None and next_node_id == owner.foreach_node_id:
|
||||
complete_item_frame(run, frame)
|
||||
wake_parent_for_child_progress(run, frame.id)
|
||||
return
|
||||
```
|
||||
|
||||
This is an ownership check, not generic cycle detection. A root or unrelated
|
||||
frame targeting the same foreach node enters it normally. An item frame
|
||||
targeting a different foreach node also enters that node normally. Only a
|
||||
return to the frame's recorded owner completes the current item.
|
||||
|
||||
The completed child records the owning foreach as its terminal graph location
|
||||
for trace and checkpoint inspection. Existing barrier code remains responsible
|
||||
for consuming its lineage-local writes, applying item-error policy, and waking
|
||||
or completing the parent.
|
||||
|
||||
Serial and concurrent modes share the same return meaning:
|
||||
|
||||
- serial mode has at most one live item frame and can admit the next item after
|
||||
its return;
|
||||
- concurrent mode may receive several independently identified item returns,
|
||||
buffers them by item identity, and completes only after its existing barrier
|
||||
policy is satisfied.
|
||||
|
||||
## Nested Foreach
|
||||
|
||||
Nested foreach remains structured by frame ownership:
|
||||
|
||||
```text
|
||||
outer.loop -> inner
|
||||
inner.loop -> work
|
||||
work -> inner
|
||||
inner.done -> after_inner
|
||||
after_inner -> outer
|
||||
outer.done -> after_outer
|
||||
```
|
||||
|
||||
The `work` frame returns only to `inner`, its immediate owner. Once the inner
|
||||
controller emits `done`, execution resumes in the outer item frame. Returning
|
||||
`after_inner -> outer` then completes the outer item.
|
||||
|
||||
Runtime metadata already carries the immediate foreach owner. Static context
|
||||
analysis must evolve from a single active foreach identifier to an ownership
|
||||
stack so inner completion restores the outer item context.
|
||||
|
||||
## Validation and Analysis
|
||||
|
||||
Validation performs structured abstract traversal over `(node_id,
|
||||
foreach_owner_stack)` rather than merely looking for graph cycles:
|
||||
|
||||
- a `loop` edge from a foreach pushes that foreach onto the stack;
|
||||
- an edge targeting the top owner is an item return and ends that child
|
||||
traversal;
|
||||
- a `done` or `completed_with_errors` edge stays in the outer context;
|
||||
- `END` or `EndNode` reached with a non-empty foreach stack is invalid;
|
||||
- nested returns must target the immediate owner before an outer owner;
|
||||
- traversal memoizes node plus owner stack so valid cycles terminate analysis.
|
||||
|
||||
A node may remain reachable in more than one execution context. Existing
|
||||
context-contract analysis can report fields as conditional in that case. The
|
||||
runtime must never infer edge meaning from graph history alone: frame ownership
|
||||
always decides whether targeting a foreach is entry or return.
|
||||
|
||||
General unreachable-node detection is useful but is not part of this change.
|
||||
It cannot establish foreach ownership because any node reached from a body is
|
||||
connected by definition. General termination is also out of scope; finite
|
||||
structural checks cannot prove that data-dependent cycles eventually exit.
|
||||
|
||||
## State and Failure Behavior
|
||||
|
||||
Back-edge return changes control representation, not state semantics.
|
||||
Iteration writes remain buffered in the item lineage. Serial behavior and the
|
||||
concurrent barrier continue to commit or merge those writes according to the
|
||||
accepted concurrent-foreach ADR and declared reducers.
|
||||
|
||||
An ordinary node outcome named `error` remains domain control. An exception
|
||||
remains a runtime item failure handled by `fail`, `skip`, or `collect`. Neither
|
||||
kind of failure is encoded by the foreach back-edge itself.
|
||||
|
||||
## Migration
|
||||
|
||||
This is a clean canonical migration. Repository tests, examples, user-facing
|
||||
docs, and generated fixtures change from:
|
||||
|
||||
```text
|
||||
foreach.loop -> body
|
||||
body -> END
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```text
|
||||
foreach.loop -> body
|
||||
body -> foreach
|
||||
```
|
||||
|
||||
Configured local artifact stores contain no persisted foreach workflows at the
|
||||
time of design. There is therefore no demonstrated persisted-data requirement
|
||||
for keeping `body -> END` as a compatibility behavior. Validation rejects that
|
||||
old shape rather than silently preserving two canonical return forms. If real
|
||||
external persisted data is identified before implementation lands, it requires
|
||||
an explicit migration decision rather than a permanent implicit shim.
|
||||
|
||||
Checkpoints from an in-progress old foreach execution are likewise not given a
|
||||
speculative compatibility path without real data. The implementation must not
|
||||
add parse-old behavior solely because repository fixtures previously used the
|
||||
old topology.
|
||||
|
||||
## Testing
|
||||
|
||||
Focused tests must prove:
|
||||
|
||||
- serial item return wakes the parent and admits the next item;
|
||||
- concurrent item returns preserve item identity and deterministic barrier
|
||||
commits;
|
||||
- a child targeting its owner returns instead of executing the foreach node;
|
||||
- a root frame targeting the same foreach enters it normally;
|
||||
- an item frame can enter a different nested foreach;
|
||||
- nested foreach restores the outer item context after inner completion;
|
||||
- item paths to `END` and explicit `EndNode` fail validation;
|
||||
- traces and serialized checkpoints retain an inspectable return location;
|
||||
- existing `fail`, `skip`, `collect`, interrupt, reducer-conflict, sync, and
|
||||
async behavior remains intact after fixture migration.
|
||||
|
||||
## Deferred Work
|
||||
|
||||
- Ergonomic/context-manager Python authoring syntax.
|
||||
- `break`, `continue` nodes, target ports, or iteration return dispositions.
|
||||
- First-completed, first-error, first-success, cancellation, or race policies.
|
||||
- General unreachable-node and non-terminating-cycle diagnostics.
|
||||
- Fork/gather nodes and activation-token persistence.
|
||||
- Nested serialized workflow blocks.
|
||||
Reference in New Issue
Block a user