fix: close workflow review gaps

This commit is contained in:
lda
2026-09-04 20:00:21 +07:00 Verified
parent 220df14314
commit 6b5c79ba21
15 changed files with 169 additions and 60 deletions
@@ -76,10 +76,14 @@ not duplicate the lineage's scope membership.
Lineage remains a virtual worldview: committed scope state plus writes visible
to one branch. Gathering several lineages does not require turning lineage
ancestry into a multi-parent graph. A partial gather can create an intermediate
lineage under the branches' common parent, retaining multi-input provenance in
activation-token metadata. A final gather can merge that lineage with remaining
siblings and resume the blocked parent continuation.
ancestry into a multi-parent graph. Every input lineage must belong to the same
runtime scope. Because lineages have one parent, their ancestry chains have at
most one deepest shared lineage: the lowest common ancestor is the deterministic
merge base. Different scopes or no shared ancestor fail before state mutation.
A partial gather creates its intermediate lineage under that merge base and
retains multi-input provenance in activation-token metadata. A final gather can
merge that lineage with remaining siblings and resume the blocked parent
continuation.
The first gather merge policy is fail-closed:
@@ -92,11 +96,30 @@ State-field reducers remain the source of truth for legitimate concurrent
merges. The gather policy determines what happens when patches cannot be
merged; the initial behavior is to fail rather than choose a last writer.
Gather slots have declaration order, and that order is the canonical reducer
replay order. After choosing the merge base, the runtime applies each selected
lineage's writes after that base in declared-slot order, never arrival,
scheduler, or frame-id order. A bucket accepts exactly one token for each slot;
a second token for the same activation and slot fails the activation instead of
making an alternative-path race decide the result. Order-sensitive reducers
such as append are therefore deterministic in synchronous and asynchronous
execution.
Branch execution order may be deterministic in the synchronous runtime and
overlap in the asynchronous runtime. Both modes must produce equivalent graph
semantics. Scheduler order decides when compatible work progresses, never which
arrivals belong together.
An unhandled branch failure makes its gather activation terminally failed and
stops further branch admission. The runtime requests cancellation of admitted
siblings, awaits every sibling's settlement, and accepts no later state or
trace commits from them; external effects that already occurred cannot be
rolled back. It then marks the failed activation's uncommitted lineages
abandoned and permanently non-mergeable, invalidates and removes every pending
gather token, marks sibling frames cancelled or failed, and persists the failed
run. Restore may inspect those frames and lineages but cannot schedule them or
consume a token from the failed activation.
`END` and explicit `EndNode` represent workflow/subgraph termination, not a
generic way to complete any child frame. A foreach item returns through a
back-edge targeting its owning `ForeachNode`; the runtime completes that item
@@ -153,9 +176,10 @@ node is a pass-through marker with no barrier contract.
- Edge identity gains gather-slot significance only when its target is a
gather; ordinary edge semantics stay unchanged.
- Workflow validation must prove that every gather slot has an incoming edge,
reject slots on non-gather targets, and preserve one successor per ordinary
`(node, outcome)` pair.
- Workflow validation must require every gather-target edge to name exactly one
declared slot, reject missing or unknown gather slots, prove that every slot
has an incoming edge, reject slots on non-gather targets, and preserve one
successor per ordinary `(node, outcome)` pair.
- Checkpoints must persist pending gather arrivals and activation provenance so
interruption/resume cannot mix loop iterations or subgraph invocations.
- Runtime operations should resolve a frame, its lineage, and its scope through
@@ -169,8 +193,9 @@ node is a pass-through marker with no barrier contract.
ordinary node output.
- Fork/gather should generalize concurrent-foreach lineage and barrier helpers,
not create a second state-patch system.
- Runtime branch failures remain execution failures in the first version. Skip,
collect, race, first-success, cancellation, and timeout policies are deferred.
- Runtime branch failures remain execution failures with the terminal cleanup
above. Skip, collect, race, first-success, configurable cancellation, and
timeout policies are deferred.
## Open Questions
@@ -4,7 +4,8 @@
Implemented on 2026-09-04. This document specifies canonical foreach
body-return semantics. It does not include the separately planned ergonomic
Python DSL or authorize fork/gather implementation.
Python DSL or the explicit fork/gather implementation described by
[ADR-0006](../../adr/0006-explicit-fork-and-topology-driven-gather.md).
## Purpose
@@ -93,6 +93,17 @@ behavior. The counter is incremented before user code or external capability
code begins, so failures and interrupts still consume the attempt that caused
them.
For a durable run, admission is checkpointed before dispatch. The checkpoint
contains the incremented counter, assigned step number, selected frame and
node, and an admitted-but-not-completed marker. Dispatch may begin only after
that checkpoint succeeds. If the process stops at that boundary, restore keeps
the attempt consumed, clears the abandoned admission marker, and requeues the
frame; a retry is a new attempt with a new step number. The in-memory executor
applies the same counter transition without requiring a persistence backend.
As with any crash after external dispatch and before result persistence, retry
may repeat external effects; the budget records attempts and does not provide
exactly-once execution.
If `steps_executed == max_steps`, the next attempted dispatch is denied. A
budget of one therefore admits exactly one step. The denied step does not
increment the counter and does not invoke a handler.
@@ -132,6 +143,15 @@ Reserved async attempts remain consumed even if one handler raises. This
matches the rule that admission, rather than successful completion, consumes
the budget and avoids making counts depend on task completion timing.
The runtime awaits every handler in an admitted batch before finalizing any
result. It then finalizes in reserved ready-queue order. Handled foreach item
failures follow their declared `skip` or `collect` policy. At the first
unhandled failure in that order, preceding successful results have committed,
the run fails, and later sibling results are discarded without state or trace
commits. Because all handler tasks have already settled, no sibling can mutate
the failed checkpoint afterward; external effects performed inside a handler
remain outside rollback.
## Exhaustion Behavior
Exhaustion is a runtime failure, not a workflow outcome. The runtime raises a
@@ -160,16 +180,19 @@ administrative rerun can raise the budget is outside ordinary resume semantics.
## Persistence and Resume
`RunLimits` and `steps_executed` are serialized inside the existing persisted
`RunState` checkpoint. An interrupted run resumes with its original maximum and
cumulative count.
`RunLimits` and `steps_executed` are serialized inside the persisted `RunState`
checkpoint. An interrupted run resumes with its original maximum and cumulative
count.
The persisted run envelope may remain at version 1 because adding dataclass
fields with defaults is structurally additive. Loading an older checkpoint
that lacks these fields yields the default limit and a zero count. The
repository has no declared production migration requirement for reconstructing
historical counts that were never recorded; if real stored checkpoints exist,
their migration policy must be established before release.
A checkpoint that predates step budgets receives one explicit, prospective
upgrade: assign the default limit and `steps_executed = 0`, mark the envelope as
budget-initialized, and persist the upgraded checkpoint before admitting any
new work. Attempts made before the upgrade cannot be reconstructed and are
explicitly outside the new budget; every attempt after it is cumulative. A
missing counter on an already budget-initialized envelope is corrupt state, not
another request for defaults. If the one-time upgrade cannot be persisted,
resume fails before dispatch. The persisted envelope version or equivalent
migration marker must distinguish these cases.
Subgraph scopes do not receive independent counters. They are part of the same
run and consume the root run's budget. This prevents an outer workflow from
@@ -258,6 +281,8 @@ with the run.
- A batch claims no more frames than the remaining budget.
- Step numbers follow ready-queue order rather than completion order.
- Reserved attempts remain counted when one async handler fails.
- A still-running sibling settles before an unhandled handler failure is
checkpointed, and its later result does not commit state or trace data.
- Sync and async runs produce the same count for equivalent serial execution.
### Persistence and API
@@ -265,7 +290,10 @@ with the run.
- Limits and counts round-trip through `dump_run_state()` and
`load_run_state()`.
- A stored interrupted run resumes without resetting or replacing its budget.
- Older additive checkpoints receive documented defaults.
- A pre-budget checkpoint receives its defaults once, persists the upgraded
envelope before dispatch, and cannot receive another fresh budget on reload.
- Stopping after the admission checkpoint but before handler start leaves the
attempt consumed; retrying the requeued frame consumes a new attempt.
- Run inspection exposes effective maximum, executed, and remaining counts.
- Trace entries expose deterministic step numbers without becoming the source
of enforcement truth.
+1
View File
@@ -234,6 +234,7 @@ An iteration body returns through its immediate owning foreach:
g.connect(each, "loop", record)
g.connect(record, "ok", each)
g.connect(each, "done", END)
g.connect(each, "completed_with_errors", END)
```
Region conflicts, unreachable nodes, body terminals, non-local returns, empty