lda-wf/docs/adr/0006-explicit-fork-and-topology-driven-gather.md

---
status: proposed
---

# Explicit Fork and Topology-Driven Gather

General graph concurrency will use explicit fork and gather steps. Outcomes
continue to select one transition; forks create several branch activations;
gathers rendezvous compatible activation tokens, merge their lineage-local
state patches, and create one continuation. This preserves topology-only edges
without making multiple matching edges silently mean broadcast.

The [draft design contract](../superpowers/specs/2026-09-06-fork-gather-design.md)
records the current requirements, open decisions, and verification gates.
The [research archive](../historical/research/2026-09-06-fork-gather/README.md)
preserves supporting evidence and known reference-model defects. This ADR is
architectural direction, not an executable implementation plan.

## Context

The scheduler, ready queue, blocked frames, lineage-local state views, and
reducer-aware barrier commits already support concurrent foreach and native
subgraphs. They do not yet define general graph-level fork/gather. A former
`JoinNode` day-one marker immediately emitted `done`; it was removed after a
census found no real persisted workflows using it. `GatherNode` therefore does
not inherit placeholder semantics or a compatibility burden.

A cross-system semantics review supported keeping `NodeResult.output` separate
from its named domain `outcome`, keeping operational failure outside that
outcome namespace, and representing concurrency with explicit control steps.
The decisive design pressure came from partial and repeated gathers rather than
from output typing.

Consider a fork producing `a`, `b`, and `c`. One gather may merge `a+b` into
`d`, while a later gather merges `d+c`. The first gather must not know which
fork originally produced its inputs, and the second must accept the merged
`d` continuation like any other arrival. Inside a loop, it must combine
`a1+b1`, never `a1+b2`.

Conditional paths add another requirement. If one branch continues through
either `d` or `e`, a later gather may require `b AND (d OR e)`. Waiting for
every incoming edge would deadlock because only one alternative can arrive.

Cross-merging raises the same issue at larger scale. If one activated region
produces `b,c` and another produces `e,f`, independent gathers may consume
`b+e` and `c+f`. Correctness cannot depend on whether the scheduler happens to
run `b,e,c,f` or `c,f,b,e` first.

## Decision

The graph model distinguishes three control meanings:

- an ordinary node outcome selects exactly one continuation;
- a fork creates one child activation for every declared branch;
- a gather consumes compatible arrivals and creates one continuation.

A gather is independent of the construct that produced its inputs. It knows
its required local input slots from the graph and owns a merge policy. Edges
targeting a gather identify the slot they can satisfy. A gather requires every
slot, while several incoming edges may be alternatives for one slot. This gives
`b AND (d OR e)` without placing executable predicates on edges.

Runtime arrivals carry activation identity, lineage identity, and provenance.
The gather buckets arrivals by compatible activation context, accepts at most
one token per slot for an activation, waits until all slots are present, then
merges their lineage-local patches. Its result is a new continuation token that
preserves combined provenance and can enter another gather.

Runtime identity follows one canonical dependency chain. A runtime scope is one
workflow invocation and its committed state universe. A lineage belongs to
exactly one scope and represents one isolated state worldview inside it. An
execution frame is a schedulable graph cursor that executes against one
lineage, so the frame's scope and parent lineage are derived through that
lineage rather than independently stored relationships. Frame ancestry remains
separate: a parent frame expresses scheduling ownership and block/wake behavior,
not state ancestry.

Lineages are not owned by frames. A completed branch frame may leave its
lineage waiting at a gather, and a later continuation frame may execute against
the lineage produced by a merge. Activation tokens bind a control-flow arrival
to its lineage plus the correlation and provenance needed by gathers; they do
not duplicate the lineage's scope membership.

Lineage remains a virtual worldview: committed scope state plus writes visible
to one branch. Gathering several lineages does not require turning lineage
ancestry into a multi-parent graph. Every input lineage must belong to the same
runtime scope. Because lineages have one parent, their ancestry chains have at
most one deepest shared lineage: the lowest common ancestor is the deterministic
merge base. Different scopes or no shared ancestor fail before state mutation.
A partial gather creates its intermediate lineage under that merge base and
retains multi-input provenance in activation-token metadata. A final gather can
merge that lineage with remaining siblings and create a continuation token.
Completion ownership survives token/frame replacement; foreach and subgraph
parents still wait for the corresponding item or call to finish.

The lineage tree is not sufficient to deduplicate shared pending history after
re-forking, cross-gathering, and reconvergence. Supporting that topology requires
stable original contribution identities or an equivalent demonstrated scheme;
otherwise the topology must be explicitly rejected. New node executions create
new contributions even when node and token identities are unchanged. Merging
existing contributions preserves their identities.

Control obligations also need two distinct ancestry queries. Referenced fork
visits cover any component of a claim and govern unresolved re-entry checks.
Enclosing visits cover every component and govern gather correlation. For a
mixed claim `{k.x, r.c}` with `k` nested under `r`, only `r` encloses the whole
claim. The proposed V1 matches a compiled anchor to that enclosing dynamic
visit, without requiring an authored originating-fork field. One gather may
fire at most once per owner/gather/anchor occurrence. Validation must account
for past firings as well as currently parked arrivals.

The first gather merge policy is fail-closed:

```python
class GatherMergePolicy:
    conflicts: Literal["error"] = "error"
```

State-field reducers remain the source of truth for legitimate concurrent
merges. The gather policy determines what happens when patches cannot be
merged; the initial behavior is to fail rather than choose a last writer.

Gather slots have declaration order, and that order is the canonical reducer
replay order for independent histories, never arrival, scheduler, or frame-id
order. Whether an earlier partial gather's serialization becomes a permanent
ordering constraint at later reconvergence remains open. Opposite histories
`[A,B]` and `[B,A]` can arise in legitimate authored graphs; treating their
combination as a merge-order conflict is a policy choice, not a corruption
check. A bucket accepts exactly one token for each slot;
a second token for the same activation and slot fails the activation instead of
making an alternative-path race decide the result. Order-sensitive reducers
such as append are therefore deterministic in synchronous and asynchronous
execution.

Branch execution order may be deterministic in the synchronous runtime and
overlap in the asynchronous runtime. Both modes must produce equivalent graph
semantics. Scheduler order decides when compatible work progresses, never which
arrivals belong together.

An unhandled branch failure makes its gather activation terminally failed and
stops further branch admission. The runtime requests cancellation of admitted
siblings, awaits every sibling's settlement, and accepts no later state or
trace commits from them; external effects that already occurred cannot be
rolled back. It then marks the failed activation's uncommitted lineages
abandoned and permanently non-mergeable, invalidates and removes every pending
gather token, marks sibling frames cancelled or failed, and persists the failed
run. Restore may inspect those frames and lineages but cannot schedule them or
consume a token from the failed activation.

`END` and explicit `EndNode` represent workflow/subgraph termination, not a
generic way to complete any child frame. A foreach item returns through a
back-edge targeting its owning `ForeachNode`; the runtime completes that item
frame and wakes its parent barrier without executing the controller inside the
child. Future fork branches likewise converge through explicit gathers rather
than independently terminating the containing workflow scope.

General break and race behavior are not part of the first fork/gather design.
The existing foreach `fail`, `skip`, and `collect` policies describe the
disposition of runtime item failures; they are distinct from completion
policies such as first-completed, first-error, or all-completed. A future break
feature would require an operational signal to the owning foreach and an
explicit policy for already admitted concurrent items. No `BreakNode` is added
without that use case and policy.

The placeholder `JoinNode` was removed rather than silently upgraded. The
repository has no real persisted artifacts outside tests, so `GatherNode` can
be introduced with its actual rendezvous and merge contract and no legacy wire
alias.

## Considered Options

**Multiple ordinary edges broadcast.** Rejected because edge cardinality would
silently change an outcome from selection to spawning and make ordinary graph
convergence ambiguous.

**Gather references its originating fork.** Rejected because partial gathers,
staged gathers, conditional paths, and cross-merges consume tokens based on
their local rendezvous contract, not one producer.

**Gather waits for every incoming edge.** Rejected because alternative paths
such as `d OR e` would require mutually exclusive arrivals. Named slots provide
AND across slots and OR within a slot.

**Lineage becomes a multi-parent DAG.** Not currently required. An intermediate
merged worldview can remain a child of the inputs' common lineage parent, but
control provenance alone is not proof of state-write uniqueness. Shared-write
identity and ordering require the separate contract described above.

**Store scope and parent-lineage identity on every frame.** Rejected because
those relationships are canonical on the lineage and duplicated frame fields
can disagree with them. Runtime operations should resolve and validate a
frame's lineage and scope together.

**Model normal foreach completion as a continue node.** Rejected because an
ordinary back-edge to the owning foreach already expresses item return in the
canonical graph. Break and race semantics remain separate future policies
rather than additional meanings assigned to ordinary outcomes.

**Reuse or rename the placeholder `JoinNode`.** Rejected and removed because
the node was a pass-through marker with no barrier contract.

## Consequences

- Edge identity gains gather-slot significance only when its target is a
  gather; ordinary edge semantics stay unchanged.
- Workflow validation must require every gather-target edge to name exactly one
  declared slot, reject missing or unknown gather slots, prove that every slot
  has an incoming edge, preserve but ignore ports on ordinary targets, and keep
  one successor per ordinary `(node, outcome)` pair.
- Checkpoints must persist pending gather arrivals and activation provenance so
  interruption/resume cannot mix loop iterations or subgraph invocations.
- Supported checkpoints must not expose partially applied transitions, including
  when an exception produces a failed stopped run. This does not require a
  durable transition journal or promise exactly-once external effects.
- Runtime operations should resolve a frame, its lineage, and its scope through
  one internal interface instead of accepting several independently supplied
  identifiers.
- Persisted frame state should not duplicate scope or parent-lineage
  relationships once real checkpoint compatibility has been checked and any
  required migration has been defined.
- Trace output must make fork activation, branch identity, gather waiting, and
  merged continuation inspectable without treating scheduler bookkeeping as
  ordinary node output.
- Fork/gather should generalize concurrent-foreach lineage and barrier helpers,
  not create a second state-patch system.
- Runtime branch failures remain execution failures with the terminal cleanup
  above. Skip, collect, race, first-success, configurable cancellation, and
  timeout policies are deferred.

## Open Questions

- The supported correlation grammar and production validation algorithm for
  loops, nested forks, partial gathers, and cross-merges. Destination edges use
  `target_port`; ordinary destinations ignore that metadata.
- Whether prior gather ordering is permanent history or a local state view.
- The concrete owner/occurrence representation and integration with blocked
  foreach and subgraph completion, including failure and checkpoint handling.
- The trace representation for waiting and merging without excessive internal
  scheduler noise.

This ADR extends the lineage and barrier direction established by
[ADR-0002](0002-concurrent-foreach-policy-and-barrier-commits.md). It remains
proposed until the open runtime-state questions are resolved in the fork/gather
design specification.