docs

CONTEXT.md

@@ -69,6 +69,14 @@ _Avoid_: Global committed state
 A merge boundary that waits for multiple child or upstream frames and combines their lineage patches before continuation.
 _Avoid_: Noop join
 
+**Gather Node**:
+A future graph step that exposes an explicit barrier for waiting on multiple branches and merging their results.
+_Avoid_: Promise.all node, converge node
+
+**Lineage Token**:
+A future runtime marker for one branch lineage that can be consumed and merged by gather-style barriers.
+_Avoid_: Edge id
+
 **Run**:
 One execution attempt of a workflow with input, state, frames, trace, and final output.
 _Avoid_: Job, invocation
@@ -87,6 +95,170 @@ _Avoid_: Job, invocation
 - Parallel foreach and native subgraphs depend on the **Scheduler Foundation**.
 - Parallel foreach requires a **Foreach Policy** before public support is
   enabled.
+- Future parallel foreach should be bounded by default. `max_active` and
+  `max_outstanding` prevent one workflow from spawning unbounded MCP, HTTP,
+  browser, or external service calls.
+- Future concurrency-specific foreach settings should live in a nested parallel
+  policy object rather than expanding `ForeachNode` with many top-level fields.
+- Item error handling is foreach-wide, not parallel-only. Serial and parallel
+  foreach can both profit from `fail`, `skip`, or `collect` item failure policy.
+- `collect` item error policy must declare an explicit destination for
+  structured item errors. Collected errors should be ordered by item index, not
+  async completion order.
+- Item error policy handles runtime failures inside an item frame, such as
+  thrown handler errors, schema failures, runtime-error utility nodes, or
+  invalid graph execution. It does not intercept normal graph outcomes like
+  `error` when those outcomes are routed.
+- A collected or skipped item failure should leave that item frame `FAILED`.
+  The foreach parent/barrier decides whether that failed child is handled and
+  whether the run may continue.
+- `skip` item error policy writes no hidden state. Failed frame status and
+  trace/observability are the record; use `collect` when structured error state
+  is needed.
+- `collect` writes structured item error records before emitting an aggregate
+  error outcome. Records include item index, frame id, failing node id, error
+  type/message, and the item value when it can be represented safely.
+- `collect` destinations must be state array fields. The foreach barrier writes
+  the full ordered error list once, so users should not expect per-item append
+  writes. Authoring and MCP validation should surface this clearly.
+- `completed_with_errors` is the canonical foreach aggregate outcome when all
+  items have finished but one or more item failures were handled by `collect` or
+  `skip`.
+- `completed_with_errors` is barrier aggregate vocabulary, not foreach-only.
+  Future explicit barriers may emit it when handled branch failures occurred.
+- Aggregate control-flow outcomes use result nouns/adjectives such as `done`,
+  `completed_with_errors`, and optionally `failed`. Policy actions use verbs
+  such as `fail`.
+- If a foreach policy can emit `completed_with_errors`, validation should treat
+  it as a required routable outcome. Users may route it to the same target as
+  `done` explicitly.
+- `collect` writes an empty list to its destination when all items succeed and
+  emits `done`. It emits `completed_with_errors` only when at least one item
+  failure was collected.
+- `skip` emits `completed_with_errors` when one or more item failures were
+  skipped. It emits `done` only when all items succeed.
+- Parallel `fail` item policy should stop scheduling new items, drain already
+  started jobs to a quiescent point, capture their results safely, and then fail
+  the run. It should not assume hard cancellation is safe.
+- After `fail` trips, drained sibling results are for trace/observability and
+  cleanup only. They should not commit normal state progress after the failure
+  boundary.
+- Foreach modes that continue after item failure, such as `collect` and `skip`,
+  should buffer item state patches until the foreach barrier completes. Future
+  parallel foreach should always use barrier-buffered commits.
+- Serial `fail` may keep immediate commits for compatibility. Commit strategy
+  should be extracted into runtime helpers instead of being smeared through
+  foreach execution code.
+- Barrier-buffered commits should store pending state patches/results, not full
+  state snapshots. Patch creation and commit must extract/reuse the existing
+  node output validation, output binding, and reducer logic rather than creating
+  a second write system.
+- Foreach barrier commits should merge item results in item index order, not
+  completion order. Reducer behavior such as list appends should therefore be
+  deterministic.
+- Successful item results may exist as pending barrier results before commit,
+  but they are not visible as workflow state until the barrier commits.
+- Pending barrier results must live in resumable `RunState`/frame metadata, not
+  only in trace. Trace records history; runtime state is what resume/checkpoint
+  uses.
+- Parallel item frames need lineage-local pending state: later nodes in the same
+  item lineage can read earlier pending patches from that item, while sibling
+  items cannot. The exact aggregate output API for committing item results to
+  parent state is deferred.
+- Lineage-local state should be represented as patch overlays over parent-visible
+  state, not deep-copied full state snapshots.
+- Patch overlays conceptually belong to lineage tokens, not execution frames.
+  A scoped foreach implementation may start by storing them in item/parent
+  metadata, but future Fork/Gather needs first-class lineage ownership.
+- `RunState.state` remains committed parent/global state. Future frame execution
+  should resolve reads against a frame-specific visible state view built from
+  committed state plus visible lineage overlays.
+- At a barrier, missing reducer means default replace only for single-writer
+  paths. Multiple sibling lineages writing the same path require an explicit
+  reducer; otherwise barrier commit raises a runtime error with writer details.
+- Barrier conflict detection should use the same write-overlap rules as normal
+  state writes. Ancestor/descendant writes from different lineages are conflicts
+  unless an explicit merge strategy covers them.
+- Reducers at barriers apply incrementally in deterministic lineage order by
+  default: item index order for foreach, declared branch token order for future
+  gather. Completion-order merging is a possible explicit barrier policy, not
+  reducer behavior.
+- Collected error records follow the same barrier merge order policy. The
+  default is deterministic lineage order.
+- Trace `state_changes` should mean committed state changes only. Do not encode
+  pending barrier patches there unless a future typed trace field is added.
+- Foreach may keep implicit barrier/iteration state on the foreach parent frame
+  because it owns item spawning and refill. General branch convergence should
+  become an explicit **Gather Node** later. Shared barrier merge/result helpers
+  should prevent foreach and future barriers from duplicating patch ordering,
+  conflict checks, and failure aggregation.
+- Future Fork/Gather needs **Lineage Tokens**, not raw edge ids. A fork produces
+  branch lineage tokens; a gather consumes a declared set of tokens and produces
+  a new merged token. This supports partial gathers such as merging branches
+  `a+b` before later merging with `c`.
+- Explicit Fork/Gather is deferred until lineage tokens are designed. Parallel
+  foreach remains the nearer target because its implicit lineage tokens are item
+  indexes owned by one foreach activation.
+- Future foreach metadata should evolve into inherited structured lineage
+  context. Alias lookup such as `context.document` can remain authoring sugar,
+  but runtime metadata should preserve nested foreach lineage without flat key
+  collisions.
+- Nested active foreach aliases should not shadow each other. Alias collisions
+  in inherited context scope should be validation errors.
+- Core runtime context should be structural, not alias-first. Foreach lineage
+  should be addressable through paths such as `context.foreach.<id>.index`;
+  `wf_authoring` can provide ergonomic alias helpers such as
+  `context_path(foreach_ref("docs").index)`.
+- Python `RuntimeContext` should eventually expose typed structured foreach
+  context, such as `ctx.foreach["docs"].index`, while serialized frame metadata
+  remains JSON-compatible dictionaries.
+- Structured foreach context keys should be foreach node ids. The `as_` alias is
+  authoring sugar for current item access, not the canonical runtime key.
+- `wf_authoring.WorkflowBuilder.foreach(...)` should eventually return a richer
+  ref exposing context selectors such as `.item` and `.index`, so users do not
+  hand-write `context.foreach.<id>...` paths.
+- Normal node authors should receive foreach values through mapped input.
+  Inspecting `RuntimeContext.foreach` is an advanced escape hatch for nodes that
+  genuinely need index/frame/lineage context.
+- `as_` remains useful ergonomic sugar and human-readable trace/docs context,
+  but structured foreach refs should be preferred for non-trivial authoring.
+- Future foreach policy shape should validate cross-field rules: `collect`
+  requires `collect_to`, non-collect actions forbid `collect_to`,
+  `mode="parallel"` requires a parallel policy, and `mode="serial"` forbids a
+  parallel policy. Deprecated top-level `on_item_error` may parse into the
+  nested item error policy, but canonical dumps should use the nested shape.
+- `ForeachParallelPolicy` should split limits into `max_active` and
+  `max_outstanding`. Defaults are `max_active=4` and `max_outstanding=20`;
+  validation requires `max_outstanding >= max_active`. Ready or running item
+  frames consume active capacity; blocked item frames consume outstanding
+  capacity but not active capacity.
+- A blocked non-interrupt item frame frees active capacity and may let foreach
+  start another item when `max_outstanding` also has room. A run-level interrupt
+  still stops scheduling.
+- The foreach parent frame owns refill decisions. Scheduler wakes/schedules
+  frames; foreach-specific policy decides whether to start more children,
+  finish, or fail.
+- When an item frame becomes blocked, it frees active capacity only for its
+  nearest foreach capacity owner. Future item metadata should identify that
+  owner rather than waking arbitrary ancestors.
+- Foreach capacity is local correctness policy, not total process protection.
+  Future runtime should also have basic global run limits in `wf_core`; source-
+  or tool-specific limits belong in the platform layer.
+- A future global `wf_core` runtime limit should count active node handler calls,
+  not all active frames. Control-flow frames are scheduler work; node calls are
+  the expensive external/user-code execution boundary.
+- Global node-call limits do not replace foreach caps. Foreach caps bound local
+  scheduling fairness, outstanding frame count, memory, and pending results;
+  global node-call limits bound expensive handler execution across the run.
+- Platform source/tool/account limits should be enforced at the node-handler
+  boundary, before invoking the external call. They layer on top of core global
+  node-call admission instead of replacing it.
+- Node calls waiting on platform source/tool/account limits still count as
+  active node calls. Core global node-call admission should happen before
+  platform-specific semaphore acquisition.
+- Waiting on a source/tool/account semaphore keeps the frame `RUNNING`; it is
+  backpressure inside the admitted node-call boundary, not workflow-level
+  `BLOCKED` state that frees foreach active capacity.
 - An **Interrupt** pauses the whole **Run**, even if the interrupted frame is a
   child of future parallel work.
 - A **Runtime Failure** is distinct from a node returning an `error` outcome.