native subgraph execution
This commit is contained in:
+10
-11
@@ -49,13 +49,12 @@ implementation state.
|
||||
scaffolding slice is complete: core has `SubgraphNode`, structural
|
||||
`WorkflowRef`, workflow-level outcomes plus explicit `EndNode` termination,
|
||||
authoring helpers (`subgraph_ref` / `WorkflowBuilder.subgraph`), and artifact
|
||||
reference conversion helpers. Runtime subgraph execution is still absent.
|
||||
The next slice is non-interrupting child execution: resolve a prepared child
|
||||
workflow, create a child scope/lineage, preserve child trace, map child
|
||||
output back through the subgraph boundary, and route by the child's terminal
|
||||
outcome. Interrupt bubbling/resume and saved/deployed child resolution follow
|
||||
after that. Wrapper helpers currently run child workflows as ordinary nodes;
|
||||
true graph-as-node behavior belongs here.
|
||||
reference conversion helpers. Core can now execute a prepared local child
|
||||
workflow through an isolated child scope/lineage, preserve its trace entries,
|
||||
map child output through the boundary, and route by the child's terminal
|
||||
outcome. Interrupt bubbling/resume and saved/deployed child resolution remain
|
||||
next. Wrapper helpers currently run child workflows as ordinary nodes; native
|
||||
`SubgraphNode` is now the graph-as-node path for prepared children.
|
||||
- **Concurrent foreach**: implemented in core with explicit scheduling,
|
||||
reducer/merge semantics, item error policy, async handler batching, and
|
||||
quiescent interrupt behavior. Remaining work is polish and future reuse of
|
||||
@@ -98,7 +97,7 @@ Frame stress points remaining for native subgraphs and future fork/gather:
|
||||
|
||||
The MCP workflow authoring path is now usable enough for real testing. The next
|
||||
bottleneck is runtime/platform correctness: resumable child execution,
|
||||
native subgraph execution, persistent run history, and protocol-native
|
||||
progress reporting. Concurrent foreach supplies scheduler/lineage precedent;
|
||||
native child graphs are now the missing runtime boundary. Those pieces should
|
||||
come before adding more high-level authoring sugar.
|
||||
saved/deployed child resolution, persistent run history, and protocol-native
|
||||
progress reporting. Concurrent foreach and prepared native child execution now
|
||||
supply scheduler/lineage precedent. Those remaining pieces should come before
|
||||
adding more high-level authoring sugar.
|
||||
|
||||
@@ -33,18 +33,17 @@ the compatibility subset needed before native subgraphs:
|
||||
`ForeachBarrierState` now keeps scheduling/result metadata plus compatibility
|
||||
patches for old serialized barrier data.
|
||||
|
||||
Direct commits currently go through `is_root_lineage_frame(frame)`, which is the
|
||||
migration shortcut for root scope/root lineage. The eventual better shape is an
|
||||
explicit scope/lineage commit target, feasible once native subgraph completion
|
||||
can declare whether child writes commit to child scope, parent lineage, or only
|
||||
through boundary output bindings.
|
||||
Direct commits now go through a scope-root commit decision: top-level frames
|
||||
commit to root state, and prepared native-child root frames commit to their
|
||||
child scope state. Descendant item/branch lineages still buffer writes until a
|
||||
barrier or future gather commits them.
|
||||
|
||||
Remaining work should avoid jumping straight into a broad rewrite. Native
|
||||
subgraph scaffolding is now present (`SubgraphNode`, structural `WorkflowRef`,
|
||||
terminal workflow outcomes, and authoring helpers). The next runtime slice can
|
||||
execute a non-interrupting prepared child graph using the current scope/lineage
|
||||
primitives; interrupt bubbling and saved/deployed workflow resolution remain
|
||||
later work.
|
||||
subgraph scaffolding and non-interrupting prepared-child execution are now
|
||||
present (`SubgraphNode`, structural `WorkflowRef`, terminal workflow outcomes,
|
||||
authoring helpers, and `PreparedSubgraph`). Child graphs execute through their
|
||||
own scope/lineage and map output back at completion. Interrupt bubbling and
|
||||
saved/deployed workflow resolution remain later work.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -34,11 +34,10 @@ The full native subgraph use of `RuntimeScope` is not implemented yet.
|
||||
`ForeachBarrierState` still owns scheduling/barrier metadata, but no longer has
|
||||
to be the primary write store for new concurrent foreach item results.
|
||||
|
||||
Direct node commits currently use the explicit root-frame helper
|
||||
`is_root_lineage_frame(frame)`. That helper still means "root scope plus root
|
||||
lineage" during migration. The better long-term shape becomes feasible when
|
||||
native subgraph completion exists: direct commits should be decided by an
|
||||
explicit scope/lineage commit target, not by root ids.
|
||||
Direct node commits now use a scope-root decision: the top-level root commits
|
||||
to `RunState.state`, while a prepared native-subgraph root commits to its child
|
||||
scope state. Descendant item/branch lineages buffer writes until a barrier or
|
||||
future gather commits them.
|
||||
|
||||
## Problem
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Native Subgraphs Design
|
||||
|
||||
Status: scaffolding implemented; runtime execution planned
|
||||
Status: prepared-child execution implemented; interrupts/artifact resolution planned
|
||||
|
||||
Native subgraphs should make a workflow usable as a workflow step without
|
||||
collapsing the child run into one opaque Python node call. The current
|
||||
@@ -8,8 +8,9 @@ collapsing the child run into one opaque Python node call. The current
|
||||
compatibility wrappers, but they hide the child trace, child frames, and child
|
||||
interrupt lifecycle from `wf_core`.
|
||||
|
||||
This design defines the core runtime shape. The boundary model is implemented;
|
||||
child execution, interruption, and saved-workflow resolution remain planned.
|
||||
This design defines the core runtime shape. The boundary model and
|
||||
non-interrupting prepared-child execution are implemented; interruption and
|
||||
saved-workflow resolution remain planned.
|
||||
|
||||
## Goals
|
||||
|
||||
@@ -76,8 +77,11 @@ declare terminal outcomes through `Workflow.outcomes` and `EndNode`.
|
||||
native boundary, while artifact helpers convert saved/capability workflow
|
||||
references into core `WorkflowRef` values.
|
||||
|
||||
Runtime execution is deliberately not implemented: stepping a `SubgraphNode`
|
||||
fails explicitly until the next slice adds child scope/frame execution.
|
||||
Runtime execution now accepts caller-supplied `PreparedSubgraph` dependencies
|
||||
for local workflow refs. It creates a child scope/lineage, schedules child
|
||||
frames in the parent run, retains child trace entries, and applies mapped
|
||||
output only at boundary completion. Saved artifact refs are not loaded by
|
||||
`wf_core`, and child interrupts fail explicitly until resume routing exists.
|
||||
|
||||
`WorkflowRef` should be structural, not a dotted string parser:
|
||||
|
||||
@@ -325,11 +329,11 @@ child = parent.subgraph(
|
||||
```
|
||||
|
||||
This copies the compiled child workflow contract into a core `SubgraphNode`,
|
||||
appends it to the builder, and returns the step for normal routing. It does not
|
||||
make the child executable yet. The core `workflow` field is structural, but
|
||||
higher layers still need dependency resolution before saved/deployed workflow
|
||||
refs can run. The lower-level `subgraph_ref(...)` helper exists for code that
|
||||
wants only the core step object.
|
||||
appends it to the builder, and returns the step for normal routing. Runtime
|
||||
execution requires the child graph and its handlers to be supplied as a
|
||||
`PreparedSubgraph`; higher layers still need dependency resolution before
|
||||
saved/deployed workflow refs can run. The lower-level `subgraph_ref(...)`
|
||||
helper exists for code that wants only the core step object.
|
||||
|
||||
Possible API:
|
||||
|
||||
@@ -389,18 +393,18 @@ selection out of `wf_core`.
|
||||
- Artifact conversion helpers bridge saved workflow identities to core
|
||||
`WorkflowRef` values.
|
||||
|
||||
### Slice 1: Non-Interrupting Inline Subgraph Runtime
|
||||
### Completed Slice 1: Non-Interrupting Prepared Subgraph Runtime
|
||||
|
||||
- Resolve local/prepared child `WorkflowRef` dependencies at runtime; do not
|
||||
load saved artifacts inside `wf_core`.
|
||||
- Execute child workflow to completion through child frames.
|
||||
- Give the child an explicit runtime scope/lineage so child state is isolated
|
||||
- Local/prepared child `WorkflowRef` dependencies resolve through
|
||||
`PreparedSubgraph`; `wf_core` does not load saved artifacts.
|
||||
- Child workflows execute through child frames in the parent scheduler.
|
||||
- Each activation owns a child runtime scope/lineage so child state is isolated
|
||||
from parent state until boundary completion.
|
||||
- Preserve child trace in a clearly-owned form.
|
||||
- Apply child output to parent state through existing output binding code.
|
||||
- Route the parent step through the child's terminal `RunState.outcome`.
|
||||
- Tests: child output mapping, child internal trace visibility, parent trace
|
||||
shape, child runtime failure fails parent.
|
||||
- Child trace entries remain in the parent run with child frame ids; completion
|
||||
records the parent `subgraph` trace entry.
|
||||
- Child output maps to parent state through existing output binding machinery.
|
||||
- The parent step routes through the child's terminal workflow outcome.
|
||||
- Child interrupts reject explicitly until structural resume routing exists.
|
||||
|
||||
### Slice 2: Interrupt Bubbling and Resume
|
||||
|
||||
@@ -450,12 +454,10 @@ selection out of `wf_core`.
|
||||
|
||||
## Recommendation
|
||||
|
||||
The typed boundary scaffold is complete. Start runtime work with Slice 1 as a
|
||||
non-interrupting inline/prepared subgraph. It gives us native trace/frame and
|
||||
scope/lineage semantics without taking on the hardest resume problem
|
||||
immediately. Do not delete the wrapper-node helpers yet; use them as
|
||||
compatibility and examples while native subgraphs mature.
|
||||
The typed boundary scaffold and non-interrupting prepared-child runtime are
|
||||
complete. Do not delete the wrapper-node helpers yet; use them as compatibility
|
||||
and examples while native subgraphs mature.
|
||||
|
||||
Then implement Slice 2 before exposing saved workflows as broadly reusable child
|
||||
graphs. Saved workflows without nested interrupt support would look reusable but
|
||||
break at exactly the moment users need persistence and resume.
|
||||
Implement Slice 2 before exposing saved workflows as broadly reusable child
|
||||
graphs. Saved workflows without nested interrupt support would look reusable
|
||||
but break at exactly the moment users need persistence and resume.
|
||||
|
||||
@@ -127,30 +127,33 @@ limits and intended adapter seam.
|
||||
- Foreach supports serial and concurrent execution. Concurrent foreach uses
|
||||
explicit policy, typed barrier state, lineage-aware patch commits, item error
|
||||
policy, and quiescent interrupt handling. Remaining gaps are higher-level
|
||||
graph constructs such as native subgraphs, explicit fork/gather nodes, and
|
||||
advanced conflict strategies beyond exact-path mergeable reducers.
|
||||
graph constructs such as explicit fork/gather nodes and advanced conflict
|
||||
strategies beyond exact-path mergeable reducers.
|
||||
- Interrupt lifecycle is still node-level and run-state-level. Long-lived
|
||||
external subscriptions or notification streams need a separate lifecycle
|
||||
design. Interrupt `request` and `resume` are canonical binding lists; nested
|
||||
child-workflow resume is still future work.
|
||||
- Native subgraphs have a core model placeholder, `SubgraphNode`, but runtime
|
||||
execution is not implemented yet. The placeholder carries a child workflow
|
||||
reference, declared input/output schemas, binding lists, and declared
|
||||
outcomes so parent graph structure can validate before execution support
|
||||
lands.
|
||||
- Native subgraphs use `SubgraphNode` plus caller-supplied `PreparedSubgraph`
|
||||
dependencies. A prepared local child executes through a child runtime scope
|
||||
and lineage; child output commits only through declared boundary bindings and
|
||||
the parent routes by the child's terminal workflow outcome. Saved/deployed
|
||||
workflow resolution remains outside core and is not implemented at this
|
||||
boundary yet.
|
||||
- Nested subgraph interruption is not first-class yet. The current
|
||||
`wf_authoring` subgraph helpers wrap a child workflow as an ordinary sync or
|
||||
async node and validate the child output; they do not preserve a child run
|
||||
state that can interrupt, bubble to the parent, and later resume inside the
|
||||
child. See `examples/authoring_workflow_as_node.py` for the current
|
||||
wrapper-node shape.
|
||||
child. Native `SubgraphNode` preserves prepared-child execution and trace,
|
||||
but rejects child interrupts until route-aware resume is implemented. See
|
||||
`examples/authoring_workflow_as_node.py` for the wrapper-node shape.
|
||||
- Saved workflow-as-node execution with interrupts requires a core runtime
|
||||
upgrade: nested run state, child-frame trace preservation, interrupt bubbling
|
||||
with path metadata, and resume back into the child workflow.
|
||||
- Frames are no longer only a serial execution stack: the runtime has a ready
|
||||
queue, `BLOCKED` frame state, lineage isolation, barrier merge semantics, and
|
||||
pending child results for concurrent foreach. Native subgraphs still need
|
||||
explicit child workflow/deployment identity and child-scope execution.
|
||||
pending child results for concurrent foreach. Native prepared subgraphs now
|
||||
use child-scope execution; saved/deployed child resolution and nested
|
||||
interruption remain outstanding.
|
||||
Concurrent foreach is the primary current use case for async concurrent node
|
||||
handler execution.
|
||||
- Runtime errors are still ordinary exceptions plus failed run status. A richer
|
||||
|
||||
Reference in New Issue
Block a user