native subgraph execution

This commit is contained in:
lda
2026-05-25 04:28:06 +07:00 Verified
parent 3880a86c26
commit a4eeb506be
20 changed files with 746 additions and 146 deletions
+10 -11
View File
@@ -49,13 +49,12 @@ implementation state.
scaffolding slice is complete: core has `SubgraphNode`, structural scaffolding slice is complete: core has `SubgraphNode`, structural
`WorkflowRef`, workflow-level outcomes plus explicit `EndNode` termination, `WorkflowRef`, workflow-level outcomes plus explicit `EndNode` termination,
authoring helpers (`subgraph_ref` / `WorkflowBuilder.subgraph`), and artifact authoring helpers (`subgraph_ref` / `WorkflowBuilder.subgraph`), and artifact
reference conversion helpers. Runtime subgraph execution is still absent. reference conversion helpers. Core can now execute a prepared local child
The next slice is non-interrupting child execution: resolve a prepared child workflow through an isolated child scope/lineage, preserve its trace entries,
workflow, create a child scope/lineage, preserve child trace, map child map child output through the boundary, and route by the child's terminal
output back through the subgraph boundary, and route by the child's terminal outcome. Interrupt bubbling/resume and saved/deployed child resolution remain
outcome. Interrupt bubbling/resume and saved/deployed child resolution follow next. Wrapper helpers currently run child workflows as ordinary nodes; native
after that. Wrapper helpers currently run child workflows as ordinary nodes; `SubgraphNode` is now the graph-as-node path for prepared children.
true graph-as-node behavior belongs here.
- **Concurrent foreach**: implemented in core with explicit scheduling, - **Concurrent foreach**: implemented in core with explicit scheduling,
reducer/merge semantics, item error policy, async handler batching, and reducer/merge semantics, item error policy, async handler batching, and
quiescent interrupt behavior. Remaining work is polish and future reuse of quiescent interrupt behavior. Remaining work is polish and future reuse of
@@ -98,7 +97,7 @@ Frame stress points remaining for native subgraphs and future fork/gather:
The MCP workflow authoring path is now usable enough for real testing. The next The MCP workflow authoring path is now usable enough for real testing. The next
bottleneck is runtime/platform correctness: resumable child execution, bottleneck is runtime/platform correctness: resumable child execution,
native subgraph execution, persistent run history, and protocol-native saved/deployed child resolution, persistent run history, and protocol-native
progress reporting. Concurrent foreach supplies scheduler/lineage precedent; progress reporting. Concurrent foreach and prepared native child execution now
native child graphs are now the missing runtime boundary. Those pieces should supply scheduler/lineage precedent. Those remaining pieces should come before
come before adding more high-level authoring sugar. adding more high-level authoring sugar.
@@ -33,18 +33,17 @@ the compatibility subset needed before native subgraphs:
`ForeachBarrierState` now keeps scheduling/result metadata plus compatibility `ForeachBarrierState` now keeps scheduling/result metadata plus compatibility
patches for old serialized barrier data. patches for old serialized barrier data.
Direct commits currently go through `is_root_lineage_frame(frame)`, which is the Direct commits now go through a scope-root commit decision: top-level frames
migration shortcut for root scope/root lineage. The eventual better shape is an commit to root state, and prepared native-child root frames commit to their
explicit scope/lineage commit target, feasible once native subgraph completion child scope state. Descendant item/branch lineages still buffer writes until a
can declare whether child writes commit to child scope, parent lineage, or only barrier or future gather commits them.
through boundary output bindings.
Remaining work should avoid jumping straight into a broad rewrite. Native Remaining work should avoid jumping straight into a broad rewrite. Native
subgraph scaffolding is now present (`SubgraphNode`, structural `WorkflowRef`, subgraph scaffolding and non-interrupting prepared-child execution are now
terminal workflow outcomes, and authoring helpers). The next runtime slice can present (`SubgraphNode`, structural `WorkflowRef`, terminal workflow outcomes,
execute a non-interrupting prepared child graph using the current scope/lineage authoring helpers, and `PreparedSubgraph`). Child graphs execute through their
primitives; interrupt bubbling and saved/deployed workflow resolution remain own scope/lineage and map output back at completion. Interrupt bubbling and
later work. saved/deployed workflow resolution remain later work.
--- ---
@@ -34,11 +34,10 @@ The full native subgraph use of `RuntimeScope` is not implemented yet.
`ForeachBarrierState` still owns scheduling/barrier metadata, but no longer has `ForeachBarrierState` still owns scheduling/barrier metadata, but no longer has
to be the primary write store for new concurrent foreach item results. to be the primary write store for new concurrent foreach item results.
Direct node commits currently use the explicit root-frame helper Direct node commits now use a scope-root decision: the top-level root commits
`is_root_lineage_frame(frame)`. That helper still means "root scope plus root to `RunState.state`, while a prepared native-subgraph root commits to its child
lineage" during migration. The better long-term shape becomes feasible when scope state. Descendant item/branch lineages buffer writes until a barrier or
native subgraph completion exists: direct commits should be decided by an future gather commits them.
explicit scope/lineage commit target, not by root ids.
## Problem ## Problem
@@ -1,6 +1,6 @@
# Native Subgraphs Design # Native Subgraphs Design
Status: scaffolding implemented; runtime execution planned Status: prepared-child execution implemented; interrupts/artifact resolution planned
Native subgraphs should make a workflow usable as a workflow step without Native subgraphs should make a workflow usable as a workflow step without
collapsing the child run into one opaque Python node call. The current collapsing the child run into one opaque Python node call. The current
@@ -8,8 +8,9 @@ collapsing the child run into one opaque Python node call. The current
compatibility wrappers, but they hide the child trace, child frames, and child compatibility wrappers, but they hide the child trace, child frames, and child
interrupt lifecycle from `wf_core`. interrupt lifecycle from `wf_core`.
This design defines the core runtime shape. The boundary model is implemented; This design defines the core runtime shape. The boundary model and
child execution, interruption, and saved-workflow resolution remain planned. non-interrupting prepared-child execution are implemented; interruption and
saved-workflow resolution remain planned.
## Goals ## Goals
@@ -76,8 +77,11 @@ declare terminal outcomes through `Workflow.outcomes` and `EndNode`.
native boundary, while artifact helpers convert saved/capability workflow native boundary, while artifact helpers convert saved/capability workflow
references into core `WorkflowRef` values. references into core `WorkflowRef` values.
Runtime execution is deliberately not implemented: stepping a `SubgraphNode` Runtime execution now accepts caller-supplied `PreparedSubgraph` dependencies
fails explicitly until the next slice adds child scope/frame execution. for local workflow refs. It creates a child scope/lineage, schedules child
frames in the parent run, retains child trace entries, and applies mapped
output only at boundary completion. Saved artifact refs are not loaded by
`wf_core`, and child interrupts fail explicitly until resume routing exists.
`WorkflowRef` should be structural, not a dotted string parser: `WorkflowRef` should be structural, not a dotted string parser:
@@ -325,11 +329,11 @@ child = parent.subgraph(
``` ```
This copies the compiled child workflow contract into a core `SubgraphNode`, This copies the compiled child workflow contract into a core `SubgraphNode`,
appends it to the builder, and returns the step for normal routing. It does not appends it to the builder, and returns the step for normal routing. Runtime
make the child executable yet. The core `workflow` field is structural, but execution requires the child graph and its handlers to be supplied as a
higher layers still need dependency resolution before saved/deployed workflow `PreparedSubgraph`; higher layers still need dependency resolution before
refs can run. The lower-level `subgraph_ref(...)` helper exists for code that saved/deployed workflow refs can run. The lower-level `subgraph_ref(...)`
wants only the core step object. helper exists for code that wants only the core step object.
Possible API: Possible API:
@@ -389,18 +393,18 @@ selection out of `wf_core`.
- Artifact conversion helpers bridge saved workflow identities to core - Artifact conversion helpers bridge saved workflow identities to core
`WorkflowRef` values. `WorkflowRef` values.
### Slice 1: Non-Interrupting Inline Subgraph Runtime ### Completed Slice 1: Non-Interrupting Prepared Subgraph Runtime
- Resolve local/prepared child `WorkflowRef` dependencies at runtime; do not - Local/prepared child `WorkflowRef` dependencies resolve through
load saved artifacts inside `wf_core`. `PreparedSubgraph`; `wf_core` does not load saved artifacts.
- Execute child workflow to completion through child frames. - Child workflows execute through child frames in the parent scheduler.
- Give the child an explicit runtime scope/lineage so child state is isolated - Each activation owns a child runtime scope/lineage so child state is isolated
from parent state until boundary completion. from parent state until boundary completion.
- Preserve child trace in a clearly-owned form. - Child trace entries remain in the parent run with child frame ids; completion
- Apply child output to parent state through existing output binding code. records the parent `subgraph` trace entry.
- Route the parent step through the child's terminal `RunState.outcome`. - Child output maps to parent state through existing output binding machinery.
- Tests: child output mapping, child internal trace visibility, parent trace - The parent step routes through the child's terminal workflow outcome.
shape, child runtime failure fails parent. - Child interrupts reject explicitly until structural resume routing exists.
### Slice 2: Interrupt Bubbling and Resume ### Slice 2: Interrupt Bubbling and Resume
@@ -450,12 +454,10 @@ selection out of `wf_core`.
## Recommendation ## Recommendation
The typed boundary scaffold is complete. Start runtime work with Slice 1 as a The typed boundary scaffold and non-interrupting prepared-child runtime are
non-interrupting inline/prepared subgraph. It gives us native trace/frame and complete. Do not delete the wrapper-node helpers yet; use them as compatibility
scope/lineage semantics without taking on the hardest resume problem and examples while native subgraphs mature.
immediately. Do not delete the wrapper-node helpers yet; use them as
compatibility and examples while native subgraphs mature.
Then implement Slice 2 before exposing saved workflows as broadly reusable child Implement Slice 2 before exposing saved workflows as broadly reusable child
graphs. Saved workflows without nested interrupt support would look reusable but graphs. Saved workflows without nested interrupt support would look reusable
break at exactly the moment users need persistence and resume. but break at exactly the moment users need persistence and resume.
+14 -11
View File
@@ -127,30 +127,33 @@ limits and intended adapter seam.
- Foreach supports serial and concurrent execution. Concurrent foreach uses - Foreach supports serial and concurrent execution. Concurrent foreach uses
explicit policy, typed barrier state, lineage-aware patch commits, item error explicit policy, typed barrier state, lineage-aware patch commits, item error
policy, and quiescent interrupt handling. Remaining gaps are higher-level policy, and quiescent interrupt handling. Remaining gaps are higher-level
graph constructs such as native subgraphs, explicit fork/gather nodes, and graph constructs such as explicit fork/gather nodes and advanced conflict
advanced conflict strategies beyond exact-path mergeable reducers. strategies beyond exact-path mergeable reducers.
- Interrupt lifecycle is still node-level and run-state-level. Long-lived - Interrupt lifecycle is still node-level and run-state-level. Long-lived
external subscriptions or notification streams need a separate lifecycle external subscriptions or notification streams need a separate lifecycle
design. Interrupt `request` and `resume` are canonical binding lists; nested design. Interrupt `request` and `resume` are canonical binding lists; nested
child-workflow resume is still future work. child-workflow resume is still future work.
- Native subgraphs have a core model placeholder, `SubgraphNode`, but runtime - Native subgraphs use `SubgraphNode` plus caller-supplied `PreparedSubgraph`
execution is not implemented yet. The placeholder carries a child workflow dependencies. A prepared local child executes through a child runtime scope
reference, declared input/output schemas, binding lists, and declared and lineage; child output commits only through declared boundary bindings and
outcomes so parent graph structure can validate before execution support the parent routes by the child's terminal workflow outcome. Saved/deployed
lands. workflow resolution remains outside core and is not implemented at this
boundary yet.
- Nested subgraph interruption is not first-class yet. The current - Nested subgraph interruption is not first-class yet. The current
`wf_authoring` subgraph helpers wrap a child workflow as an ordinary sync or `wf_authoring` subgraph helpers wrap a child workflow as an ordinary sync or
async node and validate the child output; they do not preserve a child run async node and validate the child output; they do not preserve a child run
state that can interrupt, bubble to the parent, and later resume inside the state that can interrupt, bubble to the parent, and later resume inside the
child. See `examples/authoring_workflow_as_node.py` for the current child. Native `SubgraphNode` preserves prepared-child execution and trace,
wrapper-node shape. but rejects child interrupts until route-aware resume is implemented. See
`examples/authoring_workflow_as_node.py` for the wrapper-node shape.
- Saved workflow-as-node execution with interrupts requires a core runtime - Saved workflow-as-node execution with interrupts requires a core runtime
upgrade: nested run state, child-frame trace preservation, interrupt bubbling upgrade: nested run state, child-frame trace preservation, interrupt bubbling
with path metadata, and resume back into the child workflow. with path metadata, and resume back into the child workflow.
- Frames are no longer only a serial execution stack: the runtime has a ready - Frames are no longer only a serial execution stack: the runtime has a ready
queue, `BLOCKED` frame state, lineage isolation, barrier merge semantics, and queue, `BLOCKED` frame state, lineage isolation, barrier merge semantics, and
pending child results for concurrent foreach. Native subgraphs still need pending child results for concurrent foreach. Native prepared subgraphs now
explicit child workflow/deployment identity and child-scope execution. use child-scope execution; saved/deployed child resolution and nested
interruption remain outstanding.
Concurrent foreach is the primary current use case for async concurrent node Concurrent foreach is the primary current use case for async concurrent node
handler execution. handler execution.
- Runtime errors are still ordinary exceptions plus failed run status. A richer - Runtime errors are still ordinary exceptions plus failed run status. A richer
+2
View File
@@ -24,6 +24,7 @@ from .models import (
from .runtime import ( from .runtime import (
AsyncNodeHandler, AsyncNodeHandler,
NodeHandler, NodeHandler,
PreparedSubgraph,
WorkflowExecutionError, WorkflowExecutionError,
coerce_node_result, coerce_node_result,
execute_workflow_async, execute_workflow_async,
@@ -72,6 +73,7 @@ __all__ = [
"SubgraphNode", "SubgraphNode",
"AsyncNodeHandler", "AsyncNodeHandler",
"NodeHandler", "NodeHandler",
"PreparedSubgraph",
"ExecutionFrame", "ExecutionFrame",
"FrameStatus", "FrameStatus",
"RunState", "RunState",
+6 -7
View File
@@ -158,12 +158,11 @@ class NodeUse(BaseModel):
class SubgraphNode(BaseModel): class SubgraphNode(BaseModel):
"""Workflow boundary step reserved for native subgraph execution. """Workflow boundary step for native prepared-child execution.
This is a contract-bearing placeholder, not the implementation of nested The runtime can execute an already-prepared local child graph through a
workflow execution yet. The core can validate the parent graph's bindings child scope/lineage and commit only its mapped boundary output. Resolving
and declared outcomes now; a later runtime slice will resolve ``workflow`` saved artifacts and resuming child interrupts remain platform/runtime work.
into a child graph, create a child scope/lineage, and commit its result.
""" """
id: str id: str
@@ -171,8 +170,8 @@ class SubgraphNode(BaseModel):
workflow: WorkflowRef = Field( workflow: WorkflowRef = Field(
description=( description=(
"Reference to the child workflow artifact or registry key. The core " "Reference to the child workflow artifact or registry key. The core "
"does not resolve this reference until native subgraph runtime " "executes local references only when a PreparedSubgraph dependency "
"execution is implemented." "is supplied; it does not load saved artifacts."
) )
) )
desc: str | None = None desc: str | None = None
+3
View File
@@ -5,6 +5,7 @@ from enum import StrEnum
from typing import Any from typing import Any
from wf_core.models.reducers import ReducerRef from wf_core.models.reducers import ReducerRef
from wf_core.models.workflow_refs import WorkflowRef
from wf_core.paths import StatePath from wf_core.paths import StatePath
ROOT_SCOPE_ID = "root" ROOT_SCOPE_ID = "root"
@@ -54,7 +55,9 @@ class RuntimeScope:
id: str id: str
workflow_name: str workflow_name: str
workflow_input: dict[str, Any] = field(default_factory=dict)
committed_state: dict[str, Any] = field(default_factory=dict) committed_state: dict[str, Any] = field(default_factory=dict)
workflow_ref: WorkflowRef | None = None
@dataclass(slots=True) @dataclass(slots=True)
+2
View File
@@ -13,6 +13,7 @@ from .engine import (
resume_workflow, resume_workflow,
resume_workflow_async, resume_workflow_async,
) )
from .subgraphs import PreparedSubgraph
from .step import complete_step, step_workflow, step_workflow_async from .step import complete_step, step_workflow, step_workflow_async
__all__ = [ __all__ = [
@@ -29,4 +30,5 @@ __all__ = [
"resume_workflow_async", "resume_workflow_async",
"step_workflow", "step_workflow",
"step_workflow_async", "step_workflow_async",
"PreparedSubgraph",
] ]
+85 -13
View File
@@ -3,17 +3,19 @@ from __future__ import annotations
from collections.abc import Mapping from collections.abc import Mapping
from typing import Any from typing import Any
from wf_core.errors import WorkflowExecutionError
from wf_core.models.workflow import Workflow from wf_core.models.workflow import Workflow
from wf_core.runtime.ops.flow import finalize_run from wf_core.runtime.ops.flow import finalize_run
from wf_core.runtime.ops.merges import ReducerDefinition from wf_core.runtime.ops.merges import ReducerDefinition
from wf_core.runtime.ops.nodes import AsyncNodeHandler, NodeHandler from wf_core.runtime.ops.nodes import AsyncNodeHandler, NodeHandler
from wf_core.runtime.ops.runs import create_run_state from wf_core.runtime.ops.runs import create_run_state
from wf_core.runtime.scheduler import resolve_no_ready_frames, select_next_frame from wf_core.runtime.scheduler import resolve_no_ready_frames, select_next_frame
from wf_core.run_state import RunState, RunStatus from wf_core.run_state import ROOT_SCOPE_ID, RunState, RunStatus
from wf_core.tokens import END from wf_core.tokens import END
from .preparation import prepare_new_run, prepare_resume from .preparation import prepare_new_run, prepare_resume
from .step import step_workflow, step_workflow_async from .step import step_workflow, step_workflow_async
from .subgraphs import PreparedSubgraph, resolve_prepared_subgraph
def execute_workflow( def execute_workflow(
@@ -22,13 +24,20 @@ def execute_workflow(
registry: Mapping[str, NodeHandler], registry: Mapping[str, NodeHandler],
*, *,
reducers: Mapping[str, ReducerDefinition] | None = None, reducers: Mapping[str, ReducerDefinition] | None = None,
subgraphs: Mapping[str, PreparedSubgraph[NodeHandler]] | None = None,
) -> RunState: ) -> RunState:
"""Create a run and execute a workflow synchronously until it stops.""" """Create a run and execute a workflow synchronously until it stops."""
run = create_run_state(workflow, workflow_input) run = create_run_state(workflow, workflow_input)
try: try:
prepare_new_run(workflow, workflow_input, run) prepare_new_run(workflow, workflow_input, run)
return resume_workflow(workflow, run, registry, reducers=reducers) return resume_workflow(
workflow,
run,
registry,
reducers=reducers,
subgraphs=subgraphs,
)
except Exception as exc: except Exception as exc:
run.status = RunStatus.FAILED run.status = RunStatus.FAILED
run.error = str(exc) run.error = str(exc)
@@ -41,13 +50,20 @@ async def execute_workflow_async(
registry: Mapping[str, AsyncNodeHandler], registry: Mapping[str, AsyncNodeHandler],
*, *,
reducers: Mapping[str, ReducerDefinition] | None = None, reducers: Mapping[str, ReducerDefinition] | None = None,
subgraphs: Mapping[str, PreparedSubgraph[AsyncNodeHandler]] | None = None,
) -> RunState: ) -> RunState:
"""Create a run and execute a workflow asynchronously until it stops.""" """Create a run and execute a workflow asynchronously until it stops."""
run = create_run_state(workflow, workflow_input) run = create_run_state(workflow, workflow_input)
try: try:
prepare_new_run(workflow, workflow_input, run) prepare_new_run(workflow, workflow_input, run)
return await resume_workflow_async(workflow, run, registry, reducers=reducers) return await resume_workflow_async(
workflow,
run,
registry,
reducers=reducers,
subgraphs=subgraphs,
)
except Exception as exc: except Exception as exc:
run.status = RunStatus.FAILED run.status = RunStatus.FAILED
run.error = str(exc) run.error = str(exc)
@@ -62,6 +78,7 @@ def resume_workflow(
resume_payload: dict[str, Any] | None = None, resume_payload: dict[str, Any] | None = None,
resume_outcome: str = "submitted", resume_outcome: str = "submitted",
reducers: Mapping[str, ReducerDefinition] | None = None, reducers: Mapping[str, ReducerDefinition] | None = None,
subgraphs: Mapping[str, PreparedSubgraph[NodeHandler]] | None = None,
) -> RunState: ) -> RunState:
"""Resume a synchronous run from its current state.""" """Resume a synchronous run from its current state."""
index = prepare_resume( index = prepare_resume(
@@ -77,17 +94,22 @@ def resume_workflow(
return run return run
while True: while True:
if select_next_frame(run) is None: frame = select_next_frame(run)
if frame is None:
status = resolve_no_ready_frames(run) status = resolve_no_ready_frames(run)
if status == RunStatus.COMPLETED: if status == RunStatus.COMPLETED:
break break
return run return run
active_workflow, active_registry, active_reducers = _sync_execution_target(
workflow, registry, reducers, run, subgraphs
)
step_workflow( step_workflow(
workflow, active_workflow,
run, run,
registry, active_registry,
index=index, index=index if frame.scope_id == ROOT_SCOPE_ID else None,
reducers=reducers, reducers=active_reducers,
subgraphs=subgraphs,
) )
if run.status == RunStatus.INTERRUPTED: if run.status == RunStatus.INTERRUPTED:
return run return run
@@ -103,6 +125,7 @@ async def resume_workflow_async(
resume_payload: dict[str, Any] | None = None, resume_payload: dict[str, Any] | None = None,
resume_outcome: str = "submitted", resume_outcome: str = "submitted",
reducers: Mapping[str, ReducerDefinition] | None = None, reducers: Mapping[str, ReducerDefinition] | None = None,
subgraphs: Mapping[str, PreparedSubgraph[AsyncNodeHandler]] | None = None,
) -> RunState: ) -> RunState:
"""Resume an async run from its current state.""" """Resume an async run from its current state."""
index = prepare_resume( index = prepare_resume(
@@ -118,19 +141,68 @@ async def resume_workflow_async(
return run return run
while True: while True:
if select_next_frame(run) is None: frame = select_next_frame(run)
if frame is None:
status = resolve_no_ready_frames(run) status = resolve_no_ready_frames(run)
if status == RunStatus.COMPLETED: if status == RunStatus.COMPLETED:
break break
return run return run
active_workflow, active_registry, active_reducers = _async_execution_target(
workflow, registry, reducers, run, subgraphs
)
await step_workflow_async( await step_workflow_async(
workflow, active_workflow,
run, run,
registry, active_registry,
index=index, index=index if frame.scope_id == ROOT_SCOPE_ID else None,
reducers=reducers, reducers=active_reducers,
subgraphs=subgraphs,
) )
if run.status == RunStatus.INTERRUPTED: if run.status == RunStatus.INTERRUPTED:
return run return run
return finalize_run(workflow, run) return finalize_run(workflow, run)
def _sync_execution_target(
root_workflow: Workflow,
root_registry: Mapping[str, NodeHandler],
root_reducers: Mapping[str, ReducerDefinition] | None,
run: RunState,
subgraphs: Mapping[str, PreparedSubgraph[NodeHandler]] | None,
) -> tuple[Workflow, Mapping[str, NodeHandler], Mapping[str, ReducerDefinition] | None]:
"""Return the workflow dependencies owned by the selected frame scope."""
frame = run.current_frame()
if frame.scope_id == ROOT_SCOPE_ID:
return root_workflow, root_registry, root_reducers
scope = run.scopes.get(frame.scope_id)
if scope is None or scope.workflow_ref is None:
raise WorkflowExecutionError(
f"child frame {frame.id!r} has no prepared workflow scope"
)
child = resolve_prepared_subgraph(scope.workflow_ref, subgraphs)
return child.workflow, child.registry, child.reducers
def _async_execution_target(
root_workflow: Workflow,
root_registry: Mapping[str, AsyncNodeHandler],
root_reducers: Mapping[str, ReducerDefinition] | None,
run: RunState,
subgraphs: Mapping[str, PreparedSubgraph[AsyncNodeHandler]] | None,
) -> tuple[
Workflow,
Mapping[str, AsyncNodeHandler],
Mapping[str, ReducerDefinition] | None,
]:
"""Return async workflow dependencies owned by the selected frame scope."""
frame = run.current_frame()
if frame.scope_id == ROOT_SCOPE_ID:
return root_workflow, root_registry, root_reducers
scope = run.scopes.get(frame.scope_id)
if scope is None or scope.workflow_ref is None:
raise WorkflowExecutionError(
f"child frame {frame.id!r} has no prepared workflow scope"
)
child = resolve_prepared_subgraph(scope.workflow_ref, subgraphs)
return child.workflow, child.registry, child.reducers
+35 -8
View File
@@ -7,10 +7,9 @@ from typing import Any
from wf_core.errors import WorkflowExecutionError from wf_core.errors import WorkflowExecutionError
from wf_core.run_state import ExecutionFrame, LineageState, RunState, StateWrite from wf_core.run_state import ExecutionFrame, LineageState, RunState, StateWrite
from wf_core.run_state import ROOT_LINEAGE_ID, ROOT_SCOPE_ID
from wf_core.runtime.foreach_state import ForeachBarrierState, item_frame_owner from wf_core.runtime.foreach_state import ForeachBarrierState, item_frame_owner
from wf_core.runtime.ops.state import StatePatch from wf_core.runtime.ops.state import StatePatch
from wf_core.runtime.ops.state import safe_set_nested_value from wf_core.runtime.ops.state import commit_state_patch, safe_set_nested_value
@dataclass(slots=True) @dataclass(slots=True)
@@ -79,14 +78,34 @@ def lineage_writes_for_frame(
return pending.patch.writes return pending.patch.writes
def is_root_lineage_frame(frame: ExecutionFrame) -> bool: def is_scope_root_lineage_frame(run: RunState, frame: ExecutionFrame) -> bool:
"""Return whether a frame currently commits directly to root run state. """Return whether writes from this frame commit to its scope state root."""
lineage = run.lineages.get(frame.lineage_id)
return (
lineage is not None
and lineage.scope_id == frame.scope_id
and lineage.parent_id is None
)
This is a migration shortcut, not the final commit policy. Once native
subgraphs can complete, direct commits should be decided by an explicit def commit_patch_for_frame(
scope/lineage commit target rather than only by root ids. run: RunState, frame: ExecutionFrame, patch: StatePatch
) -> dict[str, Any]:
"""Commit at a scope root or buffer writes in the frame lineage.
Child workflow root frames own a committed child-state root just like the
top-level root frame owns `RunState.state`. Descendant branch/item frames
remain isolated until an explicit barrier or future gather commits them.
""" """
return frame.scope_id == ROOT_SCOPE_ID and frame.lineage_id == ROOT_LINEAGE_ID if is_scope_root_lineage_frame(run, frame):
return commit_state_patch(scope_state_for_frame(run, frame), patch)
append_lineage_writes(
run,
scope_id=frame.scope_id,
lineage_id=frame.lineage_id,
writes=patch.writes,
)
return {}
def scope_state_for_frame(run: RunState, frame: ExecutionFrame) -> dict[str, Any]: def scope_state_for_frame(run: RunState, frame: ExecutionFrame) -> dict[str, Any]:
@@ -97,6 +116,14 @@ def scope_state_for_frame(run: RunState, frame: ExecutionFrame) -> dict[str, Any
return scope.committed_state return scope.committed_state
def scope_input_for_frame(run: RunState, frame: ExecutionFrame) -> dict[str, Any]:
"""Return the invocation input associated with the frame's workflow scope."""
scope = run.scopes.get(frame.scope_id)
if scope is None:
raise ValueError(f"unknown scope {frame.scope_id!r}")
return scope.workflow_input
def add_lineage( def add_lineage(
run: RunState, run: RunState,
*, *,
+4
View File
@@ -80,6 +80,10 @@ def advance_frame(
frame.activated_incoming_edge = frame.node_id frame.activated_incoming_edge = frame.node_id
frame.node_id = next_node_id frame.node_id = next_node_id
if next_node_id == END: if next_node_id == END:
if frame.kind in {"workflow", "subgraph_root"}:
# Legacy terminal routing emits the workflow-level `ok` outcome.
# Explicit EndNode execution stores its declared outcome first.
frame.metadata.setdefault("workflow_outcome", "ok")
frame.status = FrameStatus.COMPLETED frame.status = FrameStatus.COMPLETED
frame.finished_at_node_id = END frame.finished_at_node_id = END
wake_parent_for_child_progress(run, frame.id) wake_parent_for_child_progress(run, frame.id)
+7 -16
View File
@@ -14,18 +14,18 @@ from wf_core.runtime.foreach_state import (
) )
from wf_core.runtime.lineage import ( from wf_core.runtime.lineage import (
add_lineage, add_lineage,
append_lineage_writes, commit_patch_for_frame,
is_root_lineage_frame,
lineage_patch, lineage_patch,
scope_input_for_frame,
) )
from wf_core.runtime.ops.flow import advance_frame, append_step_result_trace from wf_core.runtime.ops.flow import advance_frame, append_step_result_trace
from wf_core.runtime.ops.frames import frame_context_values from wf_core.runtime.ops.frames import frame_context_values
from wf_core.runtime.ops.index import WorkflowIndex from wf_core.runtime.ops.index import WorkflowIndex
from wf_core.runtime.ops.merges import ReducerDefinition from wf_core.runtime.ops.merges import ReducerDefinition
from wf_core.runtime.ops.overlays import state_view_for_frame
from wf_core.runtime.ops.state import ( from wf_core.runtime.ops.state import (
StatePatch, StatePatch,
build_barrier_patch, build_barrier_patch,
commit_state_patch,
) )
from wf_core.runtime.scheduler import ( from wf_core.runtime.scheduler import (
ForeachIterationMetadata, ForeachIterationMetadata,
@@ -182,8 +182,8 @@ def _resolve_foreach_iterable(
) -> list[object]: ) -> list[object]:
iterable = safe_resolve_path( iterable = safe_resolve_path(
str(step.over), str(step.over),
state=run.state, state=state_view_for_frame(run, frame),
workflow_input=run.workflow_input, workflow_input=scope_input_for_frame(run, frame),
context=frame_context_values(frame), context=frame_context_values(frame),
) )
if not isinstance(iterable, list): if not isinstance(iterable, list):
@@ -346,19 +346,10 @@ def _finish_concurrent_foreach(
combined = build_barrier_patch( combined = build_barrier_patch(
workflow, workflow,
item_patches, item_patches,
run.state, state_view_for_frame(run, frame),
reducers=reducers, reducers=reducers,
) )
if is_root_lineage_frame(frame): state_changes = commit_patch_for_frame(run, frame, combined)
state_changes = commit_state_patch(run.state, combined)
else:
append_lineage_writes(
run,
scope_id=frame.scope_id,
lineage_id=frame.lineage_id,
writes=combined.writes,
)
state_changes = {}
append_step_result_trace( append_step_result_trace(
run, run,
frame_id=frame.id, frame_id=frame.id,
+4 -2
View File
@@ -3,9 +3,11 @@ from __future__ import annotations
from wf_core.conditions import eval_condition from wf_core.conditions import eval_condition
from wf_core.models.steps import ConditionNode, InterruptNode from wf_core.models.steps import ConditionNode, InterruptNode
from wf_core.run_state import FrameStatus, RunState, RunStatus, StepExecutionResult from wf_core.run_state import FrameStatus, RunState, RunStatus, StepExecutionResult
from wf_core.runtime.lineage import scope_input_for_frame
from wf_core.runtime.ops.flow import append_trace from wf_core.runtime.ops.flow import append_trace
from wf_core.runtime.ops.frames import frame_context_values from wf_core.runtime.ops.frames import frame_context_values
from wf_core.runtime.ops.interrupts import build_interrupt_request from wf_core.runtime.ops.interrupts import build_interrupt_request
from wf_core.runtime.ops.overlays import state_view_for_frame
def handle_condition_step( def handle_condition_step(
@@ -15,8 +17,8 @@ def handle_condition_step(
frame = run.current_frame() frame = run.current_frame()
predicate = eval_condition( predicate = eval_condition(
step.check, step.check,
run.state, state_view_for_frame(run, frame),
run.workflow_input, scope_input_for_frame(run, frame),
frame.prior_outcome, frame.prior_outcome,
) )
outcome = "true" if predicate else "false" outcome = "true" if predicate else "false"
+9 -16
View File
@@ -18,12 +18,16 @@ from wf_core.run_state import (
StepExecutionResult, StepExecutionResult,
) )
from wf_core.runtime.foreach_state import ForeachBarrierState, item_frame_owner from wf_core.runtime.foreach_state import ForeachBarrierState, item_frame_owner
from wf_core.runtime.lineage import append_lineage_writes, is_root_lineage_frame from wf_core.runtime.lineage import (
append_lineage_writes,
commit_patch_for_frame,
scope_input_for_frame,
)
from wf_core.runtime.ops.frames import frame_context_values from wf_core.runtime.ops.frames import frame_context_values
from wf_core.runtime.ops.merges import ReducerDefinition from wf_core.runtime.ops.merges import ReducerDefinition
from wf_core.runtime.ops.overlays import state_view_for_frame from wf_core.runtime.ops.overlays import state_view_for_frame
from wf_core.runtime.ops.schemas import validate_payload_against_schema from wf_core.runtime.ops.schemas import validate_payload_against_schema
from wf_core.runtime.ops.state import StatePatch, build_output_patch, commit_state_patch from wf_core.runtime.ops.state import StatePatch, build_output_patch
NodeHandler = Callable[[dict[str, Any], RuntimeContext], NodeResult | dict[str, Any]] NodeHandler = Callable[[dict[str, Any], RuntimeContext], NodeResult | dict[str, Any]]
AsyncNodeHandler = Callable[ AsyncNodeHandler = Callable[
@@ -62,7 +66,7 @@ def _resolve_node_execution(
value = safe_resolve_path( value = safe_resolve_path(
str(binding.path), str(binding.path),
state=state_view, state=state_view,
workflow_input=run.workflow_input, workflow_input=scope_input_for_frame(run, frame),
context=context_values, context=context_values,
) )
else: else:
@@ -121,18 +125,7 @@ def _finalize_node_execution(
) )
owner = item_frame_owner(frame) owner = item_frame_owner(frame)
if owner is None: if owner is None:
if is_root_lineage_frame(frame): state_changes = commit_patch_for_frame(run, frame, patch)
state_changes = commit_state_patch(run.state, patch)
else:
# Non-root frames are future subgraph/fork branch execution: writes
# become lineage-local until an explicit boundary/barrier commits.
append_lineage_writes(
run,
scope_id=frame.scope_id,
lineage_id=frame.lineage_id,
writes=patch.writes,
)
state_changes = {}
else: else:
parent_frame_id, foreach_node_id, item_index = owner parent_frame_id, foreach_node_id, item_index = owner
parent_frame = run.frames[parent_frame_id] parent_frame = run.frames[parent_frame_id]
@@ -155,7 +148,7 @@ def _finalize_node_execution(
barrier.save_to_frame(parent_frame, foreach_node_id) barrier.save_to_frame(parent_frame, foreach_node_id)
state_changes = {} state_changes = {}
else: else:
state_changes = commit_state_patch(run.state, patch) state_changes = commit_patch_for_frame(run, parent_frame, patch)
return StepExecutionResult( return StepExecutionResult(
outcome=result.outcome, outcome=result.outcome,
resolved_input=resolved_input, resolved_input=resolved_input,
+10 -1
View File
@@ -18,12 +18,20 @@ from wf_core.run_state import (
from wf_core.runtime.scheduler import add_frame from wf_core.runtime.scheduler import add_frame
def create_run_state(workflow: Workflow, workflow_input: dict[str, object]) -> RunState: def initial_state(
workflow: Workflow, workflow_input: dict[str, object]
) -> dict[str, object]:
"""Create one scope's committed state from defaults plus workflow input."""
state: dict[str, object] = {} state: dict[str, object] = {}
for field in workflow.state_schema.fields: for field in workflow.state_schema.fields:
if field.default is not None: if field.default is not None:
set_nested_value(state, list(field.path.parts), deepcopy(field.default)) set_nested_value(state, list(field.path.parts), deepcopy(field.default))
state.update(dict(workflow_input)) state.update(dict(workflow_input))
return state
def create_run_state(workflow: Workflow, workflow_input: dict[str, object]) -> RunState:
state = initial_state(workflow, workflow_input)
run = RunState( run = RunState(
workflow_name=workflow.name, workflow_name=workflow.name,
status=RunStatus.PENDING, status=RunStatus.PENDING,
@@ -33,6 +41,7 @@ def create_run_state(workflow: Workflow, workflow_input: dict[str, object]) -> R
ROOT_SCOPE_ID: RuntimeScope( ROOT_SCOPE_ID: RuntimeScope(
id=ROOT_SCOPE_ID, id=ROOT_SCOPE_ID,
workflow_name=workflow.name, workflow_name=workflow.name,
workflow_input=dict(workflow_input),
committed_state=state, committed_state=state,
) )
}, },
+32 -7
View File
@@ -38,7 +38,9 @@ from wf_core.runtime.scheduler import (
select_next_frame, select_next_frame,
wake_parent_for_child_progress, wake_parent_for_child_progress,
) )
from wf_core.runtime.subgraphs import PreparedSubgraph, step_subgraph
from wf_core.run_state import ExecutionFrame, FrameStatus, RunState, StepExecutionResult from wf_core.run_state import ExecutionFrame, FrameStatus, RunState, StepExecutionResult
from wf_core.run_state import ROOT_SCOPE_ID
from wf_core.tokens import END from wf_core.tokens import END
from .preparation import prepare_step from .preparation import prepare_step
@@ -85,6 +87,9 @@ def complete_end_step(
) -> RunState: ) -> RunState:
"""Record an explicit workflow terminal and complete the active frame.""" """Record an explicit workflow terminal and complete the active frame."""
result = StepExecutionResult(outcome=outcome) result = StepExecutionResult(outcome=outcome)
frame = run.frames[frame_id]
frame.metadata["workflow_outcome"] = outcome
if frame.parent_frame_id is None:
run.outcome = outcome run.outcome = outcome
append_step_result_trace( append_step_result_trace(
run, run,
@@ -96,7 +101,7 @@ def complete_end_step(
) )
advance_frame( advance_frame(
run, run,
run.frames[frame_id], frame,
outcome=outcome, outcome=outcome,
next_node_id=END, next_node_id=END,
) )
@@ -110,6 +115,7 @@ def step_workflow(
*, *,
index: WorkflowIndex | None = None, index: WorkflowIndex | None = None,
reducers: Mapping[str, ReducerDefinition] | None = None, reducers: Mapping[str, ReducerDefinition] | None = None,
subgraphs: Mapping[str, PreparedSubgraph[NodeHandler]] | None = None,
) -> RunState: ) -> RunState:
"""Execute at most one synchronous workflow step.""" """Execute at most one synchronous workflow step."""
frame = run.current_frame() if run.current_frame_id is not None else None frame = run.current_frame() if run.current_frame_id is not None else None
@@ -149,14 +155,23 @@ def step_workflow(
outcome=step.outcome, outcome=step.outcome,
) )
elif isinstance(step, InterruptNode): elif isinstance(step, InterruptNode):
if frame.kind == "subgraph_root" or frame.scope_id != ROOT_SCOPE_ID:
raise WorkflowExecutionError(
"child interrupts are not supported until native subgraph resume routing exists"
)
return handle_interrupt_step(run, step) return handle_interrupt_step(run, step)
elif isinstance(step, ForeachNode): elif isinstance(step, ForeachNode):
return step_foreach(workflow, run, step, index, reducers=reducers) return step_foreach(workflow, run, step, index, reducers=reducers)
elif isinstance(step, SubgraphNode): elif isinstance(step, SubgraphNode):
raise WorkflowExecutionError( step_result = step_subgraph(
f"subgraph step {step.id!r} references {step.workflow!r}, " workflow,
"but native subgraph execution is not implemented yet" run,
step,
subgraphs=subgraphs,
reducers=reducers,
) )
if step_result is None:
return run
else: else:
raise WorkflowExecutionError( raise WorkflowExecutionError(
f"unsupported step type {getattr(step, 'type', type(step).__name__)!r}" f"unsupported step type {getattr(step, 'type', type(step).__name__)!r}"
@@ -205,6 +220,7 @@ async def step_workflow_async(
*, *,
index: WorkflowIndex | None = None, index: WorkflowIndex | None = None,
reducers: Mapping[str, ReducerDefinition] | None = None, reducers: Mapping[str, ReducerDefinition] | None = None,
subgraphs: Mapping[str, PreparedSubgraph[AsyncNodeHandler]] | None = None,
) -> RunState: ) -> RunState:
"""Execute at most one async workflow step.""" """Execute at most one async workflow step."""
frame = run.current_frame() if run.current_frame_id is not None else None frame = run.current_frame() if run.current_frame_id is not None else None
@@ -255,14 +271,23 @@ async def step_workflow_async(
outcome=step.outcome, outcome=step.outcome,
) )
elif isinstance(step, InterruptNode): elif isinstance(step, InterruptNode):
if frame.kind == "subgraph_root" or frame.scope_id != ROOT_SCOPE_ID:
raise WorkflowExecutionError(
"child interrupts are not supported until native subgraph resume routing exists"
)
return handle_interrupt_step(run, step) return handle_interrupt_step(run, step)
elif isinstance(step, ForeachNode): elif isinstance(step, ForeachNode):
return step_foreach(workflow, run, step, index, reducers=reducers) return step_foreach(workflow, run, step, index, reducers=reducers)
elif isinstance(step, SubgraphNode): elif isinstance(step, SubgraphNode):
raise WorkflowExecutionError( step_result = step_subgraph(
f"subgraph step {step.id!r} references {step.workflow!r}, " workflow,
"but native subgraph execution is not implemented yet" run,
step,
subgraphs=subgraphs,
reducers=reducers,
) )
if step_result is None:
return run
else: else:
raise WorkflowExecutionError( raise WorkflowExecutionError(
f"unsupported step type {getattr(step, 'type', type(step).__name__)!r}" f"unsupported step type {getattr(step, 'type', type(step).__name__)!r}"
+273
View File
@@ -0,0 +1,273 @@
from __future__ import annotations
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass
from typing import Any, Generic, TypeVar
from wf_core.conditions import safe_resolve_path
from wf_core.errors import WorkflowExecutionError
from wf_core.local_paths import LocalPathError, set_local_value
from wf_core.models.steps import (
InputBinding,
InputPathBinding,
InputValueBinding,
SubgraphNode,
)
from wf_core.models.workflow import Workflow
from wf_core.models.workflow_refs import WorkflowRef
from wf_core.run_state import (
ExecutionFrame,
FrameStatus,
LineageState,
RunState,
RuntimeScope,
StepExecutionResult,
)
from wf_core.runtime.lineage import commit_patch_for_frame
from wf_core.runtime.ops.frames import frame_context_values
from wf_core.runtime.ops.merges import ReducerDefinition
from wf_core.runtime.ops.overlays import state_view_for_frame
from wf_core.runtime.ops.runs import initial_state
from wf_core.runtime.ops.schemas import validate_payload_against_schema
from wf_core.runtime.ops.state import build_output_patch, project_output
from wf_core.runtime.scheduler import add_frame, block_frame_on_children
HandlerT = TypeVar("HandlerT", bound=Callable[..., object])
_ACTIVATION_KEY = "subgraph_activation"
@dataclass(slots=True, frozen=True)
class PreparedSubgraph(Generic[HandlerT]):
"""Executable local child dependency supplied by the caller.
Core owns child execution semantics but does not load artifacts or resolve
deployment/source bindings. Higher layers must resolve those concerns into
this prepared dependency before a run starts.
"""
workflow: Workflow
registry: Mapping[str, HandlerT]
reducers: Mapping[str, ReducerDefinition] | None = None
@dataclass(slots=True, frozen=True)
class SubgraphActivation:
"""Runtime ownership record for one in-flight subgraph boundary."""
workflow_ref: WorkflowRef
scope_id: str
lineage_id: str
child_frame_id: str
child_input: dict[str, Any]
@classmethod
def from_frame(cls, frame: ExecutionFrame) -> SubgraphActivation | None:
raw = frame.metadata.get(_ACTIVATION_KEY)
if raw is None:
return None
if not isinstance(raw, Mapping):
raise WorkflowExecutionError(
f"malformed subgraph activation for frame {frame.id!r}"
)
try:
return cls(
workflow_ref=WorkflowRef.model_validate(raw["workflow_ref"]),
scope_id=str(raw["scope_id"]),
lineage_id=str(raw["lineage_id"]),
child_frame_id=str(raw["child_frame_id"]),
child_input=dict(raw["child_input"]),
)
except (KeyError, TypeError, ValueError) as exc:
raise WorkflowExecutionError(
f"malformed subgraph activation for frame {frame.id!r}"
) from exc
def save_to_frame(self, frame: ExecutionFrame) -> None:
frame.metadata[_ACTIVATION_KEY] = {
"workflow_ref": self.workflow_ref.model_dump(mode="json"),
"scope_id": self.scope_id,
"lineage_id": self.lineage_id,
"child_frame_id": self.child_frame_id,
"child_input": dict(self.child_input),
}
def resolve_prepared_subgraph(
ref: WorkflowRef,
subgraphs: Mapping[str, PreparedSubgraph[HandlerT]] | None,
) -> PreparedSubgraph[HandlerT]:
"""Resolve a prepared local child; artifact loading is not a core concern."""
if ref.name is None:
raise WorkflowExecutionError(
f"saved child workflow reference {ref.display!r} is not prepared for core execution"
)
prepared = None if subgraphs is None else subgraphs.get(ref.name)
if prepared is None:
raise WorkflowExecutionError(
f"no prepared child workflow registered for {ref.name!r}"
)
return prepared
def resolve_input_bindings(
bindings: Sequence[InputBinding],
*,
state: Mapping[str, Any],
workflow_input: Mapping[str, Any],
context: Mapping[str, Any],
label: str,
) -> dict[str, Any]:
"""Build a local input payload from canonical value/path bindings."""
payload: dict[str, Any] = {}
for binding in bindings:
if isinstance(binding, InputValueBinding):
value = binding.value
elif isinstance(binding, InputPathBinding):
value = safe_resolve_path(
str(binding.path),
state=state,
workflow_input=workflow_input,
context=context,
)
else:
raise WorkflowExecutionError(f"unsupported input binding for {label}")
try:
set_local_value(payload, binding.target, value)
except LocalPathError as exc:
raise WorkflowExecutionError(str(exc)) from exc
return payload
def step_subgraph(
workflow: Workflow,
run: RunState,
step: SubgraphNode,
*,
subgraphs: Mapping[str, PreparedSubgraph[HandlerT]] | None,
reducers: Mapping[str, ReducerDefinition] | None,
) -> StepExecutionResult | None:
"""Start or finish one native child activation.
Returning ``None`` means the parent frame is blocked while child frames run.
Returning a result means child execution completed and the parent boundary
can advance normally through the child's terminal workflow outcome.
"""
frame = run.current_frame()
activation = SubgraphActivation.from_frame(frame)
prepared = resolve_prepared_subgraph(step.workflow, subgraphs)
if activation is None:
_start_subgraph(run, frame, step, prepared)
return None
return _finish_subgraph(workflow, run, frame, step, activation, prepared, reducers)
def _start_subgraph(
run: RunState,
frame: ExecutionFrame,
step: SubgraphNode,
prepared: PreparedSubgraph[HandlerT],
) -> None:
prepared.workflow.validate_structure().raise_for_errors()
parent_scope = run.scopes[frame.scope_id]
child_input = resolve_input_bindings(
step.input,
state=state_view_for_frame(run, frame),
workflow_input=parent_scope.workflow_input,
context=frame_context_values(frame),
label=f"subgraph {step.id!r}",
)
validate_payload_against_schema(
step.input_schema, child_input, f"subgraph input for {step.id}"
)
validate_payload_against_schema(
prepared.workflow.input_schema,
child_input,
f"child workflow input for {step.id}",
)
scope_id = f"{frame.id}:subgraph:{step.id}"
lineage_id = f"{scope_id}:root"
child_frame_id = f"{scope_id}:frame"
if scope_id in run.scopes or lineage_id in run.lineages:
raise WorkflowExecutionError(
f"duplicate subgraph activation identifiers for step {step.id!r}"
)
run.scopes[scope_id] = RuntimeScope(
id=scope_id,
workflow_name=prepared.workflow.name,
workflow_input=dict(child_input),
committed_state=initial_state(prepared.workflow, child_input),
workflow_ref=step.workflow,
)
run.lineages[lineage_id] = LineageState(id=lineage_id, scope_id=scope_id)
add_frame(
run,
ExecutionFrame(
id=child_frame_id,
kind="subgraph_root",
node_id=prepared.workflow.start,
status=FrameStatus.PENDING,
parent_frame_id=frame.id,
scope_id=scope_id,
lineage_id=lineage_id,
),
ready=True,
)
SubgraphActivation(
workflow_ref=step.workflow,
scope_id=scope_id,
lineage_id=lineage_id,
child_frame_id=child_frame_id,
child_input=child_input,
).save_to_frame(frame)
block_frame_on_children(run, frame.id, (child_frame_id,))
def _finish_subgraph(
workflow: Workflow,
run: RunState,
frame: ExecutionFrame,
step: SubgraphNode,
activation: SubgraphActivation,
prepared: PreparedSubgraph[HandlerT],
reducers: Mapping[str, ReducerDefinition] | None,
) -> StepExecutionResult:
child_frame = run.frames.get(activation.child_frame_id)
if child_frame is None or child_frame.status != FrameStatus.COMPLETED:
raise WorkflowExecutionError(
f"subgraph step {step.id!r} resumed before its child completed"
)
child_scope = run.scopes.get(activation.scope_id)
if child_scope is None:
raise WorkflowExecutionError(
f"subgraph step {step.id!r} is missing child scope {activation.scope_id!r}"
)
child_outcome = child_frame.metadata.get("workflow_outcome")
if not isinstance(child_outcome, str):
raise WorkflowExecutionError(
f"subgraph step {step.id!r} child completed without a workflow outcome"
)
child_output = project_output(prepared.workflow, child_scope.committed_state)
validate_payload_against_schema(
prepared.workflow.output_schema,
child_output,
f"child workflow output for {step.id}",
)
validate_payload_against_schema(
step.output_schema, child_output, f"subgraph output for {step.id}"
)
patch = build_output_patch(
workflow,
step.output,
child_output,
state_view_for_frame(run, frame),
reducers=reducers,
missing_field_message="subgraph output did not include required field {field}",
)
state_changes = commit_patch_for_frame(run, frame, patch)
return StepExecutionResult(
outcome=child_outcome,
resolved_input=activation.child_input,
output=child_output,
state_changes=state_changes,
)
+1 -1
View File
@@ -68,7 +68,7 @@ def validate_subgraph_node(
workflow: Workflow, workflow: Workflow,
report: ValidationReport, report: ValidationReport,
) -> None: ) -> None:
"""Validate a subgraph boundary contract before runtime support exists.""" """Validate a subgraph boundary independently of runtime child resolution."""
_validate_boundary_bindings( _validate_boundary_bindings(
input_bindings=node.input, input_bindings=node.input,
output_bindings=node.output, output_bindings=node.output,
+205 -9
View File
@@ -1,19 +1,29 @@
from __future__ import annotations from __future__ import annotations
import asyncio
import pytest import pytest
from wf_core import ( from wf_core import (
END, END,
Edge, Edge,
EndNode,
InterruptNode,
NodeDef,
NodeUse,
PreparedSubgraph,
RunState,
SchemaRef, SchemaRef,
StateField, StateField,
StateSchema, StateSchema,
SubgraphNode, SubgraphNode,
Workflow, Workflow,
WorkflowExecutionError, WorkflowExecutionError,
execute_workflow_async,
execute_workflow, execute_workflow,
) )
from wf_core.validation.issues import ValidationIssueCode from wf_core.validation.issues import ValidationIssueCode
from wf_core.models.steps import Step
def test_subgraph_step_validates_boundary_bindings_and_outcomes() -> None: def test_subgraph_step_validates_boundary_bindings_and_outcomes() -> None:
@@ -73,33 +83,219 @@ def test_subgraph_step_rejects_unwired_declared_outcome() -> None:
) )
def test_subgraph_step_runtime_fails_explicitly_until_native_execution_exists() -> None: def test_subgraph_step_requires_prepared_child_dependency() -> None:
workflow = _workflow() workflow = _workflow()
with pytest.raises(WorkflowExecutionError, match="native subgraph execution"): with pytest.raises(WorkflowExecutionError, match="prepared child workflow"):
execute_workflow(workflow, {"text": "hello"}, {}) execute_workflow(workflow, {"text": "hello"}, {})
def _workflow(*, node: SubgraphNode | None = None) -> Workflow: def test_subgraph_step_executes_prepared_child_in_isolated_scope() -> None:
subgraph = node or SubgraphNode.model_validate( workflow = _workflow(
node=_subgraph_node(input_bindings=[{"target": "text", "value": "child-only"}]),
output_schema=_schema({"answer": {"type": "string"}}),
)
child = _child_workflow()
run = execute_workflow(
workflow,
{"text": "hello"},
{},
subgraphs={
"child.workflow": PreparedSubgraph(
workflow=child,
registry={
"answer": lambda payload, _ctx: {
"answer": f"child:{payload['text']}"
}
},
)
},
)
assert run.output["answer"] == "child:child-only"
assert run.state["answer"] == "child:child-only"
assert run.scopes["root"].committed_state["answer"] == "child:child-only"
child_scope = next(
scope for scope in run.scopes.values() if scope.workflow_name == child.name
)
assert child_scope.workflow_input["text"] == "child-only"
assert child_scope.committed_state["answer"] == "child:child-only"
assert run.trace[0].node_id == "answer"
assert run.trace[0].frame_id != "root"
assert run.trace[-1].node_id == "child"
assert run.trace[-1].step_type == "subgraph"
def test_subgraph_step_executes_prepared_async_child() -> None:
async def answer(payload: dict[str, object], _ctx: object) -> dict[str, object]:
return {"answer": f"async:{payload['text']}"}
async def execute() -> RunState:
return await execute_workflow_async(
_workflow(output_schema=_schema({"answer": {"type": "string"}})),
{"text": "hello"},
{},
subgraphs={
"child.workflow": PreparedSubgraph(
workflow=_child_workflow(),
registry={"answer": answer},
)
},
)
run = asyncio.run(execute())
assert run.output["answer"] == "async:hello"
assert run.trace[-1].step_type == "subgraph"
def test_subgraph_step_routes_through_child_terminal_outcome() -> None:
child = _child_workflow(
outcomes=["error"],
terminal=EndNode(id="child_error", type="end", outcome="error"),
edges=[
Edge.model_validate(
{"from": "answer", "outcome": "ok", "to": "child_error"}
)
],
)
subgraph = _subgraph_node(outcomes=["error"])
workflow = _workflow(
node=subgraph,
outcomes=["ok", "error"],
nodes=[subgraph, EndNode(id="parent_error", type="end", outcome="error")],
edges=[
Edge.model_validate(
{"from": "child", "outcome": "error", "to": "parent_error"}
)
],
)
run = execute_workflow(
workflow,
{"text": "hello"},
{},
subgraphs={
"child.workflow": PreparedSubgraph(
workflow=child,
registry={"answer": lambda payload, _ctx: {"answer": payload["text"]}},
)
},
)
assert run.outcome == "error"
assert any(
entry.node_id == "child_error" and entry.outcome == "error"
for entry in run.trace
)
assert any(
entry.node_id == "child" and entry.outcome == "error" for entry in run.trace
)
def test_subgraph_step_rejects_child_interrupt_until_resume_route_exists() -> None:
child = Workflow(
name="child.workflow",
input_schema=_schema({"text": {"type": "string"}}),
state_schema=StateSchema.from_field_map({}),
output_schema=_schema({}),
start="ask",
nodes=[
InterruptNode.model_validate(
{"id": "ask", "type": "interrupt", "kind": "input"}
)
],
edges=[Edge.model_validate({"from": "ask", "outcome": "submitted", "to": END})],
)
with pytest.raises(WorkflowExecutionError, match="child interrupts"):
execute_workflow(
_workflow(),
{"text": "hello"},
{},
subgraphs={"child.workflow": PreparedSubgraph(workflow=child, registry={})},
)
def _workflow(
*,
node: SubgraphNode | None = None,
outcomes: list[str] | None = None,
nodes: list[Step] | None = None,
edges: list[Edge] | None = None,
output_schema: SchemaRef | None = None,
) -> Workflow:
subgraph = node or _subgraph_node()
return Workflow(
name="subgraph_parent",
input_schema=_schema({"text": {"type": "string"}}),
state_schema=StateSchema.from_field_map({"answer": StateField(type="string")}),
output_schema=output_schema or _schema({}),
outcomes=outcomes or ["ok"],
start="child",
nodes=[subgraph] if nodes is None else nodes,
edges=edges
or [Edge.model_validate({"from": "child", "outcome": "ok", "to": END})],
)
def _subgraph_node(
*,
outcomes: list[str] | None = None,
input_bindings: list[dict[str, object]] | None = None,
) -> SubgraphNode:
return SubgraphNode.model_validate(
{ {
"id": "child", "id": "child",
"type": "subgraph", "type": "subgraph",
"workflow": "child.workflow", "workflow": "child.workflow",
"input_schema": _schema({"text": {"type": "string"}}), "input_schema": _schema({"text": {"type": "string"}}),
"output_schema": _schema({"answer": {"type": "string"}}), "output_schema": _schema({"answer": {"type": "string"}}),
"input": (
[{"target": "text", "path": "input.text"}]
if input_bindings is None
else input_bindings
),
"output": [{"source": "answer", "target": "state.answer"}],
"outcomes": outcomes or ["ok"],
}
)
def _child_workflow(
*,
outcomes: list[str] | None = None,
terminal: EndNode | None = None,
edges: list[Edge] | None = None,
) -> Workflow:
node = NodeUse.model_validate(
{
"id": "answer",
"type": "node",
"node": "answer",
"input": [{"target": "text", "path": "input.text"}], "input": [{"target": "text", "path": "input.text"}],
"output": [{"source": "answer", "target": "state.answer"}], "output": [{"source": "answer", "target": "state.answer"}],
} }
) )
return Workflow( return Workflow(
name="subgraph_parent", name="child.workflow",
input_schema=_schema({"text": {"type": "string"}}), input_schema=_schema({"text": {"type": "string"}}),
state_schema=StateSchema.from_field_map({"answer": StateField(type="string")}), state_schema=StateSchema.from_field_map({"answer": StateField(type="string")}),
output_schema=_schema({}), output_schema=_schema({"answer": {"type": "string"}}),
start="child", node_defs=[
nodes=[subgraph], NodeDef(
edges=[Edge.model_validate({"from": "child", "outcome": "ok", "to": END})], name="answer",
input_schema=_schema({"text": {"type": "string"}}),
output_schema=_schema({"answer": {"type": "string"}}),
outcomes=["ok"],
)
],
outcomes=outcomes or ["ok"],
start="answer",
nodes=[node] if terminal is None else [node, terminal],
edges=edges
or [Edge.model_validate({"from": "answer", "outcome": "ok", "to": END})],
) )