subgraph model & validation

This commit is contained in:
lda
2026-05-25 01:12:41 +07:00 Verified
parent c06df7613b
commit 2af0b3357f
16 changed files with 285 additions and 35 deletions
+6 -5
View File
@@ -45,11 +45,12 @@ implementation state.
[ADR 0002](./adr/0002-concurrent-foreach-policy-and-barrier-commits.md).
- Native subgraph design spec:
[2026-05-24 native subgraphs](./superpowers/specs/2026-05-24-native-subgraphs-design.md).
- **Native subgraphs / graph-as-node**: next major runtime feature. Add a
first-class subgraph step with child run/frame identity, child trace
preservation, interrupt bubbling, and resume back into the child workflow.
Wrapper helpers currently run child workflows as ordinary nodes; true
graph-as-node behavior belongs here.
- **Native subgraphs / graph-as-node**: next major runtime feature. A
first-class `SubgraphNode` placeholder exists and validates parent-side
bindings/outcomes, but runtime execution still needs child run/frame identity,
child trace preservation, interrupt bubbling, and resume back into the child
workflow. Wrapper helpers currently run child workflows as ordinary nodes;
true graph-as-node behavior belongs here.
- **Concurrent foreach**: implemented in core with explicit scheduling,
reducer/merge semantics, item error policy, async handler batching, and
quiescent interrupt behavior. Remaining work is polish and future reuse of
@@ -56,11 +56,19 @@ class SubgraphNode(BaseModel):
id: str
type: Literal["subgraph"]
workflow: WorkflowRef
input_schema: SchemaRef
output_schema: SchemaRef
input: list[InputBinding] = Field(default_factory=list)
output: list[OutputBinding] = Field(default_factory=list)
outcomes: list[str] = Field(default_factory=lambda: ["ok"])
```
Current implementation status: `wf_core` has a first placeholder
`SubgraphNode`, but `workflow` is still a plain string reference. The placeholder
also carries `input_schema` and `output_schema` so validation can check parent
bindings before native execution exists. Runtime execution intentionally raises
until a later slice adds child scope/frame execution.
`WorkflowRef` should be structural, not a dotted string parser:
```python
+10 -7
View File
@@ -132,9 +132,11 @@ limits and intended adapter seam.
external subscriptions or notification streams need a separate lifecycle
design. Interrupt `request` and `resume` are canonical binding lists; nested
child-workflow resume is still future work.
- Native subgraphs are not part of `wf_core` yet. The core `Step` model only
includes node, condition, foreach, join, and interrupt steps; `Workflow` does
not contain nested workflow/subgraph steps.
- Native subgraphs have a core model placeholder, `SubgraphNode`, but runtime
execution is not implemented yet. The placeholder carries a child workflow
reference, declared input/output schemas, binding lists, and declared
outcomes so parent graph structure can validate before execution support
lands.
- Nested subgraph interruption is not first-class yet. The current
`wf_authoring` subgraph helpers wrap a child workflow as an ordinary sync or
async node and validate the child output; they do not preserve a child run
@@ -145,10 +147,11 @@ limits and intended adapter seam.
upgrade: nested run state, child-frame trace preservation, interrupt bubbling
with path metadata, and resume back into the child workflow.
- Frames are no longer only a serial execution stack: the runtime has a ready
queue and `BLOCKED` frame state. Concurrent foreach and native subgraphs still
need more work: lineage isolation, barrier merge semantics, pending child
results, and explicit child workflow/deployment identity. Concurrent foreach
is the primary current use case for async concurrent node handler execution.
queue, `BLOCKED` frame state, lineage isolation, barrier merge semantics, and
pending child results for concurrent foreach. Native subgraphs still need
explicit child workflow/deployment identity and child-scope execution.
Concurrent foreach is the primary current use case for async concurrent node
handler execution.
- Runtime errors are still ordinary exceptions plus failed run status. A richer
error payload can be added later, but should be designed as part of trace/run
state rather than scattered exceptions.
+6 -4
View File
@@ -635,10 +635,12 @@ must not be parsed as a generic `CapabilityRef`.
The first implementation should prefer artifact validation and dependency
diagnostics before attempting persistent nested resume.
Native subgraphs are not in `wf_core` yet. The current core `Step` model has
node, condition, foreach, join, and interrupt steps, but no subgraph step. The
current `wf_authoring.subgraph_node` and `async_subgraph_node` helpers execute
a child workflow as a plain node and validate the child output. The async helper
Native subgraphs now have a core `SubgraphNode` placeholder. It validates the
parent-side contract: child workflow reference, declared child input/output
schemas, binding lists, and declared outcomes. Runtime execution is still not
implemented; reaching a subgraph step raises a clear runtime error. The current
`wf_authoring.subgraph_node` and `async_subgraph_node` helpers still execute a
child workflow as a plain node and validate the child output. The async helper
is explicit because hiding `asyncio.run()` inside the sync wrapper would break
inside already-running event loops. Future saved-workflow-as-node execution
needs a real child run state if child interrupts should pause the parent and
+2
View File
@@ -15,6 +15,7 @@ from .models import (
SiblingWritePolicy,
StateField,
StateSchema,
SubgraphNode,
Workflow,
)
from .runtime import (
@@ -64,6 +65,7 @@ __all__ = [
"SiblingWritePolicy",
"StateField",
"StateSchema",
"SubgraphNode",
"AsyncNodeHandler",
"NodeHandler",
"ExecutionFrame",
+2
View File
@@ -20,6 +20,7 @@ from wf_core.models.steps import (
JoinNode,
NodeUse,
Step,
SubgraphNode,
)
from wf_core.models.workflow import Edge, Workflow
@@ -48,6 +49,7 @@ __all__ = [
"StateField",
"StateSchema",
"Step",
"SubgraphNode",
"VariadicCondition",
"Workflow",
]
+44 -1
View File
@@ -6,6 +6,7 @@ from typing import Annotated, Literal, Self
from pydantic import BaseModel, ConfigDict, Field, model_validator
from wf_core.models.conditions import Condition
from wf_core.models.schemas import SchemaRef
from wf_core.paths import GraphSourcePath, LocalPath, StatePath
@@ -155,6 +156,48 @@ class NodeUse(BaseModel):
return value
class SubgraphNode(BaseModel):
"""Workflow boundary step reserved for native subgraph execution.
This is a contract-bearing placeholder, not the implementation of nested
workflow execution yet. The core can validate the parent graph's bindings
and declared outcomes now; a later runtime slice will resolve ``workflow``
into a child graph, create a child scope/lineage, and commit its result.
"""
id: str
type: Literal["subgraph"]
workflow: str = Field(
description=(
"Reference to the child workflow artifact or registry key. The core "
"does not resolve this reference until native subgraph runtime "
"execution is implemented."
)
)
desc: str | None = None
input_schema: SchemaRef = Field(
default_factory=lambda: SchemaRef(type="object"),
description="Declared child workflow input contract used to validate input bindings.",
)
output_schema: SchemaRef = Field(
default_factory=lambda: SchemaRef(type="object"),
description="Declared child workflow output contract used to validate output bindings.",
)
input: list[InputBinding] = Field(
default_factory=list,
description="Bindings that build the child workflow input payload.",
)
output: list[OutputBinding] = Field(
default_factory=list,
description="Bindings that commit child workflow output into parent state.",
)
outcomes: list[str] = Field(
default_factory=lambda: ["ok"],
min_length=1,
description="Outcomes the parent graph may wire from this subgraph boundary.",
)
class ConditionNode(BaseModel):
"""Control-flow step that routes through `true` or `false` outcomes."""
@@ -329,7 +372,7 @@ class InterruptNode(BaseModel):
Step = Annotated[
NodeUse | ConditionNode | ForeachNode | JoinNode | InterruptNode,
NodeUse | SubgraphNode | ConditionNode | ForeachNode | JoinNode | InterruptNode,
Field(discriminator="type"),
]
"""Discriminated union of all executable workflow graph steps."""
+8 -1
View File
@@ -55,6 +55,9 @@ def lineage_writes_for_frame(
)
)
# Compatibility fallback: concurrent foreach used barrier-local patches
# before `RunState.lineages` became the primary write store. Keep reading
# those patches so old serialized runs and direct barrier tests still work.
owner = item_frame_owner(frame)
if owner is None:
return ()
@@ -127,7 +130,11 @@ def lineage_patch(
scope_id: str,
lineage_id: str,
) -> StatePatch:
"""Return a replayable patch for one lineage's pending writes."""
"""Return a replayable patch for one lineage's pending writes.
Barrier/gather code should consume this instead of reconstructing a patch
from visible state. Incoming values are the replay source of truth.
"""
lineage = _lineage(run, scope_id=scope_id, lineage_id=lineage_id)
return StatePatch(writes=list(lineage.writes))
+2
View File
@@ -340,6 +340,8 @@ def _finish_concurrent_foreach(
raise WorkflowExecutionError(
"collect item error policy requires collect_to"
)
# Collect-error records are generated by the barrier itself, not by an
# item lineage, so they still enter as a compatibility `changes` patch.
item_patches.append(StatePatch(changes={str(collect_to): error_records}))
combined = build_barrier_patch(
workflow,
+4
View File
@@ -124,6 +124,8 @@ def _finalize_node_execution(
if is_root_lineage_frame(frame):
state_changes = commit_state_patch(run.state, patch)
else:
# Non-root frames are future subgraph/fork branch execution: writes
# become lineage-local until an explicit boundary/barrier commits.
append_lineage_writes(
run,
scope_id=frame.scope_id,
@@ -136,6 +138,8 @@ def _finalize_node_execution(
parent_frame = run.frames[parent_frame_id]
barrier = ForeachBarrierState.from_frame(parent_frame, foreach_node_id)
if barrier is not None and barrier.mode == "concurrent":
# New concurrent foreach stores writes in the child lineage; the
# barrier keeps only result metadata plus old patch fallback.
append_lineage_writes(
run,
scope_id=frame.scope_id,
+6 -1
View File
@@ -49,7 +49,12 @@ class StatePatch:
_staged_state: dict[str, Any] = dataclass_field(default_factory=dict, repr=False)
def __post_init__(self) -> None:
"""Keep legacy `StatePatch(changes=...)` usable during migration."""
"""Keep legacy `StatePatch(changes=...)` usable during migration.
New runtime code should prefer ordered `writes`. `changes` stays as the
public trace-facing view and as parse compatibility for old barrier
metadata/tests that predate `StateWrite`.
"""
if not self.changes and self.writes:
self.changes = {
str(write.path): write.incoming_value for write in self.writes
+11
View File
@@ -11,6 +11,7 @@ from wf_core.models.steps import (
InterruptNode,
JoinNode,
NodeUse,
SubgraphNode,
)
from wf_core.models.workflow import Workflow
from wf_core.runtime.ops.flow import advance_frame, append_step_result_trace
@@ -115,6 +116,11 @@ def step_workflow(
return handle_interrupt_step(run, step)
elif isinstance(step, ForeachNode):
return step_foreach(workflow, run, step, index, reducers=reducers)
elif isinstance(step, SubgraphNode):
raise WorkflowExecutionError(
f"subgraph step {step.id!r} references {step.workflow!r}, "
"but native subgraph execution is not implemented yet"
)
else:
raise WorkflowExecutionError(
f"unsupported step type {getattr(step, 'type', type(step).__name__)!r}"
@@ -209,6 +215,11 @@ async def step_workflow_async(
return handle_interrupt_step(run, step)
elif isinstance(step, ForeachNode):
return step_foreach(workflow, run, step, index, reducers=reducers)
elif isinstance(step, SubgraphNode):
raise WorkflowExecutionError(
f"subgraph step {step.id!r} references {step.workflow!r}, "
"but native subgraph execution is not implemented yet"
)
else:
raise WorkflowExecutionError(
f"unsupported step type {getattr(step, 'type', type(step).__name__)!r}"
+4
View File
@@ -6,6 +6,7 @@ from wf_core.models.steps import (
InterruptNode,
NodeUse,
Step,
SubgraphNode,
)
from wf_core.models.schemas import NodeDef
from wf_core.models.workflow import Edge, Workflow
@@ -17,6 +18,7 @@ from wf_core.validation.steps import (
validate_foreach_node,
validate_interrupt_node,
validate_node_use,
validate_subgraph_node,
)
@@ -69,6 +71,8 @@ def _validate_nodes(
if isinstance(node, NodeUse):
validate_node_use(node, index, node_defs, workflow, report)
elif isinstance(node, SubgraphNode):
validate_subgraph_node(node, index, workflow, report)
elif isinstance(node, ConditionNode):
validate_condition_node(
node, index, report, state_root_fields, input_root_fields
+3 -1
View File
@@ -1,7 +1,7 @@
from __future__ import annotations
from wf_core.models.schemas import NodeDef
from wf_core.models.steps import InterruptNode, NodeUse, Step
from wf_core.models.steps import InterruptNode, NodeUse, Step, SubgraphNode
from wf_core.models.workflow import Edge
from wf_core.tokens import END
@@ -10,6 +10,8 @@ def declared_outcomes_for_step(step: Step, node_defs: dict[str, NodeDef]) -> set
if isinstance(step, NodeUse):
node_def = node_defs.get(step.node)
return set(node_def.outcomes) if node_def else set()
if isinstance(step, SubgraphNode):
return set(step.outcomes)
if step.type == "condition":
return {"true", "false"}
if step.type == "foreach":
+62 -15
View File
@@ -14,9 +14,12 @@ from wf_core.models.schemas import NodeDef
from wf_core.models.steps import (
ConditionNode,
ForeachNode,
InputBinding,
InputPathBinding,
InterruptNode,
NodeUse,
OutputBinding,
SubgraphNode,
)
from wf_core.models.workflow import Workflow
from wf_core.paths import (
@@ -45,21 +48,65 @@ def validate_node_use(
)
return
input_fields = set(node_def.input_schema.properties)
output_fields = set(node_def.output_schema.properties)
state_root_fields = workflow.state_schema.root_fields()
input_root_fields = set(workflow.input_schema.properties)
_validate_boundary_bindings(
input_bindings=node.input,
output_bindings=node.output,
input_fields=set(node_def.input_schema.properties),
output_fields=set(node_def.output_schema.properties),
state_root_fields=workflow.state_schema.root_fields(),
input_root_fields=set(workflow.input_schema.properties),
report=report,
path_prefix=f"nodes[{index}]",
input_error_code=ValidationIssueCode.INVALID_NODE_INPUT_FIELD,
output_error_code=ValidationIssueCode.INVALID_NODE_OUTPUT_FIELD,
)
def validate_subgraph_node(
node: SubgraphNode,
index: int,
workflow: Workflow,
report: ValidationReport,
) -> None:
"""Validate a subgraph boundary contract before runtime support exists."""
_validate_boundary_bindings(
input_bindings=node.input,
output_bindings=node.output,
input_fields=set(node.input_schema.properties),
output_fields=set(node.output_schema.properties),
state_root_fields=workflow.state_schema.root_fields(),
input_root_fields=set(workflow.input_schema.properties),
report=report,
path_prefix=f"nodes[{index}]",
input_error_code=ValidationIssueCode.INVALID_NODE_INPUT_FIELD,
output_error_code=ValidationIssueCode.INVALID_NODE_OUTPUT_FIELD,
)
def _validate_boundary_bindings(
*,
input_bindings: list[InputBinding],
output_bindings: list[OutputBinding],
input_fields: set[str],
output_fields: set[str],
state_root_fields: set[str],
input_root_fields: set[str],
report: ValidationReport,
path_prefix: str,
input_error_code: ValidationIssueCode,
output_error_code: ValidationIssueCode,
) -> None:
"""Validate bindings for node-like boundaries with declared I/O schemas."""
input_targets = []
for input_index, binding in enumerate(node.input):
for input_index, binding in enumerate(input_bindings):
input_targets.append(binding.target)
destination_root = _local_root(binding.target)
if destination_root is None or (
destination_root != "." and destination_root not in input_fields
):
report.add(
ValidationIssueCode.INVALID_NODE_INPUT_FIELD,
f"nodes[{index}].input[{input_index}].target",
input_error_code,
f"{path_prefix}.input[{input_index}].target",
f"destination field {str(binding.target)!r} is not declared in node input schema",
)
@@ -68,40 +115,40 @@ def validate_node_use(
):
report.add(
ValidationIssueCode.INVALID_SOURCE_PATH,
f"nodes[{index}].input[{input_index}].path",
f"{path_prefix}.input[{input_index}].path",
"source path must start with input., state., or context. and reference a declared root field when applicable",
)
if has_overlapping_paths(input_targets):
report.add(
ValidationIssueCode.INVALID_NODE_INPUT_FIELD,
f"nodes[{index}].input",
input_error_code,
f"{path_prefix}.input",
"input has overlapping node-local input paths",
)
output_targets = []
for output_index, binding in enumerate(node.output):
for output_index, binding in enumerate(output_bindings):
output_targets.append(str(binding.target))
source_root = _local_root(binding.source)
if source_root is None or (
source_root != "." and source_root not in output_fields
):
report.add(
ValidationIssueCode.INVALID_NODE_OUTPUT_FIELD,
f"nodes[{index}].output[{output_index}].source",
output_error_code,
f"{path_prefix}.output[{output_index}].source",
f"source field {str(binding.source)!r} is not declared in node output schema",
)
destination_root = _state_destination_root(binding.target)
if destination_root is None or destination_root not in state_root_fields:
report.add(
ValidationIssueCode.INVALID_DESTINATION_PATH,
f"nodes[{index}].output[{output_index}].target",
f"{path_prefix}.output[{output_index}].target",
"destination path must start with state. and reference a declared root field",
)
if has_overlapping_paths(output_targets):
report.add(
ValidationIssueCode.INVALID_DESTINATION_PATH,
f"nodes[{index}].output",
f"{path_prefix}.output",
"output has overlapping state destination paths",
)
+107
View File
@@ -0,0 +1,107 @@
from __future__ import annotations
import pytest
from wf_core import (
END,
Edge,
SchemaRef,
StateField,
StateSchema,
SubgraphNode,
Workflow,
WorkflowExecutionError,
execute_workflow,
)
from wf_core.validation.issues import ValidationIssueCode
def test_subgraph_step_validates_boundary_bindings_and_outcomes() -> None:
workflow = _workflow()
report = workflow.validate_structure()
assert report.errors == []
def test_subgraph_step_rejects_undeclared_input_target() -> None:
workflow = _workflow(
node=SubgraphNode.model_validate(
{
"id": "child",
"type": "subgraph",
"workflow": "child.workflow",
"input_schema": _schema({"text": {"type": "string"}}),
"output_schema": _schema({"answer": {"type": "string"}}),
"input": [{"target": "missing", "path": "input.text"}],
"output": [{"source": "answer", "target": "state.answer"}],
}
)
)
report = workflow.validate_structure()
assert any(
issue.code == ValidationIssueCode.INVALID_NODE_INPUT_FIELD
and issue.path == "nodes[0].input[0].target"
for issue in report.errors
)
def test_subgraph_step_rejects_unwired_declared_outcome() -> None:
workflow = _workflow(
node=SubgraphNode.model_validate(
{
"id": "child",
"type": "subgraph",
"workflow": "child.workflow",
"input_schema": _schema({"text": {"type": "string"}}),
"output_schema": _schema({"answer": {"type": "string"}}),
"outcomes": ["ok", "failed"],
"input": [{"target": "text", "path": "input.text"}],
"output": [{"source": "answer", "target": "state.answer"}],
}
)
)
report = workflow.validate_structure()
assert any(
issue.code == ValidationIssueCode.MISSING_OUTCOME_EDGE
and "failed" in issue.message
for issue in report.errors
)
def test_subgraph_step_runtime_fails_explicitly_until_native_execution_exists() -> None:
workflow = _workflow()
with pytest.raises(WorkflowExecutionError, match="native subgraph execution"):
execute_workflow(workflow, {"text": "hello"}, {})
def _workflow(*, node: SubgraphNode | None = None) -> Workflow:
subgraph = node or SubgraphNode.model_validate(
{
"id": "child",
"type": "subgraph",
"workflow": "child.workflow",
"input_schema": _schema({"text": {"type": "string"}}),
"output_schema": _schema({"answer": {"type": "string"}}),
"input": [{"target": "text", "path": "input.text"}],
"output": [{"source": "answer", "target": "state.answer"}],
}
)
return Workflow(
name="subgraph_parent",
input_schema=_schema({"text": {"type": "string"}}),
state_schema=StateSchema.from_field_map({"answer": StateField(type="string")}),
output_schema=_schema({}),
start="child",
nodes=[subgraph],
edges=[Edge.model_validate({"from": "child", "outcome": "ok", "to": END})],
)
def _schema(properties: dict[str, object]) -> SchemaRef:
return SchemaRef.model_validate({"type": "object", "properties": properties})