in-memory MCP resuming an interrupted deployment

This commit is contained in:
lda
2026-05-26 01:54:04 +07:00 Verified
parent 95532b1cf9
commit 9d11f78111
14 changed files with 585 additions and 91 deletions
+8 -8
View File
@@ -61,8 +61,9 @@ implementation state.
`SubgraphNode` is now the graph-as-node path for prepared children.
`WorkflowBuilder.prepare_subgraph()` and `WorkflowBuilder.resume()` make the
local runnable/resumable path available without core-runtime plumbing.
Saved interrupting artifacts remain unrunnable through one-shot
`run_deployment` until the platform exposes persisted resume.
Saved interrupting artifacts can now pause and resume through
`run_deployment`/`resume_run` while the MCP server process stays alive.
Persisted resume remains future work.
- **Concurrent foreach**: implemented in core with explicit scheduling,
reducer/merge semantics, item error policy, async handler batching, and
quiescent interrupt behavior. Remaining work is polish and future reuse of
@@ -104,9 +105,8 @@ Frame stress points remaining for native subgraphs and future fork/gather:
## Why This Order
The MCP workflow authoring path is now usable enough for real testing. The next
bottleneck is runtime/platform correctness: persisted resume for saved
interrupting children, optional per-use-site child deployment overrides,
persistent run history, and protocol-native progress reporting. Concurrent
foreach and native saved child execution now supply scheduler/lineage
precedent. Those remaining pieces should come before adding more high-level
authoring sugar.
bottleneck is runtime/platform correctness: durable resume/run history,
optional per-use-site child deployment overrides, and protocol-native progress
reporting. Concurrent foreach, native saved child execution, and process-local
interrupt resume now supply scheduler/lineage precedent. Those remaining pieces
should come before adding more high-level authoring sugar.
+5 -3
View File
@@ -142,8 +142,9 @@ limits and intended adapter seam.
and lineage; child output commits only through declared boundary bindings and
the parent routes by the child's terminal workflow outcome. Saved/deployed
workflow resolution remains outside core; the workflow platform can now
supply non-interrupting saved child artifacts as prepared dependencies using
one inherited deployment binding environment. For local authoring,
supply saved child artifacts as prepared dependencies using one inherited
deployment binding environment, including process-local pause/resume for
child interrupts. For local authoring,
`WorkflowBuilder.prepare_subgraph()`
registers a child builder and `WorkflowBuilder.resume()` continues a paused
prepared-child interrupt without requiring direct core-runtime calls.
@@ -162,7 +163,8 @@ limits and intended adapter seam.
queue, `BLOCKED` frame state, lineage isolation, barrier merge semantics, and
pending child results for concurrent foreach. Native prepared subgraphs now
use child-scope execution and typed routed child interruption; the platform
resolves non-interrupting saved/deployed child artifacts before core starts.
resolves saved/deployed child artifacts before core starts and retains paused
deployment runs in memory for resume.
Concurrent foreach is the primary current use case for async concurrent node
handler execution.
- Runtime errors are still ordinary exceptions plus failed run status. A richer
+17 -7
View File
@@ -298,18 +298,28 @@ Inspect the returned diagnostics first. Then use the matching section above:
Do not debug runtime behavior before dependency validation is clean.
## `run_deployment` Refuses An Interrupting Artifact
## `run_deployment` Returns `interrupted`
Interrupting saved artifacts are not fully supported through the current saved
artifact execution path yet.
Expected diagnostic:
The deployment paused at an interrupt node. The response should include:
```text
interrupting_artifact_unsupported
status: interrupted
outcome: null
run_id: <process-local id>
interrupt: <request payload and metadata>
```
That is a known platform limitation, not a missing deployment binding.
Send the requested resume payload to:
```text
wf.workflow.resume_run
```
The `run_id` is intentionally process-local. It is valid only while the current
MCP server process keeps the paused run in memory. If the server restarts,
there is no durable run store yet; rerun the deployment from the beginning.
After resume completes, `status` is `completed` and `outcome` reports the
workflow terminal outcome such as `ok` or `error`.
## A Raw MCP Tool Works But The Workflow Version Is Awkward
+23 -17
View File
@@ -69,9 +69,13 @@ semantics.
Dynamic projection of saved workflows as individual MCP tools can exist later,
but it should be optional. The stable run tool is the reliable base layer.
Current `run_deployment` calls are synchronous request/response executions. They
return compact status, output, diagnostics, and `trace_count`; optional ranged
trace detail is for debugging only.
Current `run_deployment` calls are synchronous request/response executions until
the workflow pauses or completes. They return compact execution status,
terminal workflow outcome, output, diagnostics, and `trace_count`; optional
ranged trace detail is for debugging
only. If a run pauses at an interrupt, the response includes a process-local
`run_id` and interrupt payload. Use `wf.workflow.resume_run` with that `run_id`
to continue while the same MCP server process is alive.
Non-interrupting saved workflow children can now execute natively through this
deployment surface. A parent deployment resolves its saved descendants by exact
@@ -80,20 +84,21 @@ bindings for the whole child tree. This is intentionally one configured graph
environment; future per-child deployment overrides, if added, must be keyed by
the subgraph use site rather than only the child artifact id.
Interrupting saved artifacts remain unrunnable through `run_deployment`.
Although core prepared children can interrupt and resume, this one-shot public
surface does not yet persist a run for a later resume request.
Interrupting saved artifacts can pause and resume through the current
deployment surface, including interrupts raised inside saved child workflows.
This support is in-memory only: server restart, reload that replaces process
state, or another frontend process invalidates the `run_id`.
Future run history should introduce a stable `run_id` only when there is a real
run store behind it. A `run_id` without persisted state, trace paging, and
status lookup would be misleading. The likely shape is:
Future run history should replace process-local `run_id` values with durable
run records. The likely shape is:
- `run_deployment` starts or completes a run and returns `run_id`
- `inspect_run(run_id)` returns status, output, diagnostics, and trace metadata
- `read_run_trace(run_id, range)` returns bounded trace slices
Until that exists, clients should treat the current response as the complete
ephemeral run result for this request.
ephemeral run result for this request, or as a process-local resume handle when
`status` is `interrupted`.
For long-running workflow execution, prefer MCP-native execution mechanisms
where available:
@@ -664,12 +669,13 @@ parent-side contract: child workflow reference, declared child input/output
schemas, binding lists, and declared outcomes. When callers resolve a local
child into `PreparedSubgraph`, core executes it in child scope, preserves its
trace, and can bubble and resume child interrupts without exposing child state
as parent state. The current `wf_authoring.subgraph_node` and
as parent state. Saved child artifacts are resolved by the workflow surface into
the same `PreparedSubgraph` shape before execution. The current
`wf_authoring.subgraph_node` and
`async_subgraph_node` helpers still execute a child workflow as a plain node
and validate the child output. The async helper is explicit because hiding
`asyncio.run()` inside the sync wrapper would break inside already-running
event loops. Saved workflow-as-node execution still needs platform-level
artifact/deployment resolution into prepared children before core can run it.
event loops.
See `examples/authoring_workflow_as_node.py` for the compatibility wrapper-node
approach and `examples/authoring_native_subgraph.py` for native prepared-child
@@ -678,10 +684,10 @@ native child pause and builder-driven resume. In the wrapper example the
parent trace sees one node call; in the native examples child trace entries
remain in the parent run state.
Until that core upgrade exists, artifact tooling must not assume that an
interrupting saved workflow can safely be used as a child node. Top-level saved
workflows with interrupt nodes are valid, but nested interrupting workflows
should be reported as unsupported for composition.
Persisted resume is still not implemented. In-memory resume works because the
server keeps the paused `RunState`; a durable run store will need to snapshot
the root workflow, prepared child dependencies, deployment bindings, and trace
metadata before this can survive restart or move across processes.
Blocking dependency failures happen before workflow execution and are not normal
workflow outcomes. A missing source, disabled source, unresolved binding, or
+110 -3
View File
@@ -96,11 +96,101 @@ Important details:
- `steps` are keyed by stable ids so patches do not depend on array positions.
- `start` names one step id.
- `routes` map step outcomes to another step id or `__end__`.
- top-level `outcomes` declares public workflow terminal outcomes; if omitted,
it defaults to `["ok"]`.
- top-level `output` maps final graph values such as `state.result` into the
public workflow output payload. Step-level `output` only writes a node result
into workflow state.
- `capability` may be concrete during exploration, such as
`demo.personal.echo_tool`.
- When saved with source bindings, concrete refs can be normalized to logical
refs such as `demo.echo_tool`.
## Explicit Outputs And Error Outcomes
Use `__end__` as the compact terminal path for the normal `ok` workflow outcome.
For any other public terminal outcome, add an explicit `end` step and route to
it. The end step itself is terminal; do not add an edge out of it.
This complete draft shape:
```json
{
"name": "echo_with_error",
"input_schema": {
"type": "object",
"properties": {
"text": { "type": "string" },
"fail": { "type": "boolean" }
},
"required": ["text"]
},
"state_schema": {
"type": "object",
"properties": {
"raw": {
"type": "object",
"properties": {
"echoed": { "type": "string" }
}
}
}
},
"output_schema": {
"type": "object",
"properties": {
"message": { "type": "string" }
}
},
"outcomes": ["ok", "error"],
"output": [
{
"target": { "root": "local", "parts": ["message"] },
"path": { "root": "state", "parts": ["raw", "echoed"] }
}
],
"start": "call",
"steps": {
"call": {
"use": "demo.echo",
"input": [
{
"target": { "root": "local", "parts": ["text"] },
"path": { "root": "input", "parts": ["text"] }
},
{
"target": { "root": "local", "parts": ["fail"] },
"path": { "root": "input", "parts": ["fail"] }
}
],
"output": [
{
"source": { "root": "local", "parts": ["echoed"] },
"target": { "root": "state", "parts": ["raw", "echoed"] }
}
]
},
"end_error": {
"end": { "outcome": "error" }
}
},
"routes": {
"call": {
"ok": "__end__",
"error": "end_error"
}
}
}
```
Read it as:
- `call.ok` finishes the workflow with outcome `ok`.
- `call.error` executes `end_error`, which finishes the workflow with outcome
`error`.
- both terminal paths project `state.raw.echoed` into public output field
`message`.
## Binding Shape
Draft `use` steps use the same canonical binding structs as core `NodeUse`:
@@ -356,9 +446,26 @@ provided on resume back into workflow state. Older map-shaped `request` and
`resume` values are accepted only as parse compatibility and dump back to the
canonical list shape.
Saved interrupting artifacts are still limited in the current execution
surface. If a deployment reports `interrupting_artifact_unsupported`, that is a
known platform limitation rather than a draft bug.
Saved interrupting artifacts can pause and resume through deployment runs while
the MCP server process stays alive. The run response includes a process-local
`run_id`; pass that to `wf.workflow.resume_run` with the resume payload.
Persisted run storage is still future work, so a server restart invalidates
that in-memory run id.
### `end`
Declares an explicit workflow terminal outcome.
```json
{
"end": {
"outcome": "error"
}
}
```
Use explicit `end` steps for non-`ok` workflow outcomes. The legacy `__end__`
destination remains the shorthand for public workflow outcome `ok`.
### `join`