plan for durable run state storage idfk
This commit is contained in:
@@ -1,102 +0,0 @@
|
||||
# lda rambles
|
||||
|
||||
that was surprisingly tuff response from our first user. it made me think about how we handle @node atm.
|
||||
|
||||
atm only one way is allowed: path. If you want to inbuilt a value; please use the the graph input.
|
||||
|
||||
our first user doesnt understand this. AT ALL.
|
||||
|
||||
this calls to several problems.
|
||||
|
||||
## dict use, instead of list[InputMap]
|
||||
|
||||
why dict, when we can:
|
||||
|
||||
```python
|
||||
class InputMap(BaseModel, Generic[TypeT]):
|
||||
key: SupportToValue[TypeT]
|
||||
input: Path
|
||||
```
|
||||
|
||||
same thing
|
||||
|
||||
```python
|
||||
class OutputMap(BaseModel, Generic[TypeT]):
|
||||
output: Pathexpr
|
||||
key (give me a better key name): SupportToValue[TypeT]
|
||||
```
|
||||
|
||||
look at this support to value, or use another fitting name.
|
||||
what it does is:
|
||||
|
||||
Path("state.foo") -> TypeT
|
||||
Literal(TypeT) -> TypeT
|
||||
|
||||
two modes that we currently need to PATCH IN. we do not PATCH wf_core if there isnt a greater problem.
|
||||
|
||||
### default arguments, and literal()
|
||||
|
||||
expand on literal, in wf_authoring we can have some Field bs:
|
||||
|
||||
```python
|
||||
@node(
|
||||
in_map = [
|
||||
InputMap(dont support this either, use g.use()),
|
||||
]
|
||||
)
|
||||
def baz(spam: InputT = "literal input", egg: Input2T = Path(do not support this.)):
|
||||
...
|
||||
```
|
||||
|
||||
so that means wf_authoring.node shouldnt do any field augmentation.
|
||||
|
||||
```python
|
||||
g.use(
|
||||
baz,
|
||||
in_map = [
|
||||
now were talking
|
||||
]
|
||||
)
|
||||
```
|
||||
|
||||
this SHOULD support both path and literal.
|
||||
|
||||
## Context, of some sort
|
||||
|
||||
langgraph has 3 ways of storage, if you read tests/rewrite
|
||||
|
||||
lets copy that over:
|
||||
|
||||
```
|
||||
According to https://docs.langchain.com/oss/python/concepts/context, there are three types:
|
||||
|
||||
| type | mut | lifetime |
|
||||
| --- | --- | --- |
|
||||
|static runtime (context) | static | single run |
|
||||
|dynamic runtime (state) | mut | single run |
|
||||
|dynamic cross-convo (store) | mut | cross-conversation |
|
||||
|
||||
now what the hell is store
|
||||
## store
|
||||
|
||||
store is used in langgraph-demo for debugging. but it can be used for more things.
|
||||
it saves every turn. every graph nodes. I use InMemoryStore, you can use psql store!
|
||||
This allows for picking the work up again after a while for example.
|
||||
a lot more versatility there.
|
||||
```
|
||||
|
||||
now we 100% has state. we kinda have store if you skim it (trace), but trace is only 50% store. and Context is patched in through input/state.
|
||||
|
||||
this can be done with support for async, because lowk store is going to give us superpowers.
|
||||
|
||||
## how this would aid adding async + foreach parallel
|
||||
|
||||
(probably async gather)
|
||||
|
||||
currently frame feels like magic + its core i dont understand. + the current serial mode impl is lowk smart.
|
||||
its reasonable; it spawns how many child frames as how many loop in foreach container item. Serially, like how `for in` in python works.
|
||||
complaints are only API improvements; not so much aid into adding these
|
||||
|
||||
## complaints come when user uses
|
||||
|
||||
not a lot from me rn.
|
||||
@@ -77,9 +77,14 @@ implementation state.
|
||||
metadata and compatibility patches. Scope-root commits now apply to both the
|
||||
root workflow and prepared native child scopes through the explicit
|
||||
scope/lineage commit helper.
|
||||
- **Persistent run history**: add a run store before adding stable `run_id`,
|
||||
`inspect_run`, or `read_run_trace(run_id, range)` APIs. Current traces are
|
||||
returned directly from immediate run responses.
|
||||
- **Durable run history and resume**: design is recorded in
|
||||
[2026-05-26 durable workflow runs](./superpowers/specs/2026-05-26-durable-workflow-runs-and-resume-design.md).
|
||||
Add a validated `RunState` storage codec and dedicated run/checkpoint store.
|
||||
Persist stopped snapshots for interrupted, completed, and failed executions;
|
||||
replace process-local resume handles with stable `run_id` values; add compact
|
||||
`inspect_run` and bounded `read_run_trace` APIs. Resuming an interrupted run
|
||||
must revalidate its pinned dependency environment; ordinary live tool/source
|
||||
failures remain failed runs, not implicit pauses.
|
||||
- **Protocol-native long-running runs**: investigate MCP tasks/progress
|
||||
notifications for long-running workflow execution. Avoid inventing a custom
|
||||
"start" convention unless protocol-native behavior is insufficient.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,7 +1,7 @@
|
||||
# Native Subgraphs Design
|
||||
|
||||
Status: prepared-child execution and interrupt resume implemented; saved-artifact
|
||||
resolution scoped for platform implementation
|
||||
Status: prepared-child execution, saved-artifact resolution, and process-local
|
||||
interrupt resume implemented; durable resume specified separately
|
||||
|
||||
Native subgraphs should make a workflow usable as a workflow step without
|
||||
collapsing the child run into one opaque Python node call. The current
|
||||
@@ -10,10 +10,9 @@ compatibility wrappers, but they hide the child trace, child frames, and child
|
||||
interrupt lifecycle from `wf_core`.
|
||||
|
||||
This design defines the core runtime shape. The boundary model, prepared-child
|
||||
execution, and routed child interrupt resume are implemented. The remaining
|
||||
saved-workflow work is platform preparation: load immutable child artifact
|
||||
versions, resolve their runtime dependencies, and supply prepared children to
|
||||
the already-implemented core runtime.
|
||||
execution, routed child interrupt resume, and platform loading of immutable
|
||||
saved-child artifact versions are implemented. Durable persisted resume is the
|
||||
remaining run-lifecycle concern and is specified separately.
|
||||
|
||||
## Goals
|
||||
|
||||
@@ -407,15 +406,14 @@ must be keyed by the parent subgraph dependency/use site, not only by child
|
||||
artifact id: one parent graph may intentionally invoke the same immutable
|
||||
child artifact twice against different accounts or capability bindings.
|
||||
|
||||
### Saved Child Interrupt Limitation
|
||||
### Saved Child Interrupt Status
|
||||
|
||||
Native prepared children can interrupt and resume in core. The current
|
||||
workflow-surface `run_deployment` entrypoint is still a one-shot execution
|
||||
call, however, and does not expose persisted platform resume for a saved run.
|
||||
Therefore this slice preserves the existing unrunnable behavior for saved
|
||||
artifacts containing interrupts, including interrupting descendant artifacts.
|
||||
The platform must report that diagnostic before execution rather than starting
|
||||
a run it cannot resume through its public surface.
|
||||
Native prepared children can interrupt and resume in core. The workflow surface
|
||||
now exposes process-local `run_deployment` / `resume_run` support for saved
|
||||
interrupting artifacts, including interrupts raised by saved descendants.
|
||||
That support does not survive process restart yet. Durable run persistence is
|
||||
specified separately in
|
||||
[`2026-05-26-durable-workflow-runs-and-resume-design.md`](2026-05-26-durable-workflow-runs-and-resume-design.md).
|
||||
|
||||
## Implementation Slices
|
||||
|
||||
@@ -454,18 +452,18 @@ a run it cannot resume through its public surface.
|
||||
- Tests: child interrupt pauses parent, resume continues child, parent completes,
|
||||
wrong resume target fails clearly.
|
||||
|
||||
### Slice 3: Saved Workflow References
|
||||
### Completed Slice 3: Saved Workflow References
|
||||
|
||||
- Structural saved-workflow references and conversion helpers already exist;
|
||||
this slice is execution resolution, not a new identity shape.
|
||||
- Add platform-level resolution for saved workflow artifacts.
|
||||
- Apply the parent deployment binding environment transitively to each exact
|
||||
- Structural saved-workflow references and conversion helpers identify exact
|
||||
saved child versions without runtime string parsing.
|
||||
- Platform-level resolution prepares saved workflow artifacts before runtime.
|
||||
- The parent deployment binding environment applies transitively to each exact
|
||||
saved child artifact version.
|
||||
- Validate dependencies, missing artifacts, saved-child cycles, and existing
|
||||
unsupported interrupt diagnostics before execution.
|
||||
- Tests: saved child workflow runs through a deployment binding, nested saved
|
||||
child dependencies use the same binding environment, and missing,
|
||||
cyclic, or interrupting child artifacts report an unrunnable dependency.
|
||||
- Dependencies, missing artifacts, and saved-child cycles validate before
|
||||
execution.
|
||||
- Tests cover saved child execution through deployment bindings, nested saved
|
||||
child dependencies, missing/cyclic diagnostics, and process-local
|
||||
interrupt/resume for saved children.
|
||||
|
||||
### Slice 4: Optional Policy Expansion
|
||||
|
||||
@@ -498,7 +496,7 @@ resume are complete. Implement Slice 3 in the platform layer: prepare saved
|
||||
non-interrupting child artifacts recursively under one deployment environment
|
||||
and pass those prepared dependencies into core execution.
|
||||
|
||||
Do not broaden saved interrupt support through `run_deployment` until the
|
||||
platform has a public persisted resume path. Do not delete wrapper-node
|
||||
helpers yet; they remain compatibility APIs while saved native execution
|
||||
matures.
|
||||
The process-local saved-interrupt execution path is complete. Next add durable
|
||||
run/checkpoint persistence so interrupted saved children survive process
|
||||
restart. Do not delete wrapper-node helpers yet; they remain compatibility APIs
|
||||
while saved native execution matures.
|
||||
|
||||
@@ -0,0 +1,392 @@
|
||||
# Durable Workflow Runs and Resume Design
|
||||
|
||||
Status: design approved for implementation planning
|
||||
|
||||
Durable workflow runs turn the current process-local `run_deployment` /
|
||||
`resume_run` behavior into platform state. The runtime already exposes the
|
||||
essential execution snapshot as `RunState`; the missing boundary is a durable
|
||||
run repository and a strict persisted codec for that state.
|
||||
|
||||
This design intentionally implements safe stopped-run persistence first. It
|
||||
does not promise replay of an in-flight external side effect or arbitrary
|
||||
automatic retries for failed MCP tool calls.
|
||||
|
||||
## Goals
|
||||
|
||||
- Persist workflow runs that stop as `interrupted`, `completed`, or `failed`.
|
||||
- Let an explicitly interrupted deployment resume after MCP server restart or
|
||||
from another process using a durable `run_id`.
|
||||
- Pin the artifact, deployment binding environment, and prepared dependency
|
||||
environment used by a run when it starts.
|
||||
- Revalidate pinned dependencies before a persisted interrupted run continues.
|
||||
- Provide compact run inspection and bounded trace retrieval suitable for MCP
|
||||
clients and a future dashboard.
|
||||
- Keep run persistence outside `wf_core` and separate from auth/catalog storage.
|
||||
- Leave room for finer-grained checkpoints without requiring them in v1.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not pause automatically when an MCP connection or tool call fails.
|
||||
- Do not implement automatic retry or timeout behavior in this slice.
|
||||
- Do not checkpoint after every node or scheduler tick in v1.
|
||||
- Do not add time travel, replay from arbitrary historical state, or state edits.
|
||||
- Do not add LangGraph-style cross-run memory available to arbitrary nodes.
|
||||
- Do not move artifact, deployment, auth, or catalog storage into one store.
|
||||
|
||||
## Current State
|
||||
|
||||
`wf_core.RunState` already owns the data necessary to continue a stopped run:
|
||||
|
||||
- workflow input, committed state, terminal outcome, and public output
|
||||
- trace entries
|
||||
- frames and ready-frame queue
|
||||
- runtime scopes and lineage writes
|
||||
- current execution cursor
|
||||
- error information
|
||||
- interrupt request and nested subgraph interrupt route
|
||||
|
||||
`wf_mcp.workflow_surface.handlers.ActiveWorkflowRun` currently retains the
|
||||
remaining platform context only in memory:
|
||||
|
||||
- `WorkflowDeployment`
|
||||
- `WorkflowArtifact`
|
||||
- `RawWorkflowPlan`
|
||||
- `RunState`
|
||||
|
||||
That class is the temporary seam to replace. It is not evidence that run state
|
||||
belongs inside MCP: MCP is merely one front door for a workflow platform
|
||||
concern.
|
||||
|
||||
## Relationship to LangGraph Persistence
|
||||
|
||||
LangGraph distinguishes execution checkpoints from a general memory store:
|
||||
|
||||
- A **checkpointer** stores execution snapshots so interrupted graphs can be
|
||||
inspected and resumed.
|
||||
- A **store** provides arbitrary memory shared across threads/runs.
|
||||
|
||||
This design needs the first concept only. A workflow run/checkpoint repository
|
||||
is appropriate now; cross-run node-accessible memory is separate future work.
|
||||
|
||||
## Platform Ownership
|
||||
|
||||
The conceptual ownership remains:
|
||||
|
||||
```text
|
||||
wf_core
|
||||
workflow execution models and runtime
|
||||
|
||||
wf_artifacts
|
||||
immutable workflow definitions and deployment contracts
|
||||
|
||||
wf_platform (future extraction target)
|
||||
runs, checkpoints, schedules, run history, UI/admin policy
|
||||
|
||||
wf_mcp
|
||||
MCP exposure of platform operations
|
||||
```
|
||||
|
||||
The current package graph already has `wf_artifacts` depending on `wf_platform`
|
||||
for capability/source refs. Placing typed run records in `wf_platform` while
|
||||
they snapshot `WorkflowArtifact` and `WorkflowDeployment` would introduce a
|
||||
cycle or weaken those fields into untyped dictionaries. V1 should therefore
|
||||
place run models/store code under a focused `wf_artifacts.runs` subpackage.
|
||||
Names and module boundaries must make later extraction straightforward if the
|
||||
artifact/platform dependency direction is reorganized. Do not expand
|
||||
`wf_mcp.storage.Store`, which owns MCP auth/catalog snapshots with a different
|
||||
lifecycle.
|
||||
|
||||
## Domain Model
|
||||
|
||||
### Workflow Run
|
||||
|
||||
A `WorkflowRun` identifies one execution attempt of one pinned deployment
|
||||
environment.
|
||||
|
||||
Required information:
|
||||
|
||||
```text
|
||||
WorkflowRun
|
||||
id: stable run id
|
||||
deployment_id
|
||||
artifact_id
|
||||
artifact_version
|
||||
status: interrupted | completed | failed
|
||||
resume_readiness: ready | blocked | not_applicable
|
||||
created_at
|
||||
updated_at
|
||||
latest_checkpoint_id
|
||||
pinned_environment
|
||||
diagnostics
|
||||
```
|
||||
|
||||
`unrunnable` is not a run status. It is a pre-start response when deployment
|
||||
dependency validation fails before an execution begins.
|
||||
|
||||
### Run Checkpoint
|
||||
|
||||
A `RunCheckpoint` is a stored stopped-state snapshot of a run.
|
||||
|
||||
```text
|
||||
RunCheckpoint
|
||||
id
|
||||
run_id
|
||||
sequence
|
||||
reason: interrupted | completed | failed
|
||||
created_at
|
||||
persisted_run_state
|
||||
```
|
||||
|
||||
V1 writes a checkpoint only when a public run operation stops:
|
||||
|
||||
- `run_deployment` returns an interrupted, completed, or failed execution.
|
||||
- `resume_run` returns an interrupted, completed, or failed execution.
|
||||
|
||||
V1 does not guarantee recovery after a process crash while a node or external
|
||||
tool call is in progress.
|
||||
|
||||
### Pinned Execution Environment
|
||||
|
||||
Runs must not reinterpret bindings when resumed. The persisted run pins:
|
||||
|
||||
- root artifact snapshot, in addition to its exact id/version
|
||||
- deployment snapshot, including the binding selection used at start
|
||||
- resolved saved child artifact snapshots, in addition to exact child versions
|
||||
- dependency contract information needed to revalidate those bindings
|
||||
|
||||
A configuration edit after the run starts does not silently redirect a paused
|
||||
run from one MCP account/source to another. If a future repair/migration flow
|
||||
allows rebinding a stopped run, that must be explicit and auditable.
|
||||
|
||||
Snapshotting artifact contents is required by the current implementation:
|
||||
`FileWorkflowArtifactStore` stores versioned paths but does not yet reject an
|
||||
overwrite of an existing version. A durable run must not depend on a later
|
||||
filesystem overwrite preserving the graph it originally executed. A future
|
||||
store that enforces immutable artifact writes may still retain snapshots as an
|
||||
audit record.
|
||||
|
||||
## Persisted Runtime Codec
|
||||
|
||||
`RunState.to_dict()` is useful for inspection but is not by itself a durable
|
||||
storage contract. Nested dataclasses, enums, path types, reducer refs, workflow
|
||||
refs, scopes, lineages, and interrupt routes must round-trip through a
|
||||
validated persisted model or codec.
|
||||
|
||||
The storage codec must:
|
||||
|
||||
- serialize the full stopped `RunState` needed for resume
|
||||
- restore equivalent enum/path/ref/dataclass values rather than loose dicts
|
||||
- reject corrupted or unsupported stored state with a clear diagnostic
|
||||
- be versioned or structured so later checkpoint migrations are possible
|
||||
- contain JSON-compatible values only at the persistence boundary
|
||||
|
||||
Core should own the execution-state codec shape because it understands
|
||||
`RunState`. The platform store should persist the codec output, not reconstruct
|
||||
runtime internals itself.
|
||||
|
||||
## Lifecycle Semantics
|
||||
|
||||
### Start
|
||||
|
||||
```text
|
||||
validate deployment and resolved dependencies
|
||||
if blocking diagnostics:
|
||||
return unrunnable response; do not create a run
|
||||
|
||||
pin resolved execution environment
|
||||
execute workflow
|
||||
persist WorkflowRun and stopped-state checkpoint
|
||||
return run summary
|
||||
```
|
||||
|
||||
Completed and failed runs are persisted as well as interrupted runs. This makes
|
||||
inspection and debugging useful immediately without materially increasing
|
||||
storage implementation complexity.
|
||||
|
||||
### Intentional Pause
|
||||
|
||||
Only a declared `InterruptNode` creates a resumable pause.
|
||||
|
||||
```text
|
||||
InterruptNode reached
|
||||
-> RunStatus.INTERRUPTED
|
||||
-> persist interrupted checkpoint
|
||||
-> resume_readiness = ready when pinned dependencies validate
|
||||
```
|
||||
|
||||
Interrupts raised within a native saved child subgraph retain the typed
|
||||
interrupt route already carried in `RunState`, so durable resume re-enters the
|
||||
original child scope.
|
||||
|
||||
### Resume
|
||||
|
||||
```text
|
||||
load run and latest interrupted checkpoint
|
||||
verify run is interrupted and resumable
|
||||
revalidate exact pinned dependency environment
|
||||
if dependency validation fails:
|
||||
keep run status interrupted
|
||||
set resume_readiness = blocked
|
||||
return dependency diagnostics without mutating execution state
|
||||
else:
|
||||
resume from restored RunState
|
||||
persist next stopped-state checkpoint
|
||||
```
|
||||
|
||||
When a missing or disabled pinned dependency returns, a later explicit
|
||||
`validate_run`, `inspect_run`, or `resume_run` may report it ready again.
|
||||
There is no background auto-resume.
|
||||
|
||||
### Failure
|
||||
|
||||
An ordinary runtime exception, MCP transport failure, or tool connection loss
|
||||
while execution is running fails the run.
|
||||
|
||||
This is deliberately distinct from interruption. An external tool may have
|
||||
completed a side effect before losing its response; automatically presenting
|
||||
that condition as safely resumable could duplicate effects.
|
||||
|
||||
```text
|
||||
tool/source failure during execution
|
||||
-> status = failed
|
||||
-> persist failed checkpoint and error details
|
||||
-> no automatic resume
|
||||
```
|
||||
|
||||
Recovery from such failures must later be declared in workflow semantics, such
|
||||
as explicit outcome mapping or a carefully specified retry policy.
|
||||
|
||||
## Dependency Validation Semantics
|
||||
|
||||
Dependency status has three positions:
|
||||
|
||||
| Point | Result |
|
||||
| --- | --- |
|
||||
| Before start | `unrunnable`; no execution run is created |
|
||||
| At a declared interrupt | `interrupted`; persisted and potentially resumable |
|
||||
| Before resume when pinned dependency is broken | run remains `interrupted`, `resume_readiness = blocked`, diagnostics returned |
|
||||
| During live execution | `failed`, unless future workflow logic explicitly models recovery |
|
||||
|
||||
Dependency diagnostics are platform/control-plane information, not graph
|
||||
outcomes. They must not be forced through a user's declared `ok` / `error`
|
||||
workflow outcome routes.
|
||||
|
||||
## Public Surface
|
||||
|
||||
The stable MCP workflow surface should eventually expose:
|
||||
|
||||
```text
|
||||
wf.workflow.run_deployment
|
||||
starts a run; always returns a stable run_id once execution starts
|
||||
|
||||
wf.workflow.resume_run
|
||||
resumes one durable interrupted run after pinned dependency validation
|
||||
|
||||
wf.workflow.inspect_run
|
||||
returns compact run status, readiness, terminal output/outcome,
|
||||
interrupt summary, diagnostics, and trace_count
|
||||
|
||||
wf.workflow.read_run_trace
|
||||
returns a bounded trace slice selected by range
|
||||
```
|
||||
|
||||
The response style stays progressive:
|
||||
|
||||
- start/resume returns compact stopped-state status
|
||||
- inspect returns detail needed for decision making
|
||||
- trace is fetched only in bounded slices
|
||||
|
||||
## Store Interface Direction
|
||||
|
||||
Use a dedicated protocol rather than expanding MCP auth/catalog persistence:
|
||||
|
||||
```text
|
||||
RunStore
|
||||
save_run(run)
|
||||
get_run(run_id)
|
||||
save_checkpoint(checkpoint)
|
||||
get_latest_checkpoint(run_id)
|
||||
list_runs(query, cursor, limit)
|
||||
read_checkpoints(run_id, cursor, limit)
|
||||
```
|
||||
|
||||
The initial backend may be file-based under the configured application store
|
||||
root:
|
||||
|
||||
```text
|
||||
.wf_mcp_store/
|
||||
auth/
|
||||
catalog/
|
||||
workflows/
|
||||
deployments/
|
||||
runs/
|
||||
<run_id>/
|
||||
run.json
|
||||
checkpoints/
|
||||
000001.json
|
||||
000002.json
|
||||
```
|
||||
|
||||
The protocol must allow later SQLite/Postgres storage without changing the MCP
|
||||
surface or core runtime model.
|
||||
|
||||
## Retry and Timeout Fields
|
||||
|
||||
The codebase already carries `retry` and `timeout_seconds` fields in node/draft
|
||||
models, plus a runtime `retry_count` context field. They are not currently an
|
||||
implemented runtime policy.
|
||||
|
||||
For durable-run v1:
|
||||
|
||||
- treat these fields as declared-but-unsupported behavior
|
||||
- do not interpret failed MCP calls as retryable
|
||||
- do not add recovery checkpoints around external calls
|
||||
- document clearly when a future slice implements retry/timeout semantics
|
||||
|
||||
A future retry design must distinguish known-not-executed failures from
|
||||
unknown-side-effect failures, or require idempotency/author-provided policy.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
Implementation tests must cover:
|
||||
|
||||
- persisted `RunState` round-trip for root interrupts
|
||||
- persisted `RunState` round-trip for native child-subgraph interrupts
|
||||
- a completed run saved with output and terminal outcome
|
||||
- a failed run saved with error details
|
||||
- server/service recreation followed by successful `resume_run`
|
||||
- pinned dependency disabled after pause: resume is blocked and state remains
|
||||
interrupted and unmodified
|
||||
- pinned dependency restored: later resume succeeds
|
||||
- changed live configuration cannot silently redirect pinned source bindings
|
||||
- transport failure during execution produces failed, not interrupted
|
||||
- trace retrieval is ranged and reports total trace count
|
||||
|
||||
## Future Expansion
|
||||
|
||||
Once stopped-run persistence is stable:
|
||||
|
||||
1. Add checkpoint writes at explicit scheduler-safe boundaries if crash
|
||||
recovery during long execution is needed.
|
||||
2. Investigate MCP tasks/progress reporting for live long-running operations.
|
||||
3. Design retry/timeout semantics explicitly rather than activating dormant
|
||||
fields casually.
|
||||
4. Consider replay/time-travel only after external side-effect semantics are
|
||||
documented.
|
||||
5. Add general cross-run memory separately if nodes need it; it is not a
|
||||
replacement for checkpoints.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Implement durable stopped-run snapshots first:
|
||||
|
||||
1. Add a validated persisted `RunState` codec.
|
||||
2. Add `WorkflowRun`, `RunCheckpoint`, and `RunStore`.
|
||||
3. Save interrupted, completed, and failed runs after start/resume returns.
|
||||
4. Replace process-local `_active_runs` lookup with durable run retrieval.
|
||||
5. Add compact `inspect_run` and bounded `read_run_trace`.
|
||||
6. Revalidate pinned dependencies before resuming interrupted runs.
|
||||
|
||||
This delivers durable human-in-the-loop execution and run inspection while
|
||||
preserving the correctness boundary that live external-call failures are not
|
||||
safe implicit pauses.
|
||||
@@ -89,8 +89,10 @@ deployment surface, including interrupts raised inside saved child workflows.
|
||||
This support is in-memory only: server restart, reload that replaces process
|
||||
state, or another frontend process invalidates the `run_id`.
|
||||
|
||||
Future run history should replace process-local `run_id` values with durable
|
||||
run records. The likely shape is:
|
||||
Durable run history is specified in
|
||||
[`2026-05-26-durable-workflow-runs-and-resume-design.md`](superpowers/specs/2026-05-26-durable-workflow-runs-and-resume-design.md).
|
||||
It should replace process-local `run_id` values with durable run records. The
|
||||
planned surface is:
|
||||
|
||||
- `run_deployment` starts or completes a run and returns `run_id`
|
||||
- `inspect_run(run_id)` returns status, output, diagnostics, and trace metadata
|
||||
@@ -685,9 +687,11 @@ parent trace sees one node call; in the native examples child trace entries
|
||||
remain in the parent run state.
|
||||
|
||||
Persisted resume is still not implemented. In-memory resume works because the
|
||||
server keeps the paused `RunState`; a durable run store will need to snapshot
|
||||
the root workflow, prepared child dependencies, deployment bindings, and trace
|
||||
metadata before this can survive restart or move across processes.
|
||||
server keeps the paused `RunState`; the durable run design persists stopped
|
||||
snapshots for interrupted, completed, and failed runs and pins root workflow,
|
||||
prepared child dependencies, deployment bindings, and trace metadata before a
|
||||
paused run can survive restart or move across processes. Only declared
|
||||
interrupts become resumable pauses; live tool/source failures remain failures.
|
||||
|
||||
Blocking dependency failures happen before workflow execution and are not normal
|
||||
workflow outcomes. A missing source, disabled source, unresolved binding, or
|
||||
|
||||
Reference in New Issue
Block a user