docs: harden thesis architecture and evidence claims
This commit is contained in:
+65
-13
@@ -21,6 +21,25 @@ flowchart LR
|
||||
Sources --> Python[Python Sources]
|
||||
```
|
||||
|
||||
## Layered Package Boundary
|
||||
|
||||
> Package ownership view, not a runtime call graph.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
CLI[wf_cli] --> Transport[wf_transport_rpc_http]
|
||||
Transport --> Server[wf_server]
|
||||
Server --> API[wf_api]
|
||||
API --> Artifacts[wf_artifacts]
|
||||
API --> Core[wf_core]
|
||||
API --> Platform[wf_platform]
|
||||
Server --> MCP[wf_sources_mcp]
|
||||
Server --> Python[wf_sources_python]
|
||||
MCP --> Platform
|
||||
Python --> Platform
|
||||
Artifacts --> Platform
|
||||
```
|
||||
|
||||
## Workflow Lifecycle
|
||||
|
||||
> Simplified view. For the detailed platform-domain version with source inventory
|
||||
@@ -54,17 +73,32 @@ flowchart LR
|
||||
## Workflow Core
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Input[Input Schema] --> NodeUse[Node Use]
|
||||
NodeSpec[NodeSpec Contract] --> NodeUse
|
||||
NodeUse --> Outcome[Declared Outcome]
|
||||
Outcome --> Edge[Graph Edge]
|
||||
NodeUse --> StateWrite[State Write]
|
||||
StateWrite --> Reducer[Reducer]
|
||||
Reducer --> State[Workflow State]
|
||||
NodeUse --> Trace[Trace Frame]
|
||||
Interrupt[Interrupt] --> RunState[Stopped Run State]
|
||||
RunState --> Resume[Resume]
|
||||
flowchart TD
|
||||
Start[Prepare Run] --> ValidateInput[Validate Workflow Input]
|
||||
ValidateInput --> Select[Select Ready Frame]
|
||||
Select --> Step{Step Type}
|
||||
Step --> Node[NodeUse]
|
||||
Step --> Condition[Condition]
|
||||
Step --> Foreach[Foreach]
|
||||
Step --> Subgraph[Subgraph]
|
||||
Step --> Interrupt[Interrupt]
|
||||
Step --> Join[Join / Minimal Done Step]
|
||||
Step --> End[End]
|
||||
Node --> ResolveInput[Resolve Input Bindings]
|
||||
ResolveInput --> Handler[Invoke NodeDef Handler]
|
||||
Handler --> CheckOutcome[Check Declared Outcome]
|
||||
CheckOutcome --> StatePatch[Build Reducer-Aware State Patch]
|
||||
StatePatch --> Trace[Append Trace Frame]
|
||||
Condition --> Trace
|
||||
Foreach --> Trace
|
||||
Subgraph --> Trace
|
||||
Join --> Trace
|
||||
Trace --> Route[Route By Outcome Edge]
|
||||
Interrupt --> Stop[Persist Interrupt Request]
|
||||
End --> Finalize[Project Workflow Output]
|
||||
Route --> Select
|
||||
Stop --> Resume[Resume Payload + Outcome]
|
||||
Resume --> Route
|
||||
```
|
||||
|
||||
## Platform Domain
|
||||
@@ -79,10 +113,29 @@ flowchart LR
|
||||
DeploymentValidation --> Run[Workflow Run]
|
||||
Run --> RunRecord[Run Record]
|
||||
Run --> Trace[Trace Slice]
|
||||
RunRecord --> Resume[Resume]
|
||||
RunRecord --> Resume[Resume If Interrupted / Stopped]
|
||||
DeploymentValidation --> Diagnostics[Repairable Diagnostics]
|
||||
```
|
||||
|
||||
## Workflow API Lifecycle Cohesion
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Context[WorkflowOperationContext] --> API[WorkflowApi]
|
||||
API --> Caps[Capability API]
|
||||
API --> Drafts[Draft API]
|
||||
API --> Artifacts[Artifact API]
|
||||
API --> Deployments[Deployment API]
|
||||
API --> Runs[Run API]
|
||||
Caps --> Specs[WorkflowSpecProvider]
|
||||
Drafts --> DraftStore[Draft Store]
|
||||
Artifacts --> ArtifactStore[Artifact Store]
|
||||
Deployments --> ArtifactStore
|
||||
Runs --> RunStore[Run Store]
|
||||
Runs --> Runtime[WorkflowRuntimeRunner]
|
||||
Runs --> Live[Optional Live Source Checker]
|
||||
```
|
||||
|
||||
## Source Provider Boundary
|
||||
|
||||
```mermaid
|
||||
@@ -97,4 +150,3 @@ flowchart LR
|
||||
Inventory --> API[Workflow API Surface]
|
||||
API --> Runtime[Workflow Runtime]
|
||||
```
|
||||
|
||||
|
||||
@@ -93,11 +93,10 @@ deterministic report-preparation case study backed by a Python source. Section 8
|
||||
evaluates the implementation against concrete evidence. Sections 9 and 10
|
||||
discuss limitations and future work. Section 11 concludes.
|
||||
|
||||
|
||||
# Problem Statement And Requirements
|
||||
|
||||
Current agent/tool systems frequently let an LLM directly orchestrate side
|
||||
effects through ad hoc tool calls. This pattern creates several practical
|
||||
Many direct tool-calling agent patterns let an LLM orchestrate side effects
|
||||
through sequential tool calls. This pattern can create several practical
|
||||
problems for workspace automation:
|
||||
|
||||
- **Weak validation before execution.** A planner that assembles tool-call
|
||||
@@ -137,15 +136,17 @@ responsibilities of the planner.
|
||||
The design requirements that follow from this problem statement are:
|
||||
|
||||
1. Typed workflow artifact, deployment, and run lifecycle with explicit schemas.
|
||||
2. Source-provider boundary supporting MCP, Python, and future source families.
|
||||
3. Durable server, API, and CLI surface that external agents can drive.
|
||||
2. Source-provider boundary implemented for built-in, MCP, and Python sources,
|
||||
and designed to admit future source families that can be projected into the
|
||||
existing capability/source contract.
|
||||
3. Server, API, and CLI surfaces that external agents can drive, backed by
|
||||
durable lifecycle stores.
|
||||
4. Validation and inspection mechanisms intended to reduce planner
|
||||
trial-and-error.
|
||||
5. Next-action guidance that points an agent toward useful lifecycle operations
|
||||
without replacing validation.
|
||||
6. Deterministic execution for the thesis-critical evidence path.
|
||||
|
||||
|
||||
# Positioning And Related Systems
|
||||
|
||||
The system occupies a specific position in the automation landscape. It does
|
||||
@@ -199,7 +200,6 @@ source-provider correctness matters, because a source may require persistent
|
||||
sessions, auth context, catalog refresh, and prompt inventory. The platform
|
||||
places this complexity behind a neutral `CapabilitySource` interface.
|
||||
|
||||
|
||||
# Conceptual Model
|
||||
|
||||
## Working Glossary
|
||||
@@ -207,32 +207,45 @@ places this complexity behind a neutral `CapabilitySource` interface.
|
||||
The document uses these terms with specific meanings:
|
||||
|
||||
| Term | Meaning | Example |
|
||||
| --- | --- | --- |
|
||||
| --- | ----- | --- |
|
||||
| Workflow capability | A workflow-facing callable operation exposed by a source. | `local.report.extract_report` |
|
||||
| `NodeSpec` | The typed contract for a capability: input schema, output schema, and outcomes. | a Python `@node` projection |
|
||||
| `NodeSpec` | The authoring-layer typed contract produced by decorators or source adapters. | a Python `@node` projection |
|
||||
| `NodeDef` | The core-level serializable node contract: input schema, output schema, and declared outcomes. | a workflow plan node definition |
|
||||
| Source | A namespace and owner of capabilities, resources, prompts, and metadata. | `local.report`, `wf.std` |
|
||||
| Source family | A class of source implementations. | built-in, MCP, Python |
|
||||
| Source provider | Server-side code that loads or manages sources for a source family. | Python source loading |
|
||||
| Tool | A provider-native operation before projection into workflow form. | MCP tool |
|
||||
| Outcome | A control-flow label returned by a node and consumed by graph edges. | `ok`, `error`, `submitted` |
|
||||
| Output | The data payload returned by a node or workflow. | `{ "report": "..." }` |
|
||||
| Reducer | A pure state-merge operation selected by state schema. | `wf.std.replace`, `wf.std.append` |
|
||||
| Platform source | A process-provided source with fixed identity and no deployment binding. | `wf.std`, `wf.source` |
|
||||
| Deployment binding | A mapping from logical workflow source requirement to concrete source id. | `local.report=local.report` |
|
||||
| Source drift | Divergence between saved workflow requirements and the currently resolved source inventory. | missing capability or changed schema |
|
||||
|
||||
## Workflows as Typed Graphs
|
||||
|
||||
A workflow is a typed graph where nodes invoke named capabilities and edges
|
||||
route by declared outcomes. The graph model is defined by four schema contracts:
|
||||
A workflow is an outcome-routed typed graph. It is not presented here as a
|
||||
complete general DAG engine, and it is not a free-form agent state machine.
|
||||
Nodes invoke named capabilities; edges route by declared node outcomes. The
|
||||
graph model is defined by four schema contracts:
|
||||
|
||||
- `input_schema`: validates run input.
|
||||
- `state_schema`: defines workflow memory and reducer behavior.
|
||||
- `output_schema`: defines the final result shape.
|
||||
- Outcome declarations: route control flow through graph edges.
|
||||
|
||||
Each node references a `NodeSpec`---a typed contract describing input schema,
|
||||
output schema, and declared outcomes. Reducers merge repeated or otherwise
|
||||
multi-writer state updates safely. General fork/gather parallelism is future
|
||||
work, so this report does not claim complete concurrent graph semantics.
|
||||
Interrupts represent typed external input points. Subgraphs compose workflows as
|
||||
nodes.
|
||||
Each node references a core `NodeDef`---a serializable contract describing input
|
||||
schema, output schema, and declared outcomes. Source families commonly produce
|
||||
authoring-layer `NodeSpec`s first; those are projected into `NodeDef` contracts
|
||||
before the core executes a workflow. The validator checks that routed outcomes
|
||||
are declared, that a source node does not have duplicate edges for the same
|
||||
outcome, and that reachable outcome edges are present. Reducers merge state
|
||||
writes according to state-field declarations. Reducers are pure deterministic
|
||||
merge functions invoked by the runtime in workflow execution order; this report
|
||||
does not claim CRDT semantics, arbitrary concurrent writes, or order-independent
|
||||
aggregation. General fork/gather parallelism is future work, so this report
|
||||
does not claim complete concurrent graph semantics. Interrupts represent typed
|
||||
external input points. Subgraphs compose workflows as nodes.
|
||||
|
||||
The graph model improves the safety posture by making automation structure
|
||||
explicit. Node contracts, source requirements, state writes, outcomes,
|
||||
@@ -274,15 +287,18 @@ execution are distinct operations with distinct lifecycle affordances.
|
||||
|
||||
## Source Model
|
||||
|
||||
The common boundary is `CapabilitySource`. The runtime does not care where a
|
||||
`NodeSpec` came from; source-specific behavior belongs in provider packages and
|
||||
server composition.
|
||||
The common boundary is `CapabilitySource`. Source inventory can expose
|
||||
provider-derived `NodeSpec`s, reducers, resources, and prompts, but the core
|
||||
runtime ultimately executes serialized `NodeDef` contracts and handler
|
||||
functions. Source-specific behavior belongs in provider packages and server
|
||||
composition.
|
||||
|
||||
Four source families exist today:
|
||||
Representative sources and source families today:
|
||||
|
||||
| Source | Kind | Role |
|
||||
| --- | --- | --- |
|
||||
| `wf.std` | `system` | Built-in workflow nodes and reducers |
|
||||
| `wf.source` | `system` | Built-in source resource helper |
|
||||
| `wf.recipes` | `system` | First-party workflow recipes |
|
||||
| MCP sources | `connection` | Upstream MCP tools, resources, prompts |
|
||||
| Python sources | `python` | Trusted project-local `NodeSpec` registries |
|
||||
@@ -308,8 +324,9 @@ Python sources.
|
||||
|
||||
Source resolution follows a deterministic path: a logical source requirement in
|
||||
a workflow is checked against platform sources first, then resolved through
|
||||
deployment bindings to concrete sources. The runtime then delegates to the
|
||||
appropriate source handler.
|
||||
deployment bindings to concrete sources. Platform source IDs have fixed runtime
|
||||
identity: deployment validation rejects explicit bindings for platform sources.
|
||||
The runtime then delegates to the appropriate source handler.
|
||||
|
||||
## Source Resolution Path
|
||||
|
||||
@@ -327,7 +344,6 @@ This design allows workflow portability across environments: the same artifact
|
||||
can be deployed with different concrete source bindings, while the workflow
|
||||
graph references logical names only.
|
||||
|
||||
|
||||
# System Architecture
|
||||
|
||||
The architecture is organized into layered boundaries, each with a distinct
|
||||
@@ -353,13 +369,31 @@ flowchart LR
|
||||
Sources --> Python[Python Sources]
|
||||
```
|
||||
|
||||
The diagram shows the primary flow: a workflow owner expresses intent to an
|
||||
external LLM agent, which drives the `wf` CLI. The CLI communicates over
|
||||
JSON-RPC transport to a `WorkflowServer`. The server exposes a Workflow API
|
||||
Surface that delegates to the deterministic Workflow Core for graph execution and
|
||||
to platform domain objects for artifact, deployment, and run management. Source
|
||||
providers supply capabilities to the API surface without the core runtime being
|
||||
aware of provider-specific details.
|
||||
The diagram shows the primary flow from workflow owner through agent, CLI,
|
||||
transport, and server to the API surface, core, platform stores, and source
|
||||
providers. The server composes configured sources into a unified inventory
|
||||
without the core runtime being aware of provider-specific details.
|
||||
|
||||
## Layered Package Boundary
|
||||
|
||||
Unlike the previous runtime-call diagram, this one maps architectural
|
||||
responsibilities onto repository packages. It answers: which package owns each
|
||||
boundary in the current implementation?
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
CLI[wf_cli] --> Transport[wf_transport_rpc_http]
|
||||
Transport --> Server[wf_server]
|
||||
Server --> API[wf_api]
|
||||
API --> Artifacts[wf_artifacts]
|
||||
API --> Core[wf_core]
|
||||
API --> Platform[wf_platform]
|
||||
Server --> MCP[wf_sources_mcp]
|
||||
Server --> Python[wf_sources_python]
|
||||
MCP --> Platform
|
||||
Python --> Platform
|
||||
Artifacts --> Platform
|
||||
```
|
||||
|
||||
## Layer Responsibilities
|
||||
|
||||
@@ -371,10 +405,13 @@ The layered architecture separates concerns as follows:
|
||||
- **Workflow API Surface.** Application operations over capabilities, drafts,
|
||||
artifacts, deployments, and runs. The API surface consumes source DTOs through
|
||||
a neutral `WorkflowSpecProvider` and delegates to the core for execution.
|
||||
`WorkflowSpecProvider` is the API-facing reader over capability specs derived
|
||||
from source inventory; it is distinct from `WorkflowSourceProvider`, which
|
||||
loads source inventory into the server.
|
||||
|
||||
- **Platform Domain.** Draft workspaces, workflow artifacts, deployments, run
|
||||
records, source inventory snapshots, validation diagnostics, and next-action
|
||||
guidance.
|
||||
- **Platform Records And Policies.** Draft workspaces, workflow artifacts,
|
||||
deployments, run records, source inventory snapshots, validation diagnostics,
|
||||
and next-action guidance.
|
||||
|
||||
- **Server Composition.** `WorkflowServer` assembles concrete stores, sources,
|
||||
runtimes, and admin surfaces into a long-lived service. The server composes
|
||||
@@ -384,9 +421,10 @@ The layered architecture separates concerns as follows:
|
||||
The transport is protocol-neutral; the Workflow API Surface is the stable
|
||||
boundary.
|
||||
|
||||
- **Source Providers.** MCP, Python, and future source families project their
|
||||
inventory into `CapabilitySource` objects. Provider-specific behavior such as
|
||||
MCP session pools or Python module loading stays within the provider package.
|
||||
- **Source Providers.** Built-in, MCP, and Python providers project their
|
||||
inventory into `CapabilitySource` objects. The boundary is designed to admit
|
||||
future source families. Provider-specific behavior such as MCP session pools
|
||||
or Python module loading stays within the provider package.
|
||||
|
||||
## Workflow Lifecycle
|
||||
|
||||
@@ -414,6 +452,8 @@ divergence between saved artifact capability requirements and the currently
|
||||
resolved source inventory: missing bindings, missing or disabled sources,
|
||||
missing capabilities, or changed schema contracts. Run execution produces
|
||||
persisted records with trace slices and resumable stopped state.
|
||||
Only interrupted or explicitly stopped runs enter the resume path; completed
|
||||
and failed runs remain inspectable records.
|
||||
|
||||
## Workflow Core Model
|
||||
|
||||
@@ -421,23 +461,60 @@ The core model processes graph execution through typed stages:
|
||||
this diagram answers what happens during one run at the graph-execution level.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Input[Input Schema] --> NodeUse[Node Use]
|
||||
NodeSpec[NodeSpec Contract] --> NodeUse
|
||||
NodeUse --> Outcome[Declared Outcome]
|
||||
Outcome --> Edge[Graph Edge]
|
||||
NodeUse --> StateWrite[State Write]
|
||||
StateWrite --> Reducer[Reducer]
|
||||
Reducer --> State[Workflow State]
|
||||
NodeUse --> Trace[Trace Frame]
|
||||
Interrupt[Interrupt] --> RunState[Stopped Run State]
|
||||
RunState --> Resume[Resume]
|
||||
flowchart TD
|
||||
Start[Prepare Run] --> ValidateInput[Validate Workflow Input]
|
||||
ValidateInput --> Select[Select Ready Frame]
|
||||
Select --> Step{Step Type}
|
||||
Step --> Node[NodeUse]
|
||||
Step --> Condition[Condition]
|
||||
Step --> Foreach[Foreach]
|
||||
Step --> Subgraph[Subgraph]
|
||||
Step --> Interrupt[Interrupt]
|
||||
Step --> Join[Join / Minimal Done Step]
|
||||
Step --> End[End]
|
||||
Node --> ResolveInput[Resolve Input Bindings]
|
||||
ResolveInput --> Handler[Invoke NodeDef Handler]
|
||||
Handler --> CheckOutcome[Check Declared Outcome]
|
||||
CheckOutcome --> StatePatch[Build Reducer-Aware State Patch]
|
||||
StatePatch --> Trace[Append Trace Frame]
|
||||
Condition --> Trace
|
||||
Foreach --> Trace
|
||||
Subgraph --> Trace
|
||||
Join --> Trace
|
||||
Trace --> Route[Route By Outcome Edge]
|
||||
Interrupt --> Stop[Persist Interrupt Request]
|
||||
End --> Finalize[Project Workflow Output]
|
||||
Route --> Select
|
||||
Stop --> Resume[Resume Payload + Outcome]
|
||||
Resume --> Route
|
||||
```
|
||||
|
||||
Input validation gates entry. Each node use invokes a `NodeSpec` and produces a
|
||||
declared outcome, a state write, and a trace frame. Outcomes route through graph
|
||||
edges. Reducers merge state writes safely. Interrupts produce stopped run state
|
||||
that can be resumed.
|
||||
Input validation gates entry. The runtime then repeatedly selects a ready frame
|
||||
and executes one step. A `NodeUse` resolves input bindings from workflow input,
|
||||
state, and context; invokes the handler for the selected `NodeDef`; checks that
|
||||
the returned outcome is declared; builds reducer-aware state writes; records a
|
||||
trace frame; and advances through the edge for that outcome. `Condition`,
|
||||
`foreach`, `subgraph`, `join`, `interrupt`, and `end` steps are explicit core
|
||||
model variants, not provider-specific hacks. `Join` is currently a minimal step
|
||||
that returns a `"done"` outcome; it reserves a graph-level concept for future
|
||||
fork/gather semantics.
|
||||
|
||||
`foreach` is implemented as an explicit runtime step with frame and lineage
|
||||
bookkeeping for iteration and state isolation. This report does not claim a
|
||||
general parallel fork/gather model or arbitrary concurrent reducer semantics.
|
||||
|
||||
Failure has three visible forms. Structural and dependency failures are
|
||||
reported before execution through validation diagnostics. Runtime execution
|
||||
failures set the run status to `failed` and store an error string. Business
|
||||
failures are modeled as ordinary declared outcomes only when the workflow
|
||||
author defines and routes those outcomes.
|
||||
|
||||
Interrupts are first-class stop points: an `InterruptNode` builds a typed
|
||||
request payload, stores an `InterruptRequest` on the run state, and marks the
|
||||
run interrupted. Resume supplies a payload and resume outcome; resume bindings
|
||||
write the payload back into state, and routing continues from the declared
|
||||
resume outcome. This is resumability at explicit boundaries, not arbitrary
|
||||
mid-handler checkpointing.
|
||||
|
||||
## Source Provider Boundary
|
||||
|
||||
@@ -463,7 +540,6 @@ declared in the workflow config. The server composes all sources into a unified
|
||||
`CapabilitySource` inventory that the workflow API surface consumes without
|
||||
provider-specific knowledge.
|
||||
|
||||
|
||||
# Implementation
|
||||
|
||||
## Package Structure
|
||||
@@ -473,14 +549,14 @@ The implementation is organized into focused packages with clear boundaries:
|
||||
| Package | Responsibility |
|
||||
| --- | --- |
|
||||
| `wf_core` | Deterministic workflow kernel: graph execution, state, outcomes, trace, resume |
|
||||
| `wf_authoring` | Authoring support: `NodeSpec`, drafts, wrappers, API surfaces, source providers |
|
||||
| `wf_authoring` | Authoring primitives: `NodeSpec`, `WorkflowBuilder`, DSL, reducer authoring, recipes |
|
||||
| `wf_platform` | Neutral source DTOs, source visibility, permissions, and policy |
|
||||
| `wf_artifacts` | Artifact, deployment, and run models; file-backed stores; validation |
|
||||
| `wf_api` | Application surface: capabilities, drafts, artifacts, deployments, runs |
|
||||
| `wf_server` | `WorkflowServer` composition from config, stores, and source providers |
|
||||
| `wf_transport_rpc_http` | JSON-RPC over HTTP transport for CLI and future clients |
|
||||
| `wf_sources_mcp` | MCP upstream source implementation and persistent runtime pool |
|
||||
| `wf_sources_python` | Trusted in-process Python source loading and `NodeSpec` projection |
|
||||
| `wf_sources_python` | Trusted in-process Python source loading and `NodeSpec`-to-`NodeDef` projection |
|
||||
| `wf_cli` | CLI commands driving the JSON-RPC transport |
|
||||
|
||||
(Evidence: `docs/source_architecture.md`, package boundaries in `src/`.)
|
||||
@@ -490,15 +566,20 @@ The implementation is organized into focused packages with clear boundaries:
|
||||
The workflow core implements deterministic execution semantics. It processes a
|
||||
typed graph definition, validates input against `input_schema`, executes the
|
||||
selected node use, routes by declared outcomes, applies reducers to state
|
||||
writes, and produces trace frames. The current public semantics are serial graph
|
||||
execution with explicit outcomes and interrupts; general fork/gather
|
||||
parallelism is future work. The core is provider-agnostic; it sees `NodeSpec`
|
||||
contracts, not source-specific implementations.
|
||||
writes, and produces trace frames. The current public semantics are
|
||||
outcome-routed graph execution with explicit condition, foreach, subgraph,
|
||||
join, interrupt, and end steps. The async runtime has internal frame and lineage
|
||||
machinery for foreach admission and state isolation, but this report does not
|
||||
claim a complete general fork/gather programming model. The core is
|
||||
provider-agnostic; it sees `NodeDef` contracts and handler functions, not
|
||||
source-specific implementations.
|
||||
|
||||
State writes go through reducers. The platform includes a built-in `wf.std`
|
||||
reducer family. Reducers are themselves `NodeSpec` contracts that can be
|
||||
composed into workflows. Interrupts produce stopped run state with a resumable
|
||||
checkpoint.
|
||||
State writes go through reducers. The platform includes built-in `wf.std`
|
||||
reducer definitions such as `replace`, `append`, `merge_object`, `add`,
|
||||
`set_union`, and `max`. Reducers are pure merge functions paired with
|
||||
inspectable `ReducerSpec` metadata; they are exposed in source inventory, but
|
||||
they are not ordinary executable node handlers. Interrupts produce stopped run
|
||||
state with a resumable checkpoint.
|
||||
|
||||
(Evidence: `src/wf_core/`.)
|
||||
|
||||
@@ -512,9 +593,53 @@ The platform domain defines the lifecycle objects as Pydantic models:
|
||||
contract.
|
||||
- Run records track execution status, diagnostics, output, and trace counts.
|
||||
|
||||
Source binding uses `SourceBinding` objects that map logical source names to
|
||||
concrete source identifiers. The `CapabilitySource` dataclass is the neutral
|
||||
DTO that all source providers project into.
|
||||
Source binding uses `SourceBinding` objects defined in `wf_artifacts.models`
|
||||
that map logical source names to concrete source identifiers. The
|
||||
`CapabilitySource` dataclass is the neutral DTO that all source providers
|
||||
project into.
|
||||
|
||||
The lifecycle models are stored in `wf_artifacts`; orchestration of lifecycle
|
||||
operations happens one layer above, in `wf_api`.
|
||||
|
||||
The API layer holds the lifecycle together rather than acting as thin CRUD over
|
||||
files. `WorkflowApi` composes capability, draft, artifact, deployment, and run
|
||||
sub-APIs from one `WorkflowOperationContext`. That context carries stores,
|
||||
event recording, source inventory, runtime execution, and optional live-source
|
||||
checks. This is why CLI, JSON-RPC, and future transports can share the same
|
||||
domain operations without importing source-provider internals.
|
||||
|
||||
`wf_platform` is intentionally smaller than the API layer. It owns stable
|
||||
neutral source vocabulary: `CapabilitySource`, source inventory snapshots,
|
||||
declarative visibility and permission metadata, source policy, source refs,
|
||||
capability refs, and schema hashes. These flags describe source behavior for
|
||||
inventory and validation surfaces; they are not an authorization or
|
||||
policy-enforcement layer. `wf_platform` should not grow into a dumping ground
|
||||
for stores, runtimes, or provider lifecycle. Those belong in `wf_api`,
|
||||
`wf_server`, or the specific `wf_sources_*` package.
|
||||
|
||||
The API lifecycle is deliberately centralized through one facade:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Context[WorkflowOperationContext] --> API[WorkflowApi]
|
||||
API --> Caps[Capability API]
|
||||
API --> Drafts[Draft API]
|
||||
API --> Artifacts[Artifact API]
|
||||
API --> Deployments[Deployment API]
|
||||
API --> Runs[Run API]
|
||||
Caps --> Specs[WorkflowSpecProvider]
|
||||
Drafts --> DraftStore[Draft Store]
|
||||
Artifacts --> ArtifactStore[Artifact Store]
|
||||
Deployments --> ArtifactStore
|
||||
Runs --> RunStore[Run Store]
|
||||
Runs --> Runtime[WorkflowRuntimeRunner]
|
||||
Runs --> Live[Optional Live Source Checker]
|
||||
```
|
||||
|
||||
The diagram is important because it shows the design contribution at the
|
||||
application layer: drafts, artifacts, deployments, and runs are not independent
|
||||
file operations. They share source inventory, stores, event recording, runtime
|
||||
execution, validation, and live-source checks through one operation context.
|
||||
|
||||
(Evidence: `src/wf_artifacts/models.py`, `src/wf_platform/sources.py`.)
|
||||
|
||||
@@ -534,10 +659,27 @@ severity, error code, logical source reference, repair hint, and the bound
|
||||
source. These diagnostics are designed for machine clients: an LLM agent can
|
||||
read the diagnostic and determine what to fix without blind probing.
|
||||
|
||||
For example, an invalid deployment binding can produce a diagnostic shaped like
|
||||
this:
|
||||
|
||||
```json
|
||||
{
|
||||
"severity": "error",
|
||||
"code": "binding_missing",
|
||||
"logical_ref": "local.report.extract_report",
|
||||
"bound_source": null,
|
||||
"message": "No binding exists for logical source 'local.report'.",
|
||||
"repair_hint": "Bind the logical source to a compatible concrete source."
|
||||
}
|
||||
```
|
||||
|
||||
Deployment validation also detects source drift. If a source changes
|
||||
incompatibly and a deployment becomes unrunnable, the system reports the
|
||||
diagnostic with a repair hint rather than silently executing against
|
||||
incompatible capabilities.
|
||||
incompatible capabilities. In this prototype, schema drift is detected through
|
||||
saved required-capability schema hashes compared with current source inventory
|
||||
hashes when both sides provide hashes; it does not attempt semantic
|
||||
backward-compatibility analysis.
|
||||
|
||||
(Evidence: `src/wf_artifacts/validation.py`, `tests/artifacts/test_validation.py`.)
|
||||
|
||||
@@ -548,10 +690,11 @@ object tells a machine client whether there is an obvious next workflow-surface
|
||||
tool call, what that tool is, and why. It is guidance, not authority: validation
|
||||
diagnostics and runtime status remain the source of truth.
|
||||
|
||||
The `NextActions` object includes `can_continue`, `recommended_next_tool`,
|
||||
`reason`, and `patch_examples` with concrete request payloads. This makes the
|
||||
surface agent-operable: an LLM agent can read the hint and execute the suggested
|
||||
operation without reconstructing the lifecycle state.
|
||||
The `NextActions` object includes `can_continue`, `can_save_now`,
|
||||
`recommended_next_tool`, `reason`, `patch_examples` with concrete request
|
||||
payloads, and `warnings`. This makes the surface agent-operable: an LLM agent
|
||||
can read the hint and execute the suggested operation without reconstructing
|
||||
the lifecycle state.
|
||||
|
||||
(Evidence: `src/wf_api/next_actions.py`.)
|
||||
|
||||
@@ -563,8 +706,9 @@ does not own workflow semantics; it delegates to `WorkflowApi` for application
|
||||
operations.
|
||||
|
||||
The config model specifies store configuration, transport endpoints, and source
|
||||
provider declarations. Sources are declared as discriminated unions by kind:
|
||||
`mcp`, `python`, or future kinds like `openapi`.
|
||||
provider declarations. The current config model includes implemented source
|
||||
kinds such as `mcp` and `python`; future kinds such as `openapi` would extend
|
||||
the same discriminated-union pattern.
|
||||
|
||||
(Evidence: `src/wf_server/config.py`.)
|
||||
|
||||
@@ -597,9 +741,9 @@ The MCP source provider manages:
|
||||
- A persistent session pool for stateful upstream operations.
|
||||
- MCP-to-workflow converters for tools, resources, and prompts.
|
||||
|
||||
The provider projects MCP tools into `NodeSpec` contracts, making them callable
|
||||
from workflow graphs through the same `CapabilitySource` boundary as Python or
|
||||
built-in sources.
|
||||
The provider projects MCP tools into `NodeSpec` contracts and corresponding
|
||||
core `NodeDef` contracts, making them callable from workflow graphs through the
|
||||
same `CapabilitySource` boundary as Python or built-in sources.
|
||||
|
||||
(Evidence: `src/wf_sources_mcp/`, `tests/wf_sources_mcp/test_runtime.py`,
|
||||
`tests/wf_transport_rpc_http/test_mcp_backed_server_rpc.py`.)
|
||||
@@ -617,17 +761,19 @@ PythonSourceConfig(path, module, registry)
|
||||
-> PythonSourceProvider
|
||||
-> import module
|
||||
-> load NodeSpec registry
|
||||
-> project specs into NodeDef contracts
|
||||
-> qualify specs under source id
|
||||
-> CapabilitySource(kind="python")
|
||||
```
|
||||
|
||||
Python sources are static at server startup. No hot reload is implemented yet.
|
||||
The provider imports the configured module, reads the named registry attribute,
|
||||
and projects each `NodeSpec` into the workflow capability inventory.
|
||||
projects each `NodeSpec` into the workflow capability inventory, and makes the
|
||||
corresponding `NodeDef` contract available to workflow plans. This keeps the
|
||||
core provider-agnostic.
|
||||
|
||||
(Evidence: `src/wf_sources_python/`, `tests/wf_sources_python/test_loader.py`.)
|
||||
|
||||
|
||||
# Case Study: Deterministic Report Workflow
|
||||
|
||||
The thesis case study is a document/report preparation workflow backed by local
|
||||
@@ -637,6 +783,11 @@ artifact saving, deployment validation, run execution, run inspection, and
|
||||
trace viewing. The case study is deterministic and does not require an LLM
|
||||
call, remote OAuth, or provider quota.
|
||||
|
||||
This case study evaluates lifecycle integration rather than graph
|
||||
expressiveness. Graph features such as interrupts, foreach, subgraphs, joins,
|
||||
and reducer behavior are covered by targeted tests and code evidence in the
|
||||
evaluation section.
|
||||
|
||||
## Case Study Components
|
||||
|
||||
The example bundle lives at
|
||||
@@ -775,8 +926,13 @@ programmatically:
|
||||
bindings, starts a run, and asserts that the run completes with the expected
|
||||
typed output.
|
||||
|
||||
(Evidence: `tests/examples/test_report_workflow_example.py`.)
|
||||
The thesis-critical run path therefore demonstrates the full lifecycle using a
|
||||
single deterministic extraction node. The surrounding Python source includes
|
||||
additional capabilities used for capability discovery and extensibility
|
||||
evidence; it is not evaluated as a multi-node read -> extract -> render
|
||||
pipeline in this artifact.
|
||||
|
||||
(Evidence: `tests/examples/test_report_workflow_example.py`.)
|
||||
|
||||
# Evaluation
|
||||
|
||||
@@ -791,7 +947,8 @@ The evaluation is organized around criteria derived from the research question:
|
||||
| Criterion | Question | Evidence Type |
|
||||
| --- | --- | --- |
|
||||
| Representation | Can workflow intent be represented as artifacts, deployments, and runs? | model/API tests |
|
||||
| Validation | Can invalid drafts, deployments, source bindings, and source drift be reported before or during execution? | validation/diagnostic tests |
|
||||
| Validation | Can invalid drafts, deployments, source bindings, and source drift be reported before execution? | validation/diagnostic tests |
|
||||
| Runtime observability | Can runtime failures be persisted as failed run records with inspectable error state? | run API tests |
|
||||
| Execution | Can a deterministic workflow execute through the same API/CLI lifecycle used by agents? | report-workflow case study |
|
||||
| Persistence | Are lifecycle records persisted, and can stopped/interrupted runs resume at defined boundaries? | run-store and resume tests |
|
||||
| Source extensibility | Can different source families expose capabilities without changing `wf_core`? | built-in, MCP, and Python source tests |
|
||||
@@ -819,6 +976,16 @@ the prototype outperforms mature automation products.
|
||||
|
||||
The evidence supporting the thesis claims includes:
|
||||
|
||||
| Claim | Evidence | What it asserts | Result |
|
||||
| --- | --- | --- | --- |
|
||||
| Deployment validation catches source drift | `tests/artifacts/test_validation.py` | Missing, disabled, or changed capabilities produce diagnostics | Pass in focused test suite |
|
||||
| Interrupted runs resume at explicit boundaries | `tests/wf_api/test_run_api.py` and resume-concurrency tests | Stopped run state is persisted and resumed through the run API | Pass in focused test suite |
|
||||
| Python source lifecycle works | `tests/examples/test_report_workflow_example.py` | Python capability -> artifact -> deployment -> run completes | Pass in focused test suite |
|
||||
| CLI and JSON-RPC share the API surface | `tests/wf_transport_rpc_http/` and `tests/wf_cli/` | Transport and CLI delegate to the same workflow operations | Pass in focused test suite |
|
||||
|
||||
The table summarizes repository evidence; it is not a substitute for rerunning
|
||||
the verification commands before final submission.
|
||||
|
||||
### Architecture And Code Walkthrough
|
||||
|
||||
The four-layer architecture (core, API surface, server composition, transport)
|
||||
@@ -884,8 +1051,8 @@ machine-readable responses.
|
||||
### Config Validation
|
||||
|
||||
Config validation catches import and path errors before server startup. This
|
||||
prevents the server from starting with broken source configurations, reducing
|
||||
blind agent retries.
|
||||
prevents the server from starting with broken source configurations and
|
||||
provides earlier, structured failure feedback.
|
||||
|
||||
(Evidence: `src/wf_config/`.)
|
||||
|
||||
@@ -954,7 +1121,6 @@ The implementation addresses these evaluation questions:
|
||||
deployment validation reports unrunnable state with diagnostics instead of
|
||||
silently executing against incompatible capabilities.
|
||||
|
||||
|
||||
# Limitations
|
||||
|
||||
The following limitations are stated explicitly to maintain credibility and
|
||||
@@ -1017,7 +1183,6 @@ deployment concerns beyond the controlled system-design evidence in this report.
|
||||
- **No approval, roles, policy, or multi-user review.** There is no
|
||||
role-based access control or review workflow.
|
||||
|
||||
|
||||
# Future Work
|
||||
|
||||
The following areas are identified as likely future work:
|
||||
@@ -1058,7 +1223,6 @@ The following areas are identified as likely future work:
|
||||
benchmarks, and broader agent evaluation with more attempts and failure
|
||||
categories.
|
||||
|
||||
|
||||
# Conclusion
|
||||
|
||||
External LLM agents can author and operate workflows, but reusable workflow
|
||||
@@ -1071,8 +1235,10 @@ The implementation supports five claims:
|
||||
|
||||
1. A typed artifact, deployment, and run lifecycle provides persisted workflow
|
||||
records and resumability at explicit stopped/interrupted boundaries.
|
||||
2. The source-provider boundary allows MCP, Python, and future source families
|
||||
to share one workflow surface without core-runtime changes.
|
||||
2. The source-provider boundary lets built-in, MCP, and Python sources share
|
||||
one workflow surface, and is designed to admit future source families that
|
||||
can be projected into the existing capability/source contract without
|
||||
core-runtime changes.
|
||||
3. Validation and diagnostics produce machine-readable repairable failure
|
||||
states intended to reduce planner trial-and-error.
|
||||
4. The CLI and JSON-RPC transport provide an agent-operable surface that
|
||||
@@ -1086,7 +1252,6 @@ demonstrates the architecture; the thesis contribution is the platform design
|
||||
and evidence that the design can work across multiple source families under
|
||||
controlled conditions.
|
||||
|
||||
|
||||
# Appendices
|
||||
|
||||
## Appendix A: Case Study Command Transcript
|
||||
@@ -1115,49 +1280,63 @@ uv run wf --config examples/report_workflow/wf.config.json status
|
||||
### Capability Discovery
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json cap list --source local.report
|
||||
uv run wf --config examples/report_workflow/wf.config.json `
|
||||
cap list --source local.report
|
||||
```
|
||||
|
||||
### Capability Call
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json cap call local.report.extract_report --input-file examples/report_workflow/cap-input.json --format compact
|
||||
uv run wf --config examples/report_workflow/wf.config.json `
|
||||
cap call local.report.extract_report `
|
||||
--input-file examples/report_workflow/cap-input.json --format compact
|
||||
```
|
||||
|
||||
### Draft Creation
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json draft create-from-capability report_ws local.report.extract_report --name report_case_study --title "Report Case Study"
|
||||
uv run wf --config examples/report_workflow/wf.config.json `
|
||||
draft create-from-capability report_ws local.report.extract_report `
|
||||
--name report_case_study --title "Report Case Study"
|
||||
```
|
||||
|
||||
### Draft Validation
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json draft validate report_ws
|
||||
uv run wf --config examples/report_workflow/wf.config.json `
|
||||
draft validate report_ws
|
||||
```
|
||||
|
||||
### Artifact Saving
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json draft save report_ws --artifact report_case_study --version 1 --title "Report Case Study" --binding local.report=local.report
|
||||
uv run wf --config examples/report_workflow/wf.config.json `
|
||||
draft save report_ws --artifact report_case_study --version 1 `
|
||||
--title "Report Case Study" --binding local.report=local.report
|
||||
```
|
||||
|
||||
### Deployment Saving
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json deploy save report_case_study.default --artifact report_case_study --version 1 --binding local.report=local.report
|
||||
uv run wf --config examples/report_workflow/wf.config.json `
|
||||
deploy save report_case_study.default --artifact report_case_study `
|
||||
--version 1 --binding local.report=local.report
|
||||
```
|
||||
|
||||
### Deployment Validation
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json deploy validate report_case_study.default
|
||||
uv run wf --config examples/report_workflow/wf.config.json `
|
||||
deploy validate report_case_study.default
|
||||
```
|
||||
|
||||
### Run Execution
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json run start report_case_study.default --input-file examples/report_workflow/run-input.json --trace-from 0 --trace-limit 5
|
||||
uv run wf --config examples/report_workflow/wf.config.json `
|
||||
run start report_case_study.default `
|
||||
--input-file examples/report_workflow/run-input.json `
|
||||
--trace-from 0 --trace-limit 5
|
||||
```
|
||||
|
||||
### Run Inspection
|
||||
@@ -1170,7 +1349,8 @@ uv run wf --config examples/report_workflow/wf.config.json run inspect <run_id>
|
||||
### Run Trace
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json run trace <run_id> --from 0 --limit 5
|
||||
uv run wf --config examples/report_workflow/wf.config.json `
|
||||
run trace <run_id> --from 0 --limit 5
|
||||
```
|
||||
|
||||
## Appendix B: Evidence Index
|
||||
@@ -1178,7 +1358,7 @@ uv run wf --config examples/report_workflow/wf.config.json run trace <run_id> --
|
||||
See [evidence-index.md](evidence-index.md) for the full claim-to-evidence map.
|
||||
|
||||
| Claim | Evidence |
|
||||
| --- | --- |
|
||||
| --- | ----- |
|
||||
| Workflow lifecycle models | `src/wf_artifacts/models.py`, `src/wf_artifacts/runs/` |
|
||||
| Source provider boundary | `src/wf_platform/sources.py`, `src/wf_server/config.py` |
|
||||
| JSON-RPC/CLI surface | `src/wf_transport_rpc_http/`, `src/wf_cli/` |
|
||||
|
||||
Reference in New Issue
Block a user