1644 lines
74 KiB
Markdown
1644 lines
74 KiB
Markdown
---
|
|
title: "Design and Implementation of lda.chat: An AI Agent for Automating and Creating Workspace Workflows"
|
|
subtitle: ""
|
|
date: "July 1, 2026"
|
|
lang: "en-US"
|
|
documentclass: report
|
|
papersize: a4
|
|
fontsize: 10pt
|
|
toc: true
|
|
toc-depth: 2
|
|
lof: true
|
|
lot: true
|
|
numbersections: true
|
|
bibliography: references.bib
|
|
link-citations: true
|
|
figureTitle: "Figure"
|
|
figPrefix: "Figure"
|
|
chapters: true
|
|
appendix: true
|
|
syntax-highlighting: idiomatic
|
|
geometry:
|
|
- top=30mm
|
|
- bottom=30mm
|
|
- left=32mm
|
|
- right=32mm
|
|
mainfont: "Libertinus Serif"
|
|
sansfont: "Libertinus Sans"
|
|
monofont: "Libertinus Mono"
|
|
mathfont: "Libertinus Math"
|
|
colorlinks: true
|
|
linkcolor: "MidnightBlue"
|
|
urlcolor: "MidnightBlue"
|
|
toccolor: "MidnightBlue"
|
|
keywords:
|
|
- workflow
|
|
- agents
|
|
- source providers
|
|
- JSON-RPC
|
|
- MCP
|
|
- Python sources
|
|
header-includes:
|
|
# you can not specify -H and this at the same time.
|
|
diagram:
|
|
engine:
|
|
mermaid:
|
|
theme: neutral
|
|
---
|
|
|
|
# List of Abbreviations {.unnumbered}
|
|
|
|
| Abbreviation | Meaning |
|
|
| --- | --- |
|
|
| API | Application Programming Interface |
|
|
| CLI | Command-Line Interface |
|
|
| DAG | Directed Acyclic Graph |
|
|
| JSON-RPC | JavaScript Object Notation Remote Procedure Call |
|
|
| LLM | Large Language Model |
|
|
| MCP | Model Context Protocol |
|
|
| RPC | Remote Procedure Call |
|
|
| USTH | University of Science and Technology of Hanoi |
|
|
|
|
: Abbreviations used in the thesis. {#tbl:abbreviations .unnumbered}
|
|
|
|
# Abstract {.unnumbered}
|
|
|
|
Preparing reports, transforming documents, and collecting workspace information
|
|
often involve procedures that must be repeated with new inputs. An AI assistant
|
|
can help perform such work, but a successful conversation does not itself
|
|
preserve an executable procedure. This thesis presents `lda.chat`, a
|
|
programmable workflow platform for defining, checking, running, and inspecting
|
|
reusable workspace procedures.
|
|
|
|
A workflow describes the operations to perform, the data they exchange, and
|
|
the decisions that select the next step. An **artifact** is an immutable saved
|
|
version of that workflow. A **deployment** connects a saved version to the
|
|
concrete services it will use. A **run** records one execution, including its
|
|
inputs, status, result, and trace. These distinctions separate revising a
|
|
procedure, configuring where it operates, and examining what happened.
|
|
|
|
The design addresses both authoring experience and execution behavior. Authors
|
|
need to discover operations, connect their inputs and outputs, understand
|
|
validation errors, and inspect results. The runtime needs corresponding rules
|
|
for data contracts, control flow, state updates, and interruption. The thesis
|
|
compares these concerns with other workflow systems and explains the prototype's
|
|
choice to separate a node's data output from its routing outcome.
|
|
|
|
The implementation provides Python authoring objects, a workflow service, and
|
|
adapters for trusted Python functions and external tools. Evidence includes
|
|
a deterministic case study and focused conformance tests. These establish
|
|
specific engineering results, not broad usability, model superiority, or
|
|
reduced token use. The system remains in
|
|
development, with further work needed on authoring ergonomics and user
|
|
experience.
|
|
|
|
# Introduction
|
|
|
|
Preparing a weekly project report involves collecting notes, extracting actions
|
|
and risks, checking the result, and producing a document that other people can
|
|
use. A person can do this manually, write a script, or ask an AI assistant for
|
|
help. When the procedure becomes recurring, the workspace also needs a way to
|
|
preserve it, supply different inputs, and inspect unsuccessful attempts.
|
|
|
|
`lda.chat` is a programmable workflow platform for that recurring work.
|
|
It allows a human or external agent to assemble available operations into a
|
|
saved procedure and execute it through a service. The procedure exists
|
|
independently of the conversation or programming session that created it.
|
|
Its executions can be inspected separately, and explicit requests for additional
|
|
input can be resumed from saved state.
|
|
|
|
For the report task, the initial procedure is short:
|
|
|
|
Read the notes, extract a structured report, and render it as Markdown.
|
|
|
|
The operator supplies new notes each week rather than rebuilding those steps.
|
|
If the requirements change, the author revises the procedure and saves another
|
|
version. If execution fails, the operator needs to identify the affected step
|
|
and its inputs, not merely observe that no report appeared.
|
|
|
|
## From a useful procedure to a usable system
|
|
|
|
The intended beneficiary is a workspace operator who needs repeatable work.
|
|
Its author may be that person, a developer, or an external agent acting on their
|
|
behalf. Authoring a procedure and operating a saved one are different activities;
|
|
neither requires an LLM to participate in every execution.
|
|
|
|
A workflow system must help its author answer practical questions: which
|
|
operations are available, what information they require, and how one operation's
|
|
result becomes another's input. During operation, it must distinguish an invalid
|
|
definition, a missing service, a failed execution, and a request for more input.
|
|
These are interaction-design concerns as well as runtime concerns. An interface
|
|
that draws a branch without explaining whether one path or both will execute
|
|
leaves a consequential rule implicit.
|
|
|
|
The current prototype emphasizes programmable authoring and inspection.
|
|
Its Python interface presents workflow and run objects instead of requiring
|
|
authors to construct network messages. This supports code-based use, but does
|
|
not establish accessibility for non-programmers. Onboarding, edit feedback,
|
|
error presentation, and the effort needed to understand a workflow remain
|
|
areas for user-experience improvement.
|
|
|
|
## Engineering question and contribution
|
|
|
|
The engineering question is how to make a reusable workspace procedure both
|
|
operable through public interfaces and explicit enough for the runtime to
|
|
validate, execute, and record.
|
|
|
|
The contribution connects three design choices:
|
|
|
|
1. A typed graph makes operations, data movement, and routing decisions
|
|
inspectable rather than leaving the procedure only in a conversation.
|
|
2. Separate saved versions, environment bindings, and execution records make
|
|
editing, configuration, and operation distinct activities.
|
|
3. Programmable authoring, validation diagnostics, and run inspection expose
|
|
those distinctions to human and agent clients.
|
|
|
|
The work integrates established workflow techniques rather than introduces a
|
|
new autonomous planning algorithm. Its design is assessed from both sides:
|
|
what an author must understand and do, and what the runtime guarantees when
|
|
the procedure executes. The comparison with related systems uses concrete
|
|
mechanisms rather than treating visual polish or graph notation as sufficient
|
|
evidence of either usability or correctness.
|
|
|
|
## Scope of the implementation
|
|
|
|
The prototype supports conditional routes, iteration, child workflows, and
|
|
explicit interruption/resume. Its runs have an execution-attempt limit.
|
|
General parallel fork/gather remains proposed; persisted interruption does not
|
|
mean recovery halfway through arbitrary external code. The evaluation separates
|
|
tested behavior from unmeasured usability and production-readiness claims.
|
|
|
|
## Report Outline
|
|
|
|
Section 2 derives interaction and execution requirements from the workspace
|
|
task. Section 3 compares ways to author and operate that task in related
|
|
systems. Section 4 explains the prototype's concepts through an example.
|
|
Sections 5 and 6 describe the architecture and implementation; Section 7 presents
|
|
the reproducible case study. Section 8 evaluates the available evidence.
|
|
The remaining chapters discuss limitations, future work, and conclusions.
|
|
|
|
# Problem Statement And Requirements
|
|
|
|
A procedure that works once is not necessarily ready for repeated operation.
|
|
For the report example, a renamed field may invalidate extraction, a different
|
|
workspace may use another notes service, or the input may omit information
|
|
needed by the final document. The author needs feedback that distinguishes these
|
|
situations, while the runtime needs rules for handling them.
|
|
|
|
LLM tool-use approaches illustrate dynamic selection of actions
|
|
[@react-2022; @toolformer-2023]. This thesis does not assume that such systems
|
|
cannot use schemas or persistence. It asks which responsibilities should be
|
|
explicit in the reusable procedure rather than depend on the authoring session.
|
|
|
|
## Authoring and operation requirements
|
|
|
|
The following requirements describe what the interaction should support.
|
|
They are design goals, not claims that every aspect of the current experience
|
|
has been validated with users.
|
|
|
|
1. **Discover before connecting.** Show available operations and the inputs,
|
|
outputs, and services they require. The author should not need to inspect
|
|
server implementation code to learn how an operation can be used.
|
|
2. **Make data movement understandable.** Explain how a document path becomes
|
|
text, how text becomes structured fields, and which fields reach the result.
|
|
Distinguish a data connection from a decision about what executes next.
|
|
3. **Support revision and useful feedback.** An author should be able to revise
|
|
an unfinished procedure and locate errors in the relevant step or binding.
|
|
Diagnostics should distinguish what must change from what remains valid.
|
|
4. **Separate editing from running.** Make clear which saved version an execution
|
|
uses. Editing the next version should not silently change an earlier one.
|
|
5. **Explain execution state.** Distinguish completed, failed, and interrupted
|
|
runs, expose relevant intermediate evidence, and identify the input required
|
|
to resume an interruption.
|
|
|
|
For machine clients, these interactions need structured responses and stable
|
|
identities. For human authors, they also need understandable terminology and
|
|
manageable amounts of information. Providing structured output satisfies an
|
|
interface requirement; it is not proof that the overall experience is usable.
|
|
|
|
## Execution requirements behind the interaction
|
|
|
|
Those interactions require corresponding runtime contracts:
|
|
|
|
1. **Preserve definitions and executions independently.** Save an identifiable
|
|
procedure version and keep separate records for its invocations.
|
|
2. **Validate known constraints before work starts.** Check graph structure,
|
|
declared data contracts, mappings, and required service bindings without
|
|
pretending to predict every external failure.
|
|
3. **Specify routing and state changes.** Define what selects the successor,
|
|
where outputs are written, and how repeated writes affect workflow state.
|
|
4. **Keep environment choices outside the procedure's logic.** Resolve logical
|
|
service requirements to concrete configured services and report mismatches.
|
|
5. **Bound and inspect execution.** Limit execution attempts, retain status and
|
|
trace information, and support resume at defined interruption boundaries.
|
|
|
|
These requirements motivate the artifact, deployment, and run distinctions.
|
|
They do not prescribe those names for every competing system.
|
|
|
|
## Costs and boundaries
|
|
|
|
Schemas, bindings, and versioned definitions impose authoring work. A script
|
|
can be preferable for a short-lived task, especially when its author already
|
|
understands the libraries involved. The platform targets cases where reuse and
|
|
inspection justify that setup; the thesis does not establish a numerical
|
|
break-even point.
|
|
|
|
A workflow interface must also avoid promising more than its runtime supports.
|
|
Successful validation does not guarantee a remote service will succeed, and a
|
|
saved interruption does not make arbitrary external effects reversible.
|
|
Scheduling and general fork/gather remain separate development work.
|
|
|
|
# Positioning And Related Systems
|
|
|
|
The report task provides a common lens for comparison: choose operations, pass
|
|
notes between them, add a route for incomplete information, test the procedure,
|
|
and inspect its result. The comparison follows these authoring activities into
|
|
the execution rules they expose.
|
|
|
|
The accounts below use documented mechanisms consulted on September 7, 2026.
|
|
They are not hands-on usability measurements or a complete product survey.
|
|
Differences in authoring style do not establish that one system is easier for
|
|
every audience.
|
|
|
|
## Starting with code or direct tool calls
|
|
|
|
A developer can express the report procedure as function calls and use the
|
|
language's conditionals and loops. An agent can instead select successive tool
|
|
calls from the information available at each turn. Both approaches can be
|
|
combined with schemas, tests, logs, and persistence.
|
|
|
|
Code puts the procedure close to its implementation and makes ordinary debugging
|
|
tools available. It also leaves decisions about configuration, saved versions,
|
|
and run records to the program or its surrounding infrastructure. A workflow
|
|
platform makes some of those decisions part of its public contract, at the cost
|
|
of introducing another model for the author to learn.
|
|
|
|
The prototype still uses Python for authoring. The distinction is whether that
|
|
code performs the entire procedure directly or constructs a saved workflow for
|
|
the service to execute. Neither representation is automatically better for a
|
|
one-off task.
|
|
|
|
## n8n: connecting and inspecting data
|
|
|
|
For an author combining report records, n8n's Merge node exposes a concrete
|
|
choice: append the incoming collections, match records by fields or position,
|
|
or produce combinations. The node's configuration and worked examples make
|
|
these operations distinguishable. Append waits for connected inputs and emits
|
|
their items in input order [@n8n-merge-2026].
|
|
|
|
This connects an interface decision to an execution rule. Selecting a merge mode
|
|
does not merely change a diagram: it changes which records appear in the output.
|
|
For the report task, joining actions by owner is different from appending two
|
|
action lists.
|
|
|
|
The lesson for the prototype is that a connection needs an understandable data
|
|
meaning. Its state reducers describe how writes update fields; they should not
|
|
be presented as interchangeable with an item join or synchronization barrier.
|
|
This comparison concerns the documented Merge mechanism, not every n8n node.
|
|
|
|
## Zapier: configuring a decision
|
|
|
|
An author using Zapier Paths selects fields, conditions, and values, then tests
|
|
the rules against sample data. Applied to the report task, those rules could
|
|
distinguish complete records from records needing attention. Multiple paths
|
|
can qualify, so exclusivity must follow from the rules rather than the
|
|
branching appearance alone [@zapier-paths-2026].
|
|
|
|
The same documentation describes sequential execution of qualifying paths.
|
|
Paths does not provide a shared action after all branches; common steps can be
|
|
duplicated or placed in a Sub-Zap. These constraints affect how the author
|
|
organizes a common report-rendering step [@zapier-paths-2026].
|
|
|
|
The interface therefore needs to communicate both the condition being tested
|
|
and the consequence of a match. In the prototype, an ordinary outcome selects
|
|
one successor. That is a different contract, not a claim of superior usability.
|
|
|
|
## LangGraph: describing decisions in Python
|
|
|
|
LangGraph's Graph API lets an author define state, add node functions and
|
|
routing, compile the graph, and invoke it. For the report task, a developer can
|
|
represent extracted fields in state and write a routing function that selects
|
|
what happens next [@langgraph-graph-api-2026].
|
|
|
|
This authoring style exposes more behavior as code. Nodes produce state
|
|
updates, reducers determine how updates combine, and conditional routes select
|
|
subsequent execution. The documented model supports graph loops and super-step
|
|
execution; its checkpoint facilities are described separately
|
|
[@langgraph-graph-api-2026; @langgraph-persistence-2026].
|
|
|
|
The prototype shares the use of typed state and reducers, but represents
|
|
ordinary routing as a mapping from a declared outcome to a successor.
|
|
This keeps executable predicates out of ordinary edges. The cost is that some
|
|
decisions need a dedicated condition step or an additional declared outcome.
|
|
The relevant comparison is where authors express and inspect decisions, not
|
|
whether Python or a canvas is inherently the better interface.
|
|
|
|
## Implications for this design
|
|
|
|
The examples expose three connected design concerns: authors must understand
|
|
the data exchanged, the condition selecting work, and the meaning of execution
|
|
progress. A graph drawing alone does not answer any of them.
|
|
|
|
| Author's question | Execution concept it exposes |
|
|
| --- | --- |
|
|
| Which result should the next step receive? | Input/output mapping |
|
|
| Will one branch run, or several? | Exclusive routing or concurrent emission |
|
|
| What happens where paths meet? | Continuation, data merge, or a barrier |
|
|
| Why did this attempt stop? | Failure versus explicit interruption |
|
|
| What does changing the workflow affect? | Saved version versus a run |
|
|
|
|
: Interaction and execution concepts. {#tbl:positioning-summary}
|
|
|
|
The prototype prioritizes programmable authoring, explicit mappings, and saved
|
|
execution records. Its current interface must still be evaluated for the work
|
|
required to discover operations, repair definitions, and interpret results.
|
|
Neither its typed models nor the absence of executable edge predicates proves
|
|
that it achieves those user-experience goals.
|
|
|
|
External-tool protocols are a separate concern. Model Context Protocol (MCP)
|
|
exposes tools, resources, and prompts; it is not a competing graph model
|
|
[@mcp-tools-2025; @mcp-lifecycle-2025]. Here MCP is a source family, not the
|
|
product identity. Protocol details belong later, after the authoring and
|
|
execution concepts they support have been explained.
|
|
|
|
# Conceptual Model
|
|
|
|
The report example introduces the concepts in the order an author encounters
|
|
them: choose operations, connect their data, define decisions, save a version,
|
|
and inspect an execution. The following branching example explains supported
|
|
primitives; it is not an additional measured case study.
|
|
|
|
## Operations, data, and decisions
|
|
|
|
Suppose report preparation must ask for missing information before rendering:
|
|
|
|
```{.mermaid #fig:report-branch width=95% caption="Report routes."}
|
|
%%{init: {"flowchart": {"rankSpacing": 20, "nodeSpacing": 20}}}%%
|
|
flowchart LR
|
|
Read["Read<br/>notes"] --> Extract["Extract<br/>report"]
|
|
Extract --> Check{Complete?}
|
|
Check -->|ready| Render["Render<br/>report"]
|
|
Check -->|needs_information| Ask["Request<br/>information"]
|
|
Ask -->|submitted| Render
|
|
Render --> Finish([End])
|
|
```
|
|
|
|
Each named operation is a **node** in the workflow. An **edge** selects the next
|
|
step after a node produces an **outcome**, such as `ready` or
|
|
`needs_information`. The node's **output** is the data it returns, such as the
|
|
extracted report fields. A missing-information outcome is a business decision;
|
|
an exception while reading a file is a runtime failure.
|
|
|
|
The arrow to the next step does not implicitly pass all previous output into
|
|
that step. The author defines input mappings and writes relevant outputs into
|
|
workflow state. In this example, extraction writes report fields, the request
|
|
can supply missing fields on resume, and rendering reads the resulting report.
|
|
The two routes to rendering are alternatives, not concurrent branches needing
|
|
a join.
|
|
|
|
**Schemas** declare the shapes of accepted inputs and produced results.
|
|
They help the author see which fields an operation requires and allow the
|
|
validator to detect incompatible mappings. They do not establish that an
|
|
extracted fact is true or that a remote operation will succeed.
|
|
|
|
## Workflow state and iteration
|
|
|
|
**State** is the workflow's working data. A **reducer** specifies how a write
|
|
changes a state field: replace the previous value, append an action item, or add
|
|
a number, for example. Reading a field and routing to another step are separate
|
|
operations. This makes data movement inspectable but requires the author to
|
|
understand the mappings.
|
|
|
|
If the procedure processes several documents, a foreach step defines an item
|
|
body. The current item and its iteration context belong to that body; normal
|
|
item completion returns to the owning foreach. A child workflow provides a
|
|
separate invocation scope with explicit inputs and results. These boundaries
|
|
define which data is available and what completion means.
|
|
|
|
Ordinary same-region cycles are permitted, with a run-wide step budget limiting
|
|
execution attempts. General fork/gather is not implemented. Connecting paths
|
|
does not by itself promise parallel execution, synchronization, or conflict-free
|
|
state merging.
|
|
|
|
## Saving, configuring, and running the procedure
|
|
|
|
While authoring, an **editable workflow** is the mutable definition being
|
|
revised. The Python client supports this object without requiring a server-side
|
|
draft workspace. Saving creates an **artifact**: an immutable workflow version
|
|
containing its graph and declared requirements.
|
|
|
|
A **deployment** selects a saved version and connects its logical service
|
|
requirements to concrete configured services. For example, the same report
|
|
procedure could use a test notes source in one deployment and a production
|
|
notes source in another, provided both satisfy its required contracts.
|
|
Changing the bindings is different from changing how the report is assembled.
|
|
|
|
A **run** records one execution of a deployment. Last week's successful report
|
|
and this week's interrupted attempt are different runs, even if both use the
|
|
same artifact and deployment. Inspecting a run should identify the version,
|
|
inputs, status, and available execution evidence without changing the saved
|
|
procedure.
|
|
|
|
These distinctions explain the lifecycle from the operator's perspective:
|
|
|
|
```{.mermaid #fig:lifecycle-records width=95% caption="Lifecycle records."}
|
|
classDiagram
|
|
direction LR
|
|
class EditableWorkflow {
|
|
mutable graph
|
|
}
|
|
class ArtifactVersion {
|
|
artifact_id
|
|
version
|
|
saved definition
|
|
}
|
|
class Deployment {
|
|
deployment_id
|
|
source bindings
|
|
}
|
|
class Run {
|
|
run_id
|
|
input and result
|
|
status and trace
|
|
}
|
|
EditableWorkflow ..> ArtifactVersion : saves
|
|
ArtifactVersion "1" <-- "0..*" Deployment : selects
|
|
Deployment "1" <-- "0..*" Run : started from
|
|
```
|
|
|
|
## Available operations and environment binding
|
|
|
|
A **capability** is an operation available for use in a workflow.
|
|
A **source** groups capabilities under a configured identity. The report's
|
|
extraction operation might come from trusted Python code, while another
|
|
operation is supplied by an external tool service.
|
|
|
|
A deployment **binding** connects a logical source requirement in the workflow
|
|
to a concrete source in the environment. Validation checks whether that source
|
|
exists and matches the saved requirements. **Source drift** means those
|
|
requirements no longer match the currently available capabilities, for example
|
|
after an input schema changes.
|
|
|
|
Built-in sources have fixed platform identities and do not require those
|
|
deployment bindings. Configured sources remain explicit operator choices.
|
|
This provides scoped portability, not freedom from environment dependencies:
|
|
the required code, credentials, and services must still be available.
|
|
|
|
## Inspecting failure and resuming an interruption
|
|
|
|
A validation diagnostic concerns the definition or its dependencies before
|
|
execution. A failed run records an operational problem encountered during
|
|
execution. An interrupted run records an explicit request for input, together
|
|
with the state needed to continue. The interface should distinguish these
|
|
situations because they call for different actions.
|
|
|
|
In the report example, `needs_information` routes to a request step. The run
|
|
then waits for a declared resume payload; resuming applies that payload through
|
|
the workflow's bindings and continues to rendering. This is a defined pause in
|
|
the procedure, not recovery halfway through an arbitrary handler.
|
|
|
|
A **trace** records execution evidence associated with the run. It supports
|
|
questions such as which route was taken and where execution stopped, but does
|
|
not make external effects reversible. The implementation chapters explain how
|
|
these concepts become runtime records and service operations.
|
|
|
|
## Working Glossary
|
|
|
|
The core terms can now be summarized without requiring implementation vocabulary.
|
|
|
|
| Term | Meaning in the report example |
|
|
| --- | --- |
|
|
| Capability | An available operation, such as extracting report fields |
|
|
| Node | One use of an operation or control step in the procedure |
|
|
| Output | Data produced by a step |
|
|
| Outcome | A declared label selecting the next step |
|
|
| State | Working data retained during execution |
|
|
| Artifact | An immutable saved version of the procedure |
|
|
| Deployment | A saved version connected to concrete services |
|
|
| Run | One execution with its own status and evidence |
|
|
| Source | A configured collection of available operations |
|
|
| Binding | A logical source requirement mapped to a concrete source |
|
|
|
|
: Working glossary for the thesis terminology. {#tbl:working-glossary}
|
|
|
|
Python types, provider protocols, and package boundaries implement these
|
|
concepts. Their names are introduced with their responsibilities in the
|
|
architecture and implementation chapters rather than used as prerequisites
|
|
for understanding the workflow.
|
|
|
|
# System Architecture
|
|
|
|
The weekly-report example needs more than a graph executor. An author must
|
|
discover the available operations, connect them, check the resulting workflow,
|
|
and choose which saved version to run. An operator must then distinguish a
|
|
bad definition from an unavailable service or an interrupted execution. The
|
|
architecture separates these responsibilities without requiring each caller
|
|
to implement the workflow lifecycle.
|
|
|
|
Three boundaries organize the system: authoring versus server operations,
|
|
saved definitions versus individual executions, and workflow execution versus
|
|
provider-specific calls. These boundaries are visible in the user-facing
|
|
objects as well as in the implementation.
|
|
|
|
## From Authoring to Server Operations
|
|
|
|
The Python client is the main programmatic authoring interface. Its `App`
|
|
object represents a connection to the workflow service. An author can inspect
|
|
a capability, use the returned object in an editable workflow, validate that
|
|
workflow, and save it. The client reconstructs server responses as Python
|
|
objects with relevant operations, rather than requiring application code to
|
|
carry response dictionaries through every step.
|
|
|
|
Local editing does not require a server request for each graph change.
|
|
Discovery and persistence do: the service owns the available capability
|
|
inventory and stored records. Validation therefore has both a local part,
|
|
which checks the authored structure, and a server part, which checks it
|
|
against the service's contracts.
|
|
|
|
[@fig:architecture-spine] shows this separation. The CLI is another entry
|
|
point to the service; it is not a mandatory layer beneath Python authoring.
|
|
|
|
```{.mermaid #fig:architecture-spine caption="Shared service boundaries."}
|
|
flowchart TB
|
|
subgraph Authoring["Authoring and clients"]
|
|
Python["Python App and editable workflow"]
|
|
Client["Client port"]
|
|
CLI["Command-line interface"]
|
|
Python --> Client
|
|
end
|
|
subgraph Service["Server-composed services"]
|
|
API["Workflow API"]
|
|
Stores["Artifacts, deployments and run records"]
|
|
Inventory["Available operation contracts"]
|
|
end
|
|
subgraph Transport["Transport boundary"]
|
|
RPC["HTTP JSON-RPC adapter"]
|
|
end
|
|
subgraph Execution["Workflow execution"]
|
|
Core["Workflow execution core"]
|
|
end
|
|
subgraph Integrations["Provider integrations"]
|
|
Providers["Configured source providers"]
|
|
Handlers["Execution handlers"]
|
|
end
|
|
CLI --> RPC
|
|
Client --> RPC
|
|
RPC --> API
|
|
API --> Stores
|
|
API --> Core
|
|
Inventory --> API
|
|
Providers --> Inventory
|
|
Core --> Handlers
|
|
Providers --> Handlers
|
|
```
|
|
|
|
The API operation layer is independent of the wire protocol. The JSON-RPC
|
|
adapter translates requests and responses; it does not decide how a foreach
|
|
iteration returns or how a reducer applies a state update. Conversely, the
|
|
execution core does not need to know whether its caller used Python, a
|
|
command line, or another application.
|
|
|
|
This distinction also limits the role of an agent. An agent can help author
|
|
or operate a workflow through these interfaces, but the runtime follows the
|
|
saved graph. It does not ask an agent to choose the next step unless the
|
|
author has explicitly included an operation that makes such a decision.
|
|
|
|
## Keeping a Definition Separate from Its Use
|
|
|
|
Saving, deploying, and running answer different questions:
|
|
|
|
- An artifact identifies the saved workflow definition and its version.
|
|
- A deployment selects that version and supplies environment bindings.
|
|
- A run records one execution, including its input, status, and progress.
|
|
|
|
For the report workflow, changing the extraction step produces a different
|
|
definition. Choosing the configured service that supplies extraction is an
|
|
environment decision. Processing this week's notes is an individual run.
|
|
Keeping these apart allows an operator to inspect which definition and
|
|
bindings an execution used without confusing them with the current editor
|
|
contents.
|
|
|
|
The client exposes this progression through workflow artifact, deployment,
|
|
and run objects. A run object is a snapshot, not a live background
|
|
subscription. Refreshing it requests a new snapshot. This makes network
|
|
activity explicit, although applications must decide when to refresh and how
|
|
to present progress.
|
|
|
|
Draft workspaces provide a separate persisted editing surface used by the
|
|
draft API and CLI. They are not a required intermediate object for every
|
|
Python authoring operation. Both routes ultimately produce a saved
|
|
definition that the deployment and execution layers can use.
|
|
|
|
## Data Movement and Control Movement
|
|
|
|
Within a workflow, input bindings supply a step's arguments. Output bindings
|
|
select returned values to write into workflow state. Reducers determine how
|
|
those writes combine with existing values. The returned outcome selects the
|
|
next edge. These are related operations, but none substitutes for another:
|
|
routing to a renderer does not, by itself, supply the report it needs.
|
|
|
|
[@fig:node-execution-cycle] summarizes an ordinary callable step. Conditions,
|
|
iteration controllers, subgraphs, and interrupts have their own runtime
|
|
handlers rather than pretending to be remote capability calls.
|
|
|
|
```{.mermaid #fig:node-execution-cycle caption="Callable step execution."}
|
|
flowchart TB
|
|
Validation["Input validation"] --> Call["Invoke operation"]
|
|
Call --> Result["Checked result"]
|
|
subgraph Data["DATA: what becomes visible"]
|
|
Output["Output payload"] --> Bind["Output bindings"]
|
|
Bind --> Reduce["Reducers update state"]
|
|
end
|
|
subgraph Control["CONTROL: where execution goes"]
|
|
Outcome["Declared outcome"] --> Route["Select matching edge"]
|
|
Route --> Next["Next node"]
|
|
end
|
|
Result --> Output
|
|
Result --> Outcome
|
|
Reduce -.->|state available to next step| Next
|
|
```
|
|
|
|
A declared outcome such as `needs_information` is a workflow decision. A
|
|
handler exception or exhausted step budget is an execution failure. A
|
|
workflow can therefore complete with a non-success business outcome without
|
|
being a failed runtime execution. Where iteration supports collecting item
|
|
errors, that policy must be explicit; errors do not automatically become
|
|
ordinary outcome edges.
|
|
|
|
The lanes distinguish responsibilities, not concurrent tasks: the runtime
|
|
applies the result's writes before advancing along the selected route.
|
|
|
|
The trace makes the sequence inspectable, but a fixed graph does not imply
|
|
identical external results. Language-model calls, remote services, and
|
|
concurrent completion order can vary between runs. The runtime's defined
|
|
routing and state-update rules should not be confused with reproducibility
|
|
of every operation it invokes.
|
|
|
|
## Iteration and Child Workflow Boundaries
|
|
|
|
Suppose the report now covers two documents, `A.md` and `B.md`. The
|
|
workflow must render a report for each, then assemble the two reports.
|
|
For this first example, foreach is configured to process one document at
|
|
a time. An output binding appends each rendered report to a `reports`
|
|
state field; the assembly step reads that field.
|
|
|
|
The important distinction is between finishing **one document** and
|
|
finishing **the whole collection**. After A finishes, the foreach continues
|
|
with B. Only after B finishes does it follow its `done` route to assembly.
|
|
It does not begin the collection again whenever an item returns.
|
|
|
|
[@fig:foreach-region] follows one invocation from start to finish. Read
|
|
downward for time. Solid arrows request work or apply a data binding;
|
|
dashed arrows report completion. The labels identify the data and routes.
|
|
|
|
```{.mermaid #fig:foreach-region width=95% caption="Two documents, one loop."}
|
|
sequenceDiagram
|
|
participant Each as Foreach
|
|
participant Body as Render item
|
|
participant State as reports state
|
|
participant Assemble as Assemble
|
|
Note over Each,State: Start once: documents = [A.md, B.md], reports = []
|
|
Each->>Body: loop: process A.md
|
|
Body->>State: Output binding: append reportA
|
|
Body-->>Each: ok: A is finished
|
|
Each->>Body: loop: process B.md
|
|
Body->>State: Output binding: append reportB
|
|
Body-->>Each: ok: B is finished
|
|
Note over Each: Both items finished
|
|
Each->>Assemble: done: continue after the loop
|
|
Assemble->>State: Input binding: read reports
|
|
State-->>Assemble: [reportA, reportB]
|
|
```
|
|
|
|
The same authored render node executes twice, but the runtime must remember
|
|
which document each execution belongs to. It calls that per-item execution
|
|
record a **frame**. The foreach invocation that started the item is its
|
|
**owner**. Returning to that owner finishes the current item; arriving from
|
|
the preceding workflow step starts a new foreach invocation.
|
|
|
|
This explains the graph's back-edge: `render.ok -> each` is an item return,
|
|
whereas `each.done -> assemble` leaves the loop. The output binding is what
|
|
appends the report; the back-edge itself does not transport or collect data.
|
|
The graph validator rejects a body node used both inside and outside that
|
|
loop, or a nested body that returns past its immediate owner.
|
|
|
|
### Calling a Child Workflow for One Document
|
|
|
|
Now replace the rendering work with a call to a saved child workflow that
|
|
summarizes one document. Consider only the item for `A.md`. The parent may
|
|
know the whole collection, but the child receives only the input explicitly
|
|
mapped into its call: `{"document": "A.md"}`.
|
|
|
|
The child has its own input, working state, and execution context. It cannot
|
|
read the parent's current item or `reports` field merely because the parent
|
|
called it. If it needs another value, the author must add an input binding.
|
|
|
|
[@fig:scope-boundaries] shows the call and its return. The item waits while
|
|
the child runs. The child's `END` completes that child invocation, not the
|
|
parent item or the whole report workflow.
|
|
|
|
```{.mermaid #fig:scope-boundaries width=95% caption="A child call returns."}
|
|
sequenceDiagram
|
|
participant Item as Parent item A
|
|
participant Child as Child workflow
|
|
participant Result as Item A reports writes
|
|
participant Each as Foreach
|
|
Item->>Child: Input binding: document = A.md
|
|
activate Child
|
|
Note over Item: Wait for child
|
|
Note over Child: Own input,<br/>state and context
|
|
Child->>Child: Produce summaryA
|
|
Child-->>Item: END: return summaryA
|
|
deactivate Child
|
|
Item->>Result: Output binding: append summaryA to reports
|
|
Item-->>Each: ok: item A is finished
|
|
Note over Each: Continue according<br/>to foreach mode
|
|
```
|
|
|
|
The implementation calls the child's isolated data environment a **scope**.
|
|
Its input binding crosses into that scope; its output binding maps the
|
|
returned result into the caller item's pending writes, or directly into
|
|
enclosing state in serial mode. The two completion points are separate:
|
|
child `END` returns to the calling node, and the calling node's route back
|
|
to foreach finishes the item.
|
|
|
|
### When Items Run Concurrently
|
|
|
|
In concurrent mode, A and B may be in progress together. A returning from
|
|
its child does not permit assembly while B is still running. The foreach
|
|
waits for its required item completions before following `done`.
|
|
|
|
Each item keeps its own pending writes, rather than immediately exposing
|
|
them to its sibling. The runtime tracks that separate state history as a
|
|
**lineage**; the pending writes are its **buffer**. In the second diagram,
|
|
“Item A reports writes” is that buffer when the foreach is concurrent. In serial
|
|
mode, the binding instead updates the enclosing state so the next item can
|
|
read it.
|
|
|
|
For the successful two-item concurrent case, the foreach combines A's and
|
|
B's writes using the declared reducers, then assembly reads the combined
|
|
state. An append reducer and a replace reducer have different effects;
|
|
neither the arrows nor the fact that both items completed chooses a merge
|
|
policy. The serial ordering shown in the first diagram is not a promise
|
|
about concurrent completion order.
|
|
|
|
If a child or item reaches an explicit interrupt, the run must preserve
|
|
which document was active and where it was waiting. Resume continues that
|
|
saved work; it does not restart the collection or silently move the
|
|
response to another item.
|
|
|
|
## Connecting External Operations
|
|
|
|
Providers adapt external functionality to contracts the workflow system can
|
|
inspect and invoke. Discovery supplies schemas and declared outcomes;
|
|
execution supplies a handler for the selected binding. These are separate
|
|
responsibilities because discovering an operation does not mean its service
|
|
is currently reachable or authorized.
|
|
|
|
The server composes providers, stores, and the API. MCP-backed operations
|
|
may need sessions, authentication, and remote catalog handling. Python-backed
|
|
operations use configured, trusted imports. These concerns remain outside
|
|
the graph scheduler. An experimental OpenAPI provider explores another
|
|
source family without making it the system's product identity or implying
|
|
that all API descriptions are interchangeable.
|
|
|
|
This boundary reduces provider-specific logic in workflows, but cannot
|
|
erase provider differences. A schema describes a call's shape; it does not
|
|
guarantee availability, cost, side-effect safety, or semantic equivalence to
|
|
another operation with the same fields.
|
|
|
|
# Implementation
|
|
|
|
The implementation follows these boundaries through focused Python
|
|
packages. The important question is not the number of packages, but where a
|
|
change must be made. Improving a Python editing method should not require
|
|
changing the scheduler, and adapting a new remote service should not require
|
|
changing graph routing.
|
|
|
|
## Following a Python Authoring Request
|
|
|
|
The client package, `wf_client`, connects Python objects to a narrow
|
|
`WorkflowClientPort`. `App.from_http_jsonrpc(...)` configures the connection
|
|
without making a request. A subsequent capability inspection performs I/O,
|
|
decodes the response, checks the returned identity, and constructs a
|
|
`RemoteCapability`.
|
|
|
|
An `EditableWorkflow` subclasses the authoring layer's `WorkflowBuilder`,
|
|
reusing its graph-building methods. It adds remote validation and saving
|
|
rather than maintaining a second independent builder implementation.
|
|
Its validation method contains this early return:
|
|
|
|
```python
|
|
local = self.validate_local()
|
|
if not local.ok:
|
|
return WorkflowValidation(local, "not_run", ())
|
|
```
|
|
|
|
This excerpt from `wf_client/authoring.py` explains an observable behavior:
|
|
a structurally invalid edit produces local feedback without a server
|
|
request. Passing that check does not establish deployment readiness; the
|
|
server still validates the submitted plan against its inventory.
|
|
|
|
Saving validates first, submits the plan, checks the save response, and
|
|
re-inspects the exact artifact version. The resulting artifact object is
|
|
therefore reconstructed from the stored definition, not assumed to be an
|
|
unchanged copy of the editor. Identity checks reject mismatched responses
|
|
instead of quietly attaching methods to the wrong artifact or deployment.
|
|
|
|
The same principle applies to runs. In `wf_client/runs.py`, `refresh()`
|
|
returns a decoded snapshot from `inspect_run`, checking the expected run
|
|
and deployment identities. `resume()` returns another snapshot after
|
|
submitting the response. Application code must retain the returned object;
|
|
an earlier snapshot does not mutate when the server advances.
|
|
|
|
Serialization still exists at the service boundary. The benefit is that
|
|
callers need not manually rebuild the domain objects after every request.
|
|
The client does not eliminate the distinction between a local Python model
|
|
and a remote operation.
|
|
|
|
## Executing and Bounding a Run
|
|
|
|
The `wf_core` package defines the graph and execution state. Its runtime
|
|
selects ready frames and dispatches by node kind. An ordinary `NodeUse`
|
|
invokes a bound callable; `ConditionNode`, `ForeachNode`, `SubgraphNode`,
|
|
`InterruptNode`, and `EndNode` implement explicit control behavior.
|
|
|
|
Step admission happens before dispatch. The immutable `RunLimits` policy
|
|
sets a positive maximum, with a default of 10,000 attempts. The run stores
|
|
how many attempts have been admitted. Nested execution shares that run-wide
|
|
budget, and asynchronous dispatch reserves attempts before launching work.
|
|
A failed attempted step is not free simply because it returned no useful
|
|
result.
|
|
|
|
The budget bounds graph progress, including a cycle whose condition never
|
|
selects an exit. It is not a wall-clock timeout, a language-model token
|
|
allowance, or protection against a handler that blocks indefinitely. The
|
|
limit and consumed count persist with the run, so interrupting and resuming
|
|
does not reset the allowance.
|
|
|
|
## Preserving State Across Nested Execution
|
|
|
|
Workflow state updates pass through reducer-aware patches. Serial iteration
|
|
must make its writes visible to subsequent serial work, while concurrent
|
|
items need separate views until their results are combined. Nesting either
|
|
mode inside the other makes write ownership more subtle than committing
|
|
every result directly to global state.
|
|
|
|
The shared `commit_foreach_aware_patch` helper handles writes from ordinary
|
|
nodes, subgraph results, and interrupt responses. It walks serial owners
|
|
outward and selects the first concurrent item boundary, if present, as the
|
|
buffer destination. It continues checking ancestry before writing, so a
|
|
missing parent or cycle cannot cause a partial write merely because a
|
|
buffer destination was already found. With no concurrent boundary, the
|
|
patch commits through the enclosing serial owners.
|
|
|
|
When concurrent results are combined, the patch retains their constituent
|
|
write contributions for later reducer replay. Keeping only cumulative
|
|
values would allow a surrounding iteration to replay an already-counted
|
|
prefix. This distinction matters for operations such as appending report
|
|
sections: a correct visible value at one nesting level is not necessarily
|
|
a correct contribution to the next merge.
|
|
|
|
This machinery currently supports iteration. It should not be read as a
|
|
claim that arbitrary graph fork/gather semantics are already implemented.
|
|
|
|
## Giving Expressions a Consistent Context
|
|
|
|
Bindings and conditions need the same account of the current execution.
|
|
The runtime's `frame_context_view` derives structured foreach entries from
|
|
persisted frame ancestry. Entries are keyed by foreach node identity, so
|
|
nested bodies can refer to enclosing items within the same workflow scope,
|
|
rather than relying only on an innermost-item shortcut.
|
|
|
|
The walk stops at a subgraph scope boundary. An enclosing item's value must
|
|
be passed as child input if the child needs it. The reader also rejects
|
|
malformed ownership, parent cycles, and conflicting aliases; corrupt
|
|
checkpoint metadata is not treated as an innocently absent field.
|
|
|
|
Condition evaluation receives this structured mapping, as do the input
|
|
resolution paths. This connection is necessary for validation to mean
|
|
anything: accepting a context path while evaluating conditions against a
|
|
smaller stub would let a valid-looking graph silently choose the wrong
|
|
branch.
|
|
|
|
## Validation, Persistence, and Diagnostics
|
|
|
|
The authoring layer checks graph structure; the server checks saved plans
|
|
and environment bindings; the runtime checks actual values and execution
|
|
state. Each layer has information the earlier one lacks. Static validation
|
|
can reject an invalid foreach return, for example, but cannot prove that a
|
|
remote operation will remain available when a run reaches it.
|
|
|
|
Diagnostics carry a code, a location, a message, and, where available, a
|
|
repair hint. These fields let a caller identify the faulty binding or node
|
|
without parsing a prose-only error. Suggested next actions are guidance,
|
|
not authorization and not evidence that a repair has succeeded. Schema
|
|
fingerprints likewise detect a changed contract representation; they do not
|
|
prove semantic compatibility.
|
|
|
|
The API lifecycle layer persists stopped runs and their versioned
|
|
checkpoints, including interrupted runs that may later resume. Restoration
|
|
validates the stored representation and recovers the execution state before
|
|
dispatch continues. This supports explicit pause-and-resume boundaries; it
|
|
does not promise durable recovery from every instruction inside an arbitrary
|
|
handler or exactly-once external side effects.
|
|
|
|
Separating artifact, deployment, and run storage also keeps inspection
|
|
focused. Definition inspection explains what was saved; deployment
|
|
inspection explains environment selection; run inspection and trace explain
|
|
what happened during a particular attempt.
|
|
|
|
## Server and Provider Responsibilities
|
|
|
|
The remaining package boundaries put these operations into a service.
|
|
`wf_api` coordinates lifecycle operations, `wf_artifacts` supplies storage
|
|
contracts and implementations, and `wf_platform` supplies shared platform
|
|
contracts. `wf_server` composes these dependencies. `wf_transport` exposes
|
|
the JSON-RPC interface, while `wf_cli` provides terminal operations.
|
|
|
|
Provider implementations retain their own lifecycle requirements. MCP
|
|
support manages remote discovery and invocation through configured
|
|
connections. Python support loads trusted configured callables; it is not
|
|
a sandbox for arbitrary submitted code. OpenAPI support remains
|
|
experimental and is not evidence that every described HTTP service can
|
|
already be used without adaptation.
|
|
|
|
These seams make additional interfaces possible, but an interface still
|
|
needs its own interaction design. The availability of API operations and
|
|
typed client objects does not, by itself, establish that workflow authoring,
|
|
diagnosis, or recovery is easy for a new user. The case study and evaluation
|
|
therefore need to distinguish demonstrated operations from broader
|
|
usability claims.
|
|
|
|
# Case Study: Deterministic Report Workflow
|
|
|
|
Consider an author who receives weekly notes and needs two deliverables: a
|
|
structured report for further processing and Markdown for a reader. The
|
|
example workflow makes this small procedure reusable:
|
|
|
|
The three operations read notes, extract a report, and render Markdown.
|
|
|
|
The input is deliberately constrained. Notes contain named sections and
|
|
action lines with owner, task, and due-date fields. Extraction parses that
|
|
format; it is not language-model summarization of arbitrary documents. With
|
|
fixed input and local Python operations, the result can be checked without
|
|
remote credentials, service quotas, or variation in generated text.
|
|
|
|
The bundle at
|
|
[`examples/report_workflow/`](../../examples/report_workflow/) supplies the
|
|
operations, fixture notes, server configuration, and a saved raw-plan example.
|
|
The walkthrough below expresses the same procedure through the current
|
|
Python client. The example README retains a command-line route for operators
|
|
who need it; that route is not a prerequisite for using this interface.
|
|
|
|
## Starting with Available Operations
|
|
|
|
The example configuration registers three trusted Python operations under
|
|
`local.report`. Their Pydantic models describe the input and output
|
|
contracts. The author consumes those operations from the service inventory;
|
|
the client does not import their implementations to execute them locally.
|
|
|
|
The following blocks form one asynchronous Python session. They assume a
|
|
server using `examples/report_workflow/wf.config.json`, reachable at its
|
|
configured address, and a fresh artifact name or unused version. The fixture
|
|
read assumes the client is running from the repository root.
|
|
|
|
```python
|
|
from pathlib import Path
|
|
|
|
from pydantic import BaseModel
|
|
|
|
from examples.report_workflow.ops import ReportOutput
|
|
from wf_authoring import input_from, input_path, output_to, state_path
|
|
from wf_client import App
|
|
|
|
app = App.from_http_jsonrpc("http://127.0.0.1:8771/rpc")
|
|
read_notes = await app.capability("local.report.read_notes")
|
|
extract_report = await app.capability("local.report.extract_report")
|
|
render_report = await app.capability("local.report.render_markdown_report")
|
|
```
|
|
|
|
Each lookup returns a capability object with its schemas and declared
|
|
outcomes. Importing `ReportOutput` above only reuses the fixture's data model
|
|
for authoring; the three capability objects still refer to server-side
|
|
operations. An application without that shared model could use the inspected
|
|
JSON Schemas instead.
|
|
|
|
This discovery step exposes a practical requirement: authors need to know
|
|
which operations exist and what data they accept before connecting them.
|
|
A name alone is not enough to establish a compatible pipeline.
|
|
|
|
## Describing the Workflow's Data
|
|
|
|
The workflow has one public input, intermediate state, and two public
|
|
outputs. They are declared separately so that intermediate notes do not
|
|
accidentally become part of the result contract.
|
|
|
|
```python
|
|
class NotesInput(BaseModel):
|
|
text: str
|
|
|
|
|
|
class ReportState(BaseModel):
|
|
notes: str = ""
|
|
report: ReportOutput | None = None
|
|
markdown: str = ""
|
|
|
|
|
|
class ReportResult(BaseModel):
|
|
report: ReportOutput
|
|
markdown: str
|
|
|
|
|
|
graph = app.new_workflow(
|
|
"report_python_showcase",
|
|
input_schema=NotesInput,
|
|
state_schema=ReportState,
|
|
output_schema=ReportResult,
|
|
)
|
|
```
|
|
|
|
The models export schemas for the workflow contract. They do not make the
|
|
saved workflow dependent on a live Python class instance. The initially
|
|
absent report belongs to intermediate state; the public result requires a
|
|
report because a completed successful pipeline should have produced one.
|
|
|
|
## Connecting Data and Decisions
|
|
|
|
The author now creates three node uses. Each use selects an operation and
|
|
declares its data bindings:
|
|
|
|
```python
|
|
read = graph.use(
|
|
read_notes,
|
|
id="read",
|
|
input=[input_from(input_path("text"), "text")],
|
|
output=[output_to("text", state_path("notes"))],
|
|
)
|
|
extract = graph.use(
|
|
extract_report,
|
|
id="extract",
|
|
input=[input_from(state_path("notes"), "text")],
|
|
output=[output_to((), state_path("report"))],
|
|
)
|
|
render = graph.use(
|
|
render_report,
|
|
id="render",
|
|
input=[input_from(state_path("report"), "report")],
|
|
output=[output_to("markdown", state_path("markdown"))],
|
|
)
|
|
|
|
end = graph.end("ok", id="finished")
|
|
graph.set_entry_point(read)
|
|
graph.connect(read, "ok", extract)
|
|
graph.connect(extract, "ok", render)
|
|
graph.connect(render, "ok", end)
|
|
graph.set_output([
|
|
input_from(state_path("report"), "report"),
|
|
input_from(state_path("markdown"), "markdown"),
|
|
])
|
|
```
|
|
|
|
The empty tuple in `output_to((), ...)` selects the extraction step's
|
|
whole output object. The other output bindings select individual fields.
|
|
The `connect` calls then specify execution order for the `ok` outcome;
|
|
they do not implicitly carry those objects between steps.
|
|
|
|
This explicitness is both a benefit and an authoring cost. The mapping is
|
|
inspectable, and changing a route does not silently change a data source.
|
|
However, even a linear three-step procedure requires contracts, bindings,
|
|
and routes. Typed helpers reduce raw serialization work without removing
|
|
the need to understand these distinctions.
|
|
|
|
## Diagnosing and Repairing a Binding
|
|
|
|
An editable graph can temporarily be invalid. Suppose a final output binding
|
|
names a state field that does not exist:
|
|
|
|
```python
|
|
graph.set_output([
|
|
input_from(state_path("missing_report"), "report"),
|
|
input_from(state_path("markdown"), "markdown"),
|
|
])
|
|
broken = await graph.validate()
|
|
assert not broken.ok
|
|
assert broken.remote_status == "not_run"
|
|
for issue in broken.local.errors:
|
|
print(issue.code, issue.path, issue.message)
|
|
|
|
graph.set_output([
|
|
input_from(state_path("report"), "report"),
|
|
input_from(state_path("markdown"), "markdown"),
|
|
])
|
|
(await graph.validate()).raise_for_errors()
|
|
```
|
|
|
|
The local report identifies the invalid source path in the workflow's output
|
|
projection. No server validation request is made for that invalid graph.
|
|
The repair changes the projection, not the renderer or its outgoing edge.
|
|
This illustrates why data bindings and control routes need separate feedback.
|
|
The next section saves only the repaired definition.
|
|
|
|
## Saving a Version and Choosing Its Environment
|
|
|
|
Before saving, the author can request validation and inspect its diagnostics.
|
|
The example stops on errors:
|
|
|
|
```python
|
|
validation = await graph.validate()
|
|
validation.raise_for_errors()
|
|
artifact = await graph.save(version=1)
|
|
|
|
deployment = await artifact.deploy(
|
|
"report_python_showcase.local",
|
|
bindings={"local.report": "local.report"},
|
|
)
|
|
readiness = await deployment.validate()
|
|
if not readiness.runnable:
|
|
raise RuntimeError(readiness.diagnostics)
|
|
```
|
|
|
|
The artifact is the saved version of the authored procedure. The deployment
|
|
binds its logical source requirement to the configured source. Both happen
|
|
to be called `local.report` here; the mapping still records an environment
|
|
choice rather than a new graph edge.
|
|
|
|
The sequence below summarizes the public operations; it omits internal
|
|
validation and re-inspection calls made by individual client methods.
|
|
|
|
```{.mermaid #fig:python-lifecycle caption="Authoring through the service."}
|
|
sequenceDiagram
|
|
actor Author
|
|
participant Client as Python client
|
|
participant API as Workflow service
|
|
participant Provider as Bound operation
|
|
Author->>Client: Build and revise graph
|
|
Client->>Client: Check structure locally
|
|
Client->>API: Validate and save definition
|
|
API-->>Client: Saved artifact version
|
|
Client->>API: Bind deployment to version
|
|
API-->>Client: Deployment and readiness
|
|
Client->>API: Run with input
|
|
API->>Provider: Invoke graph steps
|
|
Provider-->>API: Outputs and outcomes
|
|
API-->>Client: Stopped run snapshot
|
|
Client->>API: Inspect run and bounded trace
|
|
API-->>Client: Stored execution evidence
|
|
Client-->>Author: Result or diagnostic
|
|
```
|
|
|
|
An edit to the workflow would be saved as another version, not applied
|
|
retroactively to the artifact used by this deployment. Conversely, selecting
|
|
a different source environment is a deployment concern. A readiness check
|
|
can reject a missing or incompatible binding before a run is attempted.
|
|
|
|
## Running and Inspecting the Result
|
|
|
|
The client reads the notes and sends their contents. The server therefore
|
|
does not need access to the client's file path.
|
|
|
|
```python
|
|
notes = Path("examples/report_workflow/input.md").read_text(encoding="utf-8")
|
|
run = await deployment.run({"text": notes}, max_steps=100)
|
|
if run.status != "completed":
|
|
raise RuntimeError((run.status, run.diagnostics))
|
|
|
|
result = ReportResult.model_validate(run.output)
|
|
assert result.report.title == "Weekly Project Update"
|
|
assert len(result.report.action_items) == 3
|
|
assert result.markdown.startswith("# Weekly Project Update")
|
|
|
|
run = await run.refresh()
|
|
trace = await run.trace(start=0, limit=10)
|
|
```
|
|
|
|
The run exposes a status, output, and diagnostics independently of the
|
|
editable graph. The output crosses the API as data; the explicit
|
|
`model_validate` call reconstructs the example's Pydantic result model.
|
|
Refreshing obtains the latest stored snapshot, while the bounded trace
|
|
provides step-level evidence when output alone is insufficient.
|
|
|
|
For this fixed fixture, the result includes three action items, the
|
|
recorded risks and followups, and a Markdown report headed
|
|
“Weekly Project Update.” Checking these fields establishes that the
|
|
example's data reached the intended outputs. It does not establish that
|
|
the report is useful for every reader or that extraction works on
|
|
unstructured notes.
|
|
|
|
## What This Case Demonstrates
|
|
|
|
The existing tests in
|
|
[`test_report_workflow_example.py`](../../tests/examples/test_report_workflow_example.py)
|
|
check the source's input rules, rendering and extraction, capability
|
|
discovery and invocation, and the artifact/deployment/run lifecycle using
|
|
the raw-plan fixture. Those tests are evidence for the report operations
|
|
and lifecycle. They are not a user study of the Python walkthrough.
|
|
|
|
The Python presentation makes the current authoring experience concrete:
|
|
inspect operations, declare contracts, connect data and outcomes, save,
|
|
select bindings, and inspect an execution. The CLI and draft surface offer
|
|
another way to perform related lifecycle operations; they are not required
|
|
steps in this Python walkthrough.
|
|
|
|
A direct Python script would be shorter for these three local functions.
|
|
The workflow system earns its additional structure when the definition
|
|
must be saved, bound to an environment, validated independently, and
|
|
inspected through a shared service. This example demonstrates that
|
|
integration, not a performance advantage over function calls.
|
|
|
|
Nor does a linear pipeline exercise all graph semantics. It has no
|
|
conditional branch, foreach body, child workflow, or interrupt. Targeted
|
|
runtime tests provide evidence for those mechanisms; they should not be
|
|
credited to a case that never executes them. Ease of authoring and diagnosis
|
|
also requires evidence beyond a successful fixture run.
|
|
|
|
# Evaluation
|
|
|
|
Evaluation distinguishes three questions: whether the runtime follows its
|
|
contract, whether the public lifecycle composes correctly, and whether an
|
|
author can use that lifecycle effectively. The current evidence addresses
|
|
the first two through controlled tests and an executable walkthrough.
|
|
Evaluation of the intended shell-backed authoring experience remains pending.
|
|
|
|
## Prototype Conformance Criteria
|
|
|
|
The implementation should preserve definition and execution identity, reject
|
|
invalid structures and bindings, apply declared state updates, and expose
|
|
stopped runs for inspection or explicit resume. It should also allow
|
|
configured source families to supply operations without making the graph
|
|
scheduler specific to one remote protocol.
|
|
|
|
These are conformance criteria, not claims of general reliability.
|
|
The evidence index gives the source and test paths behind each row below.
|
|
A referenced suite identifies where a behavior is tested; it does not imply
|
|
that every suite was rerun during this document revision.
|
|
|
|
| Claim | Evidence | Boundary |
|
|
| --- | --- | --- |
|
|
| Separate versions and runs | Lifecycle; E1 | No universal portability |
|
|
| Python lifecycle objects | Walkthrough; E2 | Not a usability study |
|
|
| Invalid bindings diagnosed | Validation; E3 | Not business correctness |
|
|
| Nested execution ownership | Runtime; E4 | No general fork/gather |
|
|
| Persisted step limits | Budget; E4 | Not a handler timeout |
|
|
| Provider-supplied operations | Sources; E5 | Unequal provider features |
|
|
| Explicit interrupt resume | Resume; E1 | Not arbitrary crash replay |
|
|
|
|
: Claims, evidence, and boundaries. {#tbl:prototype-conformance}
|
|
|
|
## Current Walkthrough Check
|
|
|
|
During this revision on September 7, 2026, an adapted version of the Python
|
|
blocks in the report walkthrough was executed as an in-process smoke check
|
|
against the example server configuration with an isolated temporary store.
|
|
The adapted check used an in-process client port in place of the displayed
|
|
HTTP connection; the walkthrough blocks were not executed unchanged. It
|
|
exercised discovery, graph construction, validation, saving, deployment,
|
|
execution, refresh, and trace inspection.
|
|
|
|
The fixed fixture produced the expected title, three action items, and
|
|
Markdown heading. A deliberate invalid output binding was rejected before
|
|
saving; restoring the projection allowed validation and execution to proceed.
|
|
This adds a repair example without claiming that a new user would find the
|
|
diagnostic sufficient.
|
|
|
|
The check does not exercise network startup or HTTP transport, and is not a
|
|
performance benchmark. The documentation and report-example test files are
|
|
also run together as a focused verification target:
|
|
|
|
```powershell
|
|
$suites = @(
|
|
"tests/docs/test_big_doc_links.py",
|
|
"tests/examples/test_report_workflow_example.py"
|
|
)
|
|
uv run pytest @suites -q -n 0
|
|
```
|
|
|
|
Those tests cover document integration and the existing raw-plan fixture.
|
|
They do not independently execute every Python block in the thesis; the
|
|
walkthrough execution is a separate editorial smoke check.
|
|
|
|
## Separating Design Comparison from Evaluation
|
|
|
|
The earlier comparison of n8n, Zapier, and LangGraph explains different
|
|
authoring and execution choices. The same task has not been measured across
|
|
those systems under matched conditions. This report therefore cannot rank
|
|
their usability, reliability, or performance against the prototype.
|
|
|
|
Execution tests ask whether contracts, routing, state updates, and persistence
|
|
behave as specified. Interaction evaluation asks whether an author can
|
|
discover operations, express a procedure, understand errors, and recover
|
|
without inspecting implementation code. Passing one kind of test does not
|
|
answer the other.
|
|
|
|
The report walkthrough exposes costs such as explicit bindings and deployment
|
|
selection, but does not measure whether those costs are acceptable to new
|
|
users. Similarly, structured diagnostics and inspection objects may help an
|
|
agent avoid trial and error, but reduced retries, token use, and repair time
|
|
remain hypotheses rather than measured outcomes.
|
|
|
|
## Current Evaluation Boundaries
|
|
|
|
A new agent evaluation should target the actual application-facing tools,
|
|
including the intended shell-backed workflow interface once integrated.
|
|
Its allowed operations, environment, task fixtures, and success criteria
|
|
must be fixed before collecting results. An agent's self-report should be
|
|
checked against saved artifacts, deployment identity, run output, and
|
|
recorded interactions.
|
|
|
|
The earlier CLI evaluation is retained separately in
|
|
[repository history](../historical/thesis/2026-09-07-retired-agent-evaluation.md).
|
|
It is not included as evidence for the current authoring experience, and
|
|
there are no replacement campaign results in this thesis. A demonstration
|
|
of the surrounding application will not by itself establish authoring
|
|
success across tasks or users.
|
|
|
|
## Falsifiability Criteria
|
|
|
|
The implementation would fail its stated contracts if, for example:
|
|
|
|
- a saved run could not identify the definition and bindings it used;
|
|
- an invalid foreach boundary were accepted and executed as another region;
|
|
- nested state writes were lost or counted twice;
|
|
- resume reset the run-wide budget or resumed the wrong item;
|
|
- condition evaluation used a different context model from validation;
|
|
- ordinary source invocation required provider-specific graph routing.
|
|
|
|
These cases support focused regression tests. Broader architectural claims,
|
|
such as accommodating future source families without changing the runtime,
|
|
remain design expectations to assess as those integrations are built.
|
|
|
|
# Limitations
|
|
|
|
The prototype demonstrates a workflow lifecycle under controlled conditions.
|
|
Its main limitations concern how much authors must understand, which
|
|
execution guarantees are provided, and how far the available evidence can
|
|
be generalized.
|
|
|
|
## Authoring and Diagnosis Still Require Technical Knowledge
|
|
|
|
The Python client reduces manual serialization and provides editable graphs
|
|
and inspectable objects. It does not remove the need to understand schemas,
|
|
state bindings, outcomes, and deployment selection. The report example makes
|
|
this cost visible: a short procedure requires more declarations than direct
|
|
function calls.
|
|
|
|
Diagnostics identify many invalid structures and bindings, but an accurate
|
|
message is not necessarily an understandable repair instruction. Authors
|
|
still need to distinguish a graph error from an environment problem or a
|
|
failed external operation. The current evidence does not establish that new
|
|
users can make these distinctions without assistance.
|
|
|
|
Inspection also requires judgment. A trace shows recorded execution, not
|
|
whether a report is factually correct or a remote side effect was desirable.
|
|
The system remains in development, and its authoring and operational
|
|
experience needs evaluation and refinement.
|
|
|
|
## Execution Guarantees Have Defined Boundaries
|
|
|
|
Foreach iteration, nested workflow scopes, structured context, and run-wide
|
|
step budgets are implemented foundations. They do not yet provide general
|
|
fork/gather control for arbitrary branches. In particular, concurrent
|
|
iteration should not be presented as a solution to correlating branches
|
|
that split, loop, and later meet at different gather points.
|
|
|
|
Persisted interrupted runs can resume at explicit boundaries. This is not
|
|
arbitrary mid-handler crash recovery, replay of every external call, or an
|
|
exactly-once side-effect guarantee. A run's step limit bounds admitted graph
|
|
steps; it does not bound a handler's execution time or the cost of its
|
|
external requests.
|
|
|
|
Schema validation checks declared structure, not business truth. A returned
|
|
report may satisfy its schema while containing incorrect information.
|
|
Similarly, a fixed graph specifies routing but does not make remote
|
|
responses or concurrent completion order reproducible.
|
|
|
|
## Deployment and Trust Assumptions
|
|
|
|
The controlled examples assume trusted operators and trusted Python sources.
|
|
Python operations execute in the server process without a sandbox. The
|
|
thesis does not establish multi-tenant isolation, role-based authorization,
|
|
or production-grade credential management. An explicit workflow interrupt
|
|
can request a response, but is not by itself an authenticated approval or
|
|
access-control mechanism.
|
|
|
|
Source bindings make environment choices inspectable rather than making
|
|
workflows universally portable. A destination environment still needs
|
|
compatible operations, credentials, and dependencies. Python sources are
|
|
loaded at startup; changing their code requires a server restart. The shared
|
|
provider interface does not yet unify every provider's administration,
|
|
authentication, and live health behavior.
|
|
|
|
Filesystem-backed stores support the demonstrated persistence paths. Their
|
|
use does not establish production performance, cross-process contention
|
|
behavior at scale, or disaster recovery. A future database implementation
|
|
would still need to preserve the lifecycle's transaction and ownership
|
|
contracts; changing the storage engine alone would not prove those
|
|
properties.
|
|
|
|
Scheduled deployment execution is implemented within the documented first
|
|
slice. It remains bounded by the filesystem-backed, single-process store and
|
|
the explicitly enabled local/static server composition. Nor does exposing a
|
|
provider's callable operations imply that its interactive widgets or entire
|
|
user experience are reproduced through the workflow API.
|
|
|
|
## Limits of the Evidence
|
|
|
|
The deterministic report fixture demonstrates lifecycle integration, not
|
|
broad document understanding or graph expressiveness. Targeted tests cover
|
|
additional execution mechanisms, but their passing results apply to the
|
|
cases and revisions tested.
|
|
|
|
The current walkthrough check verifies documented calls against an isolated
|
|
service API, not network deployment or usability for an independent author.
|
|
An evaluation of the intended shell-backed application remains pending.
|
|
|
|
There is no matched cross-system experiment or broad human user study.
|
|
Consequently, this report cannot claim that the prototype is easier to use,
|
|
more reliable, or faster than the systems discussed earlier. The comparison
|
|
explains design choices; the implementation evidence tests this system's
|
|
own behavior.
|
|
|
|
# Future Work
|
|
|
|
The next work should strengthen execution semantics without losing sight
|
|
of the author who must understand them. The live roadmap records engineering
|
|
order; the priorities below explain why that work matters to this design.
|
|
|
|
## Establish General Fork and Gather Semantics
|
|
|
|
The immediate runtime direction is to consolidate identity resolution and
|
|
then establish explicit fork/gather behavior. Existing frames, scopes,
|
|
iteration activations, and state lineages provide foundations, but their
|
|
relationships must remain coherent through nested execution and resume.
|
|
|
|
A fork creates concurrent execution branches; a gather must determine
|
|
which arriving branches belong together before combining their state.
|
|
Loops, partial gathers, and repeated visits make this more than waiting
|
|
for a fixed number of arrivals. The design must also preserve contribution
|
|
identity so that a write already included in one merge is not applied
|
|
again in a later merge.
|
|
|
|
This work remains planned, with reference-model verification preceding
|
|
production implementation. Pressure cases should become executable tests
|
|
for correlation, merge behavior, and recovery. The authoring contract also
|
|
needs to make clear which graphs are rejected before execution and which
|
|
decisions remain the author's responsibility.
|
|
|
|
## Evaluate the Authoring and Recovery Experience
|
|
|
|
A focused usability study should ask participants to discover an operation,
|
|
build a small workflow, change its contract, diagnose a broken binding, and
|
|
inspect a failed or interrupted run. Useful measures include task completion,
|
|
time to a correct repair, unnecessary retries, and reliance on source-code
|
|
inspection.
|
|
|
|
Execution and interaction should be evaluated together without conflating
|
|
them. For example, a validation rule may correctly reject a graph while its
|
|
diagnostic fails to explain the ownership boundary that was crossed.
|
|
Conversely, a convenient editing operation must not hide a change to the
|
|
workflow's execution meaning.
|
|
|
|
The Python client and CLI should receive evidence appropriate to their own
|
|
interaction styles. Broader agent trials can vary tasks and instruction
|
|
profiles, while human evaluation can test whether the lifecycle vocabulary
|
|
and data-binding model are understandable without implementation knowledge.
|
|
|
|
## Scheduling and the Surrounding Application
|
|
|
|
Scheduled deployment execution is implemented for the current slice. It
|
|
introduces trigger identity, overlap policy, and recovery decisions in
|
|
addition to time-expression parsing, and builds on the same run lifecycle
|
|
rather than creating a separate execution model. Extending it to distributed
|
|
workers or suspending an already-running workflow until a time or event is a
|
|
related but distinct design question; a wait node is not specified here.
|
|
|
|
The surrounding application is intended to combine assistant-backed chat
|
|
with workflow administration. A shell can let an assistant retain Python
|
|
objects across interactions, while typed client objects can support
|
|
dedicated views of artifacts, deployments, and runs. Specialized display
|
|
payloads, potentially using MIME types, are a presentation option to
|
|
investigate rather than an established public contract.
|
|
|
|
This direction does not establish a completed workflow-agent integration.
|
|
It needs a concrete interaction design, demonstrated public-client use,
|
|
and its own evaluation before contributing success claims to the thesis.
|
|
|
|
## Extend Operations When Concrete Use Requires Them
|
|
|
|
Provider expansion and operational hardening should similarly follow actual
|
|
requirements. Examples include broader OpenAPI coverage, source reload,
|
|
secret-manager integration, and an alternative storage backend. Each needs
|
|
its own compatibility, failure, and deployment evidence; none follows
|
|
automatically from the existence of a provider or store interface.
|
|
|
|
Richer debugging should clarify what can safely be resumed or repeated,
|
|
especially around external side effects. Showing more trace information is
|
|
different from promising that an earlier action can be undone.
|
|
|
|
# Conclusion
|
|
|
|
This report examined how a useful procedure can become a reusable workflow
|
|
that an author can define and an operator can inspect. The implemented
|
|
system separates the saved definition, its environment bindings, and each
|
|
execution into artifacts, deployments, and runs. Those distinctions give
|
|
workflow use a record beyond the lifetime of an editing session or a single
|
|
script invocation.
|
|
|
|
The graph model separates data movement from control movement. Contracts
|
|
and bindings describe what a step receives and writes; outcomes choose
|
|
transitions; explicit runtime constructs govern iteration, child scopes,
|
|
and interruption. The Python client exposes this model through authoring
|
|
and inspection objects, while the API and provider boundaries connect it
|
|
to configured operations.
|
|
|
|
The report case demonstrates that a small typed procedure can be saved,
|
|
deployed, executed, and inspected. Targeted tests support specific
|
|
validation, state, and persistence behaviors. These results support the
|
|
feasibility of the design under the tested conditions,
|
|
not a claim of production readiness or superior usability.
|
|
|
|
The central trade-off remains visible: explicit contracts and lifecycle
|
|
boundaries improve inspectability but ask authors to understand more than
|
|
a sequence of function calls. The next stage must therefore test both the
|
|
correctness of richer execution semantics and the clarity of the experience
|
|
used to author and diagnose them. A workflow system is useful only when its
|
|
execution rules are dependable and its users can understand what they have
|
|
asked it to do.
|
|
|
|
<!-- References -->
|
|
# References {#sec:refs .unnumbered}
|
|
|
|
::: {#refs}
|
|
:::
|
|
|
|
\appendix
|
|
<!-- Appendices -->
|
|
# Evidence Index
|
|
|
|
This appendix maps the evaluation's evidence identifiers to implementation
|
|
and tests. Paths identify inspectable evidence; they are not a claim that
|
|
all listed suites passed in one newly recorded full-system run.
|
|
|
|
## Core Workflow Lifecycle
|
|
|
|
E1: artifacts, deployments, stopped runs, and explicit resume boundaries.
|
|
|
|
- `src/wf_artifacts/models.py`
|
|
- `src/wf_artifacts/runs/`
|
|
- `src/wf_api/run_lifecycle.py`
|
|
- `tests/wf_api/test_artifact_api.py`
|
|
- `tests/wf_api/test_run_api.py`
|
|
|
|
## Python Authoring and Inspection
|
|
|
|
E2: reconstructed client objects, editable workflows, and the report fixture.
|
|
|
|
- `src/wf_client/`
|
|
- `tests/wf_client/test_authoring.py`
|
|
- `tests/wf_client/test_deployments.py`
|
|
- `tests/wf_client/test_runs.py`
|
|
- `examples/report_workflow/`
|
|
- `tests/examples/test_report_workflow_example.py`
|
|
|
|
The example's README retains the command-line route for operators who need
|
|
it. That alternative interface is not an additional evaluated case in the
|
|
current thesis.
|
|
|
|
## Validation and Diagnostics
|
|
|
|
E3: structural validation, source compatibility, and repair information.
|
|
|
|
- `src/wf_core/validation/`
|
|
- `src/wf_artifacts/validation.py`
|
|
- `tests/artifacts/test_validation.py`
|
|
- `tests/core/test_structured_context_validation.py`
|
|
- `tests/core/test_foreach_control_regions.py`
|
|
|
|
## Execution Ownership and Budgets
|
|
|
|
E4: nesting, structured context, concurrent iteration, and persisted limits.
|
|
|
|
- `src/wf_core/runtime/`
|
|
- `tests/core/test_foreach_back_edges.py`
|
|
- `tests/core/test_concurrent_foreach_interrupts.py`
|
|
- `tests/core/test_structured_runtime_context.py`
|
|
- `tests/core/test_run_step_budget.py`
|
|
- `tests/core/test_run_step_budget_codec.py`
|
|
- `tests/core/test_run_step_budget_async.py`
|
|
|
|
## Source Provider Boundary
|
|
|
|
E5: source contracts and provider-specific execution behind server composition.
|
|
|
|
- `src/wf_platform/sources.py`
|
|
- `src/wf_server/config.py`
|
|
- `src/wf_sources_python/`
|
|
- `src/wf_sources_mcp/`
|
|
- `tests/wf_sources_python/test_loader.py`
|
|
- `tests/wf_sources_mcp/test_runtime.py`
|
|
- `tests/wf_transport_rpc_http/test_mcp_backed_server_rpc.py`
|
|
|
|
The existing providers demonstrate this separation for their implemented
|
|
operations. They do not establish equal lifecycle features across providers
|
|
or prove compatibility with every future source family.
|