2070 lines
94 KiB
Markdown
2070 lines
94 KiB
Markdown
---
|
||
title: "Design and Implementation of lda.chat: An AI Agent for Automating and Creating Workspace Workflows"
|
||
subtitle: ""
|
||
date: "July 1, 2026"
|
||
lang: "en-US"
|
||
documentclass: report
|
||
papersize: a4
|
||
fontsize: 10pt
|
||
toc: true
|
||
toc-depth: 2
|
||
toc-title: "Table of Contents"
|
||
lof: true
|
||
lot: true
|
||
numbersections: true
|
||
bibliography: references.bib
|
||
link-citations: true
|
||
figureTitle: "Figure"
|
||
figPrefix: "Figure"
|
||
tblPrefix: "Table"
|
||
chapters: true
|
||
appendix: true
|
||
syntax-highlighting: idiomatic
|
||
geometry:
|
||
- top=30mm
|
||
- bottom=30mm
|
||
- left=32mm
|
||
- right=32mm
|
||
mainfont: "Libertinus Serif"
|
||
sansfont: "Libertinus Sans"
|
||
monofont: "Libertinus Mono"
|
||
mathfont: "Libertinus Math"
|
||
colorlinks: true
|
||
linkcolor: "MidnightBlue"
|
||
urlcolor: "MidnightBlue"
|
||
toccolor: "MidnightBlue"
|
||
keywords:
|
||
- workflow
|
||
- agents
|
||
- source providers
|
||
- JSON-RPC
|
||
- MCP
|
||
- Python sources
|
||
header-includes:
|
||
# you can not specify -H and this at the same time.
|
||
diagram:
|
||
engine:
|
||
mermaid:
|
||
theme: neutral
|
||
---
|
||
|
||
# List of Abbreviations {.unnumbered}
|
||
|
||
<!-- The heading identifies this captionless table; it has no table number. -->
|
||
| Abbreviation | Meaning |
|
||
| --- | --- |
|
||
| API | Application Programming Interface |
|
||
| AI | Artificial Intelligence |
|
||
| CLI | Command-Line Interface |
|
||
| JSON-RPC | JavaScript Object Notation Remote Procedure Call |
|
||
| LLM | Large Language Model |
|
||
| MCP | Model Context Protocol |
|
||
| RPC | Remote Procedure Call |
|
||
| USTH | University of Science and Technology of Hanoi |
|
||
|
||
# Abstract {.unnumbered}
|
||
|
||
Preparing reports, transforming documents, and collecting workspace information
|
||
often involve procedures that must be repeated with new inputs. An AI assistant
|
||
can help perform such work. Repeating it requires preserving the operations
|
||
and their data connections as an executable procedure. This thesis develops
|
||
the programmable workflow subsystem of `lda.chat` for defining, checking,
|
||
running, and inspecting reusable workspace procedures.
|
||
|
||
A workflow describes the operations to perform, the data they exchange, and
|
||
the decisions that select the next step. An **artifact** is an immutable saved
|
||
version of that workflow. A **deployment** connects a saved version to the
|
||
concrete services it will use. A **run** records one execution, including its
|
||
inputs, status, result, and trace. These distinctions separate revising a
|
||
procedure, configuring where it operates, and examining what happened.
|
||
|
||
The design addresses both authoring experience and execution behavior. Authors
|
||
need to discover operations, connect their inputs and outputs, understand
|
||
validation errors, and inspect results. The runtime needs corresponding rules
|
||
for data contracts, control flow, state updates, and interruption. The thesis
|
||
compares these concerns with other workflow systems and explains the prototype's
|
||
choice to separate a node's data output from its routing outcome.
|
||
|
||
The implementation includes a workflow runtime, Python authoring objects, a
|
||
workflow service, and adapters for trusted Python functions and external tools.
|
||
The evaluation focuses on workflow construction and execution through the
|
||
Python client. It uses a deterministic case study and identifies focused
|
||
tests of validation, execution, and persistence. Agent-driven authoring and
|
||
authoring usability remain to be evaluated as the interaction design develops.
|
||
|
||
# Introduction
|
||
|
||
Preparing a weekly project report involves collecting notes, extracting actions
|
||
and risks, checking the result, and producing a document that other people can
|
||
use. A person can do this manually, write a script, or ask an AI assistant for
|
||
help. When the procedure becomes recurring, the workspace also needs a way to
|
||
preserve it, supply different inputs, and inspect unsuccessful attempts.
|
||
|
||
`lda.chat` turns such a procedure into a reusable workflow: a saved description
|
||
of operations, their data connections, and their execution order. An author
|
||
can revise this description, connect it to available services, and run it
|
||
with new inputs. Each execution has a separate record of its progress and
|
||
results. The procedure remains available after the conversation or programming
|
||
session that created it has ended.
|
||
|
||
For the report task, the initial procedure is short:
|
||
|
||
Read the notes, extract a structured report, and render it as Markdown.
|
||
|
||
The operator supplies new notes each week using the saved steps.
|
||
If the requirements change, the author revises the procedure and saves another
|
||
version. If execution fails, the operator needs to identify the affected step
|
||
and its inputs.
|
||
|
||
## From a useful procedure to a usable system
|
||
|
||
The intended beneficiary is a workspace operator who needs repeatable work.
|
||
Its author may be that person, a developer, or an external agent acting on their
|
||
behalf. Once saved, the procedure can run with or without an LLM operation,
|
||
according to the steps its author included.
|
||
|
||
A workflow system must help its author answer practical questions: which
|
||
operations are available, what information they require, and how one operation's
|
||
result becomes another's input. During operation, it must distinguish an invalid
|
||
definition, a missing service, a failed execution, and a request for more input.
|
||
The **runtime** is the part of the system that executes the workflow: it invokes
|
||
steps, maintains working data, and determines where execution continues.
|
||
These are interaction-design concerns as well as runtime concerns. An interface
|
||
that draws a branch without explaining whether one path or both will execute
|
||
leaves a consequential rule implicit.
|
||
|
||
The current authoring interface is Python-based. It presents workflow and run
|
||
objects for discovering operations, revising a graph, and inspecting execution.
|
||
An external agent can use that interface through a Python execution environment.
|
||
The interface and runtime are under development, with the implemented behavior
|
||
and its supporting evidence examined in this thesis.
|
||
|
||
## Engineering question and contribution
|
||
|
||
This thesis examines how authors can define a reusable procedure while the
|
||
system manages its individual executions. Authors specify data connections
|
||
and decisions about what happens next. The system must preserve those rules
|
||
when steps repeat, when one workflow calls another, and when execution pauses
|
||
for input.
|
||
Separating these responsibilities allows a saved procedure to be executed
|
||
without retaining the conversation or programming session that produced it.
|
||
|
||
The prototype implements three design choices as one lifecycle:
|
||
|
||
1. A graph with declared data structures represents operations, working data,
|
||
and routing decisions so they can be validated and inspected.
|
||
2. Separate saved versions, service configuration, and execution records make
|
||
editing, configuration, and operation distinct activities.
|
||
3. Programmable authoring, validation diagnostics, and run inspection expose
|
||
those distinctions to human and agent clients.
|
||
|
||
The work integrates established workflow techniques into a service for reusable
|
||
procedures. Its design is examined from both sides:
|
||
what an author must understand and do, and what the runtime guarantees when
|
||
the procedure executes. The comparison with related systems examines concrete
|
||
authoring and execution mechanisms.
|
||
|
||
## Scope of the implementation
|
||
|
||
The implemented runtime supports conditional routes, iteration, calls to saved
|
||
workflows, and explicit interruption/resume. A run-wide limit bounds the number
|
||
of node executions it admits. General parallel fork/gather remains proposed.
|
||
The evaluation covers controlled execution and lifecycle tests, with usability
|
||
and production operation identified as further evaluation work. A companion
|
||
assistant backend can execute code in persistent Python shells. Its workflow
|
||
authoring integration and developing chat interface are discussed in Future
|
||
Work, outside the evaluated workflow case study.
|
||
|
||
## Report outline
|
||
|
||
Chapter 2 derives interaction and execution requirements from the workspace
|
||
task. Chapter 3 compares ways to author and operate that task in related
|
||
systems. Chapter 4 explains the prototype's concepts through an example.
|
||
Chapters 5 and 6 describe architecture and implementation; Chapter 7 presents
|
||
the reproducible case study. Chapter 8 evaluates the available evidence.
|
||
The remaining chapters discuss limitations, future work, and conclusions.
|
||
|
||
# Problem Statement And Requirements
|
||
|
||
A procedure that works once is not necessarily ready for repeated operation.
|
||
For the report example, a renamed field may invalidate extraction, a different
|
||
workspace may use another notes service, or the input may omit information
|
||
needed by the final document. The author needs feedback that distinguishes these
|
||
situations, while the runtime needs rules for handling them.
|
||
|
||
LLM tool-use approaches illustrate dynamic selection of actions
|
||
[@react-2022; @toolformer-2023]. Such approaches can incorporate schemas and
|
||
persistence. This thesis examines which responsibilities belong in the saved
|
||
procedure and which remain with its authoring environment.
|
||
|
||
The requirements are organized into two groups. R1–R5 describe the authoring
|
||
and operation experience the system should support. X1–X5 define the execution
|
||
rules needed to support that experience. Their identifiers link the design
|
||
to the evidence assessment in [@tbl:requirements-evidence].
|
||
|
||
## Authoring and operation requirements
|
||
|
||
The authoring requirements cover discovery, data movement, revision, version
|
||
selection, and execution inspection:
|
||
|
||
1. **R1 — Discover before connecting.** Show available operations and the inputs,
|
||
outputs, and services they require. The author should not need to inspect
|
||
server implementation code to learn how an operation can be used.
|
||
2. **R2 — Make data movement understandable.** Explain how a document path becomes
|
||
text, how text becomes structured fields, and which fields reach the result.
|
||
Distinguish a data connection from a decision about what executes next.
|
||
3. **R3 — Support revision and useful feedback.** An author should be able to revise
|
||
an unfinished procedure and locate errors in the relevant step or data
|
||
mapping. Diagnostics should locate the rejected part and explain the
|
||
violated constraint.
|
||
4. **R4 — Separate editing from running.** Make clear which version an execution
|
||
uses. Editing the next version should not silently change an earlier one.
|
||
5. **R5 — Explain execution state.** Distinguish completed, failed, and interrupted
|
||
runs, expose relevant intermediate evidence, and identify the input required
|
||
to resume an interruption.
|
||
|
||
Agent-assisted authoring places particular demands on this interaction. An
|
||
agent must discover the available operations, interpret a rejected definition,
|
||
and determine which changes are permitted without relying on implementation
|
||
files. This motivates three complementary forms of support: descriptions of
|
||
operations and their contracts, diagnostics that locate errors, and instructions
|
||
for the authoring and execution lifecycle. Structured responses, stable
|
||
identities, and inspection results with explicit size limits make that information
|
||
available to
|
||
programmatic clients. Human authors and other software clients use the same
|
||
information to construct and operate workflows.
|
||
|
||
A saved definition follows the same routing and data rules
|
||
whether it was written by a developer or assembled by an agent. The evaluation
|
||
therefore examines ease of use and compliance with execution rules as separate
|
||
questions.
|
||
|
||
## Execution requirements behind the interaction
|
||
|
||
Those interactions require corresponding runtime contracts (X1–X5):
|
||
|
||
1. **X1 — Preserve definitions and executions independently.** Save an identifiable
|
||
procedure version and keep separate records for its invocations.
|
||
2. **X2 — Validate known constraints before work starts.** Check graph structure,
|
||
declared data contracts, mappings, and required service connections.
|
||
3. **X3 — Specify routing and state changes.** Define what selects the successor,
|
||
where outputs are written, and how repeated writes affect the workflow's
|
||
working data, referred to here as workflow state.
|
||
4. **X4 — Separate environment choices from procedure logic.** Resolve logical
|
||
service requirements to concrete configured services and report mismatches.
|
||
5. **X5 — Bound and inspect execution.** Limit admitted node executions, retain
|
||
status and trace information, and support resume at defined interruption
|
||
boundaries.
|
||
|
||
Together, these requirements motivate separate records for saved procedures,
|
||
environment selection, and individual executions.
|
||
|
||
## Costs and boundaries
|
||
|
||
Declaring data structures, mappings, and saved versions imposes authoring work.
|
||
A script
|
||
can be preferable for a short-lived task, especially when its author already
|
||
understands the libraries involved. The platform targets cases where reuse and
|
||
inspection justify that setup. Determining when those benefits outweigh the
|
||
setup cost requires further evaluation.
|
||
|
||
A workflow interface must also avoid promising more than its runtime supports.
|
||
Successful validation does not guarantee a remote service will succeed, and a
|
||
saved interruption does not make arbitrary external effects reversible.
|
||
|
||
# Positioning And Related Systems
|
||
|
||
The report task provides a common example for comparing authoring and execution.
|
||
The comparison asks how an author connects operations and data, how the runtime
|
||
handles decisions and repeated work, and how the author tests and inspects a
|
||
run. These questions apply to graphical editors and Python interfaces alike.
|
||
|
||
The accounts below use documented mechanisms consulted in September 2026.
|
||
They describe selected interaction and execution mechanisms. Comparative
|
||
usability would require participants performing matched tasks.
|
||
|
||
## Starting with code or direct tool calls
|
||
|
||
A developer can express the report procedure as function calls and use the
|
||
language's conditionals and loops. An agent can instead select successive tool
|
||
calls from the information available at each turn. Both approaches can be
|
||
combined with schemas, tests, logs, and persistence.
|
||
|
||
Scripts, including generated scripts, are a substantive alternative rather
|
||
than merely a preliminary form of workflow automation. Code puts the procedure
|
||
close to its implementation and makes ordinary debugging
|
||
tools available. It also leaves decisions about configuration, saved versions,
|
||
and run records to the program or its surrounding infrastructure. A workflow
|
||
platform makes some of those decisions part of its public contract, at the cost
|
||
of introducing another model for the author to learn.
|
||
|
||
The prototype still uses Python for authoring. The distinction is whether that
|
||
code performs the entire procedure directly or constructs a saved workflow for
|
||
the service to execute. Neither representation is automatically better for a
|
||
one-off task.
|
||
|
||
For this comparison, state means working data retained during execution;
|
||
a reducer defines how a new write changes a state field. The prototype's
|
||
outcome is a routing label returned separately from a step's data output.
|
||
Chapter 4 develops these concepts through the report example.
|
||
|
||
The comparison asks the same questions of each system: what a step produces,
|
||
how subsequent work is selected, how data combines, and how an author sees
|
||
those rules. Suspension is considered separately from ordinary branching.
|
||
|
||
## n8n: connecting and inspecting data
|
||
|
||
n8n passes arrays of data items between connected nodes. Items contain JSON
|
||
data and may also contain binary data. Authors map fields from incoming items
|
||
into node parameters; dragging a field into a parameter creates an expression.
|
||
Thus, a connection participates in data flow as well as execution order
|
||
[@n8n-data-2026].
|
||
|
||
For an author combining report records, n8n's Merge node exposes a concrete
|
||
choice: append the incoming collections, match records by fields or position,
|
||
or produce combinations. The node's configuration and worked examples make
|
||
these operations distinguishable. Append waits for connected inputs and emits
|
||
their items in input order [@n8n-merge-2026].
|
||
|
||
[@fig:n8n-merge-append] shows the Append setting and the resulting two
|
||
output records. The fields remain in separate records; selecting Append
|
||
does not combine them into one record.
|
||
|
||
<!-- markdownlint-disable-next-line MD013 -->
|
||
{#fig:n8n-merge-append width=100%}
|
||
|
||
Selecting a merge mode changes which records appear in the output.
|
||
For the report task, joining actions by owner is different from appending two
|
||
action lists.
|
||
|
||
In this thesis, a **data binding** is a declared mapping from a value's source
|
||
location to its destination. The prototype represents data movement through
|
||
these bindings. Its reducers
|
||
define how writes update state fields. Joining records and waiting for branch
|
||
completion are separate operations.
|
||
Branching does not necessarily imply simultaneous execution. For workflows
|
||
created from n8n 1.0, the documented default completes one branch before
|
||
starting another, with ordering affected by canvas position and workflow
|
||
settings [@n8n-order-2026]. The author therefore needs both item-level data
|
||
inspection and an account of branch execution, not only a connected diagram.
|
||
|
||
## Zapier: configuring a decision
|
||
|
||
Zapier exposes the output fields of earlier steps for mapping into later
|
||
steps. Test records supply the values shown during configuration; live runs
|
||
use their own data. This makes the mapping an explicit reference to a previous
|
||
step, rather than an update to an author-declared shared state field
|
||
[@zapier-mapping-2026].
|
||
|
||
An author using Zapier Paths selects fields, conditions, and values, then tests
|
||
the rules against sample data. Applied to the report task, those rules could
|
||
distinguish complete records from records needing attention. Multiple paths
|
||
can qualify, so exclusivity must follow from the rules rather than the
|
||
branching appearance alone [@zapier-paths-2026].
|
||
|
||
[@fig:zapier-path-condition] shows a rule testing whether the selected
|
||
`Hello` field contains `world`. The test record matches, and the interface
|
||
reports that this path would have continued. This preview lets the author
|
||
check a rule against a concrete value before running the workflow.
|
||
|
||
<!-- markdownlint-disable-next-line MD013 -->
|
||
{#fig:zapier-path-condition width=75%}
|
||
|
||
The same documentation describes sequential execution of qualifying paths.
|
||
The rules can permit several branches to execute in one run. An exclusive
|
||
choice requires conditions that cannot qualify together.
|
||
Paths does not provide a shared action after all branches; common steps can be
|
||
duplicated or placed in a Sub-Zap. These constraints affect how the author
|
||
organizes a common report-rendering step [@zapier-paths-2026].
|
||
|
||
The interface therefore needs to communicate both the condition being tested
|
||
and the consequence of a match. In lda.chat, an ordinary outcome selects
|
||
one successor, whereas several Zapier Paths may qualify.
|
||
|
||
```{=latex}
|
||
\begin{minipage}{\linewidth}
|
||
```
|
||
|
||
## LangGraph: describing decisions in Python
|
||
|
||
LangGraph's Graph API lets an author define state, add node functions and
|
||
routing, compile the graph, and invoke it. For the report task, a developer can
|
||
represent extracted fields in state and write a routing function that selects
|
||
what happens next [@langgraph-graph-api-2026].
|
||
|
||
In this small example, `clean` returns an update to the `text` field.
|
||
The separate `choose` function reads that updated state and selects either
|
||
the rendering step or the end of the graph:
|
||
|
||
<!-- langgraph-comparison -->
|
||
```python
|
||
from typing import TypedDict
|
||
from langgraph.graph import START, END, StateGraph
|
||
|
||
class ReportState(TypedDict):
|
||
text: str
|
||
report: str
|
||
|
||
def clean(state: ReportState):
|
||
return {"text": state["text"].strip()}
|
||
|
||
def choose(state: ReportState):
|
||
return "render" if state["text"] else END
|
||
|
||
def render(state: ReportState):
|
||
return {"report": "# Report\n\n" + state["text"]}
|
||
|
||
builder = StateGraph(ReportState)
|
||
builder.add_node("clean", clean)
|
||
builder.add_node("render", render)
|
||
builder.add_edge(START, "clean")
|
||
builder.add_conditional_edges("clean", choose, ["render", END])
|
||
builder.add_edge("render", END)
|
||
graph = builder.compile()
|
||
```
|
||
|
||
```{=latex}
|
||
\end{minipage}
|
||
```
|
||
|
||
Calling `graph.invoke({"text": " Notes ", "report": ""})` cleans the text,
|
||
then produces a report. Whitespace-only input ends after `clean`. The state
|
||
update and the routing decision are expressed in separate functions here;
|
||
LangGraph also offers `Command` to return both from a node
|
||
[@langgraph-graph-api-2026].
|
||
|
||
This authoring style exposes more behavior as code. Nodes produce state
|
||
updates, reducers determine how updates combine, and conditional routes select
|
||
subsequent execution. The documented model supports graph loops and super-step
|
||
execution (parallel execution with a synchronization barrier); its checkpoint
|
||
facilities are described separately
|
||
[@langgraph-graph-api-2026; @langgraph-persistence-2026].
|
||
|
||
The prototype shares the use of typed state and reducers, but represents
|
||
ordinary routing as a mapping from a declared outcome to a successor.
|
||
This keeps executable predicates out of ordinary edges. The cost is that some
|
||
decisions need a dedicated condition step or an additional declared outcome.
|
||
The relevant comparison is where authors express and inspect decisions, not
|
||
whether Python or a canvas is inherently the better interface.
|
||
|
||
## Saved definitions, configuration, and execution records
|
||
|
||
Lifecycle separation is not unique to the prototype. n8n distinguishes saved
|
||
edits from the published version used for production execution, and separates
|
||
workflow history from execution history. Its execution view supports status
|
||
filtering and inspection of previous attempts
|
||
[@n8n-publish-2026; @n8n-executions-2026].
|
||
|
||
Zapier allows draft editing while a published Zap remains active. Publishing
|
||
creates a version, and run details identify the version used and the data
|
||
received and sent by individual steps. Connected application accounts are
|
||
managed separately through app connections
|
||
[@zapier-versions-2026; @zapier-history-2026; @zapier-connections-2026].
|
||
|
||
For LangGraph's library API, the graph is defined and compiled in application
|
||
code. Checkpointers store execution snapshots organized by thread identity;
|
||
the application can retrieve current state and state history. Application
|
||
configuration and dependency provision remain part of the surrounding code.
|
||
This library-level comparison does not cover hosted deployment products
|
||
[@langgraph-graph-api-2026; @langgraph-persistence-2026].
|
||
|
||
The prototype groups available operations into configured **sources**, such as
|
||
a collection of Python functions or an external tool service. It exposes
|
||
saved definitions, source selection, and run inspection
|
||
as artifact, deployment, and run objects in its Python client. Its deployment
|
||
mapping selects a configured provider that must satisfy the saved source
|
||
requirements. An app connection or credential is therefore only a partial
|
||
analogy: the source also supplies operations and their contracts.
|
||
|
||
The system combines these lifecycle responsibilities with its typed graph
|
||
and Python client. Each system must
|
||
distinguish an edit to future work from the recorded
|
||
definition and data of a past execution.
|
||
|
||
## Implications for this design
|
||
|
||
[@tbl:positioning-summary] compares the documented mechanisms with the
|
||
prototype's ordinary execution model. Its rows describe selected mechanisms,
|
||
not every extension available in each product. The sources for n8n, Zapier,
|
||
and LangGraph are discussed in the preceding subsections.
|
||
|
||
| System | Data | Control selection | Combination |
|
||
| --- | --- | --- | --- |
|
||
| n8n | Item arrays | Branch connections | Merge modes |
|
||
| Zapier | Prior-step fields | Matching Paths | Explicit later actions |
|
||
| LangGraph | State updates | Edges and routers | Field reducers |
|
||
| Prototype | Output-to-state mappings | One outcome edge | Field reducers |
|
||
|
||
: Data and control mechanisms compared. {#tbl:positioning-summary}
|
||
|
||
For the report task, these models put different work on the author. n8n
|
||
requires attention to which items reach each node; Zapier requires mappings
|
||
from earlier steps and rules for qualifying paths; LangGraph requires state
|
||
and routing code. The prototype instead requires explicit output-to-state
|
||
mappings and declared routing outcomes. These differences concern where the
|
||
procedure's meaning is expressed, not just whether its editor is visual.
|
||
|
||
Pausing also has a separate contract. n8n's Wait node can resume on a time,
|
||
webhook, or form condition [@n8n-wait-2026]. LangGraph's dynamic interrupt
|
||
uses a checkpoint and thread identity; resuming restarts the interrupted node,
|
||
so code preceding the interrupt executes again [@langgraph-interrupts-2026].
|
||
The prototype uses an explicit interruption boundary and a declared resume
|
||
payload. A failed operation is not automatically such a pause: for example,
|
||
Zapier documents that an errored step produces no output fields for subsequent
|
||
mappings [@zapier-mapping-2026]. These observations do not establish equivalent
|
||
retry or side-effect guarantees across the systems.
|
||
|
||
The design prioritizes Python authoring, explicit mappings, and saved execution
|
||
records. An external agent can use the same client operations. Evaluating these
|
||
interfaces requires observing how authors discover operations, repair
|
||
definitions, and interpret results.
|
||
|
||
Model Context Protocol (MCP) supplies another part of the integration: access
|
||
to tools, resources, and prompts [@mcp-tools-2025; @mcp-lifecycle-2025]. A
|
||
configured MCP source can supply operations and readable content. The workflow
|
||
runtime determines routing and state updates independently of that protocol.
|
||
|
||
# Conceptual Model
|
||
|
||
The report procedure connects operations that read notes, extract report
|
||
fields, and render a document. This chapter explains how the workflow
|
||
represents those operations, where their data is stored, and how execution
|
||
chooses the next step. It then follows the definition through saving,
|
||
configuration, and execution.
|
||
|
||
## Operations, data, and decisions
|
||
|
||
Suppose report preparation asks for missing information before rendering.
|
||
[@fig:report-branch] shows the two possible routes through that decision.
|
||
|
||
<!-- markdownlint-disable-next-line MD013 -->
|
||
```{.mermaid #fig:report-branch width=95% caption="Alternative routes to report rendering."}
|
||
%%{init: {"flowchart": {"rankSpacing": 20, "nodeSpacing": 20}}}%%
|
||
flowchart LR
|
||
Read["Read<br/>notes"] -->|ok| Extract["Extract<br/>report"]
|
||
Extract -->|ok| Check{Complete?}
|
||
Check -->|ready| Render["Render<br/>report"]
|
||
Check -->|needs_information| Ask["Request information<br/>(interrupt)"]
|
||
Ask -->|submitted| Render
|
||
Render -->|ok| Finish([End])
|
||
```
|
||
|
||
Each operation or control step occupies a **node** in the workflow. An operation
|
||
node performs work, such as parsing notes into report fields. On completion,
|
||
it returns **output** data and an **outcome**, a label used to choose what
|
||
executes next. An **edge** connects that node and outcome to a successor.
|
||
The runtime follows the edge whose label matches the returned outcome.
|
||
|
||
In [@fig:report-branch], successful extraction returns `ok`, which leads to
|
||
the completeness check. That check selects `ready` or `needs_information`.
|
||
The first leads directly to rendering, and the second leads to a request for
|
||
input. Each decision selects one route. An exception while reading the notes
|
||
instead fails execution and is recorded as an error.
|
||
|
||
The edge labels explain execution order. The author separately declares which
|
||
data each node receives and which results it stores. Extraction stores report
|
||
fields, the request can supply missing fields on resume, and rendering reads
|
||
the resulting report. The following section describes those data connections.
|
||
|
||
**Schemas** declare the shapes of accepted inputs and produced results.
|
||
They help the author see which fields an operation requires and allow the
|
||
validator to detect incompatible mappings. The factual accuracy of extracted
|
||
information and the availability of a remote service require separate checks.
|
||
|
||
A node output is the data returned by one step. The workflow output is the
|
||
public data selected when the procedure completes. A run records that workflow
|
||
output alongside status, diagnostics, execution identity, and trace information.
|
||
|
||
## Workflow state and data bindings
|
||
|
||
**State** is the workflow's working data. In the report procedure, it contains
|
||
the notes, extracted report, and rendered Markdown. Each run starts with its
|
||
own working data and updates it as its nodes execute.
|
||
|
||
On a node's input and output, **data bindings** map values between named
|
||
locations. An input binding supplies an argument from workflow input, state,
|
||
or execution context, runtime information such as the current iteration item.
|
||
An output binding selects a returned value to write
|
||
into state. For example, extraction reads `state.notes` into its `text`
|
||
argument, then writes its returned report to `state.report`.
|
||
|
||
[@fig:report-state] shows the linear report procedure and its data access.
|
||
Solid arrows labeled `ok` are execution edges. Dashed arrows are data bindings,
|
||
labeled with the fields they read or write. The shared State block makes the
|
||
stored intermediate results visible alongside the operations that use them.
|
||
|
||
<!-- markdownlint-disable-next-line MD013 -->
|
||
```{.mermaid #fig:report-state width=95% caption="Report nodes read and write workflow state through declared bindings."}
|
||
flowchart TB
|
||
Input(["Workflow input: text"])
|
||
subgraph Steps["Operations in execution order"]
|
||
Read["Read notes"] -->|ok| Extract["Extract report"]
|
||
Extract -->|ok| Render["Render Markdown"]
|
||
end
|
||
State["State<br/>notes · report · markdown"]
|
||
Output(["Workflow output: report + markdown"])
|
||
Input -.->|text argument| Read
|
||
Read -.->|write notes| State
|
||
State -.->|read notes as text| Extract
|
||
Extract -.->|write report| State
|
||
State -.->|read report| Render
|
||
Render -.->|write markdown| State
|
||
State -.->|select public result| Output
|
||
```
|
||
|
||
A **reducer** specifies how a write changes a state field. A replace reducer
|
||
stores the new report in place of the previous value. An append reducer can
|
||
instead accumulate reports from several documents in a list. These choices
|
||
determine the effect of a write when a node executes more than once.
|
||
|
||
The workflow's final output bindings select the public result from the
|
||
completed working data. Here they expose the report and Markdown, leaving the
|
||
intermediate notes in state. A node with no output bindings stores none of its
|
||
returned fields in workflow state.
|
||
|
||
## Saving, configuring, and running the procedure
|
||
|
||
While authoring, an **editable workflow** is the mutable definition being
|
||
revised. Saving creates an **artifact**: an immutable workflow version
|
||
containing its graph and declared requirements.
|
||
|
||
A **deployment** selects a saved version and connects its logical service
|
||
requirements to concrete configured services. For example, the same report
|
||
procedure could use a test notes source in one deployment and a production
|
||
notes source in another, provided both satisfy its required contracts.
|
||
Changing the bindings is different from changing how the report is assembled.
|
||
|
||
A **run** records one execution of a deployment. Last week's successful report
|
||
and this week's interrupted attempt are different runs, even if both use the
|
||
same artifact and deployment. Inspecting a run should identify the version,
|
||
inputs, status, and available execution evidence without changing the saved
|
||
procedure.
|
||
|
||
[@fig:lifecycle-records] relates the saved version, its deployments, and their
|
||
runs. The separate records preserve the distinction between changing a
|
||
procedure and examining an execution of it.
|
||
|
||
<!-- markdownlint-disable-next-line MD013 -->
|
||
```{.mermaid #fig:lifecycle-records width=95% caption="One saved version can serve several deployments and runs."}
|
||
classDiagram
|
||
direction LR
|
||
class EditableWorkflow {
|
||
mutable graph
|
||
}
|
||
class ArtifactVersion {
|
||
artifact_id
|
||
version
|
||
saved definition
|
||
}
|
||
class Deployment {
|
||
deployment_id
|
||
source bindings
|
||
}
|
||
class Run {
|
||
run_id
|
||
input and result
|
||
status and trace
|
||
}
|
||
EditableWorkflow ..> ArtifactVersion : saves
|
||
ArtifactVersion "1" <-- "0..*" Deployment : selects
|
||
Deployment "1" <-- "0..*" Run : started from
|
||
```
|
||
|
||
## Available operations and deployment binding
|
||
|
||
A **capability** describes functionality available from a source. The report
|
||
example uses operation capabilities, such as extraction and rendering. A
|
||
source catalog can also describe a resource, such as a readable text document.
|
||
Reading that resource is an operation that produces data for the workflow.
|
||
|
||
A **source** groups capabilities under a configured identity. The extraction
|
||
operation might come from trusted Python code, while another operation or
|
||
document is supplied by an external service.
|
||
|
||
A **deployment binding** connects a
|
||
logical source requirement in the workflow
|
||
to a concrete source in the environment. Validation checks whether that source
|
||
exists and matches the saved requirements. **Source drift** means those
|
||
requirements no longer match the currently available capabilities, for example
|
||
after an input schema changes.
|
||
|
||
Built-in sources have fixed platform identities and do not require those
|
||
deployment bindings. Configured sources remain explicit operator choices.
|
||
Portability is limited to environments with compatible code, credentials,
|
||
and services.
|
||
|
||
## Inspecting failure and resuming an interruption
|
||
|
||
A validation diagnostic concerns the definition or its dependencies before
|
||
execution. A failed run records an operational problem encountered during
|
||
execution. An interrupted run records an explicit request for input, together
|
||
with the state needed to continue. The interface should distinguish these
|
||
situations because they call for different actions.
|
||
|
||
In the report example, `needs_information` routes to a request step. The run
|
||
then saves its position and waits for the requested data. Resuming validates
|
||
the supplied response, applies the request step's declared data bindings,
|
||
and follows its continuation to rendering.
|
||
|
||
A **trace** records execution evidence associated with the run. It identifies
|
||
executed nodes and their outcomes, helping the operator locate a failed step
|
||
or follow a decision through the graph. The implementation chapters explain
|
||
how the runtime records this information.
|
||
|
||
## Working glossary
|
||
|
||
[@tbl:working-glossary] summarizes the core terms through the report example.
|
||
|
||
| Term | Meaning in the report example |
|
||
| --- | ------ |
|
||
| Capability | An available operation or resource, such as a text document |
|
||
| Node | One use of an operation or control step in the procedure |
|
||
| Output | Data produced by a step |
|
||
| Outcome | A declared label selecting the next step |
|
||
| State | Working data retained during execution |
|
||
| Artifact | An immutable saved version of the procedure |
|
||
| Deployment | A saved version connected to concrete services |
|
||
| Run | One execution with its own status and evidence |
|
||
| Source | A configured collection of available operations |
|
||
| Data binding | A mapping into node input, workflow state, or public output |
|
||
| Deployment binding | A mapping from a required source to a configured source |
|
||
|
||
: Working glossary for the thesis terminology. {#tbl:working-glossary}
|
||
|
||
The architecture chapter follows these concepts into the service and runtime.
|
||
|
||
# System Architecture
|
||
|
||
The weekly-report example needs more than a graph executor. An author must
|
||
discover the available operations, connect them, check the resulting workflow,
|
||
and choose which saved version to run. An operator must then distinguish a
|
||
bad definition from an unavailable service or an interrupted execution. The
|
||
architecture separates these responsibilities without requiring each caller
|
||
to implement the workflow lifecycle.
|
||
|
||
Three boundaries organize the system: authoring versus server operations,
|
||
saved definitions versus individual executions, and workflow execution versus
|
||
provider-specific calls. These boundaries are visible in the user-facing
|
||
objects as well as in the implementation.
|
||
|
||
## From authoring to server operations
|
||
|
||
The Python client is the main programmatic authoring interface. Its `App`
|
||
object represents a connection to the workflow service. An author can inspect
|
||
a capability, use the returned object in an editable workflow, validate that
|
||
workflow, and save it. The client reconstructs server responses as Python
|
||
objects with methods for further editing and inspection.
|
||
|
||
Local editing does not require a server request for each graph change.
|
||
Discovery and persistence do: the service owns the available capability
|
||
inventory and stored records. Validation therefore has both a local part,
|
||
which checks the authored structure, and a server part, which checks it
|
||
against the service's contracts.
|
||
|
||
[@fig:architecture-spine] shows this separation. The Python client and CLI
|
||
provide independent entry points to the same service.
|
||
|
||
<!-- markdownlint-disable-next-line MD013 -->
|
||
```{.mermaid #fig:architecture-spine caption="Clients share lifecycle services and provider-independent execution."}
|
||
flowchart TB
|
||
subgraph Authoring["Authoring and clients"]
|
||
Python["Python App and editable workflow"]
|
||
Client["Client port"]
|
||
CLI["Command-line interface"]
|
||
Python --> Client
|
||
end
|
||
subgraph Service["Server-composed services"]
|
||
API["Workflow API"]
|
||
Stores["Artifacts, deployments and run records"]
|
||
Inventory["Available operation contracts"]
|
||
end
|
||
subgraph Transport["Transport boundary"]
|
||
RPC["HTTP JSON-RPC adapter"]
|
||
end
|
||
subgraph Execution["Workflow execution"]
|
||
Core["Workflow execution core"]
|
||
end
|
||
subgraph Integrations["Provider integrations"]
|
||
Providers["Configured source providers"]
|
||
Handlers["Execution handlers"]
|
||
end
|
||
CLI --> RPC
|
||
Client --> RPC
|
||
RPC --> API
|
||
API --> Stores
|
||
API --> Core
|
||
Inventory --> API
|
||
Providers --> Inventory
|
||
Core --> Handlers
|
||
Providers --> Handlers
|
||
```
|
||
|
||
The JSON-RPC adapter translates network requests into API calls and serializes
|
||
their responses. The API coordinates validation, storage, and execution.
|
||
The runtime handles graph behavior, including iteration returns and state
|
||
updates, for all clients of that API.
|
||
|
||
An agent can author or operate a workflow through these interfaces. During a
|
||
run, the runtime follows the saved graph. Agent participation within that
|
||
execution requires an authored operation that invokes the agent.
|
||
|
||
## Keeping a definition separate from its use
|
||
|
||
The client owns the editable graph. The service owns saved artifact,
|
||
deployment, and run records. The API coordinates validation and persistence
|
||
through their respective stores. Execution reads a selected definition and
|
||
its resolved environment rather than the client's mutable editor contents.
|
||
|
||
For the report workflow, this boundary prevents an unfinished edit to the
|
||
extraction step from becoming the definition of an already-created run.
|
||
Inspection retrieves the definition identity and execution evidence recorded
|
||
for that particular run.
|
||
|
||
The client exposes this progression through workflow artifact, deployment,
|
||
and run objects. A run object contains a snapshot of the stored execution.
|
||
Refreshing it requests a new snapshot. This makes network
|
||
activity explicit, although applications must decide when to refresh and how
|
||
to present progress.
|
||
|
||
A local/static server can also start deployment runs on a schedule. The
|
||
optional scheduler supports one-time and recurring starts and records each
|
||
scheduled occurrence for inspection. The Python client's `App.create_schedule()`
|
||
registers a schedule for an existing deployment; `App.schedule()` retrieves
|
||
its configuration, and `App.schedule_occurrences()` retrieves its history.
|
||
The server must have scheduling enabled to start runs without a connected
|
||
client. The Limitations chapter discusses scheduling's storage and
|
||
concurrency constraints.
|
||
|
||
The draft API and CLI also support persisted editing workspaces, which retain
|
||
unfinished definitions between requests. Python authoring can save directly
|
||
from its editable object. Both routes produce a saved definition for deployment
|
||
and execution.
|
||
|
||
## Executing the graph
|
||
|
||
The execution core is the runtime that runs the saved graph. It holds the
|
||
workflow input, working state, and the active node for each in-flight item
|
||
or call. To execute
|
||
an ordinary operation node, it resolves the node's input bindings, calls the
|
||
selected operation, checks its result, applies output bindings, and follows
|
||
the edge selected by the outcome. Repeating that cycle advances the run until
|
||
it completes, fails, or pauses for input.
|
||
|
||
Some nodes control execution directly. A condition chooses a route, a foreach
|
||
executes a body for each item, and an interrupt requests additional input.
|
||
A subgraph node calls another saved workflow and uses its result when that
|
||
call completes. The called workflow is referred to as the **child workflow**.
|
||
Its input and state belong to that invocation, just as a function call has
|
||
its own arguments and local working data.
|
||
|
||
Within a workflow, input bindings supply a step's arguments. Output bindings
|
||
select returned values to write into workflow state. Reducers determine how
|
||
those writes combine with existing values. The returned outcome selects the
|
||
next edge. Rendering therefore requires both a route that reaches the renderer
|
||
and an input binding that supplies the report it consumes.
|
||
|
||
[@fig:node-execution-cycle] follows one operation node through this cycle.
|
||
The later iteration examples show how the runtime manages additional
|
||
execution positions while a controller or child call is active.
|
||
|
||
<!-- markdownlint-disable-next-line MD013 -->
|
||
```{.mermaid #fig:node-execution-cycle caption="A step updates state before following its selected route."}
|
||
flowchart TB
|
||
Validation["Input validation"] --> Call["Invoke operation"]
|
||
Call --> Result["Checked result"]
|
||
subgraph Data["DATA: what becomes visible"]
|
||
Output["Output payload"] --> Bind["Output bindings"]
|
||
Bind --> Reduce["Reducers update state"]
|
||
end
|
||
subgraph Control["CONTROL: where execution goes"]
|
||
Outcome["Declared outcome"] --> Route["Select matching edge"]
|
||
Route --> Next["Next node"]
|
||
end
|
||
Result --> Output
|
||
Result --> Outcome
|
||
Reduce -.->|state available to next step| Next
|
||
```
|
||
|
||
A declared outcome such as `needs_information` is a workflow decision. A
|
||
handler exception or exhausted node-execution allowance is an execution
|
||
failure. A workflow can complete with an outcome such as `needs_information`
|
||
without failing the run. Where iteration supports collecting item
|
||
errors, that policy must be explicit; errors do not automatically become
|
||
ordinary outcome edges.
|
||
|
||
The runtime performs the illustrated operations in sequence: it applies the
|
||
result's writes before advancing along the selected route.
|
||
|
||
The trace makes the sequence inspectable, but a fixed graph does not imply
|
||
identical external results. Language-model calls, remote services, and
|
||
concurrent completion order can vary between runs. The runtime's defined
|
||
routing and state-update rules should not be confused with reproducibility
|
||
of every operation it invokes.
|
||
|
||
## Iteration and child workflow boundaries
|
||
|
||
Suppose the report now covers two documents, `A.md` and `B.md`. The
|
||
workflow must render a report for each, then assemble the two reports.
|
||
For this first example, foreach is configured to process one document at
|
||
a time. An output binding appends each rendered report to a `reports`
|
||
state field, which the assembly step reads.
|
||
|
||
[@fig:foreach-graph] shows the authored graph. The `loop` edge enters the
|
||
item body. Its return edge (a back-edge), `render.ok -> each`, ends that
|
||
item's work.
|
||
The `done` edge leads to the continuation after the collection is processed.
|
||
|
||
<!-- markdownlint-disable-next-line MD013 -->
|
||
```{.mermaid #fig:foreach-graph width=90% caption="The item body returns to foreach; done continues to assembly."}
|
||
%%{init: {"flowchart": {"curve": "stepAfter"}}}%%
|
||
flowchart LR
|
||
Prepare["Prepare documents"] -->|ok| Each["each: Foreach"]
|
||
Each -->|loop| Render["render: Render item"]
|
||
Render -->|ok: item return| Each
|
||
Each -->|done| Assemble["Assemble reports"]
|
||
Assemble -->|ok| Finish(["End workflow"])
|
||
```
|
||
|
||
In serial mode, A's return advances the foreach to B. B's return completes
|
||
the collection and permits the `done` transition to assembly. Returning to
|
||
the active foreach controller therefore finishes one item. Entering that
|
||
controller from `Prepare documents` starts an invocation over the collection.
|
||
|
||
[@fig:foreach-region] follows one invocation from start to finish. Read
|
||
downward for time. Solid arrows request work or apply a data binding;
|
||
dashed arrows report completion. Here line style distinguishes requests from
|
||
completion; in [@fig:report-state], it distinguishes execution from data
|
||
bindings. The labels identify the data and routes.
|
||
|
||
<!-- markdownlint-disable-next-line MD013 -->
|
||
```{.mermaid #fig:foreach-region width=95% caption="Serial iteration finishes both documents before assembly."}
|
||
sequenceDiagram
|
||
participant Each as Foreach
|
||
participant Body as Render item
|
||
participant State as reports state
|
||
participant Assemble as Assemble
|
||
Note over Each,State: Start once: documents = [A.md, B.md], reports = []
|
||
Each->>Body: loop: process A.md
|
||
Body->>State: Output binding: append reportA
|
||
Body-->>Each: ok: A is finished
|
||
Each->>Body: loop: process B.md
|
||
Body->>State: Output binding: append reportB
|
||
Body-->>Each: ok: B is finished
|
||
Note over Each: Both items finished
|
||
Each->>Assemble: done: continue after the loop
|
||
Assemble->>State: Input binding: read reports
|
||
State-->>Assemble: [reportA, reportB]
|
||
```
|
||
|
||
The same authored render node executes twice. The runtime distinguishes
|
||
those executions so that finishing A advances to B, while finishing B permits
|
||
assembly. Each item returns to the foreach invocation that started it.
|
||
|
||
The return edge in [@fig:foreach-graph] records the item's return. Its output
|
||
binding appends the report before that return. A node in the body belongs to
|
||
that loop's control region. The validator rejects using the same node both
|
||
inside and outside the body, or returning past the immediate enclosing loop.
|
||
Ordinary cycles within one region remain valid, subject to the run-wide
|
||
node-execution limit described in the implementation chapter.
|
||
|
||
### Calling a child workflow for one document
|
||
|
||
Now replace the rendering work with a call to a saved child workflow that
|
||
summarizes one document. Consider only the item for `A.md`. The parent may
|
||
know the whole collection, but the child receives only the input explicitly
|
||
mapped into its call: `{"document": "A.md"}`.
|
||
|
||
The child has its own input, working state, and execution context. Access to
|
||
the parent's current item or `reports` field requires an input binding that
|
||
supplies the needed value.
|
||
|
||
[@fig:scope-boundaries] shows the call and its return. The item waits while
|
||
the child runs. The child's `END` returns its result to the calling node.
|
||
The parent item then continues to its own return edge.
|
||
|
||
<!-- markdownlint-disable-next-line MD013 -->
|
||
```{.mermaid #fig:scope-boundaries width=95% caption="Child completion precedes the parent item's return to foreach."}
|
||
sequenceDiagram
|
||
participant Item as Parent item A
|
||
participant Child as Child workflow
|
||
participant Result as A's pending report writes
|
||
participant Each as Foreach
|
||
Item->>Child: Input binding: document = A.md
|
||
activate Child
|
||
Note over Item: Wait for child
|
||
Note over Child: Own input,<br/>state and context
|
||
Child->>Child: Produce summaryA
|
||
Child-->>Item: END: return summaryA
|
||
deactivate Child
|
||
Item->>Result: Output binding: append summaryA to reports
|
||
Item-->>Each: ok: item A is finished
|
||
Note over Each: Continue according<br/>to foreach mode
|
||
```
|
||
|
||
Input and output bindings cross the child workflow boundary explicitly.
|
||
The two completion points are separate:
|
||
child `END` returns to the calling node, and the calling node's route back
|
||
to foreach finishes the item.
|
||
|
||
### When items run concurrently
|
||
|
||
In concurrent mode, A and B may be in progress together. A returning from
|
||
its child does not permit assembly while B is still running. The foreach
|
||
waits for its required item completions before following `done`.
|
||
|
||
Concurrent items retain separate pending writes until their results combine.
|
||
In [@fig:scope-boundaries], “A's pending report writes” represents these pending
|
||
writes. In serial mode, the output binding updates enclosing state so the next
|
||
item can read it.
|
||
|
||
For the successful two-item concurrent case, the foreach combines A's and
|
||
B's writes using the declared reducers, then assembly reads the combined
|
||
state. An append reducer and a replace reducer have different effects;
|
||
neither the arrows nor the fact that both items completed chooses a merge
|
||
policy. The serial ordering in [@fig:foreach-region] is not a promise
|
||
about concurrent completion order.
|
||
|
||
If a child or item reaches an explicit interrupt, the run preserves which
|
||
document was active and where it was waiting. Resume applies the response
|
||
to that saved position and continues the same item.
|
||
|
||
## Connecting external operations
|
||
|
||
Providers adapt external functionality to contracts the workflow system can
|
||
inspect and invoke. Discovery supplies schemas and declared outcomes;
|
||
execution supplies a handler for the selected binding. These are separate
|
||
responsibilities because discovering an operation does not mean its service
|
||
is currently reachable or authorized.
|
||
|
||
The server composes providers, stores, and the API. MCP-backed operations
|
||
may need sessions, authentication, and remote catalog handling. Python-backed
|
||
operations use configured, trusted imports. These concerns remain outside
|
||
the graph scheduler. An experimental OpenAPI provider explores another
|
||
source family without making it the system's product identity or implying
|
||
that all API descriptions are interchangeable.
|
||
|
||
This boundary reduces provider-specific logic in workflows, but cannot
|
||
erase provider differences. A schema describes a call's shape; it does not
|
||
guarantee availability, cost, side-effect safety, or semantic equivalence to
|
||
another operation with the same fields.
|
||
|
||
# Implementation
|
||
|
||
Running the report procedure involves two kinds of work. The author first
|
||
uses the Python client to construct, validate, and save a graph. The service
|
||
then executes that saved graph with supplied input and records the result.
|
||
This chapter follows those operations into the implementation, including
|
||
the additional bookkeeping needed when steps repeat, call child workflows,
|
||
or pause for input.
|
||
|
||
## Following a Python authoring request
|
||
|
||
The client package, `wf_client`, exposes the `App` connection object introduced
|
||
in the architecture chapter. Its `App.from_http_jsonrpc(...)` constructor
|
||
records the service address. A later capability lookup contacts that service,
|
||
checks the returned identity and contract, and constructs a Python capability
|
||
object for the author to inspect or add to a workflow.
|
||
|
||
The editable workflow object reuses the graph-building methods of
|
||
`WorkflowBuilder`, the authoring layer's builder. It adds validation through
|
||
the service and saving. Validation first checks the graph locally. If a
|
||
binding references an undeclared state field, the client returns that error
|
||
immediately and records that server validation was not run. Otherwise, it
|
||
submits the graph for the server to check against its available operations.
|
||
|
||
Saving validates first, submits the plan, checks the save response, and
|
||
re-inspects the exact artifact version. The resulting artifact object is
|
||
therefore reconstructed from the stored definition. Identity checks reject
|
||
responses identifying a different artifact or deployment.
|
||
|
||
The Python client's `Run.refresh()` method retrieves a stored run by its
|
||
identifier through the service's `inspect_run` operation. It verifies the
|
||
run and deployment identities and returns a new run snapshot. `Run.resume()`
|
||
submits the requested response for an interrupted run and also returns a new
|
||
snapshot. The caller retains the returned object to see the updated status
|
||
and result.
|
||
|
||
Requests and responses are serialized at the service boundary. Reconstructing
|
||
the returned objects in the client gives authors the same inspection methods
|
||
after discovery, execution, and later refreshes.
|
||
|
||
## Executing nodes and limiting a run
|
||
|
||
The `wf_core` package implements the execution cycle shown in
|
||
[@fig:node-execution-cycle]. It selects a ready execution position and handles
|
||
the selected node according to its kind. An operation node invokes the
|
||
callable chosen through the deployment. A control node instead performs its
|
||
built-in behavior, such as choosing a condition route or starting iteration.
|
||
|
||
A cycle can revisit nodes indefinitely if its condition never selects an
|
||
exit. To bound this work, the run has a **node-execution limit**, with a default
|
||
of 10,000. Immediately before dispatching a node, the runtime checks that
|
||
capacity remains and increments the run's counter. This check and increment
|
||
is called admission. The counter therefore counts admitted executions,
|
||
including control nodes, repeated visits, and executions that subsequently
|
||
fail. It does not count only successful operations.
|
||
|
||
The policy is represented by `RunLimits.max_steps`, and the stored counter
|
||
is `steps_executed`. Iteration and child workflows share this run-wide limit.
|
||
Asynchronous execution reserves capacity before launching the selected work.
|
||
When no capacity remains, the runtime fails the run before ever dispatching
|
||
another node. The limit and counter persist with saved execution state, so
|
||
resume continues with the remaining allowance.
|
||
|
||
This bounds the number of admitted node executions. The duration of a single
|
||
operation and the cost of its external requests require separate controls.
|
||
A handler that blocks indefinitely can still prevent progress after admission.
|
||
|
||
## Preserving state across nested execution
|
||
|
||
Consider the two-document iteration in [@fig:foreach-graph]. In serial mode,
|
||
document A appends its report to state before document B starts. B can read
|
||
that updated state. In concurrent mode, A and B each work with a separate
|
||
view of state. An item can read its own updates while its sibling's pending
|
||
writes remain isolated. The foreach combines their updates after the required
|
||
items finish, using the state fields' reducers.
|
||
|
||
Nesting introduces an additional case. Suppose each document is processed
|
||
concurrently, but its sections are processed serially. Section 2 can read the
|
||
work of section 1 in the same document. Those updates still belong to that
|
||
document's separate view until the outer foreach combines the documents.
|
||
Publishing every serial section's write directly into shared workflow state
|
||
would expose unfinished document results to the other concurrent items.
|
||
|
||
The runtime tracks the active node, the workflow invocation that owns its
|
||
data, and the updates held separately by concurrent items. A **frame** records one
|
||
execution position, including its current node and owning foreach when it is
|
||
an iteration item. A **scope** contains the input, working state, and context
|
||
of one workflow invocation. A child workflow has its own scope. A **lineage**
|
||
records state updates associated with an execution, allowing concurrent items
|
||
to retain their separate views.
|
||
|
||
An output binding produces a **patch**, a set of state writes together with
|
||
the information needed to apply their reducers. The runtime follows the
|
||
item's enclosing iterations to determine where that patch belongs. Serial
|
||
iterations pass their writes outward. At the nearest concurrent item boundary,
|
||
the writes are buffered in that item's lineage. With no such boundary, they
|
||
update the workflow scope's state. The runtime checks the complete ownership
|
||
chain before applying a patch, so malformed parent records cannot cause a
|
||
partial update.
|
||
|
||
Merging nested results also requires distinguishing an accumulated value from
|
||
new writes. Suppose state already contains report A, and an inner group adds
|
||
B and C. The resulting view is `[A, B, C]`. If an outer merge appends this
|
||
whole view to the existing `[A]`, it produces `[A, A, B, C]`, which is undesirable.
|
||
|
||
The patch therefore retains the new writes, or **contributions**, separately
|
||
from the accumulated value visible to readers. In this example the new
|
||
contributions are B and C. A later merge applies those writes using the
|
||
declared reducer, preserving A once. This bookkeeping supports nested
|
||
concurrent iteration even when its intermediate results are merged again.
|
||
|
||
## Giving expressions a consistent context
|
||
|
||
Suppose an outer foreach named `documents` processes documents, and an inner
|
||
foreach named `actions` processes each document's action items. A node in the
|
||
inner body may need both the document and the current action. A single
|
||
unnamed current-item value would leave it unclear which one the node receives.
|
||
|
||
**Execution context** supplies runtime information to expressions and input
|
||
bindings. Its foreach entries are keyed by the authored controller's identity.
|
||
Inside the inner body, two paths distinguish the enclosing document from
|
||
the current action:
|
||
|
||
```text
|
||
context.foreach.documents.item # enclosing document
|
||
context.foreach.actions.item # current action
|
||
```
|
||
|
||
The runtime constructs these entries from the active frames. Repeated
|
||
executions of the same node therefore resolve the paths to their own items.
|
||
|
||
A condition checking an item's fields and an input binding reading those
|
||
fields receive the same context view. Validation checks these references
|
||
against the corresponding schema. This avoids accepting a path that a
|
||
condition would treat as absent while another reader can resolve it.
|
||
|
||
Context construction stops at the current workflow scope. If a child workflow
|
||
needs the parent's document, its caller supplies that value through child
|
||
input, as shown in [@fig:scope-boundaries]. On resume, persisted frame ownership
|
||
identifies the active items. Missing owners, parent cycles, or conflicting
|
||
aliases are reported as errors before those records are used to construct
|
||
context.
|
||
|
||
## Validation, persistence, and diagnostics
|
||
|
||
Validation first checks declarations that can be evaluated without running
|
||
operations. For example, two output bindings on the same node might write
|
||
`foo` to `state."1"` and `bar` to `state."1"."2"`. The first targets an entire
|
||
field and the second targets a location inside it. Applying both could replace
|
||
the container that the nested write needs. The validator rejects overlapping
|
||
destinations within that binding list, avoiding a dependency on their order.
|
||
Here `"1"` and `"2"` are literal field names. The rule also applies to ordinary
|
||
names such as `state.report` and `state.report.title`.
|
||
|
||
This check concerns one set of mappings. Successive nodes can intentionally
|
||
update related state fields, and concurrent iteration combines writes through
|
||
its separate reducer rules. Validation also checks graph structure and rejects
|
||
illegal iteration returns. Server validation checks the available operations
|
||
and service connections. At execution time, the runtime checks actual values
|
||
and execution records as they become available.
|
||
|
||
Diagnostics carry a code, a location, a message, and, where available, a
|
||
repair hint. For an overlapping destination, the location identifies the
|
||
node's output-binding list. These fields let a caller locate the faulty
|
||
mapping and decide how to repair it. The revised definition then passes
|
||
through validation again.
|
||
|
||
The API lifecycle layer stores stopped runs and their **checkpoints**, saved
|
||
representations of execution state. An interrupted checkpoint contains the
|
||
position and working data needed to resume. Restoration validates those
|
||
records before execution continues from the explicit interruption boundary.
|
||
The limits of recovery for external operations are discussed in Limitations.
|
||
|
||
Separating artifact, deployment, and run storage also keeps inspection
|
||
focused. Definition inspection explains what was saved; deployment
|
||
inspection explains environment selection; run inspection and trace explain
|
||
what happened during a particular run.
|
||
|
||
## Server and provider responsibilities
|
||
|
||
The remaining package boundaries put these operations into a service.
|
||
`wf_api` coordinates lifecycle operations, `wf_artifacts` supplies storage
|
||
contracts and implementations, and `wf_platform` supplies shared platform
|
||
contracts. `wf_server` composes these dependencies. `wf_transport_rpc_http` exposes
|
||
the JSON-RPC interface, while `wf_cli` provides terminal operations.
|
||
|
||
Provider implementations retain their own lifecycle requirements. MCP
|
||
support manages remote discovery and invocation through configured
|
||
connections. Python support loads trusted configured callables; it is not
|
||
a sandbox for arbitrary submitted code. OpenAPI support remains
|
||
experimental and is not evidence that every described HTTP service can
|
||
already be used without adaptation.
|
||
|
||
The case study follows these components through one public authoring session:
|
||
discovery, revision, validation, persistence, execution, and inspection.
|
||
|
||
# Case Study: Deterministic Report Workflow
|
||
|
||
Consider an author who receives weekly notes and needs two deliverables: a
|
||
structured report for further processing and Markdown for a reader. The
|
||
example workflow makes this small procedure reusable:
|
||
|
||
The three operations read notes, extract a report, and render Markdown.
|
||
|
||
The input is deliberately constrained. Notes contain named sections and
|
||
action lines with owner, task, and due-date fields. Extraction parses that
|
||
format using deterministic Python code. Fixed input and local operations
|
||
allow the result to be checked independently of remote credentials, service
|
||
quotas, or variation in generated text.
|
||
|
||
The bundle at
|
||
[`examples/report_workflow/`](../../examples/report_workflow/) supplies the
|
||
operations, fixture notes, server configuration, and a saved raw-plan example.
|
||
The walkthrough below expresses the same procedure through the current
|
||
Python client. The example README retains a command-line route for operators
|
||
who need that interface.
|
||
|
||
## Starting with available operations
|
||
|
||
The operator supplies the source identities through the example configuration.
|
||
This walkthrough discovers operations within those known sources; source
|
||
administration is outside the session.
|
||
|
||
The example configuration registers three trusted Python operations under
|
||
`local.report` for discovery and `local.report_runtime` for execution.
|
||
Their Pydantic models describe the input and output
|
||
contracts. The author consumes those operations from the service inventory;
|
||
the client does not import their implementations to execute them locally.
|
||
|
||
The Python blocks form one asynchronous session. The retained test runs that
|
||
session against the real service in process. It substitutes an in-process
|
||
connection for the HTTP connection assignment; remote server startup is not
|
||
tested.
|
||
For HTTP use, the blocks assume a
|
||
server using `examples/report_workflow/wf.config.json`, reachable at its
|
||
configured address, and a fresh artifact name or unused version. The fixture
|
||
read assumes the client is running from the repository root.
|
||
|
||
```python
|
||
from pathlib import Path
|
||
|
||
from pydantic import BaseModel
|
||
|
||
from wf_authoring import input_from, input_path, output_to, state_path
|
||
from wf_client import App
|
||
|
||
app = App.from_http_jsonrpc("http://127.0.0.1:8771/rpc")
|
||
available = await app.capabilities(source_id="local.report")
|
||
for operation in available.items:
|
||
print(operation.qualified_name)
|
||
read_notes = await app.capability("local.report.read_notes")
|
||
extract_report = await app.capability("local.report.extract_report")
|
||
render_report = await app.capability("local.report.render_markdown_report")
|
||
print(extract_report.output_schema)
|
||
```
|
||
|
||
Each lookup returns a capability object with its schemas and declared
|
||
outcomes. The extraction schema exposes `title`, `summary`, `action_items`,
|
||
`risks`, and `followups`. The title names the report and the summary records its
|
||
overview. Action items describe work to do: `owner` identifies the responsible
|
||
person, `task` describes the work, and `due` records its due-date text. Risks
|
||
list potential problems, and followups record matters needing later attention.
|
||
The author can use the schema directly or declare corresponding local models.
|
||
This session uses local models without importing provider implementation code.
|
||
|
||
The listing contains `local.report.read_notes`, `local.report.extract_report`,
|
||
and `local.report.render_markdown_report`. Their inspected contracts describe
|
||
how each can be connected. The author selects the operations and constructs
|
||
the graph from those contracts.
|
||
|
||
Before connecting operations, the author checks their input and output
|
||
contracts for compatible fields.
|
||
|
||
## Describing the workflow's data
|
||
|
||
The workflow has one public input, intermediate state, and two public
|
||
outputs. They are declared separately so that intermediate notes do not
|
||
accidentally become part of the result contract.
|
||
|
||
```python
|
||
class ActionItem(BaseModel):
|
||
owner: str
|
||
task: str
|
||
due: str
|
||
|
||
|
||
class ReportOutput(BaseModel):
|
||
title: str
|
||
summary: str
|
||
action_items: list[ActionItem]
|
||
risks: list[str]
|
||
followups: list[str]
|
||
```
|
||
|
||
These author-defined models represent the discovered report fields. The
|
||
workflow then declares its own input, working state, and public output:
|
||
|
||
```python
|
||
class NotesInput(BaseModel):
|
||
text: str
|
||
|
||
|
||
class ReportState(BaseModel):
|
||
notes: str = ""
|
||
report: ReportOutput | None = None
|
||
markdown: str = ""
|
||
|
||
|
||
class ReportResult(BaseModel):
|
||
report: ReportOutput
|
||
markdown: str
|
||
|
||
|
||
graph = app.new_workflow(
|
||
"report_python_showcase",
|
||
input_schema=NotesInput,
|
||
state_schema=ReportState,
|
||
output_schema=ReportResult,
|
||
)
|
||
```
|
||
|
||
The models export schemas that are stored with the workflow contract. This
|
||
lets the service validate its data after the author's Python session ends.
|
||
The report starts as absent in intermediate state. The public result requires
|
||
a report because the successful pipeline produces one before completion.
|
||
|
||
## Connecting data and decisions
|
||
|
||
The author now creates three node uses. Each use selects an operation and
|
||
declares its data bindings. In the first mapping, `input_path("text")` selects
|
||
the workflow's public input field, and the second `"text"` names the reading
|
||
operation's argument. They happen to share a name. The extraction mapping
|
||
instead reads the state field `notes` into an argument named `text`.
|
||
|
||
In an output binding, `"."` selects the complete returned object. Extraction
|
||
stores that whole report in `state.report`, while reading and rendering store
|
||
only selected output fields:
|
||
|
||
```python
|
||
read = graph.use(
|
||
read_notes,
|
||
id="read",
|
||
input=[input_from(input_path("text"), "text")],
|
||
output=[output_to("text", state_path("notes"))],
|
||
)
|
||
extract = graph.use(
|
||
extract_report,
|
||
id="extract",
|
||
input=[input_from(state_path("notes"), "text")],
|
||
output=[output_to(".", state_path("report"))],
|
||
)
|
||
render = graph.use(
|
||
render_report,
|
||
id="render",
|
||
input=[input_from(state_path("report"), "report")],
|
||
output=[output_to("markdown", state_path("markdown"))],
|
||
)
|
||
```
|
||
|
||
The author connects each operation's `ok` outcome to the next node, ending at
|
||
a node that completes the workflow with outcome `ok`. Final output bindings
|
||
then select the report and Markdown from state:
|
||
|
||
```python
|
||
end = graph.end("ok", id="finished")
|
||
graph.set_entry_point(read)
|
||
graph.connect(read, "ok", extract)
|
||
graph.connect(extract, "ok", render)
|
||
graph.connect(render, "ok", end)
|
||
graph.set_output([
|
||
input_from(state_path("report"), "report"),
|
||
input_from(state_path("markdown"), "markdown"),
|
||
])
|
||
```
|
||
|
||
Using `"."` selects all fields of that node's output. An empty output-binding
|
||
list would store none of them in workflow state. The `connect` calls determine
|
||
execution order independently of these data selections.
|
||
|
||
The author declares data mappings and execution routes separately. Changing
|
||
one leaves the other unchanged. This requires additional declarations even
|
||
for a linear three-step procedure; typed helpers construct the serialized
|
||
representation from those declarations.
|
||
|
||
## Diagnosing and repairing a binding
|
||
|
||
An editable graph can temporarily be invalid. Suppose a final output binding
|
||
reads `missing_report`, although the declared state contains `report`:
|
||
|
||
```python
|
||
# Deliberately refer to an undeclared state field.
|
||
from textwrap import fill
|
||
|
||
graph.set_output([
|
||
input_from(state_path("missing_report"), "report"),
|
||
input_from(state_path("markdown"), "markdown"),
|
||
])
|
||
broken = await graph.validate()
|
||
assert not broken.ok
|
||
assert broken.remote_status == "not_run" # Local rejection skips the server check.
|
||
for issue in broken.local.errors:
|
||
print(issue.code, issue.path)
|
||
print(fill(issue.message, width=72))
|
||
```
|
||
|
||
The example prints the diagnostic code, its location, and the message wrapped
|
||
to fit the page:
|
||
|
||
<!-- walkthrough-output -->
|
||
```text
|
||
invalid_source_path output[0].path
|
||
source path must start with input., state., or context. and reference a
|
||
declared root field when applicable
|
||
```
|
||
|
||
The location identifies the first workflow-output mapping. Comparing its
|
||
source with the declared state reveals the misspelled reference. Local
|
||
validation rejects the graph before a server validation request is made.
|
||
The author repairs that reference and validates again:
|
||
|
||
```python
|
||
# Repair the field reference, leaving the nodes and edges unchanged.
|
||
graph.set_output([
|
||
input_from(state_path("report"), "report"),
|
||
input_from(state_path("markdown"), "markdown"),
|
||
])
|
||
(await graph.validate()).raise_for_errors()
|
||
```
|
||
|
||
The repair changes the workflow's output mapping. The renderer and its
|
||
outgoing edge remain unchanged.
|
||
This illustrates why data bindings and control routes need separate feedback.
|
||
The next section saves only the repaired definition.
|
||
|
||
## Saving a version and choosing its environment
|
||
|
||
Before saving, the author can request validation and inspect its diagnostics.
|
||
The example stops on errors:
|
||
|
||
```python
|
||
validation = await graph.validate()
|
||
validation.raise_for_errors()
|
||
artifact = await graph.save(version=1)
|
||
|
||
deployment = await artifact.deploy(
|
||
"report_python_showcase.local",
|
||
bindings={"local.report": "local.report_runtime"},
|
||
)
|
||
readiness = await deployment.validate()
|
||
if not readiness.runnable:
|
||
raise RuntimeError(readiness.diagnostics)
|
||
```
|
||
|
||
The artifact is the saved version of the authored procedure. The deployment
|
||
maps the saved `local.report` requirement to `local.report_runtime`.
|
||
The fixture registers compatible operations under both names so this choice
|
||
changes the execution source without changing the graph or requiring remote
|
||
credentials. Both source identities refer to the example's local operations.
|
||
|
||
[@fig:python-lifecycle] summarizes the public operations; it omits internal
|
||
validation and re-inspection calls made by individual client methods.
|
||
|
||
<!-- markdownlint-disable-next-line MD013 -->
|
||
```{.mermaid #fig:python-lifecycle caption="Local editing leads to saved, configured, and inspected execution."}
|
||
sequenceDiagram
|
||
actor Author
|
||
participant Client as Python client
|
||
participant API as Workflow service
|
||
participant Provider as Bound operation
|
||
Author->>Client: Build and revise graph
|
||
Client->>Client: Check structure locally
|
||
Client->>API: Validate and save definition
|
||
API-->>Client: Saved artifact version
|
||
Client->>API: Bind deployment to version
|
||
API-->>Client: Deployment and readiness
|
||
Client->>API: Run with input
|
||
API->>Provider: Invoke bound operation
|
||
Provider-->>API: Output and outcome
|
||
API-->>Client: Run snapshot
|
||
Client->>API: Inspect run and bounded trace
|
||
API-->>Client: Stored execution evidence
|
||
Client-->>Author: Result or diagnostic
|
||
```
|
||
|
||
An edit to the workflow would be saved as another version, leaving this
|
||
deployment's artifact unchanged. Selecting a different source environment
|
||
changes the deployment's bindings. A readiness check
|
||
can reject a missing or incompatible binding before a run is attempted.
|
||
|
||
## Running and inspecting the result
|
||
|
||
The client reads the notes and sends their contents. The server therefore
|
||
does not need access to the client's file path.
|
||
|
||
```python
|
||
notes = Path("examples/report_workflow/input.md").read_text(encoding="utf-8")
|
||
# Permit at most 100 admitted node executions across this run.
|
||
run = await deployment.run({"text": notes}, max_steps=100)
|
||
if run.status != "completed":
|
||
raise RuntimeError((run.status, run.diagnostics))
|
||
|
||
result = ReportResult.model_validate(run.output)
|
||
assert result.report.title == "Weekly Project Update"
|
||
assert len(result.report.action_items) == 3
|
||
assert result.markdown.startswith("# Weekly Project Update")
|
||
|
||
run = await run.refresh()
|
||
trace = await run.trace(start=0, limit=10)
|
||
|
||
print("Status:", run.status)
|
||
print("Title:", result.report.title)
|
||
print("Action items:", len(result.report.action_items))
|
||
print(result.markdown.splitlines()[0])
|
||
```
|
||
|
||
The session prints a compact result summary and the first line of the rendered
|
||
Markdown:
|
||
|
||
<!-- walkthrough-output -->
|
||
```text
|
||
Status: completed
|
||
Title: Weekly Project Update
|
||
Action items: 3
|
||
# Weekly Project Update
|
||
```
|
||
|
||
The run exposes a status, output, and diagnostics independently of the
|
||
editable graph. The output crosses the API as data; the explicit
|
||
`model_validate` call reconstructs the example's Pydantic result model.
|
||
Refreshing obtains the latest stored snapshot, while the bounded trace
|
||
provides step-level evidence when output alone is insufficient.
|
||
|
||
For this fixed fixture, the result includes three action items, the
|
||
recorded risks and followups, and a Markdown report headed
|
||
“Weekly Project Update.” Checking these fields establishes that the
|
||
example's data reached the intended outputs. The extraction contract requires
|
||
the sectioned notes format described at the start of this chapter.
|
||
|
||
## Registering a scheduled run
|
||
|
||
The same deployment can be invoked later by the server's scheduler. With
|
||
scheduling enabled on the local/static server, the client can register a
|
||
one-time start and inspect its stored configuration and occurrence history:
|
||
|
||
```python
|
||
from datetime import UTC, datetime, timedelta
|
||
|
||
schedule = await app.create_schedule(
|
||
schedule_id="report-once",
|
||
deployment_id=deployment.deployment_id,
|
||
trigger={
|
||
"kind": "oneshot",
|
||
"at": (datetime.now(UTC) + timedelta(hours=1)).isoformat(),
|
||
},
|
||
input_bindings=[{
|
||
"target": "text",
|
||
"expression": {"kind": "literal", "value": notes},
|
||
}],
|
||
max_steps=100,
|
||
)
|
||
schedule = await app.schedule(schedule.schedule_id)
|
||
history = await app.schedule_occurrences(schedule.schedule_id, limit=10)
|
||
```
|
||
|
||
The input binding stores the current contents of `notes`. The scheduled run
|
||
receives that text even if the original file changes. A recurring workflow
|
||
that needs fresh notes would need an operation that retrieves them when it
|
||
runs. Occurrence history records what happened to scheduled starts, including
|
||
admission and completion. Immediately after registration, it can be empty.
|
||
|
||
The session check verifies registration and inspection without waiting for
|
||
the scheduled time. Scheduler execution is exercised separately in
|
||
[`test_scheduled_deployment_example.py`](../../tests/examples/test_scheduled_deployment_example.py).
|
||
|
||
## What this case demonstrates
|
||
|
||
The executable check in
|
||
[`test_thesis_python_walkthrough.py`](../../tests/examples/test_thesis_python_walkthrough.py)
|
||
reads and runs this chapter's Python blocks, including the rejected binding
|
||
and its repair. It checks the stored report, the returned snapshot, and the
|
||
two displayed output excerpts against the session's captured output.
|
||
The existing tests in
|
||
[`test_report_workflow_example.py`](../../tests/examples/test_report_workflow_example.py)
|
||
check the source's input rules, rendering and extraction, capability
|
||
discovery and invocation, and the artifact/deployment/run lifecycle using
|
||
the raw-plan fixture. Those tests are evidence for the report operations
|
||
and lifecycle, complementing the Python session check.
|
||
|
||
The Python session shows the current authoring steps:
|
||
inspect operations, declare contracts, connect data and outcomes, save,
|
||
select bindings, and inspect an execution. The CLI and draft surface offer
|
||
alternative interfaces to related lifecycle operations.
|
||
|
||
A direct Python script would be shorter for these three local functions.
|
||
The additional structure becomes relevant when the definition
|
||
must be saved, bound to an environment, validated independently, and
|
||
inspected through a shared service. This example demonstrates the integration
|
||
of those operations.
|
||
|
||
The case exercises a linear pipeline. Conditional branches, foreach bodies,
|
||
child workflows, and interrupts are covered by separate runtime tests listed
|
||
in the evaluation's evidence index. Independent authoring and diagnosis tasks
|
||
are needed to evaluate the interaction beyond this scripted session.
|
||
|
||
# Evaluation
|
||
|
||
Evaluation distinguishes three questions: whether the runtime follows its
|
||
contract, whether the public lifecycle composes correctly, and whether an
|
||
author can use that lifecycle effectively. The reported in-process walkthrough
|
||
exercises the public lifecycle. Targeted tests for runtime contracts are
|
||
mapped in the Evidence Index.
|
||
The authoring assessment identifies the operations available through the
|
||
interface. Usability remains a separate evaluation question.
|
||
|
||
## Requirements and evidence
|
||
|
||
[@tbl:requirements-evidence] relates the authoring requirements R1–R5 and
|
||
execution requirements X1–X5 to the available evidence. E1–E5 refer to the
|
||
five groups in the Evidence Index appendix: E1 covers lifecycle operations,
|
||
E2 Python authoring, E3 validation, E4 runtime execution, and E5 source
|
||
providers. These labels locate implementation and test files. The table
|
||
distinguishes the walkthrough's observations from listed tests and the
|
||
usability measurements still needed.
|
||
|
||
| Requirement | Evidence | Assessment |
|
||
| --- | --- | ---- |
|
||
| R1 Discovery | Walkthrough; E2 | Exercised; usability unmeasured |
|
||
| R2 Data movement | Worked graph | Explained; comprehension unmeasured |
|
||
| R3 Revision feedback | Binding repair; E3 | Exercised; usability unmeasured |
|
||
| R4 Editing vs running | Lifecycle; E1–E2 | One saved version exercised |
|
||
| R5 Status interpretation | Inspection; E1–E2 | Exposed; usability unmeasured |
|
||
| X1 Definition/run identity | Lifecycle; E1 | Lifecycle tests listed |
|
||
| X2 Known constraints | Validation; E3 | Constraint tests listed |
|
||
| X3 Routing and state | Runtime; E4 | Runtime tests listed |
|
||
| X4 Environment choices | Sources; E5 | Provider tests listed |
|
||
| X5 Bounds/inspection | Node limits/resume; E1, E4 | Boundary tests listed |
|
||
|
||
: Requirements and available evidence. {#tbl:requirements-evidence}
|
||
|
||
The R4 observation covers saving and running version 1. The session does not
|
||
edit a later version or check its effect on earlier runs. Understanding the
|
||
version distinction remains part of the authoring evaluation.
|
||
|
||
X5 covers the run-wide node-execution limit, execution inspection, and
|
||
interruption resume. Node-execution limit tests check that the counter
|
||
persists and that exhaustion prevents further dispatch. Lifecycle tests
|
||
exercise stored inspection and resume. The report session exercises inspection
|
||
with a limit of 100 node executions. It completes within that limit and
|
||
contains no interruption. The evidence index locates the separate tests for
|
||
exhaustion and resume.
|
||
|
||
## Walkthrough method and observations
|
||
|
||
The retained walkthrough test uses the example server configuration and a
|
||
fresh pytest temporary store. It extracts the case study's Python blocks in
|
||
document order and substitutes an in-process client connection for the HTTP
|
||
connection assignment. All later calls use the real API, provider, and stores.
|
||
This setup exercises discovery, graph construction, validation, saving,
|
||
deployment, execution,
|
||
refresh, and trace inspection, while excluding network startup and transport
|
||
behavior from the observation.
|
||
|
||
[@tbl:prototype-conformance] records the concrete checks performed by that
|
||
session. The fixture expectations are fixed independently of the workflow's
|
||
result, and stored output is inspected again through the API.
|
||
|
||
| Check | Expected observation | Requirement |
|
||
| ---- | ----- | --- |
|
||
| Invalid output mapping | Local rejection at `output[0].path` | R3, X2 |
|
||
| Repaired graph | Validation permits saving and running | R3, X2 |
|
||
| Persisted report | Expected title and three action items | X1, X3 |
|
||
| Persisted rendering | Expected Markdown heading | X3 |
|
||
| Run and trace | Completed snapshot; recorded steps | R5, X5 inspection |
|
||
|
||
: Reproducible report-session checks. {#tbl:prototype-conformance}
|
||
|
||
The checked session completed with the expected title, three action items,
|
||
and Markdown heading in the saved output. The invalid binding was rejected
|
||
locally; restoring the mapping allowed validation and execution to continue.
|
||
These observations establish the displayed procedure's behavior for this
|
||
fixture, not an author's ability to construct it unaided.
|
||
|
||
The session check and the existing report-example tests can be run with:
|
||
|
||
```powershell
|
||
$suites = @(
|
||
"tests/examples/test_thesis_python_walkthrough.py",
|
||
"tests/examples/test_report_workflow_example.py"
|
||
)
|
||
uv run pytest @suites -q -n 0
|
||
```
|
||
|
||
The first suite executes the displayed case-study session with the stated
|
||
transport substitution. The second checks the operation fixtures and raw-plan
|
||
lifecycle independently. Document rendering and generated PDF assets are not
|
||
part of this command. The test and manuscript must be taken from the same
|
||
repository revision because the test reads the manuscript directly.
|
||
|
||
## Separating design comparison from evaluation
|
||
|
||
The earlier comparison of n8n, Zapier, and LangGraph explains different
|
||
authoring and execution choices. The same task has not been measured across
|
||
those systems under matched conditions. This report therefore cannot rank
|
||
their usability, reliability, or performance against the prototype.
|
||
|
||
Execution tests ask whether contracts, routing, state updates, and persistence
|
||
behave as specified. Interaction evaluation asks whether an author can
|
||
discover operations, express a procedure, understand errors, and recover
|
||
without inspecting implementation code. Passing one kind of test does not
|
||
answer the other.
|
||
|
||
The report walkthrough exposes costs such as explicit bindings and deployment
|
||
selection, but does not measure whether those costs are acceptable to new
|
||
users. Similarly, structured diagnostics and inspection objects may help an
|
||
agent avoid trial and error, but reduced retries, token use, and repair time
|
||
remain hypotheses rather than measured outcomes.
|
||
|
||
## Falsifiability criteria
|
||
|
||
The implementation would fail its stated contracts if, for example:
|
||
|
||
- a saved run could not identify the definition and bindings it used;
|
||
- an invalid foreach boundary were accepted and executed as another region;
|
||
- nested state writes were lost or counted twice;
|
||
- resume reset the node-execution counter or resumed the wrong item;
|
||
- condition evaluation used a different context model from validation;
|
||
- ordinary source invocation required provider-specific graph routing.
|
||
|
||
These cases support focused regression tests. Broader architectural claims,
|
||
such as accommodating future source families without changing the runtime,
|
||
remain design expectations to assess as those integrations are built.
|
||
|
||
# Limitations
|
||
|
||
The prototype demonstrates a workflow lifecycle under controlled conditions.
|
||
Its main limitations concern how much authors must understand, which
|
||
execution guarantees are provided, and how far the available evidence can
|
||
be generalized.
|
||
|
||
## Authoring and diagnosis still require technical knowledge
|
||
|
||
The Python client reduces manual serialization and provides editable graphs
|
||
and inspectable objects. It does not remove the need to understand schemas,
|
||
state bindings, outcomes, and deployment selection. The report example makes
|
||
this cost visible: a short procedure requires more declarations than direct
|
||
function calls.
|
||
|
||
Diagnostics identify many invalid structures and bindings, but an accurate
|
||
message is not necessarily an understandable repair instruction. Authors
|
||
still need to distinguish a graph error from an environment problem or a
|
||
failed external operation. The current evidence does not establish that new
|
||
users can make these distinctions without assistance.
|
||
|
||
Inspection also requires judgment. A trace shows recorded execution, not
|
||
whether a report is factually correct or a remote side effect was desirable.
|
||
|
||
## Execution guarantees have defined boundaries
|
||
|
||
Foreach iteration, nested workflow scopes, structured context, and run-wide
|
||
node-execution limits are implemented foundations. They do not yet provide general
|
||
fork/gather control for arbitrary branches. In particular, concurrent
|
||
iteration should not be presented as a solution to correlating branches
|
||
that split, loop, and later meet at different gather points.
|
||
|
||
Persisted interrupted runs can resume at explicit boundaries. This is not
|
||
arbitrary mid-handler crash recovery, replay of every external call, or an
|
||
exactly-once side-effect guarantee. A run's step limit bounds admitted graph
|
||
steps; it does not bound a handler's execution time or the cost of its
|
||
external requests.
|
||
|
||
Schema validation checks declared structure, not business truth. A returned
|
||
report may satisfy its schema while containing incorrect information.
|
||
Similarly, a fixed graph specifies routing but does not make remote
|
||
responses or concurrent completion order reproducible.
|
||
|
||
## Deployment and trust assumptions
|
||
|
||
The controlled examples assume trusted operators and trusted Python sources.
|
||
Python operations execute in the server process without a sandbox. The
|
||
thesis does not establish multi-tenant isolation, role-based authorization,
|
||
or production-grade credential management. An explicit workflow interrupt
|
||
can request a response, but is not by itself an authenticated approval or
|
||
access-control mechanism.
|
||
|
||
Source bindings make environment choices inspectable rather than making
|
||
workflows universally portable. A destination environment still needs
|
||
compatible operations, credentials, and dependencies. Python sources are
|
||
loaded at startup; changing their code requires a server restart. The shared
|
||
provider interface does not yet unify every provider's administration,
|
||
authentication, and live health behavior.
|
||
|
||
Filesystem-backed stores support the demonstrated persistence paths. Their
|
||
use does not establish production performance, cross-process contention
|
||
behavior at scale, or disaster recovery. A future database implementation
|
||
would still need to preserve the lifecycle's transaction and ownership
|
||
contracts; changing the storage engine alone would not prove those
|
||
properties.
|
||
|
||
Scheduled execution uses the same deployment and run records as direct
|
||
invocation. Its ownership model assumes one scheduler over the participating
|
||
file stores and is supported only by the local/static server configuration.
|
||
This bounds the deployment conditions under which the scheduling mechanism
|
||
can be used; it is not a distributed execution service.
|
||
|
||
The provider interface exposes callable operations. It does not reproduce a
|
||
provider's interactive widgets or its complete user interface.
|
||
|
||
## Limits of the evidence
|
||
|
||
The deterministic report fixture demonstrates lifecycle integration, not
|
||
broad document understanding or graph expressiveness. Targeted tests cover
|
||
additional execution mechanisms. The evidence index identifies those tests;
|
||
this report records execution results for the report-session suites.
|
||
|
||
The in-process setup also limits the walkthrough to service composition; it
|
||
does not test the displayed HTTP connection. Without independent authoring
|
||
tasks or matched cross-system measurements, its results cannot establish
|
||
ease of use, repair efficiency, or comparative performance. Those questions
|
||
require the interaction evaluation described in Future Work.
|
||
|
||
# Future Work
|
||
|
||
The remaining questions concern richer execution semantics, the effectiveness
|
||
of authoring and diagnosis, and operation beyond the controlled environment.
|
||
|
||
## Establish general fork and gather semantics
|
||
|
||
General fork/gather requires a rule for identifying which concurrent work
|
||
belongs to the same invocation. Existing frames, scopes, iteration activations,
|
||
and state lineages provide foundations, but their relationships must remain
|
||
coherent through nested execution and resume.
|
||
|
||
A fork creates concurrent execution branches; a gather must determine
|
||
which arriving branches belong together before combining their state.
|
||
Loops, partial gathers, and repeated visits make this more than waiting
|
||
for a fixed number of arrivals. The design must also preserve contribution
|
||
identity so that a write already included in one merge is not applied
|
||
again in a later merge.
|
||
|
||
This work remains planned, with reference-model verification preceding
|
||
production implementation. Pressure cases should become executable tests
|
||
for correlation, merge behavior, and recovery. The authoring contract also
|
||
needs to make clear which graphs are rejected before execution and which
|
||
decisions remain the author's responsibility.
|
||
|
||
## Evaluate the authoring and recovery experience
|
||
|
||
A focused usability study should ask participants to discover an operation,
|
||
build a small workflow, change its contract, diagnose a broken binding, and
|
||
inspect a failed or interrupted run. Useful measures include task completion,
|
||
time to a correct repair, unnecessary retries, and reliance on source-code
|
||
inspection.
|
||
|
||
Execution and interaction should be evaluated together without conflating
|
||
them. For example, a validation rule may correctly reject a graph while its
|
||
diagnostic fails to explain the ownership boundary that was crossed.
|
||
Conversely, a convenient editing operation must not hide a change to the
|
||
workflow's execution meaning.
|
||
|
||
The Python client and CLI should receive evidence appropriate to their own
|
||
interaction styles. For agent trials, fix the interface, allowed operations,
|
||
task fixtures, and success criteria before collecting results. Check an
|
||
agent's report against saved artifacts, deployment identity, run output, and
|
||
recorded interactions. Human evaluation can test whether the lifecycle
|
||
vocabulary and data-binding model are understandable without implementation
|
||
knowledge.
|
||
|
||
## Durable waiting within a run
|
||
|
||
Scheduling starts a new run of a saved deployment. Waiting until a time or
|
||
event during an existing run would instead require a durable suspension point
|
||
and rules for resuming it. Future work should distinguish these two forms of
|
||
timed execution rather than treat a wait operation as another schedule.
|
||
Distributed execution would additionally require an ownership model beyond
|
||
the current single-scheduler arrangement.
|
||
|
||
## An assistant-backed authoring application
|
||
|
||
A companion local-first assistant application has an execution backend for
|
||
persistent Python shells, streamed code output and images, execution approvals,
|
||
and saved conversations. Its chat interface is under development. The shells
|
||
allow an agent to retain Python objects across interactions, which provides
|
||
an environment in which it could use the workflow client. Code executes on
|
||
the local machine without a sandbox. Saved history survives backend restart,
|
||
but previous shells are not restored and unfinished work is not replayed.
|
||
|
||
Integrating this application with workflow authoring and administration remains
|
||
to be evaluated. An author needs to inspect and correct a proposed procedure
|
||
before running it, then follow its status and respond to interruptions.
|
||
Dedicated views of the client's artifact, deployment, and run objects could
|
||
support that interaction. The present case study evaluates the workflow
|
||
client directly and includes no agent-driven authoring session.
|
||
|
||
## Preserving contracts across operational changes
|
||
|
||
The source catalog already represents resources, and a built-in workflow
|
||
operation can read resource text with a size bound. Further authoring work
|
||
could make these resources easier to select and inspect alongside operations,
|
||
including clear treatment of content types and truncated results.
|
||
|
||
Additional providers and storage backends would test whether the architectural
|
||
boundaries hold beyond the demonstrated implementations. The question is
|
||
whether an integration can preserve source compatibility, run identity, and
|
||
recovery behavior without changing the graph's execution rules. Such work
|
||
needs failure tests and deployment evidence as well as a working adapter.
|
||
|
||
Richer debugging should clarify what can safely be resumed or repeated,
|
||
especially around external side effects. Showing more trace information is
|
||
different from promising that an earlier action can be undone.
|
||
|
||
# Conclusion
|
||
|
||
This report examined how a useful procedure can become a reusable workflow
|
||
that an author can define and an operator can inspect. The implemented
|
||
system separates the saved definition, its deployment bindings, and each
|
||
execution into artifacts, deployments, and runs. Those distinctions give
|
||
workflow use a record beyond the lifetime of an editing session or a single
|
||
script invocation.
|
||
|
||
The graph model separates data movement from control movement. Contracts
|
||
and bindings describe what a step receives and writes. Outcomes select
|
||
transitions, while runtime constructs govern iteration, child scopes,
|
||
and interruption. The Python client exposes this model through authoring
|
||
and inspection objects, while the API and provider boundaries connect it
|
||
to configured operations.
|
||
|
||
The report case demonstrates that these representations compose into a
|
||
save–deploy–run–inspect lifecycle for the fixed fixture. The repository also
|
||
contains targeted tests covering execution requirements X1–X5; the evidence
|
||
index locates them without reporting a combined execution result.
|
||
For the authoring objectives R1–R5, the
|
||
work identifies and exercises supporting interfaces, but their effectiveness
|
||
for independent human or agent authors remains an open evaluation question.
|
||
|
||
Explicit contracts expose data mappings, saved versions, and run records,
|
||
while requiring authors to learn more concepts than a sequence of function
|
||
calls. The implemented contribution is the integration of these contracts
|
||
into a programmable lifecycle, demonstrated by saving, configuring, executing,
|
||
and inspecting the report procedure through the public client.
|
||
|
||
<!-- References -->
|
||
# References {#sec:refs .unnumbered}
|
||
|
||
::: {#refs}
|
||
:::
|
||
|
||
\appendix
|
||
<!-- Appendices -->
|
||
# Evidence Index
|
||
|
||
This appendix maps the evaluation's evidence identifiers to implementation
|
||
and tests. Paths identify inspectable evidence; they are not a claim that
|
||
all listed suites passed in one newly recorded full-system run.
|
||
|
||
## Core workflow lifecycle
|
||
|
||
**E1**: artifacts, deployments, stopped runs, and explicit resume boundaries.
|
||
|
||
- `src/wf_artifacts/models.py`
|
||
- `src/wf_artifacts/runs/`
|
||
- `src/wf_api/run_lifecycle.py`
|
||
- `tests/wf_api/test_artifact_api.py`
|
||
- `tests/wf_api/test_run_api.py`
|
||
|
||
## Python authoring and inspection
|
||
|
||
**E2**: reconstructed client objects, editable workflows, and the report fixture.
|
||
|
||
- `src/wf_client/`
|
||
- `tests/wf_client/test_authoring.py`
|
||
- `tests/wf_client/test_deployments.py`
|
||
- `tests/wf_client/test_runs.py`
|
||
- `examples/report_workflow/`
|
||
- `tests/examples/test_report_workflow_example.py`
|
||
- `tests/examples/test_thesis_python_walkthrough.py`
|
||
|
||
The example's README retains the command-line route for operators who need
|
||
it. That alternative interface is not an additional evaluated case in the
|
||
current thesis.
|
||
|
||
## Validation and diagnostics
|
||
|
||
**E3**: structural validation, source compatibility, and repair information.
|
||
|
||
- `src/wf_core/validation/`
|
||
- `src/wf_artifacts/validation.py`
|
||
- `tests/artifacts/test_validation.py`
|
||
- `tests/core/test_structured_context_validation.py`
|
||
- `tests/core/test_foreach_control_regions.py`
|
||
|
||
## Execution ownership and node limits
|
||
|
||
**E4**: nesting, structured context, concurrent iteration, and persisted
|
||
node-execution limits.
|
||
|
||
- `src/wf_core/runtime/`
|
||
- `tests/core/test_foreach_back_edges.py`
|
||
- `tests/core/test_concurrent_foreach_interrupts.py`
|
||
- `tests/core/test_structured_runtime_context.py`
|
||
- `tests/core/test_run_step_budget.py`
|
||
- `tests/core/test_run_step_budget_codec.py`
|
||
- `tests/core/test_run_step_budget_async.py`
|
||
|
||
## Source provider boundary
|
||
|
||
**E5**: source contracts and provider-specific execution behind server composition.
|
||
|
||
- `src/wf_platform/sources.py`
|
||
- `src/wf_server/config.py`
|
||
- `src/wf_sources_python/`
|
||
- `src/wf_sources_mcp/`
|
||
- `src/wf_api/source_helpers.py`
|
||
- `tests/wf_sources_python/test_loader.py`
|
||
- `tests/wf_sources_mcp/test_runtime.py`
|
||
- `tests/wf_transport_rpc_http/test_mcp_backed_server_rpc.py`
|
||
|
||
The existing providers demonstrate this separation for their implemented
|
||
operations. They do not establish equal lifecycle features across providers
|
||
or prove compatibility with every future source family.
|