diff --git a/docs/thesis/references.bib b/docs/thesis/references.bib index f481aa1c..96c0c03f 100644 --- a/docs/thesis/references.bib +++ b/docs/thesis/references.bib @@ -10,7 +10,7 @@ title = {Save and publish workflows}, author = {{n8n}}, year = {2026}, - url = {https://docs.n8n.io/build/understand-workflows/save-and-publish-workflows.md}, + url = {https://docs.n8n.io/build/understand-workflows/save-and-publish-workflows}, urldate = {2026-09-11} } @@ -18,7 +18,7 @@ title = {View executions for a single workflow}, author = {{n8n}}, year = {2026}, - url = {https://docs.n8n.io/build/understand-workflows/understand-executions/view-executions-for-a-single-workflow.md}, + url = {https://docs.n8n.io/build/understand-workflows/understand-executions/view-executions-for-a-single-workflow}, urldate = {2026-09-11} } @@ -50,7 +50,7 @@ title = {Understand n8n's data structure}, author = {{n8n}}, year = {2026}, - url = {https://docs.n8n.io/build/work-with-data/understand-n8ns-data-structure.md}, + url = {https://docs.n8n.io/build/work-with-data/understand-n8ns-data-structure}, urldate = {2026-09-11} } @@ -58,7 +58,7 @@ title = {Understand execution order}, author = {{n8n}}, year = {2026}, - url = {https://docs.n8n.io/build/flow-logic/understand-execution-order.md}, + url = {https://docs.n8n.io/build/flow-logic/understand-execution-order}, urldate = {2026-09-11} } diff --git a/docs/thesis/system-design-implementation.md b/docs/thesis/system-design-implementation.md index 6f142f9e..848641c7 100644 --- a/docs/thesis/system-design-implementation.md +++ b/docs/thesis/system-design-implementation.md @@ -15,6 +15,7 @@ bibliography: references.bib link-citations: true figureTitle: "Figure" figPrefix: "Figure" +tblPrefix: "Table" chapters: true appendix: true syntax-highlighting: idiomatic @@ -65,8 +66,9 @@ diagram: Preparing reports, transforming documents, and collecting workspace information often involve procedures that must be repeated with new inputs. An AI assistant -can help perform such work, but a successful conversation does not itself -preserve an executable procedure. This thesis presents `lda.chat`, a +can help perform such work. Repeating it requires preserving the operations +and their data connections as an executable procedure. This thesis presents +`lda.chat`, a programmable workflow platform for defining, checking, running, and inspecting reusable workspace procedures. @@ -84,11 +86,12 @@ for data contracts, control flow, state updates, and interruption. The thesis compares these concerns with other workflow systems and explains the prototype's choice to separate a node's data output from its routing outcome. -The implementation provides Python authoring objects, a workflow service, and -adapters for trusted Python functions and external tools. Evidence includes +The implementation includes a workflow runtime, Python authoring objects, a +workflow service, and adapters for trusted Python functions and external tools. +Evidence includes a deterministic case study and focused tests of validation, -execution, and persistence. Authoring usability remains to be evaluated; -the prototype's interaction design is still under development. +execution, and persistence. Authoring usability remains to be evaluated as +the interaction design develops. # Introduction @@ -98,42 +101,44 @@ use. A person can do this manually, write a script, or ask an AI assistant for help. When the procedure becomes recurring, the workspace also needs a way to preserve it, supply different inputs, and inspect unsuccessful attempts. -`lda.chat` is a programmable workflow platform for that recurring work. -It allows a human or external agent to assemble available operations into a -saved procedure and execute it through a service. The procedure exists -independently of the conversation or programming session that created it. -Its executions can be inspected separately, and explicit requests for additional -input can be resumed from saved state. +`lda.chat` turns such a procedure into a reusable workflow: a saved description +of operations, their data connections, and their execution order. An author +can revise this description, connect it to available services, and run it +with new inputs. Each execution has a separate record of its progress and +results. The procedure remains available after the conversation or programming +session that created it has ended. For the report task, the initial procedure is short: Read the notes, extract a structured report, and render it as Markdown. -The operator supplies new notes each week rather than rebuilding those steps. +The operator supplies new notes each week using the saved steps. If the requirements change, the author revises the procedure and saves another version. If execution fails, the operator needs to identify the affected step -and its inputs, not merely observe that no report appeared. +and its inputs. ## From a useful procedure to a usable system The intended beneficiary is a workspace operator who needs repeatable work. Its author may be that person, a developer, or an external agent acting on their -behalf. Authoring a procedure and operating a saved one are different activities; -neither requires an LLM to participate in every execution. +behalf. Once saved, the procedure can run with or without an LLM operation, +according to the steps its author included. A workflow system must help its author answer practical questions: which operations are available, what information they require, and how one operation's result becomes another's input. During operation, it must distinguish an invalid definition, a missing service, a failed execution, and a request for more input. +The **runtime** is the part of the system that executes the workflow: it invokes +steps, maintains working data, and determines where execution continues. These are interaction-design concerns as well as runtime concerns. An interface that draws a branch without explaining whether one path or both will execute leaves a consequential rule implicit. -The current prototype emphasizes programmable authoring and inspection. -Its Python interface presents workflow and run objects instead of requiring -authors to construct network messages. The interaction considered here is -therefore code-based: discovering operations, revising a graph, and inspecting -its executions through Python objects. +The current authoring interface is Python-based. It presents workflow and run +objects for discovering operations, revising a graph, and inspecting execution. +An external agent can use that interface through a Python execution environment. +The interface and runtime are under development, with the implemented behavior +and its supporting evidence examined in this thesis. ## Engineering question and contribution @@ -147,9 +152,9 @@ without retaining the conversation or programming session that produced it. The prototype implements three design choices as one lifecycle: -1. A graph with declared input, output, and state schemas represents operations, - data movement, and routing decisions so they can be validated and inspected. -2. Separate saved versions, environment bindings, and execution records make +1. A graph with declared data structures represents operations, working data, + and routing decisions so they can be validated and inspected. +2. Separate saved versions, service configuration, and execution records make editing, configuration, and operation distinct activities. 3. Programmable authoring, validation diagnostics, and run inspection expose those distinctions to human and agent clients. @@ -162,11 +167,14 @@ authoring and execution mechanisms. ## Scope of the implementation -The prototype supports conditional routes, iteration, child workflows, and -explicit interruption/resume. Its runs have an execution-attempt limit. -General parallel fork/gather remains proposed; persisted interruption does not -mean recovery halfway through arbitrary external code. The evaluation separates -tested behavior from unmeasured usability and production-readiness claims. +The implemented runtime supports conditional routes, iteration, calls to saved +workflows, and explicit interruption/resume. A run-wide limit bounds the number +of node executions it admits. General parallel fork/gather remains proposed. +The evaluation covers controlled execution and lifecycle tests, with usability +and production operation identified as further evaluation work. A companion +assistant backend can execute code in persistent Python shells. Its workflow +authoring integration and developing chat interface are discussed in Future +Work, outside the evaluated workflow case study. ## Report Outline @@ -239,13 +247,13 @@ Those interactions require corresponding runtime contracts (X1–X5): 1. **X1 — Preserve definitions and executions independently.** Save an identifiable procedure version and keep separate records for its invocations. 2. **X2 — Validate known constraints before work starts.** Check graph structure, - declared data contracts, mappings, and required service bindings. + declared data contracts, mappings, and required service connections. 3. **X3 — Specify routing and state changes.** Define what selects the successor, where outputs are written, and how repeated writes affect the workflow's working data, referred to here as workflow state. 4. **X4 — Separate environment choices from procedure logic.** Resolve logical service requirements to concrete configured services and report mismatches. -5. **X5 — Bound and inspect execution.** Limit execution attempts, retain +5. **X5 — Bound and inspect execution.** Limit admitted node executions, retain status and trace information, and support resume at defined interruption boundaries. @@ -254,11 +262,12 @@ environment selection, and individual executions. ## Costs and boundaries -Schemas, bindings, and versioned definitions impose authoring work. A script +Declaring data structures, mappings, and saved versions imposes authoring work. +A script can be preferable for a short-lived task, especially when its author already understands the libraries involved. The platform targets cases where reuse and -inspection justify that setup; the thesis does not establish a numerical -break-even point. +inspection justify that setup. Determining when those benefits outweigh the +setup cost requires further evaluation. A workflow interface must also avoid promising more than its runtime supports. Successful validation does not guarantee a remote service will succeed, and a @@ -266,15 +275,14 @@ saved interruption does not make arbitrary external effects reversible. # Positioning And Related Systems -The report task provides a common lens for comparison: choose operations, pass -notes between them, add a route for incomplete information, test the procedure, -and inspect its result. The comparison follows these authoring activities into -the execution rules they expose. +The report task provides a common example for comparing authoring and execution. +The comparison asks how an author connects operations and data, how the runtime +handles decisions and repeated work, and how the author tests and inspects a +run. These questions apply to graphical editors and Python interfaces alike. The accounts below use documented mechanisms consulted in September 2026. -They are not hands-on usability measurements or a complete product survey. -Differences in authoring style do not establish that one system is easier for -every audience. +They describe selected interaction and execution mechanisms. Comparative +usability would require participants performing matched tasks. ## Starting with code or direct tool calls @@ -323,7 +331,9 @@ Selecting a merge mode changes which records appear in the output. For the report task, joining actions by owner is different from appending two action lists. -The prototype represents data movement through explicit mappings. Its reducers +In this thesis, a **data binding** is a declared mapping from a value's source +location to its destination. The prototype represents data movement through +these bindings. Its reducers define how writes update state fields. Joining records and waiting for branch completion are separate operations. Branching does not necessarily imply simultaneous execution. For workflows @@ -347,12 +357,14 @@ can qualify, so exclusivity must follow from the rules rather than the branching appearance alone [@zapier-paths-2026]. The same documentation describes sequential execution of qualifying paths. +The rules can permit several branches to execute in one run. An exclusive +choice requires conditions that cannot qualify together. Paths does not provide a shared action after all branches; common steps can be duplicated or placed in a Sub-Zap. These constraints affect how the author organizes a common report-rendering step [@zapier-paths-2026]. The interface therefore needs to communicate both the condition being tested -and the consequence of a match. In the prototype, an ordinary outcome selects +and the consequence of a match. In lda.chat, an ordinary outcome selects one successor, whereas several Zapier Paths may qualify. ## LangGraph: describing decisions in Python @@ -365,7 +377,8 @@ what happens next [@langgraph-graph-api-2026]. This authoring style exposes more behavior as code. Nodes produce state updates, reducers determine how updates combine, and conditional routes select subsequent execution. The documented model supports graph loops and super-step -execution; its checkpoint facilities are described separately +execution (parallel execution with a synchronization barrier); its checkpoint +facilities are described separately [@langgraph-graph-api-2026; @langgraph-persistence-2026]. The prototype shares the use of typed state and reducers, but represents @@ -396,7 +409,9 @@ configuration and dependency provision remain part of the surrounding code. This library-level comparison does not cover hosted deployment products [@langgraph-graph-api-2026; @langgraph-persistence-2026]. -The prototype exposes saved definitions, source selection, and run inspection +The prototype groups available operations into configured **sources**, such as +a collection of Python functions or an external tool service. It exposes +saved definitions, source selection, and run inspection as artifact, deployment, and run objects in its Python client. Its deployment mapping selects a configured provider that must satisfy the saved source requirements. An app connection or credential is therefore only a partial @@ -440,83 +455,113 @@ Zapier documents that an errored step produces no output fields for subsequent mappings [@zapier-mapping-2026]. These observations do not establish equivalent retry or side-effect guarantees across the systems. -The prototype prioritizes programmable authoring, explicit mappings, and saved -execution records. Its current interface must still be evaluated for the work -required to discover operations, repair definitions, and interpret results. -Neither its typed models nor the absence of executable edge predicates proves -that it achieves those user-experience goals. +The design prioritizes Python authoring, explicit mappings, and saved execution +records. An external agent can use the same client operations. Evaluating these +interfaces requires observing how authors discover operations, repair +definitions, and interpret results. -External-tool protocols are a separate concern. Model Context Protocol (MCP) -exposes tools, resources, and prompts; it is not a competing graph model -[@mcp-tools-2025; @mcp-lifecycle-2025]. In this system, a source is a configured -provider of operations. MCP is one way to obtain those operations; it does not -determine the workflow's routing or data model. +Model Context Protocol (MCP) supplies another part of the integration: access +to tools, resources, and prompts [@mcp-tools-2025; @mcp-lifecycle-2025]. A +configured MCP source can supply operations and readable content. The workflow +runtime determines routing and state updates independently of that protocol. # Conceptual Model -The report example introduces the concepts in the order an author encounters -them: choose operations, connect their data, define decisions, save a version, -and inspect an execution. The branching example in [@fig:report-branch] -explains supported -primitives; it is not an additional measured case study. +The report procedure connects operations that read notes, extract report +fields, and render a document. This chapter explains how the workflow +represents those operations, where their data is stored, and how execution +chooses the next step. It then follows the definition through saving, +configuration, and execution. ## Operations, data, and decisions -Suppose report preparation must ask for missing information before rendering: +Suppose report preparation asks for missing information before rendering. +[@fig:report-branch] shows the two possible routes through that decision. ```{.mermaid #fig:report-branch width=95% caption="Alternative routes to report rendering."} %%{init: {"flowchart": {"rankSpacing": 20, "nodeSpacing": 20}}}%% flowchart LR - Read["Read
notes"] --> Extract["Extract
report"] - Extract --> Check{Complete?} + Read["Read
notes"] -->|ok| Extract["Extract
report"] + Extract -->|ok| Check{Complete?} Check -->|ready| Render["Render
report"] Check -->|needs_information| Ask["Request
information"] Ask -->|submitted| Render - Render --> Finish([End]) + Render -->|ok| Finish([End]) ``` -Each named operation is a **node** in the workflow. An **edge** selects the next -step after a node produces an **outcome**, such as `ready` or -`needs_information`. The node's **output** is the data it returns, such as the -extracted report fields. A missing-information outcome is a business decision; -an exception while reading a file is a runtime failure. +Each operation or control step occupies a **node** in the workflow. An operation +node performs work, such as parsing notes into report fields. On completion, +it returns **output** data and an **outcome**, a label used to choose what +executes next. An **edge** connects that node and outcome to a successor. +The runtime follows the edge whose label matches the returned outcome. -The arrow to the next step does not implicitly pass all previous output into -that step. The author defines input mappings and writes relevant outputs into -workflow state. In this example, extraction writes report fields, the request -can supply missing fields on resume, and rendering reads the resulting report. -Only one of the two routes to rendering executes on each decision. +In [@fig:report-branch], successful extraction returns `ok`, which leads to +the completeness check. That check selects `ready` or `needs_information`. +The first leads directly to rendering, and the second leads to a request for +input. Each decision selects one route. An exception while reading the notes +instead fails execution and is recorded as an error. + +The edge labels explain execution order. The author separately declares which +data each node receives and which results it stores. Extraction stores report +fields, the request can supply missing fields on resume, and rendering reads +the resulting report. The following section describes those data connections. **Schemas** declare the shapes of accepted inputs and produced results. They help the author see which fields an operation requires and allow the -validator to detect incompatible mappings. They do not establish that an -extracted fact is true or that a remote operation will succeed. +validator to detect incompatible mappings. The factual accuracy of extracted +information and the availability of a remote service require separate checks. A node output is the data returned by one step. The workflow output is the public data selected when the procedure completes. A run records that workflow output alongside status, diagnostics, execution identity, and trace information. -## Workflow state and iteration +## Workflow state and data bindings -**State** is the workflow's working data. A **reducer** specifies how a write -changes a state field: replace the previous value, append an action item, or add -a number, for example. Reading a field and routing to another step are separate -operations. This makes data movement inspectable but requires the author to -understand the mappings. +**State** is the workflow's working data. In the report procedure, it contains +the notes, extracted report, and rendered Markdown. Each run starts with its +own working data and updates it as its nodes execute. -If the procedure processes several documents, a foreach step defines an item -body. The foreach distinguishes completion of one item from completion of the -collection. A child workflow provides a -separate invocation scope with explicit inputs and results. These boundaries -define which data is available and what completion means. +On a node's input and output, **data bindings** map values between named +locations. An input binding supplies an argument from workflow input, state, +or execution context, runtime information such as the current iteration item. +An output binding selects a returned value to write +into state. For example, extraction reads `state.notes` into its `text` +argument, then writes its returned report to `state.report`. -Nodes outside an iteration body, or within the same body, may form ordinary -cycles. A body and its enclosing workflow are separate control regions; -an ordinary edge cannot cross that boundary. A run-wide step budget limits -execution attempts. General fork/gather is not implemented. Connecting paths -does not by itself promise parallel execution, synchronization, or conflict-free -state merging. +[@fig:report-state] shows the linear report procedure and its data access. +Solid arrows labeled `ok` are execution edges. Dashed arrows are data bindings, +labeled with the fields they read or write. The shared State block makes the +stored intermediate results visible alongside the operations that use them. + + +```{.mermaid #fig:report-state width=95% caption="Report nodes read and write workflow state through declared bindings."} +flowchart TB + Input(["Workflow input: text"]) + subgraph Steps["Operations in execution order"] + Read["Read notes"] -->|ok| Extract["Extract report"] + Extract -->|ok| Render["Render Markdown"] + end + State["State
notes · report · markdown"] + Output(["Workflow output: report + markdown"]) + Input -.->|text argument| Read + Read -.->|write notes| State + State -.->|read notes as text| Extract + Extract -.->|write report| State + State -.->|read report| Render + Render -.->|write markdown| State + State -.->|select public result| Output +``` + +A **reducer** specifies how a write changes a state field. A replace reducer +stores the new report in place of the previous value. An append reducer can +instead accumulate reports from several documents in a list. These choices +determine the effect of a write when a node executes more than once. + +The workflow's final output bindings select the public result from the +completed working data. Here they expose the report and Markdown, leaving the +intermediate notes in state. A node with no output bindings stores none of its +returned fields in workflow state. ## Saving, configuring, and running the procedure @@ -568,13 +613,17 @@ classDiagram ## Available operations and environment binding -A **capability** is an operation available for use in a workflow. -A **source** groups capabilities under a configured identity. The report's -extraction operation might come from trusted Python code, while another -operation is supplied by an external tool service. +A **capability** describes functionality available from a source. The report +example uses operation capabilities, such as extraction and rendering. A +source catalog can also describe a resource, such as a readable text document. +Reading that resource is an operation that produces data for the workflow. -A **binding** is an explicit mapping. Input and output bindings map data; -a deployment binding connects a logical source requirement in the workflow +A **source** groups capabilities under a configured identity. The extraction +operation might come from trusted Python code, while another operation or +document is supplied by an external service. + +An **environment binding**, also called a **deployment binding**, connects a +logical source requirement in the workflow to a concrete source in the environment. Validation checks whether that source exists and matches the saved requirements. **Source drift** means those requirements no longer match the currently available capabilities, for example @@ -594,22 +643,22 @@ with the state needed to continue. The interface should distinguish these situations because they call for different actions. In the report example, `needs_information` routes to a request step. The run -then waits for a declared resume payload; resuming applies that payload through -the workflow's bindings and continues to rendering. This is a defined pause in -the procedure, not recovery halfway through an arbitrary handler. +then saves its position and waits for the requested data. Resuming validates +the supplied response, applies the request step's declared data bindings, +and follows its continuation to rendering. -A **trace** records execution evidence associated with the run. It supports -questions such as which route was taken and where execution stopped, but does -not make external effects reversible. The implementation chapters explain how -these concepts become runtime records and service operations. +A **trace** records execution evidence associated with the run. It identifies +executed nodes and their outcomes, helping the operator locate a failed step +or follow a decision through the graph. The implementation chapters explain +how the runtime records this information. ## Working Glossary [@tbl:working-glossary] summarizes the core terms through the report example. | Term | Meaning in the report example | -| --- | --- | -| Capability | An available operation, such as extracting report fields | +| --- | ------ | +| Capability | An available operation or resource, such as a text document | | Node | One use of an operation or control step in the procedure | | Output | Data produced by a step | | Outcome | A declared label selecting the next step | @@ -618,14 +667,12 @@ these concepts become runtime records and service operations. | Deployment | A saved version connected to concrete services | | Run | One execution with its own status and evidence | | Source | A configured collection of available operations | -| Binding | An explicit data mapping or source-to-environment mapping | +| Data binding | A mapping into node input, workflow state, or public output | +| Deployment binding | A mapping from a required source to a configured source | : Working glossary for the thesis terminology. {#tbl:working-glossary} -Python types, provider protocols, and package boundaries implement these -concepts. Their names are introduced with their responsibilities in the -architecture and implementation chapters rather than used as prerequisites -for understanding the workflow. +The architecture chapter follows these concepts into the service and runtime. # System Architecture @@ -647,8 +694,7 @@ The Python client is the main programmatic authoring interface. Its `App` object represents a connection to the workflow service. An author can inspect a capability, use the returned object in an editable workflow, validate that workflow, and save it. The client reconstructs server responses as Python -objects with relevant operations, rather than requiring application code to -carry response dictionaries through every step. +objects with methods for further editing and inspection. Local editing does not require a server request for each graph change. Discovery and persistence do: the service owns the available capability @@ -656,8 +702,8 @@ inventory and stored records. Validation therefore has both a local part, which checks the authored structure, and a server part, which checks it against the service's contracts. -[@fig:architecture-spine] shows this separation. The CLI is another entry -point to the service; it is not a mandatory layer beneath Python authoring. +[@fig:architecture-spine] shows this separation. The Python client and CLI +provide independent entry points to the same service. ```{.mermaid #fig:architecture-spine caption="Clients share lifecycle services and provider-independent execution."} @@ -694,29 +740,26 @@ flowchart TB Providers --> Handlers ``` -The API operation layer is independent of the wire protocol. The JSON-RPC -adapter translates requests and responses; it does not decide how a foreach -iteration returns or how a reducer applies a state update. Conversely, the -execution core does not need to know whether its caller used Python, a -command line, or another application. +The JSON-RPC adapter translates network requests into API calls and serializes +their responses. The API coordinates validation, storage, and execution. +The runtime handles graph behavior, including iteration returns and state +updates, for all clients of that API. -This distinction also limits the role of an agent. An agent can help author -or operate a workflow through these interfaces, but the runtime follows the -saved graph. It does not ask an agent to choose the next step unless the -author has explicitly included an operation that makes such a decision. +An agent can author or operate a workflow through these interfaces. During a +run, the runtime follows the saved graph. Agent participation within that +execution requires an authored operation that invokes the agent. ## Keeping a Definition Separate from Its Use -The client owns the editable graph; the service owns saved artifact, +The client owns the editable graph. The service owns saved artifact, deployment, and run records. The API coordinates validation and persistence through their respective stores. Execution reads a selected definition and its resolved environment rather than the client's mutable editor contents. For the report workflow, this boundary prevents an unfinished edit to the extraction step from becoming the definition of an already-created run. -Inspection retrieves the run's recorded identity and execution evidence; -it does not reconstruct the attempt from whatever is currently open in the -author's session. +Inspection retrieves the definition identity and execution evidence recorded +for that particular run. The client exposes this progression through workflow artifact, deployment, and run objects. A run object contains a snapshot of the stored execution. @@ -724,22 +767,36 @@ Refreshing it requests a new snapshot. This makes network activity explicit, although applications must decide when to refresh and how to present progress. -Draft workspaces provide a separate persisted editing surface used by the -draft API and CLI. They are not a required intermediate object for every -Python authoring operation. Both routes ultimately produce a saved -definition that the deployment and execution layers can use. +The draft API and CLI also support persisted editing workspaces, which retain +unfinished definitions between requests. Python authoring can save directly +from its editable object. Both routes produce a saved definition for deployment +and execution. -## Data Movement and Control Movement +## Executing the Graph + +The execution core is the runtime that runs the saved graph. It holds the +workflow input, working state, and current execution positions. To execute +an ordinary operation node, it resolves the node's input bindings, calls the +selected operation, checks its result, applies output bindings, and follows +the edge selected by the outcome. Repeating that cycle advances the run until +it completes, fails, or pauses for input. + +Some nodes control execution directly. A condition chooses a route, a foreach +executes a body for each item, and an interrupt requests additional input. +A subgraph node calls another saved workflow and uses its result when that +call completes. The called workflow is referred to as the **child workflow**. +Its input and state belong to that invocation, just as a function call has +its own arguments and local working data. Within a workflow, input bindings supply a step's arguments. Output bindings select returned values to write into workflow state. Reducers determine how those writes combine with existing values. The returned outcome selects the -next edge. These are related operations, but none substitutes for another: -routing to a renderer does not, by itself, supply the report it needs. +next edge. Rendering therefore requires both a route that reaches the renderer +and an input binding that supplies the report it consumes. -[@fig:node-execution-cycle] summarizes an ordinary callable step. Conditions, -iteration controllers, subgraphs, and interrupts have their own runtime -handlers rather than pretending to be remote capability calls. +[@fig:node-execution-cycle] follows one operation node through this cycle. +The later iteration examples show how the runtime manages additional +execution positions while a controller or child call is active. ```{.mermaid #fig:node-execution-cycle caption="A step updates state before following its selected route."} @@ -760,9 +817,9 @@ flowchart TB ``` A declared outcome such as `needs_information` is a workflow decision. A -handler exception or exhausted step budget is an execution failure. A -workflow can therefore complete with a non-success business outcome without -being a failed runtime execution. Where iteration supports collecting item +handler exception or exhausted node-execution allowance is an execution +failure. A workflow can therefore complete with a non-success business outcome +without being a failed runtime execution. Where iteration supports collecting item errors, that policy must be explicit; errors do not automatically become ordinary outcome edges. @@ -781,12 +838,27 @@ Suppose the report now covers two documents, `A.md` and `B.md`. The workflow must render a report for each, then assemble the two reports. For this first example, foreach is configured to process one document at a time. An output binding appends each rendered report to a `reports` -state field; the assembly step reads that field. +state field, which the assembly step reads. -The important distinction is between finishing **one document** and -finishing **the whole collection**. After A finishes, the foreach continues -with B. Only after B finishes does it follow its `done` route to assembly. -It does not begin the collection again whenever an item returns. +[@fig:foreach-graph] shows the authored graph. The `loop` edge enters the +item body. Its return edge, `render.ok -> each`, ends that item's work. +The `done` edge leads to the continuation after the collection is processed. + + +```{.mermaid #fig:foreach-graph width=90% caption="The item body returns to foreach; done continues to assembly."} +%%{init: {"flowchart": {"curve": "stepAfter"}}}%% +flowchart LR + Prepare["Prepare documents"] -->|ok| Each["each: Foreach"] + Each -->|loop| Render["render: Render item"] + Render -->|ok: item return| Each + Each -->|done| Assemble["Assemble reports"] + Assemble -->|ok| Finish(["End workflow"]) +``` + +In serial mode, A's return advances the foreach to B. B's return completes +the collection and permits the `done` transition to assembly. Returning to +the active foreach controller therefore finishes one item. Entering that +controller from `Prepare documents` starts an invocation over the collection. [@fig:foreach-region] follows one invocation from start to finish. Read downward for time. Solid arrows request work or apply a data binding; @@ -816,11 +888,12 @@ The same authored render node executes twice. The runtime distinguishes those executions so that finishing A advances to B, while finishing B permits assembly. Each item returns to the foreach invocation that started it. -This explains the graph's back-edge: `render.ok -> each` is an item return, -whereas `each.done -> assemble` leaves the loop. The output binding is what -appends the report; the back-edge itself does not transport or collect data. -The graph validator rejects a body node used both inside and outside that -loop, or a nested body that returns past its immediate owner. +The back-edge in [@fig:foreach-graph] records the item's return. Its output +binding appends the report before that return. A node in the body belongs to +that loop's control region. The validator rejects using the same node both +inside and outside the body, or returning past the immediate enclosing loop. +Ordinary cycles within one region remain valid, subject to the run-wide +node-execution limit described in the implementation chapter. ### Calling a Child Workflow for One Document @@ -829,13 +902,13 @@ summarizes one document. Consider only the item for `A.md`. The parent may know the whole collection, but the child receives only the input explicitly mapped into its call: `{"document": "A.md"}`. -The child has its own input, working state, and execution context. It cannot -read the parent's current item or `reports` field merely because the parent -called it. If it needs another value, the author must add an input binding. +The child has its own input, working state, and execution context. Access to +the parent's current item or `reports` field requires an input binding that +supplies the needed value. [@fig:scope-boundaries] shows the call and its return. The item waits while -the child runs. The child's `END` completes that child invocation, not the -parent item or the whole report workflow. +the child runs. The child's `END` returns its result to the calling node. +The parent item then continues to its own return edge. ```{.mermaid #fig:scope-boundaries width=95% caption="Child completion precedes the parent item's return to foreach."} @@ -879,10 +952,9 @@ neither the arrows nor the fact that both items completed chooses a merge policy. The serial ordering in [@fig:foreach-region] is not a promise about concurrent completion order. -If a child or item reaches an explicit interrupt, the run must preserve -which document was active and where it was waiting. Resume continues that -saved work; it does not restart the collection or silently move the -response to another item. +If a child or item reaches an explicit interrupt, the run preserves which +document was active and where it was waiting. Resume applies the response +to that saved position and continues the same item. ## Connecting External Operations @@ -906,156 +978,174 @@ another operation with the same fields. # Implementation -The Python package structure localizes changes to the relevant responsibility. -Improving a Python editing method should not require -changing the scheduler, and adapting a new remote service should not require -changing graph routing. +Running the report procedure involves two kinds of work. The author first +uses the Python client to construct, validate, and save a graph. The service +then executes that saved graph with supplied input and records the result. +This chapter follows those operations into the implementation, including +the additional bookkeeping needed when steps repeat, call child workflows, +or pause for input. ## Following a Python Authoring Request -The client package, `wf_client`, connects Python objects to a narrow -`WorkflowClientPort`. `App.from_http_jsonrpc(...)` configures the connection -without making a request. A subsequent capability inspection performs I/O, -decodes the response, checks the returned identity, and constructs a -`RemoteCapability`. +The client package, `wf_client`, exposes the `App` connection object introduced +in the architecture chapter. Its `App.from_http_jsonrpc(...)` constructor +records the service address. A later capability lookup contacts that service, +checks the returned identity and contract, and constructs a Python capability +object for the author to inspect or add to a workflow. -An `EditableWorkflow` subclasses the authoring layer's `WorkflowBuilder`, -reusing its graph-building methods. It adds remote validation and saving -rather than maintaining a second independent builder implementation. -Its validation method contains this early return: - -```python -local = self.validate_local() -if not local.ok: - return WorkflowValidation(local, "not_run", ()) -``` - -This excerpt from `wf_client/authoring.py` explains an observable behavior: -a structurally invalid edit produces local feedback without a server -request. Passing that check does not establish deployment readiness; the -server still validates the submitted plan against its inventory. +The editable workflow object reuses the graph-building methods of +`WorkflowBuilder`, the authoring layer's builder. It adds validation through +the service and saving. Validation first checks the graph locally. If a +binding references an undeclared state field, the client returns that error +immediately and records that server validation was not run. Otherwise, it +submits the graph for the server to check against its available operations. Saving validates first, submits the plan, checks the save response, and re-inspects the exact artifact version. The resulting artifact object is therefore reconstructed from the stored definition. Identity checks reject responses identifying a different artifact or deployment. -The same principle applies to runs. In `wf_client/runs.py`, `refresh()` -returns a decoded snapshot from `inspect_run`, checking the expected run -and deployment identities. `resume()` returns another snapshot after -submitting the response. Application code must retain the returned object; -an earlier snapshot does not mutate when the server advances. +The service's `inspect_run` operation retrieves a stored run by its identifier. +The Python client's `Run.refresh()` method calls that operation, verifies the +run and deployment identities, and returns a new run snapshot. `Run.resume()` +submits the requested response for an interrupted run and also returns a new +snapshot. The caller retains the returned object to see the updated status +and result. -Serialization still exists at the service boundary. The benefit is that -callers need not manually rebuild the domain objects after every request. -The client does not eliminate the distinction between a local Python model -and a remote operation. +Requests and responses are serialized at the service boundary. Reconstructing +the returned objects in the client gives authors the same inspection methods +after discovery, execution, and later refreshes. -## Executing and Bounding a Run +## Executing Nodes and Limiting a Run -The `wf_core` package defines the graph and execution state. Its runtime -selects ready frames and dispatches by node kind. An ordinary `NodeUse` -invokes a bound callable; `ConditionNode`, `ForeachNode`, `SubgraphNode`, -`InterruptNode`, and `EndNode` implement explicit control behavior. +The `wf_core` package implements the execution cycle shown in +[@fig:node-execution-cycle]. It selects a ready execution position and handles +the selected node according to its kind. An operation node invokes the +callable chosen through the deployment. A control node instead performs its +built-in behavior, such as choosing a condition route or starting iteration. -Step admission happens before dispatch. The immutable `RunLimits` policy -sets a positive maximum, with a default of 10,000 attempts. The run stores -how many attempts have been admitted. Nested execution shares that run-wide -budget, and asynchronous dispatch reserves attempts before launching work. -Failed step attempts also consume the budget. +A cycle can revisit nodes indefinitely if its condition never selects an +exit. To bound this work, the run has a **node-execution limit**, with a default +of 10,000. Immediately before dispatching a node, the runtime checks that +capacity remains and increments the run's counter. This check and increment +is called admission. The counter therefore counts admitted executions, +including control nodes, repeated visits, and executions that subsequently +fail. It does not count only successful operations. -The budget bounds graph progress, including a cycle whose condition never -selects an exit. It is not a wall-clock timeout, a language-model token -allowance, or protection against a handler that blocks indefinitely. The -limit and consumed count persist with the run, so interrupting and resuming -does not reset the allowance. +The policy is represented by `RunLimits.max_steps`, and the stored counter +is `steps_executed`. Iteration and child workflows share this run-wide limit. +Asynchronous execution reserves capacity before launching the selected work. +When no capacity remains, the runtime fails the run before ever dispatching +another node. The limit and counter persist with saved execution state, so +resume continues with the remaining allowance. + +This bounds the number of admitted node executions. The duration of a single +operation and the cost of its external requests require separate controls. +A handler that blocks indefinitely can still prevent progress after admission. ## Preserving State Across Nested Execution -The runtime represents an individual execution position as a **frame**. -An item frame identifies its owning foreach invocation, distinguishing a -return from an item from a new entry into the controller. A **scope** contains -the input, state, and context of one workflow invocation. Child workflows -receive their own scopes. A **lineage** records a separate state history; -its buffered writes keep concurrent items isolated until merging. +Consider the two-document iteration in [@fig:foreach-graph]. In serial mode, +document A appends its report to state before document B starts. B can read +that updated state. In concurrent mode, A and B each work with a separate +view of state. An item can read its own updates while its sibling's pending +writes remain isolated. The foreach combines their updates after the required +items finish, using the state fields' reducers. -Workflow state updates pass through reducer-aware patches. Serial iteration -must make its writes visible to subsequent serial work, while concurrent -items need separate views until their results are combined. Nesting either -mode inside the other makes write ownership more subtle than committing -every result directly to global state. +Nesting introduces an additional case. Suppose each document is processed +concurrently, but its sections are processed serially. Section 2 can read the +work of section 1 in the same document. Those updates still belong to that +document's separate view until the outer foreach combines the documents. +Publishing every serial section's write directly into shared workflow state +would expose unfinished document results to the other concurrent items. -Two rules govern this routing: serial work observes preceding serial -writes, and concurrent siblings do not observe each other's unmerged writes. -The shared `commit_foreach_aware_patch` helper handles writes from ordinary -nodes, subgraph results, and interrupt responses. It walks serial owners -outward and selects the first concurrent item boundary, if present, as the -buffer destination. It continues checking ancestry before writing, so a -missing parent or cycle cannot cause a partial write merely because a -buffer destination was already found. With no concurrent boundary, the -patch commits through the enclosing serial owners. +The runtime records where each execution belongs. A **frame** records one +execution position, including its current node and owning foreach when it is +an iteration item. A **scope** contains the input, working state, and context +of one workflow invocation. A child workflow has its own scope. A **lineage** +records state updates associated with an execution, allowing concurrent items +to retain their separate views. -Combining results must also preserve each write's contribution, rather than -count a previously merged value as new work. For example, if an enclosing -list already contains `A` and an inner group appends `B` and `C`, its visible -result is `[A, B, C]`, but its contribution is only `[B, C]`. Appending the -visible result again would duplicate `A`. +An output binding produces a **patch**, a set of state writes together with +the information needed to apply their reducers. The runtime follows the +item's enclosing iterations to determine where that patch belongs. Serial +iterations pass their writes outward. At the nearest concurrent item boundary, +the writes are buffered in that item's lineage. With no such boundary, they +update the workflow scope's state. The runtime checks the complete ownership +chain before applying a patch, so malformed parent records cannot cause a +partial update. -When concurrent results are combined, the patch retains their constituent -write contributions for later reducer replay. Keeping only cumulative -values would allow a surrounding iteration to replay an already-counted -prefix. This distinction matters for operations such as appending report -sections: a correct visible value at one nesting level is not necessarily -a correct contribution to the next merge. +Merging nested results also requires distinguishing an accumulated value from +new writes. Suppose state already contains report A, and an inner group adds +B and C. The resulting view is `[A, B, C]`. If an outer merge appends this +whole view to the existing `[A]`, it produces `[A, A, B, C]`, which is undesirable. -This implementation supports iteration; general fork/gather remains -future work. +The patch therefore retains the new writes, or **contributions**, separately +from the accumulated value visible to readers. In this example the new +contributions are B and C. A later merge applies those writes using the +declared reducer, preserving A once. This bookkeeping supports nested +concurrent iteration even when its intermediate results are merged again. ## Giving Expressions a Consistent Context -Bindings and conditions need the same account of the current execution. -The runtime's `frame_context_view` derives structured foreach entries from -persisted frame ancestry. Entries are keyed by foreach node identity, so -nested bodies can refer to enclosing items within the same workflow scope, -as well as the current item. +Suppose an outer foreach named `documents` processes documents, and an inner +foreach named `actions` processes each document's action items. A node in the +inner body may need both the document and the current action. A single +unnamed current-item value would leave it unclear which one the node receives. -The walk stops at a subgraph scope boundary. An enclosing item's value must -be passed as child input if the child needs it. The reader also rejects -malformed ownership, parent cycles, and conflicting aliases; corrupt -checkpoint metadata is not treated as an innocently absent field. +**Execution context** supplies runtime information to expressions and input +bindings. Its foreach entries are keyed by the authored controller's identity. +Inside the inner body, `context.foreach.documents.item` selects the enclosing +document, while `context.foreach.actions.item` selects the current action. +The runtime constructs these entries from the active frames. Repeated +executions of the same node therefore resolve the paths to their own items. -A context path must have the same meaning in a condition and -an input binding within one execution. Both receive the structured mapping, -while validation checks paths against the corresponding context schema. -Otherwise, a condition testing whether the current item exists could select -the false route even though an input binding can read that item. +A condition checking an item's fields and an input binding reading those +fields receive the same context view. Validation checks these references +against the corresponding schema. This avoids accepting a path that a +condition would treat as absent while another reader can resolve it. + +Context construction stops at the current workflow scope. If a child workflow +needs the parent's document, its caller supplies that value through child +input, as shown in [@fig:scope-boundaries]. On resume, persisted frame ownership +identifies the active items. Missing owners, parent cycles, or conflicting +aliases are reported as errors before those records are used to construct +context. ## Validation, Persistence, and Diagnostics -The authoring layer checks graph structure; the server checks saved plans -and environment bindings; the runtime checks actual values and execution -state. Each layer has information the earlier one lacks. Static validation -can reject an invalid foreach return, for example, but cannot prove that a -remote operation will remain available when a run reaches it. +Validation first checks declarations that can be evaluated without running +operations. For example, two output bindings on the same node might write +`foo` to `state."1"` and `bar` to `state."1"."2"`. The first targets an entire +field and the second targets a location inside it. Applying both could replace +the container that the nested write needs. The validator rejects overlapping +destinations within that binding list, avoiding a dependency on their order. +Here `"1"` and `"2"` are literal field names. The rule also applies to ordinary +names such as `state.report` and `state.report.title`. + +This check concerns one set of mappings. Successive nodes can intentionally +update related state fields, and concurrent iteration combines writes through +its separate reducer rules. Validation also checks graph structure and rejects +illegal iteration returns. Server validation checks the available operations +and service connections. At execution time, the runtime checks actual values +and execution records as they become available. Diagnostics carry a code, a location, a message, and, where available, a -repair hint. These fields let a caller identify the faulty binding or node -without parsing a prose-only error. Suggested next actions are guidance, -not authorization and not evidence that a repair has succeeded. Schema -fingerprints likewise detect a changed contract representation; they do not -prove semantic compatibility. +repair hint. For an overlapping destination, the location identifies the +node's output-binding list. These fields let a caller locate the faulty +mapping and decide how to repair it. The revised definition then passes +through validation again. -The API lifecycle layer persists stopped runs and their versioned -checkpoints, including interrupted runs that may later resume. Restoration -validates the stored representation and recovers the execution state before -dispatch continues. This supports explicit pause-and-resume boundaries; it -does not promise durable recovery from every instruction inside an arbitrary -handler or exactly-once external side effects. +The API lifecycle layer stores stopped runs and their **checkpoints**, saved +representations of execution state. An interrupted checkpoint contains the +position and working data needed to resume. Restoration validates those +records before execution continues from the explicit interruption boundary. +The limits of recovery for external operations are discussed in Limitations. Separating artifact, deployment, and run storage also keeps inspection focused. Definition inspection explains what was saved; deployment inspection explains environment selection; run inspection and trace explain -what happened during a particular attempt. +what happened during a particular run. ## Server and Provider Responsibilities @@ -1085,16 +1175,16 @@ The three operations read notes, extract a report, and render Markdown. The input is deliberately constrained. Notes contain named sections and action lines with owner, task, and due-date fields. Extraction parses that -format; it is not language-model summarization of arbitrary documents. With -fixed input and local Python operations, the result can be checked without -remote credentials, service quotas, or variation in generated text. +format using deterministic Python code. Fixed input and local operations +allow the result to be checked independently of remote credentials, service +quotas, or variation in generated text. The bundle at [`examples/report_workflow/`](../../examples/report_workflow/) supplies the operations, fixture notes, server configuration, and a saved raw-plan example. The walkthrough below expresses the same procedure through the current Python client. The example README retains a command-line route for operators -who need it; that route is not a prerequisite for using this interface. +who need that interface. ## Starting with Available Operations @@ -1136,14 +1226,17 @@ print(extract_report.output_schema) Each lookup returns a capability object with its schemas and declared outcomes. The extraction schema exposes `title`, `summary`, `action_items`, -`risks`, and `followups`; each action item has `owner`, `task`, and `due` fields. +`risks`, and `followups`. The title names the report and the summary records its +overview. Action items describe work to do: `owner` identifies the responsible +person, `task` describes the work, and `due` records its due-date text. Risks +list potential problems, and followups record matters needing later attention. The author can use the schema directly or declare corresponding local models. This session uses local models without importing provider implementation code. The listing contains `local.report.read_notes`, `local.report.extract_report`, -and `local.report.render_markdown_report`. It establishes the available names; -the inspected contracts then describe how each can be connected. The author -still selects the operations, rather than the service synthesizing a plan. +and `local.report.render_markdown_report`. Their inspected contracts describe +how each can be connected. The author selects the operations and constructs +the graph from those contracts. Before connecting operations, the author checks their input and output contracts for compatible fields. @@ -1196,15 +1289,22 @@ graph = app.new_workflow( ) ``` -The models export schemas for the workflow contract. They do not make the -saved workflow dependent on a live Python class instance. The initially -absent report belongs to intermediate state; the public result requires a -report because a completed successful pipeline should have produced one. +The models export schemas that are stored with the workflow contract. This +lets the service validate its data after the author's Python session ends. +The report starts as absent in intermediate state. The public result requires +a report because the successful pipeline produces one before completion. ## Connecting Data and Decisions The author now creates three node uses. Each use selects an operation and -declares its data bindings: +declares its data bindings. In the first mapping, `input_path("text")` selects +the workflow's public input field, and the second `"text"` names the reading +operation's argument. They happen to share a name. The extraction mapping +instead reads the state field `notes` into an argument named `text`. + +In an output binding, `"."` selects the complete returned object. Extraction +stores that whole report in `state.report`, while reading and rendering store +only selected output fields: ```python read = graph.use( @@ -1217,7 +1317,7 @@ extract = graph.use( extract_report, id="extract", input=[input_from(state_path("notes"), "text")], - output=[output_to((), state_path("report"))], + output=[output_to(".", state_path("report"))], ) render = graph.use( render_report, @@ -1225,7 +1325,13 @@ render = graph.use( input=[input_from(state_path("report"), "report")], output=[output_to("markdown", state_path("markdown"))], ) +``` +The author connects each operation's `ok` outcome to the next node, ending at +a node that completes the workflow with outcome `ok`. Final output bindings +then select the report and Markdown from state: + +```python end = graph.end("ok", id="finished") graph.set_entry_point(read) graph.connect(read, "ok", extract) @@ -1237,10 +1343,9 @@ graph.set_output([ ]) ``` -The empty tuple in `output_to((), ...)` selects the extraction step's -whole output object. The other output bindings select individual fields. -The `connect` calls then specify execution order for the `ok` outcome; -they do not implicitly carry those objects between steps. +Using `"."` selects all fields of that node's output. An empty output-binding +list would store none of them in workflow state. The `connect` calls determine +execution order independently of these data selections. The author declares data mappings and execution routes separately. Changing one leaves the other unchanged. This requires additional declarations even @@ -1250,19 +1355,21 @@ representation from those declarations. ## Diagnosing and Repairing a Binding An editable graph can temporarily be invalid. Suppose a final output binding -names a state field that does not exist: +reads `missing_report`, although the declared state contains `report`: ```python +# Deliberately refer to an undeclared state field. graph.set_output([ input_from(state_path("missing_report"), "report"), input_from(state_path("markdown"), "markdown"), ]) broken = await graph.validate() assert not broken.ok -assert broken.remote_status == "not_run" +assert broken.remote_status == "not_run" # Local rejection skips the server check. for issue in broken.local.errors: print(issue.code, issue.path, issue.message) +# Repair the field reference, leaving the nodes and edges unchanged. graph.set_output([ input_from(state_path("report"), "report"), input_from(state_path("markdown"), "markdown"), @@ -1276,9 +1383,9 @@ Its message is: > source path must start with input., state., or context. and reference a > declared root field when applicable -This locates the rejected mapping but does not name a replacement field; -the author must compare it with the declared state. No server validation -request is made for that invalid graph. +The location identifies the first workflow-output mapping. Comparing its +source with the declared state reveals the misspelled reference. Local +validation rejects the graph before a server validation request is made. The repair changes the workflow's output mapping. The renderer and its outgoing edge remain unchanged. This illustrates why data bindings and control routes need separate feedback. @@ -1307,7 +1414,7 @@ The artifact is the saved version of the authored procedure. The deployment maps the saved `local.report` requirement to `local.report_runtime`. The fixture registers compatible operations under both names so this choice changes the execution source without changing the graph or requiring remote -credentials. It demonstrates rebinding, not migration between real services. +credentials. Both source identities refer to the example's local operations. [@fig:python-lifecycle] summarizes the public operations; it omits internal validation and re-inspection calls made by individual client methods. @@ -1334,9 +1441,9 @@ sequenceDiagram Client-->>Author: Result or diagnostic ``` -An edit to the workflow would be saved as another version, not applied -retroactively to the artifact used by this deployment. Conversely, selecting -a different source environment is a deployment concern. A readiness check +An edit to the workflow would be saved as another version, leaving this +deployment's artifact unchanged. Selecting a different source environment +changes the deployment's bindings. A readiness check can reject a missing or incompatible binding before a run is attempted. ## Running and Inspecting the Result @@ -1346,6 +1453,7 @@ does not need access to the client's file path. ```python notes = Path("examples/report_workflow/input.md").read_text(encoding="utf-8") +# Permit at most 100 admitted node executions across this run. run = await deployment.run({"text": notes}, max_steps=100) if run.status != "completed": raise RuntimeError((run.status, run.diagnostics)) @@ -1368,9 +1476,8 @@ provides step-level evidence when output alone is insufficient. For this fixed fixture, the result includes three action items, the recorded risks and followups, and a Markdown report headed “Weekly Project Update.” Checking these fields establishes that the -example's data reached the intended outputs. It does not establish that -the report is useful for every reader or that extraction works on -unstructured notes. +example's data reached the intended outputs. The extraction contract requires +the sectioned notes format described at the start of this chapter. ## What This Case Demonstrates @@ -1383,25 +1490,23 @@ The existing tests in check the source's input rules, rendering and extraction, capability discovery and invocation, and the artifact/deployment/run lifecycle using the raw-plan fixture. Those tests are evidence for the report operations -and lifecycle. They complement the Python session check; neither is a user study. +and lifecycle, complementing the Python session check. The Python presentation makes the current authoring experience concrete: inspect operations, declare contracts, connect data and outcomes, save, select bindings, and inspect an execution. The CLI and draft surface offer -another way to perform related lifecycle operations; they are not required -steps in this Python walkthrough. +alternative interfaces to related lifecycle operations. A direct Python script would be shorter for these three local functions. The additional structure becomes relevant when the definition must be saved, bound to an environment, validated independently, and -inspected through a shared service. This example demonstrates that -integration, not a performance advantage over function calls. +inspected through a shared service. This example demonstrates the integration +of those operations. -Nor does a linear pipeline exercise all graph semantics. It has no -conditional branch, foreach body, child workflow, or interrupt. Targeted -runtime tests provide evidence for those mechanisms; they should not be -credited to a case that never executes them. Ease of authoring and diagnosis -also requires evidence beyond a successful fixture run. +The case exercises a linear pipeline. Conditional branches, foreach bodies, +child workflows, and interrupts are covered by separate runtime tests listed +in the evaluation's evidence index. Independent authoring and diagnosis tasks +are needed to evaluate the interaction beyond this scripted session. # Evaluation @@ -1414,13 +1519,16 @@ interface. Usability remains a separate evaluation question. ## Requirements and Evidence -[@tbl:requirements-evidence] relates the requirements to the available -evidence. The walkthrough demonstrates interface operations supporting R1–R5. -The repository also contains tests covering X1–X5. The table distinguishes -these sources from the usability measurements still needed. +[@tbl:requirements-evidence] relates the authoring requirements R1–R5 and +execution requirements X1–X5 to the available evidence. E1–E5 refer to the +five groups in the Evidence Index appendix: E1 covers lifecycle operations, +E2 Python authoring, E3 validation, E4 runtime execution, and E5 source +providers. These labels locate implementation and test files. The table +distinguishes the walkthrough's observations from listed tests and the +usability measurements still needed. | Requirement | Evidence | Assessment | -| --- | --- | --- | +| --- | --- | ---- | | R1 Discovery | Walkthrough; E2 | Exercised; usability unmeasured | | R2 Data movement | Worked graph | Explained; comprehension unmeasured | | R3 Revision feedback | Binding repair; E3 | Exercised; usability unmeasured | @@ -1430,7 +1538,7 @@ these sources from the usability measurements still needed. | X2 Known constraints | Validation; E3 | Constraint tests listed | | X3 Routing and state | Runtime; E4 | Runtime tests listed | | X4 Environment choices | Sources; E5 | Provider tests listed | -| X5 Bounds and inspection | Budget/resume; E1, E4 | Boundary tests listed | +| X5 Bounds/inspection | Node limits/resume; E1, E4 | Boundary tests listed | : Requirements and available evidence. {#tbl:requirements-evidence} @@ -1438,12 +1546,13 @@ The R4 observation covers saving and running version 1. The session does not edit a later version or check its effect on earlier runs. Understanding the version distinction remains part of the authoring evaluation. -X5 contains three distinct obligations: admission limits, execution -inspection, and interruption resume. Budget tests exercise limit persistence -and exhaustion; lifecycle tests exercise stored inspection and resume. -The report session exercises inspection but does not exhaust its budget or -interrupt. The evidence index locates these separate checks; it is not a -record of a newly executed full-system suite. +X5 covers the run-wide node-execution limit, execution inspection, and +interruption resume. Node-execution limit tests check that the counter +persists and that exhaustion prevents further dispatch. Lifecycle tests +exercise stored inspection and resume. The report session exercises inspection +with a limit of 100 node executions. It completes within that limit and +contains no interruption. The evidence index locates the separate tests for +exhaustion and resume. ## Walkthrough Method and Observations @@ -1461,7 +1570,7 @@ session. The fixture expectations are fixed independently of the workflow's result, and stored output is inspected again through the API. | Check | Expected observation | Requirement | -| --- | --- | --- | +| ---- | ----- | --- | | Invalid output mapping | Local rejection at `output[0].path` | R3, X2 | | Repaired graph | Validation permits saving and running | R3, X2 | | Persisted report | Expected title and three action items | X1, X3 | @@ -1518,7 +1627,7 @@ The implementation would fail its stated contracts if, for example: - a saved run could not identify the definition and bindings it used; - an invalid foreach boundary were accepted and executed as another region; - nested state writes were lost or counted twice; -- resume reset the run-wide budget or resumed the wrong item; +- resume reset the node-execution counter or resumed the wrong item; - condition evaluation used a different context model from validation; - ordinary source invocation required provider-specific graph routing. @@ -1553,7 +1662,7 @@ whether a report is factually correct or a remote side effect was desirable. ## Execution Guarantees Have Defined Boundaries Foreach iteration, nested workflow scopes, structured context, and run-wide -step budgets are implemented foundations. They do not yet provide general +node-execution limits are implemented foundations. They do not yet provide general fork/gather control for arbitrary branches. In particular, concurrent iteration should not be presented as a solution to correlating branches that split, loop, and later meet at different gather points. @@ -1672,18 +1781,28 @@ the current single-scheduler arrangement. ## An Assistant-Backed Authoring Application -The surrounding application is intended to combine assistant-backed chat -with workflow administration. A shell can let an assistant retain Python -objects across interactions, while typed client objects can support -dedicated views of artifacts, deployments, and runs. The design question is -how to let an author inspect and correct a proposed procedure before executing -it, while keeping subsequent run status and requests for input understandable. +A companion local-first assistant application has an execution backend for +persistent Python shells, streamed code output and images, execution approvals, +and saved conversations. Its chat interface is under development. The shells +allow an agent to retain Python objects across interactions, which provides +an environment in which it could use the workflow client. Code executes on +the local machine without a sandbox. Saved history survives backend restart, +but previous shells are not restored and unfinished work is not replayed. -This application requires an interaction design and evaluation of how the -assistant uses the public client to construct, revise, and inspect workflows. +Integrating this application with workflow authoring and administration remains +to be evaluated. An author needs to inspect and correct a proposed procedure +before running it, then follow its status and respond to interruptions. +Dedicated views of the client's artifact, deployment, and run objects could +support that interaction. The present case study evaluates the workflow +client directly and includes no agent-driven authoring session. ## Preserving Contracts Across Operational Changes +The source catalog already represents resources, and a built-in workflow +operation can read resource text with a size bound. Further authoring work +could make these resources easier to select and inspect alongside operations, +including clear treatment of content types and truncated results. + Additional providers and storage backends would test whether the architectural boundaries hold beyond the demonstrated implementations. The question is whether an integration can preserve source compatibility, run identity, and @@ -1704,8 +1823,8 @@ workflow use a record beyond the lifetime of an editing session or a single script invocation. The graph model separates data movement from control movement. Contracts -and bindings describe what a step receives and writes; outcomes choose -transitions; explicit runtime constructs govern iteration, child scopes, +and bindings describe what a step receives and writes. Outcomes select +transitions, while runtime constructs govern iteration, child scopes, and interruption. The Python client exposes this model through authoring and inspection objects, while the API and provider boundaries connect it to configured operations. @@ -1774,9 +1893,10 @@ E3: structural validation, source compatibility, and repair information. - `tests/core/test_structured_context_validation.py` - `tests/core/test_foreach_control_regions.py` -## Execution Ownership and Budgets +## Execution Ownership and Node Limits -E4: nesting, structured context, concurrent iteration, and persisted limits. +E4: nesting, structured context, concurrent iteration, and persisted +node-execution limits. - `src/wf_core/runtime/` - `tests/core/test_foreach_back_edges.py` @@ -1794,6 +1914,7 @@ E5: source contracts and provider-specific execution behind server composition. - `src/wf_server/config.py` - `src/wf_sources_python/` - `src/wf_sources_mcp/` +- `src/wf_api/source_helpers.py` - `tests/wf_sources_python/test_loader.py` - `tests/wf_sources_mcp/test_runtime.py` - `tests/wf_transport_rpc_http/test_mcp_backed_server_rpc.py`