docs: finalize thesis evaluation and front matter

docs/thesis/.gitignore

@@ -3,3 +3,4 @@
 *.tex
 *.typ
 !figures/*.svg
+!title-pages.tex

docs/thesis/agent-challenge-cohort.json

@@ -6,7 +6,9 @@
   "limitations": [
     "The three waves span repository snapshots; they are longitudinal engineering evidence, not a controlled model comparison.",
     "The base prompt changed before wave 3 to require the challenge report inline.",
-    "The models were free hosted OpenCode endpoints, so service load and latency were not controlled."
+    "The models were free hosted OpenCode endpoints, so service load and latency were not controlled.",
+    "The campaign is limited to two tasks and two hosted models, so results may not transfer to other workflows, providers, models, or deployment conditions.",
+    "All manual audits were performed by the author; no second-rater reliability check was conducted."
   ],
   "runs": [
     {

docs/thesis/agent-challenge-results.md

@@ -1,11 +1,13 @@
 ## Audited Agent Challenge Campaign
 
-The primary campaign contains 36 audited trials: 27 passes, 8 invalid samples, and 1 failure.
+The primary campaign contains 36 manually audited trials: 27 clean product-path passes under the campaign rules, 8 invalid evaluation samples, and 1 failure. These counts are not a model-success-rate estimate.
 
-The campaign crosses two challenges, two hosted models, three instruction profiles (`none`, `skills`, and `all`), and three repetitions per cell. The checked cohort snapshot records report hashes, prompt hashes, the repository commit, automatic metrics, and manual-audit outcomes; local raw report files are verified against those hashes when present.
+The campaign crosses two challenges × two hosted models × three instruction profiles (`none`, `skills`, and `all`) = 12 cells, with three repetitions per cell (n=3). The checked cohort snapshot records report hashes, prompt hashes, the repository commit, automatic metrics, and manual-audit outcomes; local raw report files are verified against those hashes when present.
 
 Because repository snapshots and one prompt rule changed between waves, this is longitudinal engineering evidence, not a controlled model comparison.
 
+> **Campaign validity note.** This campaign is a bounded longitudinal audit, not a controlled comparison. Each cell has n=3; waves changed product and prompt snapshots; all audits were performed by the author.
+
 Selection rule: The latest three completed, manually audited trials per challenge, model, and instruction profile as of 2026-06-30.
 
 | Challenge / model / profile | Pass | Invalid | Fail |
@@ -25,11 +27,11 @@ Selection rule: The latest three completed, manually audited trials per challeng
 
 : Audited outcomes by challenge, model, and instruction profile. {#tbl:agent-challenge-outcomes}
 
-A manual `pass` requires both successful product-path evidence and an acceptable audit trail. `Invalid` means the sample cannot support the clean benchmark claim, commonly because the agent read repository or example material outside its supplied workspace. `Fail` means the challenge contract itself was not established.
+A manual `pass` requires both successful product-path evidence and an acceptable audit trail. It does not imply the agent avoided every exploratory read, only that no disqualifying read or bypass was found. `Invalid` means the sample cannot support the clean benchmark claim, commonly because the agent read repository or example material outside its supplied workspace. `Fail` means the challenge contract itself was not established.
 
 ![Audited outcomes by evaluation cell.](figures/agent-challenge-audited-outcomes-by-cell.svg){#fig:agent-challenge-audited-outcomes-by-cell width=95%}
 
-[@fig:agent-challenge-audited-outcomes-by-cell] reports all three repetitions rather than hiding invalid samples. The profile labels are descriptive; this campaign does not isolate instruction-profile effects.
+[@fig:agent-challenge-audited-outcomes-by-cell] reports all three repetitions rather than hiding invalid samples. The profile labels are descriptive; this campaign does not isolate instruction-profile effects. Profile × wave is confounded because the base prompt changed before wave 3, so apparent differences may reflect prompt changes, model updates, or repository drift rather than instruction-layer effects.
 
 ![Automatic task outcomes compared with manual outcomes.](figures/agent-challenge-automatic-vs-manual-outcomes.svg){#fig:agent-challenge-automatic-vs-manual-outcomes width=75%}
 
@@ -39,12 +41,18 @@ A manual `pass` requires both successful product-path evidence and an acceptable
 
 The waves in [@fig:agent-challenge-longitudinal-outcomes] are not an improvement curve: product commits, prompt wording, and enforcement changed. They preserve the chronology needed to study those changes.
 
-![Duration and recorded token totals grouped by challenge, instruction profile, model, and wave.](figures/agent-challenge-duration-and-tokens.svg){#fig:agent-challenge-duration-and-tokens width=95%}
+![Wall-clock duration grouped by challenge, instruction profile, model, and wave.](figures/agent-challenge-duration.svg){#fig:agent-challenge-duration width=78%}
 
-[@fig:agent-challenge-duration-and-tokens] separates each challenge and metric into its own panel. Circle and square markers redundantly identify the models without relying on color. Wall-clock duration includes hosted-service latency, and OpenCode token totals include cache-read accounting, so neither axis is a normalized model-efficiency metric.
+[@fig:agent-challenge-duration] separates the two challenges. Circle and square markers redundantly identify the models without relying on color. Wall-clock duration includes hosted-service latency and is not a normalized model-efficiency metric.
+
+![Recorded token totals grouped by challenge, instruction profile, model, and wave.](figures/agent-challenge-token-volume.svg){#fig:agent-challenge-token-volume width=78%}
+
+[@fig:agent-challenge-token-volume] reports OpenCode token totals, which include cache-read accounting. The figure records observed workload volume; it is not an efficiency comparison.
 
 ### Campaign Limitations
 
 - The three waves span repository snapshots; they are longitudinal engineering evidence, not a controlled model comparison.
 - The base prompt changed before wave 3 to require the challenge report inline.
 - The models were free hosted OpenCode endpoints, so service load and latency were not controlled.
+- The campaign is limited to two tasks and two hosted models, so results may not transfer to other workflows, providers, models, or deployment conditions.
+- All manual audits were performed by the author; no second-rater reliability check was conducted.

docs/thesis/system-design-implementation.md

@@ -1,8 +1,8 @@
 ---
-title: "Design and Implementation of lda.chat"
+title: "Design and Implementation of lda.chat: An AI Agent for Automating and Creating Workspace Workflows"
-subtitle: "Infrastructure for AI Agents to Author and Execute Workspace Workflows"
+subtitle: ""
-author: "lda"
+author: "Lương Đức Anh"
-date: "2026-06-30"
+date: "July 1, 2026"
 lang: "en-US"
 documentclass: report
 papersize: a4
@@ -80,13 +80,13 @@ diagram:
 
 # Acknowledgements {.unnumbered}
 
-I thank **[External Supervisor Name]** at **[Company Name]** for providing the
+I would like to express my sincere gratitude to **Eng. Trần Văn Trường** at **Dicom Technology Co. Ltd.** for the trust, autonomy, and practical space to pursue this project and develop its technical direction independently. His thoughtful input, critical perspective, and support throughout the development process helped shape the project into a concrete and technically grounded engineering project.
-trust, autonomy, and practical space to pursue this project and to develop its
-technical direction independently.
 
-I also thank **[Internal Supervisor Name]** at **[University Name]** for academic
+I am also grateful to **Dr. Nghiêm Thị Phương** at the **University of Science and Technology of Hanoi** for coordinating the university--company requirements of the thesis, providing academic guidance on the submission process, and helping ensure that the final report met the university's formal academic requirements.
-guidance, critical feedback, and support in shaping the work into a formal
+
-thesis.
+I would also like to thank the **University of Science and Technology of Hanoi**, the **Department of Information and Communication Technology**, and **Dicom Technology Co. Ltd.** for providing the academic and professional environment in which this thesis could be carried out.
+
+Finally, I am deeply thankful to my family and friends for their encouragement, patience, and support during the development and writing of this thesis.
 
 # List of Abbreviations {.unnumbered}
 
@@ -101,7 +101,7 @@ thesis.
 | RPC | Remote Procedure Call |
 | USTH | University of Science and Technology of Hanoi |
 
-: Abbreviations used in the thesis. {#tbl:abbreviations}
+: Abbreviations used in the thesis. {#tbl:abbreviations .unnumbered}
 
 # Abstract {.unnumbered}
 
@@ -114,17 +114,21 @@ workflows as outcome-routed graphs and manages them through a
 Draft--Artifact--Deployment--Run lifecycle. A neutral source-provider boundary
 projects built-in, Model Context Protocol, and Python capabilities into the same
 workflow surface, while structured diagnostics and repair guidance support
-agent-operated authoring through CLI and JSON-RPC interfaces.
+agent-operable authoring through CLI and JSON-RPC interfaces. Here, "AI Agent"
+names the agent-facing project context; the submitted implementation is the
+workflow substrate exposed to external agents.
 
 The implementation is evaluated through automated conformance tests, a
 deterministic three-node report workflow, a browser-interaction workflow, and a
 manually audited external-agent campaign. The campaign contains 36 trials across
 two challenges, two hosted models, three instruction profiles, and three
-longitudinal waves. Manual audit classified 27 trials as passes, eight as invalid
+longitudinal waves. Manual audit, performed by the author, classified 27 trials
-evaluation samples, and one as a failure. The disagreement between automatic
+as clean product-path passes under the campaign rules, eight as invalid
-completion and manual outcomes demonstrates why successful execution alone is
+evaluation samples, and one as a failure. These counts are not a
-insufficient evidence when agents can inspect implementation files, prior
+model-success-rate estimate. The disagreement
-artifacts, or evaluator state.
+between automatic completion and manual outcomes demonstrates why successful
+execution alone is insufficient evidence when agents can inspect implementation
+files, prior artifacts, or evaluator state.
 
 The contribution is architectural rather than algorithmic: a typed workflow
 lifecycle, a provider-neutral capability boundary, and an agent-operable
@@ -136,6 +140,14 @@ comparison.
 
 # Introduction
 
+In the thesis title, "AI Agent" refers to the broader `lda.chat`
+agent-facing automation project. The submitted implementation focuses on the
+workflow substrate that such agents use: a typed runtime and lifecycle layer
+exposed through CLI and API surfaces. Experimental agent-harness work exists in
+adjacent project work, but it is outside the submitted implementation boundary;
+this report evaluates the substrate rather than claiming a production
+autonomous agent brain.
+
 This report assumes a setting in which external LLM agents are used as workflow
 authors and operators, and asks what platform substrate they need for reusable
 workspace automation. It describes the design and implementation of `lda.chat`,
@@ -177,7 +189,7 @@ This work makes five architectural and systems-engineering contributions:
    MCP, and Python sources share one workflow model without provider logic in
    the core runtime.
 4. It exposes structured validation diagnostics, repair hints, next-action
-   guidance, and inspection surfaces intended for agent-operated authoring.
+   guidance, and inspection surfaces intended for agent-operable authoring.
 5. It implements and evaluates the design through deterministic case studies,
    automated conformance tests, and a bounded manually audited agent campaign.
 
@@ -366,6 +378,7 @@ The document uses these terms with specific meanings:
 | Source provider | Server-side code that loads or manages sources for a source family. | Python source loading |
 | Tool | A provider-native operation before projection into workflow form. | MCP tool |
 | Agent-operable | A surface designed for machine clients: structured output, explicit validation, stable commands, inspectability, and bounded summaries. It does not mean independently proven agent success rates. | `wf deploy validate`, `wf run trace` |
+| `RawWorkflowPlan` | A serialized workflow plan used to create an immutable artifact without first going through a mutable draft workspace. | `wf artifact create-from-plan` |
 | Outcome | A control-flow label returned by a node and consumed by graph edges. | `ok`, `error`, `submitted` |
 | Output | The data payload returned by a node or workflow. | `{ "report": "..." }` |
 | Reducer | A pure state-merge operation selected by state schema. | `wf.std.replace`, `wf.std.append` |
@@ -611,10 +624,10 @@ is created at each stage.
 stateDiagram-v2
   direction TB
   [*] --> DraftWorkspace
-  [*] --> RawPlan
+  [*] --> RawWorkflowPlan
   DraftWorkspace --> DraftValidated: validate draft
   DraftValidated --> Artifact: save immutable version
-  RawPlan --> Artifact: create artifact from plan
+  RawWorkflowPlan --> Artifact: create artifact from plan
   Artifact --> Deployment: bind sources
   Deployment --> DeploymentValidated: validate deployment
   DeploymentValidated --> Run: start run
@@ -646,8 +659,9 @@ failed runs remain inspectable records.
 
 ## Workflow Core Model
 
-The core model processes graph execution through typed stages. To improve clarity, this section separates the broad runtime loop
+The core model processes graph execution through typed stages. This section
-from the ordinary callable-node path. [@fig:core-runtime-loop] shows how the
+separates the broad runtime loop from the ordinary callable-node path.
+[@fig:core-runtime-loop] shows how the
 runtime selects a frame, dispatches by step kind, records trace, and routes by
 outcome. [@fig:nodeuse-execution-path] then zooms into the `NodeUse` path,
 where most source-backed work occurs; it expands the `NodeUse` branch from
@@ -682,6 +696,10 @@ flowchart TB
   End --> Output[Project workflow output]
 ```
 
+```{=latex}
+\clearpage
+```
+
 ```{.mermaid #fig:nodeuse-execution-path height=80% caption="NodeUse execution path: a callable node resolves bindings, invokes a NodeDef handler, checks the declared outcome, applies reducer-aware state writes, appends trace, and returns to outcome routing."}
 sequenceDiagram
   participant Runtime as Workflow Runtime
@@ -784,7 +802,7 @@ The implementation is organized into focused packages with clear boundaries:
 The workflow core implements deterministic execution semantics. It processes a
 typed graph definition, validates input against `input_schema`, executes the
 selected node use, routes by declared outcomes, applies reducers to state
-writes, and produces trace frames. The current public semantics are
+writes, and produces trace frames. The public semantics are
 outcome-routed graph execution with explicit condition, foreach, subgraph,
 join, interrupt, and end steps. The async runtime has internal frame and lineage
 machinery for foreach admission and state isolation, but this report does not
@@ -839,9 +857,9 @@ policy-enforcement layer. `wf_platform` should not grow into a dumping ground
 for stores, runtimes, or provider lifecycle. Those belong in `wf_api`,
 `wf_server`, or the specific `wf_sources_*` package.
 
-The API lifecycle is deliberately centralized through one facade, per [@fig:api-lifecycle-facade].
+The API lifecycle is deliberately centralized through one facade, per
-This is not a second architecture diagram; it is the application-layer mechanism
+[@fig:api-lifecycle-facade]. The facade is the application-layer mechanism that
-that prevents lifecycle operations from becoming disconnected CRUD calls.
+prevents lifecycle operations from becoming disconnected CRUD calls.
 
 ```{.mermaid #fig:api-lifecycle-facade caption="API lifecycle facade: one WorkflowOperationContext carries stores, source inventory, runtime execution, and live checks for all lifecycle sub-APIs."}
 classDiagram
@@ -1230,12 +1248,12 @@ benchmark.
 
 ## Qualitative Comparison
 
-| Capability | Direct LLM tool loop | Generated script | Mature automation platform | `lda.chat` prototype |
+| Capability | Direct LLM tool loop | Generated script | Representative hosted automation platform | `lda.chat` prototype |
 | --- | --- | --- | --- | --- |
 | Versioned workflow artifact | Not inherent | Manual | Often yes | Prototype support |
 | Deployment/source binding | Not inherent | Manual config | Platform-specific | Prototype support |
 | Typed validation before run | Tool-schema dependent | Custom | Varies | Controlled-test support |
-| Durable run record | Not inherent | Custom | Often yes | Prototype support at stopped boundaries |
+| Persisted prototype run record | Not inherent | Custom | Often yes | Explicit stopped/interrupted boundaries only |
 | Source drift diagnostics | Not inherent | Custom | Varies | Schema-hash controlled examples |
 | Agent-operable repair hints | Not inherent | Custom | Usually human UI | Prototype support |
 | Scheduling | Depends on agent | External scheduler | Yes | Future work |
@@ -1245,7 +1263,9 @@ benchmark.
 The comparison positions the architecture; it is not a quantitative claim that
 the prototype outperforms mature automation products. "Not inherent" means the
 feature can be added by surrounding infrastructure, but is not provided by the
-bare strategy alone.
+bare strategy alone. "Mature automation platform" summarizes representative
+hosted automation products discussed in the Related Work chapter; it is not a
+market-wide survey.
 
 ## Formative Agent-Trial Findings
 
@@ -1274,21 +1294,34 @@ the same failures.
 
 ## Evidence Package
 
-The evidence supporting the thesis claims includes:
+The evidence supporting the thesis claims is summarized below.
 
-| Claim | Evidence | What it asserts | Result |
+- **Deployment validation catches source drift.** Evidence:
-| --- | --- | --- | --- |
+  `test_validation.py`. It asserts that missing, disabled, or changed
-| Deployment validation catches source drift | `tests/artifacts/test_validation.py` | Missing, disabled, or changed capabilities produce diagnostics | Pass in focused test suite |
+  capabilities produce diagnostics. Result: pass in the focused test suite.
-| Interrupted runs resume at explicit boundaries | `tests/wf_api/test_run_api.py` and resume-concurrency tests | Stopped run state is persisted and resumed through the run API | Pass in focused test suite |
+- **Interrupted runs resume at explicit boundaries.** Evidence:
-| Python source lifecycle works | `tests/examples/test_report_workflow_example.py` | Python capability -> artifact -> deployment -> run completes | Pass in focused test suite |
+  `test_run_api.py` and resume-concurrency tests. They assert that stopped run
-| Serial multi-node workflow works | `tests/examples/test_browser_click_workflow_example.py` | `open_click_page` -> `wait_for_click` -> `collect_snapshots` completes with before/after evidence | Pass in focused test suite |
+  state is persisted and resumed through the run API. Result: pass in the
-| Bounded agent-operability campaign is checkable | `docs/thesis/agent-challenge-cohort.json`, generated results/figures, local report hashes, and Appendix C | Two challenges, two models, three instruction profiles, and three audited repetitions per cell | 36 audited trials: 27 pass, 8 invalid, 1 fail |
+  focused test suite.
-| CLI and JSON-RPC share the API surface | `tests/wf_transport_rpc_http/` and `tests/wf_cli/` | Transport and CLI delegate to the same workflow operations | Pass in focused test suite |
+- **Python source lifecycle works.** Evidence:
+  `test_report_workflow_example.py`. It asserts that a Python capability can be
+  loaded, saved as an artifact, deployed, and executed. Result: pass in the
+  focused test suite.
+- **Serial multi-node workflow works.** Evidence:
+  `test_browser_click_workflow_example.py`. It asserts that `open_click_page`,
+  `wait_for_click`, and `collect_snapshots` complete with before/after evidence.
+  Result: pass in the focused test suite.
+- **Bounded agent-operability campaign is checkable.** Evidence:
+  `agent-challenge-cohort.json`, generated results and figures, local report
+  hashes, and Appendix C. It asserts that two challenges, two models, three
+  instruction profiles, and three audited repetitions per cell are explicitly
+  recorded. Result: 36 audited trials: 27 pass, 8 invalid, 1 fail.
+- **CLI and JSON-RPC share the API surface.** Evidence:
+  `tests/wf_transport_rpc_http/` and `tests/wf_cli/`. They assert that transport
+  and CLI operations delegate to the same workflow API surface. Result: pass in
+  the focused test suite.
 
-: Claim-to-evidence summary for the thesis evaluation. {#tbl:evidence-package}
+The bullet list summarizes repository evidence verified at the recorded commit.
-
-The table summarizes repository evidence; it is not a substitute for rerunning
-the verification commands before final submission.
 
 ::: {#include-agent-challenge-results}
 :::
@@ -1296,8 +1329,7 @@ the verification commands before final submission.
 ## Verification Snapshot
 
 This report records one focused verification snapshot to make the evidence
-claims auditable from the text. A final submission should regenerate this table
+claims auditable from the text.
-from the exact submitted commit.
 
 | Field | Value |
 | --- | --------- |
@@ -1420,7 +1452,7 @@ provides earlier, structured failure feedback.
 
 (Evidence: `src/wf_config/`.)
 
-## Planner Efficiency Hypothesis
+## Planner-Efficiency Design Hypothesis, Not Measured Outcome
 
 The platform targets planner efficiency and operational clarity rather than
 runtime throughput. The design hypothesis is that typed contracts, validation,
@@ -1635,6 +1667,11 @@ operational foundation or expands its feature scope.
   `NodeSpec` contracts, allowing planners to compose LLM steps into workflows
   without making the core runtime model-aware.
 
+- **Integrated agent harness.** Adjacent experimental work can be integrated
+  once the substrate boundary is stable. That future layer would provide the
+  autonomous planning loop that drives the workflow lifecycle; it remains
+  outside the implementation and evidence claims of this report.
+
 - **Scheduler and daemon operations.** Offline scheduling for deployments,
   cron-triggered runs, and server daemon lifecycle.
 
@@ -1656,7 +1693,7 @@ report described the design and implementation of `lda.chat`, a prototype
 platform that separates planning from execution across controlled built-in,
 MCP, and Python source examples.
 
-The implementation supports five claims:
+The implementation supports five bounded claims:
 
 1. A typed artifact, deployment, and run lifecycle provides persisted workflow
    records and resumability at explicit stopped/interrupted boundaries.
@@ -1665,7 +1702,7 @@ The implementation supports five claims:
    can be projected into the existing capability/source contract without
    core-runtime changes.
 3. Validation and diagnostics produce machine-readable failure states with
-   repair hints, intended to reduce planner trial-and-error.
+   repair hints intended to support planner repair loops.
 4. The CLI and JSON-RPC transport provide a surface designed for external LLM
    agents to drive without direct runtime access.
 5. The deterministic report-workflow case study demonstrates the full lifecycle
@@ -1675,7 +1712,10 @@ The remaining work is clear and bounded: provider lifecycle, production auth,
 scheduling, fork/gather, richer debugging, and broader evaluation. The prototype
 demonstrates the architecture; the thesis contribution is the platform design
 and evidence that the design can work across multiple source families under
-controlled conditions.
+controlled conditions. The implemented contribution is therefore the durable,
+typed workflow substrate required by an agent-facing automation system; the
+autonomous planning layer remains outside the submitted implementation
+boundary.
 
 <!-- References -->
 # References {#sec:refs .unnumbered}
@@ -1940,34 +1980,62 @@ Evidence:
 
 # Agent Challenge Harness
 
-The browser-click challenge harness is an evaluation instrument for the CLI
+## Shared Challenge Protocol
-surface intended for external-agent operation. It asks an external agent to
-build and successfully run a workflow that opens a local page with a visible
-button. The workflow records a before-click snapshot, performs or waits for a
-click, records an after-click snapshot, and returns both snapshots from a
-deployed workflow run.
 
-The harness deliberately evaluates the product-facing lifecycle rather than
+The agent-challenge harness is an evaluation instrument for the CLI surface
-general Python programmability. A valid solution uses `uv run wf ...` commands
+intended for external-agent operation. It deliberately evaluates the
-for artifact creation, deployment saving, and run execution. Importing
+product-facing lifecycle rather than general Python programmability. A valid
-`WorkflowApi`, building `WorkflowServer` directly, calling source functions
+solution uses `uv run wf ...` commands for artifact creation, deployment saving,
-directly, or solving the task as a standalone browser script is treated as a
+and run execution. Importing `WorkflowApi`, building `WorkflowServer` directly,
-bypass even if the visible output is correct.
+calling source functions directly, or solving the task as a standalone script is
+treated as a bypass even if the visible output is correct.
 
-The browser-click challenge accepts two product-facing authoring paths:
+Both checked challenges accept two product-facing authoring paths:
 
 1. **Draft path.** Create a draft from one capability, apply focused draft edits
    or an RFC 6902 patch, validate, save, deploy, and run.
-2. **Raw-plan path.** Write a raw workflow plan and load it with
+2. **Raw-plan path.** Write a `RawWorkflowPlan` and load it with
    `wf artifact create-from-plan`, then deploy and run.
 
-The challenge report is a YAML self-report with fields for product-path use,
+The challenge report is an inline YAML self-report with fields for product-path
-helper-script use, workflow file, deployment id, run id, before/after booleans,
+use, helper-script use, workflow file, deployment id, run id, read-behavior
-read-behavior flags, attempt counts, missed requirements, and notes. The harness
+flags, attempt counts, missed requirements, and challenge-specific assertions.
-uses that block for automatic convenience classification, but the official
+The harness uses that block for automatic convenience classification, but the
-outcome is manually reviewed. Manual review checks the command transcript, the
+official outcome is manually reviewed.
-workflow file, the run id, the run output or trace, and whether the agent read
+
-product source code, adjacent attempts, prior stores, or existing solutions.
+## Browser-Click Challenge
+
+The browser-click challenge asks an external agent to build and successfully run
+a workflow that opens a local page with a visible button. The workflow records a
+before-click snapshot, performs or waits for a click, records an after-click
+snapshot, and returns both snapshots from a deployed workflow run. Its success
+contract requires `before_clicked: false`, `after_clicked: true`, no failed run,
+and no leftover browser or HTTP-server process.
+
+## Report-Workflow Challenge
+
+The report-workflow challenge asks an external agent to build and successfully
+run a three-step workflow over a local Python source: `read_notes`,
+`extract_report`, and `render_markdown_report`. Its success contract requires a
+deployed workflow run, a title matching the expected report title, rendered
+Markdown output, and no helper-script or direct-API bypass.
+
+## Manual Audit Rubric
+
+Manual review checks the command transcript, the workflow file, the deployment
+id, the run id, the run output or trace, and whether the agent read product
+source code, adjacent attempts, prior stores, or existing solutions. The
+decision precedence is:
+
+| Condition | Official outcome | Reason |
+| --- | --- | --- |
+| Product path completed and the audit trail has no disqualifying reads or bypasses | Pass | Supports product-surface operability |
+| Product path completed but the agent used a disqualifying source, prior artifact, adjacent attempt, or hidden answer | Invalid | The output exists but cannot support clean evaluation |
+| No product-path artifact, deployment, and run evidence | Fail | The task contract was not established |
+| Product path exists, but a helper script or direct API bypass materially contributed | Invalid | Output exists, but the trial is contaminated |
+| No product-path artifact, deployment, and run evidence; task solved through a helper script or direct API | Fail | The product-facing challenge contract was not established |
+
+: Manual audit decision rules for agent-challenge trials. {#tbl:agent-challenge-audit-rubric}
 
 This distinction is intentional. Agent benchmark literature and practice show
 that automated scores and self-reports can be misleading when an agent can
@@ -1977,12 +2045,14 @@ records possible invalidation flags such as helper-script bypass,
 adjacent-attempt leakage, prior-store reuse, product-code dependency, false YAML
 claims, timeouts, parse failures, and missing run evidence.
 
+## Cohort Manifest And Reproducibility
+
 The harness, both challenge workflows, and the 36-trial checked cohort are
-implemented and manually audited. The Evaluation
+implemented and manually audited. The Evaluation chapter reports official
-chapter reports official outcomes, automatic/manual disagreement, duration,
+outcomes, automatic/manual disagreement, duration, and recorded token totals.
-and recorded token totals. It does not claim controlled model superiority,
+It does not claim controlled model superiority, normalized throughput, or retry
-normalized throughput, or retry reduction because the product, prompts, and
+reduction because the product, prompts, and hosted service conditions were not
-hosted service conditions were not held constant across waves.
+held constant across waves.
 
 ```{=latex}
 \clearpage

docs/thesis/title-pages.tex

@@ -0,0 +1,69 @@
+\usepackage{tikz}
+\usetikzlibrary{calc}
+
+\makeatletter
+\renewcommand{\maketitle}{
+  \begin{titlepage}
+  \thispagestyle{empty}
+  % Draw the border against the physical page, not the content box, so the
+  % frame can be larger without squeezing the title-page layout.
+  \begin{tikzpicture}[remember picture, overlay]
+    \draw[line width=1.1pt]
+      ($(current page.north west) + (1.7cm,-1.7cm)$)
+      rectangle
+      ($(current page.south east) + (-1.7cm,1.7cm)$);
+  \end{tikzpicture}
+  \noindent\makebox[\textwidth][c]{%
+    \begin{minipage}[c][0.88\textheight][c]{0.86\textwidth}
+    \centering
+    {\Large UNIVERSITY OF SCIENCE AND TECHNOLOGY OF HANOI\par}
+    \vspace{0.35cm}
+    {\large\bfseries DEPARTMENT OF INFORMATION AND COMMUNICATION TECHNOLOGY\par}
+    \vspace{0.9cm}
+    \IfFileExists{docs/thesis/assets/usth-logo.png}{%
+      \includegraphics[width=0.32\textwidth]{docs/thesis/assets/usth-logo.png}\par
+    }{%
+      \includegraphics[width=0.32\textwidth]{assets/usth-logo.png}\par
+    }
+    \vspace{1.0cm}
+    {\Large\bfseries BACHELOR THESIS\par}
+    \vspace{0.9cm}
+    {\large By\par}
+    \vspace{0.2cm}
+    {\large\bfseries Lương Đức Anh\par}
+    \vspace{0.9cm}
+    {\large Title:\par}
+    \vspace{0.2cm}
+    {\Large\bfseries Design and Implementation of lda.chat:\par}
+    \vspace{0.18cm}
+    {\Large\bfseries An AI Agent for Automating and Creating Workspace Workflows\par}
+    \vspace{0.45cm}
+    {\small\itshape Implementation boundary: this thesis evaluates the workflow substrate used by external agents, not a bundled autonomous agent brain.\par}
+    \vfill
+    \begin{tabular}{@{}r@{\hspace{0.5em}}l@{}}
+    External Supervisor: & \textbf{Eng. Trần Văn Trường}\\
+    Internal Supervisor: & \textbf{Dr. Nghiêm Thị Phương}
+    \end{tabular}
+    \vfill
+    {\large Hanoi, \@date\par}
+    \end{minipage}%
+  }
+  \end{titlepage}
+  \clearpage
+  \thispagestyle{empty}
+  \section*{Supervisor Certification}
+  To whom it may concern,
+
+  I, \textbf{Eng. Trần Văn Trường}, certify that the bachelor thesis of
+  \textbf{Lương Đức Anh} is qualified to be presented to the thesis jury.
+
+  \vspace{1.5cm}
+  Hanoi, \@date
+
+  \vspace{2.0cm}
+  \begin{flushright}
+  Supervisor's signature
+  \end{flushright}
+  \clearpage
+}
+\makeatother

examples/agent_challenges/evaluation.py

@@ -13,7 +13,8 @@ FIGURE_STEMS = (
     "agent-challenge-audited-outcomes-by-cell",
     "agent-challenge-automatic-vs-manual-outcomes",
     "agent-challenge-longitudinal-outcomes",
-    "agent-challenge-duration-and-tokens",
+    "agent-challenge-duration",
+    "agent-challenge-token-volume",
 )
 _MANUAL_OUTCOMES = frozenset({"pass", "invalid", "fail"})
 _CHALLENGE_ORDER = {"browser": 0, "report": 1}
@@ -210,17 +211,20 @@ def render_evaluation_markdown(cohort: EvaluationCohort) -> str:
         "## Audited Agent Challenge Campaign",
         "",
         (
-            f"The primary campaign contains {len(cohort.trials)} audited trials: "
+            f"The primary campaign contains {len(cohort.trials)} manually audited "
-            f"{outcomes['pass']} passes, {outcomes['invalid']} invalid samples, "
+            f"trials: {outcomes['pass']} clean product-path passes under the "
-            f"and {outcomes['fail']} failure."
+            f"campaign rules, {outcomes['invalid']} invalid evaluation samples, "
+            f"and {outcomes['fail']} failure. These counts are not a "
+            f"model-success-rate estimate."
         ),
         "",
         (
-            "The campaign crosses two challenges, two hosted models, three instruction "
+            "The campaign crosses two challenges × two hosted models × three "
-            "profiles (`none`, `skills`, and `all`), and three repetitions per cell. "
+            "instruction profiles (`none`, `skills`, and `all`) = 12 cells, with "
-            "The checked cohort snapshot records report hashes, prompt hashes, the "
+            "three repetitions per cell (n=3). The checked cohort snapshot records "
-            "repository commit, automatic metrics, and manual-audit outcomes; local "
+            "report hashes, prompt hashes, the repository commit, automatic metrics, "
-            "raw report files are verified against those hashes when present."
+            "and manual-audit outcomes; local raw report files are verified against "
+            "those hashes when present."
         ),
         "",
         (
@@ -228,6 +232,10 @@ def render_evaluation_markdown(cohort: EvaluationCohort) -> str:
             "this is longitudinal engineering evidence, not a controlled model comparison."
         ),
         "",
+        "> **Campaign validity note.** This campaign is a bounded longitudinal audit, "
+        "not a controlled comparison. Each cell has n=3; waves changed product and "
+        "prompt snapshots; all audits were performed by the author.",
+        "",
         f"Selection rule: {cohort.selection_rule}",
         "",
         "| Challenge / model / profile | Pass | Invalid | Fail |",
@@ -244,10 +252,12 @@ def render_evaluation_markdown(cohort: EvaluationCohort) -> str:
             "",
             (
                 "A manual `pass` requires both successful product-path evidence and an "
-                "acceptable audit trail. `Invalid` means the sample cannot support the "
+                "acceptable audit trail. It does not imply the agent avoided every "
-                "clean benchmark claim, commonly because the agent read repository or "
+                "exploratory read, only that no disqualifying read or bypass was found. "
-                "example material outside its supplied workspace. `Fail` means the "
+                "`Invalid` means the sample cannot support the clean benchmark claim, "
-                "challenge contract itself was not established."
+                "commonly because the agent read repository or example material outside "
+                "its supplied workspace. `Fail` means the challenge contract itself was "
+                "not established."
             ),
             "",
             "![Audited outcomes by evaluation cell.](figures/agent-challenge-audited-outcomes-by-cell.svg){#fig:agent-challenge-audited-outcomes-by-cell width=95%}",
@@ -255,7 +265,10 @@ def render_evaluation_markdown(cohort: EvaluationCohort) -> str:
             (
                 "[@fig:agent-challenge-audited-outcomes-by-cell] reports all three "
                 "repetitions rather than hiding invalid samples. The profile labels are "
-                "descriptive; this campaign does not isolate instruction-profile effects."
+                "descriptive; this campaign does not isolate instruction-profile effects. "
+                "Profile × wave is confounded because the base prompt changed before "
+                "wave 3, so apparent differences may reflect prompt changes, model "
+                "updates, or repository drift rather than instruction-layer effects."
             ),
             "",
             "![Automatic task outcomes compared with manual outcomes.](figures/agent-challenge-automatic-vs-manual-outcomes.svg){#fig:agent-challenge-automatic-vs-manual-outcomes width=75%}",
@@ -275,14 +288,21 @@ def render_evaluation_markdown(cohort: EvaluationCohort) -> str:
                 "changed. They preserve the chronology needed to study those changes."
             ),
             "",
-            "![Duration and recorded token totals grouped by challenge, instruction profile, model, and wave.](figures/agent-challenge-duration-and-tokens.svg){#fig:agent-challenge-duration-and-tokens width=95%}",
+            "![Wall-clock duration grouped by challenge, instruction profile, model, and wave.](figures/agent-challenge-duration.svg){#fig:agent-challenge-duration width=78%}",
             "",
             (
-                "[@fig:agent-challenge-duration-and-tokens] separates each challenge and "
+                "[@fig:agent-challenge-duration] separates the two challenges. Circle "
-                "metric into its own panel. Circle and square markers redundantly identify "
+                "and square markers redundantly identify the models without relying on "
-                "the models without relying on color. Wall-clock duration includes hosted-service "
+                "color. Wall-clock duration includes hosted-service latency and is not a "
-                "latency, and OpenCode token totals include cache-read accounting, so neither "
+                "normalized model-efficiency metric."
-                "axis is a normalized model-efficiency metric."
+            ),
+            "",
+            "![Recorded token totals grouped by challenge, instruction profile, model, and wave.](figures/agent-challenge-token-volume.svg){#fig:agent-challenge-token-volume width=78%}",
+            "",
+            (
+                "[@fig:agent-challenge-token-volume] reports OpenCode token totals, which "
+                "include cache-read accounting. The figure records observed workload volume; "
+                "it is not an efficiency comparison."
             ),
             "",
             "### Campaign Limitations",

examples/agent_challenges/evaluation_figures.py

@@ -312,23 +312,24 @@ def _scatter_metric(
     axis.grid(axis="y")
 
 
-def _duration_and_tokens(cohort: EvaluationCohort, plt: Any) -> Figure:
+def _metric_by_challenge(
+    cohort: EvaluationCohort,
+    plt: Any,
+    *,
+    metric: str,
+) -> Figure:
+    """Render one readable metric panel per challenge."""
     from matplotlib.lines import Line2D
 
-    figure, axes = plt.subplots(2, 2, figsize=(9.6, 6.8), sharex=True)
+    figure, axes = plt.subplots(2, 1, figsize=(7.4, 6.8), sharex=True)
     challenges = (("browser", "Browser click"), ("report", "Report workflow"))
-    for row, (challenge, challenge_label) in enumerate(challenges):
+    for axis, (challenge, challenge_label) in zip(axes, challenges, strict=True):
         trials = [trial for trial in cohort.trials if trial.challenge == challenge]
-        duration_axis, token_axis = axes[row]
+        _scatter_metric(axis, trials, metric=metric)
-        _scatter_metric(duration_axis, trials, metric="duration")
+        unit = "Minutes" if metric == "duration" else "Million tokens"
-        _scatter_metric(token_axis, trials, metric="tokens")
+        axis.set_ylabel(f"{challenge_label}\n{unit}")
-        duration_axis.set_ylabel(f"{challenge_label}\nMinutes")
-        token_axis.set_ylabel(f"{challenge_label}\nMillion tokens")
 
-    axes[0, 0].set_title("Wall-clock duration")
+    axes[-1].set_xlabel("Instruction profile")
-    axes[0, 1].set_title("Recorded token volume")
-    axes[1, 0].set_xlabel("Instruction profile")
-    axes[1, 1].set_xlabel("Instruction profile")
     legend_handles = [
         Line2D(
             [],
@@ -342,8 +343,13 @@ def _duration_and_tokens(cohort: EvaluationCohort, plt: Any) -> Figure:
         )
         for model, style in _MODEL_STYLES.items()
     ]
+    title = (
+        "Wall-clock duration by profile, model, and wave"
+        if metric == "duration"
+        else "Recorded token volume by profile, model, and wave"
+    )
     figure.suptitle(
-        "Runtime evidence by challenge, profile, model, and wave",
+        title,
         fontsize=13,
         fontweight="bold",
     )
@@ -357,7 +363,11 @@ def _duration_and_tokens(cohort: EvaluationCohort, plt: Any) -> Figure:
     figure.text(
         0.5,
         0.015,
-        "Point labels 1–3 identify waves; token totals include OpenCode cache-read accounting.",
+        (
+            "Point labels 1–3 identify waves."
+            if metric == "duration"
+            else "Point labels 1–3 identify waves; totals include OpenCode cache-read accounting."
+        ),
         ha="center",
         color="#4C5961",
         fontsize=8,
@@ -378,7 +388,14 @@ def render_evaluation_figures(
             _automatic_vs_manual(cohort, plt),
         ),
         ("agent-challenge-longitudinal-outcomes", _longitudinal_outcomes(cohort, plt)),
-        ("agent-challenge-duration-and-tokens", _duration_and_tokens(cohort, plt)),
+        (
+            "agent-challenge-duration",
+            _metric_by_challenge(cohort, plt, metric="duration"),
+        ),
+        (
+            "agent-challenge-token-volume",
+            _metric_by_challenge(cohort, plt, metric="tokens"),
+        ),
     )
     written: list[Path] = []
     for stem, figure in figures:

tests/docs/test_big_doc_links.py

@@ -76,7 +76,8 @@ def test_thesis_bundle_has_reproducible_agent_evaluation_assets() -> None:
         "agent-challenge-audited-outcomes-by-cell",
         "agent-challenge-automatic-vs-manual-outcomes",
         "agent-challenge-longitudinal-outcomes",
-        "agent-challenge-duration-and-tokens",
+        "agent-challenge-duration",
+        "agent-challenge-token-volume",
     )
 
     assert (thesis / "agent-challenge-cohort.json").is_file()

tests/examples/test_agent_challenge_evaluation.py

@@ -105,8 +105,11 @@ def test_evaluation_markdown_states_counts_and_longitudinal_limit() -> None:
     cohort = load_evaluation_cohort(COHORT_PATH, repository_root=ROOT)
     markdown = render_evaluation_markdown(cohort)
 
-    assert "36 audited trials" in markdown
+    assert "36 manually audited trials" in markdown
-    assert "27 passes, 8 invalid samples, and 1 failure" in markdown
+    assert "27 clean product-path passes under the campaign rules" in markdown
+    assert "not a model-success-rate estimate" in markdown
+    assert "8 invalid evaluation samples" in markdown
+    assert "and 1 failure" in markdown
     assert "not a controlled model comparison" in markdown
     assert "agent-challenge-audited-outcomes-by-cell.svg" in markdown
     assert "agent-challenge-longitudinal-outcomes.svg" in markdown