lda-wf/docs/historical/superpowers/plans/2026-06-14-thesis-system-design-doc.md

Historical: This plan has been completed. The Big Doc lives at docs/add/system-design-implementation.md.

Thesis System Design Document Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Produce a formal thesis-style system design and implementation document grounded in the current codebase, tests, diagrams, and reproducible evidence.

Architecture: The document should lead with concepts, then support them with verified implementation facts. Claims must be backed by code paths, tests, smoke output, or the runnable case-study bundle; do not invent product properties from aspirational roadmap text.

Tech Stack: Markdown with Pandoc-compatible frontmatter, Mermaid diagrams, PowerShell generation script in docs/add/generate.ps1, pytest docs smoke tests, ruff, basedpyright.

---

Dependency

This plan should run after the completed case-study evidence plan archived at docs/historical/superpowers/plans/2026-06-14-thesis-case-study-evidence.md or after an equivalent report-workflow example exists at examples/report_workflow/.

If examples/report_workflow/README.md does not exist, stop and implement the evidence plan first.

Files

• Create: docs/add/system-design-implementation.md — the formal thesis/system-design document.
• Create: docs/add/evidence-index.md — concise map from thesis claims to code/tests/docs evidence.
• Modify: docs/add/diagrams.md — add stable Mermaid diagrams used by the Big Doc.
• Modify: docs/add/thesis-outline.md — mark which sections have been transferred to the Big Doc.
• Modify: docs/project_map.md — link the Big Doc and evidence index from the docs map.
• Modify: docs/current_roadmap.md — mark the Big Doc draft completed.
• Modify or create: tests/docs/test_big_doc_links.py — smoke-test links to key evidence files.

Claim Verification Rule

Every factual claim in docs/add/system-design-implementation.md must be traceable to one of:

• a source file under src/
• a test under tests/
• a runnable example under examples/report_workflow/
• an existing live architecture doc such as docs/source_architecture.md
• a smoke/runbook file that states it is current

Use these commands before drafting implementation claims:

rg 'class WorkflowArtifact|class WorkflowDeployment|class WorkflowRun' src/wf_artifacts -n
rg 'class WorkflowApi|WorkflowServer|WorkflowSourceProvider|CapabilitySource' src -n
rg 'McpRuntimePool|StatefulMcpRuntime|McpSourceClient' src/wf_sources_mcp src/wf_mcp -n
rg 'PythonSourceConfig|wf_sources_python|load_sources' src tests -n
rg 'next_actions|diagnostics|validate_deployment|validate_draft' src/wf_api src/wf_cli tests -n

Do not cite old plans in docs/historical/** as current behavior unless the text explicitly says the citation is historical motivation.

Task 1: Build an Evidence Index

Files:

• Create: docs/add/evidence-index.md

• Step 1: Create the evidence index

Create docs/add/evidence-index.md with this structure:

# Thesis Evidence Index

This file maps thesis claims to implementation evidence. It is not prose for the
final report; it is a guardrail against unsupported claims.

## Core Workflow Lifecycle

Claim: The platform separates mutable drafts, immutable artifacts, deployments,
runs, and traces.

Evidence:

- `src/wf_artifacts/models.py` — artifact/deployment models.
- `src/wf_artifacts/runs/` — run records and run store.
- `src/wf_api/service.py` — facade for workflow lifecycle operations.
- `tests/wf_api/test_artifact_api.py`
- `tests/wf_api/test_run_api.py`

## Source Provider Boundary

Claim: Workflow execution consumes source-provided capabilities without making
the core runtime MCP-specific.

Evidence:

- `src/wf_platform/sources.py` — neutral source DTOs and source policy.
- `src/wf_server/config.py` — server composition for configured sources.
- `src/wf_sources_mcp/` — MCP source family.
- `src/wf_sources_python/` — Python source family.
- `docs/source_architecture.md`

## Agent-Operable Surface

Claim: External agents can operate the workflow lifecycle through stable CLI/API
surfaces.

Evidence:

- `src/wf_cli/`
- `src/wf_transport_rpc_http/`
- `tests/wf_cli/`
- `tests/wf_transport_rpc_http/`
- `docs/wf_cli.md`

## Validation And Diagnostics

Claim: Validation and diagnostics make failed workflow states repairable.

Evidence:

- `src/wf_artifacts/validation.py`
- `src/wf_api/next_actions.py`
- `src/wf_api/source_admin.py`
- `tests/artifacts/test_validation.py`
- `tests/wf_api/test_source_admin_api.py`

## Stateful MCP Source Correctness

Claim: MCP-backed sources can preserve stateful sessions across workflow calls.

Evidence:

- `src/wf_sources_mcp/runtime/`
- `src/wf_sources_mcp/client/`
- `tests/wf_sources_mcp/test_runtime.py`
- `tests/wf_transport_rpc_http/test_mcp_backed_server_rpc.py`

## Python Source Case Study

Claim: The source-provider model is not MCP-only.

Evidence:

- `examples/report_workflow/`
- `src/wf_sources_python/`
- `tests/examples/test_report_workflow_example.py`
- `tests/wf_sources_python/test_loader.py`

## Limitations

Claim: This is a prototype platform substrate, not a finished automation product.

Evidence:

- `docs/add/thesis-outline.md`
- `docs/current_roadmap.md`
- absence of scheduler/visual-editor/secret-manager production packages in
  current source tree.

• Step 2: Verify evidence paths exist

Run:

Test-Path docs/add/evidence-index.md
Test-Path src/wf_artifacts/models.py
Test-Path src/wf_platform/sources.py
Test-Path src/wf_sources_mcp/runtime
Test-Path src/wf_sources_python
Test-Path examples/report_workflow

Expected: all commands print True. If examples/report_workflow prints False, stop and implement the case-study evidence plan first.

• Step 3: Commit evidence index

Run:

git add docs/add/evidence-index.md
git commit -m "docs: add thesis evidence index"

Expected: commit succeeds.

Task 2: Add Stable Mermaid Diagrams

Files:

• Modify: docs/add/diagrams.md

• Step 1: Add workflow core diagram

Append this section to docs/add/diagrams.md:

## Workflow Core

```mermaid
flowchart LR
  Input[Input Schema] --> NodeUse[Node Use]
  NodeSpec[NodeSpec Contract] --> NodeUse
  NodeUse --> Outcome[Declared Outcome]
  Outcome --> Edge[Graph Edge]
  NodeUse --> StateWrite[State Write]
  StateWrite --> Reducer[Reducer]
  Reducer --> State[Workflow State]
  NodeUse --> Trace[Trace Frame]
  Interrupt[Interrupt] --> RunState[Stopped Run State]
  RunState --> Resume[Resume]
```

• Step 2: Add platform domain diagram

Append this section to docs/add/diagrams.md:

## Platform Domain

```mermaid
flowchart LR
  Draft[Draft Workspace] --> DraftValidation[Draft Validation]
  DraftValidation --> Artifact[Workflow Artifact]
  Artifact --> Deployment[Workflow Deployment]
  SourceInventory[Source Inventory] --> DeploymentValidation[Deployment Validation]
  Deployment --> DeploymentValidation
  DeploymentValidation --> Run[Workflow Run]
  Run --> RunRecord[Run Record]
  Run --> Trace[Trace Slice]
  RunRecord --> Resume[Resume]
  DeploymentValidation --> Diagnostics[Repairable Diagnostics]
```

• Step 3: Add source provider diagram

Append this section to docs/add/diagrams.md:

## Source Provider Boundary

```mermaid
flowchart LR
  Config[Workflow Config Sources] --> Server[WorkflowServer Composition]
  Server --> Builtin[Platform Sources]
  Server --> MCP[MCP Source Provider]
  Server --> Python[Python Source Provider]
  Builtin --> Inventory[CapabilitySource Inventory]
  MCP --> Inventory
  Python --> Inventory
  Inventory --> API[Workflow API Surface]
  API --> Runtime[Workflow Runtime]
```

• Step 4: Commit diagrams

Run:

git add docs/add/diagrams.md
git commit -m "docs: add thesis diagrams"

Expected: commit succeeds.

Task 3: Draft the Formal Big Doc

Files:

• Create: docs/add/system-design-implementation.md

• Step 1: Create frontmatter and introduction

Create docs/add/system-design-implementation.md with this frontmatter and opening:

---
title: "Design and Implementation of lda.chat"
subtitle: "Infrastructure for AI Agents to Author and Execute Workspace Workflows"
author: "draft"
date: "2026-06-14"
lang: "en-US"
documentclass: report
papersize: a4
fontsize: 10pt
toc: true
toc-depth: 2
numbersections: true
geometry:
  - top=30mm
  - bottom=30mm
  - left=32mm
  - right=32mm
mainfont: "Libertinus Serif"
sansfont: "Libertinus Sans"
monofont: "Libertinus Mono"
mathfont: "Libertinus Math"
colorlinks: true
linkcolor: "MidnightBlue"
urlcolor: "MidnightBlue"
toccolor: "MidnightBlue"
keywords:
  - workflow
  - agents
  - source providers
  - JSON-RPC
  - MCP
  - Python sources
header-includes:
  - \usepackage{graphicx}
  - \usepackage{booktabs}
  - \usepackage{hyperref}
  - \usepackage{hyperxmp}
  - \usepackage[dvipsnames]{xcolor}
  - \usepackage{fancyhdr}
  - \pagestyle{fancy}
  - \fancyhead[L]{\small lda.chat}
  - \fancyhead[R]{\small\leftmark}
  - \fancyfoot[C]{\thepage}
  - \setlength{\parskip}{0.6em}
  - \setlength{\parindent}{0pt}
  - \setkeys{Gin}{width=\linewidth,height=0.55\textheight,keepaspectratio}
  - \renewcommand{\arraystretch}{1.3}
  - \hypersetup{pdfauthor={lda.chat}, pdftitle={Design and Implementation of lda.chat}}
---

# Introduction

External LLM agents are useful workflow authors and operators, but durable
workspace automation needs a typed execution substrate. This report describes
the design and implementation of `lda.chat`, a prototype platform where agents
can author, validate, execute, and inspect reusable workspace workflows without
making the LLM itself responsible for runtime state, validation, source binding,
or persistence.

The central claim is that agent-facing workflow automation should separate
planning from execution. The LLM or human author can propose and revise workflow
structure, while the platform owns artifacts, deployments, runs, source
inventory, validation diagnostics, traces, and resumability.

• Step 2: Add the required section skeleton

Append these headings to docs/add/system-design-implementation.md:

# Problem Statement And Requirements

# Positioning And Related Systems

# Conceptual Model

# System Architecture

# Implementation

# Case Study: Deterministic Report Workflow

# Evaluation

# Limitations

# Future Work

# Conclusion

# Appendices

• Step 3: Fill sections from the outline

Use docs/add/thesis-outline.md as the source for section content. Copy ideas, not whole notes. Required rules:

• Keep package names secondary to concepts.

• Include the main architecture Mermaid diagram in # System Architecture.

• Include workflow lifecycle and source provider diagrams where they explain the text.

• Include the report-workflow example in # Case Study.

• Keep long command transcripts in # Appendices.

• Include no claim about production secret management, production reliability, broad user study, full MCP frontend, MCP widget proxying, visual editor, or scheduler.

• Step 4: Insert claim references

Add short parenthetical evidence references in prose, using this style:

(Evidence: `src/wf_artifacts/models.py`, `tests/wf_api/test_run_api.py`.)

Minimum required evidence references:

• workflow lifecycle models/tests

• source provider boundary

• JSON-RPC/CLI surface

• Python source case study

• MCP stateful runtime tests

• validation/diagnostics tests

• Step 5: Commit Big Doc draft

Run:

git add docs/add/system-design-implementation.md
git commit -m "docs: draft thesis system design document"

Expected: commit succeeds.

Task 4: Add Link Tests And Documentation Map Entries

Files:

• Create: tests/docs/test_big_doc_links.py

• Modify: docs/project_map.md

• Modify: docs/current_roadmap.md

• Step 1: Add docs link test

Create tests/docs/test_big_doc_links.py with this test module:

from __future__ import annotations

from pathlib import Path


ROOT = Path(__file__).resolve().parents[2]


def test_big_doc_links_case_study_and_evidence_index() -> None:
    doc = (ROOT / "docs" / "add" / "system-design-implementation.md").read_text(
        encoding="utf-8"
    )

    assert "examples/report_workflow" in doc
    assert "docs/add/evidence-index.md" in doc or "evidence-index.md" in doc


def test_project_map_links_big_doc() -> None:
    project_map = (ROOT / "docs" / "project_map.md").read_text(encoding="utf-8")

    assert "system-design-implementation.md" in project_map
    assert "evidence-index.md" in project_map


def test_big_doc_keeps_mcp_as_source_family() -> None:
    doc = (ROOT / "docs" / "add" / "system-design-implementation.md").read_text(
        encoding="utf-8"
    )

    assert "MCP" in doc
    assert "source family" in doc
    assert "product identity" in doc

• Step 2: Update project map

In docs/project_map.md, add a docs/add entry:

- `docs/add/system-design-implementation.md` — formal thesis/system-design draft.
- `docs/add/evidence-index.md` — claim-to-evidence map for the thesis draft.

Place it near existing docs/add or architecture document entries.

• Step 3: Update roadmap

In docs/current_roadmap.md, add:

- Completed thesis system-design draft: `docs/add/system-design-implementation.md`
  now frames the platform as a formal system design/implementation report backed
  by `docs/add/evidence-index.md` and the report-workflow case study.

• Step 4: Run docs tests

Run:

uv run pytest tests/docs/test_big_doc_links.py -q

Expected: 3 passed.

• Step 5: Commit docs links

Run:

git add tests/docs/test_big_doc_links.py docs/project_map.md docs/current_roadmap.md
git commit -m "docs: link thesis system design draft"

Expected: commit succeeds.

Task 5: Generate HTML/PDF Smoke Outputs

Files:

• Read: docs/add/generate.ps1

• Generate locally: docs/add/system-design-implementation.html

• Generate locally: docs/add/system-design-implementation.pdf

• Step 1: Generate HTML

Run from docs/add:

.\generate.ps1 -type html -- system-design-implementation.md -o system-design-implementation.html

Expected: command exits 0 and creates docs/add/system-design-implementation.html.

• Step 2: Generate PDF

Run from docs/add:

.\generate.ps1 -type pdf -- system-design-implementation.md -o system-design-implementation.pdf

Expected: command exits 0 and creates docs/add/system-design-implementation.pdf.

• Step 3: Decide whether generated files are tracked

Check .gitignore and current tracking:

git ls-files docs/add/*.pdf docs/add/*.html
git status --short docs/add

If generated HTML/PDF are already tracked in this folder, add them. If not tracked or ignored, leave them uncommitted and mention generation success in the report.

• Step 4: Commit generated outputs only if tracked

If tracked outputs changed, run:

git add docs/add/system-design-implementation.html docs/add/system-design-implementation.pdf
git commit -m "docs: generate thesis system design outputs"

If files are ignored/untracked, do not commit this task.

Task 6: Final Verification And Archive Plan

Files:

• Verify all changed files.

• Move this plan to historical after completion.

• Step 1: Run docs tests

Run:

uv run pytest tests/docs tests/examples -q

Expected: all selected docs/example tests pass.

• Step 2: Run ruff on tests

Run:

uv run ruff check tests/docs tests/examples

Expected: All checks passed!

• Step 3: Run typecheck on tests

Run:

uv run basedpyright --level error tests/docs tests/examples

Expected: 0 errors.

• Step 4: Run whitespace check

Run:

git diff --check

Expected: no whitespace errors. CRLF warnings on Windows are acceptable.

• Step 5: Archive this plan

Move this file to:

docs/historical/superpowers/plans/2026-06-14-thesis-system-design-doc.md

Commit:

git add docs/superpowers/plans/2026-06-14-thesis-system-design-doc.md docs/historical/superpowers/plans/2026-06-14-thesis-system-design-doc.md
git commit -m "docs: archive thesis system design plan"

Expected: commit succeeds.

Self-Review Checklist

• The Big Doc is thesis/system-design-first, not a presentation deck.
• The Big Doc cites code/tests/examples for implementation claims.
• MCP is described as a source family, not product identity.
• Google Drive MCP is not required for thesis-critical evidence.
• Diagrams appear before package-heavy implementation detail.
• Limitations are explicit and not hidden in future work.
• Long command transcripts are in appendices or linked runbooks.
• No current docs introduce wf.std=wf.std deployment self-bindings.