docs: add thesis report workflow evidence

This commit is contained in:
lda
2026-06-14 17:16:40 +07:00 Verified
parent 03762b1aa9
commit 1aa40285b1
10 changed files with 328 additions and 3 deletions
+3
View File
@@ -59,6 +59,9 @@ The case study should exist as a runnable example, not only prose. Target shape:
`examples/report_workflow/ops.py`, `input.md`, `wf.config.json`, a short `examples/report_workflow/ops.py`, `input.md`, `wf.config.json`, a short
`README.md`, and commands for config validation, server startup, capability `README.md`, and commands for config validation, server startup, capability
calls, draft/artifact/deployment creation, run, inspect, and trace. calls, draft/artifact/deployment creation, run, inspect, and trace.
The runnable evidence bundle for this case study lives at
`examples/report_workflow/README.md`.
Keep the thesis-critical path deterministic. Do not require an LLM call inside Keep the thesis-critical path deterministic. Do not require an LLM call inside
the case-study workflow. LLM nodes can be discussed as future work or an the case-study workflow. LLM nodes can be discussed as future work or an
+3
View File
@@ -180,6 +180,9 @@ stable.
- `wf run watch` provides polling-based progress UX. - `wf run watch` provides polling-based progress UX.
- CLI expected errors are compact by default; `wf --verbose ...` preserves raw - CLI expected errors are compact by default; `wf --verbose ...` preserves raw
tracebacks for debugging. tracebacks for debugging.
- Completed thesis case-study evidence bundle: `examples/report_workflow/`
provides a deterministic report workflow with Python source, fixture input,
config, runbook, and tests.
## Historical References ## Historical References
@@ -243,7 +243,7 @@ Create `examples/report_workflow/wf.config.json` with this exact JSON:
Run from repo root: Run from repo root:
```powershell ```powershell
uv run wf --config examples/report_workflow/wf.config.json config validate uv run wf config validate examples/report_workflow/wf.config.json
``` ```
Expected: command exits `0` and reports a valid config. If this command name differs in current code, inspect `uv run wf config --help` and update the runbook to the real command. Expected: command exits `0` and reports a valid config. If this command name differs in current code, inspect `uv run wf config --help` and update the runbook to the real command.
@@ -450,7 +450,7 @@ remote OAuth, LLM calls, or provider quota.
From the repository root: From the repository root:
```powershell ```powershell
uv run wf --config examples/report_workflow/wf.config.json config validate uv run wf config validate examples/report_workflow/wf.config.json
uv run wf-rpc-server --config examples/report_workflow/wf.config.json uv run wf-rpc-server --config examples/report_workflow/wf.config.json
``` ```
@@ -12,7 +12,9 @@
## Dependency ## Dependency
This plan should run after `docs/superpowers/plans/2026-06-14-thesis-case-study-evidence.md` or after an equivalent report-workflow example exists at `examples/report_workflow/`. This plan should run after the completed case-study evidence plan archived at
`docs/historical/superpowers/plans/2026-06-14-thesis-case-study-evidence.md` or
after an equivalent report-workflow example exists at `examples/report_workflow/`.
If `examples/report_workflow/README.md` does not exist, stop and implement the evidence plan first. If `examples/report_workflow/README.md` does not exist, stop and implement the evidence plan first.
+66
View File
@@ -0,0 +1,66 @@
# Report Workflow Example
This example is the deterministic thesis case study. It demonstrates a trusted
Python source that turns project notes into a typed report object without using
remote OAuth, LLM calls, or provider quota.
## Files
- `input.md` — fixture notes.
- `cap-input.json` — capability-call payload generated from the fixture notes.
- `run-input.json` — workflow-run payload generated from the fixture notes.
- `ops.py` — Python source exposing `read_notes`, `extract_report`, and
`render_markdown_report`.
- `wf.config.json` — local server/client config using the `local.report` Python
source.
## Run
From the repository root:
```powershell
uv run wf config validate examples/report_workflow/wf.config.json
uv run wf-rpc-server --config examples/report_workflow/wf.config.json
```
In another terminal:
```powershell
uv run wf --config examples/report_workflow/wf.config.json status
uv run wf --config examples/report_workflow/wf.config.json cap list --source local.report
uv run wf --config examples/report_workflow/wf.config.json cap call local.report.extract_report --input-file examples/report_workflow/cap-input.json --format compact
```
The full artifact/deployment/run path is covered by
`tests/examples/test_report_workflow_example.py`. To exercise the same lifecycle
manually through the CLI, use the source capability as the draft seed:
```powershell
uv run wf --config examples/report_workflow/wf.config.json draft create-from-capability report_ws local.report.extract_report --name report_case_study --title "Report Case Study"
uv run wf --config examples/report_workflow/wf.config.json draft validate report_ws
uv run wf --config examples/report_workflow/wf.config.json draft save report_ws --artifact report_case_study --version 1 --title "Report Case Study" --binding local.report=local.report
uv run wf --config examples/report_workflow/wf.config.json deploy save report_case_study.default --artifact report_case_study --version 1 --binding local.report=local.report
uv run wf --config examples/report_workflow/wf.config.json deploy validate report_case_study.default
uv run wf --config examples/report_workflow/wf.config.json run start report_case_study.default --input-file examples/report_workflow/run-input.json --trace-from 0 --trace-limit 5
uv run wf --config examples/report_workflow/wf.config.json run list --limit 5
uv run wf --config examples/report_workflow/wf.config.json run inspect <run_id>
uv run wf --config examples/report_workflow/wf.config.json run trace <run_id> --from 0 --limit 5
```
The expected report includes:
- title: `Weekly Project Update`
- three action items
- at least one risk mentioning Google Drive MCP quota
- followups for Markdown rendering and baseline comparison
## Thesis Evidence
The example supports these claims:
- Python sources can expose typed capabilities through the same workflow surface
as built-in and MCP sources.
- The case-study path is deterministic and does not depend on an LLM or remote
provider.
- The workflow lifecycle can be exercised through config validation, capability
inventory, capability calls, artifacts, deployments, runs, inspect, and trace.
+3
View File
@@ -0,0 +1,3 @@
{
"text": "# Weekly Project Update\n\nSummary:\nThe workflow platform demo is ready for a deterministic thesis case study. The\nteam wants a repeatable report that does not depend on remote OAuth, LLM output,\nor provider quotas.\n\nActions:\n- Alice | Prepare demo config | Friday\n- Bao | Run five agent attempts | Monday\n- Casey | Capture trace screenshots | Tuesday\n\nRisks:\n- Google Drive MCP quota is too low for regression evidence\n- Unbounded provider output can waste tokens\n\nFollowups:\n- Add optional Markdown renderer\n- Compare direct script baseline against workflow lifecycle\n"
}
+19
View File
@@ -0,0 +1,19 @@
# Weekly Project Update
Summary:
The workflow platform demo is ready for a deterministic thesis case study. The
team wants a repeatable report that does not depend on remote OAuth, LLM output,
or provider quotas.
Actions:
- Alice | Prepare demo config | Friday
- Bao | Run five agent attempts | Monday
- Casey | Capture trace screenshots | Tuesday
Risks:
- Google Drive MCP quota is too low for regression evidence
- Unbounded provider output can waste tokens
Followups:
- Add optional Markdown renderer
- Compare direct script baseline against workflow lifecycle
+110
View File
@@ -0,0 +1,110 @@
from __future__ import annotations
from pathlib import Path
from pydantic import BaseModel, Field
from wf_authoring import node
class ReadInput(BaseModel):
path: str = Field(description="Path to a UTF-8 Markdown notes file.")
class ReadOutput(BaseModel):
text: str
class ExtractInput(BaseModel):
text: str
class ActionItem(BaseModel):
owner: str
task: str
due: str
class ReportOutput(BaseModel):
title: str
summary: str
action_items: list[ActionItem]
risks: list[str]
followups: list[str]
class MarkdownInput(BaseModel):
report: ReportOutput
class MarkdownOutput(BaseModel):
markdown: str
@node(name="read_notes")
def read_notes(payload: ReadInput) -> ReadOutput:
return ReadOutput(text=Path(payload.path).read_text(encoding="utf-8"))
@node(name="extract_report")
def extract_report(payload: ExtractInput) -> ReportOutput:
title = ""
summary_lines: list[str] = []
actions: list[ActionItem] = []
risks: list[str] = []
followups: list[str] = []
section: str | None = None
for raw_line in payload.text.splitlines():
line = raw_line.strip()
if not line:
continue
if line.startswith("# "):
title = line.removeprefix("# ").strip()
continue
if line.endswith(":"):
section = line[:-1].lower()
continue
if section == "summary":
summary_lines.append(line)
elif section == "actions" and line.startswith("- "):
parts = [part.strip() for part in line.removeprefix("- ").split("|")]
if len(parts) == 3:
owner, task, due = parts
actions.append(ActionItem(owner=owner, task=task, due=due))
elif section == "risks" and line.startswith("- "):
risks.append(line.removeprefix("- ").strip())
elif section == "followups" and line.startswith("- "):
followups.append(line.removeprefix("- ").strip())
return ReportOutput(
title=title,
summary=" ".join(summary_lines),
action_items=actions,
risks=risks,
followups=followups,
)
@node(name="render_markdown_report")
def render_markdown_report(payload: MarkdownInput) -> MarkdownOutput:
report = payload.report
lines = [
f"# {report.title}",
"",
report.summary,
"",
"## Action Items",
]
lines.extend(
f"- {item.owner}: {item.task} (due: {item.due})"
for item in report.action_items
)
lines.extend(["", "## Risks"])
lines.extend(f"- {risk}" for risk in report.risks)
lines.extend(["", "## Followups"])
lines.extend(f"- {followup}" for followup in report.followups)
return MarkdownOutput(markdown="\n".join(lines))
registry = [read_notes, extract_report, render_markdown_report]
+3
View File
@@ -0,0 +1,3 @@
{
"text": "# Weekly Project Update\n\nSummary:\nThe workflow platform demo is ready for a deterministic thesis case study. The\nteam wants a repeatable report that does not depend on remote OAuth, LLM output,\nor provider quotas.\n\nActions:\n- Alice | Prepare demo config | Friday\n- Bao | Run five agent attempts | Monday\n- Casey | Capture trace screenshots | Tuesday\n\nRisks:\n- Google Drive MCP quota is too low for regression evidence\n- Unbounded provider output can waste tokens\n\nFollowups:\n- Add optional Markdown renderer\n- Compare direct script baseline against workflow lifecycle\n"
}
@@ -0,0 +1,116 @@
from __future__ import annotations
from pathlib import Path
import pytest
from wf_config import load_workflow_config
from wf_server.config import build_workflow_server_from_workflow_config
EXAMPLE_DIR = Path(__file__).resolve().parents[2] / "examples" / "report_workflow"
@pytest.mark.asyncio
async def test_report_workflow_python_source_loads_and_calls_capability(
tmp_path,
) -> None:
config = load_workflow_config(EXAMPLE_DIR / "wf.config.json")
config.server.store.root = tmp_path / "store"
server = build_workflow_server_from_workflow_config(config)
listed = await server.api.list_capabilities(source_id="local.report")
names = {capability["name"] for capability in listed["capabilities"]}
assert "local.report.extract_report" in names
result = await server.api.call_capability(
qualified_name="local.report.extract_report",
payload={"text": (EXAMPLE_DIR / "input.md").read_text(encoding="utf-8")},
)
assert result["outcome"] == "ok"
assert result["output"]["title"] == "Weekly Project Update"
assert result["output"]["action_items"][0] == {
"owner": "Alice",
"task": "Prepare demo config",
"due": "Friday",
}
assert "Google Drive MCP quota" in result["output"]["risks"][0]
@pytest.mark.asyncio
async def test_report_workflow_artifact_deployment_run_path(tmp_path) -> None:
config = load_workflow_config(EXAMPLE_DIR / "wf.config.json")
config.server.store.root = tmp_path / "store"
server = build_workflow_server_from_workflow_config(config)
plan = {
"name": "report_case_study",
"input_schema": {
"type": "object",
"properties": {"text": {"type": "string"}},
"required": ["text"],
},
"state_schema": {
"type": "object",
"properties": {"report": {"type": "object", "reducer": "wf.std.replace"}},
},
"output_schema": {
"type": "object",
"properties": {"report": {"type": "object"}},
"required": ["report"],
},
"outcomes": ["ok"],
"start": "extract",
"nodes": [
{
"id": "extract",
"type": "node",
"node": "local.report.extract_report",
"input": [
{
"path": {"root": "input", "parts": ["text"]},
"target": {"root": "local", "parts": ["text"]},
}
],
"output": [
{
"source": {"root": "local", "parts": []},
"target": {"root": "state", "parts": ["report"]},
}
],
}
],
"edges": [{"from": "extract", "outcome": "ok", "to": "__end__"}],
"output": [
{
"path": {"root": "state", "parts": ["report"]},
"target": {"root": "local", "parts": ["report"]},
}
],
}
await server.api.create_artifact_from_plan(
artifact_id="report_case_study",
version=1,
title="Report Case Study",
plan=plan,
outcomes=["ok"],
source_bindings={"local.report": "local.report"},
)
await server.api.save_deployment(
{
"id": "report_case_study.default",
"artifact_id": "report_case_study",
"artifact_version": 1,
"bindings": {"local.report": "local.report"},
}
)
run = await server.api.run_deployment(
deployment_id="report_case_study.default",
workflow_input={"text": (EXAMPLE_DIR / "input.md").read_text(encoding="utf-8")},
)
assert run["status"] == "completed"
assert run["output"]["report"]["title"] == "Weekly Project Update"
assert len(run["output"]["report"]["action_items"]) == 3