docs: plan thesis evidence and reject platform bindings
This commit is contained in:
+4
-3
@@ -84,7 +84,7 @@ _Avoid_: Ad-hoc environment variables, hidden source lookup
|
||||
A process-provided source with a fixed identity, such as `wf.std` or
|
||||
`wf.source`. Platform sources can satisfy workflow requirements without
|
||||
deployment self-bindings because their logical source id is also their concrete
|
||||
source id.
|
||||
source id, and deployments should not override them with custom bindings.
|
||||
_Avoid_: Configured account source, user-installed provider
|
||||
|
||||
**Configured Source**:
|
||||
@@ -106,14 +106,15 @@ An inert pass-by-value reference to source-owned content, carrying a logical
|
||||
source and provider URI. The URI is not globally meaningful by itself; runtime
|
||||
must resolve the logical source through deployment/platform context before a
|
||||
helper such as `wf.source.read_resource` can dereference it.
|
||||
_Avoid_: Bare URI, immediate content fetch
|
||||
_Avoid_: Resource Ref as the canonical term, bare URI, immediate content fetch
|
||||
|
||||
**Prompt Inventory**:
|
||||
Source-owned prompt/template names and metadata. Listing prompts is safe
|
||||
inventory; rendering a prompt is an upstream operation that may be stateful and
|
||||
needs a concrete graph use case, argument schema, and bounded output policy
|
||||
before becoming a workflow helper.
|
||||
_Avoid_: Treating prompt list entries as already-rendered text
|
||||
_Avoid_: Source Prompt Ref as premature symmetry, treating prompt list entries
|
||||
as already-rendered text
|
||||
|
||||
**Workflow Portability**:
|
||||
The goal that a workflow artifact can describe logical requirements separately
|
||||
|
||||
@@ -117,9 +117,10 @@ trace, and commits reducer-aware state changes.
|
||||
|
||||
Platform sources such as `wf.std` and `wf.source` are process-provided sources.
|
||||
They can appear in artifacts and runs without deployment self-bindings like
|
||||
`wf.std=wf.std`; configured sources such as `local.ops` or `everything.default`
|
||||
still use deployment bindings when a workflow needs portability across accounts
|
||||
or workspaces.
|
||||
`wf.std=wf.std`. Deployment validation rejects explicit platform-source
|
||||
bindings as stale configuration. Configured sources such as `local.ops` or
|
||||
`everything.default` still use deployment bindings when a workflow needs
|
||||
portability across accounts or workspaces.
|
||||
|
||||
## Source Model
|
||||
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
# Diagram Scratchpad
|
||||
|
||||
This file is a working library of Mermaid diagrams for the thesis/report. Keep
|
||||
diagrams here while they are being shaped, then copy stable versions into the
|
||||
final document when the surrounding prose is ready.
|
||||
|
||||
## Main Architecture Spine
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Owner[Workflow Owner] --> Agent[External LLM Agent]
|
||||
Agent --> CLI[wf CLI]
|
||||
CLI --> Transport[JSON-RPC Transport]
|
||||
Transport --> Server[WorkflowServer]
|
||||
Server --> API[Workflow API Surface]
|
||||
API --> Core[Workflow Core]
|
||||
API --> Platform[Artifacts / Deployments / Runs]
|
||||
Server --> Sources[Source Providers]
|
||||
Sources --> Builtins[Platform Sources]
|
||||
Sources --> MCP[MCP Sources]
|
||||
Sources --> Python[Python Sources]
|
||||
```
|
||||
|
||||
## Workflow Lifecycle
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Draft[Draft Workspace] --> ValidateDraft[Draft Validation]
|
||||
ValidateDraft --> Artifact[Workflow Artifact]
|
||||
Artifact --> Deployment[Workflow Deployment]
|
||||
Deployment --> ValidateDeploy[Deployment Validation]
|
||||
ValidateDeploy --> Run[Workflow Run]
|
||||
Run --> Trace[Run Trace]
|
||||
Run --> Inspect[Run Inspect / List]
|
||||
Run --> Resume[Resume If Interrupted]
|
||||
```
|
||||
|
||||
## Source Resolution
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Ref[Logical Source Requirement] --> PlatformCheck{Platform Source?}
|
||||
PlatformCheck -- yes --> Fixed[Fixed Source Id]
|
||||
PlatformCheck -- no --> Binding[Deployment Binding]
|
||||
Binding --> Concrete[Concrete Source]
|
||||
Fixed --> Runtime[Source Runtime / Handler]
|
||||
Concrete --> Runtime
|
||||
Runtime --> Capability[Workflow Capability]
|
||||
```
|
||||
|
||||
+212
-17
@@ -3,11 +3,17 @@
|
||||
This is a writing scaffold for a thesis/report about the workflow platform. It
|
||||
should guide the argument; it is not a changelog.
|
||||
|
||||
The final document should read like a formal system design and implementation
|
||||
report. Keep detailed command transcripts and long CLI outputs in appendices or
|
||||
linked runbooks; inline chapters should show only the commands/results needed to
|
||||
support the argument.
|
||||
|
||||
## Core Argument
|
||||
|
||||
Agent work should be represented as typed, durable workflows. The LLM should
|
||||
plan and revise workflow structure, but a deterministic runtime should own
|
||||
execution, state, validation, persistence, and source binding.
|
||||
External LLM agents are useful workflow authors and operators, but durable
|
||||
workspace automation needs a typed execution substrate. Artifacts, deployments,
|
||||
source bindings, validation, runs, traces, and resumability should be owned by
|
||||
the platform, not improvised through raw tool-call loops.
|
||||
|
||||
The short version:
|
||||
|
||||
@@ -39,6 +45,65 @@ The thesis contribution is a platform architecture, not a new foundation model:
|
||||
5. next-action guidance that points an agent toward useful lifecycle operations
|
||||
without replacing validation
|
||||
|
||||
## Evidence Strategy
|
||||
|
||||
Use one representative workspace case study as the narrative spine: a
|
||||
document/report preparation workflow backed by local fixtures and trusted Python
|
||||
sources. The case study should read or receive a small document-like input,
|
||||
extract structured information such as action items or sections, normalize the
|
||||
result, and produce report-shaped JSON or Markdown. MCP-backed sources can be
|
||||
secondary evidence, but the thesis-critical demo should not depend on remote
|
||||
MCP auth, quota, or provider availability.
|
||||
|
||||
The case study should exist as a runnable example, not only prose. Target shape:
|
||||
`examples/report_workflow/ops.py`, `input.md`, `wf.config.json`, a short
|
||||
`README.md`, and commands for config validation, server startup, capability
|
||||
calls, draft/artifact/deployment creation, run, inspect, and trace.
|
||||
|
||||
Keep the thesis-critical path deterministic. Do not require an LLM call inside
|
||||
the case-study workflow. LLM nodes can be discussed as future work or an
|
||||
optional variant, but the evidence path should be reproducible without model
|
||||
credentials, cost, or output variance.
|
||||
|
||||
Prefer typed report JSON as the primary output, with Markdown rendering optional
|
||||
later. A useful output contract is:
|
||||
|
||||
```json
|
||||
{
|
||||
"title": "Weekly Project Update",
|
||||
"summary": "...",
|
||||
"action_items": [
|
||||
{"owner": "Alice", "task": "Prepare demo config", "due": "Friday"}
|
||||
],
|
||||
"risks": ["..."],
|
||||
"followups": ["..."]
|
||||
}
|
||||
```
|
||||
|
||||
This makes schemas and validation visible in the case study.
|
||||
|
||||
Use CLI lifecycle commands for the thesis narrative because they demonstrate the
|
||||
agent-operable surface. Tests may seed `RawWorkflowPlan` objects directly when
|
||||
that makes assertions tighter, but the case-study runbook should show config
|
||||
validation, server startup, capability inspection/call, draft or artifact
|
||||
creation, deployment save/validate, run start, run inspect, run trace, and run
|
||||
list.
|
||||
|
||||
Support that case study with platform evidence: automated tests, CLI/server
|
||||
smoke runs, source-provider examples, run persistence/resume checks, source
|
||||
drift producing unrunnable deployments, stateful MCP session reuse, and a small
|
||||
failed-attempt case study showing how validation/diagnostics reduced blind
|
||||
retries.
|
||||
|
||||
Do not frame the evaluation as a broad user study unless that study actually
|
||||
exists. The evidence claim is that the prototype demonstrates the architecture
|
||||
and workflow lifecycle under controlled examples.
|
||||
|
||||
Do not make runtime throughput a central claim. The project targets planner
|
||||
efficiency and operational clarity: fewer blind retries through typed contracts,
|
||||
validation, diagnostics, compact outputs, and traces. Performance optimization
|
||||
is future work unless backed by explicit measurements.
|
||||
|
||||
## Working Title
|
||||
|
||||
Safer current title:
|
||||
@@ -137,7 +202,31 @@ The design goals should be stated early and then revisited in evaluation:
|
||||
- source-provider correctness, especially for external systems whose tools,
|
||||
resources, prompts, or authentication depend on initialized stateful sessions
|
||||
|
||||
## 4. Architecture
|
||||
Auth should be described as prototype source-readiness plumbing, not a completed
|
||||
production secret system. The implementation includes typed auth records, OAuth
|
||||
refresh-token support, source auth diagnostics, and MCP auth binding. This is
|
||||
enough to show how credentials participate in source readiness and diagnostics,
|
||||
but encrypted-at-rest storage, production secret-manager integration, and broad
|
||||
provider verification remain future work.
|
||||
|
||||
## 4. Positioning And Related Systems
|
||||
|
||||
Keep this section short and category-oriented. The goal is to position the
|
||||
system, not to claim full feature parity with mature platforms.
|
||||
|
||||
Compare against:
|
||||
|
||||
- direct LLM tool orchestration: flexible but weak durability and validation
|
||||
- generated scripts: simple and maintainable for some tasks, but lifecycle
|
||||
affordances are manual
|
||||
- Zapier/RPA/workflow automation platforms: mature integrations and scheduling,
|
||||
but less agent-native typed authoring/repair flow in this prototype's terms
|
||||
- LangGraph-style agent graphs/durable agents: adjacent durability ideas, but a
|
||||
different emphasis from source-provider-backed reusable workspace workflows
|
||||
- MCP: useful protocol for tools/resources/prompts, but not itself the workflow
|
||||
artifact/deployment/run lifecycle
|
||||
|
||||
## 5. Architecture
|
||||
|
||||
Explain the active package boundaries:
|
||||
|
||||
@@ -162,6 +251,11 @@ Important layers:
|
||||
- `wf_sources_python`: trusted in-process Python source loading
|
||||
- `wf_mcp`: legacy/special-purpose MCP compatibility package
|
||||
|
||||
Use package names as implementation evidence, not as the main argument. The
|
||||
conceptual architecture should lead: workflow core, platform domain, workflow
|
||||
API surface, server/transport composition, and source providers. Package names
|
||||
then show how those concepts were implemented in this codebase.
|
||||
|
||||
The thesis should explain why the old “everything in MCP” shape was split:
|
||||
transport, source provider, workflow API, and runtime concerns are different.
|
||||
|
||||
@@ -185,7 +279,13 @@ If MCP is discussed, distinguish upstream MCP sources from a future client-facin
|
||||
MCP frontend. The former exists as a source family; the latter should not be
|
||||
claimed as a completed clean platform surface.
|
||||
|
||||
## 5. Workflow Model
|
||||
MCP is an important source-provider case study, not the product identity. It
|
||||
demonstrates why source-provider correctness matters: a source may require
|
||||
persistent sessions, auth context, catalog refresh, resources, and prompt
|
||||
inventory. The platform treats MCP as one source family behind the workflow
|
||||
boundary, not as the whole architecture.
|
||||
|
||||
## 6. Workflow Model
|
||||
|
||||
Describe workflows as typed graphs:
|
||||
|
||||
@@ -212,12 +312,18 @@ Key distinction:
|
||||
- outcome controls routing
|
||||
- output carries business data
|
||||
|
||||
The graph model is also a safety boundary. It is not safer because it can make
|
||||
all tools safe; it is safer than arbitrary generated scripts because structure,
|
||||
schemas, source bindings, state, outcomes, and review points are explicit. This
|
||||
is the answer to “why not just have the AI write a Playwright script?” Scripts
|
||||
can be simple and maintainable, but they do not automatically provide the same
|
||||
validation and lifecycle affordances.
|
||||
The graph model improves the safety posture by making automation structure
|
||||
explicit. Node contracts, source requirements, state writes, outcomes,
|
||||
validation gates, and trace records are visible before and after execution. It
|
||||
does not guarantee safe behavior from provider code, credentials, or external
|
||||
side effects.
|
||||
|
||||
Use generated scripts as a serious baseline, not a strawman. Scripts can be
|
||||
simpler and maintainable for many tasks. The platform argument is that reusable
|
||||
workspace automation benefits from lifecycle affordances that scripts do not
|
||||
automatically provide: typed validation, source binding, artifact/deployment
|
||||
separation, run records, resumability, trace inspection, and repairable
|
||||
diagnostics.
|
||||
|
||||
Code ends at the source-provider boundary. A workflow can call trusted Python,
|
||||
Playwright, API, MCP, or future LLM capabilities, but those should appear as
|
||||
@@ -235,7 +341,7 @@ Trace claims should be grounded in the current code: run summaries expose
|
||||
Do not overstate this as production observability, distributed tracing, metrics,
|
||||
or OpenTelemetry support.
|
||||
|
||||
## 6. Source Model
|
||||
## 7. Source Model
|
||||
|
||||
The common boundary is `CapabilitySource`.
|
||||
|
||||
@@ -287,7 +393,7 @@ For MCP, source-provider correctness includes stateful runtime behavior. A
|
||||
workflow capability call should not silently turn a stateful external provider
|
||||
into a fresh one-off client call when provider state is part of correctness.
|
||||
|
||||
## 7. Implementation Vertical Slice
|
||||
## 8. Implementation Vertical Slice
|
||||
|
||||
Use the working product path as evidence:
|
||||
|
||||
@@ -317,11 +423,49 @@ A strong demonstration is the Python source flow:
|
||||
|
||||
This shows the source abstraction is not MCP-only.
|
||||
|
||||
Use diagrams as first-class explanation, especially Mermaid diagrams that can be
|
||||
rendered by the existing document generation flow. Each major part should have
|
||||
at least one diagram that explains its role and boundaries before code excerpts
|
||||
or package names appear. Prefer diagrams for:
|
||||
|
||||
- main architecture spine:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Owner[Workflow Owner] --> Agent[External LLM Agent]
|
||||
Agent --> CLI[wf CLI]
|
||||
CLI --> Transport[JSON-RPC Transport]
|
||||
Transport --> Server[WorkflowServer]
|
||||
Server --> API[Workflow API Surface]
|
||||
API --> Core[Workflow Core]
|
||||
API --> Platform[Artifacts / Deployments / Runs]
|
||||
Server --> Sources[Source Providers]
|
||||
Sources --> Builtins[Platform Sources]
|
||||
Sources --> MCP[MCP Sources]
|
||||
Sources --> Python[Python Sources]
|
||||
```
|
||||
|
||||
- layer architecture: CLI/transport/server/API/core/source providers
|
||||
- workflow core: schemas, nodes, outcomes, reducers, trace, interrupts/resume
|
||||
- platform domain: draft workspaces, artifacts, deployments, source inventory,
|
||||
validation diagnostics, run records
|
||||
- lifecycle: draft -> artifact -> deployment -> run -> trace/list/resume
|
||||
- source resolution: logical source -> deployment binding/platform context ->
|
||||
concrete source/runtime
|
||||
- source-provider comparison: built-in, MCP, Python, future OpenAPI
|
||||
- runtime call path: cap call/run -> Workflow API -> source runtime/client
|
||||
|
||||
Use source excerpts sparingly. Include small snippets for key seams such as the
|
||||
artifact/deployment/run lifecycle shape, `CapabilitySource`, the
|
||||
`WorkflowSourceProvider` protocol, a compact Python source `@node` example, and
|
||||
selected CLI/JSON responses. Avoid long file listings; the implementation
|
||||
chapter should explain the architecture, not reproduce the repository.
|
||||
|
||||
Frame Python sources as trusted developer extensibility. They are useful because
|
||||
project-local code can become typed workflow capabilities quickly, but they are
|
||||
not sandboxed non-programmer plugins yet.
|
||||
|
||||
## 8. Evaluation
|
||||
## 9. Evaluation
|
||||
|
||||
Evaluation should use concrete evidence:
|
||||
|
||||
@@ -347,6 +491,35 @@ Evaluation should use concrete evidence:
|
||||
or source assumptions caused repeated failed runs
|
||||
- source-drift cases where old deployments become unrunnable with diagnostics
|
||||
instead of silently executing against incompatible capabilities
|
||||
- current agent/tool evaluation with a small repeat count. A practical prototype
|
||||
target is five end-to-end attempts where free or commodity LLM agents try to
|
||||
use the CLI/API surface to complete the deterministic case study. Report
|
||||
pass/fail counts, failure categories, and whether diagnostics were actionable;
|
||||
do not present this as a statistical reliability benchmark.
|
||||
|
||||
For the agent/tool evaluation, success means end-to-end completion: the agent
|
||||
creates or selects a valid workflow artifact/deployment and completes a run whose
|
||||
output matches the expected typed report schema and key content. Merely calling
|
||||
a capability, producing a draft, or returning freeform text outside the schema is
|
||||
not success.
|
||||
|
||||
Count failures explicitly. Useful categories include:
|
||||
|
||||
- config/setup failure
|
||||
- source discovery or source binding failure
|
||||
- draft validation failure
|
||||
- deployment validation failure
|
||||
- run failure
|
||||
- output schema/content mismatch
|
||||
- excessive manual intervention
|
||||
- agent gave up or looped without progress
|
||||
|
||||
Track autonomous and assisted success separately. Autonomous success means the
|
||||
agent completes the task with only the initial prompt/runbook. Assisted success
|
||||
means completion after limited documented help such as confirming that the
|
||||
server is running or pointing at the intended config path. Manual artifact edits,
|
||||
code fixes, or changing the expected output should count as failures for
|
||||
autonomous evaluation.
|
||||
|
||||
Avoid vague claims such as “robust” or “production-ready” unless backed by
|
||||
specific checks.
|
||||
@@ -392,7 +565,7 @@ Possible evaluation questions:
|
||||
- Does deployment validation surface source drift as runnable/unrunnable state
|
||||
with diagnostics rather than silent behavior changes?
|
||||
|
||||
## 8.1 Positioning Against Existing Automation Platforms
|
||||
## 9.1 Positioning Against Existing Automation Platforms
|
||||
|
||||
The thesis should discuss the space it fits into through multiple baselines:
|
||||
direct LLM tool use, manual scripts, Zapier-style automation platforms, RPA
|
||||
@@ -421,7 +594,7 @@ This prototype explores a different center of gravity:
|
||||
Use the comparison to position the work, not as a claim that the prototype
|
||||
outperforms existing automation products.
|
||||
|
||||
## 9. Limitations
|
||||
## 10. Limitations
|
||||
|
||||
State limitations explicitly:
|
||||
|
||||
@@ -430,6 +603,8 @@ State limitations explicitly:
|
||||
- Source provider lifecycle is early, especially for non-MCP mutable sources.
|
||||
- Workflow portability is scoped; local Python code, MCP catalogs, auth records,
|
||||
and source stores can differ between environments.
|
||||
- The prototype has not been evaluated against a broad external provider catalog
|
||||
or a large user study.
|
||||
- File-backed stores are the current implementation proof for durable lifecycle;
|
||||
durability itself should not be framed as filesystem-specific.
|
||||
- Auth records/admin surfaces exist as prototype plumbing, but end-to-end
|
||||
@@ -439,12 +614,14 @@ State limitations explicitly:
|
||||
are not carried through the durable workflow path.
|
||||
- Crash recovery is at stopped boundaries, not arbitrary mid-node checkpoints.
|
||||
- Offline scheduling is not implemented yet.
|
||||
- There is no visual workflow editor yet.
|
||||
- There is no bundled autonomous agent brain.
|
||||
- General fork/gather workflow control is future work.
|
||||
- There is no full approval, roles, policy, or multi-user review system.
|
||||
|
||||
Limitations make the thesis more credible. They also motivate future work.
|
||||
|
||||
## 10. Future Work
|
||||
## 11. Future Work
|
||||
|
||||
Likely future-work sections:
|
||||
|
||||
@@ -462,6 +639,24 @@ Likely future-work sections:
|
||||
- first-party workflow UI for listing, inspecting, and editing workflows
|
||||
- richer evaluation with real workflows and larger source catalogs
|
||||
|
||||
## 12. Conclusion
|
||||
|
||||
Restate the core argument: external LLM agents can author and operate workflows,
|
||||
but the durable workflow lifecycle should live in a typed platform substrate.
|
||||
Summarize what the implementation proves: artifacts, deployments, runs,
|
||||
validation, source providers, server/API/CLI surfaces, and reproducible evidence
|
||||
across built-in, MCP, and Python sources.
|
||||
|
||||
## Appendices
|
||||
|
||||
Keep long operational material out of the main argument:
|
||||
|
||||
- reproducible command transcript for the case study
|
||||
- smoke-test commands and abbreviated outputs
|
||||
- selected config files
|
||||
- generated workflow/artifact/deployment examples
|
||||
- test/evidence index
|
||||
|
||||
## What Not To Do
|
||||
|
||||
Do not make the thesis a commit history. The reader does not need every
|
||||
|
||||
@@ -114,8 +114,9 @@ auth admin are implemented. The next work is polish, not new broad surfaces.
|
||||
MCP HTTP, MCP stdio, Python sources, auth refs, OAuth refresh-token setup,
|
||||
diagnostics, and the Google Drive MCP caveat.
|
||||
- Completed platform source policy: `wf.*` process-provided sources are marked
|
||||
as platform sources and no longer require self-bindings such as
|
||||
`wf.std=wf.std` in deployments.
|
||||
as platform sources. They resolve by fixed source id, do not require
|
||||
self-bindings, and deployment validation rejects explicit platform-source
|
||||
bindings as stale configuration.
|
||||
- Completed `wf.source.read_resource`: resource refs are inert pass-by-value
|
||||
data using `logical_source`; explicit platform helper nodes dereference them
|
||||
through runtime/platform context with bounded output.
|
||||
|
||||
@@ -104,7 +104,9 @@ source ids. For example, a resource ref should store:
|
||||
The deployment binding decides whether `drive` means `drive.personal`,
|
||||
`drive.work`, or another concrete source. Platform sources such as `wf.std` and
|
||||
`wf.source` are special because their logical source id is also their concrete
|
||||
source id, so they do not require deployment bindings.
|
||||
source id, so they do not require deployment bindings. Deployments should not
|
||||
bind platform sources explicitly; validation rejects those bindings as stale
|
||||
or misleading configuration.
|
||||
|
||||
Runtime dereference is explicit. Passing a resource ref by value does not fetch
|
||||
content. A helper capability such as `wf.source.read_resource` receives the ref,
|
||||
@@ -157,8 +159,8 @@ operator-configured project sources.
|
||||
|
||||
Generated draft workflows may still use built-in helper sources such as
|
||||
`wf.std`. Platform sources do not need deployment bindings, so deployment
|
||||
examples should bind configured sources only unless validation reports a
|
||||
non-platform logical source requirement.
|
||||
examples should bind configured sources only. If validation reports a platform
|
||||
source binding, remove it rather than changing the concrete source id.
|
||||
|
||||
## `wf_sources_mcp` Internal Layers
|
||||
|
||||
|
||||
@@ -0,0 +1,586 @@
|
||||
# Thesis Case Study Evidence Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build a reproducible deterministic report-workflow example and evidence bundle that the thesis/system-design document can cite.
|
||||
|
||||
**Architecture:** The example should use a trusted Python source as the deterministic provider and the existing `wf` / `wf-rpc-server` lifecycle as the product surface. The evidence must prove the current code path, not restate claims from docs without verification.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic, `wf_authoring.node`, `wf_config`, `wf_server`, JSON-RPC HTTP transport, Typer CLI, pytest, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Files
|
||||
|
||||
- Create: `examples/report_workflow/ops.py` — deterministic Python source with typed report extraction and Markdown rendering capabilities.
|
||||
- Create: `examples/report_workflow/input.md` — stable fixture input for the case study.
|
||||
- Create: `examples/report_workflow/wf.config.json` — self-contained server/client config for the example.
|
||||
- Create: `examples/report_workflow/README.md` — runbook commands and expected result shape.
|
||||
- Create: `tests/examples/test_report_workflow_example.py` — regression tests proving the example source loads and runs through the workflow API.
|
||||
- Modify: `docs/add/thesis-outline.md` — link the concrete example as the case-study evidence artifact.
|
||||
- Modify: `docs/add/diagrams.md` — add or refine the case-study lifecycle diagram if needed.
|
||||
- Modify: `docs/current_roadmap.md` — mark the thesis case-study evidence bundle completed after implementation.
|
||||
|
||||
## Claim Verification Rule
|
||||
|
||||
Before writing any evidence claim, verify it against code or tests. Use direct code searches such as:
|
||||
|
||||
```powershell
|
||||
rg 'class WorkflowArtifact|class WorkflowDeployment|class WorkflowRun' src/wf_artifacts -n
|
||||
rg 'WorkflowSourceProvider|CapabilitySource|PythonSourceConfig' src -n
|
||||
rg 'build_workflow_server_from_workflow_config|build_local_static_workflow_server' src/wf_server -n
|
||||
```
|
||||
|
||||
For every claim in `README.md`, either point to a command in the runbook, a test in `tests/examples/test_report_workflow_example.py`, or a source file path.
|
||||
|
||||
## Task 1: Create the Deterministic Python Source
|
||||
|
||||
**Files:**
|
||||
- Create: `examples/report_workflow/ops.py`
|
||||
- Create: `examples/report_workflow/input.md`
|
||||
|
||||
- [ ] **Step 1: Create fixture input**
|
||||
|
||||
Create `examples/report_workflow/input.md` with this exact content:
|
||||
|
||||
```md
|
||||
# Weekly Project Update
|
||||
|
||||
Summary:
|
||||
The workflow platform demo is ready for a deterministic thesis case study. The
|
||||
team wants a repeatable report that does not depend on remote OAuth, LLM output,
|
||||
or provider quotas.
|
||||
|
||||
Actions:
|
||||
- Alice | Prepare demo config | Friday
|
||||
- Bao | Run five agent attempts | Monday
|
||||
- Casey | Capture trace screenshots | Tuesday
|
||||
|
||||
Risks:
|
||||
- Google Drive MCP quota is too low for regression evidence
|
||||
- Unbounded provider output can waste tokens
|
||||
|
||||
Followups:
|
||||
- Add optional Markdown renderer
|
||||
- Compare direct script baseline against workflow lifecycle
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Create `ops.py`**
|
||||
|
||||
Create `examples/report_workflow/ops.py` with this exact source:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from wf_authoring import node
|
||||
|
||||
|
||||
class ReadInput(BaseModel):
|
||||
path: str = Field(description="Path to a UTF-8 Markdown notes file.")
|
||||
|
||||
|
||||
class ReadOutput(BaseModel):
|
||||
text: str
|
||||
|
||||
|
||||
class ExtractInput(BaseModel):
|
||||
text: str
|
||||
|
||||
|
||||
class ActionItem(BaseModel):
|
||||
owner: str
|
||||
task: str
|
||||
due: str
|
||||
|
||||
|
||||
class ReportOutput(BaseModel):
|
||||
title: str
|
||||
summary: str
|
||||
action_items: list[ActionItem]
|
||||
risks: list[str]
|
||||
followups: list[str]
|
||||
|
||||
|
||||
class MarkdownInput(BaseModel):
|
||||
report: ReportOutput
|
||||
|
||||
|
||||
class MarkdownOutput(BaseModel):
|
||||
markdown: str
|
||||
|
||||
|
||||
@node(name="read_notes")
|
||||
def read_notes(payload: ReadInput) -> ReadOutput:
|
||||
return ReadOutput(text=Path(payload.path).read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
@node(name="extract_report")
|
||||
def extract_report(payload: ExtractInput) -> ReportOutput:
|
||||
title = ""
|
||||
summary_lines: list[str] = []
|
||||
actions: list[ActionItem] = []
|
||||
risks: list[str] = []
|
||||
followups: list[str] = []
|
||||
section: str | None = None
|
||||
|
||||
for raw_line in payload.text.splitlines():
|
||||
line = raw_line.strip()
|
||||
if not line:
|
||||
continue
|
||||
if line.startswith("# "):
|
||||
title = line.removeprefix("# ").strip()
|
||||
continue
|
||||
if line.endswith(":"):
|
||||
section = line[:-1].lower()
|
||||
continue
|
||||
if section == "summary":
|
||||
summary_lines.append(line)
|
||||
elif section == "actions" and line.startswith("- "):
|
||||
parts = [part.strip() for part in line.removeprefix("- ").split("|")]
|
||||
if len(parts) == 3:
|
||||
owner, task, due = parts
|
||||
actions.append(ActionItem(owner=owner, task=task, due=due))
|
||||
elif section == "risks" and line.startswith("- "):
|
||||
risks.append(line.removeprefix("- ").strip())
|
||||
elif section == "followups" and line.startswith("- "):
|
||||
followups.append(line.removeprefix("- ").strip())
|
||||
|
||||
return ReportOutput(
|
||||
title=title,
|
||||
summary=" ".join(summary_lines),
|
||||
action_items=actions,
|
||||
risks=risks,
|
||||
followups=followups,
|
||||
)
|
||||
|
||||
|
||||
@node(name="render_markdown_report")
|
||||
def render_markdown_report(payload: MarkdownInput) -> MarkdownOutput:
|
||||
report = payload.report
|
||||
lines = [
|
||||
f"# {report.title}",
|
||||
"",
|
||||
report.summary,
|
||||
"",
|
||||
"## Action Items",
|
||||
]
|
||||
lines.extend(
|
||||
f"- {item.owner}: {item.task} (due: {item.due})"
|
||||
for item in report.action_items
|
||||
)
|
||||
lines.extend(["", "## Risks"])
|
||||
lines.extend(f"- {risk}" for risk in report.risks)
|
||||
lines.extend(["", "## Followups"])
|
||||
lines.extend(f"- {followup}" for followup in report.followups)
|
||||
return MarkdownOutput(markdown="\n".join(lines))
|
||||
|
||||
|
||||
registry = [read_notes, extract_report, render_markdown_report]
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit the source fixture**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git add examples/report_workflow/input.md examples/report_workflow/ops.py
|
||||
git commit -m "docs: add deterministic report source fixture"
|
||||
```
|
||||
|
||||
Expected: commit succeeds.
|
||||
|
||||
## Task 2: Add Example Workflow Config
|
||||
|
||||
**Files:**
|
||||
- Create: `examples/report_workflow/wf.config.json`
|
||||
|
||||
- [ ] **Step 1: Create config**
|
||||
|
||||
Create `examples/report_workflow/wf.config.json` with this exact JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"client": {
|
||||
"target": {
|
||||
"kind": "rpc_http",
|
||||
"url": "http://127.0.0.1:8771/rpc",
|
||||
"timeout_seconds": 30
|
||||
}
|
||||
},
|
||||
"server": {
|
||||
"store": {
|
||||
"kind": "filesystem",
|
||||
"root": ".wf_report_store"
|
||||
},
|
||||
"transports": [
|
||||
{
|
||||
"kind": "rpc_http",
|
||||
"host": "127.0.0.1",
|
||||
"port": 8771,
|
||||
"path": "/rpc"
|
||||
}
|
||||
],
|
||||
"sources": [
|
||||
{
|
||||
"kind": "python",
|
||||
"id": "local.report",
|
||||
"path": ".",
|
||||
"module": "ops",
|
||||
"registry": "registry"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Validate config manually**
|
||||
|
||||
Run from repo root:
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json config validate
|
||||
```
|
||||
|
||||
Expected: command exits `0` and reports a valid config. If this command name differs in current code, inspect `uv run wf config --help` and update the runbook to the real command.
|
||||
|
||||
- [ ] **Step 3: Commit config**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git add examples/report_workflow/wf.config.json
|
||||
git commit -m "docs: add report workflow example config"
|
||||
```
|
||||
|
||||
Expected: commit succeeds.
|
||||
|
||||
## Task 3: Test the Example Source and Workflow API Path
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/examples/test_report_workflow_example.py`
|
||||
|
||||
- [ ] **Step 1: Write tests**
|
||||
|
||||
Create `tests/examples/test_report_workflow_example.py` with this exact test module:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from wf_config import load_workflow_config
|
||||
from wf_server.config import build_workflow_server_from_workflow_config
|
||||
|
||||
|
||||
EXAMPLE_DIR = Path(__file__).resolve().parents[2] / "examples" / "report_workflow"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_report_workflow_python_source_loads_and_calls_capability(tmp_path) -> None:
|
||||
config = load_workflow_config(EXAMPLE_DIR / "wf.config.json")
|
||||
config.server.store.root = tmp_path / "store"
|
||||
server = build_workflow_server_from_workflow_config(config)
|
||||
|
||||
listed = await server.api.list_capabilities(source="local.report")
|
||||
names = {capability["qualified_name"] for capability in listed["capabilities"]}
|
||||
|
||||
assert "local.report.extract_report" in names
|
||||
|
||||
result = await server.api.call_capability(
|
||||
qualified_name="local.report.extract_report",
|
||||
payload={"text": (EXAMPLE_DIR / "input.md").read_text(encoding="utf-8")},
|
||||
)
|
||||
|
||||
assert result["outcome"] == "ok"
|
||||
assert result["output"]["title"] == "Weekly Project Update"
|
||||
assert result["output"]["action_items"][0] == {
|
||||
"owner": "Alice",
|
||||
"task": "Prepare demo config",
|
||||
"due": "Friday",
|
||||
}
|
||||
assert "Google Drive MCP quota" in result["output"]["risks"][0]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_report_workflow_artifact_deployment_run_path(tmp_path) -> None:
|
||||
config = load_workflow_config(EXAMPLE_DIR / "wf.config.json")
|
||||
config.server.store.root = tmp_path / "store"
|
||||
server = build_workflow_server_from_workflow_config(config)
|
||||
|
||||
plan = {
|
||||
"name": "report_case_study",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {"text": {"type": "string"}},
|
||||
"required": ["text"],
|
||||
},
|
||||
"state_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"report": {"type": "object", "reducer": "wf.std.replace"}
|
||||
},
|
||||
},
|
||||
"output_schema": {
|
||||
"type": "object",
|
||||
"properties": {"report": {"type": "object"}},
|
||||
"required": ["report"],
|
||||
},
|
||||
"outcomes": ["ok"],
|
||||
"start": "extract",
|
||||
"nodes": [
|
||||
{
|
||||
"id": "extract",
|
||||
"type": "node",
|
||||
"node": "local.report.extract_report",
|
||||
"input": [
|
||||
{
|
||||
"source": {"root": "input", "parts": ["text"]},
|
||||
"target": {"root": "local", "parts": ["text"]},
|
||||
}
|
||||
],
|
||||
"output": [
|
||||
{
|
||||
"source": {"root": "local", "parts": []},
|
||||
"target": {"root": "state", "parts": ["report"]},
|
||||
}
|
||||
],
|
||||
}
|
||||
],
|
||||
"edges": [{"from": "extract", "outcome": "ok", "to": "__end__"}],
|
||||
"output": [
|
||||
{
|
||||
"path": {"root": "state", "parts": ["report"]},
|
||||
"target": {"root": "local", "parts": ["report"]},
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
await server.api.create_artifact_from_plan(
|
||||
artifact_id="report_case_study",
|
||||
version=1,
|
||||
title="Report Case Study",
|
||||
plan=plan,
|
||||
outcomes=["ok"],
|
||||
source_bindings={"local.report": "local.report"},
|
||||
)
|
||||
await server.api.save_deployment(
|
||||
{
|
||||
"id": "report_case_study.default",
|
||||
"artifact_id": "report_case_study",
|
||||
"artifact_version": 1,
|
||||
"bindings": {"local.report": "local.report"},
|
||||
}
|
||||
)
|
||||
run = await server.api.run_deployment(
|
||||
deployment_id="report_case_study.default",
|
||||
workflow_input={"text": (EXAMPLE_DIR / "input.md").read_text(encoding="utf-8")},
|
||||
)
|
||||
|
||||
assert run["status"] == "completed"
|
||||
assert run["output"]["report"]["title"] == "Weekly Project Update"
|
||||
assert len(run["output"]["report"]["action_items"]) == 3
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests and inspect failures**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/examples/test_report_workflow_example.py -q
|
||||
```
|
||||
|
||||
Expected first run may fail if current APIs use different method names or response fields. Fix only the test calls to match current code; do not weaken the assertions about title/action items/completed run.
|
||||
|
||||
- [ ] **Step 3: Run final focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/examples/test_report_workflow_example.py tests/wf_sources_python/test_loader.py tests/wf_server/test_config_composition.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
- [ ] **Step 4: Commit tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git add tests/examples/test_report_workflow_example.py
|
||||
git commit -m "test: prove report workflow example"
|
||||
```
|
||||
|
||||
Expected: commit succeeds.
|
||||
|
||||
## Task 4: Write the Case Study Runbook
|
||||
|
||||
**Files:**
|
||||
- Create: `examples/report_workflow/README.md`
|
||||
- Modify: `docs/add/thesis-outline.md`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Create README**
|
||||
|
||||
Create `examples/report_workflow/README.md` with this content:
|
||||
|
||||
```md
|
||||
# Report Workflow Example
|
||||
|
||||
This example is the deterministic thesis case study. It demonstrates a trusted
|
||||
Python source that turns project notes into a typed report object without using
|
||||
remote OAuth, LLM calls, or provider quota.
|
||||
|
||||
## Files
|
||||
|
||||
- `input.md` — fixture notes.
|
||||
- `ops.py` — Python source exposing `read_notes`, `extract_report`, and
|
||||
`render_markdown_report`.
|
||||
- `wf.config.json` — local server/client config using the `local.report` Python
|
||||
source.
|
||||
|
||||
## Run
|
||||
|
||||
From the repository root:
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json config validate
|
||||
uv run wf-rpc-server --config examples/report_workflow/wf.config.json
|
||||
```
|
||||
|
||||
In another terminal:
|
||||
|
||||
```powershell
|
||||
uv run wf --config examples/report_workflow/wf.config.json status
|
||||
uv run wf --config examples/report_workflow/wf.config.json cap list --source local.report
|
||||
uv run wf --config examples/report_workflow/wf.config.json cap call local.report.extract_report --input "{\"text\":\"$(Get-Content examples/report_workflow/input.md -Raw)\"}" --format compact
|
||||
```
|
||||
|
||||
The expected report includes:
|
||||
|
||||
- title: `Weekly Project Update`
|
||||
- three action items
|
||||
- at least one risk mentioning Google Drive MCP quota
|
||||
- followups for Markdown rendering and baseline comparison
|
||||
|
||||
## Thesis Evidence
|
||||
|
||||
The example supports these claims:
|
||||
|
||||
- Python sources can expose typed capabilities through the same workflow surface
|
||||
as built-in and MCP sources.
|
||||
- The case-study path is deterministic and does not depend on an LLM or remote
|
||||
provider.
|
||||
- The workflow lifecycle can be exercised through config validation, capability
|
||||
inventory, capability calls, artifacts, deployments, runs, inspect, and trace.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Link from thesis outline**
|
||||
|
||||
In `docs/add/thesis-outline.md`, ensure the case-study paragraph names
|
||||
`examples/report_workflow/README.md` as the runnable evidence bundle.
|
||||
|
||||
- [ ] **Step 3: Update roadmap**
|
||||
|
||||
In `docs/current_roadmap.md`, add one completed bullet under the thesis/docs area:
|
||||
|
||||
```md
|
||||
- Completed thesis case-study evidence bundle: `examples/report_workflow/`
|
||||
provides a deterministic report workflow with Python source, fixture input,
|
||||
config, runbook, and tests.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run docs smoke**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/docs tests/examples/test_report_workflow_example.py -q
|
||||
```
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
- [ ] **Step 5: Commit docs**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git add examples/report_workflow/README.md docs/add/thesis-outline.md docs/current_roadmap.md
|
||||
git commit -m "docs: document report workflow case study"
|
||||
```
|
||||
|
||||
Expected: commit succeeds.
|
||||
|
||||
## Task 5: Final Verification
|
||||
|
||||
**Files:**
|
||||
- Verify all changed files.
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/examples/test_report_workflow_example.py tests/docs -q
|
||||
```
|
||||
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 2: Run lint**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run ruff check examples/report_workflow tests/examples
|
||||
```
|
||||
|
||||
Expected: `All checks passed!`
|
||||
|
||||
- [ ] **Step 3: Run typecheck**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error examples/report_workflow tests/examples
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 4: Run whitespace check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: no whitespace errors. CRLF warnings on Windows are acceptable.
|
||||
|
||||
- [ ] **Step 5: Move this plan after completion**
|
||||
|
||||
After all tasks are implemented and committed, move this plan to:
|
||||
|
||||
```text
|
||||
docs/historical/superpowers/plans/2026-06-14-thesis-case-study-evidence.md
|
||||
```
|
||||
|
||||
Then commit:
|
||||
|
||||
```powershell
|
||||
git add docs/superpowers/plans/2026-06-14-thesis-case-study-evidence.md docs/historical/superpowers/plans/2026-06-14-thesis-case-study-evidence.md
|
||||
git commit -m "docs: archive thesis case study plan"
|
||||
```
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- The example is deterministic and local-first.
|
||||
- The test proves both capability call and workflow run paths.
|
||||
- The runbook avoids Google Drive MCP, remote OAuth, and LLM calls as required evidence.
|
||||
- Every thesis claim in the README points to a command, test, or source file.
|
||||
- No `wf.std=wf.std` platform-source self-binding appears in new current docs or tests.
|
||||
@@ -0,0 +1,610 @@
|
||||
# Thesis System Design Document Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Produce a formal thesis-style system design and implementation document grounded in the current codebase, tests, diagrams, and reproducible evidence.
|
||||
|
||||
**Architecture:** The document should lead with concepts, then support them with verified implementation facts. Claims must be backed by code paths, tests, smoke output, or the runnable case-study bundle; do not invent product properties from aspirational roadmap text.
|
||||
|
||||
**Tech Stack:** Markdown with Pandoc-compatible frontmatter, Mermaid diagrams, PowerShell generation script in `docs/add/generate.ps1`, pytest docs smoke tests, ruff, basedpyright.
|
||||
|
||||
---
|
||||
|
||||
## Dependency
|
||||
|
||||
This plan should run after `docs/superpowers/plans/2026-06-14-thesis-case-study-evidence.md` or after an equivalent report-workflow example exists at `examples/report_workflow/`.
|
||||
|
||||
If `examples/report_workflow/README.md` does not exist, stop and implement the evidence plan first.
|
||||
|
||||
## Files
|
||||
|
||||
- Create: `docs/add/system-design-implementation.md` — the formal thesis/system-design document.
|
||||
- Create: `docs/add/evidence-index.md` — concise map from thesis claims to code/tests/docs evidence.
|
||||
- Modify: `docs/add/diagrams.md` — add stable Mermaid diagrams used by the Big Doc.
|
||||
- Modify: `docs/add/thesis-outline.md` — mark which sections have been transferred to the Big Doc.
|
||||
- Modify: `docs/project_map.md` — link the Big Doc and evidence index from the docs map.
|
||||
- Modify: `docs/current_roadmap.md` — mark the Big Doc draft completed.
|
||||
- Modify or create: `tests/docs/test_big_doc_links.py` — smoke-test links to key evidence files.
|
||||
|
||||
## Claim Verification Rule
|
||||
|
||||
Every factual claim in `docs/add/system-design-implementation.md` must be traceable to one of:
|
||||
|
||||
- a source file under `src/`
|
||||
- a test under `tests/`
|
||||
- a runnable example under `examples/report_workflow/`
|
||||
- an existing live architecture doc such as `docs/source_architecture.md`
|
||||
- a smoke/runbook file that states it is current
|
||||
|
||||
Use these commands before drafting implementation claims:
|
||||
|
||||
```powershell
|
||||
rg 'class WorkflowArtifact|class WorkflowDeployment|class WorkflowRun' src/wf_artifacts -n
|
||||
rg 'class WorkflowApi|WorkflowServer|WorkflowSourceProvider|CapabilitySource' src -n
|
||||
rg 'McpRuntimePool|StatefulMcpRuntime|McpSourceClient' src/wf_sources_mcp src/wf_mcp -n
|
||||
rg 'PythonSourceConfig|wf_sources_python|load_sources' src tests -n
|
||||
rg 'next_actions|diagnostics|validate_deployment|validate_draft' src/wf_api src/wf_cli tests -n
|
||||
```
|
||||
|
||||
Do not cite old plans in `docs/historical/**` as current behavior unless the text explicitly says the citation is historical motivation.
|
||||
|
||||
## Task 1: Build an Evidence Index
|
||||
|
||||
**Files:**
|
||||
- Create: `docs/add/evidence-index.md`
|
||||
|
||||
- [ ] **Step 1: Create the evidence index**
|
||||
|
||||
Create `docs/add/evidence-index.md` with this structure:
|
||||
|
||||
```md
|
||||
# Thesis Evidence Index
|
||||
|
||||
This file maps thesis claims to implementation evidence. It is not prose for the
|
||||
final report; it is a guardrail against unsupported claims.
|
||||
|
||||
## Core Workflow Lifecycle
|
||||
|
||||
Claim: The platform separates mutable drafts, immutable artifacts, deployments,
|
||||
runs, and traces.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `src/wf_artifacts/models.py` — artifact/deployment models.
|
||||
- `src/wf_artifacts/runs/` — run records and run store.
|
||||
- `src/wf_api/service.py` — facade for workflow lifecycle operations.
|
||||
- `tests/wf_api/test_artifact_api.py`
|
||||
- `tests/wf_api/test_run_api.py`
|
||||
|
||||
## Source Provider Boundary
|
||||
|
||||
Claim: Workflow execution consumes source-provided capabilities without making
|
||||
the core runtime MCP-specific.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `src/wf_platform/sources.py` — neutral source DTOs and source policy.
|
||||
- `src/wf_server/config.py` — server composition for configured sources.
|
||||
- `src/wf_sources_mcp/` — MCP source family.
|
||||
- `src/wf_sources_python/` — Python source family.
|
||||
- `docs/source_architecture.md`
|
||||
|
||||
## Agent-Operable Surface
|
||||
|
||||
Claim: External agents can operate the workflow lifecycle through stable CLI/API
|
||||
surfaces.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `src/wf_cli/`
|
||||
- `src/wf_transport_rpc_http/`
|
||||
- `tests/wf_cli/`
|
||||
- `tests/wf_transport_rpc_http/`
|
||||
- `docs/wf_cli.md`
|
||||
|
||||
## Validation And Diagnostics
|
||||
|
||||
Claim: Validation and diagnostics make failed workflow states repairable.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `src/wf_artifacts/validation.py`
|
||||
- `src/wf_api/next_actions.py`
|
||||
- `src/wf_api/source_admin.py`
|
||||
- `tests/artifacts/test_validation.py`
|
||||
- `tests/wf_api/test_source_admin_api.py`
|
||||
|
||||
## Stateful MCP Source Correctness
|
||||
|
||||
Claim: MCP-backed sources can preserve stateful sessions across workflow calls.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `src/wf_sources_mcp/runtime/`
|
||||
- `src/wf_sources_mcp/client/`
|
||||
- `tests/wf_sources_mcp/test_runtime.py`
|
||||
- `tests/wf_transport_rpc_http/test_mcp_backed_server_rpc.py`
|
||||
|
||||
## Python Source Case Study
|
||||
|
||||
Claim: The source-provider model is not MCP-only.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `examples/report_workflow/`
|
||||
- `src/wf_sources_python/`
|
||||
- `tests/examples/test_report_workflow_example.py`
|
||||
- `tests/wf_sources_python/test_loader.py`
|
||||
|
||||
## Limitations
|
||||
|
||||
Claim: This is a prototype platform substrate, not a finished automation product.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `docs/add/thesis-outline.md`
|
||||
- `docs/current_roadmap.md`
|
||||
- absence of scheduler/visual-editor/secret-manager production packages in
|
||||
current source tree.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify evidence paths exist**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
Test-Path docs/add/evidence-index.md
|
||||
Test-Path src/wf_artifacts/models.py
|
||||
Test-Path src/wf_platform/sources.py
|
||||
Test-Path src/wf_sources_mcp/runtime
|
||||
Test-Path src/wf_sources_python
|
||||
Test-Path examples/report_workflow
|
||||
```
|
||||
|
||||
Expected: all commands print `True`. If `examples/report_workflow` prints `False`, stop and implement the case-study evidence plan first.
|
||||
|
||||
- [ ] **Step 3: Commit evidence index**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git add docs/add/evidence-index.md
|
||||
git commit -m "docs: add thesis evidence index"
|
||||
```
|
||||
|
||||
Expected: commit succeeds.
|
||||
|
||||
## Task 2: Add Stable Mermaid Diagrams
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/add/diagrams.md`
|
||||
|
||||
- [ ] **Step 1: Add workflow core diagram**
|
||||
|
||||
Append this section to `docs/add/diagrams.md`:
|
||||
|
||||
````md
|
||||
## Workflow Core
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Input[Input Schema] --> NodeUse[Node Use]
|
||||
NodeSpec[NodeSpec Contract] --> NodeUse
|
||||
NodeUse --> Outcome[Declared Outcome]
|
||||
Outcome --> Edge[Graph Edge]
|
||||
NodeUse --> StateWrite[State Write]
|
||||
StateWrite --> Reducer[Reducer]
|
||||
Reducer --> State[Workflow State]
|
||||
NodeUse --> Trace[Trace Frame]
|
||||
Interrupt[Interrupt] --> RunState[Stopped Run State]
|
||||
RunState --> Resume[Resume]
|
||||
```
|
||||
````
|
||||
|
||||
- [ ] **Step 2: Add platform domain diagram**
|
||||
|
||||
Append this section to `docs/add/diagrams.md`:
|
||||
|
||||
````md
|
||||
## Platform Domain
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Draft[Draft Workspace] --> DraftValidation[Draft Validation]
|
||||
DraftValidation --> Artifact[Workflow Artifact]
|
||||
Artifact --> Deployment[Workflow Deployment]
|
||||
SourceInventory[Source Inventory] --> DeploymentValidation[Deployment Validation]
|
||||
Deployment --> DeploymentValidation
|
||||
DeploymentValidation --> Run[Workflow Run]
|
||||
Run --> RunRecord[Run Record]
|
||||
Run --> Trace[Trace Slice]
|
||||
RunRecord --> Resume[Resume]
|
||||
DeploymentValidation --> Diagnostics[Repairable Diagnostics]
|
||||
```
|
||||
````
|
||||
|
||||
- [ ] **Step 3: Add source provider diagram**
|
||||
|
||||
Append this section to `docs/add/diagrams.md`:
|
||||
|
||||
````md
|
||||
## Source Provider Boundary
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Config[Workflow Config Sources] --> Server[WorkflowServer Composition]
|
||||
Server --> Builtin[Platform Sources]
|
||||
Server --> MCP[MCP Source Provider]
|
||||
Server --> Python[Python Source Provider]
|
||||
Builtin --> Inventory[CapabilitySource Inventory]
|
||||
MCP --> Inventory
|
||||
Python --> Inventory
|
||||
Inventory --> API[Workflow API Surface]
|
||||
API --> Runtime[Workflow Runtime]
|
||||
```
|
||||
````
|
||||
|
||||
- [ ] **Step 4: Commit diagrams**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git add docs/add/diagrams.md
|
||||
git commit -m "docs: add thesis diagrams"
|
||||
```
|
||||
|
||||
Expected: commit succeeds.
|
||||
|
||||
## Task 3: Draft the Formal Big Doc
|
||||
|
||||
**Files:**
|
||||
- Create: `docs/add/system-design-implementation.md`
|
||||
|
||||
- [ ] **Step 1: Create frontmatter and introduction**
|
||||
|
||||
Create `docs/add/system-design-implementation.md` with this frontmatter and opening:
|
||||
|
||||
```md
|
||||
---
|
||||
title: "Design and Implementation of lda.chat"
|
||||
subtitle: "Infrastructure for AI Agents to Author and Execute Workspace Workflows"
|
||||
author: "draft"
|
||||
date: "2026-06-14"
|
||||
lang: "en-US"
|
||||
documentclass: report
|
||||
papersize: a4
|
||||
fontsize: 10pt
|
||||
toc: true
|
||||
toc-depth: 2
|
||||
numbersections: true
|
||||
geometry:
|
||||
- top=30mm
|
||||
- bottom=30mm
|
||||
- left=32mm
|
||||
- right=32mm
|
||||
mainfont: "Libertinus Serif"
|
||||
sansfont: "Libertinus Sans"
|
||||
monofont: "Libertinus Mono"
|
||||
mathfont: "Libertinus Math"
|
||||
colorlinks: true
|
||||
linkcolor: "MidnightBlue"
|
||||
urlcolor: "MidnightBlue"
|
||||
toccolor: "MidnightBlue"
|
||||
keywords:
|
||||
- workflow
|
||||
- agents
|
||||
- source providers
|
||||
- JSON-RPC
|
||||
- MCP
|
||||
- Python sources
|
||||
header-includes:
|
||||
- \usepackage{graphicx}
|
||||
- \usepackage{booktabs}
|
||||
- \usepackage{hyperref}
|
||||
- \usepackage{hyperxmp}
|
||||
- \usepackage[dvipsnames]{xcolor}
|
||||
- \usepackage{fancyhdr}
|
||||
- \pagestyle{fancy}
|
||||
- \fancyhead[L]{\small lda.chat}
|
||||
- \fancyhead[R]{\small\leftmark}
|
||||
- \fancyfoot[C]{\thepage}
|
||||
- \setlength{\parskip}{0.6em}
|
||||
- \setlength{\parindent}{0pt}
|
||||
- \setkeys{Gin}{width=\linewidth,height=0.55\textheight,keepaspectratio}
|
||||
- \renewcommand{\arraystretch}{1.3}
|
||||
- \hypersetup{pdfauthor={lda.chat}, pdftitle={Design and Implementation of lda.chat}}
|
||||
---
|
||||
|
||||
# Introduction
|
||||
|
||||
External LLM agents are useful workflow authors and operators, but durable
|
||||
workspace automation needs a typed execution substrate. This report describes
|
||||
the design and implementation of `lda.chat`, a prototype platform where agents
|
||||
can author, validate, execute, and inspect reusable workspace workflows without
|
||||
making the LLM itself responsible for runtime state, validation, source binding,
|
||||
or persistence.
|
||||
|
||||
The central claim is that agent-facing workflow automation should separate
|
||||
planning from execution. The LLM or human author can propose and revise workflow
|
||||
structure, while the platform owns artifacts, deployments, runs, source
|
||||
inventory, validation diagnostics, traces, and resumability.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add the required section skeleton**
|
||||
|
||||
Append these headings to `docs/add/system-design-implementation.md`:
|
||||
|
||||
```md
|
||||
|
||||
# Problem Statement And Requirements
|
||||
|
||||
# Positioning And Related Systems
|
||||
|
||||
# Conceptual Model
|
||||
|
||||
# System Architecture
|
||||
|
||||
# Implementation
|
||||
|
||||
# Case Study: Deterministic Report Workflow
|
||||
|
||||
# Evaluation
|
||||
|
||||
# Limitations
|
||||
|
||||
# Future Work
|
||||
|
||||
# Conclusion
|
||||
|
||||
# Appendices
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Fill sections from the outline**
|
||||
|
||||
Use `docs/add/thesis-outline.md` as the source for section content. Copy ideas, not whole notes. Required rules:
|
||||
|
||||
- Keep package names secondary to concepts.
|
||||
- Include the main architecture Mermaid diagram in `# System Architecture`.
|
||||
- Include workflow lifecycle and source provider diagrams where they explain the text.
|
||||
- Include the report-workflow example in `# Case Study`.
|
||||
- Keep long command transcripts in `# Appendices`.
|
||||
- Include no claim about production secret management, production reliability, broad user study, full MCP frontend, MCP widget proxying, visual editor, or scheduler.
|
||||
|
||||
- [ ] **Step 4: Insert claim references**
|
||||
|
||||
Add short parenthetical evidence references in prose, using this style:
|
||||
|
||||
```md
|
||||
(Evidence: `src/wf_artifacts/models.py`, `tests/wf_api/test_run_api.py`.)
|
||||
```
|
||||
|
||||
Minimum required evidence references:
|
||||
|
||||
- workflow lifecycle models/tests
|
||||
- source provider boundary
|
||||
- JSON-RPC/CLI surface
|
||||
- Python source case study
|
||||
- MCP stateful runtime tests
|
||||
- validation/diagnostics tests
|
||||
|
||||
- [ ] **Step 5: Commit Big Doc draft**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git add docs/add/system-design-implementation.md
|
||||
git commit -m "docs: draft thesis system design document"
|
||||
```
|
||||
|
||||
Expected: commit succeeds.
|
||||
|
||||
## Task 4: Add Link Tests And Documentation Map Entries
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/docs/test_big_doc_links.py`
|
||||
- Modify: `docs/project_map.md`
|
||||
- Modify: `docs/current_roadmap.md`
|
||||
|
||||
- [ ] **Step 1: Add docs link test**
|
||||
|
||||
Create `tests/docs/test_big_doc_links.py` with this test module:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def test_big_doc_links_case_study_and_evidence_index() -> None:
|
||||
doc = (ROOT / "docs" / "add" / "system-design-implementation.md").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
|
||||
assert "examples/report_workflow" in doc
|
||||
assert "docs/add/evidence-index.md" in doc or "evidence-index.md" in doc
|
||||
|
||||
|
||||
def test_project_map_links_big_doc() -> None:
|
||||
project_map = (ROOT / "docs" / "project_map.md").read_text(encoding="utf-8")
|
||||
|
||||
assert "system-design-implementation.md" in project_map
|
||||
assert "evidence-index.md" in project_map
|
||||
|
||||
|
||||
def test_big_doc_keeps_mcp_as_source_family() -> None:
|
||||
doc = (ROOT / "docs" / "add" / "system-design-implementation.md").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
|
||||
assert "MCP" in doc
|
||||
assert "source family" in doc
|
||||
assert "product identity" in doc
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update project map**
|
||||
|
||||
In `docs/project_map.md`, add a docs/add entry:
|
||||
|
||||
```md
|
||||
- `docs/add/system-design-implementation.md` — formal thesis/system-design draft.
|
||||
- `docs/add/evidence-index.md` — claim-to-evidence map for the thesis draft.
|
||||
```
|
||||
|
||||
Place it near existing docs/add or architecture document entries.
|
||||
|
||||
- [ ] **Step 3: Update roadmap**
|
||||
|
||||
In `docs/current_roadmap.md`, add:
|
||||
|
||||
```md
|
||||
- Completed thesis system-design draft: `docs/add/system-design-implementation.md`
|
||||
now frames the platform as a formal system design/implementation report backed
|
||||
by `docs/add/evidence-index.md` and the report-workflow case study.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run docs tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/docs/test_big_doc_links.py -q
|
||||
```
|
||||
|
||||
Expected: `3 passed`.
|
||||
|
||||
- [ ] **Step 5: Commit docs links**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git add tests/docs/test_big_doc_links.py docs/project_map.md docs/current_roadmap.md
|
||||
git commit -m "docs: link thesis system design draft"
|
||||
```
|
||||
|
||||
Expected: commit succeeds.
|
||||
|
||||
## Task 5: Generate HTML/PDF Smoke Outputs
|
||||
|
||||
**Files:**
|
||||
- Read: `docs/add/generate.ps1`
|
||||
- Generate locally: `docs/add/system-design-implementation.html`
|
||||
- Generate locally: `docs/add/system-design-implementation.pdf`
|
||||
|
||||
- [ ] **Step 1: Generate HTML**
|
||||
|
||||
Run from `docs/add`:
|
||||
|
||||
```powershell
|
||||
.\generate.ps1 -type html -- system-design-implementation.md -o system-design-implementation.html
|
||||
```
|
||||
|
||||
Expected: command exits `0` and creates `docs/add/system-design-implementation.html`.
|
||||
|
||||
- [ ] **Step 2: Generate PDF**
|
||||
|
||||
Run from `docs/add`:
|
||||
|
||||
```powershell
|
||||
.\generate.ps1 -type pdf -- system-design-implementation.md -o system-design-implementation.pdf
|
||||
```
|
||||
|
||||
Expected: command exits `0` and creates `docs/add/system-design-implementation.pdf`.
|
||||
|
||||
- [ ] **Step 3: Decide whether generated files are tracked**
|
||||
|
||||
Check `.gitignore` and current tracking:
|
||||
|
||||
```powershell
|
||||
git ls-files docs/add/*.pdf docs/add/*.html
|
||||
git status --short docs/add
|
||||
```
|
||||
|
||||
If generated HTML/PDF are already tracked in this folder, add them. If not tracked or ignored, leave them uncommitted and mention generation success in the report.
|
||||
|
||||
- [ ] **Step 4: Commit generated outputs only if tracked**
|
||||
|
||||
If tracked outputs changed, run:
|
||||
|
||||
```powershell
|
||||
git add docs/add/system-design-implementation.html docs/add/system-design-implementation.pdf
|
||||
git commit -m "docs: generate thesis system design outputs"
|
||||
```
|
||||
|
||||
If files are ignored/untracked, do not commit this task.
|
||||
|
||||
## Task 6: Final Verification And Archive Plan
|
||||
|
||||
**Files:**
|
||||
- Verify all changed files.
|
||||
- Move this plan to historical after completion.
|
||||
|
||||
- [ ] **Step 1: Run docs tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run pytest tests/docs tests/examples -q
|
||||
```
|
||||
|
||||
Expected: all selected docs/example tests pass.
|
||||
|
||||
- [ ] **Step 2: Run ruff on tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run ruff check tests/docs tests/examples
|
||||
```
|
||||
|
||||
Expected: `All checks passed!`
|
||||
|
||||
- [ ] **Step 3: Run typecheck on tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
uv run basedpyright --level error tests/docs tests/examples
|
||||
```
|
||||
|
||||
Expected: `0 errors`.
|
||||
|
||||
- [ ] **Step 4: Run whitespace check**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: no whitespace errors. CRLF warnings on Windows are acceptable.
|
||||
|
||||
- [ ] **Step 5: Archive this plan**
|
||||
|
||||
Move this file to:
|
||||
|
||||
```text
|
||||
docs/historical/superpowers/plans/2026-06-14-thesis-system-design-doc.md
|
||||
```
|
||||
|
||||
Commit:
|
||||
|
||||
```powershell
|
||||
git add docs/superpowers/plans/2026-06-14-thesis-system-design-doc.md docs/historical/superpowers/plans/2026-06-14-thesis-system-design-doc.md
|
||||
git commit -m "docs: archive thesis system design plan"
|
||||
```
|
||||
|
||||
Expected: commit succeeds.
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- The Big Doc is thesis/system-design-first, not a presentation deck.
|
||||
- The Big Doc cites code/tests/examples for implementation claims.
|
||||
- MCP is described as a source family, not product identity.
|
||||
- Google Drive MCP is not required for thesis-critical evidence.
|
||||
- Diagrams appear before package-heavy implementation detail.
|
||||
- Limitations are explicit and not hidden in future work.
|
||||
- Long command transcripts are in appendices or linked runbooks.
|
||||
- No current docs introduce `wf.std=wf.std` deployment self-bindings.
|
||||
@@ -4,7 +4,9 @@ Date: 2026-06-09
|
||||
|
||||
Historical note: this smoke captured older platform-source behavior. Current
|
||||
deployments no longer need self-bindings such as `wf.std=wf.std` for built-in
|
||||
platform sources.
|
||||
platform sources, and current validation rejects explicit platform-source
|
||||
bindings. The two rows below mentioning `wf.std=wf.std` are preserved as
|
||||
historical output, not current guidance.
|
||||
|
||||
Target:
|
||||
|
||||
@@ -39,8 +41,8 @@ The remote CLI path is usable end-to-end:
|
||||
| `wf --url ... draft create-from-capability smoke_ws_20260609 wf.std.constant ...` | OK; valid draft, high-confidence wrapper hints. |
|
||||
| `wf --url ... draft validate smoke_ws_20260609` | OK; valid. |
|
||||
| `wf --url ... draft inspect smoke_ws_20260609 --include-draft` | OK; full draft returned. |
|
||||
| `wf --url ... draft save smoke_ws_20260609 --artifact smoke_artifact_20260609 --version 1 ...` | OK; artifact saved, suggested binding `wf.std=wf.std`. |
|
||||
| `wf --url ... deploy save smoke_deploy_20260609 --artifact smoke_artifact_20260609 --version 1 --binding wf.std=wf.std` | OK. |
|
||||
| `wf --url ... draft save smoke_ws_20260609 --artifact smoke_artifact_20260609 --version 1 ...` | Historical: artifact saved and suggested `wf.std=wf.std`; current behavior should not suggest platform bindings. |
|
||||
| `wf --url ... deploy save smoke_deploy_20260609 --artifact smoke_artifact_20260609 --version 1 --binding wf.std=wf.std` | Historical: accepted then; current validation rejects explicit platform bindings. |
|
||||
| `wf --url ... deploy validate smoke_deploy_20260609` | OK; `status: runnable`. |
|
||||
| `wf --url ... run start smoke_deploy_20260609 --input '{"value":"remote lifecycle smoke"}'` | OK; completed with expected output. |
|
||||
| `wf --url ... run inspect run_7afdda9f958a4c258866192a78d1ef6b` | OK; compact completed run summary. |
|
||||
|
||||
@@ -22,6 +22,26 @@ def validate_deployment_dependencies(
|
||||
bindings = deployment.binding_map()
|
||||
diagnostics: list[DependencyDiagnostic] = []
|
||||
|
||||
for logical_source, concrete_source in bindings.items():
|
||||
source = sources_by_id.get(logical_source)
|
||||
if source is not None and source.platform:
|
||||
diagnostics.append(
|
||||
DependencyDiagnostic(
|
||||
severity=DiagnosticSeverity.ERROR,
|
||||
code="platform_binding_forbidden",
|
||||
logical_ref=logical_source,
|
||||
bound_source=concrete_source,
|
||||
message=(
|
||||
f"Platform source {logical_source!r} has a fixed runtime "
|
||||
"identity and cannot be deployment-bound."
|
||||
),
|
||||
repair_hint=(
|
||||
"Remove this binding. Platform sources resolve by their "
|
||||
"source id without deployment bindings."
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
for logical_ref, required in artifact.required_capability_map().items():
|
||||
platform_source = sources_by_id.get(required.logical_source)
|
||||
if platform_source is not None and platform_source.platform:
|
||||
|
||||
@@ -234,6 +234,38 @@ def test_platform_source_requirement_does_not_need_binding() -> None:
|
||||
assert diagnostics == []
|
||||
|
||||
|
||||
def test_platform_source_rejects_explicit_deployment_binding() -> None:
|
||||
artifact = artifact_with(
|
||||
required_capability(logical_source="wf.std", capability_name="replace")
|
||||
)
|
||||
deployment = WorkflowDeployment(
|
||||
id="demo.default",
|
||||
artifact_id=artifact.id,
|
||||
artifact_version=artifact.version,
|
||||
bindings={"wf.std": "custom.std"},
|
||||
)
|
||||
|
||||
diagnostics = validate_deployment_dependencies(
|
||||
artifact=artifact,
|
||||
deployment=deployment,
|
||||
sources=[
|
||||
AvailableSource(
|
||||
id="wf.std",
|
||||
platform=True,
|
||||
capabilities={
|
||||
"replace": AvailableCapability(name="replace", kind="node_spec")
|
||||
},
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
assert [diagnostic.code for diagnostic in diagnostics] == [
|
||||
"platform_binding_forbidden"
|
||||
]
|
||||
assert diagnostics[0].logical_ref == "wf.std"
|
||||
assert diagnostics[0].bound_source == "custom.std"
|
||||
|
||||
|
||||
def test_missing_platform_source_still_reports_binding_missing() -> None:
|
||||
artifact = artifact_with(
|
||||
required_capability(logical_source="wf.std", capability_name="replace")
|
||||
|
||||
@@ -665,8 +665,6 @@ def test_wf_remote_draft_artifact_deploy_lifecycle(monkeypatch, tmp_path) -> Non
|
||||
"Remote Artifact",
|
||||
"--outcome",
|
||||
"ok",
|
||||
"--binding",
|
||||
"wf.std=wf.std",
|
||||
],
|
||||
)
|
||||
assert saved_artifact.exit_code == 0, saved_artifact.output
|
||||
@@ -690,8 +688,6 @@ def test_wf_remote_draft_artifact_deploy_lifecycle(monkeypatch, tmp_path) -> Non
|
||||
"remote_artifact",
|
||||
"--version",
|
||||
"1",
|
||||
"--binding",
|
||||
"wf.std=wf.std",
|
||||
],
|
||||
)
|
||||
assert saved_deployment.exit_code == 0, saved_deployment.output
|
||||
@@ -774,7 +770,7 @@ def test_wf_status_uses_rpc_url_override(monkeypatch, tmp_path) -> None:
|
||||
title="Status Constant",
|
||||
plan=_constant_plan(),
|
||||
outcomes=("ok",),
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
)
|
||||
asyncio.run(
|
||||
@@ -783,7 +779,7 @@ def test_wf_status_uses_rpc_url_override(monkeypatch, tmp_path) -> None:
|
||||
"id": "status_constant.default",
|
||||
"artifact_id": "status_constant",
|
||||
"artifact_version": 1,
|
||||
"bindings": [{"logical_source": "wf.std", "concrete_source": "wf.std"}],
|
||||
"bindings": {},
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
@@ -84,14 +84,14 @@ async def test_local_static_server_runs_deployment_and_persists_run(tmp_path) ->
|
||||
title="Server Constant",
|
||||
plan=plan,
|
||||
outcomes=["ok"],
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
deployment_result = await api.save_deployment(
|
||||
{
|
||||
"id": "server_constant.default",
|
||||
"artifact_id": "server_constant",
|
||||
"artifact_version": 1,
|
||||
"bindings": [{"logical_source": "wf.std", "concrete_source": "wf.std"}],
|
||||
"bindings": {},
|
||||
}
|
||||
)
|
||||
run_result = await api.run_deployment(
|
||||
@@ -119,14 +119,14 @@ async def test_local_static_server_inspects_and_reads_bounded_trace(tmp_path) ->
|
||||
title="Server Trace",
|
||||
plan=plan.model_copy(update={"name": "server_trace"}),
|
||||
outcomes=["ok"],
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
await api.save_deployment(
|
||||
{
|
||||
"id": "server_trace.default",
|
||||
"artifact_id": "server_trace",
|
||||
"artifact_version": 1,
|
||||
"bindings": [{"logical_source": "wf.std", "concrete_source": "wf.std"}],
|
||||
"bindings": {},
|
||||
}
|
||||
)
|
||||
run_result = await api.run_deployment(
|
||||
|
||||
@@ -175,7 +175,7 @@ async def test_rpc_draft_artifact_deployment_lifecycle(tmp_path) -> None:
|
||||
},
|
||||
"outcomes": ["ok"],
|
||||
"required_capabilities": {},
|
||||
"source_bindings": {"wf.std": "wf.std"},
|
||||
"source_bindings": {},
|
||||
"plan": compiled_plan,
|
||||
},
|
||||
},
|
||||
@@ -188,9 +188,7 @@ async def test_rpc_draft_artifact_deployment_lifecycle(tmp_path) -> None:
|
||||
"id": "constant_rpc.default",
|
||||
"artifact_id": "constant_rpc",
|
||||
"artifact_version": 1,
|
||||
"bindings": [
|
||||
{"logical_source": "wf.std", "concrete_source": "wf.std"}
|
||||
],
|
||||
"bindings": {},
|
||||
},
|
||||
},
|
||||
)
|
||||
@@ -215,14 +213,14 @@ async def test_rpc_artifact_and_deployment_catalog_methods(tmp_path) -> None:
|
||||
title="RPC Lifecycle",
|
||||
plan=_constant_plan(),
|
||||
outcomes=["ok"],
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
await server.api.save_deployment(
|
||||
{
|
||||
"id": "rpc_lifecycle.default",
|
||||
"artifact_id": "rpc_lifecycle",
|
||||
"artifact_version": 1,
|
||||
"bindings": [{"logical_source": "wf.std", "concrete_source": "wf.std"}],
|
||||
"bindings": {},
|
||||
}
|
||||
)
|
||||
|
||||
@@ -303,7 +301,7 @@ async def test_rpc_draft_workspace_methods(tmp_path) -> None:
|
||||
"title": "Remote Artifact",
|
||||
"outcomes": ["ok"],
|
||||
"kind": "workflow",
|
||||
"source_bindings": {"wf.std": "wf.std"},
|
||||
"source_bindings": {},
|
||||
},
|
||||
)
|
||||
|
||||
@@ -346,7 +344,7 @@ async def test_rpc_artifact_delete(tmp_path) -> None:
|
||||
title="Delete Me",
|
||||
plan=_constant_plan(),
|
||||
outcomes=["ok"],
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
|
||||
app = create_rpc_app(server)
|
||||
@@ -419,14 +417,14 @@ async def test_rpc_runs_deployment_and_reads_bounded_trace(tmp_path) -> None:
|
||||
title="RPC Constant",
|
||||
plan=_constant_plan(),
|
||||
outcomes=["ok"],
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
await server.api.save_deployment(
|
||||
{
|
||||
"id": "rpc_constant.default",
|
||||
"artifact_id": "rpc_constant",
|
||||
"artifact_version": 1,
|
||||
"bindings": [{"logical_source": "wf.std", "concrete_source": "wf.std"}],
|
||||
"bindings": {},
|
||||
}
|
||||
)
|
||||
|
||||
@@ -473,14 +471,14 @@ async def test_rpc_run_list_method(tmp_path) -> None:
|
||||
title="List Runs RPC",
|
||||
plan=_constant_plan(),
|
||||
outcomes=["ok"],
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
await server.api.save_deployment(
|
||||
{
|
||||
"id": "list_runs_rpc.default",
|
||||
"artifact_id": "list_runs_rpc",
|
||||
"artifact_version": 1,
|
||||
"bindings": [{"logical_source": "wf.std", "concrete_source": "wf.std"}],
|
||||
"bindings": {},
|
||||
}
|
||||
)
|
||||
started = await server.api.run_deployment(
|
||||
@@ -584,7 +582,7 @@ async def test_rpc_runs_workflow_from_python_source_capability(tmp_path) -> None
|
||||
"title": "Python Echo",
|
||||
"outcomes": ["ok"],
|
||||
"kind": "workflow",
|
||||
"source_bindings": {"local.ops": "local.ops", "wf.std": "wf.std"},
|
||||
"source_bindings": {"local.ops": "local.ops"},
|
||||
},
|
||||
)
|
||||
deployment = await _rpc(
|
||||
@@ -597,7 +595,6 @@ async def test_rpc_runs_workflow_from_python_source_capability(tmp_path) -> None
|
||||
"artifact_version": 1,
|
||||
"bindings": [
|
||||
{"logical_source": "local.ops", "concrete_source": "local.ops"},
|
||||
{"logical_source": "wf.std", "concrete_source": "wf.std"},
|
||||
],
|
||||
}
|
||||
},
|
||||
|
||||
@@ -145,14 +145,14 @@ async def test_rpc_workflow_client_runs_and_reads_trace(tmp_path) -> None:
|
||||
title="Client Constant",
|
||||
plan=_constant_plan(),
|
||||
outcomes=["ok"],
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
await server.api.save_deployment(
|
||||
{
|
||||
"id": "client_constant.default",
|
||||
"artifact_id": "client_constant",
|
||||
"artifact_version": 1,
|
||||
"bindings": [{"logical_source": "wf.std", "concrete_source": "wf.std"}],
|
||||
"bindings": {},
|
||||
}
|
||||
)
|
||||
app = create_rpc_app(server)
|
||||
@@ -215,7 +215,7 @@ async def test_rpc_workflow_client_lists_and_inspects_artifacts(tmp_path) -> Non
|
||||
title="Client Art",
|
||||
plan=_constant_plan(),
|
||||
outcomes=["ok"],
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
app = create_rpc_app(server)
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
@@ -242,14 +242,14 @@ async def test_rpc_workflow_client_lists_inspects_validates_and_deletes_deployme
|
||||
title="Client Deploy Art",
|
||||
plan=_constant_plan(),
|
||||
outcomes=["ok"],
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
await server.api.save_deployment(
|
||||
{
|
||||
"id": "client_deploy_art.default",
|
||||
"artifact_id": "client_deploy_art",
|
||||
"artifact_version": 1,
|
||||
"bindings": [{"logical_source": "wf.std", "concrete_source": "wf.std"}],
|
||||
"bindings": {},
|
||||
}
|
||||
)
|
||||
app = create_rpc_app(server)
|
||||
@@ -310,7 +310,7 @@ async def test_rpc_workflow_client_draft_workspace_lifecycle(tmp_path) -> None:
|
||||
title="Client WS Art",
|
||||
outcomes=("ok",),
|
||||
kind="workflow",
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
|
||||
assert created["workspace_id"] == "client_ws"
|
||||
@@ -357,7 +357,7 @@ async def test_rpc_workflow_client_deletes_artifact(tmp_path) -> None:
|
||||
title="Delete Me",
|
||||
plan=_constant_plan(),
|
||||
outcomes=["ok"],
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
|
||||
app = create_rpc_app(server)
|
||||
@@ -383,14 +383,14 @@ async def test_rpc_client_lists_runs(tmp_path) -> None:
|
||||
title="Client List Runs",
|
||||
plan=_constant_plan(),
|
||||
outcomes=["ok"],
|
||||
source_bindings={"wf.std": "wf.std"},
|
||||
source_bindings={},
|
||||
)
|
||||
await server.api.save_deployment(
|
||||
{
|
||||
"id": "client_list_runs.default",
|
||||
"artifact_id": "client_list_runs",
|
||||
"artifact_version": 1,
|
||||
"bindings": [{"logical_source": "wf.std", "concrete_source": "wf.std"}],
|
||||
"bindings": {},
|
||||
}
|
||||
)
|
||||
started = await server.api.run_deployment(
|
||||
|
||||
@@ -476,7 +476,6 @@ async def test_mcp_backed_rpc_workflow_reuses_runtime_session_across_runs(
|
||||
"logical_source": "fixture.default",
|
||||
"concrete_source": "fixture.default",
|
||||
},
|
||||
{"logical_source": "wf.std", "concrete_source": "wf.std"},
|
||||
],
|
||||
}
|
||||
)
|
||||
@@ -569,7 +568,6 @@ async def test_mcp_backed_rpc_workflow_reuses_runtime_session_direct_setup(
|
||||
"logical_source": "fixture.default",
|
||||
"concrete_source": "fixture.default",
|
||||
},
|
||||
{"logical_source": "wf.std", "concrete_source": "wf.std"},
|
||||
],
|
||||
}
|
||||
)
|
||||
@@ -665,7 +663,6 @@ async def test_mcp_backed_rpc_deployment_becomes_unrunnable_after_source_removed
|
||||
"logical_source": "fixture.default",
|
||||
"concrete_source": "fixture.default",
|
||||
},
|
||||
{"logical_source": "wf.std", "concrete_source": "wf.std"},
|
||||
],
|
||||
}
|
||||
)
|
||||
@@ -752,7 +749,6 @@ async def test_mcp_backed_rpc_workflow_reuses_real_stdio_fixture_session(
|
||||
"logical_source": "fixture.personal",
|
||||
"concrete_source": "fixture.personal",
|
||||
},
|
||||
{"logical_source": "wf.std", "concrete_source": "wf.std"},
|
||||
],
|
||||
}
|
||||
)
|
||||
@@ -781,7 +777,6 @@ async def test_mcp_backed_rpc_workflow_reuses_real_stdio_fixture_session(
|
||||
"logical_source": "fixture.personal",
|
||||
"concrete_source": "fixture.personal",
|
||||
},
|
||||
{"logical_source": "wf.std", "concrete_source": "wf.std"},
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user