docs: add python source runbook

2026-06-12 09:47:53 +07:00
parent 03c388d654
commit dee95c89d2
8 changed files with 280 additions and 0 deletions
@@ -48,6 +48,9 @@ implementation plans are kept for context, not as active instructions.
  lifecycle flow, and common diagnostics.
 - [`runbooks/rpc-cli-smoke.md`](runbooks/rpc-cli-smoke.md): bounded
  end-to-end smoke test for `wf-rpc-server` plus remote `wf` CLI.
+- [`runbooks/python-source.md`](runbooks/python-source.md): configure trusted
+  project-local Python `NodeSpec` registries, validate them, and run them
+  through `wf-rpc-server`.

 ## MCP Platform

@@ -92,6 +92,10 @@ auth admin are implemented. The next work is polish, not new broad surfaces.
  including config-relative path resolution and trusted static Python source
  imports. MCP sources are shape-validated only; live upstream checks remain a
  server/status concern.
+- Completed: Python source operator docs and RPC integration coverage now prove
+  `ops.py` source config, capability call, draft artifact creation, deployment,
+  and workflow run. Runbook:
+  [`Python source`](runbooks/python-source.md).
 - Completed: server startup policy moved to `wf_server.cli`; JSON-RPC HTTP
  remains in `wf_transport_rpc_http`:
  [`server CLI and transport boundary`](superpowers/specs/2026-06-10-server-cli-transport-boundary.md).
@@ -0,0 +1,181 @@
+# Python Source Runbook
+
+This runbook shows how to expose project-local Python `NodeSpec` functions as
+workflow capabilities through `wf-rpc-server`.
+
+Python sources are trusted in-process code. Importing the configured module uses
+normal Python import semantics, so top-level module code can run during
+`wf config validate` and server startup. Keep module top-level work small and
+side-effect free; put real work inside `@node` functions.
+
+## 1. Write `ops.py`
+
+Create a module that exports one or more `NodeSpec` objects. The simplest path
+is to decorate typed functions with `wf_authoring.node` and put them in a
+`registry` list.
+
+```python
+from __future__ import annotations
+
+from pydantic import BaseModel
+
+from wf_authoring import node
+
+
+class EchoInput(BaseModel):
+    text: str
+
+
+class EchoOutput(BaseModel):
+    echoed: str
+
+
+@node(name="echo")
+def echo(payload: EchoInput) -> EchoOutput:
+    return EchoOutput(echoed=payload.text)
+
+
+registry = [echo]
+```
+
+The registry can also be a mapping or a callable returning a sequence/mapping.
+Every exported value must be a `NodeSpec`.
+
+## 2. Configure The Source
+
+Add a `kind: "python"` entry under `server.sources[]`:
+
+```json
+{
+  "version": 1,
+  "client": {
+    "target": {
+      "kind": "rpc_http",
+      "url": "http://127.0.0.1:8766/rpc",
+      "timeout_seconds": 30
+    }
+  },
+  "server": {
+    "store": {"kind": "filesystem", "root": ".wf_python_store"},
+    "transports": [
+      {"kind": "rpc_http", "host": "127.0.0.1", "port": 8766, "path": "/rpc"}
+    ],
+    "sources": [
+      {
+        "kind": "python",
+        "id": "local.ops",
+        "path": ".",
+        "module": "ops",
+        "registry": "registry"
+      }
+    ]
+  }
+}
+```
+
+`path` is resolved relative to the config file. It is important for console
+scripts: `uv run python` and installed entrypoints do not always have the same
+import path.
+
+The source id prefixes local names. A node named `echo` becomes
+`local.ops.echo`. A node named `authoring.echo` is also exposed as
+`local.ops.echo`.
+
+## 3. Validate And Start
+
+Preflight the config:
+
+```powershell
+uv run wf config validate wf.python.config.json
+```
+
+Then start the server:
+
+```powershell
+uv run wf-rpc-server --config wf.python.config.json
+```
+
+If the config includes `client.target`, the CLI can use the config directly:
+
+```powershell
+uv run wf --config wf.python.config.json status
+```
+
+You can also pass the URL explicitly:
+
+```powershell
+uv run wf --url http://127.0.0.1:8766/rpc source list
+```
+
+## 4. Call A Capability
+
+List and call the Python capability:
+
+```powershell
+uv run wf --url http://127.0.0.1:8766/rpc cap list --source local.ops
+uv run wf --url http://127.0.0.1:8766/rpc cap call local.ops.echo --input '{"text":"hello"}'
+```
+
+Expected output includes:
+
+```json
+{
+  "outcome": "ok",
+  "output": {"echoed": "hello"}
+}
+```
+
+## 5. Save And Run A Workflow
+
+Create a draft from the Python capability:
+
+```powershell
+uv run wf --url http://127.0.0.1:8766/rpc draft create-from-capability `
+  python_echo_ws local.ops.echo --name python_echo
+```
+
+Save it as an artifact. Include both the Python source binding and `wf.std`;
+the generated scaffold can use built-in helper nodes such as `wf.std.replace`.
+
+```powershell
+uv run wf --url http://127.0.0.1:8766/rpc draft save python_echo_ws `
+  --artifact python_echo `
+  --version 1 `
+  --title "Python Echo" `
+  --outcome ok `
+  --binding local.ops=local.ops `
+  --binding wf.std=wf.std
+```
+
+Save a deployment with the same bindings:
+
+```powershell
+uv run wf --url http://127.0.0.1:8766/rpc deploy save python_echo.default `
+  --artifact python_echo `
+  --version 1 `
+  --binding local.ops=local.ops `
+  --binding wf.std=wf.std
+```
+
+Run it:
+
+```powershell
+uv run wf --url http://127.0.0.1:8766/rpc run start python_echo.default `
+  --input '{"text":"hello workflow"}'
+```
+
+Expected output includes:
+
+```json
+{
+  "outcome": "ok",
+  "output": {"echoed": "hello workflow"}
+}
+```
+
+## Current Limits
+
+- Python sources are static at server startup; there is no hot reload yet.
+- Python sources are trusted in-process code; there is no sandbox.
+- Source registry mutation/apply support for Python sources is deferred.
+- Reducer exports are deferred until a real source needs them.
@@ -84,6 +84,11 @@ PythonSourceConfig(path, module, registry)
  -> CapabilitySource(kind="python")
 ```

+For a concrete operator flow, see the
+[`Python source runbook`](runbooks/python-source.md). It covers writing `ops.py`,
+configuring `path`/`module`/`registry`, validating the config, calling a
+capability, and running a saved workflow deployment.
+
 ## Built-Ins Versus Configured Sources

 `wf.std` and `wf.recipes` are built-in local sources owned by `wf_api.local_sources`.
@@ -98,6 +103,10 @@ Configured sources are explicit server/operator choices:
 Do not move `wf.std` or `wf.recipes` into `wf_sources_python`. They are not
 operator-configured project sources.

+Generated draft workflows may still require built-in helper sources such as
+`wf.std`; deployment examples should bind both the configured source and any
+built-in requirements reported by validation.
+
 ## `wf_sources_mcp` Internal Layers

 `wf_sources_mcp` is clearer if read from bottom to top:
@@ -126,6 +126,9 @@ Implemented:
  missing modules, missing registries, invalid registry shapes, and duplicate
  specs before server startup.
 - Capability listing/calling works over JSON-RPC.
+- JSON-RPC integration coverage now proves a Python source capability can be
+  converted into a draft artifact, deployed with required source bindings, and
+  run as a workflow.

 Still deferred:

@@ -61,6 +61,9 @@ server startup. MCP sources are shape-validated only; use `wf status`,
 `wf source list`, or `wf deploy validate --live` against a running server for
 live upstream checks.

+For a complete Python-source flow from `ops.py` through deployment/run, see the
+[`Python source runbook`](runbooks/python-source.md).
+
 The old `store_root` field maps to
 `server.store: {"kind": "filesystem", "root": ...}`; old `connections[]` map to
 `server.sources[]` entries with `kind: "mcp"`.
@@ -47,6 +47,8 @@ wf_cli
 - [docs/README.md](docs/README.md): documentation index.
 - [docs/wf_cli.md](docs/wf_cli.md): CLI lifecycle, remote targets, and common
  diagnostics.
+- [docs/runbooks/python-source.md](docs/runbooks/python-source.md): trusted
+  project-local Python source setup and workflow-run flow.
 - [docs/wf_api_architecture.md](docs/wf_api_architecture.md): current API,
  server, transport, and source package boundaries.
 - [docs/project_map.md](docs/project_map.md): package map, entrypoints, examples,
@@ -541,3 +541,78 @@ async def test_rpc_calls_python_source_capability(tmp_path) -> None:
    assert listed["result"]["total"] == 2
    assert called["result"]["outcome"] == "ok"
    assert called["result"]["output"] == {"echoed": "hello python"}
+
+
+async def test_rpc_runs_workflow_from_python_source_capability(tmp_path) -> None:
+    config = WorkflowConfigFile.model_validate(
+        {
+            "version": 1,
+            "server": {
+                "store": {"kind": "filesystem", "root": str(tmp_path / "store")},
+                "sources": [
+                    {
+                        "kind": "python",
+                        "id": "local.ops",
+                        "module": "tests.fixtures.python_source_ops",
+                        "registry": "registry",
+                    }
+                ],
+            },
+        }
+    )
+    server = build_workflow_server_from_workflow_config(config)
+    app = create_rpc_app(server)
+    transport = httpx.ASGITransport(app=app)
+    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
+        await _rpc(
+            client,
+            "workflow.draft_workspaces.create_from_capability",
+            {
+                "workspace_id": "python_echo_ws",
+                "capability_name": "local.ops.echo",
+                "name": "python_echo",
+                "title": "Python Echo",
+            },
+        )
+        artifact = await _rpc(
+            client,
+            "workflow.draft_workspaces.create_artifact",
+            {
+                "workspace_id": "python_echo_ws",
+                "artifact_id": "python_echo",
+                "version": 1,
+                "title": "Python Echo",
+                "outcomes": ["ok"],
+                "kind": "workflow",
+                "source_bindings": {"local.ops": "local.ops", "wf.std": "wf.std"},
+            },
+        )
+        deployment = await _rpc(
+            client,
+            "workflow.deployments.save",
+            {
+                "deployment": {
+                    "id": "python_echo.default",
+                    "artifact_id": "python_echo",
+                    "artifact_version": 1,
+                    "bindings": [
+                        {"logical_source": "local.ops", "concrete_source": "local.ops"},
+                        {"logical_source": "wf.std", "concrete_source": "wf.std"},
+                    ],
+                }
+            },
+        )
+        run = await _rpc(
+            client,
+            "workflow.runs.start",
+            {
+                "deployment_id": "python_echo.default",
+                "workflow_input": {"text": "hello workflow"},
+                "trace_range": {"start": 0, "limit": 5},
+            },
+        )
+
+    assert artifact["result"]["artifact_id"] == "python_echo"
+    assert deployment["result"]["deployment_id"] == "python_echo.default"
+    assert run["result"]["outcome"] == "ok"
+    assert run["result"]["output"] == {"echoed": "hello workflow"}