docs: add python source runbook
This commit is contained in:
@@ -48,6 +48,9 @@ implementation plans are kept for context, not as active instructions.
|
|||||||
lifecycle flow, and common diagnostics.
|
lifecycle flow, and common diagnostics.
|
||||||
- [`runbooks/rpc-cli-smoke.md`](runbooks/rpc-cli-smoke.md): bounded
|
- [`runbooks/rpc-cli-smoke.md`](runbooks/rpc-cli-smoke.md): bounded
|
||||||
end-to-end smoke test for `wf-rpc-server` plus remote `wf` CLI.
|
end-to-end smoke test for `wf-rpc-server` plus remote `wf` CLI.
|
||||||
|
- [`runbooks/python-source.md`](runbooks/python-source.md): configure trusted
|
||||||
|
project-local Python `NodeSpec` registries, validate them, and run them
|
||||||
|
through `wf-rpc-server`.
|
||||||
|
|
||||||
## MCP Platform
|
## MCP Platform
|
||||||
|
|
||||||
|
|||||||
@@ -92,6 +92,10 @@ auth admin are implemented. The next work is polish, not new broad surfaces.
|
|||||||
including config-relative path resolution and trusted static Python source
|
including config-relative path resolution and trusted static Python source
|
||||||
imports. MCP sources are shape-validated only; live upstream checks remain a
|
imports. MCP sources are shape-validated only; live upstream checks remain a
|
||||||
server/status concern.
|
server/status concern.
|
||||||
|
- Completed: Python source operator docs and RPC integration coverage now prove
|
||||||
|
`ops.py` source config, capability call, draft artifact creation, deployment,
|
||||||
|
and workflow run. Runbook:
|
||||||
|
[`Python source`](runbooks/python-source.md).
|
||||||
- Completed: server startup policy moved to `wf_server.cli`; JSON-RPC HTTP
|
- Completed: server startup policy moved to `wf_server.cli`; JSON-RPC HTTP
|
||||||
remains in `wf_transport_rpc_http`:
|
remains in `wf_transport_rpc_http`:
|
||||||
[`server CLI and transport boundary`](superpowers/specs/2026-06-10-server-cli-transport-boundary.md).
|
[`server CLI and transport boundary`](superpowers/specs/2026-06-10-server-cli-transport-boundary.md).
|
||||||
|
|||||||
@@ -0,0 +1,181 @@
|
|||||||
|
# Python Source Runbook
|
||||||
|
|
||||||
|
This runbook shows how to expose project-local Python `NodeSpec` functions as
|
||||||
|
workflow capabilities through `wf-rpc-server`.
|
||||||
|
|
||||||
|
Python sources are trusted in-process code. Importing the configured module uses
|
||||||
|
normal Python import semantics, so top-level module code can run during
|
||||||
|
`wf config validate` and server startup. Keep module top-level work small and
|
||||||
|
side-effect free; put real work inside `@node` functions.
|
||||||
|
|
||||||
|
## 1. Write `ops.py`
|
||||||
|
|
||||||
|
Create a module that exports one or more `NodeSpec` objects. The simplest path
|
||||||
|
is to decorate typed functions with `wf_authoring.node` and put them in a
|
||||||
|
`registry` list.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
from wf_authoring import node
|
||||||
|
|
||||||
|
|
||||||
|
class EchoInput(BaseModel):
|
||||||
|
text: str
|
||||||
|
|
||||||
|
|
||||||
|
class EchoOutput(BaseModel):
|
||||||
|
echoed: str
|
||||||
|
|
||||||
|
|
||||||
|
@node(name="echo")
|
||||||
|
def echo(payload: EchoInput) -> EchoOutput:
|
||||||
|
return EchoOutput(echoed=payload.text)
|
||||||
|
|
||||||
|
|
||||||
|
registry = [echo]
|
||||||
|
```
|
||||||
|
|
||||||
|
The registry can also be a mapping or a callable returning a sequence/mapping.
|
||||||
|
Every exported value must be a `NodeSpec`.
|
||||||
|
|
||||||
|
## 2. Configure The Source
|
||||||
|
|
||||||
|
Add a `kind: "python"` entry under `server.sources[]`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"client": {
|
||||||
|
"target": {
|
||||||
|
"kind": "rpc_http",
|
||||||
|
"url": "http://127.0.0.1:8766/rpc",
|
||||||
|
"timeout_seconds": 30
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"server": {
|
||||||
|
"store": {"kind": "filesystem", "root": ".wf_python_store"},
|
||||||
|
"transports": [
|
||||||
|
{"kind": "rpc_http", "host": "127.0.0.1", "port": 8766, "path": "/rpc"}
|
||||||
|
],
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"kind": "python",
|
||||||
|
"id": "local.ops",
|
||||||
|
"path": ".",
|
||||||
|
"module": "ops",
|
||||||
|
"registry": "registry"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`path` is resolved relative to the config file. It is important for console
|
||||||
|
scripts: `uv run python` and installed entrypoints do not always have the same
|
||||||
|
import path.
|
||||||
|
|
||||||
|
The source id prefixes local names. A node named `echo` becomes
|
||||||
|
`local.ops.echo`. A node named `authoring.echo` is also exposed as
|
||||||
|
`local.ops.echo`.
|
||||||
|
|
||||||
|
## 3. Validate And Start
|
||||||
|
|
||||||
|
Preflight the config:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
uv run wf config validate wf.python.config.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Then start the server:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
uv run wf-rpc-server --config wf.python.config.json
|
||||||
|
```
|
||||||
|
|
||||||
|
If the config includes `client.target`, the CLI can use the config directly:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
uv run wf --config wf.python.config.json status
|
||||||
|
```
|
||||||
|
|
||||||
|
You can also pass the URL explicitly:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
uv run wf --url http://127.0.0.1:8766/rpc source list
|
||||||
|
```
|
||||||
|
|
||||||
|
## 4. Call A Capability
|
||||||
|
|
||||||
|
List and call the Python capability:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
uv run wf --url http://127.0.0.1:8766/rpc cap list --source local.ops
|
||||||
|
uv run wf --url http://127.0.0.1:8766/rpc cap call local.ops.echo --input '{"text":"hello"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected output includes:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"outcome": "ok",
|
||||||
|
"output": {"echoed": "hello"}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 5. Save And Run A Workflow
|
||||||
|
|
||||||
|
Create a draft from the Python capability:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
uv run wf --url http://127.0.0.1:8766/rpc draft create-from-capability `
|
||||||
|
python_echo_ws local.ops.echo --name python_echo
|
||||||
|
```
|
||||||
|
|
||||||
|
Save it as an artifact. Include both the Python source binding and `wf.std`;
|
||||||
|
the generated scaffold can use built-in helper nodes such as `wf.std.replace`.
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
uv run wf --url http://127.0.0.1:8766/rpc draft save python_echo_ws `
|
||||||
|
--artifact python_echo `
|
||||||
|
--version 1 `
|
||||||
|
--title "Python Echo" `
|
||||||
|
--outcome ok `
|
||||||
|
--binding local.ops=local.ops `
|
||||||
|
--binding wf.std=wf.std
|
||||||
|
```
|
||||||
|
|
||||||
|
Save a deployment with the same bindings:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
uv run wf --url http://127.0.0.1:8766/rpc deploy save python_echo.default `
|
||||||
|
--artifact python_echo `
|
||||||
|
--version 1 `
|
||||||
|
--binding local.ops=local.ops `
|
||||||
|
--binding wf.std=wf.std
|
||||||
|
```
|
||||||
|
|
||||||
|
Run it:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
uv run wf --url http://127.0.0.1:8766/rpc run start python_echo.default `
|
||||||
|
--input '{"text":"hello workflow"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected output includes:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"outcome": "ok",
|
||||||
|
"output": {"echoed": "hello workflow"}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Current Limits
|
||||||
|
|
||||||
|
- Python sources are static at server startup; there is no hot reload yet.
|
||||||
|
- Python sources are trusted in-process code; there is no sandbox.
|
||||||
|
- Source registry mutation/apply support for Python sources is deferred.
|
||||||
|
- Reducer exports are deferred until a real source needs them.
|
||||||
@@ -84,6 +84,11 @@ PythonSourceConfig(path, module, registry)
|
|||||||
-> CapabilitySource(kind="python")
|
-> CapabilitySource(kind="python")
|
||||||
```
|
```
|
||||||
|
|
||||||
|
For a concrete operator flow, see the
|
||||||
|
[`Python source runbook`](runbooks/python-source.md). It covers writing `ops.py`,
|
||||||
|
configuring `path`/`module`/`registry`, validating the config, calling a
|
||||||
|
capability, and running a saved workflow deployment.
|
||||||
|
|
||||||
## Built-Ins Versus Configured Sources
|
## Built-Ins Versus Configured Sources
|
||||||
|
|
||||||
`wf.std` and `wf.recipes` are built-in local sources owned by `wf_api.local_sources`.
|
`wf.std` and `wf.recipes` are built-in local sources owned by `wf_api.local_sources`.
|
||||||
@@ -98,6 +103,10 @@ Configured sources are explicit server/operator choices:
|
|||||||
Do not move `wf.std` or `wf.recipes` into `wf_sources_python`. They are not
|
Do not move `wf.std` or `wf.recipes` into `wf_sources_python`. They are not
|
||||||
operator-configured project sources.
|
operator-configured project sources.
|
||||||
|
|
||||||
|
Generated draft workflows may still require built-in helper sources such as
|
||||||
|
`wf.std`; deployment examples should bind both the configured source and any
|
||||||
|
built-in requirements reported by validation.
|
||||||
|
|
||||||
## `wf_sources_mcp` Internal Layers
|
## `wf_sources_mcp` Internal Layers
|
||||||
|
|
||||||
`wf_sources_mcp` is clearer if read from bottom to top:
|
`wf_sources_mcp` is clearer if read from bottom to top:
|
||||||
|
|||||||
@@ -126,6 +126,9 @@ Implemented:
|
|||||||
missing modules, missing registries, invalid registry shapes, and duplicate
|
missing modules, missing registries, invalid registry shapes, and duplicate
|
||||||
specs before server startup.
|
specs before server startup.
|
||||||
- Capability listing/calling works over JSON-RPC.
|
- Capability listing/calling works over JSON-RPC.
|
||||||
|
- JSON-RPC integration coverage now proves a Python source capability can be
|
||||||
|
converted into a draft artifact, deployed with required source bindings, and
|
||||||
|
run as a workflow.
|
||||||
|
|
||||||
Still deferred:
|
Still deferred:
|
||||||
|
|
||||||
|
|||||||
@@ -61,6 +61,9 @@ server startup. MCP sources are shape-validated only; use `wf status`,
|
|||||||
`wf source list`, or `wf deploy validate --live` against a running server for
|
`wf source list`, or `wf deploy validate --live` against a running server for
|
||||||
live upstream checks.
|
live upstream checks.
|
||||||
|
|
||||||
|
For a complete Python-source flow from `ops.py` through deployment/run, see the
|
||||||
|
[`Python source runbook`](runbooks/python-source.md).
|
||||||
|
|
||||||
The old `store_root` field maps to
|
The old `store_root` field maps to
|
||||||
`server.store: {"kind": "filesystem", "root": ...}`; old `connections[]` map to
|
`server.store: {"kind": "filesystem", "root": ...}`; old `connections[]` map to
|
||||||
`server.sources[]` entries with `kind: "mcp"`.
|
`server.sources[]` entries with `kind: "mcp"`.
|
||||||
|
|||||||
@@ -47,6 +47,8 @@ wf_cli
|
|||||||
- [docs/README.md](docs/README.md): documentation index.
|
- [docs/README.md](docs/README.md): documentation index.
|
||||||
- [docs/wf_cli.md](docs/wf_cli.md): CLI lifecycle, remote targets, and common
|
- [docs/wf_cli.md](docs/wf_cli.md): CLI lifecycle, remote targets, and common
|
||||||
diagnostics.
|
diagnostics.
|
||||||
|
- [docs/runbooks/python-source.md](docs/runbooks/python-source.md): trusted
|
||||||
|
project-local Python source setup and workflow-run flow.
|
||||||
- [docs/wf_api_architecture.md](docs/wf_api_architecture.md): current API,
|
- [docs/wf_api_architecture.md](docs/wf_api_architecture.md): current API,
|
||||||
server, transport, and source package boundaries.
|
server, transport, and source package boundaries.
|
||||||
- [docs/project_map.md](docs/project_map.md): package map, entrypoints, examples,
|
- [docs/project_map.md](docs/project_map.md): package map, entrypoints, examples,
|
||||||
|
|||||||
@@ -541,3 +541,78 @@ async def test_rpc_calls_python_source_capability(tmp_path) -> None:
|
|||||||
assert listed["result"]["total"] == 2
|
assert listed["result"]["total"] == 2
|
||||||
assert called["result"]["outcome"] == "ok"
|
assert called["result"]["outcome"] == "ok"
|
||||||
assert called["result"]["output"] == {"echoed": "hello python"}
|
assert called["result"]["output"] == {"echoed": "hello python"}
|
||||||
|
|
||||||
|
|
||||||
|
async def test_rpc_runs_workflow_from_python_source_capability(tmp_path) -> None:
|
||||||
|
config = WorkflowConfigFile.model_validate(
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"server": {
|
||||||
|
"store": {"kind": "filesystem", "root": str(tmp_path / "store")},
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"kind": "python",
|
||||||
|
"id": "local.ops",
|
||||||
|
"module": "tests.fixtures.python_source_ops",
|
||||||
|
"registry": "registry",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
server = build_workflow_server_from_workflow_config(config)
|
||||||
|
app = create_rpc_app(server)
|
||||||
|
transport = httpx.ASGITransport(app=app)
|
||||||
|
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
|
||||||
|
await _rpc(
|
||||||
|
client,
|
||||||
|
"workflow.draft_workspaces.create_from_capability",
|
||||||
|
{
|
||||||
|
"workspace_id": "python_echo_ws",
|
||||||
|
"capability_name": "local.ops.echo",
|
||||||
|
"name": "python_echo",
|
||||||
|
"title": "Python Echo",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
artifact = await _rpc(
|
||||||
|
client,
|
||||||
|
"workflow.draft_workspaces.create_artifact",
|
||||||
|
{
|
||||||
|
"workspace_id": "python_echo_ws",
|
||||||
|
"artifact_id": "python_echo",
|
||||||
|
"version": 1,
|
||||||
|
"title": "Python Echo",
|
||||||
|
"outcomes": ["ok"],
|
||||||
|
"kind": "workflow",
|
||||||
|
"source_bindings": {"local.ops": "local.ops", "wf.std": "wf.std"},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
deployment = await _rpc(
|
||||||
|
client,
|
||||||
|
"workflow.deployments.save",
|
||||||
|
{
|
||||||
|
"deployment": {
|
||||||
|
"id": "python_echo.default",
|
||||||
|
"artifact_id": "python_echo",
|
||||||
|
"artifact_version": 1,
|
||||||
|
"bindings": [
|
||||||
|
{"logical_source": "local.ops", "concrete_source": "local.ops"},
|
||||||
|
{"logical_source": "wf.std", "concrete_source": "wf.std"},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
},
|
||||||
|
)
|
||||||
|
run = await _rpc(
|
||||||
|
client,
|
||||||
|
"workflow.runs.start",
|
||||||
|
{
|
||||||
|
"deployment_id": "python_echo.default",
|
||||||
|
"workflow_input": {"text": "hello workflow"},
|
||||||
|
"trace_range": {"start": 0, "limit": 5},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert artifact["result"]["artifact_id"] == "python_echo"
|
||||||
|
assert deployment["result"]["deployment_id"] == "python_echo.default"
|
||||||
|
assert run["result"]["outcome"] == "ok"
|
||||||
|
assert run["result"]["output"] == {"echoed": "hello workflow"}
|
||||||
|
|||||||
Reference in New Issue
Block a user