wf reorg 5
This commit is contained in:
@@ -0,0 +1,62 @@
|
||||
# Project Map
|
||||
|
||||
This repository has three main packages plus examples and tests.
|
||||
|
||||
## Packages
|
||||
|
||||
| Package | Purpose | Usual callers |
|
||||
| --- | --- | --- |
|
||||
| `wf_core` | Deterministic workflow kernel: models, validation, runtime, run state, traces, interrupts, foreach, and path/state operations. | Runtime users, `wf_authoring`, workflow adapters. |
|
||||
| `wf_authoring` | Ergonomic workflow construction: `@node`, `NodeSpec`, builder DSL, conditions, path helpers, reusable ops, subgraph nodes. | Humans, tests, future LLM workflow builders. |
|
||||
| `wf_mcp` | MCP integration: SDK adapters, broker/proxy runtime, storage, config control, transparent proxy, and workflow wrappers for discovered tools. | MCP-facing CLI/server code and future UI/control surfaces. |
|
||||
|
||||
## Important Entry Points
|
||||
|
||||
- `wf_core`: public kernel facade for common runtime/model imports.
|
||||
- `wf_core.runtime`: `execute_workflow`, `resume_workflow`, `step_workflow`,
|
||||
and async variants.
|
||||
- `wf_core.models`: concrete Pydantic workflow model package.
|
||||
- `wf_core.validation`: structural workflow validation.
|
||||
- `wf_authoring`: public authoring facade.
|
||||
- `wf_authoring.WorkflowBuilder`: graph construction.
|
||||
- `wf_authoring.node`: typed Python function to `NodeSpec`.
|
||||
- `wf_mcp`: public MCP facade.
|
||||
- `wf-mcp`: CLI script from `pyproject.toml`.
|
||||
|
||||
## Examples
|
||||
|
||||
`examples/demo_workflow.py` contains the declared demo workflow and demo node
|
||||
registry used by `main.py` and workflow tests. It is intentionally outside
|
||||
`wf_core` so the kernel package does not carry fixture/demo code.
|
||||
|
||||
## Tests
|
||||
|
||||
- `tests/authoring`: builder, node decorator, ops, async runtime, subgraph, and
|
||||
demo workflow comparisons.
|
||||
- `tests/wf_mcp`: MCP SDK adapter, broker, transparent proxy, storage, CLI, and
|
||||
naming behavior.
|
||||
- `tests/rewrite`: local rewrite/port experiments that should keep exercising
|
||||
real user ergonomics.
|
||||
- `tests/fixtures`: test-only helper servers and fixtures.
|
||||
|
||||
## Verification Commands
|
||||
|
||||
```powershell
|
||||
uv run --with pytest pytest -q
|
||||
uv run ruff check src tests main.py examples
|
||||
uv run basedpyright src\wf_core tests\authoring tests\rewrite examples main.py --level error
|
||||
```
|
||||
|
||||
Use `uv run --env-file .env --with pytest pytest -q` when live MCP-backed tests
|
||||
need local environment configuration.
|
||||
|
||||
## Where To Add Things
|
||||
|
||||
- Add new executable workflow semantics in `wf_core.runtime` / `wf_core.runtime.ops`.
|
||||
- Add new graph/model syntax in `wf_core.models`, then validate it in
|
||||
`wf_core.validation`.
|
||||
- Add author convenience helpers in `wf_authoring`, not `wf_core`.
|
||||
- Add MCP transport/proxy/config behavior in `wf_mcp` concern packages.
|
||||
- Add runnable examples in `examples`.
|
||||
- Add test-only servers or helpers in `tests/fixtures`.
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
# Schema Validation Boundary
|
||||
|
||||
`wf_core` uses `SchemaRef` to carry JSON-schema-like shapes on workflow input,
|
||||
node input, node output, and workflow output. The current runtime does not
|
||||
implement full JSON Schema validation.
|
||||
|
||||
## Current Behavior
|
||||
|
||||
`wf_core.runtime.ops.schemas.validate_payload_against_schema` currently checks:
|
||||
|
||||
- if `schema.type == "object"`, the payload must be a `dict`
|
||||
- required top-level keys must be present
|
||||
|
||||
It does not currently check:
|
||||
|
||||
- property value types, such as `string`, `boolean`, `array`, or nested objects
|
||||
- array item schemas
|
||||
- `additionalProperties`
|
||||
- formats, enums, minimums, maximums, unions, discriminators, or nested required
|
||||
fields
|
||||
- whether a schema is valid JSON Schema
|
||||
|
||||
This means schema fields are mostly contracts for authoring, planning,
|
||||
documentation, and mapping validation today. They are not yet strong runtime
|
||||
guards.
|
||||
|
||||
## Why This Matters
|
||||
|
||||
The engine currently looks stricter than it is. A node can return a required
|
||||
field with the wrong type and pass runtime validation as long as the field is
|
||||
present. That is acceptable during early design work, but it is a real product
|
||||
boundary before workflows are generated by an LLM or backed by arbitrary MCP
|
||||
tools.
|
||||
|
||||
## Intended Seam
|
||||
|
||||
The schema adapter should live behind:
|
||||
|
||||
```text
|
||||
wf_core.runtime.ops.schemas.validate_payload_against_schema
|
||||
```
|
||||
|
||||
Callers should not choose or invoke the validation backend directly. The runtime
|
||||
should keep one small validation interface and hide whether the implementation
|
||||
uses Pydantic, `jsonschema`, a generated model cache, or a stricter custom
|
||||
adapter.
|
||||
|
||||
## Future Requirements
|
||||
|
||||
- Validate workflow input, node input, node output, and final workflow output
|
||||
with the same semantics.
|
||||
- Return errors that name the failing boundary and path.
|
||||
- Avoid silently accepting unsupported schema features once schemas are
|
||||
user/LLM-authored.
|
||||
- Keep schema validation separate from graph structure validation.
|
||||
- Keep `wf_authoring` free to generate schemas from Pydantic models without
|
||||
making the core runtime depend on authoring internals.
|
||||
|
||||
## Non-Goals For Now
|
||||
|
||||
- Do not add ad hoc type checks throughout runtime state operations.
|
||||
- Do not let each node wrapper invent separate validation behavior.
|
||||
- Do not conflate graph validation with payload validation.
|
||||
|
||||
@@ -62,6 +62,13 @@ raising at the first failure.
|
||||
- `wf_core.__init__` should stay a curated public facade, not a dump of runtime
|
||||
internals.
|
||||
|
||||
## Schema Validation
|
||||
|
||||
Payload schema validation is intentionally isolated behind
|
||||
`wf_core.runtime.ops.schemas.validate_payload_against_schema`. That function is
|
||||
not a full JSON Schema engine today; see `docs/schema_validation.md` for the
|
||||
current limits and intended adapter seam.
|
||||
|
||||
## What This Cleanup Does Not Solve Yet
|
||||
|
||||
- Foreach is still serial-only. Parallel foreach needs an explicit scheduling
|
||||
@@ -72,3 +79,5 @@ raising at the first failure.
|
||||
- Runtime errors are still ordinary exceptions plus failed run status. A richer
|
||||
error payload can be added later, but should be designed as part of trace/run
|
||||
state rather than scattered exceptions.
|
||||
- Payload schema validation is still shallow. The runtime checks object payloads
|
||||
and required top-level keys, not full JSON Schema semantics.
|
||||
|
||||
@@ -3,7 +3,9 @@
|
||||
## Related docs
|
||||
|
||||
- [docs/authoring_sketch.md](docs/authoring_sketch.md): `wf_authoring` direction, including `@node`, `NodeSpec`, builder ergonomics, async registry seams, and subgraph-as-node.
|
||||
- [docs/project_map.md](docs/project_map.md): current package map, entrypoints, examples, tests, and verification commands.
|
||||
- [docs/wf_core_architecture.md](docs/wf_core_architecture.md): current `wf_core` package boundaries, runtime flow, validation flow, and remaining cleanup seams.
|
||||
- [docs/schema_validation.md](docs/schema_validation.md): current payload schema validation limits and intended validation seam.
|
||||
- [docs/wf_mcp_plan.md](docs/wf_mcp_plan.md): `wf_mcp` direction as a namespaced MCP capability broker plus workflow build/run layer.
|
||||
- [docs/wf_mcp_architecture.md](docs/wf_mcp_architecture.md): current `wf_mcp` package boundaries and dependency rules.
|
||||
- [docs/scratchpad.md](docs/scratchpad.md): rougher design history and intermediate spec notes that fed the current model.
|
||||
|
||||
@@ -18,7 +18,7 @@ from wf_core import (
|
||||
execute_workflow,
|
||||
)
|
||||
from wf_core.errors import WorkflowExecutionError
|
||||
from wf_core.models import Condition as CoreCondition
|
||||
from wf_core.models.conditions import Condition as CoreCondition
|
||||
|
||||
from ..dsl import Expr, PathArg, compile_condition
|
||||
from ..nodes.callables import SyncRegistryHandler
|
||||
|
||||
@@ -3,7 +3,7 @@ from __future__ import annotations
|
||||
from dataclasses import dataclass
|
||||
from typing import Literal
|
||||
|
||||
from wf_core.models import (
|
||||
from wf_core.models.conditions import (
|
||||
BinaryCondition,
|
||||
Condition,
|
||||
ExistsCondition,
|
||||
|
||||
@@ -4,15 +4,15 @@ from collections.abc import Mapping
|
||||
from typing import Any
|
||||
|
||||
from .errors import WorkflowExecutionError
|
||||
from .models import (
|
||||
from .models.conditions import (
|
||||
BinaryCondition,
|
||||
Condition,
|
||||
ExistsCondition,
|
||||
LiteralOperand,
|
||||
NotCondition,
|
||||
PathOperand,
|
||||
VariadicCondition,
|
||||
)
|
||||
from .models.conditions import VariadicCondition
|
||||
from .paths import PathResolutionError, path_exists, resolve_graph_path
|
||||
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ from __future__ import annotations
|
||||
from collections.abc import Mapping
|
||||
from typing import Any
|
||||
|
||||
from wf_core.models import Workflow
|
||||
from wf_core.models.workflow import Workflow
|
||||
from wf_core.runtime.ops.flow import finalize_run
|
||||
from wf_core.runtime.ops.frames import collapse_completed_frames
|
||||
from wf_core.runtime.ops.nodes import AsyncNodeHandler, NodeHandler
|
||||
|
||||
@@ -2,7 +2,7 @@ from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from wf_core.models import Workflow
|
||||
from wf_core.models.workflow import Workflow
|
||||
from wf_core.run_state import (
|
||||
ExecutionFrame,
|
||||
FrameStatus,
|
||||
|
||||
@@ -2,7 +2,8 @@ from __future__ import annotations
|
||||
|
||||
from wf_core.conditions import safe_resolve_path
|
||||
from wf_core.errors import WorkflowExecutionError
|
||||
from wf_core.models import ForeachNode, Workflow
|
||||
from wf_core.models.steps import ForeachNode
|
||||
from wf_core.models.workflow import Workflow
|
||||
from wf_core.run_state import ExecutionFrame, FrameStatus, RunState, StepExecutionResult
|
||||
from wf_core.runtime.ops.flow import advance_frame, append_step_result_trace
|
||||
from wf_core.runtime.ops.frames import frame_context_values
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_core.conditions import eval_condition
|
||||
from wf_core.models import ConditionNode, InterruptNode
|
||||
from wf_core.models.steps import ConditionNode, InterruptNode
|
||||
from wf_core.run_state import FrameStatus, RunState, RunStatus, StepExecutionResult
|
||||
from wf_core.runtime.ops.flow import append_trace
|
||||
from wf_core.runtime.ops.frames import frame_context_values
|
||||
|
||||
@@ -4,7 +4,8 @@ from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from wf_core.errors import WorkflowExecutionError
|
||||
from wf_core.models import NodeDef, Workflow
|
||||
from wf_core.models.schemas import NodeDef
|
||||
from wf_core.models.workflow import Workflow
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
|
||||
@@ -4,7 +4,8 @@ from typing import Any
|
||||
|
||||
from wf_core.conditions import safe_resolve_path
|
||||
from wf_core.errors import WorkflowExecutionError
|
||||
from wf_core.models import InterruptNode, Workflow
|
||||
from wf_core.models.steps import InterruptNode
|
||||
from wf_core.models.workflow import Workflow
|
||||
from wf_core.run_state import InterruptRequest, RunState, StepExecutionResult
|
||||
from wf_core.runtime.ops.flow import advance_frame, append_step_result_trace
|
||||
from wf_core.runtime.ops.index import WorkflowIndex
|
||||
|
||||
@@ -5,7 +5,10 @@ from typing import Any, cast
|
||||
|
||||
from wf_core.conditions import safe_resolve_path
|
||||
from wf_core.errors import WorkflowExecutionError
|
||||
from wf_core.models import NodeDef, NodeResult, NodeUse, Workflow
|
||||
from wf_core.models.results import NodeResult
|
||||
from wf_core.models.schemas import NodeDef
|
||||
from wf_core.models.steps import NodeUse
|
||||
from wf_core.models.workflow import Workflow
|
||||
from wf_core.run_state import RunState, RuntimeContext, StepExecutionResult
|
||||
from wf_core.runtime.ops.frames import frame_context_values
|
||||
from wf_core.runtime.ops.schemas import validate_payload_against_schema
|
||||
|
||||
@@ -2,7 +2,7 @@ from __future__ import annotations
|
||||
|
||||
from copy import deepcopy
|
||||
|
||||
from wf_core.models import Workflow
|
||||
from wf_core.models.workflow import Workflow
|
||||
from wf_core.run_state import ExecutionFrame, FrameStatus, RunState, RunStatus
|
||||
|
||||
|
||||
|
||||
@@ -3,7 +3,8 @@ from __future__ import annotations
|
||||
from typing import Any
|
||||
|
||||
from wf_core.errors import WorkflowExecutionError
|
||||
from wf_core.models import NodeUse, Workflow
|
||||
from wf_core.models.steps import NodeUse
|
||||
from wf_core.models.workflow import Workflow
|
||||
from wf_core.paths import (
|
||||
PathResolutionError,
|
||||
get_nested_value,
|
||||
|
||||
@@ -3,7 +3,7 @@ from __future__ import annotations
|
||||
from typing import Any
|
||||
|
||||
from wf_core.errors import WorkflowExecutionError
|
||||
from wf_core.models import Workflow
|
||||
from wf_core.models.workflow import Workflow
|
||||
from wf_core.runtime.ops.frames import collapse_completed_frames
|
||||
from wf_core.runtime.ops.index import WorkflowIndex, build_workflow_index
|
||||
from wf_core.runtime.ops.interrupts import resume_interrupt
|
||||
|
||||
@@ -4,14 +4,14 @@ from collections.abc import Mapping
|
||||
from typing import Any
|
||||
|
||||
from wf_core.errors import WorkflowExecutionError
|
||||
from wf_core.models import (
|
||||
from wf_core.models.steps import (
|
||||
ConditionNode,
|
||||
ForeachNode,
|
||||
InterruptNode,
|
||||
JoinNode,
|
||||
NodeUse,
|
||||
Workflow,
|
||||
)
|
||||
from wf_core.models.workflow import Workflow
|
||||
from wf_core.runtime.ops.flow import advance_frame, append_step_result_trace
|
||||
from wf_core.runtime.ops.foreach import step_foreach
|
||||
from wf_core.runtime.ops.handlers import (
|
||||
|
||||
@@ -1,15 +1,14 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_core.models import (
|
||||
from wf_core.models.steps import (
|
||||
ConditionNode,
|
||||
Edge,
|
||||
ForeachNode,
|
||||
InterruptNode,
|
||||
NodeDef,
|
||||
NodeUse,
|
||||
Step,
|
||||
Workflow,
|
||||
)
|
||||
from wf_core.models.schemas import NodeDef
|
||||
from wf_core.models.workflow import Edge, Workflow
|
||||
from wf_core.tokens import END
|
||||
from wf_core.validation.issues import ValidationIssueCode, ValidationReport
|
||||
from wf_core.validation.outcomes import declared_outcomes_for_step, reachable_node_ids
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_core.models import Edge, InterruptNode, NodeDef, NodeUse, Step
|
||||
from wf_core.models.schemas import NodeDef
|
||||
from wf_core.models.steps import InterruptNode, NodeUse, Step
|
||||
from wf_core.models.workflow import Edge
|
||||
from wf_core.tokens import END
|
||||
|
||||
|
||||
|
||||
@@ -1,20 +1,17 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from wf_core.models import (
|
||||
from wf_core.models.conditions import (
|
||||
BinaryCondition,
|
||||
Condition,
|
||||
ConditionNode,
|
||||
ExistsCondition,
|
||||
ForeachNode,
|
||||
InterruptNode,
|
||||
LiteralOperand,
|
||||
NodeDef,
|
||||
NodeUse,
|
||||
NotCondition,
|
||||
PathOperand,
|
||||
VariadicCondition,
|
||||
Workflow,
|
||||
)
|
||||
from wf_core.models.schemas import NodeDef
|
||||
from wf_core.models.steps import ConditionNode, ForeachNode, InterruptNode, NodeUse
|
||||
from wf_core.models.workflow import Workflow
|
||||
from wf_core.paths import is_valid_destination_path, is_valid_source_path
|
||||
from wf_core.validation.issues import ValidationIssueCode, ValidationReport
|
||||
|
||||
|
||||
Reference in New Issue
Block a user