use json schema for state schema
This commit is contained in:
@@ -160,31 +160,30 @@ creates a reusable boundary for:
|
||||
## State Declarations and Merge Rules
|
||||
|
||||
State merge behavior is attached to declared exact state paths. The canonical
|
||||
schema shape is a list of declarations:
|
||||
schema shape is ordinary JSON Schema. `reducer` is a wf_core extension keyword
|
||||
on property schemas; JSON Schema validators ignore it, while wf_core validates
|
||||
and uses it for state writes.
|
||||
|
||||
```json
|
||||
{
|
||||
"fields": [
|
||||
{
|
||||
"path": "state.person.name",
|
||||
"schema": {"type": "string"},
|
||||
"reducer": {"name": "wf.std.replace"}
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"person": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string", "reducer": "wf.std.replace"},
|
||||
"tags": {"type": "array", "reducer": "wf.std.append"}
|
||||
}
|
||||
},
|
||||
{
|
||||
"path": "state.person.tags",
|
||||
"schema": {"type": "array"},
|
||||
"reducer": {"name": "wf.std.append"}
|
||||
},
|
||||
{
|
||||
"path": "state.profile",
|
||||
"schema": {"type": "object"},
|
||||
"reducer": {"name": "wf.std.merge_object"}
|
||||
"profile": {
|
||||
"type": "object",
|
||||
"reducer": "wf.std.merge_object"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Deprecated dict-shaped state fields are still accepted at parse boundaries:
|
||||
Deprecated `fields` state declarations are still accepted at parse boundaries:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -194,14 +193,15 @@ Deprecated dict-shaped state fields are still accepted at parse boundaries:
|
||||
}
|
||||
```
|
||||
|
||||
Validated `StateSchema` models store and dump the canonical list shape.
|
||||
Presentation layers may rebuild a tree for humans.
|
||||
Validated `StateSchema` models store and dump the canonical JSON Schema shape.
|
||||
`StateSchema.field_map()` compiles an internal exact-path index for runtime
|
||||
reducer lookup.
|
||||
|
||||
`wf_authoring` keeps authored schemas nested for humans and LLM clients, but
|
||||
projects nested authored state into this flat exact-path index. For example, a
|
||||
Pydantic `person: Person` field may produce declarations for `person`,
|
||||
`person.name`, and `person.tags` without forcing the author to spell those
|
||||
paths manually.
|
||||
`wf_authoring` keeps authored schemas nested for humans and LLM clients, and
|
||||
injects state metadata such as `reducer` into the generated JSON Schema
|
||||
properties. For example, a Pydantic `person: Person` field can produce nested
|
||||
properties for `person.name` and `person.tags` without forcing the author to
|
||||
spell those paths manually.
|
||||
|
||||
### Exact-path ownership
|
||||
|
||||
|
||||
@@ -139,7 +139,7 @@ Overlap rules:
|
||||
|
||||
State path validation and write behavior:
|
||||
|
||||
- writable `StatePath` must have its root declared in `state_schema.fields`
|
||||
- writable `StatePath` must have its root declared in `state_schema.properties`
|
||||
- whole-state write targets such as bare `state` stay out of scope for now
|
||||
- nested state subpaths are allowed once the root exists in the schema
|
||||
- exact nested state declarations are reducer/schema hints, not root ownership
|
||||
@@ -476,29 +476,28 @@ Core explicitness:
|
||||
|
||||
State schema fields:
|
||||
|
||||
- move toward list-of-structs instead of dict keys
|
||||
- canonical shape:
|
||||
- canonical shape is normal JSON Schema
|
||||
- state field metadata such as `reducer` lives as a wf_core extension keyword on
|
||||
each property schema
|
||||
- JSON Schema validators ignore `reducer`; wf_core validates it separately and
|
||||
compiles it into an exact-path runtime index
|
||||
- accept old `fields` shapes at parse time for compatibility:
|
||||
|
||||
```text
|
||||
StateSchema.fields: list[StateFieldDecl]
|
||||
|
||||
StateFieldDecl:
|
||||
path: StatePath
|
||||
type: string
|
||||
reducer: ReducerRef
|
||||
```
|
||||
|
||||
- serialized field paths include `state.` prefix, e.g. `state.person.tags`
|
||||
- accept old dict shape at parse time for compatibility:
|
||||
|
||||
```text
|
||||
fields = {
|
||||
"person.tags": {"type": "array", "reducer": "wf.std.append"}
|
||||
state_schema = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"person": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"tags": {"type": "array", "reducer": "wf.std.append"}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- normalize old shape to canonical list internally
|
||||
- canonical serialization emits list shape
|
||||
- canonical serialization emits JSON Schema shape
|
||||
- duplicate field paths are validation errors
|
||||
- exact reducer matching uses exact `StatePath`
|
||||
|
||||
|
||||
@@ -0,0 +1,249 @@
|
||||
# JSON Schema State Reducers Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make `Workflow.state_schema` a normal JSON Schema object, with `reducer` as an explicit workflow extension keyword on field schemas.
|
||||
|
||||
**Architecture:** `StateSchema` should validate as JSON Schema first, then expose helper indexes for workflow runtime metadata. Runtime reducer lookup should compile from `properties` paths instead of requiring a separate path declaration list. Legacy `fields` inputs remain parse-only compatibility during the transition.
|
||||
|
||||
**Tech Stack:** Python, Pydantic v2, `jsonschema`, `wf_core` path models, pytest, basedpyright, ruff.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Canonical State Schema Tests
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/core/test_nested_state_paths.py`
|
||||
- Modify: `tests/core/test_schema_validation.py`
|
||||
|
||||
- [ ] **Step 1: Add a test for JSON Schema property reducers**
|
||||
|
||||
```python
|
||||
def test_state_schema_uses_json_schema_properties_as_canonical_shape() -> None:
|
||||
schema = StateSchema.model_validate(
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"person": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"description": "Display name",
|
||||
"reducer": "wf.std.replace",
|
||||
}
|
||||
},
|
||||
},
|
||||
"count": {"type": "integer", "reducer": "wf.std.add"},
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
fields = schema.field_map()
|
||||
assert fields["person.name"].validation_schema.type == "string"
|
||||
assert fields["person.name"].reducer == ReducerRef(name="wf.std.replace")
|
||||
assert fields["count"].reducer == ReducerRef(name="wf.std.add")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add a dump test proving the canonical output is still JSON Schema**
|
||||
|
||||
```python
|
||||
def test_state_schema_dumps_canonical_json_schema_with_reducer_keyword() -> None:
|
||||
schema = StateSchema.model_validate(
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"count": {"type": "integer", "reducer": "wf.std.add"}
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
dumped = schema.model_dump(mode="json")
|
||||
assert dumped["type"] == "object"
|
||||
assert dumped["properties"]["count"]["type"] == "integer"
|
||||
assert dumped["properties"]["count"]["reducer"] == "wf.std.add"
|
||||
Draft202012Validator.check_schema(dumped)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add a runtime reducer lookup test from canonical schema**
|
||||
|
||||
```python
|
||||
def test_exact_nested_state_path_uses_reducer_from_json_schema_property() -> None:
|
||||
workflow = _workflow_from_state_schema(
|
||||
StateSchema.model_validate(
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"person": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"tags": {"type": "array", "reducer": "wf.std.append"}
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
)
|
||||
)
|
||||
state = {"person": {"tags": ["seed"]}}
|
||||
|
||||
write_state_value(workflow, state, "state.person.tags", ["next"])
|
||||
|
||||
assert state["person"]["tags"] == ["seed", "next"]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run focused tests and confirm failures**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py -q`
|
||||
|
||||
Expected: new tests fail because `StateSchema` still serializes as `fields: [...]` and reducer lookup is compiled from field declarations only.
|
||||
|
||||
### Task 2: Implement JSON-Schema-Native `StateSchema`
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_core/models/schemas.py`
|
||||
|
||||
- [ ] **Step 1: Make `StateSchema` inherit JSON Schema fields directly**
|
||||
|
||||
`StateSchema` should expose common JSON Schema object fields:
|
||||
|
||||
```python
|
||||
title: str | None = None
|
||||
type: str | list[str] | None = "object"
|
||||
properties: dict[str, Any] = Field(default_factory=dict)
|
||||
required: list[str] = Field(default_factory=list)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Preserve legacy `fields` as parse-only input**
|
||||
|
||||
Keep accepting:
|
||||
|
||||
```json
|
||||
{"fields": [{"path": "state.count", "type": "integer", "reducer": "wf.std.add"}]}
|
||||
```
|
||||
|
||||
and:
|
||||
|
||||
```json
|
||||
{"fields": {"count": {"type": "integer", "reducer": "wf.std.add"}}}
|
||||
```
|
||||
|
||||
by converting both into:
|
||||
|
||||
```json
|
||||
{"type": "object", "properties": {"count": {"type": "integer", "reducer": "wf.std.add"}}}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add `field_map()` as an internal compiled index**
|
||||
|
||||
`field_map()` should walk explicit object `properties` and return `StateFieldDecl` values keyed by rootless state path. It must:
|
||||
|
||||
- include every explicit property path
|
||||
- parse `reducer` with `ReducerRef`
|
||||
- default missing reducer to `wf.std.replace`
|
||||
- preserve `trace` and `default` workflow extension keywords
|
||||
- remove workflow extension keywords from `StateFieldDecl.validation_schema`
|
||||
|
||||
- [ ] **Step 4: Validate JSON Schema and extension keyword types**
|
||||
|
||||
Use `SchemaRef`/`jsonschema` validation for the complete state schema. Add explicit validation that:
|
||||
|
||||
- `reducer` is a string or `ReducerRef`-compatible object
|
||||
- `trace` is a boolean when present
|
||||
- `default` is allowed as JSON Schema/default metadata
|
||||
|
||||
### Task 3: Update Artifact Reducer Extraction
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_artifacts/factory.py`
|
||||
|
||||
- [ ] **Step 1: Extract reducer dependencies from `state_schema.properties`**
|
||||
|
||||
Add a helper that walks explicit JSON Schema properties and yields reducer payloads from every property schema.
|
||||
|
||||
- [ ] **Step 2: Keep legacy `fields` extraction only as compatibility**
|
||||
|
||||
If `state_schema.fields` exists in old artifacts, continue reading it. Prefer canonical `properties` when present.
|
||||
|
||||
- [ ] **Step 3: Add tests through existing workflow surface/artifact tests**
|
||||
|
||||
Use an existing artifact/dependency test and assert a reducer declared at:
|
||||
|
||||
```json
|
||||
state_schema.properties.count.reducer
|
||||
```
|
||||
|
||||
is included in required capabilities.
|
||||
|
||||
### Task 4: Update Authoring Conversion
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/wf_authoring/schemas.py`
|
||||
- Modify: `tests/authoring/test_schemas.py`
|
||||
|
||||
- [ ] **Step 1: Attach reducer metadata directly to generated property schemas**
|
||||
|
||||
When `state_schema_from(BaseModel)` sees `Annotated[..., state_field(reducer=...)]`, inject `reducer` and `trace` into that property schema instead of building a separate field map.
|
||||
|
||||
- [ ] **Step 2: Preserve model JSON Schema as the state schema**
|
||||
|
||||
Return `StateSchema.model_validate(schema_with_reducer_keywords)` so generated state schema remains JSON Schema-shaped.
|
||||
|
||||
### Task 5: Update Docs and Examples
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/core_state_mapping_and_merge.md`
|
||||
- Modify: `docs/workflow_drafts.md`
|
||||
- Modify: `docs/wf_mcp_operator_manual.md`
|
||||
- Modify: `docs/wf_mcp_end_to_end_runbook.md`
|
||||
- Modify: `examples/raw_canonical_workflow.py`
|
||||
|
||||
- [ ] **Step 1: Replace canonical `fields: [...]` examples**
|
||||
|
||||
Use JSON Schema:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"count": {
|
||||
"type": "integer",
|
||||
"description": "Counter value",
|
||||
"reducer": "wf.std.add"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Document extension semantics**
|
||||
|
||||
State clearly that `reducer` is not standard JSON Schema behavior. JSON Schema validators ignore it; `wf_core` reads it for workflow state writes.
|
||||
|
||||
### Task 6: Verification
|
||||
|
||||
**Files:**
|
||||
- All touched files
|
||||
|
||||
- [ ] **Step 1: Run focused tests**
|
||||
|
||||
Run: `uv run --with pytest pytest tests/core/test_nested_state_paths.py tests/core/test_schema_validation.py tests/authoring/test_schemas.py -q`
|
||||
|
||||
- [ ] **Step 2: Run full tests**
|
||||
|
||||
Run: `uv run --with pytest pytest -q`
|
||||
|
||||
- [ ] **Step 3: Run static checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
uvx ruff check
|
||||
uv run basedpyright --level error
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: covers canonical JSON Schema state shape, reducer extension keyword, compatibility, runtime lookup, artifact dependency extraction, authoring generation, docs, and verification.
|
||||
- Placeholder scan: no placeholders remain.
|
||||
- Type consistency: `StateSchema`, `StateFieldDecl`, `ReducerRef`, and `SchemaRef` names match current code.
|
||||
@@ -244,14 +244,13 @@ arguments:
|
||||
"required": ["text"]
|
||||
},
|
||||
"state_schema": {
|
||||
"fields": [
|
||||
{
|
||||
"path": "state.echoed",
|
||||
"schema": {
|
||||
"type": "string"
|
||||
}
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"echoed": {
|
||||
"type": "string",
|
||||
"reducer": "wf.std.replace"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"output_schema": {
|
||||
"type": "object",
|
||||
|
||||
@@ -389,14 +389,13 @@ Minimal example:
|
||||
"required": ["text"]
|
||||
},
|
||||
"state_schema": {
|
||||
"fields": [
|
||||
{
|
||||
"path": "state.echoed",
|
||||
"schema": {
|
||||
"type": "string"
|
||||
}
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"echoed": {
|
||||
"type": "string",
|
||||
"reducer": "wf.std.replace"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"output_schema": {
|
||||
"type": "object",
|
||||
|
||||
@@ -48,14 +48,13 @@ A minimal draft looks like this:
|
||||
"required": ["text"]
|
||||
},
|
||||
"state_schema": {
|
||||
"fields": [
|
||||
{
|
||||
"path": "state.echoed",
|
||||
"schema": {
|
||||
"type": "string"
|
||||
}
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"echoed": {
|
||||
"type": "string",
|
||||
"reducer": "wf.std.replace"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"output_schema": {
|
||||
"type": "object",
|
||||
|
||||
Reference in New Issue
Block a user