fmt, fix
This commit is contained in:
@@ -333,6 +333,8 @@ Add:
|
||||
def validate_barrier_writes(
|
||||
item_patches: Sequence[StatePatch],
|
||||
state_fields: Mapping[StatePath, StateFieldDecl],
|
||||
*,
|
||||
reducers: Mapping[str, ReducerDefinition] | None = None,
|
||||
) -> None:
|
||||
"""Reject ambiguous sibling writes before replaying a foreach barrier.
|
||||
|
||||
@@ -369,7 +371,7 @@ In `build_barrier_patch(...)`, after:
|
||||
add:
|
||||
|
||||
```python
|
||||
validate_barrier_writes(item_patches, state_fields)
|
||||
validate_barrier_writes(item_patches, state_fields, reducers=reducers)
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Verify focused unit tests**
|
||||
|
||||
@@ -135,7 +135,11 @@ def test_sync_concurrent_foreach_item_reads_own_buffered_write() -> None:
|
||||
assert run.state["seen"] == ["scratch:a", "scratch:b", "scratch:c"]
|
||||
```
|
||||
|
||||
Important detail: update the existing `record` `NodeUse` in `_workflow(...)` or inside this test so it writes `scratch` to `state.scratch` for this workflow. If `_workflow(...)` is too fixed for that, create a small dedicated helper for this test instead of making `_workflow(...)` harder to read.
|
||||
Important detail: do not mutate the generic `_workflow(...)` helper for this test.
|
||||
Create a dedicated helper such as `_multi_step_overlay_workflow()` whose `record`
|
||||
`NodeUse` writes `scratch` to `state.scratch`. The test is specifically about an
|
||||
item-local write followed by an item-local read, so the workflow shape should be
|
||||
self-contained and obvious.
|
||||
|
||||
- [ ] **Step 2: Add a sibling isolation test**
|
||||
|
||||
@@ -498,7 +502,8 @@ def test_sync_concurrent_foreach_allows_multi_step_item_body_with_overlay() -> N
|
||||
# Assert the workflow completes and output contains all expected values.
|
||||
```
|
||||
|
||||
Prefer not duplicating the full workflow; extract a helper:
|
||||
Prefer not duplicating the full workflow; reuse the dedicated helper introduced
|
||||
for the overlay read/write tests:
|
||||
|
||||
```python
|
||||
def _multi_step_overlay_workflow() -> Workflow:
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
**Goal:** Implement concurrent foreach incrementally without breaking serial workflows or duplicating state-write logic.
|
||||
|
||||
**Architecture:** The work is split into four independently shippable layers: policy models, state patch extraction, barrier runtime state, and concurrent execution. Each layer preserves current serial behavior and adds tests before implementation. `foreach(mode="concurrent")` remains unsupported until the final layer. Sync runtime should support deterministic interleaving once the mode is enabled; async runtime can additionally run admitted async node handlers simultaneously.
|
||||
**Architecture:** The work was split into four independently shippable layers: policy models, state patch extraction, barrier runtime state, and concurrent execution. Each layer preserves current serial behavior and adds tests before implementation. Sync runtime supports deterministic interleaving for concurrent foreach; async runtime can additionally run admitted async node handlers simultaneously.
|
||||
|
||||
**Tech Stack:** Python 3.14, Pydantic v2, dataclasses, pytest, basedpyright, ruff, existing `wf_core` scheduler/runtime modules.
|
||||
|
||||
@@ -19,10 +19,10 @@
|
||||
- Phase 3 is implemented: `wf_core.runtime.foreach_state` owns typed barrier
|
||||
metadata and serial foreach progress now uses that metadata instead of ad hoc
|
||||
`foreach_progress`.
|
||||
- Phase 4 is not implemented: `foreach(mode="concurrent")` still validates as a
|
||||
model shape but runtime execution rejects it until concurrent scheduling,
|
||||
barrier commits, and item failure handling are implemented.
|
||||
- Phase 4 is expanded into a dedicated roadmap:
|
||||
- Phase 4 is implemented: `foreach(mode="concurrent")` supports sync
|
||||
interleaving, async item-node batching, barrier commits, item error policies,
|
||||
and quiescent interrupt handling.
|
||||
- Phase 4 details live in the dedicated roadmap:
|
||||
[`2026-05-22-concurrent-foreach-phase4-roadmap.md`](2026-05-22-concurrent-foreach-phase4-roadmap.md).
|
||||
Start with
|
||||
[`2026-05-22-concurrent-foreach-v1-sync-fail-only.md`](2026-05-22-concurrent-foreach-v1-sync-fail-only.md).
|
||||
@@ -466,7 +466,8 @@ async def test_async_runtime_accepts_concurrent_foreach() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
Expected before implementation: both tests fail because runtime still rejects concurrent mode.
|
||||
Historical expectation before Phase 4: both tests failed because runtime rejected
|
||||
concurrent mode. Current implementation status: these tests should pass.
|
||||
|
||||
- [ ] **Step 2: Add capacity tests**
|
||||
|
||||
@@ -575,10 +576,10 @@ Ship these as separate commits/PRs:
|
||||
3. Phase 3: barrier metadata with serial behavior unchanged
|
||||
4. Phase 4: concurrent execution
|
||||
|
||||
Do not start Phase 4 until Phase 2 and Phase 3 are stable. Concurrent foreach depends on patch extraction and resumable barrier state.
|
||||
Phase 4 depended on Phase 2 and Phase 3 because concurrent foreach needs patch extraction and resumable barrier state. Keep future fork/gather or native subgraph work layered on top of those runtime primitives instead of replacing them.
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: ADR 0002 decisions are represented across the four phases.
|
||||
- Intentional gaps: explicit Fork/Gather, lineage-token graph nodes, OpenTelemetry, platform source/tool caps, and full run persistence are not included.
|
||||
- Risk control: phases 1-3 preserve serial behavior and keep `mode="concurrent"` unsupported until phase 4.
|
||||
- Risk control: phases 1-3 preserved serial behavior until phase 4 enabled `mode="concurrent"`.
|
||||
|
||||
@@ -118,10 +118,11 @@ limits and intended adapter seam.
|
||||
state patch commits. The remaining mapping design notes for future reducer
|
||||
metadata are documented in
|
||||
[`core_state_mapping_and_merge.md`](core_state_mapping_and_merge.md).
|
||||
- Foreach is still serial-only. The scheduler foundation exists, but concurrent
|
||||
foreach still needs explicit policy, implicit barrier state, lineage-aware
|
||||
patch commits, and quiescent interrupt handling. `ForeachNode.over` is typed
|
||||
as a `GraphSourcePath`, but execution is still serial.
|
||||
- Foreach supports serial and concurrent execution. Concurrent foreach uses
|
||||
explicit policy, typed barrier state, lineage-aware patch commits, item error
|
||||
policy, and quiescent interrupt handling. Remaining gaps are higher-level
|
||||
graph constructs such as native subgraphs, explicit fork/gather nodes, and
|
||||
advanced conflict strategies beyond exact-path mergeable reducers.
|
||||
- Interrupt lifecycle is still node-level and run-state-level. Long-lived
|
||||
external subscriptions or notification streams need a separate lifecycle
|
||||
design. Interrupt `request` and `resume` are canonical binding lists; nested
|
||||
|
||||
Reference in New Issue
Block a user