# Scheduled Caller Cancellation Capacity Implementation Plan > **For agentic workers:** Execute this plan inline in the supplied isolated > worktree. Preserve the existing shutdown fence and use a run-scoped > reconciliation path; never perform whole-store recovery for caller > cancellation. **Goal:** Reconcile a cancelled scheduled resume while the scheduler remains running so its durable ambiguity is terminal and its execution slot is reclaimed without permitting replay. **Architecture:** Keep shutdown cancellation on the existing `fence()` path, because shutdown must retain the durable executing marker and ACTIVE attempt until restart recovery decides the ambiguous outcome. Add a distinct normal- operation cancellation path that invokes the existing `recover(..., only_run_id=...)` reconciliation under the scheduler's ownership/lock, then releases only that run's in-memory and durable capacity state. Healthy sibling runs and the rest of the store remain untouched. **Tech Stack:** Python 3.14, asyncio, pytest/pytest-asyncio, file-backed run and schedule stores, Ruff, basedpyright. **Spec:** `docs/superpowers/specs/2026-09-08-deployment-scheduling-design.md` ## Global Constraints - Scheduled execution is bounded by the configured shared capacity. - A cancelled or abandoned execution must disclose unknown outcome and must never replay the occurrence. - Restart recovery is run-scoped when repairing a live execution; do not use whole-store recovery for caller cancellation. - Shutdown cancellation must preserve the existing durable fence and cancel-and-join-before-ownership-release ordering. - Existing B1–B4, drain, restart, sibling, and capacity behavior must remain green. --- ### Task 1: Add the caller-cancellation regression test **Files:** - Modify: `tests/wf_server/test_scheduler_integration.py` **Interfaces:** - Reuse `_interrupt_then_gate_plan`, `_gate_open`, `_scheduler`, `_wait_for`, `_entries`, and `_kinds` from the integration test module. - The test must observe the public `server.api.resume_run` behavior plus the durable `FileRunStore` state, not only private bookkeeping. - [x] **Step 1: Write the failing test** Add an async integration test with `capacity=1` and `auto_tick=False` that: 1. schedules `_interrupt_then_gate_plan`, polls until it is interrupted; 2. starts `server.api.resume_run` while `_gate_open` is closed; 3. waits until the run is in `service._live_resumes`, then cancels and awaits the task with `pytest.raises(asyncio.CancelledError)`; 4. asserts the service is still running, the live-resume set is empty, the run is durably `failed`, its `executing` marker is cleared, and its resume attempt remains `ACTIVE` as the no-replay ambiguity record; 5. schedules an unrelated ordinary constant workflow and polls until it completes, proving the cancelled run no longer occupies capacity; 6. asserts the cancelled run has exactly one failed occurrence entry and that the sibling has only its normal admitted/completed history; 7. attempts to resume the cancelled run and asserts the API rejects the ambiguous attempt without dispatching or adding completion history. - [x] **Step 2: Run the focused test to verify the red failure** Run: ```powershell uv run pytest -q tests/wf_server/test_scheduler_integration.py -k caller_cancellation ``` Expected: the new test fails because the cancelled run remains `interrupted` with `executing=True`, the attempt remains active, and the ordinary sibling cannot dispatch. ### Task 2: Separate normal-operation cancellation reconciliation from shutdown fencing **Files:** - Modify: `src/wf_api/runs.py` - Modify: `src/wf_scheduling/resume_gate.py` - Modify: `tests/wf_server/test_scheduler_integration.py` if a focused assertion needs a small adjustment after the first red run **Interfaces:** - Preserve `SchedulerResumeGate.fence(run_id)` for shutdown cancellation. - Add a narrowly named gate operation for a cancelled live scheduled resume; it must reconcile only `run_id` through `wf_scheduling.recovery.recover` with `only_run_id=run_id`, while holding the service lock and requiring the live service's ownership. - Keep `WorkflowRunApi._resume_run_unlocked` re-raising `CancelledError`. - [x] **Step 1: Implement the minimal normal-operation branch** In the `CancelledError` handler, choose the normal-operation reconciliation only while the scheduler is still started and not stopping. If shutdown has begun, retain the existing `fence()` call unchanged. The normal branch must run the existing scoped recovery for this `run_id`, preserve the `ACTIVE` attempt and ambiguous failure reason, clear the durable executing marker, remove the run from `_live_resumes`/`_resume_tasks`, and re-raise the cancellation. If scoped reconciliation fails, retain the durable marker and report the failure through the scheduler's existing error path rather than claiming capacity was reclaimed. - [x] **Step 2: Run the regression and shutdown tests** Run: ```powershell uv run pytest -q tests/wf_server/test_scheduler_integration.py -k "caller_cancellation or shutdown or sibling" ``` Expected: the new caller-cancellation test passes, and the existing shutdown timeout/restart, drain rejection, healthy sibling, and live resume tests remain green. - [x] **Step 3: Run the focused scheduling/API suites** Run: ```powershell uv run pytest -q tests/scheduling tests/wf_api/test_resume_attempt.py tests/wf_api/test_resume_concurrency.py tests/wf_server/test_scheduler_integration.py uv run ruff check src/wf_api/runs.py src/wf_scheduling/resume_gate.py tests/wf_server/test_scheduler_integration.py uv run basedpyright --level error src/wf_api/runs.py src/wf_scheduling/resume_gate.py ``` Expected: all selected tests pass, Ruff reports no violations, and basedpyright reports no errors. - [x] **Step 4: Review the full feature range independently** Review `git diff d5fa1809...HEAD` plus the new fix commit against the deployment scheduling spec and historical plan. Inspect admission, dispatch, resume, cancellation, shutdown, ownership, persistence, integration, pagination, and test boundaries. Reproduce any suspected defect before changing it; fix only confirmed in-scope blockers and then re-review the fix and its interactions. - [x] **Step 5: Run final verification and commit locally** Run the appropriately scoped suite, `uv run ruff check`, `uv run basedpyright --level error`, and a diff/status check. Commit the focused implementation and regression coverage locally on `opencode/sched-verify-plan`; do not merge or push.