docs: archive caller cancellation plan

2026-09-09 20:10:28 +07:00
parent 34e026a8f2
commit d8a6795193
1 changed files with 147 additions and 0 deletions
@@ -0,0 +1,147 @@
 # Scheduled Caller Cancellation Capacity Implementation Plan
 > **For agentic workers:** Execute this plan inline in the supplied isolated
 > worktree. Preserve the existing shutdown fence and use a run-scoped
 > reconciliation path; never perform whole-store recovery for caller
 > cancellation.
 **Goal:** Reconcile a cancelled scheduled resume while the scheduler remains
 running so its durable ambiguity is terminal and its execution slot is
 reclaimed without permitting replay.
 **Architecture:** Keep shutdown cancellation on the existing `fence()` path,
 because shutdown must retain the durable executing marker and ACTIVE attempt
 until restart recovery decides the ambiguous outcome. Add a distinct normal-
 operation cancellation path that invokes the existing `recover(...,
 only_run_id=...)` reconciliation under the scheduler's ownership/lock, then
 releases only that run's in-memory and durable capacity state. Healthy sibling
 runs and the rest of the store remain untouched.
 **Tech Stack:** Python 3.14, asyncio, pytest/pytest-asyncio, file-backed run
 and schedule stores, Ruff, basedpyright.
 **Spec:** `docs/superpowers/specs/2026-09-08-deployment-scheduling-design.md`
 ## Global Constraints
 - Scheduled execution is bounded by the configured shared capacity.
 - A cancelled or abandoned execution must disclose unknown outcome and must
  never replay the occurrence.
 - Restart recovery is run-scoped when repairing a live execution; do not use
  whole-store recovery for caller cancellation.
 - Shutdown cancellation must preserve the existing durable fence and
  cancel-and-join-before-ownership-release ordering.
 - Existing B1–B4, drain, restart, sibling, and capacity behavior must remain
  green.
 ---
 ### Task 1: Add the caller-cancellation regression test
 **Files:**
 - Modify: `tests/wf_server/test_scheduler_integration.py`
 **Interfaces:**
 - Reuse `_interrupt_then_gate_plan`, `_gate_open`, `_scheduler`, `_wait_for`,
  `_entries`, and `_kinds` from the integration test module.
 - The test must observe the public `server.api.resume_run` behavior plus the
  durable `FileRunStore` state, not only private bookkeeping.
 - [x] **Step 1: Write the failing test**
  Add an async integration test with `capacity=1` and `auto_tick=False` that:
  1. schedules `_interrupt_then_gate_plan`, polls until it is interrupted;
  2. starts `server.api.resume_run` while `_gate_open` is closed;
  3. waits until the run is in `service._live_resumes`, then cancels and
     awaits the task with `pytest.raises(asyncio.CancelledError)`;
  4. asserts the service is still running, the live-resume set is empty, the
     run is durably `failed`, its `executing` marker is cleared, and its
     resume attempt remains `ACTIVE` as the no-replay ambiguity record;
  5. schedules an unrelated ordinary constant workflow and polls until it
     completes, proving the cancelled run no longer occupies capacity;
  6. asserts the cancelled run has exactly one failed occurrence entry and
     that the sibling has only its normal admitted/completed history;
  7. attempts to resume the cancelled run and asserts the API rejects the
     ambiguous attempt without dispatching or adding completion history.
 - [x] **Step 2: Run the focused test to verify the red failure**
  Run:
  ```powershell
  uv run pytest -q tests/wf_server/test_scheduler_integration.py -k caller_cancellation
  ```
  Expected: the new test fails because the cancelled run remains
  `interrupted` with `executing=True`, the attempt remains active, and the
  ordinary sibling cannot dispatch.
 ### Task 2: Separate normal-operation cancellation reconciliation from shutdown fencing
 **Files:**
 - Modify: `src/wf_api/runs.py`
 - Modify: `src/wf_scheduling/resume_gate.py`
 - Modify: `tests/wf_server/test_scheduler_integration.py` if a focused
  assertion needs a small adjustment after the first red run
 **Interfaces:**
 - Preserve `SchedulerResumeGate.fence(run_id)` for shutdown cancellation.
 - Add a narrowly named gate operation for a cancelled live scheduled resume;
  it must reconcile only `run_id` through `wf_scheduling.recovery.recover` with
  `only_run_id=run_id`, while holding the service lock and requiring the live
  service's ownership.
 - Keep `WorkflowRunApi._resume_run_unlocked` re-raising `CancelledError`.
 - [x] **Step 1: Implement the minimal normal-operation branch**
  In the `CancelledError` handler, choose the normal-operation reconciliation
  only while the scheduler is still started and not stopping. If shutdown has
  begun, retain the existing `fence()` call unchanged. The normal branch must
  run the existing scoped recovery for this `run_id`, preserve the `ACTIVE`
  attempt and ambiguous failure reason, clear the durable executing marker,
  remove the run from `_live_resumes`/`_resume_tasks`, and re-raise the
  cancellation. If scoped reconciliation fails, retain the durable marker and
  report the failure through the scheduler's existing error path rather than
  claiming capacity was reclaimed.
 - [x] **Step 2: Run the regression and shutdown tests**
  Run:
  ```powershell
  uv run pytest -q tests/wf_server/test_scheduler_integration.py -k "caller_cancellation or shutdown or sibling"
  ```
  Expected: the new caller-cancellation test passes, and the existing shutdown
  timeout/restart, drain rejection, healthy sibling, and live resume tests
  remain green.
 - [x] **Step 3: Run the focused scheduling/API suites**
  Run:
  ```powershell
  uv run pytest -q tests/scheduling tests/wf_api/test_resume_attempt.py tests/wf_api/test_resume_concurrency.py tests/wf_server/test_scheduler_integration.py
  uv run ruff check src/wf_api/runs.py src/wf_scheduling/resume_gate.py tests/wf_server/test_scheduler_integration.py
  uv run basedpyright --level error src/wf_api/runs.py src/wf_scheduling/resume_gate.py
  ```
  Expected: all selected tests pass, Ruff reports no violations, and
  basedpyright reports no errors.
 - [x] **Step 4: Review the full feature range independently**
  Review `git diff d5fa1809...HEAD` plus the new fix commit against the
  deployment scheduling spec and historical plan. Inspect admission,
  dispatch, resume, cancellation, shutdown, ownership, persistence,
  integration, pagination, and test boundaries. Reproduce any suspected
  defect before changing it; fix only confirmed in-scope blockers and then
  re-review the fix and its interactions.
 - [x] **Step 5: Run final verification and commit locally**
  Run the appropriately scoped suite, `uv run ruff check`,
  `uv run basedpyright --level error`, and a diff/status check. Commit the
  focused implementation and regression coverage locally on
  `opencode/sched-verify-plan`; do not merge or push.