sched: fence scheduled resumes at shutdown so none outlive ownership
This commit is contained in:
@@ -310,7 +310,13 @@ server capacity default is deployment configuration, with deterministic tests
|
||||
using a small injected limit.
|
||||
|
||||
On shutdown stop admission first and drain active tasks within a configured
|
||||
grace period. Record cancellation/failure when possible; abrupt termination
|
||||
grace period. New scheduled resumes are rejected once shutdown begins
|
||||
(no dispatch, no ungated fallback); in-flight scheduled resumes share
|
||||
the grace window, then are cancelled and joined before ownership is
|
||||
released, so no old execution persists after a new owner takes over.
|
||||
A cancelled resume keeps its executing mark and ACTIVE attempt, and
|
||||
startup recovery fails it closed exactly like a crash mid-resume.
|
||||
Record cancellation/failure when possible; abrupt termination
|
||||
uses startup recovery. Paused/deleted schedule definitions must not prevent
|
||||
run completion or resume from updating retained occurrence history.
|
||||
|
||||
@@ -321,9 +327,11 @@ canonical ownership, recovers without executing, then ticks calendar
|
||||
polling without blocking on long workflows: each dispatch spawns exactly
|
||||
one bounded execution task behind the async-completion seam, and the
|
||||
scheduler's own capacity gate is the execution-slot bound (an executing
|
||||
run keeps its slot until it stops). Shutdown stops admission, drains
|
||||
within `drain_grace_s`, leaves unfinished work under its executing mark
|
||||
for startup recovery to abandon truthfully, and releases ownership last.
|
||||
run keeps its slot until it stops). Shutdown stops admission, rejects
|
||||
new scheduled resumes for the duration of the drain, cancels and joins
|
||||
unfinished scheduled resumes after `drain_grace_s` (cancelled work keeps
|
||||
its executing mark and ACTIVE attempt for startup recovery to abandon
|
||||
truthfully), and releases ownership last.
|
||||
A failed startup releases the lock and raises.
|
||||
|
||||
Administration (`WorkflowApi` schedules methods, `workflow.schedules.*`
|
||||
@@ -344,7 +352,9 @@ resumed through the run API acquires a server execution slot through the
|
||||
scheduler's own accounting before dispatch — rejection leaves no resume
|
||||
attempt behind — holds the durable executing mark for the re-execution
|
||||
(visible to capacity and drain like any live execution), and releases
|
||||
the slot when its stopped result is persisted. A resumed scheduled run
|
||||
the slot when its stopped result is persisted. Shutdown drain rejects
|
||||
further scheduled resumes; a resume cancelled by the drain keeps its
|
||||
marks for recovery instead of releasing them. A resumed scheduled run
|
||||
reconciles its terminal history live through the same idempotent
|
||||
recording as dispatch; restart recovery still repairs torn boundaries.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user