[colin99d]

Problem

When a handler returns {"refresh_worker": True}, the SDK sends stopPod: True with the job result but keeps its job-take loop running. Once the slot frees, get_jobs can take the next queued job before the platform acts on the stop. The restart then kills that job, which is retried or fails as stale_job.

The start-time flag (runpod.serverless.start({"handler": h, "refresh_worker": True})) never had this race: JobScaler.handle_job calls kill_worker() before its finally frees the slot. The per-job return value skipped that step.

Prod example (endpoint 9d7pu3kccylu0w, pod 5ys0tdv6mxhb07, 2026-10-28 UTC):

Time	Event
14:22:41.0	job A's job-done carries stopPod; ai-api removes the worker and stops the pod
14:22:42.5	the scaler resumes the same pod for queued job B, which re-registers it
14:22:43.4	the old process takes job B
14:23:10	the host recreates the container; job B dies and is later requeued

ai-api can't close this on its side: the old and new process send identical job-take requests, and ai-api has no signal for when the old container is gone. Only the worker process knows it is the one being replaced.

Fix

• rp_job.handle_job returns the result it sent to job-done.
• JobScaler.handle_job calls kill_worker() when that result carries stopPod, as well as for the start-time flag. The call runs before the slot frees and with no await in between, so get_jobs can't take another job.
• Logs Refresh requested, taking no more jobs. Until now the per-job path logged nothing, which made affected endpoints hard to find.

Tests

tests/test_serverless/test_refresh_worker.py, 32 tests on four levels:

1. JobScaler with a fake queue:
   • refresh on the first and on a later job;
   • the worker is already marked dead when the slot frees;
   • the start-time flag still refreshes;
   • jobs already running finish at concurrency 2;
   • 9 result shapes that must not refresh, e.g. stopPod: False or stopPod nested in output.
2. rp_job.handle_job returns exactly the payload it sent.
3. The real JobScaler over real HTTP against a fake ai-api, which asserts that no job-take arrives after stopPod.
4. A real worker process started via runpod.serverless.start, with platform-style env vars, against the same fake ai-api.

Without the fix, 17 of these fail and the 15 control tests pass. The real-process test reproduces the prod sequence: job B is taken after stopPod. With the fix:

• all 32 pass;
• the full suite passes (2213 tests, 93.7% coverage) on macOS and on Linux aarch64 (fork);
• the new tests pass on Linux x86_64 in Docker, where the reverted fix fails the same 17;
• the related tests (104) pass on Python 3.10 to 3.14;
• 14 repeated runs, 4 of them in parallel, showed no flakes.

Not covered / worth knowing

• Release path. main is on the 2.0.0rc line, so this ships in 2.0.0. The reporting customer is on 1.10.0. If they shouldn't have to take a major upgrade for this, it needs a 1.12.1 backport from v1.12.0, which has the same code. Until then they can use the start-time flag.
• Concurrency > 1. If a job-take long-poll is already in flight when the refresh happens, that job can still be lost. The start-time flag has the same limit.
• Behavior change. After a per-job refresh, the process now exits right after job-done instead of idling until the platform kills it. This matches the start-time flag, but daemon threads that flush logs after the handler returns may get less time.
• Platform-level race. A fast ResumePod after any StopPod on a non-FlashBoot endpoint can still hand a job to the old process. That lives in ai-api and is out of scope here.

Linear: SLS-900

🤖 Generated with Claude Code