Skip to content

Pin orkes-conductor-queues to 2.0.0.rc2 (CxODEV-2125) - #10

Merged
milansvitlica-codaxy merged 1 commit into
mainfrom
CxODEV-2125-orkes-queues-rc2
Oct 19, 2026
Merged

milansvitlica-codaxy merged 1 commit into
mainfrom
CxODEV-2125-orkes-queues-rc2

Conversation

@milansvitlica-codaxy

Copy link
Copy Markdown

Pins orkes-conductor-queues to 2.0.0.rc2. This is a CxODEV-2125 follow-up to the 3.32.5 release (server-3.32.5-rmq-kafka). The image from this change would be server-3.32.5-rmq-kafka.1.

Why

  • Conductor 3.31.0 and later bundle orkes-conductor-queues 2.0.0.rc3. Its new background poller has a race (fix: prevent permanent per-queue poller stall in QueueMonitor orkes-io/orkes-queues#30, open, not released):
    • wakePoller() can cancel a backoff poll that is already running, so two poll chains share one slot.
    • When demand drops, both release it and the slot count goes to -1.
    • From then on that queue never polls Redis again until the JVM restarts. No error is logged, and /health stays up.
  • The JOIN queue is hit first, so workflows stay RUNNING on a JOIN whose branches are done. We saw exactly that in one of our e2e runs on server-3.32.5-rmq-kafka.
  • rc2 has no background poller: its QueueMonitor fetches on the caller's thread. It is the version 3.30.0–3.30.2 shipped.

Compatibility (checked)

  • 3.32.5 uses only ConductorQueue, QueueMessage and the three-argument ConductorRedisQueue / ConductorRedisClusterQueue constructors. rc2 has them unchanged.
  • JedisCommands (redis-api), which rc2 calls, has the same methods in 3.30.2 and 3.32.5.
  • There are no lockfiles, and the server config has no rc3-only settings.
  • Cost: rc3's speed work goes away (dedicated poller, push wake-ups, doorbell). Queue pickup goes back to 3.30.x behaviour.

Tested

  • A local image built from this branch: its server jar contains only orkes-conductor-queues-2.0.0.rc2.jar.
  • The CxO end-to-end suite on that image: 254 passed, 26 failed (all known pending), 0 flaky. All 286 scenarios have the same status as on server-3.32.5-rmq-kafka, with the same suite commit.
  • :conductor-redis-persistence:test passed 63/63, and spotlessCheck is green.
  • Not run locally: :conductor-test-harness:test. It hardcodes localhost for its Redis container, which my containerised Gradle can't reach. This PR's CI runs it.
  • No new regression test. A JOIN-like delivery test against real Redis never stalled rc3 in 4 × 60 s, so it couldn't show the bug. The PINNED comment and the AGENTS.md row are the guard.

🤖 Generated with Claude Code

Conductor 3.31.0 and later bundle orkes-conductor-queues 2.0.0.rc3. Its
new background poller has a race: wakePoller() can cancel a backoff poll
that is already running, so two poll chains share one slot. When demand
drops, both release it and the slot count goes to -1. From then on the
queue never polls Redis again until the JVM restarts, with no error and a
healthy /health. The JOIN queue is hit first, because an unfinished JOIN
is re-queued at offset 0. The upstream fix, orkes-io/orkes-queues#30, is
not released.

rc2 has no background poller and is what 3.30.x shipped. 3.32.5 uses only
ConductorQueue, QueueMessage and the three-argument queue constructors,
and rc2 has them unchanged. The pin is listed with the other hard pins in
AGENTS.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@milansvitlica-codaxy
milansvitlica-codaxy merged commit 22da857 into main Oct 19, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant