Repository navigation
fix(dojo-verify): speed tests measure CPU time; release-check reruns a mod run that only timed out - #18
Merged
Conversation
…a mod run that only timed out On 2026-11-02 the release check for PR 16 failed both dojo-verify rows at load ~550. The same code passed 153/153 and 109/109 (mod suite in 4s) once the machine was quiet; nothing in dojo-verify was wrong. - unittest: the TestSpeed and TestTailWindow limits read wall-clock time, which counts time spent waiting for a CPU. They now read CPU time of the test process and its finished children (tests/_helpers.py cpu_seconds). Windows has no child CPU accounting there and keeps the wall clock. Proven able to fail: a quadratic loop added to detect_claim fails 18 speed checks at ~0.38s against 0.2s. - mod: the kit gives each test a fixed 5000 ms that a mod can't raise (probed: an options timeout is ignored). release-check now reruns a mod run once when every failure is that timeout, and the row reads WARN with the first run's count and the load average. A test that hangs times out again and stays FAIL; a real failure beside a timeout is never rerun. Three new self-tests; the two rerun tests fail against the old script. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The PR 16 release check failed both
dojo-verifyrows at load ~550 (unittest:dojo-verifywith 5 failures,mod:dojo-verifywith exit 1). The same code onmainpasses 153/153 unit tests and 109/109 mod tests (the whole mod suite in about 4s) once the machine is quiet. Running the suites at background CPU priority brings the failures back on unchangedmain. Nothing indojo-verify's logic was wrong; both checks were timing the machine.Cause
unittestTestSpeedandTestTailWindowlimits (0.2s, 1.0s, 1.5s)time.time(), which also counts time spent waiting for a CPUmodoptionstimeout was probed and ignored)At low load the slowest mod test takes 265 ms. Under load, a test that does almost nothing (
DOJO_VERIFY_OFF=1) took 2.6s.The fix
tests/_helpers.pycpu_seconds()sums the CPU time of the test process and its finished children, and the hook runs as a waited subprocess. The limits are unchanged. Windows has no child CPU accounting there, so it keeps the wall clock.release-check.shreruns a mod run once when every failure is a kit timeout. The row then readsWARNwith the first run's failure count and the load average, so a rerun never passes silently. A test that really hangs times out again and staysFAIL. A real failure beside a timeout is never rerun.Checks
dojo-verifyunittestrelease-check.sh --only unittest,mod --plugin dojo-verifydetect_claimfails 18 speed checks at ~0.38s against 0.2sStill open: at background priority, which also lowers the clock speed,
command_eventswith 3,000ifbranches takes 1.09s of CPU against its 1.0s limit (0.35s at normal priority). Real load runs on full-speed cores, so the limit is unchanged, but it has the least headroom of the speed checks. I haven't reproduced load near 550 against this branch.🤖 Generated with Claude Code