Repository navigation
The benchmark is measured after the library's set-up - #2191
Merged
Rafael-SOWNet merged 1 commit intoOct 28, 2026
Merged
Conversation
The Kernel Benchmark gate failed twice today on unchanged library code: on #2187's first run, and on #2188 after its rebase. Both times it was SimplifyEasy, SolveEasy and EvalTrigPrecise, at 3-12x their time, and SimplifyEasy at 3% more allocation. Their re-runs matched the baseline to the byte. BenchmarkDotNet times one call before it decides how to measure, and where that call takes more than two thirds of its 500 ms iteration, it times one call per iteration from then on. The first call of those three took 349-362 ms on the failing runs and 252-300 ms on the passing ones. For a call of tens of microseconds, one call per iteration is another measurement. That first call paid for parsing every static expression of the class and for the library's own set-up: EvalEasy, a cache lookup, took 133 ms the first time on one machine. A [GlobalSetup] now touches the class and runs one simplification, one solve and one evaluation before the timed call. Measured on one machine, master against this, the first call drops from 172 to 8 ms (SimplifyEasy), 164 to 5 ms (SolveEasy) and 158 to 45 ms (EvalTrigPrecise). What the gated benchmarks allocate is unchanged: SimplifyEasy is 79,682 B/op on both, and the largest move is ParseEasy's +0.11% (21 B/op). The two compilation benchmarks, which the gate leaves out for their own spread, moved -0.5% and +2.0%. SolveHard, a call of 70 ms, is now timed one call per iteration rather than piloted, and allocates 9,832,912 B/op against 9,831,370. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #2191 +/- ##
==========================================
+ Coverage 86.52% 86.53% +0.01%
==========================================
Files 297 297
Lines 66161 66324 +163
Branches 19237 19267 +30
==========================================
+ Hits 57245 57395 +150
- Misses 4851 4856 +5
- Partials 4065 4073 +8
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Rafael-SOWNet
deleted the
the-benchmark-is-measured-after-the-librarys-set-up
branch
October 29, 2026 00:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Kernel Benchmark gate failed twice today on unchanged library code: on #2187's first run, and on #2188 after its rebase. Each time SimplifyEasy, SolveEasy and EvalTrigPrecise took 3–12x their time, and SimplifyEasy allocated 3% more. Both re-runs matched the baseline to the byte.
BenchmarkDotNet times one call first, and when that call takes more than two thirds of its 500 ms iteration, it times one call per iteration from then on. The first call took 349–362 ms on the failing runs and 252–300 ms on the passing ones. It includes parsing the class's static expressions and the library's own set-up.
A
[GlobalSetup]now does both before the timed call. Measured on one machine, master against this branch:Allocation is unchanged. SimplifyEasy is 79,682 B/op on both, and the largest move among gated benchmarks is ParseEasy, +0.11% (21 B/op). SolveHard, at 70 ms a call, is now timed one call per iteration instead of piloted, at 9,832,912 B/op against 9,831,370.
🤖 Generated with Claude Code
https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura