Repository navigation
Perf/ticker wheel - #1672
Draft
nightNR wants to merge 2 commits into
Draft
Perf/ticker wheel#1672nightNR wants to merge 2 commits into
nightNR wants to merge 2 commits into
Conversation
The timers of the objects and the periodic ticks of the characters were kept in sorted vectors, rebuilt every tick from queued add/erase requests. With many timers this dominated the tick time. Use a timing wheel instead (CTickWheel): slots of 25 ms covering a window of TimerWheelSlots slots (sphere.ini, default 131072, ~55 minutes), plus an overflow list for later timers, scanned incrementally and moved to the wheel before they expire. The link lives inside the object, so adding, removing or rescheduling a timer is O(1) and never allocates. Each tick only visits the slots elapsed since the previous one. The entries collected in a tick are sorted by expiration time, then by insertion order, the same order given by the previous sorted container.
Check the slot count normalization, the expiration boundary and the order of entries expiring at the same time, the window boundary, time jumps around one revolution, the clock not moving, going back or negative, very far timers, many entries in one slot, and the overflow entries firing on time. Game-like cases: entries removed before expiring, or removed and rescheduled while the overflow scan is in progress (with the memory of the removed entry overwritten, as for a deleted item), timers set again from their own tick or changed while pending, sleeping objects, clear and reuse. A randomized test compares every collected batch, including its order, with a reference ordered container.
6 of 12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Problem.
CWorldTickerkeeps every pending object timer (TIMER, decay, spawns, memory items, spell effects, ...) and the periodic tick of every awake character in sortedstd::vectors. In every tick with at least one add or remove request the whole vector is rebuilt: removed entries are copied out, added ones arestd::merged in, and expired ones are erased from the front. On a live shard awake NPCs reschedule every tick, so the rebuild runs in practically every tick and costs O(total timers), even though almost all timers are just waiting. With many timers this dominates the tick time.Change. The timers are scheduled by a timing wheel (
src/game/CTickWheel.h):TimerWheelSlotsslots (newsphere.inisetting, default 131072, about 55 minutes; rounded up to a power of 2, clamped to 256..4194304, read at startup, the slot count and window length are logged);(N = pending timers, C = add/remove requests in the tick, k = timers expiring in the tick.)
Benchmarks (Release, Windows,
NetworkThreads=0, client logged in, 45 s runs,Tick()time summed over 10 s windows):Tick()per 10 s@Timerscripts/s)Tick()per 10 sTick()per 10 sWith many timers the tick is about 5x faster and uses about 3x less CPU. Under heavy script load (scripts take ~85 % of the tick) it still saves about 16 % of the tick time and 17 % of the CPU. In a small world the difference is within the measurement noise (picking the expired timers is only 5-10 % of the tick there). The machine is noisy (up to 2x between runs of the same binary), so the numbers above compare runs of the same batch.
Type of Change
Related Issues
Addresses item P1 of the performance audit in #1671 (world timers rebuilt every tick).
Alternative implementation of the same change: #1673 (ordered container,
perf/ticker-tree). Only one of the two should be merged; see "Alternative" below.Testing
Unit tests (
tests/src/t_CTickWheel.cpp, doctest,-DUNIT_TESTING=ON): 25/25 test cases, 3,601,519 assertions, 0 warnings on Linux GCC and Clang (Nightly). They cover slot count normalization, the expiry boundary and the order of equal timeouts, the overflow list (each timer fires in the first tick after its time, never earlier), the clock going back or jumping over many revolutions, sleeping entries,Clear(), removal or rescheduling while the overflow scan is in progress (with the removed entry's memory overwritten), and a randomized comparison (2 wheel sizes x 40 seeds x 3000 ticks of random add, cancel, reschedule, sleep and clock moves) where every collected batch, including its order, must match a reference ordered container. A mutant without the cursor fix in the overflow scan is caught by two test cases.In-game tests on a clean instance with the latest ScriptPack and a real client, Debug builds (all asserts active), before vs after. Every timer checks it fires not early, not more than 500 ms late, not twice, not after being removed or cancelled, and in order where required. 24 scenarios: ordering (also through the overflow list), equal due times, removal before/after firing, cancel and re-arm, self-rescheduling and self-removal from
@Timer,TIMERMS=0,TIMER/TIMERDunits, negative timers, a stress test of 1000 timers with random removals and reschedules, equipped items, NPC periodic ticks with chars killed/removed during the tick, contained timers, decay,CAN_O_NOSLEEPin sleeping sectors, spawns, items moved between ground/backpack/equipped while a timer is pending.ItemTimers=1No error, assert or crash in any run. Lateness and the behavior of the informational scenarios (sleeping sectors, containers, dupe) are identical before and after.
Checklist
sphere.ini,Changelog.txt)No script-visible change: timers fire at the same times. The only difference is the order of timers with exactly the same expiration time: before it was arbitrary (sorted by object pointer), now it is the order they were set.
Alternative: ordered container (#1673)
The other PR keeps the same scheduling model but replaces the sorted vectors with a
std::multimap; each object keeps its own node and reuses it when rescheduled. O(log N) insert, and the tick only looks at the front of the container.Tick()per 10 s, 200k timersTick()per 10 s, heavy reschedulingTree pros:
sphere.inisetting;Tree cons:
Wheel cons:
The wheel is faster or equal in every scenario; the tree never beats it. The tree is the simpler option if reviewability matters more than the last 10-15 %.
Additional Notes
@Timerof A removes B, or sets B's timer to later, and B expired in the same tick, B's@Timerstill runs (on an object being deleted), and the new timeout is cleared afterwards (the reschedule is lost). Fixing it changes script-visible behavior, so it is left for a separate PR; the wheel already keeps the data needed for that check.NetworkThreads=0: withNetworkThreads=1the current code (also without this PR) crashes intermittently inCNetworkOutput(purecall / access violation), about 3 runs out of 15.