Repository navigation
GPU funnel ~1.25x faster with identical candidate output - #36
Closed
legendaryshirt wants to merge 1 commit into
Closed
legendaryshirt wants to merge 1 commit into
legendaryshirt wants to merge 1 commit into
Conversation
legendaryshirt
force-pushed
the
pr/gpu-speedup
branch
from
November 6, 2026 21:44
23f3e61 to
2fbb0a8
Compare
See the pull request description for the per-change breakdown, the measurements and the set-identity check (tools/dump_patch.py). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
legendaryshirt
force-pushed
the
pr/gpu-speedup
branch
from
November 6, 2026 21:46
2fbb0a8 to
9713638
Compare
|
this was closed as per the following discord discussion: https://discord.com/channels/720723932738486323/1406730684537766088/1568747590366007356 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This makes the GPU funnel about 1.25x faster end to end (about 1.29x with
--pipelines 2) without changing what it finds: for the same seeds, every stage produces the same candidate set asmain. That was checked as a sorted (seed, x, z) set comparison, not just a count comparison, for small biomes, large biomes and unbound.Measured on an RTX 4070 Ti (sm_89), CUDA 13.4, 60 s wall clock including startup, default
SEED_ORDER=linear:--pipelines 2Per stage, fixed work (
tools/bench.sh 512: 512 iterations x 2^28 seeds, small biomes, bounded, one pipeline):(Runs on different days drift by about ±3% with GPU clocks; each table above was measured in a single session.)
How output identity was checked
tools/dump_patch.pyinserts the same dump code into any tree. After each offilter_2_01a,filter_gradvecs_2,filter_2_01b,filter_2aandfilter_2b, it syncs the stream and writes every candidate asstage seed x z, before a later stage reuses the buffer. I applied it tomainand to this branch, ran both from the same--start, and compared the sorted outputs:All 15 sets are identical (same SHA-256 of the sorted lines). The earlier stages feed these, and the later stages run the same kernels on identical inputs. A 64-iteration run on small biomes was identical as well.
There are two deliberate differences outside the GPU candidate sets:
output.txt2-3 times. On 4.4 T seeds with every candidate verified, this gave 48 -> 37 output lines with the identical 37 islands.XrsrRandom's constructor. The constructor also replaces an all-zero state, which can only differ for a 2^-128 event.Changes
filter_seeds (integer-ALU bound, ~96% ALU pipe)
nextFloatvalue is converted fromuint32instead ofuint64(exact, cheaper conversion).filter_gradvecs_1 (shared-memory LSU bound)
[513][6]put lanes 24 B apart, a 2-way bank conflict. This overlaps with optimize gpu.cu which leads to a ~4% speedup #28.float2, so one load serves two consecutive candidates, and the last load of each 8-candidate group is carried into the next. That is one load per y-row per candidate.nz,nz+128): the byte indexv + nzis unpacked once, and the second row is the same register plus an immediate offset.score_center_2x2/score_full_12.Noise kernels (filter_2_01a/b and filter_2a..d, LSU-issue bound)
2*gmakes each one a float whose only non-zero byte is the top one (0x40000000,0xC0000000,0). One 32-bit word then holds all three, and one__byte_permrebuilds each, which is 1 shared load per corner instead of 3. Doubling is exact, and callers fold the 0.5 into the octave weight, so every noise value is bit-identical.q[i] = p[i] | p[i+1] << 8. Every lookup in a noise cell comes as an(i, i+1)pair, so a cell needs 7 shared loads instead of 14. The pairs are built from the global octave during the copy each warp already did.Pipeline and host
filter_2_01bcompacted its output over its own input buffer (buffer_2), so a warp could overwrite an input entry that another warp had not read yet. It now writes to its own 4 MB buffer. This is the same fix as in optimize gpu.cu which leads to a ~4% speedup #28. It changes no result in practice, because outputs lagged far behind inputs, but nothing guaranteed that.--pipelines <n>(default 1) runs n independent pipelines per device: oneGpuThreadeach, with its own stream and buffers, sharing the atomic seed range. A second pipeline fills the GPU while the other one is in its small, latency-bound late stages. It needs about 3 GB of VRAM per pipeline, so the default and the README's VRAM requirement stay as they are.MAX_ITERS=n(make): run n iterations, then shut down through the normal graceful path, so queued CPU candidates are still verified.tools/bench.sh [iters] [start]uses it for fixed-work benchmarks. stdout is line-buffered so that logs redirected to a file survive any exit.Opt-in, off by default:
SEED_ORDER=goldenSeed index n is checked as seed
(n * GOLDEN) ^ SILVER. That is a bijection, since GOLDEN is odd, so it still covers every 64-bit seed exactly once. Seed n's secondmix64is then seed n+1's first, which makes filter_seeds about 1.08x faster. It changes which seeds a given--startcovers, so it is not the default, and such builds use a different network handshake (OMISSION-GPZ) so they cannot share a server with normal builds.Not included / related
ifeq (LARGE_BIOMES, 1)suffix bug is left to Fix variant suffix resolution and binary target name in makefile #33.CI replica: the Linux build job of
build.yml(nvidia/cuda:12.9.1-devel-ubuntu22.04,make ARCH=all CC=gcc-11 CXX=g++-11) builds cleanly forLARGE_BIOMES=0 UNBOUND=0andLARGE_BIOMES=1 UNBOUND=1. I have not built it on Windows, and I have tested only on sm_89. Results on other GPUs would be welcome.🤖 Generated with Claude Code