Skip to content

Broad desync fixes (join flow, simulation contexts, multifaction, tick loop) and smaller bug fixes - #996

Open
WoodyWizard wants to merge 198 commits into
rwmt:devfrom
WoodyWizard:fix/desync-review
Open

Broad desync fixes (join flow, simulation contexts, multifaction, tick loop) and smaller bug fixes#996
WoodyWizard wants to merge 198 commits into
rwmt:devfrom
WoodyWizard:fix/desync-review

Conversation

@WoodyWizard

@WoodyWizard WoodyWizard commented Oct 19, 2026

Copy link
Copy Markdown

This PR makes broad, global changes aimed at fixing desyncs, the most common complaint, and fixes a number of smaller bugs along the way. It comes from a full review of the desync-relevant code (command log and join points, server, RNG and simulation contexts, sync framework, sessions, multifaction, async time and the tick loop, storyteller, gravship), followed by a review of all 28 open PRs and about 60 open issues against the result. Useful parts of open PRs were ported or reimplemented (credited below), and the remaining issues were fixed.

It is large: 182 commits, 192 files, +8957/−1468, on top of dev at e760d30. If you prefer, it can be split into smaller PRs. The branch was assembled from 15 topic branches (command log, net, determinism, sync, rng, sessions, ...), which are natural split points, though later ones build on earlier ones. The full categorized changelog, with every commit, is in docs/desync-review.md. That file is only for review and can be dropped before merging.

Highlights

  • Joiners replayed commands in a different order than live clients. The server stored commands per map, and joiners ran "world first, then each map". Live clients run one queue in arrival order. There is now one globally ordered command log (7d09265).
  • Join points are atomic and have one uploader. On standalone servers only the issuing client reloaded, and it drifted from everyone else. Now every client reloads. The server picks the uploader (arbiter, host, or the longest-connected in-sync player) and swaps world, maps and log together. Stalled join points are aborted (bdf1995, 940a691, edd7d3c).
  • Arbiter games froze for everyone when someone joined. In batch mode the arbiter never started the join point long event, and the server waited for it forever. This was reproduced on dev (c877df8).
  • Reloads ran under ProgramState.Playing, so load-time code behaved differently on reloading clients than on joiners (d67d559, Resyncs run with wrong ProgramState #854).
  • VTR viewer counts differed or were wrong.
    • The counts weren't saved, so a reload reset them.
    • The host's map was never counted, so it ran at the no-viewer rate.
    • Multifaction joiners were never counted.
    • The arbiter counted as a viewer.
    • Fixed in 7b666d7, 6ad2ef2, 65bfd9d.
  • RNG isolation.
  • Sim long events ran outside any context. Long events queued from ticks and commands ran later in the frame. In multifaction this meant map generation for each client's own faction, and actions deferred from such events ran as interface code. They now run in a captured SimLongEventContext (619bc19, 389678c).
  • Map time contexts didn't nest. A quest time getter inside a map tick left the map's clock installed, so the world clock jumped. The same commit makes the tick loop skip maps that are still being generated (822b44c, BUG: Launching Gravship Throws Null Reference Exception #789).
  • Storyteller and quest incidents were multiplied.
    • Adaptation grew 1+N times as fast with N maps.
    • Every player faction's storyteller fired the same quest threats.
    • Other quest incident makers fired 1+N times.
    • Quest threats never fired in singleplayer with the mod loaded.
    • Fixed in 5fb7c98, 46c06a7, 99d1da2, 17c7354.
  • World and maps tick in lockstep without async time. Previously they drifted apart by a few ticks (291c3c4, World ticks can get ahead of map ticks/map ticks can get out of sync with each other (without async mode) #339).
  • Multifaction faction data is deterministic.
    • Forced faction pushes apply the faction to every map.
    • Faction creation produces the same data on every client.
    • Every client keeps the same letter stack.
    • Season notifications run per faction.
  • Gravship takeoff/landing ends ran per frame in the interface, at a different moment on each client. They now run as synced commands, and the ship is placed and removed in its map's context (f8d9f49, bdb2515, 2b952b9).
  • Server thread races.
    • The server join flow ran on a thread-pool thread, concurrently with the server loop (d2cf042).
    • Standalone servers never compared sync opinions, so desyncs went undetected (6c48789).
  • False desyncs and stalls from the stack tracer.
  • Exception safety. Field watches and sync flags are restored in finalizers. An exception in one tick subsystem no longer skips the rest of the tick.
  • Big commands are sent fragmented, up to 1 MiB (Certain interactions can not sync and cause an error due to packet size being too big or cause remote packet read error #415).

Fixes

Likely fixes (not confirmed, so not auto-closing)

Partially addresses

Can be closed separately:

Ported / superseded PRs

Testing

Builds and unit tests

  • The solution builds and 284 unit tests pass.
  • New tests cover:
    • the command log, join points and the joining state
    • fragmented packets, large commands and reader bounds
    • reconnects and server stop
    • the sync reference, standalone persistence and world data size
    • selection packets, line endings and log trimming
  • Each stage was reviewed separately, and its findings were fixed before it was merged into the branch.

In-game tests

  • Setup: several real RimWorld 1.6.4850 instances with all DLCs on Linux, headless (Xvfb, software GL).
  • A harness mod drives the game through normal sync paths and logs the sync state every 5 s. The paths are draft/undraft, mining, stockpiles, work priorities, forbid, plant cutting, raids, fires and incidents.
  • Scenarios:
    • hosted with an arbiter and a joiner joining mid-game
    • hosted without an arbiter
    • multifaction, with the joiner creating its own faction and map
    • async time
    • standalone server: two players, a server-driven autosave join point, a restart from the persisted state, and a start from an older save

Results

  • dev:
    • 1 desync in 4 valid runs ("Random state from commands doesn't match" during the joiner's catch-up).
    • The arbiter game froze for everyone as soon as someone joined.
  • This branch:
    • No desyncs in about 30 sessions, the last ones a full run of every scenario on the complete branch.
    • VTR viewer counts were identical on all instances.
    • World and map ticks were identical without async time.

Not covered in game

These paths were checked by code review only: Steam, Windows/macOS clients, other mods, gravship flights, rituals, trading/caravans and large mod lists.

Compatibility and behavior changes

Protocol

  • The protocol goes from 56 to 57.
  • What changed on the wire:
    • Server_WorldData now carries a single command list.
    • The join data now includes missing defs and the game language.
    • The CreateJoinPoint payload names the uploader, and uploads and save requests carry the tick.
    • There is a new Server_SyncReference packet.
    • Large commands are sent fragmented.
    • The sync data for unheld pawns, genes and WorldSelected changed.
  • Coordination needed: Fix VTR viewed-map flush after reload #956 also uses 57 and Desync fixes, multifaction routing, and failure surfacing #961 uses 58, both with different wire formats. Whichever PR lands first needs the number adjusted in the others.

Replays and saved data

  • Replays recorded before this branch are not compatible: sync handler ids, command formats and tick order changed.
  • Standalone servers store the log in Saved/cmds.dat, with its protocol in Saved/cmds_protocol.txt. A log from an older server is discarded on start; the saved world, maps and session still load.

Join and server behavior

  • Every client reloads at standalone join points, and only the sync reference uploads.
  • Join points abort after 60 s without progress. Forced join points are limited to one per 10 s.
  • Minute-based standalone autosaves are issued by the server. Join point requests from world travel are ignored.
  • Clients can't send server-only commands. Only the host can move other players between factions.
  • Timeouts:
    • handshake: 2 min, or 5 min for the init data upload, or 15 min for steps behind user input
    • in game: a player is dropped after 10 min without a keep-alive reply
  • The join check also compares Multiplayer's own assemblies, missing def types and the game language. It ignores line endings in configs and XML.

Gameplay-visible changes (all deterministic)

  • Storyteller adaptation grows once per game tick. Before, it grew 1+N times with N maps.
  • Multifaction storytellers only target their own maps, or unowned maps where they have pawns.
  • Quest threats fire once per map.
  • Quest threats fire again in singleplayer with the mod loaded.
  • Without async time, world and maps tick in lockstep, with no 1–3 tick drift. At speeds above 1x the order is W M M W M M instead of W W M M M M.
  • The arbiter doesn't count as a VTR viewer; the host's map is counted.
  • Caravan meeting and demand dialogs pause the game.
  • Rituals start with the starting player's assignments.
  • Zone/plan cells are shuffled again in singleplayer.

New UI

  • a confirmation dialog before hosting or joining when there are loading errors
  • a Prepatcher-missing warning
  • a Save button after a disconnect
  • a language line in the join data window
  • an explanation for server state exception disconnects

Translation keys

  • New keys, with English fallbacks in code, to add to Multiplayer-Locale: MpMismatchLanguage, MpPrepatcherMissing, MpPrepatcherMissingDesc, MpSaveGameAfterDisconnect, MpServerStateException, MpServerStateExceptionDesc.
  • A few new messages are English only for now:
    • the loading errors confirmation
    • "action too big to send"
    • the skipped simulation error notice
    • "world data too big"

Removed

  • the consistentCommandOrder tweak value
  • the PlaceGravship sync method registration
  • PushFactionOutsideContext/PopToLocalFaction
  • the dev tick-once hotkey outside replays

Not addressed

🤖 Generated with Claude Code

WoodyWizard and others added 30 commits October 17, 2026 15:34
@WoodyWizard @claude
Isolate RNG in sound and music entry points
675ec35 
Fire.TickInterval calls SustainerAggregatorUtility.AggregateOrSpawnSustainerFor,
which constructs a SoundSizeAggregator (Rand.Value) only when no aggregated
sustainer exists locally. Sustainer lifetime depends on the local sound manager
(camera, frame timing), so the map RNG advanced on some clients only (rwmt#618).

Push/pop the RNG around the sound entry points reachable from ticks
(PlayOneShot, TrySpawnSustainer, Sustainer ctor, the aggregator and its ctor)
and the public music manager triggers. The push is skipped off the main thread
so async long events don't race on Rand's state stack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Run gravship takeoff/landing end in a synced command
f8d9f49 
TakeoffEnded and LandingEnded are called by vanilla from
WorldComponentUpdate, i.e. per frame in the interface, at a different
moment on every client. The landing outcome, session closing and map
speed change ran there with interface semantics: synced methods called
inside were sent as commands by every client (rwmt#640), unique IDs were
local, Faction.OfPlayer was the local faction and RNG was popped.

When the local cutscene ends, each client now unfreezes and sends a
synced command (landing as a command of the landing map, takeoff as a
world command). The first one to execute runs the vanilla method on all
clients; later ones are ignored using a pending flag that is only
changed from command execution. The cutscene is held on its last frame
until then. In multifaction the gravship's faction is pushed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Fix inverted check in the zone/plan cell shuffle patch
83d0cc8 
The transpiler emitted 'call ShouldShuffle; not; or'. OpCodes.Not is a
bitwise not, so the condition was always non-zero and Zone/Plan cells
were never shuffled, even in singleplayer.

Restore the vanilla shuffle in singleplayer and keep MP on the
long-standing (deterministic) no-shuffle behaviour: the shuffle uses
Rand and the shuffled flag isn't saved, so after an asymmetric reload or
a UI access it could differ between clients. Log an error if the patch
site isn't found.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Keep the server join flow on the server thread
d2cf042 
TypedPacket<T>() wrapped the packet task in ContinueWith without
ExecuteSynchronously. The server thread has no SynchronizationContext,
so everything after the first awaited typed packet in
ServerJoiningState (PauseAll, join point creation, OnJoin, the
InitPlayerData command, disconnects) ran on a thread pool thread,
concurrently with the server loop. That raced CommandHandler.Send
(mapCmds, SentCmds, per-peer send order) and player list mutations.

Typed packet waits now use PacketAwaitable directly: the packet is
bound and the awaiting state resumed inline by the packet handler, so
the whole async state runs on the thread that handles the packet.
PacketAwaitable tracks completion explicitly and supports exceptions.

Add MultiplayerServer.CheckServerThread() and call it from
CommandHandler.Send and PlayerManager.OnJoin/SetDisconnected. It logs
(rate limited) when they run off the server thread once it started.
The loopback join test now fails on a regression; the join point test
ends the join point on the server thread like the real code does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Allow PriorityWork.Clear during ticks
393ac90 
The patch meant to stop the interface (IsPrioritized timeout check
from the gizmos) from clearing prioritized work, but its guard only
allowed command execution. Vanilla's tick-time calls (JobGiver_Work
after the prioritized work is exhausted, the timeout check while
thinking) were dropped too, leaving the pawn prioritized until the
timeout. Block only interface calls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Make the deferred stack tracer fail safely
93d6f86 
The tracer runs as a postfix of Rand.Value/Rand.Int. Any exception in it
aborted the game code after the RNG had advanced, on that client only:
- GetStackUsage threw "Unknown function header" (and left a half-filled cache
  entry that later walks followed with a bogus frame size)
- frames skipped for AggressiveInlining were stored without counting toward
  the 32-frame cap, so traceIn could overflow
- the LMF walk dereferenced previous_lmf before the zero check and without
  masking Mono's flag bits (the NRE from rwmt#944)

Now unknown frames are cached as such and end the walk, LMF pointers are
masked and checked (ext/trampoline LMFs end the walk), rbp must move up the
stack, the walk has a step cap and the postfix never throws. A failed walk
still records a placeholder trace so trace counts stay aligned, and marks the
opinion so its trace hashes aren't compared (sent as "simulating") instead of
producing a false "Trace hashes don't match". The failure is logged once.

Frames skipped for AggressiveInlining no longer count toward the recorded
depth or the hashed frames either, since whether they appear can differ
between platforms. Traces are only taken on the main thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Keep trace hash depth independent of the tick loop call site
059510e 
The desync trace hash includes the stack depth. Commands run either from
RunCmds in TickPatch.Prefix or one frame deeper from the DoUpdate loop,
depending on local frame timing, so identical clients could report "Trace
hashes don't match" (rwmt#994). Route the Prefix call through a NoInlining
wrapper so both paths have the same depth.

The dev tick-once hotkey called DoUpdate from OnGUI on a single client; it is
meant for replays only, so restrict it to them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Use deterministic job-loop counter for pawns without a map
fa7f328 
Pawn_JobTracker.StartJob compares RealTime.frameCount to detect job
loops. The patch replaced it with the map's event counter, but for
pawns that aren't spawned (carried, in containers, in caravans) Map is
null and it fell back to the frame count, which differs between
clients. Use MapHeld and fall back to the tick timer. Also log an
error if the transpiler finds nothing to patch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Tick the UI tile temperature cache from the interface
873f837 
The tick patch only ticked uiTemperatures when the real comp's
WorldComponentTick ran in the interface, which never happens (it's only
called from the world tick). UI temperature entries were therefore
never refreshed or evicted. Tick the UI cache at most once per frame
when the interface reads from it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Fix init data requests that hang every later join
7c88a31 
- SetDisconnected marks the state dead before OnDisconnect. OnDisconnect
  synchronously resumes a pending init data request; the joining loop
  then saw InitDataState.Waiting on a still "alive" state and requested
  the init data again from the dying connection. That orphaned request
  kept the server in InitDataState.Requested, so every later joiner
  waited forever (standalone first joiner leaving mid-handshake).
- RequestInitData always completes the source, also when
  ServerInitData.FromNet throws (e.g. duplicate def names). The player
  is then disconnected and the next joiner gets asked.
- Waits in AsyncConnectionState take an optional timeout; the server
  loop disconnects players whose awaited packet doesn't arrive in time
  (PlayerManager.CheckTimeouts, once per second). ServerJoiningState
  uses 2 minutes for automatic replies (protocol, init data, join data)
  and 15 minutes for packets behind user input (password, mismatch
  window). Idle handshakes no longer hold a player slot forever.
- Restore a real limit for ClientInitDataPacket.rawMods: rwmt#913 moved it
  back to the default 32 KiB of BindRemaining. Raise the compressed mod
  list limit to 4 MiB and use the same limit for the forwarded copy in
  ServerJoinDataPacket (was 512 KiB, too small for big synced configs).
- Stop the joining flow right after a protocol/username disconnect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Restore watched sync fields even if the watched code throws
a59e1c8 
The field watch postfix didn't run when the patched UI method threw, leaving
the user's change (or the pending buffered value) in the simulation of that
client only, and the watch stack unbalanced.

- Patch watched methods with a finalizer instead of a postfix.
- Float menu/dropdown watch wrappers use try/finally instead of a multicast
  delegate.
- Always pop the whole watch scope, even if handling one entry throws.
- Flush leaked scopes (e.g. API WatchBegin without WatchEnd) once per frame.
- Skip nested watches of a field already watched by an enclosing scope
  (Bill_Production.paused is watched in both Bill.DoInterface and
  ShouldDoNow, which reverted the buffered value mid-scope).
- Restore a null actual value instead of falling back to the UI value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Complete path jobs right after PathFinderTick
9a7c278 
The only protection against path jobs racing main thread changes
(rwmt#900) is ForceCompleteScheduledJobs after MapPreTick; the snapshot
patch in PathFinderPatch.cs is compiled out. Also complete the jobs in
a PathFinderTick postfix so they can't overlap the rest of MapPreTick
or calls from elsewhere, and mark the dead code as reference only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Reject server-only commands and foreign faction changes from clients
dd969f9 
HandleClientCommand forwarded any CommandType with any map id. Clients
only send Sync, Designator, DebugTools, GlobalTimeSpeed, MapTimeSpeed,
TimeSpeedVote and PlayerCount; PauseAll, CreateJoinPoint and
InitPlayerData are issued by the server alone. A modified client could
e.g. send InitPlayerData to grant dev mode to any player, or spam
pauses and join points. Unknown types and map ids outside the scope
the client uses (Global for global commands, >= 0 for map commands,
either for Sync/DebugTools) now fail as a bad packet, which disconnects
the sender. Validating against the set of existing maps is not done:
the server doesn't know maps created since the last join point.

HandleSetFaction let anyone move anyone to any faction. Players may now
only change their own faction; moving others (factions window drag and
drop, which the UI only offers on the hosting client) is host-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Balance faction context pushes in trading UI and map faction ticks
b6f1126 
FactionContext.Push always pushes an entry (null when the push is a
no-op), but TradingWindow only popped when Push returned non-null, so
every frame of an own-faction trade leaked a stack entry. Always pop.

MultiplayerMapComp.DoTick popped the faction without try/finally, so an
exception in a faction's lister/resource tick left the faction context
and map managers of that faction applied for the rest of the tick.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Don't mutate live sessions while saving; fix inverted checks
22f958b 
SessionManager.WriteSessionData removed invalid semi-persistent sessions
from the live lists (without PostRemoveSession). Saving can happen on a
single client (host autosave, manual save), so the session lists, and
with them map pausing, could differ between clients. Invalid sessions
are now only left out of the written data. The 'log once per type'
conditions were inverted (logged only from the second occurrence on).

CaravanFormingSession scribed its onClosed Action with Scribe_Values,
which can't round-trip (logs a parse error on load). Stop saving it and
document that it's intentionally never invoked.

HandleMissingDialogsGameStart returned when there were trade sessions,
making the rest of the method dead code (inverted in rwmt#401).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Reset sync state flags in finalizers instead of postfixes
2aec6ac 
Add an MpFinalizer patch attribute and use it where a skipped postfix leaves
global state broken for the rest of the session:

- StorageSettingsClipboard.Copy set Multiplayer.dontSync in a prefix and
  cleared it in a postfix. If Copy threw, every later sync handler ran
  local-only. Restore the previous value (so an outer dontSync scope isn't
  clobbered) in a finalizer.
- ThingFilterMarkers cleared the drawn filter context in postfixes. One
  exception while drawing made the setter throw "context already set" on
  every following frame (or left a stale context for unrelated filters).
- The growth moment letter's seeded Rand state is popped in a finalizer,
  only if it was pushed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Drop partially received fragmented packets on state change
e3edd35 
The fragment buffer allows one fragmented packet in flight and survived
connection state changes. When a client changed state mid-transfer
(e.g. rejoining while a Server_SyncInfo was arriving), the remaining
fragments were dropped in lenient mode but the partial entry stayed;
the first Server_WorldData fragment then failed with "High number of
fragmented packets at once!" and the rejoin broke (likely rwmt#691).

ChangeState now discards partial packets and the remaining fragments of
a discarded packet are skipped by fragment id, whatever the new state
handles. A fragment without a handler (lenient mode) drops the rest of
its packet the same way instead of logging each fragment, so a later
state can't mistake a continuation fragment for a new packet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Keep one globally ordered command log for joiners
7d09265 
Live clients execute every command from a single queue in arrival order
(rwmt#734), but the server stored commands per map and joiners rebuilt per-map
queues, running "world first, then each map". Any tick with commands for
two tickables then ran in a different order on joiners, and AsyncTimeComp
.FinalizeInit replaced a map's queue with the snapshot list, which could
drop map commands received during the asynchronous load or execute them
twice for maps generated during play.

Server: WorldData keeps a single list of serialized commands in send order
and Server_WorldData sends it as one list. Standalone persistence stores it
in Saved/cmds.dat; legacy world_cmds.dat + maps/N_cmds.dat are still read
(merged by tick) and removed on the next join point.

Client: GameDataSnapshot holds one ordered command list. Every command is
queued in the world queue; while a (re)load is in progress commands are
only appended to the snapshot list and Loader.PostLoad (main thread) builds
the queue from it, so nothing is lost or duplicated. FinalizeInit no longer
touches queues. Replays store all commands in world/N_cmds and still read
old per-map entries. The consistentCommandOrder tweak value is removed.

RunCmds no longer blocks forever on a command for a past tick; it logs an
error and drops it (the loaded state already contains it).

Protocol bumped to 57 (Server_WorldData format).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Keep thing filter edits out of the sender-only simulation
85066f3 
- The hit points / quality / mental break range setters invoke the filter's
  settingsChangedCallback, which for storage rechecks the map's haulables
  (StorageSettings.TryNotifyChanged). The buffered field watch wrote the
  pending value and restored the real one through these setters every frame,
  and the slider itself calls them in the UI, so the haulables list changed
  on the sender only. In the interface, write the backing field only for the
  drawn (synced) filter or a storage filter; the callback runs everywhere
  when the synced field is applied.
- When the storage tab's selection has no map no context is created and
  SetAllow/range edits ran locally. Block them instead.
- TabStorageWrapper now supplies ITab_Storage.HiddenSpecialThingFilters(),
  so "Clear all" and category toggles leave hidden special filters (e.g.
  AllowFresh) alone like in vanilla.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Validate command size before adding it to the log
ec1900b 
Client command data was allowed up to 65535 bytes, but the server's
rebroadcast adds a 17 byte header and Send throws above
MaxSinglePacketSize. The throw happened after the command was appended to
the log, so joiners executed a command live clients never received.

Command data is now capped at what fits the rebroadcast
(ServerCommandPacket.MaxDataLength) for both packet directions, and
CommandHandler.Send rejects oversized data before logging anything.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Make growth moment sessions survive the child leaving the map
de1999a 
The session's map was letter.pawn.Map, so once the child left the map
(caravan, transport pod, death, kidnapping) Tick() threw inside
TickMapSessions every tick, GetSessionFor threw and the session could
never be removed. The window's PostClose also ticked the session from
UI code, which could remove it on one client only.

- Store the session's map (saved sessions get it from the loader) and
  treat a child that's dead or not on that map as an invalid session,
  removed deterministically by the session's tick.
- Look sessions up on every map, never through pawn.Map.
- PostClose only reopens the window (UI), it no longer ticks the session.
- The timeout window opened by the tick is only shown to players of the
  child's faction.
- Generate the letter's options when the session is created on every
  client, instead of only on clients that open the window.
- SessionManager.TickSessions catches and logs exceptions per session so
  one broken session doesn't skip the rest of the map/world tick.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Sync colonist bar reordering with pawns resolved on the sender
dce32db 
The reorder was synced as the ColonistBar.Entry closure, which only captures
the entry group and then calls Find.ColonistBar.Reorder(from, to, group) on
each client. The indices and group were resolved against every client's own
cached bar entries, which differ in multifaction, with hidden maps, or when
the receiver hasn't recached its bar yet, so the saved displayOrder diverged.

Intercept ColonistBar.Reorder in the interface instead, resolve the dragged
pawn, the target pawn and the group's pawns from the local bar and sync
those. The synced method applies the vanilla reorder rule to the group and
all pawns of the faction (a deterministic superset of the bar's pawns).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Apply deterministic HashCode.Combine patches early and always
6e8249c 
The int-only System.HashCode.Combine patches switched between the
random-seeded and the deterministic hash on Multiplayer.Client != null,
so the hash function changed under hash containers that were alive
across connecting/disconnecting, and containers filled during def
loading kept a per-process order. Apply them as early patches (before
def loading) and unconditionally.

Also document why Tradeable's identity hash is acceptable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Validate remote lengths before allocating
dba4ff2 
A length prefix claiming int.MaxValue made the host try to allocate
2 GB before noticing the data isn't there:
- ByteReader.ReadRaw checks 0 <= len <= Left before allocating (all
  prefixed byte reads go through it); SubArray validates its range.
- ReadPrefixedInts/UInts/ULongs/Strings check the count against the
  remaining bytes.
- PacketReader.BindBytes honours maxLength (it ignored it, so the
  Gzipped mod list limit from rwmt#878 never applied). The writer checks it
  too. Desync traces used the 32 KiB default unknowingly and now
  explicitly allow up to the fragmented packet size.
- List/Dictionary binds don't preallocate more than the remaining data.
- Gzipped caps the decompressed size (64 MiB default, also for the mod
  list with synced configs) against decompression bombs.

Large legit payloads (world data, snapshots, saves, traces) are bounded
by the packet size limits only, as before. No wire format change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Look up portal/transporter sessions by their target, fix forming faction
ee6e9af 
- MapPortalProxy found its session with GetFirstOfType although a map
  can have one portal session per portal, so a second portal's window
  edited the first portal's transferables. Look the session up by portal.
- MapPortalSession.Map was portal.Map, null once the portal despawned
  (Remove() threw). Store the map; a despawned portal no longer pauses
  the map or offers the blocking-window option, and its window closes.
- CreateTransporterLoadingSession reused any loading session on the map
  for a different group of transporters, and the proxy looked up the
  first one as well. Sessions are now matched by transporters, and the
  owner faction is taken from the transporters.
- Caravan forming sessions created during ticks used Faction.OfPlayer,
  which is the spectator on maps without a player owner: every player's
  controls were disabled while the session paused the map. Use a player
  faction that has pawns on the map instead (multifaction only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Keep sync serializer reader and writer aligned
5238392 
- Writers of implementation-indexed types (SyncWithImpl, BindType, the
  ThingComp/HediffComp/AbilityComp/MapComponent/WorldObjectComp/
  WorldComponent/GameComponent workers and ISyncSimple) wrote 0xFFFF for an
  unknown type and then the object's data anyway. The reader treats 0xFFFF
  as null and doesn't consume that data, misaligning the rest of the
  command. Throw on the writer instead, so the command isn't sent at all.
- When an implicit sync worker returned false, the writer was truncated to
  the start of the worker, dropping the parent type's data already written,
  while the reader re-read the parent's data. Roll both back only to the
  position after the parent's data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Don't run caravan splitting commands through the local dialog
e0161aa 
CaravanSplittingSession's synced methods used its 'dialog' field, a UI
window that only exists on clients that opened it (and is null after a
reload on every client):
- CancelSplittingSession threw on other clients before RemoveSession,
  leaving the session (which pauses everything) alive there.
- AcceptSplitSession split the caravan only where the dialog existed,
  using its possibly stale mass caches.

Accept now splits through a fresh proxy built from the session's
transferables, UI closes are null-safe and open proxies close
themselves once the session is removed. The window is no longer opened
from session creation on every client: only the caravan's players get
it (unchanged for single faction games), and only if the session was
actually added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Recache trade deals in ticks and commands instead of the UI
3f41f92 
MpTradeDeal.Recache() adds/removes tradeables and reassigns stacks, but
it was only called by TradingWindow, i.e. on clients that had the window
open, using the local client's faction data. Count changes are resolved
by thing id against deal.tradeables, so once one client recached the
others resolved them differently, and TryExecute traded different items.

The deal is now recached (when dirty) by the session's world tick and
before resolving a synced transferable reference, both of which run on
every client, with the negotiator's faction data forced. The UI only
follows the deal through uiShouldReset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Save VTR player counts and clamp the world count
7b666d7 
The per-map and world player counts decide VTR, which affects the
simulation, but they weren't saved, so every reload reset them to 0.
On standalone servers a join point reloads only the requesting client,
leaving its VTR different from everyone else's; elsewhere viewed maps
ran at the slow rate until someone switched maps. Save them with the
async time comps. Also clamp the world count at 0 like the map count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Time out unresponsive players and clamp client timing reports
b2fb3f1 
keepAliveAt was written but never read. LiteNet pings from its own
thread, so a hung game stays connected while its ticksBehind pauses
the simulation for everyone, indefinitely. CheckTimeouts now drops a
player in game (ServerPlaying, status Playing) that hasn't answered a
keep alive for 10 minutes. The in-memory host connection is exempt and
the timer restarts when the playing state starts and on world ready,
because loading a big world blocks the client's main thread.

Client-reported values are sanitized; the pause-when-behind logic is
unchanged:
- ticksBehind is clamped to [0, 2^24]. Negative values sped the server
  up through the slowdown factor, huge ones overflowed
  ExtrapolatedTicksBehind.
- frameTime: NaN becomes 0, the rest is clamped to [0, 10s]. A NaN went
  into serverTimePerTick and made realTime NaN, which stopped the
  server's game timer for good. The serverTimePerTick clamp is also
  NaN-safe now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WoodyWizard and others added 29 commits October 18, 2026 19:34
@WoodyWizard @claude
Start rituals with the starting player's role assignments
6472678 
Without Prepatcher the generic role selection widget methods aren't synced,
so each client can hold its own assignments when a ritual or psychic ritual
starts and the lords diverge. Send the starting player's assignments
(exposed) with the start command and copy them into the session's object on
every client before starting. With Prepatcher this is a no-op.

Addresses rwmt#687 (desync when the ritual starts without Prepatcher)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Make FastTileFinder.Closest deterministic
52cd03d 
Closest runs the same parallel job as Query and keeps a closest tile per worker
thread, so equally close tiles went to whichever thread got them first. Run it
in a single batch like Query. Query and Closest also share the flag that lets
them reuse their last result: a call made only by one client's interface could
make the simulation reuse a result from before a tile change, so the
simulation now always recomputes.

Fixes rwmt#648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Read the gravship move designator through its lazy accessor
3682aea 
The Designator_MoveGravship sync worker used the controller's moveDesignator
field, which only the landing UI's OnGUI creates (and refreshes after an
abort). On clients viewing the world, in screenshot mode or just loaded it
was null or stale, the command threw there and the landing marker, and so
the landing cells, diverged. Use MoveDesignator() and return no designator
when the landing already ended; HandleDesignator skips a null designator.

Fixes rwmt#791

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Pause when a caravan meeting or demand dialog opens
016815f 
Dialog_NodeTree force-pauses in vanilla, but MP forces WindowsForcePause to
false since windows differ between clients, so time kept running while the
caravan meeting or demand dialog waited for an answer. When either incident
fires on a caravan, pause from the simulation the way a pausing letter does,
unless the host set pause on letter to never.

Fixes rwmt#512

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Place the landing gravship in its map's context
bdb2515 
PlaceGravship runs in a long event queued from a cutscene render callback,
outside any tick or command, so spawning the ship and clearing its area used
the interface's RNG and the world's clock. Run it inside the landing map's
context (time, random state, faction) like a command of that map and update
the managers a command updates afterwards, which replaces the glow grid
workaround from rwmt#635. Drop the PlaceGravship sync method registration: it
never synced inside the long event and couldn't be read as a command.

Fixes rwmt#646

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Time extended camera shakes with the MP timer
44a946a 
ExtendedShakeRequest stores TicksGame when created and CameraShaker.Update
compares it with TicksGame in the interface. Under async time the request
may come from a map's clock and the check uses the world's, so after a
landing on a map ahead of the world the camera kept shaking until the world
caught up. In MP use TickPatch.Timer for both and clamp loaded requests to
it. UI only.

Fixes rwmt#696

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Only click the ritual gizmo the player clicked
0d73a38 
Grouped Command_Ritual gizmos (shared rituals of several ideoligions) also
process the click of another one in their group, before it. In MP each sync
call opens a session and the clicked ritual's came last and was rejected,
so the dialog opened for the wrong ideoligion. Don't inherit interactions
from grouped ritual gizmos in MP. UI only.

Fixes rwmt#506

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Sync the dev remove hediff button of the health tab
1fa6df1 
HealthCardUtility.DrawHediffRow's dev mode button removed the hediff
locally from its drawer lambda. The lambda runs every frame, so instead of
syncing it, redirect its RemoveHediff call to a debug-only sync method.

Fixes rwmt#358

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Warn when Prepatcher didn't patch the ritual role widget
ab76273 
Rituals rely on Prepatcher to sync the generic role selection widget, but
nothing told players when it was missing or failed to load (e.g. on macOS).
Detect it from the patched method itself (a flag set by Prepatcher's pass
is lost when it restarts the game with the patched assemblies), log a
warning and show it in the host window, the server browser and the ritual
and psychic ritual dialogs. English text with TryTranslate fallbacks.

Fixes rwmt#501

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Mark traces unreliable before logging a stack walk failure
6c5bf26 
If logging the failure threw, the outer catch skipped marking the window,
which then kept a truncated trace hash list and would report a false trace
desync. Mark right after turning tracing off, and again in the outer catch
when tracing was turned off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Match vanilla more closely when picking interface melee verbs
03d89bd 
Clear the per-frame cache when a tick ran within the frame, require the
reused current verb not to be in the Worst category like any other pick,
and ignore NaN weights for the highest weight like vanilla's comparison.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Merge fix/is-ux: interface melee verb lag, mod XML hashes, save after…
5e4e77e 
… disconnect, tracer kill switch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Report not paused while a world or map tick runs
6fcaecb 
TickManager.Paused is patched to the paused state of the map the local player
is looking at, also while the simulation ticks, so code in ticks saw different
values per client (e.g. Pawn_JobTracker.StartJob counts jobs by the local frame
count while paused). This became reachable more often with lockstep ticking:
the maps after a mid-tick pause (session, letter) still finish the game tick.
Ticks now always see not paused, as in vanilla where only unpaused games tick.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Start new maps at the game's speed without async time
34e19f8 
A new map took TickManager.CurTimeSpeed, the speed of the last context entered.
After a pausing letter earlier in the same world tick that was still the old
speed, and since the world runs at the fastest map's speed the new map
unpaused the game (with lockstep ticking every map). Use the speed last set
for the whole game instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Hide the per-client GameInitData while reading a starting pawn
9992f50 
Outside Playing, GenTicks.TicksAbs, Faction.OfPlayer and PawnsFinder read the
GameInitData, which differs between clients when SendPawn runs (the issuer's
configuration pages, an earlier faction creation, none). Nothing on the
starting pawn load path reaches them today, but a mod's pawn data could.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Keep the landing marker on the confirming client until it lands
e47be47 
Vanilla's confirm landing button clears the controller's landingMarker
right after BeginLanding, which MP turns into a command, so the confirming
client alone had no landing in progress until its command executed: a join
point in between didn't restore its landing session, and a move designation
or abort from another player failed there only. Restore the field after
the OnGUI call and ignore a second confirmation of a destroyed marker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Set the speed of every tickable after landing without async time
74a851a 
Landing set Normal speed only on the landing map. Without async time all
tickables share one speed, as a time speed command sets it, so the other
maps and the world kept the old one. Use SetTimeEverywhere there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
End gravship cutscenes that lost their gravship
2bca475 
A join point during a gravship cutscene (e.g. executed right after the
unfreeze, before the synced end) loses the gravship: nothing saves it while
it's neither on a map nor a world object. LandingEnded then threw and
TakeoffEnded threw after abandoning the map, leaving the game in the
cutscene for good (no world map, no saving, the map paused). Without a
gravship, end the cutscene instead: in the synced end command, and right
after loading when a cutscene was pending (no client would end it then).
This also closes the landing session of a catching-up joiner that executes
the end before its own PlaceGravship ran.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Apply only the widget-changed role assignments at ritual start
425c18d 
Copying every field of the starting player's exposed assignments also
replaced the ritual and the role change selection, which are saved by
reference: precepts created during the game (a new faction's ideoligion, a
reform) aren't in the shared cross refs and read back as null, which left
the lord without roles or threw. Copy only what the role selection widgets
change (assigned roles, spectators, a non-null role change selection; for
psychic rituals the role and pawn assignments and spectators) and drop
pawns which didn't resolve.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Land the gravship with the confirming player's rotation
218c7e5 
Other players can still have the move designator selected, and rotating
its preview changes the landing marker's (and the gravship's) rotation
locally, which BeginLanding and later PlaceGravship use. Send the
confirming player's rotation with SyncBeginLanding, deselect the move
designator on every client and then apply the rotation before landing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Don't let the simulation use a tile cache the interface regenerated
f145f9a 
When FastTileFinder's cache is dirty (after a reload, WorldReachability
clearing it) the first Query or Closest regenerates it, which can be interface
code on one client only (world targeter, launch and pilot console targeting).
Later DirtyTile calls refresh only changed tiles, so the cache kept e.g. the
neighbours' validForSettlement from that client's interface time. The
simulation now regenerates a cache the interface regenerated.

Part of rwmt#648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Remove the departing gravship in its map's context
2b952b9 
The takeoff counterpart of PlaceGravship: the long event queued from the
gravship capture callback removes the ship from the takeoff map (creating
its world object) and rebuilds the vacuum data with the interface's RNG and
clock. Run that lambda in the takeoff map's context too and update the
managers a command updates afterwards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Report the simulated paused state to commands and sim long events
2336353 
Commands, the constant ticker and sim long events still got TickManager.Paused
from the local view (viewed map, world view, vanilla ForcePaused incl.
TilePicker.Active). They now get whether their context's tickable is paused.
StartJob counts a job given while paused only for the tracker's second one in
the same step (its frame count is already replaced by the map's event count),
so the paused value itself made jobsGivenThisTick differ between clients. This
also corrects the reason given in 6fcaecb, which wasn't the frame count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Fire other quest incident makers once and keep owners without data fi…
99d1da2 
…ring

Incident makers other than threat generators (from mods) passed the filter in
the world tick and in every map tick, so they fired 1+N times; they now run in
the world tick only. A map owned by a player faction without world data (so
without a storyteller pass) never got its threats; it now falls back to the
pawn-based choice.

Part of rwmt#659 and rwmt#884

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Start maps generated anywhere in the world tick at the maps' tick
925534c 
worldTicks is updated right after DoSingleTick, so a map generated later in the
world tick (world sessions, dissolution updates) started at the new value and,
being in the same game tick's map snapshot, ended a tick ahead. During the
world tick new maps now start at worldTicks from before it.

Part of rwmt#339

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Merge fix/is-story: storyteller adaptation, quest threats, starting p…
255df8c 
…awn sync, lockstep ticking, tile finder

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Ignore late gravship aborts and stale cutscene freeze state
273b61b 
An AbortLanding command executed after the landing began (another player
confirmed meanwhile, or the abort dialog was left open) closed the landing
session mid-landing and vanilla then threw: skip it when no landing area
confirmation is in progress. Tie the cutscene freeze to the connection it
was sent on and clear this client's leftover freeze state when a game
without a pending cutscene loads (e.g. rejoining after one ended). Correct
the comment on when EndWithoutGravship runs identically on every client.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Merge fix/is-odyssey: gravship landing state, ritual role snapshots, …
7f555bb 
…dev hediff removal, Prepatcher warning

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WoodyWizard @claude
Describe the desync review changes for the pull request
48afdb4 
docs/desync-review.md lists every change on this branch, grouped by area, with the commits and issues, plus the
compatibility and behavior changes, known limitations and how it was tested. It is meant for reviewing the pull
request and can be removed before merging.

The branch is a broad attempt at fixing desyncs and a number of smaller bugs: a full review of the desync-relevant
code (command log and join points, server, RNG and simulation contexts, sync framework, sessions, multifaction, async
time and the tick loop, storyteller, gravship), followed by a review of all open pull requests and desync issues,
porting the useful parts and fixing the remaining issues.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mkiplm-svg mkiplm-svg mentioned this pull request Oct 19, 2026
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@WoodyWizard