Repository navigation
Expose checkpoint progress and buffer replica checkpoint writes - #680
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Standalone replicas now expose bounded binary protobuf journal status and an explicit checkpoint request. Clients verify publication against both the requested epoch and journal identity, so a replaced or reopened legacy journal cannot satisfy an old checkpoint ticket. Memory-only and Raft nodes refuse these local maintenance operations; journal I/O failures fence the replica.
Default replica checkpoints now coalesce fragmented encoder output with a fixed 64 KiB buffer and explicitly flush it before rewriting the checkpoint header, synchronizing the file, and publishing the replacement. Stored bytes, checksums, synchronization, and recovery rules remain unchanged. Final write and flush failures propagate to the existing failure fence.
The paired corpus test drains warmup and final checkpoints before sampling Linux data-process I/O. It reports acknowledgment latency and QPS separately from the full cycle, includes kernel writes and cancellations, and checks retained acknowledged pages on every replica after restart. Reads use resident payloads. The timed requests directly address data replicas; metadata publication QPS and Windows device I/O are outside their scope.
In one local WSL release run with 64 MiB of background payload per data replica, 16 clients and 512 fixed-suffix appends, the three data processes made 12,001 write calls, versus about 921,000 in three preceding unbuffered runs. Kernel write bytes remained near 658 MB. CPU ticks were 542, versus 705–839 before buffering. These short stress phases include completed checkpoints and use a 1 MiB checkpoint interval, three acknowledged copies and one-copy reads. Timing varied substantially; this is not a reliable latency or sustained-QPS claim.
Validation covers canonical bounded codecs, both layouts over TCP, checkpoint identity across restart, fragmented reads, short/interrupted writes, final write/flush failures, changed stream length, library tests, Star replication through metadata failover and full process restart, zero-filled WAL storage, paired release measurements at 1/16/64 MiB per replica, feature combinations, documentation, and the CI test-name inventory.
The segmented layout remains opt-in, retains old segments, and still rewrites its complete metadata catalog. Buffering reduces default checkpoint syscall overhead; it does not remove full-corpus payload rewriting. Flat append latency, incremental metadata checkpoints, and bounded storage retention remain future work.