Repository navigation
fix(agent): a group member's holder appends facts through a scheduler stop; fence clock never pauses (#271) - #273
Merged
Merged
Conversation
… stop; fence clock never pauses (#271)
…ng facts, so a fence lifts on a landed fact (#271) Review: with no scheduler wanted and no member held, the writer stopped and the fence latched, so a group member could never be claimed. Count a group member's task for its whole life instead of only while held. Docs follow; sys.facts growth with no scheduler is #274.
Contributor
Author
|
Revert check (#272, fix disabled: writes_facts back to the scheduler watch alone, fence_self pause restored): |
el-bri-ga
enabled auto-merge (squash)
November 18, 2026 20:36
el-bri-ga
disabled auto-merge
November 18, 2026 22:18
…eduler to step a stop (#271) Every node with a group member task appends facts through the stop, so the restarted scheduler has nothing to change and, short of a 10 minute table tick, outputs nothing. Waiting for its output count to grow passed only when it moved the group, the bug the test checks. A fact carrying a relist nonce, appended after the stop's facts, gets an answer once the scheduler has stepped all of them.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Fixes #271
What
#251 (#270) paused a node's fence clock over a confirmed scheduler stop, and the scheduler pauses its forget clock only over a gap with no facts from anyone. A node with a stale watch keeps appending facts through the stop, so on restart there is no gap: a holder that went quiet during the stop is counted silent and forgotten while it still runs its group members, and the group can run split.
The fix uses the holder's own evidence. A node holding a group member appends facts whether or not a scheduler is wanted, and the fence clock never pauses. So the only node the scheduler can forget while it runs a member is one whose facts are not landing, and that node fences (releases its members) first.
Invariant and entry points
Invariant: a node runs a group member only while its own facts land within the fence bound, which is under half the forget window.
crates/agent/src/assignments.rs:498write_facts,:548writes_factsa_node_holding_a_group_member_writes_facts_through_a_scheduler_stopcrates/agent/src/agent.rs:379GroupMember, taken at:936at the top ofown(), before any claim, dropped when the task endsa_group_member_keeps_the_node_writing_facts_whether_held_or_waiting(revert-checked)crates/agent/src/agent.rs:751fence_selfscheduler_stoppedread removed; only a landed fact (:733->facts_due_from:238) moves the clocka_node_whose_facts_stop_landing_fences_itself_and_must_be_relistedagent.rs:1038(claim refused),:1344(wait to claim),:1810(held member released)crates/core/src/schedule.rs:360forget_silent,:374pause_over_gapcrates/felixd/tests/scheduler.rsa_stale_node_s_facts_during_a_scheduler_stop_do_not_forget_a_group_s_holdersTLA: new
StopPauseconstant andpausedvariable with aPause(n)action (the #270 behaviour).FelixdPlacementForgetStopPause.cfgsets it and is expected to violate GroupTogether (twins job andcheck_tla.sh); every other config setsStopPause = FALSE.Design calls
Pause(n)models the fix(agent): fence a node whose facts stop landing; no forget over a fact gap (#251) #270 behaviour as a free action, so ForgetStopPause shows a pause unmatched by the scheduler breaks GroupTogether. The spec does not model the writer rule itself; withStopPause = FALSEit keeps ForgetLive's assumption that a live node is forgotten only once fenced, which the writer rule is what makes true in code.Revert check
Draft #272 runs only the new integration test with the fix disabled (writes_facts back to the scheduler watch, fence_self pause restored). Result noted below once CI reports.
Local checks
cargo test -p felixd-agent --lib(210 passed), clippy--workspace --all-targets --all-features -D warnings, fmt. TLC runs in CI.