Skip to content

Coalesce repeated events so no program can flood trace (0.7.0) - #30

Merged
dmccoystephenson merged 1 commit into
mainfrom
feat/coalesce-repeated-events
Oct 28, 2026
Merged

dmccoystephenson merged 1 commit into
mainfrom
feat/coalesce-repeated-events

Conversation

@dmccoystephenson

Copy link
Copy Markdown
Member

Why

One third-party server runs Medieval Factions' /power command roughly every 100 ms (most likely another plugin dispatching it). Medieval Factions reports a command event per execution, so trace received about 425k events in five days from that one server. The fix is made in the client, so no consumer can do this, with no change to the consumer API.

What was changed

  • Coalescing in report(). An event is its name, value and tags (after version, install and server-wide tags are merged; tag order is ignored). The first report is sent at once and opens a 60 s window (COALESCE_WINDOW_MS). Identical reports inside the window are counted, and a count above zero is sent as one event when the window closes. A window that sent a count reopens, so a steady flood costs one event a minute per event; a window that counted nothing ends.
  • close() flushes every open count before the existing bounded drain.
  • startup is never coalesced or delayed.
  • Bounded memory: at most MAX_COALESCED_EVENTS (256) distinct events are counted at once. A new event beyond that is dropped silently, like a report that finds the queue full. The 256-report send queue bound is kept.
  • The sender became a single-thread ScheduledThreadPoolExecutor, so the same daemon thread (trace-client/<application>) closes windows. No second thread was added. The queue bound is now enforced by counting queued sends.
  • Version bumped to 0.7.0 (header, VERSION, pom, README JitPack snippet). This is a minor bump: new public constants (COALESCE_WINDOW_MS, MAX_COALESCED_EVENTS, COALESCED_TAG) and changed send behaviour, with no API removed or changed.
  • The README gained a "Repeated events are coalesced" section and a "Throttled" row in the promises table.

How the count is encoded

The trace server's MetricDto has a nullable Double value and a string→string tags map (at most 32). The server counts rows today and never sums value. A coalesced event carries:

  • value = count, when the original event had no value (every command and most other events). value is the only numeric field, so SUM(COALESCE(value, 1)) gives true occurrences for valueless events without any string-to-number cast.
  • tag coalesced = count, always (when there is room under 32 tags). This marks the row as a summary and carries the count for events that already had a value, which is kept unchanged because its meaning differs per program. coalesced was chosen over count so that it cannot collide with a consumer's own count tag.

Occurrences = Σ (value where tagged coalesced, else 1). Row counts still answer "was it used, by how many servers". By design they now undercount "how many times".

Tests

Eight tests were added: the first report goes out at once and the repeats become one counted event; a steady flood gives about one event per window, the counts add up to every report, and a quiet window ends; close() sends open counts without waiting for the window; the key includes name, value and tags but not tag order, and NaN counts as no value; startup is never held back; the distinct-event cap holds; nothing is counted after close(); and withCount respects an own tag and MAX_TAGS. The queue-bound test now floods startup, because that is the one event that still goes straight to the queue. The tests use a package-private Builder.coalesceWindowMillis to shorten the window, which is not public API.

mvn -B verify was run locally on Java 17 and 21: 94 tests, 0 failures, repeated three times. Java 8 was not available locally, so CI covers it.

Releasing and re-vendoring into consumers are left to the owner.


drafted by Claude on behalf of Daniel Stephenson

🤖 Generated with Claude Code

One server whose /power command was dispatched every ~100 ms by another
plugin sent ~425k command events in five days, one per run. The client
now coalesces: the first report of an event (name, value and tags) is
sent at once and opens a 60 s window; identical reports inside it are
counted and sent as one event when it closes, tagged coalesced=<n> and
with value=<n> when the event had none. A window that sent a count opens
the next, so a steady flood is one event a minute; close() sends every
open count. startup is never coalesced. At most 256 events are counted
at once, so memory stays bounded.

The sender becomes a single-thread ScheduledThreadPoolExecutor so the
same daemon thread closes windows; the 256-report queue bound is kept
by counting queued sends. The consumer API is unchanged; new public
constants make this a minor release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dmccoystephenson
dmccoystephenson merged commit ce3baa4 into main Oct 28, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant