Skip to content

0.2.0: async-profiler 4.6, span filters, endpoint fixes, library API and new docs - #63

Merged
dpsoft merged 46 commits into
mainfrom
integration/modernize
Nov 15, 2026
Merged

dpsoft merged 46 commits into
mainfrom
integration/modernize

Conversation

@dpsoft

@dpsoft dpsoft commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Summary

Modernizes ap-agent for a 0.2.0 release: async-profiler 4.6, span filtering, endpoint bug fixes, no Vavr, a
supported library API, a test suite (there was none) and a new README with docs and images.

Every phase was developed on its own branch, reviewed, and merged into integration/modernize; CI runs on JDK
11/17/21/25 (x64) and JDK 21 (arm64).

Phase What
1. Tests JUnit 5 + JaCoCo. Unit tests, endpoint tests against the real profiler, smoke tests attaching the shaded jar with -javaagent to a separate JVM (standard mode, Go mode, configuration).
2. CI pull_request was nested under push (PRs never ran CI). JDK matrix + arm64, current actions, Dependabot weekly and grouped.
3. Upgrade ap-loader 4.4-13 → 4.6-14 (async-profiler 4.6), typesafe config 1.4.10, tinylog 2.8.1, Maven plugins; --release 11; drop the retired OSSRH snapshots repository.
4. Span filters New tag=<regex> and latency=<ms> parameters on top of the async-profiler Span API; query parameters decoded once.
5. Bug fixes See Behaviour changes below.
6. Remove Vavr Shaded jar 2.82 MB → 1.79 MB (−37%), startup unchanged (scripts/bench-startup.sh).
7. Docs and API ProfilerService / Profile library API, short README in the style of perf-agent + docs/, images and demo GIF generated with async-profiler 4.6, release workflow fixed.

Behaviour changes against 0.1.10

CompatibilityIT downloads 0.1.10 from Maven Central (SHA-1 checked), sends the same 22 requests to it and to this
build, and compares status, payload kind, sample types and samples. 16 responses are identical; these are the only
differences, all intended:

Request 0.1.10 Now Why
output=collapsed gzipped JFR collapsed stacks collapsed returned the JFR recording
event=wall&output=flame empty flame graph wall samples wall samples were read as CPU samples
event=nativemem&output=pprof pprof cpu pprof malloc pprof ignored event
duration=abc connection dropped 400 + message invalid parameters
?debug&... connection dropped 200 parameters without =
unknown path 404 empty 404 + message

Also: Content-Type on every response and Content-Encoding: gzip for JFR/pprof (it was set after the headers were
sent), 500 with the cause when the profiler cannot record or convert, 409 when another session is running (the
server now handles requests concurrently), the profiler is always stopped and temporary files are always deleted.

Bugs found along the way

  • A second ap-loader on the application classpath killed the JVM at startup (Multiple versions of async-profiler found, thrown from premain). The agent now logs it and the application keeps running. Applications
    that record spans must use tools.profiler:async-profiler, not ap-loader-* (documented, with a smoke test).
  • ?output=pprof always recorded CPU; event=wall flame graphs were empty; interval=0 returned a truncated body.
  • interval had no effect with event=alloc, lock or nativemem: async-profiler only reads it for CPU and wall
    clock. It is now passed as alloc=, lock= or nativemem=, and ap-agent.profiler.interval only applies to CPU
    and wall clock.

Migration (library users)

Command and ProfilerExecutor no longer expose Vavr types and are internal; use ProfilerService and Profile
(see docs/library-usage.md and docs/upgrading.md). The old README example did not compile against 0.1.10 either.
Documentation examples are now compiled and run by the build (ReadmeExamplesTest).

Release

release.yml failed on the last releases. It now only attaches ap-agent-<version>.jar and ap-agent.jar to a
published GitHub release; the Maven Central deploy stays manual (see CONTRIBUTING.md). Nothing is released by
this PR.

Notes for review

  • The embedded-server example uses a non-daemon newCachedThreadPool(); production uses daemon threads.
  • Images in docs/assets are regenerated with mvn -Passets verify -Dgpg.skip=true (Chrome, ffmpeg and Go).

Fixes #40
Closes #3 — async-profiler rejected per-sample context ids (async-profiler#576) in favour of the Span API, which
tag=/latency= now expose.

Add JUnit 5, surefire/failsafe and JaCoCo. Cover the current behaviour of
query parsing, Command, configuration, banner and build info with unit
tests, drive AsyncProfilerHandler against the real profiler in-process,
and add smoke tests that attach the shaded agent to a separate JVM with
-javaagent (standard mode, go mode, banner and interval overrides).

Known bugs are pinned as @disabled tests describing the expected
behaviour, to be enabled as each one is fixed.
- AgentProcess cleans up the child JVM and its log when startup fails and
  waits for both startup log lines before returning.
- Decode pprof responses (sample types and sample count) instead of
  matching substrings, and assert the expected type per go endpoint.
- Assert the effective profiler interval through jdk.ActiveSetting.
- Add monitor contention to the fixture so lock profiles have samples.
FixtureApp.start returns a Workload handle that interrupts and joins
every thread, so contention threads no longer outlive the handler test.
The pprof decoder validates varints and length-delimited segments against
the message end and no longer loses bytes when skipping unknown fields.
- build.yml: pull_request was nested under push, so PRs never ran CI.
  Run on main and integration/** pushes and PRs, with a JDK matrix,
  linux-arm64, Maven cache, and test/coverage report artifacts.
- release.yml: move to actions/checkout@v7 and setup-java@v6.
- dependabot: weekly, grouped Maven plugin and test updates, and keep
  GitHub Actions up to date.
- ap-loader-all 4.4-13 -> 4.6-14 (async-profiler 4.6)
- vavr 0.10.6 -> 1.1.0, typesafe config 1.4.5 -> 1.4.10,
  tinylog 2.7.0 -> 2.8.1
- maven-shade 3.6.2, maven-source 3.4.0, central-publishing 0.12.0
- pin maven-compiler-plugin 3.16.0 and compile with --release 11
- drop the s01 OSSRH snapshots repository (service retired)

Notable async-profiler changes for users passing raw options through
params=: 4.4 removed check/safemode/cstack=lbr, 4.6 makes 'filter' take
no arguments and raises the macOS minimum to 11.
async-profiler 4.5+ records profiler.Span events through its Span API.
Expose the converter span filters as dedicated query parameters:

  tag=<regex>      keep samples inside spans whose tag fully matches
  latency=<ms>     keep samples inside spans lasting at least <ms>

They are separate from params= because params is split on commas,
which would break tags such as 'SparkTask:.*,taskAttemptId=123,.*'.

The application records spans with one.profiler.Span from the
async-profiler API artifact (unrelocated); it binds to the library the
agent already loaded. Smoke tests cover that setup with the shaded jar.
Split the raw query and percent-decode each part exactly once, keeping a
literal '+', so span tag regexes such as 'fixture:.+' or values with
encoded '&', '=' or '%' reach the converter intact. Reject latencies that
would overflow once converted to nanoseconds (they silently disabled the
filter). Check tag filters on a single recording instead of comparing
independent captures.
Decoding once is enough: '+' still means a space for clients that
encode spaces that way (URLEncoder, title=My+Title), and %2B yields '+'.
…point

- Send the 200 headers only after the recording succeeded, after
  Content-Type and Content-Encoding are set: 400 for invalid parameters
  (duration, latency), 500 with the cause when the profiler cannot
  record, 404 with a body and an exact length.
- output=collapsed returns collapsed stacks as text instead of the JFR.
- An explicit event is honoured for output=pprof; the Go path mapping
  only applies when no event is given.
- Wall-clock profiles are converted as wall samples (they were read as
  CPU samples and came out empty).
- Always stop the profiler once started and delete the temporary
  recording when it fails.
- Skip query parameters without '=' instead of failing.
- StrSubstitutor keeps unknown variables and treats replacements as
  literal text.

Supersedes the fixes proposed in #57.
- Build the converter arguments when the request is accepted, so an
  invalid tag regex is a 400 before anything is recorded.
- Convert into a temporary file before sending 200: conversion errors
  (e.g. a tag filter on a recording without spans) become a 500 with the
  cause, and the response carries an exact Content-Length.
- Handle InterruptedException explicitly: restore the interrupt status,
  delete the recording and report a regular failure (vavr rethrows it
  from Try.failure).
- Reject durations that overflow once converted to milliseconds.
Replace Try/Option/List/Match with plain Java (exceptions, Optional,
java.util.List, switch). The shaded jar goes from 2.82 MB to 1.79 MB
(-37%); startup time is unchanged (~167 ms median on JDK 21, measured
with the new scripts/bench-startup.sh).

ProfilerExecutor.run() and convert() now throw instead of returning a
Try. Runner logs LinkageError too, so a missing native library cannot
abort the application's startup.
Use port 0, fail on JVM errors, missing 'AP Agent started', ERROR lines or
a missing startup time, and require every run to be valid.
AsyncProfilerLoader.isSupported() throws when the application ships
another ap-loader version ('Multiple versions of async-profiler found').
It was called outside the try block, so the exception escaped premain and
killed the JVM. Log it like any other loading failure instead.
CompatibilityIT downloads ap-agent 0.1.10 from Maven Central
(SHA-1 checked; resolving it through Maven would pick the current build
while the versions match), sends the same requests to it and to the
current build, and compares status, payload kind, sample types and
whether there are samples. Every difference must be declared with its old
and new shape; a markdown report is written and uploaded by CI.

Smoke tests now give the application the official async-profiler API
(tools.profiler:async-profiler) for spans, and check that a conflicting
ap-loader on the application classpath does not kill it.
- ProfilerService / Profile / ProfilerBusyException: a supported library
  API that hides the relocated async-profiler types and shares the
  agent's validation, status codes and temporary file handling; one
  profiling session at a time (409 when busy).
- README rewritten as a short landing page in the style of perf-agent,
  with the details in docs/ (usage, span filtering, Go mode,
  configuration, library usage, upgrading) and a CONTRIBUTING guide.
- Java examples live in src/test/java/.../examples, compiled and run by
  the build; ReadmeExamplesTest checks the docs show exactly that code.
- docs/assets: images and a demo GIF generated with async-profiler 4.6
  from a DemoShop workload (mvn -Passets verify), plus brand SVGs.
- release.yml only attaches the jar to a published GitHub release; the
  Maven Central deploy stays manual.
- Version 0.2.0.
… examples

- The session lock is shared by every ProfilerService instance: Async
  Profiler is one per JVM, so a second service now gets
  ProfilerBusyException (409) instead of a 500.
- The agent's HTTP server handles requests on daemon worker threads, so
  a request arriving during a recording gets an immediate 409 instead of
  waiting in line; the embedded server example does the same.
- The servlet example defaults to pprof output for the /debug/pprof paths
  and is now executed by the tests (with minimal Servlet API fakes).
- profile() reports a failure to create the temporary recording as an
  IOException, and the release guide notes that EXPECTED_CHANGES must be
  emptied when the baseline moves.
…vent

async-profiler only reads 'interval' for CPU and wall-clock profiling: with
event=alloc, lock or nativemem it had no effect (and the docs said it was
in bytes). An explicit interval is now passed as alloc=, lock= or
nativemem=, and the configured ap-agent.profiler.interval only applies to
CPU and wall clock.

The fixture allocates a few MB per second: with 1 KB arrays some
one-second alloc profiles had no samples on CI runners.
@dpsoft
dpsoft marked this pull request as ready for review November 15, 2026 18:56
@dpsoft
dpsoft requested a balanced review from Copilot November 15, 2026 19:00
@dpsoft
dpsoft marked this pull request as draft November 15, 2026 19:01
@dpsoft
dpsoft marked this pull request as ready for review November 15, 2026 19:03
The alloc, lock and nativemem options of async-profiler disable the event
for an unparsable or negative value, and async-profiler then falls back
to CPU profiling: a bad interval answered 200 with the wrong event.
Validate explicit intervals against the units async-profiler reads
(n/u/m/s for times, b/k/m/g for sizes) and answer 400 otherwise.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Malformed intervals can silently produce the wrong profile, rejected concurrent requests retain delete-on-exit entries, and compatibility tests can mistake bookkeeping events for samples.

3 open issues
What changed in this PR

Prepares ap-agent 0.2.0 by modernizing its profiler integration, improving HTTP profiling reliability, and adding a supported library API with regression coverage and documentation.

Changes:

  • Upgrades to async-profiler 4.6, adds span filters, and corrects event selection, output conversion, error handling, and temporary-file cleanup.
  • Introduces ProfilerService and Profile, removes Vavr, and adds unit, endpoint, compatibility, and packaged-agent tests.
  • Updates CI and release workflows, reorganizes documentation, and adds reproducible profiling demos.
File Description
src/​test/​java/​io/​github/​dpsoft/​ap/​util/​StrSubstitutorTest.java Tests literal substitution and unknown variables.
src/​test/​java/​io/​github/​dpsoft/​ap/​util/​ProfilerExecutorTest.java Tests validation, interruption, and recording cleanup.
src/​test/​java/​io/​github/​dpsoft/​ap/​util/​BuildInfoTest.java Checks filtered build-version metadata.
src/​test/​java/​io/​github/​dpsoft/​ap/​util/​BannerTest.java Verifies banner text and versions.
src/​test/​java/​io/​github/​dpsoft/​ap/​testing/​Profiles.java Adds profile-payload inspection helpers.
src/​test/​java/​io/​github/​dpsoft/​ap/​testing/​PprofTest.java Tests protobuf decoding and workload shutdown.
src/​test/​java/​io/​github/​dpsoft/​ap/​testing/​Pprof.java Adds a lightweight test-only pprof decoder.
src/​test/​java/​io/​github/​dpsoft/​ap/​testing/​FixtureApp.java Supplies sampling, contention, and span workloads.
src/​test/​java/​io/​github/​dpsoft/​ap/​ReadmeExamplesTest.java Checks documentation snippets and runs examples.
src/​test/​java/​io/​github/​dpsoft/​ap/​ProfilerServiceTest.java Tests library behavior and session exclusivity.
src/​test/​java/​io/​github/​dpsoft/​ap/​it/​CompatibilityIT.java Compares response shapes against the previous release.
src/​test/​java/​io/​github/​dpsoft/​ap/​it/​AgentSmokeIT.java Exercises the packaged agent in separate JVMs.
src/​test/​java/​io/​github/​dpsoft/​ap/​it/​AgentProcess.java Manages test JVM startup, requests, and cleanup.
src/​test/​java/​io/​github/​dpsoft/​ap/​handler/​AsyncProfilerHandlerTest.java Covers HTTP outputs, filters, errors, and cleanup.
src/​test/​java/​io/​github/​dpsoft/​ap/​functions/​FunctionsTest.java Tests query decoding and string helpers.
src/​test/​java/​io/​github/​dpsoft/​ap/​examples/​ServletFakesAccess.java Exposes servlet test helpers across packages.
src/​test/​java/​io/​github/​dpsoft/​ap/​examples/​ServletFakes.java Supplies minimal servlet request and response fakes.
src/​test/​java/​io/​github/​dpsoft/​ap/​examples/​ProfilerServlet.java Demonstrates serving profiles through a servlet.
src/​test/​java/​io/​github/​dpsoft/​ap/​examples/​OrderService.java Demonstrates tagging work with spans.
src/​test/​java/​io/​github/​dpsoft/​ap/​examples/​EmbeddedServerExample.java Demonstrates an embedded profiling endpoint.
src/​test/​java/​io/​github/​dpsoft/​ap/​demo/​DemoShop.java Provides the documentation-demo workload.
src/​test/​java/​io/​github/​dpsoft/​ap/​config/​AgentConfigurationTest.java Verifies default configuration.
src/​test/​java/​io/​github/​dpsoft/​ap/​command/​CommandTest.java Covers command options, intervals, and span filters.
src/​main/​resources/​reference.conf Clarifies sampling-interval scope in comments.
src/​main/​java/​io/​github/​dpsoft/​ap/​util/​StrSubstitutor.java Preserves unknown variables and literal replacements.
src/​main/​java/​io/​github/​dpsoft/​ap/​util/​Server.java Adds concurrent request handling with daemon workers.
src/​main/​java/​io/​github/​dpsoft/​ap/​util/​Runner.java Logs profiler-loading failures without propagating them.
src/​main/​java/​io/​github/​dpsoft/​ap/​util/​ProfilerExecutor.java Revises recording, conversion, interruption, and cleanup.
src/​main/​java/​io/​github/​dpsoft/​ap/​util/​BuildInfo.java Replaces Vavr and closes the metadata stream.
src/​main/​java/​io/​github/​dpsoft/​ap/​util/​Banner.java Replaces Vavr and explicitly decodes UTF-8.
src/​main/​java/​io/​github/​dpsoft/​ap/​ProfilerService.java Adds the library entry point and session coordination.
src/​main/​java/​io/​github/​dpsoft/​ap/​ProfilerBusyException.java Represents concurrent-session rejection.
src/​main/​java/​io/​github/​dpsoft/​ap/​Profile.java Encapsulates profile output and temporary-file ownership.
src/​main/​java/​io/​github/​dpsoft/​ap/​handler/​AsyncProfilerHandler.java Uses the service API and sets response metadata and errors.
src/​main/​java/​io/​github/​dpsoft/​ap/​functions/​Functions.java Decodes query components once and skips valueless parameters.
src/​main/​java/​io/​github/​dpsoft/​ap/​config/​AgentConfiguration.java Replaces Vavr-based configuration error handling.
src/​main/​java/​io/​github/​dpsoft/​ap/​command/​Command.java Adds span filters and event-specific interval options.
src/​main/​java/​io/​github/​dpsoft/​ap/​Agent.java Connects agent startup to the service API.
scripts/​bench-startup.sh Measures startup timing and loaded classes.
scripts/​assets/​GenerateAssets.java Generates profile pages, screenshots, and a demo GIF.
README.md Adds a concise quickstart and documentation index.
pom.xml Updates dependencies and configures tests, coverage, and assets.
docs/​usage.md Documents parameters, outputs, errors, and recipes.
docs/​upgrading.md Explains behavior and library API migration.
docs/​span-filtering.md Explains span instrumentation and filtering.
docs/​library-usage.md Documents supported APIs and integration examples.
docs/​go-mode.md Describes Go-compatible profiling endpoints.
docs/​configuration.md Documents settings, deployment, and requirements.
docs/​brand/​ap-agent-mark.svg Adds the standalone brand mark.
docs/​brand/​ap-agent-lockup.svg Adds the themed logo and wordmark.
docs/​assets/​flame-span-filter.html Adds an interactive span-filtered profile example.
docs/​assets/​flame-nativemem.html Adds an interactive native-memory profile example.
CONTRIBUTING.md Documents builds, tests, assets, and release steps.
.github/​workflows/​release.yml Builds and attaches jars to published releases.
.github/​workflows/​build.yml Fixes PR triggers and expands CI coverage.
.github/​dependabot.yml Adds weekly grouped dependency and action updates.

Review effort: Balanced

async-profiler parses intervals with strtol base 0 (a leading zero means
octal) and multiplies by the unit without overflow checks; for alloc,
lock and nativemem a bad result disables the event and it silently
profiles CPU. Parse the value in base 10, apply the unit with overflow
checks, and pass the normalized nanoseconds or bytes; anything else is
a 400.
…ofiles

- New ap-agent mark: an icicle (async-profiler flame graph turned upside
  down, as the docs show them) in async-profiler frame colours, with a
  monospace wordmark; it no longer reuses the perf-agent flame.
- docs: demo, span filter and heatmap GIFs generated by the assets
  profile; the heatmap is recorded for 60 s and the GIF selects a range.
- 'Interactive version' links opened the HTML source on GitHub: they now
  go through htmlpreview.github.io, which renders the live page.
- ProfilerService takes the session lock before creating the executor:
  a busy request (409) no longer creates a recording file. The temporary
  file is no longer registered with deleteOnExit(), which kept every path
  in memory for the life of the JVM (every path deletes it already).
- CompatibilityIT only counts real sample events, not bookkeeping events
  such as profiler.Log or profiler.Window.
- Demo GIF: zoom into a frame and back to the whole graph, search 'regex'
  (a partial match) instead of a frame present in every stack; GIFs loop.
- ProfilerExecutor.with only builds and validates the converter arguments;
  the temporary recording is created by run(). ProfilerService validates
  everything before taking the session, so an invalid tag regex is a 400
  even while another session runs, and a busy request allocates nothing.
- run() deletes the recording on any failure, Errors included (it relied
  on deleteOnExit for those before).
- CompatibilityIT retries Maven Central with backoff (Retry-After) on 429,
  5xx and I/O errors, alternating repo1/repo.maven.apache.org: parallel CI
  jobs got rate limited.
- scripts/assets/CheckLiveLinks.java opens every htmlpreview link of the
  docs in headless Chrome and checks that the profile canvas is painted;
  documented in CONTRIBUTING (the links only work once on main).
- The smoke tests attach the JaCoCo agent to the JVM running the current
  shaded agent jar (never the baseline release), so premain, Runner and
  Server are measured too.
- scripts/coverage-it.sh reports that data against the classes inside the
  shaded jar (shading rewrites their bytecode; line numbers are the same).
- CI uploads the unit and smoke test reports to Codecov through OIDC;
  Codecov merges them (95% of lines locally, 87% from unit tests alone).
- Coverage badge in the README.
@dpsoft
dpsoft merged commit 1bdfa99 into main Nov 15, 2026
15 checks passed
@dpsoft
dpsoft deleted the integration/modernize branch November 15, 2026 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Not clear/delete temp file if profile's execution failed Add support for Context ID functionality

2 participants