Repository navigation
0.2.0: async-profiler 4.6, span filters, endpoint fixes, library API and new docs - #63
Merged
Merged
Conversation
Add JUnit 5, surefire/failsafe and JaCoCo. Cover the current behaviour of query parsing, Command, configuration, banner and build info with unit tests, drive AsyncProfilerHandler against the real profiler in-process, and add smoke tests that attach the shaded agent to a separate JVM with -javaagent (standard mode, go mode, banner and interval overrides). Known bugs are pinned as @disabled tests describing the expected behaviour, to be enabled as each one is fixed.
- AgentProcess cleans up the child JVM and its log when startup fails and waits for both startup log lines before returning. - Decode pprof responses (sample types and sample count) instead of matching substrings, and assert the expected type per go endpoint. - Assert the effective profiler interval through jdk.ActiveSetting. - Add monitor contention to the fixture so lock profiles have samples.
FixtureApp.start returns a Workload handle that interrupts and joins every thread, so contention threads no longer outlive the handler test. The pprof decoder validates varints and length-delimited segments against the message end and no longer loses bytes when skipping unknown fields.
- build.yml: pull_request was nested under push, so PRs never ran CI. Run on main and integration/** pushes and PRs, with a JDK matrix, linux-arm64, Maven cache, and test/coverage report artifacts. - release.yml: move to actions/checkout@v7 and setup-java@v6. - dependabot: weekly, grouped Maven plugin and test updates, and keep GitHub Actions up to date.
- ap-loader-all 4.4-13 -> 4.6-14 (async-profiler 4.6) - vavr 0.10.6 -> 1.1.0, typesafe config 1.4.5 -> 1.4.10, tinylog 2.7.0 -> 2.8.1 - maven-shade 3.6.2, maven-source 3.4.0, central-publishing 0.12.0 - pin maven-compiler-plugin 3.16.0 and compile with --release 11 - drop the s01 OSSRH snapshots repository (service retired) Notable async-profiler changes for users passing raw options through params=: 4.4 removed check/safemode/cstack=lbr, 4.6 makes 'filter' take no arguments and raises the macOS minimum to 11.
async-profiler 4.5+ records profiler.Span events through its Span API. Expose the converter span filters as dedicated query parameters: tag=<regex> keep samples inside spans whose tag fully matches latency=<ms> keep samples inside spans lasting at least <ms> They are separate from params= because params is split on commas, which would break tags such as 'SparkTask:.*,taskAttemptId=123,.*'. The application records spans with one.profiler.Span from the async-profiler API artifact (unrelocated); it binds to the library the agent already loaded. Smoke tests cover that setup with the shaded jar.
Split the raw query and percent-decode each part exactly once, keeping a literal '+', so span tag regexes such as 'fixture:.+' or values with encoded '&', '=' or '%' reach the converter intact. Reject latencies that would overflow once converted to nanoseconds (they silently disabled the filter). Check tag filters on a single recording instead of comparing independent captures.
Decoding once is enough: '+' still means a space for clients that encode spaces that way (URLEncoder, title=My+Title), and %2B yields '+'.
…point - Send the 200 headers only after the recording succeeded, after Content-Type and Content-Encoding are set: 400 for invalid parameters (duration, latency), 500 with the cause when the profiler cannot record, 404 with a body and an exact length. - output=collapsed returns collapsed stacks as text instead of the JFR. - An explicit event is honoured for output=pprof; the Go path mapping only applies when no event is given. - Wall-clock profiles are converted as wall samples (they were read as CPU samples and came out empty). - Always stop the profiler once started and delete the temporary recording when it fails. - Skip query parameters without '=' instead of failing. - StrSubstitutor keeps unknown variables and treats replacements as literal text. Supersedes the fixes proposed in #57.
- Build the converter arguments when the request is accepted, so an invalid tag regex is a 400 before anything is recorded. - Convert into a temporary file before sending 200: conversion errors (e.g. a tag filter on a recording without spans) become a 500 with the cause, and the response carries an exact Content-Length. - Handle InterruptedException explicitly: restore the interrupt status, delete the recording and report a regular failure (vavr rethrows it from Try.failure). - Reject durations that overflow once converted to milliseconds.
Replace Try/Option/List/Match with plain Java (exceptions, Optional, java.util.List, switch). The shaded jar goes from 2.82 MB to 1.79 MB (-37%); startup time is unchanged (~167 ms median on JDK 21, measured with the new scripts/bench-startup.sh). ProfilerExecutor.run() and convert() now throw instead of returning a Try. Runner logs LinkageError too, so a missing native library cannot abort the application's startup.
Use port 0, fail on JVM errors, missing 'AP Agent started', ERROR lines or a missing startup time, and require every run to be valid.
AsyncProfilerLoader.isSupported() throws when the application ships
another ap-loader version ('Multiple versions of async-profiler found').
It was called outside the try block, so the exception escaped premain and
killed the JVM. Log it like any other loading failure instead.
CompatibilityIT downloads ap-agent 0.1.10 from Maven Central (SHA-1 checked; resolving it through Maven would pick the current build while the versions match), sends the same requests to it and to the current build, and compares status, payload kind, sample types and whether there are samples. Every difference must be declared with its old and new shape; a markdown report is written and uploaded by CI. Smoke tests now give the application the official async-profiler API (tools.profiler:async-profiler) for spans, and check that a conflicting ap-loader on the application classpath does not kill it.
- ProfilerService / Profile / ProfilerBusyException: a supported library API that hides the relocated async-profiler types and shares the agent's validation, status codes and temporary file handling; one profiling session at a time (409 when busy). - README rewritten as a short landing page in the style of perf-agent, with the details in docs/ (usage, span filtering, Go mode, configuration, library usage, upgrading) and a CONTRIBUTING guide. - Java examples live in src/test/java/.../examples, compiled and run by the build; ReadmeExamplesTest checks the docs show exactly that code. - docs/assets: images and a demo GIF generated with async-profiler 4.6 from a DemoShop workload (mvn -Passets verify), plus brand SVGs. - release.yml only attaches the jar to a published GitHub release; the Maven Central deploy stays manual. - Version 0.2.0.
… examples - The session lock is shared by every ProfilerService instance: Async Profiler is one per JVM, so a second service now gets ProfilerBusyException (409) instead of a 500. - The agent's HTTP server handles requests on daemon worker threads, so a request arriving during a recording gets an immediate 409 instead of waiting in line; the embedded server example does the same. - The servlet example defaults to pprof output for the /debug/pprof paths and is now executed by the tests (with minimal Servlet API fakes). - profile() reports a failure to create the temporary recording as an IOException, and the release guide notes that EXPECTED_CHANGES must be emptied when the baseline moves.
…vent async-profiler only reads 'interval' for CPU and wall-clock profiling: with event=alloc, lock or nativemem it had no effect (and the docs said it was in bytes). An explicit interval is now passed as alloc=, lock= or nativemem=, and the configured ap-agent.profiler.interval only applies to CPU and wall clock. The fixture allocates a few MB per second: with 1 KB arrays some one-second alloc profiles had no samples on CI runners.
dpsoft
marked this pull request as ready for review
November 15, 2026 18:56
dpsoft
marked this pull request as draft
November 15, 2026 19:01
dpsoft
marked this pull request as ready for review
November 15, 2026 19:03
The alloc, lock and nativemem options of async-profiler disable the event for an unparsable or negative value, and async-profiler then falls back to CPU profiling: a bad interval answered 200 with the wrong event. Validate explicit intervals against the units async-profiler reads (n/u/m/s for times, b/k/m/g for sizes) and answer 400 otherwise.
There was a problem hiding this comment.
🟡 Changes recommended
Malformed intervals can silently produce the wrong profile, rejected concurrent requests retain delete-on-exit entries, and compatibility tests can mistake bookkeeping events for samples.
3 open issues
What changed in this PR
Prepares ap-agent 0.2.0 by modernizing its profiler integration, improving HTTP profiling reliability, and adding a supported library API with regression coverage and documentation.
Changes:
- Upgrades to async-profiler 4.6, adds span filters, and corrects event selection, output conversion, error handling, and temporary-file cleanup.
- Introduces
ProfilerServiceandProfile, removes Vavr, and adds unit, endpoint, compatibility, and packaged-agent tests. - Updates CI and release workflows, reorganizes documentation, and adds reproducible profiling demos.
| File | Description |
|---|---|
| src/test/java/io/github/dpsoft/ap/util/StrSubstitutorTest.java | Tests literal substitution and unknown variables. |
| src/test/java/io/github/dpsoft/ap/util/ProfilerExecutorTest.java | Tests validation, interruption, and recording cleanup. |
| src/test/java/io/github/dpsoft/ap/util/BuildInfoTest.java | Checks filtered build-version metadata. |
| src/test/java/io/github/dpsoft/ap/util/BannerTest.java | Verifies banner text and versions. |
| src/test/java/io/github/dpsoft/ap/testing/Profiles.java | Adds profile-payload inspection helpers. |
| src/test/java/io/github/dpsoft/ap/testing/PprofTest.java | Tests protobuf decoding and workload shutdown. |
| src/test/java/io/github/dpsoft/ap/testing/Pprof.java | Adds a lightweight test-only pprof decoder. |
| src/test/java/io/github/dpsoft/ap/testing/FixtureApp.java | Supplies sampling, contention, and span workloads. |
| src/test/java/io/github/dpsoft/ap/ReadmeExamplesTest.java | Checks documentation snippets and runs examples. |
| src/test/java/io/github/dpsoft/ap/ProfilerServiceTest.java | Tests library behavior and session exclusivity. |
| src/test/java/io/github/dpsoft/ap/it/CompatibilityIT.java | Compares response shapes against the previous release. |
| src/test/java/io/github/dpsoft/ap/it/AgentSmokeIT.java | Exercises the packaged agent in separate JVMs. |
| src/test/java/io/github/dpsoft/ap/it/AgentProcess.java | Manages test JVM startup, requests, and cleanup. |
| src/test/java/io/github/dpsoft/ap/handler/AsyncProfilerHandlerTest.java | Covers HTTP outputs, filters, errors, and cleanup. |
| src/test/java/io/github/dpsoft/ap/functions/FunctionsTest.java | Tests query decoding and string helpers. |
| src/test/java/io/github/dpsoft/ap/examples/ServletFakesAccess.java | Exposes servlet test helpers across packages. |
| src/test/java/io/github/dpsoft/ap/examples/ServletFakes.java | Supplies minimal servlet request and response fakes. |
| src/test/java/io/github/dpsoft/ap/examples/ProfilerServlet.java | Demonstrates serving profiles through a servlet. |
| src/test/java/io/github/dpsoft/ap/examples/OrderService.java | Demonstrates tagging work with spans. |
| src/test/java/io/github/dpsoft/ap/examples/EmbeddedServerExample.java | Demonstrates an embedded profiling endpoint. |
| src/test/java/io/github/dpsoft/ap/demo/DemoShop.java | Provides the documentation-demo workload. |
| src/test/java/io/github/dpsoft/ap/config/AgentConfigurationTest.java | Verifies default configuration. |
| src/test/java/io/github/dpsoft/ap/command/CommandTest.java | Covers command options, intervals, and span filters. |
| src/main/resources/reference.conf | Clarifies sampling-interval scope in comments. |
| src/main/java/io/github/dpsoft/ap/util/StrSubstitutor.java | Preserves unknown variables and literal replacements. |
| src/main/java/io/github/dpsoft/ap/util/Server.java | Adds concurrent request handling with daemon workers. |
| src/main/java/io/github/dpsoft/ap/util/Runner.java | Logs profiler-loading failures without propagating them. |
| src/main/java/io/github/dpsoft/ap/util/ProfilerExecutor.java | Revises recording, conversion, interruption, and cleanup. |
| src/main/java/io/github/dpsoft/ap/util/BuildInfo.java | Replaces Vavr and closes the metadata stream. |
| src/main/java/io/github/dpsoft/ap/util/Banner.java | Replaces Vavr and explicitly decodes UTF-8. |
| src/main/java/io/github/dpsoft/ap/ProfilerService.java | Adds the library entry point and session coordination. |
| src/main/java/io/github/dpsoft/ap/ProfilerBusyException.java | Represents concurrent-session rejection. |
| src/main/java/io/github/dpsoft/ap/Profile.java | Encapsulates profile output and temporary-file ownership. |
| src/main/java/io/github/dpsoft/ap/handler/AsyncProfilerHandler.java | Uses the service API and sets response metadata and errors. |
| src/main/java/io/github/dpsoft/ap/functions/Functions.java | Decodes query components once and skips valueless parameters. |
| src/main/java/io/github/dpsoft/ap/config/AgentConfiguration.java | Replaces Vavr-based configuration error handling. |
| src/main/java/io/github/dpsoft/ap/command/Command.java | Adds span filters and event-specific interval options. |
| src/main/java/io/github/dpsoft/ap/Agent.java | Connects agent startup to the service API. |
| scripts/bench-startup.sh | Measures startup timing and loaded classes. |
| scripts/assets/GenerateAssets.java | Generates profile pages, screenshots, and a demo GIF. |
| README.md | Adds a concise quickstart and documentation index. |
| pom.xml | Updates dependencies and configures tests, coverage, and assets. |
| docs/usage.md | Documents parameters, outputs, errors, and recipes. |
| docs/upgrading.md | Explains behavior and library API migration. |
| docs/span-filtering.md | Explains span instrumentation and filtering. |
| docs/library-usage.md | Documents supported APIs and integration examples. |
| docs/go-mode.md | Describes Go-compatible profiling endpoints. |
| docs/configuration.md | Documents settings, deployment, and requirements. |
| docs/brand/ap-agent-mark.svg | Adds the standalone brand mark. |
| docs/brand/ap-agent-lockup.svg | Adds the themed logo and wordmark. |
| docs/assets/flame-span-filter.html | Adds an interactive span-filtered profile example. |
| docs/assets/flame-nativemem.html | Adds an interactive native-memory profile example. |
| CONTRIBUTING.md | Documents builds, tests, assets, and release steps. |
| .github/workflows/release.yml | Builds and attaches jars to published releases. |
| .github/workflows/build.yml | Fixes PR triggers and expands CI coverage. |
| .github/dependabot.yml | Adds weekly grouped dependency and action updates. |
Review effort: Balanced
async-profiler parses intervals with strtol base 0 (a leading zero means octal) and multiplies by the unit without overflow checks; for alloc, lock and nativemem a bad result disables the event and it silently profiles CPU. Parse the value in base 10, apply the unit with overflow checks, and pass the normalized nanoseconds or bytes; anything else is a 400.
…ofiles - New ap-agent mark: an icicle (async-profiler flame graph turned upside down, as the docs show them) in async-profiler frame colours, with a monospace wordmark; it no longer reuses the perf-agent flame. - docs: demo, span filter and heatmap GIFs generated by the assets profile; the heatmap is recorded for 60 s and the GIF selects a range. - 'Interactive version' links opened the HTML source on GitHub: they now go through htmlpreview.github.io, which renders the live page.
- ProfilerService takes the session lock before creating the executor: a busy request (409) no longer creates a recording file. The temporary file is no longer registered with deleteOnExit(), which kept every path in memory for the life of the JVM (every path deletes it already). - CompatibilityIT only counts real sample events, not bookkeeping events such as profiler.Log or profiler.Window. - Demo GIF: zoom into a frame and back to the whole graph, search 'regex' (a partial match) instead of a frame present in every stack; GIFs loop.
- ProfilerExecutor.with only builds and validates the converter arguments; the temporary recording is created by run(). ProfilerService validates everything before taking the session, so an invalid tag regex is a 400 even while another session runs, and a busy request allocates nothing. - run() deletes the recording on any failure, Errors included (it relied on deleteOnExit for those before).
- CompatibilityIT retries Maven Central with backoff (Retry-After) on 429, 5xx and I/O errors, alternating repo1/repo.maven.apache.org: parallel CI jobs got rate limited. - scripts/assets/CheckLiveLinks.java opens every htmlpreview link of the docs in headless Chrome and checks that the profile canvas is painted; documented in CONTRIBUTING (the links only work once on main).
- The smoke tests attach the JaCoCo agent to the JVM running the current shaded agent jar (never the baseline release), so premain, Runner and Server are measured too. - scripts/coverage-it.sh reports that data against the classes inside the shaded jar (shading rewrites their bytecode; line numbers are the same). - CI uploads the unit and smoke test reports to Codecov through OIDC; Codecov merges them (95% of lines locally, 87% from unit tests alone). - Coverage badge in the README.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Modernizes ap-agent for a 0.2.0 release: async-profiler 4.6, span filtering, endpoint bug fixes, no Vavr, a
supported library API, a test suite (there was none) and a new README with docs and images.
Every phase was developed on its own branch, reviewed, and merged into
integration/modernize; CI runs on JDK11/17/21/25 (x64) and JDK 21 (arm64).
-javaagentto a separate JVM (standard mode, Go mode, configuration).pull_requestwas nested underpush(PRs never ran CI). JDK matrix + arm64, current actions, Dependabot weekly and grouped.--release 11; drop the retired OSSRH snapshots repository.tag=<regex>andlatency=<ms>parameters on top of the async-profiler Span API; query parameters decoded once.scripts/bench-startup.sh).ProfilerService/Profilelibrary API, short README in the style of perf-agent +docs/, images and demo GIF generated with async-profiler 4.6, release workflow fixed.Behaviour changes against 0.1.10
CompatibilityITdownloads 0.1.10 from Maven Central (SHA-1 checked), sends the same 22 requests to it and to thisbuild, and compares status, payload kind, sample types and samples. 16 responses are identical; these are the only
differences, all intended:
output=collapsedevent=wall&output=flameevent=nativemem&output=pprofcpumalloceventduration=abc400+ message?debug&...200=404empty404+ messageAlso:
Content-Typeon every response andContent-Encoding: gzipfor JFR/pprof (it was set after the headers weresent),
500with the cause when the profiler cannot record or convert,409when another session is running (theserver now handles requests concurrently), the profiler is always stopped and temporary files are always deleted.
Bugs found along the way
Multiple versions of async-profiler found, thrown frompremain). The agent now logs it and the application keeps running. Applicationsthat record spans must use
tools.profiler:async-profiler, notap-loader-*(documented, with a smoke test).?output=pprofalways recorded CPU;event=wallflame graphs were empty;interval=0returned a truncated body.intervalhad no effect withevent=alloc,lockornativemem: async-profiler only reads it for CPU and wallclock. It is now passed as
alloc=,lock=ornativemem=, andap-agent.profiler.intervalonly applies to CPUand wall clock.
Migration (library users)
CommandandProfilerExecutorno longer expose Vavr types and are internal; useProfilerServiceandProfile(see
docs/library-usage.mdanddocs/upgrading.md). The old README example did not compile against 0.1.10 either.Documentation examples are now compiled and run by the build (
ReadmeExamplesTest).Release
release.ymlfailed on the last releases. It now only attachesap-agent-<version>.jarandap-agent.jarto apublished GitHub release; the Maven Central deploy stays manual (see
CONTRIBUTING.md). Nothing is released bythis PR.
Notes for review
newCachedThreadPool(); production uses daemon threads.docs/assetsare regenerated withmvn -Passets verify -Dgpg.skip=true(Chrome, ffmpeg and Go).Fixes #40
Closes #3 — async-profiler rejected per-sample context ids (async-profiler#576) in favour of the Span API, which
tag=/latency=now expose.