CoreTrace Compiler

CoreTrace Compiler is a Clang/LLVM-based wrapper that can emit LLVM IR, build binaries, and optionally instrument code with runtime checks (alloc/bounds/trace/vtable). It can run in a file-based mode or an in-memory mode for tooling pipelines.

Build

Requirements: CMake, a C++20 compiler and LLVM/Clang 16 to 23.

CI builds and tests every combination below in its own job (the LLVM versions workflow), and first checks that the installed LLVM is the version the job is named after:

System	LLVM	Project built with	Tests
Linux (Ubuntu 24.04)	16 to 23	the same version's clang; GCC for LLVM 20	unit, smoke, install, runtime fixtures at -O0 and -O2, pass fuzzing
macOS 15	19, 20, 23 (Homebrew)	AppleClang	unit, smoke, install, runtime fixtures at -O0 and -O2, pass fuzzing
Windows (Server 2022)	19.1.7, 20.1.0, 22.1.8, 23.1.2 (official archives)	the archive's clang-cl	unit, smoke, install, runtime fixtures at -O0 and -O2 under Git Bash, without those that need POSIX

How instrumented code is compiled, and what differs between LLVM versions: see LLVM versions.

The Build workflow adds, with LLVM 20 on Linux: AddressSanitizer, LeakSanitizer and UndefinedBehaviorSanitizer builds, a build without the runtime, multi-arch Docker tests on main, and a line coverage report of compilerlib, cc and the runtime. The report is in the job's summary, and in the coverage-html artifact; it enforces no threshold. Locally, on Linux, -DENABLE_COVERAGE=ON builds with Clang's source-based coverage. A last job builds the runtime with ThreadSanitizer (-DENABLE_TSAN_RUNTIME=ON) and runs its unit tests and the runtime fixture suites with programs built with -fsanitize=thread: a data race in the runtime fails it.

The Benchmarks workflow measures the run-time overhead of instrumentation, on demand: it builds the programs in bench/ without instrumentation and with several sets of modules, and writes the median time of each, as a multiple of the plain build's, to its summary. Locally: python3 scripts/bench/overhead.py --cc build/cc.

Quick build:

mkdir -p build && cd build
./build.sh

Windows (native, produces cc.exe under dist/windows/bin):

powershell -ExecutionPolicy Bypass -File .\scripts\build-windows.ps1 `
  -LLVMDir "C:\LLVM\lib\cmake\llvm" `
  -LoggerSourceDir "C:\Users\shookapic\Documents\coretrace-log" `
  -Configuration Release

The LLVM 23 archive links with zlib and zstd, which it does not ship (#149). Install them for the static C runtime the project uses, for example with vcpkg, and let CMake find them before building:

vcpkg install zlib zstd --triplet x64-windows-static
$env:CMAKE_PREFIX_PATH = "<vcpkg root>\installed\x64-windows-static"

macOS:

mkdir -p build && cd build
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_DIR=$(brew --prefix llvm)/lib/cmake/llvm \
  -DClang_DIR=$(brew --prefix llvm)/lib/cmake/clang \
  -DUSE_SHARED_LIB=OFF

Linux:

mkdir -p build && cd build
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_DIR=/usr/lib/llvm-${LLVM_VERSION}/lib/cmake/llvm \
  -DClang_DIR=/usr/lib/llvm-${LLVM_VERSION}/lib/cmake/clang \
  -DCLANG_LINK_CLANG_DYLIB=ON \
  -DLLVM_LINK_LLVM_DYLIB=ON \
  -DUSE_SHARED_LIB=OFF \
  && cmake --build build -j"$(nproc)"

Without the instrumentation runtime: a tool that only embeds compilerlib to produce LLVM IR can pass -DCORETRACE_COMPILER_BUILD_RUNTIME=OFF. The runtime and its logger, coretrace-log, are then neither fetched, built nor installed; compilerlib, cc and the compilerlib unit tests still build. --instrument then links a runtime built elsewhere, from the directory CT_RUNTIME_LIB_DIR names, and otherwise fails with "instrumentation runtime archives not found".

Install

cmake --install build --prefix /opt/coretrace
/opt/coretrace/bin/cc --version

The prefix contains bin/cc, lib/libct_instrument_runtime.a, lib/libcoretrace_logger.a and include/. cc finds the runtime archives relative to its own location, so the prefix can be moved. Overrides: CT_RUNTIME_LIB_DIR (directory holding both archives), CT_CLANG (clang executable), CT_CLANG_RESOURCE_DIR (clang resource directory). Compilation runs in process in every mode: the clang executable is only run to assemble .s sources. Without any clang, Clang's resource directory is enough (CT_CLANG_RESOURCE_DIR, or the one recorded at build time): the driver then assumes the standard LLVM layout, <prefix>/bin/clang next to <prefix>/lib/clang/<version>.

Tests

After building into build/:

ctest --test-dir build --output-on-failure # unit tests (GoogleTest 1.17.0, fetched at configure time)
python3 test/examples/test_smoke.py        # compiler smoke tests (needs coretrace-testkit)
bash test/run_autofree_tests.sh            # auto-free fixtures under test/
bash test/run_runtime_tests.sh             # alloc, new/delete, shadow and vtable fixtures
bash test/run_differential_tests.sh        # instrumented vs plain builds of bug-free programs

The shell runners accept an output directory as first argument, CC_BIN to point at another compiler binary, and CT_TEST_OPT to build the programs with an optimization flag (for instance CT_TEST_OPT=-O2; the default is -O0). CI runs all of them on Linux and macOS, and runs the shell runners a second time at -O2. Unit tests are built only when this project is the top-level CMake project; pass -DCORETRACE_BUILD_UNIT_TESTS=OFF to skip them.

Code Style (clang-format)

• Target version: clang-format 18 (CI uses this).
• The scripts run clang-format from PATH, or the binary that CLANG_FORMAT names: CLANG_FORMAT=clang-format-18 ./scripts/format.sh
• Format locally: ./scripts/format.sh
• Check without modifying: ./scripts/format-check.sh
• CMake targets: cmake --build build --target format or --target format-check
• CI: the clang-format GitHub Actions job fails if a file is not formatted.

Usage

./cc -S -emit-llvm test.cc
./cc -S test.cc
./cc -c test.c
./cc -c test.c -O2
./cc --instrument -o app main.c
./cc --instrument -o=app main.c
./cc -x c++ -o=appCpp main.cpp
./cc -x=c++ -o appCpp main.cpp
./cc --instrument --ct-modules=trace,alloc,bounds,vtable main.cpp
./cc --instrument --ct-shadow -o app main.c
./cc --instrument --ct-shadow-aggressive --ct-bounds-no-abort -o app main.c
./cc --instrument --ct-modules=vtable --ct-vcall-trace -o app main.cpp
./cc --in-mem -S -emit-llvm test.c

CLI Options

Core options:

• --instrument: enable CoreTrace instrumentation (required for --ct-* flags).
• --in-mem, --in-memory: print LLVM IR to stdout (use with -S -emit-llvm).

Instrumentation toggles:

• --ct-modules=<list>: comma-separated list trace,alloc,bounds,vtable,all.
• --ct-shadow: enable shadow memory in the produced binary.
• --ct-shadow-aggressive, --ct-shadow=aggressive: aggressive shadow mode.
• --ct-bounds-no-abort: do not abort on bounds errors.
• --ct-no-trace / --ct-trace: disable/enable function entry/exit instrumentation.
• --ct-no-alloc / --ct-alloc: disable/enable malloc/free instrumentation.
• --ct-no-bounds / --ct-bounds: disable/enable bounds checks.
• --ct-no-autofree / --ct-autofree: disable/enable auto-free on unreachable allocations.
• --ct-no-alloc-trace / --ct-alloc-trace: disable/enable malloc/free tracing logs.
• --ct-no-vcall-trace / --ct-vcall-trace: disable/enable virtual call tracing (Itanium ABI).
• --ct-no-vtable-diag / --ct-vtable-diag: enable/disable vtable diagnostics.

Frontend toggles:

• --ct-optnone: add optnone and noinline to user-defined functions.
• --ct-no-optnone: disable optnone injection.

Notes:

• All other arguments are forwarded to clang (e.g. -O2, -g, -I, -D, -L, -l, -std=...).
• Alloc instrumentation rewrites malloc/free/calloc/realloc/aligned_alloc/posix_memalign, mmap/munmap and sbrk/brk, the global C++ operator new (scalar and array, plain and nothrow), and the global operator delete in its scalar, array, sized, nothrow and destroying forms. Aligned operator new (std::align_val_t) is not rewritten, so its blocks are not tracked. Aligned operator delete is rewritten on Itanium targets (Linux, macOS) only: the Microsoft CRT allocates aligned blocks with _aligned_malloc, which only the aligned delete may release.
• Allocations are tracked where user code makes them: an allocation made by system-header code, such as a standard container's, stays untracked, even once inlined into user code. Releases go through the runtime everywhere, system headers included, since library code such as std::unique_ptr releases blocks user code allocated. The release of an untracked block is only traced, marked (unknown).
• Released blocks go to a quarantine instead of back to the allocator, as with AddressSanitizer: while a block is there, no other allocation receives its address, so an access through a dangling pointer is reported as heap-use-after-free. realloc of a tracked block moves it to a new block and quarantines the old one. The quarantine holds up to CT_QUARANTINE_MB megabytes (256 by default); past that, the oldest blocks are released for real, and a use-after-free or double free of them is no longer reported. Unmapped memory (munmap) is not quarantined.
• Bounds checks cover heap blocks and the stack objects of running frames, reported as stack-buffer-overflow. A local array or struct is tracked while its function runs when its address escapes, for instance to a callee, or when an access to it cannot be proven in bounds at compile time; accesses proven in bounds are not checked at all. As for heap blocks, an access is checked when its base is the start of the object, or any address inside it with --ct-shadow-aggressive. Variable-length arrays are not tracked, nor objects beyond the 512 innermost ones each thread registers.
• Vtable tooling requires C++ and an Itanium ABI (macOS/Linux).
• Clang automatically adds optnone at -O0. Use --ct-optnone to force the attribute even when passing -Xclang -disable-O0-optnone.

Leak report

When an instrumented program exits, the runtime lists the tracked blocks still allocated: ct: leaks detected count=N, then one ct: leak line per block, with the site that allocated it. The report is meant to run after the program's own exit-time code, so that the blocks that code releases are not reported.

What the tests check, in CI:

Exit-time code that releases blocks	Linux	macOS	Windows
Destructors of global objects of instrumented code	yes	yes	yes
atexit handlers registered by instrumented code	yes	yes	yes
Destructor functions (__attribute__((destructor))) of default priority, in instrumented code	yes	yes	yes
The same in objects compiled without --instrument and linked before the instrumented ones	yes	yes (#161)	yes
The same in a shared library the program links with	yes (#158)	yes (#158)	yes (#187)
A terminator placed in .CRT$XTU, in instrumented code	—	—	yes

On Linux, the report is registered from the program's last destructor function, and glibc runs it once every shared library has run its exit-time code. On macOS, the exit-time code of the program and of its dylibs is one list, run in reverse order of registration; the report's handler runs the rest of that list before reporting, so the report comes last whichever tool links the program.

On Windows, the report runs from a TLS callback of the executable, when it detaches at process exit: after the program's exit-time code and that of every DLL, whether the program returns from main, calls exit() or calls ExitProcess, which skips the C runtime's exit handlers. The loader lock is held then, and every other thread has been terminated: if one of them held the runtime's allocation table, the report says ct: leak report skipped instead of waiting for it (#187).

A DLL that links the runtime, as an instrumented DLL does, has a copy of the runtime of its own, with its own table. When the DLL uses the static C runtime, as cc links it by default, that copy reports the DLL's leaks when the process exits, after the DLL's atexit handlers, the destructors of its global objects and its destructor functions: from a terminator of the DLL, or, when the DLL itself ends the process with exit(), when the loader detaches it. That comes before the exit-time code of the modules that detach after the DLL. It reports nothing when FreeLibrary unloads the DLL (#232).

DbgHelp on Windows

On Windows, the runtime names the targets of virtual calls in vtable boxes, and the frames of crash backtraces (CT_BACKTRACE), through DbgHelp. A program that uses DbgHelp itself shares it with the runtime as follows:

• Sessions. Each copy of the runtime, the executable's and that of each DLL that links the runtime, opens a session of its own, on a handle of the process that it duplicates, as SymInitialize's documentation asks of each component that inspects the current process. The runtime does not initialize, clean up or change the modules of the session on GetCurrentProcess(): the program's session keeps its modules and search path. The copy in a DLL closes its session and its handle of the lock below when FreeLibrary unloads the DLL, after the destructors of the DLL's global objects. When another thread still holds that lock after the 500 ms wait, the session and its handle of the process stay open instead: closing the session needs the lock, and waiting longer could deadlock with a thread that waits for the loader lock, which FreeLibrary holds. When the process terminates, the copy in a DLL leaves them to the system, as the DllMain guidance asks of a DLL that detaches then. The executable's copy keeps them until the process ends.
• Options. DbgHelp's options are the process's. The runtime adds SYMOPT_DEFERRED_LOADS, SYMOPT_LOAD_LINES and SYMOPT_UNDNAME to those the program set, and removes none.
• Lock. DbgHelp is single-threaded for the whole process, whatever the session. Every copy of the runtime calls it with one lock held: the semaphore Local\coretrace-dbghelp-<pid>, the process ID in decimal. A program that calls DbgHelp from a thread while instrumented code runs takes that semaphore around its calls (CreateSemaphoreW(nullptr, 1, 1, name), then WaitForSingleObject and ReleaseSemaphore); otherwise its calls may run at the same time as the runtime's.
• When DbgHelp is busy. A vtable box waits 500 ms at most for the lock, then names nothing (<unknown>); a thread that waited in vain does not wait again until it gets the lock. The crash handler does not wait, and prints bare addresses then. A box printed with the loader lock held, from DllMain or a DLL's constructors, thus never waits forever for a thread that waits for the loader lock.
• Modules. The runtime follows the loader's notifications (LdrRegisterDllNotification), so that a vtable box names the functions of the DLLs loaded when it is printed, a DLL replaced by another one at the same address included. When its session cannot be brought up to date, the box names nothing rather than a function of a DLL unloaded since. The crash handler, which cannot list the modules, prints bare the frames of the modules that the loader has unloaded since they were last brought up to date: when the handler was installed, or at a vtable box since. When the loader has unloaded more DLLs since than the runtime keeps the bases of (64), the handler names only the frames of the executable and of the module that holds the runtime's copy, which the loader cannot have unloaded. Once that copy has stopped following the loader, when the module that holds it detaches, the handler prints every frame bare.

LLVM versions

CoreTrace Compiler builds and instruments with LLVM 16 to 23. Most behaviour is the same on every version; this section lists what differs, and why.

How instrumented code is compiled

An instrumented compilation goes through the same steps on every LLVM version:

1. Clang's frontend produces the module, without optimizing it.
2. In the calls to operator new and operator delete that the alloc module tracks, compilerlib removes the mark that lets the optimizer omit the allocation of a new-expression, as C++ allows.
3. Clang's optimization pipeline runs once, with the options of the compilation.
4. The instrumentation passes run on the optimized module, which is then verified.
5. Clang's backend writes the module, with every code generation option of the compilation, such as -ffunction-sections, -fdata-sections, -gsplit-dwarf and -fstack-usage, and without optimizing it again.

With --instrument, the compiler also adds to clang's arguments:

• -gline-tables-only, unless debug information is already requested, as clang decides it: by an option such as -g, -g2 or -gdwarf-5, not undone by a later -g0. Options that only change debug information, such as -gcolumn-info or -gsplit-dwarf, do not request it. The passes record the source location of each allocation;
• -fPIE on Linux, unless -fPIC or -fPIE is given;
• one -fno-builtin-<name> per C allocation function the alloc module rewrites (malloc, calloc, realloc, aligned_alloc, posix_memalign, free). Otherwise clang may remove or merge their calls before the passes run, and the allocations would go untracked. Other C library functions keep their builtin status: setjmp keeps returns_twice, and explicit memcpy, memset and memmove calls become memory intrinsics, which the bounds module checks.

Link-time optimization is not supported with --instrument: -flto and -flto=<mode> make an instrumented compilation fail with an error, unless -fno-lto turns them off (#163).

-fsave-optimization-record records the remarks of steps 3 to 5 in one file, as for a plain compilation; a failed compilation leaves no record. The diagnostics of steps 3 to 5 are Clang's, as for a plain compilation (#167): -Rpass, -Rpass-missed and -Rpass-analysis select the remarks printed, warning options such as -Werror and -w apply, and each diagnostic is located in the source. -ftime-trace writes its file, which also covers the instrumentation (#166).

An instrumented compilation prints its diagnostics as clang prints them, those of the frontend included (#172): with source excerpts, notes such as the inline assembly an error comes from, and the option of each warning or remark, for example [-Wunused-variable]. CompileResult::diagnostics holds this text, so a host of compilerlib that parses it sees that format too.

Other notes per version

• Windows with LLVM 23: the official archive links with zlib and zstd, which it does not ship; the Windows build section says how to provide them (#149).
• Clang 23, Apple targets: Objective-C messages go through selector stubs such as objc_msgSend$new. The alloc module recognizes them, so the objects are tracked as with earlier versions.
• LLVM 19, arm64: LLVM's own AArch64 code generator crashes on some IR at -O0, with or without --instrument. This is an LLVM bug: plain clang -c crashes on the same input. The pass fuzzer reports such modules as skipped; in CI, it has met this crash with LLVM 19 only.

Objective-C and Objective-C++

On macOS, .m and .mm files (or -x objective-c and -x objective-c++) are compiled against the Apple Objective-C runtime; Objective-C++ sources also link the C++ standard library. The usual clang flags apply, such as -fobjc-arc, and Foundation programs link with -framework Foundation:

./cc --instrument --ct-modules=alloc -fobjc-arc -framework Foundation -o app main.m

• trace prints methods under their Objective-C name, such as -[Greeter greet:].
• alloc tracks malloc/free and C++ new/delete called from methods, and the Objective-C objects the program creates with +alloc, +allocWithZone:, +new or [[X alloc] init], with or without ARC, until the Objective-C runtime deallocates them. Objects still alive at exit are reported with the other leaks. To see deallocations, the runtime replaces -[NSObject dealloc] at start-up with a wrapper that calls the original.

Limits:

• Objects of Apple's system classes (Foundation, CoreFoundation, libobjc) and of classes outside the NSObject hierarchy, such as NSProxy subclasses, are not tracked: some live for the whole process or are released without -[NSObject dealloc], and would read as leaks.
• Only macOS with the Apple runtime is tested; Linux (GNUstep) and Windows are not.

Auto-free GC Scan (Conservative)

The runtime can run a conservative root scan (stack/regs/globals) to decide whether an allocation is still reachable. The periodic scan keeps every block reachable from those roots, directly or through other tracked blocks, and releases the others. It runs on macOS only, and is optional and controlled by environment variables.

Typical usage:

CT_AUTOFREE_SCAN=1 CT_AUTOFREE_SCAN_START=1 \
CT_AUTOFREE_SCAN_PTR=0 \
CT_AUTOFREE_SCAN_GLOBALS=0 \
CT_AUTOFREE_SCAN_PERIOD_MS=200 \
CT_AUTOFREE_SCAN_BUDGET_MS=100 \
CT_DEBUG_AUTOFREE_SCAN=1 \
./app

Without GC scan (auto-free only from compile-time analysis):

./app

With GC scan (conservative root scan):

CT_AUTOFREE_SCAN=1 CT_AUTOFREE_SCAN_START=1 \
CT_AUTOFREE_SCAN_PTR=1 \
CT_AUTOFREE_SCAN_GLOBALS=1 \
CT_AUTOFREE_SCAN_PERIOD_MS=200 \
CT_AUTOFREE_SCAN_BUDGET_MS=100 \
CT_DEBUG_AUTOFREE_SCAN=1 \
./app

Environment variables (ms can be floating-point; US/NS override MS):

• CT_AUTOFREE_SCAN=1: enable conservative scanning.
• CT_AUTOFREE_SCAN_START=1: run a scan at startup and launch a periodic scan thread.
• CT_AUTOFREE_SCAN_PERIOD_MS=N: period between scans when START=1 (default: 1000ms).
• CT_AUTOFREE_SCAN_PERIOD_US=N: period between scans in microseconds.
• CT_AUTOFREE_SCAN_PERIOD_NS=N: period between scans in nanoseconds.
• CT_AUTOFREE_SCAN_BUDGET_MS=N: time budget per scan; if exceeded, no frees are performed (default: 5ms; 0 for no limit).
• CT_AUTOFREE_SCAN_BUDGET_US=N: time budget per scan in microseconds.
• CT_AUTOFREE_SCAN_BUDGET_NS=N: time budget per scan in nanoseconds.
• CT_AUTOFREE_SCAN_STACK=0/1: scan thread stacks (default: 1).
• CT_AUTOFREE_SCAN_REGS=0/1: scan registers (default: 1).
• CT_AUTOFREE_SCAN_GLOBALS=0/1: scan globals (__DATA segments) (default: 1).
• CT_AUTOFREE_SCAN_INTERIOR=0/1: treat interior pointers as roots (default: 1).
• CT_AUTOFREE_SCAN_PTR=0/1: enable per-pointer scan before auto-free (default: 1).
• CT_DEBUG_AUTOFREE_SCAN=1: log scan activity only when a scan frees or times out.
• CT_DEBUG_AUTOFREE_SCAN=2: log every scan plus per-pointer scans.

Use cases:

• Keep CT_AUTOFREE_SCAN_PTR=1 for a conservative safety check before any auto-free.
• Set CT_AUTOFREE_SCAN_PTR=0 for immediate auto-free on unreachable sites, and rely on periodic scans.
• Disable CT_AUTOFREE_SCAN_GLOBALS=0 to reduce scan cost when you see timeouts: the periodic scan then releases blocks that only globals refer to.

Notes:

• This is conservative: stale values on stack/regs/globals can keep a pointer "reachable".
• If a scan times out (budget exceeded), nothing is freed to avoid false positives.

Vtable diagnostics (--ct-vtable-diag):

• Logs an init line with alloc-tracking state: enabled/disabled and the reason if disabled.
• Warns when vptr/vtable data is invalid: null this pointer, missing vptr, or missing typeinfo.
• Warns when vtable address cannot be resolved to a module (dladdr/phdr/dyld fallback).
• Warns on module mismatch when both vtable module and target module are resolved.
• Notes partial resolution cases (only vtable or target resolved) and escalates if target is non-exec.
• Warns on static vs dynamic type mismatch when a static type is available.
• Warns when the object appears freed (alloc table says state=freed).

Using coretrace-compiler in Your Project

You can integrate coretrace-compiler into your own CMake project using FetchContent or by building it as a standalone library. The sample project in extern-project shows a minimal setup.

cd extern-project
mkdir -p build && cd build

cmake .. -DLLVM_DIR=$(brew --prefix llvm@20)/lib/cmake/llvm/ \
  -DClang_DIR=$(brew --prefix llvm@20)/lib/cmake/clang

make

This example demonstrates how to:

• Link against compilerlib_static or compilerlib_shared.
• Use the public API from include/compilerlib/.
• Build with the correct LLVM/Clang version (20 on macOS).

You can use the same pattern in any external project by passing the correct LLVM_DIR and Clang_DIR paths to CMake.

compilerlib::compile runs each -cc1 job on a thread of its own, with the 8 MiB stack Clang expects, and waits for it: a host can compile from a thread with a small stack, such as a secondary thread on macOS, whose stack is 512 KiB (#204). The time-trace profiler a host runs on its own thread, started with llvm::timeTraceProfilerInitialize, therefore does not record the compilation. With -ftime-trace, each job writes its own trace file, as clang does.