Repository navigation
[build] Add a Bazel build: shared module, standalone CppInterOp pin and wheel-layout parity - #55
Conversation
fbe1830 to
e098147
Compare
|
@aaronj0, here's the bazel infra added back in for cppjit. I'll open another PR against CppInterOp to adjust properly once this is in, and we should be cleanly bazel capable again! |
e098147 to
7b8a790
Compare
|
One more thing the 3.14 bump surfaced: a latent bug. |
470e2ef to
a89bf1f
Compare
a89bf1f to
ec15cb1
Compare
|
@aaronj0, can this merge now? it'd be great to get the bazel stuff settled and off the docket. |
Move the shared cppjit_bazel module out of the archived cppyy repo, add a root Bazel build that mirrors the CMake/wheel layout, and restore the best-effort CI job. - bazel/: the shared module (copts, the @llvm extension, the wheel-layout staging rule, the test macros) as its own Bazel module. - Root build: one merged cc_shared_library links into python/cppjit/libcppjit.so; stage_files recreates the wheel layout; test_selflocation gates the dladdr-relative resource lookup. - CppInterOp comes from a pinned archive, so a fresh clone needs no sibling checkout. Only LLVM_DIR comes from the host. - libcppjit.so expands the ${ORIGIN} token in CPPINTEROP_EXTRA_INTERPRETER_ARGS. - cppinterop_cc_test takes extra_tags for --test_tag_filters. - test/test_main.py imports the cppjit_bazel path fixups. The py_test bootstrap builds sys.path in-process, so site cannot auto-import them. - ci: the best-effort, non-gating bazel workflow, restored from the archived repo without the sibling clones. Co-developed-with-the-help-of: Claude Code (Fable 5, human in the loop)
ec15cb1 to
b16c316
Compare
aaronj0
left a comment
There was a problem hiding this comment.
Thanks @conrade-ctc! This LGTM % minor comment on the patch file
| diff --git a/BUILD.bazel b/BUILD.bazel | ||
| index dd26bb5..f64dd0b 100644 | ||
| --- a/BUILD.bazel | ||
| +++ b/BUILD.bazel |
There was a problem hiding this comment.
Do we still need this patch file?
There was a problem hiding this comment.
Yes, for now. CppInterOp main still declares bazel_dep(name = "cppyy_bazel") and loads from @cppyy_bazel. The patch renames these in the fetched tree.
The order is fixed: this PR gives cppjit_bazel a home. Right after this merges, a CppInterOp PR switches to it, and its CI clones cppjit instead of cppyy. After that merges, I bump the pin here and delete the patch.
CppInterOp's Bazel build depends on a shared module,
cppyy_bazel, that livesin
compiler-research/cppyyatcppyy/bazel/. That repo is archived and thefour Python repos merged into this one, so the module has no home in the
surviving stack. CppInterOp's
bazel (best-effort)job still clones cppyy tofind it, and both
bazel build //...andbazel test //:testsfail theretoday.
This PR gives the module a home here and adds a Bazel build for cppjit.
What it contains
bazel/— the shared module, moved out of the archived cppyy repo andrenamed
cppjit_bazel. It holds the copts and linkopts, the@llvmmoduleextension, the wheel-layout staging rule and the test macros. It is its own
Bazel module, so cppjit and CppInterOp can both depend on it without a
bzlmod cycle.
cc_shared_library, mirroring CMake:every
src/cpyrt/*.cxxplus the two interop translation units link intopython/cppjit/libcppjit.so.stage_filesrecreates the wheel layoutbeside it, which is what
cppinterop_paths()anchors at its owndladdrlocation.
pinned archive, so a fresh clone needs no sibling checkout.
Only
LLVM_DIRcomes from the host, as with CMake. A commentedlocal_path_overridekeeps the sibling dev loop available — the Bazelequivalent of CMake's
CPPINTEROP_SOURCE_DIR.${ORIGIN}expansion in interpreter args. No installer knows itsabsolute prefix at build time.
libcppjit.sonow rewritesCPPINTEROP_EXTRA_INTERPRETER_ARGS, so a consumer passes the token throughverbatim.
Non-gating (
continue-on-error), on LLVM 22 and 23; the standalone pinremoves the sibling clones the old job needed.
extra_tagsoncppinterop_cc_test. A consumer can select or excludeone test with
--test_tag_filters.Standalone constraint
The build must work from a fresh clone with nothing beside it. Verified: this
branch cloned into an empty directory, no sibling checkout, a fresh
--output_base, and--nosystem_rc --nohome_rcso no hostbazelrccansupply a registry.
test_memoryanalysis, added since, passes in a normal checkout.//:test_selflocationis the layout gate: it imports and JITs with cwd/and no path environment variables, so a regression in the
dladdr-relativeresource lookup fails here instead of only in an installed wheel.
Every commit in the series builds on its own.
CppInterOp pin
Pinned at
f8f0689a, the CppInterOp commit with the Bazel fix for thegenerated
BakedArgs.h(compiler-research/CppInterOp#1162).CPPINTEROP_GIT_TAGnames
d3c8cf7a, which predates that fix, so the two pins differ until theCMake pin moves to
f8f0689aor later. The in-tree patch onlyrenames the shared module in the fetched tree; it goes away once the pinned
commit carries the rename.
Parity with
pip install .//:site_packagesreproduces thepip install .payload class for class —one unversioned
libclangCppInterOp.so, the CppInterOp headers, the clangbuiltin resource dir,
libcppjit.sowith a matching soname, and thepure-Python sources — with a single remaining difference: Bazel ships the
library unstripped (181 MB against 156 MB stripped).
.dynsymsurvivesstripping, so the dlsym-based dispatch resolves either way. Happy to add a
strip step if you want exact bytes.
Follow-up
A companion PR in CppInterOp repoints its
bazel_depandlocal_path_overrideatcppjit_bazel, updates the CI clone, wires the testsources CMake already lists, and tags
VTableOverlayCrossTUBenchas abenchmark. It needs this PR first.