Skip to content

ci: keep Go's module and build caches between Lorx jobs - #100

Merged
hallelx2 merged 2 commits into
mainfrom
halleluyaholudele/hal-3505-engine-ci-keep-gos-module-and-build-caches-between-lorx-jobs
Oct 18, 2026
Merged

hallelx2 merged 2 commits into
mainfrom
halleluyaholudele/hal-3505-engine-ci-keep-gos-module-and-build-caches-between-lorx-jobs

Conversation

@hallelx2

@hallelx2 hallelx2 commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Keeps Go's module downloads and compiled packages between Lorx CI jobs, so each job stops starting from nothing.

Why

Every Go job started from a bare golang:1.25 container: it downloaded every module and compiled every package from nothing. On 2026-10-17 vet took 96.8 s and build 98.7 s. Long jobs hold the Lorx server longer, and that server (lorx-1, an e2-small) peaked at 282% CPU and cancelled running jobs (HAL-3404).

How Lorx caches

A job's cache: directories are paths inside the checkout. Each is mounted at /work/<dir> from a volume named after the repository, the directory and the image. Volumes unused for a week are removed.

Change

  • vet, build, test: GOMODCACHE=/work/.cache/gomod and GOCACHE=/work/.cache/gobuild, with cache: [.cache/gomod, .cache/gobuild].
  • lint: the same, plus GOLANGCI_LINT_CACHE=/work/.cache/golangci.
  • Why .cache/ is safe: ./... skips it as a dot-directory, and it is already git-ignored.
  • fmt stays uncached: gofmt -l . would walk the module cache, and it takes a second anyway.

Paths: skip the Go jobs when no Go input changed

Lorx's step cache keys every step on the whole checkout, so any change, a README included, re-runs every step; it skips a step only when nothing at all changed. A job's paths: decides whether it runs at all. Each Go job now runs only when a change touches **/*.go, go.mod, go.sum, pkg/**, internal/**, cmd/**, testdata/**, .golangci.yml or .lorx/**. That covers everything Go reads: the migrations embedded from pkg/db/migrations, both testdata folders, the lint config and this file. A push touching only docs/, charts/, selfhost/ or the READMEs runs none of them.

The three layers together:

  • nothing changed: the step cache skips every step;
  • no Go input changed: paths skips the Go jobs;
  • Go changed: the jobs run, and the Go cache reuses every package that did not change.

Verification

  • bit ci check: valid, 1 workflow, 5 jobs in 3 levels, 5 path-filtered.
  • The Lorx run on this push is the first cold run; the next run of the same jobs should reuse the caches. That depends on the Lorx server coping (HAL-3404).

Closes HAL-3505

Summary by Sourcery

Speed up Go CI by reusing dependency and compilation caches and skipping validation for unrelated changes.

Enhancements:

  • Persist Go module, build, and golangci-lint caches between CI jobs to reduce repeated downloads and compilation.
  • Restrict Go validation jobs to changes that can affect Go code, dependencies, test data, lint configuration, or CI configuration.

Every Go job started from a bare golang:1.25 container: it downloaded every
module and compiled every package from nothing (vet 96.8 s and build
98.7 s on 2026-10-17). Long jobs hold the Lorx server longer, and that
server, an e2-small, peaked at 282% CPU and cancelled running jobs.

vet, build, lint and test now point GOMODCACHE and GOCACHE (and, for lint,
GOLANGCI_LINT_CACHE) at directories under .cache/ in the checkout. They
name those directories in cache:, which Lorx keeps between jobs as volumes
per repository, directory and image. ./... skips dot-directories, and
.cache/ is git-ignored. fmt stays uncached, because gofmt -l . would walk
the module cache. bit ci check: valid, 5 jobs in 3 levels.
@sourcery-ai

sourcery-ai Bot commented Oct 18, 2026

Copy link
Copy Markdown

Sorry @hallelx2, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 17 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@sourcery-ai

sourcery-ai Bot commented Oct 18, 2026

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Updates the Lorx CI workflow to persist Go module, build, and golangci-lint caches under git-ignored .cache directories, configures the corresponding environment variables, and leaves fmt uncached to avoid traversing the module cache.

File-Level Changes

Change Details Files
Persist Go dependency and compilation caches across CI jobs and point Go tooling at the mounted cache paths.
  • Configure GOMODCACHE and GOCACHE for vet, build, and test.
  • Register module and build cache directories with Lorx via each job’s cache list.
  • Keep fmt uncached because directory-wide formatting would traverse the module cache.
.lorx/actions/ci.yml
Add persistent lint caching alongside the shared Go caches.
  • Configure GOLANGCI_LINT_CACHE under the checkout’s .cache directory.
  • Register the golangci-lint cache together with the Go module and build caches.
.lorx/actions/ci.yml
Document the cache layout, lifecycle, and safety rationale in the CI configuration.
  • Explain Lorx volume mounting and reuse behavior.
  • Document why .cache is skipped by ./... and remains git-ignored.
  • Preserve existing GOFLAGS and job dependencies while applying caching to the relevant jobs.
.lorx/actions/ci.yml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai

coderabbitai Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: aa325f30-00b2-4c68-ae0d-5c4854552a49
📥 Commits

Reviewing files that changed from the base of the PR and between 3d28397 and 2acdfc0.

📒 Files selected for processing: 1
  • .lorx/actions/ci.yml

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

The CI configuration documents where job caches are mounted and configures Go module and build caches for vet, build, lint, and test. The lint job also configures the golangci-lint cache.

Changes: CI cache configuration

Layer / File(s) Summary
Configure job caches
.lorx/actions/ci.yml
The workflow documents its cache layout and configures Go cache paths for vet, build, lint, and test. The lint job also configures the golangci-lint cache. The fmt job remains uncached.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to 2acdf

The Go jobs can reuse their matching caches, while lint uses separate caches for its different image. No concrete merge-blocking risk is established.

Pre-merge checks: Passed 5
Check name Status Explanation
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: preserving Go module and build caches between Lorx CI jobs.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches: 💡 1
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Lorx's step cache keys every step on the whole checkout, so a docs-only
push re-ran all five Go jobs. Each job now names the paths it depends on:
Go files, go.mod and go.sum, pkg/, internal/, cmd/ and testdata/ (the
embedded migrations and test fixtures live there), the lint config and
the CI file. docs/, charts/, selfhost/ and the READMEs run none of them.
bit ci check: valid, 5 jobs, 5 path-filtered.
@hallelx2
hallelx2 merged commit 2859c80 into main Oct 18, 2026
2 of 8 checks passed
@hallelx2
hallelx2 deleted the halleluyaholudele/hal-3505-engine-ci-keep-gos-module-and-build-caches-between-lorx-jobs branch October 18, 2026 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant