Skip to content

Fill gaps from assistant conversations: Prometheus vs billing, multi-repo git metadata - #446

Open
mintlify[bot] wants to merge 4 commits into
mainfrom
mintlify/b40fd0ef
Open

mintlify[bot] wants to merge 4 commits into
mainfrom
mintlify/b40fd0ef

Conversation

@mintlify

@mintlify mintlify Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Address two recurring assistant-conversation gaps: Prometheus check-result counter vs billing usage, and attaching git metadata across multiple source repositories.

Changes

  • integrations/observability/prometheus-v2.mdx: Note that checkly_check_result_total counts individual check-run results and is not intended to reconcile with billing usage; point to the billing view and checkly account CLI for actual plan usage.
  • cli/attaching-git-metadata.mdx: Add a "Working with multiple repositories" section describing the supported pattern of running checkly test/checkly deploy per-repo pipeline with CHECKLY_REPO_URL set for that repo, plus isolation guidance via groups/tags.

Context

Recent assistant threads showed users confused about a mismatch between the Prometheus counter and billed usage, and asking whether git metadata could be attached from multiple repos (backend + frontend) — cases where the assistant had to infer answers not clearly stated in the docs.

@mintlify

mintlify Bot commented Jul 17, 2026 •

Copy link
Copy Markdown
Contributor Author

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
checkly-422f444a 🟢 Ready View Preview Nov 4, 2026, 2:36 PM

@mintlify

mintlify Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
checkly-422f444a 🟡 Building – Jul 17, 2026, 1:08 PM

@hlenke hlenke left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs changes · The multi-repository advice would make one repository's deploy delete the other's checks: checkly deploy has no --tags flag, and groups or check globs do not separate two pipelines that deploy the same project. Give each repository its own project (logicalId) instead; a suggestion is inline.

Merge readiness

  • CI: all checks pass (docs-gate, preview-checks, static-docs-checks, Mintlify link-rot and vale). GitHub reports the PR as mergeable and clean.
  • Staleness: main changed integrations/observability/prometheus-v2.mdx since this PR's base (bc4370b added metrics and labels, 4d01c2a updated the settings navigation, 749370d casing). None of them touch the lines this PR edits, so no conflict and nothing reintroduced. cli/attaching-git-metadata.mdx is unchanged on main since the base.
  • Overlap: #449 edits the same page in a different hunk (no conflict between the two). #776 deletes integrations/observability/prometheus-v2.mdx, redirects it to /integrations/observability/prometheus, and rewrites that page without a billing note. If #776 lands first, this hunk becomes a modify/delete conflict: move the Note into the new prometheus.mdx, next to the checkly_check_result_total explanation. If this PR lands first, #776 has to carry the Note over.
  • Cross-file impact (pr_impact.py): nothing reported. All link targets exist on main (/platform/groups, /cli/checkly-deploy, /cli/checkly-account, the Billing deep link).

Inline comments: 1 Blocker, 1 Important.

What I checked
  • Read the whole diff from the merge base fe23fb8b to the head, and both pages on origin/main.
  • gh pr checks 446, gh pr view 446 --json mergeable,mergeStateStatus, git log <base>..origin/main on both files, open PRs touching the same files, scripts/pr_impact.py.
  • checkly-cli at origin/main@4e450744: packages/cli/src/commands/deploy.ts (its flags: no --tags; --tags exists on checkly test/trigger, not on deploy), packages/cli/src/services/util.ts (CHECKLY_REPO_URL takes precedence over repoUrl from the config, one URL per run), packages/cli/src/commands/account/plan.ts (shows plan, entitlements and feature limits, not usage).
  • main docs: cli/checkly-account.mdx (usage is reported by checkly account usage summary/series), cli/checkly-deploy.mdx (resources removed from code are deleted on deploy), constructs/project.mdx (logicalId, repoUrl), snippets/cli-env-vars.mdx.
  • Billing deep link: the route settings/account/billing exists in the webapp (monorepo origin/main@28c905592a, apps/webapp/src/pages/settings/account/billing/) and is already linked from two pages on main.
  • End-to-end, tested against local Checkly (monorepo 23a0c61682, CLI 10.1.0): two projects with the same logicalId and different repoUrl (a "backend" and a "frontend" repo, one API check each). Deployed repo A, then repo B with checkly deploy --force. Afterwards only repo B's check existed in the account; repo A's check had been deleted. Destroyed the project afterwards and confirmed nothing was left.
  • Could not verify: what the billing system counts as "active checks" and "check run volume", so the suggestion leaves those terms out.

Comment thread cli/attaching-git-metadata.mdx Resolved

@hlenke hlenke left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ready to merge · Both suggestions from the last review are applied as written. The only open item is merge order with #776, which deletes the page that holds the billing Note.

This is a re-review of the two commits since my review at adb614b7.

From the last review

  • Blocker, multi-repository setup: fixed. The section now gives each repository its own project with a unique logicalId and repoUrl, and it warns that a shared logicalId deletes the other repository's checks. The invalid --tags advice for checkly deploy is gone.
  • Important, billing Note: fixed. It now points at checkly account usage summary instead of checkly account plan and no longer names billing terms I couldn't verify.

Merge readiness

  • CI: the required checks docs-gate and static-docs-checks pass, as does the Mintlify validation. The optional preview-checks failed in "Resolve preview URL" after 21 minutes, the same preview timeout as in July. That step never reaches the content. The branch is based on fe23fb8b from July. Merging main into it, as on #506 and #449 today, makes the preview resolve again.
  • Mergeable, no conflicts. On main, prometheus-v2.mdx has changed since the base (bc4370b, 4d01c2a, 749370d), but none of those commits touch these lines.
  • Overlap with #776: #776 deletes integrations/observability/prometheus-v2.mdx and redirects it to /integrations/observability/prometheus.
    • If this PR lands first, #776 gets a modify/delete conflict on that file. When resolving it, keep the deletion and move this Note into the new prometheus.mdx, right after line 96 ("checkly_check_status can be useful for viewing the current status of a check…"). The same sentence sits right above the Note here.
    • If #776 lands first, rebase this PR and move the Note to the same spot.
  • #449 no longer touches prometheus-v2.mdx, so it no longer overlaps.

Inline comments

None.

What I checked
  • Incremental diff adb614b7..f8ba9506 (2 files) and the full head version of cli/attaching-git-metadata.mdx. Merge base with origin/main is fe23fb8b.
  • gh pr checks 446. For run 41754634717, the failed step is "Resolve preview URL".
  • git log fe23fb8b..origin/main on both files: the three commits above, none on the edited lines.
  • Anchors: the live page https://www.checklyhq.com/docs/constructs/project/ renders id="param-logical-id" and id="param-repo-url", so both new links resolve.
  • Placement: the new multi-repository section is appended after "Environment variables" at the end of the page. It doesn't push existing content down.
  • Where the Note would go in #776: refs/docs-review/pr-776:integrations/observability/prometheus.mdx:96.
  • The facts behind both fixes were verified in the last review and are unchanged:
    • checkly deploy flags in checkly-cli origin/main@4e450744.
    • checkly account subcommands.
    • The end-to-end test against local Checkly (monorepo 23a0c61682, CLI 10.1.0), which showed a second deploy with the same logicalId deleting the first repository's checks.
  • No new end-to-end tests: the commits only applied the suggested text.

#776 deleted integrations/observability/prometheus-v2.mdx. Keep the deletion and move the billing Note into integrations/observability/prometheus.mdx, after the checkly_check_status / checkly_check_result_total sentence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@hlenke hlenke left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ready to merge · The merge with main put the billing Note in the right spot in the new prometheus.mdx, and the content is unchanged. The failing preview-checks is a workflow problem, not this PR's content (see CI below).

This is a re-review of f8ba9506..64a3f6f0, a single merge of origin/main that brings in #776.

Since the last review

  • prometheus-v2.mdx is gone, as #776 deleted it and redirected /integrations/observability/prometheus-v2 (docs.json:2481). The Note now sits in integrations/observability/prometheus.mdx right after line 96 ("checkly_check_status can be useful…"), the spot I suggested last time. Its text is unchanged.
  • cli/attaching-git-metadata.mdx: unchanged since the last review.

Merge readiness

  • Mergeable, no conflicts (MERGEABLE / UNSTABLE). The merge base is now e905ff08, current main.
  • CI: docs-gate, static-docs-checks, and the Mintlify checks pass. The optional preview-checks failed again with "Timed out waiting for Mintlify preview URL", even though Mintlify Deployment succeeded 75 seconds into the wait.
    • Last time I said merging main would make the preview resolve. That was wrong. Here's the actual cause.
    • This PR has two Mintlify preview comments with the same marker. The first (created Jul 17 13:11) is the one Mintlify keeps updating. It shows 🟢 Ready as of today. The second (Jul 17 13:26) is stuck at 🟡 Building.
    • The "Resolve preview URL" step takes the last matching comment in creation order. It reads the stale one every 15 seconds until the 20-minute timeout.
    • Fix it by deleting or hiding the stale Jul 17 13:26 comment, then re-running the job. The lasting fix is to make the workflow pick the most recently updated comment (sort_by(.updated_at) | last) instead of last. That belongs in a separate PR.
  • Overlap: no other non-draft PR touches these files. Draft #774 also edits cli/attaching-git-metadata.mdx. Whichever lands second will need a rebase.
  • Cross-file impact: scripts/pr_impact.py reports nothing for either file.

Inline comments

None.

What I checked
  • git diff f8ba9506 64a3f6f0 on this PR's files. Only prometheus-v2.mdx (deleted) and prometheus.mdx (from #776) differ, both from the main merge.
  • Full diff from the new merge base e905ff08: the multi-repository section in cli/attaching-git-metadata.mdx and the 4-line Note in prometheus.mdx.
  • origin/main:integrations/observability/prometheus.mdx: line 96 is the sentence the Note follows, and line 60 defines checkly_check_result_total.
  • origin/main:cli/checkly-account.mdx:32: the usage summary subcommand the Note links to.
  • gh pr checks 446. gh run view 41849068944 --log-failed: "Mintlify Deployment: success" from 75s on, then the timeout at 1200s.
  • gh api repos/checkly/docs/issues/446/comments: the two mintlify[bot] comments with the mintlify-preview-comment-checkly-422f444a-mintlify-b40fd0ef marker and their status.
  • gh pr view 446 --json mergeable,mergeStateStatus. gh pr list --json number,files for overlap.
  • The facts behind both additions were verified in earlier reviews and are unchanged:
    • checkly deploy behavior: end-to-end test against local Checkly (monorepo 23a0c61682, CLI 10.1.0). A second deploy with the same logicalId deleted the first repository's checks.
    • checkly account subcommands.
    • The param-logical-id and param-repo-url anchors on the live project construct page.
  • No new end-to-end tests: this merge changed no content written by this PR.

This branch was successfully deployed

1 active deployment
staging — 64a3f6f0 Deployed Nov 4, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant