From 56f58d83ce5d3cec62e4bb2b90d3ebd405f6f28a Mon Sep 17 00:00:00 2001 From: "mintlify[bot]" <109931778+mintlify[bot]@users.noreply.github.com> Date: Fri, 24 Jul 2026 13:09:04 +0000 Subject: [PATCH 1/4] docs: cover SMS/email OTP and clarify Prometheus histogram percentiles --- integrations/observability/prometheus-v2.mdx | 6 ++++++ learn/playwright/authentication.mdx | 13 +++++++++++++ learn/playwright/bypass-totp.mdx | 4 ++++ 3 files changed, 23 insertions(+) diff --git a/integrations/observability/prometheus-v2.mdx b/integrations/observability/prometheus-v2.mdx index b6c924c3..f90cd9d0 100644 --- a/integrations/observability/prometheus-v2.mdx +++ b/integrations/observability/prometheus-v2.mdx @@ -191,6 +191,12 @@ The different histogram metrics can all be used to compute averages. For example sum by(type) (rate(checkly_browser_check_web_vitals_seconds_sum{name="Check Name"}[30m])) / sum by(type) (rate(checkly_browser_check_web_vitals_seconds_count{name="Check Name"}[30m])) ``` + +The histogram metrics exposed by the Checkly exporter only ship a single `le="+Inf"` bucket — they don't include the finite bucket boundaries that `histogram_quantile()` needs. That means percentile queries such as `histogram_quantile(0.95, rate(checkly_api_check_timing_seconds_bucket[5m]))` will not return a useful value. + +Use the `_sum / _count` average pattern shown above for aggregations. If you need percentile-style latency signals, compute them in your alerting layer from the raw check run data rather than from the `_bucket` series. + + ## Heartbeat Metrics [Heartbeat monitors](/detect/uptime-monitoring/heartbeat-monitors/overview) are passive checks that wait for your scheduled jobs and automated processes to send a ping. The Prometheus exporter includes heartbeat checks in the standard `checkly_check_status` and `checkly_check_result_total` metrics, plus two heartbeat-specific gauges designed for [dead man's switch](https://en.wikipedia.org/wiki/Dead_man%27s_switch) alerting. diff --git a/learn/playwright/authentication.mdx b/learn/playwright/authentication.mdx index 61f6d709..f8761086 100644 --- a/learn/playwright/authentication.mdx +++ b/learn/playwright/authentication.mdx @@ -138,6 +138,19 @@ Notice the following in this code example: As mentioned, there is some prep you need to do. Refer to our full [article on 2FA login](https://www.checklyhq.com/learn/playwright/bypass-totp/) for all instructions. +### SMS and email one-time passcodes + +Unlike TOTP, one-time passcodes sent via SMS or email cannot be generated inside your test — the code is only known after the provider sends it. Playwright cannot read a text message or open a mailbox on its own, so you need a way to fetch the code from outside the browser and then type it into the login form. + +You have a few practical options: + +- **Use a test-only bypass.** The most reliable approach is to work with your application team to add a test-mode account, a shared static code, or a header/query flag that skips the SMS/email step in a non-production environment. This keeps the test deterministic and avoids depending on third-party delivery. +- **Poll a programmable inbox.** For email OTP, sign up your test user with a mailbox that exposes an API (for example, Mailosaur, Mailslurp, or a Gmail account you can query via the Gmail API). In your Playwright test, poll the API for the most recent message, extract the code with a regex, and `fill()` it into the OTP input. +- **Poll a programmable phone number.** For SMS OTP, use a provider that lets you receive SMS via API (for example, Twilio or a similar service) against a dedicated test number, then poll for the latest message body and extract the code. +- **Read the code from your own backend.** If your app stores the last issued OTP in a database or exposes it on an internal endpoint in staging, query that from the test instead of the delivery channel. + +Whichever path you pick, wrap the fetch step in a short polling loop with a timeout so the test fails clearly if the code never arrives, and store any credentials or API keys for the inbox provider as [environment variables](/platform/variables) rather than committing them to the repo. + ## Passkey & **WebAuthn** Passkeys are a fairly recent authentication scheme. Passkeys are passwordless, so passwords can’t be stolen or phished. There is also no need to memorize a passkey. Often, a passkey is tied to biometric data: every time you authenticate on some app or service with Face ID or Touch ID or using [YubiKey](https://www.yubico.com/) you are using a passkey. diff --git a/learn/playwright/bypass-totp.mdx b/learn/playwright/bypass-totp.mdx index 069aea54..f3a91ede 100644 --- a/learn/playwright/bypass-totp.mdx +++ b/learn/playwright/bypass-totp.mdx @@ -38,6 +38,10 @@ Following the flow, people log in with their credentials and are prompted with a And indeed, testing a passcode coming from a mobile application or SMS is challenging, but as it turns out, authenticator apps don’t include as much magic as I initially thought! + +This article covers **authenticator app / TOTP** flows only. If your login uses an OTP delivered by **SMS or email**, `otpauth` can't help — the code is generated server-side and delivered out of band. See the [SMS and email one-time passcodes](/learn/playwright/authentication#sms-and-email-one-time-passcodes) section of the authentication guide for the patterns we recommend (test-mode bypass, programmable inbox providers, or reading the code from your own backend). + + ### Authenticator apps and TOTP Let’s look at GitHub’s 2FA flow. When setting up 2FA with an authenticator app, the service greets you with the following screen. From 920892f81d4045adc949dfc5aacb00816e449903 Mon Sep 17 00:00:00 2001 From: Hannes Lenke Date: Tue, 3 Nov 2026 20:37:22 -0500 Subject: [PATCH 2/4] Update learn/playwright/authentication.mdx --- learn/playwright/authentication.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/learn/playwright/authentication.mdx b/learn/playwright/authentication.mdx index f8761086..4a3fbb6b 100644 --- a/learn/playwright/authentication.mdx +++ b/learn/playwright/authentication.mdx @@ -140,7 +140,7 @@ As mentioned, there is some prep you need to do. Refer to our full [article on 2 ### SMS and email one-time passcodes -Unlike TOTP, one-time passcodes sent via SMS or email cannot be generated inside your test — the code is only known after the provider sends it. Playwright cannot read a text message or open a mailbox on its own, so you need a way to fetch the code from outside the browser and then type it into the login form. +Unlike TOTP codes, one-time passcodes sent by SMS or email cannot be generated inside your test. The code is only known after the provider sends it. Playwright cannot read a text message or open a mailbox, so the test has to fetch the code from somewhere else and type it into the login form. You have a few practical options: From 1053dfcbb1bc1f7d2b69dde2c962063b6d1044ab Mon Sep 17 00:00:00 2001 From: Hannes Lenke Date: Tue, 3 Nov 2026 20:37:32 -0500 Subject: [PATCH 3/4] Update learn/playwright/bypass-totp.mdx --- learn/playwright/bypass-totp.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/learn/playwright/bypass-totp.mdx b/learn/playwright/bypass-totp.mdx index f3a91ede..f62ffe33 100644 --- a/learn/playwright/bypass-totp.mdx +++ b/learn/playwright/bypass-totp.mdx @@ -39,7 +39,7 @@ Following the flow, people log in with their credentials and are prompted with a And indeed, testing a passcode coming from a mobile application or SMS is challenging, but as it turns out, authenticator apps don’t include as much magic as I initially thought! -This article covers **authenticator app / TOTP** flows only. If your login uses an OTP delivered by **SMS or email**, `otpauth` can't help — the code is generated server-side and delivered out of band. See the [SMS and email one-time passcodes](/learn/playwright/authentication#sms-and-email-one-time-passcodes) section of the authentication guide for the patterns we recommend (test-mode bypass, programmable inbox providers, or reading the code from your own backend). +This article covers TOTP codes from authenticator apps only. `otpauth` cannot produce a passcode sent by SMS or email, because the server creates it and delivers it out of band. For those flows, see [SMS and email one-time passcodes](/learn/playwright/authentication#sms-and-email-one-time-passcodes). ### Authenticator apps and TOTP From 2634fc7f462184c85bc21b0583cd97c4b3e18b3b Mon Sep 17 00:00:00 2001 From: Hannes Lenke Date: Tue, 3 Nov 2026 20:38:38 -0500 Subject: [PATCH 4/4] docs: drop the Prometheus histogram note, covered by #776 Co-Authored-By: Claude Opus 5.5 --- integrations/observability/prometheus-v2.mdx | 6 ------ 1 file changed, 6 deletions(-) diff --git a/integrations/observability/prometheus-v2.mdx b/integrations/observability/prometheus-v2.mdx index 2d1ffea2..74f05c34 100644 --- a/integrations/observability/prometheus-v2.mdx +++ b/integrations/observability/prometheus-v2.mdx @@ -205,12 +205,6 @@ The different histogram metrics can all be used to compute averages. For example sum by(type) (rate(checkly_browser_check_web_vitals_seconds_sum{name="Check Name"}[30m])) / sum by(type) (rate(checkly_browser_check_web_vitals_seconds_count{name="Check Name"}[30m])) ``` - -The histogram metrics exposed by the Checkly exporter only ship a single `le="+Inf"` bucket — they don't include the finite bucket boundaries that `histogram_quantile()` needs. That means percentile queries such as `histogram_quantile(0.95, rate(checkly_api_check_timing_seconds_bucket[5m]))` will not return a useful value. - -Use the `_sum / _count` average pattern shown above for aggregations. If you need percentile-style latency signals, compute them in your alerting layer from the raw check run data rather than from the `_bucket` series. - - ## Heartbeat Metrics [Heartbeat monitors](/detect/uptime-monitoring/heartbeat-monitors/overview) are passive checks that wait for your scheduled jobs and automated processes to send a ping. The Prometheus exporter includes heartbeat checks in the standard `checkly_check_status` and `checkly_check_result_total` metrics, plus two heartbeat-specific gauges designed for [dead man's switch](https://en.wikipedia.org/wiki/Dead_man%27s_switch) alerting.