[Pijukatel]

Description

After an Actor migration, resurrection or abort, crawler_runtime misbehaved: the first periodic log of the resumed run included the whole downtime (e.g. 38.01s right after resurrection of a run whose real runtime was ~0.5 s), the downtime could get permanently baked into the statistics, and with repeated resurrections only the last run's segment survived.

This is a v2-targeted rework that makes the runtime a plain accumulated statistic:

• StatisticsState.accumulated_runtime (persisted as crawlerRuntimeMillis) holds the runtime accumulated by the previous runs; the read-only crawler_runtime property adds the elapsed time of the run in progress, measured by the crawler_last_started_at/crawler_finished_at timestamps.
• The field serializer for accumulated_runtime writes the live total, so every persist is an exact checkpoint: a state persisted mid-run (periodic persist, migration event) carries the runtime up to that moment. After an unclean termination, the resumed run continues from the last checkpoint, so no downtime is ever counted and at most the tail since the last persist is lost.
• Statistics.__aenter__ clears the previous run's timestamps before the periodic logger starts, so the restored total is reported as-is and the downtime between runs can never leak into the runtime. The documented invariant: timestamps of a loaded state are history, not measurement anchors.
• A backwards wall clock jump during a run cannot push the runtime below the accumulated total (the live segment is clamped at zero).
• No new public API beyond the documented accumulated_runtime field: Statistics communicates with the state only through the existing public timestamp fields.

Guarantees (all covered by tests in tests/unit/_statistics/test_persistence.py):

1. A fresh run's first periodic statistics log reports a runtime of exactly 0.
2. A resumed run's first periodic log reports exactly the previously persisted runtime.
3. The runtime accumulates correctly over any number of persist/resume cycles.
4. Aborts/migrations (unclean termination, no final persist) resume from the last periodic checkpoint, excluding downtime.

Breaking changes (v2)

• StatisticsState.crawler_runtime_for_serialization is removed; crawler_runtime is a read-only property (the setter was already removed by refactor!: remove deprecated StatisticsState.crawler_runtime setter #2315) backed by the new accumulated_runtime field.
• crawlerRuntimeMillis is serialized as a number of milliseconds, consistent with the other *Millis fields and with Crawlee for JavaScript, instead of an ISO 8601 duration string. (A v1 state with an ISO duration string still loads, since the timedelta_ms validator falls back to ISO parsing, but v1 compatibility is not a goal of this PR.)
• The unused StatisticsState fields errors, retry_errors, requests_finished_per_minute and requests_failed_per_minute are removed. They were never populated; the FinalStatistics fields of the same names are unaffected.

The upgrading guide (docs/upgrading/upgrading_to_v2.md) documents all of the above.

Related

• Related to Remove deprecated StatisticsState.crawler_runtime setter #1567 (the deprecated crawler_runtime setter itself was removed in refactor!: remove deprecated StatisticsState.crawler_runtime setter #2315).

🤖 Generated with Claude Code

https://claude.ai/code/session_01VMHwDx87jV4rnu5LkbpNxy

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (v2@758e546). Learn more about missing BASE report.

Additional details and impacted files

@@          Coverage Diff          @@
##             v2    #2204   +/-   ##
=====================================
  Coverage      ?   94.03%           
=====================================
  Files         ?      184           
  Lines         ?    13568           
  Branches      ?        0           
=====================================
  Hits          ?    12759           
  Misses        ?      809           
  Partials      ?        0           

Flag	Coverage Δ
unit	94.03% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Pijukatel]

Waiting for V2 as this might be breaking change and will inlcude removal of deprecated fields