Skip to content

fix: do not count downtime towards crawler_runtime after migration or resurrection - #2204

Draft
Pijukatel wants to merge 3 commits into
masterfrom
claude/actor-resurrection-time-tracking-bixxkl
Draft

fix: do not count downtime towards crawler_runtime after migration or resurrection#2204
Pijukatel wants to merge 3 commits into
masterfrom
claude/actor-resurrection-time-tracking-bixxkl

Conversation

@Pijukatel

@Pijukatel Pijukatel commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Description

When a crawler run is resumed from a persisted state (Actor migration or resurrection), the reported crawler_runtime was inflated by the downtime between the runs, in several ways:

  • Statistics.__aenter__ started the periodic logger while crawler_last_started_at still pointed at the previous run's start, so the first statistics log of the resumed run included all the downtime between the runs. Later stats were computed from the new start time, which made the value appear to jump from a large number (e.g. 38s) back down to the real runtime (e.g. 542ms). The run timestamps are now reset before the periodic logger starts and the new start time is set after the initial log entry, so the first log reports exactly the runtime accumulated so far: zero for a fresh start, the previous total for a resumed run.
  • When the persisted state came from a run that did not finish cleanly (migration, abort), StatisticsState.model_post_init computed the accumulated runtime offset using the current time, so the downtime was permanently baked into all subsequent statistics of the resumed run.
  • The offset was reconstructed only from the last run's timestamps, so with two or more resurrections, the runtime of all runs before the most recent one was silently dropped from the total.

The accumulated runtime is now restored directly from the persisted crawlerRuntimeMillis value, which is correct for both clean and unclean shutdowns and accumulates across any number of resurrections. The timestamp-based reconstruction is kept as a fallback for states persisted by older versions, using stats_persisted_at (the moment the state was last saved) to approximate when an uncleanly finished run stopped. crawlerRuntimeMillis is also serialized as milliseconds now, matching its name and the other *Millis fields, instead of an ISO 8601 duration string.

Issues

  • None filed; reported via a resurrected Actor run log where the initial statistics table showed crawler_runtime of 38.01s that later dropped to 542.8ms.

Testing

  • Added regression tests in tests/unit/_statistics/test_persistence.py: the first periodic log of a fresh run reports exactly zero, the first log of a resumed run reports the previous runtime without a simulated 2-hour downtime, the runtime offset stays correct after an unclean shutdown, and the total runtime accumulates across multiple resurrections instead of keeping only the last run's segment. All fail without the fix and pass with it.
  • poe lint, poe type-check, and the statistics and basic-crawler unit tests pass locally.

Checklist

  • CI passed

🤖 Generated with Claude Code

https://claude.ai/code/session_01VMHwDx87jV4rnu5LkbpNxy

… resurrection

When a crawler run was resumed from a persisted state, two issues inflated
the reported `crawler_runtime`:

- `Statistics.__aenter__` started the periodic logger before updating
  `crawler_last_started_at`, so the first statistics log of the resumed run
  computed the runtime from the previous run's start time and included all
  the downtime between the runs (later stats were computed correctly, which
  made the value appear to "jump back").
- When the state came from a run that did not finish cleanly (migration,
  abort), `StatisticsState.model_post_init` computed the accumulated runtime
  offset using the current time, permanently baking the downtime between the
  runs into all subsequent statistics. The offset is now computed from the
  moment the state was last persisted instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VMHwDx87jV4rnu5LkbpNxy
@github-actions github-actions Bot added this to the 148th sprint - Tooling team milestone Aug 31, 2026
@github-actions github-actions Bot added t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics. labels Aug 31, 2026
@codecov

codecov Bot commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.74%. Comparing base (142ec58) to head (5c119b1).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #2204   +/-   ##
=======================================
  Coverage   93.74%   93.74%           
=======================================
  Files         181      181           
  Lines       12852    12853    +1     
=======================================
+ Hits        12048    12049    +1     
  Misses        804      804           
Flag Coverage Δ
unit 93.74% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The first periodic statistics log must report exactly the runtime
accumulated by previous runs: zero for a fresh start, the previous total
for a resumed run. `Statistics.__aenter__` now resets the run timestamps
before starting the periodic logger and sets the new start time after the
initial log entry is printed.

The accumulated runtime is now also restored from the persisted
`crawlerRuntimeMillis` value instead of being reconstructed from the last
run's timestamps, so runs before the most recent one are no longer lost
from the total, and a state persisted mid-run keeps the correct runtime.
The timestamp-based reconstruction is kept as a fallback for states
persisted by older versions. `crawlerRuntimeMillis` is also serialized as
milliseconds now, matching its name and the other `*Millis` fields,
instead of an ISO 8601 duration string.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VMHwDx87jV4rnu5LkbpNxy
… on Python 3.10

`datetime.fromisoformat` does not accept the 'Z' suffix until Python 3.11,
which failed the resumed-run test on the 3.10 CI jobs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VMHwDx87jV4rnu5LkbpNxy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants