Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement graceful shutdown #2456

Merged
merged 25 commits into from
Dec 27, 2024
Merged
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
c480c41
feat: add util crate with the task module
CHr15F0x Dec 16, 2024
9fbeab3
refactor: use rayon where tokio::task::spawn_blocking was sub-optimal
CHr15F0x Dec 17, 2024
256c63f
feat(util/task): add cancellation aware std::thread::spawn wrapper
CHr15F0x Dec 17, 2024
be05b27
refactor: merge make_stream into the util crate
CHr15F0x Dec 17, 2024
4ec79f2
refactor: move AnyhowExt into the util crate
CHr15F0x Dec 17, 2024
4c70576
feat(make_stream/from_blocking): bail out upon cancellation
CHr15F0x Dec 17, 2024
aa63cd9
doc(util/task): minor refinement of doc comments
CHr15F0x Dec 17, 2024
d73b015
fix(sync/checkpoint): rollback to anchor is not committed to the DB
CHr15F0x Dec 17, 2024
03df8da
feat: track tasks in pathfinder-ethereum and pathfinder crates
CHr15F0x Dec 17, 2024
eaf2689
chore: cargo sort
CHr15F0x Dec 17, 2024
9b857e6
chore: remove dead code
CHr15F0x Dec 17, 2024
1969346
feat(rpc): enable graceful shutdown for the rpc server
CHr15F0x Dec 18, 2024
f7442dc
feat: track tasks in pathfinder-rpc
CHr15F0x Dec 18, 2024
27cf8f6
feat(pathfinder): force exit after a grace period
CHr15F0x Dec 18, 2024
8bc8a28
feat(pathfinder/config): add grace period to config
CHr15F0x Dec 18, 2024
7b4dadb
feat(monitoring): add graceful shutdown
CHr15F0x Dec 18, 2024
e3abcea
refactor(main): reorder initialization towards disallowing task detac…
CHr15F0x Dec 19, 2024
1bc3889
fix(p2p): storage connection pool too small
CHr15F0x Dec 23, 2024
52577a8
fix: critical task errors not propagated to process exit
CHr15F0x Dec 23, 2024
0900358
feat: make sure the last RW db connection pool is dropped as the last
CHr15F0x Dec 23, 2024
8bed9c2
feat: force orderly cancellation even if rpc server or p2p fail to start
CHr15F0x Dec 23, 2024
13643ee
fixup: allow interrupting migrations and trie pruning
CHr15F0x Dec 23, 2024
e4ef8e7
chore: update changelog
CHr15F0x Dec 23, 2024
423d84a
chore: remove surplus comments from cargo.toml files
CHr15F0x Dec 27, 2024
4b8cbd4
fix: post rebase build error
CHr15F0x Dec 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat(pathfinder): force exit after a grace period
  • Loading branch information
CHr15F0x committed Dec 27, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 27cf8f6f788efb8c3a1d5935cd98ff5587b266e6
16 changes: 12 additions & 4 deletions crates/pathfinder/src/bin/pathfinder/main.rs
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ use std::num::NonZeroU32;
use std::path::PathBuf;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use std::time::{SystemTime, UNIX_EPOCH};
use std::time::{Duration, SystemTime, UNIX_EPOCH};

use anyhow::Context;
use metrics_exporter_prometheus::PrometheusBuilder;
@@ -338,9 +338,17 @@ Hint: This is usually caused by exceeding the file descriptor limit of your syst

util::task::tracker::close();
tracing::info!("Waiting for all tasks to finish...");
util::task::tracker::wait().await;
tracing::info!("Waiting for all tasks to finish... done!");
Ok(())
// Force exit after a grace period
match tokio::time::timeout(Duration::from_secs(10), util::task::tracker::wait()).await {
Ok(_) => {
tracing::info!("All tasks finished successfully");
Ok(())
}
Err(_) => {
tracing::warn!("Graceful shutdown timed out");
Err(anyhow::anyhow!("Graceful shutdown timed out"))
}
}
}

#[cfg(feature = "tokio-console")]