Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue1035 retry too much, reduce cpu to some extent... #1086

Merged
merged 7 commits into from
Jun 4, 2024

Conversation

petersilva
Copy link
Contributor

improves #1035 hopefully.

trying to reduce cpu time usage. It creeps up after having run for a long time. Not sure if this fixes it, but it should help.
So this eliminates all file i/o in the main run loop. there will be no i/o if:

  • the retry_queue is empty
  • the metrics file has been recently written.

eliminated a lot of file i/o noise for checks for directory existence also.

We'll see how much this helps. It does help... but the extent is unclear.

Copy link

github-actions bot commented Jun 3, 2024

Test Results

242 tests   234 ✅  19s ⏱️
  1 suites    8 💤
  1 files      0 ❌

Results for commit 409f3eb.

♻️ This comment has been updated with latest results.

@petersilva petersilva marked this pull request as ready for review June 4, 2024 02:53
@andreleblanc11
Copy link
Member

andreleblanc11 commented Jun 4, 2024

Even if this doesn't fix the main load problem, it should still help drag it down at least a bit.

After we merge to development, we should pull the latest changes on ddsr development and see if the CPU keeps climbing high.

@petersilva
Copy link
Contributor Author

I think the basic test was failing because the metrics dir was not being created in one case. Re-added to beginnning of run() routine.

@petersilva petersilva merged commit 4ca6117 into development Jun 4, 2024
4 checks passed
@petersilva petersilva deleted the issue1035_retry_too_much branch June 6, 2024 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants