[v2] S3 progress indicator frequency #9545
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kdaily
force-pushed
the
s3-progress-indicators
branch
from
1983e09 to
d4e4f48
Compare
kdaily
force-pushed
the
cli-accessibility
branch
3 times, most recently
from
e0f7597 to
c522dbd
Compare
kdaily
force-pushed
the
s3-progress-indicators
branch
from
d4e4f48 to
2366b20
Compare
Adds parameter for output progress frequency, defaulting to 0 to keep current behavior. Add multiline output control to allow for writing the progress report to a new line.
kdaily
force-pushed
the
s3-progress-indicators
branch
from
4a32a93 to
7c922cc
Compare
aemous
requested changes
Jun 27, 2025
| @@ -331,6 +333,8 @@ def __init__(self, result_recorder, out_file=None, error_file=None): | |||
| """ | |||
There was a problem hiding this comment.
Can we add docs for frequency and oneline?
|
|
||
| def _override_progress_result_handler(self, result, result_handler): | ||
| if ( | ||
| type(result) in [ProgressResult] |
There was a problem hiding this comment.
nit: could this be type(result) is ProgressResult to be consistent with the same check in __call__ ?
| @@ -347,12 +351,31 @@ def __init__(self, result_recorder, out_file=None, error_file=None): | |||
| DryRunResult: self._print_dry_run, | |||
| FinalTotalSubmissionsResult: self._clear_progress_if_no_more_expected_transfers, | |||
| } | |||
| self._now = time.time() | |||
There was a problem hiding this comment.
non-blocking: seems this initial value is never used, consider initializing to None
| ) | ||
| result_handler(result=result) | ||
|
|
||
| def _override_progress_result_handler(self, result, result_handler): |
There was a problem hiding this comment.
non-blocking question: have we considered modifying _print_progress to check frequency and first with a similar check, and only print as needed, instead of adding a new override handler function?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
Add the ability to control the frequency of updating the S3 progress indicator bar.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.