Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Block Upload Manager Implementation #16974

Open
Tracked by #16768
derektriley opened this issue Dec 6, 2024 · 0 comments · May be fixed by #17135
Open
Tracked by #16768

Block Upload Manager Implementation #16974

derektriley opened this issue Dec 6, 2024 · 0 comments · May be fixed by #17135
Assignees
Milestone

Comments

@derektriley
Copy link
Contributor

derektriley commented Dec 6, 2024

Background
Implement the main upload manager responsible for coordinating block uploads to multiple providers.

See https://github.com/hashgraph/hedera-services/blob/16770-block-stream-bucket-uploader-design/hedera-node/docs/proposals/block-stream-bucket-uploader.md#configuration-classes for more details.

Acceptance Criteria

  • Implement BucketUploadManager class
  • Create BucketUploadListener interface
  • Update FileBlockItemWriter to support upload listeners
    • closeBlock() should notify registered listeners
  • Implement directory management logic
    • uploaded folder for block files which have been successfully uploaded to all buckets.
    • hashmismatch folder for block files which has a different MD5 hash than what's in a particular cloud bucket
  • Add proper error handling for upload failures
  • Implement hash checking logic
  • Create comprehensive unit tests
  • Document all public APIs (JavaDocs)

Dependencies
N/A

@github-project-automation github-project-automation bot moved this to 📋 Backlog in Services Team Dec 6, 2024
@derektriley derektriley moved this from 📋 Backlog to To Do in Services Team Dec 6, 2024
@derektriley derektriley self-assigned this Dec 11, 2024
@derektriley derektriley moved this from To Do to 👷🏼‍♀️ In Progress in Services Team Dec 11, 2024
@derektriley derektriley added this to the v0.58 milestone Dec 11, 2024
@derektriley derektriley linked a pull request Dec 19, 2024 that will close this issue
2 tasks
@derektriley derektriley linked a pull request Dec 19, 2024 that will close this issue
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 👷🏼‍♀️ In Progress
Development

Successfully merging a pull request may close this issue.

1 participant