Skip to content

Implement Data Collection and Visualization for Web3.Storage Measurement Batch #511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
pyropy opened this issue Feb 10, 2025 · 3 comments · May be fixed by #560
Open

Implement Data Collection and Visualization for Web3.Storage Measurement Batch #511

pyropy opened this issue Feb 10, 2025 · 3 comments · May be fixed by #560
Assignees
@Winter-Soren
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@pyropy
Copy link
Contributor

pyropy commented Feb 10, 2025 •
edited
Loading

Description:

We need to collect and visualize the size of measurement batches submitted to web3.storage by spark-publish. Existing metric publish is already being sent to InfluxDB here and should be simply extended by adding new point. The collected data should be sent to InfluxDB for storage and further analysis.

Task Breakdown:

  1. Data Collection:

    • Capture the size of each measurement batch submitted to web3.storage by spark-publish.
    • Extend existing InfluxDB "publish" metric with new point that captures batch size.

  2. Data Visualization:

    • Create a chart to display size of measurement batches over time.
    • Create a chart to display size of measurement batches per round.

Acceptance Criteria:

  • The system should accurately capture and store the size of each measurement batch submitted to web3.storage.
  • The visualization dashboard should provide real-time insights into the measurement batch sizes.
@pyropy pyropy added enhancement New feature or request good first issue 🤗 Good for newcomers help wanted Extra attention is needed and removed good first issue 🤗 Good for newcomers labels Feb 10, 2025
@Winter-Soren
Copy link

Hey @pyropy, I would love to contribute to this issue. This is very interesting. Can you assign this issue to me?

@pyropy
Copy link
Contributor Author

pyropy commented Mar 10, 2025

Sure thing @Winter-Soren ! I'm afraid that you wouldn't be able to visualise the data yourself since that is supposed to be done in our internal Grafana dashboard but I can help you with that part.

I'm going to assinge the issue to you and If there's anything that you would need help with, let me know.

@Winter-Soren
Copy link

Sure thing @Winter-Soren ! I'm afraid that you wouldn't be able to visualise the data yourself since that is supposed to be done in our internal Grafana dashboard but I can help you with that part.

I'm going to assinge the issue to you and If there's anything that you would need help with, let me know.

Hey @pyropy,

Thanks for assigning me the issue! I appreciate the heads-up about the visualization part—I'll focus on the data collection and storage aspects first. Further I'll jump on visualisation part. :)

@Winter-Soren Winter-Soren linked a pull request Mar 28, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Winter-Soren Winter-Soren

Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
Space Meridian
Status: No status
Milestone
No milestone
2 participants
@pyropy @Winter-Soren