Apply InfluxDB and Grafana Changes for Data Aggregation and Alignment to Optimize Query Performance

[davidpepm]

Summary

This issue focuses on changes required in InfluxDB data processing and Grafana dashboard configurations to ensure that key performance indicators (KPIs) are supported efficiently. Specifically, the changes will streamline data aggregation in InfluxDB and enhance data alignment in Grafana by combining aggregated and default buckets for more comprehensive and accurate KPI tracking.

InfluxDB Changes

• Data Filtering and Aggregation:
  • Filter and aggregate data specifically for the required KPIs.
  • Perform data aggregation at 6-hour intervals to capture periodic metrics.
  • Utilize two distinct buckets—two for aggregated data (_agg_stats, _agg_topic) and another for raw data (default)—to efficiently manage and query data based on time range.

Grafana Changes

• Data Alignment:

  • Implement logic to combine data from both default and _agg_stats, _agg_topic buckets, enabling the Grafana dashboard to use both raw and aggregated data for accurate visualizations.
  • Align data sources within Grafana panels to support KPI tracking by seamlessly integrating with InfluxDB’s 6-hour interval data.

• Enhanced KPI Support:

  • Ensure that aggregated data from InfluxDB is correctly visualized and filtered in Grafana, making KPI tracking more efficient and accurate.
  • Update panel transformations and queries to use the adjusted data sources, providing consistent results even as data is aligned across both raw and aggregated sources.

Acceptance Criteria

• InfluxDB queries filter and aggregate data at 6-hour intervals, aligning with KPI tracking requirements.
• Grafana panels accurately display combined data from both default and _agg_stats, _agg_topic buckets.
• Tests confirm accurate KPI tracking, with optimized query performance on both the InfluxDB and Grafana sides.