Skip to content

Conversation

eguguchkin
Copy link
Contributor

replace map with slice to improve performance for billion-access queries

Fixes #170


  • I have read and followed all requirements in CONTRIBUTING.md;
  • I used LLM/AI assistance to make this pull request;

@eguguchkin eguguchkin force-pushed the 170-replace-map-in-hist branch 2 times, most recently from 53ac030 to 480019b Compare October 1, 2025 15:47
@codecov-commenter
Copy link

codecov-commenter commented Oct 1, 2025

Codecov Report

❌ Patch coverage is 81.81818% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.58%. Comparing base (abd6bfb) to head (5251276).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
frac/processor/search.go 79.31% 5 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #171      +/-   ##
==========================================
+ Coverage   71.37%   71.58%   +0.20%     
==========================================
  Files         200      200              
  Lines       18191    18216      +25     
==========================================
+ Hits        12984    13040      +56     
+ Misses       4492     4458      -34     
- Partials      715      718       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@eguguchkin eguguchkin force-pushed the 170-replace-map-in-hist branch from 480019b to 5251276 Compare October 1, 2025 23:55
@eguguchkin eguguchkin changed the title perf(histogram): replace map with slice to improve performance perf(search): replace map with slice to improve histogram performance Oct 1, 2025
@eguguchkin eguguchkin modified the milestone: v0.62.1 Oct 2, 2025
@eguguchkin eguguchkin requested review from cheb0 and forshev October 10, 2025 14:11
@eguguchkin eguguchkin added this to the v0.62.2 milestone Oct 10, 2025
Copy link
Member

@cheb0 cheb0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. However, it seems performance is not for free, because this hist can be quite big if user sets a low hist interval like 1s (or even lower). In that case, hists will take almost 700 kbytes of memory for 24h fraction. If number of hit buckets is low, map will probably win, because updating small memory set (which fit to L1 cache) is more efficient even despite of map.put call overhead.

However, with map it might be even worse if hist interval is low and every bucket has hits. In such scenarios, we run build histograms, but they can not be sent by GRPC (size limit).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Speed up histogram queries by using slices instead of maps

4 participants