Use SkipBlockRangeIterator as approximation in SortedNumericDocValuesRangeQuery#15954
Open
sgup432 wants to merge 3 commits intoapache:mainfrom
Open
Use SkipBlockRangeIterator as approximation in SortedNumericDocValuesRangeQuery#15954sgup432 wants to merge 3 commits intoapache:mainfrom
sgup432 wants to merge 3 commits intoapache:mainfrom
Conversation
romseygeek
requested changes
Apr 14, 2026
Contributor
romseygeek
left a comment
There was a problem hiding this comment.
Thanks @sgup432, this is a great start. I think we can improve things further by caching the docid of the end of a block and avoiding comparisons on every match call for fully matching blocks.
| new TwoPhaseIterator(skipApprox) { | ||
| @Override | ||
| public boolean matches() throws IOException { | ||
| int blockMatch = classifyBlock(); |
Contributor
There was a problem hiding this comment.
Can we cache this somehow so that we're not doing the comparison on every match request? For example, if you know that the whole block matches then you can just check that the current docId() is lower than the block end.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Replace DocValuesRangeIterator with SkipBlockRangeIterator as the two-phase approximation when a DocValuesSkipper is available. This makes the approximation block-level only (no DV decoding), deferring value reads to matches(). In
conjunctions, this avoids wasted DV decoding when another field's block is NO.
This was done as suggested in the comment here - #15770 (comment) by @romseygeek
Also includes benchmark:
Results:
JMH Benchmark Results (1M docs, ops/s, higher is better)
Data patterns:
Query: BooleanQuery with N FILTER clauses, each a
SortedNumericDocValuesField.newSlowRangeQueryon a distinct field with skip index enabled.