You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reuse the same reader across ReadChunk calls so that prefetched blocks are not discared.
In the sequential non-parallel path (ReadRecord->ReadAheadFromBuffer->ReadChunk) ReadChunk reads the whole chunk into memory. Most readers will round up the read to the blocksize and thereby prefetch some data from the next chunk.
Currently ReadChunk recreates the reader each time and the previously prefetched data is lost and must be read again.
Instead use get_backing_reader() directly each time. We can do this since ReadChunk is only called in single threaded context (Initialize, ReadRecord (parallelism disabled)) so there cannot be any concurrent access to the underlying reader.
ReadTrace (before):
```
offset=64, size=1M <-- 1st chunk
offset=1M, size=1M
...
offset=8M, size=1M <-- block aligned read crosses chunk boundary
offset=8.3M, size=0.7M <-- redundant read for 2nd chunk
offset=9M, size=1M
...
```
ReadTrace (after):
```
offset=64, size=1M <-- 1st chunk
offset=1M, size=1M
...
offset=8M, size=1M <-- block aligned read crosses chunk boundary and is reused for the 2nd chunk
offset=9M, size=1M
...
```
PiperOrigin-RevId: 577312179
0 commit comments