Skip to content

GH-1217: Bounds check before alloc in var-width view vectors - #1291

Open
lidavidm wants to merge 3 commits into
apache:mainfrom
lidavidm:view-buffer-validate
Open

GH-1217: Bounds check before alloc in var-width view vectors#1291
lidavidm wants to merge 3 commits into
apache:mainfrom
lidavidm:view-buffer-validate

Conversation

@lidavidm

Copy link
Copy Markdown
Member

Also, document that when enable_unsafe_memory_access is enabled, all bets are off.

Reported by n0mi1k.

Also, document that when enable_unsafe_memory_access is enabled,
all bets are off.

Reported by n0mi1k.
@github-actions

This comment has been minimized.

@lidavidm lidavidm added the bug-fix PRs that fix a big. label Sep 10, 2026
@lidavidm
lidavidm marked this pull request as ready for review September 10, 2026 07:10

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit surprised as #1217 names this method as affected, but it isn't touched by this PR.

buffer.set(dataBuf, dataOffset, dataLength) -> ReusableByteArray.set() still does bytes = new byte[len] before the source getBytes() bounds check runs. The same allocate-before-check pattern just fixed two methods above in getData(int).

Should we include the same change here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, adding manual checks everywhere is going to be fairly brittle...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I consolidated all of these methods to use a single accessor that always checks the buffer size.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here: it's mentioned in #1217 but not changed in this PR.

allocateOrGetLastDataBuffer(viewLength) allocates a buffer sized by viewLength before the bounds-checked thisDataBuf.setBytes(..., dataBuf, dataOffset, viewLength) ever runs. A corrupted source view (let's say copyFrom/copyFromSafe on data loaded from an untrusted IPC stream) drives the same large-allocation DoS this PR fixes in getData, through the copy path instead.

final ArrowBuf dataBuf = dataBuffers.get(readBufIndex);

// allocate data buffer
ArrowBuf currentDataBuf = target.allocateOrGetLastDataBuffer(stringLength);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Named in #1217 😄

target.allocateOrGetLastDataBuffer(stringLength) allocated before currentDataBuf.setBytes(currentOffset, dataBuf, readBufOffset, stringLength) bounds-checks the source read, and stringLength (from getValueLength(i)) is never validated first.

This runs on every split/slice/transfer of a View vector, so it's a fairly reachable path for the same bug class this PR addresses elsewhere.

}

@Override
public ArrowBufPointer getDataPointer(int index, ArrowBufPointer reuse) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Named in #1217 😄

No array allocation here, but bufIndex/dataOffset from the view record are used to build an ArrowBufPointer with zero validation. Under arrow.enable_unsafe_memory_access=true, this is the "reads arbitrary native heap" case, not just a DoS.

}

@Override
public int hashCode(int index, ArrowBufHasher hasher) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as getDataPointer: dataOffset/bufIndex feed ByteFunctionHelpers.hash(hasher, dataBuf, dataOffset, dataOffset + length) with no validation.

}

@Test
public void testValidateInvalidOffsets() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ArrowBuf.setInt(index, value) takes a byte offset, not a field-slot index. The view record layout is length@0, prefix@4, bufferIndex@8, dataOffset@12 (4 bytes each), but setInt(1, 0), setInt(2, 0), setInt(3, 1024) write to byte offsets 1/2/3, overlapping the length field instead of landing on prefix/bufferIndex/dataOffset.

I just tested it: length ends up 64 (unchanged), prefix=4 (collateral overlap), bufferIndex=0, dataOffset=0 (not the intended 1024).

The test passes, but I believe because dataLength=64 exceeds the tiny data buffer's capacity, not because of an out-of-range dataOffset like the test name implies.

I suggest to use setInt(12, 1024) to actually cover the corrupted-offset scenario.

@github-actions github-actions Bot added this to the 20.0.0 milestone Sep 11, 2026
@lidavidm
lidavidm requested a balanced review from Copilot September 11, 2026 03:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The split-and-transfer path can still allocate from an unvalidated length.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds eager bounds checks for variable-width view vectors to prevent oversized allocations from corrupt offsets.

Changes:

  • Centralizes checked view-data access.
  • Adds regression coverage for invalid offsets.
  • Documents risks of disabling bounds checks.
File summaries
File Description
BaseVariableWidthViewVector.java Adds checked data access and updates callers.
TestVariableWidthViewVector.java Tests invalid view offsets.
ArrowBuf.java Adds checked array-copy helper.
BoundsChecking.java Documents unsafe-access risks.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +1531 to +1532
/** Helper to get a single view value with sanity checking. */
protected <T> T getData(int index, ViewElementConsumer<T> consumer) {
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix PRs that fix a big.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Out-of-bounds read for corrupt view offsets in BaseVariableWidthViewVector

3 participants