Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows Access Violation with fill_null_forward() on ChunkedArray #45086

Open
aiudirog opened this issue Dec 20, 2024 · 0 comments
Open

Windows Access Violation with fill_null_forward() on ChunkedArray #45086

aiudirog opened this issue Dec 20, 2024 · 0 comments

Comments

@aiudirog
Copy link

aiudirog commented Dec 20, 2024

Describe the bug, including details regarding any error messages, version, and platform.

I've encountered a consistent segfault when trying to use Pandas .ffill() with a boolean ChunkedArray on Windows (no problems on Linux):

import pandas as pd
import pyarrow as pa
print("Pandas:", pd.__version__)
print("PyArrow:", pa.__version__)

for i in range(1, 32):
    mx = 2 ** i
    print(i, mx)
    x = pd.Series(pd.NA, index=range(mx), dtype='boolean[pyarrow]')
    x[mx // 2] = True
    x.ffill()

This always crashes on the 12th iteration, when the array has 4096 values, across multiple computers with the following error:

Process finished with exit code -1073741819 (0xC0000005)

I dug under the hood to see what Pandas was doing and was able to create this pure PyArrow example (though I'm not sure if I'm using the API correctly):

import pyarrow as pa 
import pyarrow.compute as pc

pad = [None] * 4000
a = pa.chunked_array([pad, [True], pad], type=pa.bool_())
pc.fill_null_forward(a)

Additionally, I also tried to reproduce this with just pa.array() but that worked fine.


Operating System: Windows 10 (10.0.19045 Build 19045)
Processor: Intel Core i7-12850HX
Python Version: 3.12.7 (additionally confirmed by co-workers on 3.11 & 3.10)
PyArrow Versions Tested: 16.1, 17.0, 18.0, 18.1

Component(s)

Python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant