Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Add narwhals tests to our test suite #17662

Open
vyasr opened this issue Dec 30, 2024 · 3 comments
Open

[FEA] Add narwhals tests to our test suite #17662

vyasr opened this issue Dec 30, 2024 · 3 comments
Labels
feature request New feature or request

Comments

@vyasr
Copy link
Contributor

vyasr commented Dec 30, 2024

Is your feature request related to a problem? Please describe.
@MarcoGorelli has reported a number of issues to cudf stemming from narwhals. narwhals is sussing out a number of issues that we don't currently see anywhere else except likely in the pandas test suite, where our pass rate remains too low to be used as an absolute pass/fail indicator in CI.

Describe the solution you'd like
We would benefit from running the narwhals test suite as part of cudf's CI similarly to how we run the pandas test suite.

Describe alternatives you've considered
None

Additional context
The narwhals tests look like they take <5 min, so they shouldn't be too expensive, but I would still recommend only running them on a single CUDA ver/Python ver/arch combination to limit resource usage.

@vyasr vyasr added the feature request New feature or request label Dec 30, 2024
@MarcoGorelli
Copy link
Contributor

MarcoGorelli commented Dec 30, 2024

Thanks for the ping! Glad to hear our test suite has been useful

You can run the Narwhals test suite in ~60-70 seconds: https://www.kaggle.com/code/marcogorelli/testing-cudf-in-narwhals?scriptVersionId=215469980

git clone https://github.com/narwhals-dev/narwhals --depth=1
cd narwhals
pip install -U -e ".[dev]"
pytest tests

So long as cuDF is installed, it will include cuDF in the parametrised tests

@vyasr
Copy link
Contributor Author

vyasr commented Dec 31, 2024

Awesome. From a quick look at the configuration it looks like the test suite automatically selects which narwhals backends to test. Is there a way to select some explicitly? If not, would you be open to adding some custom flags to your pytest_addoption implementation to allow for selecting the pandas tests, polars tests, etc? The reason that I ask is that if your test suite is that fast, we might even want to consider running it for each of our front-ends separately, i.e. run the cuDF tests, then run the pandas tests with cudf.pandas enabled, then run the polars test suite with GPU support via cuDF enabled.

@MarcoGorelli
Copy link
Contributor

Definitely in-scope, we'll look into how to provide this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants