Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFE: Take contiguity caching into nvFuser #4043

Open
csarofeen opened this issue Mar 7, 2025 · 0 comments
Open

RFE: Take contiguity caching into nvFuser #4043

csarofeen opened this issue Mar 7, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@csarofeen
Copy link
Collaborator

At the moment the caching of contiguity changes is done in python which has some latency associated with it. It's not as bad with Ivan's PR: Lightning-AI/lightning-thunder@56b922a

However, this seems to be something that nvFuser should take ownership of in its caching system. Today we require a new fusion definition on any contiguity changes, as it can be a valuable optimization within a kernel. One idea is to put this aspect in the concretization pass in nvFuser if the latency of that would be tolerable.

This would prevent having any caching logic within python for nvfuser execution.

Original discussion in:
Lightning-AI/lightning-thunder#1840

@csarofeen csarofeen added the enhancement New feature or request label Mar 7, 2025
@csarofeen csarofeen changed the title Take contiguity caching into nvFuser RFE: Take contiguity caching into nvFuser Mar 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant