Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement eager mode caching #394

Open
pranavm-nvidia opened this issue Nov 19, 2024 · 2 comments
Open

Implement eager mode caching #394

pranavm-nvidia opened this issue Nov 19, 2024 · 2 comments
Assignees

Comments

@pranavm-nvidia
Copy link
Collaborator

We should implement a caching mechanism for eager mode so that we don't need to compile each time. We should make it so that every constant (i.e. produced by a Storage op) becomes an input. That way, we only need to compare the text IRs (i.e. the graph structure) to check for cache hits.

@farazkh80
Copy link
Collaborator

farazkh80 commented Dec 4, 2024

Per discussion we should also inlcude

  • cache mode enable/disable using environment variables
  • size for cache memory (i.e. 2GB)
  • make sure the cache key string includes all ops vars and kwargs

@pranavm-nvidia
Copy link
Collaborator Author

We can add the environment variable in config.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants