Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FR]: add a new pass MergeModel to use with SplitModel #1605

Open
7 tasks
xieofxie opened this issue Feb 10, 2025 · 2 comments
Open
7 tasks

[FR]: add a new pass MergeModel to use with SplitModel #1605

xieofxie opened this issue Feb 10, 2025 · 2 comments
Labels
enhancement New feature or request

Comments

@xieofxie
Copy link
Contributor

Proposal Summary

For static quantization, some model is too big to run (I will get OOM). So I need to split it first.
But the result models are smaller, so I think we had better have a MergeModel pass to merge them back to one model

What component(s) does this request affect?

  • OliveModels
  • OliveSystems
  • OliveEvaluator
  • Metrics
  • Engine
  • Passes
  • Other
@xieofxie xieofxie added the enhancement New feature or request label Feb 10, 2025
@devang-ml
Copy link
Contributor

Are the resulting parts smaller because you optimized/quantized each split before trying to merge them together?

@xieofxie
Copy link
Contributor Author

Yes, for uint8, usually only 25%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants