[FR]: save model from pass OnnxStaticQuantization quant_preprocess = true

[xieofxie]

Proposal Summary

do we have a way to this? Sometimes I need to quantize from the model here and preprocess is not needed to be reruned multiple times.

We could do

• Add a new pass to run this only
• Add a parameter to OnnxStaticQuantization so only save the mode from this and no quantize

What component(s) does this request affect?

• OliveModels
• OliveSystems
• OliveEvaluator
• Metrics
• Engine
• Passes
• Other

[jambayk]

This is a good idea. Having to rerun this multiple times was a concern for us so we tried to do some caching with

Olive/olive/passes/onnx/quantization.py

Line 395 in 51d2c8a

preprocessed_temp_model_path = (

but that only persists during the lifetime of the pass and was meant for use with search.

I think for a general use case, we could create a new pass to do quant preprocess. Providing a directory to save the preprocessed model is a possibility too but I am not sure if we want the user to have to manage that.

[xieofxie]

Yes, a new pass is very useful. Because many parameters of OnnxStaticQuantization are based on the preprocessed model not the input model of the pass