-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Trt refactor remove mixin #420
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Trt refactor remove mixin #420
Conversation
Trt refactor remove exporters
…ild trt engines from onnx model
This PR is in charge of adding a trt-config class. This class is responsible to build the trt engine given an onnx path and various trt-flags. As the same model might need different trt-configurations depending on which precision is used, a registry is used to collect all the model configuration. Based on the provided key the get_config method will return the appropriate model configuration to use. Each configuration is a dataclass containing: the needed trt flags a from_model factory method to feed all needed parameters to the config class an get_input_profile method that return the max and mix input supported by the build engine Engine classes are changed to: use trt-config class instead of mixin classes
|
merge 8 description This PR is in charge of removing onnx-exporter class as it is not needed. |
|
merge 9 description This PR is in charge of adding a trt-config class. As the same model might need different trt-configurations depending on which precision is used, a registry is used to collect all the model configuration. @timudk here are the main changes
Engine classes are changed to:
|
Trt refactor trt manager
|
merge 10 description This PR implements the changes on trt-manager to use trt-config classes instead of exporters and mixin. for each porvided model a trt-config is provided by _get_trt_configs method. @timudk this PR greatly simplify this class by reducing code lines by half. |
|
merge 11 description This PR change the CLI script to support the new TRT interface.
a different set of inputs are used:
The following are additional inputs args that can be provided:
Note that:
|
This is the main PR used to refactor TRT inference code to: