[Feature Request]: TensorRT support.

[bropines]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Is it possible to add tensorRT like here: https://github.com/ddPn08/Lsmith ?
If so, why not add it, since this method speeds up the generation on 20/30/40 series of video cards by 2 or more times

Proposed workflow

Add support TensorRT

Additional information

No response

[anapnoe]

Yes this is really cool I need to find some more time to look into thanks

[midcoastal]

I imagine that this would/may come around the same time any Vlad merging? Looking forward to this. I bought a 4070 TI, and, while it is doing better than my 1080 TI was, for sure, I know I am not getting the best out of it. Considering doing some compiling myself... But we know how that goes... ;-)

[rushuna86]

tensorRT feature right now is very very limited generation or of use TBH. Automatic has an extension now that supports this feature, but the conversion process has a limit on "size" during conversion. And the TensorRT Unet file is restricted once again for generation, with the image size/batch that was "converted", lora doesn't work on it, and neither does hires. So unless you're generating as is images it's not very useful yet. Wait for nvidia release of the implementation.I went from 47-49its/sec up to 87its on a batch 4, performance isn't really 2x as hyped about.