Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add fp16 inference support (trt) #875

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

OrangeSodahub
Copy link
Contributor

@OrangeSodahub OrangeSodahub commented Dec 8, 2022

Add float16 inference support for tensorrt.

@OrangeSodahub OrangeSodahub deleted the add-fp16-inference-support-trt branch December 8, 2022 08:41
@OrangeSodahub OrangeSodahub restored the add-fp16-inference-support-trt branch December 8, 2022 08:42
@OrangeSodahub OrangeSodahub reopened this Dec 8, 2022
@OrangeSodahub OrangeSodahub force-pushed the add-fp16-inference-support-trt branch from af15a6b to 1fe3a5a Compare December 8, 2022 08:43
@OrangeSodahub OrangeSodahub reopened this Dec 8, 2022
@codecov
Copy link

codecov bot commented Dec 8, 2022

Codecov Report

Merging #875 (32a1dd8) into main (0b293ec) will decrease coverage by 14.22%.
The diff coverage is 0.00%.

@@             Coverage Diff             @@
##             main     #875       +/-   ##
===========================================
- Coverage   83.06%   68.83%   -14.23%     
===========================================
  Files          22       22               
  Lines        1529     1534        +5     
===========================================
- Hits         1270     1056      -214     
- Misses        259      478      +219     
Flag Coverage Δ
cas 68.83% <0.00%> (-14.23%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/clip_server/executors/clip_tensorrt.py 0.00% <0.00%> (-94.60%) ⬇️
server/clip_server/model/clip_trt.py 0.00% <0.00%> (-85.72%) ⬇️
server/clip_server/model/trt_utils.py 0.00% <0.00%> (-83.52%) ⬇️
server/clip_server/model/clip.py 68.75% <0.00%> (-18.75%) ⬇️
server/clip_server/model/pretrained_models.py 84.12% <0.00%> (-14.29%) ⬇️
client/clip_client/client.py 79.87% <0.00%> (-7.51%) ⬇️
server/clip_server/model/clip_onnx.py 82.53% <0.00%> (+17.46%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@ZiniuYu
Copy link
Member

ZiniuYu commented Dec 15, 2022

Please fix open_clip version to v2.7.0 for now. I will take a look a the latset 2.8

@github-actions github-actions bot added size/s and removed size/m labels Dec 15, 2022
@@ -113,6 +113,7 @@ jobs:
pip install --no-cache-dir "server/[onnx]"
pip install --no-cache-dir "server/[transformers]"
pip install --no-cache-dir "server/[search]"
pip install open-clip-torch==2.7.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please specify the open-clip-torch and tensorrt version in setup

@github-actions github-actions bot added size/m and removed size/s labels Dec 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants