Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dead kernel #74

Open
benzom opened this issue Aug 17, 2023 · 0 comments
Open

Dead kernel #74

benzom opened this issue Aug 17, 2023 · 0 comments

Comments

@benzom
Copy link

benzom commented Aug 17, 2023

I follow the example from README except for the part where model is being quantized

from fastT5 import (OnnxT5, get_onnx_runtime_sessions,
                    generate_onnx_representation, quantize)
from transformers import AutoTokenizer

model_or_model_path = 't5-small'

# Step 1. convert huggingfaces t5 model to onnx
onnx_model_paths = generate_onnx_representation(model_or_model_path)
model_sessions = get_onnx_runtime_sessions(onnx_model_paths)

however get_onnx_runtime_sessions is killing the kernel, what's wrong? same for

from fastT5 import export_and_get_onnx_model
from transformers import AutoTokenizer

model_name = 't5-small'
model = export_and_get_onnx_model(model_name, quantized=False)

I'm using
fastt5==0.1.4
torch==1.13.1
onnx==1.8.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant