Replies: 1 comment
-
See #15434 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi,
I am trying to get all the hidden states from the forward pass of a llama model using vllm. Sort of like what is possible with hugging face's
out_hidden_states = True
but with all the optimizations that film does to speed up inference that one doesn't get using vanilla hf.Is this even possible?
I have tried setting task = embedding in the LLM class, but that only gives the last layer's last embedding. I would want to have access to the outputs from each layer.
Beta Was this translation helpful? Give feedback.
All reactions