How do I propagate intermediate results in an integration model?

[callmezhangchenchenokay]

Because of this problem
triton-inference-server/tensorrtllm_backend#71

How do I propagate intermediate results in an integration model?

I want to get the output of the first model

I tried to use the output of the first model directly as the input in the third model, but it didn't work

[oandreeva-nv]

Did the solution provided in triton-inference-server/tensorrtllm_backend#71 worked for your case?

[callmezhangchenchenokay]

Thanks for your reply!
At the bottom of this solution is my comment that the problem has been solved

[callmezhangchenchenokay]

Sorry to interrupt again!

The solution mentioned above requires model_transaction_policy to be set to True,
Can only be used if stream = False,

However, this problem occurs when stream = True

So there needs to be a way to export REQUEST_INPUT_LEN from stream =True, model_transaction_policy =True