What does "Send Tool Output in User Message" mean?

[siben168]

There is a toggle in smart threads setting as following, run several tests and i didn't see any differences in answers, what does it use for?

"Send Tool Output in User Message"

[brianpetro]

On the backend, when sending data to the chat model, there are a few different "roles" for labelling a message.

For a typical OpenAI model request, the role for sending tool output is "tool". While other models try to emulate this, they often fail to perform well with this setup. So that same tool output is sent in the "user" message when that tool is toggled on.

I expect it's impact to decrease as the models continue to improve and standardize tool usage. In the meantime, the impact is model-dependent and in best-case will improve the chat output by better utilizing context provided by tools 🎄

[siben168]

Thank you for the detailed explanation. It makes a lot of sense!🎄