Allow agent to return logprobs

[ratulsaha2025]

Enable agents to return the logprobs for their response. This could be used downstream to determine confidence of the agent's response and use the response appropriately.

Copied from
openai/openai-agents-python#715

[yusuf-eren]

hey @ratulsaha2025 ,

It could be done but currently it's not supported in openai-node package yet for Responses API . And this repository also works on the openai npm package. related code

I created a PR for it in the original repo : openai/openai-node#1573

However, if you don't want to wait, you can pass it via modelSettings.providerData in Agent

import { Agent, run } from '@openai/agents';

const agent = new Agent({
  name: 'My Agent',
  instructions: 'You are a helpful assistant.',
  modelSettings: {
    providerData: {
      top_logprobs: 2,
      include: ['message.output_text.logprobs'],
    },
  },
});

async function main() {
  const result = await run(agent, 'What is the capital of France?');
  const content = (result.output[0] as any).content;
  console.log(JSON.stringify(content, null, 2));
  //   [
  //     {
  //       "type": "output_text",
  //       "text": "The capital of France is Paris.",
  //       "annotations": [],
  //       "logprobs": [
  //         {
  //           "bytes": [
  //             84,
  //             104,
  //             101
  //           ],
  //           "logprob": 0,
  //           "token": "The",
  //           "top_logprobs": [
  //             {
  //               "bytes": [
  //                 84,
  //                 104,
  //                 101
  //               ],
  //               "logprob": 0,
  //               "token": "The"
  //             },
  //             {
  //               "bytes": [
  //                 80,
  //                 97,
  //                 114,
  //                 105,
  //                 115
  //               ],
  //               "logprob": -20.25,
  //               "token": "Paris"
  //             }
  //           ]
  //         },
  //         ...
  //       ]
  //     }
  //   ]
}

main();