feat: Complete dangling tool requests to avoid API errors

[gadenbuie]

Fixes #459 by completing incomplete tool requests with an empty tool result.

The implementation focuses on dangling tool requests only. In other words, when calling $chat() or $stream() or related async function via Chat, we check if the last message is an assistant message with unanswered tool requests and we add a new user turn with the empty tool results before processing the new input.

Here's a simple example with a tool to get a random number:

pkgload::load_all()
#> ℹ Loading ellmer

random_number_tool <- tool(
  function(n) sample(1:100, n),
  name = "pick_random_number",
  description = "Pick a random number between 1 and 100.",
  arguments = list(
    n = type_number("Number of random numbers to pick.")
  )
)

chat <- chat_openai(model = "gpt-4.1-nano")
# chat <- chat_anthropic(model = "claude-haiku-4-5-20251001")
chat$register_tool(random_number_tool)

chat$chat("Pick a single random number for me.")
#> ◯ [tool call] pick_random_number(n = 1L)
#> ● #> 95
#> I picked the number 95 for you.

If we pretend the chat was truncated, i.e. the user interrupted the action before we could send the tool result back to the LLM, the turns would look something like this:

chat$set_turns(chat$get_turns()[1:2]) # Keep user input and tool request
chat$get_turns()
#> [[1]]
#> <Turn: user>
#> Pick a single random number for me.
#> 
#> [[2]]
#> <Turn: assistant>
#> [tool request (call_EdrkWPAmiN5KbaGSghAUpuwo)]: pick_random_number(n = 1L)

Currently, trying to pick up the conversation at this point would result in an API error because the tool request wasn't completed. With this PR, however, the chat can be continued normally.

chat$chat("Try again")
#> ◯ [tool call] pick_random_number(n = 1L)
#> ● #> 52
#> The random number I picked for you is 52.

Inspecting the chat shows the new user turn with the empty tool request telling the LLM that the chat was interrupted and the tool wasn't invoked.

chat
#> <Chat OpenAI/gpt-4.1-nano turns=7 input=308 output=42 cost=$0.00>
#> ── user ────────────────────────────────────────────────────────────────────────
#> Pick a single random number for me.
#> ── assistant [input=65 output=15 cost=$0.00] ───────────────────────────────────
#> [tool request (call_EdrkWPAmiN5KbaGSghAUpuwo)]: pick_random_number(n = 1L)
#> ── user ────────────────────────────────────────────────────────────────────────
#> [tool result  (call_EdrkWPAmiN5KbaGSghAUpuwo)]: Error: Chat ended before the tool could be invoked.
#> ── user ────────────────────────────────────────────────────────────────────────
#> Try again
#> ── assistant [input=109 output=15 cost=$0.00] ──────────────────────────────────
#> [tool request (call_9qXCmh4p8QMwLi0TzXaeRrHe)]: pick_random_number(n = 1L)
#> ── user ────────────────────────────────────────────────────────────────────────
#> [tool result  (call_9qXCmh4p8QMwLi0TzXaeRrHe)]: 52
#> ── assistant [input=134 output=12 cost=$0.00] ──────────────────────────────────
#> The random number I picked for you is 52.

Note that we insert a separate user message primarily because of the bug discussed in #735: inserting tool results into the new user message hits the incorrect behavior described in the linked comment and doesn't work with OpenAI (and a few other providers). I also think conceptually it's better to have this stored as separate user messages, but it also wouldn't be hard to inject to tool results into the new actual user message after #735.

TODO

• Add tests

[gadenbuie]

@hadley I requested a review to see if you have initial thoughts and reactions. We could tackle this separately or in combination with #735, and I still need to add tests.

[hadley]

This approach of providing a generic "tool request failed" result seems perfect to me!

[gadenbuie]

I'm in favor of relaxing the user-assistant paired turns constraint, but I also understand that you probably don't want to do that in this release and that it's better to stay internally consistent.

I think this approach is clean and simple except for the faked assistant turn. In my initial implementation, which I'd recommend over adding an assistant turn, was to attach the tool results to the incoming user message.

That approach introduces a dependency on #735 (in the sense that it fixes a bug with tool results and text content being sent out of order), but that PR is almost done. I'll add a commit that takes this PR in that direction so you can see how that feels (and we can revert it if you prefer this approach).

[gadenbuie]

Here's the commit 4037506. I consolidated the tests into one that uses vcr and updated that cassette in a follow up: 70313b1

[gadenbuie]

Small note here, but now that tokens are structured, could this code use names instead of indices? Also, very minor nit, but I think res would be better than out; on my first read I thought out was for output tokens.

[hadley]

Tokens aren't structured in Turn objects yet 😬

[hadley]

This is much better!