Fix index field not populated in streaming mode with n>1 and tool calls

[Sameerlite]

Title

Fix index field not populated in streaming mode with n>1 and tool calls

Relevant issues

Fixes #8977
Fixes LIT-1340

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

Problem

When using OpenAI's streaming mode with n>1 and tool calls, the index field in choice objects was not being populated correctly. LiteLLM always returned index=0 for all choices, while OpenAI correctly returns different indices (0, 1, 2, etc.) to distinguish between the different completions requested via the n parameter.

Root Cause

The issue was in the is_chunk_non_empty() method in streaming_handler.py. When processing tool call chunks:

• The completion_obj["content"] is empty (since tool calls don't have text content)
• Tool calls exist only in model_response.choices[0].delta.tool_calls
• The method wasn't checking for tool_calls or function_call in the delta object
• This caused is_chunk_non_empty() to return False, skipping the code path that properly processes and preserves the index from the original OpenAI chunk
• As a result, the default index=0 was always used

Solution

Extended the is_chunk_non_empty() method to check for tool_calls and function_call in the delta object:

or (
    "tool_calls" in model_response.choices[0].delta
    and model_response.choices[0].delta["tool_calls"] is not None
)
or (
    "function_call" in model_response.choices[0].delta
    and model_response.choices[0].delta["function_call"] is not None
)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
litellm	Ready	Preview	Comment	Nov 1, 2025 8:34pm

[ishaan-jaff]

[ishaan-jaff]

move both tests to test_openai.py in llm_translation directory

[Sameerlite]

moved