OpenAI Generator Assertion Error due to empty chunks

[alexeyKivelInit]

Describe the bug
Streaming with (Azure)OpenAI Generator gives Assertion Error, because first (and last) received chunk have empty choices. However, this is to be expected according to openai/openai-python#1266 . The empty chunks have to be diregarded and method _convert_streaming_chunks_to_chat_message needs to get a list without disregarded chunks and a valid chunk.

Following monkey patch solved the issue for me:

def _handle_stream_response_patched(self, chat_completion: Stream, callback: StreamingCallbackT) -> List[ChatMessage]:
    chunks: List[StreamingChunk] = []
    valid_chunk = None

    for chunk in chat_completion:  # pylint: disable=not-an-iterable
        #assert len(chunk.choices) == 1, "Streaming responses should have only one choice."
        if chunk.choices:
            valid_chunk = chunk
            chunk_delta: StreamingChunk = self._convert_chat_completion_chunk_to_streaming_chunk(chunk)
            chunks.append(chunk_delta)

            callback(chunk_delta)

    return [self._convert_streaming_chunks_to_chat_message(valid_chunk, chunks)]

Error message
AssertionError: Streaming responses should have only one choice.

Expected behavior
Streaming from (Azure) OpenAI model.

Additional context
Add any other context about the problem here, like document types / preprocessing steps / settings of reader etc.

To Reproduce
Run a Generator from AzureOpenAI and stream.

FAQ Check

• Have you had a look at our new FAQ page?

System:

• OS:
• GPU/CPU:
• Haystack version (commit or version number): 2.9.0
• DocumentStore:
• Reader:
• Retriever:

[anakin87]

Hello!

Please clarify a bit the issue:

• given the piece of code you reported, I think you are referring to AzureOpenAIChatGenerator (not AzureOpenAIGenerator). Right?
• could you provide a reproducible example that triggers the error?

What I tried

from haystack.components.generators.chat import AzureOpenAIChatGenerator

from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage

generator = AzureOpenAIChatGenerator(streaming_callback=print_streaming_chunk,
                                     azure_deployment="gpt-35-turbo",
                                     azure_endpoint="...")

msg = ChatMessage.from_user("Summarize what is NLP in 3 paragraphs")

result=generator.run(messages=[msg])

print(result)

This works properly: no errors, streaming works, and the final response is correct.