Update openai sdk version

[artur-krawczyk-epam]

It seems that openai sdk is not up to date in ai-dial-adapter-openai https://github.com/epam/ai-dial-adapter-openai .
Official example from microsoft https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/function-calling#single-toolfunction-calling-example doesn't work with the current version of openai adapter.

Whereas calling Azure OpenAI service directly example works with the example from MSFT documentation.

In our opinion the problem is related with
https://platform.openai.com/docs/api-reference/chat/create
and tool_choice parameter

Error:
openai.BadRequestError: Error code: 400 - {'error': {'message': 'Your request contained invalid structure on path messages.1.refusal. extra fields not permitted', 'type': 'invalid_request_error', 'code': '400'}}

[adubovik]

Hi @artur-krawczyk-epam, what is the model you are requesting that returns 400 error response? Is it Azure OpenAI model?

[artur-krawczyk]

Hi @adubovik yes, it's gpt-4o

[adubovik]

@artur-krawczyk @artur-krawczyk-epam I'm struggling to reproduce the error. Could you please provide the exact request that leads to the error? It could be a Python program or a curl request.

[artur-krawczyk-epam]

@adubovik sure, this is Python code that breaks:

import os
import json
from openai import AzureOpenAI
from datetime import datetime
from zoneinfo import ZoneInfo
import requests

client = AzureOpenAI(
    azure_endpoint="<endpoint_url>",
    azure_ad_token="<access_token>",
    api_version="2024-05-01-preview"
)

# Define the deployment you want to use for your chat completions API calls

deployment_name = "gpt-4o"

# Simplified timezone data
TIMEZONE_DATA = {
    "tokyo": "Asia/Tokyo",
    "san francisco": "America/Los_Angeles",
    "paris": "Europe/Paris"
}


def get_current_time(location):
    """Get the current time for a given location"""
    print(f"get_current_time called with location: {location}")
    location_lower = location.lower()

    for key, timezone in TIMEZONE_DATA.items():
        if key in location_lower:
            print(f"Timezone found for {key}")
            current_time = datetime.now(ZoneInfo(timezone)).strftime("%I:%M %p")
            return json.dumps({
                "location": location,
                "current_time": current_time
            })

    print(f"No timezone data found for {location_lower}")
    return json.dumps({"location": location, "current_time": "unknown"})


def run_conversation():
    # Initial user message
    messages = [{"role": "user", "content": "What's the current time in San Francisco"}]  # Single function call

    # Define the function for the model
    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_current_time",
                "description": "Get the current time in a given location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city name, e.g. San Francisco",
                        },
                    },
                    "required": ["location"],
                },
            }
        }
    ]

    # First API call: Ask the model to use the function
    response = client.chat.completions.create(
        model=deployment_name,
        messages=messages,
        tools=tools,
        tool_choice="required",
    )

    # Process the model's response
    response_message = response.choices[0].message
    messages.append(response_message)

    print("Model's response:")
    print(response_message)

    # Handle function calls
    if response_message.tool_calls:
        for tool_call in response_message.tool_calls:
            if tool_call.function.name == "get_current_time":
                function_args = json.loads(tool_call.function.arguments)
                print(f"Function arguments: {function_args}")
                time_response = get_current_time(
                    location=function_args.get("location")
                )
                messages.append({
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": "get_current_time",
                    "content": time_response,
                })
    else:
        print("No tool calls were made by the model.")

        # Second API call: Get the final response from the model
    final_response = client.chat.completions.create(
        model=deployment_name,
        messages=messages,
    )

    return final_response.choices[0].message.content


# Run the conversation and print the result
print(run_conversation())

I've masked the endpoint_url and access_token parameters, as they're project specific & irrelevant here. This code works when tool_choice="auto", but breaks when tool_choice="required".
According to docs here https://platform.openai.com/docs/api-reference/chat/create the latest sdk supports both auto and required.
Hope that helps pinpointing the error

[adubovik]

@artur-krawczyk-epam @artur-krawczyk

Could you please elaborate how exactly this code breaks?
What is the error message along with the stack trace?

I got the following error message when I ran the provided code on our prod env:

openai.BadRequestError: Error code: 400 - {'error': {'code': 'BadRequest', 'message': 'tool_choice value as required is enabled only for api versions 2024-06-01 and later'}}

and after I fixed the api version to 2024-06-01 I get the correct response:

Model's response: ChatCompletionMessage(...)
Function arguments: {'location': 'San Francisco'}
get_current_time called with location: San Francisco
Timezone found for san francisco
The current time in San Francisco is 2:56 AM.

So it appear to be a configuration issue. Could you please provide more details about your environment:

1. the DIAL version
2. how "gpt-4o" is defined in the DIAL Core configuration
3. ai-dial-adapter-openai version
4. the full error message along with the stack trace from the ai-dial-adapter-openai logs
5. the value of GPT4O_DEPLOYMENTS env variable set in the ai-dial-adapter-openai container

[artur-krawczyk-epam]

@adubovik my error is a little different than what you got initially, it is:

Error:
openai.BadRequestError: Error code: 400 - {'error': {'message': 'Your request contained invalid structure on path messages.1.refusal. extra fields not permitted', 'type': 'invalid_request_error', 'code': '400'}}

Changing the API version did not help.
Answering your specific questions:

1. the Dial version is docker.io/epam/ai-dial-core:0.21.1
2. I'm not sure what you're asking here
3. docker.io/epam/ai-dial-adapter-openai:0.19.0
4. skipping for now, maybe the rest will help
5. GPT4O_DEPLOYMENTS: gpt-4o