bug: openai.py: Received openai error - 'NoneType' object has no attribute '__dict__'

[GildeshAbhay]

Describe the bug

So i have been trying to use two things in my code which uses litellm, one is prompt management and the other is tracing. So i don't get the prompts stored in my langfuse UI and moreover, I get the following error

Manual fields: ['company_name', 'notes', 'url']
Updating an existing record for the company_name - Freenome in the company table.
Record Id: 2
LLM fields: ['ceo']
Starting smart_search...

Request to litellm:
litellm.completion(model='gpt-4o', messages=[{'role': 'user', 'content': "\n        Given the company name 'Freenome', the field 'ceo' with description 'Name of the company's Chief Executive Officer.', generate 3 effective Google search queries that would help find the most recent and relevant information. \n        Include boolean operators (AND, OR, NOT) in these queries to refine the search results.\n        The current month and year is October 2024. Depending on the field, it may be necessary to specifically search for the latest information using date-related terms or the current year.\n        Focus on 2024 and 2023 for recent data.\n         Provide the queries as a comma-separated list, without any additional text."}])

20:48:49 - LiteLLM:WARNING: utils.py:329 - `litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'extra_body': {}}

POST Request Sent from LiteLLM:
curl -X POST \
https://api.openai.com/v1/ \
-d '{'model': 'gpt-4o', 'messages': [{'role': 'user', 'content': "\n        Given the company name 'Freenome', the field 'ceo' with description 'Name of the company's Chief Executive Officer.', generate 3 effective Google search queries that would help find the most recent and relevant information. \n        
Include boolean operators (AND, OR, NOT) in these queries to refine the search results.\n        The current month and year is October 2024. Depending on the field, it may be necessary to specifically search for the latest information using date-related terms or the current year.\n        Focus on 2024 and 2023 for recent data.\n         Provide the queries as a comma-separated list, without any additional text."}], 'extra_body': {}}'

'NoneType' object has no attribute '__dict__'
openai.py: Received openai error - 'NoneType' object has no attribute '__dict__'
RAW RESPONSE:
'NoneType' object has no attribute '__dict__'
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.
Langfuse Logging - Enters logging function for model {'model': 'gpt-4o', 'messages': [{'role': 'user', 'content': "\n        Given the company name 'Freenome', the field 'ceo' with description 'Name of the company's Chief Executive Officer.', generate 3 effective Google search queries that would help find the most recent and relevant information. \n        Include boolean operators (AND, OR, NOT) in these queries to refine the search results.\n        The current month and year is October 2024. Depending on the field, it may be necessary to specifically search for the latest information using date-related terms or the current year.\n        Focus on 2024 and 2023 for recent data.\n         Provide the queries as a comma-separated list, without any additional text."}], 'optional_params': {'extra_body': {}}, 'litellm_params': {'acompletion': False, 'api_key': None, 'force_timeout': 600, 'logger_fn': None, 'verbose': False, 'custom_llm_provider': 'openai', 'api_base': 'https://api.openai.com/v1/', 'litellm_call_id': '1dd73cf1-7573-47b9-a21f-ffff3b1a62c7', 'model_alias_map': {}, 'completion_call_id': None, 'metadata': None, 'model_info': None, 'proxy_server_request': None, 'preset_cache_key': None, 'no-log': False, 'stream_response': {}, 'input_cost_per_token': None, 'input_cost_per_second': None, 'output_cost_per_token': None, 'output_cost_per_second': None, 'cooldown_time': 
None, 'text_completion': None, 'azure_ad_token_provider': None, 'user_continue_message': None, 'base_model': None}, 'start_time': datetime.datetime(2024, 10, 15, 20, 48, 49, 714892), 'stream': False, 'user': None, 'call_type': 'completion', 'litellm_call_id': '1dd73cf1-7573-47b9-a21f-ffff3b1a62c7', 'completion_start_time': None, 'standard_callback_dynamic_params': {}, 'extra_body': {}, 'custom_llm_provider': 'openai', 'input': [{'role': 'user', 'content': "\n  
      Given the company name 'Freenome', the field 'ceo' with description 'Name of the company's Chief Executive Officer.', generate 3 effective Google search queries that would help find the most recent and relevant information. \n        Include boolean operators (AND, OR, NOT) in these queries to refine the search results.\n        The current month and year is October 2024. Depending on the field, it may be necessary to specifically search for the latest information using date-related terms or the current year.\n        Focus on 2024 and 2023 for recent data.\n         Provide the queries as a comma-separated list, without any additional text."}], 'api_key': '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', 'additional_args': {'headers': None}, 'log_event_type': 'failed_api_call', 'api_call_start_time': datetime.datetime(2024, 10, 15, 20, 48, 51, 795964), 'original_response': "'NoneType' object has no attribute '__dict__'", 'exception': litellm.APIError: APIError: OpenAIException - 'NoneType' object has no attribute '__dict__', 'traceback_exception': 'Traceback (most recent call last):\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\litellm\\llms\\OpenAI\\openai.py", line 810, in completion\n    raise 
e\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\litellm\\llms\\OpenAI\\openai.py", line 746, in completion\n    self.make_sync_openai_chat_completion_request(\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\litellm\\llms\\OpenAI\\openai.py", line 605, in make_sync_openai_chat_completion_request\n    raise e\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\litellm\\llms\\OpenAI\\openai.py", line 594, in make_sync_openai_chat_completion_request\n    raw_response = openai_client.chat.completions.with_raw_response.create(\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\openai\\_legacy_response.py", line 356, in wrapped\n    return cast(LegacyAPIResponse[R], func(*args, **kwargs))\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\langfuse\\openai.py", line 179, in wrapper\n    return func(open_ai_definitions, initialize, wrapped, args, kwargs)\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\langfuse\\openai.py", line 582, in _wrap\n    raise ex\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\langfuse\\openai.py", line 559, in _wrap\n    model, completion, usage = _get_langfuse_data_from_default_response(\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\langfuse\\openai.py", line 519, in _get_langfuse_data_from_default_response\n    return model, completion, usage.__dict__ if 
_is_openai_v1() else usage\nAttributeError: \'NoneType\' object has no attribute \'__dict__\'. Did you mean: \'__dir__\'?\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\litellm\\main.py", line 1527, in completion\n    raise e\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\litellm\\main.py", line 1500, in completion\n    response = openai_chat_completions.completion(\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\litellm\\llms\\OpenAI\\openai.py", line 820, in completion\n    raise OpenAIError(\nlitellm.llms.OpenAI.openai.OpenAIError: \'NoneType\' object has no attribute \'__dict__\'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\litellm\\utils.py", line 959, in wrapper\n    result = original_function(*args, **kwargs)\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\litellm\\main.py", line 2957, in completion\n    raise exception_type(\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\litellm\\litellm_core_utils\\exception_mapping_utils.py", line 2131, in exception_type\n    raise e\n  File "C:\\Users\\abhay.saini\\AppData\\Local\\miniconda\\envs\\abhay_darwin\\lib\\site-packages\\litellm\\litellm_core_utils\\exception_mapping_utils.py", line 419, in exception_type\n    raise APIError(\nlitellm.exceptions.APIError: litellm.APIError: APIError: OpenAIException - \'NoneType\' object has no attribute \'__dict__\'\n', 'end_time': datetime.datetime(2024, 10, 15, 20, 48, 55, 282669), 'response_cost': 0, 'standard_logging_object': {'id': '1dd73cf1-7573-47b9-a21f-ffff3b1a62c7', 'call_type': 'completion', 'cache_hit': False, 'status': 'failure', 'saved_cache_cost': 0.0, 'startTime': 1729005529.714892, 'endTime': 1729005535.282669, 'completionStartTime': 1729005535.282669, 'model': 'gpt-4o', 'metadata': {}, 'cache_key': None, 'response_cost': 0.0, 'total_tokens': 0, 'prompt_tokens': 0, 'completion_tokens': 0, 'request_tags': [], 'end_user': '', 'api_base': 'https://api.openai.com/v1/', 'model_group': '', 'model_id': '', 'requester_ip_address': None, 'messages': [{'role': 'user', 'content': "\n        Given the company name 'Freenome', the field 'ceo' with description 'Name of the company's Chief Executive Officer.', generate 3 effective Google search queries that would help find the most recent and relevant information. \n        Include boolean operators (AND, OR, NOT) in these queries to refine the search results.\n        The current month and year is October 2024. Depending on the field, it may be necessary to specifically search for the latest information using date-related terms or the current year.\n        Focus on 2024 and 2023 for recent data.\n         Provide the queries as a comma-separated list, without any additional text."}], 'response': {}, 'model_parameters': {'extra_body': {}}, 'hidden_params': {'model_id': None, 'cache_key': None, 'api_base': None, 'response_cost': None, 'additional_headers': None}, 'model_map_information': {'model_map_key': '', 'model_map_value': None}, 'error_str': "litellm.APIError: APIError: OpenAIException - 'NoneType' object has no attribute '__dict__'", 'response_cost_failure_debug_info': None}}
OUTPUT IN LANGFUSE: litellm.APIError: APIError: OpenAIException - 'NoneType' object has no attribute '__dict__'; original: None
Langfuse Layer Logging - logging to langfuse v2
trace: 0
Langfuse Layer Logging - final response object: None
Failed to update field after 0 attempts: Failed to update field with smart search: litellm.APIError: APIError: OpenAIException - 'NoneType' object has no attribute '__dict__'
processing: product
processing: indication

However, have been unsuccessful so far.

To reproduce
Run this code snippet

Below is the code snippet

from app.database import *
import os, asyncio, aiohttp, litellm, warnings, json
from litellm import completion
from typing import List, Dict, Any, Optional
from datetime import datetime, timezone, timedelta
from dotenv import load_dotenv
from langfuse import Langfuse
from langfuse.openai import OpenAI
from langfuse.decorators import langfuse_context, observe
os.environ["LANGFUSE_PUBLIC_KEY"] = os.getenv("LANGFUSE_PUBLIC_KEY")
os.environ["LANGFUSE_SECRET_KEY"] = os.getenv("LANGFUSE_SECRET_KEY")
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
os.environ["LANGFUSE_HOST"]="https://cloud.langfuse.com" # EU region
# set callbacks
litellm.success_callback = ["langfuse"]
litellm.failure_callback = ["langfuse"]
litellm.set_verbose=True
langfuse = Langfuse()
#langfuse.auth_check()
warnings.filterwarnings("ignore", category=UserWarning, module="pydantic.main")
#PROXY_URL="http://0.0.0.0:4000"
#client = OpenAI(base_url=PROXY_URL)
litellm.drop_params = True

@observe
async def _extract_information(self, content: str, parent_entity: Dict[str, str], child_entities: Optional[List[Dict[str, str]]], field: Dict[str, str]) -> str:
        prompt = f"""
        Based on the following content, extract information relevant to '{field['description']}' for the {parent_entity.get("name")} '{parent_entity.get("value")}'.
        """
        if child_entities:
            for child_entity in child_entities:
                prompt += f" Include information specific to the {child_entity.get('name')} '{child_entity.get('value')}'."
        
        prompt += f" Provide a concise paragraph that could be used to update the '{field['name']}' field.\n\nContent:\n{content}"
        langfuse.create_prompt(name="extract_information_darwin", prompt=prompt,labels=["production"])
        fuse_prompt = langfuse.get_prompt("extract_information_darwin")
        #fuse_prompt.compile(json_schema="TEST SCHEMA")
        #response = client.completions.create(
        #    model=self.MODEL,
        #    messages=[{"role": "user", "content":prompt}],
        #    langfuse_prompt = fuse_prompt
        #)
        response1 = completion(
            model=self.MODEL, 
            messages=[{"role": "user", "content": prompt}],
            langfuse_prompt= fuse_prompt#
            #metadata={"trace_id": langfuse_context.get_current_trace_id()}
            )

        #extraction = response['choices'][0]['message']['content'].strip()
        extraction1 = response1['choices'][0]['message']['content'].strip()
        return extraction1

SDK and container versions
Name: langfuse
Version: 2.52.0
Summary: A client library for accessing langfuse
Home-page:
Author: langfuse
Author-email: [email protected]
License: MIT