Feedback for “Log Levels”

[subbiah-cape]

I was working on extracting the article title and abstract from a PDF. I have already implemented the code, and it is functioning correctly. I used GPT-4.0 from Azure. However, when I integrated this code into Langfuse, the model cost, input, and output token counts were not being tracked.

For the API request, I used a request URL and implemented parallel processing using OpenAI's API directly. Additionally, I imported AzureAI from the OpenAI Python package.

My question is: If I integrate my code into Langfuse without using Langfuse modules (i.e., from langfuse.openai import azureai), will it still track tokens, model costs, and performance? I also used @decorator(), but tracking did not work when executing parallel processing through the request URL.

Is it mandatory to use Langfuse modules and adjust the implementation according to Langfuse documentation for tracking features to work?

[marcklingen]

For auto-instrumentation to work, you need to use the import from langfuse.openai. Have you tried this?

[subbiah-cape]

When using langfuse.openai, everything is tracked correctly. However, if I import from langfuse.openai but make requests using the request URL, tracking does not work.

from langfuse.openai import AzureOpenAI

client = AzureOpenAI(  
    azure_endpoint=GPT4o_BASE_URL,
    azure_deployment=GPT4o_ENGINE_Name,
    api_key=GPT4o_KEY,
    api_version=GPT4o_API_VERSION
)

base_url = str(client._base_url).rstrip('/')
GPT4o_Request_URL = f"{base_url}/chat/completions?api-version={GPT4o_API_VERSION}"

Is there a separate module or function in Langfuse that enables auto-instrumentation when making requests using GPT4o_Request_URL?

[marcklingen]

That's expected because then you are not making the request via the instrumented sdk. Why can't you use the SDK?