Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecation wrapper poor performance (latency) #26918

Open
5 tasks done
MartinGotelli opened this issue Sep 26, 2024 · 0 comments
Open
5 tasks done

Deprecation wrapper poor performance (latency) #26918

MartinGotelli opened this issue Sep 26, 2024 · 0 comments
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature

Comments

@MartinGotelli
Copy link
Contributor

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangChain documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

pip install snakeviz

script.py

from pydantic import BaseModel
import cProfile
from langchain_openai import ChatOpenAI

class Model(BaseModel):
    value: str

profiler = cProfile.Profile()
profiler.enabled()
model = ChatOpenAI(model="gpt-4o", api_key="key")
model.with_structured_output(Model)
profiler.disable()
profile.dump_stats("make_martin_happy.prof")

python script.py
snakeviz make_martin_happy.prof

Error Message and Stack Trace (if applicable)

No response

Description

We are trying to build a performant model that calls gpt-4o-mini, and the main issue is latency. While trying to improve latency, I discovered that the check of the module from the inspect library can take around 0.25s. It's not a lot, but it adds to the total.

I don't want you to "fix" inspect.get_module. But I would like to have an alternative to disable LangChain warnings during the execution of my code.

image

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 23.6.0: Wed Jul 31 20:49:39 PDT 2024; root:xnu-10063.141.1.700.5~1/RELEASE_ARM64_T6000
Python Version: 3.11.4 (main, Nov 30 2023, 11:49:37) [Clang 15.0.0 (clang-1500.0.40.1)]

Package Information

langchain_core: 0.3.2
langchain: 0.3.0
langchain_community: 0.3.0
langsmith: 0.1.127
langchain_aws: 0.2.1
langchain_openai: 0.2.0
langchain_postgres: 0.0.12
langchain_text_splitters: 0.3.0
langgraph: 0.2.27

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.10.5
async-timeout: Installed. No version info available.
boto3: 1.34.162
dataclasses-json: 0.6.7
httpx: 0.27.2
jsonpatch: 1.33
langgraph-checkpoint: 1.0.9
numpy: 1.26.4
openai: 1.45.0
orjson: 3.10.7
packaging: 24.1
pgvector: 0.2.5
psycopg: 3.2.1
psycopg-pool: 3.2.2
pydantic: 2.9.1
pydantic-settings: 2.5.2
PyYAML: 6.0.2
requests: 2.32.3
SQLAlchemy: 2.0.34
sqlalchemy: 2.0.34
tenacity: 8.5.0
tiktoken: 0.7.0
typing-extensions: 4.12.2

@dosubot dosubot bot added the 🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature label Sep 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature
Projects
None yet
Development

No branches or pull requests

1 participant