Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added o3 and updated openai pricing #108

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Added o3 and updated openai pricing #108

wants to merge 3 commits into from

Conversation

williamjameshandley
Copy link
Contributor

@williamjameshandley williamjameshandley commented Feb 1, 2025 •
edited
Loading

Summary:
This PR expands our support for OpenAI models by introducing additional pricing constants for several GPT-3.5, GPT-4, and related models, while also adding detection for the "o3" model type in the assistant. These changes ensure that our pricing logic reflects the latest API offerings and model variants.

Changes:
• In gptcli/assistant.py:

  • Updated the get_completion_provider() function to also return OpenAICompletionProvider when the model name starts with "o3". This extends our support for the new o3 variant.

• In gptcli/providers/openai.py:

  • Renamed and updated pricing constants for GPT-3.5 models (e.g., renamed GPT_3_5_TURBO_PRICE_PER_TOKEN to GPT_3_5_TURBO_0125_PRICE_PER_TOKEN and added constants for instruct, 1106, 0613, 16K_0613, 0301).
  • Added several new pricing dictionaries for GPT-4 models including preview, turbo, and vision variants, as well as additional pricing for GPT-4 O models (with various release dates) and GPT-4 O mini/audio editions.
  • Added constants for O_1 and O_3 series models, and also introduced new constants for Davinci and Babbage variants.
  • Refactored the gpt_pricing() function to use explicit equality checks for model names instead of prefix matching in many cases so that pricing retrieval is more precise and maintainable.

Rationale:
• Ensures support for the latest and upcoming OpenAI models, reflecting current pricing structures.
• Improves clarity and maintainability by having explicit mappings for each model variant.
• Future-proofs our codebase against potential changes in model naming conventions and pricing updates.

Testing:
• Verified that gpt_pricing() returns the appropriate pricing dictionary for each supported model.
• Confirmed that the new "o3" prefix correctly instantiates an OpenAICompletionProvider in the assistant.

Please review the changes and let me know if any adjustments are needed.

For reference, below is the prompt I used to update the pricing.

I also then checked that these were all consistent with a large language model which did not find any issues. I checked that LLMs were able to find injected errors.

Here is the openai pricing as copied from the openai website on 2025-02-01:

GPT-4o
GPT-4o is our most advanced multimodal model that’s faster and cheaper than GPT-4 Turbo with stronger vision capabilities. The model has 128K context and an October 2023 knowledge cutoff.

Learn about GPT-4o(opens in a new window)
Model
Pricing
Pricing with Batch API*
gpt-4o
$2.50 / 1M input tokens
$1.25 / 1M input tokens
$1.25 / 1M cached** input tokens
$10.00 / 1M output tokens
$5.00 / 1M output tokens
gpt-4o-2024-11-20
$2.50 / 1M input tokens
$1.25 / 1M input tokens
$1.25 / 1M cached** input tokens
$10.00 / 1M output tokens
$5.00 / 1M output tokens
gpt-4o-2024-08-06
$2.50 / 1M input tokens
$1.25 / 1M input tokens
$1.25 / 1M cached** input tokens
$10.00 / 1M output tokens
$5.00 / 1M output tokens
gpt-4o-audio-preview
Text
$2.50 / 1M input tokens
$10.00 / 1M output tokens
Audio
$40.00 / 1M input tokens
$80.00 / 1M output tokens
gpt-4o-audio-preview-2024-12-17
Text
$2.50 / 1M input tokens
$10.00 / 1M output tokens
Audio
$40.00 / 1M input tokens
$80.00 / 1M output tokens
gpt-4o-audio-preview-2024-10-01
Text
$2.50 / 1M input tokens
$10.00 / 1M output tokens
Audio
$100.00 / 1M input tokens
$200.00 / 1M output tokens
gpt-4o-2024-05-13
$5.00 / 1M input tokens
$2.50 / 1M input tokens
$15.00 / 1M output tokens
$7.50 / 1M output tokens

GPT-4o mini
GPT-4o mini is our most cost-efficient small model that’s smarter and cheaper than GPT-3.5 Turbo, and has vision capabilities.

Learn about GPT-4o mini(opens in a new window)
Model
Pricing
Pricing with Batch API*
gpt-4o-mini
$0.150 / 1M input tokens
$0.075 / 1M input tokens
$0.075 / 1M cached** input tokens
$0.600 / 1M output tokens
$0.300 / 1M output tokens
gpt-4o-mini-2024-07-18
$0.150 / 1M input tokens
$0.075 / 1M input tokens
$0.075 / 1M cached** input tokens
$0.600 / 1M output tokens
$0.300 / 1M output tokens
gpt-4o-mini-audio-preview
Text
$0.150 / 1M input tokens
$0.600 / 1M output tokens
Audio
$10.000 / 1M input tokens
$20.000 / 1M output tokens
gpt-4o-mini-audio-preview-2024-12-17
Text
$0.150 / 1M input tokens
$0.600 / 1M output tokens
Audio
$10.000 / 1M input tokens
$20.000 / 1M output tokens

OpenAI o1
o1 is our frontier reasoning model that supports tools, Structured Outputs, and vision. The model has 200K context and an October 2023 knowledge cutoff.

Learn about o1
Model
Pricing
Pricing with Batch API***
o1
$15.00 / 1M input tokens
$7.50 / 1M input tokens
$7.50 / 1M cached* input tokens
$60.00 / 1M output** tokens
$30.00 / 1M output** tokens
o1-2024-12-17
$15.00 / 1M input tokens
$7.50 / 1M input tokens
$7.50 / 1M cached* input tokens
$60.00 / 1M output** tokens
$30.00 / 1M output** tokens
o1-preview
$15.00 / 1M input tokens
$7.50 / 1M input tokens
$7.50 / 1M cached* input tokens
$60.00 / 1M output** tokens
$30.00 / 1M output** tokens
o1-preview-2024-09-12
$15.00 / 1M input tokens
$7.50 / 1M input tokens
$7.50 / 1M cached* input tokens
$60.00 / 1M tokens
$30.00 / 1M output** tokens

OpenAI o3-mini
o3-mini is our cost-efficient reasoning model that’s optimized for coding, math, and science, and supports tools and Structured Outputs.

Learn about o3-mini >⁠(opens in a new window)

Learn more about o3(opens in a new window)
Model
Pricing
Pricing with Batch API***
o3-mini
$1.10 / 1M input tokens
$0.55 / 1M input tokens
$0.55 / 1M cached* input tokens
$4.40 / 1M output** tokens
$2.20 / 1M output** tokens
03-mini-2025-01-31
$1.10 / 1M input tokens
$0.55 / 1M input tokens
$0.55 / 1M cached* input tokens
$4.40 / 1M output** tokens
$2.20 / 1M output** tokens


o1-mini-2024-09-12
$1.10 / 1M tokens
$4.40 / 1M tokens
$0.55 / 1M cached* tokens
chatgpt-4o-latest
$5.00 / 1M tokens
$15.00 / 1M tokens
gpt-4-turbo
$10.00 / 1M tokens
$30.00 / 1M tokens
gpt-4-turbo-2024-04-09
$10.00 / 1M tokens
$30.00 / 1M tokens
gpt-4
$30.00 / 1M tokens
$60.00 / 1M tokens
gpt-4-32k
$60.00 / 1M tokens
$120.00 / 1M tokens
gpt-4-0125-preview
$10.00 / 1M tokens
$30.00 / 1M tokens
gpt-4-1106-preview
$10.00 / 1M tokens
$30.00 / 1M tokens
gpt-4-vision-preview
$10.00 / 1M tokens
$30.00 / 1M tokens
gpt-3.5-turbo-0125
$0.50 / 1M tokens
$1.50 / 1M tokens
gpt-3.5-turbo-instruct
$1.50 / 1M tokens
$2.00 / 1M tokens
gpt-3.5-turbo-1106
$1.00 / 1M tokens
$2.00 / 1M tokens
gpt-3.5-turbo-0613
$1.50 / 1M tokens
$2.00 / 1M tokens
gpt-3.5-turbo-16k-0613
$3.00 / 1M tokens
$4.00 / 1M tokens
gpt-3.5-turbo-0301
$1.50 / 1M tokens
$2.00 / 1M tokens
davinci-002
$2.00 / 1M tokens
$2.00 / 1M tokens
babbage-002
$0.40 / 1M tokens
$0.40 / 1M tokens

These are copied from a table, so in general if it's ambiguous, the first price is the one that I want (i.e. without batching or caching)

Please update this python code to include all the prices above:

GPT_3_5_TURBO_PRICE_PER_TOKEN: Pricing = {
    "prompt": 0.50 / 1_000_000,
    "response": 1.50 / 1_000_000,
}

GPT_3_5_TURBO_16K_PRICE_PER_TOKEN: Pricing = {
    "prompt": 0.003 / 1000,
    "response": 0.004 / 1000,
}

GPT_4_PRICE_PER_TOKEN: Pricing = {
    "prompt": 30.0 / 1_000_000,
    "response": 60.0 / 1_000_000,
}

GPT_4_TURBO_PRICE_PER_TOKEN: Pricing = {
    "prompt": 10.0 / 1_000_000,
    "response": 30.0 / 1_000_000,
}

GPT_4_32K_PRICE_PER_TOKEN: Pricing = {
    "prompt": 60.0 / 1_000_000,
    "response": 120.0 / 1_000_000,
}

GPT_4_O_2024_05_13_PRICE_PER_TOKEN: Pricing = {
    "prompt": 5.0 / 1_000_000,
    "response": 15.0 / 1_000_000,
}

GPT_4_O_2024_08_06_PRICE_PER_TOKEN: Pricing = {
    "prompt": 2.50 / 1_000_000,
    "response": 10.0 / 1_000_000,
}

GPT_4_O_MINI_PRICE_PER_TOKEN: Pricing = {
    "prompt": 0.150 / 1_000_000,
    "response": 0.600 / 1_000_000,
}

O_1_PREVIEW_PRICE_PER_TOKEN: Pricing = {
    "prompt": 15.0 / 1_000_000,
    "response": 60.0 / 1_000_000,
}

O_1_MINI_PRICE_PER_TOKEN: Pricing = {
    "prompt": 3.0 / 1_000_000,
    "response": 12.0 / 1_000_000,
}


def gpt_pricing(model: str) -> Optional[Pricing]:
    if model.startswith("gpt-3.5-turbo-16k"):
        return GPT_3_5_TURBO_16K_PRICE_PER_TOKEN
    elif model.startswith("gpt-3.5-turbo"):
        return GPT_3_5_TURBO_PRICE_PER_TOKEN
    elif model.startswith("gpt-4-32k"):
        return GPT_4_32K_PRICE_PER_TOKEN
    elif model.startswith("gpt-4o-mini"):
        return GPT_4_O_MINI_PRICE_PER_TOKEN
    elif model.startswith("gpt-4o-2024-05-13") or model.startswith("chatgpt-4o-latest"):
        return GPT_4_O_2024_05_13_PRICE_PER_TOKEN
    elif model.startswith("gpt-4o"):
        return GPT_4_O_2024_08_06_PRICE_PER_TOKEN
    elif model.startswith("gpt-4-turbo") or re.match(r"gpt-4-\d\d\d\d-preview", model):
        return GPT_4_TURBO_PRICE_PER_TOKEN
    elif model.startswith("gpt-4"):
        return GPT_4_PRICE_PER_TOKEN
    elif model.startswith("o1-preview"):
        return O_1_PREVIEW_PRICE_PER_TOKEN
    elif model.startswith("o1-mini"):
        return O_1_MINI_PRICE_PER_TOKEN
    else:
        return None

The new price should supercede the old price. Please give me the full code output (do not abbreviate) so I can copy it in as a new pull request

kharvd
kharvd requested changes Feb 3, 2025
View reviewed changes
Copy link
Owner

@kharvd kharvd left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're changing the logic to explicitly handle every supported model name, let's use a dictionary and not a huge AI-generated if-elif

gptcli/providers/openai.py Outdated
return O_1_PREVIEW_2024_09_12_PRICE_PER_TOKEN
elif model == "o3-mini":
return O_3_MINI_PRICE_PER_TOKEN
elif model == "03-mini-2025-01-31":
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be o3, not 03

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good spot (this was a typo on the website apparently).

@williamjameshandley
Copy link
Contributor Author

Is the above what you had in mind, or would you like to move it into a nested dict (or yaml file of prices?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kharvd kharvd Awaiting requested review from kharvd

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@williamjameshandley @kharvd