Invocation of model ID anthropic.claude-3-5-sonnet-20241022-v2:0 with on-demand throughput isn’t supported.

[aripovula]

I am using gem 'ruby_llm', '1.2.0' due to my current ruby version.

And my Ruby LLM GEM based chat service works great when I use OpenAI api keys.

And when I use AWS Bedrock directly (not through Ruby LLM) my Bedrock API calls work great.

Issues arise when I try to use AWS Bedrock through Ruby LLM.

When I select "Bedrock" as the Provider and use claude-3-5-sonnet as the model I get:
RubyLLM API error: 400 {"message":"Invocation of model ID anthropic.claude-3-5-sonnet-20241022-v2:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model."}.

And when I try to use us.anthropic.claude-3-5-sonnet-20240620-v1:0 as the model name I get Unknown model: us.anthropic.claude-3-5-sonnet-20240620-v1:0 for provider: bedrock. Same for us.anthropic.claude-3-5-sonnet-20240620-v1 or us.anthropic.claude-3-5-sonnet.

In AWS Bedrock console I do not see any model where I can add provisioned throughput. What should I do given the error message above?

I also tried using models like claude-sonnet-4 - I get error similar to Unknown model: claude-sonnet-4 for provider: bedrock when I try:
claude-sonnet-4
claude-4-sonnet
anthropic.claude-sonnet-4-20250514-v1:0
deepseek.r1-v1:0 and many others as model name ( as ENV.fetch("AWS_BEDROCK_MODEL", ...).

What I am doing wrong?

My code is

  RubyLLM.configure do |config|
    if ENV["PRECOMPILING_ASSETS"].present?
      Rails.logger.warn "Skipping ruby_llm initialization"
    else
      begin
        if ENV["AWS_BEDROCK_REGION"].present? && ENV["AWS_ACCESS_KEY_ID"].present?
          config.bedrock_api_key = ENV["AWS_ACCESS_KEY_ID"]
          config.bedrock_secret_key = ENV["AWS_SECRET_ACCESS_KEY"]
          config.bedrock_region = ENV["AWS_BEDROCK_REGION"]
          config.default_model = ENV.fetch("AWS_BEDROCK_MODEL", "claude-3-5-sonnet")
        elsif ENV["ANTHROPIC_API_KEY"].present?
          ....```

Thank you in advance !

[crmne]

I am using gem 'ruby_llm', '1.2.0' due to my current ruby version.

That's ancient. We've always supported Ruby 3.1.4 and above. Do you use Ruby 3.0 or older?

[aripovula]

Yes, we use ruby '3.1.0'.

[crmne]

Please upgrade to Ruby 3.1.4 so you can upgrade to the last RubyLLM.