Skip to content

Commit

Permalink
increased limits on anthropic batches
Browse files Browse the repository at this point in the history
  • Loading branch information
RyanMarten committed Dec 18, 2024
1 parent be2123c commit 69109c9
Showing 1 changed file with 3 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,9 @@
logger = logging.getLogger(__name__)

# https://docs.anthropic.com/en/api/creating-message-batches
MAX_REQUESTS_PER_BATCH = 10_000
MAX_BYTES_PER_BATCH = 32 * 1024 * 1024
# https://docs.anthropic.com/en/docs/build-with-claude/message-batches#batch-limitations
MAX_REQUESTS_PER_BATCH = 100_000
MAX_BYTES_PER_BATCH = 256 * 1024 * 1024

MAX_CONCURRENT_BATCH_OPERATIONS = 100 # this might need to be reduced
MAX_RETRIES_PER_OPERATION = 10
Expand Down

0 comments on commit 69109c9

Please sign in to comment.