Skip to content

fix: implement lazy client creation in replicate.use()#57

Merged
zeke merged 12 commits intonextfrom
fix-lazy-client-creation
Aug 28, 2025
Merged

fix: implement lazy client creation in replicate.use()#57
zeke merged 12 commits intonextfrom
fix-lazy-client-creation

Conversation

@zeke
Copy link
Member

@zeke zeke commented Aug 27, 2025

Problem

replicate.use() was creating a client immediately, causing errors when no API token was available at call time. This prevented usage in Cog pipelines where tokens are provided through cog.current_scope() at runtime.

Solution

Modified Function and AsyncFunction classes to support lazy client creation by always accepting client classes and instantiating them on demand.

Changes

  • _module_client.py: Pass Replicate and AsyncReplicate classes directly instead of factory functions
  • lib/_predictions_use.py: Simplified to always use Type[Client] approach:
    • Removed Union[Client, Type[Client]] in favor of just Type[Client]
    • Updated constructors to store _client_class instead of mixed types
    • Simplified _client property to always instantiate from class
    • Updated all use() overloads to accept class types only
    • Use issubclass() for async client detection
  • Added simple test demonstrating the fix

Result

replicate.use("model") works even when no token is initially available
✅ Client created lazily when model is actually called
✅ Token retrieved from current cog scope at call time
✅ Simplified, consistent approach using only class types
✅ Clean type inference without complex Union handling

Fixes the original error:

replicate.ReplicateError: The bearer_token client option must be set either by passing bearer_token to the client or by setting the REPLICATE_API_TOKEN environment variable

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants