fix: implement lazy client creation in replicate.use()#57
Merged
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
replicate.use()was creating a client immediately, causing errors when no API token was available at call time. This prevented usage in Cog pipelines where tokens are provided throughcog.current_scope()at runtime.Solution
Modified
FunctionandAsyncFunctionclasses to support lazy client creation by always accepting client classes and instantiating them on demand.Changes
_module_client.py: PassReplicateandAsyncReplicateclasses directly instead of factory functionslib/_predictions_use.py: Simplified to always useType[Client]approach:Union[Client, Type[Client]]in favor of justType[Client]_client_classinstead of mixed types_clientproperty to always instantiate from classuse()overloads to accept class types onlyissubclass()for async client detectionResult
✅
replicate.use("model")works even when no token is initially available✅ Client created lazily when model is actually called
✅ Token retrieved from current cog scope at call time
✅ Simplified, consistent approach using only class types
✅ Clean type inference without complex Union handling
Fixes the original error: