Why does generate_content against a tuned model require transport='grpc'

[LindaLawton]

Description of the bug:

When Accessing a tuned model i need to configure grcp.

genai.configure(api_key=os.getenv("API_KEY"))
genai.configure(transport='grpc')

Why is this needed

Actual vs expected behavior:

I am really just curious its more of a question then a bug.

full error stacktrace

Traceback (most recent call last):
File "C:\Development\Gemini\python\gemini_samples_python\GeminiSamples\tuning\access_tuned_model.py", line 30, in
tuned_models = list_my_tuned_models()
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Development\Gemini\python\gemini_samples_python\GeminiSamples\tuning\access_tuned_model.py", line 23, in list_my_tuned_models
for m in genai.list_tuned_models():
File "C:\Development\Gemini\python\GeminiAnalyitcs\google-analytics-gemini\venv\Lib\site-packages\google\generativeai\models.py", line 237, in list_tuned_models
for model in client.list_tuned_models(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Development\Gemini\python\GeminiAnalyitcs\google-analytics-gemini\venv\Lib\site-packages\google\ai\generativelanguage_v1beta\services\model_service\client.py", line 1144, in list_tuned_models
response = rpc(
^^^^
File "C:\Development\Gemini\python\GeminiAnalyitcs\google-analytics-gemini\venv\Lib\site-packages\google\api_core\gapic_v1\method.py", line 131, in call
return wrapped_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Development\Gemini\python\GeminiAnalyitcs\google-analytics-gemini\venv\Lib\site-packages\google\api_core\retry\retry_unary.py", line 293, in retry_wrapped_func
return retry_target(
^^^^^^^^^^^^^
File "C:\Development\Gemini\python\GeminiAnalyitcs\google-analytics-gemini\venv\Lib\site-packages\google\api_core\retry\retry_unary.py", line 153, in retry_target
_retry_error_helper(
File "C:\Development\Gemini\python\GeminiAnalyitcs\google-analytics-gemini\venv\Lib\site-packages\google\api_core\retry\retry_base.py", line 212, in _retry_error_helper
raise final_exc from source_exc
File "C:\Development\Gemini\python\GeminiAnalyitcs\google-analytics-gemini\venv\Lib\site-packages\google\api_core\retry\retry_unary.py", line 144, in retry_target
result = target()
^^^^^^^^
File "C:\Development\Gemini\python\GeminiAnalyitcs\google-analytics-gemini\venv\Lib\site-packages\google\api_core\timeout.py", line 120, in func_with_timeout
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Development\Gemini\python\GeminiAnalyitcs\google-analytics-gemini\venv\Lib\site-packages\google\api_core\grpc_helpers.py", line 78, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.Unauthenticated: 401 API keys are not supported by this API. Expected OAuth2 access token or other authentication credentials that assert a principal. See https://cloud.google.com/docs/authentication [reason: "CREDENTIALS_MISSING"
domain: "googleapis.com"
metadata {
key: "service"
value: "generativelanguage.googleapis.com"
}
metadata {
key: "method"
value: "google.ai.generativelanguage.v1beta.ModelService.ListTunedModels"
}
]

Any other information you'd like to share?

My full sample

from dotenv import load_dotenv
import os
import google.generativeai as genai

load_dotenv()

genai.configure(api_key=os.getenv("API_KEY"))
genai.configure(transport='grpc')   # comment this out and it will break


def get_response(use_tuned_model, text):
    try:
        print(f"Making request: {text} against Model: {use_tuned_model}")
        model = genai.GenerativeModel(model_name=use_tuned_model)
        response = model.generate_content(text)
        print(response)
    except Exception as e:
        print("Error: ", e)


def list_my_tuned_models():
    models = []
    for m in genai.list_tuned_models():
        print(f"Name: {m.name} Description: {m.description} Created: {m.create_time} Base Model: {m.base_model}")
        models.append(m.name)
    return models


if __name__ == "__main__":
    tuned_models = list_my_tuned_models()

    for tuned_model in tuned_models:
        get_response(tuned_model, "55")

[MarkDaoust]

Thanks for the report Linda.

1. The default is grpc, and I see you're not in Colab, so this has nothing to do with Colab's default being transport='rest'.

2. I'm working on a replication, but I didn't have a tuned model and my tuning job is at the back of the queue for now.

Hmmm....

genai.configure(api_key=os.getenv("API_KEY"))
genai.configure(transport='grpc')

I'm not sure about most of these steps, but it's possible that:

• You have the oauth credentials in the default location (gcloud auth application-default ...), from when you tuned the model.
• Setting api_key= overrides the application-default-credentials.
• Tuned models require the oauth credentials.
• .configure doesn't edit the configuration. It replaces the configuration.
  • So if the previous points are true, .configure(transport='grpc') throws out the api-key, and puts you on the application-default credentials. .configure() or configure(transport='rest') might have the same effect.

While I wait for tuning to wake up could you confirm any of those other points?

[LindaLawton]

• Its the same project which has GOOGLE_APPLICATION_CREDENTIALS set in the .env var.
• Why would setting apikey overwrite application default credetinals and why would it work again adding grpc?
• Yes tuned models require service account auth hence the GOOGLE_APPLICATION_CREDENTIALS setting
• So what setting API key forces it to be transport rest?

Let me know if you want my tuning sample its basically the one from the docs though. Note im just testing so just use the sample tuned data.

PS: didnt know you could do checkboxes in Git markdown. #cool

[MarkDaoust]

I think this all fits my interpretation of what's going wrong here.

GOOGLE_APPLICATION_CREDENTIALS set in the .env var.

That sets the ADC as the default credentials.

Why would setting apikey overwrite application default credentials

Because the API can't take both the ADC and an API key at the same time. It has to choose one or the other.

and why would it work again adding grpc?

ADC works for Tuned Models, API key doesn't.
When you call genai.configure(api_key="...") it stores configuration= {'api_key': ...}.

• now tuned models are broken because the api_key takes precedence.

When you call genai.configure(transport="grpc") it stores configuration={'transport': 'grpc'}, and forgets the api_key.

• now tuned models work because it falls back to using the ADC.

It doesn't have anything to do with actually switching between rest and grpc, just clearing the api-key from the config:

Steps to reproduce: https://colab.research.google.com/drive/1C8nKL6cKjrwGom_Ye3ssB2iqw7gQyB4T?usp=sharing

[github-actions]

Marking this issue as stale since it has been open for 14 days with no activity. This issue will be closed if no further activity occurs.