400 INVALID_ARGUMENT error when attempting to use Function Calling with Vertex AI Search retrieval tool

[dtrenz]

Environment details

• Programming language: TS/JS
• OS: macOS
• Language runtime version: Typescript 5.6.2
• Package version: 1.7.0

Steps to reproduce

1. Configure a Vertex AI generative model (getGenerativeModel) with a function calling tool and a Vertex AI Search retrieval tool:

  tools: [
    {
      functionDeclarations: [
        {
          name: "get_current_weather",
          description: "get weather in a given location",
          parameters: {
            type: FunctionDeclarationSchemaType.OBJECT,
            properties: {
              location: { type: FunctionDeclarationSchemaType.STRING },
            },
            required: ["location"],
          },
        },
      ],
      retrieval: {
        vertexAiSearch: {
          datastore: "<datastore>",
        },
        disableAttribution: false,
      },
    },
  ],

2. Create a ChatSession using startChat()
3. Call sendMessageStream on a ChatSession

Error output

 ⨯ Internal error: ClientError: [VertexAI.ClientError]: got status: 400 Bad Request. {"error":{"code":400,"message":"Request contains an invalid argument.","status":"INVALID_ARGUMENT"}}
    at throwErrorIfNotOK (./node_modules/@google-cloud/vertexai/build/src/functions/post_fetch_processing.js:32:27)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async generateContentStream (./node_modules/@google-cloud/vertexai/build/src/functions/generate_content.js:97:5)
digest: "3774699693"
Cause: GoogleApiError: Request contains an invalid argument.
    at throwErrorIfNotOK (webpack-internal:///(action-browser)/./node_modules/@google-cloud/vertexai/build/src/functions/post_fetch_processing.js:32:66)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async generateContentStream (webpack-internal:///(action-browser)/./node_modules/@google-cloud/vertexai/build/src/functions/generate_content.js:97:5) {
  code: 400,
  status: 'INVALID_ARGUMENT',
  errorDetails: undefined
}

[marius-ciocoiu]

There is a similar issue with googleSearchRetrieval, which can be resolved by including only the googleSearchRetrieval tool in the tools array. If we include any other tool besides that, we encounter a 400 INVALID_ARGUMENT error. It appears that when grounding is being used, we cannot currently use any other tool.