BedrockAgentRuntimeAsyncClient InvokeAgent Attributions ALWAYS Empty

[kmazurek13]

Describe the bug

I have an agent linked to a knowledge base that uses the Claude 3.5 Haiku model to orchestrate response.

I can't get the attribution information containing retrieved references to pass all the way through.
Whenever I'm looking at the attribution property on my PayloadPart object in the visitChunk method it's always null.

What's interesting is, if I query the Knowledge Base directly using the same model in the AWS console, I get the attribution's references in-line exactly how I want. When I go up to the agent level, those attributions are dropped in the response even though they're still clearly available in the trace.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

I'd expect the attribution information to pass all the way through to the request's response.

Current Behavior

I get a good response from the agent but there is no attribution info. The attribution property is null.

Reproduction Steps

Send any InvokeAgent request to an agent using a knowledge base and Claude 3.5 Haiku. Investigate the attribution property on the PayloadPart when visitChunk is called on the response.

Possible Solution

No response

Additional Information/Context

This same issue appears to be occurring in boto3 as well:

boto/boto3#4197

AWS Java SDK version used

2.30.3

JDK version used

openjdk version "21.0.5" 2024-10-15 LTS OpenJDK Runtime Environment Temurin-21.0.5+11 (build 21.0.5+11-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.5+11 (build 21.0.5+11-LTS, mixed mode, sharing)

Operating System and version

Ubuntu 22.04.5 LTS

[debora-ito]

Hi @kmazurek13 thank you for reporting this. Looks like an issue on the Bedrock side, they are aware and investigating.

When I go up to the agent level, those attributions are dropped in the response even though they're still clearly available in the trace.

By "go up the agent level" you mean calling the API instead of using the console? Can you show how the trace looks like?

[kmazurek13]

@debora-ito really appreciate you responding!

I'm sorry for not sharing my trace information as the tests I'm running right now are against a knowledge base I can't share on GitHub.

When I mentioned "go up to the agent level" I was specifically talking about in the AWS console.

In the AWS Bedrock Console, under Knowledge Bases when I select the Claude 3.5 Haiku model to test my specific knowledge base and then ask it a question, it answers that question and includes inline citations with the retrieved results. I guess I'm assuming, this means at that point the attribution is present in whatever implementation the console is using.

Still in the AWS console, when I select the agent I created that uses the aforementioned knowledge base and the same Claude 3.5 Haiku model and test it with the same question, I get a response but those inline citations are no longer there nor do they show up in the attribution object when I run InvokeAgent via the API.

Is there somewhere I can follow progress on the known Bedrock issue? I'm assuming it's out of our hands if that's where the issue lies?

[debora-ito]

Got it, thank you for the clarification.

Is there somewhere I can follow progress on the known Bedrock issue? I'm assuming it's out of our hands if that's where the issue lies?

No way to follow progress, we usually suggest to keep an eye in the CHANGELOG for Bedrock updates. I'll add your use case to the internal ticket cut to the Bedrock team, and try to keep this issue up to date.