Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat (provider/amazon-bedrock): Add reasoning support #4980

Merged
merged 35 commits into from
Mar 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
476bfe4
feat(bedrock): Added bedrock reasoning support
Und3rf10w Feb 25, 2025
69c4dd6
fix(bedrock): In `doGenerate` make the reasoning be a reasoning part,…
Und3rf10w Feb 25, 2025
a7cfb3d
fix(bedrock): Was parsing geneate output reasoning part incorrectly
Und3rf10w Feb 25, 2025
2562bbe
fix(bedrock): Fixed type checks and stream reasoning part
Und3rf10w Feb 25, 2025
6cdb3ff
fix(bedrock): Fixed stream signature part
Und3rf10w Feb 25, 2025
e7aeec0
fix(bedrock): Verified that signature part parses correctly on bedroc…
Und3rf10w Feb 25, 2025
31bef7c
fix(ai-sdk/amazon-bedrock): Fix tool calling with reasoning support o…
Und3rf10w Feb 26, 2025
6fa8205
Merge branch 'main' into bedrock/add-reasoning
Und3rf10w Feb 26, 2025
cdda477
fix(provider/amazon-bedrock): Add redacted reasoning support to strea…
Und3rf10w Feb 27, 2025
a43445c
Merge branch 'main' into bedrock/add-reasoning
Und3rf10w Feb 27, 2025
df977b1
fix(provider/amazon-bedrock): Properly discriminate and handle reason…
Und3rf10w Feb 28, 2025
d3cfac2
Merge branch 'main' into bedrock/add-reasoning
Und3rf10w Feb 28, 2025
22c5486
feat(provider/amazon-bedrock): Add tests for reasoning, tool usage, a…
Und3rf10w Feb 28, 2025
6f23643
fix(provider/amazon-bedrock): Shape the reasoning details types corre…
Und3rf10w Feb 28, 2025
468a750
fix(provider/amazon-bedrock): Mimic doGenerate reasoning response for…
Und3rf10w Feb 28, 2025
f5afe75
Merge branch 'main' into bedrock/add-reasoning
Und3rf10w Feb 28, 2025
96000c1
Merge branch 'main' into bedrock/add-reasoning
Und3rf10w Mar 3, 2025
ac55421
fix(ai): Append reasoning details on turns
Und3rf10w Mar 3, 2025
fdb87b5
fix(providers/amazon-bedrock): Fix json response test in bedrock
Und3rf10w Mar 4, 2025
9988ee8
docs(providers/amazon-bedrock): Add reasoning documentation
Und3rf10w Mar 4, 2025
d3dc27b
chore: Add changeset
Und3rf10w Mar 4, 2025
64d2f17
Merge branch 'main' into bedrock/add-reasoning
Und3rf10w Mar 4, 2025
74c7bda
tweak(providers/amazon-bedrock): code cleanup
Und3rf10w Mar 4, 2025
d85704e
fix(ai/core): Revert change to generate-text
Und3rf10w Mar 5, 2025
a2c9eb4
Merge branch 'main' into bedrock/add-reasoning
Und3rf10w Mar 5, 2025
8eed8cd
feat(provider/amazon-bedrock): Add examples for amazon-bedrock reasoning
Und3rf10w Mar 5, 2025
ce6e24d
Merge remote-tracking branch 'origin/main' into bedrock/add-reasoning
shaper Mar 6, 2025
050e309
edit doc, add convert tests, simplify test types, share trim logic in…
shaper Mar 6, 2025
74e5a60
fix typo in docs, add "us." prefix to model ids, add api doc link to …
shaper Mar 6, 2025
7a157d8
rm unused imports, avoid duplicate reasoning_config in request body
shaper Mar 6, 2025
c19c5e5
fix model id for one example
shaper Mar 6, 2025
d3d23cb
Merge branch 'main' into bedrock/add-reasoning
Und3rf10w Mar 6, 2025
4ac00d3
fix(providers/amazon-bedrock): Reimplement provider reasoning config
Und3rf10w Mar 6, 2025
3bde0d1
tweak(examples/amazon-bedrock): Clean bedrock examples
Und3rf10w Mar 6, 2025
54c10c8
fix(providers/amazon-bedrock): Remove min max from reasoning config
Und3rf10w Mar 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/two-mayflies-shave.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@ai-sdk/amazon-bedrock': minor
---

feat (providers/amazon-bedrock): Add reasoning support to amazon-bedrock
28 changes: 28 additions & 0 deletions content/providers/01-ai-sdk-providers/08-amazon-bedrock.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -316,6 +316,34 @@ console.log(
// }
```

## Reasoning

Amazon Bedrock has reasoning support for the `claude-3-7-sonnet-20250219` model.

You can enable it using the `reasoning_config` provider option and specifying a thinking budget in tokens (minimum: `1024`, maximum: `64000`).

```ts
import { bedrock } from '@ai-sdk/amazon-bedrock';
import { generateText } from 'ai';

const { text, reasoning, reasoningDetails } = await generateText({
model: bedrock('us.anthropic.claude-3-7-sonnet-20250219-v1:0'),
prompt: 'How many people will live in the world in 2040?',
providerOptions: {
bedrock: {
reasoningConfig: { type: 'enabled', budgetTokens: 12000 },
},
},
});

console.log(reasoning); // reasoning text
console.log(reasoningDetails); // reasoning details including redacted reasoning
console.log(text); // text response
```

See [AI SDK UI: Chatbot](/docs/ai-sdk-ui/chatbot#reasoning) for more details
on how to integrate reasoning into your chatbot.

### Model Capabilities

| Model | Image Input | Object Generation | Tool Usage | Tool Streaming |
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import { bedrock } from '@ai-sdk/amazon-bedrock';
import { CoreMessage, generateText } from 'ai';
import 'dotenv/config';
import * as readline from 'node:readline/promises';
import { weatherTool } from '../tools/weather-tool';

const terminal = readline.createInterface({
input: process.stdin,
output: process.stdout,
});

const messages: CoreMessage[] = [];

async function main() {
while (true) {
const userInput = await terminal.question('You: ');
messages.push({ role: 'user', content: userInput });

const { steps, response } = await generateText({
model: bedrock('us.anthropic.claude-3-7-sonnet-20250219-v1:0'),
tools: { weatherTool },
system: `You are a helpful, respectful and honest assistant.`,
messages,
maxSteps: 5,
providerOptions: {
bedrock: {
reasoning_config: { type: 'enabled', budget_tokens: 2048 },
},
},
});

for (const step of steps) {
console.log(step);
if (step.reasoning) {
console.log(`\x1b[36m${step.reasoning}\x1b[0m`);
}

if (step.text) {
console.log(step.text);
}

if (step.toolCalls) {
for (const toolCall of step.toolCalls) {
console.log(
`\x1b[33m${toolCall.toolName}\x1b[0m` +
JSON.stringify(toolCall.args),
);
}
}
}

console.log('\n');

messages.push(...response.messages);
}
}

main().catch(console.error);
30 changes: 30 additions & 0 deletions examples/ai-core/src/generate-text/amazon-bedrock-reasoning.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import { bedrock } from '@ai-sdk/amazon-bedrock';
import { generateText } from 'ai';
import 'dotenv/config';

async function main() {
const result = await generateText({
model: bedrock('us.anthropic.claude-3-7-sonnet-20250219-v1:0'),
prompt: 'How many "r"s are in the word "strawberry"?',
temperature: 0.5, // should get ignored (warning)
providerOptions: {
bedrock: {
reasoning_config: { type: 'enabled', budget_tokens: 2048 },
},
},
maxRetries: 0,
maxSteps: 5,
});

console.log('Reasoning:');
console.log(result.reasoningDetails);
console.log();

console.log('Text:');
console.log(result.text);
console.log();

console.log('Warnings:', result.warnings);
}

main().catch(console.error);
64 changes: 29 additions & 35 deletions examples/ai-core/src/stream-text/amazon-bedrock-fullstream.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { bedrock } from '@ai-sdk/amazon-bedrock';
import { streamText } from 'ai';
import { streamText, ToolCallPart, ToolResultPart } from 'ai';
import 'dotenv/config';
import { z } from 'zod';
import { weatherTool } from '../tools/weather-tool';
Expand All @@ -14,57 +14,51 @@ async function main() {
},
},
prompt: 'What is the weather in San Francisco?',
maxSteps: 5,
});

let enteredReasoning = false;
let enteredText = false;
const toolCalls: ToolCallPart[] = [];
const toolResponses: ToolResultPart[] = [];

for await (const part of result.fullStream) {
switch (part.type) {
case 'reasoning': {
if (!enteredReasoning) {
enteredReasoning = true;
console.log('\nREASONING:\n');
}
process.stdout.write(part.textDelta);
break;
}

case 'text-delta': {
console.log('Text delta:', part.textDelta);
if (!enteredText) {
enteredText = true;
console.log('\nTEXT:\n');
}
process.stdout.write(part.textDelta);
break;
}

case 'tool-call': {
switch (part.toolName) {
case 'cityAttractions': {
console.log('TOOL CALL cityAttractions');
console.log(`city: ${part.args.city}`); // string
break;
}

case 'weather': {
console.log('TOOL CALL weather');
console.log(`location: ${part.args.location}`); // string
break;
}
}
toolCalls.push(part);

process.stdout.write(
`\nTool call: '${part.toolName}' ${JSON.stringify(part.args)}`,
);
break;
}

case 'tool-result': {
switch (part.toolName) {
// NOT AVAILABLE (NO EXECUTE METHOD)
// case 'cityAttractions': {
// console.log('TOOL RESULT cityAttractions');
// console.log(`city: ${part.args.city}`); // string
// console.log(`result: ${part.result}`);
// break;
// }

case 'weather': {
console.log('TOOL RESULT weather');
console.log(`location: ${part.args.location}`); // string
console.log(`temperature: ${part.result.temperature}`); // number
break;
}
}
toolResponses.push(part);

process.stdout.write(
`\nTool response: '${part.toolName}' ${JSON.stringify(part.result)}`,
);
break;
}

case 'error':
console.error('Error:', part.error);
break;
}
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
import { createAmazonBedrock } from '@ai-sdk/amazon-bedrock';
import { CoreMessage, streamText, tool } from 'ai';
import 'dotenv/config';
import * as readline from 'node:readline/promises';
import { z } from 'zod';

const bedrock = createAmazonBedrock({
// example fetch wrapper that logs the input to the API call:
fetch: async (url, options) => {
console.log('URL', url);
console.log('Headers', JSON.stringify(options!.headers, null, 2));
console.log(
`Body ${JSON.stringify(JSON.parse(options!.body! as string), null, 2)}`,
);
return await fetch(url, options);
},
});

const terminal = readline.createInterface({
input: process.stdin,
output: process.stdout,
});

const messages: CoreMessage[] = [];

async function main() {
while (true) {
const userInput = await terminal.question('You: ');

messages.push({ role: 'user', content: userInput });

const result = streamText({
model: bedrock('us.anthropic.claude-3-7-sonnet-20250219-v1:0'),
messages,
tools: {
weather: tool({
description: 'Get the weather in a location',
parameters: z.object({
location: z
.string()
.describe('The location to get the weather for'),
}),
execute: async ({ location }) => ({
location,
temperature: 72 + Math.floor(Math.random() * 21) - 10,
}),
}),
},
maxSteps: 5,
maxRetries: 0,
providerOptions: {
bedrock: {
reasoning_config: { type: 'enabled', budget_tokens: 2048 },
},
},
onError: error => {
console.error(error);
},
});

process.stdout.write('\nAssistant: ');
for await (const part of result.fullStream) {
if (part.type === 'reasoning') {
process.stdout.write('\x1b[34m' + part.textDelta + '\x1b[0m');
} else if (part.type === 'redacted-reasoning') {
process.stdout.write('\x1b[31m' + '<redacted>' + '\x1b[0m');
} else if (part.type === 'text-delta') {
process.stdout.write(part.textDelta);
}
}
process.stdout.write('\n\n');

messages.push(...(await result.response).messages);
}
}

main().catch(console.error);
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import { bedrock } from '@ai-sdk/amazon-bedrock';
import { streamText, ToolCallPart, ToolResultPart } from 'ai';
import 'dotenv/config';
import { weatherTool } from '../tools/weather-tool';

async function main() {
const result = streamText({
model: bedrock('us.anthropic.claude-3-7-sonnet-20250219-v1:0'),
tools: {
weather: weatherTool,
},
prompt: 'What is the weather in San Francisco?',
providerOptions: {
bedrock: {
reasoning_config: { type: 'enabled', budget_tokens: 12000 },
},
},
maxSteps: 5,
});

let enteredReasoning = false;
let enteredText = false;
const toolCalls: ToolCallPart[] = [];
const toolResponses: ToolResultPart[] = [];

for await (const part of result.fullStream) {
switch (part.type) {
case 'reasoning': {
if (!enteredReasoning) {
enteredReasoning = true;
console.log('\nREASONING:\n');
}
process.stdout.write(part.textDelta);
break;
}

case 'text-delta': {
if (!enteredText) {
enteredText = true;
console.log('\nTEXT:\n');
}
process.stdout.write(part.textDelta);
break;
}

case 'tool-call': {
toolCalls.push(part);

process.stdout.write(
`\nTool call: '${part.toolName}' ${JSON.stringify(part.args)}`,
);
break;
}

case 'tool-result': {
toolResponses.push(part);

process.stdout.write(
`\nTool response: '${part.toolName}' ${JSON.stringify(part.result)}`,
);
break;
}
}
}
}

main().catch(console.error);
Loading
Loading