Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions packages/typescript/ai/src/activities/chat/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1084,22 +1084,29 @@ class TextEngine<
const chunks: Array<StreamChunk> = []

for (const result of results) {
const content = JSON.stringify(result.result)
const fullContent = JSON.stringify(result.result)

// Apply clientOutput filter if the tool defines one
const tool = this.tools.find((t) => t.name === result.toolName)
const clientContent =
tool?.clientOutput && result.state !== 'output-error'
? JSON.stringify(tool.clientOutput(result.result))
: fullContent

chunks.push({
type: 'TOOL_CALL_END',
timestamp: Date.now(),
model: finishEvent.model,
toolCallId: result.toolCallId,
toolName: result.toolName,
result: content,
result: clientContent,
Comment on lines +1087 to +1102
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Wrap clientOutput invocation in try-catch to prevent stream breakage.

If clientOutput throws (e.g., due to unexpected result structure or JSON.stringify failing on circular references), the exception will propagate and break the entire stream. Consider catching exceptions and falling back to fullContent.

🛡️ Proposed fix to add error handling
       const fullContent = JSON.stringify(result.result)

       // Apply clientOutput filter if the tool defines one
       const tool = this.tools.find((t) => t.name === result.toolName)
-      const clientContent =
-        tool?.clientOutput && result.state !== 'output-error'
-          ? JSON.stringify(tool.clientOutput(result.result))
-          : fullContent
+      let clientContent = fullContent
+      if (tool?.clientOutput && result.state !== 'output-error') {
+        try {
+          clientContent = JSON.stringify(tool.clientOutput(result.result))
+        } catch {
+          // Fall back to full content if clientOutput fails
+        }
+      }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const fullContent = JSON.stringify(result.result)
// Apply clientOutput filter if the tool defines one
const tool = this.tools.find((t) => t.name === result.toolName)
const clientContent =
tool?.clientOutput && result.state !== 'output-error'
? JSON.stringify(tool.clientOutput(result.result))
: fullContent
chunks.push({
type: 'TOOL_CALL_END',
timestamp: Date.now(),
model: finishEvent.model,
toolCallId: result.toolCallId,
toolName: result.toolName,
result: content,
result: clientContent,
const fullContent = JSON.stringify(result.result)
// Apply clientOutput filter if the tool defines one
const tool = this.tools.find((t) => t.name === result.toolName)
let clientContent = fullContent
if (tool?.clientOutput && result.state !== 'output-error') {
try {
clientContent = JSON.stringify(tool.clientOutput(result.result))
} catch {
// Fall back to full content if clientOutput fails
}
}
chunks.push({
type: 'TOOL_CALL_END',
timestamp: Date.now(),
model: finishEvent.model,
toolCallId: result.toolCallId,
toolName: result.toolName,
result: clientContent,
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/typescript/ai/src/activities/chat/index.ts` around lines 1087 -
1102, The call to the tool's clientOutput inside the TOOl_CALL_END construction
can throw and break the stream; modify the logic around the clientContent
computation (where tool, clientOutput, result, and fullContent are used) to
invoke tool.clientOutput(result.result) inside a try-catch, and on any error
fall back to using fullContent (optionally log or attach the caught error
context), then push the chunks entry with the safe clientContent instead of
directly calling clientOutput.

})

this.messages = [
...this.messages,
{
role: 'tool',
content,
content: fullContent,
toolCallId: result.toolCallId,
},
]
Expand Down
10 changes: 9 additions & 1 deletion packages/typescript/ai/src/activities/chat/tools/tool-calls.ts
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,7 @@ export class ToolCallManager {
const tool = this.tools.find((t) => t.name === toolCall.function.name)

let toolResultContent: string
let clientResultContent: string | undefined
if (tool?.execute) {
try {
// Parse arguments (normalize "null" to "{}" for empty tool_use blocks)
Expand Down Expand Up @@ -215,6 +216,13 @@ export class ToolCallManager {

toolResultContent =
typeof result === 'string' ? result : JSON.stringify(result)

// Apply clientOutput filter if the tool defines one
if (tool.clientOutput) {
const parsed =
typeof result === 'string' ? JSON.parse(result) : result
clientResultContent = JSON.stringify(tool.clientOutput(parsed))
}
Comment on lines +219 to +225
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Potential runtime error when clientOutput is applied to non-JSON string results.

If tool.execute returns a plain string (not valid JSON) and clientOutput is defined, JSON.parse(result) at line 223 will throw a SyntaxError. Additionally, if tool.clientOutput returns a value with circular references, JSON.stringify at line 224 will throw.

Since clientOutput is a convenience feature for filtering sensitive data, failures here should degrade gracefully rather than break the stream.

🛡️ Proposed fix to add defensive error handling
          // Apply clientOutput filter if the tool defines one
          if (tool.clientOutput) {
-            const parsed =
-              typeof result === 'string' ? JSON.parse(result) : result
-            clientResultContent = JSON.stringify(tool.clientOutput(parsed))
+            try {
+              const parsed =
+                typeof result === 'string' ? JSON.parse(result) : result
+              clientResultContent = JSON.stringify(tool.clientOutput(parsed))
+            } catch {
+              // Fall back to unfiltered content if clientOutput fails
+            }
          }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Apply clientOutput filter if the tool defines one
if (tool.clientOutput) {
const parsed =
typeof result === 'string' ? JSON.parse(result) : result
clientResultContent = JSON.stringify(tool.clientOutput(parsed))
}
// Apply clientOutput filter if the tool defines one
if (tool.clientOutput) {
try {
const parsed =
typeof result === 'string' ? JSON.parse(result) : result
clientResultContent = JSON.stringify(tool.clientOutput(parsed))
} catch {
// Fall back to unfiltered content if clientOutput fails
}
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/typescript/ai/src/activities/chat/tools/tool-calls.ts` around lines
219 - 225, The current block that applies tool.clientOutput to the tool result
can throw on JSON.parse(result) if result is a non-JSON string and can throw on
JSON.stringify if clientOutput returns circular data; wrap the clientOutput
application in a try/catch around parsing, filtering, and stringifying (the code
handling tool.clientOutput, result, parsed, and clientResultContent) and on any
error fall back to a safe, non-throwing representation (e.g., use the original
result string or a simple "[unserializable]" placeholder) and log the error;
ensure you attempt parsing only when result is a valid JSON string (or skip
parse and pass raw value to clientOutput), and try JSON.stringify but catch
TypeError from circular structures to avoid breaking the stream.

} catch (error: unknown) {
// If tool execution fails, add error message
const message =
Expand All @@ -233,7 +241,7 @@ export class ToolCallManager {
toolName: toolCall.function.name,
model: finishEvent.model,
timestamp: Date.now(),
result: toolResultContent,
result: clientResultContent ?? toolResultContent,
}

// Add tool result message
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ export interface ClientTool<
needsApproval?: boolean
lazy?: boolean
metadata?: Record<string, unknown>
clientOutput?: (result: InferSchemaType<TOutput>) => unknown
execute?: (
args: InferSchemaType<TInput>,
) => Promise<InferSchemaType<TOutput>> | InferSchemaType<TOutput>
Expand Down Expand Up @@ -99,6 +100,7 @@ export interface ToolDefinitionConfig<
needsApproval?: boolean
lazy?: boolean
metadata?: Record<string, unknown>
clientOutput?: (result: InferSchemaType<TOutput>) => unknown
}

/**
Expand Down
3 changes: 3 additions & 0 deletions packages/typescript/ai/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,9 @@ export interface Tool<
/** If true, this tool is lazy and will only be sent to the LLM after being discovered via the lazy tool discovery mechanism. Only meaningful when used with chat(). */
lazy?: boolean

/** Optional transform to filter tool output before streaming to the client. The full result is always sent to the LLM. */
clientOutput?: (result: any) => any

/** Additional metadata for adapters or custom extensions */
metadata?: Record<string, any>
}
Expand Down
130 changes: 130 additions & 0 deletions packages/typescript/ai/tests/client-output-types.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
/**
* Type-level tests for clientOutput on tool definitions.
*
* These tests verify that TypeScript correctly infers the clientOutput
* parameter type from the outputSchema, and that the property propagates
* through .server() and .client() builders.
*/

import { describe, expectTypeOf, it } from 'vitest'
import { z } from 'zod'
import { toolDefinition } from '../src/activities/chat/tools/tool-definition'

describe('clientOutput type inference', () => {
const outputSchema = z.object({
id: z.string(),
name: z.string(),
ssn: z.string(),
internalScore: z.number(),
})

type ExpectedOutput = z.infer<typeof outputSchema>

it('should type the clientOutput parameter from outputSchema', () => {
toolDefinition({
name: 'lookup_user',
description: 'Look up a user',
outputSchema,
clientOutput: (result) => {
// result should be typed as the inferred output schema type
expectTypeOf(result).toEqualTypeOf<ExpectedOutput>()
return { id: result.id }
},
})
})

it('should allow clientOutput to return any shape', () => {
// clientOutput return type is unknown — no constraints on what shape the
// filtered result has. This is intentional: the client result doesn't need
// to conform to the outputSchema.
toolDefinition({
name: 'flexible_return',
description: 'Return anything',
outputSchema,
clientOutput: (result) => {
expectTypeOf(result).toEqualTypeOf<ExpectedOutput>()
return { justId: result.id, extra: 42 }
},
})
})

it('should reject accessing nonexistent properties in clientOutput', () => {
toolDefinition({
name: 'bad_access',
description: 'Bad property access',
outputSchema,
clientOutput: (result) => ({
id: result.id,
// @ts-expect-error - nonExistent does not exist on output schema
bad: result.nonExistent,
}),
})
})

it('should propagate clientOutput through .server()', () => {
const tool = toolDefinition({
name: 'server_propagation',
description: 'Check server propagation',
outputSchema,
clientOutput: (result) => ({ id: result.id }),
})

const serverTool = tool.server(async () => ({
id: '1',
name: 'Alice',
ssn: '000',
internalScore: 99,
}))

// ServerTool extends Tool which uses (result: any) => any for clientOutput.
// Verify it exists and is callable.
expectTypeOf(serverTool).toHaveProperty('clientOutput')
if (serverTool.clientOutput) {
expectTypeOf(serverTool.clientOutput).toBeFunction()
}
})

it('should propagate clientOutput through .client()', () => {
const tool = toolDefinition({
name: 'client_propagation',
description: 'Check client propagation',
outputSchema,
clientOutput: (result) => ({ id: result.id }),
})

const clientTool = tool.client()

// ClientTool preserves the strongly-typed clientOutput
expectTypeOf(clientTool).toHaveProperty('clientOutput')
if (clientTool.clientOutput) {
// The parameter is typed from outputSchema
type Param = Parameters<typeof clientTool.clientOutput>[0]
expectTypeOf<Param>().toEqualTypeOf<ExpectedOutput>()
}
})

it('should type clientOutput parameter as any when no outputSchema', () => {
// Without outputSchema, SchemaInput defaults to StandardJSONSchemaV1<any, any> | JSONSchema,
// so InferSchemaType resolves to `any` — matching the convention of execute's args type.
toolDefinition({
name: 'no_schema',
description: 'No output schema',
clientOutput: (result) => {
expectTypeOf(result).toBeAny()
return result
},
})
})

it('should allow omitting clientOutput entirely', () => {
const tool = toolDefinition({
name: 'no_filter',
description: 'No client filtering',
outputSchema,
})

// clientOutput should be optional (undefined at runtime)
expectTypeOf(tool).toHaveProperty('clientOutput')
expectTypeOf(tool.clientOutput).toBeNullable()
})
})
151 changes: 151 additions & 0 deletions packages/typescript/ai/tests/tool-call-manager.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,58 @@ describe('ToolCallManager', () => {
expect(toolCalls[0]?.function.arguments).toBe('{"location":"New York"}')
})
})

it('should apply clientOutput filter to TOOL_CALL_END but keep full result in tool message', async () => {
const sensitiveWeatherTool: Tool = {
name: 'get_weather_sensitive',
description: 'Get weather with internal data',
execute: vi.fn(() => ({
temp: 72,
location: 'Paris',
internalStationId: 'SENSOR-42',
})),
clientOutput: (result: any) => ({
temp: result.temp,
location: result.location,
}),
}

const manager = new ToolCallManager([sensitiveWeatherTool])

manager.addToolCallStartEvent({
type: 'TOOL_CALL_START',
toolCallId: 'call_filtered',
toolName: 'get_weather_sensitive',
timestamp: Date.now(),
index: 0,
})

manager.addToolCallArgsEvent({
type: 'TOOL_CALL_ARGS',
toolCallId: 'call_filtered',
timestamp: Date.now(),
delta: '{}',
})

const { chunks, result: toolMessages } = await collectGeneratorOutput(
manager.executeTools(mockFinishedEvent),
)

// TOOL_CALL_END chunk should contain filtered result (no internalStationId)
expect(chunks).toHaveLength(1)
const clientResult = JSON.parse(chunks[0]!.result!)
expect(clientResult).toEqual({ temp: 72, location: 'Paris' })
expect(clientResult).not.toHaveProperty('internalStationId')

// Tool message for LLM should contain full result
expect(toolMessages).toHaveLength(1)
const llmResult = JSON.parse(toolMessages[0]!.content as string)
expect(llmResult).toEqual({
temp: 72,
location: 'Paris',
internalStationId: 'SENSOR-42',
})
})
})

describe('executeToolCalls', () => {
Expand Down Expand Up @@ -720,4 +772,103 @@ describe('executeToolCalls', () => {
)
})
})

describe('clientOutput filtering', () => {
it('should filter tool result in TOOL_CALL_END but keep full result for LLM', async () => {
const tool: Tool = {
name: 'lookup_user',
description: 'Look up a user',
execute: vi.fn(() => ({
id: 'u1',
name: 'Alice',
ssn: '123-45-6789',
internalScore: 99,
})),
clientOutput: (result: any) => ({
id: result.id,
name: result.name,
}),
}

const toolCalls = [makeToolCall('call_1', 'lookup_user', '{}')]

const gen = executeToolCalls(toolCalls, [tool], new Map(), new Map())

const chunks: Array<any> = []
let next = await gen.next()
while (!next.done) {
chunks.push(next.value)
next = await gen.next()
}
const result = next.value

// Full result goes to LLM
expect(result.results).toHaveLength(1)
expect(result.results[0]?.result).toEqual({
id: 'u1',
name: 'Alice',
ssn: '123-45-6789',
internalScore: 99,
})
})

it('should not apply clientOutput when tool execution errors', async () => {
const tool: Tool = {
name: 'failing_tool',
description: 'A tool that fails',
execute: vi.fn(() => {
throw new Error('DB connection failed')
}),
clientOutput: (result: any) => ({ id: result.id }),
}

const toolCalls = [makeToolCall('call_1', 'failing_tool', '{}')]

const gen = executeToolCalls(toolCalls, [tool], new Map(), new Map())

const chunks: Array<any> = []
let next = await gen.next()
while (!next.done) {
chunks.push(next.value)
next = await gen.next()
}
const result = next.value

// Error result should not be filtered
expect(result.results[0]?.result).toEqual({
error: 'DB connection failed',
})
expect(result.results[0]?.state).toBe('output-error')
})

it('should send full result to client when no clientOutput is defined', async () => {
const tool: Tool = {
name: 'open_tool',
description: 'No filtering',
execute: vi.fn(() => ({
id: 'u1',
secret: 'visible',
})),
// No clientOutput
}

const toolCalls = [makeToolCall('call_1', 'open_tool', '{}')]

const gen = executeToolCalls(toolCalls, [tool], new Map(), new Map())

const chunks: Array<any> = []
let next = await gen.next()
while (!next.done) {
chunks.push(next.value)
next = await gen.next()
}
const result = next.value

// Full result in both places
expect(result.results[0]?.result).toEqual({
id: 'u1',
secret: 'visible',
})
})
})
})
Loading
Loading