-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Open
Labels
Description
Confirm this is a feature request for the Node library and not the underlying OpenAI API.
- This is a feature request for the Node libraryTo pick up a draggable item, press the space bar. While dragging, use the arrow keys to move the item. Press space again to drop the item in its new position, or press escape to cancel.
Describe the feature or improvement you're requesting
I currently have a pattern where I need to pass context to my tools to allow them to act on my app. For example:
function updateEvent(context: { eventId: string}, args: ArgsFromOpenAi) {
const event = await fetchEvent(eventId);
}
It'd be great if there were some way to pass a global context to the runner since the runner is passed into each function call. Then, I could do something like this:
function updateEvent( args: ArgsFromOpenAi, runner: ChatCompletionStreamingRunner<EventContext>) {
const { eventId } = runner.context;
const event = await fetchEvent(eventId);
}
Additional context
A workaround is to build my own runner that leverages the existing helpers. However, this is complicated because of the types integration.
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
Select code repository
Activity
[-]Passing global context into runTools helper[/-][+]Passing global context into tools called by the runTools helper[/+]rattrayalex commentedon Dec 21, 2023
Could you use a closure for this?
tday commentedon Dec 21, 2023
I considered a closure like you wrote, but it would require merging tools defined across multiple files into a single (very) large file. I'm currently leveraging an inherited class to provide the closure, but running into some typing limitations with it that are worked around with casting and use of
any
.LMK if I'm missing a simpler solution 🙏
rattrayalex commentedon Dec 21, 2023
tday commentedon Dec 22, 2023
I'm imagining something that matches the lifecycle of runTools. The context should stay the same throughout that run. It is probably simplest to pass down a context through the
runTools
interface. E.g.My implementation is kind of a hack that I worked up after realizing that
runner
was passed to each tool call. I simply added a context class variable to a class that extendsChatCompletionStreamingRunner
. This doesn't work out well because other references to tool type don't expect my custom class.rattrayalex commentedon Dec 23, 2023
Thanks. Could you provide a more complete code sample of what you're trying to do / how you're trying to use this? Including how you update and reference the context?
rattrayalex commentedon Dec 23, 2023
Have you tried using
.bind(context)
on the functions before passing them in, and referencingthis
for context? Or even a pattern like this?:tday commentedon Jan 3, 2024
That pattern works! Thanks for the suggestion.
The caveat is that the tool definition would have to be managed within the scope of the context which requires a good bit of refactor for me.
You can close this issue if you think it's best that providing context not be built into the library!
rattrayalex commentedon Jan 4, 2024
Thanks!
Hmm, it might be optimal, but I'd like to provide the best possible experience. Would you be willing to share a more complete code sample of what you'd ideally like to see, including how you update & read from context?
tday commentedon Jan 18, 2024
Sorry for the late response-- I've been pushing to get the feature launched and left this as tech debt.
I was able to circle back to clean it up.
Here's how my implementation looks with the function closure:
This feels fairly good. The only caveat is I have to override and maintain my own type and wrapper to convert to the function type that
RunnableFunction
expects. The abstraction gets a bit leakyIt would be great to simplify the above by being able to simply change
completionStreamWithTools
to a function signature like this:Then, I would call runTools like this:
I imagine that the tool context would then be provided to each tool with something like this:
Alternatively, the tool context could be destructured into Args, but that might be more complicated than it is worth.
rattrayalex commentedon Jan 24, 2024
Interesting. Thank you very much for sharing, this is quite helpful. The
toolContext
suggestion is interesting and we'll take that back to the team.What do you think about something like this, so you don't have to subclass or write
toTools
?tday commentedon Jan 24, 2024
Oh, that's a nice suggestion! Though, I think that might still lead to folks wanting to DRY up the
RunnableFunction
instantiation with a helper liketoTools
to reduce boilerplatee.g. to DRY up this