Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

generate assistant message id from backend side #3061

Open
himself65 opened this issue Sep 19, 2024 · 2 comments · May be fixed by #3110
Open

generate assistant message id from backend side #3061

himself65 opened this issue Sep 19, 2024 · 2 comments · May be fixed by #3110
Labels
ai/ui enhancement New feature or request

Comments

@himself65
Copy link
Contributor

Feature Description

We have a chat backend with persistence layer, so everytime user refresh page will keep track of the history messages. So it's necessary that keep message id constant everytime user popup a new message.

But I found that in StreamResponse protocol there is only possible to return text and data. But there's no id. and the reason of generateId is not working because it means id is generated from frontend, but for the ai message, it's actually generated from backend. So it won't work.

Use Case

I have some features that will allow user to custimize ai chat message per different browser session. Like double click edit, mark, quote, and index sources. So it's important to keep id constant with backend.

Additional context

my current workaround

  onFinish: useCallback(
      (message: Message) => {
        // handle the AI message generated by the backend, because id is not returned in the response,
        //  we need to fetch the latest session to get the id of the last message
        getChatSessionFromBackend({
          sessionId,
        })
          .then((newSession) => {
            const lastAIMessage = newSession.chat_messages
              ?.sort((a, b) => (a.index > b.index ? 1 : -1))
              .at(-1);
            if (lastAIMessage?.role !== "assistant") {
              throw unexpected()
              return;
            } else {
              const id = lastAIMessage.id;
              storeMessage({ ...message, id }) // store message to indexeddb
              });
            }
          })
          .then(async () => {
            await refetch();
          })
      },
      [pipelineId, refetch],
    ),
@lgrammel lgrammel added enhancement New feature or request ai/ui labels Sep 19, 2024
@himself65
Copy link
Contributor Author

I think the streaming protocol should allow gives the id:id for the first chunk

@himself65 himself65 linked a pull request Sep 24, 2024 that will close this issue
Draft
@LucasYver
Copy link

LucasYver commented Oct 15, 2024 •
edited
Loading

Hey,

Can't you use the 2: custom JSON added by the user using Data using toDataStream ?

    const data = new StreamData()
    data.append('my-id')
    const reader = this.stream_result.toDataStream({ data }).getReader()

Like this the first chunk will be your custom json.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ai/ui enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: support generate id from server side himself65/ai
3 participants
@lgrammel @himself65 @LucasYver