Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Asynchronous Chat #1

Open
Fab8573 opened this issue Aug 29, 2024 · 2 comments
Open

Asynchronous Chat #1

Fab8573 opened this issue Aug 29, 2024 · 2 comments
Labels
question Further information is requested

Comments

@Fab8573
Copy link

Fab8573 commented Aug 29, 2024

Hello,

First, I would like to express my sincere thanks for sharing your source code.

Currently, I'm using the HemulGM wrapper for OpenAI, and your code will be invaluable as I integrate Mistral into my tests. With the HemulGM code, implementing asynchronous chat functionality is straightforward, as events are triggered when the result is ready. Specifically, these events include:

TOpenAIChat.OnChat: Triggered when the response is complete.

TOpenAIChat.OnBeginWork: Marks the start of the process.
TOpenAIChat.OnEndWork: Indicates the end of the process.
 => These events are particularly useful for displaying a wait cursor or similar indicators during processing

Can we imagine implementing something similar on your side?

Best regards,

@MaxiDonkey
Copy link
Owner

Hello,

First and foremost, thank you for your interest in this project. Your request is both insightful and greatly appreciated. I’ll make sure to add it to my list of pending tasks.

Currently, I am waiting for Mistral.AI to release the API extension for agent creation, which is a critical piece that I am eagerly anticipating.

In the meantime, if you’re looking to quickly establish an asynchronous mode, you could implement a "temporary" solution as follows:

Create methods:

procedure MyClass.DoOnWorkStart(Sender: TObject);
…
procedure MyClass.DoOnWorkEnd(Sender: TObject);
…
procedure MyClass.DoOnChat(Sender: TObject);
…

Using the first Chat (non-Stream) example, we can quickly create an asynchronous task as follows:

// system.Threading;

var Task: ITask := TTask.Create(
  procedure()
  begin
    TThread.Synchronize(TThread.Current,
      procedure
      begin
        var Chat := MistralAI.Chat.Create(
          procedure (Params: TChatParams)
          begin
            Params.Model('mistral-tiny');
            Params.Messages([TChatMessagePayload.User(Memo2.Text)]);
            Params.MaxTokens(1024);
          end);
        try
          for var Choice in Chat.Choices do
            Memo1.Lines.Add(Choice.Message.Content);
          DoOnChat(Chat);
        finally
          Chat.Free;
        end;
        DoOnWorkEnd(Self);
      end);
  end);
DoOnWorkStart(Self);
Task.Start;

I am aware that this is only a draft solution. I will dedicate more time to it as soon as I have finished the Wrapper for Delphi, which will allow the use of Anthropic APIs for its models.

Best regards,

@MaxiDonkey
Copy link
Owner

Hello,

A new version has been released to handle asynchronous chat (streamed and non-streamed).

Best regards

@MaxiDonkey MaxiDonkey pinned this issue Sep 17, 2024
@MaxiDonkey MaxiDonkey added the question Further information is requested label Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants