LiteAgent core Dart Server

English · 中文

LLM AI Agent multi session HTTP/WebSocket service

Feature

• Support OpenAPI/OpenRPC/OpenModbus/OpenTool JSON Spec.
• Support LLM Function calling to HTTP API/json-rpc 2.0 over HTTP/Modbus and more custom tools.
• HTTP Server wrapper Lite Agent core Dart
• Base on Lite Agent Core AgentService(DTO included), add Controller、Router, wrapper to HTTP/WS API.

Usage

1. Prepare

1. Some OpenSpec json file, according to /example/json/open*/*.json, which is callable.
2. Run your tool server, which is described in json file.
3. Add .env file in the example folder, and add below content in the .env file：

   baseUrl = https://xxx.xxx.com         # LLM API BaseURL
   apiKey = sk-xxxxxxxxxxxxxxxxxxxx      # LLM API ApiKey

2. Develop run server

1. debug or run mode run /bin/server.dart file main()

3. HTTP/WS API

• HTTP API
• WebSocket API
• Typical Interaction

3.1 HTTP API

• session control command, include：
  • /version：get version number, to confirm server running
  • /init：initial new session, server return session id
  • /history：get session messages not be cleared
  • /stop：stop session, when current message done, will not run next message
  • /clear：clear session context messages, and close websocket connection

BaseURL

• http://127.0.0.1:9527/api

[GET] /version

• Feature：get version number, to confirm server running

• Request params: empty

• Response body sample：

  {
      "version": "0.1.0"
  }

[POST] /init

• Feature: initial new session agent

• Request body：

  • LLM config: baseUrl, apiKey, model
  • System Prompt: Agent character, ToDo/NotToDo description
  • Tools Description: openapi、openmodbus Spec. According to third APIs in Spec to set apiKey or net
  • Timeout：3600 seconds in default. When agent stopped, massages context will be clear
  • Sample:

    {
        "llmConfig": {
            "baseUrl": "<LLM API baseUrl, e.g. https://api.openai.com/v1>",
            "apiKey": "<LLM API apiKey, e.g. sk-xxxxxxxxxx>",
            "model": "<LLM API model name, e.g. gpt-3.5-turbo. And temperature、maxTokens、topP can be changed below >",
            "temperature": 0,
            "maxTokens": 4096,
            "topP": 1
        },
        "systemPrompt": "<System Prompt. LLM character, capabilities, need to help user fixed what problems>",
        "openSpecList": [
            {
                "openSpec": "<tool spec json string, support openapi、openmodbus、openrpc>",
                "apiKey": {
                    "type": "<basic or bearer>",
                    "apiKey": "<Third APIs apiKey>"
                },
                "protocol": "Support openapi, openmodbus, jsonrpcHttp"
            },
            {
                "openSpec": "<Another spec json string, can be another protocol>",
                "protocol": "Support openapi, openmodbus, jsonrpcHttp"
            }
        ],
        "timeoutSeconds": 3600
    }

• Response body：

  • sessionId, will be used as session websocket subscribe, stop and clear operations.
  • Sample:

    {
        "id": "b2ac9280-70d6-4651-bd3a-45eb81cd8c30"
    }

[GET] /history?id=<sessionId>

• Response body:
  • agent messages context as list
  • Sample:

  [
  {
    "sessionId": "b2ac9280-70d6-4651-bd3a-45eb81cd8c30",
    "from": "system | user | agent | llm | tool",
    "to": "user | agent | llm | tool | client",
    "type": "text | imageUrl | functionCallList | toolReturn | contentList",
    "message": "<need to parse according type>",
    "completions": {
        "tokenUsage": {
          "promptTokens": 100,
          "completionTokens": 522,
          "totalTokens": 622
        },
        "id": "chatcmpl-9bgYkOjpdtLV0o0JugSmnNzGrRFMG",
        "model": "gpt-3.5-turbo"
     },
    "createTime": "2023-06-18T15:45:30.000+0800"
  }
  ]

[GET] /stop?id=<sessionId>

• Response body:
  • sessionId, to confirm the operation of the session
  • Sample:

  {
      "id": "b2ac9280-70d6-4651-bd3a-45eb81cd8c30"
  }

[GET] /clear?id=<sessionId>

• Response body:
  • sessionId, to confirm the operation of the session
  • Sample:

  {
      "id": "b2ac9280-70d6-4651-bd3a-45eb81cd8c30"
  }

3.2 WebSocket API

• Send and subscribe session AgentMessage

Endpoint

• ws://127.0.0.1:9527/api/chat?id=<sessionId>

3.2.1 alive

• client(ping) -> server: send "ping" to server
• client <- server(pong): respond "pong" to client

3.2.2 client send UserMessageDto List to server

• client([UserMessageDto]) -> server ：Wrap and send server
• Sample:

[
  {
    "type": "text",
    "message": "Get some tool status"
  }
]

3.2.3 server feedback AgentMessage to client

• client <- server(AgentMessage) ：server will keep sending AgentMessage to client
• Sample:

{
    "sessionId": "b2ac9280-70d6-4651-bd3a-45eb81cd8c30",
    "from": "system | user | agent | llm | tool",
    "to": "user | agent | llm | tool | client",
    "type": "text | imageUrl | functionCallList | toolReturn | contentList",
    "message": "<need to parse according type>",
    "completions": {
      "tokenUsage": {
        "promptTokens": 100,
        "completionTokens": 522,
        "totalTokens": 622
      },
      "id": "chatcmpl-9bgYkOjpdtLV0o0JugSmnNzGrRFMG",
      "model": "gpt-3.5-turbo"
    },
    "createTime": "2023-06-18T15:45:30.000+0800"
}

• According type to parse message
  • text、imageUrl：
    • String
    • Sample: "Tool result: PASS"
  • functionCallList：
    • Struct:

    [
        {
            "id":"<LLM respond id in function call>",
            "name":"<function name>",
            "parameters": "<LLM respond parameters in map>"
        }
    ]

    • Sample:

    [
      {
        "id":"call_z5FK2dAfU8TXzn61IJXzRl5I",
        "name":"SomeFunction",
        "parameters": {
          "operation":"result"
        }
      }
    ]

  • toolReturn：
    • Struct:

    {
      "id":"<LLM respond id in function call>",
      "result": <JSON Map, different tools in defferent result>
    }

    • Sample:

    {
      "id":"call_z5FK2dAfU8TXzn61IJXzRl5I",
      "result": {
        "statusCode":200,
        "body":"{\"code\":200,\"message\":\"PASS\"}"
      }
    }

  • contentList:
    • Struct:

    [
      {
        "type":"text | imageUrl",
        "message":"String"
      }
    ]

    • Sample:

    [
      {
        "type":"text",
        "message":"What’s in this image?"
      },
      {
        "type":"imageUrl",
        "message":"https://www.xxx.com/xxx.jpg"
      }
    ]

• When to=Client, message in below status:
  • "[TASK_START]"：agent receive user messages, and ready to run task
  • "[TOOLS_START]": ready to call Tools
  • "[TOOLS_DONE]": Tools return finished
  • "[TASK_STOP]"：agent receive stop or clear command, stop task
  • "[TASK_DONE]"：agent run task finished

3.3 Typical Interaction

[/init request] {llmConfig: ..., systemPrompt:..., openSpecList: [...]}
[/init response] {id: eccdacc8-a1a8-463f-b0af-7aebc278c842}
[After /chat connect ws, send userMessageDtoList] [{type: text, message: Get some tool status}]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 🤖AGENT -> 🔗CLIENT: [text] [TASK_START]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 👤USER -> 🤖AGENT: [text] Get some tool status
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 🤖AGENT -> 💡LLM: [text] Get some tool status
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 💡LLM -> 🤖AGENT: [functionCallList] [{"id":"call_73xLVZDe70QgLHsURgY5BNT0","name":"SomeFunction","parameters":{"operation":"result"}}]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 🤖AGENT -> 🔧TOOL: [functionCallList] [{"id":"call_73xLVZDe70QgLHsURgY5BNT0","name":"SomeFunction","parameters":{"operation":"result"}}]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 🤖AGENT -> 🔗CLIENT: [text] [TOOLS_START]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 🔧TOOL -> 🤖AGENT: [toolReturn] {"id":"call_73xLVZDe70QgLHsURgY5BNT0","result":{"statusCode":200,"body":"{\"code\":200,\"message\":\"FAIL\"}"}}
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 🤖AGENT -> 💡LLM: [toolReturn] {"id":"call_73xLVZDe70QgLHsURgY5BNT0","result":{"statusCode":200,"body":"{\"code\":200,\"message\":\"FAIL\"}"}}
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 🔧TOOL -> 🤖AGENT: [text] [TOOLS_DONE]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 🤖AGENT -> 🔗CLIENT: [text] [TOOLS_DONE]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 💡LLM -> 🤖AGENT: [text] Tool status: FAIL。
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 🤖AGENT -> 👤USER: [text] Tool status: FAIL。
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 🤖AGENT -> 🔗CLIENT: [text] [TASK_DONE]
[ws push] id: eccdacc8-a1a8-463f-b0af-7aebc278c842# 🤖AGENT -> 🔗CLIENT: [text] [TASK_STOP]
[/stop request] {id: eccdacc8-a1a8-463f-b0af-7aebc278c842}
[/clear request] {id: eccdacc8-a1a8-463f-b0af-7aebc278c842}
[ws close] WebSocket connection closed

Build and Run

1. Build in shell script:

   dart compile exe bin/server.dart -o build/lite_agent_core_dart_server

2. Then the lite_agent_core_dart_server file will be in build folder
3. Copy config.json file to lite_agent_core_dart_server same folder
4. Run in shell script:

   ./lite_agent_core_dart_server

5. Terminal will show:

   INFO: 2024-06-24 14:48:05.862057: PID 34567: [HTTP] Start Server - http://0.0.0.0:9527/api
   

6. After server running, will create log folder and agent.log file in the folder, to record server running logs.