{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":646174248,"defaultBranch":"main","name":"aiavatarkit","ownerLogin":"uezo","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-05-27T14:22:56.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/19625242?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1717837220.0","currentOid":""},"activityList":{"items":[{"before":null,"after":"0751ab8294dc38ce2fa340f7a8ba86746eaf7e00","ref":"refs/heads/v0.5.8","pushedAt":"2024-06-08T08:58:51.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Merge pull request #59 from uezo/add-video-device-for-vision\n\nAdd VideoDevice to control camera for vision input📷","shortMessageHtmlLink":"Merge pull request #59 from uezo/add-video-device-for-vision"}},{"before":"7b2381220c7d9f46e330a92c81d6acdd24369544","after":"0751ab8294dc38ce2fa340f7a8ba86746eaf7e00","ref":"refs/heads/main","pushedAt":"2024-06-08T08:55:22.000Z","pushType":"pr_merge","commitsCount":3,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Merge pull request #59 from uezo/add-video-device-for-vision\n\nAdd VideoDevice to control camera for vision input📷","shortMessageHtmlLink":"Merge pull request #59 from uezo/add-video-device-for-vision"}},{"before":null,"after":"606211367465c159c51686a0f8ffa6bdeeacf50a","ref":"refs/heads/add-video-device-for-vision","pushedAt":"2024-06-08T08:52:52.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"e9e55dc39c7e224f0300992a3cda4c8b2bf8ca6b","after":"7b2381220c7d9f46e330a92c81d6acdd24369544","ref":"refs/heads/main","pushedAt":"2024-06-08T07:25:25.000Z","pushType":"pr_merge","commitsCount":3,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Merge pull request #58 from uezo/support-vision-input-gemini-claude\n\nAdd vision input support for the Gemini and Claude","shortMessageHtmlLink":"Merge pull request #58 from uezo/support-vision-input-gemini-claude"}},{"before":"f6a163699beaae86f3d09e263bf066c837ad2a57","after":"503e61ec14501064c9a969ae300d5f024417dc57","ref":"refs/heads/support-vision-input-gemini-claude","pushedAt":"2024-06-08T07:24:27.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":null,"after":"f6a163699beaae86f3d09e263bf066c837ad2a57","ref":"refs/heads/support-vision-input-gemini-claude","pushedAt":"2024-06-08T07:17:39.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Add vision input support for the Gemini and Claude\n\nInstruct vision tag in the system message.\n\n```python\nsystem_message_content = \"\"\"\n### Using Vision\n\nIf you need an image to process a user's request, you can obtain it using the following methods:\n\n- screenshot\n- camera\n\nIf an image is needed to process the request, add an instruction like [vision:screenshot] to your response to request an image from the user.\n\nBy adding this instruction, the user will provide an image in their next utterance. No comments about the image itself are necessary.\n\nExample:\n\nuser: Look! This is the sushi I had today.\nassistant: [vision:screenshot] Let me take a look.\n\"\"\"\n```\n\nCreate instance of GeminiProcessor/ClaudeProcessor with this system message and set `True` to `use_vision`.\n\n```python\n# Gemini\nchat_processor_gemini = GeminiProcessor(\n api_key=GEMINI_API_KEY,\n model=\"gemini-1.5-pro-latest\",\n system_message_content=PROMPT,\n use_vision=True\n)\n\n# Claude\nchat_processor_claude = ClaudeProcessor(\n api_key=ANTHROPIC_API_KEY,\n system_message_content=PROMPT,\n model=\"claude-3-opus-20240229\",\n use_vision=True\n)\n```","shortMessageHtmlLink":"Add vision input support for the Gemini and Claude"}},{"before":"36ec88c5721de3fb065b3e3c49146c5bc792388d","after":"e9e55dc39c7e224f0300992a3cda4c8b2bf8ca6b","ref":"refs/heads/main","pushedAt":"2024-06-08T07:07:25.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Merge pull request #57 from uezo/fix-chatgpt-incorrect-context-when-vision\n\nFix bug that context is incorrect when using vision","shortMessageHtmlLink":"Merge pull request #57 from uezo/fix-chatgpt-incorrect-context-when-v…"}},{"before":null,"after":"aab4e8365efb9d58c9dc7fbee11fe480c0b9b50f","ref":"refs/heads/fix-chatgpt-incorrect-context-when-vision","pushedAt":"2024-06-08T07:07:08.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Fix bug that context is incorrect when using vision","shortMessageHtmlLink":"Fix bug that context is incorrect when using vision"}},{"before":"7e166e02628b5024178d13c3cce4ae6a3d65b94a","after":"36ec88c5721de3fb065b3e3c49146c5bc792388d","ref":"refs/heads/main","pushedAt":"2024-06-02T16:36:04.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Merge pull request #56 from uezo/support-dify-experimental\n\nAdd experimental support for Dify","shortMessageHtmlLink":"Merge pull request #56 from uezo/support-dify-experimental"}},{"before":null,"after":"dedc8abbcf74f564816da27a74ba2b5ae7f6dfba","ref":"refs/heads/support-dify-experimental","pushedAt":"2024-06-02T16:35:43.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Add experimental support for Dify\n\nYou can use the Dify API instead of a specific LLM's API. This eliminates the need to manage code for tools or RAG locally.\n\n```python\nfrom aiavatar import AIAvatar\nfrom aiavatar.processors.dify import DifyProcessor\n\nchat_processor_dify = DifyProcessor(\n api_key=DIFY_API_KEY,\n user=DIFY_USER\n)\n\napp = AIAvatar(\n google_api_key=GOOGLE_API_KEY,\n chat_processor=chat_processor_dify\n)\n\napp.start_listening_wakeword()\n```","shortMessageHtmlLink":"Add experimental support for Dify"}},{"before":"81f46ea7265c42dd1b069e2882a2bea064b8427e","after":"7e166e02628b5024178d13c3cce4ae6a3d65b94a","ref":"refs/heads/main","pushedAt":"2024-06-02T16:16:10.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Merge pull request #55 from uezo/support-chatgpt-vision\n\nAdd support for ChatGPT high-performance vision input👀","shortMessageHtmlLink":"Merge pull request #55 from uezo/support-chatgpt-vision"}},{"before":null,"after":"aaa1a85d91fa9fe90bafcd9ac022f6b3bf7b2b15","ref":"refs/heads/support-chatgpt-vision","pushedAt":"2024-06-02T16:15:35.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Add support for ChatGPT high-performance vision input👀\n\nAdd high-performance and dynamic vision control feature.\nTo use vision, implement `get_image()` and configure ChatGPTProcessor.\n\n```python\nimport io\nimport pyautogui\nfrom aiavatar.processors.chatgpt import ChatGPTProcessor\n\n# Implement get_image\nasync def get_image(source: str=None) -> bytes:\n buffered = io.BytesIO()\n image = pyautogui.screenshot(region=(0, 0, 1280, 720))\n image.save(buffered, format=\"PNG\")\n image.save(\"image_to_gemini.png\") # Save current image for debug\n return buffered.getvalue()\n\n# Configure ChatGPTProcessor\nchat_processor_gpt = ChatGPTProcessor(\n api_key=OPENAI_API_KEY,\n model=\"gpt-4o\",\n system_message_content=\"ユーザーからの要求を処理するために画像データが必要な場合、[vision:screenshot]を応答メッセージに含めてください。\\n\\n例\\n[vision:screenshot]承知しました。画像を確認しています。\"\n)\nchat_processor_gpt.use_vision = True\nchat_processor_gpt.get_image = get_image\n```","shortMessageHtmlLink":"Add support for ChatGPT high-performance vision input👀"}},{"before":"ea9de24fdf9a4b83125b02f1e2f3b7ee22214f88","after":"81f46ea7265c42dd1b069e2882a2bea064b8427e","ref":"refs/heads/main","pushedAt":"2024-06-01T03:25:21.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Merge pull request #54 from uezo/support-chatgpt-tool-call\n\nSupport Tools Call for ChatGPT","shortMessageHtmlLink":"Merge pull request #54 from uezo/support-chatgpt-tool-call"}},{"before":null,"after":"4c7f94f1f6591b10b3bba2ab6db6fee12a426894","ref":"refs/heads/support-chatgpt-tool-call","pushedAt":"2024-06-01T03:24:41.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Support Tools Call for ChatGPT\n\nUse tools call instead of function calling internally.","shortMessageHtmlLink":"Support Tools Call for ChatGPT"}},{"before":null,"after":"ea9de24fdf9a4b83125b02f1e2f3b7ee22214f88","ref":"refs/heads/v0.5.7","pushedAt":"2024-05-21T15:35:46.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"v0.5.7","shortMessageHtmlLink":"v0.5.7"}},{"before":"0f9ae80d0f36e37d5ad4ba36674207cdee68610f","after":"ea9de24fdf9a4b83125b02f1e2f3b7ee22214f88","ref":"refs/heads/main","pushedAt":"2024-05-21T15:35:26.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"v0.5.7","shortMessageHtmlLink":"v0.5.7"}},{"before":"53263951397f866a88e88d753d4c89eaa0b7137d","after":"0f9ae80d0f36e37d5ad4ba36674207cdee68610f","ref":"refs/heads/main","pushedAt":"2024-05-21T15:20:35.000Z","pushType":"pr_merge","commitsCount":3,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Merge pull request #53 from uezo/update-claude-processor\n\nAdd support for Claude function calling and vision input","shortMessageHtmlLink":"Merge pull request #53 from uezo/update-claude-processor"}},{"before":"cfe791d2a4d4cab0b0d9aaea0438b2f493d0adf1","after":"b550aa57e0c6a56b0d9815680b699b7012175361","ref":"refs/heads/update-claude-processor","pushedAt":"2024-05-21T15:18:39.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Add support for Claude vision input\n\nUsage:\n\n```python\nfrom aiavatar.processors.claude import ClaudeProcessorWithVisionBase\n\n# Create ChatProcessor that captures screen shot\nclass ClaudeProcessorWithVisionScreenShot(ClaudeProcessorWithVisionBase):\n async def get_image(self) -> bytes:\n buffered = io.BytesIO()\n image = pyautogui.screenshot(region=(0, 0, 1280, 720))\n image.save(buffered, format=\"PNG\")\n image.save(\"image_to_claude.png\") # Save current image for debug\n return buffered.getvalue()\n\nchat_processor_claude = ClaudeProcessorWithVisionScreenShot(\n api_key=YOUR_API_KEY\n)\n```","shortMessageHtmlLink":"Add support for Claude vision input"}},{"before":null,"after":"cfe791d2a4d4cab0b0d9aaea0438b2f493d0adf1","ref":"refs/heads/update-claude-processor","pushedAt":"2024-05-21T15:16:30.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Add support for Claude function calling\n\nUsage:\n\n```python\nfrom aiavatar.processors.claude import ClaudeProcessor, ClaudeFunction\n\n# Make function\nweather_func_claude = ClaudeFunction(\n name=\"get_current_weather\",\n description=\"Get the current weather in a given location\",\n input_schema={\n \"type\": \"object\",\n \"properties\": {\"location\": {\"type\": \"string\", \"description\": \"The city name of the location for which to get the weather.\"}},\n },\n func=weather\n)\n\n# Create ClaudeProcessor with functions\nchat_processor = ClaudeProcessor(\n api_key=YOUR_API_KEY,\n functions={\"get_current_weather\": weather_func_claude}\n)\n```","shortMessageHtmlLink":"Add support for Claude function calling"}},{"before":"8d8b4e72a93c829cf94ffa6bc74c04dc916f3836","after":null,"ref":"refs/heads/update-claude-processor","pushedAt":"2024-05-21T15:15:14.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"}},{"before":"03aa456b7baeb62f4e8a60910867faa74119b844","after":"8d8b4e72a93c829cf94ffa6bc74c04dc916f3836","ref":"refs/heads/update-claude-processor","pushedAt":"2024-05-21T15:09:45.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Add support for Claude vision input\n\nUsage:\n\n```python\nfrom aiavatar.processors.claude import ClaudeProcessorWithVisionBase\n\n# Create ChatProcessor that captures screen shot\nclass ClaudeProcessorWithVisionScreenShot(ClaudeProcessorWithVisionBase):\n async def get_image(self) -> bytes:\n buffered = io.BytesIO()\n image = pyautogui.screenshot(region=(0, 0, 1280, 720))\n image.save(buffered, format=\"PNG\")\n image.save(\"image_to_claude.png\") # Save current image for debug\n return buffered.getvalue()\n\nchat_processor_claude = ClaudeProcessorWithVisionScreenShot(\n api_key=YOUR_API_KEY\n)\n```","shortMessageHtmlLink":"Add support for Claude vision input"}},{"before":null,"after":"03aa456b7baeb62f4e8a60910867faa74119b844","ref":"refs/heads/update-claude-processor","pushedAt":"2024-05-21T15:07:18.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"v0.5.6","shortMessageHtmlLink":"v0.5.6"}},{"before":"03aa456b7baeb62f4e8a60910867faa74119b844","after":"53263951397f866a88e88d753d4c89eaa0b7137d","ref":"refs/heads/main","pushedAt":"2024-05-21T14:08:57.000Z","pushType":"pr_merge","commitsCount":4,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Merge pull request #52 from uezo/update-gemini-processor\n\nAdd support for ✨Gemini function calling and image input","shortMessageHtmlLink":"Merge pull request #52 from uezo/update-gemini-processor"}},{"before":"283d3d24490ebe5942e0594628918f97649e43b5","after":"af99c55ad7c99ad64b330d3e4cda31bf9e8b7b18","ref":"refs/heads/update-gemini-processor","pushedAt":"2024-05-21T14:05:47.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Add support for Gemini image input\n\nUsage:\n\n```python\n# Implement `get_image()`\nfrom aiavatar.processors.gemini import GeminiProcessorWithVisionBase\n\nclass GeminiProcessorWithVisionScreenShot(GeminiProcessorWithVisionBase):\n async def get_image(self) -> bytes:\n buffered = io.BytesIO()\n image = pyautogui.screenshot(region=(0, 0, 1280, 720))\n image.save(buffered, format=\"PNG\")\n image.save(\"image_to_gemini.png\") # Save current image for debug\n return buffered.getvalue()\n\n# Create ChatProcessor\nchat_processor_gemini = GeminiProcessorWithVisionScreenShot(\n api_key=GOOGLE_API_KEY\n)\n```","shortMessageHtmlLink":"Add support for Gemini image input"}},{"before":"f69407a70d5ceebfc7ea692ed58c38f2631e544f","after":"283d3d24490ebe5942e0594628918f97649e43b5","ref":"refs/heads/update-gemini-processor","pushedAt":"2024-05-21T13:59:20.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Add experimental support for Gemini function calling\n\nUsage\n\n```python\n# Make function\nasync def weather(location: str):\n return {\"location\": location, \"weahter\": random.choice([\"clear\", \"rain\", \"cloudy\"])}\n\nweather_func = GeminiFunction(\n name=\"get_current_weather\",\n description=\"Get the current weather in a given location\",\n parameters={\n \"type\": \"object\",\n \"properties\": {\"location\": {\"type\": \"string\", \"description\": \"The city name of the location for which to get the weather.\"}},\n },\n func=weather\n)\n\n# Create ChatProcessor with functions\nchat_processor_gemini = GeminiProcessor(\n api_key=YOUR_API_KEY,\n functions={\"get_current_weather\": weather_func}\n)\n```","shortMessageHtmlLink":"Add experimental support for Gemini function calling"}},{"before":null,"after":"f69407a70d5ceebfc7ea692ed58c38f2631e544f","ref":"refs/heads/update-gemini-processor","pushedAt":"2024-05-20T13:23:54.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Use system instructions","shortMessageHtmlLink":"Use system instructions"}},{"before":null,"after":"03aa456b7baeb62f4e8a60910867faa74119b844","ref":"refs/heads/v0.5.6","pushedAt":"2024-05-19T15:08:22.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"v0.5.6","shortMessageHtmlLink":"v0.5.6"}},{"before":"716b2e2f96f536b907b77d4c4c00a6506bfd34a3","after":"03aa456b7baeb62f4e8a60910867faa74119b844","ref":"refs/heads/main","pushedAt":"2024-05-19T15:07:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"v0.5.6","shortMessageHtmlLink":"v0.5.6"}},{"before":"bd36e8daaf0985ad99a2071ffc21bd2fef9b5991","after":"716b2e2f96f536b907b77d4c4c00a6506bfd34a3","ref":"refs/heads/main","pushedAt":"2024-05-19T14:52:05.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Small changes🐤","shortMessageHtmlLink":"Small changes🐤"}},{"before":"63a29fcd68126e5c52c06d39cb803f01302ad59e","after":"bd36e8daaf0985ad99a2071ffc21bd2fef9b5991","ref":"refs/heads/main","pushedAt":"2024-05-19T14:50:41.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"uezo","name":"uezo","path":"/uezo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19625242?s=80&v=4"},"commit":{"message":"Merge pull request #51 from uezo/add-vision-input-support-for-chatgpt\n\nAdd experimental support for vision input to ChatGPT","shortMessageHtmlLink":"Merge pull request #51 from uezo/add-vision-input-support-for-chatgpt"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEX9XZ_gA","startCursor":null,"endCursor":null}},"title":"Activity · uezo/aiavatarkit"}