Drop Node canvas for manual sharp conversion
#3221
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lohawk-azalea
pushed a commit
to azalea-gograbcode/anything-llm
that referenced
this pull request
Feb 17, 2025
* Patch missing folder autogenerate for plugins (Mintplex-Labs#2273) * bump ref to browser ext * bump Perplexity models (Mintplex-Labs#2275) * Make `userId` actually optional for workspaceThread endpoint (Mintplex-Labs#2276) * update doc links and readme * Support `@agent` custom skills (Mintplex-Labs#2280) * Support `@agent` custom skills * move import * Patch UI bug with agent skill web-search and sql-connector (Mintplex-Labs#2282) * Patch UI bug with agent skill * wrap call in try/catch for failures res?. optional call for settings since null is default * uncheck * Patch 11Labs selection UI bug (Mintplex-Labs#2284) * Patch 11Labs selection UI bug * remove log * 1943 add fireworksai support (Mintplex-Labs#2300) * Issue Mintplex-Labs#1943: Add support for LLM provider - Fireworks AI * Update UI selection boxes Update base AI keys for future embedder support if needed Add agent capabilites for FireworksAI * class only return --------- Co-authored-by: Aaron Van Doren <[email protected]> * Appearance setting for show/hide scroll bar on chat window (Mintplex-Labs#2187) * implement appearance setting for show/hide scrollbar * put back comments * revert backend for show_scrollbar * show scrollbar save to localstorage * old model function * lint * edit --------- Co-authored-by: Timothy Carambat <[email protected]> * Fix gitlab data connector for self-hosted instances (Mintplex-Labs#2315) (Mintplex-Labs#2316) * Fix gitlab data connector for self-hosted instances (Mintplex-Labs#2315) * Linting fix. * Add more verbose error messages in embed chat (Mintplex-Labs#2306) * publish embed updates * server sided error messages * publish embed chat widget * sync submodule * unset change * update embed to merged changes for error --------- Co-authored-by: timothycarambat <[email protected]> * Load all branches in gitlab data connector (Mintplex-Labs#2325) * Fix gitlab data connector for self-hosted instances (Mintplex-Labs#2315) * Linting fix. * Load all branches in the GitLab data connector Mintplex-Labs#2319 * Mintplex-Labs#2319 lint fixes. * update fetch on fail --------- Co-authored-by: BΕaΕΌej Owczarczyk <[email protected]> * Add ability to copy/paste images, files, and text from web, local, or otherwise (Mintplex-Labs#2326) * Fix custom domain in confluence (Mintplex-Labs#2328) confluence custom domain fix * Enable Mistral Multimodal (Mintplex-Labs#2343) * Enable Mistral Multimodal * remove console * Added JSONSchema for `plugin.json` files (Mintplex-Labs#2344) Added JSONSchema for agent skill plugin manifest files Signed-off-by: Jaid <[email protected]> * Make streaming behavior more natural (Mintplex-Labs#2336) * fix scrolling behavior + add cursor to streaming chats * lint * linting --------- Co-authored-by: timothycarambat <[email protected]> * Dont overwrite content in input on paste linting * Add select/unselect all context menu to directory component (Mintplex-Labs#2337) add select/unselect all context menu to directory component * PR#2355 Continued + expanded scope (Mintplex-Labs#2365) * Mintplex-Labs#2317 Fetch pinned documents once per folder to reduce the number of queries. * Reorder the lines to keeps const declarations together. * Add some comments to functions move pinned document fetch for folder to function move watched documents per-folder to also function the same remove unused function in documents model --------- Co-authored-by: BΕaΕΌej Owczarczyk <[email protected]> * Export embedded chat history (Mintplex-Labs#2329) export embedded chat history Co-authored-by: Timothy Carambat <[email protected]> * Bulk document removal from workspace * wip improve remove document ux * fix border ui bugs when adding files to workspace * sort workspacedirectory put adding files at top * fix workspace file row ui shifting * fix selected items bug when adding another item with items already selected on workspace * fix tooltip * lint * refactor * fix bug where unadding single item while selected would stay selected --------- Co-authored-by: timothycarambat <[email protected]> * Workspace agent autoselection (Mintplex-Labs#2357) * refactor agent to add fallback to workspace, then to chat provider/model * commenting update logic for bedrock and fireworks fallbacks --------- Co-authored-by: timothycarambat <[email protected]> * Support attachments in developer API (Mintplex-Labs#2373) * support attachments in developer api * lint --------- Co-authored-by: Timothy Carambat <[email protected]> * 1417 completion timeout (Mintplex-Labs#2374) * Refactor handleDefaultStreamResponseV2 function for better error handling * run yarn lint * small error handling changes * update error handling flow and scope of vars * add back space --------- Co-authored-by: Roman <[email protected]> * Support more Confluence URL formats (Mintplex-Labs#2118) * support more confluence url formats * use pattern matching for confluence urls and manual splitting as fallback * rework entire Confluence flow to prevent issues with custom, local, and cloud spaces * remove dep --------- Co-authored-by: Timothy Carambat <[email protected]> * Add dropdown for confluence connector deployment (Mintplex-Labs#2376) * Added an option to fetch issues from gitlab. Made the file fetching aβ¦ (Mintplex-Labs#2335) * Added an option to fetch issues from gitlab. Made the file fetching asynchornous to improve performance. Mintplex-Labs#2334 * Fixed a typo in loadGitlabRepo. * Convert issues to markdown. * Fixed an issue with time estimate field names in issueToMarkdown. * handle rate limits more gracefully + update checkbox to toggle switch * lint --------- Co-authored-by: Timothy Carambat <[email protected]> Co-authored-by: shatfield4 <[email protected]> * 1959 filetype filters (Mintplex-Labs#2378) * Updated the `GitHubRepoLoader` class to use the new import syntax and adjust the `recursiveLoader` method accordingly. * add @langchain/community to collector package.json * fix: Improve handling of complex ignore patterns in GitLabRepoLoader * refactor: use ignore package for simplified ignore logic * run yarn lint * add @langchain/community@^0.2.23 * remove unused dep lint --------- Co-authored-by: Emil Rofors (aider) <[email protected]> * Support DeepSeek (Mintplex-Labs#2377) * add deepseek support * lint * update deepseek context length * add deepseek to onboarding --------- Co-authored-by: Timothy Carambat <[email protected]> * Once again, modify Confluence to support every esoteric combination and undocumented way of running Confluence ever devised resolves Mintplex-Labs#2379 * Patch bug with pasted text not being detected (Mintplex-Labs#2386) * [FEAT] Add Llama 3.2 models to Fireworks AI's LLM selection dropdown (Mintplex-Labs#2384) Add Llama 3.2 3B and 1B models to Fireworks AI LLM selection * Added voyage-3 and voyage-3-lite. (Mintplex-Labs#2394) * Add 3GB file size limit to body parser middlewares (Mintplex-Labs#2390) * Tavily search web search agent support (Mintplex-Labs#2395) * support tavily search web search agent * lint * remove unneeded comments * Add exception handling for special case files like `Dockerfile` and `Jenkinsfile` (Mintplex-Labs#2410) * Support XLSX files (Mintplex-Labs#2403) * support xlsx files * lint * create seperate docs for each xlsx sheet * lint * use node-xlsx pkg for parsing xslx files * lint * update error handling --------- Co-authored-by: timothycarambat <[email protected]> * hide config for agent skills to manager (perm not allowed) (Mintplex-Labs#2469) * Integrate Apipie support directly (Mintplex-Labs#2470) resolves Mintplex-Labs#2464 resolves Mintplex-Labs#989 Note: Streaming not supported * LiteLLM agent support (Mintplex-Labs#2460) * litellm agent support * lint * add LiteLLM provider config --------- Co-authored-by: Timothy Carambat <[email protected]> * Pasting text bug fix (Mintplex-Labs#2425) pasting text bug fix Co-authored-by: Timothy Carambat <[email protected]> * Fix file name/folder name truncation (Mintplex-Labs#2456) fix filename/foldername truncation Co-authored-by: Timothy Carambat <[email protected]> * Daily message limit per user (Mintplex-Labs#2417) * set message limit per user * remove old limit user messages + unused admin page * fix daily message validation * refactor message limit input refactor canSendChat on user to a method on user model --------- Co-authored-by: timothycarambat <[email protected]> * Add backfilling on `query` for chat widget to improve UX (Mintplex-Labs#2482) * Support free-form input for workspace model for providers with no `/models` endpoint (Mintplex-Labs#2397) * support generic openai workspace model * Update UI for free form input for some providers --------- Co-authored-by: Timothy Carambat <[email protected]> * OpenAI o1 model support (Mintplex-Labs#2427) * support openai o1 models * Prevent O1 use for agents getter for isO1Model; --------- Co-authored-by: timothycarambat <[email protected]> * Tts open ai compatible endpoints (Mintplex-Labs#2487) * Update OpenAI TTS config to allow a custom BaseURL * uncheck config file * break openai generic TTS into its own provider * add space * hide TTS on user msg --------- Co-authored-by: Adam <[email protected]> * Handle Bedrock models that cannot use `system` prompts (Mintplex-Labs#2489) * Handle non-ascii characters in single and bulk link scraper URLs (Mintplex-Labs#2495) handle non-ascii characters in urls * linting * Passthrough agentModel for LMStudio (Mintplex-Labs#2499) * Patch `DNDFilePicker` useEffect callback (Mintplex-Labs#2506) * Add Groq vision preview support (Mintplex-Labs#2511) Adds support for only the llama3.2 vision models on groq. This comes with many conditionals and nuances to handle as Groqs vision implemention is quite bad right now * Feature/thread creation slug name (Mintplex-Labs#2512) * thread creation additional params name and slug, with api * typo fix * Rebuild openai Swagger docs Handle validations for fields to prevent invalid field inputs for .new Enforce sluggification of `slug` to prevent breaking of URL structs --------- Co-authored-by: abrakadobr <[email protected]> * Enable the ability to disable the chat history UI (Mintplex-Labs#2501) * Enable the ability to disable the chat history UI * forgot files * Fix tooling examples (Mintplex-Labs#2513) * Fix incorrect json API description. * small edits and validity checks * remove console.logs * unset and recheck changes --------- Co-authored-by: Adam <[email protected]> * Rework failover agent invocation selection for agent and ephemmeral agent (Mintplex-Labs#2515) * Rework failover agent invocation selection for agent and ephemmeral agent * update order of check * update order of check * lint * Github data connector improvements (Mintplex-Labs#2439) * fix tree/blob github urls from branches not being loaded * improve ux of github data connector * lint * patch Github URL parser to just validate with `URL` native parser * uncheck LocalStorage of PAT for security reasons --------- Co-authored-by: Timothy Carambat <[email protected]> * Add Grok/XAI support for LLM & agents (Mintplex-Labs#2517) * Add Grok/XAI support for LLM & agents * forgot files * bump readme * Alignment crime fixed (Mintplex-Labs#2528) fix alignment crime * Add filtering to sessionID for workspace chats (Mintplex-Labs#2531) * Patch `v1/document/upload` filename charset encoding (Mintplex-Labs#2535) * add `ino` filetype to text/plain support * patch bad reference for `EphemeralAgentHandler` * Simple SSO feature for login flows from external services (Mintplex-Labs#2553) * Simple SSO feature for login flows from external services * linting * add `sessionToken` validation connection auth for AWSbedrock (Mintplex-Labs#2554) * Add index leftover from prisma migration from PR Mintplex-Labs#2553 * Feature/update api example outputs (Mintplex-Labs#2547) * /v1/workspace/{slug} actually returns an array - update example output * update swagger example to show array output * Add optional limit and orderBy to dev api chat endpoint (Mintplex-Labs#2559) add optional limit and orderBy to dev api chat endpoint * Allow 127.0.0.1 as valid URL for scraping (Mintplex-Labs#2560) * allow 127.0.0.1 as valid url for scraping * update comments and lint --------- Co-authored-by: timothycarambat <[email protected]> * 2488 novita ai llm integration (Mintplex-Labs#2582) * feat: add new model provider: Novita AI * feat: finished novita AI * fix: code lint * remove unneeded logging * add back log for novita stream not self closing * Clarify ENV vars for LLM/embedder seperation for future Patch ENV check for workspace/agent provider --------- Co-authored-by: Jason <[email protected]> Co-authored-by: shatfield4 <[email protected]> * Add header static class for metadata assembly (Mintplex-Labs#2567) * Add header static class for metadata assembly * update comments * patch header parsing for links * DuckDuckGo web search agent skill support (Mintplex-Labs#2584) * duckduckgo web search agent skill support * move ddg to first option in dropdown menu * lint * add duckduckgo option stating no config required * bump @anthropic-ai/sdk to 0.32.1 * bump Anthropic models * Patch VoyageAI implementation from LC resolves Mintplex-Labs#2593 * Added translation dutch (nl) (Mintplex-Labs#2595) * Create common.js Dutch translation * Update resources.js Add NL * Update GitLab PAT token url (Mintplex-Labs#2598) Update gitlab PAT token url for Gitlab * Fix the problem of web-searching can not complete searching sessions when using bing search. (Mintplex-Labs#2611) Fix the problem of web-searching can not complete searching sessions. * Handle OpenAI whisper transcription edge case (Mintplex-Labs#2621) remove openai whisper transcription provider response_format option * Fix chat window scrollbar appearance option (Mintplex-Labs#2589) * wip fix chat window scrollbar * fix chat/default chat window scrollbare option * lint * small lint and refactor --------- Co-authored-by: timothycarambat <[email protected]> * linting * Add undo/redo functionality Mintplex-Labs#2591 (Mintplex-Labs#2623) * initial work - undo works except typed keys * working but clunky code * single letter and paste with no selection working * add comments and keep the previous selection * optimizations + add redo feature * linting --------- Co-authored-by: Mr Simon C <[email protected]> Co-authored-by: shatfield4 <[email protected]> * Frontend performance improvements (Mintplex-Labs#2627) * Frontend performance improvements * test docker build * Patch bad models endpoint path in LM Studio embedding engine (Mintplex-Labs#2628) * patch bad models endpoint path in lm studio embedding engine * convert to OpenAI wrapper compatibility * add URL force parser/validation for LMStudio connections * remove comment --------- Co-authored-by: timothycarambat <[email protected]> * update onboarding survery endpoint * Add external app section in README closes Mintplex-Labs#2639 * Bump TogetherAI models (Mintplex-Labs#2645) * bump together ai models * Run post-bump command --------- Co-authored-by: timothycarambat <[email protected]> * Light/dark mode UI overhaul (Mintplex-Labs#2629) * Refactor workspace sidebar component styles (Mintplex-Labs#2380) rely on css for conditional styles * New sidebar colors (Mintplex-Labs#2381) new sidebar colors * Main container color update (Mintplex-Labs#2382) * Dark mode setup themes (Mintplex-Labs#2411) * setup generic tailwind theme + ability to add new themes * add theme context * use correct colors from design for sidebar + fix padding * Settings sidebar UI updates (Mintplex-Labs#2416) settings sidebar ui updates * fix sidebar resizing/truncate issue on hover * Dark mode chat window (Mintplex-Labs#2443) * Support XLSX files (Mintplex-Labs#2403) * support xlsx files * lint * create seperate docs for each xlsx sheet * lint * use node-xlsx pkg for parsing xslx files * lint * update error handling --------- Co-authored-by: timothycarambat <[email protected]> * wip chat window * ux+ux improvements and update new colors * chat window dark mode * remove comment --------- Co-authored-by: timothycarambat <[email protected]> * Dark mode welcome page (Mintplex-Labs#2444) * dark mode welcome page styles + refactor * remove AI_BACKGROUND_COLOR and USER_BACKGROUND_COLOR constants * Dark mode UI for admin and tools pages + mobile view improvements (Mintplex-Labs#2454) * dark mode ui for admin and tools pages + mobile view improvements * lint * ai provider pages + options darkmode ui * placeholder generic class * appearance settings styles * ai providers mobile margin * dark mode styles for agent skills + experimental features * mobile styles on security settings * fine tune flow ui dark mode * workspace settings page * lint * Dark mode onboarding (Mintplex-Labs#2461) dark mode onboarding * update all modals + normalize styles (Mintplex-Labs#2471) * lint * Dark mode privacy & experimental pages (Mintplex-Labs#2479) * document watch + privacy pages ui + mobile modal darkmode * lint * Dark mode login screens (Mintplex-Labs#2483) * multi-user auth screen ui update * dark mode login screen + recovery key modals * remove unneeded import * Workspace preset commands modals dark mode (Mintplex-Labs#2484) update workspace preset dark mode modal * Document pinning modal ui update (Mintplex-Labs#2490) document pinning modal ui update * Experimental agreement modal dark mode (Mintplex-Labs#2491) experimental agreement modal dark mode * Serp options dark mode (Mintplex-Labs#2492) serp options dark mode * field fixes * attempt light mode wip * setting sidebar * Toasts and threads * main page content and privacy page * force rewrite for light * add border for light mode rightside content * more fixes * wip * wip * wip light mode implementation * wip dark light mode file picker * document picker light mode ui * slight ui tweaks * light mode fine tuning flow * light mode tweaks + qa fixes * fix md rendering of light mode + tooltip fixes * lint * qa bug fixes * Add developer hook for theme move provider to outmost layer * qa light mode bug fixes * Linting and hotfixes for UI * Light mode to dev * accept invite light mode ui fix * Fix onboarding inputs in dark mode * fix close icons last minute items * patch z-index on tooltips * patch light mode citations --------- Co-authored-by: timothycarambat <[email protected]> * update uploader status containers for new theme * show error in upload * add back timeout for documents to fade out * UI light QA * update light/dark codeblock styles for visibility patch embed modal text invisible patch toggle colors * remove dupe key in ModelMap * refresh theme without reloading page * Pr 2660 branch (Mintplex-Labs#2662) * Add a basic Traditional Chinese(zh_TW) locale * patch lang code --------- Co-authored-by: Peter Dave Hello <[email protected]> * Add more expiermental models from Gemini (Mintplex-Labs#2663) * Add plaintext file extensions (Mintplex-Labs#2664) * TogetherAI Llama 3.2 vision models support (Mintplex-Labs#2666) * togetherai llama 3.2 vision models support * remove console log * fix listing to reflect what is on the chart --------- Co-authored-by: timothycarambat <[email protected]> * Add gemini-exp-1121 resolves Mintplex-Labs#2657 * Mistral embedding engine support (Mintplex-Labs#2667) * add mistral embedding engine support * remove console log + fix data handling onboarding * update data handling description --------- Co-authored-by: Timothy Carambat <[email protected]> * Pr 2653 branch (Mintplex-Labs#2672) * feat: initial setup for Dependabot * remove embed folder check --------- Co-authored-by: Philipp Eckel <[email protected]> * feat: support setting maxConcurrentChunks for Generic OpenAI embedder (Mintplex-Labs#2655) * exposes `maxConcurrentChunks` parameter for the generic openai embedder through configuration. This allows setting a batch size for endpoints which don't support the default of 500 * Update new field to new UI make getting to ensure proper type and format --------- Co-authored-by: timothycarambat <[email protected]> * Adjustable font size in chat input (Mintplex-Labs#2692) * adjustable prompt input text sizing * dev build --------- Co-authored-by: timothycarambat <[email protected]> * β¨ feat: Add Vietnamese translations and configuration (Mintplex-Labs#2641) * Create common.js update file common.js language vietnam * Update resources.js * Update common.js * Update resources.js * Update common.js * Update resources.js --------- Co-authored-by: Timothy Carambat <[email protected]> * patch baseURL in builder for logo * linting * add gemini-exp-1121 to expiermental set * fix mistral key on embedder selection * Fix typo with GCP deploy guideline (Mintplex-Labs#2718) * Fix markdown issue where type is missing in backticks (Mintplex-Labs#2719) * Normalize Input Fields (Mintplex-Labs#2720) * WIP field normalization * more norms * fix advanced setting fields hover onlight mode * Community hub integration (Mintplex-Labs#2555) * wip hub connection page fe + backend * lint * implement backend for local hub items + placeholder endpoints to fetch hub app data * fix hebrew translations * revamp community integration flow * change sidebar * Auto import if id in URL param remove preview in card screen and instead go to import flow * get user's items + team items from hub + ui improvements to hub settings * lint * fix merge conflict * refresh hook for community items * add fallback for user items * Disable bundle items by default on all instances * remove translations (will complete later) * loading skeleton * Make community hub endpoints admin only show visibility on items combine import/apply for items to they are event logged for review * improve middleware and import flow * community hub ui updates * Adjust importing process * community hub to dev * Add webscraper preload into imported plugins * add runtime property to plugins * Fix button status on imported skill change show alert on skill change Update markdown type and theme on import of agent skill * update documentaion paths * remove unused import * linting * review loading state --------- Co-authored-by: Timothy Carambat <[email protected]> * Patch light mode on Community Hub import flow * Community hub integration UX improvements (Mintplex-Labs#2727) * add empty states to your account hub page + disconnect button for api key * lint * lint --------- Co-authored-by: timothycarambat <[email protected]> * patch button normalization for desktop builds * Add Support for NVIDIA NIM (Mintplex-Labs#2766) * Add Support for NVIDIA NIM * update README * linting * persist token window for NIM and not only on model change * patch docs with proper input for OpenAI compatible embeddings resolves Mintplex-Labs#2801 * update chat model filters for openai (Mintplex-Labs#2803) * add gemini 1206 and gemini-2.0-flash exp models connect Mintplex-Labs#2788 * Fixed a typo in README (Mintplex-Labs#2805) * Fix scraping failed bug in link/bulk link scrapers (Mintplex-Labs#2807) * fix scraping failed bug in link/bulk link scrapers * reset submodule * swap to networkidle2 as a safe mix for SPA and API-loaded pages, but also not hang on request heavy pages * lint --------- Co-authored-by: timothycarambat <[email protected]> * Fix timestamp bug in embed widget (Mintplex-Labs#2806) * fix timestamp bug in embed widget * bump embed for formatDate fallback * bump embed module --------- Co-authored-by: timothycarambat <[email protected]> * silence dependabot for now * patch gemini-2.0-key * Redirect to default thread on thread deletion (Mintplex-Labs#2808) * redirect to default thread on thread deletion * reset submodule * handle redirect on bulk delete thread * redirect on bulk thread delete only if deleted thread is selected * redirect only on single delete of active thread * lint * patch threadSlug for clarity --------- Co-authored-by: Timothy Carambat <[email protected]> * Add support for hiding sidebar (Mintplex-Labs#2809) * Add support for hiding sidebar Support cmd/ctrl + shift +s for quick hide/show * patch sidebar padding on collapse * update Key for sidebar setting * Add vector search API endpoint (Mintplex-Labs#2815) * Add vector search API endpoint * Add missing import * Modify the data that is returned * Change similarityThreshold to scoreThreshold As this is what is actually returned by the search * Removing logging (oops!) * chore: regen swagger docs for new endpoint fix: update function to sanity check values to prevent crashes during search --------- Co-authored-by: Scott Bowler <[email protected]> * patch ZH typo (Mintplex-Labs#2816) * add Gitlab to watchable documents (Mintplex-Labs#2817) * Fix(2753): Replace React Router Navigate with window.location for SSO state refresh (Mintplex-Labs#2818) * fix(2753): update SSO redirect to use window.location.replace * fix(2753): remove unused import --------- Co-authored-by: Timothy Carambat <[email protected]> * Pin all docker-scout CI versions to 1.15.1 for now * APIPie LLM provider improvements (Mintplex-Labs#2695) * fix apipie streaming/sort by chat models * lint * linting --------- Co-authored-by: timothycarambat <[email protected]> * Add thread name to workspaces get endpoint (Mintplex-Labs#2822) add thread name to workspaces get endpoint Co-authored-by: Timothy Carambat <[email protected]> * repatch scout cli * Add new workspace user management endpoint (Mintplex-Labs#2842) deprecate old endpoint which users workspace ID vs slug resolves Mintplex-Labs#2838 * Add attachments to GenericOpenAI prompt (Mintplex-Labs#2831) * added attachments to genericopenai prompt * add devnote --------- Co-authored-by: timothycarambat <[email protected]> * Purge cached docs and remove docs from all workspaces on vectorDB/embedder changes (Mintplex-Labs#2819) * wip remove all docs clear vector db on embedder/vector db change * purge all cached docs and remove docs from workspaces on vectordb/embedder change * lint * remove unneeded console log * remove reset vector stores endpoint and move to server side updateENV with postUpdate check * reset embed module * remove unused import * simplify deletion process rescoped document deletion to be more general for speed, everything needs to be reset anyway fixed issue where unembedded docs not in any workspaces, but cached, were not removed * add back missing readme file update warning text modals --------- Co-authored-by: timothycarambat <[email protected]> * remove dependabot * LLM performance metric tracking (Mintplex-Labs#2825) * WIP performance metric tracking * fix: patch UI trying to .toFixed() null metric Anthropic tracking migraiton cleanup logs * Apipie implmentation, not tested * Cleanup Anthropic notes, Add support for AzureOpenAI tracking * bedrock token metric tracking * Cohere support * feat: improve default stream handler to track for provider who are actually OpenAI compliant in usage reporting add deepseek support * feat: Add FireworksAI tracking reporting fix: improve handler when usage:null is reported (why?) * Add token reporting for GenericOpenAI * token reporting for koboldcpp + lmstudio * lint * support Groq token tracking * HF token tracking * token tracking for togetherai * LiteLLM token tracking * linting + Mitral token tracking support * XAI token metric reporting * native provider runner * LocalAI token tracking * Novita token tracking * OpenRouter token tracking * Apipie stream metrics * textwebgenui token tracking * perplexity token reporting * ollama token reporting * lint * put back comment * Rip out LC ollama wrapper and use official library * patch images with new ollama lib * improve ollama offline message * fix image handling in ollama llm provider * lint * NVIDIA NIM token tracking * update openai compatbility responses * UI/UX show/hide metrics on click for user preference * update bedrock client --------- Co-authored-by: shatfield4 <[email protected]> * Append writable fields to dev API new workspace endpoint (Mintplex-Labs#2843) * add writible fields to dev api new workspace endpoint * lint * implement validations for workspace model * update swagger comments * simplify validations for workspace on frontend and API * cleanup validations --------- Co-authored-by: Timothy Carambat <[email protected]> * Enable `default` roles to be able to use `attachments` but not upload documents (Mintplex-Labs#2860) * Enable default users to be able to add attachments, but not files to chats * Add support for gemini authenticated models endpoint (Mintplex-Labs#2868) * Add support for gemini authenticated models endpoint add customModels entry add un-authed fallback to default listing separate models by expiermental status resolves Mintplex-Labs#2866 * add back improved logic for apiVersion decision making * Remove fine-tuning flow (Mintplex-Labs#2872) remove fine-tuning flow * fix wrong metadata assignment in MilvusProvider (Mintplex-Labs#2870) fixed wrong metadata assignment * update ollama performance mode (Mintplex-Labs#2874) * Fix light mode table colors (Mintplex-Labs#2873) * fix light mode table colors * fix table styles from being overwritten in other sections that are not chat messages --------- Co-authored-by: Timothy Carambat <[email protected]> * more table styles for lightmode * Fix styles of ChangeWarningModal (Mintplex-Labs#2876) fix styles of ChangeWarningModal * Patch gemini dropdown to model id since names are wrong for the different versions connect Mintplex-Labs#2898 * enable leftover mlock setting * Show scrollbar on rendered tables in chat (Mintplex-Labs#2884) show scrollbar on rendered tables in chat Co-authored-by: Timothy Carambat <[email protected]> * Fix typo in frontend for SearXNG agent setup (Mintplex-Labs#2917) fix typo in frontend for SearXNG agent setup * Normalize paths on files uploaded to prevent arbitrary file writes (Mintplex-Labs#2905) * normalize paths on files uploaded to prevent arbitrary file writes * force normalize path in string parse --------- Co-authored-by: timothycarambat <[email protected]> * Patch unauthorized access to other user's pfps (Mintplex-Labs#2904) * patch unauthorized viewing of other user's pfps * inline return responses --------- Co-authored-by: Timothy Carambat <[email protected]> * Audio file validations (Mintplex-Labs#2902) * add audio file validations * patch sharp to support wavfile parsing --------- Co-authored-by: timothycarambat <[email protected]> * Patch custom models endpoint (Mintplex-Labs#2903) * prevent non admin users from changing llm settings via custom-models endpoint * permission middleware to JSDOC --------- Co-authored-by: timothycarambat <[email protected]> * Add support for Google Generative AI (Gemini) embedder (Mintplex-Labs#2895) * Add support for Google Generative AI (Gemini) embedder * Add missing example in docker Fix UI key elements in options Add Gemini to data handling section Patch issues with chunk handling during embedding * remove dupe in env --------- Co-authored-by: timothycarambat <[email protected]> * Return timestamps in OpenAI compat endpoints in proper format (Mintplex-Labs#2921) * support attachments via thread API chat/stream-chat endpoints * resolves Mintplex-Labs#2926 floor ms value * Update README.md (Mintplex-Labs#2924) * make `similarityResponse` object arguments and not positional (Mintplex-Labs#2930) * make `similarityResponse` object arguments and not positional * reuse client for qdrant * Reranker option for RAG (Mintplex-Labs#2929) * Reranker WIP * add cacheing and singleton loading * Add field to workspaces for vectorSearchMode Add UI for lancedb to change mode update all search endpoints to pass in reranker prop if provider can use it * update hint text * When reranking, swap score to rerank score * update optchain * Update example ENV files (Mintplex-Labs#2931) update example env files * Migrate CDN download URL from S3 bucket * patch logo var typo * Add fallback URL for reranking model * patch unnessecary scrollbars resolves Mintplex-Labs#2943 * feat: add support for voyage-3-large and voyage-code-3 embedding models (Mintplex-Labs#2955) * feat: add support for voyage-3-large and voyage-code-3 embedding models - Add voyage-3-large and voyage-code-3 to VoyageAiOptions dropdown - Update getMaxEmbeddingLength to support 32k context for new models - Update .env.example with new model options * unset env example --------- Co-authored-by: timothycarambat <[email protected]> * patch scrollbars - with regress Mintplex-Labs#2943 * update .nvmrc * Add caching to Gemini /models (Mintplex-Labs#2969) rename file typo * patch gemini embedding key preference resolves Mintplex-Labs#2972 * add `index` to response object for marshalling resolves Mintplex-Labs#2956 * Support historical message image inputs/attachments for n+1 queries (Mintplex-Labs#2919) * Support historical message image inputs/attachments for n+1 queries * patch gemini * OpenRouter vision support cleanup * xai vision history support * Mistral logging --------- Co-authored-by: shatfield4 <[email protected]> * Swap to GH `arm64` runner (Mintplex-Labs#2980) * test arm runner speed * test arm runner speed with arm * update primary worker flows * update dev build name * remove QEMU step * Fix similarity score bug in lance/chroma dbs (Mintplex-Labs#2986) * fix similarity score bug in lance/chroma dbs * batch lower bound case --------- Co-authored-by: timothycarambat <[email protected]> * Farsi (Persian) translations (Mintplex-Labs#2989) * Initial commit | Farsi (Persian) translations * Farsi (Persian) translations - Pt. 2 * Farsi (Persian) translations - Pt. 3 * Farsi (Persian) translations - `README.fa-IR.md` file * remove greptile link * linting * wip agent ui animation (Mintplex-Labs#2999) * wip agent ui animation * WIP agent ui revision * linting * simplify css * memoize agent responses --------- Co-authored-by: shatfield4 <[email protected]> * Display `thinking` in the prompt response - model agnostic (Mintplex-Labs#3001) * CoT Display * forgot file * preformance optimizations * match agent ui on thinking model ui when collapsed * style cleanup * spacing fixes --------- Co-authored-by: shatfield4 <[email protected]> * Fix stop generation button UI (Mintplex-Labs#3011) * fix stop generation button ui * keep action color theme and match rest of the UI rescale button to be consistent with UI --------- Co-authored-by: timothycarambat <[email protected]> * Revert StatusResponse Changes due to crashing frontend * Bump perplexity models (Mintplex-Labs#3014) * bump perplexity models --------- Co-authored-by: Timothy Carambat <[email protected]> * linting * Dynamic fetching of TogetherAI models (Mintplex-Labs#3017) * implement dynamic fetching of togetherai models * implement caching for togetherai models * update gitignore for togetherai model caching * Remove models.json from git tracking * Remove .cached_at from git tracking * lint * revert unneeded change --------- Co-authored-by: Timothy Carambat <[email protected]> * Add Version to AzureOpenAI (Mintplex-Labs#3023) * Allow editing of LLM settings from workspace settings (Mintplex-Labs#2865) * wip change workspace llm settings * allow editing of workspace llm and agent config inside workspace settings * lint + put back deleted comment --------- Co-authored-by: Timothy Carambat <[email protected]> * feat: update novita AI logo and default model (Mintplex-Labs#3037) * [Chore] Add better data-handling for unknown providers (Mintplex-Labs#3026) Add better data-handling for unknown providers * breakout latex plugin for delims (Mintplex-Labs#3040) * Breakout LaTeX plugin for modification * backport regular markdown link * Remove native LLM option (Mintplex-Labs#3024) * remove native llm * remove node-llama-cpp from dockerfile * remove unneeded items from dockerfile --------- Co-authored-by: Timothy Carambat <[email protected]> * Add ability to disable default agent skills (Mintplex-Labs#3041) * Add ability to disable default agent skills * debug build * rever ubuntu arm runner - fickle and fails often * Update NVIDIA branding * Agent UI animations (Mintplex-Labs#3025) * wip agent ui animation * WIP agent ui revision * linting * simplify css * memoize agent responses * patch hook memo issue * dev build --------- Co-authored-by: shatfield4 <[email protected]> * Patch gear showing on llms with no settings * Enable `include_reasoning` in request (Mintplex-Labs#3067) * parse streams from `reasoning_content` from DeepSeek API (Mintplex-Labs#3068) * Add tokenizer improvments via Singleton class and estimation (Mintplex-Labs#3072) * Add tokenizer improvments via Singleton class linting * dev build * Estimation fallback when string exceeds a fixed byte size * Add notice to tiktoken on backend * autodetect parseable text file contents (Mintplex-Labs#3079) * Fix inifite loading when bad file is uploaded alone in uploader * o3 model patch * inherit font size from UI for thoughts (Mintplex-Labs#3099) * bump cdn * Add patch for `o#` models on Azure connect Mintplex-Labs#3023 Note: depends on user naming the deployment correctly. * Enable consistent styling on chart items (Mintplex-Labs#3126) * Add reasoning flag for azure models with "default" fallback (Mintplex-Labs#3128) * Enable `num_ctx` to match defined chunk length in ollama embedder (Mintplex-Labs#3129) * Enable `num_ctx` to match defined chunk length in ollama embedder * remove console * Patch PPLX streaming for timeouts (Mintplex-Labs#3130) Add in-text citations as well for PPLX token streaming handle timeouts for stream/buffer hanging * Add improved logging and introspection to web-browsing (Mintplex-Labs#3140) * [FEAT] Add Turkish Language Support (tr.js) Mintplex-Labs#3108 (Mintplex-Labs#3141) * Patch light mode remove on SQL Connection resolves Mintplex-Labs#3160 * linting * [CHORE] Onboarding language translations (Mintplex-Labs#3176) * wip onboarding translations * add normalized EN reference for languages * add zh/zh-tw translations * Fix translations * fix UserMenu rendered twice on Main page (Mintplex-Labs#3110) fix: fix UserMenu rendered twice on Main page * Add handling to reasoning models for Generic OpenAI connector (Mintplex-Labs#3183) * Add handling to resoning models for Generic OpenAI connector resolves Mintplex-Labs#3177 * linting * Patch code block header oddity resolves Mintplex-Labs#3117 * Agent flow builder (Mintplex-Labs#3077) * wip agent builder * refactor structure for agent builder * improve ui for add block menu and sidebar * lint * node ui improvement * handle deleting variable in all nodes * add headers and body to apiCall node * lint * Agent flow builder backend (Mintplex-Labs#3078) * wip agent builder backend * save/load agent tasks * lint * refactor agent task to use uuids instead of names * placeholder for run task * update frontend sidebar + seperate backend to agent-tasks utils * lint * add deleting of agent tasks * create AgentTasks class + wip load agent tasks into aibitat * lint * inject + call agent tasks * wip call agent tasks * add llm instruction + fix api calling blocks * add ui + backend for editing/toggling agent tasks * lint * add back middlewares * disable run task + add navigate to home on logo click * implement normalizePath to prevent path traversal * wip make api calling more consistent * lint * rename all references from task to flow * patch load flow bug when on editing page * remove unneeded files/comments * lint * fix delete endpoint + rename load flows * add move block to ui + fix api-call backend + add telemetry * lint * add web scraping block * only allow admin for agent builder --------- Co-authored-by: timothycarambat <[email protected]> * Move AgentFlowManager flows to static simplify UI states Handle LLM prompt flow when provided non-string * delete/edit menu for agent flow panel + update flow icon * lint * fix open builder button hidden bug * add tooltips to move up/down block buttons * add tooltip to delete block * truncate block description to fit on blocklist component * light mode agent builder sidebar * light mode api call block * fix light mode styles for agent builder blocks * agent flow fetch in UI * sync delete flow * agent flow ui/ux improvements * remove unused AgentSidebar component * comment out /run * UI changes and updates for flow builder * format flow panel info * update link handling * ui tweaks to header menu * remove unused import * update doc links update block icons * bump readme * Patch code block header oddity resolves Mintplex-Labs#3117 * bump dev image --------- Co-authored-by: Timothy Carambat <[email protected]> * patch flow link * chore: rename Github to GitHub (Mintplex-Labs#3199) * chore: rename Github to GitHub Signed-off-by: Adam Setch <[email protected]> * chore: rename Github to GitHub Signed-off-by: Adam Setch <[email protected]> * Undo some code changes for references --------- Signed-off-by: Adam Setch <[email protected]> Co-authored-by: timothycarambat <[email protected]> * update dead link resolves Mintplex-Labs#3201 * chore: rename `Gitlab` to `GitLab` (Mintplex-Labs#3200) * chore: rename `Gitlab` to `GitLab` Signed-off-by: Adam Setch <[email protected]> * undo code changes - breaks refs --------- Signed-off-by: Adam Setch <[email protected]> Co-authored-by: timothycarambat <[email protected]> * Add `querySelectorAll` capability to web-scraping block (Mintplex-Labs#3186) * Add `querySelectorAll` capability to web-scraping block * patches and fallbacks * fix styles of text in web scraping block --------- Co-authored-by: shatfield4 <[email protected]> * Turkish translations (Mintplex-Labs#3207) * OCR PDFs as fallback during upload (Mintplex-Labs#3204) * OCR PDFs as fallback in spawn thread * wip * build our own worker fanout and wrapper * norm pkgs * bump dev * Add OCR of image support (Mintplex-Labs#3219) * OCR PDFs as fallback in spawn thread * wip * build our own worker fanout and wrapper * norm pkgs * Add image OCR support * feat(dataconnectors): support confluence personal access token (Mintplex-Labs#3206) * feat(dataconnectors): support confluence personal access token * fix: change select option * linting change name on accesstype field --------- Co-authored-by: timothycarambat <[email protected]> * fix agent builder move block bug (Mintplex-Labs#3220) patch bug allowing user to move block above vars in agent builder * Drop Node `canvas` for manual `sharp` conversion (Mintplex-Labs#3221) * Drop Node `canvas` for manual `sharp` conversion * bump dev * sync-fork-250217 --------- Signed-off-by: Jaid <[email protected]> Signed-off-by: Adam Setch <[email protected]> Co-authored-by: Timothy Carambat <[email protected]> Co-authored-by: Aaron Van Doren <[email protected]> Co-authored-by: Sean Hatfield <[email protected]> Co-authored-by: Blazej Owczarczyk <[email protected]> Co-authored-by: Jaid <[email protected]> Co-authored-by: Roman <[email protected]> Co-authored-by: Emil Rofors (aider) <[email protected]> Co-authored-by: a4v2d4 <[email protected]> Co-authored-by: Adam <[email protected]> Co-authored-by: abrakadobr <[email protected]> Co-authored-by: Location <[email protected]> Co-authored-by: Mr Simon C <[email protected]> Co-authored-by: Jason <[email protected]> Co-authored-by: Melroy <[email protected]> Co-authored-by: Gabriel Le Breton <[email protected]> Co-authored-by: WinSun <[email protected]> Co-authored-by: Peter Dave Hello <[email protected]> Co-authored-by: Philipp Eckel <[email protected]> Co-authored-by: hdelossantos <[email protected]> Co-authored-by: Debugs <[email protected]> Co-authored-by: Tuan Nguyen <[email protected]> Co-authored-by: Logan Reeder <[email protected]> Co-authored-by: Scott Bowler <[email protected]> Co-authored-by: Andy Santana <[email protected]> Co-authored-by: wolfganghuse <[email protected]> Co-authored-by: Chaiwat Saithongcum <[email protected]> Co-authored-by: Shixian Sheng <[email protected]> Co-authored-by: ReindeerFlotilla <[email protected]> Co-authored-by: okaeiz <[email protected]> Co-authored-by: TunahanDilercan <[email protected]> Co-authored-by: Sander de Leeuw <[email protected]> Co-authored-by: Adam Setch <[email protected]> Co-authored-by: Mert Cobanov <[email protected]> Co-authored-by: mr-chenguang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Type
What is in this change?
canvasparser forsharpPDF to image parsing to it will work with tesseract. It is about 50% faster and canvas binary support is pretty bad.Additional Information
Developer Validations
yarn lintfrom the root of the repo & committed changes