feat(llm): redo llm feature and tools #7270

perfectra1n · 2025-10-10T19:26:42Z

No description provided.

gemini-code-assist · 2025-10-10T19:27:12Z

Summary of Changes

Hello @perfectra1n, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant overhaul of the LLM feature set, focusing on simplification, efficiency, and maintainability. It re-architects the core LLM pipeline into a leaner, three-stage process and consolidates numerous specialized tools into a few versatile ones. These changes are supported by a new configuration service, a dynamic tool filtering mechanism, and a pipeline adapter to ensure a smooth transition and improved performance across different LLM providers.

Highlights

LLM Pipeline Simplification (V2): The core LLM pipeline has been re-architected and simplified from 8 stages down to 3 essential stages: Message Preparation, LLM Execution (including tool handling), and Response Formatting. This new V2 pipeline reduces code complexity by approximately 60%.
Tool Consolidation: Twelve legacy LLM tools have been consolidated into four more powerful and unified tools: smart_search, manage_note, navigate_hierarchy, and calendar_integration. This consolidation significantly reduces the token overhead for tool definitions by an estimated 600 tokens.
Dynamic Tool Filtering: A new ToolFilterService has been introduced to intelligently filter tools based on the LLM provider (e.g., Ollama, OpenAI) and the user's query intent. This optimizes tool selection, especially for local models like Ollama which perform better with a limited number of relevant tools.
Pipeline Adapter for Gradual Migration: A PipelineAdapter now provides a unified interface for both the legacy and new V2 LLM pipelines. It uses feature flags (environment variables or input options) to allow for a seamless and gradual migration to the V2 pipeline, defaulting to V2.
Centralized Configuration: A PipelineConfigService has been added to provide centralized, type-safe access to LLM pipeline settings, including parameters for tool execution, streaming, debugging, context management, and provider-specific configurations like Ollama's context window and max tools.
New Hierarchy Navigation Tool: A new navigate_hierarchy tool has been added, enabling the LLM to efficiently traverse Trilium's note tree to find children, parents, ancestors, or siblings of a given note.
Structured Logging: A StructuredLogger has been implemented to provide more detailed and trackable logging for the LLM pipeline, including request IDs, log levels, and performance timing.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a major and well-executed refactoring of the LLM feature, including a new V2 pipeline, consolidated tools, a feature-flagging mechanism, improved configuration management, and structured logging. The changes are well-tested and significantly improve the modularity and maintainability of the codebase. My feedback focuses on improving type safety, ensuring consistency, and enhancing error logging for better debuggability.

gemini-code-assist · 2025-10-10T19:29:01Z

apps/server/src/services/llm/pipeline/pipeline_v2.spec.ts

+
+import { describe, it, expect, beforeEach, vi } from 'vitest';
+import type { PipelineV2Input } from './pipeline_v2.js';
+import type { Message } from '../ai_interface.js';


This test file is missing several type imports, which will cause compilation errors. Specifically, PipelineV2, AIService, and ChatResponse are used without being imported.

Please add the necessary imports to ensure the file is type-safe and compilable.

Suggested change

import type { Message } from '../ai_interface.js';

import type { Message, AIService, ChatResponse } from '../ai_interface.js';

import { PipelineV2 } from './pipeline_v2.js';

gemini-code-assist · 2025-10-10T19:29:02Z

apps/server/src/services/llm/config/pipeline_config.ts

+     */
+    private getBoolOption(key: string, defaultValue: boolean): boolean {
+        try {
+            const value = (options as any).getOptionBool(key);


Casting options to any bypasses TypeScript's type safety. While this might be necessary if the options object is not strongly typed, it's a practice that should be avoided. Consider defining a minimal interface for the parts of options you are using, or use a more specific type assertion than any to improve maintainability and prevent potential runtime errors.

gemini-code-assist · 2025-10-10T19:29:02Z

apps/server/src/services/llm/config/pipeline_config.ts

+    private getStringOption(key: string, defaultValue: string): string {
+        try {
+            const value = (options as any).getOption(key);
+            return value !== null && value !== undefined ? String(value) : defaultValue;
+        } catch {
+            return defaultValue;
+        }
+    }


The getStringOption method is defined but appears to be unused within this class, as it's not called from loadConfiguration. If it's not intended for future use, it should be removed to eliminate dead code.

gemini-code-assist · 2025-10-10T19:29:02Z

apps/server/src/services/llm/pipeline/pipeline_adapter.ts

+                ...response,
+                pipelineVersion: 'v2',
+                requestId: response.requestId,
+                processingTime: response.processingTime


There's an inconsistency in how processingTime is handled between executeLegacy and executeV2. In executeLegacy, the time measured by the adapter is used for both metrics and the returned output. However, in executeV2, the adapter-measured time is used for metrics, but the processingTime from the V2 pipeline's response is returned in the output.

For consistency, it would be better to use the time measured within the adapter for the output in both methods. This ensures that the reported processing time always reflects the total time taken by the adapter's execution path.

Suggested change

processingTime: response.processingTime

processingTime: processingTime

gemini-code-assist · 2025-10-10T19:29:02Z

apps/server/src/services/llm/pipeline/pipeline_v2.ts

+            }
+
+            return null;
+        } catch (error: any) {


Using any for caught errors is generally discouraged as it bypasses type safety. It's better to type the error as unknown and then perform type checks to safely access its properties. This makes the error handling more robust.

Suggested change

} catch (error: any) {

} catch (error: unknown) {

gemini-code-assist · 2025-10-10T19:29:02Z

apps/server/src/services/llm/tools/consolidated/smart_search_tool.ts

+    /**
+     * Get or create vector search tool
+     */
+    private async getVectorSearchTool(): Promise<any> {


The method getVectorSearchTool has a return type of Promise<any>. Using any weakens type safety. It would be beneficial to define a specific interface for the vector search tool and use that as the return type. This will improve code clarity and prevent potential runtime errors.

gemini-code-assist · 2025-10-10T19:29:02Z

apps/server/src/services/llm/utils/structured_logger.ts

+        switch (entry.level) {
+            case LogLevel.ERROR:
+                if (entry.error) {
+                    log.error(`${formattedMessage}: ${entry.error.message}`);


When logging an error, only the message property is being logged. This can make debugging difficult as the stack trace is lost. It would be more helpful to log the full error object or at least its stack trace to provide more context for debugging.

Suggested change

log.error(`${formattedMessage}: ${entry.error.message}`);

log.error(`${formattedMessage}: ${entry.error.stack || entry.error.message}`);

feat(llm): redo llm feature and tools

74a2fcd

dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Oct 10, 2025

gemini-code-assist bot reviewed Oct 10, 2025

View reviewed changes

perfectra1n added 2 commits October 10, 2025 13:28

feat(llm): get rid of unused code

4a23924

feat(llm): migrate the calendar tool into the manage_note tool

5710bec

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(llm): redo llm feature and tools #7270

feat(llm): redo llm feature and tools #7270

Uh oh!

perfectra1n commented Oct 10, 2025

Uh oh!

gemini-code-assist bot commented Oct 10, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Oct 10, 2025

Uh oh!

gemini-code-assist bot Oct 10, 2025

Uh oh!

gemini-code-assist bot Oct 10, 2025

Uh oh!

gemini-code-assist bot Oct 10, 2025

Uh oh!

gemini-code-assist bot Oct 10, 2025

Uh oh!

gemini-code-assist bot Oct 10, 2025

Uh oh!

gemini-code-assist bot Oct 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	import type { Message } from '../ai_interface.js';
	import type { Message, AIService, ChatResponse } from '../ai_interface.js';
	import { PipelineV2 } from './pipeline_v2.js';

	processingTime: response.processingTime
	processingTime: processingTime

	log.error(`${formattedMessage}: ${entry.error.message}`);
	log.error(`${formattedMessage}: ${entry.error.stack \|\| entry.error.message}`);

Uh oh!

feat(llm): redo llm feature and tools #7270

Are you sure you want to change the base?

feat(llm): redo llm feature and tools #7270

Uh oh!

Conversation

perfectra1n commented Oct 10, 2025

Uh oh!

gemini-code-assist bot commented Oct 10, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant