Skip to content

Conversation

@anasdorbani
Copy link
Member

Fixes an issue where llm_filter only worked when context_columns was provided. Now it supports simple prompts without context columns.

Before (only worked with context_columns):

SELECT llm_filter(
    {'model_name': 'gpt-4o'}, 
    {'prompt': 'Is this sentiment positive?', 'context_columns': [{'data': text}]}
) FROM unnest(['I love this product!']) as tbl(text);

After (now works without context_columns):

SELECT llm_filter(
    {'model_name': 'gpt-4o'}, 
    {'prompt': 'Is paris the best capital in the world?'}
);

The function now handles both cases:

  • Without context_columns: Makes a single completion request and returns the boolean result
  • With context_columns: Continues to work as before with batch processing

Copilot AI review requested due to automatic review settings December 11, 2025 16:17
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds comprehensive audio transcription support to the Flock extension. The main purpose is to enable LLM functions (like llm_complete, llm_filter, llm_rerank, etc.) to process audio files by transcribing them first, then using the transcriptions as context for LLM operations.

Key Changes:

  • Implemented audio transcription functionality via OpenAI's Whisper API
  • Added URL/file handling utilities for downloading and processing audio files
  • Extended all LLM functions to support audio context columns with automatic transcription
  • Added metrics tracking system for monitoring LLM function performance

Reviewed changes

Copilot reviewed 78 out of 81 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/model_manager/providers/adapters/openai.cpp Added transcription request handling for OpenAI provider
src/model_manager/providers/adapters/ollama.cpp Added error handling for unsupported transcription in Ollama
src/model_manager/providers/adapters/azure.cpp Added transcription support for Azure provider
src/prompt_manager/prompt_manager.cpp Added audio transcription processing in prompt rendering
src/include/flock/model_manager/providers/handlers/url_handler.hpp New utility for handling file downloads and base64 conversion
src/metrics/*.cpp New metrics tracking system implementation
test/unit/prompt_manager/prompt_manager_test.cpp Added tests for audio transcription functionality
test/unit/model_manager/model_providers_test.cpp Updated model name and added transcription tests
test/integration/src/integration/conftest.py Added audio file path helper and secrets setup
test/integration/src/integration/tests/functions/**/*.py Added audio transcription integration tests across all functions
Comments suppressed due to low confidence (3)

test/unit/model_manager/model_providers_test.cpp:1

  • The model version 'gemma3:4b' does not exist. The Gemma 3 family was released after your knowledge cutoff (January 2025), but typical Gemma model naming uses '2b', '7b', '9b', '27b' parameter counts. The '4b' variant is not a standard Gemma model size. Consider using a valid Gemma model like 'gemma2:2b' or 'gemma2:9b'.
    test/unit/model_manager/model_manager_test.cpp:1
  • The model version 'gemma3:4b' does not exist. Please use a valid Gemma model version such as 'gemma2:2b' or 'gemma2:9b'.
    test/integration/src/integration/tests/functions/scalar/test_llm_embedding.py:1
  • The model 'gpt-4o-mini-transcribe' referenced in other test files does not exist. OpenAI's transcription models use 'whisper-1' as the model identifier. Please ensure transcription model references use the correct model name.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant