document_query Tool Returns 0 Results Despite Successful Chunking

[BdM-15]

document_query Tool Returns 0 Results Despite Successful Chunking

Issue Description

The document_query tool successfully chunks uploaded documents but fails to retrieve any relevant content via vector search, always returning 0 results and falling back to full raw text retrieval. This defeats the purpose of the RAG-based document query functionality.

Environment

• Agent-Zero Version: Latest 0.9.7 (fresh container, deployed: 1/23/2026)
• Models:
  • Chat/Utility/Web Browser: xAI Grok 4.1 Fast Reasoning
  • Embeddings: sentence-transformers/all-MiniLM-L6-v2 (default, unchanged)
• API Keys: OpenAI and xAI configured
• Setup: Brand new container, only configuration changes made were setting models and API keys

Steps to Reproduce

1. Create fresh agent-zero container
2. Configure models in Model Settings:
   • Set Chat, Utility, and Web Browser models to xAI Grok 4.1 Fast Reasoning
   • Leave embeddings model at default (HuggingFace sentence-transformers/all-MiniLM-L6-v2)
3. Add OpenAI and xAI API keys in API Keys settings
4. Upload document (PDF): Best_Value_Selection.pdf
5. Query: "what are the management techniques to consider in best value selection?"

Expected Behavior

The document_query tool should:

1. Chunk the document ✅ (works - 36 chunks created)
2. Generate embeddings for chunks ✅ (works - 2 embedding files created, 8.2 KB each)
3. Perform vector similarity search ❌ (returns 0 results)
4. Return relevant chunks to answer the query ❌ (fails)

Actual Behavior

Added document 'file:///a0/tmp/uploads/Best_Value_Selection.pdf' with 36 chunks
Search 'best value selection management techniques methods strategies approaches procurement evaluation trade-off' returned 0 results
Search 'best value selection techniques considerations management procurement project key methods criteria evaluation value-based tender bidding' returned 0 results

!!! No content found for documents: ["/a0/tmp/uploads/Best_Value_Selection.pdf"] matching queries: ["What are the management techniques to consider in best value selection?", "List and describe key techniques or considerations for best value selection in management."]

The tool then falls back to retrieving the full document without queries.

Complete Interaction Log

First Interaction (Greeting)

User: Hi
Agent: Greeting user back
Tool: response
Result: "Hi there! How can I assist you today?"

Second Interaction (Document Query - ISSUE)

User: what are the management techniques to consider in best value selection?
Attachments: Best_Value_Selection.pdf

Agent Tool Call:
{
  "thoughts": [
    "User is asking about management techniques for best value selection, with a PDF attachment likely containing the info.",
    "Best to query the document directly using document_query tool.",
    "Query: 'What are the management techniques to consider in best value selection?' to get precise answers.",
    "After results, summarize in a structured response."
  ],
  "headline": "Analyzing PDF for best value selection management techniques",
  "tool_name": "document_query",
  "tool_args": {
    "document": "/a0/tmp/uploads/Best_Value_Selection.pdf",
    "queries": [
      "What are the management techniques to consider in best value selection?",
      "List and describe key techniques or considerations for best value selection in management."
    ]
  }
}

Tool Output:
Added document 'file:///a0/tmp/uploads/Best_Value_Selection.pdf' with 36 chunks
Search 'best value selection management techniques methods strategies approaches procurement evaluation trade-off' returned 0 results
Search 'best value selection techniques considerations management procurement project key methods criteria evaluation value-based tender bidding' returned 0 results

!!! No content found for documents: ["/a0/tmp/uploads/Best_Value_Selection.pdf"] matching queries: [...]

Agent Fallback (second attempt):
{
  "thoughts": [
    "Previous document_query with queries found no matching content.",
    "Now fetching full document content without queries to inspect what's in the PDF.",
    "If empty or irrelevant, will search web next.",
    "PDF path confirmed from attachment."
  ],
  "headline": "Retrieving full content from Best_Value_Selection.pdf",
  "tool_name": "document_query",
  "tool_args": {
    "document": "/a0/tmp/uploads/Best_Value_Selection.pdf"
  }
}

Analysis

The issue appears to be in the vector similarity search pipeline:

1. ✅ Document Loading: Document is successfully loaded
2. ✅ Chunking: 36 chunks created successfully
3. ✅ Embedding Generation: Confirmed working - 2 embedding files created (8.2 KB each, HuggingFace sentence-transformers format)
4. ❌ Vector Search: Returns 0 results for highly relevant queries
5. ⚠️ Fallback: System falls back to raw text retrieval

Critical Finding: Embeddings ARE being generated and stored correctly. The problem is specifically in the vector similarity search/retrieval mechanism, not in the embedding generation pipeline.

Potential Root Causes

1. Similarity Threshold Too Restrictive: Search threshold may be set too high, rejecting all matches
2. Vector Search Algorithm Issue: The cosine similarity or distance calculation may be broken/misconfigured
3. Vector Store Query Issue: The vector database query mechanism may not be properly retrieving/comparing embeddings
4. Embedding Dimensionality Mismatch: Query embeddings and chunk embeddings may have different dimensions
5. Index Not Built/Updated: The vector index may not be built correctly after embedding generation

Questions

• ✅ Are embeddings being generated? YES - confirmed via file system (2 files, 8.2 KB each)
• What is the configured similarity threshold for vector search?
• What vector store/search implementation is being used (FAISS, ChromaDB, etc.)?
• Can you provide debug logs showing the vector search query process?
• Are the query embeddings being generated with the same model/parameters as chunk embeddings?
• Is the vector index being properly built/loaded after embedding generation?

Expected Fix

The document_query tool should successfully retrieve relevant chunks via vector similarity search, eliminating the need for raw text fallback except in edge cases where truly no relevant content exists.

Additional Context

This is a fresh installation test - no custom modifications, extensions, or previous usage. This behavior occurs on the very first document query attempt, suggesting a default configuration or initialization issue rather than a corruption or state problem.

Evidence of embedding generation: File system shows 2 embedding files were created successfully:

• huggingface_sentence-transformers_all-MiniLM-L6-v20b010d60-ef0b-5ce0-bbc6-56774988177d4 (8.2 KB, added 26 minutes ago)
• huggingface_sentence-transformers_all-MiniLM-L6-v2798ab3a8-fa91-59ab-b571-10b6425e8587 (8.2 KB, added 26 minutes ago)

This confirms the embedding pipeline works correctly, isolating the issue to the vector search/retrieval mechanism.

Workaround

Currently, the system falls back to full document retrieval which works but:

• Defeats the purpose of RAG/chunking
• May hit token limits with large documents
• Loses the benefits of semantic search
• Reduces response quality and accuracy

[TheOneBestAgent]

Additional Test Results: Query Parameter Completely Broken

I've conducted comprehensive testing that reveals another manifestation of this bug:

Test Results Summary

✅ What Works:

• Content extraction WITHOUT queries (HTML, PDF, DOCX, GitHub raw, multiple docs)
• Error handling (404s, invalid URLs)
• Multiple document processing

❌ What's Broken:

• Query functionality returns "No content found" 100% of the time
• Even when content is successfully extracted without queries
• Tested with simple queries on known content - all failed

Reproduction Steps

1. Extract content without query: document_query(document="https://example.com") ✅ Works
2. Same document WITH query: document_query(document="https://example.com", queries=["What is..."]) ❌ Returns "No content found"

Test Coverage

• Static HTML pages ✅/❌
• JavaScript SPAs ✅/❌
• PDF documents ✅/❌
• Microsoft Word docs ✅/❌
• GitHub raw content ✅/❌
• Multiple combined documents ✅/❌

(✅ = extraction works, ❌ = queries fail)

Workaround

Current solution: Extract full content first, then use the LLM to answer questions:

# Instead of:
result = document_query(document=url, queries=["question"])

# Use:
content = document_query(document=url)  # No queries
# Then ask LLM to analyze the content
Analysis
This appears related to your findings but manifests differently:

Your issue: Vector search returns 0 results, falls back to full document
My finding: Query parameter fails completely with "No content found"
Both point to the same underlying problem in the query/retrieval mechanism.

Additional Notes
Excel files (.xlsx) fail due to unsupported mimetype
PowerPoint files may have permission issues
Full test documentation available if needed
Hope this additional data helps isolate the root cause! 🔍