Advanced RAG using RAG + LOTR + Remove Redundancy + Long Context Reorder "Lord of the Retrievers", also referred to as MergerRetriever, is designed to consolidate the outputs of various retrievers into a unified list by integrating the outcomes of their get_relevant_documents() functions. This integrated list consists of documents pertinent to the search query, ranked according to their relevance by different retrievers.
Regardless of the underlying model architecture, introducing more than ten retrieved documents significantly hampers performance. Essentially, when models are required to sift through extensive information to find relevant data, they often overlook the documents provided. According to research (https://arxiv.org/abs/2307.03172), models encounter difficulties when trying to pinpoint crucial information located in the middle of lengthy contexts. It has been found that optimal results are typically achieved when vital information is situated at the beginning or end of the input context. Moreover, the longer the input context, the more pronounced the decline in model performance, even for models specifically engineered to handle long contexts. To mitigate this issue, this module rearranges the retrieved items, which proves beneficial in scenarios requiring a substantial number of top-k results.