Skip to content

[Feature] Modify expansion of BaseRetriever to match BaseGenerator for multimodal considerations #481

@nerdai

Description

@nerdai

Tasks:

  • Update fed_rag.base.retriever to use Query and Context
  • Update fed_rag.base.retriever to return a new encoder type that flexible accepts separate encoders for text and image (audio and video later)
  • Update fed_rag.retrievers.hf_sentence_transformer to be able to take in Query and Context inputs — ignoring the other modalities, namely image (audio and video later).

Metadata

Metadata

Assignees

Labels

Projects

Status

In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions