Mixture-of-Experts Language Models Can Be Strong and Efficient Retrievers
Recent work has shown that fine-tuning decoder-only large language models (LLMs) for retrieval yields strong first-stage retrievers, with effectiveness improving as backbones grow in size. However, every query and document must pass through the full model, so encoding cost increases with model size. Mixture-of-Experts...
Anubhav Shrestha, Safal Shrestha, Minwu Kim et al.
· 0 citations