Analysis of Benign Overfitting in Retrieval Augmented Models

Faculty-development competitive research grants program for 2023-2025

Develop an adequate mathematical framework for the analysis of retrieval augmented models (RAMs). Under such framework, prove that RAMs overfit benignly and have lower sample complexity than their counterparts without retrieval mechanisms. Decouple memorization and generalization in RAMs with possible applications to natural language processing (NLP) tasks such as bilingual/multilingual neural machine translation (NMT).

The results of the research will be published in international journals and conference proceedings, and all the software will be made publicly available as free/open-source software. This will facilitate research on retrieval augmented models. Successful implementation of the Project will result in (a) a better understanding of the underlying structure of RAMs; (b) theoretically driven and competitive RAMs.