Analysis of Benign Overfitting in Retrieval Augmented Models

Project: FDCRGP

Project Details

Grant Program

Faculty-development competitive research grants program for 2023-2025

Project Description

Develop an adequate mathematical framework for the analysis of retrieval augmented models (RAMs). Under such framework, prove that RAMs overfit benignly and have lower sample complexity than their counterparts without retrieval mechanisms. Decouple memorization and generalization in RAMs with possible applications to natural language processing (NLP) tasks such as bilingual/multilingual neural machine translation (NMT).

Project Impact

The results of the research will be published in international journals and conference proceedings, and all the software will be made publicly available as free/open-source software. This will facilitate research on retrieval augmented models. Successful implementation of the Project will result in (a) a better understanding of the underlying structure of RAMs; (b) theoretically driven and competitive RAMs.
Effective start/end date1/1/2312/31/25


Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint.