[2411.07820] Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models

View a PDF of the paper titled Question Optimization for Parametric Information Refinement in Retrieval-Augmented Giant Language Fashions, by Youan Cong and three different authors

View PDF
HTML (experimental)

Summary:We introduce the Extract-Refine-Retrieve-Learn (ERRR) framework, a novel method designed to bridge the pre-retrieval data hole in Retrieval-Augmented Technology (RAG) methods by question optimization tailor-made to fulfill the particular data necessities of Giant Language Fashions (LLMs). In contrast to typical question optimization methods utilized in RAG, the ERRR framework begins by extracting parametric data from LLMs, adopted by utilizing a specialised question optimizer for refining these queries. This course of ensures the retrieval of solely probably the most pertinent data important for producing correct responses. Furthermore, to reinforce flexibility and scale back computational prices, we suggest a trainable scheme for our pipeline that makes use of a smaller, tunable mannequin because the question optimizer, which is refined by data distillation from a bigger instructor mannequin. Our evaluations on varied question-answering (QA) datasets and with totally different retrieval methods present that ERRR persistently outperforms present baselines, proving to be a flexible and cost-effective module for enhancing the utility and accuracy of RAG methods.