A Model-Agnostic Framework Boosting LLM-based Machine Translation to the Next Level

View a PDF of the paper titled Ladder: A Mannequin-Agnostic Framework Boosting LLM-based Machine Translation to the Subsequent Degree, by Zhaopeng Feng and 4 different authors

View PDF
HTML (experimental)

Summary:Normal-purpose Giant Language Fashions (LLMs) like GPT-4 have achieved outstanding developments in machine translation (MT) by leveraging intensive internet content material. Alternatively, translation-specific LLMs are constructed by pre-training on domain-specific monolingual corpora and fine-tuning with human-annotated translation information. Regardless of the superior efficiency, these strategies both demand an unprecedented scale of computing and information or substantial human modifying and annotation efforts. On this paper, we develop MT-Ladder, a novel model-agnostic and cost-effective software to refine the efficiency of normal LLMs for MT. MT-Ladder is skilled on pseudo-refinement triplets which might be simply obtained from current LLMs with out extra human price. Throughout coaching, we suggest a hierarchical fine-tuning technique with an easy-to-hard schema, bettering MT-Ladder’s refining efficiency progressively. The skilled MT-Ladder might be seamlessly built-in with any general-purpose LLMs to spice up their translation efficiency. By using Gemma-2B/7B because the spine, MT-Ladder-2B can elevate uncooked translations to the extent of top-tier open-source fashions (e.g., refining BigTranslate-13B with +6.91 BLEU and +3.52 COMET for XX-En), and MT-Ladder-7B can additional improve mannequin efficiency to be on par with the state-of-the-art GPT-4. In depth ablation and evaluation corroborate the effectiveness of MT-Ladder in numerous settings. Our code is obtainable at this https URL