[2310.20246] Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations

View a PDF of the paper titled Breaking Language Boundaries in Multilingual Mathematical Reasoning: Insights and Observations, by Nuo Chen and 5 different authors

View PDF
HTML (experimental)

Summary:Present analysis predominantly focuses on growing highly effective language studying fashions (LLMs) for mathematical reasoning inside monolingual languages, with few explorations in preserving efficacy in a multilingual context. To bridge this hole, this paper pioneers exploring and coaching highly effective Multilingual Math Reasoning (xMR) LLMs. Firstly, by using translation, we assemble the primary multilingual math reasoning instruction dataset, MGSM8KInstruct, encompassing ten distinct languages, thus addressing the difficulty of coaching knowledge shortage in xMR duties. Based mostly on the collected dataset, we suggest completely different coaching methods to construct highly effective xMR LLMs, named MathOctopus, notably outperform typical open-source LLMs and exhibit superiority over ChatGPT in few-shot eventualities. Notably, MathOctopus-13B reaches 47.6% accuracy which exceeds ChatGPT 46.3% on MGSM testset. Past exceptional outcomes, we unearth a number of pivotal observations and insights from intensive experiments: (1) When extending the rejection sampling technique to the multilingual context, it proves efficient for mannequin performances, albeit restricted. (2) Using parallel corpora for math Supervised Nice-Tuning (SFT) throughout a number of languages not solely considerably enhances mannequin efficiency multilingually but in addition elevates their monolingual efficiency. This means that crafting multilingual corpora may be thought to be a significant technique for enhancing mannequin efficiency in a particular language, particularly in mathematical reasoning duties. For example, MathOctopus-7B improves its counterparts that skilled on English from 42.2% to 50.8% on GSM8K testset. Codes can be found at this https URL.