View a PDF of the paper titled Thank You, Stingray: Multilingual Giant Language Fashions Can Not (But) Disambiguate Cross-Lingual Phrase Sense, by Samuel Cahyawijaya and Ruochen Zhang and Holy Lovenia and Jan Christian Blaise Cruz and Elisa Gilbert and Hiroki Nomoto and Alham Fikri Aji
Summary:Multilingual giant language fashions (LLMs) have gained prominence, however issues come up relating to their reliability past English. This examine addresses the hole in cross-lingual semantic analysis by introducing a novel benchmark for cross-lingual sense disambiguation, StingrayBench. On this paper, we show utilizing false associates — phrases which might be orthographically comparable however have fully totally different meanings in two languages — as a attainable strategy to pinpoint the limitation of cross-lingual sense disambiguation in LLMs. We accumulate false associates in 4 language pairs, particularly Indonesian-Malay, Indonesian-Tagalog, Chinese language-Japanese, and English-German; and problem LLMs to differentiate using them in context. In our evaluation of varied fashions, we observe they are typically biased towards higher-resource languages. We additionally suggest new metrics for quantifying the cross-lingual sense bias and comprehension primarily based on our benchmark. Our work contributes to growing extra numerous and inclusive language modeling, selling fairer entry for the broader multilingual group.
Submission historical past
From: Samuel Cahyawijaya [view email]
[v1]
Mon, 28 Oct 2024 22:09:43 UTC (9,057 KB)
[v2]
Wed, 30 Oct 2024 11:56:17 UTC (9,057 KB)
Source link
#Multilingual #Giant #Language #Fashions #Disambiguate #CrossLingual #Phrase #Sense
Unlock the potential of cutting-edge AI options with our complete choices. As a number one supplier within the AI panorama, we harness the ability of synthetic intelligence to revolutionize industries. From machine studying and knowledge analytics to pure language processing and laptop imaginative and prescient, our AI options are designed to reinforce effectivity and drive innovation. Discover the limitless potentialities of AI-driven insights and automation that propel your enterprise ahead. With a dedication to staying on the forefront of the quickly evolving AI market, we ship tailor-made options that meet your particular wants. Be part of us on the forefront of technological development, and let AI redefine the best way you use and achieve a aggressive panorama. Embrace the longer term with AI excellence, the place potentialities are limitless, and competitors is surpassed.
