![[2405.18348] Can Automatic Metrics Assess High-Quality Translations?](https://i0.wp.com/arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png?w=1200&resize=1200,700&ssl=1 "[2405.18348] Can Automatic Metrics Assess High-Quality Translations?"){kind=logo}
View a PDF of the paper titled Can Automated Metrics Assess Excessive-High quality Translations?, by Sweta Agrawal and three different authors
Summary:Automated metrics for evaluating translation high quality are sometimes validated by measuring how nicely they correlate with human assessments. Nonetheless, correlation strategies are inclined to seize solely the flexibility of metrics to distinguish between good and unhealthy source-translation pairs, overlooking their reliability in distinguishing various translations for a similar supply. On this paper, we verify that that is certainly the case by displaying that present metrics are insensitive to nuanced variations in translation high quality. This impact is most pronounced when the standard is excessive and the variance amongst options is low. Given this discovering, we shift in direction of detecting high-quality right translations, an necessary drawback in sensible decision-making eventualities the place a binary examine of correctness is prioritized over a nuanced analysis of high quality. Utilizing the MQM framework because the gold customary, we systematically stress-test the flexibility of present metrics to determine translations with no errors as marked by people. Our findings reveal that present metrics usually over or underestimate translation high quality, indicating vital room for enchancment in automated analysis strategies.
Submission historical past
From: Sweta Agrawal [view email]
[v1]
Tue, 28 Could 2024 16:44:02 UTC (8,892 KB)
[v2]
Thu, 10 Oct 2024 10:04:20 UTC (9,007 KB)
Source link
#Automated #Metrics #Assess #HighQuality #Translations
Unlock the potential of cutting-edge AI options with our complete choices. As a number one supplier within the AI panorama, we harness the facility of synthetic intelligence to revolutionize industries. From machine studying and knowledge analytics to pure language processing and laptop imaginative and prescient, our AI options are designed to boost effectivity and drive innovation. Discover the limitless potentialities of AI-driven insights and automation that propel what you are promoting ahead. With a dedication to staying on the forefront of the quickly evolving AI market, we ship tailor-made options that meet your particular wants. Be a part of us on the forefront of technological development, and let AI redefine the way in which you use and achieve a aggressive panorama. Embrace the longer term with AI excellence, the place potentialities are limitless, and competitors is surpassed.
