3 天之前 · As a step toward this goal, we propose an evaluation methodology grounded in explicit error analysis, based on the Multidimensional Quality Metrics (MQM) framework. We carry out the largest MQM research study to date, scoring the outputs of top systems from the WMT 2020 shared task in two language pairs using annotations provided by ...

As of July 2021, MQM 2.0 is under development. Some components, such as the error typology, are mostly stable and widely implemented. Others are under active development and subject to change. Development is ongoing in the ASTM WK46396 committee and will contribute to the resulting standard.

Multidimensional Quality Metrics (MQM) is a framework for analytic Translation Quality Evaluation (TQE). It can be applied to both human translation, machine translation and AI-generated translation.

2021年12月17日 · MQM correlations are again higher than WMT at this granularity, and are higher for ZhEn than EnDe, reversing the pattern from system-level results and suggesting a potential for improved system-level metric performance through better aggregation of …

2021年4月29日 · As a step toward this goal, we propose an evaluation methodology grounded in explicit error analysis, based on the Multidimensional Quality Metrics (MQM) framework. We carry out the largest MQM research study to date, scoring the outputs of top systems from the WMT 2020 shared task in two language pairs using annotations provided by ...

Expert-based Human Evaluations for the Submissions of WMT 2020, WMT 2021, WMT 2022 and WMT 2023. The contents of this repository are not an official Google product.

2025年3月23日 · In this paper, we present the joint contribution of Unbabel and IST to the WMT 2021 Metrics Shared Task. With this year’s focus on Multidimensional Quality Metric (MQM) as the ground-truth human assessment, our aim was to …

2021年10月19日 · An update to the MQM Core is now available at https://themqm.info/typology/. Please see https://www.w3.org/community/mqmcg/updated-mqm-core-posted/ for more details.

An updated MQM Core is now available at https://themqm.info/typology/. IT contains a subset of MQM recommended for general usage and is substantially the same as a proposed update to the TAUS Dynamic Quality Framework subset of MQM (DQF-MQM).

2021年10月19日 · MQM Core Update (October 202... An update to the MQM Core is now available at https://themqm.info/typology/. Please see https://www.w3.org/community/mqmcg/updated-mqm-core-posted/ for more details. Your email address will not be published.