COLING 2022 Highlights
More robust evaluation metrics, language models that don’t understand anything, and better evaluation for grammatical error correction COLING 2022 was held in mid-October in Gyeongju (Republic of Korea). This natural language…
MBR Decoding: Get Better Results from Many Systems
Even your state-of-the-art system has flaws that others don’t have Illustration by the author. Made with Noto Color Emoji. New large language models (BLOOM, OPT, GPT-3, NLLB, …) are released almost…
A Large-Scale Automatic Evaluation of Machine Translation
Like every year since 2006, the Conference on Machine Translation (WMT) organized extensive machine translation shared tasks. Numerous participants from all over the world submitted their machine translation (MT) outputs…
AMTA 2022 Highlights
I highlight and sum up the AMTA 2022 papers that I found the most original and interesting. I picked papers from the users (see the proceedings) and the research (see…
compare-mt: Because Scoring Your Systems Is Not Enough
I present compare-mt. A very simple tool that gives the user a high-level and coherent view of the salient differences between systems. It exploits statistics usually computed by automatic metrics such…
Comparing the Uncomparable to Claim the State of the Art: A Concerning Trend
I describe how uncomparable numbers happen to be compared in scientific papers. Then, I review some concrete examples of flawed comparisons from very well-known and recent work, namely, Open AI’s…