A rule of thumb may yield correct results but can’t be scientifically credible. Illustration by the author. Take any research paper or blog post presenting a new method for AI,…
More robust evaluation metrics, language models that don’t understand anything, and better evaluation for grammatical error correction COLING 2022 was held in mid-October in Gyeongju (Republic of Korea). This natural language…
A heuristic-based method exploiting Unicode tables
Even your state-of-the-art system has flaws that others don’t have Illustration by the author. Made with Noto Color Emoji. New large language models (BLOOM, OPT, GPT-3, NLLB, …) are released almost…
Like every year since 2006, the Conference on Machine Translation (WMT) organized extensive machine translation shared tasks. Numerous participants from all over the world submitted their machine translation (MT) outputs…
I highlight and sum up the AMTA 2022 papers that I found the most original and interesting. I picked papers from the users (see the proceedings) and the research (see…
I present compare-mt. A very simple tool that gives the user a high-level and coherent view of the salient differences between systems. It exploits statistics usually computed by automatic metrics such…
I describe how uncomparable numbers happen to be compared in scientific papers. Then, I review some concrete examples of flawed comparisons from very well-known and recent work, namely, Open AI’s…