Announcing the two Best Paper Awards for this year’s workshop. Congratulations to all the authors for your great work!
09.11.2025 08:26
👍 1
🔁 0
💬 0
📌 0
Announcing the two Best Paper Awards for this year’s workshop. Congratulations to all the authors for your great work!
Finally, we recognize the Hawaiian submission to the shared task. Thank you for your contributions!
The second is 7PERFECTION, a dataset for seven Nigerian languages!
The first of our best contribution awards goes to AraPIQA!
We would like to recognize three honorable mention submissions. Great work!
This afternoon, @catherinearnett.bsky.social presented the result of the shared task and presented the best contribution awards. Congratulations to all contributors!
@aliceatkaist.bsky.social joins us to give the final keynote of the day about code switching in multilingual language models!
Join us in hall C3 at posters 137-168 for our in-person poster session. Join us online on Zoom via Underline for our virtual poster session!
Now Pontus Stenetorp shares an oral history of UK-LLM!
Research poster of the paper "Sub-1B Language Models for Low-resource Languages: Training Strategies and Insights for Basque."
Tomorrow, I'll be presenting (virtually) our research from @orainlp.bsky.social on pre-training SLMs for low-resource languages as a poster during the @mrl-workshop.bsky.social.
Come check it out!
📝 aclanthology.org/2025.mrl-mai...
@kellymarchisio.bsky.social from Cohere presents “Building Multilingual LLMs in Industry”, sharing insights on training multilinguality at scale!
We have kicked off proceedings with some brief opening remarks from @catherinearnett.bsky.social
We are kicking off this year’s workshop in Suzhou at #EMNLP2025! Come join us in room A106-107 or online!
Preprint: arxiv.org/abs/2510.24081
Dataset: huggingface.co/datasets/mrl...
It’s not too late to get involved! Until early 2026, we will be accepting submissions for languages not already represented in Global PIQA. If you’re interested, please fill out this form and we will contact you with details!
docs.google.com/forms/d/e/1F...
There are seven languages where even the best proprietary LLM scores less than 80% (chance: 50%). Sub-Saharan African languages lag behind Western European languages by ~15%. Thus Global PIQA highlights languages which are very poorly served by large, proprietary models.
The top proprietary models achieve ~90% accuracy, which falls short of human accuracy (~95%). The best open models perform significantly worse, with the best open model performance from Gemma 3 (27B) at 82.4%.
This dataset is created and owned by the contributors, all of whom were offered authorship. We believe this is more fair to annotators and is likely to result in a higher-quality dataset, as it is constructed by the NLP researchers who will use it.
Global PIQA includes subsets for 116 unique language varieties. These cover five continents, 14 language families, and 23 writing systems. Over 50% of examples reference local foods, customs, traditions, or other culturally-specific elements.
Introducing Global PIQA, a new multilingual benchmark for 100+ languages. This benchmark is the outcome of this year’s MRL shared task, in collaboration with 300+ researchers from 65 countries. This dataset evaluates physical commonsense reasoning in culturally relevant contexts.
We are in need of some emergency reviewers for MRL. If you are available, please fill out this form!
This year MRL is also accepting papers that have been submitted to ARR and have received reviews and a metareview! Submit your papers by September 23rd! See the workshop website for details on how to submit ⬇️
Correct, you can submit ARR papers that already have their reviews by Sep 23. More instructions soon!
TBD on whether the workshop will be hybrid.
We extended the deadline by one day, so you have until the end of today (Aug 24) AoE to submit! Good luck!
Check out more information, including answers to FAQs: docs.google.com/presentation...
If you plan to participate, fill in this google form so we can better plan the shared task: forms.gle/zxhpCfL6wvBz...
We have over 200 volunteers now for 90+ languages! We are hoping to expand the diversity of our language coverage and are still looking for participants who speak these languages. Check out how to get involved below, and please help us spread the word!
The deadline for MRL at #EMNLP2025 is next week!
⏰ Submission Deadline: August 23rd (AoE)
🔗 CfP: sigtyp.github.io/ws2025-mrl.h...
See the shared task page for more information: sigtyp.github.io/st2025-mrl.h...