Tamara Broderick

I am very behind the times, but this read was fantastic. Thanks for sharing!

An Automatic Finite-Sample Robustness Metric: When Can Dropping a Little Data Make a Big Difference? Study samples often differ from the target populations of inference and policy decisions in non-random ways. Researchers typically believe that such departures from random sampling -- due to changes i...

This work builds on our cool previous work on sensitivity of data-analysis conclusions to dropping a super small fraction of data (led by the amazing Ryan Giordano and @economeager.bsky.social ):
arxiv.org/abs/2011.14999
github.com/rgiordan/AMI...
github.com/rgiordan/zam...

Dropping Just a Handful of Preferences Can Change Top Large Language Model Rankings We propose a method for evaluating the robustness of widely used LLM ranking systems -- variants of a Bradley--Terry model -- to dropping a worst-case very small fraction of preference data. Our appro...

Shared first authors are my awesome PhD students Jenny Huang and Yunyi Shen. This work is with our fantastic collaborator Dennis Wei and will be appearing at ICLR 2026. The paper itself can be found here:
arxiv.org/abs/2508.11847

Study: Platforms that rank the latest LLMs can be unreliable The results of popular LLM ranking platforms can be skewed by just a few data points, possibly providing an unreliable report about which LLM would perform best in real situations, according MIT resea...

I’m excited that MIT News covered our new paper on robustness of popular LLM rankings! We find that dropping just 2 out of 57,477 matches can change the top-ranked model on Chatbot Arena (based on historical preference data shared by Chatbot Arena on Hugging Face). news.mit.edu/2026/study-p...

🚨🧪 Announcing our #ICLR2026 Workshop, Generative AI in Genomics (Gen2): Barriers and Frontiers! @iclr-conf.bsky.social

📣Call for: Full workshop papers (5-8 pages) and Tiny papers (2-4 pages)
📅Submission deadline: 7 February 2026 AoE
🌐Learn more: genai-in-genomics.github.io
(1/7)

Organized by: @tamarabroderick.bsky.social, @vdebortoli.bsky.social, @pinard.bsky.social, @arnauddoucet.bsky.social, Dongshunyi "Dora" Li, Maria Skoularidou, Renzo Soatto, and Max Welling @amlab.bsky.social
Feel free to reach out if you’d like to be involved!

Stay tuned for more updates!
(7/7)

And thanks for the interest!

If you're asking about the link to the paper itself, it can be found here on the arxiv: arxiv.org/pdf/2502.06067 It just appeared at NeurIPS 2025.

Shared first authors are my amazing postdoc David Burt and fantastic PhD student @renberlinghieri.bsky.social . Work is with our wonderful collaborator Stephen Bates.

Our work shows that existing confidence intervals can provide way less than nominal coverage (sometimes near 0) in spatial association settings, but we provide a new method that achieves nominal coverage across our experiments.

Scientists are often interested in understanding an association like how much air pollution exposure varies with proximity to a highway. And data (e.g. air pollution sensors) may not be available exactly where they want to estimate the association.

New method improves the reliability of statistical estimations MIT researchers developed a method that generates more accurate uncertainty measures for certain types of estimation. This could help improve the reliability of data analyses in areas like economics, ...

I’m excited that MIT News covered our new paper on confidence intervals for associations in spatial settings!
news.mit.edu/2025/new-met...

Kudos to the researchers who retracted once they realised their results hinged on a single point.

To check this in your own work, for small fractions of your data set (not just single points), check out our R package! cc @tamarabroderick.bsky.social (ryan not on here?)

github.com/rgiordan/zam...

Big thanks to @natematias.bsky.social for this extremely touching and kind post about some of our research! (Amazing co-first authors are my PhD student @renberlinghieri.bsky.social and postdoc David Burt, and awesome collaborators are Paolo Giani and Arlene Fiore.)