New NeurIPS paper! Why do LMs represent concepts linearly? We focus on LMs's tendency to linearly separate true and false assertions, and provide an analysis of the truth circuit in a toy model. A joint work with Gilad Yehudai, @tallinzen.bsky.social, Joan Bruna and @albertobietti.bsky.social.
✨ Thank you #AISTATS for the best paper award!!
📜 arxiv.org/abs/2410.11067
💡What does VI learn and under what conditions? The answer lies in symmetry.
🤝 Honored to share this award with my co-author Lawrence Saul from @flatironinstitute.org
Come hear Matt Smart's talk about in-context denoising with transformers at the Associative memory workshop #ICLR25, 2:15pm! This task refines the connection between transformers and associative memories. w/ M Smart and @albertobietti.bsky.social at @flatironinstitute.org
Given a high-quality verifier, language model accuracy can be improved by scaling inference-time compute (e.g., w/ repeated sampling). When can we expect similar gains without an external verifier?
New paper: Self-Improvement in Language Models: The Sharpening Mechanism
arxiv.org/abs/2412.01951
📣I'm hiring PhD interns for combined theory+empirical projects in: exploration in post-training, multi-task learning in autoregressive models, distillation, reasoning beyond CoT.
Apply on the link below. If you're at #NeurIPS2024, message me to chat.
jobs.careers.microsoft.com/global/en/jo...
Applications to our Research Fellow position at
@flatironinstitute.org CCM are closing soon on Dec 15! It's a great place for doing fundamental ML research with a lot of freedom, in the heart of NYC. Apply here: apply.interfolio.com/155357
Generating cat videos is nice, but what if you could tackle real scientific problems with the same methods? 🧪🌌
Introducing The Well: 16 datasets (15TB) for Machine Learning, from astrophysics to fluid dynamics and biology.
🐙: github.com/PolymathicAI...
📜: openreview.net/pdf?id=00Sx5...
