Newish work (arXived in December):
Prompts can be ambig., but handling ambiguity is context/user dependent. Sometimes the right thing is to ask a clarifying question, sometimes to give multi. answers, and sometimes to just guess. Can we train steerable models that change their strategy per context?
For more discussion, please see the paper! arxiv.org/abs/2602.24188
While AI models may struggle to collaborate, at Google DeepMind my collaborators are proactive and fully coherent. Thanks to @fantinehuot.bsky.social , Adam Fisch, @jonathanberant.bsky.social, and Mirella Lapata!
AI systems are also overconfident, terminating dialogues long before exhausting their turn budget - even after explicit reminders.
On most games, performance is flat or even decreasing. What went wrong?
Using the classic NLP toolbox, we find that AI models suffer from low discourse coherence, leading to weak performance despite relatively high information density - even when using twice as many tokens as humans.
So how well do today's models do?
To answer this, we design a new multi-turn scaling analysis, called *isotoken evaluation*: fix a total token budget, and partition it into variable numbers of turns.
Performance should be non-decreasing in the number of turns... and yet!
We believe these games are more naturalistic and proactive than most existing multi-turn evaluations, which often employ user simulators to create multi-turn user-assistant scenarios.
Here's another game, which requires answering a question about two privately-held images.
This task is part of 🏓MT-PingEval, a new benchmark of verifiable collaborative private information games that involve multi-turn dialogue.
In this game, the "describer" sees only a single image, and the "guesser" has to identify which one it is.
Are AI models effective collaborators, or mere assistants awaiting your next command? (Preprint: arxiv.org/abs/2602.24188)
To find out, we make AI collaborate with itself, in private information games: tasks that require sharing private information, like this chess board ordering task.
This looks like it’ll be a fantastic intro to transformers ⚡️
[1/n] Just wrapped up 7 months interning with @pcastr.bsky.social at Google DeepMind and I'm so excited to share our work: arxiv.org/abs/2602.10324.
TLDR: We used LLM-powered program synthesis to automatically model and discover differences between human and LLM strategic behavior
🚨 🔬 PhD positions at Google DeepMind in France 🇫🇷
We are advertising Master Level Intern positions at Google DeepMind within our Frontier AI Unit.
These could lead to co-advised PhD positions with Google DeepMind and French academic institutions.
job-boards.greenhouse.io/deepmind/job...
Hello! 👋
Are you interested in AI for board games using language models? Want to do some hobby tinkering with fine-tuning or RL?
We've released an easy-to-follow example colab that fine-tunes Gemma models via Kauldron to mimic an MCTS player.
Details here: github.com/google-deepm...
♟️🎲♦️♠️♥️♣️✨🎉
Day 1 of #BooksAreMyJam!
Blueberry Maple jam, with Linguaphile: A life of language love by Julie Sedivy.
A classic Canadian flavour duo + this book about @juliesedivy.bsky.social's relationship with language through her childhood in Montreal, later research as a linguist, and more
![Dwarkesh Patel @dwarkesh_sp
X.com
"The thing that happened with AGI and pretraining is that in some sense they overshot the target.
You will realize that a human being is not an AGI.
Because a human being lacks a huge amount of knowledge. Instead, we rely on continual learning.
If I produce a super intelligent 15-year-old, they don't know very much at all. A great student, very eager. [You can say,] 'You go and be a programmer. You go and be a doctor. Go and learn.'
So you could imagine that the deployment itself will involve some kind of a learning trial and error period. It's a process as opposed to, you drop the finished thing."
@ilyasut](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:ckaz32jwl6t2cno6fmuw2nhn/bafkreic7t5vazon6lavac3vywxuyvs65pfzz35a6t74v2u6u75dytzm3di)
Dwarkesh Patel @dwarkesh_sp X.com "The thing that happened with AGI and pretraining is that in some sense they overshot the target. You will realize that a human being is not an AGI. Because a human being lacks a huge amount of knowledge. Instead, we rely on continual learning. If I produce a super intelligent 15-year-old, they don't know very much at all. A great student, very eager. [You can say,] 'You go and be a programmer. You go and be a doctor. Go and learn.' So you could imagine that the deployment itself will involve some kind of a learning trial and error period. It's a process as opposed to, you drop the finished thing." @ilyasut
this is the theme — you can’t have AGI without existing in and learning from the real world
Photo of Cornelll University building surrounded by colorful trees
No better time to start learning about that #AI thing everyone's talking about...
📢 I'm recruiting PhD students in Computer Science or Information Science @cornellbowers.bsky.social!
If you're interested, apply to either department (yes, either program!) and list me as a potential advisor!
knowing how to tie your shoes or order a drink in a crowded bar: not agi
naming the big five personality traits: definitely agi
nice summary of everybody’s new fave
Nicholas Carlini asking the right questions at #COLM2025
Here’s a #COLM2025 feed!
Pin it 📌 to follow along with the conference this week!
Screenshot of paper title: Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence
AI always calling your ideas “fantastic” can feel inauthentic, but what are sycophancy’s deeper harms? We find that in the common use case of seeking AI advice on interpersonal situations—specifically conflicts—sycophancy makes people feel more right & less willing to apologize.
Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformers
Excited to share a new paper that aims to narrow the conceptual gap between the idealized notion of Kolmogorov complexity and practical complexity measures for neural networks.
🛸
thanks! i was more confused about the “kugel” part but TIL that this is apparently inspired by an airy globe?
Cannot stress enough how good it is that you can come across a post about gorgeous little Yiddish book sitting in someone’s family collection, and within a few seconds you can find the full scanned version of the book available for free through the Yiddish Book Center’s website
i think my great grandmother was the last owner of these books that knew how to read them
yiddish book cover
automatic translation: autonomy by dr. b hoffman
found some books at my parents’ house
On the positive side, this vario grinder, which i bought second hand, is the best technological upgrade of the summer in my house.
(Its grind settings are 1-10, a-z, so the chatgpt output is clearly wrong and the claude output is nonsensical)
