People overuse the singularity as the term to describe the large acceleration in ai progress that'll come due to better agents. There are still real points of friction that the current models don't address. The singularity is catchy but misleading. Progress isn't infinite.
GPT 5.4 is the first time I've used codex for multiple hours straight and not ragequit back to claude code.
GPT 5.4 in codex cli/app is much more approachable than any of their models that came before. This is really big for them, excited to keep trying it vis a vis Claude as my agent daily driver.
Hoping OpenClaw convinces OpenAI to build GPT OSS 2. It’d be a great fit.
Must read on Chinese open source from Kevin Xu with the very similarly named substack (story for another time)
interconnect.substack.com/p/chinese-op...
We talk about open models as political insurance, the widening frontier gap, and the ever weirder futures of AI. This is a very important time for open models, the weight of it is obvious, but the economic challenges are so extreme.
New conversation on Interconnects w Dean Ball on why the Anthropic v DoW moment could strengthen the long-run case for open models - even if the next few years get rough for open.
www.interconnects.ai/p/how-anthro...
Waiting for deepseek v4
I've written up a blog post that explains why this matters and hybrid models didn't work a few years ago when Mamba was super popular. Plus, this paper is a great entry point for modern deep learning / language modeling scaling theory. Enjoy and send feedback!
www.interconnects.ai/p/olmo-hybri...
In particular, the OSS tools for these new architectures is really limited. New architectures are much slower than standard transformers or popular models like DeepSeek MoEs. This is work that we can do together to keep pushing the frontier of efficient, open models.
It's incredible timing to release a fully open model so people can study how these architecture changes impact the full stack.
Personally, I learned a lot in making the post-training work. Even with the data being identical for pretraining, post-training is very different!
Excited to share the latest Olmo model: Olmo Hybrid. This is a model with gated delta net (GDN) layers in a 3:1 ratio with full attention. It follows lots of other developments like Qwen 3.5 and Kimi Linear.
I’m doing my part to save Qwen.
Yes they dm me regularly.
Lots of core team members of Alibaba Qwen are resigning publicly on X.
The gaping hole that Qwen imploding would leave in the open research ecosystem will be hard to fill. The small models are irreplaceable.
I’ll do my best to keep carrying that torch. Every bit matters.
Latest open artifacts (#19): Qwen 3.5, GLM 5, MiniMax 2.5 — Chinese labs' latest push of the frontier
Welcome to the year of the horse! I always learn something new doing these w Florian.
www.interconnects.ai/p/latest-ope...
All of this happening with Anthropic/DoW etc will push a lot more investment in open models, so there’s transparency in the tools that’re being used across high stakes domains.
At the same time, these models won’t be received well if they’re built in an overly prescriptive way by any government.
It gives me a glimmer of hope in challenging times to see such a deeply respectable, principled stance being held in face of unjust pressure.
Doubly so to see so many I respect and admire standing in support of it.
Stay the course and stand with Anthropic.
If people are working on open research for scaling RL in llms i'd love to talk to you.
How much does distillation really matter for Chinese LLMs?
DeepSeek's usage was a rounding error. MiniMax's was substantial. But distillation is getting less important as RL takes over — it's easier to access "banned" APIs than to smuggle GPUs.
www.interconnects.ai/p/how-much-d...
Made a language model RL cheatsheet for the extra page on the inside back cover of the physical edition RLHF Book.
The RLHF Book should be sent off for printing in the next month or two.
Working on final edits and reviews :D.
Thanks all for your patience.
now that exceeded even my expectations, props Claude
Open models are in a perpetual race to stay relevant at the frontier. While they're doing better than I, and many experts would expect given the cost of models, I don't see evidence that open models are accelerating and surpassing the best closed models.
www.interconnects.ai/p/open-model...
Using the Claude app more due to the personality of the latest Opus and it’s under the radar how much better Claude’s search has gotten. The top end isn’t as good as GPT Thinking/Pro for research, but the speed is a big upside.
Here are my slides from my recent CMU talk, as I'm transitioning from the Olmo 3 era of just building a reasoning model to thinking about how to do impactful research for agentic systems.
docs.google.com/presentation...
First time at CMU
Fun to set up real analytics and learn that my RLHF Book pdf is downloaded 50-100 times a day from my site (doesnt include Arxiv downloads/views).
Thanks for reading!
Codex app is nice.
Im just a few minutes in and think it'll make some of the crazy things i was doing way easier to monitor.
