Dan Cross

You jerk. Now that's stuck in my head.

Yup. So when do they predict these things top out?

Google is actively partnering with companies to build small scale nuclear power reactors directly at data center sites.

The breakthrough required is that they need to figure out how to make energy demand sublinear with respect to increased capability. I see no evidence suggesting that's anywhere close to being on the horizon.

But that's not sustainable.

I'm not sure I care about business viability: I care about environmental impact and sustainability. Sure, they can boil the oceans and burn all the coal and dam up all the rivers to generate the gigawatts to meet demand, and they can drain all the water tables to keep the computers cool, and profit.

I think it's a combination. I think it's some gains in efficiency (MoE, for instance), some increases in revenue, and a _lot_ of VC cash.

That's the thing: no one _really_ knows, and people are making a _lot_ of assumptions as they speculate. That does _not_ give me a warm fuzzy.

It seems to me, at a minimum, that a lot of optimism is predicated on the assumption of breaking some pretty fundamental barriers, and soon.

I see data that suggest that they are, and no evidence suggesting that they are not. And yeah, the consequences for the rest of us are going to be bad.

There's clearly utility here. Some impressive things can be done. The limitations are obvious, though.

Moore's law necessarily stopped because of physics: you can only double density so many times per unit volume before you hit fundamental limitations of physical reality. But LLMs just aren't the same; when do they top out?

Again, I go back to Moore's law: compute was driven by semiconductors getting twice as dense per unit area _at half the cost_ for ~40 years. There is nothing comparable for LLMs: an LLM may get twice as good in 2 years, but at 4x the cost, even after efficiency gain factors.

I know.

But I'm not sure I have seen enough evidence to agree, and some data I have seen suggests this is not true.

They're also huge and and can afford to divert the assets in the short term, banking they'll have an advantage once the bubbble bursts.

It's like how the US ran up the national debt to bankrupt the Soviet Union during the cold war.

Not really, no. The energy cost to produce new computers relative to volume of computation went _down_. That's not happening here at all. Energy cost to create new models is going _up_.

There's nothing remotely comparable to Moore's law for LLMs. Once the VC cash runs out, the cost will skyrocket.

I'm aware of the math, but it doesn't really change the point. Energy cost is growing superlinearly to increase the capacity of LLMs; that's not sustainable.

Compare to computers writ large. Contrast the energy efficiency of your laptop vs ENIAC, in comparison to the raw aggregate power of both.

Ironically, I just asked Gemini about this and it responded that a 2x increase in model capability will require a 10x increase in energy consumption. That is, linear increases in capability require exponentially more resources. That's not sustainable.

Did it hallucinate?

But it WILL slice a chunk of your thumb off...(Ask me how I know!).

The Wu is coming through.

Neighbors left their shovel out (which has been very handy as I don’t have one with a big scoop!) and my UniFi doorbell keeps periodically detecting it as a package.

Is this AI

Every day I am filled with hate because my heart overflows with my love of the computer. aftermath.site/ram-prices-h...

It's just... like... you know....vibes, man.

Having a linguistics background I think there's a huge misconception when it comes to prompting LLM's, the vast majority thinks that coding is "hard" compared with speaking a natural language. Well it's actually the contrary from a utilitarian POV less verbs at your disposal => easier to be specific

That's certainly true, but has anyone checked up on whether the cavalry is on its way yet? Cause we need that as a prerequisite as soon as possible.

I noticed the same thing. The decentralized, DIY, samizdat, word-of-mouth, and most importantly "we're in this together" ethos of the punk and hardcore youth movements of the 1980s and 90s are good models of resistance.

😶

OK "god bless america" and then naming every country in the americas from south to north is absolute king shit

I can't help but smile at all the racists and maga types who must have had their minds explode.

The maha food commercial with Mike Tyson immediately after was a bit of a letdown, though.