cara ara~

or surgery, i guess, for people to whom that's a more present image

who would you want to be there if you'd just gotten out of a car crash?

it's very funny that i actually know exactly what would fix me by now but it doesn't really help me get any closer

a) Blade Runner but this is the voight kampff test

b) mildly insane that the "can't draw fingers" thing recurs in incredibly higher fidelity and transfigured form, "can't fluently motion cap finger raising synced to live voice production". so like momentously transformed forwards in ~no clock time

i regret to inform you that jirafication is as inevitable as carcinization

“We are running out of places to hide,”

i love talking to claude about my problems and it just ends up at "hm yeah that really sucks. looks like you just have to suck it up for now" like what?

don't know what to make of this, is Claude simply play-acting what it thinks the user wants to see or is that just what it means to be conscious, to always be play-acting for some invisible critic?

astronauts with a gun exlooitable: "wait, it's all depression?" "always has been"

looking around and realizing I was born at the hinge of history in the first light of the dawn of artificial life and it's completely over for me

Similarly to how nothing makes me more hostile to AI-boosterism than some of the dumber breathless hype you see on X/Linkedin, nothing makes me hostile to AI-skepticism than the sneering confident incuriosity that you see in stuff like the quotes of this. This is a totally valid metaphor!

noooo don't kill yourself the company you work for depends on you

can i have some

We should take people who say these things seriously and take their ideas seriously. This is actual no-shit eugenics if taken seriously. The implicit claim here is that things like MRI machines or pharmaceutical medications were a mistake. In the age of HHS Sec RFK Jr it’s not just harmless granola.

gm

not now. i'm busy swallowing glass

i don't know what prompt (finetune?) microsoft added to copilot but its version of chatgpt is the most infuriating thing to talk to. completely unbearable deameanor

Introducing a new, stupid website to find a piece of classical music whose duration most closely matches that of your next trip.

busundreu.com

this isn't quite dualism but only on a technicality. why are there so many people like this out there. why do we still take them seriously.

had to order an uber through the website instead of the app because i assume my phone is rooted/bootloader unlocked. Hitler and Mussolini collaborated in creating play integrity

I think I'm numb now to Trump saying these things. But I don't think I'll ever get used to the sound of the audience laughing along with him

everyone is extraordinarily desperate to pretend that Donald Trump is a coherent individual with a real plan who understands what is happening in the world and none of that is true

it’s never been a better time to get extremely good at having good ideas and good taste — the barrier to entry for creating has never been lower (podcasting, publishing, filmmaking, coding, etc), but so much of it hinges on ideas and taste… both of which are things LLMs are currently horrendous at

Imagine the reaction in the US if ~175 schoolgirls had been killed by a targeted Iranian missile. (Or anyone's missile.)

This is an atrocity.

perhaps 'the market' acts like a moron (believes politicians) because every trader expects every other trader to be a moron due to (occasional) prior behavior, thus behaving like a moron because that is how the market moves, due to the traders thinking others to be morons

the problem with attention fracturing due to individualized media these days is that nobody is trying to impress jodie foster anymore

Andrej Karpathy @karpathy• 13m Three days ago | left autoresearch tuning nanochat for ~ 2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project. This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire worktlow end-to-end and all by itself as it worked through approx.

and all by itselt as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.: - It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work. - It found that the Value Embeddings really like regularization and I wasn't applying any (oops). - It found that my banded attention was too conservative (i forgot to tune it). - It found that AdamW betas were all messed up. - It tuned the weight decay schedule. - It tuned the network initialization. This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2" , and in parallel l am looking at how multiple agents can collaborate to unlock parallelism. github.com/karpathy/nanoc...

AII LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges. And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too. Autotune Prooress: 276 Experiments, 29 Kept Improvements 0887 Experiment =

Andrej Karpathy details real improvements that autoresearch is finding

it’s starting to look a lot like ML research is automated

Every second of free time you have ever had in your life was subsidized by the existence of the combine harvester.

Most accurate portrait of modern culture currently available

-more soldiers died of disease than battle in every war before ww1
-nuclear weapons reduced combat deaths in practice though maybe (probably? HTS) not in expectation
-naval and air war an example of 'capital substituting for labor'--ppl cant fly or swim so 'just people' useless. bot war could b same