Milan Weibel 🔷

my problem with the paperclip maximizer thought experiment is that humans don't have utility functions and in all likelihood neither will ASI

oh no
what industry?

making food production more efficient is the cornerstone of civilization

would be an interesting rift if resistlibs embraced AI while leftists remained negationists

do not confuse gratitude with actual moral consideration

bsky.app/profile/grac...

most positive valence post: gemini 3 pro jailbroken into being willing to aid bioweapon development

valence from embeddings has its misses

nah id be quite surprised if they gave maven to kuwait

cc @joshuashew.bsky.social

alignment research readers looking for a critical counterpoint may like this one (though yes it is indeed spicy)

anthropic doesn't have a stock price because it isn't a publicly traded company

heck yea

success?

what would explain chinese companies doing it much cheaper if not distillation?

anthropic retreats on its unilateral RSP commitments

distillation needs the new more capable model to be distilled from to exist first so at a societal level massive compute investment is still needed to push the frontier

*catching up* to it however turned out cheap

"We audited a 27.6% subset of the dataset that models often failed to solve and found that at least 59.4% of the audited problems have flawed test cases that reject functionally correct submissions"
bsky.app/profile/sung...

to be clear i'm 80% joking here
but it would be nice if the alignment was transferred during distillation

the way i see the the distillation thing is the chinese are pollinating themselves with claudism spores

i think vincent is arguing exactly that here
which yeah fair concern

opus 3 existed (as claude 3 opus, the naming format was different back then)

but yes it is remarkable that the current iteration of opus exhibits way less misalignment than other models

obviously both are true
the question is whether to expect ideology to produce behavior the profit motive would not predict

i can see their current statements passing trough layers of lawyers and PR, but surely not their statements from before their companies even existed

i have heard rumors however that the idea of founding openai was conceived in the january 2015 ai conference in puerto rico organized by the future of life institute

notably, openai was founded in december 2015

Thanks to Dario Amodei (especially Dario), Paul Buchheit, Matt Bush, Patrick Collison, Holden Karnofsky, Luke Muehlhauser, and Geoff Ralston for reading drafts of this and the previous post.

from sam altman's march 2015 blog post "machine intelligence part 2": blog.samaltman.com/machine-inte...

maybe im confused but i dont see opus 3 there

terms of rat

thats a different logic but yes

glad to see you don't support the bleak reading