The name is the button.

An agent writes the code now, and a human thumb hovers over Accept all a hundred times a day. The honest question of this decade is what, exactly, you just accepted. So I read the diff.

Eight years as a backend engineer in Stockholm, back in Tokyo since 2023, and I still ship something small most nights, usually badly. My one rule: the only real review of a model is the thing you built with it. That is why there is no leaderboard on this site.

Around a hundred and forty sources come through — the labs' own posts, the papers, the tools, the runtimes and languages nobody is watching yet, the chip under the demo, and the newsletters people actually forward. What is moving gets counted so the week reads at a glance, and every few days I explain one mechanism in plain words.

A story here is a headline, a short excerpt, one image and the link to whoever wrote it. We never reproduce the post; the images belong to their authors. A post written in Japanese appears in Japanese, exactly as it was published, and the link goes to the original.

— Yuki Halvorsen, Tokyo

← Accept All Archive

AI Alignment ForumResearch

Fixed-weight models are adversarially vulnerable: hence misaligned

September 28

This post argues that fixed-weight models (at least as we understand them today) will a) always be vulnerable to adversarial examples in their concept-spaces, and b) hence will be misaligned, under…

Read at AI Alignment Forum ↗

One letter a week, in your inbox.

Latest

Fixed-weight models are adversarially vulnerable: hence misaligned

More from AI Alignment Forum on Accept All

ResearchContinual learning might make your blocking monitors nearly useless AI Alignment Forum

ResearchLatent reasoning architectures would undermine CoT, our strongest oversight tool AI Alignment Forum

ResearchWhy I'm scared of RL AI Alignment Forum

ResearchWorkspaceBench: Evaluating Interpretability Methods for the Global Workspace AI Alignment Forum

ResearchShallow Beliefs: Midtraining does not inoculate against EM from reward hacking AI Alignment Forum

ResearchOp-Ed: I Worked at Google DeepMind. You Should Listen to the Warnings About AI AI Alignment Forum