The letter

One letter a week, in your inbox.

The signal of the week, what shipped, what to try, and the editor's note. No tracking, no ads, nothing else.

We keep your address, your language and the date you joined, nothing else. Every letter has a one-click unsubscribe link that deletes the record.

← Accept All   Archive
AI Alignment ForumResearch

Training on probes: What's going on

September 8
Training on probes: What's going on

TL;DR If you train a probe for some property (like "honesty") and do gradient descent against this probe while continuing training that incentivizes dishonesty, the model will change its internal representation to evade

Read at AI Alignment Forum ↗

Related

More from AI Alignment Forum on Accept All.