The letter

One letter a week, starting soon.

The week's signal, what shipped, what is worth running tonight, and the editor's note. No tracking, no ads, nothing else.

We keep your address, your language and the date you joined, nothing else. Every letter has a one-click unsubscribe link that deletes the record.

← Accept All   Archive
Qiita (LLM)Japan

事前学習モデルを事後学習しようとしたら入力埋め込みの勾配ノルムが infになって学習できなかったお話し

11.06

ことの経緯 llm-jp/llm-jp-4-8b-baseや、このモデルにCPT(継続事前学習)したikedachin/llm-jp-4-8b-base_cpt_v3にSFTをしようとしたら勾配異常が発生しました。 grad normが延々とinf・・・学習できへん😭 ...

Models & releases
Read at Qiita (LLM) ↗

Related

More from Qiita (LLM) on Accept All.