Skip to content

Translation in progress Some entries still include text in the original language.

Failure-First·Indexed by this site · Originally published

论文主张:深度学习将出现自己的科学理论 Learning Mechanics

[Daily Paper] There Will Be a Scientific Theory of Deep Learning

AI summary

论文主张深度学习将出现一门可预测的科学理论,作者称之为 Learning Mechanics。文中列出五类支撑进展:Deep Linear Networks、Kernel Regression 等可解设定,Mean-Field 无限宽极限,Neural Scaling Laws 与 Edge of Stability 等宏观规律,以 μP 为核心的超参数理论,以及不同架构收敛到相似内部表示的 Platonic 普适性。作者还区分 lazy 与 rich 两类训练阶段,提出 Discretization Hypothesis,并称 μP 支撑的 μTransfer 可把小模型调好的超参数零样本迁移到大模型。

Read originalfailurefirst.org