spectral-bias.md

tags
from_paper

On the Spectral Bias of Neural Networks

src: OR

The theory of [[implicit-regularisation]] is used to explain why overparameterised neural networks don't overfit even when no explicit regularisation techniques are used. One of the key points is that we're working in the overparameterised regime – you have multiple interpolating solutions, but during training, neural networks tend to converge to the "simpler" solutions