Maximal update parametrization allows AI researchers to predict optimal hyperparameters for large neural network training runs using models that are 100 times smaller

Opening sign-in…