Symmetry breaking refer to a requirement of initializing machine learning models like neural networks.

When some machine learning models have weights all initialized to the same value, it can be difficult or impossible for the weights to differ as the model is trained. This is the “symmetry”.

Initializing the model to small random values breaks the symmetry and allows different weights to learn independently of each other.