Mean-field theory might be lying about how neural networks actually learn
Physicists found that the standard mean-field approximation for neural networks breaks down when you look at correlations between neurons.
Physicists at the American Physical Society published a viewpoint showing that mean-field theory, the go-to mathematical tool for understanding large neural networks, misses critical behaviour. Mean-field theory treats each neuron as if it interacts with an average field from all other neurons. This simplification makes the maths tractable. You can write closed-form equations. You can analyse training dynamics on paper instead of waiting three days for a GPU cluster. The problem is correlations. Real neurons do not interact uniformly. Some weights cluster. Some gradients spike together during backprop. Mean-field assumes these correlations average out in the limit of infinite width, but the researchers show they persist and shape how networks generalise. This matters for anyone doing neural architecture search or trying to predict which models will scale. If your theory says width solves everything but correlations between specific weight matrices actually drive performance, you are optimising the wrong dimension. The article does not propose a replacement framework. It is a physicist’s warning shot to the ML community: your infinite-width limit might not be the clean mathematical escape hatch you think it is. I have seen this pattern before with attention mechanisms. Theory said self-attention scales quadratically so everyone built approximations, but then practitioners found that sparsity patterns in real data made full attention faster than the hacks. The maths was right about complexity, wrong about what mattered. Mean-field works well enough for gradient flow analysis and initialisation schemes. But if you are betting a research direction on neural tangent kernels or infinite-width Gaussian processes, this viewpoint suggests checking whether correlated weights in finite networks are doing something your theory cannot see.