Normalization (machine learning): Difference between revisions

Revision as of 21:30, 26 September 2024

Revision as of 21:39, 26 September 2024

Line 31:

# Mean and variance of each feature

mu = np.mean(x, axis=0) # shape (N,)

sigma2var = np.var(x, axis=0) # shape (N,)

# Normalize the activations

x_hat = (x - mu) / np.sqrt(sigma2var + epsilon) # shape (B, N)

# Apply the linear transform