Scale, Standardize, and Normalize Data

Note: Contents and examples in this article are partially from Scikit-learn-Preprocessing data and faqs.org-Should I normalize/standardize/rescale the data

Scaling

Scaling a vector means to add/substract a constant, then multiply/divide by another constant, so the features can lie between given minimum and maximum values. The motivation to use this scaling include robustness to very small standard deviations of features and preserving zero entries in sparse data. Normally, the given range is [0,1]

$x^*_{i,j}=\frac{x_{i,j}-x^{min}_{j}}{x^{max}_{j}-x^{min}_{j}}$

For example, if we have a dataset like below,

$\left(\begin{array}{rrr}1&-1&2\2&0&0\0&1&-1\end{array}\right)$

The scaling process will be

take the first column vector x₀ = [1, 2, 0]^T;
x₀^max=2 and x₀^max=0
scale x_i,0,

$X=\begin{array}{lll}x_{0,0}&=\frac{x_{0,0}-0}{2}&=0.5\x_{1,0}&=\frac{x_{1,0}-0}{2}&=1\x_{2,0}&=\frac{x_{2,0}-0}{2}&=0\\end{array}$

Therefore, the scaling x₀ is [0.5, 1, 0]^T. Repeating the same process for x₁ and x₂, the scaling dataset is

$\left(\begin{array}{rrr}0.5&0&1\1&0.5&0.33\0&1&0\end{array}\right)$

Standardizing

Standardization of dataset enables the individual feature look like standard normally distributed data: Gaussian with zero mean and unit variance.

$x^*_{i,j}=\frac{x_{i,j}-\bar{x_j}}{\sigma}$

where:

x̄_j is the mean of the vector, and
σ is the standard deviation of the vector.

Let’t take X as the example again. The standardizing process will be

take the first column vector x₀ = [1, 2, 0]^T;
calculate the mean of x̄₀ = (1+2+0)/3 = 1
calculate the standard deviation of x₀,
$\sigma=\sqrt{\frac{(1-1)^2+(2-1)^2+(0-1)^2}{3}}=\sqrt{2/3}$
calculate

$\begin{array}{lllr}x_{0,0}&=\frac{x_{0,0}-1}{\sqrt{2/3}}&=&0.41\x_{1,0}&=\frac{x_{1,0}-1}{\sqrt{2/3}}&=&-0.41\x_{2,0}&=\frac{x_{2,0}-1}{\sqrt{2/3}}&=&0.82\\end{array}$

Therefore, the standardizing x₀ is [0, 1.22, -1.22]^T. Repeating the same process for x₁ and x₂, the standardizing dataset is

$\left(\begin{array}{rrr}0&-1.22&1.33\1.22&0&0.26\-1.22&1.221&-1.06\end{array}\right)$

Normalizing

Normalizing a vector is the process of scaling vectors to have unit norm. The motivation is to qualify the similarity of any pair of vectors while using dot-product.

$x^*_{i,j}=\frac{x_{i,j}}{|x_j|}$

Let’t take X as the example again. The scaling process will be

take the first row vector x₀ = [1, -1, 2];
calculate norm of x₀
$|x_0|=\sqrt{1^2+(-1)^2+2^2}=\sqrt{6}$
normalize x₀,

$\begin{array}{lllr}x_{0,0}&=\frac{x_{0,0}}{\sqrt{6}}&=&0.41\x_{0,1}&=\frac{x_{0,1}}{\sqrt{6}}&=&-0.41\x_{0,2}&=\frac{x_{0,2}}{\sqrt{6}}&=&0.82\\end{array}$

Therefore, the normalizing x₀ is [0.41, -0.41, 0.82]. Repeating the same process for x₁ and x₂, the normalizing dataset is

$\left(\begin{array}{rrr}0.41&-0.41&0.82\1&0&0\0&0.71&-0.71\end{array}\right)$

Scaling

Standardizing

Normalizing

Leave a Reply Cancel reply