(PCA): - Dimension reduction analysis
-> It is a technique for feature extraction from a given data set.
-> There are N-number of Principal Component corresponding to the N-number of data.
-> 95% of the features of the extracted data belong to the first principal component.
-> Therefore, we have to select the first n-number of the principal component corresponding to the N-number of data; the choice of the n-number principal component is determined by the precision we are aiming for.
-> So, PCA reduces the N-number of the principal components corresponding to the N-number of data into the n-number of the features; N >> n.
-> Consequently, another name for this method is a dimension reduction analysis.
-> Example: - Considering the situation for the data sets X and Y.
X = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10.
Y = 1, 4, 9, 16, 25, 36, 49, 64, 81, 100.
Here, the number of features = 2 and the number of samples = 10.
The steps for computing PCA are given as follows:
Step 1: Generate the covariance matrix for datasets X and Y.
| Cov (X, X) Cov (X, Y) |
A _{(2 X 2)} =
| Cov (Y, X) Cov (Y, Y) |
\begin{align}Cov{(X, Y)}=\sum_{i=1}^N\frac{(x_i-\mu_X)(y_i-\mu_Y)}{N}\end{align}
Where, \; \mu_X and \mu_Y are the mean of the given data sets X and Y respectively.
| 8.25 90.75 |
A _{(2 X 2)} =
| 90.75 1051.05 |
Step 2: Generate the characteristics equation by using covariance matrix A_{(2 X 2)}.
Note:- det (A_{(2 X 2)} - \lambda I ) = 0; represents the characteristics equation and I = unit matrix.
| 8.25 - \lambda 90.75 |
\det = 0
| 90.75 1051.05 - \lambda |
\implies (8.25 - \lambda) (1051.05 - \lambda) - 90.75 * 90.75 = 0
\implies \lambda^{2} - 1059.3 \lambda + 435.6 = 0 . . . (1)
\implies \lambda_{1}=1058.89, \lambda_{2}=0.411375 .
The \lambda_{1}, \lambda_{2} represents the Eigen Values of the matrix A_{(2 X 2)}.
The first principal component is defined by the largest eigenvalue, the second principal component by the second-largest eigenvalue, and so on.
Step 3: The computation of the Eigen Vectors corresponding to the Eigen Values.
(A_{(2 X 2)} - \lambda_{i} I) U_{i} = 0 . . . (2)
When \lambda_{1} = 1058.89, then the (A_{(2 X 2)} - \lambda_{i} I) U_{i} =
| -1050.64 90.75 | | u_{1} | | 0 |
=
| 90.75 -7.84 | | u_{2} | | 0 |
Now equating the matrix on both sides, we get.
-1050.64 * u_{1} + 90.75 * u_{2} = 0 . . . (3)
and 90.75 * u_{1} - 7.84 * u_{2} = 0 ... (4)
The Eigen Vectors corresponding to equations (3) and (4) are as follows:
|u_{1}| | 90.75 * k| |7.84*k|
= OR
|u_{2}| |1050.64*k| |90.75*k|
Where 'k=1' is a constant.
When \lambda_{2} = 0.411375, then the (A_{(2 X 2)} - \lambda_{i} I) U_{i} =
| 7.838625 90.75 | | u_{1} | | 0 |
=
| 90.75 1050.638625 | | u_{2} | | 0 |
Now equating the matrix on both sides, we get.
7.838625 * u_{1} + 90.75 * u_{2} = 0 . . . (5)
and
90.75 * u_{1} + 1050.638625 * u_{2} = 0 . . . (6)
The Eigen Vectors corresponding to equations (5) and (6) are as follows:
|u_{1}| |90.75*k| |1050.64*k|
= OR
|u_{2}| |-7.84*k| |-90.75*k|
Where 'k=1' is a constant.
.png)