Decomposition¶
Decomposition is the process of using an orthogonal transformation to convert a set of observations of possibly-correlated variables (with numerical values) into a set of values of linearly-uncorrelated variables called principal components.
Since some algorithms are sensitive to correlated predictors, it can be a good idea to use the PCA (Principal Component Analysis: Decomposition Technique) before applying the algorithm. Since some algorithms are also sensitive to the number of predictors, we’ll have to be picky with which variables we include.
To demonstrate data decomposition in vastorbit, we’ll use the well-known iris dataset.
from vastorbit.datasets import load_iris
iris = load_iris()
iris.head(100)
123 sepallengthcmDecimal(5,2) | 123 sepalwidthcmDecimal(5,2) | 123 petallengthcmDecimal(5,2) | 123 petalwidthcmDecimal(5,2) | Abc speciesVarchar(30) | |
|---|---|---|---|---|---|
| 1 | 5.1 | 3.5 | 1.4 | 0.2 | Iris-setosa |
| 2 | 4.9 | 3.0 | 1.4 | 0.2 | Iris-setosa |
| 3 | 4.7 | 3.2 | 1.3 | 0.2 | Iris-setosa |
| 4 | 4.6 | 3.1 | 1.5 | 0.2 | Iris-setosa |
| 5 | 5.0 | 3.6 | 1.4 | 0.2 | Iris-setosa |
| 6 | 5.4 | 3.9 | 1.7 | 0.4 | Iris-setosa |
| 7 | 4.6 | 3.4 | 1.4 | 0.3 | Iris-setosa |
| 8 | 5.0 | 3.4 | 1.5 | 0.2 | Iris-setosa |
| 9 | 4.4 | 2.9 | 1.4 | 0.2 | Iris-setosa |
| 10 | 4.9 | 3.1 | 1.5 | 0.1 | Iris-setosa |
| 11 | 5.4 | 3.7 | 1.5 | 0.2 | Iris-setosa |
| 12 | 4.8 | 3.4 | 1.6 | 0.2 | Iris-setosa |
| 13 | 4.8 | 3.0 | 1.4 | 0.1 | Iris-setosa |
| 14 | 4.3 | 3.0 | 1.1 | 0.1 | Iris-setosa |
| 15 | 5.8 | 4.0 | 1.2 | 0.2 | Iris-setosa |
| 16 | 5.7 | 4.4 | 1.5 | 0.4 | Iris-setosa |
| 17 | 5.4 | 3.9 | 1.3 | 0.4 | Iris-setosa |
| 18 | 5.1 | 3.5 | 1.4 | 0.3 | Iris-setosa |
| 19 | 5.7 | 3.8 | 1.7 | 0.3 | Iris-setosa |
| 20 | 5.1 | 3.8 | 1.5 | 0.3 | Iris-setosa |
| 21 | 5.4 | 3.4 | 1.7 | 0.2 | Iris-setosa |
| 22 | 5.1 | 3.7 | 1.5 | 0.4 | Iris-setosa |
| 23 | 4.6 | 3.6 | 1.0 | 0.2 | Iris-setosa |
| 24 | 5.1 | 3.3 | 1.7 | 0.5 | Iris-setosa |
| 25 | 4.8 | 3.4 | 1.9 | 0.2 | Iris-setosa |
| 26 | 5.0 | 3.0 | 1.6 | 0.2 | Iris-setosa |
| 27 | 5.0 | 3.4 | 1.6 | 0.4 | Iris-setosa |
| 28 | 5.2 | 3.5 | 1.5 | 0.2 | Iris-setosa |
| 29 | 5.2 | 3.4 | 1.4 | 0.2 | Iris-setosa |
| 30 | 4.7 | 3.2 | 1.6 | 0.2 | Iris-setosa |
| 31 | 4.8 | 3.1 | 1.6 | 0.2 | Iris-setosa |
| 32 | 5.4 | 3.4 | 1.5 | 0.4 | Iris-setosa |
| 33 | 5.2 | 4.1 | 1.5 | 0.1 | Iris-setosa |
| 34 | 5.5 | 4.2 | 1.4 | 0.2 | Iris-setosa |
| 35 | 4.9 | 3.1 | 1.5 | 0.1 | Iris-setosa |
| 36 | 5.0 | 3.2 | 1.2 | 0.2 | Iris-setosa |
| 37 | 5.5 | 3.5 | 1.3 | 0.2 | Iris-setosa |
| 38 | 4.9 | 3.1 | 1.5 | 0.1 | Iris-setosa |
| 39 | 4.4 | 3.0 | 1.3 | 0.2 | Iris-setosa |
| 40 | 5.1 | 3.4 | 1.5 | 0.2 | Iris-setosa |
| 41 | 5.0 | 3.5 | 1.3 | 0.3 | Iris-setosa |
| 42 | 4.5 | 2.3 | 1.3 | 0.3 | Iris-setosa |
| 43 | 4.4 | 3.2 | 1.3 | 0.2 | Iris-setosa |
| 44 | 5.0 | 3.5 | 1.6 | 0.6 | Iris-setosa |
| 45 | 5.1 | 3.8 | 1.9 | 0.4 | Iris-setosa |
| 46 | 4.8 | 3.0 | 1.4 | 0.3 | Iris-setosa |
| 47 | 5.1 | 3.8 | 1.6 | 0.2 | Iris-setosa |
| 48 | 4.6 | 3.2 | 1.4 | 0.2 | Iris-setosa |
| 49 | 5.3 | 3.7 | 1.5 | 0.2 | Iris-setosa |
| 50 | 5.0 | 3.3 | 1.4 | 0.2 | Iris-setosa |
| 51 | 7.0 | 3.2 | 4.7 | 1.4 | Iris-versicolor |
| 52 | 6.4 | 3.2 | 4.5 | 1.5 | Iris-versicolor |
| 53 | 6.9 | 3.1 | 4.9 | 1.5 | Iris-versicolor |
| 54 | 5.5 | 2.3 | 4.0 | 1.3 | Iris-versicolor |
| 55 | 6.5 | 2.8 | 4.6 | 1.5 | Iris-versicolor |
| 56 | 5.7 | 2.8 | 4.5 | 1.3 | Iris-versicolor |
| 57 | 6.3 | 3.3 | 4.7 | 1.6 | Iris-versicolor |
| 58 | 4.9 | 2.4 | 3.3 | 1.0 | Iris-versicolor |
| 59 | 6.6 | 2.9 | 4.6 | 1.3 | Iris-versicolor |
| 60 | 5.2 | 2.7 | 3.9 | 1.4 | Iris-versicolor |
| 61 | 5.0 | 2.0 | 3.5 | 1.0 | Iris-versicolor |
| 62 | 5.9 | 3.0 | 4.2 | 1.5 | Iris-versicolor |
| 63 | 6.0 | 2.2 | 4.0 | 1.0 | Iris-versicolor |
| 64 | 6.1 | 2.9 | 4.7 | 1.4 | Iris-versicolor |
| 65 | 5.6 | 2.9 | 3.6 | 1.3 | Iris-versicolor |
| 66 | 6.7 | 3.1 | 4.4 | 1.4 | Iris-versicolor |
| 67 | 5.6 | 3.0 | 4.5 | 1.5 | Iris-versicolor |
| 68 | 5.8 | 2.7 | 4.1 | 1.0 | Iris-versicolor |
| 69 | 6.2 | 2.2 | 4.5 | 1.5 | Iris-versicolor |
| 70 | 5.6 | 2.5 | 3.9 | 1.1 | Iris-versicolor |
| 71 | 5.9 | 3.2 | 4.8 | 1.8 | Iris-versicolor |
| 72 | 6.1 | 2.8 | 4.0 | 1.3 | Iris-versicolor |
| 73 | 6.3 | 2.5 | 4.9 | 1.5 | Iris-versicolor |
| 74 | 6.1 | 2.8 | 4.7 | 1.2 | Iris-versicolor |
| 75 | 6.4 | 2.9 | 4.3 | 1.3 | Iris-versicolor |
| 76 | 6.6 | 3.0 | 4.4 | 1.4 | Iris-versicolor |
| 77 | 6.8 | 2.8 | 4.8 | 1.4 | Iris-versicolor |
| 78 | 6.7 | 3.0 | 5.0 | 1.7 | Iris-versicolor |
| 79 | 6.0 | 2.9 | 4.5 | 1.5 | Iris-versicolor |
| 80 | 5.7 | 2.6 | 3.5 | 1.0 | Iris-versicolor |
| 81 | 5.5 | 2.4 | 3.8 | 1.1 | Iris-versicolor |
| 82 | 5.5 | 2.4 | 3.7 | 1.0 | Iris-versicolor |
| 83 | 5.8 | 2.7 | 3.9 | 1.2 | Iris-versicolor |
| 84 | 6.0 | 2.7 | 5.1 | 1.6 | Iris-versicolor |
| 85 | 5.4 | 3.0 | 4.5 | 1.5 | Iris-versicolor |
| 86 | 6.0 | 3.4 | 4.5 | 1.6 | Iris-versicolor |
| 87 | 6.7 | 3.1 | 4.7 | 1.5 | Iris-versicolor |
| 88 | 6.3 | 2.3 | 4.4 | 1.3 | Iris-versicolor |
| 89 | 5.6 | 3.0 | 4.1 | 1.3 | Iris-versicolor |
| 90 | 5.5 | 2.5 | 4.0 | 1.3 | Iris-versicolor |
| 91 | 5.5 | 2.6 | 4.4 | 1.2 | Iris-versicolor |
| 92 | 6.1 | 3.0 | 4.6 | 1.4 | Iris-versicolor |
| 93 | 5.8 | 2.6 | 4.0 | 1.2 | Iris-versicolor |
| 94 | 5.0 | 2.3 | 3.3 | 1.0 | Iris-versicolor |
| 95 | 5.6 | 2.7 | 4.2 | 1.3 | Iris-versicolor |
| 96 | 5.7 | 3.0 | 4.2 | 1.2 | Iris-versicolor |
| 97 | 5.7 | 2.9 | 4.2 | 1.3 | Iris-versicolor |
| 98 | 6.2 | 2.9 | 4.3 | 1.3 | Iris-versicolor |
| 99 | 5.1 | 2.5 | 3.0 | 1.1 | Iris-versicolor |
| 100 | 5.7 | 2.8 | 4.1 | 1.3 | Iris-versicolor |
Notice that all the predictors are well-correlated with each other.
iris.corr()
Let’s compute the PCA of the different elements.
from vastorbit.machine_learning.vast import PCA
model = PCA()
model.fit(
iris,
[
"PetalLengthCm",
"SepalWidthCm",
"SepalLengthCm",
"PetalWidthCm",
],
)
Let’s compute the correlation matrix of the result of the PCA.
model.transform().corr()
Notice that the predictors are now independant and combined together and they have the exact same amount of information than the previous variables. Let’s look at the accumulated explained variance of the PCA components.
model.explained_variance_ratio_
Most of the information is in the first two components with more than 97.7% of explained variance. We can export this result to a VastFrame.
model.transform(n_components = 2)
123 sepallengthcmDecimal(5,2) | 123 sepalwidthcmDecimal(5,2) | 123 petallengthcmDecimal(5,2) | 123 petalwidthcmDecimal(5,2) | Abc speciesVarchar(30) | 123 col0Decimal(38,33) | 123 col1Decimal(38,33) | |
|---|---|---|---|---|---|---|---|
| 1 | 5.1 | 3.5 | 1.4 | 0.2 | Iris-setosa | -2.6842071251039528 | 0.32660731476437427 |
| 2 | 4.9 | 3.0 | 1.4 | 0.2 | Iris-setosa | -2.7153906156341328 | -0.1695568475560382 |
| 3 | 4.7 | 3.2 | 1.3 | 0.2 | Iris-setosa | -2.8898195396179185 | -0.1373456096050428 |
| 4 | 4.6 | 3.1 | 1.5 | 0.2 | Iris-setosa | -2.746437197308736 | -0.3111243157520073 |
| 5 | 5.0 | 3.6 | 1.4 | 0.2 | Iris-setosa | -2.7285929818313193 | 0.3339245635684391 |
| 6 | 5.4 | 3.9 | 1.7 | 0.4 | Iris-setosa | -2.279897361009604 | 0.7477827132251201 |
| 7 | 4.6 | 3.4 | 1.4 | 0.3 | Iris-setosa | -2.8208906821806323 | -0.0821045110246971 |
| 8 | 5.0 | 3.4 | 1.5 | 0.2 | Iris-setosa | -2.626481993323823 | 0.17040534896027557 |
| 9 | 4.4 | 2.9 | 1.4 | 0.2 | Iris-setosa | -2.8879585653356346 | -0.5707980263316074 |
| 10 | 4.9 | 3.1 | 1.5 | 0.1 | Iris-setosa | -2.6738446867191237 | -0.10669170375275287 |
| 11 | 5.4 | 3.7 | 1.5 | 0.2 | Iris-setosa | -2.5065267893389094 | 0.6519350136725583 |
| 12 | 4.8 | 3.4 | 1.6 | 0.2 | Iris-setosa | -2.6131427182710594 | 0.021520631960241694 |
| 13 | 4.8 | 3.0 | 1.4 | 0.1 | Iris-setosa | -2.787433975997099 | -0.22774018887112002 |
| 14 | 4.3 | 3.0 | 1.1 | 0.1 | Iris-setosa | -3.225200446274981 | -0.503279909485443 |
| 15 | 5.8 | 4.0 | 1.2 | 0.2 | Iris-setosa | -2.6435432169411546 | 1.1861948994134386 |
| 16 | 5.7 | 4.4 | 1.5 | 0.4 | Iris-setosa | -2.383869323799385 | 1.3447543445598482 |
| 17 | 5.4 | 3.9 | 1.3 | 0.4 | Iris-setosa | -2.6225262031258154 | 0.8180896745965832 |
| 18 | 5.1 | 3.5 | 1.4 | 0.3 | Iris-setosa | -2.648322732479131 | 0.319136667750872 |
| 19 | 5.7 | 3.8 | 1.7 | 0.3 | Iris-setosa | -2.19907796143077 | 0.8792440880917256 |
| 20 | 5.1 | 3.8 | 1.5 | 0.3 | Iris-setosa | -2.5873461889177434 | 0.5204736388059529 |