b
This commit is contained in:
lda committed
1 parent
95b5e546dc
commit
6bc9d0fc69
6 files changed
+297
-28
No files matched your search
+20
-28
@@ -12,7 +12,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 27,
|
||||
"execution_count": 60,
|
||||
"id": "6015efdc",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -35,7 +35,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 28,
|
||||
"execution_count": 61,
|
||||
"id": "dcc03ce5",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -165,7 +165,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 29,
|
||||
"execution_count": 62,
|
||||
"id": "d8ddea0f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -214,7 +214,7 @@
|
||||
"type": "float"
|
||||
}
|
||||
],
|
||||
"ref": "b9750d7c-8269-4010-ab71-458e17630535",
|
||||
"ref": "68b8f569-e767-4e9b-9bbf-d434b264434a",
|
||||
"rows": [
|
||||
[
|
||||
"0",
|
||||
@@ -364,7 +364,7 @@
|
||||
"4 1.364217 -1.296612 -0.384658 -0.553006 3.030874 -1.303849 0.501984"
|
||||
]
|
||||
},
|
||||
"execution_count": 29,
|
||||
"execution_count": 62,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
@@ -386,7 +386,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 30,
|
||||
"execution_count": 63,
|
||||
"id": "42d915c7",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -425,7 +425,7 @@
|
||||
"type": "boolean"
|
||||
}
|
||||
],
|
||||
"ref": "a3b22897-f525-4245-890b-e98e5a69d041",
|
||||
"ref": "961f26d9-842d-4b3a-8b89-0defb6ea3200",
|
||||
"rows": [
|
||||
[
|
||||
"0",
|
||||
@@ -560,7 +560,7 @@
|
||||
"4 True "
|
||||
]
|
||||
},
|
||||
"execution_count": 30,
|
||||
"execution_count": 63,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
@@ -610,7 +610,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 31,
|
||||
"execution_count": 64,
|
||||
"id": "b155bb2a",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -687,7 +687,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 32,
|
||||
"execution_count": 65,
|
||||
"id": "7e89760b",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -760,7 +760,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 33,
|
||||
"execution_count": 66,
|
||||
"id": "b230b630",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -820,7 +820,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 34,
|
||||
"execution_count": 67,
|
||||
"id": "f66d9cfb",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -875,7 +875,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 35,
|
||||
"execution_count": 68,
|
||||
"id": "7ec9e853",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -896,7 +896,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 36,
|
||||
"execution_count": 69,
|
||||
"id": "17d051c3",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -918,7 +918,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 37,
|
||||
"execution_count": 70,
|
||||
"id": "d7550be2",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -967,7 +967,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 38,
|
||||
"execution_count": 71,
|
||||
"id": "03ecd5a4",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -996,7 +996,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 39,
|
||||
"execution_count": 72,
|
||||
"id": "08458903",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -1025,23 +1025,15 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "babec866",
|
||||
"id": "5eafddad",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Interpretation of PCA 2D Scatter\n",
|
||||
"\n",
|
||||
"For both datasets, the PCA 2D scatter plots (PC1 vs PC2) do not show clearly separate clusters corresponding to the labels:\n",
|
||||
"\n",
|
||||
"- In the apple dataset, ‘good’ and ‘bad’ quality apples overlap heavily in the first two principal components. This suggests that the directions of greatest variance in the data are not strongly aligned with the quality labels.\n",
|
||||
"\n",
|
||||
"- In the elevator dataset, the ‘peak hour’ vs ‘non-peak hour’ samples are also not clearly separable in the first two PCs. This indicates that the primary sources of variance in the elevator features do not correspond strongly with peak hour status.\n",
|
||||
"\n",
|
||||
"In both cases, PCA captures high variance directions, but because it is unsupervised, it does not necessarily capture class distinctions — especially when those distinctions don’t drive the strongest variance in the data.\n"
|
||||
"## Scree plot"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 40,
|
||||
"execution_count": 73,
|
||||
"id": "d210701d",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
|
||||
+91
@@ -604,3 +604,94 @@ plt.show()
|
||||
|
||||
|
||||
|
||||
|
||||
Interpretation of PCA 2D Scatter
|
||||
|
||||
For both datasets, the PCA 2D scatter plots (PC1 vs PC2) do not show clearly separate clusters corresponding to the labels:
|
||||
|
||||
- In the apple dataset, ‘good’ and ‘bad’ quality apples overlap heavily in the first two principal components. This suggests that the directions of greatest variance in the data are not strongly aligned with the quality labels.
|
||||
|
||||
- In the elevator dataset, the ‘peak hour’ vs ‘non-peak hour’ samples are also not clearly separable in the first two PCs. This indicates that the primary sources of variance in the elevator features do not correspond strongly with peak hour status.
|
||||
|
||||
In both cases, PCA captures high variance directions, but because it is unsupervised, it does not necessarily capture class distinctions — especially when those distinctions don’t drive the strongest variance in the data.
|
||||
|
||||
|
||||
|
||||
```python
|
||||
plt.plot(np.cumsum(pca_apple.explained_variance_ratio_))
|
||||
plt.xlabel("Number of PCs")
|
||||
plt.ylabel("Cumulative explained variance")
|
||||
plt.title("Apple explained variance")
|
||||
plt.show()
|
||||
|
||||
plt.plot(np.cumsum(pca_elev.explained_variance_ratio_))
|
||||
plt.xlabel("Number of PCs")
|
||||
plt.ylabel("Cumulative explained variance")
|
||||
plt.title("Elevator explained variance")
|
||||
plt.show()
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## **PCA Analysis Write-Up (for Apple & Elevator)**
|
||||
|
||||
<https://chatgpt.com/share/69561813-30c8-8012-994a-c6a44f8a2387>
|
||||
|
||||
### **Principal Component Selection**
|
||||
|
||||
To apply PCA, all numerical features were standardized so that each has mean 0 and unit variance. PCA then finds new orthogonal axes (principal components) ordered by how much variance in the data they capture. In both datasets, the first two principal components explain the largest portion of total variance, and these were selected for 2D visualization.
|
||||
|
||||
The principal components with highest variance represent the most significant directions of variation in the data. Lower-variance components represent subtler variation or noise. Using only the highest-variance components prioritizes the most meaningful structure, while using low-variance components typically does not reveal useful structure.
|
||||
|
||||
---
|
||||
|
||||
### **Effect of Number of Components**
|
||||
|
||||
Varying the number of principal components affects how much information about the original dataset is retained:
|
||||
|
||||
* With **2 components**, we can visualize the data in 2D while capturing a portion of the variance.
|
||||
* Increasing the number of components (e.g., to 3 or more) captures more of the dataset’s overall variance. This improves reconstruction fidelity, but makes visualization harder.
|
||||
* A scree plot (cumulative explained variance vs. number of components) can be used to select the number of components needed to reach a target variance threshold (e.g., 90%).
|
||||
|
||||
Overall, using more components retains more information, but the first few components often suffice to capture broad patterns.
|
||||
|
||||
---
|
||||
|
||||
### **Visualizing Data in 2D**
|
||||
|
||||
Below are the interpretations of the 2D PCA scatter plots:
|
||||
|
||||
#### **Apple Dataset**
|
||||
|
||||
In the Apple PCA scatter plot, the data points colored by quality (good vs. bad) are distributed in a cloud without clear separation between the groups. This indicates that the directions of greatest variance (first two principal components) do not create distinct clusters aligned with the quality labels. In other words, the main axes of variability in the dataset do not correspond strongly to the supervised quality classification.
|
||||
|
||||
> *Interpretation:* The lack of visible clusters suggests that apple quality is influenced by subtler combinations of features that are not captured in the top two principal components.
|
||||
|
||||
---
|
||||
|
||||
#### **Elevator Dataset**
|
||||
|
||||
Similarly, in the Elevator PCA plot where points are colored by peak hour vs. non-peak hour, the two groups are mixed throughout the 2D scatter. There is no obvious cluster separation along PC1 or PC2. This implies that peak hour status is not strongly associated with the dominant directions of variance in the dataset.
|
||||
|
||||
> *Interpretation:* Peak hour vs. non-peak hour differences do not define the principal axes of variation. Instead, other patterns (e.g., variation in wait time, load, floor requests) contribute more to overall variance.
|
||||
|
||||
---
|
||||
|
||||
### **Summary**
|
||||
|
||||
* PCA identifies directions of maximum variance without using label information (unsupervised).
|
||||
* In both datasets, the first two principal components do not produce clearly separated clusters by label.
|
||||
* This is expected when labels are not strongly linked to the features that account for the most variation.
|
||||
* Increasing the number of principal components would capture more information but may still not separate classes unless the class structure is aligned with variance.
|
||||
|
||||
BIN
Binary file not shown.
BIN
Binary file not shown.
+177
@@ -0,0 +1,177 @@
|
||||
---
|
||||
title: "**Principal Component Analysis**"
|
||||
author:
|
||||
- "Mai Minh Vũ - 22BI13477"
|
||||
- "Lương Đức Anh - 23BI14021"
|
||||
header-includes:
|
||||
- \usepackage{float}
|
||||
- \usepackage{tabularx}
|
||||
- \usepackage{booktabs}
|
||||
---
|
||||
|
||||
## **1. Study the Dataset**
|
||||
|
||||
### **1.1 Dataset Selection**
|
||||
|
||||
Two datasets with more than six dimensions were selected:
|
||||
|
||||
1. **Apple Quality Dataset**
|
||||
Source: [Kaggle – Apple Quality](https://www.kaggle.com/datasets/nelgiriyewithana/apple-quality)
|
||||
This dataset contains multiple numerical measurements describing apple characteristics and a quality label.
|
||||
|
||||
2. **Elevator Traffic Analytics Dataset**
|
||||
Source: [Kaggle – Elevator Traffic Analytics](https://www.kaggle.com/datasets/prince7489/elevator-traffic-analytics-dataset)
|
||||
This dataset contains elevator usage statistics including temporal, load, and traffic-related features.
|
||||
|
||||
Both datasets satisfy the dimensionality requirement and are suitable for PCA.
|
||||
|
||||
### **1.2 Feature Types**
|
||||
|
||||
#### **Apple Dataset**
|
||||
|
||||
| Feature | Discrete / Continuous | Quantitative / Qualitative | Numerical / Categorical | Explanation |
|
||||
| :------------ | :-------------------- | :------------------------- | :---------------------- | :---------------------------- |
|
||||
| `A_id` | Discrete | Qualitative | Categorical | Identifier, not a measurement |
|
||||
| `Size` | Continuous | Quantitative | Numerical | Standardized size value |
|
||||
| `Weight` | Continuous | Quantitative | Numerical | Standardized weight value |
|
||||
| `Sweetness` | Continuous | Quantitative | Numerical | Sensor-based score |
|
||||
| `Crunchiness` | Continuous | Quantitative | Numerical | Sensor-based score |
|
||||
| `Juiciness` | Continuous | Quantitative | Numerical | Sensor-based score |
|
||||
| `Ripeness` | Continuous | Quantitative | Numerical | Maturity score |
|
||||
| `Acidity` | Continuous | Quantitative | Numerical | Chemical property |
|
||||
| `Quality` | Discrete | Qualitative | Categorical | Target label (Good/Bad) |
|
||||
|
||||
Table: Apple Feature Classification
|
||||
|
||||
Negative values appear because the dataset is standardized (mean $\approx$ 0), not due to incorrect measurements.
|
||||
|
||||
#### **Elevator Dataset**
|
||||
|
||||
| Feature | Discrete / Continuous | Quantitative / Qualitative | Numerical / Categorical | Explanation |
|
||||
| :------------------ | :-------------------- | :------------------------- | :---------------------- | :------------------- |
|
||||
| `timestamp` | Continuous | Quantitative | Numerical | Time of request |
|
||||
| `floor_requested` | Discrete | QuFantitative | Numerical | Integer floor number |
|
||||
| `wait_time_seconds` | Continuous | Quantitative | Numerical | Waiting duration |
|
||||
| `direction` | Discrete | Qualitative | Categorical | Up/Down |
|
||||
| `people_count` | Discrete | Quantitative | Numerical | Number of passengers |
|
||||
| `peak_hour` | Discrete | Qualitative | Categorical | Peak / Non-peak |
|
||||
| `load_percent` | Continuous | Quantitative | Numerical | Elevator load |
|
||||
|
||||
Table: Elevator Feature Classification
|
||||
|
||||
Discrete numerical variables (e.g., floor number) are still quantitative because they represent measurable magnitude.
|
||||
|
||||
### **1.3 Data Quality and Preparation**
|
||||
|
||||
- **Apple dataset**
|
||||
|
||||
- No missing values
|
||||
- No incorrect or impossible values
|
||||
- Data is already standardized
|
||||
- Minimal preparation required
|
||||
- Identifier (`A_id`) removed before PCA
|
||||
|
||||
- **Elevator dataset**
|
||||
|
||||
- No missing values
|
||||
- Categorical features (`direction`) require encoding
|
||||
- Timestamp removed as it does not contribute to variance structure
|
||||
|
||||
Thus, light preprocessing was required mainly for encoding and feature selection.
|
||||
|
||||
### **1.4 Labels in the Dataset**
|
||||
|
||||
- **Apple dataset**:
|
||||
`Quality` (Good / Bad) is a clear label.
|
||||
|
||||
- **Elevator dataset**:
|
||||
`peak_hour` acts as a binary label (Yes / No).
|
||||
|
||||
PCA is an **unsupervised** method, so labels are not used to compute principal components, but they are used for visualization and interpretation.
|
||||
|
||||
### **1.5 Statistical Measures**
|
||||
|
||||
For numerical features, the following statistics were computed:
|
||||
|
||||
- **Mean**: average value
|
||||
- **Variance**: measure of spread
|
||||
- **Covariance**: joint variation between two features
|
||||
- **Correlation**: normalized covariance in range [-1, 1]
|
||||
|
||||
{latex-placement="H"}
|
||||
|
||||
{latex-placement="H"}
|
||||
|
||||
**Categorical features** do not have meaningful mean, variance, or covariance unless encoded numerically. Therefore, PCA and statistical analysis were applied only to numerical features.
|
||||
|
||||
### **1.6 Most Correlated Feature Pairs**
|
||||
|
||||
From the correlation analysis, we can see some correlated pairs of data points:
|
||||
|
||||
- **Apple dataset**
|
||||
|
||||
- Size $\leftrightarrow$ Sweetness
|
||||
- Sweetness $\leftrightarrow$ Ripeness
|
||||
- Juiciness $\leftrightarrow$ Crunchiness
|
||||
- Juiciness $\leftrightarrow$ Acidity
|
||||
- Sweetness $\leftrightarrow$ Juiciness
|
||||
- Ripeness $\leftrightarrow$ Acidity
|
||||
|
||||
These correlations are intuitive, as sweeter apples tend to be juicier, and acidity changes with ripeness.
|
||||
|
||||
- **Elevator dataset**
|
||||
|
||||
- People count $\leftrightarrow$ Load percent
|
||||
- Wait time $\leftrightarrow$ People count
|
||||
- Wait time $\leftrightarrow$ Load percent
|
||||
|
||||
Higher passenger volume naturally increases load and waiting time.
|
||||
|
||||
### **1.7 Missing Data**
|
||||
|
||||
No missing values were detected in either dataset. Therefore, no imputation or removal strategies were required.
|
||||
|
||||
## **2. Principal Component Analysis (PCA)**
|
||||
|
||||
### **2.1 Applying PCA**
|
||||
|
||||
Before applying PCA, all numerical features were standardized to ensure equal contribution. PCA was then applied to both datasets to transform the original features into orthogonal principal components ranked by explained variance.
|
||||
|
||||
### **2.2 Selection of Principal Components**
|
||||
|
||||
Principal components were selected based on **explained variance ratio**:
|
||||
|
||||
- Components with **high variance** capture meaningful structure
|
||||
- Components with **low variance** mainly capture noise
|
||||
|
||||
Using low-variance components leads to information loss and reduced interpretability.
|
||||
|
||||
### **2.3 Effect of Number of Components**
|
||||
|
||||
- Using **2 components** enables visualization but captures limited variance
|
||||
- Increasing components improves information retention
|
||||
- A scree plot can be used to identify how many components are required to retain ~90% variance
|
||||
|
||||
{latex-placement="H"}
|
||||
|
||||
{latex-placement="H"}
|
||||
|
||||
### **2.4 2D Visualization**
|
||||
|
||||
{latex-placement="H"}
|
||||
|
||||
{latex-placement="H"}
|
||||
|
||||
### **2.5 Interpretation of PCA Plots**
|
||||
|
||||
- **Apple dataset**:
|
||||
No clear separation between good and bad apples is visible. This indicates that apple quality is not strongly aligned with the directions of highest variance.
|
||||
|
||||
- **Elevator dataset**:
|
||||
Peak-hour and non-peak-hour data points overlap significantly, suggesting that peak hour status does not dominate the variance structure.
|
||||
|
||||
In both cases, the lack of clear clusters is expected because PCA does not use label information and focuses solely on variance.
|
||||
|
||||
## **Conclusion**
|
||||
|
||||
This study applied PCA to two real-world datasets. While PCA successfully reduced dimensionality and revealed variance structure, it did not produce clear label-based clusters. This highlights the distinction between unsupervised variance-based methods and supervised classification tasks.
|
||||
LFS
BIN
Binary file not shown.
Reference in new issue
Block a user