claude Cooked

lw1/README.md

@@ -0,0 +1,19 @@
+# T1 - Principal component analysis
+
+## Study the dataset
+
+1. Choose 2 datasets from Kaggle or any source in Internet with more than 6 dimensions.
+1. For each dataset, determine which feature is discrete or continuous? quantitative or qualitative? numerical or categorical? Explain.
+1. What are the issues lying within data? for example: missing data, incorrect data? Do you
+1. need to prepare data?
+1. Is there any label in the datasets? Explain.
+1. Calculate mean, variance, covariance, correlation of the selected datasets. How do you calculate these measures for categorical features (if any)?
+1. Find the most corelated couple of features of each dataset. Comment on the results.
+   Is there any missing data? How do you handle this issue?
+
+## PCA
+
+1. Apply PCA on the two selected datasets. Describe how you select the principal components.
+   Is there any difference while using principal components with highest and lowest values?
+1. Vary the number of used principal components, analyze and comment on the obtained results.
+1. Visualize data distribution in 2D.

lw2/README.md

@@ -0,0 +1,19 @@
+# Labwork 2 - Clustering
+
+## K-means
+
+Select two datasets from UCI, and for each of them:
+
+1. Run experiments with k-means. Explain the experimental protocol.
+2. Explain the centroid initialization of the k-means used in the first question.
+3. Analyze and compare the results with different values of k.
+4. Calculate the clustering quality (any criteria in slides).
+
+## Subspace clustering
+
+1. Select one high-dimensional dataset (>100 features)
+2. Use PCA or SVD to visualize the data distribution in 2D/3D.
+3. Apply any clustering method (DBSCAN, K-means, SOM, GNG, etc.) for the selected datasets
+   in 2D/3D after using PCA/SVD. Compare the performance before and after the dimensionality reduction.
+4. Select a random subspace of the dataset. Re-apply any clustering method on the subspace.
+   Discuss and compare the performance with the previous results.

lw2/c.bib

@@ -5,6 +5,9 @@
   howpublished = {UCI Machine Learning Repository},
   note         = {{DOI}: https://doi.org/10.24432/C53W3X}
 }
+
+Redmond, Michael. 2002. Communities and Crime. UCI Machine Learning Repository. https://doi.org/10.24432/C53W3X.
+
 @misc{abalone_1,
   author       = {Nash, Warwick, Sellers, Tracy, Talbot, Simon, Cawthorn, Andrew, and Ford, Wes},
   title        = {{Abalone}},
@@ -12,10 +15,15 @@
   howpublished = {UCI Machine Learning Repository},
   note         = {{DOI}: https://doi.org/10.24432/C55C7W}
 }
+
+Nash, Warwick, Tracy Sellers, Simon Talbot, Andrew Cawthorn, and Wes Ford. 1994. Abalone. UCI Machine Learning Repository. https://doi.org/10.24432/C55C7W.
+
 @misc{covertype_31,
   author       = {Blackard, Jock},
   title        = {{Covertype}},
   year         = {1998},
   howpublished = {UCI Machine Learning Repository},
   note         = {{DOI}: https://doi.org/10.24432/C50K5N}
-}
+}
+
+Blackard, Jock. 1998. Covertype. UCI Machine Learning Repository. https://doi.org/10.24432/C50K5N.

lw2/lw2.ipynb

@@ -2,7 +2,7 @@
  "cells": [
   {
    "cell_type": "code",
-   "execution_count": 116,
+   "execution_count": 32,
    "id": "e377bc9b",
    "metadata": {},
    "outputs": [],
@@ -13,14 +13,18 @@
     "from sklearn.cluster import KMeans, MiniBatchKMeans\n",
     "from sklearn.decomposition import PCA\n",
     "from sklearn.preprocessing import StandardScaler\n",
-    "from sklearn.metrics import silhouette_score\n",
+    "from sklearn.metrics import (\n",
+    "    silhouette_score,\n",
+    "    davies_bouldin_score,\n",
+    "    calinski_harabasz_score,\n",
+    ")\n",
     "\n",
-    "from pathlib import Path\n"
+    "from pathlib import Path"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 117,
+   "execution_count": 33,
    "id": "d7275d15",
    "metadata": {},
    "outputs": [
@@ -30,7 +34,7 @@
        "WindowsPath('c:/Users/Admin/Documents/lecture ass shit/3/ml2/lw2')"
       ]
      },
-     "execution_count": 117,
+     "execution_count": 33,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -51,7 +55,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 118,
+   "execution_count": 34,
    "id": "4de18adb",
    "metadata": {},
    "outputs": [],
@@ -78,7 +82,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 119,
+   "execution_count": 35,
    "id": "50b3eb52",
    "metadata": {},
    "outputs": [],
@@ -105,45 +109,28 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 120,
+   "execution_count": 36,
    "id": "a89a6974",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "([0.4307184563069171,\n",
-       "  0.3308881531841743,\n",
-       "  0.34454881088271005,\n",
-       "  0.3330973959780588,\n",
-       "  0.37350567261284556,\n",
-       "  0.39200932042425607,\n",
-       "  0.3905976225423938,\n",
-       "  0.38566484346143776,\n",
-       "  0.3910649022379036],\n",
-       " [18066.009207127645,\n",
-       "  13418.727667404704,\n",
-       "  11157.455698492053,\n",
-       "  9330.808564389003,\n",
-       "  7660.230080453761,\n",
-       "  6764.331897239782,\n",
-       "  5939.9031003732325,\n",
-       "  5315.323604570619,\n",
-       "  4805.410264465898])"
+       "'silhouettes = [0.4307184563069171, 0.3308881531841743, 0.34454881088271005, 0.3330973959780588, 0.37350567261284556, 0.39200932042425607, 0.3905976225423938, 0.38566484346143776, 0.3910649022379036], inertias = [18066.00920712765, 13418.727667404704, 11157.455698492053, 9330.808564389003, 7660.230080453761, 6764.331897239782, 5939.903100373233, 5315.323604570618, 4805.410264465897]'"
       ]
      },
-     "execution_count": 120,
+     "execution_count": 36,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "silhouettes, inertias"
+    "f\"{silhouettes = }, {inertias = }\""
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 121,
+   "execution_count": 37,
    "id": "c70bfa21",
    "metadata": {},
    "outputs": [],
@@ -154,7 +141,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 122,
+   "execution_count": 38,
    "id": "926a255e",
    "metadata": {},
    "outputs": [
@@ -179,7 +166,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 123,
+   "execution_count": 39,
    "id": "7b8c1750",
    "metadata": {},
    "outputs": [
@@ -204,7 +191,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 124,
+   "execution_count": 40,
    "id": "acab8d95",
    "metadata": {},
    "outputs": [
@@ -227,6 +214,33 @@
     "plt.show()\n"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 41,
+   "id": "b7447f18",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Abalone: n_samples=4177, n_features=9\n",
+      "Best k: 2\n",
+      "Cluster sizes: [1940 2237]\n",
+      "Davies-Bouldin Index: 0.8836\n",
+      "Calinski-Harabasz Index: 4512.6320\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Abalone dataset stats\n",
+    "print(f\"Abalone: n_samples={X.shape[0]}, n_features={X.shape[1]}\")\n",
+    "print(f\"Best k: {best_k}\")\n",
+    "print(f\"Cluster sizes: {np.bincount(best_labels)}\")\n",
+    "print(f\"Davies-Bouldin Index: {davies_bouldin_score(X, best_labels):.4f}\")\n",
+    "print(f\"Calinski-Harabasz Index: {calinski_harabasz_score(X, best_labels):.4f}\")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "3841acb2",
@@ -237,7 +251,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 125,
+   "execution_count": 42,
    "id": "92b692bc",
    "metadata": {},
    "outputs": [],
@@ -252,7 +266,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 126,
+   "execution_count": 43,
    "id": "e6bbfcb7",
    "metadata": {},
    "outputs": [],
@@ -277,41 +291,28 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 127,
+   "execution_count": 44,
    "id": "05d4bd31",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "([-0.030689560868985374,\n",
-       "  0.014267829228328097,\n",
-       "  0.052170441001039715,\n",
-       "  0.0763160133542805,\n",
-       "  0.05712772597563029,\n",
-       "  0.11281790580873496,\n",
-       "  0.1241172489653034],\n",
-       " [1033855.8510931745,\n",
-       "  983031.2462602488,\n",
-       "  933199.3143504256,\n",
-       "  914886.5211138967,\n",
-       "  889334.3440412064,\n",
-       "  868852.814198654,\n",
-       "  848656.9204411206])"
+       "'silhouettes = [-0.030689560868985374, 0.014267829228328097, 0.052170441001039715, 0.0763160133542805, 0.05712772597563029, 0.11281790580873496, 0.1241172489653034], inertias = [1033855.8510931745, 983031.2462602488, 933199.3143504256, 914886.5211138967, 889334.3440412064, 868852.814198654, 848656.9204411206]'"
       ]
      },
-     "execution_count": 127,
+     "execution_count": 44,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "silhouettes, inertias"
+    "f\"{silhouettes = }, {inertias = }\""
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 128,
+   "execution_count": 45,
    "id": "da938557",
    "metadata": {},
    "outputs": [
@@ -336,7 +337,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 129,
+   "execution_count": 46,
    "id": "0070102e",
    "metadata": {},
    "outputs": [
@@ -361,7 +362,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 130,
+   "execution_count": 47,
    "id": "c312a24a",
    "metadata": {},
    "outputs": [],
@@ -370,6 +371,33 @@
     "best_labels = labels_per_k[best_k]"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 48,
+   "id": "71ae758d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Covertype: n_samples=20000, n_features=54\n",
+      "Best k: 8\n",
+      "Cluster sizes: [1407 6092 3757 2258 1139 4226   88 1033]\n",
+      "Davies-Bouldin Index: 2.3425\n",
+      "Calinski-Harabasz Index: 714.6171\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Covertype dataset stats\n",
+    "print(f\"Covertype: n_samples={X.shape[0]}, n_features={X.shape[1]}\")\n",
+    "print(f\"Best k: {best_k}\")\n",
+    "print(f\"Cluster sizes: {np.bincount(best_labels)}\")\n",
+    "print(f\"Davies-Bouldin Index: {davies_bouldin_score(X, best_labels):.4f}\")\n",
+    "print(f\"Calinski-Harabasz Index: {calinski_harabasz_score(X, best_labels):.4f}\")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "aecfb074",
@@ -380,7 +408,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 131,
+   "execution_count": 49,
    "id": "bc26af01",
    "metadata": {},
    "outputs": [],
@@ -408,7 +436,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 132,
+   "execution_count": 50,
    "id": "a3e10a92",
    "metadata": {},
    "outputs": [
@@ -434,6 +462,32 @@
     "plt.show()\n"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 51,
+   "id": "1a38e68b",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Communities: n_samples=1994, n_features=122\n",
+      "Components for 90% variance: 30\n",
+      "First 2 components explain: 35.0%\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Communities dataset stats + PCA 90% variance\n",
+    "print(f\"Communities: n_samples={X.shape[0]}, n_features={X.shape[1]}\")\n",
+    "n_90 = np.argmax(np.cumsum(pca.explained_variance_ratio_) >= 0.90) + 1\n",
+    "print(f\"Components for 90% variance: {n_90}\")\n",
+    "print(\n",
+    "    f\"First 2 components explain: {pca.explained_variance_ratio_[:2].sum() * 100:.1f}%\"\n",
+    ")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "a7acc3d1",
@@ -444,7 +498,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 133,
+   "execution_count": 52,
    "id": "69a1249a",
    "metadata": {},
    "outputs": [
@@ -477,7 +531,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 134,
+   "execution_count": 53,
    "id": "f41491b4",
    "metadata": {},
    "outputs": [],
@@ -501,7 +555,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 135,
+   "execution_count": 54,
    "id": "873caac5",
    "metadata": {},
    "outputs": [],
@@ -526,7 +580,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 136,
+   "execution_count": 55,
    "id": "6846b929",
    "metadata": {},
    "outputs": [
@@ -561,7 +615,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 137,
+   "execution_count": 56,
    "id": "fc244f0c",
    "metadata": {},
    "outputs": [
@@ -594,7 +648,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 138,
+   "execution_count": 57,
    "id": "813e9b68",
    "metadata": {},
    "outputs": [],
@@ -607,7 +661,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 139,
+   "execution_count": 58,
    "id": "6aa93bec",
    "metadata": {},
    "outputs": [],
@@ -630,7 +684,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 140,
+   "execution_count": 59,
    "id": "1fb883f8",
    "metadata": {},
    "outputs": [
@@ -658,40 +712,84 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 141,
+   "execution_count": 60,
    "id": "a54ce8fb",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "([0.1447573886443752,\n",
-       "  0.1513815804645033,\n",
-       "  0.1622363759648628,\n",
-       "  0.10473845344821518,\n",
-       "  0.11854956392780605,\n",
-       "  0.11689825990588008],\n",
-       " [0.3885783888959868,\n",
-       "  0.43308922486937373,\n",
-       "  0.3730220026311381,\n",
-       "  0.38759443998496274,\n",
-       "  0.35733354404743983,\n",
-       "  0.36750626178240353],\n",
-       " [0.16609258945009867,\n",
-       "  0.13111313797495522,\n",
-       "  0.13898110095002836,\n",
-       "  0.12515895253959083,\n",
-       "  0.12109768667928446,\n",
-       "  0.10798315908348614])"
+       "'sil_orig = [0.1447573886443752, 0.1513815804645033, 0.1622363759648628, 0.10473845344821518, 0.11854956392780605, 0.11689825990588008], sil_pca = [0.3885783888959868, 0.43308922486937373, 0.3730220026311381, 0.38759443998496274, 0.35733354404743983, 0.36750626178240353], sil_sub = [0.16609258945009867, 0.13111313797495522, 0.13898110095002836, 0.12515895253959083, 0.12109768667928446, 0.10798315908348614]'"
       ]
      },
-     "execution_count": 141,
+     "execution_count": 60,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "sil_orig, sil_pca, sil_sub"
+    "f\"{sil_orig = }, {sil_pca = }, {sil_sub = }\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 61,
+   "id": "271dc568",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Communities & Crime - Full Metrics Table\n",
+      "----------------------------------------------------------------------\n",
+      "k    Orig Sil     PCA Sil      Rand Sil    \n",
+      "----------------------------------------------------------------------\n",
+      "2    0.1448       0.3886       0.1661      \n",
+      "3    0.1514       0.4331       0.1311      \n",
+      "4    0.1622       0.3730       0.1390      \n",
+      "5    0.1047       0.3876       0.1252      \n",
+      "6    0.1185       0.3573       0.1211      \n",
+      "7    0.1169       0.3675       0.1080      \n"
+     ]
+    }
+   ],
+   "source": [
+    "# Full metrics table for all k values - Communities\n",
+    "print(\"Communities & Crime - Full Metrics Table\")\n",
+    "print(\"-\" * 70)\n",
+    "print(f\"{'k':<4} {'Orig Sil':<12} {'PCA Sil':<12} {'Rand Sil':<12}\")\n",
+    "print(\"-\" * 70)\n",
+    "for i, k in enumerate(ks):\n",
+    "    print(f\"{k:<4} {sil_orig[i]:<12.4f} {sil_pca[i]:<12.4f} {sil_sub[i]:<12.4f}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 62,
+   "id": "d6dcd4f5",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Best PCA k: 3\n",
+      "Cluster sizes (PCA): [786 253 955]\n",
+      "Davies-Bouldin (PCA): 0.7937\n",
+      "Calinski-Harabasz (PCA): 1704.2171\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Best PCA clustering - additional metrics\n",
+    "best_pca_k = ks[np.argmax(sil_pca)]\n",
+    "best_pca_labels = labels_pca[best_pca_k]\n",
+    "\n",
+    "print(f\"Best PCA k: {best_pca_k}\")\n",
+    "print(f\"Cluster sizes (PCA): {np.bincount(best_pca_labels)}\")\n",
+    "print(f\"Davies-Bouldin (PCA): {davies_bouldin_score(Xp, best_pca_labels):.4f}\")\n",
+    "print(f\"Calinski-Harabasz (PCA): {calinski_harabasz_score(Xp, best_pca_labels):.4f}\")"
    ]
   }
  ],

lw2/report.md

@@ -7,34 +7,7 @@ header-includes:
   - \usepackage{float}
   - \usepackage{tabularx}
   - \usepackage{booktabs}
-references:
-- author:
-  - family: Redmond
-    given: Michael
-  id: communities_and_crime_183
-  issued: 2002
-  note: "DOI: https://doi.org/10.24432/C53W3X"
-  publisher: UCI Machine Learning Repository
-  title: "[Communities and Crime]{.nocase}"
-- author:
-  - family: Nash
-    given: Sellers
-    suffix: Warwick
-  - family: Ford
-    given: Wes
-  id: abalone_1
-  issued: 1994
-  note: "DOI: https://doi.org/10.24432/C55C7W"
-  publisher: UCI Machine Learning Repository
-  title: Abalone
-- author:
-  - family: Blackard
-    given: Jock
-  id: covertype_31
-  issued: 1998
-  note: "DOI: https://doi.org/10.24432/C50K5N"
-  publisher: UCI Machine Learning Repository
-  title: Covertype
+bibliography: "c.bib"
 nocite: "[@*]"
 ---
 
@@ -52,6 +25,15 @@ We applied K-means clustering to the Abalone dataset. Preprocessing included:
 
 We varied k from 2 to 10, used the **k-means++** centroid initialization, and ran each clustering with multiple initializations (`n_init = 10`) to reduce random seed sensitivity.
 
+### Centroid Initialization: k-means++
+
+The k-means++ algorithm improves upon random initialization by selecting initial centroids that are spread out across the data:
+
+1. **First centroid**: chosen uniformly at random from all data points.
+2. **Subsequent centroids**: each new centroid is selected with probability proportional to $D(x)^2$, where $D(x)$ is the distance from point $x$ to the nearest already-chosen centroid.
+
+This weighted sampling ensures centroids are unlikely to start close together, leading to faster convergence and better final solutions compared to naive random initialization.
+
 ### Results
 
 #### Silhouette Analysis
@@ -70,6 +52,31 @@ The elbow plot showing inertia (within-cluster sum of squares) across k:
 
 The silhouette score peaked at _k = 2_, indicating that this value of k yields the best separation given the dataset. The elbow plot shows diminishing returns after this point. Overall, the clustering quality suggests moderate structure in the data.
 
+### Dataset Statistics
+
+| Metric                  | Value         |
+| ----------------------- | ------------- |
+| Samples                 | 4,177         |
+| Features                | 9             |
+| Best k                  | 2             |
+| Cluster sizes           | 1,940 / 2,237 |
+| Davies-Bouldin Index    | 0.8836        |
+| Calinski-Harabasz Index | 4,512.63      |
+
+### Full Results Table
+
+| k   | Silhouette | Inertia (WCSS) |
+| --- | ---------- | -------------- |
+| 2   | **0.431**  | 18,066         |
+| 3   | 0.331      | 13,419         |
+| 4   | 0.345      | 11,157         |
+| 5   | 0.333      | 9,331          |
+| 6   | 0.374      | 7,660          |
+| 7   | 0.392      | 6,764          |
+| 8   | 0.391      | 5,940          |
+| 9   | 0.386      | 5,315          |
+| 10  | 0.391      | 4,805          |
+
 ---
 
 ## 1.2 Dataset: Covertype [@covertype_31]
@@ -98,6 +105,38 @@ We tested k from 2 to 8, tracked inertia and silhouette scores, and recorded res
 
 The Covertype silhouette scores remained relatively low across k values, indicating weak separation in the original feature space. The elbow plot does not show a clear bend, further suggesting that classic k-means struggles with high dimensionality and feature noise in this dataset.
 
+### Dataset Statistics
+
+| Metric                  | Value                                                      |
+| ----------------------- | ---------------------------------------------------------- |
+| Samples                 | 20,000 (subsampled)                                        |
+| Features                | 54                                                         |
+| Best k                  | 8                                                          |
+| Cluster sizes           | 1,407 / 6,092 / 3,757 / 2,258 / 1,139 / 4,226 / 88 / 1,033 |
+| Davies-Bouldin Index    | 2.3425                                                     |
+| Calinski-Harabasz Index | 714.62                                                     |
+
+### Full Results Table
+
+| k   | Silhouette | Inertia (WCSS) |
+| --- | ---------- | -------------- |
+| 2   | -0.031     | 1,033,856      |
+| 3   | 0.014      | 983,031        |
+| 4   | 0.052      | 933,199        |
+| 5   | 0.076      | 914,887        |
+| 6   | 0.057      | 889,334        |
+| 7   | 0.113      | 868,853        |
+| 8   | **0.124**  | 848,657        |
+
+### Comparison of Datasets
+
+| Dataset   | Best k | Silhouette | Davies-Bouldin | Calinski-Harabasz |
+| --------- | ------ | ---------- | -------------- | ----------------- |
+| Abalone   | 2      | 0.431      | 0.88 (better)  | 4,513 (better)    |
+| Covertype | 8      | 0.124      | 2.34           | 715               |
+
+The low silhouette scores for Covertype suggest the dataset may benefit from dimensionality reduction or alternative clustering methods (e.g., DBSCAN for density-based clusters).
+
 ---
 
 # Part 2 — Subspace Clustering
@@ -114,6 +153,15 @@ Before clustering:
 - Replaced missing values (`?`) with column means.
 - Standard scaled all features.
 
+### Dataset Statistics
+
+| Metric                      | Value |
+| --------------------------- | ----- |
+| Samples                     | 1,994 |
+| Features                    | 122   |
+| Components for 90% variance | 30    |
+| First 2 PCs explain         | 35.0% |
+
 ## 2.2 PCA Variance Explained
 
 We applied PCA to analyze variance distribution across components and determine an appropriate dimensionality for visualization/clustering.
@@ -145,16 +193,54 @@ For the PCA space, the best silhouette score occurred at _k = 3_. The clusters a
 
 ## 2.5 Random Subspace Clustering
 
-We also selected a random subset of 30 features and repeated k-means clustering, comparing silhouette scores with previous methods. This helped assess the impact of arbitrary feature selection vs. PCA-based dimensionality reduction.
+We also selected a random subset of 30 features (out of 100+) and repeated k-means clustering. This simulates a naive dimensionality reduction approach without considering feature importance or variance.
 
 ### Random Subspace Results
 
 The random subspace silhouette scores are included in the comparison plot above.
 
+### Discussion
+
+#### Full Comparison Table
+
+| k   | Original | PCA 2D    | Random 30 |
+| --- | -------- | --------- | --------- |
+| 2   | 0.145    | 0.389     | 0.166     |
+| 3   | 0.151    | **0.433** | 0.131     |
+| 4   | 0.162    | 0.373     | 0.139     |
+| 5   | 0.105    | 0.388     | 0.125     |
+| 6   | 0.119    | 0.357     | 0.121     |
+| 7   | 0.117    | 0.368     | 0.108     |
+
+#### Best PCA Clustering (k=3)
+
+| Metric                  | Value           |
+| ----------------------- | --------------- |
+| Cluster sizes           | 786 / 253 / 955 |
+| Davies-Bouldin Index    | 0.7937          |
+| Calinski-Harabasz Index | 1,704.22        |
+
+**Key findings:**
+
+- **PCA outperforms random subspace**: By projecting onto directions of maximum variance, PCA preserves cluster structure better than random feature selection.
+- **Random subspace is unstable**: Different random seeds would yield different subsets, leading to inconsistent clustering results.
+- **Dimensionality reduction is beneficial**: Both methods outperform the original high-dimensional space, confirming that reducing noise dimensions improves cluster separability.
+
 ---
 
 # Conclusion
 
-We successfully ran k-means clustering on two UCI datasets (Abalone and Covertype). For Abalone, moderate cluster structure was identified, while Covertype presented weak separability in the original space.
+We successfully ran k-means clustering on two UCI datasets:
 
-In the subspace clustering section, PCA significantly improved clustering quality relative to both original and random subspace clustering. This demonstrates that **dimensionality reduction can reveal latent structure** that is otherwise obscured in high-dimensional data.
+- **Abalone** (4,177 samples, 9 features): Best at k=2 with silhouette 0.431, Davies-Bouldin 0.88, showing moderate cluster structure.
+- **Covertype** (20,000 samples, 54 features): Best at k=8 with silhouette 0.124, indicating weak separability due to high dimensionality.
+
+For subspace clustering on **Communities & Crime** (1,994 samples, 122 features):
+
+- PCA to 2D achieved the highest silhouette (0.433 at k=3), a 3× improvement over the original space (0.151).
+- Random 30-feature subspace performed worse (0.131 at k=3) than PCA, demonstrating that informed dimensionality reduction outperforms arbitrary feature selection.
+- Only 30 PCA components are needed to capture 90% of variance, while the first 2 components explain 35%.
+
+This confirms that **dimensionality reduction can reveal latent cluster structure** otherwise obscured by the curse of dimensionality.
+
+# References