claude Cooked Again
This commit is contained in:
lda committed
1 parent
6b91d6efca
commit
e0eeb3a1e9
12 files changed
+819
-4
No files matched your search
+1
-1
@@ -1,8 +1,8 @@
|
|||||||
---
|
---
|
||||||
title: "**Principal Component Analysis**"
|
title: "**Principal Component Analysis**"
|
||||||
author:
|
author:
|
||||||
- "Mai Minh Vũ - 22BI13477"
|
|
||||||
- "Lương Đức Anh - 23BI14021"
|
- "Lương Đức Anh - 23BI14021"
|
||||||
|
- "Mai Minh Vũ - 22BI13477"
|
||||||
header-includes:
|
header-includes:
|
||||||
- \usepackage{float}
|
- \usepackage{float}
|
||||||
- \usepackage{tabularx}
|
- \usepackage{tabularx}
|
||||||
|
|||||||
+1
-1
@@ -1,8 +1,8 @@
|
|||||||
---
|
---
|
||||||
title: Clustering Analysis Report
|
title: Clustering Analysis Report
|
||||||
author:
|
author:
|
||||||
- "Mai Minh Vũ - 22BI13477"
|
|
||||||
- "Lương Đức Anh - 23BI14021"
|
- "Lương Đức Anh - 23BI14021"
|
||||||
|
- "Mai Minh Vũ - 22BI13477"
|
||||||
header-includes:
|
header-includes:
|
||||||
- \usepackage{float}
|
- \usepackage{float}
|
||||||
- \usepackage{tabularx}
|
- \usepackage{tabularx}
|
||||||
|
|||||||
LFS
BIN
Binary file not shown.
@@ -0,0 +1,15 @@
|
|||||||
|
@misc{seeds_236,
|
||||||
|
author = {Charytanowicz, Magorzata, and Niewczas, Jerzy, and Kulczycki, Piotr, and Kowalski, Piotr, and Lukasik, Szymon},
|
||||||
|
title = {{Seeds}},
|
||||||
|
year = {2010},
|
||||||
|
howpublished = {UCI Machine Learning Repository},
|
||||||
|
note = {{DOI}: https://doi.org/10.24432/C5H30K}
|
||||||
|
}
|
||||||
|
|
||||||
|
@misc{banknote_authentication_267,
|
||||||
|
author = {Lohweg, Volker},
|
||||||
|
title = {{Banknote Authentication}},
|
||||||
|
year = {2012},
|
||||||
|
howpublished = {UCI Machine Learning Repository},
|
||||||
|
note = {{DOI}: https://doi.org/10.24432/C55P57}
|
||||||
|
}
|
||||||
+326
@@ -0,0 +1,326 @@
|
|||||||
|
{
|
||||||
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 3,
|
||||||
|
"id": "d2aa9221",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"\n",
|
||||||
|
"===== Banknote Authentication =====\n",
|
||||||
|
"[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0024271844660194164, 0.009708737864077666] [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0024271844660194164, 0.009708737864077666]\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkgAAAHHCAYAAABEEKc/AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAWaJJREFUeJzt3Qd8k9X6wPEn3VA2hZZCgVLK3nspKggq7gmiIKA4EQQvCgro1SsKF0UURVw4QBS9ohcRRUABQUYZgrIpm5bdsrrz/zynN/m3aQtNacn6fT+f9H3z5s2bk5M0eXLOc85rsVqtVgEAAICd3/+vAgAAgAAJAAAgH7QgAQAAOCBAAgAAcECABAAA4IAACQAAwAEBEgAAgAMCJAAAAAcESAAAAA4IkIALeOGFF8RiscixY8eopyK66qqrpEmTJm5Xf7Vr15YHHnjAZe8pb/Hrr7+a5/P111+7uihAsSJAglubMWOG+fDNealatapcffXV8uOPP4q3OnfunPki1S+fknLq1CkJCQkxdbply5ZLOtahQ4dMeTds2CDuZMWKFaZc+ly97fXzFraAMTw83NRbfoHsjTfemGub7bNg0qRJBX5mrF27tlCBXUGX2bNnF8OzgycLcHUBgML45z//KdHR0aKnDkxMTDQfgjfccIP897//zfPh6Q30i+LFF1+0t8CUhDlz5pgvgoiICJk5c6a8/PLLlxQgaXn1y6xFixbiTgGSlktbiipUqJDrtm3btomfn99lf/2ef/55efbZZ0vkcT3ZkSNH5N1335URI0YU+j4TJ06URx99VEqXLl3kx33yySelbdu2ebZ37NixyMeEdyBAgke4/vrrpU2bNvbrgwYNMr84v/jiC68MkC6Hzz//3ASZtWrVklmzZl1SgOSJgoODXfK4AQEB5oLcNLDWgOexxx6TUqVKFWp/bbGcNm2aDB8+vMjVecUVV8idd97p1H2ysrIkLS3NtMA6Onv2rISGhsqlBtiXEvSheNDFBo+krQH6Ier4RfPvf/9bOnXqJJUrVza3t27dOt/cCG05eeKJJ2Tu3LkmP0a/LBs3biwLFiy46GPv3btX6tata+6nrVk582z+/vtv0/2nH27Vq1eXCRMm5PtL2Rbg6Qds8+bN5ZNPPrHfvmfPHqlSpYpZ11YIW5O/dkXYbN261XyoV6pUyRxDg8fvv/++0PW3b98+WbZsmfTu3dtc4uPjTWtLYfN09PnaWka0q8L2C3zAgAH28morX06FqZvU1FQZN26cqV99TaKiomTkyJFmu7Ovn9bXP/7xD7OurY+2cmn9FvTctCvuqaeeMrfpMWvUqCH9+vWz56Dpl+LYsWPN+6p8+fLmi1C/YJcsWVLo1y+/HKSMjAx56aWXJCYmxjyuPv7o0aPzPG9bd9Py5culXbt25rWvU6eOfPrpp+JOtNxaTq2j/N5X+dF61f8nbUUqjM6dO8s111xj3kfnz5+XkmR7v2lLq77P9DXS95qtO++3334zgZ12/+t7xuadd96x7x8ZGSmPP/54nu5e22dHXFycXHnlleb/Q197uB4BEjxCUlKS+ZI6evSo/PXXX6ZZ/cyZM3Lffffl2u/NN9+Uli1bmi65V155xQRQd911l/zwww95jqlfMvqhpgGCfsimpKTIHXfcIcePHy+wHLt27TIfYmXLljWBgQY5NidPnpTrrrvOBDyaG9GgQQN55plncuVK6Qe5fiB+9tln0rdvX/OLWb9E9Itay670y9X2JXHbbbeZffVy++23m236/Dt06GDyhrSrRh9Lv6hvvfVW+fbbbwtVn9rypvfRLzH9otUvZv3wL4qGDRua+laDBw+2l1fryZm60V/lN998swlyb7rpJnnrrbfMc3rjjTfknnvucfr10/rq06ePWddj2MplC14c6ftJgx193B49epjX45FHHjHB6IEDB8w+ycnJ8sEHH5jX8LXXXjPBjr4ne/bsac+/utjrl58HH3zQBAitWrUyZe3atauMHz/ePDdHO3fuNMHxtddea+qyYsWK5v2j7wt3oO9xff00MPrll1/MD5bC0Lp3NuDR+ncmqMrP6dOnzWeL40W783NavHixCZ71vajvDQ1WbfR9qD8A9DW0dZ9q2TQg0sBIXyd9b7733nvmvZWenp7r2Pqe1VZybRWbPHmy+SEBN2AF3NjHH3+sn1J5LsHBwdYZM2bk2f/cuXO5rqelpVmbNGliveaaa3Jt12MEBQVZd+7cad+2ceNGs/2tt96ybxs3bpzZdvToUeuWLVuskZGR1rZt21pPnDiR63hdu3Y1+3366af2bampqdaIiAjrHXfcYd82efJks9/nn3+eq4wdO3a0lilTxpqcnGy26ePpfvr4jrp162Zt2rSpNSUlxb4tKyvL2qlTJ2tsbGwhatVq7t+3b1/79dGjR1vDwsKs6enpufarVauWtX///nnur89XLzZr1qwx5dXXK799C1M3n332mdXPz8+6bNmyXPefNm2auf/vv//u9Os3ceJEsy0+Pj5PuRyf29ixY82+//nPf/Lsq/WrMjIyTNlzOnnypDU8PNw6cOBA+7YLvX6295TNhg0bzPUHH3ww135PP/202b548eJcZdZtS5cutW87cuSI+X8YMWKE1RWWLFliyjRnzhzr6dOnzeut76X169cX6v45/8d+++03s/7666/nes69evXKdR/d5/HHHzfrV199tXkv2f73bZ8Z+p4sTLkLuhw+fDjX4+l786+//sp1DNtjdenSxbw3cr4m+v7s0aOHNTMz07797bffNvt/9NFHef4/9H0O90ILEjzC1KlTZeHCheaiuTP6C0t/df/nP//JtV/O3AVttdCWJ/1lum7dujzH7N69u2k5sWnWrJmUK1dOdu/enWffzZs3m1/1+qtRfxXrr3ZHZcqUydWiFRQUZFpnch5v/vz5Jina1rKhAgMDTaKotmBoU/2FnDhxwvySvfvuu3P98tVfoNqKsWPHDjl48OAFj/Hnn3/Kpk2bcpVB1/U4P/30k5SEwtSNJo1ra5S2LuX8Ja+tCipnN5azr19hfPPNN6aFS1t9HNm6xPz9/U3ZbS1e+npo95h2ceb3HisMfU8oxzwaW7KyY+tno0aNzHvaRlus6tevX+TnXVz0f01bR7TFTVtXi5Ksr62O+r/tbCtSQkKCyUUqCm31sX225Lxo93VO+v+vdZ+fhx56yLw3bPQzQrtjhw0blmsggO6n71HH11S74LR7Gu6FTEF4BP0yzZmkrV/o2pWmeQHaTWT70po3b55JNtbujpz5G/nNO1OzZs082zTw0cDKkXYZaHeaBhD6ZZ8fzT1wfBw9ngYkOfOXYmNj84ye0sDAdvuFaPeK/qAdM2aMueRHc5w0x6cgGmBq95rmrujxlOayaPCn3Wy9evWS4laYutHgTrsNC+oC0+dV1NevMLT7VLtBLkbzxbTLRAOBnF0lmudUFPqa6/tB865y0kBac+0c3xNFfd7aFZiZmVmkMuprkjMAyI8GA9rNuX79epN3k5MGCxpMFuaYGvBoMKIBj3ZpORNUaZeos5o2bWqC7Yu50OvreJvtNdPANSf9nNL/O8fXVP9fbZ9hcB8ESPBI+oWiH4qaC6BfrPqBrEnHmsOiH5iaHFmtWjXTOvPxxx+bUVqOCvrAd8w9UPrFqV+MGkA8/PDD+d7PmeMVlbZaqKefftq0GOXH8YvWsSyaf6QjbfL7NaxBiLZk2YLAgiY01C/ai31hOls3+tz0y+r111/Pd19N2Hb2mMVNg0vN99HcKE0A16RcLYfmC2mAdSkKO3lkUZ+3JtJfLAAviCbx58y5yc8tt9xi5g569dVXTdJ4zh8Bmo/kmFdT0DH1/1dzvJwJeDSxX++jOT6O0zkUlwuNrCvMqLuiHhuuQ4AEj6VdG0q/0G1dJNoSoq08OYdwa4B0qTSZWhO+NRlTE7TvvffeIh1Hh9Rrq4kGAzm/QLQ1wnb7hb4s9den0sCvML96HWkXniYca1K1rdXKRlsgNMlaR4bZusO0ZSK/SRb1i9ZWlguV1xnaXbZx40bp1q1bsc007cxx9PG1K/VCdESkPm/t2s15bP2CLurj6muu7wcN9HO+Jpp8rHVve09cKg3uizraS1uzLkaDRu1i0wBS/0dyJk5r16V2WxX2mNqKZAt4CkNbnGyJ89pl5mq210zn2sr5f6ItaRoYFuV/F5cfARI8knZt/Pzzz6ZZ2valor+s9YspZzeCDrnWL/xLpcedPn26yfvp37+/aWHR1ipn6bxDWu4vv/zSngOkgZ6OnNJj6ge9ss2B4hicaIuF7YtjyJAhppXMsRuloC6qnN1r2vqR3xwuGgjqF6ktQNKgQVvm9IM9Zzfm/v37c33w2+Z9uZQZqzWvSvNx3n//fROo5aRf7BpEODu/jDPl0lZCDRx1JKBjHpK2zuh7wNZ6Y7uuVq1aJStXrszV9VXQ61fQe0KHdevopZwBga0lrbi6PHVYfEnTKRF0pJ++NzXXRgMWW6DtTFCQM+ApbIugLajS/1NX0+eq/y9Tpkwxozdt75UPP/zQ5GqVRDc2ih8BEjyCDge3tbJoN5B2mekvbh1Sqx/ESj909EtFP5C0hUf30+Ru7XLKmetSVNriowGG/lK2fZnbEogLS7/49UtQf2XrvCfaxaCtEr///rv5gtRf3rYmd+0C00CqXr16JmFU50rRiz6nLl26mO4oTfrUQEVbG/RLWluHtBUmP5qTpa1sOjw8v+BIadCn3ZZadxqMaSK8lk/rVJ+zdiNpHeRMjlZ6Xbs2NG9En4MGJu3bt3cqL+f++++Xr776ynSraEK2fqFrsKuvu27XlsGceWiFofMVqeeee84MmdeWN80nyy/Q0qBRn6tOCzFw4EBzX82b0fml9HlpK4jmu2nrkQZQ+n7T1gC9TV8rW0vmxV4/R3pcDbr1i10DKg0OVq9ebbp09b3maUO+NS9QgyStc53Coqhz+mirnDPPXetNLxcb6OBIfwBLine truncated
|
||||||
|
"text/plain": [
|
||||||
|
"<Figure size 640x480 with 1 Axes>"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAHHCAYAAABHp6kXAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQABAABJREFUeJzsnQV4VNcWhdfEAxEI7u6uxbVQnKLFHQrFWtyKS3EpDi3u7lDcpVDc3S1AEuJ237fOZCYjdyaTEPSd/3vzaEbunCtzzz5b1tYoiqJAIpFIJBKJ5CvH7nMPQCKRSCQSiSQ+kEaNRCKRSCSSbwJp1EgkEolEIvkmkEaNRCKRSCSSbwJp1EgkEolEIvkmkEaNRCKRSCSSbwJp1EgkEolEIvkmkEaNRCKRSCSSbwJp1EgkEolEIvkmkEaN5Iti+PDh0Gg08Pb2/txD+WqpUKEC8ubNiy+NjBkzok2bNp/tmpJ8OyxevFic0wcPHvzfXMcS25BGjUT1ZmH4SJ48OSpWrIhdu3Z9s0crMDBQTH6HDh36aN/h4+MDFxcXcUyvX7/+Qdt69uyZGO+FCxfwJXHixAkxLu7rt3b+vhQ4kRv+Pu3t7ZE+fXrUq1dP9XoIDg7G1KlT8d1338HT01Ncg9mzZ0e3bt1w69Yt1e/o16+f2PZPP/0U6wnf9N5RtmxZbNq0CV8Tn+s6lsQD7P0kkehYtGgRe4EpI0eOVJYtW6YsXbpUmThxopInTx7x/LZt2z7qwRo2bJj4ntevX3/Sk8Lv4/fy+z8W8+fPV1xcXJSUKVMqgwcP/qBt/fvvv2K8PF+mlC9fXpyvzwGvFY7r/v37Zq8FBwcroaGhn/z8hYWFKUFBQcq3Ao8t97Vp06biN7p48WKlf//+ioeHh+Ls7KycP3/e6LgUKVJEvL9WrVrKtGnTlIULFyp9+/ZV0qVLpzg6OpptPzIyUkmbNq2SMWNGxdXVVfHz87N5bBkyZFAKFiwoxsXH+PHjlcyZM4vvnzNnTrwdg/DwcHFOOdZv6TqWfDgO8WEYSb49qlevjqJFi+r/bt++PVKkSIFVq1ahVq1an3VsXyvLly9HjRo1kCFDBqxcuRKjR4/G/xPOzs6f5XsdHBzE41ujcOHCaNGihf7v0qVLo06dOpgzZw7mzZsnnmOY5Pz581i/fj0aNGhg9PlRo0Zh8ODBZtult+vJkyc4cOAAfvjhB2zcuBGtW7e2eVxp0qQxGlerVq2QNWtW4S3q3Lmz6mfCw8MRGRkJJycnm76D3ik+/p+uY4mNxINhJPkGPTX0BBjCFRFXgq1atTJb0ZQsWVLx8vISXojChQsr69atM9sut9m1a1dl06ZNwovg5OSk5M6dW9m1a1eMnpoHDx4oWbJkEZ978eKFkTfi6tWrSoUKFcSKMnXq1GJlaMrLly+Vdu3aKcmTJxcr2fz584vVrenK1/RhuOq/fv260qBBAyVx4sRiG1z9btmyxebj+vDhQ0Wj0Shr165VTp8+LbZ//Phx1ZVu69atzZ7n/vJBDh48qDpendcmNseGq86hQ4eK48tzwhU6V/F8PrbnT3fuTB+61a7avr1790759ddfxWvcZpo0aZSWLVvqz39ISIjy+++/i+uK11+CBAmUMmXKKAcOHLD5/OnGZeq9oTeSXgR+L79/4MCBZvvN52vWrKkcPXpUKVasmDj3mTJlUpYsWaJ8LnT7y9+eIf7+/uL5KlWqiL9PnTol/u7YsWOstt++fXtxbkn16tX127MF3fEypWjRonqvkOH4p06dKs6BnZ2d3sO0f/9+cY55rj09PZU6deoo165dU71PmXpSdu7cqf+sm5ubUqNGDeXKlStm4+HvuVGjRkrSpEnFfSt79uzKoEGD4nwd3717V2nYsKG4P/D39t133ynbt283eo/ud7tmzRpl9OjR4lrn9VSpUiXl9u3bNh9jiXW+veWLJF7w9fUVybqcz169eoU///wT/v7+RiswMn36dLE6bN68OUJDQ7F69Wo0atQI27dvR82aNY3ee+zYMbHq++WXX+Du7o4ZM2aI1eOjR4+QJEkS1XHcvXsXlSpVgpeXF/bu3YukSZPqX3v37h2qVauG+vXro3HjxmI12r9/f+TLl094mkhQUJBInL1z547IIciUKRPWrVsnVrCMl/fs2RPJkiUTq9suXbqIvARuj+TPn1/8e/XqVbEK5gp0wIABSJgwIdauXYsff/wRGzZsEJ+JCXq4+Dl6uVxdXZElSxasWLECpUqVivW5yZUrF0aOHImhQ4eiU6dOImeBGG7LlmPDlTHPHc8Lt8PtXr58WayomWuxefPmWJ0/fhc/x33lNnTnisdXDV5PHDvzi9q1ayc8D7zmtm7dKjwF/Lyfnx8WLlyIpk2bomPHjnj//j3++usv4UE4c+YMChYsGOP5U6NDhw5YsmQJGjZsiN69e+P06dMYN26cGItp/gevHb6P3kp6LP7++29x/RQpUgR58uTBlwJ/K0T3W+JxJC1btrR5GyEhIeKa5jEhPO5t27bFixcvkDJlyjiNKywsDI8fPzb7jS9atEjk+/Dao/eDv/F9+/aJ6zNz5swip4W/X957+Pv777//RM6OJZYtWybOD6+N8ePHizwrXhdlypQR3irdZy9duiSuO0dHR/HdfJ7Hbtu2bRgzZkysr+OXL1+K3x6/r0ePHmI/eW3xt8Xfnen94Y8//oCdnR369Okj7rMTJkwQ909eg5J4IAajR/J/hm4FZPrgisLQu6EjMDDQ6G/GmvPmzStWH4ZwG1wR37lzR//cxYsXxfN//vmnqqeGqyl6GLhCfvv2rdH26I3g+5jzo4Oreuar0KOigzkEfN/y5cuNxkjvEldyunwBazkZlStXVvLly2e0iqfnqlSpUkq2bNlsOKqK+Hzz5s31f3NVyFUiPQax9dTYklNjy7FhzgNXyPRCGDJ37lwzT5Kt589aLoLpvtFDxPdu3LjR7L26XAnmTnDspt6dFClSCO+bDmvnz9RTc+HCBfF3hw4djN7Xp08f8byhF4hj5nNHjhzRP/fq1Svxe+jdu7fyOdB5OkaMGCH2m97LQ4cOKYUKFRLPb9iwQbyvXr164m8eL1tZv369+IzOc8DfBz0Z9KjYAo9X1apVxbj44DXSpEkTsc3u3bsbjZ+eNx5LQ5iPQ4/qmzdv9M9xG7xODb3Epp6a9+/fK4kSJTLzSvHY0Ntj+Hy5cuUUd3d34T01xDA/JzbXMT2NfK/h74jjoUePeUkRERFGnppcuXIZXdPTp08Xz1++fNmmYyyxjqx+kqgya9Ys4Rnhg7kgrH7i6pYrdUPodTD0DnDlwVUQV1WmfP/998JDoYMraQ8PD9y7d8/svVeuXEH58uXFKoqrt8SJE5u9x83NzchzxHh88eLFjba3c+dOscLkilMHV2hcUdFTcPjwYatXwNu3b0VuAb0d9BLQk8DHmzdvxIrw9u3bePr0qdVtcGVID4jhGPjf3M6ePXvwMbDl2NBjRe9Mzpw59fvFBz1j5ODBg3E+f7ZAj0CBAgVUPV26EmzmTejyLOhZ4vlg/gXzvdSuMVvgNUF69epl9LzOO7Fjxw6j53Pnzq33hulW7Dly5IjzfscXw4YNE2Ph9U1vJL0N9FDoPFX0chF61WyF3kMeW+bA6D5Ljyuft5V//vlHjIsPnl9eZ/QWcWyG0Mtn6P14/vy5qN6iF4xeG8PrrEqVKvrzpgbvU/S86n5XugevH1Z96a7l169f48iRI8IzyIoxQ+Ja9s9x8bdFj5Dh749eIFaqXbt2zej99HwZ5g7prq3PfT19K8jwk0QV/kgNE4V5syhUqJAI4TCEovtRMszEhFfejOi6tnaDML2JEBorNIZMqV27tkhM5qTPG4QaadOmNfsebo9GhI6HDx8iW7Zswt1rCCdz3evWYOiBjorff/9dPNRgeI6hKUvQKGToiS51bo+wrJYGGycL0zBdfGDLsaFBxnCLJbc69yuu588WOAmbJq+qQVf+5MmTcePGDRHK0MFQYlzgOef1oJu4ddA4SJQokdk1Edf95gQaERERpzHynMSUCMtJk6Fe7gvHzVCYYRIrDU5CY5yvxwSNAk7Q/I3rrlPC0A8NUIZkWAoeEzQieE/g9ZcgQQLxW1P7ftPzpzvuNBhN4TZ4LwgICBC/JVN4LROdQW6K7ljoDIf41HHiuLnPamPWvW74fabXk27BFtffkcQYadRIbII3TnprmEPDGwhvoEePHhVx43LlymH27NlIlSqV8IIwVs7qHlMs3aS10Q1jONlxMuOk//PPP6t+Ljbbiyv0DhDGv+mZUcN0cjQdC2PzvBlzxa9mONBjpDPcLK0WOTnGptrDlmPDfWOOzZQpU1Tfmy5dulhvM76hQciVO/OX+vbtK3RPOA7mv+hySOKKrSvzuO53sWLFYjSaLXH//n2r+SOExjq9Z5agB47QS2joabIEPSpcmNCA5MMU/hZHjBgR43aYg2JtXGpe3vj6nTKvRi3350uqfvscv6P/J76cMy354qHbn3ASJly90ePAFZThCpFGzYcyceJEcSPSJaU2a9YsTtth+TS9E7zpGXpruOrXvW5tgqN3hdBYs+VGbQrDW0x6ZWKvbuWmgyszrraZkKsLFXHVpib4xclRNxZr440NDCVdvHgRlStXjjfF3dhsh9/PMKM1mGjJ/WbY03DbDL3E9Xt5znk90Dg3PCdM+OSx110THwqNACa6xoW4JuWaejtp/NEwtMWo4XjpUTA9toQl4lyo2GLUxBXdcb9586bZa/y90lhS89IQXViURq+136nuNxTTdRfb68nSmHWvSz4d0qiR2ATd/oyVM+ykmwi44uCP39DFzhiyadVMXOB258+fL1znrGigJ4NeodhCXRiOe82aNfqcFhpnrKjgNpm3Q+gmJ6YGBW+SzFfgTb179+7CG2UaYrAUvjEMPdHLQANQzXjjZKIzanhzpgeMlWSGIT5WjxgaNbqb+4conjJPiOGGBQsLine truncated
|
||||||
|
"text/plain": [
|
||||||
|
"<Figure size 640x480 with 1 Axes>"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"k-NN error after PCA (k=5): 0.1238\n",
|
||||||
|
"Best k via CV: 15\n",
|
||||||
|
"CV accuracy: 0.9993\n",
|
||||||
|
"Leave-One-Out error: 0.0015\n",
|
||||||
|
"SVM accuracy: 1.0\n",
|
||||||
|
"\n",
|
||||||
|
"===== Seeds =====\n",
|
||||||
|
"[0.17460317460317465, 0.17460317460317465, 0.12698412698412698, 0.12698412698412698, 0.15873015873015872, 0.15873015873015872, 0.15873015873015872, 0.1428571428571429] [0.09523809523809523, 0.12698412698412698, 0.12698412698412698, 0.12698412698412698, 0.11111111111111116, 0.1428571428571429, 0.1428571428571429, 0.1428571428571429]\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAHHCAYAAABXx+fLAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAaHxJREFUeJzt3Qd4U1UbB/B/96ZAS1tWKWXvVfaoIAKKA0UFZAmKioAi+rmF73OBiogKCqIMBQQXigtUZFMoFJC9C4VCB6uFlu58z3suKS202Jmb5P5/z3P1JrlJTm5C8+ac97zHwWQymUBERERkII56N4CIiIjI0hgAERERkeEwACIiIiLDYQBEREREhsMAiIiIiAyHARAREREZDgMgIiIiMhwGQERERGQ4DICIiIjIcBgAEVGp/fe//4WDg4PVnsnjx4+r9k2dOlXvphCRlWAARGSDdu/ejfvvvx+1atWCu7s7qlevjttuuw0ff/yx3k2zS/Pnz1cBlJzr2NjYG26/5ZZb0LRp03zXhYSEqPuMGzfuhuPXrFmjbvvuu++KFLgVtk2ZMqUMXh2RMTnr3QAiKp5Nmzahe/fuCA4OxqhRoxAUFISTJ09i8+bN+PDDDwv8wqWykZ6eroKO4gSac+bMwUsvvYRq1aqV+HkHDRqEO+6444brW7VqVeLHJDI6BkBENuatt96Cr68vtm7diooVK+a7LSEhQbd2GUHLli2LFdA0adIEBw8eVEHTRx99VOLnbd26NYYMGVKs+8g612lpafDw8LjhNrne1dUVjo4lHwRISUmBl5dXie9PpDcOgRHZmKNHj6ov1uuDHxEQEHDDdQsXLkSbNm3UF2HlypUxcOBA1WN0vS1btqBPnz4quPL09ER4eDg2btx4w3EbNmxA27Zt1XBQnTp1MHv27ALb+eeff6JLly6qnd7e3mjQoAFefvllWAsJEB577DEVCPzwww9Fuo+0Pzs7u8hDTzIMNmzYMBU0nT59upQt/vfnuvPOO7Fy5UqEhYWp91veG/Nw25IlS/Dqq6+q4VJ5f5OTk9X9vv3229zPh7+/vwq0rh/me/jhh9V7KJ896Yny8fHB4MGDy/X1EJU3BkBENkbyfqKiorBnz54i9RbJF3C9evUwbdo0jB8/HqtWrUK3bt1w8eLF3OP+/vtvdZ18KU6aNAlvv/22ur1Hjx6IjIzMl3vUq1cv1dMkic8jRoxQxy9btizf8+7du1d9GcuQ0euvv473338fd999d4EBlR4kiJEv9S+//FK1/b777ivS/WrXrl3sgOaVV15BVlZWqfJ1UlNTcfbs2Rs2edy8pLdJhsskH0yGQ6XHyuyNN97Ar7/+iueee069vxL4SW7Tgw8+CCcnJ0yePFkNqUowKIFr3s+HkOfq3bu3CrIlmbx///4lfj1EVsFERDbljz/+MDk5OamtY8eOpueff960cuVKU0ZGRr7jjh8/ro5566238l2/e/duk7Ozc+71OTk5pnr16pl69+6t9s1SU1NNtWvXNt1222251/Xr18/k7u5uOnHiRO51+/btU8+T98/JBx98oC4nJiaarEF0dLRqz3vvvWfKzMw0DRgwwOTh4aHOW1HMmzdP3X/r1q2mo0ePqvP31FNP5d4eHh5uatKkSb771KpVy9S3b1+1P2LECHXeTp8+rS6vXr1aPd63335bpHYXtkVEROR7PrluxYoV+R7D/FyhoaHqPTWTz0tAQICpadOmpitXruRe/8svv6jjJ06cmHvd8OHD1XUvvvhikc4XkS1gDxCRjZFf9xEREapH5Z9//sG7776rfpnL0Mby5ctzj5Nf8jk5OeoXft5eA0malh6h1atXq+N27tyJw4cP46GHHsK5c+dyj5Mcj1tvvRXr1q1TjyO9JjK80q9fP5WAbdaoUSP1/HmZh+d++ukndV9rkZGRgQceeAC//PILfvvtN9WbVVyhoaEYOnQoPvvsM5w5c6ZI95Ghp9L0AslQnQwpXr81btz4hh6q698Ls+HDh+fLB9q2bZvqyXvyySfVcKZZ37590bBhQ9VbdL3Ro0eXqP1E1ohJ0EQ2SHJwJMCRL3QJgmQY54MPPlBT4yWgkS9GCWokz0WCnYK4uLio/8tx5i/IwiQlJanhrCtXrhT4eJLfIwGF2YABA/D555/j0UcfxYsvvqgCKRlmkvbdLPH2/Pnz6jWVhOQ3ybDOzcgwz+XLl/H777+rqet5SYCXmJhYpMeUgOarr75SAY0MNRUnaJLzUVxyznv27Pmvx0kAVNTbTpw4kfveXU8CIMn1ysvZ2Rk1atQoRquJrBsDICIbJl/OEgzJVr9+fZWTI0mtkpcjPS+S/Cpf9pLjcT1JahXmHpr33nsvX87I9cdKAFRU0tMgPUfSyyQ9CStWrMDSpUtVTtEff/xRYHuEBElr165FSchzXR/UXE96R6Qt0msmx+bt+ZDE8OuDhMIeUwIaSRYuTkAjuUASNL3zzjuqF608FDTjqyi3FYWbm1upZo0RWRsGQER2Qmb+CPOwjMzQkh4g+VKX4KgwcpyoUKHCTXsZqlSpor5EzT1G1yffXk++LKXnRzZJwJbEWwkCJKgo7HkkWfrChQsoiRYtWvzrMR06dMATTzyhErRlKEx6zqRnQ8jQoAwrFfUxpRdIZthJQFMUcp4laJKZWe3bt4c1JNOb3zsJTPOS68y3E9krBkBENsbcK3H90hPmISjzkIb0pki9mv/973/qizrv8RIYyXCTn5+fmgItX84ys0fygMw9Q2YyLCTBj/TaSA/Kjz/+iJiYmNw8oP3796vcoLzksWX4KC9z79LNepKkLeVNgi+ZEi4BkAxLLVq0SAVr0htUlGGmggIaCRbMgdTNmIfOpAfKGgJmmdE1a9YsjBw5UvXwCOkxlPd04sSJejeRqFwxACKyMVLpWaZF33vvvSpXQ3JmpDq0DDFJLRgZBjN/Qb/55psqCJIlFWTYReq3REdHq54PSayVKdHy5S/5OrfffruqLyT3l4RqqQUjwZb0DP3888/qMSWYkiGkrl27quRZSeyVqshyv127duW2Uaa+yxCYJNRKcCDJtp988onKIZEp1nqTczFv3jw1pV1eX2G1jIo6rCU9JnIOiho0LViwoFjPs337dhXEFvR4HTt2RElIDpj0Xsn7LTWfZPp8fHy8ymmSz9EzzzxTosclshl6T0MjouL5/fffTSNHjjQ1bNjQ5O3tbXJ1dTXVrVvXNG7cOFN8fPwNx3///femLl26mLy8vNQm9xszZozp4MGD+Y7bsWOH6b777jP5+fmZ3Nzc1LTqBx980LRq1ap8x61du9bUpk0b9bwytXrWrFmmSZMm5ZsGL/e55557TNWqVVPHyf8HDRpkOnTokO7T4PP65JNP1PXPPfdckafBX888Rfxm0+DzOnz4cG7ZgNJOg5fn/rfn+7cp90uXLjW1atVKveeVK1c2DR482HTq1KkbXqN8dojsiYP8R+8gjIiIiMiSmNJPREREhsMAiIiIiAyHARAREREZDgMgIiIiMhwGQERERGQ4DICIiIjIcFgIsQCyNtLp06dV0bjrq+0SERGRdZLKPpcuXUK1atX+de06BkAFkOCnZs2a5fX+EBERUTmSxY2l8vzNMAAqgPT8mE+glMknIiIi65ecnKw6MMzf4zfDAKgA5mEvCX4YABEREdmWoqSvMAmaiIiIDIcBEBERERkOAyAiIiIyHOYAERGRTcrOzkZmZqbezSALcnFxgZOTU5k8FgMgIiKyuVovcXFxuHjxot5NIR1UrFgRQUFBpa7TxwCIiIhsijn4CQgIgKenJwvWGijwTU1NRUJCgrpctWrVUj0eAyAiIrKpYS9z8OPn56d3c8jCPDw81P8lCJLPQGmGw5gETURENsOc8yM9P2RMnlff+9LmfzEAIiIim8N1Go3LoYzW6GQARERERIbDAIiIiIgMhwEQERGRBTz88MNq+EY2qWdTu3ZtPP/880hLS+P51wFngVlQakYWzqdkwOj8vNzg4Vo2hayIiGxJnz59MG/ePJXAGxUVheHDh6uA6J133tG7aYbDAMiC/tqfgKe+3gGjq+zlilUTwlHJy1XvphARWZSbm5sq4idq1qyJnj174s8//1QB0Llz5zB27FisW7cOFy5cQJ06dfDyyy9j0KBB6vhffvkFQ4YMUcfJ9O+dO3eiVatWeOGFFzBlyhR1zKOPPqp6lBYuXMh39l8wALIgJwcHuDkbe9QxMztH9YKt3BuHge2C9W4OEdlJgbwrmdm6PLeHi1OJZyXt2bMHmzZtQq1atdRlCVzatGmjApoKFSrg119/xdChQ1Ug1K5dO3Tt2hWXLl3Cjh07EBYWhrVr18Lf3x9r1qzJfUy5Tu5P/44BkAX1bV5VbUY24+/DmPrHIaxgAEREZUSCn8YTV+pyPve93huerkX/KpVeHG9vb2RlZSE9PR2Ojo6YMWOGuq169ep47rnnco8dN24cVq5ciW+++UYFQL6+vmjZsqUKeCQAkv8/88wz+N///ofLly8jKSkJR44cQXh4eLm8Vntj7O4Isrg+TbWu301HziE5jYsYEpGxdO/eXQ1dbdmyReX/jBgxAv3798+tcv3GG2+gWbNmqFy5sgqUJACKiYnJvb8ENxL4SK/X+vXrcd9996FRo0bYsGGD6v2pVq0a6tWrp+MrtB3sASKLqhvggzpVvHA0MQWrDyTgnpbV+Q4QUamHoaQnRq/nLg4vLy/UrVtX7c+dOxctWrTAF198gUceeQTvvfcePvzwQ0yfPl0FQXLs+PHjkZFxbfLMLbfcou73zz//qJlkDRs2VNdJUCR5Q+z9KTr2AJFuvUCSB0REVFqSgyPDUHpspalKLMNfkuT86quv4sqVK9i4cSPuuecelegsgVFoaCgOHTqU7z7mPKAPPvggN9gxB0CyyT4VDQMgsrjeTbQAaPWBRKTplLhIRGQNHnjgATWja+bMmWroSmaESWL0/v378fjjjyM+Pj7f8ZULine truncated
|
||||||
|
"text/plain": [
|
||||||
|
"<Figure size 640x480 with 1 Axes>"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAHHCAYAAABHp6kXAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAppJJREFUeJzsnQWcFOUbx3+ze91Jd3d3dykpLaWIioAiBqJ/A5VQRFRUUMQW6Va6W7q7++647rvd9/953r09LnaPO9je5+tnPHZmd/bdmdl9f/OkIoQQYBiGYRiGsXNU1h4AwzAMwzCMKWBRwzAMwzCMQ8CihmEYhmEYh4BFDcMwDMMwDgGLGoZhGIZhHAIWNQzDMAzDOAQsahiGYRiGcQhY1DAMwzAM4xCwqGEYhmEYxiFgUcMwDspHH30ERVGsPQwmD7Zv3y7PEf21NG3atJELwzgSLGoYxkScPHkSffv2RenSpeHh4YHixYujY8eOmD17Nh9jI9CErl9UKhWKFSuGTp06GZzkNRoNfvnlFzkRBwUFwd3dHWXKlMFzzz2HQ4cOGdz/999/L/fduHHjAp0Deo+sY6P3a9iwIX7++WdotVq7OZ9nzpyR4vbatWvWHgrDWASFez8xzJOzd+9etG3bFqVKlcLw4cNRpEgR3Lx5E/v378fly5dx6dIlix9mmswmT54MW27vRoKBhN+wYcPkOK9evSqFSFhYGP755x907dpVPi8pKQl9+vTB+vXr0apVK3Tv3l0KDZqsFy9ejAsXLuDGjRsoUaJEtv03b94cd+7ckc+7ePEiKlSokG9RQ+dt2rRp8nF4eDh+//13HDt2DBMnTsT06dNN8vlJIKWmpsLNzU2KOlOzdOlS9OvXD9u2bctllaH3Jei9GcZhIFHDMMyT0a1bNxEaGiqioqJybbt//75VDu+HH35IakbYMjS+MWPGZFt34sQJub5Tp06Z6+g5tG7WrFm59pGeni5mzJghbt68mW39lStX5GuWL18uz81HH32U73G1bt1aVK9ePdu6hIQEUaJECeHt7S1SU1MNvk6j0YikpCRhKyxZskQeg23btll7KAxjEdj9xDAmgO7qq1evjoCAgFzbChUqlGvdn3/+ifr168PT01NaHAYOHCgtOzk5cOAAunTpAn9/f3h5eaF169bYs2dPruft3r1bukfI7VW+fHn88MMPBse5adMmtGjRQo7Tx8cHlStXxrvvvgtbombNmggJCZFWG+LWrVvy85BFZ/z48bmer1ar8eabb+ay0vz1118IDAzEU089Jd2C9PhJoOPfpEkTJCQkSMuN3tI0duxYuW86/+QSI2sScfToUWlp8vPzk8e6ffv20nKXn5ia/J7327dvY+TIkdJtR+9dtmxZjB49Wlphfv31V2mlIciKqHel6d/LUEwNWchof4ULF5bXUu3atfHbb79lew5ZvWg/X3zxBX788Ud5vdF70/V38ODBJzrGDPOkuDzxHhiGkXE0+/btw6lTp1CjRo08j8iUKVPw/vvvo3///njhhRfkBElxN+RWoYlQL4y2bt0qJ0USPx9++KF0T1BMSbt27bBr1y40atQoM5aH4lBCQ0Olyyk9PV0+nyamrJw+fRpPP/00atWqhY8//lhOROQWMzRZWpOoqCi56F1F69atk59p6NChBdoPCQ1yWZF7ZdCgQZgzZ46cdGnyfVyuXLkiRVRW8UrniVxgJG5IjFGcDx3rli1bSkHz9ttvw9XVVQozEhE7duzIM8Ynv+ed3Gr07+joaLz44ouoUqWKFDnkckpMTJTX06uvvopvvvlGCteqVavK1+n/5oRcfDQ+uibos5BAWrJkCUaMGCHf47XXXsv2/AULFiAuLg4vvfSSFDmff/65PN50jOjzMoxVsIxBiGEcm40bNwq1Wi2Xpk2birffflts2LAhl5vi2rVr8jlTpkzJtv7kyZPCxcUlc71WqxUVK1YUnTt3lv/Wk5iYKMqWLSs6duyYua5Xr17Cw8NDXL9+PXPdmTNn5Ptk/YqT64Yeh4eHC1uBxjNy5Eg5prCwMHHgwAHRvn17uX7mzJnyOa+//rp8fPTo0Xzv99ChQ/I1mzZtko/pGJLr6LXXXsu3+6lKlSpyXLScPXtWvPrqq3Kf3bt3zzZ+lUolTp8+ne31dE7c3NzE5cuXM9fduXNH+Pr6ilatWmWuI7dQVvdQQc77sGHD5HsfPHgw1/j1r83L/USfkRY9X331lXzun3/+mbmOrl+6nn18fERsbKxcd/XqVfm84OBgERkZmfncVatWyfVr1qzJ1zFmGHPAooZhTMR///0nevfuLby8vOSPOy0Uy0E/9nq+/PJLoSiKuHjxYuaEqV+qVq0qOnToIJ935MgR+frffvst1/NeeOEF4e7uLuM3KJ7E09NTDBw40GCcT1ZR88svv8jHP/30k3ytLaA/TlkXEmgTJkzIHCOJHlp/6dKlfO+XhFDhwoXl8dHzxhtv5FpnDJrsc46LzttTTz2VTRTS+rZt22Z7Le2froH+/fvn2u9LL70khUhMTIxBUZPf806Ln5+f6NmzZ56foyCihmKYihQpkuva+Pvvv7OJFb2oeeWVV7I9jwQOrf/666/zHBPDmBN2PzGMiSC3xvLly2U8w/Hjx7FixQrMmjVLxnNQ1ky1atVkBg7NhRUrVjS4D73Znp5HUCaVMWJiYpCSkiLdBob2R/Ey//77b+bjAQMG4KeffpIur3feeUfGeJC7gMaXV+ZNZGRkZqZMQaF4oUdl1/Ts2VO6O8iF4evrK2NTvL29M7eTC4cgV0d+oNTvhQsXyjgSfVwOQS6fmTNnYsuWLdJd9yjIjTRv3jw5LoovoWNsKD6K3DRZIXciuX/o+OeEXD+U8UTxU/Q5c5Lf807nIzY29pGuzoJw/fp1+RlzXgt6dxVtzwpl+mWF4pcIch0yjLVgUcMwJoYmcRI4tFSqVEnWUaHYBIqPoAmNJkmKE6HYjJxQQCmhr4UyY8YM1KlTx+D70HNJ1OQXCkreuXOnTO+ldGkKaF20aJGM1di4caPB8RAkfCgO5HEwlEqcEwrw7dChg9HtFCuijx0ydixyxqTcvXtXChtaDMXa5EfUkLDKa1xZj6upyO95J6FpbYxdL7ZcQoBxfFjUMIwZadCggfxLkyxBmSL0o0939yR4jEHP01sp8ppYKTiYJlX9HX5Wzp8/n2sd3YWThYaWL7/8ElOnTsV7770nxYex9yHrxuPefVP2zJNCQbM0gVLGWH6ChUm0kEXlu+++y7WNLGlkQZs7d65JxUjOc0IZS4aO/7lz5+Q5KFmy5BOfd3oOBabnRUEqSlOw+4kTJ6SwymqtoTHrtzOMzWNW5xbDOAlbt27NFtip57PPPpNxBhRLQ1BcCAXwDh48ONfz6XFERIT8N8U1lC9fXgaNxsXF5dovBdUWNFD4wYMHufbzzz//yOesXbtW2EqdGkO8/PLL8rnffPNNrm10rL744gtZp4YCaikY9/nnnze4nz179sj9LFy4sMB1agoyfjonFP9C8Sd67t27J+Ng8goULsh5z0+g8Lp16+T+V6xYke9A4QULFmSuS0tLE82bNzcYKEy1gQwdD6qPxDDWgi01DGMCxo0bJ+MoevfuLd0lFPNAVYbJvaMv5a+/E//0008xadIkWe+jV69eMo6EYj/IgkCpuVRzhe6UKf6FrBQUe0Gvp7YLlLJLVhW6S1+zZo3cJ1UNJlcSpRC/8sorMv2ZUsTpdXTnrYfSuMn9RHVb6K6bapJQ9V5y/1DtGluGrEVUC4hSlMnaQqnpFMNBVYTJtUfWBKr1s3r1ahl706NHD4P7oTozZOUgaw7FGJkLOsf6mkB0TlxcXGRKN7kLKfXZGAU572RlI7ch1bCh64ZiX8giSMeD6hZR2jm5sMjK9dlnn8lYHErjJ3ejodgg2geNkVK4Dx8+LK9bSg+nlP+vvvpKXqcMY/NYTU4xjANBd8RkHaA0YLqrpXTeChUqiHHjxhmsKLxs2TLRokULWZ2WFnod3fGfP38+2/MojblPnz4yfZbu/EuXLi2zarZs2ZLteTt27BD169eX71uuXDkxd+7cXBWF6TWULVOsWDH5PPo7aNAgceHCBWEt8mup0WcVUeZWy5Ythb+/v3B1dZXH47nnnstM96Z0a7JaUfVfY4wYMUK+Vm8VM4elRp/JRKnZdD1QNhRlSe3duzfbc3Jaagp63sk6RxYbyrKj59G5p/GkpKRkPmfevHlyvd5yp3+vnJYagq5VOp4hISHyGqlZs6bMmssKW2oYW4Z7PzEMw1gJysSi2Bkqqmfr1jKGsQe4TQLDMIyV0AeQUyVihmGeHLbUMAzDWBjqH0VxPV9//bWsN0M1YMzRpZthnA3+FjEMw1gYKtBHweWUVr5s2TIWNAxjIthSwzAMwzCMQ8CWGoZhGIZhHAIWNQzDMAzDOAROVXyPyn/fuXNHFpEqSPlwhmEYhmGsB5WFosKaxYoVyzMGzalEDQkaYz1XGIZhGIaxbajDPVVBN4ZTiRp9mW86KFRunGEYhmEY24dKH5BR4lHtOpxK1OhdTiRoWNQwDMMwjH3xqNARDhRmGIZhGMYhYFHDMAzDMIxDwKKGYRiGYRiHgEUNwzAMwzAOAYsahmEYhmEcAhY1DMMwDMM4BCxqGIZhGIZxCFjUMAzDMAzjELCoYRiGYRjGIWBRwzAMwzCMQ+BUbRIYhmEYx2fHtauYf/QQLkdFoWxAAJ6v2wDtypaz9rAYC8CihmEYhnEYFp46gXe3boJaUaARAvcT4rH31k181LodhtWua+3hMWaG3U8MwzCMQ5CUloYpu7bLf5OgIbQZf6fv2Yn41FSrjo8xPyxqGIZhGIfg2L27SEhLM7gtOT0dh+7ctviYGMvCooZhGIZxCFzUeU9pro/Yztg/HFPj5AgyzabuB9IvAOoigHtbKIqbtYfFMAxTYOoWKYZQLy9EJCZC53TSoQDw9/BAw2Il+Kg6OCxqnBihjYSLine truncated
|
||||||
|
"text/plain": [
|
||||||
|
"<Figure size 640x480 with 1 Axes>"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"k-NN error after PCA (k=5): 0.1587\n",
|
||||||
|
"Best k via CV: 11\n",
|
||||||
|
"CV accuracy: 0.9238\n",
|
||||||
|
"Leave-One-Out error: 0.0762\n",
|
||||||
|
"SVM accuracy: 0.873\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"from typing import Callable, Literal\n",
|
||||||
|
"import numpy as np\n",
|
||||||
|
"import pandas as pd\n",
|
||||||
|
"import matplotlib.pyplot as plt\n",
|
||||||
|
"\n",
|
||||||
|
"from sklearn.model_selection import train_test_split, cross_val_score, LeaveOneOut\n",
|
||||||
|
"from sklearn.neighbors import KNeighborsClassifier\n",
|
||||||
|
"from sklearn.preprocessing import StandardScaler\n",
|
||||||
|
"from sklearn.decomposition import PCA\n",
|
||||||
|
"from sklearn.metrics import accuracy_score\n",
|
||||||
|
"from sklearn.svm import SVC\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"# ---------- Utility functions ----------\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def load_banknote():\n",
|
||||||
|
" url = \"https://archive.ics.uci.edu/ml/machine-learning-databases/00267/data_banknote_authentication.txt\"\n",
|
||||||
|
" df = pd.read_csv(url, header=None)\n",
|
||||||
|
" X = df.iloc[:, :-1].values\n",
|
||||||
|
" y = df.iloc[:, -1].values\n",
|
||||||
|
" return X, y, \"Banknote Authentication\"\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def load_seeds():\n",
|
||||||
|
" url = \"https://archive.ics.uci.edu/ml/machine-learning-databases/00236/seeds_dataset.txt\"\n",
|
||||||
|
" cols = [\n",
|
||||||
|
" \"area\",\n",
|
||||||
|
" \"perimeter\",\n",
|
||||||
|
" \"compactness\",\n",
|
||||||
|
" \"length\",\n",
|
||||||
|
" \"width\",\n",
|
||||||
|
" \"asymmetry\",\n",
|
||||||
|
" \"groove\",\n",
|
||||||
|
" \"class\",\n",
|
||||||
|
" ]\n",
|
||||||
|
" df = pd.read_csv(url, sep=r\"\\s+\", names=cols)\n",
|
||||||
|
" X = df.iloc[:, :-1].values\n",
|
||||||
|
" y = df[\"class\"].values\n",
|
||||||
|
" return X, y, \"Seeds\"\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def knn_error(X, y, k):\n",
|
||||||
|
" Xtr, Xte, ytr, yte = train_test_split(\n",
|
||||||
|
" X, y, test_size=0.3, random_state=42, stratify=y\n",
|
||||||
|
" )\n",
|
||||||
|
" model = KNeighborsClassifier(n_neighbors=k)\n",
|
||||||
|
" model.fit(Xtr, ytr)\n",
|
||||||
|
" preds = model.predict(Xte)\n",
|
||||||
|
" return 1 - accuracy_score(yte, preds)\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def evaluate_knn_over_k(X, y, k_values):\n",
|
||||||
|
" return [knn_error(X, y, k) for k in k_values]\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def normalize(X):\n",
|
||||||
|
" return StandardScaler().fit_transform(X)\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def apply_pca(X, n_components=2):\n",
|
||||||
|
" return PCA(n_components=n_components).fit_transform(X)\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def knn_cross_val(X, y, k_values, cv=10):\n",
|
||||||
|
" scores = {}\n",
|
||||||
|
" for k in k_values:\n",
|
||||||
|
" knn = KNeighborsClassifier(n_neighbors=k)\n",
|
||||||
|
" scores[k] = cross_val_score(knn, X, y, cv=cv).mean()\n",
|
||||||
|
" return scores\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def leave_one_out_error(X, y, k):\n",
|
||||||
|
" loo = LeaveOneOut()\n",
|
||||||
|
" knn = KNeighborsClassifier(n_neighbors=k)\n",
|
||||||
|
" scores = cross_val_score(knn, X, y, cv=loo)\n",
|
||||||
|
" return 1 - scores.mean()\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def run_svm(X, y, kernel=\"rbf\"):\n",
|
||||||
|
" Xtr, Xte, ytr, yte = train_test_split(\n",
|
||||||
|
" X, y, test_size=0.3, random_state=42, stratify=y\n",
|
||||||
|
" )\n",
|
||||||
|
" svm = SVC(kernel=kernel, C=1, gamma=\"scale\")\n",
|
||||||
|
" svm.fit(Xtr, ytr)\n",
|
||||||
|
" return svm.score(Xte, yte)\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"# ---------- Plotting ----------\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def plot_error_vs_k(k_vals, err_raw, err_scaled, title):\n",
|
||||||
|
" plt.figure()\n",
|
||||||
|
" plt.plot(k_vals, err_raw, label=\"Raw\")\n",
|
||||||
|
" plt.plot(k_vals, err_scaled, label=\"Normalized\")\n",
|
||||||
|
" plt.xlabel(\"k\")\n",
|
||||||
|
" plt.ylabel(\"Classification Error\")\n",
|
||||||
|
" plt.title(title)\n",
|
||||||
|
" plt.legend()\n",
|
||||||
|
" plt.show()\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def plot_pca_2d(X_pca, y, title):\n",
|
||||||
|
" plt.figure()\n",
|
||||||
|
" plt.scatter(X_pca[:, 0], X_pca[:, 1], c=y, cmap=\"viridis\", s=20)\n",
|
||||||
|
" plt.title(title)\n",
|
||||||
|
" plt.xlabel(\"PC1\")\n",
|
||||||
|
" plt.ylabel(\"PC2\")\n",
|
||||||
|
" plt.show()\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"# ---------- Main pipeline ----------\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"def run_full_pipeline(loader_fn):\n",
|
||||||
|
" X, y, name = loader_fn()\n",
|
||||||
|
" print(f\"\\n===== {name} =====\")\n",
|
||||||
|
"\n",
|
||||||
|
" k_vals = list(range(1, 16, 2))\n",
|
||||||
|
"\n",
|
||||||
|
" # Raw k-NN\n",
|
||||||
|
" err_raw = evaluate_knn_over_k(X, y, k_vals)\n",
|
||||||
|
"\n",
|
||||||
|
" # Normalized k-NN\n",
|
||||||
|
" X_scaled = normalize(X)\n",
|
||||||
|
" err_scaled = evaluate_knn_over_k(X_scaled, y, k_vals)\n",
|
||||||
|
" print(err_raw, err_scaled)\n",
|
||||||
|
" plot_error_vs_k(k_vals, err_raw, err_scaled, f\"{name} — k-NN Error\")\n",
|
||||||
|
"\n",
|
||||||
|
" # PCA + k-NN\n",
|
||||||
|
" X_pca = apply_pca(X_scaled, 2)\n",
|
||||||
|
" plot_pca_2d(X_pca, y, f\"{name} — PCA Projection\")\n",
|
||||||
|
"\n",
|
||||||
|
" pca_err = knn_error(X_pca, y, k=5)\n",
|
||||||
|
" print(\"k-NN error after PCA (k=5):\", round(pca_err, 4))\n",
|
||||||
|
"\n",
|
||||||
|
" # Cross-validation\n",
|
||||||
|
" cv_scores = knn_cross_val(X_scaled, y, k_vals)\n",
|
||||||
|
" best_k = max(cv_scores, key=cv_scores.get)\n",
|
||||||
|
" print(\"Best k via CV:\", best_k)\n",
|
||||||
|
" print(\"CV accuracy:\", round(cv_scores[best_k], 4))\n",
|
||||||
|
"\n",
|
||||||
|
" # Leave-One-Out\n",
|
||||||
|
" loo_err = leave_one_out_error(X_scaled, y, best_k)\n",
|
||||||
|
" print(\"Leave-One-Out error:\", round(loo_err, 4))\n",
|
||||||
|
"\n",
|
||||||
|
" # SVM\n",
|
||||||
|
" svm_acc = run_svm(X_scaled, y)\n",
|
||||||
|
" print(\"SVM accuracy:\", round(svm_acc, 4))\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"# ---------- Run on both datasets ----------\n",
|
||||||
|
"\n",
|
||||||
|
"if __name__ == \"__main__\":\n",
|
||||||
|
" run_full_pipeline(load_banknote)\n",
|
||||||
|
" run_full_pipeline(load_seeds)\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 4,
|
||||||
|
"id": "bbd5e7a8",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"=== Dataset Sizes ===\n",
|
||||||
|
"Banknote: 1372 samples, 4 features\n",
|
||||||
|
"Seeds: 210 samples, 7 features\n",
|
||||||
|
"\n",
|
||||||
|
"=== PCA Explained Variance (2 components) ===\n",
|
||||||
|
"Banknote: 54.5% + 32.3% = 86.8% total\n",
|
||||||
|
"Seeds: 71.9% + 17.1% = 89.0% total\n",
|
||||||
|
"\n",
|
||||||
|
"=== Feature Ranges (min-max span) ===\n",
|
||||||
|
"Banknote: ['13.87', '26.72', '23.21', '11.00']\n",
|
||||||
|
"Seeds: ['10.59', '4.84', '0.11', '1.78', '1.40', '7.69', '2.03']\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"# Additional data for report\n",
|
||||||
|
"from sklearn.decomposition import PCA\n",
|
||||||
|
"\n",
|
||||||
|
"# Load datasets\n",
|
||||||
|
"X_bank, y_bank, _ = load_banknote()\n",
|
||||||
|
"X_seeds, y_seeds, _ = load_seeds()\n",
|
||||||
|
"\n",
|
||||||
|
"print(\"=== Dataset Sizes ===\")\n",
|
||||||
|
"print(f\"Banknote: {X_bank.shape[0]} samples, {X_bank.shape[1]} features\")\n",
|
||||||
|
"print(f\"Seeds: {X_seeds.shape[0]} samples, {X_seeds.shape[1]} features\")\n",
|
||||||
|
"\n",
|
||||||
|
"# PCA variance explained\n",
|
||||||
|
"print(\"\\n=== PCA Explained Variance (2 components) ===\")\n",
|
||||||
|
"for name, X in [(\"Banknote\", normalize(X_bank)), (\"Seeds\", normalize(X_seeds))]:\n",
|
||||||
|
" pca = PCA(n_components=2)\n",
|
||||||
|
" pca.fit(X)\n",
|
||||||
|
" total = sum(pca.explained_variance_ratio_) * 100\n",
|
||||||
|
" print(\n",
|
||||||
|
" f\"{name}: {pca.explained_variance_ratio_[0] * 100:.1f}% + {pca.explained_variance_ratio_[1] * 100:.1f}% = {total:.1f}% total\"\n",
|
||||||
|
" )\n",
|
||||||
|
"\n",
|
||||||
|
"# Feature scale ranges (to explain why normalization matters)\n",
|
||||||
|
"print(\"\\n=== Feature Ranges (min-max span) ===\")\n",
|
||||||
|
"print(\n",
|
||||||
|
" \"Banknote:\",\n",
|
||||||
|
" [f\"{X_bank[:, i].max() - X_bank[:, i].min():.2f}\" for i in range(X_bank.shape[1])],\n",
|
||||||
|
")\n",
|
||||||
|
"print(\n",
|
||||||
|
" \"Seeds:\",\n",
|
||||||
|
" [\n",
|
||||||
|
" f\"{X_seeds[:, i].max() - X_seeds[:, i].min():.2f}\"\n",
|
||||||
|
" for i in range(X_seeds.shape[1])\n",
|
||||||
|
" ],\n",
|
||||||
|
")"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"metadata": {
|
||||||
|
"kernelspec": {
|
||||||
|
"display_name": "ml2",
|
||||||
|
"language": "python",
|
||||||
|
"name": "python3"
|
||||||
|
},
|
||||||
|
"language_info": {
|
||||||
|
"codemirror_mode": {
|
||||||
|
"name": "ipython",
|
||||||
|
"version": 3
|
||||||
|
},
|
||||||
|
"file_extension": ".py",
|
||||||
|
"mimetype": "text/x-python",
|
||||||
|
"name": "python",
|
||||||
|
"nbconvert_exporter": "python",
|
||||||
|
"pygments_lexer": "ipython3",
|
||||||
|
"version": "3.14.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"nbformat": 4,
|
||||||
|
"nbformat_minor": 5
|
||||||
|
}
|
||||||
+256
@@ -0,0 +1,256 @@
|
|||||||
|
```python
|
||||||
|
from typing import Callable, Literal
|
||||||
|
import numpy as np
|
||||||
|
import pandas as pd
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
|
||||||
|
from sklearn.model_selection import train_test_split, cross_val_score, LeaveOneOut
|
||||||
|
from sklearn.neighbors import KNeighborsClassifier
|
||||||
|
from sklearn.preprocessing import StandardScaler
|
||||||
|
from sklearn.decomposition import PCA
|
||||||
|
from sklearn.metrics import accuracy_score
|
||||||
|
from sklearn.svm import SVC
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- Utility functions ----------
|
||||||
|
|
||||||
|
|
||||||
|
def load_banknote():
|
||||||
|
url = "https://archive.ics.uci.edu/ml/machine-learning-databases/00267/data_banknote_authentication.txt"
|
||||||
|
df = pd.read_csv(url, header=None)
|
||||||
|
X = df.iloc[:, :-1].values
|
||||||
|
y = df.iloc[:, -1].values
|
||||||
|
return X, y, "Banknote Authentication"
|
||||||
|
|
||||||
|
|
||||||
|
def load_seeds():
|
||||||
|
url = "https://archive.ics.uci.edu/ml/machine-learning-databases/00236/seeds_dataset.txt"
|
||||||
|
cols = [
|
||||||
|
"area",
|
||||||
|
"perimeter",
|
||||||
|
"compactness",
|
||||||
|
"length",
|
||||||
|
"width",
|
||||||
|
"asymmetry",
|
||||||
|
"groove",
|
||||||
|
"class",
|
||||||
|
]
|
||||||
|
df = pd.read_csv(url, sep=r"\s+", names=cols)
|
||||||
|
X = df.iloc[:, :-1].values
|
||||||
|
y = df["class"].values
|
||||||
|
return X, y, "Seeds"
|
||||||
|
|
||||||
|
|
||||||
|
def knn_error(X, y, k):
|
||||||
|
Xtr, Xte, ytr, yte = train_test_split(
|
||||||
|
X, y, test_size=0.3, random_state=42, stratify=y
|
||||||
|
)
|
||||||
|
model = KNeighborsClassifier(n_neighbors=k)
|
||||||
|
model.fit(Xtr, ytr)
|
||||||
|
preds = model.predict(Xte)
|
||||||
|
return 1 - accuracy_score(yte, preds)
|
||||||
|
|
||||||
|
|
||||||
|
def evaluate_knn_over_k(X, y, k_values):
|
||||||
|
return [knn_error(X, y, k) for k in k_values]
|
||||||
|
|
||||||
|
|
||||||
|
def normalize(X):
|
||||||
|
return StandardScaler().fit_transform(X)
|
||||||
|
|
||||||
|
|
||||||
|
def apply_pca(X, n_components=2):
|
||||||
|
return PCA(n_components=n_components).fit_transform(X)
|
||||||
|
|
||||||
|
|
||||||
|
def knn_cross_val(X, y, k_values, cv=10):
|
||||||
|
scores = {}
|
||||||
|
for k in k_values:
|
||||||
|
knn = KNeighborsClassifier(n_neighbors=k)
|
||||||
|
scores[k] = cross_val_score(knn, X, y, cv=cv).mean()
|
||||||
|
return scores
|
||||||
|
|
||||||
|
|
||||||
|
def leave_one_out_error(X, y, k):
|
||||||
|
loo = LeaveOneOut()
|
||||||
|
knn = KNeighborsClassifier(n_neighbors=k)
|
||||||
|
scores = cross_val_score(knn, X, y, cv=loo)
|
||||||
|
return 1 - scores.mean()
|
||||||
|
|
||||||
|
|
||||||
|
def run_svm(X, y, kernel="rbf"):
|
||||||
|
Xtr, Xte, ytr, yte = train_test_split(
|
||||||
|
X, y, test_size=0.3, random_state=42, stratify=y
|
||||||
|
)
|
||||||
|
svm = SVC(kernel=kernel, C=1, gamma="scale")
|
||||||
|
svm.fit(Xtr, ytr)
|
||||||
|
return svm.score(Xte, yte)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- Plotting ----------
|
||||||
|
|
||||||
|
|
||||||
|
def plot_error_vs_k(k_vals, err_raw, err_scaled, title):
|
||||||
|
plt.figure()
|
||||||
|
plt.plot(k_vals, err_raw, label="Raw")
|
||||||
|
plt.plot(k_vals, err_scaled, label="Normalized")
|
||||||
|
plt.xlabel("k")
|
||||||
|
plt.ylabel("Classification Error")
|
||||||
|
plt.title(title)
|
||||||
|
plt.legend()
|
||||||
|
plt.show()
|
||||||
|
|
||||||
|
|
||||||
|
def plot_pca_2d(X_pca, y, title):
|
||||||
|
plt.figure()
|
||||||
|
plt.scatter(X_pca[:, 0], X_pca[:, 1], c=y, cmap="viridis", s=20)
|
||||||
|
plt.title(title)
|
||||||
|
plt.xlabel("PC1")
|
||||||
|
plt.ylabel("PC2")
|
||||||
|
plt.show()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- Main pipeline ----------
|
||||||
|
|
||||||
|
|
||||||
|
def run_full_pipeline(loader_fn):
|
||||||
|
X, y, name = loader_fn()
|
||||||
|
print(f"\n===== {name} =====")
|
||||||
|
|
||||||
|
k_vals = list(range(1, 16, 2))
|
||||||
|
|
||||||
|
# Raw k-NN
|
||||||
|
err_raw = evaluate_knn_over_k(X, y, k_vals)
|
||||||
|
|
||||||
|
# Normalized k-NN
|
||||||
|
X_scaled = normalize(X)
|
||||||
|
err_scaled = evaluate_knn_over_k(X_scaled, y, k_vals)
|
||||||
|
print(err_raw, err_scaled)
|
||||||
|
plot_error_vs_k(k_vals, err_raw, err_scaled, f"{name} — k-NN Error")
|
||||||
|
|
||||||
|
# PCA + k-NN
|
||||||
|
X_pca = apply_pca(X_scaled, 2)
|
||||||
|
plot_pca_2d(X_pca, y, f"{name} — PCA Projection")
|
||||||
|
|
||||||
|
pca_err = knn_error(X_pca, y, k=5)
|
||||||
|
print("k-NN error after PCA (k=5):", round(pca_err, 4))
|
||||||
|
|
||||||
|
# Cross-validation
|
||||||
|
cv_scores = knn_cross_val(X_scaled, y, k_vals)
|
||||||
|
best_k = max(cv_scores, key=cv_scores.get)
|
||||||
|
print("Best k via CV:", best_k)
|
||||||
|
print("CV accuracy:", round(cv_scores[best_k], 4))
|
||||||
|
|
||||||
|
# Leave-One-Out
|
||||||
|
loo_err = leave_one_out_error(X_scaled, y, best_k)
|
||||||
|
print("Leave-One-Out error:", round(loo_err, 4))
|
||||||
|
|
||||||
|
# SVM
|
||||||
|
svm_acc = run_svm(X_scaled, y)
|
||||||
|
print("SVM accuracy:", round(svm_acc, 4))
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- Run on both datasets ----------
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
run_full_pipeline(load_banknote)
|
||||||
|
run_full_pipeline(load_seeds)
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
===== Banknote Authentication =====
|
||||||
|
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0024271844660194164, 0.009708737864077666] [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0024271844660194164, 0.009708737864077666]
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
k-NN error after PCA (k=5): 0.1238
|
||||||
|
Best k via CV: 15
|
||||||
|
CV accuracy: 0.9993
|
||||||
|
Leave-One-Out error: 0.0015
|
||||||
|
SVM accuracy: 1.0
|
||||||
|
|
||||||
|
===== Seeds =====
|
||||||
|
[0.17460317460317465, 0.17460317460317465, 0.12698412698412698, 0.12698412698412698, 0.15873015873015872, 0.15873015873015872, 0.15873015873015872, 0.1428571428571429] [0.09523809523809523, 0.12698412698412698, 0.12698412698412698, 0.12698412698412698, 0.11111111111111116, 0.1428571428571429, 0.1428571428571429, 0.1428571428571429]
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
k-NN error after PCA (k=5): 0.1587
|
||||||
|
Best k via CV: 11
|
||||||
|
CV accuracy: 0.9238
|
||||||
|
Leave-One-Out error: 0.0762
|
||||||
|
SVM accuracy: 0.873
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Additional data for report
|
||||||
|
from sklearn.decomposition import PCA
|
||||||
|
|
||||||
|
# Load datasets
|
||||||
|
X_bank, y_bank, _ = load_banknote()
|
||||||
|
X_seeds, y_seeds, _ = load_seeds()
|
||||||
|
|
||||||
|
print("=== Dataset Sizes ===")
|
||||||
|
print(f"Banknote: {X_bank.shape[0]} samples, {X_bank.shape[1]} features")
|
||||||
|
print(f"Seeds: {X_seeds.shape[0]} samples, {X_seeds.shape[1]} features")
|
||||||
|
|
||||||
|
# PCA variance explained
|
||||||
|
print("\n=== PCA Explained Variance (2 components) ===")
|
||||||
|
for name, X in [("Banknote", normalize(X_bank)), ("Seeds", normalize(X_seeds))]:
|
||||||
|
pca = PCA(n_components=2)
|
||||||
|
pca.fit(X)
|
||||||
|
total = sum(pca.explained_variance_ratio_) * 100
|
||||||
|
print(
|
||||||
|
f"{name}: {pca.explained_variance_ratio_[0] * 100:.1f}% + {pca.explained_variance_ratio_[1] * 100:.1f}% = {total:.1f}% total"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Feature scale ranges (to explain why normalization matters)
|
||||||
|
print("\n=== Feature Ranges (min-max span) ===")
|
||||||
|
print(
|
||||||
|
"Banknote:",
|
||||||
|
[f"{X_bank[:, i].max() - X_bank[:, i].min():.2f}" for i in range(X_bank.shape[1])],
|
||||||
|
)
|
||||||
|
print(
|
||||||
|
"Seeds:",
|
||||||
|
[
|
||||||
|
f"{X_seeds[:, i].max() - X_seeds[:, i].min():.2f}"
|
||||||
|
for i in range(X_seeds.shape[1])
|
||||||
|
],
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
=== Dataset Sizes ===
|
||||||
|
Banknote: 1372 samples, 4 features
|
||||||
|
Seeds: 210 samples, 7 features
|
||||||
|
|
||||||
|
=== PCA Explained Variance (2 components) ===
|
||||||
|
Banknote: 54.5% + 32.3% = 86.8% total
|
||||||
|
Seeds: 71.9% + 17.1% = 89.0% total
|
||||||
|
|
||||||
|
=== Feature Ranges (min-max span) ===
|
||||||
|
Banknote: ['13.87', '26.72', '23.21', '11.00']
|
||||||
|
Seeds: ['10.59', '4.84', '0.11', '1.78', '1.40', '7.69', '2.03']
|
||||||
|
|
||||||
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
+203
@@ -0,0 +1,203 @@
|
|||||||
|
---
|
||||||
|
title: Classification 1 - Comparative Analysis of k-NN and SVM Classifiers
|
||||||
|
author:
|
||||||
|
- "Lương Đức Anh - 23BI14021"
|
||||||
|
- "Mai Minh Vũ - 22BI13477"
|
||||||
|
bibliography: "cite.bib"
|
||||||
|
header-includes:
|
||||||
|
- \usepackage{float}
|
||||||
|
nocite: "[@*]"
|
||||||
|
---
|
||||||
|
|
||||||
|
# 1. k-Nearest Neighbor (k-NN) Classification
|
||||||
|
|
||||||
|
## 1.1 Selected datasets
|
||||||
|
|
||||||
|
Two labeled datasets were selected:
|
||||||
|
|
||||||
|
- **Banknote Authentication** [@banknote_authentication_267]
|
||||||
|
- Binary classification (genuine vs forged banknotes)
|
||||||
|
- 1,372 samples with 4 continuous features extracted from wavelet transforms
|
||||||
|
- Well-separated classes with similar feature scales (ranges 11–27)
|
||||||
|
|
||||||
|
- **Seeds Dataset** [@seeds_236]
|
||||||
|
- Multi-class classification (3 types of wheat seeds)
|
||||||
|
- 210 samples with 7 continuous geometric features
|
||||||
|
- Feature scales vary significantly (ranges 0.11–10.59), requiring normalization
|
||||||
|
|
||||||
|
Both datasets contain numerical features and ground-truth labels, making them suitable for k-NN and SVM classification.
|
||||||
|
|
||||||
|
## 1.2 k-NN classification and error computation
|
||||||
|
|
||||||
|
Each dataset was split into **70% training** and **30% testing** subsets. k-NN classification was applied across multiple values of $k$, and the **classification error** was computed as:
|
||||||
|
|
||||||
|
$$
|
||||||
|
\text{Error} = 1 - \text{Accuracy}
|
||||||
|
$$
|
||||||
|
|
||||||
|
## 1.3 Effect of varying $k$
|
||||||
|
|
||||||
|
The choice of $k$ reflects a bias-variance tradeoff: small $k$ values lead to flexible decision boundaries with high variance, while large $k$ values produce smoother boundaries with higher bias.
|
||||||
|
|
||||||
|
### Banknote Authentication
|
||||||
|
|
||||||
|
{latex-placement="H"}
|
||||||
|
|
||||||
|
- For small to moderate values of $k$, the classification error is **almost zero**
|
||||||
|
- Error increases slightly for larger $k$, indicating **over-smoothing**
|
||||||
|
- The dataset is easy to classify due to strong feature separation
|
||||||
|
|
||||||
|
**Observation:**
|
||||||
|
k-NN performs extremely well even with simple settings.
|
||||||
|
|
||||||
|
### Seeds Dataset
|
||||||
|
|
||||||
|
{latex-placement="H"}
|
||||||
|
|
||||||
|
- Error varies noticeably with $k$
|
||||||
|
- Small $k$ values show higher variance
|
||||||
|
- Intermediate values of $k$ yield the best performance
|
||||||
|
|
||||||
|
**Observation:**
|
||||||
|
The Seeds dataset is more sensitive to the choice of $k$, indicating overlapping classes.
|
||||||
|
|
||||||
|
## 1.4 Effect of normalization
|
||||||
|
|
||||||
|
Normalization was applied using standard scaling.
|
||||||
|
|
||||||
|
### Banknote Authentication
|
||||||
|
|
||||||
|
- No visible difference between raw and normalized data
|
||||||
|
- Features are already on comparable scales
|
||||||
|
|
||||||
|
### Seeds Dataset
|
||||||
|
|
||||||
|
- Normalization **dramatically reduces classification error** at small $k$: from 17.5% (raw) to 9.5% (normalized) at $k=1$
|
||||||
|
- The gap narrows at intermediate $k$ values, confirming that normalization has the greatest impact when few neighbors dominate the decision
|
||||||
|
|
||||||
|
**Conclusion:**
|
||||||
|
Normalization improves k-NN performance when feature scales differ, and is essential for distance-based classifiers.
|
||||||
|
|
||||||
|
## 1.5 PCA projection and k-NN performance
|
||||||
|
|
||||||
|
Principal Component Analysis (PCA) was applied to reduce the data to **2 dimensions**.
|
||||||
|
|
||||||
|
### Banknote Authentication
|
||||||
|
|
||||||
|
{latex-placement="H"}
|
||||||
|
|
||||||
|
- PCA projection reveals **significant class overlap** in the central region (PC1 between 0 and 2)
|
||||||
|
- The two principal components retain only **86.8%** of the total variance (54.5% + 32.3%)
|
||||||
|
- k-NN error increases from near 0% to **12.4%** after PCA, confirming that the remaining 13% of variance contains critical discriminative information
|
||||||
|
|
||||||
|
### Seeds Dataset
|
||||||
|
|
||||||
|
{latex-placement="H"}
|
||||||
|
|
||||||
|
- PCA reveals three visually distinct clusters with moderate overlap between classes
|
||||||
|
- The two principal components retain **89.0%** of variance (71.9% + 17.1%)
|
||||||
|
- k-NN error after PCA is **15.9%**, slightly worse than the normalized full-feature space
|
||||||
|
|
||||||
|
**Conclusion:**
|
||||||
|
PCA is useful for visualization, but dimensionality reduction may reduce classification accuracy when informative features are discarded.
|
||||||
|
|
||||||
|
## 1.6 k-fold cross-validation
|
||||||
|
|
||||||
|
A **10-fold cross-validation** was used to select the optimal $k$.
|
||||||
|
|
||||||
|
- **Banknote Authentication**
|
||||||
|
- Best $k = 15$ (selected by CV on normalized data)
|
||||||
|
- Cross-validation accuracy $\approx$ **99.9%**
|
||||||
|
- Note: on the held-out test set, error increases at higher $k$ due to over-smoothing
|
||||||
|
|
||||||
|
- **Seeds Dataset**
|
||||||
|
- Best $k = 11$
|
||||||
|
- Cross-validation accuracy $\approx$ **92.4%**
|
||||||
|
|
||||||
|
**Conclusion:**
|
||||||
|
Cross-validation provides a more reliable estimate of performance and avoids dependence on a single train/test split.
|
||||||
|
|
||||||
|
## 1.7 Leave-One-Out validation
|
||||||
|
|
||||||
|
Leave-One-Out (LOO) validation was applied using the best $k$.
|
||||||
|
|
||||||
|
- **Banknote Authentication**
|
||||||
|
- Error $\approx$ **0.15%**
|
||||||
|
- Confirms extremely high generalization performance
|
||||||
|
|
||||||
|
- **Seeds Dataset**
|
||||||
|
- Error $\approx$ **7.6%**
|
||||||
|
- Higher error due to class overlap and smaller dataset size
|
||||||
|
|
||||||
|
**Observation:**
|
||||||
|
LOO provides an accurate but computationally expensive estimate of classification error.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# 2. Support Vector Machine (SVM) Classification
|
||||||
|
|
||||||
|
## 2.1 Data separability analysis
|
||||||
|
|
||||||
|
- **Banknote Authentication**
|
||||||
|
- The data is nearly linearly separable
|
||||||
|
- Simple decision boundaries are sufficient
|
||||||
|
|
||||||
|
- **Seeds Dataset**
|
||||||
|
- Classes overlap
|
||||||
|
- Non-linear decision boundaries are required
|
||||||
|
|
||||||
|
## 2.2 SVM configuration
|
||||||
|
|
||||||
|
- RBF (Gaussian) kernel was selected
|
||||||
|
- Regularization parameter $C = 1$
|
||||||
|
- Automatic kernel coefficient scaling (`gamma='scale'`) was used
|
||||||
|
|
||||||
|
This configuration allows SVM to model non-linear class boundaries.
|
||||||
|
|
||||||
|
## 2.3 SVM performance
|
||||||
|
|
||||||
|
- **Banknote Authentication**
|
||||||
|
- Accuracy = **100%**
|
||||||
|
- SVM perfectly separates the classes
|
||||||
|
|
||||||
|
- **Seeds Dataset**
|
||||||
|
- Accuracy $\approx$ **87.3%**
|
||||||
|
- Lower than k-NN, likely due to suboptimal hyperparameters or the specific class structure
|
||||||
|
|
||||||
|
**Conclusion:**
|
||||||
|
SVM achieves perfect separation on linearly separable data (Banknote) but does not always outperform k-NN. Performance depends on hyperparameter tuning and dataset characteristics.
|
||||||
|
|
||||||
|
## 2.4 Multi-class handling in SVM
|
||||||
|
|
||||||
|
SVM is inherently a binary classifier.
|
||||||
|
Multi-class classification is handled using strategies such as:
|
||||||
|
|
||||||
|
- **One-vs-Rest (OvR)**
|
||||||
|
- **One-vs-One (OvO)**
|
||||||
|
|
||||||
|
The scikit-learn implementation automatically applies these strategies, allowing SVM to handle multi-class datasets transparently.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Summary
|
||||||
|
|
||||||
|
| Dataset | k-NN (CV) | k-NN (LOO) | SVM |
|
||||||
|
| -------- | --------- | ---------- | ----- |
|
||||||
|
| Banknote | 99.9% | 99.85% | 100% |
|
||||||
|
| Seeds | 92.4% | 92.4% | 87.3% |
|
||||||
|
|
||||||
|
Table: Classification accuracy comparison across methods.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Overall Conclusion
|
||||||
|
|
||||||
|
This study compared k-NN and SVM classifiers on two datasets with different characteristics. Key findings include:
|
||||||
|
|
||||||
|
- k-NN performs well on well-separated datasets but is sensitive to feature scaling and parameter choice
|
||||||
|
- Normalization significantly improves k-NN performance
|
||||||
|
- PCA is effective for visualization but may reduce classification accuracy
|
||||||
|
- Cross-validation is essential for selecting optimal hyperparameters
|
||||||
|
- SVM provides strong performance, especially for non-linear and multi-class problems
|
||||||
|
|
||||||
|
# References
|
||||||
LFS
BIN
Binary file not shown.
Reference in new issue
Block a user