S5.KNN_from_Scratch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# k-Nearest Neighbors (KNN) from Scratch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## KNN Model Representation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The model representation for KNN is the entire training dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* KNN has no model other than storing the entire dataset, so there is **no learning required**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making Predictions with KNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* KNN makes predictions using the training dataset directly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Predictions are made for a new data point by searching through the entire training set for the k most similar instances (the neighbors) and summarizing the output variable for those k instances."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* For regression this might be **the mean output variable**, in classification this might be **the mode (or most common) class value**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* To determine which of the k instances in the training dataset are most similar to a new input **a distance measure** is used. For real-valued input variables, the most popular distance measure is **Euclidean distance**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* **Euclidean distance** is calculated as the square root of the sum of the squared differences between a point a and point b across all input attributes i."
   ]
  },
  {
   "attachments": {
    "image.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAACHCAYAAAAr+jJeAAAgAElEQVR4nO3dXUxU56I38PXenYvJm8wlFyQmhoQLkqbheFFi3uCFpDsmaNoYQ2sINm3AtA1o00EbB41CjYyxm4ktetpJtxPdHZtOzC72lO7j1Ion4D6yW4xw7FjFDMq4AUEnIxlgDc//vZi1YD7WrPlgZtbA/H8JF7Uws2bW86z1X8+nBCIiIiKDSEYfABEREZUuBhEiIiIyDIMIERERGYZBhIiIiAzDIEJERESGYRAhIiIiwzCIEBERkWEYRIiIiMgwDCJERERkGAYRIiIiMgyDCBERERmGQYSIiIgMwyBCREREhmEQISIiIsMwiBAREZFhGESIiIjIMAwiREREZBgGESIiIjIMgwgREREZhkGEiIiIDMMgQkRERIZhECEiIiLDMIgQERlqATPe3zA8PKz7c2ciCGH0oRLlAYMIEZGBwmO92CpJkPjDnwL9FJviOyIiohIiD9uwWWqG2y8bfShEhmAQISIyEIMIlToGESIiAzGIUKljECEiMhCDCJU6BhEiIgMxiFCpYxAhIjIQgwiVOgYRIiIDMYhQqWMQISIyEIMIlToGESIiAzGIUKljECEiMhCDCJU6BhEiIgMxiFCpYxAhIjIQgwiVOgYRIiIDMYhQqWMQISIyEIMIlToGESIiAy18+wb+TXoF1v9dyNM7hBEcvYj2tjY01u5Bl+cBJocvoaOlDdbuo2jetgNt3/2OUJ7enSgVBhEiIgPlPYgs34ez0Yah+ecY6twKadMrqD/6d/hlAWAJE659kMwfoX82nJ/3J0qBQYSIyED57poRE5fxbucg5pfvwVFXBtNOB7xLQvm/IXgduyGVHYYnsJyX9ydKhUGEiMhAhRojIvxuNJnKsdP5B8TKP47D1bAZpgYXfELvr4nyh0GEiMhAhQkiYcz2fwSztBO9Y/Mr/yp8LjSYNqPBNQ7mEDIKgwgRkYEKE0RmMWB9DVL5UQy8VCPHInyud2Ayt6JvSsbi6JdodtwDO2io0BhEiIgMVJAgEr6L3q1mlFk8CKz84yT6mish7XFhQkzBc/gI3JNL+TsGoiQYRIiIDFSQIDJ3DZaK19DW74/qgpExPWBDfW0jPm5txaeex+BKJmQEBhEiIgNxQTMqdQwiREQGYhChUscgQkRkIAYRKnUMIkREBmIQoVLHIEJEZCAGESp1DCJERAZiEKFSxyBCRGQgBhEqdQwiREQGYhChUscgQkRkoPSDyEuM9LwOSZKM/ak8hgHu1Es5xCBCRGSgxe8b8H//z7+j6+Fiit8MI3D7M2w3xQYD0zYLvvrODbd7bT/fOXths9nQbW1F0+5abE4aRiqxz+3jJnmUMwwiREQGyqxrJojR3j0wxQSDWhzy+PMQDMIITvwKj/tLnGisiXlP004HvEuMIpQbDCJERAbKeIzI0n249r0a20pR0Qq3L5THowwjOH4dX7VuVwLJ67ANv8jj+1EpYRAhIjJQ5oNVBWSfGy0VpqgwYkLFPhce5ruVQgTw8GoX6jebUNbWj1k2ilAOMIgQbVQigPF//oHnRX2zCOO59y7Gg2GjD8Qw2c2akTHt6cCWmC6aTajvvYN8totEhBG8+yX2lO2F8+FCen+yLspiMcqkfgjI/v/Guba3sXvvXuyufQXVu4/i8tiLoh/PwyBCtBGJFxi99DkujRb/RUgE7+DS6W8xWqJhJOvpu2IWt207Y8eLmHbCdnu2AOc8BJ/7AOps/4P5lMe5fspiMUq3fojALdjesODyg0Dke5b9uNFVD1NFM5zeYEGONVtZBxEx0YdDjY1ojBpdbaregb2NjWiM/9m7A9UrI73LUGP7R+rCmxfzGO+3w9LcpBxTNSyemZy+g5gbhrOjFU2Ne7G7tgJSnQPe9TzTTfjh6WpGY+ObqN2sNAWbqrFjb/x5fhO11dvR2N4D18BDBHnFMY54gVFHOw71+WDYyhRiEgP2Q2hubMTeHdUwlR2GJ+mUTwF58ioOvX+hJMPIWtYREcFh2Os3xY4X2dIBz3QBzvzSH3Cf+AZjizqVvRjKohEyKv8pXyyN+rGESfcBvO0ajwl7YqoPLWYJ5n1uTBbxNXntLSLL9+CoK4Mk1aJz6Hny35NfYPzGWTRWlKGm5zekmqiWLyL0HE/9d+FqroJk0mtaXIDfcwr11XtgG5rJIMnLCM48xZMBG2okEyqtNxDIzaEbLASvYzckyYzXkj0FybPwXrOjsWITattcGEt1U5Efw9O1G9X1n2ForvRuQPkRxtygDbsO/ogpQy88ywg9n4L/dxeazRJMDS74dI9HxlT/Eezquom5Ir5g5sPaFjQTWHpwEQ1mKWa8SGVbHyZlo7/IYimLRsi0/KeSqn78C/2tr0Ay7ULPSNQdR/wB585ySOaP0D9bvNfYNQcRNXFJ5lb0TaWqSC8wbHsDDXGpreAWf0NPjRlSjR0jydK88KO/bQukrJq1lhEYOIZKqQotfU82RnOkeIK+lipI0lZYB57p/GIYwdG/oKliE7aluKmI6R/RVmFGRdNFeEO5+pbCmBv6DDu3HcaP/qUcveb6IQK3YKtrgsNrTJtjvGWvA3WSOb2Hj6UxOHY1xF5IS8DaV1YNweduRUXMeJFXsc91H0bWgGIri0bIqPynols/Ahjp2QVTfBDBBNyNmyEV+cq9awwi6g033cS3CJ/rINpz3B2SKeFzocFkQlXnoH4XkfwSz4PZnLwgRnp2QDK9A5fPqLafHAvcgLXSlEHgfD0xnScQkIMvEMzlk5samNI6zo1mET7Xu6hqv1YksxlkTPW1wpyqtTTq96f7LagssTUqcrLEu5hE/4HXYrtozI1w/G7U2IBiK4tGyLT8p369jOvHywFYy6WiHyKwxiCi3HDTTnyL8LmOwT5i5MAZtXDksbVi+T6cuzbpt7isKwKLI3bUSBJMu5x4mLJAC8wPfYoqKY2wl2tKYDK39JVec/DiHfTWbUO7Z6pIWuGeYcC6NaNQKGavob3qjZJqFcnVXjNi7ia6tpXHrrq6/TPcDhjQJF90ZdEImZf/VDKrH0uYdO+HWdqGjoHpoj4Pawsi6g03SXO9CE7hacw4gRC8jpMGtxI8x1BnbV5bKyLdVQbchPNmAQ+de2HKoIkx0iRZ6CSuBqbNxnf/FZwS/iqPwFMsfcFKF2hG/ePiCfpaXtlAdSe13G16F0Zw5HPUxywBb8aWjl8KPO6mCMuiEbIp/6mkXT/UtWaq0eS4U/STB9YURPTHhyzCd/k0LsYMBg0j+Oghnho5iCqd8SFrUsrjQ1atBJE1jRbP1Dy8jrdgknagx9BWNyNEWieLqSVo+aETu0yZ9o8vwud6B6YN05qYWm533w3id0cjzDHjRV5DW0FnrRRfWTRCduU/lfTqR2R8zna0OIs/hABrCiJq87ukfVNfGoPjvd6UFxMRvIvLXUfR2W1FW/tXGPQnzmKJNDluxf6+ieQ3dhHA+MBl9LQfgKWzG7buDlg6LmAo7vX0x4cIyP5fYO84hlPt72N/zy/w64Um+RlG+86ivbUdnbaT6Og4jxv+Z6nHh4h5+Id/gKPzI7RZT8JmO4H29rPofxhI8vmWMDd6FXbls3V3HIf9xhMsBUdx5fz3cbNTwgiOXkZXRyesbe2w33gCGYAI+TDkOo329hOw2U7A0noMjqHJ9C5O84PorEp3fAgQUzYSgsgC/DfOo6PjBNpbDqBHOb6k5CmMuM/gQJsV3TYbbJ2H0W67Cq/6mRdH4Xy/aXUK8WYTpM212K1OK24+DY/WoFXNczeP4Nj3OH9lNLbyihcYvdSNjlNWtB34Ajf8CwCWEZoYgsv2Cdo7u2HrbEerNbG8JQojOH4Trp5P0Go5AZvtJKyWE0nPhQg9wfBVBzrbDsLarXz+np/wMH5GkhiHq6Eyg4teJmUqG2oX6FZYb4zjYf8X6Og4mXj+EiitWhtpfFUKuQ0iAORxuFuqC7wEfJSiK4tGyLb8p5K6fojgHTj2vY2Oa4+Va4qMmTu/YjxnEwJybw1BRB0fEn9TX0Zo7iEGz72HmlRTV5fu4/Kxv2AkGFamAZdrBAT1hNYk7W8Uwbtwte1Gg+0afKHVm56Yvo5j7/RiZF7EvZZ2a4WYHsDZ80ORZsyAB5Yyvfe8A2fLLuw7dwvTaliRx3HlaBve+1Py8SGRY92B7da/wfs86ga5NAbHXhuG5uP+RrzAqLMV2/f9B25Pqze5EHxXPkXre68nPP2L2es4c/YfCIhlBDyHUVZxAN//PoxLpy9F/X0Ys54jqDS9lcaI9kzHhwCr33N814yM6QEHzg/NQEA5Pr3mW/kxrh39AEdXKlTkeOTZQdhP/h3T8V+VOmc+xZNYZucujNmfz+Hs7VkIzMBj2YKK/Vfw+91vcfrrf6z+vZiCp71GfzMw8QJjLgvqG87A43u5Wq7EU9w4dgD2mH7fMIJjLrRt2wXr9/fwfCUQCyx5v8berrh6EvDAUpZmK1yGZSo7Sv+4qRaNFjuuroTseXgdb6NSZ6ZU5Dzmfo2fYpXzIAIBebIPbZWmmDBi3vM1fi/EzajoyqIRsi//qejVDxG8A0fLu+jyRF0zxSO4Gm0YWtiIQUSMw9WwGZJUhuodb0UtbPUWdlSXQUrZNSFj+qezOKdefJWn7sT+NGVMR5KncRG8A0dTDbYdvR57Y5r/Db07K+NaJvQGDwUwcv48fp4NI3Kxd2CnSXusQeQ9q1HR4oYvpsVE7ZbRbnERc4Ow1Vdp/B0AzMDTfjA25YoXGHW8hwqtpxl1FkvVp1HhRcbU1bP42juP1TBgRsVOe8KAtUjXSTpjKZSmwIymoCnnTDKhMnrU/PyvOH/muvLfSjdK0mS/BH/fAVTFL8QjnmLg0zexOeZzA6utMPqfKeNzJ57g6sm/RsKF2kVlqsLOnn8gEPMmyjoryT6Pci4rt3XhRsxiUwGM9L6FzTHHHZmCXL95C1rc44ktJQEP2vfG1pPI+Uzj5p1xmcqS2gWq8T6RY00e8iM3sjLUOe4h/U69RUy439fZuj7NH9OfYLtd2M3cch9EACCMuYETcUvAl2O77VZcuc29oiuLRlhL+U8lWf2QH+Naxw5U7v440nqs/pxqxY43nBgv4q8x+5VV1fEhWhde+QFcTfvg8Oo0BYpxXP7kovL0qDyha91E1AE/Wk/jahOkxlN1pDunGvVd11a7V3ReS8x6cPz0kHIDeonR3jdh0moxEDMY7Nqh/f9WPodGIVOn1yVphRCzHhz94HLUplVhzA12Y5tpE3Y6xhLWAxCz19AePztEPEHfJ18qT/PqFGKtKbTp3bRXXrOlClI6FxaVWgljwmgYs54/47Q6jW3xDnrrypO3ICjvm9C6sfQAfUc/Qsflu3F9n2oLnc4TVFrnLjZAi6kf8IldCWBq+dluj2plU+kNglbXedAqFzMY7KpHRf0peJRuHTH1Iw5UmpJ8N2HMek7hg5g1ItRWq936dS6bMpUlvf7xlGsrLN+Do26ToQsfFtLCt2/g36RXYP3fNPdtSZfmEvA70DWYyQKNGb9p0ZVFI6yp/Kd8ca36MYfbtvrYcx31U24dwMu1fKA8yzKIpBgfghl4jtn1k6wI4unTYKRCKM3aUoUF/XFLEyc/oeqNoxx1vXfSm8mhUzhEcBKPZpQLwfz/wPaaWWNZXLWlJNmSuclabwTmR+zYbortNhCh5/CP38WA+yw6ur6L7QtdGoNj5yZI5v1wTyZUU+0gIYJ4/OhZ5AladwqxOgsmjWZPdXxI2n32q99R7PkMI/jYhxlZRI5/+DRekyqxz+3TviiutD5sR2vv33Bz9BGm9dZ0UT9v0ieoNM9d3OeMLhe6g890vm/1omqqO4fRlAMwlYWJYgLRMkLP/RgfHYDbfgJdCSFM6eZKdfHPpkxlRW/9hNVrR5nFo911q6zWnPT/bzB5CyIAEBqFY09F7I2psh39eVtjp1BlcQE+9weoMO1JEXiiyDPwDg9jOOuf+8r1K+UbZVX+l33f4Z2KTXjdcQ+6I0g2YP3IMogkGx+iWBzB+VM/xy1kswT/1a9wReNmpk53rUxY/CZqwE/8bI2VJ/V0+xB1XiuGGnC2oK3fH1cJ1M+dpIIkbXFR/64ctc1HlSYzOxwuN/o8w7g/E4p7rahxGZpTv1I//esOylW71VI2e64eR/qzjJTFzKRybO/5p/YUM53guUpdoTW6n7sC21o+VwaLxr+kUoaSjktK79wl/5xqF5X24kTJv+/VLrK0noDU4zBtQ/OJ7khZ6XHA5b4Kz/AfmAlpdVakc/Ffe5lKn14XqBqCdbpeNuCFVk9+umZUAksPXdgXXY9M9bDdnsvDewGFK4sypgfsaOu4kuZYi9XFN7Pvukt31mB25V9M38Dpti64U63kvQHrR3ZBZGV8SAZTVOf/Cfsnf9N4EtUbjKpzQgMeWMqkDPoQ01xcRl3aXWsQ5cq+OtoVJHIz0rjhrHxf6V7kV/d10bx5qa+X9KapH7rSXll25eab/poowv899leYYNrWjcEk+8eoS7snBs+E34T83IsBtwN/tn6IPbWRJ7vE105jynS2527lF5TgW3kMAwnTkfVCygw8lurUezHFHEcmwQ9I7+K/1jKVAb31E9KpCxvwQqsnv0EEABYw2fexchPO99LvRVYWjbDW8p/KBqwfWQUR3fEhmpbg7zuBo5qDc3QCQkILwwKe3vchKFbXqUi7D1HntWI+m3JDWt3YbQHP/LOR7g7d8JN4819p1ldvgmmvqaF/80p4+o/ukgGgP8BXvWlGVQR5Fv5nGs3CK61O6Y6AV8fB6G1Frr7/67ANv9B//3jyM4w43kNFwveijoeJ28RQBPDo/tTazp1K6aLSLG8JF04B+dm/8EwWq+c+zanParnO9CIT+Tu9i9tay1QGAh5YypJ0gSrXDt3ZRYu/oadmEwer5op4gbvn9sJc0MGqRVIWjbDW8p9KVvWjuGURRFKND9H4i8At2N7uwbBWy4V6odZYgTPhKXXxN9g/+QFTYvWE6l+wBeSnD/EoGE7s3496rVXqTI430TuqDO2ZH8KZM8ogVrXZXHO10PhAJWOqz6YsZ69UvFRBRJ7C/UcBiJWmSa2nisSn/5gBlVHHqTnAV+mbXR2vIDA/dB5ntJ7WMxofogzINP0pav66hlTvL15g1PkhtlTuh+uhRhtMwANLWdxFTvMJSmDJexmnriprz2R97iKvFWlK3oRdzvtxlV8dexI9Vuk5hs6cjwQeNcyle+6VwJRuuV75F58LDSa9AcVrLFOZSDrrRa1fKQZMBjywlK3T1XHFU9z8aTSjVWHzG0QW4O8/gi2SKclsvdzLa1mUH8Njs8By/EPU17Tim4w3JC2AjMr/AvyeHrRZjqC1/nW0fHMPKUe8rOf6kUQWQSTF+JB48mNc63gDuzRGRgNYvYkk3CDUVTLVAi2w5P0rTl5VnsyVLpTkLSLLCPn+Drv9Z/jlsNJcmOS14o5lNa0uwX/1z6tTjNWxDQnHKhDyXoyMZ1D/n5iC59QF5cYY2ayoQm8RmtAj/GT/Upk5oY5T0aiooTE4m16NqsRhzHq+wPmoG7PuoNz4cCem8POZCxqzQDIYHyI/w8g3H6O28m3YUixQFnn/qJHyYgJXj3+9+v7qNO4thzR2z40M+q3bFfc0oXXzlsdx5dNLGFX7j7M+d4D+4N7EFiYxex1nzv+q1A3l3Ou0iMSce6VcJ18WOrpcR/3C/CA6qyp0Lk5rK1NAGEHvf+Kc/ZvUi7YlWdAqMmh3U8qn8kgZWY/rRyih9JXMppzmL4isLvluqv88sl5TIeStLM7jgfNjHPZMQbwcgLXcjK29d/UHdhoh7fIvsPTgEt49fA2zYhYD1tcgbe3FWIoPtH7rR3KZBxF1tHOq5no5gImRH/FF81aYKg6gL+mW7JH1IipixmQswO/5EvYLPWiu2BIJD2IKnmOno35HGYRVuQf2kecxxyFCTzDs/gKnL49gTrlYx6T0hNdS/zBys1JvaGJuCOfVBc6U9wx5L6Kp8u2o6Z9LmBtx4fSl6/jxRK1yMxOQfd/j5MV7q+FLHoe75f+h3j4c2x0kB+C7fQX2099hZC7qOwqNwdm0NWpqm4A89ysunnbh1o/HUaVW1KUxOD5yRt2Y9Xd8jFQG5TtFGHNDF5QFxuKpQTD507kITcM7+B1sTduwrdWBmxMvUyb0lO8vJtF/+BCcCU86ywhN/AJ7+2e4Fn8jFBPo21+9epzyM4xc/AKXY14j/XO35HXio+jgrLZqaHbrKBdVtcVDzGDo/AUMRY9hWboP176tiedezMM/HH/ulT0iEsq1gBz04XZcuV7935Fj1G1JybpMIapvO53u0MjUzO1RgTGyfksd9tiu669WrLa4rsfdk9WwK22HbTj9jrX8BBF1rxETpC1H0J9yxd9cvnWeyuLyPTg+vADvkqwEmVztaptr6Zb/ELyOY3B455XrojmtPWTWbf3QkWYQeQmv6xM0Ne7F7lp1Klj8QmZRP3t3oHpl46VybOu6qb/pkngBb99naGuxRJbbttrgGv4XZCzAf+MLHGiz4lRHNy6Nvoi70S0jNHETzpWl0k/C2vYROh0/YNg/H/e7qV4LACJLvPfsfx/tp46hw661xHsYQe9V2A4chLW7G53WkyvLc4vgXVzuOAhL5wl0xD+xAoA8iSHnSVjaT8Bms6Hb2o52mxM/3fFDa+C3CN5Dn80S+WydR9HpGIy8pniBscudaLUcR2dH78r6ExFL8PcdxJb6z2JvhjHfw1nsbzmEU8pSytGrli56L+NgUyMad9eu9Lebqndgb8I53oPdDR+i44tvccM7m0H/rd77K0cR8uFm1Pe0cl4v/By7Gm3Uccv+X2A/cBDWbr3l8rM8d2ICffu3o942qF2O5Se40XMALe0nlKXiNWb1hHy46fwUbcpS9d3Wg2jrdODq8BONcy8g+4fg7DwUWT7edhJWyyewOf+OOwnlWpXmHhRZlSkACML7TSu2VNRh91t/TuOJfwH+wa/Q3mZFd7cVbQfOwK35WeOt131KoqatZ7hGRD6CyMpOvBXvwaF5rcunPJfFdTGYNZPyrzFuL6n1Wj/0rW33XSIqGsLnQoM5bsBuzoXg/ct/5G8jw+X7cO7aknx9mWIlpuA51IjGvTWRB7AMbpI5DyJq90aq8Vp5lL+yqDdea53SG9MXb73WjxQYRIg2CuFHf1ut5kqVufMMN7/+z9xtax5DaVWoWm/bx0f2/3n70H/hqTquKoPpmTkNIvJjXOv4E0zSq8Zu/563shg7I3Bx9Es0r+vZI+rkD2Wow+Id9DZf0BhQH/nd9Vk/UmMQIdowlMG8dVpL0OfoHWZ/wdlL9/IUdAIY6dmDumQL4RUrMQXPodbI2KOV7Q3S757JWRBR926RyhP33iq4PJXF8F30bjVHliwXU/Ac/hjOB+uqtMSZx1jvTkjlRzHwUsaspxPvOr1J6tc6rR9pYBAh2kjEDAa7GvF+30Tum27FLG6fPadsDJl7wv893t9pS7oQXnFSW0PUxfnUWYXpd8/kJohETdPd54rasyqPxAt4r9/GRNLdpvNRFiNjlWrqP8RxiwU2jzFdT7kjEPJ+g5aaXWg93o42myfpYO71WT/SwyBCtMGIwC3Y3jiS4/1Ewpi9+dckg7xzQEyi/+C7OgvhFS8xfQe3HqnPqFHT3qWd6B1L/ey69iASNU13+2cJO23nR6Sb4I13tPZtivqtvJTFErSO60c6GESINpzI9PHDR3/AZAEWsFq7BUxe/QwnDRpYmXMLt2F7xQRJbz+dKGsLIlHTdDW2nM8XERyGvf61NFZcXm9lsRhtsPqhgUGEaEMSkP0/wx6zDk4xCiMwfAln130TezRlQKUkJVnJN9ZagogI3IJtezkk0+6E9ZTyJbImxqtJ9l3S/It1UhaL0UasH4kYRIiIcipqGwzdzd8isg4iS/fh2vcqJKkW7f0TBbhRLWB62Im22vL0V9YmSgODCBFRrqn7NKXRPZNVEBFPceNogabpytMY++kCTjTWwLSyQeDGWmKcjMUgQkSUc+l3z2QcRKKn6Xb8PcWS+ekTwQncGR7G8JAHfd850WuzonVPrfaOxtvPYawQM3OoJDCIEBHlXPrdM5kFkQX4r53ANpNGOCjYz8ba+ZWMxyBCRJQPK90z+jfu9IOIQOj3r7HHbGQIkSCZ98M9mb+1e6n0MIgQEeXFDDyW6sjGkQ2upMvipx9Ewgh6f8EVtxtuA3+u/Ow1bul42pAYRIiI8mIZAc9hlEkSJNM7cPm0F3zPx+67ROsJgwgRUb4EPLCU6Y+rYBChUscgQkSUN6m7ZxhEqNQxiBAR5U1090yLZthgEKFSxyBCRJRPK90zVZp7szCIUKljECEiyqvV7hlzSx+m4pIIgwiVOgYRIqK8kjHV1wqzJEEyt6JvKjZwFHsQEYFbsNUdRN902OhDoQ2KQYSIKM/EVB9azNrdM7kJIgvwuT9AhWlPyk320ibPwDvwV3TUV0Iq4qBE6x+DCBFRvokn6Gup0uyeyU0QkTE9YEdbxxV4QzlYbSzsheuD/bB0fY7eI/UMIpRXDCJERHkX3T3zEfpnV7s5irtrRobf3cwgQnnFIEJEVACr3TPVsHhmVv6dQYRKHYMIEVEhRHXPlFk8CCj/vKYgIj+Gx2aB5fiHqK9pxTfeYE4PmUGECoFBhIioIGT43S0wSRKkssPwBJYj/5p1EJnHA+fHOOyZgng5AGu5GVt77yIMAMv34dy1KbNddSuPYUA5pthjZhCh/GIQISIqEOFzocEU2z2TdRBZvgfHhxfgXZIx6zmCSqkWnUPPc3zEDCKUfwwiRESFIsbhatgc0z2z5jEi6mvW2DGymIMZMzEYRCj/GESIiApmET7XOzHdM2sLIgKLI3bUSJuwy3kf8R0ra8cgQvnHIEJEVECr3TORrpS1BZHnGOqsXVmxdXH0SzQ77mGZY0RoHWEQISIqpJXuGROqOgcRWNP70XsAAAFfSURBVEsQCd9F71Yzyq0DeCmm4Dn8MZwP5nN4sAwilH8MIkREBbWAcefbkVaIqk8xcP3UGlpEgvB+04qa+g9x3GKBzfMYOYkLYS9cHzShsfEt7KgugySVoXrHW2hsbMIHLi+46wzlEoMIEVGBLXsdqJMi3TPWztYiXtCMKP8YRIiICm35Hhx1ZZAkEzZt3gQTgwiVMAYRIqKCC8Hr2B01WJRBhEoXgwgRkQFWu2cYRKi0MYgQERlhpXuGQYRKG4MIEZEhortnGESodDGIEBEZJDzWi60MIlTiGESIiIyy+Bt6aswMIlTSGESIiAwTxEjPDgYRKmkMIkREhhFYGv8JZx034ZdzvXMu0frAIEJERESGYRAhIiIiwzCIEBERkWEYRIiIiMgwDCJERERkGAYRIiIiMgyDCBERERmGQYSIiIgMwyBCREREhmEQISIiIsMwiBAREZFhGESIiIjIMP8fUpe0GM1EHYQAAAAASUVORK5CYII="
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![image.png](attachment:image.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Other popular distance measures include:\n",
    "    * **Hamming Distance:** Calculate the distance between binary vectors.\n",
    "    * **Manhattan Distance:** Calculate the distance between real vectors using the sum of their absolute difference. Also called **City Block Distance**.\n",
    "    * **Minkowski Distance:** Generalization of Euclidean and Manhattan distance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from math import sqrt\n",
    "# Calculate the Euclidean distance between two vectors\n",
    "def euclidean_distance(row1, row2):\n",
    "    distance = 0.0\n",
    "    #print(row1)\n",
    "    for i in range(len(row1)-1):\n",
    "        distance += (row1[i] - row2[i])**2\n",
    "    return sqrt(distance)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Locate the most similar neighbors\n",
    "def get_neighbors(train, test_row, num_neighbors):\n",
    "    distances = list()\n",
    "    for train_row in train.values:\n",
    "        dist = euclidean_distance(test_row, train_row)\n",
    "        distances.append((train_row, dist))\n",
    "    distances.sort(key=lambda tup: tup[1])\n",
    "    neighbors = list()\n",
    "    for i in range(num_neighbors):\n",
    "        neighbors.append(distances[i][0])\n",
    "    return neighbors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make a prediction with neighbors\n",
    "def predict_classification(train, test_row, num_neighbors):\n",
    "    neighbors = get_neighbors(train, test_row, num_neighbors)\n",
    "    output_values = [row[-1] for row in neighbors]\n",
    "    prediction = max(set(output_values), key=output_values.count)\n",
    "    return prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# kNN Algorithm\n",
    "def k_nearest_neighbors(X_train, X_test, num_neighbors):\n",
    "    predictions = list()\n",
    "    for row in X_test.values:\n",
    "        output = predict_classification(X_train, row, num_neighbors)\n",
    "        predictions.append(output)\n",
    "    return(predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd \n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"Data/abalone.csv\",header=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>M</td>\n",
       "      <td>0.455</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.095</td>\n",
       "      <td>0.5140</td>\n",
       "      <td>0.2245</td>\n",
       "      <td>0.1010</td>\n",
       "      <td>0.150</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>M</td>\n",
       "      <td>0.350</td>\n",
       "      <td>0.265</td>\n",
       "      <td>0.090</td>\n",
       "      <td>0.2255</td>\n",
       "      <td>0.0995</td>\n",
       "      <td>0.0485</td>\n",
       "      <td>0.070</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>F</td>\n",
       "      <td>0.530</td>\n",
       "      <td>0.420</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.6770</td>\n",
       "      <td>0.2565</td>\n",
       "      <td>0.1415</td>\n",
       "      <td>0.210</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>M</td>\n",
       "      <td>0.440</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.125</td>\n",
       "      <td>0.5160</td>\n",
       "      <td>0.2155</td>\n",
       "      <td>0.1140</td>\n",
       "      <td>0.155</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>I</td>\n",
       "      <td>0.330</td>\n",
       "      <td>0.255</td>\n",
       "      <td>0.080</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.0895</td>\n",
       "      <td>0.0395</td>\n",
       "      <td>0.055</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   0      1      2      3       4       5       6      7   8\n",
       "0  M  0.455  0.365  0.095  0.5140  0.2245  0.1010  0.150  15\n",
       "1  M  0.350  0.265  0.090  0.2255  0.0995  0.0485  0.070   7\n",
       "2  F  0.530  0.420  0.135  0.6770  0.2565  0.1415  0.210   9\n",
       "3  M  0.440  0.365  0.125  0.5160  0.2155  0.1140  0.155  10\n",
       "4  I  0.330  0.255  0.080  0.2050  0.0895  0.0395  0.055   7"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "M    1528\n",
       "I    1342\n",
       "F    1307\n",
       "Name: 0, dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[0].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[0] = df[0].map({'M':0, 'I':1, 'F':2})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.455</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.095</td>\n",
       "      <td>0.5140</td>\n",
       "      <td>0.2245</td>\n",
       "      <td>0.1010</td>\n",
       "      <td>0.150</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0.350</td>\n",
       "      <td>0.265</td>\n",
       "      <td>0.090</td>\n",
       "      <td>0.2255</td>\n",
       "      <td>0.0995</td>\n",
       "      <td>0.0485</td>\n",
       "      <td>0.070</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0.530</td>\n",
       "      <td>0.420</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.6770</td>\n",
       "      <td>0.2565</td>\n",
       "      <td>0.1415</td>\n",
       "      <td>0.210</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0.440</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.125</td>\n",
       "      <td>0.5160</td>\n",
       "      <td>0.2155</td>\n",
       "      <td>0.1140</td>\n",
       "      <td>0.155</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0.330</td>\n",
       "      <td>0.255</td>\n",
       "      <td>0.080</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.0895</td>\n",
       "      <td>0.0395</td>\n",
       "      <td>0.055</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   0      1      2      3       4       5       6      7   8\n",
       "0  0  0.455  0.365  0.095  0.5140  0.2245  0.1010  0.150  15\n",
       "1  0  0.350  0.265  0.090  0.2255  0.0995  0.0485  0.070   7\n",
       "2  2  0.530  0.420  0.135  0.6770  0.2565  0.1415  0.210   9\n",
       "3  0  0.440  0.365  0.125  0.5160  0.2155  0.1140  0.155  10\n",
       "4  1  0.330  0.255  0.080  0.2050  0.0895  0.0395  0.055   7"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df.drop(8, axis=1)\n",
    "y = df[8]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test = train_test_split(df, test_size=0.33, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1593</th>\n",
       "      <td>1</td>\n",
       "      <td>0.525</td>\n",
       "      <td>0.380</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.6150</td>\n",
       "      <td>0.2610</td>\n",
       "      <td>0.1590</td>\n",
       "      <td>0.1750</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>111</th>\n",
       "      <td>0</td>\n",
       "      <td>0.465</td>\n",
       "      <td>0.360</td>\n",
       "      <td>0.105</td>\n",
       "      <td>0.4310</td>\n",
       "      <td>0.1720</td>\n",
       "      <td>0.1070</td>\n",
       "      <td>0.1750</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3271</th>\n",
       "      <td>0</td>\n",
       "      <td>0.520</td>\n",
       "      <td>0.425</td>\n",
       "      <td>0.155</td>\n",
       "      <td>0.7735</td>\n",
       "      <td>0.2970</td>\n",
       "      <td>0.1230</td>\n",
       "      <td>0.2550</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1089</th>\n",
       "      <td>1</td>\n",
       "      <td>0.450</td>\n",
       "      <td>0.330</td>\n",
       "      <td>0.105</td>\n",
       "      <td>0.3715</td>\n",
       "      <td>0.1865</td>\n",
       "      <td>0.0785</td>\n",
       "      <td>0.0975</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2918</th>\n",
       "      <td>1</td>\n",
       "      <td>0.600</td>\n",
       "      <td>0.445</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.9205</td>\n",
       "      <td>0.4450</td>\n",
       "      <td>0.2035</td>\n",
       "      <td>0.2530</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      0      1      2      3       4       5       6       7   8\n",
       "1593  1  0.525  0.380  0.135  0.6150  0.2610  0.1590  0.1750   8\n",
       "111   0  0.465  0.360  0.105  0.4310  0.1720  0.1070  0.1750   9\n",
       "3271  0  0.520  0.425  0.155  0.7735  0.2970  0.1230  0.2550  17\n",
       "1089  1  0.450  0.330  0.105  0.3715  0.1865  0.0785  0.0975   7\n",
       "2918  1  0.600  0.445  0.135  0.9205  0.4450  0.2035  0.2530   9"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>866</th>\n",
       "      <td>0</td>\n",
       "      <td>0.605</td>\n",
       "      <td>0.455</td>\n",
       "      <td>0.160</td>\n",
       "      <td>1.1035</td>\n",
       "      <td>0.4210</td>\n",
       "      <td>0.3015</td>\n",
       "      <td>0.325</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1483</th>\n",
       "      <td>0</td>\n",
       "      <td>0.590</td>\n",
       "      <td>0.440</td>\n",
       "      <td>0.150</td>\n",
       "      <td>0.8725</td>\n",
       "      <td>0.3870</td>\n",
       "      <td>0.2150</td>\n",
       "      <td>0.245</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>599</th>\n",
       "      <td>2</td>\n",
       "      <td>0.560</td>\n",
       "      <td>0.445</td>\n",
       "      <td>0.195</td>\n",
       "      <td>0.9810</td>\n",
       "      <td>0.3050</td>\n",
       "      <td>0.2245</td>\n",
       "      <td>0.335</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1702</th>\n",
       "      <td>2</td>\n",
       "      <td>0.635</td>\n",
       "      <td>0.490</td>\n",
       "      <td>0.170</td>\n",
       "      <td>1.2615</td>\n",
       "      <td>0.5385</td>\n",
       "      <td>0.2665</td>\n",
       "      <td>0.380</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>670</th>\n",
       "      <td>0</td>\n",
       "      <td>0.475</td>\n",
       "      <td>0.385</td>\n",
       "      <td>0.145</td>\n",
       "      <td>0.6175</td>\n",
       "      <td>0.2350</td>\n",
       "      <td>0.1080</td>\n",
       "      <td>0.215</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      0      1      2      3       4       5       6      7   8\n",
       "866   0  0.605  0.455  0.160  1.1035  0.4210  0.3015  0.325   9\n",
       "1483  0  0.590  0.440  0.150  0.8725  0.3870  0.2150  0.245   8\n",
       "599   2  0.560  0.445  0.195  0.9810  0.3050  0.2245  0.335  16\n",
       "1702  2  0.635  0.490  0.170  1.2615  0.5385  0.2665  0.380   9\n",
       "670   0  0.475  0.385  0.145  0.6175  0.2350  0.1080  0.215  14"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_neighbors = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "predicted = k_nearest_neighbors(X_train, X_test, num_neighbors)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.24147933284989123\n"
     ]
    }
   ],
   "source": [
    "print (accuracy_score(X_test[8].values, predicted))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = KNeighborsClassifier(n_neighbors=num_neighbors)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier()"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = model.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.24655547498187091\n"
     ]
    }
   ],
   "source": [
    "print (accuracy_score(y_test, predictions))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.3 64-bit ('base': conda)",
   "language": "python",
   "name": "python38364bitbaseconda5399315a2a954424bee177201f1b943d"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}