kNN

k-Nearest Neighbors

Birds of the same feather are expected to flock together

---

Examples:

• R code and Python code to classify/predict a flower into species of iris [virginica, Versicolor, or setosa] by [petal.length, sepal.length, petal.width and sepal.length].

• R code to classify/predict a diamond into quality of the cut [Fair, Good, Very Good, Premium, Ideal] by [carat, depth, table, price, x, y, z].

---

Basic Concept

An object is assigned to the class most common among its k nearest neighbors. For example, if the neighbors based on some values of predictors all like to buy product X, we will predict that the new customer would also like to buy product X.

---

Classification Steps

Example: to classify a new sample

Step#1	Step#2	Step#3

---

How to pick the right k?

• A common practice is to use the square root of the number of features as the starting point to select k.
• Then use a validation error curve to pick k.

---

References:

• Jupyter Notebook example using the UCI iris dataset.