16.MachineLearning_4z5_UnsupervisedLearning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Warsztaty Python w Data Science\n",
    "\n",
    "---\n",
    "\n",
    "## Machine Learning - część 4 z 5. Unsupervised Learning\n",
    "\n",
    "- ### Clustering\n",
    "- ### Przekleństwo wymiarowości (ang. __*Dimensionality Curse*__)\n",
    "- ### Reguły asocjacyjne - algorytm Apriori\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://scikit-learn.org/stable/modules/clustering.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Dane syntetyczne"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.datasets import make_classification\n",
    "from sklearn.datasets import make_blobs\n",
    "from sklearn.datasets import make_gaussian_quantiles\n",
    "plt.style.use(\"dark_background\")\n",
    "plt.figure(figsize=(8, 8))\n",
    "\n",
    "plt.title(\"Three blobs\", fontsize='small')\n",
    "X1, Y1 = make_blobs(n_features=2, centers=3, random_state=4)\n",
    "plt.scatter(X1[:, 0], X1[:, 1], marker='o', c=Y1,\n",
    "            s=25, edgecolor='w')\n",
    "\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# Clustering K-means"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline \n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "from sklearn.cluster import KMeans\n",
    "from sklearn import datasets\n",
    "plt.style.use(\"dark_background\")\n",
    "\n",
    "df = pd.DataFrame(X1)\n",
    "\n",
    "kmeanModel = KMeans(n_clusters=3)\n",
    "kmeanModel.fit(df)\n",
    "\n",
    "df['k_means']=kmeanModel.predict(df)\n",
    "df['target']=Y1\n",
    "fig, axes = plt.subplots(1, 2, figsize=(16,8))\n",
    "axes[0].scatter(df[0], df[1], c=df['target'])\n",
    "axes[1].scatter(df[0], df[1], c=df['k_means'])\n",
    "axes[0].set_title('Actual', fontsize=18);\n",
    "axes[1].set_title('K-Means', fontsize=18);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Przekleństwo wymiarowości (ang. __*Dimensionality Curse*__)\n",
    "\n",
    "### \"Przekleństwo gęstości informacji\"\n",
    "- 2-wymiarowy sześcian o boku 1 \n",
    "  - każdy bok próbkujemy co 0.1 - $10^2$ próbek \n",
    "- 10-wymiarowy sześcian o boku 1 \n",
    "  - każdy bok próbkujemy co 0.1 - $10^{10}$ próbek \n",
    "- 100-wymiarowy sześcian o boku 1 \n",
    "  - każdy bok próbkujemy co 0.1 - $10^{100}$ próbek \n",
    "\n",
    "### \"Przeklęństwo odległości\"\n",
    "- 2-wymiarowy sześcian o boku 1 \n",
    "  - ma przekątną $\\sqrt{2} \\approx 1.41$ \n",
    "- 10-wymiarowy sześcian o boku 1 \n",
    "  - ma przekątną $\\sqrt{10} \\approx 3.16$  \n",
    "- 100-wymiarowy sześcian o boku 1 \n",
    "  - ma przekątną $\\sqrt{100} = 10$  \n",
    "\n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Iris Dataset\n",
    "\n",
    "This data sets consists of 3 different types of irises’ (Setosa, Versicolour, and Virginica) petal and sepal length, stored in a 150x4 numpy.ndarray\n",
    "\n",
    "The rows being the samples and the columns being: \n",
    "- Sepal Length \n",
    "- Sepal Width \n",
    "- Petal Length \n",
    "- Petal Width\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline \n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "from sklearn.cluster import KMeans\n",
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()\n",
    "\n",
    "data=pd.DataFrame(iris['data'])\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "from sklearn import datasets\n",
    "from sklearn.decomposition import PCA\n",
    "\n",
    "# import some data to play with\n",
    "iris = datasets.load_iris()\n",
    "X = iris.data[:, :2]  # we only take the first two features.\n",
    "y = iris.target\n",
    "\n",
    "x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5\n",
    "y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5\n",
    "\n",
    "\n",
    "\n",
    "fig = plt.figure(1, figsize=(8, 6))\n",
    "ax = Axes3D(fig, elev=-150, azim=110)\n",
    "X_reduced = PCA(n_components=3).fit_transform(iris.data)\n",
    "ax.scatter(X_reduced[:, 0], X_reduced[:, 1], X_reduced[:, 2], c=y,\n",
    "           cmap=plt.cm.Set1, edgecolor='k', s=40)\n",
    "ax.set_title(\"First three PCA directions\")\n",
    "ax.set_xlabel(\"1st eigenvector\")\n",
    "ax.w_xaxis.set_ticklabels([])\n",
    "ax.set_ylabel(\"2nd eigenvector\")\n",
    "ax.w_yaxis.set_ticklabels([])\n",
    "ax.set_zlabel(\"3rd eigenvector\")\n",
    "ax.w_zaxis.set_ticklabels([])\n",
    "\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "distortions = []\n",
    "K = range(1,10)\n",
    "for k in K:\n",
    "    kmeanModel = KMeans(n_clusters=k)\n",
    "    kmeanModel.fit(data)\n",
    "    distortions.append(kmeanModel.inertia_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(16,8))\n",
    "plt.plot(K, distortions, 'bx-')\n",
    "plt.xlabel('k')\n",
    "plt.ylabel('Distortion')\n",
    "plt.title('The Elbow Method showing the optimal k')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## n_clusters=3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df=pd.DataFrame(iris['data'])\n",
    "\n",
    "kmeanModel = KMeans(n_clusters=3)\n",
    "kmeanModel.fit(df)\n",
    "\n",
    "df['k_means']=kmeanModel.predict(df)\n",
    "df['target']=iris['target']\n",
    "fig, axes = plt.subplots(1, 2, figsize=(16,8))\n",
    "axes[0].scatter(df[0], df[1], c=df['target'])\n",
    "axes[1].scatter(df[0], df[1], c=df['k_means'])\n",
    "axes[0].set_title('Actual', fontsize=18);\n",
    "axes[1].set_title('K-Means N=3', fontsize=18);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## n_clusters=2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df=pd.DataFrame(iris['data'])\n",
    "\n",
    "kmeanModel = KMeans(n_clusters=2)\n",
    "kmeanModel.fit(df)\n",
    "\n",
    "df['k_means']=kmeanModel.predict(df)\n",
    "df['target']=iris['target']\n",
    "fig, axes = plt.subplots(1, 2, figsize=(16,8))\n",
    "axes[0].scatter(df[0], df[1], c=df['target'])\n",
    "axes[1].scatter(df[0], df[1], c=df['k_means'])\n",
    "axes[0].set_title('Actual', fontsize=18);\n",
    "axes[1].set_title('K-Means N=2', fontsize=18);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## n_clusters=4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df=pd.DataFrame(iris['data'])\n",
    "\n",
    "kmeanModel = KMeans(n_clusters=4)\n",
    "kmeanModel.fit(df)\n",
    "\n",
    "df['k_means']=kmeanModel.predict(df)\n",
    "df['target']=iris['target']\n",
    "fig, axes = plt.subplots(1, 2, figsize=(16,8))\n",
    "axes[0].scatter(df[0], df[1], c=df['target'])\n",
    "axes[1].scatter(df[0], df[1], c=df['k_means'])\n",
    "axes[0].set_title('Actual', fontsize=18);\n",
    "axes[1].set_title('K-Means N=4', fontsize=18);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### Hierarchical Clustering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "from scipy.cluster.hierarchy import dendrogram\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.cluster import AgglomerativeClustering\n",
    "\n",
    "\n",
    "def plot_dendrogram(model, **kwargs):\n",
    "    # Create linkage matrix and then plot the dendrogram\n",
    "\n",
    "    # create the counts of samples under each node\n",
    "    counts = np.zeros(model.children_.shape[0])\n",
    "    n_samples = len(model.labels_)\n",
    "    for i, merge in enumerate(model.children_):\n",
    "        current_count = 0\n",
    "        for child_idx in merge:\n",
    "            if child_idx < n_samples:\n",
    "                current_count += 1  # leaf node\n",
    "            else:\n",
    "                current_count += counts[child_idx - n_samples]\n",
    "        counts[i] = current_count\n",
    "\n",
    "    linkage_matrix = np.column_stack([model.children_, model.distances_,\n",
    "                                      counts]).astype(float)\n",
    "\n",
    "    # Plot the corresponding dendrogram\n",
    "    dendrogram(linkage_matrix, **kwargs)\n",
    "\n",
    "\n",
    "iris = load_iris()\n",
    "X = iris.data\n",
    "\n",
    "# setting distance_threshold=0 ensures we compute the full tree.\n",
    "model = AgglomerativeClustering(distance_threshold=0, n_clusters=None)\n",
    "\n",
    "model = model.fit(X)\n",
    "plt.title('Hierarchical Clustering Dendrogram')\n",
    "# plot the top three levels of the dendrogram\n",
    "plot_dendrogram(model, truncate_mode='level', p=3)\n",
    "plt.xlabel(\"Number of points in node (or index of point if no parenthesis).\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# Reguły asocjacyjne - algorytm Apriori\n",
    "\n",
    "Tworzy reguły \"jeśli A to B\" pisane $A \\rightarrow B$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__*Support*__ dla $X$ - jaka część zbioru tranzakcyjnego zawiera $X$\n",
    "\n",
    "$$supp(X) = \\frac{|\\{X \\subset T\\}|}{|T|}$$\n",
    "\n",
    "\n",
    "__*Confidence*__ dla $X \\rightarrow Y$ - jak często dana reguła jest prawdziwa\n",
    "\n",
    "$$conf(X \\rightarrow Y) = supp(X \\cup Y) \\div supp(X)$$\n",
    "jaka część zbioru tranzakcyjnego zawiera $X$\n",
    "\n",
    "\n",
    "__*lift*__ dla $X \\rightarrow Y$ - jak zależne są $X$ i $Y$\n",
    "$$lift(X \\rightarrow Y) = \\frac{supp(X \\cup Y)}{supp(X) \\times supp(Y)}$$\n",
    "\n",
    "- $lift < 1$ - przedmioty są komplementarne\n",
    "- $lift > 1$ - przedmioty są skorelowane\n",
    "- $lift = 1$ - przedmioty są niezależne\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "!pip install efficient-apriori"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from efficient_apriori import apriori\n",
    "\n",
    "itemsetlist = [['eggs', 'bacon', 'soup'],\n",
    "                ['eggs', 'bacon', 'apple'],\n",
    "                ['soup', 'bacon', 'banana']]\n",
    "\n",
    "freqitemset, rules = apriori(itemsetlist, min_support=0.5, min_confidence=0.5)\n",
    "\n",
    "print(rules)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MovieLens\n",
    "\n",
    "100,000 ratings and 3,600 tag applications applied to 9,000 movies by 600 users. Last updated 9/2018.\n",
    "\n",
    "https://grouplens.org/datasets/movielens/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "movies = pd.read_csv('data/ml-latest-small/movies.csv')\n",
    "movies.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "id_title = {}\n",
    "for index, row in movies.iterrows():\n",
    "    id_title[row['movieId']] = row['title']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "id_title[260]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratings = pd.read_csv('data/ml-latest-small/ratings.csv')\n",
    "ratings.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratings['rating'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratings = ratings[ ratings['rating'] > 3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratings['rating'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def to_item_set_list(df):\n",
    "    usermap = {}\n",
    "    for index, row in df.iterrows():\n",
    "        userlist = usermap.get(row['userId'], [])\n",
    "        userlist.append(int(row['movieId']))\n",
    "        usermap[row['userId']] = userlist\n",
    "    return usermap.values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "to_item_set_list(ratings.head(50))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "itemsetlist = to_item_set_list(ratings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(itemsetlist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "freqitemset, rules = apriori(itemsetlist, min_support=0.1, min_confidence=0.8)\n",
    "len(rules)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "\n",
    "\n",
    "def get_recommendations(rules, movieid):\n",
    "    ret = []\n",
    "    for rule in rules:\n",
    "        if rule.lhs[0]==movieid:\n",
    "            ret.append(rule.rhs[0])\n",
    "    return sorted(Counter(ret).items(),key = lambda x: x[1], reverse=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_recommendations(rules, 260)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for recommendation in get_recommendations(rules, 260):\n",
    "    print(id_title[recommendation[0]], recommendation[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SPMF data-mining\n",
    "\n",
    "- 202 algorytmy\n",
    "- 41 w tej samej kategorii co Apriori\n",
    "\n",
    "http://www.philippe-fournier-viger.com/spmf/index.php?link=algorithms.php"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}