Recommendation_Systems_Colab.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Recommendation Systems Colab.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "code",
      "metadata": {
        "id": "ELkeHGciW9wG",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 62
        },
        "outputId": "fbbd8795-d46d-499d-f515-c71257bb40ba"
      },
      "source": [
        "from __future__ import print_function\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import collections\n",
        "from mpl_toolkits.mplot3d import Axes3D\n",
        "from IPython import display\n",
        "from matplotlib import pyplot as plt\n",
        "import sklearn\n",
        "import sklearn.manifold\n",
        "import tensorflow as tf\n",
        "from sklearn.preprocessing import MinMaxScaler"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<p style=\"color: red;\">\n",
              "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.<br>\n",
              "We recommend you <a href=\"https://www.tensorflow.org/guide/migrate\" target=\"_blank\">upgrade</a> now \n",
              "or ensure your notebook will continue to use TensorFlow 1.x via the <code>%tensorflow_version 1.x</code> magic:\n",
              "<a href=\"https://colab.research.google.com/notebooks/tensorflow_version.ipynb\" target=\"_blank\">more info</a>.</p>\n"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2tx2QWqaPCnY",
        "colab_type": "text"
      },
      "source": [
        "# Download and create dataframes \n",
        "\n",
        "The *ml100k* datasets containts different items for different usecases. We are only interested in the .user and .data items."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WJ-376M1XECM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Read the users and ratings datasets\n",
        "\n",
        "users_cols = ['user_id', 'age', 'sex', 'occupation', 'zip_code']\n",
        "users = pd.read_csv('/content/drive/My Drive/Colab Notebooks/ml-100k/u.user', sep='|', names=users_cols)\n",
        "\n",
        "ratings_cols = ['user_id', 'movie_id', 'rating', 'unix_timestamp']\n",
        "ratings = pd.read_csv('/content/drive/My Drive/Colab Notebooks/ml-100k/u.data', sep='\\t', names=ratings_cols, encoding='latin-1')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "VE7qgOqAXZsp",
        "colab": {}
      },
      "source": [
        "# Read in the movies datasets\n",
        "\n",
        "genre_cols = [\n",
        "    \"genre_unknown\", \"Action\", \"Adventure\", \"Animation\", \"Children\", \"Comedy\",\n",
        "    \"Crime\", \"Documentary\", \"Drama\", \"Fantasy\", \"Film-Noir\", \"Horror\",\n",
        "    \"Musical\", \"Mystery\", \"Romance\", \"Sci-Fi\", \"Thriller\", \"War\", \"Western\"]\n",
        "    \n",
        "movies_cols = ['movie_id', 'title', 'release_date', 'video_release_date', 'imdb_url'] + genre_cols\n",
        "movies = pd.read_csv('/content/drive/My Drive/Colab Notebooks/ml-100k/u.item', sep='|', names=movies_cols, encoding='latin-1')\n",
        "movies = movies.drop(columns=['release_date', 'video_release_date', 'imdb_url'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z99CXFA1XHK-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Split the data\n",
        "\n",
        "def split_dataframe(df, holdout_fraction=0.1):\n",
        "    \n",
        "  test = df.sample(frac=0.1, replace=False)\n",
        "  train = df[~df.index.isin(test.index)]\n",
        "\n",
        "  return train, test"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gw5htnNrQBCr",
        "colab_type": "text"
      },
      "source": [
        "# Matrix Factorization\n",
        "\n",
        "The goal of matrix factorization is to decompose the feedback matrix, $A_{nm}$ into a product approximation of 2 embedding vector that is $U_n$ and $V_m$. \n",
        "\n",
        "To put it in simple terms:\n",
        "\n",
        "$A_{NM}$ approximates\n",
        "$U = \\begin{bmatrix} u_{1} \\\\ \\hline u_{N} \\end{bmatrix}$ x\n",
        "$V= (\\begin{bmatrix} v_{1} \\\\ \\hline \\hline v_{M} \\end{bmatrix})^T$ \n",
        "\n",
        "where,\n",
        "\n",
        "- N = Number of users\n",
        "- M = Number of movies\n",
        "- $U$ = user embedding vector \n",
        "- $V$ = movie embedding vector\n",
        "\n",
        "*source: [Link](https://machinelearningmastery.com/introduction-to-matrix-decompositions-for-machine-learning/)*\n",
        "\n",
        "\n",
        "# Objective\n",
        "\n",
        "To create a matrix factorization model, the inputs $U$ and $V$ has to be a sparse matrix. This is because generally in a user-movie model, not all the movies are rated by the user. The general steps to build a MF model is detailed below:\n",
        "\n",
        " - Create and initialise both $U$ and $V$ sparse tensor matrix with random variables \n",
        "\n",
        " - Select the loss function to approximate the UT\n",
        "\n",
        " - Add regularization function (L1 or L2) to reduce overfitting of outlier features\n",
        "\n",
        " -  Perform user recommendation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AIWTAAs7Yyma",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Build the sparse ratings matrix for our datasets\n",
        "\n",
        "def build_rating_sparse_tensor(ratings_df):\n",
        "\n",
        "  return tf.SparseTensor(indices=ratings[['user_id', 'movie_id']].values, values=np.array(ratings['rating']), dense_shape=(users.shape[0], movies.shape[0]))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X5lpgHqWWDqs",
        "colab_type": "text"
      },
      "source": [
        " - Select the loss function to approximate the UV"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ma6W7RViZJ0X",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def mse_error(sparse_matrix_UV, vectorU, vectorV):\n",
        "    \n",
        "  # Matmul to find the dot product of U and V for approximation\n",
        "  #**tf.gather_nd() needs GPU runtime so that it does not check the index bound. CPU does check and will cause error**#\n",
        "  predictions = tf.gather_nd(tf.matmul(vectorU, vectorV, transpose_b=True),sparse_matrix_UV.indices)\n",
        "  \n",
        "  # Finding the MSE between A and U*V_transpose\n",
        "  return tf.losses.mean_squared_error(sparse_matrix_UV.values, predictions)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eiv-T_p_ZKbC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_ratings, test_ratings = split_dataframe(ratings)\n",
        "\n",
        "# initialise a U and V matrix\n",
        "\n",
        "fakeA = tf.Variable(tf.random.normal([build_rating_sparse_tensor(test_ratings).dense_shape[0], 3]))\n",
        "fakeB = tf.Variable(tf.random.normal([build_rating_sparse_tensor(test_ratings).dense_shape[1], 3]))\n",
        "gatherPredictions = tf.gather(tf.matmul(fakeA, fakeB, transpose_b=True), [942, 2])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZVBDfDAAZdhs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Creating the training class\n",
        "\n",
        "class Model(object):\n",
        "\n",
        "  def __init__(self, embedding_vars, loss, metrics=None):\n",
        "\n",
        "    self.embedding_vars = embedding_vars\n",
        "    self.loss = loss\n",
        "    self.metrics = metrics\n",
        "    self.embeddings = {k: None for k in embedding_vars}\n",
        "    self.session = None\n",
        "\n",
        "  def embeddings(self):\n",
        "\n",
        "    return self.embeddings\n",
        "\n",
        "  def train(self, num_iterations=100, learning_rate=1.0, plot_results=True, optimizer=tf.train.GradientDescentOptimizer):\n",
        "\n",
        "    with self.loss.graph.as_default():\n",
        "\n",
        "      # Minimize loss function\n",
        "      train_op = optimizer(learning_rate).minimize(self.loss)\n",
        "\n",
        "      # Initialise the operation\n",
        "      local_init_op = tf.group(tf.variables_initializer(optimizer(learning_rate).variables()), tf.local_variables_initializer())\n",
        "      \n",
        "      if self.session is None:\n",
        "        self.session = tf.Session()\n",
        "\n",
        "        with self.session.as_default():\n",
        "          self.session.run([tf.global_variables_initializer(), tf.tables_initializer()])\n",
        "\n",
        "    with self.session.as_default():\n",
        "      local_init_op.run()\n",
        "      iterations = []\n",
        "      metrics = {}\n",
        "      metrics_vals = {}\n",
        "\n",
        "      # Train and append results.\n",
        "      for i in range(num_iterations + 1):\n",
        "        _, results = self.session.run((train_op, metrics))\n",
        "        if i == num_iterations:\n",
        "\n",
        "          for metric_val, result in zip(metrics_vals, results):\n",
        "\n",
        "            # Embeddings are u and k respectively\n",
        "            for k, v in result.items():\n",
        "              metric_val[k].append(v)\n",
        "\n",
        "      for k, v in self.embedding_vars.items():\n",
        "        self.embeddings[k] = v.eval()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q2rIt-268IHU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Compute the embedding scores\n",
        "# Once the model is trained, the embeddings can be inspected to understand the learned features between the users and the matrix\n",
        "# Given the feedback matrix A:(MxN), the model can learn the embeddings, U (Mxd) and V(Nxd) such that the dot product of U and V_transpose will be the approximated A matrix.\n",
        "\n",
        "def userRecommendation(model, userID):\n",
        "\n",
        "  # Upon getting the user and movie embedding vector, the score recreates the feedback matrix. \n",
        "  userEmbedding = model.embeddings['user_id'][userID]\n",
        "  movieEmbedding = model.embeddings['movie_id']\n",
        "\n",
        "  scores = userEmbedding.dot(movieEmbedding.T)\n",
        "\n",
        "  df = pd.DataFrame({ \n",
        "      'SCORE': list(scores),\n",
        "      'MOVIE': movies['movie_id'],\n",
        "      'TITLE': movies['title'],})\n",
        "  \n",
        "  return display.display(df.sort_values('SCORE', ascending=False).head(5)), df"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZQLjAeIOv6fZ",
        "colab_type": "text"
      },
      "source": [
        "- Add regularization function L1 to reduce overfitting of outlier features"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_ARTW0ypD40O",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# One of the common issue with training only with observed pairs is the problem of overfitting it. To solve this, L1 and L2 regularization can be used. These regularizations are basically penalty terms to reduce overfitting\n",
        "\n",
        "def build_model(ratings, embedding_dim=3, regularization_coeff=0.1, init_stddev=0.1):\n",
        "\n",
        "  # Split the training and test datasets\n",
        "  train_ratings, test_ratings = split_dataframe(ratings)\n",
        "\n",
        "  # Creating a random variable vector for the embeddings\n",
        "  vectorU = tf.Variable(tf.random.normal([build_rating_sparse_tensor(train_ratings).dense_shape[0], embedding_dim], stddev=init_stddev))\n",
        "  vectorV = tf.Variable(tf.random.normal([build_rating_sparse_tensor(train_ratings).dense_shape[1], embedding_dim], stddev=init_stddev))\n",
        "        \n",
        "  # Computing the mean squared error for feedback matrix approximation\n",
        "  train_loss = mse_error(build_rating_sparse_tensor(train_ratings), vectorU, vectorV)\n",
        "  test_loss = mse_error(build_rating_sparse_tensor(test_ratings), vectorU, vectorV)\n",
        "\n",
        "  # Buiding the cost function for L1 regularization. \n",
        "  feedbackLoss = train_loss + (regularization_coeff * (tf.math.reduce_sum(tf.math.abs(vectorU))/vectorU.shape[0].value + \n",
        "                                                       tf.math.reduce_sum(tf.math.abs(vectorV))/vectorV.shape[0].value))\n",
        "  loss_components = {'loss': feedbackLoss}\n",
        "\n",
        "  embeddings = {\"user_id\": vectorU, \"movie_id\": vectorV}\n",
        "\n",
        "  return Model(embeddings, feedbackLoss, [loss_components])\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "el0hCLnl1gDJ",
        "colab_type": "text"
      },
      "source": [
        "- Perform user recommendation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ec-9DVAAigCt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = build_model(ratings, embedding_dim=5, regularization_coeff=0.1, init_stddev=0.1)\n",
        "model.train(num_iterations=2000, learning_rate=1.)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ioUQ0Ydz_WyD",
        "colab_type": "code",
        "outputId": "fb2d8861-0078-4e92-ef6d-25edea253122",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 217
        }
      },
      "source": [
        "# Let's try with user id number 500 and see what the model recommends the user to watch\n",
        "\n",
        "user = 500\n",
        "print('Top 5 movie recommendation for User {}: '.format(user))\n",
        "df2, df_F = userRecommendation(model, user)"
      ],
      "execution_count": 132,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Top 5 movie recommendation for User 500: \n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>SCORE</th>\n",
              "      <th>MOVIE</th>\n",
              "      <th>TITLE</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>408</th>\n",
              "      <td>4.389266</td>\n",
              "      <td>409</td>\n",
              "      <td>Jack (1996)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>483</th>\n",
              "      <td>4.338067</td>\n",
              "      <td>484</td>\n",
              "      <td>Maltese Falcon, The (1941)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>169</th>\n",
              "      <td>4.323239</td>\n",
              "      <td>170</td>\n",
              "      <td>Cinema Paradiso (1988)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>114</th>\n",
              "      <td>4.285742</td>\n",
              "      <td>115</td>\n",
              "      <td>Haunted World of Edward D. Wood Jr., The (1995)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>318</th>\n",
              "      <td>4.284071</td>\n",
              "      <td>319</td>\n",
              "      <td>Everyone Says I Love You (1996)</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "        SCORE  MOVIE                                            TITLE\n",
              "408  4.389266    409                                      Jack (1996)\n",
              "483  4.338067    484                       Maltese Falcon, The (1941)\n",
              "169  4.323239    170                           Cinema Paradiso (1988)\n",
              "114  4.285742    115  Haunted World of Edward D. Wood Jr., The (1995)\n",
              "318  4.284071    319                  Everyone Says I Love You (1996)"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    }
  ]
}