Week 9/03 - Recommender Systems.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Recommender Systems\n",
    "\n",
    "<img width=500 src=\"https://miro.medium.com/max/1064/1*mz9tzP1LjPBhmiWXeHyQkQ.png\">\n",
    "\n",
    "\n",
    "Other Examples: \n",
    "- [Datacamp](https://www.datacamp.com/community/tutorials/recommender-systems-python?utm_source=adwords_ppc&utm_campaignid=1455363063&utm_adgroupid=65083631748&utm_device=c&utm_keyword=&utm_matchtype=b&utm_network=g&utm_adpostion=&utm_creative=278443377095&utm_targetid=dsa-473406569915&utm_loc_interest_ms=&utm_loc_physical_ms=1003659&gclid=CjwKCAjwqJ_1BRBZEiwAv73uwEfN8mroCUBwlK610d129PtFHKH_y-uxHtYyL4kK4Oc985Y7MIU5dRoCAdUQAvD_BwE)\n",
    "- [towardsdatascience 1](https://towardsdatascience.com/how-to-build-a-simple-recommender-system-in-python-375093c3fb7d)\n",
    "- [towardsdatascience 2](https://towardsdatascience.com/beginners-recommendation-systems-with-python-ee1b08d2efb6)\n",
    "- [kaggle](https://www.kaggle.com/gspmoreira/recommender-systems-in-python-101)\n",
    "- [realpython](https://realpython.com/build-recommendation-engine-collaborative-filtering/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from math import sqrt\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Advantages and Disadvantages of Collaborative Filtering\n",
    "\n",
    "##### Advantages\n",
    "* Takes other user's ratings into consideration\n",
    "* Doesn't need to study or extract information from the recommended item\n",
    "* Adapts to the user's interests which might change over time\n",
    "\n",
    "##### Disadvantages\n",
    "* Approximation function can be slow\n",
    "* There might be a low of amount of users to approximate\n",
    "* Privacy issues when trying to learn the user's preferences"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Content based"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(   movieId                               title  \\\n",
       " 0        1                    Toy Story (1995)   \n",
       " 1        2                      Jumanji (1995)   \n",
       " 2        3             Grumpier Old Men (1995)   \n",
       " 3        4            Waiting to Exhale (1995)   \n",
       " 4        5  Father of the Bride Part II (1995)   \n",
       " \n",
       "                                         genres  \n",
       " 0  Adventure|Animation|Children|Comedy|Fantasy  \n",
       " 1                   Adventure|Children|Fantasy  \n",
       " 2                               Comedy|Romance  \n",
       " 3                         Comedy|Drama|Romance  \n",
       " 4                                       Comedy  ,\n",
       "    userId  movieId  rating   timestamp\n",
       " 0       1      169     2.5  1204927694\n",
       " 1       1     2471     3.0  1204927438\n",
       " 2       1    48516     5.0  1204927435\n",
       " 3       2     2571     3.5  1436165433\n",
       " 4       2   109487     4.0  1436165496)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_df = pd.read_csv('../Datasets/Week9/moviedataset/movies.csv')\n",
    "\n",
    "files = [file for file in os.listdir('../Datasets/Week9/moviedataset/ratings/')]\n",
    "\n",
    "ratings_df = pd.DataFrame()\n",
    "\n",
    "for file in files:\n",
    "    df = pd.read_csv('../Datasets/Week9/moviedataset/ratings/'+file)\n",
    "    ratings_df = pd.concat([ratings_df,df])\n",
    "\n",
    "movies_df.head(), ratings_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((34208, 3), (22884377, 4))"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_df.shape, ratings_df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use regular expressions to find the information in parentheses\n",
    "\n",
    "And we specify the parentheses to avoid problems with movies that have years in their names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                        title  \\\n",
       "0        1                    Toy Story   \n",
       "1        2                      Jumanji   \n",
       "2        3             Grumpier Old Men   \n",
       "3        4            Waiting to Exhale   \n",
       "4        5  Father of the Bride Part II   \n",
       "\n",
       "                                        genres  year  \n",
       "0  Adventure|Animation|Children|Comedy|Fantasy  1995  \n",
       "1                   Adventure|Children|Fantasy  1995  \n",
       "2                               Comedy|Romance  1995  \n",
       "3                         Comedy|Drama|Romance  1995  \n",
       "4                                       Comedy  1995  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_df['year'] = movies_df.title.str.extract('(\\(\\d\\d\\d\\d\\))',expand=False)\n",
    "\n",
    "#Removing the parentheses\n",
    "movies_df['year'] = movies_df.year.str.extract('(\\d\\d\\d\\d)',expand=False)\n",
    "\n",
    "#Removing the years from the 'title' column\n",
    "movies_df['title'] = movies_df.title.str.replace('(\\(\\d\\d\\d\\d\\))', '')\n",
    "\n",
    "#Applying the strip function to get rid of any ending whitespace characters that may have appeared\n",
    "movies_df['title'] = movies_df['title'].apply(lambda x: x.strip())\n",
    "\n",
    "movies_df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each genre is separated by |, so to simplify we will use the split function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>[Adventure, Children, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men</td>\n",
       "      <td>[Comedy, Romance]</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>[Comedy, Drama, Romance]</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>[Comedy]</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                        title  \\\n",
       "0        1                    Toy Story   \n",
       "1        2                      Jumanji   \n",
       "2        3             Grumpier Old Men   \n",
       "3        4            Waiting to Exhale   \n",
       "4        5  Father of the Bride Part II   \n",
       "\n",
       "                                              genres  year  \n",
       "0  [Adventure, Animation, Children, Comedy, Fantasy]  1995  \n",
       "1                     [Adventure, Children, Fantasy]  1995  \n",
       "2                                  [Comedy, Romance]  1995  \n",
       "3                           [Comedy, Drama, Romance]  1995  \n",
       "4                                           [Comedy]  1995  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_df['genres'] = movies_df.genres.str.split('|')\n",
    "movies_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we make a copy of our original dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>year</th>\n",
       "      <th>Adventure</th>\n",
       "      <th>Animation</th>\n",
       "      <th>Children</th>\n",
       "      <th>Comedy</th>\n",
       "      <th>Fantasy</th>\n",
       "      <th>Romance</th>\n",
       "      <th>...</th>\n",
       "      <th>Horror</th>\n",
       "      <th>Mystery</th>\n",
       "      <th>Sci-Fi</th>\n",
       "      <th>IMAX</th>\n",
       "      <th>Documentary</th>\n",
       "      <th>War</th>\n",
       "      <th>Musical</th>\n",
       "      <th>Western</th>\n",
       "      <th>Film-Noir</th>\n",
       "      <th>(no genres listed)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>[Adventure, Children, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men</td>\n",
       "      <td>[Comedy, Romance]</td>\n",
       "      <td>1995</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>[Comedy, Drama, Romance]</td>\n",
       "      <td>1995</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>[Comedy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                        title  \\\n",
       "0        1                    Toy Story   \n",
       "1        2                      Jumanji   \n",
       "2        3             Grumpier Old Men   \n",
       "3        4            Waiting to Exhale   \n",
       "4        5  Father of the Bride Part II   \n",
       "\n",
       "                                              genres  year  Adventure  \\\n",
       "0  [Adventure, Animation, Children, Comedy, Fantasy]  1995        1.0   \n",
       "1                     [Adventure, Children, Fantasy]  1995        1.0   \n",
       "2                                  [Comedy, Romance]  1995        0.0   \n",
       "3                           [Comedy, Drama, Romance]  1995        0.0   \n",
       "4                                           [Comedy]  1995        0.0   \n",
       "\n",
       "   Animation  Children  Comedy  Fantasy  Romance  ...  Horror  Mystery  \\\n",
       "0        1.0       1.0     1.0      1.0      0.0  ...     0.0      0.0   \n",
       "1        0.0       1.0     0.0      1.0      0.0  ...     0.0      0.0   \n",
       "2        0.0       0.0     1.0      0.0      1.0  ...     0.0      0.0   \n",
       "3        0.0       0.0     1.0      0.0      1.0  ...     0.0      0.0   \n",
       "4        0.0       0.0     1.0      0.0      0.0  ...     0.0      0.0   \n",
       "\n",
       "   Sci-Fi  IMAX  Documentary  War  Musical  Western  Film-Noir  \\\n",
       "0     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n",
       "1     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n",
       "2     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n",
       "3     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n",
       "4     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n",
       "\n",
       "   (no genres listed)  \n",
       "0                 0.0  \n",
       "1                 0.0  \n",
       "2                 0.0  \n",
       "3                 0.0  \n",
       "4                 0.0  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "moviesWithGenres_df = movies_df.copy()\n",
    "\n",
    "# we assign a 1 if the movie has the genre and a 0 if not (to avoid NaN values)\n",
    "for index, row in movies_df.iterrows():\n",
    "    for genre in row['genres']:\n",
    "        moviesWithGenres_df.at[index, genre] = 1\n",
    "\n",
    "moviesWithGenres_df = moviesWithGenres_df.fillna(0)\n",
    "moviesWithGenres_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>169</td>\n",
       "      <td>2.5</td>\n",
       "      <td>1204927694</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2471</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1204927438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>48516</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1204927435</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2571</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1436165433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>109487</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1436165496</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userId  movieId  rating   timestamp\n",
       "0       1      169     2.5  1204927694\n",
       "1       1     2471     3.0  1204927438\n",
       "2       1    48516     5.0  1204927435\n",
       "3       2     2571     3.5  1436165433\n",
       "4       2   109487     4.0  1436165496"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>169</td>\n",
       "      <td>2.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2471</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>48516</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2571</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>109487</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userId  movieId  rating\n",
       "0       1      169     2.5\n",
       "1       1     2471     3.0\n",
       "2       1    48516     5.0\n",
       "3       2     2571     3.5\n",
       "4       2   109487     4.0"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we delete the columns we don't need\n",
    "\n",
    "ratings_df = ratings_df.drop('timestamp', 1)\n",
    "ratings_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we are going to implement a __Content-Based__ or __Item-Item recommendation systems__. With this we seek to discover what are the important and / or favorite aspects of a user, and after that it recommends elements that have a certain similarity with those aspects. In this case, we are going to try to find out the favorite genres of the movie entry and ratings given.\n",
    "\n",
    "We created a login user to recommend movies for:\n",
    "\n",
    "Notice: To add more movies add the items in __userInput__. Just be sure to write it in Capital Case format and if a movie starts with \"The\" like \"The Nightmare Before Christmas\" write it like this: 'Nightmare Before Christmas, The'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>Breakfast Club, The</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>Pulp Fiction</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>Akira</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>Blade Runner</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 title  rating\n",
       "0  Breakfast Club, The     5.0\n",
       "1            Toy Story     3.5\n",
       "2              Jumanji     2.0\n",
       "3         Pulp Fiction     5.0\n",
       "4                Akira     4.5\n",
       "5         Blade Runner     5.0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "userInput = [\n",
    "            {'title':'Breakfast Club, The', 'rating':5},\n",
    "            {'title':'Toy Story', 'rating':3.5},\n",
    "            {'title':'Jumanji', 'rating':2},\n",
    "            {'title':\"Pulp Fiction\", 'rating':5},\n",
    "            {'title':'Akira', 'rating':4.5},\n",
    "            {'title':'Blade Runner', 'rating':5.0}\n",
    "         ] \n",
    "inputMovies = pd.DataFrame(userInput)\n",
    "inputMovies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We filter the movies by title\n",
    "\n",
    "Then merging it so we can get the movieId. It's implicitly merging it by title.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>296</td>\n",
       "      <td>Pulp Fiction</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>541</td>\n",
       "      <td>Blade Runner</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>1274</td>\n",
       "      <td>Akira</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>1968</td>\n",
       "      <td>Breakfast Club, The</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                title  rating\n",
       "0        1            Toy Story     3.5\n",
       "1        2              Jumanji     2.0\n",
       "2      296         Pulp Fiction     5.0\n",
       "3      541         Blade Runner     5.0\n",
       "4     1274                Akira     4.5\n",
       "5     1968  Breakfast Club, The     5.0"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "inputId = movies_df[movies_df['title'].isin(inputMovies['title'].tolist())]\n",
    "\n",
    "inputMovies = pd.merge(inputId, inputMovies)\n",
    "# Remove information we won't use \n",
    "inputMovies = inputMovies.drop('genres', 1).drop('year', 1)\n",
    "#Final input dataframe\n",
    "#If a movie you added in above isn't here, then it might not be in the original \n",
    "#dataframe or it might spelled differently, please check capitalisation.\n",
    "inputMovies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>year</th>\n",
       "      <th>Adventure</th>\n",
       "      <th>Animation</th>\n",
       "      <th>Children</th>\n",
       "      <th>Comedy</th>\n",
       "      <th>Fantasy</th>\n",
       "      <th>Romance</th>\n",
       "      <th>...</th>\n",
       "      <th>Horror</th>\n",
       "      <th>Mystery</th>\n",
       "      <th>Sci-Fi</th>\n",
       "      <th>IMAX</th>\n",
       "      <th>Documentary</th>\n",
       "      <th>War</th>\n",
       "      <th>Musical</th>\n",
       "      <th>Western</th>\n",
       "      <th>Film-Noir</th>\n",
       "      <th>(no genres listed)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>[Adventure, Children, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>293</td>\n",
       "      <td>296</td>\n",
       "      <td>Pulp Fiction</td>\n",
       "      <td>[Comedy, Crime, Drama, Thriller]</td>\n",
       "      <td>1994</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>537</td>\n",
       "      <td>541</td>\n",
       "      <td>Blade Runner</td>\n",
       "      <td>[Action, Sci-Fi, Thriller]</td>\n",
       "      <td>1982</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1246</td>\n",
       "      <td>1274</td>\n",
       "      <td>Akira</td>\n",
       "      <td>[Action, Adventure, Animation, Sci-Fi]</td>\n",
       "      <td>1988</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1885</td>\n",
       "      <td>1968</td>\n",
       "      <td>Breakfast Club, The</td>\n",
       "      <td>[Comedy, Drama]</td>\n",
       "      <td>1985</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>6 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      movieId                title  \\\n",
       "0           1            Toy Story   \n",
       "1           2              Jumanji   \n",
       "293       296         Pulp Fiction   \n",
       "537       541         Blade Runner   \n",
       "1246     1274                Akira   \n",
       "1885     1968  Breakfast Club, The   \n",
       "\n",
       "                                                 genres  year  Adventure  \\\n",
       "0     [Adventure, Animation, Children, Comedy, Fantasy]  1995        1.0   \n",
       "1                        [Adventure, Children, Fantasy]  1995        1.0   \n",
       "293                    [Comedy, Crime, Drama, Thriller]  1994        0.0   \n",
       "537                          [Action, Sci-Fi, Thriller]  1982        0.0   \n",
       "1246             [Action, Adventure, Animation, Sci-Fi]  1988        1.0   \n",
       "1885                                    [Comedy, Drama]  1985        0.0   \n",
       "\n",
       "      Animation  Children  Comedy  Fantasy  Romance  ...  Horror  Mystery  \\\n",
       "0           1.0       1.0     1.0      1.0      0.0  ...     0.0      0.0   \n",
       "1           0.0       1.0     0.0      1.0      0.0  ...     0.0      0.0   \n",
       "293         0.0       0.0     1.0      0.0      0.0  ...     0.0      0.0   \n",
       "537         0.0       0.0     0.0      0.0      0.0  ...     0.0      0.0   \n",
       "1246        1.0       0.0     0.0      0.0      0.0  ...     0.0      0.0   \n",
       "1885        0.0       0.0     1.0      0.0      0.0  ...     0.0      0.0   \n",
       "\n",
       "      Sci-Fi  IMAX  Documentary  War  Musical  Western  Film-Noir  \\\n",
       "0        0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n",
       "1        0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n",
       "293      0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n",
       "537      1.0   0.0          0.0  0.0      0.0      0.0        0.0   \n",
       "1246     1.0   0.0          0.0  0.0      0.0      0.0        0.0   \n",
       "1885     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n",
       "\n",
       "      (no genres listed)  \n",
       "0                    0.0  \n",
       "1                    0.0  \n",
       "293                  0.0  \n",
       "537                  0.0  \n",
       "1246                 0.0  \n",
       "1885                 0.0  \n",
       "\n",
       "[6 rows x 24 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Filtering out the movies from the input\n",
    "userMovies = moviesWithGenres_df[moviesWithGenres_df['movieId'].isin(inputMovies['movieId'].tolist())]\n",
    "userMovies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Adventure</th>\n",
       "      <th>Animation</th>\n",
       "      <th>Children</th>\n",
       "      <th>Comedy</th>\n",
       "      <th>Fantasy</th>\n",
       "      <th>Romance</th>\n",
       "      <th>Drama</th>\n",
       "      <th>Action</th>\n",
       "      <th>Crime</th>\n",
       "      <th>Thriller</th>\n",
       "      <th>Horror</th>\n",
       "      <th>Mystery</th>\n",
       "      <th>Sci-Fi</th>\n",
       "      <th>IMAX</th>\n",
       "      <th>Documentary</th>\n",
       "      <th>War</th>\n",
       "      <th>Musical</th>\n",
       "      <th>Western</th>\n",
       "      <th>Film-Noir</th>\n",
       "      <th>(no genres listed)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Adventure  Animation  Children  Comedy  Fantasy  Romance  Drama  Action  \\\n",
       "0        1.0        1.0       1.0     1.0      1.0      0.0    0.0     0.0   \n",
       "1        1.0        0.0       1.0     0.0      1.0      0.0    0.0     0.0   \n",
       "2        0.0        0.0       0.0     1.0      0.0      0.0    1.0     0.0   \n",
       "3        0.0        0.0       0.0     0.0      0.0      0.0    0.0     1.0   \n",
       "4        1.0        1.0       0.0     0.0      0.0      0.0    0.0     1.0   \n",
       "5        0.0        0.0       0.0     1.0      0.0      0.0    1.0     0.0   \n",
       "\n",
       "   Crime  Thriller  Horror  Mystery  Sci-Fi  IMAX  Documentary  War  Musical  \\\n",
       "0    0.0       0.0     0.0      0.0     0.0   0.0          0.0  0.0      0.0   \n",
       "1    0.0       0.0     0.0      0.0     0.0   0.0          0.0  0.0      0.0   \n",
       "2    1.0       1.0     0.0      0.0     0.0   0.0          0.0  0.0      0.0   \n",
       "3    0.0       1.0     0.0      0.0     1.0   0.0          0.0  0.0      0.0   \n",
       "4    0.0       0.0     0.0      0.0     1.0   0.0          0.0  0.0      0.0   \n",
       "5    0.0       0.0     0.0      0.0     0.0   0.0          0.0  0.0      0.0   \n",
       "\n",
       "   Western  Film-Noir  (no genres listed)  \n",
       "0      0.0        0.0                 0.0  \n",
       "1      0.0        0.0                 0.0  \n",
       "2      0.0        0.0                 0.0  \n",
       "3      0.0        0.0                 0.0  \n",
       "4      0.0        0.0                 0.0  \n",
       "5      0.0        0.0                 0.0  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Resetting the index to avoid future issues\n",
    "userMovies = userMovies.reset_index(drop=True)\n",
    "# Remove unnecessary issues to save memory and avoid future issues\n",
    "userGenreTable = userMovies.drop('movieId', 1).drop('title', 1).drop('genres', 1).drop('year', 1)\n",
    "userGenreTable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    3.5\n",
       "1    2.0\n",
       "2    5.0\n",
       "3    5.0\n",
       "4    4.5\n",
       "5    5.0\n",
       "Name: rating, dtype: float64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inputMovies['rating']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Adventure             10.0\n",
       "Animation              8.0\n",
       "Children               5.5\n",
       "Comedy                13.5\n",
       "Fantasy                5.5\n",
       "Romance                0.0\n",
       "Drama                 10.0\n",
       "Action                 9.5\n",
       "Crime                  5.0\n",
       "Thriller              10.0\n",
       "Horror                 0.0\n",
       "Mystery                0.0\n",
       "Sci-Fi                 9.5\n",
       "IMAX                   0.0\n",
       "Documentary            0.0\n",
       "War                    0.0\n",
       "Musical                0.0\n",
       "Western                0.0\n",
       "Film-Noir              0.0\n",
       "(no genres listed)     0.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Dot produt to get weights\n",
    "userProfile = userGenreTable.transpose().dot(inputMovies['rating'])\n",
    "#The user profile\n",
    "userProfile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Adventure</th>\n",
       "      <th>Animation</th>\n",
       "      <th>Children</th>\n",
       "      <th>Comedy</th>\n",
       "      <th>Fantasy</th>\n",
       "      <th>Romance</th>\n",
       "      <th>Drama</th>\n",
       "      <th>Action</th>\n",
       "      <th>Crime</th>\n",
       "      <th>Thriller</th>\n",
       "      <th>Horror</th>\n",
       "      <th>Mystery</th>\n",
       "      <th>Sci-Fi</th>\n",
       "      <th>IMAX</th>\n",
       "      <th>Documentary</th>\n",
       "      <th>War</th>\n",
       "      <th>Musical</th>\n",
       "      <th>Western</th>\n",
       "      <th>Film-Noir</th>\n",
       "      <th>(no genres listed)</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>movieId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Adventure  Animation  Children  Comedy  Fantasy  Romance  Drama  \\\n",
       "movieId                                                                    \n",
       "1              1.0        1.0       1.0     1.0      1.0      0.0    0.0   \n",
       "2              1.0        0.0       1.0     0.0      1.0      0.0    0.0   \n",
       "3              0.0        0.0       0.0     1.0      0.0      1.0    0.0   \n",
       "4              0.0        0.0       0.0     1.0      0.0      1.0    1.0   \n",
       "5              0.0        0.0       0.0     1.0      0.0      0.0    0.0   \n",
       "\n",
       "         Action  Crime  Thriller  Horror  Mystery  Sci-Fi  IMAX  Documentary  \\\n",
       "movieId                                                                        \n",
       "1           0.0    0.0       0.0     0.0      0.0     0.0   0.0          0.0   \n",
       "2           0.0    0.0       0.0     0.0      0.0     0.0   0.0          0.0   \n",
       "3           0.0    0.0       0.0     0.0      0.0     0.0   0.0          0.0   \n",
       "4           0.0    0.0       0.0     0.0      0.0     0.0   0.0          0.0   \n",
       "5           0.0    0.0       0.0     0.0      0.0     0.0   0.0          0.0   \n",
       "\n",
       "         War  Musical  Western  Film-Noir  (no genres listed)  \n",
       "movieId                                                        \n",
       "1        0.0      0.0      0.0        0.0                 0.0  \n",
       "2        0.0      0.0      0.0        0.0                 0.0  \n",
       "3        0.0      0.0      0.0        0.0                 0.0  \n",
       "4        0.0      0.0      0.0        0.0                 0.0  \n",
       "5        0.0      0.0      0.0        0.0                 0.0  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Now let's get the genres of every movie in our original dataframe\n",
    "genreTable = moviesWithGenres_df.set_index(moviesWithGenres_df['movieId'])\n",
    "#And remove unnecessary information\n",
    "genreTable = genreTable.drop('movieId', 1).drop('title', 1).drop('genres', 1).drop('year', 1)\n",
    "genreTable.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(34208, 20)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "genreTable.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "movieId\n",
       "1    0.491329\n",
       "2    0.242775\n",
       "3    0.156069\n",
       "4    0.271676\n",
       "5    0.156069\n",
       "dtype: float64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Multiply the genres by the weights and then take the weighted average\n",
    "recommendationTable_df = ((genreTable*userProfile).sum(axis=1))/(userProfile.sum())\n",
    "recommendationTable_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "movieId\n",
       "5018      0.734104\n",
       "6902      0.676301\n",
       "117646    0.676301\n",
       "64645     0.670520\n",
       "122787    0.670520\n",
       "dtype: float64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Sort our recommendations in descending order\n",
    "recommendationTable_df = recommendationTable_df.sort_values(ascending=False)\n",
    "#Just a peek at the values\n",
    "recommendationTable_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>4625</td>\n",
       "      <td>4719</td>\n",
       "      <td>Osmosis Jones</td>\n",
       "      <td>[Action, Animation, Comedy, Crime, Drama, Roma...</td>\n",
       "      <td>2001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4923</td>\n",
       "      <td>5018</td>\n",
       "      <td>Motorama</td>\n",
       "      <td>[Adventure, Comedy, Crime, Drama, Fantasy, Mys...</td>\n",
       "      <td>1991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6793</td>\n",
       "      <td>6902</td>\n",
       "      <td>Interstate 60</td>\n",
       "      <td>[Adventure, Comedy, Drama, Fantasy, Mystery, S...</td>\n",
       "      <td>2002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9296</td>\n",
       "      <td>27344</td>\n",
       "      <td>Revolutionary Girl Utena: Adolescence of Utena...</td>\n",
       "      <td>[Action, Adventure, Animation, Comedy, Drama, ...</td>\n",
       "      <td>1999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10575</td>\n",
       "      <td>40339</td>\n",
       "      <td>Chicken Little</td>\n",
       "      <td>[Action, Adventure, Animation, Children, Comed...</td>\n",
       "      <td>2005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11785</td>\n",
       "      <td>52287</td>\n",
       "      <td>Meet the Robinsons</td>\n",
       "      <td>[Action, Adventure, Animation, Children, Comed...</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11806</td>\n",
       "      <td>52462</td>\n",
       "      <td>Aqua Teen Hunger Force Colon Movie Film for Th...</td>\n",
       "      <td>[Action, Adventure, Animation, Comedy, Fantasy...</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13109</td>\n",
       "      <td>62956</td>\n",
       "      <td>Futurama: Bender's Game</td>\n",
       "      <td>[Action, Adventure, Animation, Comedy, Fantasy...</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13250</td>\n",
       "      <td>64645</td>\n",
       "      <td>The Wrecking Crew</td>\n",
       "      <td>[Action, Adventure, Comedy, Crime, Drama, Thri...</td>\n",
       "      <td>1968</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>14397</td>\n",
       "      <td>71999</td>\n",
       "      <td>Aelita: The Queen of Mars (Aelita)</td>\n",
       "      <td>[Action, Adventure, Drama, Fantasy, Romance, S...</td>\n",
       "      <td>1924</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15001</td>\n",
       "      <td>75408</td>\n",
       "      <td>Lupin III: Sweet Lost Night (Rupan Sansei: Swe...</td>\n",
       "      <td>[Action, Animation, Comedy, Crime, Drama, Myst...</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15073</td>\n",
       "      <td>76153</td>\n",
       "      <td>Lupin III: First Contact (Rupan Sansei: Faasut...</td>\n",
       "      <td>[Action, Animation, Comedy, Crime, Drama, Myst...</td>\n",
       "      <td>2002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16055</td>\n",
       "      <td>81132</td>\n",
       "      <td>Rubber</td>\n",
       "      <td>[Action, Adventure, Comedy, Crime, Drama, Film...</td>\n",
       "      <td>2010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16884</td>\n",
       "      <td>85261</td>\n",
       "      <td>Mars Needs Moms</td>\n",
       "      <td>[Action, Adventure, Animation, Children, Comed...</td>\n",
       "      <td>2011</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>22145</td>\n",
       "      <td>106240</td>\n",
       "      <td>Free Birds</td>\n",
       "      <td>[Action, Adventure, Animation, Children, Comed...</td>\n",
       "      <td>2013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24565</td>\n",
       "      <td>115479</td>\n",
       "      <td>Whip Hand, The</td>\n",
       "      <td>[Action, Adventure, Crime, Drama, Sci-Fi, Thri...</td>\n",
       "      <td>1951</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25218</td>\n",
       "      <td>117646</td>\n",
       "      <td>Dragonheart 2: A New Beginning</td>\n",
       "      <td>[Action, Adventure, Comedy, Drama, Fantasy, Th...</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26442</td>\n",
       "      <td>122787</td>\n",
       "      <td>The 39 Steps</td>\n",
       "      <td>[Action, Adventure, Comedy, Crime, Drama, Thri...</td>\n",
       "      <td>1959</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33509</td>\n",
       "      <td>148775</td>\n",
       "      <td>Wizards of Waverly Place: The Movie</td>\n",
       "      <td>[Adventure, Children, Comedy, Drama, Fantasy, ...</td>\n",
       "      <td>2009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33692</td>\n",
       "      <td>149488</td>\n",
       "      <td>Christmas Town</td>\n",
       "      <td>[Action, Children, Comedy, Drama, Fantasy, Thr...</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       movieId                                              title  \\\n",
       "4625      4719                                      Osmosis Jones   \n",
       "4923      5018                                           Motorama   \n",
       "6793      6902                                      Interstate 60   \n",
       "9296     27344  Revolutionary Girl Utena: Adolescence of Utena...   \n",
       "10575    40339                                     Chicken Little   \n",
       "11785    52287                                 Meet the Robinsons   \n",
       "11806    52462  Aqua Teen Hunger Force Colon Movie Film for Th...   \n",
       "13109    62956                            Futurama: Bender's Game   \n",
       "13250    64645                                  The Wrecking Crew   \n",
       "14397    71999                 Aelita: The Queen of Mars (Aelita)   \n",
       "15001    75408  Lupin III: Sweet Lost Night (Rupan Sansei: Swe...   \n",
       "15073    76153  Lupin III: First Contact (Rupan Sansei: Faasut...   \n",
       "16055    81132                                             Rubber   \n",
       "16884    85261                                    Mars Needs Moms   \n",
       "22145   106240                                         Free Birds   \n",
       "24565   115479                                     Whip Hand, The   \n",
       "25218   117646                     Dragonheart 2: A New Beginning   \n",
       "26442   122787                                       The 39 Steps   \n",
       "33509   148775                Wizards of Waverly Place: The Movie   \n",
       "33692   149488                                     Christmas Town   \n",
       "\n",
       "                                                  genres  year  \n",
       "4625   [Action, Animation, Comedy, Crime, Drama, Roma...  2001  \n",
       "4923   [Adventure, Comedy, Crime, Drama, Fantasy, Mys...  1991  \n",
       "6793   [Adventure, Comedy, Drama, Fantasy, Mystery, S...  2002  \n",
       "9296   [Action, Adventure, Animation, Comedy, Drama, ...  1999  \n",
       "10575  [Action, Adventure, Animation, Children, Comed...  2005  \n",
       "11785  [Action, Adventure, Animation, Children, Comed...  2007  \n",
       "11806  [Action, Adventure, Animation, Comedy, Fantasy...  2007  \n",
       "13109  [Action, Adventure, Animation, Comedy, Fantasy...  2008  \n",
       "13250  [Action, Adventure, Comedy, Crime, Drama, Thri...  1968  \n",
       "14397  [Action, Adventure, Drama, Fantasy, Romance, S...  1924  \n",
       "15001  [Action, Animation, Comedy, Crime, Drama, Myst...  2008  \n",
       "15073  [Action, Animation, Comedy, Crime, Drama, Myst...  2002  \n",
       "16055  [Action, Adventure, Comedy, Crime, Drama, Film...  2010  \n",
       "16884  [Action, Adventure, Animation, Children, Comed...  2011  \n",
       "22145  [Action, Adventure, Animation, Children, Comed...  2013  \n",
       "24565  [Action, Adventure, Crime, Drama, Sci-Fi, Thri...  1951  \n",
       "25218  [Action, Adventure, Comedy, Drama, Fantasy, Th...  2000  \n",
       "26442  [Action, Adventure, Comedy, Crime, Drama, Thri...  1959  \n",
       "33509  [Adventure, Children, Comedy, Drama, Fantasy, ...  2009  \n",
       "33692  [Action, Children, Comedy, Drama, Fantasy, Thr...  2008  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#The final recommendation table\n",
    "movies_df.loc[movies_df['movieId'].isin(recommendationTable_df.head(20).keys())]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Collaborative Filtering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "movies_df = pd.read_csv('../Datasets/Week9/moviedataset/movies.csv')\n",
    "\n",
    "files = [file for file in os.listdir('../Datasets/Week9/moviedataset/ratings/')]\n",
    "\n",
    "ratings_df = pd.DataFrame()\n",
    "\n",
    "for file in files:\n",
    "    df = pd.read_csv('../Datasets/Week9/moviedataset/ratings/'+file)\n",
    "    ratings_df = pd.concat([ratings_df,df])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                               title  \\\n",
       "0        1                    Toy Story (1995)   \n",
       "1        2                      Jumanji (1995)   \n",
       "2        3             Grumpier Old Men (1995)   \n",
       "3        4            Waiting to Exhale (1995)   \n",
       "4        5  Father of the Bride Part II (1995)   \n",
       "\n",
       "                                        genres  \n",
       "0  Adventure|Animation|Children|Comedy|Fantasy  \n",
       "1                   Adventure|Children|Fantasy  \n",
       "2                               Comedy|Romance  \n",
       "3                         Comedy|Drama|Romance  \n",
       "4                                       Comedy  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Head is a function that gets the first N rows of a dataframe. N's default is 5.\n",
    "movies_df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use regular expressions to find the information in parentheses\n",
    "\n",
    "And we specify the parentheses to avoid problems with movies that have years in their names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "movies_df['year'] = movies_df.title.str.extract('(\\(\\d\\d\\d\\d\\))',expand=False)\n",
    "\n",
    "movies_df['year'] = movies_df.year.str.extract('(\\d\\d\\d\\d)',expand=False)\n",
    "\n",
    "movies_df['title'] = movies_df.title.str.replace('(\\(\\d\\d\\d\\d\\))', '')\n",
    "\n",
    "movies_df['title'] = movies_df['title'].apply(lambda x: x.strip())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                        title  \\\n",
       "0        1                    Toy Story   \n",
       "1        2                      Jumanji   \n",
       "2        3             Grumpier Old Men   \n",
       "3        4            Waiting to Exhale   \n",
       "4        5  Father of the Bride Part II   \n",
       "\n",
       "                                        genres  year  \n",
       "0  Adventure|Animation|Children|Comedy|Fantasy  1995  \n",
       "1                   Adventure|Children|Fantasy  1995  \n",
       "2                               Comedy|Romance  1995  \n",
       "3                         Comedy|Drama|Romance  1995  \n",
       "4                                       Comedy  1995  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# remove the genres column\n",
    "movies_df = movies_df.drop('genres', 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                        title  year\n",
       "0        1                    Toy Story  1995\n",
       "1        2                      Jumanji  1995\n",
       "2        3             Grumpier Old Men  1995\n",
       "3        4            Waiting to Exhale  1995\n",
       "4        5  Father of the Bride Part II  1995"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>169</td>\n",
       "      <td>2.5</td>\n",
       "      <td>1204927694</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2471</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1204927438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>48516</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1204927435</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2571</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1436165433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>109487</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1436165496</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userId  movieId  rating   timestamp\n",
       "0       1      169     2.5  1204927694\n",
       "1       1     2471     3.0  1204927438\n",
       "2       1    48516     5.0  1204927435\n",
       "3       2     2571     3.5  1436165433\n",
       "4       2   109487     4.0  1436165496"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Drop removes a specified row or column from a dataframe\n",
    "ratings_df = ratings_df.drop('timestamp', 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>169</td>\n",
       "      <td>2.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2471</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>48516</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2571</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>109487</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userId  movieId  rating\n",
       "0       1      169     2.5\n",
       "1       1     2471     3.0\n",
       "2       1    48516     5.0\n",
       "3       2     2571     3.5\n",
       "4       2   109487     4.0"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, it is time to start our work on recommendation systems.\n",
    "\n",
    "The first technique is called __Colaborative Filtering__, which is also called by __User-User Filtering__. As hinted at by its alternate name, this technique uses other users to recommend items to the input user. Try to find users who have similar preferences and opinions to the post, and then recommend items they liked to the post. There are several methods to find similar users (including some that use Machine Learning), and the one we will use here will be based on the __Pearson Correlation Function__.\n",
    "\n",
    "\n",
    "The process or step by step to create a user-based recommendation system is as follows:\n",
    "- Select a user with the movies that the user has seen\n",
    "- Based on your movie rating, find the best neighbors X\n",
    "- Obtain the user view movie record for each neighbor.\n",
    "- Calculate a similarity score using some formula\n",
    "- Recommend the articles with the highest score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>Breakfast Club, The</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>Pulp Fiction</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>Akira</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>Blade Runner</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 title  rating\n",
       "0  Breakfast Club, The     5.0\n",
       "1            Toy Story     3.5\n",
       "2              Jumanji     2.0\n",
       "3         Pulp Fiction     5.0\n",
       "4                Akira     4.5\n",
       "5         Blade Runner     5.0"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "userInput = [\n",
    "            {'title':'Breakfast Club, The', 'rating':5},\n",
    "            {'title':'Toy Story', 'rating':3.5},\n",
    "            {'title':'Jumanji', 'rating':2},\n",
    "            {'title':\"Pulp Fiction\", 'rating':5},\n",
    "            {'title':'Akira', 'rating':4.5},\n",
    "            {'title':'Blade Runner', 'rating':5.0}\n",
    "         ] \n",
    "inputMovies = pd.DataFrame(userInput)\n",
    "inputMovies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>296</td>\n",
       "      <td>Pulp Fiction</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>541</td>\n",
       "      <td>Blade Runner</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>1274</td>\n",
       "      <td>Akira</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>1968</td>\n",
       "      <td>Breakfast Club, The</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                title  rating\n",
       "0        1            Toy Story     3.5\n",
       "1        2              Jumanji     2.0\n",
       "2      296         Pulp Fiction     5.0\n",
       "3      541         Blade Runner     5.0\n",
       "4     1274                Akira     4.5\n",
       "5     1968  Breakfast Club, The     5.0"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Filtering out the movies by title\n",
    "inputId = movies_df[movies_df['title'].isin(inputMovies['title'].tolist())]\n",
    "\n",
    "inputMovies = pd.merge(inputId, inputMovies)\n",
    "\n",
    "inputMovies = inputMovies.drop('year', 1)\n",
    "\n",
    "inputMovies\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>19</td>\n",
       "      <td>4</td>\n",
       "      <td>296</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>399</td>\n",
       "      <td>12</td>\n",
       "      <td>541</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>441</td>\n",
       "      <td>12</td>\n",
       "      <td>1968</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>479</td>\n",
       "      <td>13</td>\n",
       "      <td>2</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>531</td>\n",
       "      <td>13</td>\n",
       "      <td>1274</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     userId  movieId  rating\n",
       "19        4      296     4.0\n",
       "399      12      541     4.0\n",
       "441      12     1968     3.0\n",
       "479      13        2     2.0\n",
       "531      13     1274     5.0"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Filtering out users that have watched movies that the input has watched and storing it\n",
    "userSubset = ratings_df[ratings_df['movieId'].isin(inputMovies['movieId'].tolist())]\n",
    "userSubset.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Groupby creates several sub dataframes where they all have the same value in the column specified as the parameter\n",
    "userSubsetGroup = userSubset.groupby(['userId'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>104167</td>\n",
       "      <td>1130</td>\n",
       "      <td>1</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>104168</td>\n",
       "      <td>1130</td>\n",
       "      <td>2</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>104214</td>\n",
       "      <td>1130</td>\n",
       "      <td>296</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>104247</td>\n",
       "      <td>1130</td>\n",
       "      <td>541</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>104363</td>\n",
       "      <td>1130</td>\n",
       "      <td>1274</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>104443</td>\n",
       "      <td>1130</td>\n",
       "      <td>1968</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        userId  movieId  rating\n",
       "104167    1130        1     0.5\n",
       "104168    1130        2     4.0\n",
       "104214    1130      296     4.0\n",
       "104247    1130      541     4.5\n",
       "104363    1130     1274     4.5\n",
       "104443    1130     1968     4.5"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "userSubsetGroup.get_group(1130)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Sorting it so users with movie most in common with the input will have priority\n",
    "userSubsetGroup = sorted(userSubsetGroup,  key=lambda x: len(x[1]), reverse=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(106,       userId  movieId  rating\n",
       "  9083     106        1     2.5\n",
       "  9084     106        2     3.0\n",
       "  9115     106      296     3.5\n",
       "  9135     106      541     3.0\n",
       "  9198     106     1274     3.0\n",
       "  9238     106     1968     3.5), (686,        userId  movieId  rating\n",
       "  61336     686        1     4.0\n",
       "  61337     686        2     3.0\n",
       "  61377     686      296     4.0\n",
       "  61411     686      541     3.0\n",
       "  61478     686     1274     4.0\n",
       "  61569     686     1968     5.0), (815,        userId  movieId  rating\n",
       "  73747     815        1     4.5\n",
       "  73748     815        2     3.0\n",
       "  73922     815      296     5.0\n",
       "  74070     815      541     3.0\n",
       "  74362     815     1274     3.0\n",
       "  74678     815     1968     4.5)]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "userSubsetGroup[0:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Similarity of users to input user\n",
    "Next, we are going to compare all users (not really all !!!) to our specified user and find the one that is most similar.  \n",
    "we're going to find out how similar each user is to the input through the __Pearson Correlation Coefficient__. It is used to measure the strength of a linear association between two variables. The formula for finding this coefficient between sets X and Y with N values can be seen in the image below. \n",
    "\n",
    "Why Pearson Correlation?\n",
    "\n",
    "Pearson correlation is invariant to scaling, i.e. multiplying all elements by a nonzero constant or adding any constant to all elements. For example, if you have two vectors X and Y,then, pearson(X, Y) == pearson(X, 2 * Y + 3). This is a pretty important property in recommendation systems because for example two users might rate two series of items totally different in terms of absolute rates, but they would be similar users (i.e. with similar ideas) with similar rates in various scales .\n",
    "\n",
    "![alt text](https://wikimedia.org/api/rest_v1/media/math/render/svg/bd1ccc2979b0fd1c1aec96e386f686ae874f9ec0 \"Pearson Correlation\")\n",
    "\n",
    "The values given by the formula vary from r = -1 to r = 1, where 1 forms a direct correlation between the two entities (it means a perfect positive correlation) and -1 forms a perfect negative correlation. \n",
    "\n",
    "In our case, a 1 means that the two users have similar tastes while a -1 means the opposite."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "userSubsetGroup = userSubsetGroup[0:100]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Store the Pearson Correlation in a dictionary, where the key is the user Id and the value is the coefficient\n",
    "pearsonCorrelationDict = {}\n",
    "\n",
    "#For every user group in our subset\n",
    "for name, group in userSubsetGroup:\n",
    "    #Let's start by sorting the input and current user group so the values aren't mixed up later on\n",
    "    group = group.sort_values(by='movieId')\n",
    "    inputMovies = inputMovies.sort_values(by='movieId')\n",
    "    #Get the N for the formula\n",
    "    nRatings = len(group)\n",
    "    #Get the review scores for the movies that they both have in common\n",
    "    temp_df = inputMovies[inputMovies['movieId'].isin(group['movieId'].tolist())]\n",
    "    #And then store them in a temporary buffer variable in a list format to facilitate future calculations\n",
    "    tempRatingList = temp_df['rating'].tolist()\n",
    "    #Let's also put the current user group reviews in a list format\n",
    "    tempGroupList = group['rating'].tolist()\n",
    "    #Now let's calculate the pearson correlation between two users, so called, x and y\n",
    "    Sxx = sum([i**2 for i in tempRatingList]) - pow(sum(tempRatingList),2)/float(nRatings)\n",
    "    Syy = sum([i**2 for i in tempGroupList]) - pow(sum(tempGroupList),2)/float(nRatings)\n",
    "    Sxy = sum( i*j for i, j in zip(tempRatingList, tempGroupList)) - sum(tempRatingList)*sum(tempGroupList)/float(nRatings)\n",
    "    \n",
    "    #If the denominator is different than zero, then divide, else, 0 correlation.\n",
    "    if Sxx != 0 and Syy != 0:\n",
    "        pearsonCorrelationDict[name] = Sxy/sqrt(Sxx*Syy)\n",
    "    else:\n",
    "        pearsonCorrelationDict[name] = 0\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_items([(106, 0.5118906968889928), (686, 0.47532707568263655), (815, 0.34000462769372364), (1130, 0.35049981861639046), (1502, 0.8636363636363663), (1599, 0.5330017908890288), (1625, 0.7462025072446367), (1950, 0.05330017908890424), (2065, 0.5330017908890288), (2432, 0.34645116407786797), (2791, 0.876459821202215), (2839, 0.7312724241271304), (2948, 0.15745916432444343), (3025, 0.4999999999999995), (3040, 0.9061030445113399), (3186, 0.7018644002680551), (3271, 0.28653494154687914), (3429, -0.10660035817780393), (3734, -0.16480856327180468), (4208, 0.3450172512938587), (4282, -0.3370999312316288), (4292, 0.703123049310654), (4415, -0.08762495465409724), (4586, -0.7272727272727297), (4818, 0.5310665934349466), (5104, 0.746128578230016), (5165, -0.47193990372427286), (5547, 0.3060730460762064), (6082, 0.06998656386948264), (6366, 0.674199862463241), (6482, -0.03656362120635603), (6530, -0.16854996561581298), (7235, 0.7286524160629291), (7403, -0.09231861823449959), (7641, 0.7462025072446379), (7996, 0.6158817620514411), (8008, -0.38766492091636556), (8086, 0.6662522386112839), (8245, 0.02799462554779351), (8675, 0.6030226891555274), (9101, 0.09417632186960104), (9358, 0.7335514572338323), (9663, 0.2991830368027056), (9994, 0.5550277520814231), (10315, 0.5863019699779285), (10368, 0.5237849266164991), (10607, 0.511890696888991), (10707, 0.9570136726330055), (10863, 0.6590909090909114), (11314, 0.8043996665398446), (11399, 0.5850179393017063), (11769, 0.9255913610168006), (11827, 0.5598925109558536), (12069, 0.15075567228888184), (12325, 0.9518451791086072), (12916, 0.6404898693400816), (12921, 0.7006490497453709), (13053, 0.9045340337332911), (13142, 0.5886020116850136), (13366, 0.8427498280790529), (13768, 0.6908492797077577), (13923, 0.4382299106011088), (13934, 0.35527985620321306), (14529, 0.6415029025857749), (14551, 0.3015113445777637), (14588, -0.1889822365046148), (14984, 0.5330017908890288), (15137, 0.5850179393017063), (15157, 0.9090909090909103), (15466, 0.7534105749568204), (15670, 0.39364791081110856), (15834, 0.2354408046740046), (16456, 0.7450131987531122), (16506, 0.5875120888504805), (17246, 0.5415138507502151), (17438, 0.7418575770165083), (17501, 0.8238110443533808), (17502, 0.4792338382985232), (17735, 0.6981582997692484), (17757, 0.5640760748177659), (17854, 0.5863019699779253), (17897, 0.6908492797077577), (17944, 0.40024936794580906), (18301, 0.4339166959907863), (18509, 0.040291148201270215), (18765, 0.32828533472737936), (18905, 0.1088931012960942), (19208, 0.6592342530872167), (19381, 0.19583736295016013), (19607, 0.47532707568263655), (19890, 0.7727272727272727), (20278, 0.7108057085060273), (20319, 0.0337099931231631), (20341, 0.876459821202215), (20444, 0.7180703308172538), (20650, 0.9327531340557992), (20708, -0.14625448482542736), (20759, 0.29545454545454464), (20838, 0.15075567228888184), (20969, 0.3863636363636383)])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pearsonCorrelationDict.items()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>similarityIndex</th>\n",
       "      <th>userId</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.511891</td>\n",
       "      <td>106</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.475327</td>\n",
       "      <td>686</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.340005</td>\n",
       "      <td>815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.350500</td>\n",
       "      <td>1130</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.863636</td>\n",
       "      <td>1502</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   similarityIndex  userId\n",
       "0         0.511891     106\n",
       "1         0.475327     686\n",
       "2         0.340005     815\n",
       "3         0.350500    1130\n",
       "4         0.863636    1502"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pearsonDF = pd.DataFrame.from_dict(pearsonCorrelationDict, orient='index')\n",
    "pearsonDF.columns = ['similarityIndex']\n",
    "pearsonDF['userId'] = pearsonDF.index\n",
    "pearsonDF.index = range(len(pearsonDF))\n",
    "pearsonDF.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>similarityIndex</th>\n",
       "      <th>userId</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>47</td>\n",
       "      <td>0.957014</td>\n",
       "      <td>10707</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>54</td>\n",
       "      <td>0.951845</td>\n",
       "      <td>12325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>95</td>\n",
       "      <td>0.932753</td>\n",
       "      <td>20650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>51</td>\n",
       "      <td>0.925591</td>\n",
       "      <td>11769</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>68</td>\n",
       "      <td>0.909091</td>\n",
       "      <td>15157</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    similarityIndex  userId\n",
       "47         0.957014   10707\n",
       "54         0.951845   12325\n",
       "95         0.932753   20650\n",
       "51         0.925591   11769\n",
       "68         0.909091   15157"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "topUsers=pearsonDF.sort_values(by='similarityIndex', ascending=False)[0:50]\n",
    "topUsers.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's start recommending movies to the input user.\n",
    "\n",
    "#### Rating of selected users to all movies\n",
    "We're going to do this by taking the weighted average of the ratings of the movies using the Pearson Correlation as the weight. But to do this, we first need to get the movies watched by the users in our __pearsonDF__ from the ratings dataframe and then store their correlation in a new column called \"*similarityIndex*\". This is achieved below by merging of these two tables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>similarityIndex</th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.957014</td>\n",
       "      <td>10707</td>\n",
       "      <td>1</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.957014</td>\n",
       "      <td>10707</td>\n",
       "      <td>2</td>\n",
       "      <td>1.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.957014</td>\n",
       "      <td>10707</td>\n",
       "      <td>6</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.957014</td>\n",
       "      <td>10707</td>\n",
       "      <td>10</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.957014</td>\n",
       "      <td>10707</td>\n",
       "      <td>16</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   similarityIndex  userId  movieId  rating\n",
       "0         0.957014   10707        1     3.0\n",
       "1         0.957014   10707        2     1.5\n",
       "2         0.957014   10707        6     4.0\n",
       "3         0.957014   10707       10     4.0\n",
       "4         0.957014   10707       16     4.0"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "topUsersRating=topUsers.merge(ratings_df, left_on='userId', right_on='userId', how='inner')\n",
    "topUsersRating.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>similarityIndex</th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "      <th>weightedRating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.957014</td>\n",
       "      <td>10707</td>\n",
       "      <td>1</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.871041</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.957014</td>\n",
       "      <td>10707</td>\n",
       "      <td>2</td>\n",
       "      <td>1.5</td>\n",
       "      <td>1.435521</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.957014</td>\n",
       "      <td>10707</td>\n",
       "      <td>6</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.828055</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.957014</td>\n",
       "      <td>10707</td>\n",
       "      <td>10</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.828055</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.957014</td>\n",
       "      <td>10707</td>\n",
       "      <td>16</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.828055</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   similarityIndex  userId  movieId  rating  weightedRating\n",
       "0         0.957014   10707        1     3.0        2.871041\n",
       "1         0.957014   10707        2     1.5        1.435521\n",
       "2         0.957014   10707        6     4.0        3.828055\n",
       "3         0.957014   10707       10     4.0        3.828055\n",
       "4         0.957014   10707       16     4.0        3.828055"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Multiplies the similarity by the user's ratings\n",
    "topUsersRating['weightedRating'] = topUsersRating['similarityIndex']*topUsersRating['rating']\n",
    "topUsersRating.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sum_similarityIndex</th>\n",
       "      <th>sum_weightedRating</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>movieId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>36.065776</td>\n",
       "      <td>135.733976</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>36.065776</td>\n",
       "      <td>92.558986</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>11.168428</td>\n",
       "      <td>28.021296</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.659234</td>\n",
       "      <td>1.318469</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>7.460800</td>\n",
       "      <td>15.112080</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         sum_similarityIndex  sum_weightedRating\n",
       "movieId                                         \n",
       "1                  36.065776          135.733976\n",
       "2                  36.065776           92.558986\n",
       "3                  11.168428           28.021296\n",
       "4                   0.659234            1.318469\n",
       "5                   7.460800           15.112080"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Applies a sum to the topUsers after grouping it up by userId\n",
    "tempTopUsersRating = topUsersRating.groupby('movieId').sum()[['similarityIndex','weightedRating']]\n",
    "tempTopUsersRating.columns = ['sum_similarityIndex','sum_weightedRating']\n",
    "tempTopUsersRating.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>weighted average recommendation score</th>\n",
       "      <th>movieId</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>movieId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>3.763512</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>2.566394</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>2.508974</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>2.025531</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         weighted average recommendation score  movieId\n",
       "movieId                                                \n",
       "1                                     3.763512        1\n",
       "2                                     2.566394        2\n",
       "3                                     2.508974        3\n",
       "4                                     2.000000        4\n",
       "5                                     2.025531        5"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Creates an empty dataframe\n",
    "recommendation_df = pd.DataFrame()\n",
    "#Now we take the weighted average\n",
    "recommendation_df['weighted average recommendation score'] = tempTopUsersRating['sum_weightedRating']/tempTopUsersRating['sum_similarityIndex']\n",
    "recommendation_df['movieId'] = tempTopUsersRating.index\n",
    "recommendation_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>weighted average recommendation score</th>\n",
       "      <th>movieId</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>movieId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>91199</td>\n",
       "      <td>5.0</td>\n",
       "      <td>91199</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26171</td>\n",
       "      <td>5.0</td>\n",
       "      <td>26171</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1493</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5605</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5605</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>987</td>\n",
       "      <td>5.0</td>\n",
       "      <td>987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26342</td>\n",
       "      <td>5.0</td>\n",
       "      <td>26342</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3451</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3451</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4038</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4038</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26350</td>\n",
       "      <td>5.0</td>\n",
       "      <td>26350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1477</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1477</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         weighted average recommendation score  movieId\n",
       "movieId                                                \n",
       "91199                                      5.0    91199\n",
       "26171                                      5.0    26171\n",
       "1493                                       5.0     1493\n",
       "5605                                       5.0     5605\n",
       "987                                        5.0      987\n",
       "26342                                      5.0    26342\n",
       "3451                                       5.0     3451\n",
       "4038                                       5.0     4038\n",
       "26350                                      5.0    26350\n",
       "1477                                       5.0     1477"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommendation_df = recommendation_df.sort_values(by='weighted average recommendation score', ascending=False)\n",
    "recommendation_df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>928</td>\n",
       "      <td>945</td>\n",
       "      <td>Top Hat</td>\n",
       "      <td>1935</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>969</td>\n",
       "      <td>987</td>\n",
       "      <td>Bliss</td>\n",
       "      <td>1997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1439</td>\n",
       "      <td>1477</td>\n",
       "      <td>Love Jones</td>\n",
       "      <td>1997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1451</td>\n",
       "      <td>1493</td>\n",
       "      <td>Love and Other Catastrophes</td>\n",
       "      <td>1996</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1958</td>\n",
       "      <td>2041</td>\n",
       "      <td>Condorman</td>\n",
       "      <td>1981</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2763</td>\n",
       "      <td>2848</td>\n",
       "      <td>Othello (Tragedy of Othello: The Moor of Venic...</td>\n",
       "      <td>1952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3363</td>\n",
       "      <td>3451</td>\n",
       "      <td>Guess Who's Coming to Dinner</td>\n",
       "      <td>1967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3379</td>\n",
       "      <td>3467</td>\n",
       "      <td>Hud</td>\n",
       "      <td>1963</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3945</td>\n",
       "      <td>4038</td>\n",
       "      <td>Kill Me Again</td>\n",
       "      <td>1989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5507</td>\n",
       "      <td>5605</td>\n",
       "      <td>Ratcatcher</td>\n",
       "      <td>1999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5669</td>\n",
       "      <td>5767</td>\n",
       "      <td>Teddy Bear (Mis)</td>\n",
       "      <td>1981</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6874</td>\n",
       "      <td>6985</td>\n",
       "      <td>Passion of Joan of Arc, The (Passion de Jeanne...</td>\n",
       "      <td>1928</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8662</td>\n",
       "      <td>26171</td>\n",
       "      <td>Play Time (a.k.a. Playtime)</td>\n",
       "      <td>1967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8785</td>\n",
       "      <td>26342</td>\n",
       "      <td>Farewell, My Lovely</td>\n",
       "      <td>1975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8792</td>\n",
       "      <td>26350</td>\n",
       "      <td>Passenger, The (Professione: reporter)</td>\n",
       "      <td>1975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8826</td>\n",
       "      <td>26400</td>\n",
       "      <td>Gates of Heaven</td>\n",
       "      <td>1978</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10670</td>\n",
       "      <td>41712</td>\n",
       "      <td>Room for Romeo Brass, A</td>\n",
       "      <td>1999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18272</td>\n",
       "      <td>91199</td>\n",
       "      <td>Weekend</td>\n",
       "      <td>2011</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20402</td>\n",
       "      <td>99917</td>\n",
       "      <td>Upstream Color</td>\n",
       "      <td>2013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20463</td>\n",
       "      <td>100106</td>\n",
       "      <td>Pervert's Guide to Ideology, The</td>\n",
       "      <td>2012</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       movieId                                              title  year\n",
       "928        945                                            Top Hat  1935\n",
       "969        987                                              Bliss  1997\n",
       "1439      1477                                         Love Jones  1997\n",
       "1451      1493                        Love and Other Catastrophes  1996\n",
       "1958      2041                                          Condorman  1981\n",
       "2763      2848  Othello (Tragedy of Othello: The Moor of Venic...  1952\n",
       "3363      3451                       Guess Who's Coming to Dinner  1967\n",
       "3379      3467                                                Hud  1963\n",
       "3945      4038                                      Kill Me Again  1989\n",
       "5507      5605                                         Ratcatcher  1999\n",
       "5669      5767                                   Teddy Bear (Mis)  1981\n",
       "6874      6985  Passion of Joan of Arc, The (Passion de Jeanne...  1928\n",
       "8662     26171                        Play Time (a.k.a. Playtime)  1967\n",
       "8785     26342                                Farewell, My Lovely  1975\n",
       "8792     26350             Passenger, The (Professione: reporter)  1975\n",
       "8826     26400                                    Gates of Heaven  1978\n",
       "10670    41712                            Room for Romeo Brass, A  1999\n",
       "18272    91199                                            Weekend  2011\n",
       "20402    99917                                     Upstream Color  2013\n",
       "20463   100106                   Pervert's Guide to Ideology, The  2012"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_df.loc[movies_df['movieId'].isin(recommendation_df.head(20)['movieId'].tolist())]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}