04-content-based-embedding.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Content Based Embedding\n",
    "\n",
    "In this experiment we will use the movie attributes (content features) to generate dense vectors that will preserve content similarity between events.\n",
    "\n",
    "The use of sparse matrices here is very important! First we will create tf-idf sparse features from the text attributes, then we will one-hot encode categorical features into sparse binary features. Finally, the Truncated SVD algorithm will be used to convert the sparse features into low-dimensional and dense embedding features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.decomposition import TruncatedSVD\n",
    "from sklearn.preprocessing import normalize\n",
    "\n",
    "from scipy.sparse import coo_matrix, hstack\n",
    "\n",
    "from src.text import clean_text\n",
    "from src.factorization import describe_csr_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Text Embedding Features\n",
    "\n",
    "Here we combine all text fields in a single one. Then we apply the `TfidfVectorizer` to enconde text to numerical sparse features. Before applying SVD, we normalize each row by its l2-norm. This normalization allows to compute the cosine similarity as the dot product between thge vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>original_title</th>\n",
       "      <th>title</th>\n",
       "      <th>overview</th>\n",
       "      <th>tagline</th>\n",
       "      <th>tags</th>\n",
       "      <th>keywords</th>\n",
       "      <th>idx</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>When siblings Judy and Peter discover an encha...</td>\n",
       "      <td>Roll the dice and unleash the excitement!</td>\n",
       "      <td>fantasy adapted from:book animals bad cgi base...</td>\n",
       "      <td>board game disappearance based on children's b...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men</td>\n",
       "      <td>Grumpier Old Men</td>\n",
       "      <td>A family wedding reignites the ancient feud be...</td>\n",
       "      <td>Still Yelling. Still Fighting. Still Ready for...</td>\n",
       "      <td>moldy old Ann Margaret Burgess Meredith Daryl ...</td>\n",
       "      <td>fishing best friend duringcreditsstinger old men</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>Cheated on, mistreated and stepped on, the wom...</td>\n",
       "      <td>Friends are the people who let you be yourself...</td>\n",
       "      <td>characters girl movie characters chick flick b...</td>\n",
       "      <td>based on novel interracial relationship single...</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>Just when George Banks has recovered from his ...</td>\n",
       "      <td>Just When His World Is Back To Normal... He's ...</td>\n",
       "      <td>steve martin steve martin pregnancy remake agi...</td>\n",
       "      <td>baby midlife crisis confidence aging daughter ...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>Heat</td>\n",
       "      <td>Heat</td>\n",
       "      <td>Obsessive master thief, Neil McCauley leads a ...</td>\n",
       "      <td>A Los Angeles Crime Saga</td>\n",
       "      <td>overrated bank robbery crime heists relationsh...</td>\n",
       "      <td>robbery detective bank obsession chase shootin...</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id               original_title                        title  \\\n",
       "0   2                      Jumanji                      Jumanji   \n",
       "1   3             Grumpier Old Men             Grumpier Old Men   \n",
       "2   4            Waiting to Exhale            Waiting to Exhale   \n",
       "3   5  Father of the Bride Part II  Father of the Bride Part II   \n",
       "4   6                         Heat                         Heat   \n",
       "\n",
       "                                            overview  \\\n",
       "0  When siblings Judy and Peter discover an encha...   \n",
       "1  A family wedding reignites the ancient feud be...   \n",
       "2  Cheated on, mistreated and stepped on, the wom...   \n",
       "3  Just when George Banks has recovered from his ...   \n",
       "4  Obsessive master thief, Neil McCauley leads a ...   \n",
       "\n",
       "                                             tagline  \\\n",
       "0          Roll the dice and unleash the excitement!   \n",
       "1  Still Yelling. Still Fighting. Still Ready for...   \n",
       "2  Friends are the people who let you be yourself...   \n",
       "3  Just When His World Is Back To Normal... He's ...   \n",
       "4                           A Los Angeles Crime Saga   \n",
       "\n",
       "                                                tags  \\\n",
       "0  fantasy adapted from:book animals bad cgi base...   \n",
       "1  moldy old Ann Margaret Burgess Meredith Daryl ...   \n",
       "2  characters girl movie characters chick flick b...   \n",
       "3  steve martin steve martin pregnancy remake agi...   \n",
       "4  overrated bank robbery crime heists relationsh...   \n",
       "\n",
       "                                            keywords  idx  \n",
       "0  board game disappearance based on children's b...    0  \n",
       "1   fishing best friend duringcreditsstinger old men    1  \n",
       "2  based on novel interracial relationship single...    2  \n",
       "3  baby midlife crisis confidence aging daughter ...    3  \n",
       "4  robbery detective bank obsession chase shootin...    4  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies = pd.read_csv('data/movie_info.csv', index_col='id')\n",
    "tags = pd.read_csv('data/movie_tags.csv', index_col='id')\n",
    "keywords = pd.read_csv('data/movie_keywords.csv', index_col='id')\n",
    "\n",
    "movies = movies.join(tags)\n",
    "movies = movies.join(keywords)\n",
    "\n",
    "movies.fillna('', inplace=True)\n",
    "movies.reset_index(drop=False, inplace=True)\n",
    "movies['idx'] = movies.index\n",
    "\n",
    "movies.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After joining all text fields, we apply some cleaning procedures:\n",
    "- Lower case.\n",
    "- Remove numbers, punctuation and special chars.\n",
    "- Remove single char words.\n",
    "- Remove stopwords."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 5.15 s, sys: 8 ms, total: 5.16 s\n",
      "Wall time: 5.15 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "movies['text'] = movies['title'] + \" \" + movies['overview'] + \" \" + movies['tagline'] + \" \" + movies['tags'] + \" \" + movies['keywords']\n",
    "text = list(map(clean_text, movies['text']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'jumanji siblings judy peter discover enchanted board game opens door magical world unwittingly invite alan adult trapped inside game years living room alan hope freedom finish game proves risky three find running giant rhinoceroses evil monkeys terrifying creatures roll dice unleash excitement fantasy adapted book animals bad cgi based book board game childhood recaptured children chris van allsburg fantasy filmed bc jungle kid flick kirsten dunst monkey robin williams saturn award best special effects saturn award best supporting actress scary time time travel fantasy robin williams adapted book childish children kid flick time travel robin williams time travel robin williams joe johnston robin williams children kid flick itaege fantasy robin williams scary time travel game animals comedy fiction thrill dynamic cgi action bad cgi horrifying horror genre kirsten dunst magic board game monkey kids based children book board game disappearance giant insect new home recluse animals fantasy lebbat bad cgi magic board game children fantasy robin williams kirsten dunst robin williams robin williams animals based book fantasy magic board game monkey kids robin williams scary time travel animals bad cgi board game kids robin williams time travel animals bad cgi based book board game chris van allsburg baker viu robin williams time travel board game robin williams filmed bc fantasy fantasy time travel adventure family fantasy kids magic board game robin williams adaptation book robin williams time travel animals bad cgi board game fantasy robin williams time travel children kid flick bad cgi kid flick robin williams monkey fantasy robin williams jungle adapted book board game childhood recaptured game herds cgi animals scary see also zathura time time travel robin williams robin williams time travel kid flick bad cgi kid flick scary time travel adventure board game fantasy robin williams time travel fantasy fantasy time travel fantasy robin williams fantasy magic board game robin williams joe johnston fantasy robin williams time travel animals board game children fantasy kirsten dunst robin williams time travel animals fantasy robin williams scary animals childhood recaptured herds cgi animals kirsten dunst robin williams rainy day watchlist robin williams robin williams animals robin williams board game family fantasy fiction robin williams scary time travel board game time travel fantasy robin williams animals board game fantasy robin williams clv bad cgi kid flick robin williams board game fantasy robin williams scary time travel board game disappearance based children book new home recluse giant insect'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We consider a vocabulary of 30k terms, taking into account unigrams and bigrams."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20624 x 30000 sparse matrix with 99.79% of sparsity.\n",
      "CPU times: user 7.47 s, sys: 192 ms, total: 7.66 s\n",
      "Wall time: 7.66 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "vectorizer = TfidfVectorizer(lowercase=False, max_df=0.5, min_df=5, ngram_range=(1, 2), \n",
    "                             norm='l2', max_features=30000, sublinear_tf=True)\n",
    "tfidf = vectorizer.fit_transform(text)\n",
    "\n",
    "print(describe_csr_matrix(tfidf))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we reduce dimension to 300, using SVD. The `arpack` parameter is necessary to deal with a sparse matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2min 54s, sys: 2.68 s, total: 2min 57s\n",
      "Wall time: 15.7 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "N_COMPONENTS = 300\n",
    "\n",
    "svd = TruncatedSVD(n_components=N_COMPONENTS, algorithm=\"arpack\", random_state=0)\n",
    "tfidf_embedding = svd.fit_transform(tfidf)\n",
    "tfidf_embedding = normalize(tfidf_embedding, norm=\"l2\", axis=1, copy=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.25727685,  0.14228107, -0.05615754, -0.22410679, -0.04457764],\n",
       "       [ 0.28550536, -0.00267135, -0.01574177, -0.07095338, -0.13337099],\n",
       "       [ 0.26452568, -0.00370943,  0.04026973, -0.19422941, -0.00220639],\n",
       "       [ 0.26318172, -0.01038827,  0.04837783, -0.08737987, -0.10562586],\n",
       "       [ 0.28532055,  0.05990469, -0.17970695, -0.02468343,  0.08128035],\n",
       "       [ 0.24741332,  0.02394533,  0.06111758, -0.22589893, -0.01820054],\n",
       "       [ 0.24301806,  0.06613637, -0.05265269, -0.13214851, -0.00093491],\n",
       "       [ 0.09992267,  0.03170735, -0.04626095,  0.02680382, -0.03229045],\n",
       "       [ 0.18925709,  0.09980298, -0.11548792, -0.01287145,  0.01059282],\n",
       "       [ 0.25855054,  0.08158593,  0.03407546, -0.08312593, -0.03829589]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tfidf_embedding[:10, :5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's use the [Embedding Projector](https://projector.tensorflow.org/) to visualize the embedding space:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 8.82 s, sys: 44 ms, total: 8.87 s\n",
      "Wall time: 8.87 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "movies[['id','title']].to_csv('output/content_embedding_meta.tsv', sep='\\t', header=True, index=False)\n",
    "pd.DataFrame(tfidf_embedding).to_csv('output/tfidf_embedding_vectors.tsv', sep='\\t', \n",
    "                                     float_format='%.5f', header=False, index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/tfidf_lord_rings.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/tfidf_star_wars.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/tfidf_pulp_fiction.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Categorical Embedding Features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For categorical features we need to carefully transform them to a binary sparse matrix where movies are put as rows and categories are put as columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>person_id</th>\n",
       "      <th>job</th>\n",
       "      <th>person_name</th>\n",
       "      <th>num_movies</th>\n",
       "      <th>movie_idx</th>\n",
       "      <th>person_idx</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>361</th>\n",
       "      <td>1</td>\n",
       "      <td>Director</td>\n",
       "      <td>George Lucas</td>\n",
       "      <td>61</td>\n",
       "      <td>204</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>362</th>\n",
       "      <td>1</td>\n",
       "      <td>Writer</td>\n",
       "      <td>George Lucas</td>\n",
       "      <td>61</td>\n",
       "      <td>204</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3389</th>\n",
       "      <td>1</td>\n",
       "      <td>Director</td>\n",
       "      <td>George Lucas</td>\n",
       "      <td>61</td>\n",
       "      <td>1923</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4327</th>\n",
       "      <td>1</td>\n",
       "      <td>Director</td>\n",
       "      <td>George Lucas</td>\n",
       "      <td>61</td>\n",
       "      <td>2461</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6794</th>\n",
       "      <td>1</td>\n",
       "      <td>Director</td>\n",
       "      <td>George Lucas</td>\n",
       "      <td>61</td>\n",
       "      <td>3921</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     person_id       job   person_name  num_movies  movie_idx  person_idx\n",
       "361          1  Director  George Lucas          61        204           0\n",
       "362          1    Writer  George Lucas          61        204           0\n",
       "3389         1  Director  George Lucas          61       1923           0\n",
       "4327         1  Director  George Lucas          61       2461           0\n",
       "6794         1  Director  George Lucas          61       3921           0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "crew = pd.read_csv('data/movie_producer.csv', index_col='id')\n",
    "\n",
    "crew = crew.join(movies[['id','idx']].set_index('id'), how='inner')\n",
    "crew.reset_index(drop=True, inplace=True)\n",
    "crew.rename(columns={'idx':'movie_idx'}, inplace=True)\n",
    "\n",
    "crew['person_id'] = crew['person_id'].astype('category')\n",
    "crew['person_idx'] = crew['person_id'].cat.codes\n",
    "\n",
    "crew.sort_values(['person_idx','movie_idx'], inplace=True)\n",
    "\n",
    "crew.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some people will have more than one job in the same movie. So, we will count the number of roles and use this number in the sparse matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>person_idx</th>\n",
       "      <th>movie_idx</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>204</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1923</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>2461</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>3921</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>4758</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   person_idx  movie_idx  count\n",
       "0           0        204      2\n",
       "1           0       1923      1\n",
       "2           0       2461      1\n",
       "3           0       3921      2\n",
       "4           0       4758      1"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "crew = crew.groupby(['person_idx','movie_idx']).agg(\n",
    "    count=pd.NamedAgg(column='movie_idx', aggfunc='count')\n",
    ")\n",
    "crew.reset_index(drop=False, inplace=True)\n",
    "\n",
    "crew.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20624 x 3635 sparse matrix with 99.97% of sparsity.\n"
     ]
    }
   ],
   "source": [
    "values = crew['count'].values\n",
    "idx = (crew.movie_idx.values, crew.person_idx.values,)\n",
    "dim = (movies.idx.max()+1, crew.person_idx.max()+1)\n",
    "x_crew = coo_matrix((values, idx), shape=dim).tocsr()\n",
    "\n",
    "print(describe_csr_matrix(x_crew))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After creating the sparse matrix, we normalize its rows and bind them with the tf-idf matrix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20624 x 33635 sparse matrix with 99.81% of sparsity.\n"
     ]
    }
   ],
   "source": [
    "x_crew = normalize(x_crew, norm=\"l2\", axis=1, copy=False)\n",
    "x_tfidf_crew = hstack([tfidf, x_crew])\n",
    "\n",
    "print(describe_csr_matrix(x_tfidf_crew))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we apply the SVD in all sparse features (text + categorical):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2min 55s, sys: 2.46 s, total: 2min 57s\n",
      "Wall time: 15.8 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "tfidf_crew_embedding = svd.fit_transform(x_tfidf_crew)\n",
    "tfidf_crew_embedding = normalize(tfidf_crew_embedding, norm=\"l2\", axis=1, copy=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.31059373,  0.17439317, -0.07966628, -0.24312864,  0.05104854],\n",
       "       [ 0.33851136, -0.00991364,  0.00576794, -0.06548582, -0.1383411 ],\n",
       "       [ 0.32878417, -0.01374984,  0.06625597, -0.204537  , -0.05281414],\n",
       "       [ 0.30297741, -0.03919086,  0.08155608, -0.1031477 , -0.0885925 ],\n",
       "       [ 0.15918579,  0.04355397, -0.09853233, -0.04052476, -0.02758957],\n",
       "       [ 0.12162072,  0.02209307,  0.03885346, -0.12141141, -0.0453824 ],\n",
       "       [ 0.32863383,  0.08996191, -0.0563822 , -0.15020375, -0.03584434],\n",
       "       [ 0.06833647,  0.02258088, -0.04616372,  0.02299206, -0.00446049],\n",
       "       [ 0.17945057,  0.10574306, -0.12189527, -0.02358589,  0.01602491],\n",
       "       [ 0.15631755,  0.04875546,  0.03154279, -0.07359079, -0.04261311]])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tfidf_crew_embedding[:10, :5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 8.76 s, sys: 36 ms, total: 8.79 s\n",
      "Wall time: 8.79 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "pd.DataFrame(tfidf_crew_embedding).to_csv('output/tfidf_crew_embedding_vectors.tsv', sep='\\t', \n",
    "                                          float_format='%.5f', header=False, index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/tfidf_crew_lord_rings.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/tfidf_crew_star_wars.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/tfidf_crew_pulp_fiction.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will apply the same process to the cast information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>actor_id</th>\n",
       "      <th>actor_name</th>\n",
       "      <th>num_movies</th>\n",
       "      <th>movie_idx</th>\n",
       "      <th>actor_idx</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>14338</th>\n",
       "      <td>1</td>\n",
       "      <td>George Lucas</td>\n",
       "      <td>20</td>\n",
       "      <td>1516</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23600</th>\n",
       "      <td>1</td>\n",
       "      <td>George Lucas</td>\n",
       "      <td>20</td>\n",
       "      <td>2547</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55588</th>\n",
       "      <td>1</td>\n",
       "      <td>George Lucas</td>\n",
       "      <td>20</td>\n",
       "      <td>6392</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59375</th>\n",
       "      <td>1</td>\n",
       "      <td>George Lucas</td>\n",
       "      <td>20</td>\n",
       "      <td>6935</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84758</th>\n",
       "      <td>1</td>\n",
       "      <td>George Lucas</td>\n",
       "      <td>20</td>\n",
       "      <td>10202</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      actor_id    actor_name  num_movies  movie_idx  actor_idx  count\n",
       "14338        1  George Lucas          20       1516          0    1.0\n",
       "23600        1  George Lucas          20       2547          0    1.0\n",
       "55588        1  George Lucas          20       6392          0    1.0\n",
       "59375        1  George Lucas          20       6935          0    1.0\n",
       "84758        1  George Lucas          20      10202          0    1.0"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cast = pd.read_csv('data/movie_actor.csv', index_col='id')\n",
    "\n",
    "cast = cast.join(movies[['id','idx']].set_index('id'), how='inner')\n",
    "cast.reset_index(drop=True, inplace=True)\n",
    "cast.rename(columns={'idx':'movie_idx'}, inplace=True)\n",
    "\n",
    "cast['actor_id'] = cast['actor_id'].astype('category')\n",
    "cast['actor_idx'] = cast['actor_id'].cat.codes\n",
    "cast['count'] = 1.0\n",
    "\n",
    "cast.sort_values(['actor_idx','movie_idx'], inplace=True)\n",
    "\n",
    "cast.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20624 x 10269 sparse matrix with 99.94% of sparsity.\n"
     ]
    }
   ],
   "source": [
    "values = cast['count'].values\n",
    "idx = (cast.movie_idx.values, cast.actor_idx.values,)\n",
    "dim = (movies.idx.max()+1, cast.actor_idx.max()+1)\n",
    "x_cast = coo_matrix((values, idx), shape=dim).tocsr()\n",
    "\n",
    "print(describe_csr_matrix(x_cast))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20624 x 43904 sparse matrix with 99.84% of sparsity.\n"
     ]
    }
   ],
   "source": [
    "x_tfidf_crew_cast = hstack([tfidf, x_crew, x_cast])\n",
    "\n",
    "print(describe_csr_matrix(x_tfidf_crew_cast))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 3min 34s, sys: 3.27 s, total: 3min 37s\n",
      "Wall time: 19.2 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "tfidf_crew_cast_embedding = svd.fit_transform(x_tfidf_crew_cast)\n",
    "tfidf_crew_cast_embedding = normalize(tfidf_crew_cast_embedding, norm=\"l2\", axis=1, copy=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 5.97115181e-02, -1.68955073e-01, -2.03521438e-02,\n",
       "        -3.50106713e-03,  4.41017307e-02],\n",
       "       [ 5.27002925e-02, -8.92850924e-02, -2.73559422e-02,\n",
       "         2.60500081e-04, -6.02938200e-02],\n",
       "       [ 7.78474004e-02, -1.93302408e-01, -4.77872985e-02,\n",
       "        -2.22740679e-03,  2.55805720e-02],\n",
       "       [ 5.09061308e-02, -1.27617687e-01, -1.85314134e-02,\n",
       "        -2.61425765e-03,  1.86144973e-02],\n",
       "       [ 5.03717413e-02, -1.59873186e-01, -2.73723736e-02,\n",
       "        -3.54352878e-03,  5.11350651e-02],\n",
       "       [ 5.50013444e-02, -1.46239852e-01, -2.02701450e-02,\n",
       "        -1.80958369e-03,  2.07394806e-02],\n",
       "       [ 1.85454047e-01, -3.60821251e-01, -8.32022824e-02,\n",
       "        -3.71871403e-03, -1.30683332e-02],\n",
       "       [ 5.56385271e-02, -1.27784343e-01, -3.40226278e-02,\n",
       "        -1.44905896e-03, -1.11192659e-03],\n",
       "       [ 3.82868947e-02, -1.04685110e-01, -2.44340864e-02,\n",
       "        -2.14479531e-03,  3.64669762e-02],\n",
       "       [ 5.87999692e-02, -1.52994027e-01, -2.44519542e-02,\n",
       "        -1.98539931e-03,  1.14726547e-02]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tfidf_crew_cast_embedding[:10, :5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 8.62 s, sys: 28 ms, total: 8.65 s\n",
      "Wall time: 8.64 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "pd.DataFrame(tfidf_crew_cast_embedding).to_csv('output/tfidf_crew_cast_embedding_vectors.tsv', sep='\\t', \n",
    "                                          float_format='%.5f', header=False, index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/tfidf_crew_cast_lord_rings.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/tfidf_crew_cast_star_wars.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/tfidf_crew_cast_pulp_fiction.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have the content embedding features to be used in the similarity match task."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}