genre_analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intro\n",
    "Some websites allow writers and publishers to publish their books in a simple way. Each person has a preference for literary genres when choosing their next reading, so choosing genres well when publishing a book can make your book reach the right audience, thereby increasing your sales or advertising.\n",
    "The focus of this project will be to create a method for recommending literary genre tags for writers and publishers to publish their books. Thus, when filling in filling out the book's information on a platform, it indicates some genre tags that best fit the description of the work. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data\n",
    "We will use a dataset with information of published books on the Goodreads platform. The dataset contains information such as the name of the authors, title, genres, description of the book, and so on. Thus, we will analyze it and try to obtain a classification method based on this information to provide genres for an unpublished work, based on it's description.\n",
    "All information about the dataset can be found [here](https://www.kaggle.com/bahramjannesarr/goodreads-book-datasets-10m)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Load packages\n",
    "import pandas as pd\n",
    "import re\n",
    "import numpy as np\n",
    "import nltk\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from nltk.tokenize import word_tokenize\n",
    "from nltk.stem import PorterStemmer\n",
    "from nltk.stem import WordNetLemmatizer\n",
    "from nltk.corpus import stopwords\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.multiclass import OneVsRestClassifier\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.metrics import make_scorer\n",
    "from sklearn.metrics import hamming_loss\n",
    "\n",
    "# Download nltk data. Uncomment if you need to download NLTK data\n",
    "# nltk.download('punkt')\n",
    "# nltk.download('wordnet')\n",
    "# nltk.download('stopwords')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Name</th>\n      <th>Authors</th>\n      <th>Description</th>\n      <th>Genres</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>Haroun and the Sea of Stories</td>\n      <td>Salman Rushdie</td>\n      <td>The author of The Satanic Verses returns with ...</td>\n      <td>['Fiction']</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>The Desire and Pursuit of the Whole: A Romance...</td>\n      <td>Frederick Rolfe</td>\n      <td>&lt;i&gt;The Desire and Pursuit of the Whole&lt;/i&gt; sta...</td>\n      <td>['Fiction']</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>Green Arrow, Vol. 2: Sounds of Violence</td>\n      <td>Kevin Smith</td>\n      <td>The reinvention of a classic comics character ...</td>\n      <td>['Comic books, strips, etc']</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>Trojan Odyssey (Dirk Pitt, #17)</td>\n      <td>Clive Cussler</td>\n      <td>Long hailed as the grand master of adventure f...</td>\n      <td>['Fiction']</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>Strontium Dog: Search/Destroy Agency Files, Vo...</td>\n      <td>John Wagner</td>\n      <td>Earth, the late 22nd century. Following the at...</td>\n      <td>['Bounty hunters']</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "                                                Name          Authors  \\\n0                      Haroun and the Sea of Stories   Salman Rushdie   \n1  The Desire and Pursuit of the Whole: A Romance...  Frederick Rolfe   \n2            Green Arrow, Vol. 2: Sounds of Violence      Kevin Smith   \n3                    Trojan Odyssey (Dirk Pitt, #17)    Clive Cussler   \n4  Strontium Dog: Search/Destroy Agency Files, Vo...      John Wagner   \n\n                                         Description  \\\n0  The author of The Satanic Verses returns with ...   \n1  <i>The Desire and Pursuit of the Whole</i> sta...   \n2  The reinvention of a classic comics character ...   \n3  Long hailed as the grand master of adventure f...   \n4  Earth, the late 22nd century. Following the at...   \n\n                         Genres  \n0                   ['Fiction']  \n1                   ['Fiction']  \n2  ['Comic books, strips, etc']  \n3                   ['Fiction']  \n4            ['Bounty hunters']  "
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "books_df = pd.read_csv(\"new_data/books_data.csv\", usecols=[\"Name\", \"Authors\", \"Description\", \"Genres\"])\n",
    "books_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(111436, 4)\n"
     ]
    }
   ],
   "source": [
    "print(books_df.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 111436 entries, 0 to 111435\n",
      "Data columns (total 4 columns):\n",
      " #   Column       Non-Null Count   Dtype \n",
      "---  ------       --------------   ----- \n",
      " 0   Name         111436 non-null  object\n",
      " 1   Authors      111436 non-null  object\n",
      " 2   Description  111436 non-null  object\n",
      " 3   Genres       111436 non-null  object\n",
      "dtypes: object(4)\n",
      "memory usage: 3.4+ MB\n"
     ]
    }
   ],
   "source": [
    "books_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset has 111436 samples. It is a very good amount of examples for a machine learning problem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will focus on text analysis on the \"Description\" column, so we will drop the other columns. This way, we will reduce the memory usage and leave the dataset cleaner for a better modeling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "books_description_and_genres = books_df.drop(columns=[\"Name\", \"Authors\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this phase, we will clean the texts, removing symbols, accents, punctuations, etc. Next, we will use some Natural Language Processing techniques to better process the text, such as tokenization, removing stop words and text normalization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Description</th>\n      <th>Genres</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>the author of the satanic verses returns with ...</td>\n      <td>fiction</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>ithe desire and pursuit of the wholei stands a...</td>\n      <td>fiction</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>the reinvention of a classic comics character ...</td>\n      <td>comic books, strips, etc</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>long hailed as the grand master of adventure f...</td>\n      <td>fiction</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>earth the late nd century following the atomic...</td>\n      <td>bounty hunters</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "                                         Description                    Genres\n0  the author of the satanic verses returns with ...                   fiction\n1  ithe desire and pursuit of the wholei stands a...                   fiction\n2  the reinvention of a classic comics character ...  comic books, strips, etc\n3  long hailed as the grand master of adventure f...                   fiction\n4  earth the late nd century following the atomic...            bounty hunters"
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "books_description_and_genres[\"Genres\"] = books_description_and_genres[\"Genres\"].apply(lambda x: re.sub(r\"[\\[\\]\\']|&amp\", \"\", str(x).lower()))\n",
    "books_description_and_genres[\"Description\"] = books_description_and_genres[\"Description\"].apply(lambda x: re.sub(r\"[\\,\\.\\\\\\/\\_\\:\\;\\>\\<\\}\\{\\´\\`\\(\\)\\+\\-\\%\\$\\#\\@\\!\\?\\[\\]\\&]|\\d\", \"\", str(x).lower().strip()))\n",
    "books_description_and_genres.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def text_preprocessing(text):\n",
    "    # Tokenization\n",
    "    tokenized = word_tokenize(text)\n",
    "    # Text normalization\n",
    "    # Stemming\n",
    "    stemmer = PorterStemmer()\n",
    "    stemmed = [stemmer.stem(token) for token in tokenized]\n",
    "    # Lemmatization\n",
    "    # lemmatizer = WordNetLemmatizer()\n",
    "    # lemmatized = [lemmatizer.lemmatize(token) for token in stemmed]\n",
    "    # Remove stopwords\n",
    "    stop_words = stopwords.words(\"english\")\n",
    "    text_no_stop = [token for token in stemmed if token not in stop_words]\n",
    "    \n",
    "    processed_text = \" \".join(text_no_stop)\n",
    "\n",
    "    return processed_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": "0    author satan vers return hi humor access novel...\n1    ith desir pursuit wholei stand uniqu scurril s...\n2    reinvent classic comic charact continuesoliv q...\n3    long hail grand master adventur fiction clive ...\n4    earth late nd centuri follow atom war britain ...\nName: Description, dtype: object"
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "books_description_and_genres[\"Description\"] = books_description_and_genres[\"Description\"].apply(lambda x: text_preprocessing(x))\n",
    "books_description_and_genres[\"Description\"].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will create a list containing all the literary genres present in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of genres 6282\n"
     ]
    },
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Genres</th>\n      <th>Number of books</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>fiction</td>\n      <td>28767</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>juvenile fiction</td>\n      <td>10559</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>history</td>\n      <td>7477</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>religion</td>\n      <td>3334</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>biography &amp; autobiography</td>\n      <td>3167</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>95</th>\n      <td>christian life</td>\n      <td>58</td>\n    </tr>\n    <tr>\n      <th>96</th>\n      <td>africa</td>\n      <td>58</td>\n    </tr>\n    <tr>\n      <th>97</th>\n      <td>europe</td>\n      <td>58</td>\n    </tr>\n    <tr>\n      <th>98</th>\n      <td>detective and mystery stories, english</td>\n      <td>58</td>\n    </tr>\n    <tr>\n      <th>99</th>\n      <td>artists</td>\n      <td>58</td>\n    </tr>\n  </tbody>\n</table>\n<p>100 rows × 2 columns</p>\n</div>",
      "text/plain": "                                    Genres  Number of books\n0                                  fiction            28767\n1                         juvenile fiction            10559\n2                                  history             7477\n3                                 religion             3334\n4                biography & autobiography             3167\n..                                     ...              ...\n95                          christian life               58\n96                                  africa               58\n97                                  europe               58\n98  detective and mystery stories, english               58\n99                                 artists               58\n\n[100 rows x 2 columns]"
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"Number of genres {num_genres}\".format(num_genres=books_description_and_genres[\"Genres\"].nunique()))\n",
    "top_100_genres = books_description_and_genres[\"Genres\"].value_counts().head(100).reset_index()\n",
    "top_100_genres.rename(columns={\n",
    "    \"Genres\": \"Number of books\",\n",
    "    \"index\": \"Genres\"\n",
    "}, inplace=True)\n",
    "\n",
    "top_100_genres"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have 6282 genres, some of them are only for a small list of books. So we will reduce the number of genres to 100, to reduce memory usage. One think to notice here is that the majority of the books are fictions, having almost 3 times the number of books of the second most higher genre, \"juvenile fiction\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "genres = []\n",
    "\n",
    "for genre in top_100_genres[\"Genres\"]:\n",
    "    if genre not in genres:\n",
    "        genres.append(genre)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Turn each genre into a column, so our dataset will be in a wide-form. Whenever a sample has that genre in its list of genres, then we put a value of 1 in the column that represents the genre, if it does not have it, we add 0. For example, if a book is only \"adventure\", then its \"adventure\" column will have a value of 1 and the rest of the genre columns will have a value of 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Description</th>\n      <th>Genres</th>\n      <th>fiction</th>\n      <th>juvenile fiction</th>\n      <th>history</th>\n      <th>religion</th>\n      <th>biography &amp; autobiography</th>\n      <th>comics &amp; graphic novels</th>\n      <th>juvenile nonfiction</th>\n      <th>business &amp; economics</th>\n      <th>...</th>\n      <th>fantasy fiction, american</th>\n      <th>authors, american</th>\n      <th>australian fiction</th>\n      <th>london (england)</th>\n      <th>children</th>\n      <th>christian life</th>\n      <th>africa</th>\n      <th>europe</th>\n      <th>detective and mystery stories, english</th>\n      <th>artists</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>author satan vers return hi humor access novel...</td>\n      <td>fiction</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>ith desir pursuit wholei stand uniqu scurril s...</td>\n      <td>fiction</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>reinvent classic comic charact continuesoliv q...</td>\n      <td>comic books, strips, etc</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>long hail grand master adventur fiction clive ...</td>\n      <td>fiction</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>earth late nd centuri follow atom war britain ...</td>\n      <td>bounty hunters</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>...</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 102 columns</p>\n</div>",
      "text/plain": "                                         Description  \\\n0  author satan vers return hi humor access novel...   \n1  ith desir pursuit wholei stand uniqu scurril s...   \n2  reinvent classic comic charact continuesoliv q...   \n3  long hail grand master adventur fiction clive ...   \n4  earth late nd centuri follow atom war britain ...   \n\n                     Genres  fiction  juvenile fiction  history  religion  \\\n0                   fiction        1                 0        0         0   \n1                   fiction        1                 0        0         0   \n2  comic books, strips, etc        0                 0        0         0   \n3                   fiction        1                 0        0         0   \n4            bounty hunters        0                 0        0         0   \n\n   biography & autobiography  comics & graphic novels  juvenile nonfiction  \\\n0                          0                        0                    0   \n1                          0                        0                    0   \n2                          0                        0                    0   \n3                          0                        0                    0   \n4                          0                        0                    0   \n\n   business & economics  ...  fantasy fiction, american  authors, american  \\\n0                     0  ...                          0                  0   \n1                     0  ...                          0                  0   \n2                     0  ...                          0                  0   \n3                     0  ...                          0                  0   \n4                     0  ...                          0                  0   \n\n   australian fiction  london (england)  children  christian life  africa  \\\n0                   0                 0         0               0       0   \n1                   0                 0         0               0       0   \n2                   0                 0         0               0       0   \n3                   0                 0         0               0       0   \n4                   0                 0         0               0       0   \n\n   europe  detective and mystery stories, english  artists  \n0       0                                       0        0  \n1       0                                       0        0  \n2       0                                       0        0  \n3       0                                       0        0  \n4       0                                       0        0  \n\n[5 rows x 102 columns]"
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for genre in genres:\n",
    "    books_description_and_genres[genre] = books_description_and_genres[\"Genres\"].apply(lambda x: 1 if genre == x else 0)\n",
    "\n",
    "books_description_and_genres.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Modeling\n",
    "First, we will separate the data into two sets, a training set, which will serve to train the Machine Learning (ML) algorithm, and a test set to check how well the model does with data not yet seen. Next, we will use a pipeline so that the data can be processed with tf-idf (term frequency-inverse document frequency) and bag-of-words (BoW), then use the Naive Bayes ML algorithm using the One-vs-Rest technique (OvR), which allows us to do a multilabel classification, that is, of all classes (Multiclass) the result for being more than 1 of them. More info about multilabel classification go [here](https://towardsdatascience.com/multi-label-text-classification-with-scikit-learn-30714b7819c5) and more info about OvR go [here](https://machinelearningmastery.com/one-vs-rest-and-one-vs-one-for-multi-class-classification/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = books_description_and_genres[\"Description\"]\n",
    "y = books_description_and_genres.drop(columns=[\"Description\", \"Genres\"])\n",
    "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, shuffle=True, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use unigrams (one word) and bigrams (combination of two terms) to better analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": "Pipeline(steps=[('vectorizer',\n                 TfidfVectorizer(max_features=10000, ngram_range=(1, 2))),\n                ('model',\n                 OneVsRestClassifier(estimator=MultinomialNB(), n_jobs=-1))])"
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe_tf_idf = Pipeline([\n",
    "    (\"vectorizer\", TfidfVectorizer(ngram_range=(1, 2), max_features=10000)),\n",
    "    (\"model\", OneVsRestClassifier(MultinomialNB(), n_jobs=-1))\n",
    "])\n",
    "\n",
    "pipe_tf_idf.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": "Pipeline(steps=[('vectorizer',\n                 CountVectorizer(max_features=10000, ngram_range=(1, 2))),\n                ('model',\n                 OneVsRestClassifier(estimator=MultinomialNB(), n_jobs=-1))])"
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe_bag_of_words = Pipeline([\n",
    "    (\"vectorizer\", CountVectorizer(ngram_range=(1, 2), max_features=10000)),\n",
    "    (\"model\", OneVsRestClassifier(MultinomialNB(), n_jobs=-1))\n",
    "])\n",
    "\n",
    "pipe_bag_of_words.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluation\n",
    "Hamming loss is one of the most used valuation techniques in cases of multilabel classification. Accuracy would measure the hit rate if all possible genres in a sample were classified correctly, for example: if the algorithm predicted that the genres of a hypothetical book would be action and adventure, but the book is of the genre adventure and romance, the accuracy test would consider this result as an error, despite the fact that the algorithm got one of the values right. Hamming loss checks each forecast, label by label, and gives us a result based on the rate of errors we make in each assessment, so the lower the value of hamming loss, the better our result. More information [here](https://www.geeksforgeeks.org/an-introduction-to-multilabel-classification/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TF-IDF results:\n",
      "Accuracy: 0.38258255563531945\n",
      "Hamming loss: 0.007022164393395549\n"
     ]
    }
   ],
   "source": [
    "print(\"TF-IDF results:\")\n",
    "print(\"Accuracy: {accuracy}\".format(accuracy=pipe_tf_idf.score(x_test, y_test)))\n",
    "\n",
    "predicted_tf_idf_labels = pipe_tf_idf.predict(x_test)\n",
    "\n",
    "hamming_loss_result = hamming_loss(y_test, predicted_tf_idf_labels)\n",
    "print(\"Hamming loss: {hamming_loss}\".format(hamming_loss=hamming_loss_result))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bag-of-Words results:\n",
      "Accuracy: 0.13419777458722182\n",
      "Hamming loss: 0.03543386575735822\n"
     ]
    }
   ],
   "source": [
    "print(\"Bag-of-Words results:\")\n",
    "print(\"Accuracy: {accuracy}\".format(accuracy=pipe_bag_of_words.score(x_test, y_test)))\n",
    "\n",
    "predicted_bow_labels = pipe_bag_of_words.predict(x_test)\n",
    "\n",
    "hamming_loss_result = hamming_loss(y_test, predicted_bow_labels)\n",
    "print(\"Hamming loss: {hamming_loss}\".format(hamming_loss=hamming_loss_result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, the accuracy was extremely low, as it will penalize the result even in cases where only 1 of the labels is erroneously predicted. However, looking at a metric more focused on multilabel, such as hamming_loss, we see that the error rate was very small, which indicates that our model did well in most cases. Again, the less the Hamming loss result, the better.\n",
    "The results for TF-IDF are better than using only bag-of-words model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Results and discussion\n",
    "After pre-processing the description of the books, using Natural Language Processing (NLP) techniques and Naive Bayes ML Algorithm, we were able to have a good result in identifying the literary genre of a book through a brief description of it. This system can help authors and publishers to better publish their books, where a book publishing platform can use this algorithm to recommend genre tags at the time of publication, or can be used to categorize already published books that do not have a specified genre.\n",
    "Although the results are very good, we can expand the project using other Machine Learning algorithms, doing hyperparametrization to obtain even more accurate results and using Part-of-Speech (POS) tags to use Lemmatization in Text Preprocessing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['fiction']\n"
     ]
    }
   ],
   "source": [
    "# A Game of Thrones\n",
    "# A Song of Ice and Fire\n",
    "book_description = \"Winter is coming. Such is the stern motto of House Stark, the northernmost of the fiefdoms that owe allegiance to King Robert Baratheon in far-off King’s Landing. There Eddard Stark of Winterfell rules in Robert’s name. There his family dwells in peace and comfort: his proud wife, Catelyn; his sons Robb, Brandon, and Rickon; his daughters Sansa and Arya; and his bastard son, Jon Snow. Far to the north, behind the towering Wall, lie savage Wildings and worse—unnatural things relegated to myth during the centuries-long summer, but proving all too real and all too deadly in the turning of the season.\"\n",
    "\n",
    "processed_book_description = text_preprocessing(book_description)\n",
    "\n",
    "y_pred = pipe_tf_idf.predict([processed_book_description])\n",
    "\n",
    "# If you want to see the entire array, uncomment the line bellow\n",
    "# print(y_pred)\n",
    "\n",
    "genres_pred = []\n",
    "for i in range(len(y.columns)):\n",
    "    if y_pred[0][i] == 1:\n",
    "        genres_pred.append(y.columns[i])\n",
    "\n",
    "print(\"Predicted genres: {genres_pred}\".format(genres_pred=genres_pred))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.6 64-bit",
   "name": "python396jvsc74a57bd0767d51c1340bd893661ea55ea3124f6de3c7a262a8b4abca0554b478b1e2ff90"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "metadata": {
   "interpreter": {
    "hash": "767d51c1340bd893661ea55ea3124f6de3c7a262a8b4abca0554b478b1e2ff90"
   }
  },
  "orig_nbformat": 2
 },
 "nbformat": 4,
 "nbformat_minor": 2
}