Sentiment Classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sentiment Classifier using Python\n",
    "\n",
    "1. preprocess the data\n",
    "2. convert English data to numerical representations\n",
    "3. prepare it to be fed as input for our deeplearning model with GRUs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing the modules :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "tf.random.set_seed(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the dataset :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_keys(['test', 'train', 'unsupervised'])\n"
     ]
    }
   ],
   "source": [
    "#importing dataset\n",
    "import tensorflow_datasets as tfds\n",
    "datasets, info = tfds.load(\"imdb_reviews\", as_supervised=True, with_info=True)\n",
    "print(datasets.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25000 25000\n"
     ]
    }
   ],
   "source": [
    "train_size = info.splits[\"train\"].num_examples\n",
    "test_size = info.splits[\"test\"].num_examples\n",
    "print(train_size, test_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploring the dataset :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Review :  This was an absolutely terrible movie. Don't be lured in by Christopher Walken or Michael Ironside. Both are great actors, but this must simply be their worst role in history. Even their great acting  ...\n",
      "Label :  0  = Negative\n",
      "\n",
      "Review :  I have been known to fall asleep during films, but this is usually due to a combination of things including, really tired, being warm and comfortable on the sette and having just eaten a lot. However  ...\n",
      "Label :  0  = Negative\n",
      "\n",
      "Review :  Mann photographs the Alberta Rocky Mountains in a superb fashion, and Jimmy Stewart and Walter Brennan give enjoyable performances as they always seem to do. <br /><br />But come on Hollywood - a Moun ...\n",
      "Label :  0  = Negative\n",
      "\n",
      "Review :  This is the kind of film for a snowy Sunday afternoon when the rest of the world can go ahead with its own business as you descend into a big arm-chair and mellow for a couple of hours. Wonderful perf ...\n",
      "Label :  1  = Positive\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for X_batch, y_batch in datasets[\"train\"].batch(2).take(2):\n",
    "    for review, label in zip(X_batch.numpy(), y_batch.numpy()):\n",
    "        print(\"Review : \", review.decode(\"utf-8\")[:200], \"...\")\n",
    "        print(\"Label : \", label, \" = Positive\" if label else \" = Negative\")\n",
    "        print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining the preprocess function :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def preprocess(X_batch, y_batch):\n",
    "    X_batch = tf.strings.substr(X_batch, 0, 300)\n",
    "    X_batch = tf.strings.regex_replace(X_batch, rb\"<br\\s*/?>\", b\" \")\n",
    "    X_batch = tf.strings.regex_replace(X_batch, b\"[^a-zA-Z]\", b\" \")\n",
    "    X_batch = tf.strings.split(X_batch)\n",
    "    return X_batch.to_tensor(default_value = b\"<pad>\"), y_batch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<tf.Tensor: shape=(2, 59), dtype=string, numpy=\n",
       " array([[b'Mann', b'photographs', b'the', b'Alberta', b'Rocky',\n",
       "         b'Mountains', b'in', b'a', b'superb', b'fashion', b'and',\n",
       "         b'Jimmy', b'Stewart', b'and', b'Walter', b'Brennan', b'give',\n",
       "         b'enjoyable', b'performances', b'as', b'they', b'always',\n",
       "         b'seem', b'to', b'do', b'But', b'come', b'on', b'Hollywood',\n",
       "         b'a', b'Mountie', b'telling', b'the', b'people', b'of',\n",
       "         b'Dawson', b'City', b'Yukon', b'to', b'elect', b'themselves',\n",
       "         b'a', b'marshal', b'yes', b'a', b'marshal', b'and', b'to', b'e',\n",
       "         b'<pad>', b'<pad>', b'<pad>', b'<pad>', b'<pad>', b'<pad>',\n",
       "         b'<pad>', b'<pad>', b'<pad>', b'<pad>'],\n",
       "        [b'This', b'is', b'the', b'kind', b'of', b'film', b'for', b'a',\n",
       "         b'snowy', b'Sunday', b'afternoon', b'when', b'the', b'rest',\n",
       "         b'of', b'the', b'world', b'can', b'go', b'ahead', b'with',\n",
       "         b'its', b'own', b'business', b'as', b'you', b'descend', b'into',\n",
       "         b'a', b'big', b'arm', b'chair', b'and', b'mellow', b'for', b'a',\n",
       "         b'couple', b'of', b'hours', b'Wonderful', b'performances',\n",
       "         b'from', b'Cher', b'and', b'Nicolas', b'Cage', b'as', b'always',\n",
       "         b'gently', b'row', b'the', b'plot', b'along', b'There', b'are',\n",
       "         b'no', b'rapids', b'to', b'cr']], dtype=object)>,\n",
       " <tf.Tensor: shape=(2,), dtype=int64, numpy=array([0, 1])>)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preprocess(X_batch, y_batch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Constructing the Vocabulary :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "vocabulary = Counter()\n",
    "for X_batch, y_batch in datasets[\"train\"].batch(2).map(preprocess):\n",
    "    for review in X_batch:\n",
    "        vocabulary.update(list(review.numpy()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(b'<pad>', 65828),\n",
       " (b'the', 61156),\n",
       " (b'a', 38569),\n",
       " (b'of', 33984),\n",
       " (b'and', 33432)]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocabulary.most_common()[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "49739"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(vocabulary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Truncating the Vocabulary :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "vocab_size = 10000\n",
    "truncated_vocabulary = [words for words, count in vocabulary.most_common()[:vocab_size]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a lookup table :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "words = tf.constant(truncated_vocabulary)\n",
    "word_ids = tf.range(len(truncated_vocabulary), dtype = tf.int64)\n",
    "vocab_init = tf.lookup.KeyValueTensorInitializer(words, word_ids)\n",
    "#print(vocab_init)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_oov_buckets = 1000\n",
    "table = tf.lookup.StaticVocabularyTable(vocab_init, num_oov_buckets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(1, 4), dtype=int64, numpy=array([[   24,    12,    13, 10053]])>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table.lookup(tf.constant([b\"This movie was faaaaaantastic\".split()]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating the Final Train and Test sets :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def encode_words(X_batch, y_batch):\n",
    "    return table.lookup(X_batch), y_batch\n",
    "\n",
    "train_set = datasets[\"train\"].repeat().batch(32).map(preprocess)\n",
    "train_set = train_set.map(encode_words).prefetch(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_set = datasets[\"test\"].batch(1000).map(preprocess)\n",
    "test_set = test_set.map(encode_words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(\n",
      "[[  24   13   31 ...    0    0    0]\n",
      " [   5   23   73 ...    0    0    0]\n",
      " [3454 6898    1 ...    0    0    0]\n",
      " ...\n",
      " [  24   12  122 ...    0    0    0]\n",
      " [1681 3980  451 ...    0    0    0]\n",
      " [3361 4259    5 ...    0    0    0]], shape=(32, 63), dtype=int64)\n",
      "tf.Tensor([0 0 0 1 1 1 0 0 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 1 1 0 0 0 1 0 0 0], shape=(32,), dtype=int64)\n"
     ]
    }
   ],
   "source": [
    "for X_batch, y_batch in train_set.take(1):\n",
    "    print(X_batch)\n",
    "    print(y_batch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building the Model :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "embed_size = 128\n",
    "model = keras.models.Sequential([\n",
    "    keras.layers.Embedding(vocab_size + num_oov_buckets, embed_size,\n",
    "                          mask_zero = True,\n",
    "                          input_shape = [None]),\n",
    "    keras.layers.GRU(4, return_sequences = True),\n",
    "    keras.layers.GRU(2),\n",
    "    keras.layers.Dense(1, activation = 'sigmoid')\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss = 'binary_crossentropy', optimizer=\"adam\", metrics = [\"accuracy\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training and Testing the Model :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "start = time.time()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/2\n",
      "781/781 [==============================] - 55s 63ms/step - loss: 0.6561 - accuracy: 0.5757\n",
      "Epoch 2/2\n",
      "781/781 [==============================] - 44s 56ms/step - loss: 0.4205 - accuracy: 0.8105\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fdde5d40220>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(train_set, steps_per_epoch=train_size//32, epochs=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time of execution :  99.370929479599\n"
     ]
    }
   ],
   "source": [
    "end=time.time()\n",
    "print(\"Time of execution : \", end-start)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25/25 [==============================] - 4s 99ms/step - loss: 0.5492 - accuracy: 0.7525\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.549174427986145, 0.7524799704551697]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(test_set)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}