17.Pytorch Transfer learning with Caltech101.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "**Caltech 101 Dataset Image Classification with Pytorch Pretrained Model Resnet34**\r\n",
    "---\r\n",
    "\r\n",
    "* Introduction of Caltech Dataset 101\r\n",
    "* Approach to Train a Model\r\n",
    "    * Neural Network(Resnet34)\r\n",
    "    * Tools and Libraries\r\n",
    "    * Directory Structure"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**Dataset Download : [Caltech101](http://www.vision.caltech.edu/Image_Datasets/Caltech101/#Download)** \r\n",
    "---\r\n",
    "![](https://debuggercafe.com/wp-content/uploads/2020/03/caltech101exp-min.png)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**Total Images : 8677 Images**\r\n",
    "---\r\n",
    "* If we ignore the BACKGROUND_Google tag and its images, then we have 8677 images in total. Now if you have been learning in depth for a while, you know that these are not enough images for very high precision. Still, we will do our best.\r\n",
    "* Second issue with datset is distribution, which is not upto the mark as illustrated in Image.\r\n",
    "![Bar plot showing the distribution of number of images in each category in the Caltech101 dataset.](https://debuggercafe.com/wp-content/uploads/2020/03/dist.png)  \r\n",
    "Bar plot showing the distribution of number of images in each category in the Caltech101 dataset.\r\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**1. Load Modules**\r\n",
    "---"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "source": [
    "!pip install pretrainedmodels\r\n",
    "!pip install imutils"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Requirement already satisfied: pretrainedmodels in c:\\users\\shiva\\miniconda3\\envs\\pytorch19\\lib\\site-packages (0.7.4)\n",
      "Requirement already satisfied: torchvision in c:\\users\\shiva\\miniconda3\\envs\\pytorch19\\lib\\site-packages (from pretrainedmodels) (0.10.0)\n",
      "Requirement already satisfied: tqdm in c:\\users\\shiva\\miniconda3\\envs\\pytorch19\\lib\\site-packages (from pretrainedmodels) (4.61.2)\n",
      "Requirement already satisfied: munch in c:\\users\\shiva\\miniconda3\\envs\\pytorch19\\lib\\site-packages (from pretrainedmodels) (2.5.0)\n",
      "Requirement already satisfied: torch in c:\\users\\shiva\\miniconda3\\envs\\pytorch19\\lib\\site-packages (from pretrainedmodels) (1.9.0)\n",
      "Requirement already satisfied: six in c:\\users\\shiva\\miniconda3\\envs\\pytorch19\\lib\\site-packages (from munch->pretrainedmodels) (1.16.0)\n",
      "Requirement already satisfied: typing_extensions in c:\\users\\shiva\\miniconda3\\envs\\pytorch19\\lib\\site-packages (from torch->pretrainedmodels) (3.10.0.0)\n",
      "Requirement already satisfied: numpy in c:\\users\\shiva\\miniconda3\\envs\\pytorch19\\lib\\site-packages (from torchvision->pretrainedmodels) (1.21.0)\n",
      "Requirement already satisfied: pillow>=5.3.0 in c:\\users\\shiva\\miniconda3\\envs\\pytorch19\\lib\\site-packages (from torchvision->pretrainedmodels) (8.3.1)\n",
      "Requirement already satisfied: colorama in c:\\users\\shiva\\miniconda3\\envs\\pytorch19\\lib\\site-packages (from tqdm->pretrainedmodels) (0.4.4)\n",
      "Requirement already satisfied: imutils in c:\\users\\shiva\\miniconda3\\envs\\pytorch19\\lib\\site-packages (0.5.4)\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "import matplotlib.pyplot as plt\r\n",
    "%matplotlib inline\r\n",
    "import matplotlib\r\n",
    "import joblib\r\n",
    "import cv2\r\n",
    "import os\r\n",
    "import time\r\n",
    "import random\r\n",
    "import pretrainedmodels\r\n",
    "import numpy as np\r\n",
    "\r\n",
    "from imutils import paths\r\n",
    "from sklearn.preprocessing import LabelEncoder\r\n",
    "from sklearn.model_selection import train_test_split\r\n",
    "from tqdm import tqdm\r\n",
    "\r\n",
    "# Load torch...!!!\r\n",
    "import torch\r\n",
    "import torch.nn as nn\r\n",
    "import torch.nn.functional as F\r\n",
    "import torch.optim as optim\r\n",
    "from torch.utils.data import Dataset, DataLoader\r\n",
    "\r\n",
    "# Load torchvision ...!!!\r\n",
    "from torchvision import transforms\r\n",
    "\r\n",
    "'''SEED Everything'''\r\n",
    "def seed_everything(SEED=42):\r\n",
    "    random.seed(SEED)\r\n",
    "    np.random.seed(SEED)\r\n",
    "    torch.manual_seed(SEED)\r\n",
    "    torch.cuda.manual_seed(SEED)\r\n",
    "    torch.cuda.manual_seed_all(SEED)\r\n",
    "    torch.backends.cudnn.benchmark = True # keep True if all the input have same size.\r\n",
    "SEED=42\r\n",
    "seed_everything(SEED=SEED)\r\n",
    "'''SEED Everything'''"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "'SEED Everything'"
      ]
     },
     "metadata": {},
     "execution_count": 5
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Going over some of the important imports in the above code block.\r\n",
    "\r\n",
    "* `torch.nn` will help us access the neural network layers in the PyTorch library.\r\n",
    "* `torch.optim` to access all the optimizer functions in PyTorch.\r\n",
    "* `pretrainedmodels` to access all the pre-trained models like ResNet34 and many more. We installed this library in one of the previous sections.\r\n",
    "* Using `torchvision.transforms` we can apply transforms to our image like normalization and resizing.\r\n",
    "* `DataLoader` and `Dataset` from the torchvision.transforms will help us to create our own custom image dataset module and iterable data loaders.\r\n",
    "* `cv2` to read images in the dataset."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**2. Define the device, Epochs, and BatchSize**\r\n",
    "---"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\") # GPU\r\n",
    "epochs = 5 # Number of epochs\r\n",
    "BS = 16 # Batch size"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**3. Preparing the Labels and Images**\r\n",
    "---"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "source": [
    "image_paths = list(paths.list_images('./101_ObjectCategories'))\r\n",
    "\r\n",
    "data = []\r\n",
    "labels = []\r\n",
    "for img_path in tqdm(image_paths):\r\n",
    "    label = img_path.split(os.path.sep)[-2]\r\n",
    "    if label == \"BACKGROUND_Google\":\r\n",
    "        continue\r\n",
    "    img = cv2.imread(img_path)\r\n",
    "    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\r\n",
    "    \r\n",
    "    data.append(img)\r\n",
    "    labels.append(label)\r\n",
    "    \r\n",
    "data = np.array(data)\r\n",
    "labels = np.array(labels)\r\n",
    "    "
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "100%|██████████| 9144/9144 [01:10<00:00, 130.34it/s]\n",
      "C:\\Users\\SHIVA\\AppData\\Local\\Temp/ipykernel_12256/712078175.py:15: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n",
      "  data = np.array(data)\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**4. Label Encoder**\r\n",
    "---"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "source": [
    "lb = LabelEncoder()\r\n",
    "labels = lb.fit_transform(labels)\r\n",
    "print(f\"Total Number of Classes: {len(lb.classes_)}\")"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Total Number of Classes: 101\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**5.Define the Transforms**\r\n",
    "---"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "source": [
    "train_transforms = transforms.Compose([\r\n",
    "    transforms.ToPILImage(),\r\n",
    "    transforms.Resize((224, 224)),\r\n",
    "    transforms.ToTensor(),\r\n",
    "    transforms.Normalize(mean = [0.485,0.456,0.406], std=[0.229,0.224,0.225]),\r\n",
    "])\r\n",
    "\r\n",
    "val_transform = transforms.Compose([\r\n",
    "    transforms.ToPILImage(),\r\n",
    "    transforms.Resize((224, 224)),\r\n",
    "    transforms.ToTensor(),\r\n",
    "    transforms.Normalize(mean = [0.485,0.456,0.406], std=[0.229,0.224,0.225]),\r\n",
    "])    "
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**6. Divide the Train,Validation and Test split**\r\n",
    "---"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "source": [
    "# divide the data into train, validation, and test set\r\n",
    "(X, x_val , Y, y_val) = train_test_split(data, labels, test_size=0.2,  stratify=labels,random_state=42)\r\n",
    "(x_train, x_test, y_train, y_test) = train_test_split(X, Y, test_size=0.25, random_state=42)\r\n",
    "print(f\"x_train examples: {x_train.shape}\\nx_test examples: {x_test.shape}\\nx_val examples: {x_val.shape}\")"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "x_train examples: (5205,)\n",
      "x_test examples: (1736,)\n",
      "x_val examples: (1736,)\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**7. Creating Custom Dataset**\r\n",
    "---"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "source": [
    "# custom dataset class\r\n",
    "class CustomDataset(Dataset):\r\n",
    "    def __init__(self, images, labels= None, transforms = None):\r\n",
    "        self.labels = labels\r\n",
    "        self.images = images\r\n",
    "        self.transforms = transforms\r\n",
    "        \r\n",
    "    def __len__(self):\r\n",
    "        return len(self.images)\r\n",
    "    \r\n",
    "    def __getitem__(self, index):\r\n",
    "        data = self.images[index][:]\r\n",
    "        \r\n",
    "        if self.transforms:\r\n",
    "            data = self.transforms(data)\r\n",
    "            \r\n",
    "        if self.y is not None:\r\n",
    "            return (data, self.labels[index])\r\n",
    "        else:\r\n",
    "            return data\r\n",
    "\r\n",
    "train_data = CustomDataset(x_train, y_train, train_transforms)\r\n",
    "val_data = CustomDataset(x_val, y_val, val_transform)\r\n",
    "test_data = CustomDataset(x_test, y_test, val_transform)       \r\n",
    "\r\n",
    "trainLoader = DataLoader(train_data, batch_size=BS, shuffle=True, num_workers=4)\r\n",
    "valLoader = DataLoader(val_data, batch_size=BS, shuffle=True, num_workers=4)\r\n",
    "testLoader = DataLoader(test_data, batch_size=BS, shuffle=True, num_workers=4) \r\n"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**8.ResNet34 Model**\r\n",
    "---"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "source": [
    "# the resnet34 model\r\n",
    "class ResNet34(nn.Module):\r\n",
    "    def __init__(self, pretrained):\r\n",
    "        super(ResNet34, self).__init__()\r\n",
    "        if pretrained is True:\r\n",
    "            self.model = pretrainedmodels.__dict__['resnet34'](pretrained='imagenet')\r\n",
    "        else:\r\n",
    "            self.model = pretrainedmodels.__dict__['resnet34'](pretrained = None)\r\n",
    "        # change the classification layer\r\n",
    "        self.l0= nn.Linear(512, len(lb.classes_))\r\n",
    "        self.dropout = nn.Dropout2d(0.4)\r\n",
    "        \r\n",
    "    def forward(self, x):\r\n",
    "        # get the batch size only, ignore(c, h, w)\r\n",
    "        batch, _, _. _ = x.shape\r\n",
    "        x = self.model.features(x)\r\n",
    "        x = F.adaptive_avg_pool2d(x, 1).reshape(batch, -1)\r\n",
    "        x = self.dropout(x)\r\n",
    "        l0 = self.l0(x)\r\n",
    "        return l0\r\n",
    "\r\n",
    "model = ResNet34(pretrained=True).to(device)\r\n",
    "print(model)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "ResNet34(\n",
      "  (model): ResNet(\n",
      "    (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n",
      "    (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "    (relu): ReLU(inplace=True)\n",
      "    (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n",
      "    (layer1): Sequential(\n",
      "      (0): BasicBlock(\n",
      "        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "      (1): BasicBlock(\n",
      "        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "      (2): BasicBlock(\n",
      "        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "    )\n",
      "    (layer2): Sequential(\n",
      "      (0): BasicBlock(\n",
      "        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (downsample): Sequential(\n",
      "          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
      "          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        )\n",
      "      )\n",
      "      (1): BasicBlock(\n",
      "        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "      (2): BasicBlock(\n",
      "        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "      (3): BasicBlock(\n",
      "        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "    )\n",
      "    (layer3): Sequential(\n",
      "      (0): BasicBlock(\n",
      "        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (downsample): Sequential(\n",
      "          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
      "          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        )\n",
      "      )\n",
      "      (1): BasicBlock(\n",
      "        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "      (2): BasicBlock(\n",
      "        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "      (3): BasicBlock(\n",
      "        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "      (4): BasicBlock(\n",
      "        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "      (5): BasicBlock(\n",
      "        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "    )\n",
      "    (layer4): Sequential(\n",
      "      (0): BasicBlock(\n",
      "        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (downsample): Sequential(\n",
      "          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
      "          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        )\n",
      "      )\n",
      "      (1): BasicBlock(\n",
      "        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "      (2): BasicBlock(\n",
      "        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "        (relu): ReLU(inplace=True)\n",
      "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      )\n",
      "    )\n",
      "    (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))\n",
      "    (fc): None\n",
      "    (last_linear): Linear(in_features=512, out_features=1000, bias=True)\n",
      "  )\n",
      "  (l0): Linear(in_features=512, out_features=101, bias=True)\n",
      "  (dropout): Dropout2d(p=0.4, inplace=False)\n",
      ")\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "source": [
    "from torchsummary import summary\r\n",
    "print(summary(model, input_size=(3, 224, 224)))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "=================================================================\n",
      "Layer (type:depth-idx)                   Param #\n",
      "=================================================================\n",
      "├─ResNet: 1-1                            --\n",
      "|    └─Conv2d: 2-1                       9,408\n",
      "|    └─BatchNorm2d: 2-2                  128\n",
      "|    └─ReLU: 2-3                         --\n",
      "|    └─MaxPool2d: 2-4                    --\n",
      "|    └─Sequential: 2-5                   --\n",
      "|    |    └─BasicBlock: 3-1              73,984\n",
      "|    |    └─BasicBlock: 3-2              73,984\n",
      "|    |    └─BasicBlock: 3-3              73,984\n",
      "|    └─Sequential: 2-6                   --\n",
      "|    |    └─BasicBlock: 3-4              230,144\n",
      "|    |    └─BasicBlock: 3-5              295,424\n",
      "|    |    └─BasicBlock: 3-6              295,424\n",
      "|    |    └─BasicBlock: 3-7              295,424\n",
      "|    └─Sequential: 2-7                   --\n",
      "|    |    └─BasicBlock: 3-8              919,040\n",
      "|    |    └─BasicBlock: 3-9              1,180,672\n",
      "|    |    └─BasicBlock: 3-10             1,180,672\n",
      "|    |    └─BasicBlock: 3-11             1,180,672\n",
      "|    |    └─BasicBlock: 3-12             1,180,672\n",
      "|    |    └─BasicBlock: 3-13             1,180,672\n",
      "|    └─Sequential: 2-8                   --\n",
      "|    |    └─BasicBlock: 3-14             3,673,088\n",
      "|    |    └─BasicBlock: 3-15             4,720,640\n",
      "|    |    └─BasicBlock: 3-16             4,720,640\n",
      "|    └─AdaptiveAvgPool2d: 2-9            --\n",
      "|    └─Linear: 2-10                      513,000\n",
      "├─Linear: 1-2                            51,813\n",
      "├─Dropout2d: 1-3                         --\n",
      "=================================================================\n",
      "Total params: 21,849,485\n",
      "Trainable params: 21,849,485\n",
      "Non-trainable params: 0\n",
      "=================================================================\n",
      "=================================================================\n",
      "Layer (type:depth-idx)                   Param #\n",
      "=================================================================\n",
      "├─ResNet: 1-1                            --\n",
      "|    └─Conv2d: 2-1                       9,408\n",
      "|    └─BatchNorm2d: 2-2                  128\n",
      "|    └─ReLU: 2-3                         --\n",
      "|    └─MaxPool2d: 2-4                    --\n",
      "|    └─Sequential: 2-5                   --\n",
      "|    |    └─BasicBlock: 3-1              73,984\n",
      "|    |    └─BasicBlock: 3-2              73,984\n",
      "|    |    └─BasicBlock: 3-3              73,984\n",
      "|    └─Sequential: 2-6                   --\n",
      "|    |    └─BasicBlock: 3-4              230,144\n",
      "|    |    └─BasicBlock: 3-5              295,424\n",
      "|    |    └─BasicBlock: 3-6              295,424\n",
      "|    |    └─BasicBlock: 3-7              295,424\n",
      "|    └─Sequential: 2-7                   --\n",
      "|    |    └─BasicBlock: 3-8              919,040\n",
      "|    |    └─BasicBlock: 3-9              1,180,672\n",
      "|    |    └─BasicBlock: 3-10             1,180,672\n",
      "|    |    └─BasicBlock: 3-11             1,180,672\n",
      "|    |    └─BasicBlock: 3-12             1,180,672\n",
      "|    |    └─BasicBlock: 3-13             1,180,672\n",
      "|    └─Sequential: 2-8                   --\n",
      "|    |    └─BasicBlock: 3-14             3,673,088\n",
      "|    |    └─BasicBlock: 3-15             4,720,640\n",
      "|    |    └─BasicBlock: 3-16             4,720,640\n",
      "|    └─AdaptiveAvgPool2d: 2-9            --\n",
      "|    └─Linear: 2-10                      513,000\n",
      "├─Linear: 1-2                            51,813\n",
      "├─Dropout2d: 1-3                         --\n",
      "=================================================================\n",
      "Total params: 21,849,485\n",
      "Trainable params: 21,849,485\n",
      "Non-trainable params: 0\n",
      "=================================================================\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**9.Calculate the Loss and Optimization Method**\r\n",
    "---\r\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "source": [
    "# loss function\r\n",
    "criterion = nn.CrossEntropyLoss()\r\n",
    "\r\n",
    "# optimizer\r\n",
    "optimizer = optim.Adam(model.parameters(), lr = 1e-4)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**10.Model Training**\r\n",
    "---"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "source": [
    "# training function\r\n",
    "train_loss , train_accuracy = [], []\r\n",
    "for epoch in range(epochs):\r\n",
    "    print('Training')\r\n",
    "    model.train()\r\n",
    "    running_loss = 0.0\r\n",
    "    running_correct = 0\r\n",
    "    for data in tqdm(trainLoader):\r\n",
    "        tqdm.set_description(f\"Epoch {epoch+1}/{epochs}\")\r\n",
    "        data, target = data[0].to(device), data[1].to(device)\r\n",
    "        optimizer.zero_grad()\r\n",
    "        outputs = model(data)\r\n",
    "        loss = criterion(outputs, torch.max(target, 1)[1])\r\n",
    "        running_loss += loss.item()\r\n",
    "        _, preds = torch.max(outputs.data, 1)\r\n",
    "        running_correct += (preds == torch.max(target, 1)[1]).sum().item()\r\n",
    "        loss.backward()\r\n",
    "        optimizer.step()\r\n",
    "        tqdm.set_description(f\"Loss: {loss.item()}\")\r\n",
    "        \r\n",
    "    loss = running_loss/len(trainLoader.dataset)\r\n",
    "    accuracy = 100. * running_correct/len(trainLoader.dataset)\r\n",
    "    \r\n",
    "    print(f\"Train Loss: {loss:.4f}, Train Acc: {accuracy:.2f}\")"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Training\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "  0%|          | 0/326 [00:00<?, ?it/s]"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "#validation function\r\n",
    "def validate(model, dataloader):\r\n",
    "    print('Validating')\r\n",
    "    model.eval()\r\n",
    "    running_loss = 0.0\r\n",
    "    running_correct = 0\r\n",
    "    with torch.no_grad():\r\n",
    "        for i, data in tqdm(enumerate(dataloader), total=int(len(val_data)/dataloader.batch_size)):\r\n",
    "            data, target = data[0].to(device), data[1].to(device)\r\n",
    "            outputs = model(data)\r\n",
    "            loss = criterion(outputs, torch.max(target, 1)[1])\r\n",
    "            \r\n",
    "            running_loss += loss.item()\r\n",
    "            _, preds = torch.max(outputs.data, 1)\r\n",
    "            running_correct += (preds == torch.max(target, 1)[1]).sum().item()\r\n",
    "        \r\n",
    "        loss = running_loss/len(dataloader.dataset)\r\n",
    "        accuracy = 100. * running_correct/len(dataloader.dataset)\r\n",
    "        print(f'Val Loss: {loss:.4f}, Val Acc: {accuracy:.2f}')\r\n",
    "        \r\n",
    "        return loss, accuracy"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "def test(model, dataloader):\r\n",
    "    correct = 0\r\n",
    "    total = 0\r\n",
    "    with torch.no_grad():\r\n",
    "        for data in testLoader:\r\n",
    "            inputs, target = data[0].to(device), data[1].to(device)\r\n",
    "            outputs = model(inputs)\r\n",
    "            _, predicted = torch.max(outputs.data, 1)\r\n",
    "            total += target.size(0)\r\n",
    "            correct += (predicted == torch.max(target, 1)[1]).sum().item()\r\n",
    "    return correct, total"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**11.Model Training**\r\n",
    "---"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "train_loss , train_accuracy = [], []\r\n",
    "val_loss , val_accuracy = [], []\r\n",
    "print(f\"Training on {len(train_data)} examples, validating on {len(val_data)} examples...\")\r\n",
    "start = time.time()\r\n",
    "for epoch in range(epochs):\r\n",
    "    print(f\"Epoch {epoch+1} of {epochs}\")\r\n",
    "    train_epoch_loss, train_epoch_accuracy = fit(model, trainLoader)\r\n",
    "    val_epoch_loss, val_epoch_accuracy = validate(model, valLoader)\r\n",
    "    train_loss.append(train_epoch_loss)\r\n",
    "    train_accuracy.append(train_epoch_accuracy)\r\n",
    "    val_loss.append(val_epoch_loss)\r\n",
    "    val_accuracy.append(val_epoch_accuracy)\r\n",
    "end = time.time()\r\n",
    "print((end-start)/60, 'minutes')"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# accuracy plots\r\n",
    "plt.figure(figsize=(10, 7))\r\n",
    "plt.subplot(121)\r\n",
    "plt.plot(train_accuracy, color='green', label='train accuracy')\r\n",
    "plt.plot(val_accuracy, color='blue', label='validataion accuracy')\r\n",
    "plt.xlabel('Epochs')\r\n",
    "plt.ylabel('Accuracy')\r\n",
    "plt.legend()\r\n",
    "# plt.savefig('../outputs/plots/accuracy.png')\r\n",
    "# loss plots\r\n",
    "plt.subplot(122)\r\n",
    "plt.plot(train_loss, color='orange', label='train loss')\r\n",
    "plt.plot(val_loss, color='red', label='validataion loss')\r\n",
    "plt.xlabel('Epochs')\r\n",
    "plt.ylabel('Loss')\r\n",
    "plt.legend()\r\n",
    "# plt.savefig('../outputs/plots/loss.png')\r\n",
    "plt.show()"
   ],
   "outputs": [],
   "metadata": {}
  }
 ],
 "metadata": {
  "orig_nbformat": 4,
  "language_info": {
   "name": "python",
   "version": "3.8.0",
   "mimetype": "text/x-python",
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "pygments_lexer": "ipython3",
   "nbconvert_exporter": "python",
   "file_extension": ".py"
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.8.0 64-bit ('pytorch19': conda)"
  },
  "interpreter": {
   "hash": "884a8f81666a19c0851426c83cd6eaa7b212468ad852fb3caa21591c98d4369f"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}