DNN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DNN (Multi-Layer Perceptron)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Run the comment below only when using Google Colab\n",
    "# !pip install torch torchvision"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Google Colab에서 실행할 경우 위 주석을 해제하고 실행하면 됨. Colab에는 pytorch가 설치되어있지 않으므로 설치 필요."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torch.nn as nn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**torch**는 pytorch의 가장 상위에 있는 package로써 라이브러리에서 이것저것 불러오기 위해 import\n",
    "\n",
    "**torch.nn**은 모델을 정의할 때 사용하는 Class들을 포함하고 있음.\n",
    "\n",
    "**torchvision**은 Computer vision에서 사용하는 각종 테크닉들을 torch와 연동하여 구현한 라이브러리."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "from torchvision import datasets\n",
    "from torchvision import transforms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import datetime\n",
    "import os, sys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**numpy**는 python으로 data science를 할 때 가장 기본이 되는 라이브러리 중 하나. 각종 행렬 연산에 필요한 함수들을 다수 포함하고 있음.\n",
    "**datetime** 시간찍을려고...\n",
    "**os, sys** os로부터 정보를 얻고 싶을 때 쓰는 라이브러리"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from matplotlib.pyplot import imshow, imsave\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**matplotlib**는 python visualization library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "MODEL_NAME = 'DNN'\n",
    "DEVICE = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GPU가 있다면 GPU를 통해 학습을 가속화하고, 없으면 CPU로 학습하기 위해 device를 정해준다.\n",
    "\n",
    "**torch.cuda.is_avaliable()**은 GPU가 사용가능한지를 판단하는 함수"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class HelloDNN(nn.Module):\n",
    "    \"\"\"\n",
    "        Simple Discriminator w/ MLP\n",
    "    \"\"\"\n",
    "    def __init__(self, input_size=784, num_classes=10):\n",
    "        super(HelloDNN, self).__init__()\n",
    "        self.mlp = nn.Sequential(\n",
    "            nn.Linear(input_size, 64),\n",
    "            nn.Tanh(),\n",
    "            nn.Linear(64, 64),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(64, num_classes),\n",
    "            # nn.Softmax(),\n",
    "        )\n",
    "    \n",
    "    def forward(self, x):\n",
    "        y_ = x.view(x.size(0), -1) # (N, 784)\n",
    "        y_ = self.mlp(y_)\n",
    "        return y_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 모델 정의를 위한 코드\n",
    "\n",
    "**nn.Module**은 뉴럴넷 구현을 위한 base class. forward, parameter 등 모델을 만들고 사용할 때 필요한 부분들이 내부적으로 구현되어 있음.\n",
    "\n",
    "**__init__**은 python class의 constructor. 필요한 멤버변수들을 초기화하고, **nn.Sequential** or **nn.ModuleList**를 이용하여 모델의 구성을 정의한다.\n",
    "\n",
    "**forward**는 모델의 input을 받고 output을 return하는 함수."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = HelloDNN().to(DEVICE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "정의한 모델을 메모리에 올리는 작업."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "transform = transforms.Compose([transforms.ToTensor(),\n",
    "                                transforms.Normalize(mean=(0.5, 0.5, 0.5),\n",
    "                                std=(0.5, 0.5, 0.5))]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**transforms** torchvision에서 제공하는 transform 함수들이 있는 패키지.\n",
    "\n",
    "**ToTensor**는 numpy array를 torch tensor로 변환.\n",
    "\n",
    "**Normalize**는 다음과 같이 계산함. input[channel] = (input[channel] - mean[channel]) / std[channel]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mnist_train = datasets.MNIST(root='../data/', train=True, transform=transform, download=True)\n",
    "mnist_test = datasets.MNIST(root='../data/', train=False, transform=transform, download=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**datasets**에는 여러 데이터들에 대해 다운로드하고 처리하는 클래스가 내장되어 있음. [참고](https://pytorch.org/docs/stable/torchvision/datasets.html)\n",
    "\n",
    "root 폴더에 없을 시에 download하고, 앞서 정의한 transform에 따라 전처리 된 데이터를 return함."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "batch_size = 64"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_loader = DataLoader(dataset=mnist_train, batch_size=batch_size, shuffle=True, drop_last=True)\n",
    "test_loader = DataLoader(dataset=mnist_test, batch_size=100, shuffle=False, drop_last=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**DataLoader**는 pytorch에서 학습 시에 데이터를 배치 사이즈만큼씩 효율적으로 불러오도록 돕는 클래스. 잘 사용할수록 GPU의 사용률이 올라간다.\n",
    "\n",
    "**shuffle**: every epochs 마다 데이터의 순서를 랜덤하게 섞는다.\n",
    "\n",
    "**drop_last**: 데이터의 개수가 배치 사이즈로 나눠떨어지지 않는 경우, 마지막 배치를 버린다. 주로 학습시에만 사용."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "criterion = nn.CrossEntropyLoss()\n",
    "optim = torch.optim.Adam(model.parameters(), lr=0.001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**nn.CrossEntropyLoss**: Cross entropy를 계산하는 Loss. softmax가 내부적으로 수행된다.\n",
    "\n",
    "**optim.Adam**: optim에는 여러 optimizer가 있고, Adam Optimizer는 대표적으로 많이 사용된다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "max_epoch = 5\n",
    "step = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "최대 epoch 조정"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training\n",
    "\n",
    "첫번째 for문: 원하는 epoch만큼 반복\n",
    "\n",
    "두번째 for문: training datset에서 배치 사이즈 만큼씩 모두 샘플링 될 때까지 반복.\n",
    "\n",
    "**Line 2**: MNIST dataset은 DataLoader를 통해 image와 label을 return.\n",
    "\n",
    "**Line 4**: 각각 Device에 올린다 (GPU or CPU)\n",
    "\n",
    "**Line 5**: 모델에 이미지를 넣고 forward propagation 한다.\n",
    "\n",
    "**Line 7**: 결과값 y_hat과 실제 정답 y에 대한 loss를 계산한다.\n",
    "\n",
    "**zero_grad (Line 9)**: 모델의 gradient를 0으로 초기화한다.\n",
    "\n",
    "**backward (Line 10)**: loss를 계산하는 것까지 연결되어있는 graph를 따라 gradient를 계산한다.\n",
    "\n",
    "**step (Line 11)**: 계산된 gradient를 모두 parameter에 적용한다.\n",
    "\n",
    "**eval (Line 17)**: 모델을 evaluation mode로 바꿔준다 (dropout 조정, Batch normalization 조정 등)\n",
    "\n",
    "**torch.no_grad (Line 19)**: gradient를 계산하기 위해 추적하는 수고를 하지 않음\n",
    "\n",
    "**torch.max (Line 24)**: max value와 indices(즉, argmax)를 return.\n",
    "\n",
    "**train (Line 29)**: evaluation mode였던 모델을 train mode로 전환"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0/5, Step: 5000, Loss: 0.08296909183263779\n",
      "********** Test **********\n",
      "Step: 5000, Loss: 0.14294588565826416, Accuracy: 96.55 %\n",
      "Epoch: 0/5, Step: 5500, Loss: 0.19698578119277954\n",
      "Epoch: 1/5, Step: 6000, Loss: 0.19575947523117065\n",
      "********** Test **********\n",
      "Step: 6000, Loss: 0.20920392870903015, Accuracy: 96.85000000000001 %\n",
      "Epoch: 1/5, Step: 6500, Loss: 0.07467810809612274\n",
      "Epoch: 2/5, Step: 7000, Loss: 0.12162145972251892\n",
      "********** Test **********\n",
      "Step: 7000, Loss: 0.11932624876499176, Accuracy: 96.64 %\n",
      "Epoch: 3/5, Step: 7500, Loss: 0.12283851206302643\n",
      "Epoch: 3/5, Step: 8000, Loss: 0.09724417328834534\n",
      "********** Test **********\n",
      "Step: 8000, Loss: 0.10302124917507172, Accuracy: 95.98 %\n",
      "Epoch: 4/5, Step: 8500, Loss: 0.0469556525349617\n",
      "Epoch: 4/5, Step: 9000, Loss: 0.09951281547546387\n",
      "********** Test **********\n",
      "Step: 9000, Loss: 0.05420346185564995, Accuracy: 96.64 %\n"
     ]
    }
   ],
   "source": [
    "for epoch in range(max_epoch):\n",
    "    for idx, (images, labels) in enumerate(train_loader):\n",
    "        # Training Discriminator\n",
    "        x, y = images.to(DEVICE), labels.to(DEVICE) # (N, 1, 28, 28), (N, )\n",
    "        y_hat = model(x) # (N, 10)\n",
    "        \n",
    "        loss = criterion(y_hat, y)\n",
    "        \n",
    "        optim.zero_grad()\n",
    "        loss.backward()\n",
    "        optim.step()\n",
    "        \n",
    "        if step % 500 == 0:\n",
    "            print('Epoch: {}/{}, Step: {}, Loss: {}'.format(epoch, max_epoch, step, loss.item()))\n",
    "        \n",
    "        if step % 1000 == 0:\n",
    "            model.eval()\n",
    "            acc = 0.\n",
    "            with torch.no_grad():\n",
    "                for idx, (images, labels) in enumerate(test_loader):\n",
    "                    x, y = images.to(DEVICE), labels.to(DEVICE) # (N, 1, 28, 28), (N, )\n",
    "                    y_hat = model(x) # (N, 10)\n",
    "                    loss = criterion(y_hat, y)\n",
    "                    _, indices = torch.max(y_hat, dim=-1)\n",
    "                    acc += torch.sum(indices == y).item()\n",
    "            print('*'*20, 'Test', '*'*20)\n",
    "            print('Step: {}, Loss: {}, Accuracy: {} %'.format(step, loss.item(), acc/len(mnist_test)*100))\n",
    "            print('*'*46)\n",
    "            model.train()\n",
    "        step += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test and Visualize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "******************** Test ********************\n",
      "Step: 9370, Loss: 0.09149564802646637, Accuracy: 96.67999999999999 %\n",
      "**********************************************\n"
     ]
    }
   ],
   "source": [
    "# Test\n",
    "model.eval()\n",
    "acc = 0.\n",
    "with torch.no_grad():\n",
    "    for idx, (images, labels) in enumerate(test_loader):\n",
    "        x, y = images.to(DEVICE), labels.to(DEVICE) # (N, 1, 28, 28), (N, )\n",
    "        y_hat = model(x) # (N, 10)\n",
    "        loss = criterion(y_hat, y)\n",
    "        _, indices = torch.max(y_hat, dim=-1)\n",
    "        acc += torch.sum(indices == y).item()\n",
    "print('*'*20, 'Test', '*'*20)\n",
    "print('Step: {}, Loss: {}, Accuracy: {} %'.format(step, loss.item(), acc/len(mnist_test)*100))\n",
    "print('*'*46)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([1, 28, 28]), tensor(5))"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "idx = 7777 # 0 to 9999\n",
    "img, y = mnist_test[idx]\n",
    "img.shape, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1a1a3f335f8>"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADeNJREFUeJzt3X+MVfWZx/HPg4BGSiKs6YRQs4jBTdBYTCawiWTTdRXQ\nNIH+oRYTQyPpoFJc4mo0+oeYzUbTUKvyB8kYScFQCok/wCrbICFl12waRqP4qy2W0JQJDioGRBNw\n8Nk/5tAMMPd7L/eeX+PzfiWTufc899zz5GY+c86933Pu19xdAOIZU3UDAKpB+IGgCD8QFOEHgiL8\nQFCEHwiK8ANBEX4gKMIPBDW2zI2ZGacTAgVzd2vlcR3t+c1sgZn9ycw+MrOHOnkuAOWyds/tN7ML\nJP1Z0o2SDkraI2mxu3+QWIc9P1CwMvb8syV95O773f2kpN9IWtjB8wEoUSfhnyrpb8PuH8yWncHM\nesysz8z6OtgWgJwV/oGfu/dK6pU47AfqpJM9f7+ky4bd/162DMAo0En490iaYWaXm9l4ST+WtC2f\ntgAUre3DfncfNLOfSfqdpAskrXP393PrDECh2h7qa2tjvOcHClfKST4ARi/CDwRF+IGgCD8QFOEH\ngiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTh\nB4IqdYpuFGPcuHENaxdddFGh277hhhuS9fnz5zesLVu2LO92zrBu3bqGtYGBgeS6e/bsSdZfe+21\nZP3EiRPJeh2w5weCIvxAUIQfCIrwA0ERfiAowg8ERfiBoDqapdfMDkj6QtIpSYPu3t3k8czS24a5\nc+cm66tWrWpYu/766zvatll6wtcyZ3muk8cffzxZf+SRR0rq5FytztKbx0k+/+run+bwPABKxGE/\nEFSn4XdJr5vZm2bWk0dDAMrR6WH/XHfvN7PvStphZn90993DH5D9U+AfA1AzHe353b0/+31Y0kuS\nZo/wmF537272YSCAcrUdfjObYGYTT9+WNE/Se3k1BqBYnRz2d0l6KRsKGivp1+7+37l0BaBwbYff\n3fdL+n6OvYR1xRVXJOtr165N1q+66qo824Gko0ePJuuvvPJKSZ0Uh6E+ICjCDwRF+IGgCD8QFOEH\ngiL8QFAdXdJ73hvjkt623HLLLcn65s2bC9t2nS/pPXbsWLK+f//+tp97xYoVyfobb7zR9nMXrdVL\netnzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQTNFdA5MnT07Wly5dWlIn+du3b1/D2jvvvJNcd8eO\nHcn6J598kqy//PLLyXp07PmBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjG+Wtgzpw5yfq8efNK6uRc\np06dStbvv//+ZH3Tpk0NawMDA231hHyw5weCIvxAUIQfCIrwA0ERfiAowg8ERfiBoJqO85vZOkk/\nlHTY3a/Olk2WtFnSNEkHJN3q7p8X1+a3W52v17/99tuT9S1btpTUCfLWyp7/V5IWnLXsIUk73X2G\npJ3ZfQCjSNPwu/tuSUfOWrxQ0vrs9npJi3LuC0DB2n3P3+Xuh7LbH0vqyqkfACXp+Nx+d/fUHHxm\n1iOpp9PtAMhXu3v+ATObIknZ78ONHujuve7e7e7dbW4LQAHaDf82SUuy20skbc2nHQBlaRp+M9sk\n6f8k/ZOZHTSzpZKekHSjme2TdEN2H8AoYmXOr576bCCyxYsXJ+sbN24sqZNzff55+vSNI0fOHgg6\n06uvvtqwtmvXruS6W7dyQNkOd7dWHscZfkBQhB8IivADQRF+ICjCDwRF+IGgGOqrgZkzZybrq1ev\nTtYXLDj7osv8mKVHjTr5+xkcHEzWP/vss2T9+eefT9ZTQ4nbt29PrjuaMdQHIInwA0ERfiAowg8E\nRfiBoAg/EBThB4JinH8UGDs2/W1rK1eubFh79NFHk+tOmDAhWS9ynL9oqenFn3rqqeS6jz32WLJ+\n/PjxtnoqA+P8AJIIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvmDu+6665L1efPmJev33Xdfsj5mTOP9\ny8UXX5xct0pr1qxJ1u+9996SOjl/jPMDSCL8QFCEHwiK8ANBEX4gKMIPBEX4gaCajvOb2TpJP5R0\n2N2vzpatkvRTSZ9kD3vY3V9rujHG+cO55JJLGtbuuOOO5LrN6t3d3W311Iq9e/cm63PmzEnWT5w4\nkWc75yXPcf5fSRppVohfuvus7Kdp8AHUS9Pwu/tuSUdK6AVAiTp5z7/CzPaa2Tozm5RbRwBK0W74\n10qaLmmWpEOSftHogWbWY2Z9ZtbX5rYAFKCt8Lv7gLufcvdvJD0raXbisb3u3u3uxX06A+C8tRV+\nM5sy7O6PJL2XTzsAypL+TmhJZrZJ0g8kXWpmByU9KukHZjZLkks6IGlZgT0CKADX86O2urq6kvXd\nu3cn6zNmzMiznTNMnDgxWf/yyy8L23YzXM8PIInwA0ERfiAowg8ERfiBoAg/EFTTcX4U78orr0zW\n6/wV1/Pnz0/Wp06d2rDW7Ouvmw2XFTmctn379mS9ykt288KeHwiK8ANBEX4gKMIPBEX4gaAIPxAU\n4QeCYpw/M378+GT9pptuali76667Otr27NkNvwhJkjRpUnVfkWiWvjq02SXh/f39DWtbtmxJrnvP\nPfck67NmzUrWO7Fr165kfXBwsLBtl4U9PxAU4QeCIvxAUIQfCIrwA0ERfiAowg8EFearu8eOTZ/S\n8PTTTyfrd999d57tjBqdjvPX1Zo1a5L1Bx54IFk/efJknu3kiq/uBpBE+IGgCD8QFOEHgiL8QFCE\nHwiK8ANBNb2e38wuk7RBUpckl9Tr7k+b2WRJmyVNk3RA0q3u/nlxrXZm+fLlyXrUcfxvs2eeeaZh\n7cEHH0yuW+dx/Ly0sucflPQf7j5T0j9LWm5mMyU9JGmnu8+QtDO7D2CUaBp+dz/k7m9lt7+Q9KGk\nqZIWSlqfPWy9pEVFNQkgf+f1nt/Mpkm6VtIfJHW5+6Gs9LGG3hYAGCVa/g4/M/uOpBckrXT3Y8PP\n+XZ3b3Tevpn1SOrptFEA+Wppz29m4zQU/I3u/mK2eMDMpmT1KZIOj7Suu/e6e7e7d+fRMIB8NA2/\nDe3in5P0obs/Oay0TdKS7PYSSVvzbw9AUVo57L9O0h2S3jWzt7NlD0t6QtIWM1sq6a+Sbi2mxXwM\nDAxU3QJy1uyy3NRw3rdhiu1ONQ2/u/+vpEbXB/9bvu0AKAtn+AFBEX4gKMIPBEX4gaAIPxAU4QeC\nCvPV3WPGpP/P9fb2Jut33nlnnu2MGp1+dffRo0cb1jZs2JBcd/Pmzcl6X19fsh7hstyR8NXdAJII\nPxAU4QeCIvxAUIQfCIrwA0ERfiCoMOP8zVx44YXJ+qJFjb+f9Jprrkmue9tttyXr06dPT9aLtHr1\n6mT966+/TtaPHz+erKemPv/qq6+S66I9jPMDSCL8QFCEHwiK8ANBEX4gKMIPBEX4gaAY5we+ZRjn\nB5BE+IGgCD8QFOEHgiL8QFCEHwiK8ANBNQ2/mV1mZrvM7AMze9/M/j1bvsrM+s3s7ezn5uLbBZCX\npif5mNkUSVPc/S0zmyjpTUmLJN0q6bi7p78N4szn4iQfoGCtnuQztoUnOiTpUHb7CzP7UNLUztoD\nULXzes9vZtMkXSvpD9miFWa218zWmdmkBuv0mFmfmaXnVgJQqpbP7Tez70j6vaT/cvcXzaxL0qeS\nXNJ/auitQXJCOw77geK1etjfUvjNbJyk30r6nbs/OUJ9mqTfuvvVTZ6H8AMFy+3CHhuapvU5SR8O\nD372QeBpP5L03vk2CaA6rXzaP1fS/0h6V9I32eKHJS2WNEtDh/0HJC3LPhxMPRd7fqBguR7254Xw\nA8Xjen4ASYQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgmn6B\nZ84+lfTXYfcvzZbVUV17q2tfEr21K8/e/rHVB5Z6Pf85Gzfrc/fuyhpIqGtvde1Lord2VdUbh/1A\nUIQfCKrq8PdWvP2UuvZW174kemtXJb1V+p4fQHWq3vMDqEgl4TezBWb2JzP7yMweqqKHRszsgJm9\nm808XOkUY9k0aIfN7L1hyyab2Q4z25f9HnGatIp6q8XMzYmZpSt97eo243Xph/1mdoGkP0u6UdJB\nSXskLXb3D0ptpAEzOyCp290rHxM2s3+RdFzShtOzIZnZzyUdcfcnsn+ck9z9wZr0tkrnOXNzQb01\nmln6J6rwtctzxus8VLHnny3pI3ff7+4nJf1G0sIK+qg9d98t6chZixdKWp/dXq+hP57SNeitFtz9\nkLu/ld3+QtLpmaUrfe0SfVWiivBPlfS3YfcPql5Tfruk183sTTPrqbqZEXQNmxnpY0ldVTYzgqYz\nN5fprJmla/PatTPjdd74wO9cc919lqSbJC3PDm9ryYfes9VpuGatpOkamsbtkKRfVNlMNrP0C5JW\nuvux4bUqX7sR+qrkdasi/P2SLht2/3vZslpw9/7s92FJL2nobUqdDJyeJDX7fbjifv7O3Qfc/ZS7\nfyPpWVX42mUzS78gaaO7v5gtrvy1G6mvql63KsK/R9IMM7vczMZL+rGkbRX0cQ4zm5B9ECMzmyBp\nnuo3+/A2SUuy20skba2wlzPUZebmRjNLq+LXrnYzXrt76T+SbtbQJ/5/kfRIFT006Gu6pHeyn/er\n7k3SJg0dBn6toc9Glkr6B0k7Je2T9LqkyTXq7XkNzea8V0NBm1JRb3M1dEi/V9Lb2c/NVb92ib4q\ned04ww8Iig/8gKAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8E9f/KHJ5wsWXfUgAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a1a3eb3438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "imshow(img[0], cmap='gray')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([5], device='cuda:0')"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample = img.to(DEVICE)\n",
    "out = model(sample)\n",
    "_, idx = out.max(dim=-1)\n",
    "idx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Saving params.\n",
    "torch.save(model.state_dict(), 'model.pkl')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}