docs/MNIST_tf/mnist_tf_2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1 style=\"color:Blue;\">MNIST handwritten digit classification with TensorFlow<br>\n",
    "    Part 2 - Preparing for deployment</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>In Part 2 of our tutorial, we will use the Xilinx&trade; DNNDK toolsuite to convert our trained floating-point model into instructions and data for the DPU.\n",
    "<br><br>\n",
    "There are 4 main steps:\n",
    "    <ul>\n",
    "      <li>Freeze the floating model (often referred to as 'freezing the graph')\n",
    "      <li>Download images for quantization\n",
    "      <li>Quantize the floating-point model using DECENT_Q\n",
    "      <li>Compile the quantized model using DNNC          \n",
    "    </UL>\n",
    "<br>\n",
    "We will not be using Pruning in this tutorial.\n",
    "</b>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 style=\"color:Blue;\">Freeze the Floating-Point Model</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Now that we have saved the trained parameters of our network as a checkpoint and graph, we need to 'freeze' it by converting all the variables into constants and stripping out the training nodes to leave just the nodes that we need for deployment.\n",
    "<br><br>\n",
    "Luckily, TensorFlow provides a script called 'freeze_graph.py' which will do this for us..</b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# delete previous results\n",
    "!rm -rf ./freeze\n",
    "!mkdir ./freeze\n",
    "\n",
    "!freeze_graph --input_graph=./chkpts/training_graph.pb \\\n",
    "              --input_checkpoint=./chkpts/float_model.ckpt \\\n",
    "              --input_binary=true \\\n",
    "              --output_graph=./freeze/frozen_graph.pb \\\n",
    "              --output_node_names=dense_1/BiasAdd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 style=\"color:Blue;\">Quantize the floating-point model</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>This is the first step that involves the Xilinx&trade; DNNDK toolkit. We will now quantize the floating point model by running DECENT_Q.\n",
    "<br>\n",
    "<b>The quantization process has a calibration phase which requires approximately 1000 images files in a format that is compatible with openCV.\n",
    "<br>\n",
    "This section of python code will fetch the MNIST dataset as numpy arrays and the convert the test dataset (10k arrays) into 10k .png image files and write them into a separate folder.\n",
    "<br>\n",
    "It will also create a text file in that same folder which lists the images - this is also required for calibration.\n",
    "</b>\n",
    "</b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import shutil\n",
    "import numpy as np\n",
    "import cv2\n",
    "\n",
    "from keras.datasets import mnist\n",
    "\n",
    "SCRIPT_DIR = os.getcwd()\n",
    "CALIB_DIR = os.path.join(SCRIPT_DIR, 'calib_dir')\n",
    "IMAGE_LIST_FILE = 'calib_list.txt'\n",
    "\n",
    "if (os.path.exists(CALIB_DIR)):\n",
    "    shutil.rmtree(CALIB_DIR)\n",
    "os.makedirs(CALIB_DIR)\n",
    "print('Directory', CALIB_DIR, 'created')\n",
    "\n",
    "# get MNIST datasets as numpy arrays\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "\n",
    "# create file for list of calibration images\n",
    "f = open(os.path.join(CALIB_DIR, IMAGE_LIST_FILE), 'w')\n",
    "\n",
    "# use openCV to save .png images\n",
    "for i in range(len(x_test)):\n",
    "    cv2.imwrite(os.path.join(CALIB_DIR,'x_test_'+str(i)+'.png'), x_test[i])\n",
    "    f.write('x_test_'+str(i)+'.png\\n')\n",
    "\n",
    "f.close()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Now we run the decent_q quantizer tool.\n",
    "<br>\n",
    "Users are encouraged to refer to the DECENT_Q User Guide to see the complete list of command options.\n",
    "</b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# delete previous results\n",
    "!rm -rf ./quantize_results\n",
    "\n",
    "!decent_q quantize \\\n",
    "  --input_frozen_graph ./freeze/frozen_graph.pb \\\n",
    "  --input_nodes images_in \\\n",
    "  --input_shapes ?,28,28,1 \\\n",
    "  --output_nodes dense_1/BiasAdd \\\n",
    "  --method 1 \\\n",
    "  --input_fn graph_input_fn.calib_input \\\n",
    "  --gpu 0 \\\n",
    "  --calib_iter 200\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>The quantize_results folder will contain two .pb files - quantize_eval_model.pb which we use to verify the quality of our now quantized model and deploy_model.pb which is the .pb file that will be passed to the DNNC compiler.</b>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 style=\"color:Blue;\">Evaluate Quantized Model</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>This is an optional step that can be skipped but is very useful in understanding how much the quantization process has effected the accuracy of our classifier.\n",
    "<br>\n",
    "The evaluation process uses the quantized evaluation model (quantize_eval_model.pb) which was generated during the quantization process.\n",
    "<br>\n",
    "The top-1 and top-5 accuracies are calculated - the top-1 accuracy should be compared to the final accuracy calculated during training and evaluation to understand how much accuracy was lost to quantization.\n",
    "</b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python eval_graph.py \\\n",
    "  --graph ./quantize_results/quantize_eval_model.pb \\\n",
    "  --input_node images_in \\\n",
    "  --output_node dense_1/BiasAdd \\\n",
    "  --gpu 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 style=\"color:Blue;\">Compile the quantized model</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>\n",
    "The final step is to use the DNNC compiler which will parse the quantized model and generate one or more .ELF format files which are to be integrated into the final hardware/software platform.\n",
    "</b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# delete previous results\n",
    "!rm -rf ./compile\n",
    "\n",
    "!dnnc \\\n",
    "   --parser=tensorflow \\\n",
    "   --frozen_pb=./quantize_results/deploy_model.pb \\\n",
    "   --dpu=1152FA \\\n",
    "   --cpu_arch=arm64 \\\n",
    "   --output_dir=compile \\\n",
    "   --save_kernel \\\n",
    "   --mode normal \\\n",
    "   --net_name=mnist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>The 'compile' folder should contain the .elf file for execution on the DPU.</b>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}