imagenet_pretrained.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Running Pretrained ImageNet: the Easy way\n",
    "=========================================\n",
    "\n",
    "For easier use of pretrained models, we provide a wrapper specifically written for the case of ImageNet, so one can take an image and directly compute features or predictions from them. Both Python and Matlab wrappers are provided. We will describe the use of the Python wrapper here, and the Matlab wrapper usage is very similar.\n",
    "\n",
    "We assume that you have successfully compiled Caffe and set the correct `PYTHONPATH`. If not, please refer to the [installation instructions](installation.html). You will use our pre-trained imagenet model, which you can download (232.57MB) by running `examples/imagenet/get_caffe_reference_imagenet_model.sh`. Note that this pre-trained model is licensed for academic research / non-commercial use only.\n",
    "\n",
    "Ready? Let's start."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "ename": "ImportError",
     "evalue": "cannot import name imagenet",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mImportError\u001b[0m                               Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-4-f456f95e6137>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0msys\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      2\u001b[0m \u001b[0msys\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'/home/philipp/caffe/caffe-master/python'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 3\u001b[1;33m \u001b[1;32mfrom\u001b[0m \u001b[0mcaffe\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mimagenet\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      4\u001b[0m \u001b[1;32mfrom\u001b[0m \u001b[0mmatplotlib\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mpyplot\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mImportError\u001b[0m: cannot import name imagenet"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "sys.path.append('/home/philipp/caffe/caffe-master/python') \n",
    "from caffe import imagenet\n",
    "from matplotlib import pyplot\n",
    "\n",
    "# Set the right path to your model file, pretrained model,\n",
    "# and the image you would like to classify.\n",
    "MODEL_FILE = 'imagenet/imagenet_deploy.prototxt'\n",
    "PRETRAINED = 'imagenet/caffe_reference_imagenet_model'\n",
    "IMAGE_FILE = 'images/cat.jpg'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Loading a network is easy. imagenet.ImagenetClassifier wraps everything. In default, the classifier will crop the center and corners of an image, as well as their mirrored versions, thus creating a batch of 10 images. If you look at the provided MODEL_FILE you can actually see that we are defining the input batch size to be 10.\n",
    "\n",
    "If you would like to just do the center, you need to specify center_only=1, and also change the batch size from 10 to 1 in the prototxt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net = imagenet.ImageNetClassifier(\n",
    "    MODEL_FILE, PRETRAINED)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will set the phase to test since we are doing testing, and will first use CPU for the computation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net.caffenet.set_phase_test()\n",
    "net.caffenet.set_mode_cpu()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So now, we can do a prediction. Let's show some output as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "prediction = net.predict(IMAGE_FILE)\n",
    "print 'prediction shape:', prediction.shape\n",
    "pyplot.plot(prediction)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see that the prediction is 1000-dimensional, and is pretty sparse.\n",
    "\n",
    "Our pretrained model uses the synset ID ordering of the classes, as listed in `../data/ilsvrc12/synset_words.txt` if you fetch the auxiliary imagenet data by `../data/ilsvrc12/get_ilsvrc_aux.sh`. If you look at the top indices that maximize the prediction score, they are foxes, cats, and other cute mammals. Not unreasonable predictions, right?\n",
    "\n",
    "Now, why don't we see how long it takes to perform the classification end to end? This result is run from an Intel i5 CPU, so you may observe some performance differences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%timeit net.predict(IMAGE_FILE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It may look a little slow, but note that it also includes image loading, cropping, and python interfacing time, and it is running 10 images. As a performance notice, if you really want to make prediction fast, you can optionally write things in C and also pipeline the image loading part. But for most applications, the current speed might be fine I guess?\n",
    "\n",
    "OK, so how about GPU? it is actually pretty easy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net.caffenet.set_mode_gpu()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Voila! Now we are in GPU mode. Let's see if the code gives the same result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "prediction = net.predict(IMAGE_FILE)\n",
    "print 'prediction shape:', prediction.shape\n",
    "pyplot.plot(prediction)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Good, everything is the same. And how about time consumption? The following benchmark is obtained on the same machine with a K20 GPU:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%timeit net.predict(IMAGE_FILE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pretty fast right? Not as fast as you expected? Indeed, in this python demo you are seeing only 4 times speedup. But remember - the GPU code is actually very fast, and the data loading, transformation and interfacing actually start to take **more** time than the actual convnet computation itself!\n",
    "\n",
    "To fully utilize the power of GPUs, you really want to use one of these ideas:\n",
    "* Use larger batches, and minimize python call and data transfer overheads.\n",
    "* Pipeline data load operations, like using a subprocess.\n",
    "* Code in C++. A little inconvenient, but maybe worth it if your dataset is really, really large."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Parting Words\n",
    "-------------\n",
    "\n",
    "So this is python! We hope the interface is easy enough for one to use. The python wrapper is interfaced with boost::python, and source code can be found at `python/caffe` while the ImageNet wrapper used here is at `python/caffe/imagenet`. If you would like to achieve some custom functions, you are more than welcome to look at them!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}