intro_profiling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "83f0cd9d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# High performance Python 🚀\n",
    "### Zbyszek & Aitor\n",
    "### ASPP 2023, Heraklion, Greece\n",
    "\n",
    "fork and clone the respository now please! :)\n",
    "\n",
    "Link is on the schedule! (Or here: https://github.com/ASPP/2023-high-performance-python)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2089e7e0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Outline\n",
    "\n",
    "* Introduction\n",
    "* Analyze what makes your code slow and inefficient with **profiling** (Aitor)\n",
    "* Speed & *convenience* with **Numba** (Zbyszek)\n",
    "* Speed & *flexibility* with **Cython** (Zbyszek)\n",
    "* Outroduction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30cd66fd",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Introduction\n",
    "\n",
    "* By now you are the *Master of Research*(TM).\n",
    "\n",
    "<!-- ![Master of research](figures/mor.png) -->\n",
    "\n",
    "* Using your new skills you can confidently transform any idea into a great manuscript!\n",
    "\n",
    "* It seems like the only things holding you back are the **execution speed** and **memory usage** of your scripts!\n",
    "\n",
    "* Profiling can help you identify which parts of your code are slow and use too much memory -> **optimization**\n",
    "\n",
    "* For extra faster code both `Cython` and `Numba` can take your scripts to the next level."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed68ad80",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Exercise\n",
    "\n",
    "Who thinks that they would benefit from faster code?\n",
    "\n",
    "Please raise your hand."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61dcef05",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Who has had code that uses too much memory?\n",
    "\n",
    "Please raise your hand."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50b08d72",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Who has spent countless hours fiddeling with code trying to optimize it?\n",
    "\n",
    "Please raise your hand."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5aace232",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The three rules of optimization\n",
    "(adapted from Sebastian Witowski, EuroPython 2016)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93e91691",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### 1. Don't.\n",
    "- Likely you don't need it.\n",
    "- Optimization comes with costs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f8d8c87",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### 2. Don't yet.\n",
    "- Is your code finished?\n",
    "- Did you write tests?\n",
    "- Are you sure it's worth the investment?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e98dca3b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### 3. Profile\n",
    "- Collect data - don't guess which part of your code you should optimize!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db5fe2b6",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Profiling demo: the Mandelbrot set\n",
    "\n",
    "Let us use the Mandelbrot set (`profiling/mandelbrot.py`) as an example for this part, because fractals are pretty!\n",
    "\n",
    "Using this script Aitor will demonstrate time and memory profiling in python.\n",
    "\n",
    "Don't worry about understanding every line in the code!\n",
    "\n",
    "<img src=\"./figures/mandelbrot.png\" alt=\"mandelbrot\" style=\"width:900px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c28f284",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Collect basic data (`runtime`)\n",
    "- while optimizing it's a good idea to keep track of the total runtime of your script\n",
    "- even though modern profilers introduce little overhead this makes sure that your code changes translate into actual speedups\n",
    "- the simplest way to do this is via `time` (or the equivalent on your OS):\n",
    "```\n",
    "time python mandelbrot_example.py \n",
    "\n",
    "    real    0m5,994s\n",
    "    user    0m5,967s\n",
    "    sys\t    0m0,961s\n",
    "```\n",
    "  - you're typically interested in \"user time\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53701bed",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "**EXERCISE** How long does it take in your machine?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2dcd2c0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## More data (`runtime`, `memory`...)\n",
    "\n",
    "- One can also get data about CPU availability, memory usage and more\n",
    "- In Linux you can use the GNU binary, which can be easily called as `\\time`\n",
    "- Use the `-v` verbose flag to get more information\n",
    "- Thus: \n",
    "\n",
    "```\n",
    "\\time -v python mandelbrot_example.py\n",
    "```\n",
    "\n",
    "We are interested in `User time` and `Maximum resident set size`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54584db8",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "**EXERCISE** How much memory in your machine?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1093ca7d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.1829643249511719"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Convenience box to convert from KB to GB\n",
    "1240428 / 1024 / 1024"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9355227b",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "```\n",
    "\tCommand being timed: \"python mandelbrot_example.py\"\n",
    "\tUser time (seconds): 6.14                                <---------\n",
    "\tSystem time (seconds): 0.94\n",
    "\tPercent of CPU this job got: 115%\n",
    "\tElapsed (wall clock) time (h:mm:ss or m:ss): 0:06.10\n",
    "\tAverage shared text size (kbytes): 0\n",
    "\tAverage unshared data size (kbytes): 0\n",
    "\tAverage stack size (kbytes): 0\n",
    "\tAverage total size (kbytes): 0\n",
    "\tMaximum resident set size (kbytes): 370700               <---------\n",
    "\tAverage resident set size (kbytes): 0\n",
    "\tMajor (requiring I/O) page faults: 0\n",
    "\tMinor (reclaiming a frame) page faults: 104585\n",
    "\tVoluntary context switches: 565\n",
    "\tInvoluntary context switches: 148\n",
    "\tSwaps: 0\n",
    "\tFile system inputs: 0\n",
    "\tFile system outputs: 2056\n",
    "\tSocket messages sent: 0\n",
    "\tSocket messages received: 0\n",
    "\tSignals delivered: 0\n",
    "\tPage size (bytes): 4096\n",
    "\tExit status: 0\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06c423f1",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Collect fine grained `runtime` data with `py-spy`\n",
    "\n",
    "**profilers** monitor the execution of your script, record statistics, and thus can **provide an understanding of the performance characteristics of your code**\n",
    "\n",
    "Here we consider [py-spy](https://github.com/benfred/py-spy), a sampling-based runtime profiler for Python\n",
    "  - simply speaking `py-spy` examines your program at regular intervals and records which function (or rather line) is currently being executed\n",
    "- No code modifications needed!\n",
    "\n",
    "<img src=\"./figures/sampling.svg\" alt=\"sampling\" style=\"width:1100px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adf58484",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Usage of `py-spy`\n",
    "To time profile your script:\n",
    "```\n",
    "py-spy record -o flamegraph_mandelbrot.svg python mandelbrot_example.py\n",
    "```\n",
    "  - to make timings accurate it needs to collect enough of data; you can control the \"sampling rate\" using the `-r` argument\n",
    "  - get more info on arguments with `-h`\n",
    "\n",
    "- `py-spy` will will produce a \"flamegraph\" like the following (here `flamegraph_mandelbrot.svg`; open it with firefox)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db9c5111",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "<img src=\"./figures/flamegraph_mandelbrot.svg\" alt=\"flamegraph\" style=\"width:1100px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98db228b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Collect fine grained `memory` data with `memray`\n",
    "\n",
    "Here we will use [memray](https://github.com/bloomberg/memray), which tracks and reports memory allocations, both in Python code and in compiled extension modules. \n",
    "- No code modifications needed!\n",
    "\n",
    "### Usage of `memray`\n",
    "You can memory profile your script:\n",
    "```\n",
    "memray run mandelbrot_example.py\n",
    "```\n",
    "\n",
    "This will create a bin file in your current directory `memray-mandelbrot_example.py.XXXXX.bin`\n",
    "\n",
    "Create a flamegraph like so (an equivalent command will we printed by memray for you in screen! You can just copy that!)\n",
    "```\n",
    "memray flamegraph memray-mandelbrot_example.py.XXXXX.bin\n",
    "```\n",
    "Finally creating an `.html` file, open it with firefox"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "957c844a",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "<img src=\"./figures/memray_flamegraph.png\" alt=\"memory\" style=\"width:900px;\"/>\n",
    "<img src=\"./figures/mandelbrot_memory.png\" alt=\"memory\" style=\"width:900px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b53d7e6",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Exercise\n",
    "\n",
    "It's time to put theory into practice. We have prepared an example script (see [./profiling/numerical_integration.py](./profiling/numerical_integration.py)) which numerically computes the integral of a function and measures the error with respect to analytical integration.\n",
    "\n",
    "#### Refresher: numerical integration\n",
    "\n",
    "![RiemannSum](figures/MidRiemann2.svg)\n",
    "\n",
    "Riemann sum: $\\int_a^b dx f(x) \\approx \\sum_{i = 0}^{n - 1} f(a + (i + 0.5) \\Delta x) \\Delta x$ with $\\Delta x = (b - a)/n$\n",
    "\n",
    "here $a=0, b=2, n=4$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ccf6384",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Exercise\n",
    "\n",
    "It's time to put theory into practice. We have prepared an example script (see [./profiling/numerical_integration.py](./profiling/numerical_integration.py)) which numerically computes the integral of a function and measures the error with respect to analytical integration.\n",
    "\n",
    "0. Fork & clone this repository.\n",
    "1. Familarize yourself with the script and exectute it: `python numerical_integration.py`\n",
    "2. Profile the script both for speed (py-spy) and memory (memray). Look at the flamegraphs from both!\n",
    "3. Optimize the script (**Only the indicated functions**), can you make it faster and more memory efficient?\n",
    "4. Commit your changes in a new branch and create a PR. Include the duration before/after optimization in the PR message.\n",
    "\n",
    "#### Hints\n",
    "- Each profiler will tell you where more time is spent or memory is allocated, these are the optimization opportunities!\n",
    "- Avoid using external libraries\n",
    "\n",
    "Afterwards we will discuss the results jointly."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be8cb0af",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Exercise discussion\n",
    "\n",
    "What did we learn?\n",
    "- ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e51e2b5a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Profiling conclusion\n",
    "\n",
    "- Before optimizing, first finish your code & write tests!\n",
    "- Then *measure* to find functions(/lines) that take up most of the time.\n",
    "- Only optimize the relevant functions(/lines), measure again, and *know when to stop*!\n",
    "  - 1min script you run 5 times\n",
    "  - 8h script you run 1000 times\n",
    "- To gain some basic data, you can use builtin tools\n",
    "  - `time` (commandline)\n",
    "  - `%timeit` (ipython, jupyter)\n",
    "  - `import timeit; timeit.time('some_func()')` (requires code changes)\n",
    "- profilers collect more fine grained data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25e0cb0f",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Beyond `py-spy` and `memray`\n",
    "For *runtime*\n",
    "- [py-spy](https://github.com/benfred/py-spy) is just one of many *runtime* profilers; alternatives include\n",
    "  - [cProfile](https://docs.python.org/3/library/profile.html) (builtin!) + [snakeviz](https://github.com/jiffyclub/snakeviz)\n",
    "  - [pyinstrument](https://github.com/joerick/pyinstrument)\n",
    "  - [austin](https://github.com/P403n1x87/austin)\n",
    "  - ...\n",
    "\n",
    "For *memory* \n",
    "- [memray](https://github.com/bloomberg/memray) is just one of many *memory* profilers; alternatives include\n",
    "  - [memory-profiler](https://pypi.org/project/memory-profiler/)\n",
    "  - [pympler](https://pypi.org/project/Pympler/)\n",
    "  - [guppy3](https://pypi.org/project/guppy3/)\n",
    "  - ...\n",
    " \n",
    " \n",
    "With modern tools, **profiling is easy! Use it!**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d62aa09a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Optimization: what to do (in order of [subjective] increasing complexity)\n",
    "\n",
    "- **Do nothing**\n",
    "- Vectorization (`numpy`!!)\n",
    "- Data structures and algorithms\n",
    "- Memoization / caching\n",
    "- Non-Python libraries (`blas`, `openblas`, `blis`, `atlas`, Intel `mkl`, ...)\n",
    "- Buy better hardware\n",
    "- **Numba**\n",
    "- **Cython** / pythran\n",
    "- **Parallelization** (->tomorrow)\n",
    "- GPUs (`cuda`, `opencl`, `directml`, ...)\n",
    "- Low-level code"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}