lecture04_numpy/notebook_basic_numpy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8e139647",
   "metadata": {},
   "source": [
    "# Week 04: Basic NumPy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bd8edae",
   "metadata": {},
   "source": [
    "Make sure to have NumPy installed before proceeding!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f55caed4",
   "metadata": {},
   "source": [
    "## Importing Modules\n",
    "\n",
    "To use more specialized functionality of Python, as well as third-party modules, you will need to know about the import statement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c4736ad6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021\n",
      "5\n",
      "1.20.2\n"
     ]
    }
   ],
   "source": [
    "# this will import the standard library module 'time'\n",
    "import time\n",
    "\n",
    "# its functions, classes and objects are available with the prefix 'time.'\n",
    "print(time.localtime().tm_year)\n",
    "\n",
    "# you can also import specific objects directly, in which case you don't need the prefix when using them\n",
    "from time import localtime\n",
    "print(localtime().tm_mon)\n",
    "\n",
    "# lastly, the keyword 'as' is used to import something under a different name\n",
    "# numpy, for instance, is usually import as 'np' to keep code shorter\n",
    "import numpy as np\n",
    "print(np.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6975bd3e",
   "metadata": {},
   "source": [
    "Import statements are also used to construct programs that consist of multiple files. As an exercise, try creating a file called `my_library.py` in the same directory as this notebook and giving it a function called `my_function` (it doesn't need to do much, perhaps print something). Then, try executing the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "5bfdfc63",
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'my_library'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-2-47325e7005eb>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mmy_library\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mmy_function\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mmy_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'my_library'"
     ]
    }
   ],
   "source": [
    "from my_library import my_function\n",
    "\n",
    "my_function()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3467f49f",
   "metadata": {},
   "source": [
    "It should be noted that when importing even just one object from a script, the entire script will be executed! Make sure to wrap any code that should *not* be executed when being imported in a block like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "26fae755",
   "metadata": {},
   "outputs": [],
   "source": [
    "if __name__ == \"__main__\":\n",
    "    # code here\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccebed4a",
   "metadata": {},
   "source": [
    "## The `np.ndarray`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d104ad96",
   "metadata": {},
   "source": [
    "Everything in NumPy is built around the `ndarray`. Arrays, like lists, store values, but with two limitations:\n",
    "\n",
    "1. They have a fixed datatype that is defined at their creation and must apply to all elements. Numpy usually detects an appropriate datatype automatically, but it can also be explicitly given with the `dtype` parameter.\n",
    "2. They contain a fixed number of elements (also defined at creation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8f13c426",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 3])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# NumPy arrays can be created directly from Python sequences like lists or tuples\n",
    "np.array([1, 2, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0cf30ee3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      "[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      "[0 1 2 3 4 5 6 7 8 9]\n",
      "[0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]\n"
     ]
    }
   ],
   "source": [
    "# ... or using helper functions\n",
    "print(np.zeros(10))\n",
    "print(np.ones(10))\n",
    "\n",
    "print(np.arange(10))\n",
    "print(np.arange(10, dtype=float))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "25d1cf09",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "float64 int64\n"
     ]
    }
   ],
   "source": [
    "# you can access the datatype of any ndarray with the dtype attribute\n",
    "arr1 = np.zeros(10)\n",
    "arr2 = np.arange(10)\n",
    "\n",
    "print(arr1.dtype, arr2.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f132ac96",
   "metadata": {},
   "source": [
    "Note that the datatypes used in NumPy arrays are a lot closer to those used in C and Java, a `uint32` array for example can't have numbers larger than 4,294,967,295, everything above \"overflows\" and starts again at 0."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4e53e9d",
   "metadata": {},
   "source": [
    "## Multidimensional Arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5054300d",
   "metadata": {},
   "source": [
    "One of the great advantages of NumPy arrays is that they are designed to be **multidimensional**. The vanilla Python equivalent would be **nested lists**, but here lies another difference: NumPy arrays must be **rectangular**, meaning for example all \"rows\" must have the same length."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0db52dfa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 2],\n",
       "       [3, 4]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# creating a multidimensional array from a nested list:\n",
    "np.array([[1, 2], [3, 4]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "efaaea03",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-8-6b2b97dcec9d>:2: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n",
      "  np.array([[1, 2, 3], [4]])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([list([1, 2, 3]), list([4])], dtype=object)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# this won't produce the intended result because the nested list is not rectangular:\n",
    "np.array([[1, 2, 3], [4]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7bd31af",
   "metadata": {},
   "source": [
    "Most array-creating functions have a parameter `shape`, which can be used to specify the number of dimensions and their respective sizes. A shape is a `tuple` of `int`s, where each `int` determines the size of a dimension. The dimensions are ordered from **outermost** to **innermost** (when thinking of it as a nested array)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "fa78aa2a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 0., 0.],\n",
       "       [0., 0., 0., 0.],\n",
       "       [0., 0., 0., 0.]])"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.zeros(shape=(3, 4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "44c7dd4e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1.]],\n",
       "\n",
       "       [[1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1.]]])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.ones(shape=(2, 3, 4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6172ae2a",
   "metadata": {},
   "source": [
    "In fact, the positional parameter we used earlier *was* the shape parameter, so this also works:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "40e0f6fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 0.],\n",
       "       [0., 0., 0.],\n",
       "       [0., 0., 0.]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.zeros((3, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57780a5a",
   "metadata": {},
   "source": [
    "Arrays can be reshaped into different shapes. The number of dimensions may change, but the total number of elements may not. The use of a -1 signals NumPy to automatically determine the size of that dimension, given the other sizes and the total number of elements. There may only be a single -1 in each `reshape` call. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "6245eed4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0  1  2  3]\n",
      " [ 4  5  6  7]\n",
      " [ 8  9 10 11]\n",
      " [12 13 14 15]] \n",
      "\n",
      "[[ 0  1  2  3  4  5  6  7]\n",
      " [ 8  9 10 11 12 13 14 15]] \n",
      "\n",
      "[[ 0]\n",
      " [ 1]\n",
      " [ 2]\n",
      " [ 3]\n",
      " [ 4]\n",
      " [ 5]\n",
      " [ 6]\n",
      " [ 7]\n",
      " [ 8]\n",
      " [ 9]\n",
      " [10]\n",
      " [11]\n",
      " [12]\n",
      " [13]\n",
      " [14]\n",
      " [15]] \n",
      "\n",
      "[[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15]]\n"
     ]
    }
   ],
   "source": [
    "array = np.arange(16)\n",
    "\n",
    "print(array.reshape(4, 4), \"\\n\")\n",
    "print(array.reshape(2, -1), \"\\n\")\n",
    "print(array.reshape(16, 1), \"\\n\")\n",
    "print(array.reshape(1, 16))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c0eb6f6",
   "metadata": {},
   "source": [
    "The `.flatten()` method is equivalent to `.reshape(-1)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "4a7b7414",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 0., 0., 0., 0., 0., 0., 0.])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "array = np.zeros((3, 3))\n",
    "array.flatten()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0278a8c",
   "metadata": {},
   "source": [
    "First, note that like most NumPy functions and methods, `.reshape` does **not** work **in-place**, meaning the original array is not changed. \n",
    "\n",
    "However, perhaps confusingly, `.reshape` also does **not** (necessarily) return a **copy** with changed dimensions of the array. Instead, it returns a new **view** of the same array, which means that if we mutate the original array, the view is also changed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "4803efc2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0. 0. 0. 0. 0. 0. 0. 0. 0.] \n",
      "\n",
      "[[0. 0. 0.]\n",
      " [0. 0. 0.]\n",
      " [0. 0. 0.]]\n"
     ]
    }
   ],
   "source": [
    "array = np.zeros(9)\n",
    "view = array.reshape(3, 3)\n",
    "\n",
    "print(array, \"\\n\")\n",
    "print(view)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "6554c93a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.  0.  0.  0. 42.  0.  0.  0.  0.] \n",
      "\n",
      "[[ 0.  0.  0.]\n",
      " [ 0. 42.  0.]\n",
      " [ 0.  0.  0.]]\n"
     ]
    }
   ],
   "source": [
    "# now change only original array\n",
    "array[4] = 42\n",
    "\n",
    "# and print both again\n",
    "print(array, \"\\n\")\n",
    "print(view)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "812cabe5",
   "metadata": {},
   "source": [
    "It's sometimes confusing when this kind of behavior occurs, some functions work like this and others don't. Aside from looking it up in the documentation, you can also use `array.copy()` if you want to make sure to work with a copy."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e10bca3",
   "metadata": {},
   "source": [
    "## Indexing Arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4434a16d",
   "metadata": {},
   "source": [
    "There are many ways to index the elements of a NumPy array, some of which will be shown in the next lecture. NumPy differentiates between *simple* and *advanced* indexing. Simple indexing can be thought of as a generalization of Python's slicing to multiple dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "1728d046",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15]\n"
     ]
    }
   ],
   "source": [
    "array = np.arange(16)\n",
    "print(array)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "fa5c0ea9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n"
     ]
    }
   ],
   "source": [
    "# a single element\n",
    "print(array[4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "a356a776",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2 3 4 5 6]\n",
      "[0 1 2 3]\n",
      "[ 9 10 11 12 13]\n"
     ]
    }
   ],
   "source": [
    "# some slices\n",
    "print(array[2:7])\n",
    "print(array[:4])\n",
    "print(array[-7:-2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "6a681c4e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[15 14 13 12 11 10  9  8  7  6  5  4  3  2  1  0]\n"
     ]
    }
   ],
   "source": [
    "# extended slicing also works!\n",
    "print(array[::-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbbb5aa6",
   "metadata": {},
   "source": [
    "Slices can also be used for assignment. The right side of the assignment operator must be a Python sequence or a NumPy array with the correct datatype and shape (or a shape that is broadcastable to the required shape, see below for more on broadcasting)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "id": "2b00404e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0  1  2 42 43 44 45  7  8  9]\n"
     ]
    }
   ],
   "source": [
    "array = np.arange(10)\n",
    "array[3:7] = [42, 43, 44, 45]\n",
    "\n",
    "print(array)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49a93672",
   "metadata": {},
   "source": [
    "For higher-dimensional arrays, each dimension is indexed individually, separated by `,`. Again, dimensions are ordered from outermost to innermost."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "b92abf2d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0  1  2  3]\n",
      " [ 4  5  6  7]\n",
      " [ 8  9 10 11]\n",
      " [12 13 14 15]]\n"
     ]
    }
   ],
   "source": [
    "array = np.arange(16).reshape(4, 4)\n",
    "print(array)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "f3dd7485",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6\n"
     ]
    }
   ],
   "source": [
    "# a single element (row 1, column 2)\n",
    "print(array[1, 2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "d989dac9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2 3]\n",
      " [5 6 7]] \n",
      "\n",
      "[[ 0  1  2  3]\n",
      " [ 4  5  6  7]\n",
      " [ 8  9 10 11]] \n",
      "\n",
      "[[ 0  1  2]\n",
      " [ 4  5  6]\n",
      " [ 8  9 10]\n",
      " [12 13 14]] \n",
      "\n",
      "[[12 14]\n",
      " [ 8 10]\n",
      " [ 4  6]\n",
      " [ 0  2]]\n"
     ]
    }
   ],
   "source": [
    "# some slices\n",
    "print(array[0:2, 1:4], \"\\n\")    # rows 0 and 1, columns 1, 2 and 3\n",
    "print(array[:-1], \"\\n\")         # rows 0, 1, and 2, no restriction on columns\n",
    "print(array[:, :-1], \"\\n\")      # no restriction on rows, columns 0, 1 and 2\n",
    "print(array[::-1, ::2])         # all rows (but in reverse), every second column"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7dd66ff",
   "metadata": {},
   "source": [
    "If you want your resulting slice to be of higher dimension than the original array, you can insert a `np.newaxis` object at the corresponding position. Alternatively, `None` does the same thing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "4cb9fd21",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[ 0  1  2  3]]\n",
      "\n",
      " [[ 4  5  6  7]]\n",
      "\n",
      " [[ 8  9 10 11]]\n",
      "\n",
      " [[12 13 14 15]]] \n",
      "\n",
      "[[[ 0  1  2  3]\n",
      "  [ 4  5  6  7]\n",
      "  [ 8  9 10 11]\n",
      "  [12 13 14 15]]]\n"
     ]
    }
   ],
   "source": [
    "print(array[:, np.newaxis, :], \"\\n\")\n",
    "print(array[None, :, :])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "826d43c7",
   "metadata": {},
   "source": [
    "Lastly, for arrays with many dimensions, of which you only want to slice a couple, the use of the **ellipsis** object (literally `...`) makes sense. This lets NumPy automatically fill in as many `:, ` as needed to reach all dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "id": "c81e4819",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[[27, 28, 29],\n",
       "         [30, 31, 32],\n",
       "         [33, 34, 35]],\n",
       "\n",
       "        [[36, 37, 38],\n",
       "         [39, 40, 41],\n",
       "         [42, 43, 44]],\n",
       "\n",
       "        [[45, 46, 47],\n",
       "         [48, 49, 50],\n",
       "         [51, 52, 53]]]])"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "array = np.arange(3 * 3 * 3 * 3).reshape(3, 3, 3, 3)\n",
    "\n",
    "array[1:2, ...]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b400a410",
   "metadata": {},
   "source": [
    "In other languages you may have seen a syntax like this `array[row][column]` to access an element of a multidimensional array. While this technically works in NumPy, it is discouraged because it tends to be slower and less elegant. Consider using the `array[row, column]` syntax described above."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76f63b7a",
   "metadata": {},
   "source": [
    "## Mathematical Operations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d51b2ce",
   "metadata": {},
   "source": [
    "NumPy arrays work with the usual mathematical operators on an element-wise basis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "55b351cc",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0  1  2  3]\n",
      " [ 4  5  6  7]\n",
      " [ 8  9 10 11]\n",
      " [12 13 14 15]] \n",
      "\n",
      "[[1 1 1 1]\n",
      " [1 1 1 1]\n",
      " [1 1 1 1]\n",
      " [1 1 1 1]]\n"
     ]
    }
   ],
   "source": [
    "a = np.arange(16).reshape(4, 4)\n",
    "b = np.ones((4, 4), dtype=np.int32)\n",
    "\n",
    "print(a, \"\\n\")\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "cf5ebde3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1  2  3  4]\n",
      " [ 5  6  7  8]\n",
      " [ 9 10 11 12]\n",
      " [13 14 15 16]]\n"
     ]
    }
   ],
   "source": [
    "print(a + b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "65653703",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[       inf 1.         0.5        0.33333333]\n",
      " [0.25       0.2        0.16666667 0.14285714]\n",
      " [0.125      0.11111111 0.1        0.09090909]\n",
      " [0.08333333 0.07692308 0.07142857 0.06666667]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-52-ffb769c58b4d>:3: RuntimeWarning: divide by zero encountered in true_divide\n",
      "  print(b / a)\n"
     ]
    }
   ],
   "source": [
    "# unlike vanilla Python, NumPy will only produce a warning when a division by zero occurs!\n",
    "# the code will still run and result in 'inf' (or 'nan', if 0 / 0)\n",
    "print(b / a)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c43f8c0",
   "metadata": {},
   "source": [
    "The `@` operator can be used to perform matrix multiplication."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "bfd33167",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1. 1.]\n",
      " [5. 5.]]\n"
     ]
    }
   ],
   "source": [
    "matrix1 = np.arange(4).reshape(2, 2)\n",
    "matrix2 = np.ones((2, 2))\n",
    "\n",
    "print(matrix1 @ matrix2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66e14958",
   "metadata": {},
   "source": [
    "For the element-wise operations, it is not necessary that the two operands have the same dimensions. For example, it's possible to multiply an entire array element wise with a scalar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "883a7d7b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[4. 4. 4.]\n",
      " [4. 4. 4.]\n",
      " [4. 4. 4.]]\n"
     ]
    }
   ],
   "source": [
    "print(np.ones((3, 3)) * 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e927096a",
   "metadata": {},
   "source": [
    "However, something like this also works:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "c63e4976",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0. 1. 2.]\n",
      " [0. 1. 2.]\n",
      " [0. 1. 2.]]\n"
     ]
    }
   ],
   "source": [
    "array1 = np.ones((3, 3))    # shape: (3, 3)\n",
    "array2 = np.arange(3)       # shape: (3)\n",
    "\n",
    "print(array1 * array2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c692145a",
   "metadata": {},
   "source": [
    "In both cases, NumPy automatically performs what's called **Broadcasting**. Broadcasting tries to expand the dimensions of both operands until they match. This is not always possible. To understand what's going on we have to look at the steps by which NumPy performs broadcasting.#\n",
    "\n",
    "**1. Step** If the arrays have different numbers of dimensions, the smaller shape is padded with ones on its left side.\n",
    "\n",
    "    Example: (5 x 3) + (3) → (5 x 3) + (1 x 3)\n",
    "\n",
    "\n",
    "**2. Step** If the number of the dimensions matches, but the size of a dimension does not, dimensions with the size of 1 are expanded.\n",
    "\n",
    "    Example: (5 x 3) + (1 x 3) → (5 x 3) + (5 x 3)\n",
    "\n",
    "**3. Step** If the shapes of the arrays still defer after applying the steps 1 and 2, a broadcasting error is raised.\n",
    "\n",
    "NumPy documentation on broadcasting: https://numpy.org/doc/stable/user/basics.broadcasting.html\n",
    "\n",
    "![Broadcasting Illustration](https://jakevdp.github.io/PythonDataScienceHandbook/figures/02.05-broadcasting.png)\n",
    "\n",
    "Image Source: https://jakevdp.github.io/PythonDataScienceHandbook/figures/02.05-broadcasting.png"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb94d3dc",
   "metadata": {},
   "source": [
    "## Element-wise Functions\n",
    "\n",
    "NumPy provides many functions that take entire arrays as inputs and perform element-wise operations.\n",
    "\n",
    "A special type of element-wise functions are called **ufuncs** (universal functions). If given multiple arguments, they will perform automatic broadcasting where required.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "89a79808",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.          0.84147098  0.90929743]\n",
      " [ 0.14112001 -0.7568025  -0.95892427]\n",
      " [-0.2794155   0.6569866   0.98935825]]\n"
     ]
    }
   ],
   "source": [
    "array = np.arange(9).reshape(3, 3)\n",
    "\n",
    "print(np.sin(array))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30c68e72",
   "metadata": {},
   "source": [
    "A list of all available ufuncs can be found here:\n",
    "https://numpy.org/doc/stable/reference/ufuncs.html#available-ufuncs\n",
    "\n",
    "A complete list of all NumPy functions (many of which work element-wise) can be found here:\n",
    "https://numpy.org/doc/stable/reference/routines.html\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3aafb30",
   "metadata": {},
   "source": [
    "## Constants\n",
    "\n",
    "Of course, NumPy also has common constants used in mathematics like $\\pi$ and $e$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "4f37080a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.141592653589793\n",
      "2.718281828459045\n"
     ]
    }
   ],
   "source": [
    "print(np.pi)\n",
    "print(np.e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "765ee2c0",
   "metadata": {},
   "source": [
    "## `np.linspace`\n",
    "\n",
    "`np.linspace(start, stop, num)` is a useful function that, similarly to `np.arange`, creates an array spanning from `start` to `stop`. However, the third parameter `num` determines how many evenly distributed elements the array should have."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "b5acd10c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.  0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1. ]\n"
     ]
    }
   ],
   "source": [
    "print(np.linspace(0, 1, 11))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69b4eca1",
   "metadata": {},
   "source": [
    "## Aggregation Functions\n",
    "\n",
    "Contrary to element-wise functions, aggregation functions reduce the dimensionality of an array. Examples include `np.mean`, `np.sum` and `np.max`/`np.min`.\n",
    "\n",
    "They all have a special parameter `axis`, which specifies along which dimension (or which dimensions) to perform the aggregation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "fabf5802",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0  1  2  3]\n",
      " [ 4  5  6  7]\n",
      " [ 8  9 10 11]\n",
      " [12 13 14 15]]\n"
     ]
    }
   ],
   "source": [
    "array = np.arange(16).reshape(4, 4)\n",
    "print(array)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "6b1d70c3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[24 28 32 36]\n",
      "[ 6 22 38 54]\n",
      "120\n",
      "120\n"
     ]
    }
   ],
   "source": [
    "print(np.sum(array, axis=0))       # sums of all columns\n",
    "print(np.sum(array, axis=1))       # sums of all rows\n",
    "print(np.sum(array, axis=(0, 1)))  # sum over dimensions 0 and 1\n",
    "print(np.sum(array))               # sum over all dimensions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4703a943",
   "metadata": {},
   "source": [
    "A special aggregation function is `np.cumsum`, which like `np.sum` sums along one or more axes, but returns all intermediate results in an array of the same shape as the input array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "4cb1c94c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0  1  3  6]\n",
      " [ 4  9 15 22]\n",
      " [ 8 17 27 38]\n",
      " [12 25 39 54]]\n"
     ]
    }
   ],
   "source": [
    "print(np.cumsum(array, axis=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9baa3800",
   "metadata": {},
   "source": [
    "A less known trick is that every *ufunc* (see above) can be turned into a aggregation function using `ufunc.reduce`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "0c7dd38a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[   0  585 1680 3465]\n"
     ]
    }
   ],
   "source": [
    "print(np.multiply.reduce(array, axis=0))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db081c59",
   "metadata": {},
   "source": [
    "## Comparing Arrays\n",
    "\n",
    "Like mathematical operators, comparison operators also work element-wise with NumPy Arrays. They will return a boolean array containing the result of each comparison."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "cf41b1d0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ True  True  True False  True]\n"
     ]
    }
   ],
   "source": [
    "array1 = np.arange(5)\n",
    "array2 = array1.copy()\n",
    "array2[3] = 42\n",
    "\n",
    "print(array1 == array2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe0d7273",
   "metadata": {},
   "source": [
    "`np.all` is an aggregation function that returns `True` if and only if all values in an array (or along an axis if an `axis` is specified) are `True`. To compare two entire arrays as a whole, it seems intuitive to do something like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "8381c655",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.all(array1 == array2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a01ff9f",
   "metadata": {},
   "source": [
    "While this usually works, in some edge cases this can produce unintended results. It is safer to use NumPy's provided function `np.array_equal`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "723fd3fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array_equal(array1, array2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9930a3e",
   "metadata": {},
   "source": [
    "A similar function is `np.allclose`, which returns `True` if two arrays have almost the same value in all places. An error margin can be specified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "6cb6bcc5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "epsilon = 1e-10\n",
    "\n",
    "array3 = array1 + epsilon\n",
    "\n",
    "print(np.array_equal(array1, array3))\n",
    "print(np.allclose(array1, array3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da995ff0",
   "metadata": {},
   "source": [
    "## Performance\n",
    "\n",
    "There is nothing you can do with NumPy that you couldn't in principle also do with normal Python, but by now you have hopefully noticed the convenience of NumPy when working with numerical data. In addition to convenience, NumPy also offers significantly better performance when working with large amounts of numbers.\n",
    "\n",
    "As a quick demonstration, let's compute the mean of the integers from 1 to 10,000,000 using plain Python and using NumPy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "id": "da202ecc",
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_mean_naive(data):\n",
    "    mean = 0\n",
    "    \n",
    "    for d in data:\n",
    "        mean += d\n",
    "    \n",
    "    mean /= len(data)\n",
    "    \n",
    "    return mean\n",
    "\n",
    "def compute_mean_pythonic(data):\n",
    "    return sum(data) / len(data)\n",
    "\n",
    "def compute_mean_numpy(data):\n",
    "    return np.mean(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "id": "835de536",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_python = list(range(10_000_000))\n",
    "data_numpy = np.array(data_python)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "id": "f9f1d5a9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "332 ms ± 14.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n",
      "61.8 ms ± 760 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n",
      "9.75 ms ± 14.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit compute_mean_naive(data_python)\n",
    "%timeit compute_mean_pythonic(data_python)\n",
    "%timeit compute_mean_numpy(data_numpy)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d77d281",
   "metadata": {},
   "source": [
    "As is evident from the results, using Python's built-in functions already drastically improves upon a naive approach, but NumPy is another order of magnitude faster."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3186b08f",
   "metadata": {},
   "source": [
    "## Random Numbers\n",
    "\n",
    "The module `numpy.random` provides options for generating pseudo-random numbers. For instance, there are various distributions to sample from."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "233f4658",
   "metadata": {},
   "outputs": [],
   "source": [
    "uniform = np.random.uniform(0, 1, size=10)\n",
    "normal = np.random.normal(0, 1, size=10)\n",
    "gumbel = np.random.gumbel(0, 1, size=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd6e0bd6",
   "metadata": {},
   "source": [
    "In some cases, you may want the generated random numbers to be the same every time you run your program. In that case, you can set a seed like this before generating."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "a5f5c87d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.37454012 0.95071431 0.73199394 0.59865848 0.15601864 0.15599452\n",
      " 0.05808361 0.86617615 0.60111501 0.70807258]\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(42)\n",
    "print(np.random.uniform(0, 1, size=10))    # these will always be the same"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d024241",
   "metadata": {},
   "source": [
    "`np.random.shuffle` and `np.random.permutation` both shuffle an array, however the first works in-place while the latter returns a copy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "2f045692",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[8 2 0 6 7 9 3 1 4 5]\n",
      "[1 7 6 2 8 0 3 4 5 9]\n"
     ]
    }
   ],
   "source": [
    "array1 = np.arange(10)\n",
    "np.random.shuffle(array1)\n",
    "\n",
    "print(array1)\n",
    "\n",
    "array2 = np.arange(10)\n",
    "print(np.random.permutation(array2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "869e1835",
   "metadata": {},
   "source": [
    "`np.random.choice` allows to pick random elements from an array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "id": "950d0170",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['o' 't' 'n']\n"
     ]
    }
   ],
   "source": [
    "array = np.array(list(\"Python\"))\n",
    "\n",
    "print(np.random.choice(array, size=3, replace=False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a18932af",
   "metadata": {},
   "source": [
    "## More NumPy\n",
    "\n",
    "NumPy has much more to offer. Next week, we will discuss a couple features that were left out this time, most notably advanced indexing methods.\n",
    "\n",
    "https://numpy.org/doc/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b16e4c2f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}