Iterables, Generators, Yield.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>The Python yield keyword explained</h1>\n",
    "\n",
    "This is from https://pythontips.com/2013/09/29/the-python-yield-keyword-explained/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Iterables</h2>\n",
    "\n",
    "<p>When you create a list, you can read its items one by one, and it’s called iteration:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "2\n",
      "3\n"
     ]
    }
   ],
   "source": [
    "mylist = [1, 2, 3]\n",
    "for i in mylist:\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>List Comprehension</h2>\n",
    "<p>Mylist is an iterable. When you use a list comprehension, you create a list, and so an iterable:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "mylist = [x*x for x in range(3)]\n",
    "for i in mylist:\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4, 5, 6, 7]\n"
     ]
    }
   ],
   "source": [
    "# another exmaple\n",
    "mylist = [1,2,3,4,5,6,7]\n",
    "newList = [x for x in mylist if x> 3]\n",
    "print newList"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Everything you can use “for… in…” on is an iterable: </p>\n",
    "lists, strings, files…\n",
    "\n",
    "<p>These iterables are handy because you can read them as much as you wish, but you store all the values in memory and it’s not always what you want when you have a lot of values.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Generators</h2>\n",
    "<p>Generators are iterators, but you can only iterate over them once. It’s because they do not store all the values in memory, <b>they generate the values on the fly</b>:</p>\n",
    "\n",
    "<p>The syntax for list comprehension is the same as generator except you use <b>()</b> instead of <b>[]</b>. BUT, you can not perform for i in mygenerator a second time since generators can only be used once: they calculate 0, then forget about it and calculate 1, and end calculating 4, one by one.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "mygenerator = (x*x for x in range(3)) # notice generators look just like list comprehensions but surrounded by () instead of []\n",
    "for i in mygenerator:\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<type 'generator'>\n",
      "0\n",
      "1\n",
      "4\n",
      "<type 'generator'>\n"
     ]
    }
   ],
   "source": [
    "mygenerator = (x*x for x in range(3)) # notice generators look just like list comprehensions but surrounded by () instead of []\n",
    "print type(mygenerator)\n",
    "for i in mygenerator:\n",
    "    print(i)\n",
    "\n",
    "\n",
    "# notice the second time we access mygenerator it's returns an empty result.\n",
    "print type(mygenerator)\n",
    "for i in mygenerator:\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'generator' object has no attribute '__getitem__'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-14-df0c02633cf9>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[0mmygenerator\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0mx\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m3\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;31m# notice generators look just like list comprehensions but surrounded by () instead of []\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      2\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 3\u001b[1;33m \u001b[1;32mprint\u001b[0m \u001b[0mmygenerator\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m2\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;31m# it's not really a list, it's only something you can iterate over\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m: 'generator' object has no attribute '__getitem__'"
     ]
    }
   ],
   "source": [
    "mygenerator = (x*x for x in range(3)) # notice generators look just like list comprehensions but surrounded by () instead of []\n",
    "\n",
    "print mygenerator[2] # it's not really a list, it's only something you can iterate over"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Yield</h2>\n",
    "\n",
    "<p>To master yield, you must understand that when you call the function, the code you have written in the function body does not run. The function only returns the generator object, this is a bit tricky</p>\n",
    "\n",
    "<p>Then, your code will be run each time the <b>for</b> uses the generator.</p>\n",
    "\n",
    "<p>Now the hard part:</p>\n",
    "\n",
    "<p>The first time the <b>for</b> calls the generator object created from your function, it will run the code in your function from the beginning until it hits yield, then it’ll return the first value of the loop. Then, each subsequent call will run the loop you have written in the function one more time, and return the next value, until there is no value to return.</p>\n",
    "\n",
    "</p>The generator is considered empty once the function runs but does not hit yield anymore. It can be because the loop had come to an end, or because you do not satisfy a “if/else” anymore."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<generator object createGenerator at 0x000000000402CA20>\n",
      "0\n",
      "1\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "# define a generator function\n",
    "def createGenerator():\n",
    "    mylist = range(3)\n",
    "    for i in mylist:\n",
    "        yield i*i\n",
    "        \n",
    "# invoke the create generator function, note a this time the function is not called!        \n",
    "mygenerator = createGenerator()\n",
    "print(mygenerator) # mygenerator is an object!\n",
    "\n",
    "for i in mygenerator:\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "def _make_gen(reader):\n",
    "    b = reader(1024 * 1024)\n",
    "    while b:\n",
    "        yield b\n",
    "        b = reader(1024*1024)\n",
    "\n",
    "def rawgencount(filename):\n",
    "    f = open(filename, 'rb')\n",
    "    f_gen = _make_gen(f.raw.read)\n",
    "    return sum( buf.count(b'\\n') for buf in f_gen )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<h2> Iterable Built-in Functions </h2>\n",
    "\n",
    "<table style=\"width:100%;border-collapse: collapse;text-align:left\"> \n",
    "\n",
    "<tr>\n",
    "    <th>Function</th>\n",
    "    <th>Description</th> \n",
    "</tr>\n",
    "\n",
    "\n",
    "<tr><td>all()</td><td>Returns True if all elements of an iterable are true</td></tr>\n",
    "<tr><td>any()</td><td>Returns True if any elements of an iterable are true</td></tr>\n",
    "<tr><td>enumerate()</td><td>Returns a list of tuples containing indices and values from an iterable</td></tr>\n",
    "<tr><td>filter()</td><td>Filters elements from an iterable</td></tr>\n",
    "<tr><td>iter()</td><td>Returns an iterator object</td></tr>\n",
    "<tr><td>len()</td><td>Returns the length of an object</td></tr>\n",
    "<tr><td>map()</td><td>Applies a function to every item of an iterable</td></tr>\n",
    "<tr><td>next()</td><td>Retrieves the next item from an iterator</td></tr>\n",
    "<tr><td>range()</td><td>Generates a range of integer values</td></tr>\n",
    "<tr><td>reversed()</td><td>Returns a reverse iterator</td></tr>\n",
    "<tr><td>slice()</td><td>Returns a slice object</td></tr>\n",
    "<tr><td>sorted()</td><td>Returns a sorted list from an iterable</td></tr>\n",
    "<tr><td>zip()</td><td>Creates an iterator that aggregates elements from iterables</td></tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}