From 1e936b0eed6f9115b91f224f237975b9fbab0634 Mon Sep 17 00:00:00 2001 From: mmamedli Date: Mon, 11 Jan 2021 20:38:54 +0300 Subject: [PATCH] Proofread recaps and week 1 assignments --- week1_intro/crossentropy_method.ipynb | 30 ++-- week1_intro/deep_crossentropy_method.ipynb | 36 ++--- week1_intro/gym_interface.ipynb | 38 +++--- week1_intro/primer/recap_ml.ipynb | 152 ++++++++++++--------- week1_intro/primer/recap_pytorch.ipynb | 84 +++++++----- 5 files changed, 187 insertions(+), 153 deletions(-) diff --git a/week1_intro/crossentropy_method.ipynb b/week1_intro/crossentropy_method.ipynb index 1f21258b9..f91ad4efb 100644 --- a/week1_intro/crossentropy_method.ipynb +++ b/week1_intro/crossentropy_method.ipynb @@ -6,7 +6,7 @@ "source": [ "# Crossentropy method\n", "\n", - "This notebook will teach you to solve reinforcement learning problems with crossentropy method. We'll follow-up by scaling everything up and using neural network policy." + "This notebook will teach you to solve reinforcement learning problems with crossentropy method. After that we'll scale everything up using neural network policy." ] }, { @@ -24,8 +24,8 @@ "\n", " !touch .setup_complete\n", "\n", - "# This code creates a virtual display to draw game images on.\n", - "# It will have no effect if your machine has a monitor.\n", + "# This code creates a virtual display for drawing game images on.\n", + "# It won't have any effect if your machine has a monitor.\n", "if type(os.environ.get(\"DISPLAY\")) is not str or len(os.environ.get(\"DISPLAY\")) == 0:\n", " !bash ../xvfb start\n", " os.environ['DISPLAY'] = ':1'" @@ -69,7 +69,7 @@ "\n", "Since we still use integer state and action representations, you can use a 2-dimensional array to represent the policy.\n", "\n", - "Please initialize the policy __uniformly__, that is, probabililities of all actions should be equal." + "Please initialize the policy __uniformly__, that is, the probabililities of all actions should be equal." ] }, { @@ -114,9 +114,9 @@ "source": [ "def generate_session(env, policy, t_max=10**4):\n", " \"\"\"\n", - " Play game until end or for t_max ticks.\n", - " :param policy: an array of shape [n_states,n_actions] with action probabilities\n", - " :returns: list of states, list of actions and sum of rewards\n", + " Play the game until the end or for t_max ticks.\n", + " :param policy: an array of shape [n_states,n_actions] with the action probabilities\n", + " :returns: list of states, list of actions and the sum of rewards\n", " \"\"\"\n", " states, actions = [], []\n", " total_reward = 0.\n", @@ -198,7 +198,7 @@ " [i.e. sorted by session number and timestep within session]\n", "\n", " If you are confused, see examples below. Please don't assume that states are integers\n", - " (they will become different later).\n", + " (their type will change later).\n", " \"\"\"\n", "\n", " reward_threshold = \n", @@ -267,7 +267,7 @@ " policy[s_i,a_i] ~ #[occurrences of s_i and a_i in elite states/actions]\n", "\n", " Don't forget to normalize the policy to get valid probabilities and handle the 0/0 case.\n", - " For states that you never visited, use a uniform distribution (1/n_actions for all states).\n", + " For states, that you never visited, use a uniform distribution (1/n_actions for all states).\n", "\n", " :param elite_states: 1D list of states from elite sessions\n", " :param elite_actions: 1D list of actions from elite sessions\n", @@ -387,7 +387,7 @@ "\n", " policy = learning_rate * new_policy + (1 - learning_rate) * policy\n", "\n", - " # display results on chart\n", + " # display results on the chart\n", " show_progress(rewards_batch, log, percentile)" ] }, @@ -395,13 +395,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Reflecting on results\n", + "### Reflecting on the results\n", "\n", - "You may have noticed that the taxi problem quickly converges from less than -1000 to a near-optimal score and then descends back into -50/-100. This is in part because the environment has some innate randomness. Namely, the starting points of passenger/driver change from episode to episode.\n", + "You may have noticed that the taxi problem quickly converges from less than -1000 to a near-optimal score and then descends back to -50/-100. This is in part because the environment has some innate randomness. Namely, the starting points of passenger/driver change from episode to episode.\n", "\n", - "In case CEM failed to learn how to win from one distinct starting point, it will simply discard it because no sessions from that starting point will make it into the \"elites\".\n", + "In case CEM failed to learn, how to win from one distinct starting point, it will simply discard it because no sessions from that starting point will make it into the \"elites\".\n", "\n", - "To mitigate that problem, you can either reduce the threshold for elite sessions (duct tape way) or change the way you evaluate strategy (theoretically correct way). For each starting state, you can sample an action randomly, and then evaluate this action by running _several_ games starting from it and averaging the total reward. Choosing elite sessions with this kind of sampling (where each session's reward is counted as the average of the rewards of all sessions with the same starting state and action) should improve the performance of your policy." + "To mitigate that problem, you can either reduce the threshold for elite sessions (duct tape way) or change the way you evaluate the strategy (theoretically correct way). For each starting state, you can sample an action randomly, and then evaluate this action by running _several_ games starting from it and averaging the total reward. Choosing elite sessions with this kind of sampling (where each session reward is counted as the average of the rewards of all sessions with the same starting state and action) should improve the performance of your policy." ] }, { @@ -429,5 +429,5 @@ } }, "nbformat": 4, - "nbformat_minor": 1 + "nbformat_minor": 0 } diff --git a/week1_intro/deep_crossentropy_method.ipynb b/week1_intro/deep_crossentropy_method.ipynb index 9e3681f50..446ddebaa 100644 --- a/week1_intro/deep_crossentropy_method.ipynb +++ b/week1_intro/deep_crossentropy_method.ipynb @@ -27,8 +27,8 @@ "\n", " !touch .setup_complete\n", "\n", - "# This code creates a virtual display to draw game images on.\n", - "# It will have no effect if your machine has a monitor.\n", + "# This code creates a virtual display for drawing game images on.\n", + "# It won't have any effect if your machine has a monitor.\n", "if type(os.environ.get(\"DISPLAY\")) is not str or len(os.environ.get(\"DISPLAY\")) == 0:\n", " !bash ../xvfb start\n", " os.environ['DISPLAY'] = ':1'" @@ -65,8 +65,8 @@ "\n", "For this assignment we'll utilize the simplified neural network implementation from __[Scikit-learn](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html)__. Here's what you'll need:\n", "\n", - "* `agent.partial_fit(states, actions)` - make a single training pass over the data. Maximize the probabilitity of :actions: from :states:\n", - "* `agent.predict_proba(states)` - predict probabilities of all actions, a matrix of shape __[len(states), n_actions]__\n" + "* `agent.partial_fit(states, actions)` - makes a single training pass over the data. Maximize the probabilitity of :actions: from :states:\n", + "* `agent.predict_proba(states)` - predicts probabilities of all actions, a matrix of shape __[len(states), n_actions]__\n" ] }, { @@ -82,7 +82,7 @@ " activation='tanh',\n", ")\n", "\n", - "# initialize agent to the dimension of state space and number of actions\n", + "# initialize agent to the dimension of state space and a number of actions\n", "agent.partial_fit([env.reset()] * n_actions, range(n_actions), range(n_actions))" ] }, @@ -107,7 +107,7 @@ " # use agent to predict a vector of action probabilities for state :s:\n", " probs = \n", "\n", - " assert probs.shape == (env.action_space.n,), \"make sure probabilities are a vector (hint: np.reshape)\"\n", + " assert probs.shape == (env.action_space.n,), \"make sure that the probabilities are a vector (hint: np.reshape)\"\n", " \n", " # use the probabilities you predicted to pick an action\n", " # sample proportionally to the probabilities, don't just take the most likely action\n", @@ -168,7 +168,7 @@ " [i.e. sorted by session number and timestep within session]\n", "\n", " If you are confused, see examples below. Please don't assume that states are integers\n", - " (they will become different later).\n", + " (their type will change later).\n", " \"\"\"\n", "\n", " \n", @@ -274,7 +274,7 @@ "metadata": {}, "outputs": [], "source": [ - "# Show video. This may not work in some setups. If it doesn't\n", + "# Show video. In some setups this may not work. If it doesn't\n", "# work for you, you can download the videos and view them locally.\n", "\n", "from pathlib import Path\n", @@ -297,23 +297,23 @@ "\n", "By this moment you should have got enough score on [CartPole-v0](https://gym.openai.com/envs/CartPole-v0) to consider it solved (see the link). It's time to try something harder.\n", "\n", - "_if you have any trouble with CartPole-v0 and feel stuck, take a look at the forums_\n", + "_if you have any trouble with CartPole-v0 and feel stuck, take a look on forums_\n", "\n", - "Your assignment is to obtain average reward of __at least -150__ on `MountainCar-v0`.\n", + "Your assignment is to obtain an average reward of __at least -150__ on `MountainCar-v0`.\n", "\n", "See the tips section below, it's kinda important.\n", " \n", "* Bonus quest: Devise a way to speed up training against the default version\n", " * Obvious improvement: use [joblib](https://www.google.com/search?client=ubuntu&channel=fs&q=joblib&ie=utf-8&oe=utf-8)\n", - " * Try re-using samples from 3-5 last iterations when computing threshold and training\n", - " * Experiment with amount of training iterations and learning rate of the neural network (see params)\n", + " * Try re-using samples from 3-5 last iterations when computing threshold and during training\n", + " * Experiment with an amount of training iterations and the learning rate of the neural network (see params)\n", " \n", " \n", "### Tips\n", "* Gym page: [MountainCar](https://gym.openai.com/envs/MountainCar-v0)\n", "* Sessions for MountainCar may last for 10k+ ticks. Make sure ```t_max``` param is at least 10k.\n", - " * Also it may be a good idea to cut rewards via \">\" and not \">=\". If 90% of your sessions get reward of -10k and 10% are better, than if you use percentile 20% as threshold, R >= threshold __fails cut off bad sessions__ whule R > threshold works alright.\n", - "* _issue with gym_: Some versions of gym limit game time by 200 ticks. This will prevent cem training in most cases. Make sure your agent is able to play for the specified __t_max__, and if it isn't, try `env = gym.make(\"MountainCar-v0\").env` or otherwise get rid of TimeLimit wrapper.\n", + " * Also it may be a good idea to cut rewards via \">\" and not \">=\". If 90% of your sessions get reward of -10k and 10% are better, than if you use percentile 20% as the threshold, R >= threshold __fails cut off bad sessions__ whule R > threshold works alright.\n", + "* _issue with gym_: Some versions of gym limit game time by 200 ticks. This will prevent the training in most cases. Make sure your agent is able to play for the specified __t_max__, and if it isn't, try `env = gym.make(\"MountainCar-v0\").env` or otherwise get rid of TimeLimit wrapper.\n", "* If it won't train it's a good idea to plot reward distribution and record sessions: they may give you some clue. If they don't, call course staff :)\n", "* 20-neuron network is probably not enough, feel free to experiment.\n", "\n", @@ -332,7 +332,9 @@ "
" ] }, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "display_data" } ], @@ -359,7 +361,7 @@ " ax.set_xlabel('position (x)')\n", " ax.set_ylabel('velocity (v)')\n", " \n", - " # Sample a trajectory and draw it\n", + " # Sample the trajectory and draw it\n", " states, actions, _ = generate_session(env, agent)\n", " states = np.array(states)\n", " ax.plot(states[:, 0], states[:, 1], color='white')\n", @@ -416,5 +418,5 @@ } }, "nbformat": 4, - "nbformat_minor": 1 + "nbformat_minor": 0 } diff --git a/week1_intro/gym_interface.ipynb b/week1_intro/gym_interface.ipynb index 4ef90609f..db6295a90 100644 --- a/week1_intro/gym_interface.ipynb +++ b/week1_intro/gym_interface.ipynb @@ -15,8 +15,8 @@ "\n", " !touch .setup_complete\n", "\n", - "# This code creates a virtual display to draw game images on.\n", - "# It will have no effect if your machine has a monitor.\n", + "# This code creates a virtual display for drawing game images.\n", + "# It has no effect if your machine has a monitor.\n", "if type(os.environ.get(\"DISPLAY\")) is not str or len(os.environ.get(\"DISPLAY\")) == 0:\n", " !bash ../xvfb start\n", " os.environ['DISPLAY'] = ':1'" @@ -39,9 +39,9 @@ "source": [ "### OpenAI Gym\n", "\n", - "We're gonna spend several next weeks learning algorithms that solve decision processes. We are then in need of some interesting decision problems to test our algorithms.\n", + "We're gonna spend several next weeks learning algorithms that solve decision processes. So we need a few interesting decision problems to test our algorithms.\n", "\n", - "That's where OpenAI Gym comes into play. It's a Python library that wraps many classical decision problems including robot control, videogames and board games.\n", + "That's where OpenAI Gym comes into play. It's a Python library that wraps many classical decision problems, including robot control, videogames and board games.\n", "\n", "So here's how it works:" ] @@ -66,7 +66,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note: if you're running this on your local machine, you'll see a window pop up with the image above. Don't close it, just alt-tab away." + "Note: if you're running this on your local machine, you'll see a window popping up with the image above. Don't close it, just alt-tab away." ] }, { @@ -75,13 +75,13 @@ "source": [ "### Gym interface\n", "\n", - "The three main methods of an environment are\n", - "* `reset()`: reset environment to the initial state, _return first observation_\n", - "* `render()`: show current environment state (a more colorful version :) )\n", - "* `step(a)`: commit action `a` and return `(new_observation, reward, is_done, info)`\n", + "The three main methods of this environment are:\n", + "* `reset()`: resets an environment to the initial state, _return first observation_\n", + "* `render()`: shows the current environment state (a more colorful version :) )\n", + "* `step(a)`: commits an action `a` and returns `(new_observation, reward, is_done, info)`\n", " * `new_observation`: an observation right after committing the action `a`\n", - " * `reward`: a number representing your reward for committing action `a`\n", - " * `is_done`: True if the MDP has just finished, False if still in progress\n", + " * `reward`: a number which represents your reward for committing action `a`\n", + " * `is_done`: True if the MDP has just finished, False if it is still in progress\n", " * `info`: some auxiliary stuff about what just happened. For now, ignore it." ] }, @@ -94,7 +94,7 @@ "obs0 = env.reset()\n", "print(\"initial observation code:\", obs0)\n", "\n", - "# Note: in MountainCar, observation is just two numbers: car position and velocity" + "# Note: in MountainCar, an observation is just two numbers: car position and velocity" ] }, { @@ -110,7 +110,7 @@ "print(\"reward:\", reward)\n", "print(\"is game over?:\", is_done)\n", "\n", - "# Note: as you can see, the car has moved to the right slightly (around 0.0005)" + "# Note: as you can see, the car has moved slightly to the right (around 0.0005)" ] }, { @@ -119,7 +119,7 @@ "source": [ "### Play with it\n", "\n", - "Below is the code that drives the car to the right. However, if you simply use the default policy, the car will not reach the flag at the far right due to gravity.\n", + "Below is the code that drives the car to the right. However, if you simply use the default policy, the car won't reach the flag at the far right due to the gravity.\n", "\n", "__Your task__ is to fix it. Find a strategy that reaches the flag. \n", "\n", @@ -151,14 +151,14 @@ "source": [ "def policy(obs, t):\n", " # Write the code for your policy here. You can use the observation\n", - " # (a tuple of position and velocity), the current time step, or both,\n", + " # (a tuple of the position and the velocity), the current time step, or both,\n", " # if you want.\n", " position, velocity = obs\n", " \n", - " # This is an example policy. You can try running it, but it will not work.\n", + " # This is an example policy. You can try running it, but it won't work.\n", " # Your goal is to fix that. You don't need anything sophisticated here,\n", " # and you can hard-code any policy that seems to work.\n", - " # Hint: think how you would make a swing go farther and faster.\n", + " # Hint: think how you would make a swing go faster and faster.\n", " return actions['right']" ] }, @@ -178,7 +178,7 @@ " action = policy(obs, t) # Call your policy\n", " obs, reward, done, _ = env.step(action) # Pass the action chosen by the policy to the environment\n", " \n", - " # We don't do anything with reward here because MountainCar is a very simple environment,\n", + " # We won't do anything with reward here because MountainCar is a very simple environment,\n", " # and reward is a constant -1. Therefore, your goal is to end the episode as quickly as possible.\n", "\n", " # Draw game image on display.\n", @@ -214,5 +214,5 @@ } }, "nbformat": 4, - "nbformat_minor": 1 + "nbformat_minor": 0 } diff --git a/week1_intro/primer/recap_ml.ipynb b/week1_intro/primer/recap_ml.ipynb index 920699541..c67c789c3 100644 --- a/week1_intro/primer/recap_ml.ipynb +++ b/week1_intro/primer/recap_ml.ipynb @@ -14,7 +14,7 @@ "\n", "---\n", "\n", - "This notebook is going to teach you to use the basic data science stack for Python: Jupyter, Numpy, matplotlib, and sklearn." + "This notebook will teach you how to use the basic data science stack in Python: Jupyter, Numpy, matplotlib and sklearn." ] }, { @@ -23,13 +23,13 @@ "source": [ "### Part I: Jupyter notebooks in a nutshell\n", "* You are reading this line in a jupyter notebook.\n", - "* A notebook consists of cells. A cell can contain either code or hypertext. \n", - " * This cell contains hypertext. The next cell contains code.\n", - "* You can __run a cell__ with code by selecting it (click) and pressing `Ctrl + Enter` to execute the code and display output(if any).\n", - "* If you're running this on a device with no keyboard, ~~you are doing it wrong~~ use the top bar (esp. play/stop/restart buttons) to run code.\n", - "* Behind the curtains, there's a Python interpreter that runs that code and remembers anything you defined.\n", + "* A notebook consists of cells, which can contain either code or hypertext. \n", + " * This cell contains hypertext, while the next one contains code.\n", + "* You can __run a cell__ with code by selecting it (click) and pressing `Ctrl + Enter` to execute the code and display the output (if any).\n", + "* If you're running this on a device with no keyboard, ~~you are doing it wrong~~ use the top bar (esp. play/stop/restart buttons) to run the code.\n", + "* Behind the curtains, there's a Python interpreter, that runs the code and remembers everything, that was defined.\n", "\n", - "Run these cells to get started" + "Run these cells to get started:" ] }, { @@ -62,9 +62,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "* `Ctrl + S` to save changes (or use the button that looks like a floppy disk)\n", - "* Top menu → Kernel → Interrupt (or Stop button) if you want it to stop running cell midway.\n", - "* Top menu → Kernel → Restart (or cyclic arrow button) if interrupt doesn't fix the problem (you will lose all variables).\n", + "* Press `Ctrl + S` to save changes (or use the button, that looks like a floppy disk)\n", + "* Use top menu → Kernel → Interrupt (or Stop button), if you want to stop running the cell midway.\n", + "* Use top menu → Kernel → Restart (or cyclic arrow button), if interruption doesn't fix the problem (be aware, you will lose all defined variables).\n", "* For shortcut junkies like us: Top menu → Help → Keyboard Shortcuts\n", "\n", "\n", @@ -72,7 +72,7 @@ "\n", "Now __the most important feature__ of jupyter notebooks for this course: \n", "* if you're typing something, press `Tab` to see automatic suggestions, use arrow keys + enter to pick one.\n", - "* if you move your cursor inside some function and press `Shift + Tab`, you'll get a help window. `Shift + (Tab , Tab)` (press `Tab` twice) will expand it." + "* if you move your cursor inside a function and press `Shift + Tab`, a help window will appear. `Shift + (Tab, Tab)` (press `Tab` twice) expands it." ] }, { @@ -92,7 +92,7 @@ "outputs": [], "source": [ "# place your cursor at the end of the unfinished line below to find a function\n", - "# that computes arctangent from two parameters (should have 2 in it's name)\n", + "# that computes arctangent from two parameters (has '2' in it's name)\n", "# once you chose it, press shift + tab + tab(again) to see the docs\n", "\n", "math.a # <---" @@ -103,9 +103,9 @@ "metadata": {}, "source": [ "### Part II: Loading data with Pandas\n", - "Pandas is a library that helps you load the data, prepare it and perform some lightweight analysis. The god object here is the `pandas.DataFrame` - a 2D table with batteries included. \n", + "Pandas is the library that helps you to load the data, prepare it and perform some lightweight analysis. The god object here is the `pandas.DataFrame` - a 2D table with included batteries. \n", "\n", - "In the cells below we use it to read the data on the infamous titanic shipwreck.\n", + "We use it in the cells below to read the data on the infamous titanic shipwreck.\n", "\n", "__please keep running all the code cells as you read__" ] @@ -116,8 +116,8 @@ "metadata": {}, "outputs": [], "source": [ - "# If you are running in Google Colab, this cell will download the dataset from our repository.\n", - "# Otherwise, this cell will do nothing.\n", + "# In Google Colab this cell will download the dataset from our repository.\n", + "# Otherwise, this cell won't do anything.\n", "\n", "import sys\n", "if 'google.colab' in sys.modules:\n", @@ -131,7 +131,7 @@ "outputs": [], "source": [ "import pandas as pd\n", - "# this yields a pandas.DataFrame\n", + "# this creates a pandas.DataFrame\n", "data = pd.read_csv(\"train.csv\", index_col='PassengerId')" ] }, @@ -375,7 +375,9 @@ ] }, "execution_count": null, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "execute_result" } ], @@ -391,17 +393,17 @@ "metadata": {}, "source": [ "#### About the data\n", - "Here's some of the columns\n", - "* Name - a string with person's full name\n", - "* Survived - 1 if a person survived the shipwreck, 0 otherwise.\n", - "* Pclass - passenger class. Pclass == 3 is cheap'n'cheerful, Pclass == 1 is for moneybags.\n", - "* Sex - a person's gender (in those good ol' times when there were just 2 of them)\n", - "* Age - age in years, if available\n", - "* Sibsp - number of siblings on a ship\n", - "* Parch - number of parents on a ship\n", - "* Fare - ticket cost\n", - "* Embarked - port where the passenger embarked\n", - " * C = Cherbourg; Q = Queenstown; S = Southampton" + "Here are some of the columns:\n", + "* Name - a string with person's full name;\n", + "* Survived - 1 - if a person survived the shipwreck, 0 - otherwise;\n", + "* Pclass - passenger class. Pclass == 3 is cheap'n'cheerful, Pclass == 1 is for moneybags;\n", + "* Sex - a person's gender (in those good ol' times, when there were just 2 of them);\n", + "* Age - age in years, if available;\n", + "* Sibsp - number of siblings on a ship;\n", + "* Parch - number of parents on a ship;\n", + "* Fare - ticket cost;\n", + "* Embarked - port, where the passenger embarked;\n", + " * C = Cherbourg; Q = Queenstown; S = Southampton." ] }, { @@ -507,7 +509,7 @@ } ], "source": [ - "# select a single column.\n", + "# select a single column\n", "ages = data[\"Age\"]\n", "print(ages[:10]) # alternatively: data.Age" ] @@ -594,7 +596,9 @@ ] }, "execution_count": null, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "execute_result" } ], @@ -617,7 +621,7 @@ "metadata": {}, "outputs": [], "source": [ - "# Select passengers number 13 and 666 (with these PassengerId values). Did they survive?\n", + "# Select passengers under number 13 and 666 (with these PassengerId values). Did they survive?\n", "\n", "" ] @@ -644,7 +648,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Pandas also has some basic data analysis tools. For one, you can quickly display statistical aggregates for each column using `.describe()`" + "Pandas also has some basic data analysis tools. For one, you can quickly display statistical aggregates for each column using `.describe()`." ] }, { @@ -660,9 +664,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Some columns contain __NaN__ values - this means that there is no data there. For example, passenger `#6` has unknown age. To simplify the future data analysis, we'll replace NaN values by using pandas `fillna` function.\n", + "Some columns contain __NaN__ values - this means that the data is missing. For example, age of the passenger `#6` is unknown. To simplify the future data analysis, we'll replace all NaN values by using pandas `fillna` function.\n", "\n", - "_Note: we do this so easily because it's a tutorial. In general, you think twice before you modify data like this._" + "_Note: we do this so easy because it's a tutorial. In general, you should think twice before you modify the data like this._" ] }, { @@ -697,9 +701,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "More pandas: \n", - "* A neat [tutorial](http://pandas.pydata.org/) from pydata\n", - "* Official [tutorials](https://pandas.pydata.org/pandas-docs/stable/tutorials.html), including this [10 minutes to pandas](https://pandas.pydata.org/pandas-docs/stable/10min.html#min)\n", + "More on pandas: \n", + "* A neat [tutorial](http://pandas.pydata.org/) from pydata;\n", + "* Official [tutorials](https://pandas.pydata.org/pandas-docs/stable/tutorials.html), including this [10 minutes to pandas](https://pandas.pydata.org/pandas-docs/stable/10min.html#min);\n", "* Bunch of cheat sheets awaits just one google query away from you (e.g. [basics](http://blog.yhat.com/static/img/datacamp-cheat.png), [combining datasets](https://pbs.twimg.com/media/C65MaMpVwAA3v0A.jpg) and so on). " ] }, @@ -709,9 +713,9 @@ "source": [ "### Part III: Numpy and vectorized computing\n", "\n", - "Almost any machine learning model requires some computational heavy lifting usually involving linear algebra problems. Unfortunately, raw Python is terrible at this because each operation is interpreted at runtime. \n", + "Almost any machine learning model requires some computational heavy lifting, usually involving linear algebra. Unfortunately, raw Python is terrible at this, because each operation is interpreted at runtime. \n", "\n", - "So instead, we'll use `numpy` - a library that lets you run blazing fast computation with vectors, matrices and other tensors. Again, the god object here is `numpy.ndarray`:" + "So instead, we'll use `numpy` - the library, that lets you run blazing fast computation with vectors, matrices and other tensors. Again, the god object here is `numpy.ndarray`:" ] }, { @@ -781,8 +785,8 @@ "### How fast is it, Harry?\n", "![img](https://img.buzzfeed.com/buzzfeed-static/static/2015-11/6/7/enhanced/webdr10/enhanced-buzz-22847-1446811476-0.jpg)\n", "\n", - "Let's compare computation time for Python and Numpy\n", - "* Two arrays of $10^6$ elements\n", + "Let's compare the computation time for Python and Numpy:\n", + "* Two arrays of $10^6$ elements:\n", " * first one: from 0 to 1 000 000\n", " * second one: from 99 to 1 000 099\n", " \n", @@ -801,7 +805,7 @@ "outputs": [], "source": [ "%%time\n", - "# ^-- this \"magic\" measures and prints cell computation time\n", + "# ^-- this \"magic\" measures and prints the cell computation time\n", "\n", "# Option I: pure Python\n", "arr_1 = range(1000000)\n", @@ -872,7 +876,7 @@ "---\n", "\n", "\n", - "There's also a bunch of pre-implemented operations including logarithms, trigonometry, vector/matrix products and aggregations." + "There's also a bunch of pre-implemented operations: logarithms, trigonometry, vector/matrix products, aggregations and others." ] }, { @@ -954,7 +958,7 @@ "metadata": {}, "outputs": [], "source": [ - "# your code: compute mean passenger age and the oldest guy on the ship\n", + "# your code: compute the mean passenger age and define the oldest guy on the ship\n", "" ] }, @@ -2079,7 +2083,9 @@ ] }, "execution_count": null, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "execute_result" } ], @@ -2087,11 +2093,11 @@ "a = np.array([0, 1, 4, 9, 16, 25])\n", "ix = np.array([1, 2, 5])\n", "print(\"a =\", a)\n", - "print(\"Select by element index\")\n", + "print(\"Select by the element index\")\n", "print(\"a[[1,2,5]] =\", a[ix])\n", "\n", "print(\"\\nSelect by boolean mask\")\n", - "# select all elementts in a that are greater than 5\n", + "# select all elements in 'a' that are greater than 5\n", "print(\"a[a > 5] =\", a[a > 5])\n", "print(\"(a % 2 == 0) =\", a % 2 == 0) # True for even, False for odd\n", "print(\"a[a % 2 == 0] =\", a[a % 2 == 0]) # select all elements in a that are even\n", @@ -2106,9 +2112,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Your turn\n", + "It's your turn\n", "\n", - "Use numpy and pandas to answer a few questions about data" + "Use numpy and pandas to answer a few questions about the data" ] }, { @@ -2117,7 +2123,7 @@ "metadata": {}, "outputs": [], "source": [ - "# who on average paid more for their ticket, men or women?\n", + "# who, men or women, paid more on average for their ticket?\n", "\n", "mean_fare_men = \n", "mean_fare_women = \n", @@ -2147,7 +2153,7 @@ "\n", "Using Python to visualize the data is covered by yet another library: matplotlib.\n", "\n", - "Just like Python itself, matplotlib has an awesome tendency of keeping simple things simple while still allowing you to write complicated stuff with convenience (e.g. super-detailed plots or custom animations)." + "Just like Python itself, matplotlib has an awesome tendency of keeping simple things simple, while still allowing you to write complicated stuff with convenience (e.g. super-detailed plots or custom animations)." ] }, { @@ -2162,7 +2168,9 @@ ] }, "execution_count": null, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "execute_result" }, { @@ -2172,14 +2180,16 @@ "" ] }, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", - "# ^-- this \"magic\" tells all future matplotlib plots to be drawn inside notebook and not in a separate window.\n", + "# ^-- this \"magic\" tells all future matplotlib plots to be drawn inside a notebook, not in a separate window.\n", "\n", "# line plot\n", "plt.plot([0, 1, 2, 3, 4, 5], [0, 1, 4, 9, 16, 25])" @@ -2197,7 +2207,9 @@ "" ] }, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "display_data" } ], @@ -2205,7 +2217,7 @@ "# scatter-plot\n", "plt.scatter([0, 1, 2, 3, 4, 5], [0, 1, 4, 9, 16, 25])\n", "\n", - "plt.show() # show the first plot and begin drawing next one" + "plt.show() # shows the first plot and begins to draw the next one" ] }, { @@ -2220,7 +2232,9 @@ ] }, "execution_count": null, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "execute_result" }, { @@ -2230,7 +2244,9 @@ "" ] }, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "display_data" } ], @@ -2262,7 +2278,9 @@ "" ] }, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "display_data" }, { @@ -2274,7 +2292,9 @@ ] }, "execution_count": null, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "execute_result" }, { @@ -2284,7 +2304,9 @@ "" ] }, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "display_data" } ], @@ -2303,7 +2325,7 @@ "metadata": {}, "outputs": [], "source": [ - "# plot a histogram of age and a histogram of ticket fares on separate plots\n", + "# plot the histogram of age and the histogram of ticket fares on separate plots\n", "\n", "\n", "\n", @@ -2342,7 +2364,7 @@ "\n", "Scikit-learn is _the_ tool for simple machine learning pipelines. \n", "\n", - "It's a single library that unites a whole bunch of models under the common interface:\n", + "It's a single library that unites a whole bunch of models under a common interface:\n", "* Create: `model = sklearn.whatever.ModelNameHere(parameters_if_any)`\n", "* Train: `model.fit(X, y)`\n", "* Predict: `model.predict(X_test)`\n", @@ -2418,5 +2440,5 @@ } }, "nbformat": 4, - "nbformat_minor": 1 + "nbformat_minor": 0 } diff --git a/week1_intro/primer/recap_pytorch.ipynb b/week1_intro/primer/recap_pytorch.ipynb index d11679fa7..fb0f7c823 100644 --- a/week1_intro/primer/recap_pytorch.ipynb +++ b/week1_intro/primer/recap_pytorch.ipynb @@ -8,11 +8,11 @@ "\n", "![img](https://pytorch.org/tutorials/_static/pytorch-logo-dark.svg)\n", "\n", - "__This notebook__ will teach you to use PyTorch low-level core. If you're running this notebook outside the course environment, you can install it [here](https://pytorch.org).\n", + "__This notebook__ will teach you how to use PyTorch low-level core. If you're running this notebook outside the course environment, you can install it [here](https://pytorch.org).\n", "\n", - "__PyTorch feels__ differently than tensorflow/theano on almost every level. TensorFlow makes your code live in two \"worlds\" simultaneously: symbolic graphs and actual tensors. First you declare a symbolic \"recipe\" of how to get from inputs to outputs, then feed it with actual minibatches of data. In PyTorch, __there's only one world__: all tensors have a numeric value.\n", + "__PyTorch feels__ differently than tensorflow/theano at almost every level. TensorFlow makes your code live in two \"worlds\" simultaneously: symbolic graphs and actual tensors. First you declare a symbolic \"recipe\" of how to get from inputs to outputs, then feed it with actual minibatches of data. In PyTorch, __there's only one world__: all tensors have a numeric value.\n", "\n", - "You compute outputs on the fly without pre-declaring anything. The code looks exactly as in pure numpy with one exception: PyTorch computes gradients for you. And can run stuff on GPU. And has a number of pre-implemented building blocks for your neural nets. [And a few more things.](https://medium.com/towards-data-science/pytorch-vs-tensorflow-spotting-the-difference-25c75777377b)\n", + "You compute outputs on the fly without pre-declaring anything. The code looks exactly as in pure numpy with one exception: PyTorch computes gradients for you, can run stuff on GPU and has a number of pre-implemented building blocks for your neural nets. [And a few more things.](https://medium.com/towards-data-science/pytorch-vs-tensorflow-spotting-the-difference-25c75777377b)\n", "\n", "And now we finally shut up and let PyTorch do the talking." ] @@ -159,9 +159,9 @@ "source": [ "## NumPy and PyTorch\n", "\n", - "As you can notice, PyTorch allows you to hack stuff much the same way you did with NumPy. No graph declaration, no placeholders, no sessions. This means that you can _see the numeric value of any tensor at any moment of time_. Debugging such code can be done with by printing tensors or using any debug tool you want (e.g. [PyCharm debugger](https://www.jetbrains.com/help/pycharm/part-1-debugging-python-code.html) or [gdb](https://wiki.python.org/moin/DebuggingWithGdb)).\n", + "As you can notice, PyTorch allows you to hack stuff more or less the same way you did with NumPy. No graph declaration, no placeholders, no sessions. This means that you can _see the numeric value of any tensor at any moment of time_. Debugging such code can be done by printing tensors or using any debug tool you want (e.g. [PyCharm debugger](https://www.jetbrains.com/help/pycharm/part-1-debugging-python-code.html) or [gdb](https://wiki.python.org/moin/DebuggingWithGdb)).\n", "\n", - "You could also notice the a few new method names and a different API. So no, there's no compatibility with NumPy [yet](https://github.com/pytorch/pytorch/issues/2228) and yes, you'll have to memorize all the names again. Get excited!\n", + "You could also notice a few new method names and a different API. So no, there's no compatibility with NumPy [yet](https://github.com/pytorch/pytorch/issues/2228) and yes, you'll need to memorize all the names again. Get excited!\n", "\n", "![img](http://i0.kym-cdn.com/entries/icons/original/000/017/886/download.jpg)\n", "\n", @@ -170,14 +170,14 @@ " * `x.reshape([1,2,8]) -> x.view(1,2,8)`\n", "* You should swap `axis` for `dim` in operations like `mean` or `cumsum`\n", " * `x.sum(axis=-1) -> x.sum(dim=-1)`\n", - "* Most mathematical operations are the same, but types an shaping is different\n", + "* Most mathematical operations are the same, but types and shaping is different\n", " * `x.astype('int64') -> x.type(torch.LongTensor)`\n", "\n", - "To help you acclimatize, there's a [table](https://github.com/torch/torch7/wiki/Torch-for-NumPy-users) covering most new things. There's also a neat [documentation page](http://pytorch.org/docs/master/).\n", + "To help you acclimatize, there's a [table](https://github.com/torch/torch7/wiki/Torch-for-NumPy-users) covering most of new things. There's also a neat [documentation page](http://pytorch.org/docs/master/).\n", "\n", - "Finally, if you're stuck with a technical problem, we recommend searching [PyTorch forums](https://discuss.pytorch.org/). Or just googling, which usually works just as efficiently. \n", + "Finally, if you're stuck with a technical problem, we recommend searching [PyTorch forums](https://discuss.pytorch.org/) or googling, which usually works just as efficiently. \n", "\n", - "If you feel like you almost give up, remember two things: __GPU__ and __free gradients__. Besides you can always jump back to NumPy with `x.numpy()`." + "If you feel like you are ready to give up, remember two things: __GPU__ and __free gradients__. Besides you can always jump back to NumPy with `x.numpy()`." ] }, { @@ -215,7 +215,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "If you're done early, try adjusting the formula and seeing how it affects the function." + "If you're done, try adjusting the formula and seeing how it affects the function." ] }, { @@ -231,7 +231,7 @@ "source": [ "## Automatic gradients\n", "\n", - "Any self-respecting DL framework must do your backprop for you. Torch handles this with the `autograd` module.\n", + "Any self-respecting DL framework must do backprop for you. Torch handles this with the `autograd` module.\n", "\n", "The general pipeline looks like this:\n", "* When creating a tensor, you mark it as `requires_grad`:\n", @@ -256,7 +256,9 @@ ] }, "execution_count": null, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "execute_result" }, { @@ -267,7 +269,8 @@ ] }, "metadata": { - "needs_background": "light" + "needs_background": "light", + "tags": [] }, "output_type": "display_data" } @@ -308,7 +311,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The gradients are now stored in `.grad` of those variables that require them." + "The gradients are now stored in `.grad` of the variables that require them." ] }, { @@ -336,7 +339,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "If you compute gradient from multiple losses, the gradients will add up at variables, therefore it's useful to __zero the gradients__ between iteratons." + "If you compute gradient from multiple losses, the gradients will be added to variables, therefore, it's useful to __zero the gradients__ between iteratons." ] }, { @@ -352,7 +355,8 @@ ] }, "metadata": { - "needs_background": "light" + "needs_background": "light", + "tags": [] }, "output_type": "display_data" }, @@ -379,7 +383,7 @@ " w.grad.data.zero_()\n", " b.grad.data.zero_()\n", "\n", - " # the rest of code is just bells and whistles\n", + " # the rest of the code are just bells and whistles\n", " if (i + 1) % 5 == 0:\n", " clear_output(True)\n", " plt.scatter(x.numpy(), y.numpy())\n", @@ -437,7 +441,9 @@ "" ] }, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "display_data" } ], @@ -501,7 +507,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "There's a vast library of popular layers and architectures already built for ya'.\n", + "There's a detailed library of popular layers and architectures already built for you.\n", "\n", "This is a binary classification problem, so we'll train __Logistic Regression__.\n", "$$P(y_i | X_i) = \\sigma(W \\cdot X_i + b) ={ 1 \\over {1+e^{- [W \\cdot X_i + b]}} }$$\n" @@ -553,7 +559,9 @@ ] }, "execution_count": null, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "execute_result" } ], @@ -562,7 +570,7 @@ "x = torch.tensor(X_train[:3], dtype=torch.float32)\n", "y = torch.tensor(y_train[:3], dtype=torch.float32)\n", "\n", - "# compute outputs given inputs, both are variables\n", + "# compute outputs with given inputs, both are variables\n", "y_predicted = model(x)[:, 0]\n", "\n", "y_predicted # display what we've got" @@ -590,19 +598,19 @@ "loss = \n", "\n", "assert tuple(crossentropy.size()) == (\n", - " 3,), \"Crossentropy must be a vector with element per sample\"\n", + " 3,), \"Crossentropy must be a vector with an element per sample\"\n", "assert tuple(loss.size()) == tuple(\n", "), \"Loss must be scalar. Did you forget the mean/sum?\"\n", - "assert loss.data.numpy() > 0, \"Crossentropy must non-negative, zero only for perfect prediction\"\n", + "assert loss.data.numpy() > 0, \"Crossentropy must be non-negative, zero only for perfect prediction\"\n", "assert loss.data.numpy() <= np.log(\n", - " 3), \"Loss is too large even for untrained model. Please double-check it.\"" + " 3), \"Loss is too large even for an untrained model. Please double-check it.\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "__Note:__ you can also find many such functions in `torch.nn.functional`, just type __`F.`__." + "__Note:__ you can also find many other functions in `torch.nn.functional`, just type __`F.`__." ] }, { @@ -611,7 +619,7 @@ "source": [ "__Torch optimizers__\n", "\n", - "When we trained Linear Regression above, we had to manually `.zero_()` gradients on both our variables. Imagine that code for a 50-layer network.\n", + "When we trained Linear Regression above, we had to manually zero-out (`.zero_()`) gradients of both variables. Imagine this code for a 50-layer network.\n", "\n", "Again, to keep it from getting dirty, there's `torch.optim` module with pre-implemented algorithms:" ] @@ -723,10 +731,10 @@ "source": [ "__Debugging tips:__\n", "* Make sure your model predicts probabilities correctly. Just print them and see what's inside.\n", - "* Don't forget the _minus_ sign in the loss function! It's a mistake 99% people do at some point.\n", + "* Don't forget the _minus_ sign in the loss function! This is the mistake 99% people do at some point.\n", "* Make sure you zero-out gradients after each step. Seriously:)\n", - "* In general, PyTorch's error messages are quite helpful, read 'em before you google 'em.\n", - "* if you see nan/inf, print what happens at each iteration to find our where exactly it occurs.\n", + "* In general, PyTorch's error messages are quite helpful, read them before you google them.\n", + "* if you see nan/inf, print what happens at each iteration to find out, where exactly it occurs.\n", " * If loss goes down and then turns nan midway through, try smaller learning rate. (Our current loss formula is unstable)." ] }, @@ -774,8 +782,8 @@ "## More about PyTorch:\n", "* Using torch on GPU and multi-GPU - [link](http://pytorch.org/docs/master/notes/cuda.html)\n", "* More tutorials on PyTorch - [link](http://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html)\n", - "* PyTorch examples - a repo that implements many cool DL models in PyTorch - [link](https://github.com/pytorch/examples)\n", - "* Practical PyTorch - a repo that implements some... other cool DL models... yes, in PyTorch - [link](https://github.com/spro/practical-pytorch)\n", + "* PyTorch examples - a repo, that implements many cool DL models in PyTorch - [link](https://github.com/pytorch/examples)\n", + "* Practical PyTorch - a repo, that implements some... other cool DL models... yes, in PyTorch - [link](https://github.com/spro/practical-pytorch)\n", "* And some more - [link](https://www.reddit.com/r/pytorch/comments/6z0yeo/pytorch_and_pytorch_tricks_for_kaggle/)\n", "\n", "---" @@ -787,7 +795,7 @@ "source": [ "# Bonus tasks\n", "\n", - "If you get stuck with no progress, try switching to the next task and returning later." + "If you get stuck with no progress, try switching to the next task and returning to them later." ] }, { @@ -1034,7 +1042,9 @@ "" ] }, - "metadata": {}, + "metadata": { + "tags": [] + }, "output_type": "display_data" } ], @@ -1078,7 +1088,7 @@ " - Softmax (exp over sum of exps) can be implemented manually or as `nn.Softmax` (layer) or `F.softmax` (function)\n", " - Probably better to use STOCHASTIC gradient descent (minibatch) for greater speed\n", " - You can also try momentum/rmsprop/adawhatever\n", - " - in which case the dataset should probably be shuffled (or use random subsamples on each iteration)\n", + " - in this case the dataset should probably be shuffled (or use random subsamples on each iteration)\n", "* Add a hidden layer. Now your logistic regression uses hidden neurons instead of inputs.\n", " - Hidden layer uses the same math as output layer (ex-logistic regression), but uses some nonlinearity (e.g. sigmoid) instead of softmax\n", " - You need to train both layers, not just the output layer :)\n", @@ -1086,9 +1096,9 @@ " - In ideal case this totals to 2 `torch.matmul`'s, 1 softmax and 1 ReLU/sigmoid\n", " - __Make sure this neural network works better than logistic regression!__\n", " \n", - "* Now's the time to try improving the network. Consider layers (size, neuron count), nonlinearities, optimization methods, initialization — whatever you want, but please avoid convolutions for now.\n", + "* Now's the time to try improving the network. Consider layers (size, neuron count), nonlinearities, optimization methods, initialization — whatever you want, but, please, avoid convolutions for now.\n", " \n", - "* If anything seems wrong, try going through one step of training and printing everything you compute.\n", + "* If something seems wrong, try going through each step of training and print everything you compute.\n", "* If you see NaNs midway through optimization, you can estimate $\\log P(y \\mid x)$ as `F.log_softmax(layer_before_softmax)`." ] } @@ -1100,5 +1110,5 @@ } }, "nbformat": 4, - "nbformat_minor": 2 + "nbformat_minor": 0 }