exercise-a-single-neuron.ipynb

{"cells":[{"metadata":{},"cell_type":"markdown","source":"**This notebook is an exercise in the [Intro to Deep Learning](https://www.kaggle.com/learn/intro-to-deep-learning) course.  You can reference the tutorial at [this link](https://www.kaggle.com/ryanholbrook/a-single-neuron).**\n\n---\n"},{"metadata":{},"cell_type":"markdown","source":"# Introduction #\n\nIn the tutorial we learned about the building blocks of neural networks: *linear units*. We saw that a model of just one linear unit will fit a linear function to a dataset (equivalent to linear regression). In this exercise, you'll build a linear model and get some practice working with models in Keras.\n\nBefore you get started, run the code cell below to set everything up."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Setup plotting\nimport matplotlib.pyplot as plt\n\nplt.style.use('seaborn-whitegrid')\n# Set Matplotlib defaults\nplt.rc('figure', autolayout=True)\nplt.rc('axes', labelweight='bold', labelsize='large',\n       titleweight='bold', titlesize=18, titlepad=10)\n\n# Setup feedback system\nfrom learntools.core import binder\nbinder.bind(globals())\nfrom learntools.deep_learning_intro.ex1 import *","execution_count":1,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The *Red Wine Quality* dataset consists of physiochemical measurements from about 1600 Portuguese red wines.  Also included is a quality rating for each wine from blind taste-tests. \n\nFirst, run the next cell to display the first few rows of this dataset."},{"metadata":{"trusted":true},"cell_type":"code","source":"import pandas as pd\n\nred_wine = pd.read_csv('../input/dl-course-data/red-wine.csv')\nred_wine.head()","execution_count":2,"outputs":[{"output_type":"execute_result","execution_count":2,"data":{"text/plain":"   fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n0            7.4              0.70         0.00             1.9      0.076   \n1            7.8              0.88         0.00             2.6      0.098   \n2            7.8              0.76         0.04             2.3      0.092   \n3           11.2              0.28         0.56             1.9      0.075   \n4            7.4              0.70         0.00             1.9      0.076   \n\n   free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \\\n0                 11.0                  34.0   0.9978  3.51       0.56   \n1                 25.0                  67.0   0.9968  3.20       0.68   \n2                 15.0                  54.0   0.9970  3.26       0.65   \n3                 17.0                  60.0   0.9980  3.16       0.58   \n4                 11.0                  34.0   0.9978  3.51       0.56   \n\n   alcohol  quality  \n0      9.4        5  \n1      9.8        5  \n2      9.8        5  \n3      9.8        6  \n4      9.4        5  ","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>fixed acidity</th>\n      <th>volatile acidity</th>\n      <th>citric acid</th>\n      <th>residual sugar</th>\n      <th>chlorides</th>\n      <th>free sulfur dioxide</th>\n      <th>total sulfur dioxide</th>\n      <th>density</th>\n      <th>pH</th>\n      <th>sulphates</th>\n      <th>alcohol</th>\n      <th>quality</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>7.4</td>\n      <td>0.70</td>\n      <td>0.00</td>\n      <td>1.9</td>\n      <td>0.076</td>\n      <td>11.0</td>\n      <td>34.0</td>\n      <td>0.9978</td>\n      <td>3.51</td>\n      <td>0.56</td>\n      <td>9.4</td>\n      <td>5</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>7.8</td>\n      <td>0.88</td>\n      <td>0.00</td>\n      <td>2.6</td>\n      <td>0.098</td>\n      <td>25.0</td>\n      <td>67.0</td>\n      <td>0.9968</td>\n      <td>3.20</td>\n      <td>0.68</td>\n      <td>9.8</td>\n      <td>5</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>7.8</td>\n      <td>0.76</td>\n      <td>0.04</td>\n      <td>2.3</td>\n      <td>0.092</td>\n      <td>15.0</td>\n      <td>54.0</td>\n      <td>0.9970</td>\n      <td>3.26</td>\n      <td>0.65</td>\n      <td>9.8</td>\n      <td>5</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>11.2</td>\n      <td>0.28</td>\n      <td>0.56</td>\n      <td>1.9</td>\n      <td>0.075</td>\n      <td>17.0</td>\n      <td>60.0</td>\n      <td>0.9980</td>\n      <td>3.16</td>\n      <td>0.58</td>\n      <td>9.8</td>\n      <td>6</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>7.4</td>\n      <td>0.70</td>\n      <td>0.00</td>\n      <td>1.9</td>\n      <td>0.076</td>\n      <td>11.0</td>\n      <td>34.0</td>\n      <td>0.9978</td>\n      <td>3.51</td>\n      <td>0.56</td>\n      <td>9.4</td>\n      <td>5</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"You can get the number of rows and columns of a dataframe (or a Numpy array) with the `shape` attribute."},{"metadata":{"trusted":true},"cell_type":"code","source":"red_wine.shape # (rows, columns)","execution_count":3,"outputs":[{"output_type":"execute_result","execution_count":3,"data":{"text/plain":"(1599, 12)"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"# 1) Input shape #\n\nHow well can we predict a wine's perceived quality from the physiochemical measurements?  \n\nThe target is `'quality'`, and the remaining columns are the features.  How would you set the `input_shape` parameter for a Keras model on this task?"},{"metadata":{"lines_to_next_cell":0,"trusted":true},"cell_type":"code","source":"# YOUR CODE HERE\ninput_shape = [11]\n\n# Check your answer\nq_1.check()","execution_count":4,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"1_Q1\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Correct","text/markdown":"<span style=\"color:#33cc33\">Correct</span>"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Lines below will give you a hint or solution code\nq_1.hint()\nq_1.solution()","execution_count":5,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"interactionType\": 2, \"questionType\": 2, \"questionId\": \"1_Q1\", \"learnToolsVersion\": \"0.3.4\", \"valueTowardsCompletion\": 0.0, \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\", \"outcomeType\": 4}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Hint: Remember to *only* count the input features when determining `input_shape`. You should not count the target (the `quality` column).","text/markdown":"<span style=\"color:#3366cc\">Hint:</span> Remember to *only* count the input features when determining `input_shape`. You should not count the target (the `quality` column)."},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"interactionType\": 3, \"questionType\": 2, \"questionId\": \"1_Q1\", \"learnToolsVersion\": \"0.3.4\", \"valueTowardsCompletion\": 0.0, \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\", \"outcomeType\": 4}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Solution: \n```python\n\ninput_shape = [11]\n# you could also use a 1-tuple, like input_shape = (11,)\n\n```","text/markdown":"<span style=\"color:#33cc99\">Solution:</span> \n```python\n\ninput_shape = [11]\n# you could also use a 1-tuple, like input_shape = (11,)\n\n```"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"# 2) Define a linear model\n\nNow define a linear model appropriate for this task. Pay attention to how many inputs and outputs the model should have."},{"metadata":{"lines_to_next_cell":0,"trusted":true},"cell_type":"code","source":"from tensorflow import keras\nfrom tensorflow.keras import layers\n\n# YOUR CODE HERE\nmodel = keras.Sequential([\n    layers.Dense(units=1, input_shape=[11])\n])\n\n# Check your answer\nq_2.check()","execution_count":6,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"2_Q2\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Correct","text/markdown":"<span style=\"color:#33cc33\">Correct</span>"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Lines below will give you a hint or solution code\nq_2.hint()\nq_2.solution()","execution_count":7,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"interactionType\": 2, \"questionType\": 2, \"questionId\": \"2_Q2\", \"learnToolsVersion\": \"0.3.4\", \"valueTowardsCompletion\": 0.0, \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\", \"outcomeType\": 4}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Hint: Your answer should look something like:\n```python\nmodel = keras.Sequential([\n    ____\n])\n```","text/markdown":"<span style=\"color:#3366cc\">Hint:</span> Your answer should look something like:\n```python\nmodel = keras.Sequential([\n    ____\n])\n```\n"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"interactionType\": 3, \"questionType\": 2, \"questionId\": \"2_Q2\", \"learnToolsVersion\": \"0.3.4\", \"valueTowardsCompletion\": 0.0, \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\", \"outcomeType\": 4}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Solution: \n```python\n\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\n\nmodel = keras.Sequential([\n    layers.Dense(units=1, input_shape=[11])\n])\n\n```","text/markdown":"<span style=\"color:#33cc99\">Solution:</span> \n```python\n\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\n\nmodel = keras.Sequential([\n    layers.Dense(units=1, input_shape=[11])\n])\n\n```"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"# 3) Look at the weights\n\nInternally, Keras represents the weights of a neural network with **tensors**. Tensors are basically TensorFlow's version of a Numpy array with a few differences that make them better suited to deep learning. One of the most important is that tensors are compatible with [GPU](https://www.kaggle.com/docs/efficient-gpu-usage) and [TPU](https://www.kaggle.com/docs/tpu)) accelerators. TPUs, in fact, are designed specifically for tensor computations.\n\nA model's weights are kept in its `weights` attribute as a list of tensors. Get the weights of the model you defined above. (If you want, you could display the weights with something like: `print(\"Weights\\n{}\\n\\nBias\\n{}\".format(w, b))`)."},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Weights\\n{}\\n\\nBias\\n{}\".format(w, b)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"red_wine.columns","execution_count":8,"outputs":[{"output_type":"execute_result","execution_count":8,"data":{"text/plain":"Index(['fixed acidity', 'volatile acidity', 'citric acid', 'residual sugar',\n       'chlorides', 'free sulfur dioxide', 'total sulfur dioxide', 'density',\n       'pH', 'sulphates', 'alcohol', 'quality'],\n      dtype='object')"},"metadata":{}}]},{"metadata":{"lines_to_next_cell":0,"trusted":true},"cell_type":"code","source":"# YOUR CODE HERE\nw, b = model.weights\n\nprint(\"Weights\\n{}\\n\\nBias\\n{}\".format(w, b))\n\n# Check your answer\nq_3.check()","execution_count":15,"outputs":[{"output_type":"error","ename":"SyntaxError","evalue":"invalid syntax (<ipython-input-15-ed5f2fbb5b6c>, line 4)","traceback":["\u001b[0;36m  File \u001b[0;32m\"<ipython-input-15-ed5f2fbb5b6c>\"\u001b[0;36m, line \u001b[0;32m4\u001b[0m\n\u001b[0;31m    print(\"Weights\\n{}\\n\\nBias\\n{}\".format(w, b)))\u001b[0m\n\u001b[0m                                                 ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"]}]},{"metadata":{"lines_to_next_cell":0,"trusted":true},"cell_type":"code","source":"# Lines below will give you a hint or solution code\nq_3.hint()\nq_3.solution()","execution_count":12,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"interactionType\": 2, \"questionType\": 2, \"questionId\": \"3_Q3\", \"learnToolsVersion\": \"0.3.4\", \"valueTowardsCompletion\": 0.0, \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\", \"outcomeType\": 4}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Hint: You can get the attribute of an object using the 'dot' notation: like `object.attribute`.","text/markdown":"<span style=\"color:#3366cc\">Hint:</span> You can get the attribute of an object using the 'dot' notation: like `object.attribute`."},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"interactionType\": 3, \"questionType\": 2, \"questionId\": \"3_Q3\", \"learnToolsVersion\": \"0.3.4\", \"valueTowardsCompletion\": 0.0, \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\", \"outcomeType\": 4}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Solution: \n```python\n\n# Uncomment if you need the model from the previous question:\n# model = keras.Sequential([\n#     layers.Dense(units=1, input_shape=[11])\n# ])\n\nw, b = model.weights\n\nprint(\"Weights\\n{}\\n\\nBias\\n{}\".format(w, b))\n\n```","text/markdown":"<span style=\"color:#33cc99\">Solution:</span> \n```python\n\n# Uncomment if you need the model from the previous question:\n# model = keras.Sequential([\n#     layers.Dense(units=1, input_shape=[11])\n# ])\n\nw, b = model.weights\n\nprint(\"Weights\\n{}\\n\\nBias\\n{}\".format(w, b))\n\n```"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"(By the way, Keras represents weights as tensors, but also uses tensors to represent data. When you set the `input_shape` argument, you are telling Keras the dimensions of the array it should expect for each example in the training data. Setting `input_shape=[3]` would create a network accepting vectors of length 3, like `[0.2, 0.4, 0.6]`.)\n \n\n# Optional: Plot the output of an untrained linear model\n \nThe kinds of problems we'll work on through Lesson 5 will be *regression* problems, where the goal is to predict some numeric target. Regression problems are like \"curve-fitting\" problems: we're trying to find a curve that best fits the data. Let's take a look at the \"curve\" produced by a linear model. (You've probably guessed that it's a line!)\n \nWe mentioned that before training a model's weights are set randomly. Run the cell below a few times to see the different lines produced with a random initialization. (There's no coding for this exercise -- it's just a demonstration.)"},{"metadata":{"lines_to_next_cell":0,"trusted":false},"cell_type":"code","source":"import tensorflow as tf\nimport matplotlib.pyplot as plt\n\nmodel = keras.Sequential([\n    layers.Dense(1, input_shape=[1]),\n])\n\nx = tf.linspace(-1.0, 1.0, 100)\ny = model.predict(x)\n\nplt.figure(dpi=100)\nplt.plot(x, y, 'k')\nplt.xlim(-1, 1)\nplt.ylim(-1, 1)\nplt.xlabel(\"Input: x\")\nplt.ylabel(\"Target y\")\nw, b = model.weights # you could also use model.get_weights() here\nplt.title(\"Weight: {:0.2f}\\nBias: {:0.2f}\".format(w[0][0], b[0]))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Keep Going #\n\nAdd hidden layers and [**make your models deep**](https://www.kaggle.com/ryanholbrook/deep-neural-networks) in Lesson 2."},{"metadata":{},"cell_type":"markdown","source":"---\n\n\n\n\n*Have questions or comments? Visit the [Learn Discussion forum](https://www.kaggle.com/learn-forum/191966) to chat with other Learners.*"}],"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.7.9","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":4}