Copy_of_M5Lab_Neural_Networks_Predict.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Copy of M5Lab-Neural Networks-Predict.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# In-Class Lab: Neural Networks\n",
        "\n",
        "In this lab, we will implement a neural network to solve the XOR problem.  For now, we will only implement the predict method, given weights that have been pre-computed (that were extracted from an already trained network).\n",
        "\n",
        "The architecture of the network would be similar to the main example we \n",
        "showed in class:\n",
        "\n",
        "* Input layer: 2 neurons (for the two inputs of XOR function)\n",
        "\n",
        "* Hidden layer: 3 neurons (we will have only 1 hidden layer with three neurons)\n",
        "\n",
        "* Output layer: 1 neuron (1 neuron in the output as output will be a single digit (0 or 1))\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "VbUw_vkL9OFJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Part 1: Setup & Initialization"
      ],
      "metadata": {
        "id": "RG30xBxWpZDJ"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "x_seJiPMNmjd"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "\n",
        "# helper function to implement tanh\n",
        "def tanh(x): \n",
        "  return np.tanh(x)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Initializing the weights for the different layers of the network"
      ],
      "metadata": {
        "id": "uySXcFYfyI9I"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# weights for the layer between the inputs and the hidden layer\n",
        "weights_l1 = (np.reshape([1.33802204,  0.27270439, -2.32711844,1.06193803,  0.77875626, -2.36624024], (2,3)))\n",
        "\n",
        "# weights for the layer between the hidden layer and the output layer\n",
        "weights_l2 = [[-2.16869671], [-1.09397363], [-2.61705925]]\n",
        "\n",
        "# concatenating all the weights in the neural network together\n",
        "weights = [weights_l1, np.array(weights_l2)]\n",
        "weights"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oBPYdxMpzjbd",
        "outputId": "61d3c711-86f0-4def-b250-daf497f4ef4b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[array([[ 1.33802204,  0.27270439, -2.32711844],\n",
              "        [ 1.06193803,  0.77875626, -2.36624024]]), array([[-2.16869671],\n",
              "        [-1.09397363],\n",
              "        [-2.61705925]])]"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# bias weights for the nodes in the hidden layer\n",
        "bias_l1 = [-1.71269062, -0.40820044,  0.81479502]\n",
        "\n",
        "# bias weights for the output layer node\n",
        "bias_l2 = [-0.69492816]\n",
        "\n",
        "# concatenating all the bias values in the neural network together\n",
        "bias = [np.array([bias_l1]), np.array([bias_l2])]\n",
        "bias"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Fo8N9sfSzs7u",
        "outputId": "a9c2658c-ad40-44ba-a8f4-9ee4a4b5f184"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[array([[-1.71269062, -0.40820044,  0.81479502]]), array([[-0.69492816]])]"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Exercise 1: Compute the vector of activations that is output by a network layer\n",
        "\n",
        "Hint: Use the np.dot function, and tanh activation."
      ],
      "metadata": {
        "id": "CGBZGHE0zYU7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def compute_activation(input, weights, bias):\n",
        "  result = np.dot(input, weights) + bias\n",
        "\n",
        "  return tanh(result)"
      ],
      "metadata": {
        "id": "nwSRXbgQN3DN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Exercise 2: Compute a full forward propagation pass"
      ],
      "metadata": {
        "id": "VUATfidp0ld7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# use the neural network to make predictions for each sample in the test data\n",
        "def predict(test_data):\n",
        "        \n",
        "        result = []\n",
        "\n",
        "        for i in range(len(test_data)):\n",
        "            \n",
        "            # compute the vector of activations output by the hidden layer\n",
        "            output1_activation = compute_activation(test_data[i], weights_l1, bias_l1)\n",
        "            print('output1_activation:', output1_activation)\n",
        "            # compute the activatiion output by the output layer (the single output neuron)\n",
        "            output2_activation = compute_activation(output1_activation, weights_l2, bias_l2)\n",
        "            print('output2_activation:', output2_activation)\n",
        "\n",
        "            result.append(output2_activation)\n",
        "\n",
        "        return result"
      ],
      "metadata": {
        "id": "IWJj4QI8W7i2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# use the neural network to make predictions for each sample in the test data\n",
        "def predict(test_data):\n",
        "        \n",
        "        result = []\n",
        "\n",
        "        for i in range(len(test_data)):\n",
        "            \n",
        "            # compute the vector of activations output by the hidden layer\n",
        "            output1_activation = compute_activation(test_data[i], weights_l1, bias_l1)\n",
        "            print('output1_activation:', output1_activation)\n",
        "            # compute the activatiion output by the output layer (the single output neuron)\n",
        "            output2_activation = compute_activation(output1_activation, weights_l2, bias_l2)\n",
        "            print('output2_activation:', output2_activation)\n",
        "\n",
        "            result.append(output2_activation)\n",
        "\n",
        "        return result"
      ],
      "metadata": {
        "id": "utE5Tz5TrU42"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x_test = np.array([[[0,0]], [[0,1]], [[1,0]], [[1,1]]])\n",
        "x_test[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VNUjycOxpC9c",
        "outputId": "ab3c7c74-8247-45dc-9c4b-beee3e544bfd"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0, 0]])"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# predict the first test sample\n",
        "prediction = predict(x_test[0])\n",
        "prediction"
      ],
      "metadata": {
        "id": "z4kn1TCz0k6C",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6be9a7dc-2ef7-4ade-d431-0eee6fbf897d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "output1_activation: [-0.93697683 -0.38694363  0.67222694]\n",
            "output2_activation: [0.00113879]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[array([0.00113879])]"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# predict all test samples\n",
        "predictions = predict(x_test)\n",
        "predictions"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KFy7CauRpIP7",
        "outputId": "556cdd90-2bcd-48b4-8e38-bdce4a48f7a5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "output1_activation: [[-0.93697683 -0.38694363  0.67222694]]\n",
            "output2_activation: [[0.00113879]]\n",
            "output1_activation: [[-0.57217639  0.35447779 -0.91402363]]\n",
            "output2_activation: [[0.98788554]]\n",
            "output1_activation: [[-0.35806851 -0.1346729  -0.9073505 ]]\n",
            "output2_activation: [[0.98910423]]\n",
            "output1_activation: [[ 0.59622498  0.5671153  -0.999145  ]]\n",
            "output2_activation: [[0.00645307]]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[array([[0.00113879]]),\n",
              " array([[0.98788554]]),\n",
              " array([[0.98910423]]),\n",
              " array([[0.00645307]])]"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# predict all test samples\n",
        "predictions = predict(x_test)\n",
        "predictions"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7VK9DlRJ5fTN",
        "outputId": "f1be8b0c-bcfc-4ea4-f2c4-50cf2ed19e98"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[array([[0.00113879]]),\n",
              " array([[0.98788554]]),\n",
              " array([[0.98910423]]),\n",
              " array([[0.00645307]])]"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    }
  ]
}