data_exploration&cleaning.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "**Data Exploration & Cleaning**\n",
        "---\n",
        "The first part of any data analysis or predictive modeling task is an initial exploration of the data. Even if you collected the data yourself and you already have a list of questions in mind that you want to answer, it is important to explore the data before doing any serious analysis, since oddities in the data can cause bugs and muddle your results. Before exploring deeper questions, you have to answer many simpler ones about the form and quality of data. That said, it is important to go into your initial data exploration with a big picture question in mind since the goal of your analysis should inform how you prepare the data."
      ],
      "metadata": {
        "id": "c1sCw1Tuk9Za"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 45,
      "metadata": {
        "id": "bfhy44HDkL3y"
      },
      "outputs": [],
      "source": [
        "# Load in some packages\n",
        "import calendar\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "import warnings\n",
        "\n",
        "warnings.filterwarnings(\"ignore\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Exploring The Variables**\n",
        "The first step in exploratory analysis is reading in the data and then exploring the variables. It is important to get a sense of how many variables and cases there are, the data types of the variables and the range of values they take on.\n",
        "\n",
        "We'll start by reading in the data:"
      ],
      "metadata": {
        "id": "wVpMOT6HlgIo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data = pd.read_csv('./datasets/merged/merged.csv') # Read the data"
      ],
      "metadata": {
        "id": "aefYmy5DlgnB"
      },
      "execution_count": 46,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "It's a good idea to start off by checking the dimensions of your data set with df.shape and the variable data types of df.dtypes."
      ],
      "metadata": {
        "id": "rbblN03Lls54"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data.shape # Check dimensions"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IgHS7UDGlua3",
        "outputId": "8b551811-89b0-4c82-efb7-d3eb7ead8c5e"
      },
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(186850, 6)"
            ]
          },
          "metadata": {},
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data.dtypes"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xlJJbmzhl0tX",
        "outputId": "5aeeb53b-7e63-4459-b44d-6bce451bc712"
      },
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Order ID            object\n",
              "Product             object\n",
              "Quantity Ordered    object\n",
              "Price Each          object\n",
              "Order Date          object\n",
              "Purchase Address    object\n",
              "dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The output shows us that we're working with a set of 186850 records and 6 columns. All of the column variables are encoded as \"object\". Let's check the head of the data to get a better sense of what the variables look like:"
      ],
      "metadata": {
        "id": "xuO9acjRlu3o"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data.head(5)  # Check the first 5 rows"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "8KqY3H9Ll2eH",
        "outputId": "8d9a60b4-b659-41cc-ea00-6149adb29b70"
      },
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  Order ID                   Product Quantity Ordered Price Each  \\\n",
              "0   141234                    iPhone                1      700.0   \n",
              "1   141235  Lightning Charging Cable                1      14.95   \n",
              "2   141236          Wired Headphones                2      11.99   \n",
              "3   141237          27in FHD Monitor                1     149.99   \n",
              "4   141238          Wired Headphones                1      11.99   \n",
              "\n",
              "       Order Date                       Purchase Address  \n",
              "0  01/22/19 21:25        944 Walnut St, Boston, MA 02215  \n",
              "1  01/28/19 14:15       185 Maple St, Portland, OR 97035  \n",
              "2  01/17/19 13:33  538 Adams St, San Francisco, CA 94016  \n",
              "3  01/05/19 20:33     738 10th St, Los Angeles, CA 90001  \n",
              "4  01/25/19 11:59          387 10th St, Austin, TX 73301  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-193d5a89-1c23-4893-bae7-fb9737162bb4\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Order ID</th>\n",
              "      <th>Product</th>\n",
              "      <th>Quantity Ordered</th>\n",
              "      <th>Price Each</th>\n",
              "      <th>Order Date</th>\n",
              "      <th>Purchase Address</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>141234</td>\n",
              "      <td>iPhone</td>\n",
              "      <td>1</td>\n",
              "      <td>700.0</td>\n",
              "      <td>01/22/19 21:25</td>\n",
              "      <td>944 Walnut St, Boston, MA 02215</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>141235</td>\n",
              "      <td>Lightning Charging Cable</td>\n",
              "      <td>1</td>\n",
              "      <td>14.95</td>\n",
              "      <td>01/28/19 14:15</td>\n",
              "      <td>185 Maple St, Portland, OR 97035</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>141236</td>\n",
              "      <td>Wired Headphones</td>\n",
              "      <td>2</td>\n",
              "      <td>11.99</td>\n",
              "      <td>01/17/19 13:33</td>\n",
              "      <td>538 Adams St, San Francisco, CA 94016</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>141237</td>\n",
              "      <td>27in FHD Monitor</td>\n",
              "      <td>1</td>\n",
              "      <td>149.99</td>\n",
              "      <td>01/05/19 20:33</td>\n",
              "      <td>738 10th St, Los Angeles, CA 90001</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>141238</td>\n",
              "      <td>Wired Headphones</td>\n",
              "      <td>1</td>\n",
              "      <td>11.99</td>\n",
              "      <td>01/25/19 11:59</td>\n",
              "      <td>387 10th St, Austin, TX 73301</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-193d5a89-1c23-4893-bae7-fb9737162bb4')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-193d5a89-1c23-4893-bae7-fb9737162bb4 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-193d5a89-1c23-4893-bae7-fb9737162bb4');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It appears we have a mixture of numeric columns and columns with text data, despite that all of the column variables are encoded as \"object\". We will convert them if necessary, as we'll see later.\n",
        "\n",
        "After getting a sense of the data's structure, it is a good idea to look at a statistical summary of the variables with df.describe():"
      ],
      "metadata": {
        "id": "S-x7VU8il94v"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data.describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "2LgW8YTfl-gP",
        "outputId": "bc96747e-dddf-415f-fa75-c8ee4e5b9fa4"
      },
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        Order ID               Product  Quantity Ordered  Price Each  \\\n",
              "count     186305                186305            186305   186305.00   \n",
              "unique    178438                    20                18       41.00   \n",
              "top     Order ID  USB-C Charging Cable                 1       11.95   \n",
              "freq         355                 21903            118804    15358.00   \n",
              "\n",
              "        Order Date  Purchase Address  \n",
              "count       186305            186305  \n",
              "unique      142396            140788  \n",
              "top     Order Date  Purchase Address  \n",
              "freq           355               355  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-242c2d63-b104-49ec-9ec3-2a66f8b5ed35\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Order ID</th>\n",
              "      <th>Product</th>\n",
              "      <th>Quantity Ordered</th>\n",
              "      <th>Price Each</th>\n",
              "      <th>Order Date</th>\n",
              "      <th>Purchase Address</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>186305</td>\n",
              "      <td>186305</td>\n",
              "      <td>186305</td>\n",
              "      <td>186305.00</td>\n",
              "      <td>186305</td>\n",
              "      <td>186305</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>unique</th>\n",
              "      <td>178438</td>\n",
              "      <td>20</td>\n",
              "      <td>18</td>\n",
              "      <td>41.00</td>\n",
              "      <td>142396</td>\n",
              "      <td>140788</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>top</th>\n",
              "      <td>Order ID</td>\n",
              "      <td>USB-C Charging Cable</td>\n",
              "      <td>1</td>\n",
              "      <td>11.95</td>\n",
              "      <td>Order Date</td>\n",
              "      <td>Purchase Address</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>freq</th>\n",
              "      <td>355</td>\n",
              "      <td>21903</td>\n",
              "      <td>118804</td>\n",
              "      <td>15358.00</td>\n",
              "      <td>355</td>\n",
              "      <td>355</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-242c2d63-b104-49ec-9ec3-2a66f8b5ed35')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-242c2d63-b104-49ec-9ec3-2a66f8b5ed35 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-242c2d63-b104-49ec-9ec3-2a66f8b5ed35');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Describe() give us enough information to determine what each variable means. So, after looking at the data for the first time, you should ask yourself a few questions:\n",
        "\n",
        "1.   Do I need all of the variables?\n",
        "2.   Should I transform any variables?\n",
        "2.   Are there NA values, outliers or other strange values?\n",
        "2.   Should I create new variables?"
      ],
      "metadata": {
        "id": "pfgSsy1xmN8H"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Do I Need All of the Variables?**"
      ],
      "metadata": {
        "id": "EINtnYp5oNnI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Getting rid of unnecessary variables is a good first step when dealing with any data set, since dropping variables reduces complexity and can make computation on the data faster. Whether you should get rid of a variable or not will depend on size of the data set and the goal of your analysis. With a data set as small as the Titanic data, there's no real need to drop variables from a computing perspective (we have plenty of memory and processing power to deal with such a small data set) but it can still be helpful to drop variables that will only distract from your goal.\n",
        "\n",
        "From each variable we can take valuable information. In this case, we need all of the variables."
      ],
      "metadata": {
        "id": "YepGvidemeDX"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Should I Transform Any Variables?**"
      ],
      "metadata": {
        "id": "XN_ThVLfoKp3"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "When you first load a data set, some of the variables may be encoded as data types that don't fit well with what the data really is or what it means."
      ],
      "metadata": {
        "id": "TyA5cTyboPW1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data['Order Date'].describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8EjjpxKPp7be",
        "outputId": "cacc424f-99b6-4df2-ede5-94259f9dd0c7"
      },
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count         186305\n",
              "unique        142396\n",
              "top       Order Date\n",
              "freq             355\n",
              "Name: Order Date, dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Notice that as top frequency for the date feature 'Order Date' is the string value 'Order Date'. This doesn't make any sense, since it should be only date formatted info in this feature. We get rid of it, holding all the rows that don't have 'Order Date' as value in this particular feature:"
      ],
      "metadata": {
        "id": "keGIdRsftigU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data = data[data['Order Date'] != 'Order Date']\n",
        "data.describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "LyCDrmXWuIGD",
        "outputId": "a714dffd-e071-4931-8232-5f983793ce50"
      },
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        Order ID               Product  Quantity Ordered  Price Each  \\\n",
              "count     185950                185950            185950   185950.00   \n",
              "unique    178437                    19                17       40.00   \n",
              "top       160873  USB-C Charging Cable                 1       11.95   \n",
              "freq           5                 21903            118804    15358.00   \n",
              "\n",
              "            Order Date                        Purchase Address  \n",
              "count           185950                                  185950  \n",
              "unique          142395                                  140787  \n",
              "top     12/15/19 20:16  193 Forest St, San Francisco, CA 94016  \n",
              "freq                 8                                       9  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-f210cdda-7aac-430b-bffc-936c2c0a6434\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Order ID</th>\n",
              "      <th>Product</th>\n",
              "      <th>Quantity Ordered</th>\n",
              "      <th>Price Each</th>\n",
              "      <th>Order Date</th>\n",
              "      <th>Purchase Address</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>185950</td>\n",
              "      <td>185950</td>\n",
              "      <td>185950</td>\n",
              "      <td>185950.00</td>\n",
              "      <td>185950</td>\n",
              "      <td>185950</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>unique</th>\n",
              "      <td>178437</td>\n",
              "      <td>19</td>\n",
              "      <td>17</td>\n",
              "      <td>40.00</td>\n",
              "      <td>142395</td>\n",
              "      <td>140787</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>top</th>\n",
              "      <td>160873</td>\n",
              "      <td>USB-C Charging Cable</td>\n",
              "      <td>1</td>\n",
              "      <td>11.95</td>\n",
              "      <td>12/15/19 20:16</td>\n",
              "      <td>193 Forest St, San Francisco, CA 94016</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>freq</th>\n",
              "      <td>5</td>\n",
              "      <td>21903</td>\n",
              "      <td>118804</td>\n",
              "      <td>15358.00</td>\n",
              "      <td>8</td>\n",
              "      <td>9</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f210cdda-7aac-430b-bffc-936c2c0a6434')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-f210cdda-7aac-430b-bffc-936c2c0a6434 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-f210cdda-7aac-430b-bffc-936c2c0a6434');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you have oddly formatted date time objects, you might have to specify the exact format to get it to convert correctly into a Timestamp. For instance, consider a date format that gives date times of the form hour:minute:second year-day-month:\n",
        "\n",
        "\n",
        "\n",
        "```\n",
        "odd_date = \"12:30:15 2015-29-11\"\n",
        "```\n",
        "\n",
        "\n",
        "\n",
        "The default to_datetime parser will fail to convert this date because it expects dates in the form year-month-day. In cases like this, specify the date's format to convert it to Timestamp:"
      ],
      "metadata": {
        "id": "A5fM5CQioxq9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "new_Dates = pd.to_datetime(data['Order Date'], format= \"%m/%d/%y %H:%M\") \n",
        "data['Order Date'] = new_Dates"
      ],
      "metadata": {
        "id": "euRriko8oRsG"
      },
      "execution_count": 53,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Numerical features 'Quantity Ordered' and 'Price Each' are object types. We should trasform them to numerical type, in order to perform mathematical calculations, with pandas function to_numeric, which converts each feature to the coresponding numerical type:"
      ],
      "metadata": {
        "id": "w-bpDf9_fApv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data['Quantity Ordered'] = pd.to_numeric(data['Quantity Ordered']) # Convert to int\n",
        "data['Price Each'] = pd.to_numeric(data['Price Each']) # Convert to float\n",
        "\n",
        "data.describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "ESkZuSjufngA",
        "outputId": "0e2b01e9-a97f-4849-b092-fa5aad14c2bc"
      },
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       Quantity Ordered     Price Each\n",
              "count     185950.000000  185950.000000\n",
              "mean           1.124383     184.399735\n",
              "std            0.442793     332.731330\n",
              "min            1.000000       2.990000\n",
              "25%            1.000000      11.950000\n",
              "50%            1.000000      14.950000\n",
              "75%            1.000000     150.000000\n",
              "max            9.000000    1700.000000"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-de36731e-c271-4ac4-8d84-058dcc8c5259\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Quantity Ordered</th>\n",
              "      <th>Price Each</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>185950.000000</td>\n",
              "      <td>185950.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>1.124383</td>\n",
              "      <td>184.399735</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.442793</td>\n",
              "      <td>332.731330</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>2.990000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>11.950000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>14.950000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>150.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>9.000000</td>\n",
              "      <td>1700.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-de36731e-c271-4ac4-8d84-058dcc8c5259')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-de36731e-c271-4ac4-8d84-058dcc8c5259 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-de36731e-c271-4ac4-8d84-058dcc8c5259');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Are there NA Values, Outliers or Other Strange Values?**"
      ],
      "metadata": {
        "id": "AiFSUi1StFCb"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Data sets are often littered with missing data, extreme data points called outliers and other strange values. Missing values, outliers and strange values can negatively affect statistical tests and models and may even cause certain functions to fail.\n",
        "\n",
        "In Python, you can detect missing values with the pd.dropna().\n",
        "\n",
        "Detecting missing values is the easy part: it is far more difficult to decide how to handle them. In cases where you have a lot of data and only a few missing values, it might make sense to simply delete records with missing values present. On the other hand, if you have more than a handful of missing values, removing records with missing values could cause you to get rid of a lot of data. Missing values in categorical data are not particularly troubling because you can simply treat NA as an additional category. Missing values in numeric variables are more troublesome, since you can't just treat a missing value as number. As it happens, the Sales dataset has some NA's:"
      ],
      "metadata": {
        "id": "OGhBb8tkxOSn"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data.shape[0] - data.dropna().shape[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MqoY2CowxGl_",
        "outputId": "41b7cc09-6c5e-45f3-f5ee-ebbd9a200e17"
      },
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "545"
            ]
          },
          "metadata": {},
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "There are 545 NA values, so much smaller number that the original 186495 data. In this case, we will drop all NA values, using pd.dropna()."
      ],
      "metadata": {
        "id": "eXeHpNbxxm9_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data.dropna(inplace = True)\n",
        "data.describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "tvL1qukDrqiE",
        "outputId": "0889ba15-4f27-4a03-cee0-d450abd74dde"
      },
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       Quantity Ordered     Price Each\n",
              "count     185950.000000  185950.000000\n",
              "mean           1.124383     184.399735\n",
              "std            0.442793     332.731330\n",
              "min            1.000000       2.990000\n",
              "25%            1.000000      11.950000\n",
              "50%            1.000000      14.950000\n",
              "75%            1.000000     150.000000\n",
              "max            9.000000    1700.000000"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-8987438d-3dfd-4d6c-8846-625cc21f7ea6\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Quantity Ordered</th>\n",
              "      <th>Price Each</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>185950.000000</td>\n",
              "      <td>185950.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>1.124383</td>\n",
              "      <td>184.399735</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.442793</td>\n",
              "      <td>332.731330</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>2.990000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>11.950000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>14.950000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>150.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>9.000000</td>\n",
              "      <td>1700.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8987438d-3dfd-4d6c-8846-625cc21f7ea6')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-8987438d-3dfd-4d6c-8846-625cc21f7ea6 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-8987438d-3dfd-4d6c-8846-625cc21f7ea6');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data.describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "Vw_sgzmVte0j",
        "outputId": "39d7a2d6-1cef-42e6-ded0-b11c24949f32"
      },
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       Quantity Ordered     Price Each\n",
              "count     185950.000000  185950.000000\n",
              "mean           1.124383     184.399735\n",
              "std            0.442793     332.731330\n",
              "min            1.000000       2.990000\n",
              "25%            1.000000      11.950000\n",
              "50%            1.000000      14.950000\n",
              "75%            1.000000     150.000000\n",
              "max            9.000000    1700.000000"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-6cb55dec-1d6f-4eb0-9f36-576e4856cac4\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Quantity Ordered</th>\n",
              "      <th>Price Each</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>185950.000000</td>\n",
              "      <td>185950.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>1.124383</td>\n",
              "      <td>184.399735</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.442793</td>\n",
              "      <td>332.731330</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>2.990000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>11.950000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>14.950000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>150.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>9.000000</td>\n",
              "      <td>1700.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6cb55dec-1d6f-4eb0-9f36-576e4856cac4')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-6cb55dec-1d6f-4eb0-9f36-576e4856cac4 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-6cb55dec-1d6f-4eb0-9f36-576e4856cac4');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "categorical = data.dtypes[data.dtypes == \"object\"].index\n",
        "\n",
        "data[categorical].describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "XiDYUtkFuIQ0",
        "outputId": "ddb4a72f-048c-4d8a-8085-d22a9c333344"
      },
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        Order ID               Product                        Purchase Address\n",
              "count     185950                185950                                  185950\n",
              "unique    178437                    19                                  140787\n",
              "top       160873  USB-C Charging Cable  193 Forest St, San Francisco, CA 94016\n",
              "freq           5                 21903                                       9"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-34187b82-006c-43f3-995b-3fdaec1a545c\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Order ID</th>\n",
              "      <th>Product</th>\n",
              "      <th>Purchase Address</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>185950</td>\n",
              "      <td>185950</td>\n",
              "      <td>185950</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>unique</th>\n",
              "      <td>178437</td>\n",
              "      <td>19</td>\n",
              "      <td>140787</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>top</th>\n",
              "      <td>160873</td>\n",
              "      <td>USB-C Charging Cable</td>\n",
              "      <td>193 Forest St, San Francisco, CA 94016</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>freq</th>\n",
              "      <td>5</td>\n",
              "      <td>21903</td>\n",
              "      <td>9</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-34187b82-006c-43f3-995b-3fdaec1a545c')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-34187b82-006c-43f3-995b-3fdaec1a545c button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-34187b82-006c-43f3-995b-3fdaec1a545c');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 58
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Should I Create New Variables?**"
      ],
      "metadata": {
        "id": "65KgJ5JZvUsY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The variables present when you load a data set aren't always the most useful variables for analysis. Creating new variables that are derivations or combinations existing ones is a common step to take before jumping into an analysis or modeling task.\n",
        "\n",
        "For example, imagine you are analyzing web site auctions where one of the data fields is a text description of the item being sold. A raw block of text is difficult to use in any sort of analysis, but you could create new variables from it such as a variable storing the length of the description or variables indicating the presence of certain keywords.\n",
        "\n",
        "Creating a new variable can be as simple as taking one variable and adding, multiplying or dividing by another. Let's create a new variable, Month, from 'Order Date':"
      ],
      "metadata": {
        "id": "OXt5rBu2v11X"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data['Month'] = data['Order Date'].dt.month\n",
        "data['Month'].describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DB_Vg0v6vZbw",
        "outputId": "32fb9de2-f6ca-4459-fc6f-ef0aa1883fe5"
      },
      "execution_count": 59,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count    185950.000000\n",
              "mean          7.059140\n",
              "std           3.502996\n",
              "min           1.000000\n",
              "25%           4.000000\n",
              "50%           7.000000\n",
              "75%          10.000000\n",
              "max          12.000000\n",
              "Name: Month, dtype: float64"
            ]
          },
          "metadata": {},
          "execution_count": 59
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, we should really create a column featuring 'Sales', which we will create using 'Quantity Ordered' and 'Price Each':"
      ],
      "metadata": {
        "id": "Fy650jkReYMg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data['Sales'] = data['Quantity Ordered'] * data['Price Each']\n",
        "data['Sales'].describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sswM5QFkeqn2",
        "outputId": "cfb7b760-7d90-4141-c4a5-2613cd559be7"
      },
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count    185950.000000\n",
              "mean        185.490917\n",
              "std         332.919771\n",
              "min           2.990000\n",
              "25%          11.950000\n",
              "50%          14.950000\n",
              "75%         150.000000\n",
              "max        3400.000000\n",
              "Name: Sales, dtype: float64"
            ]
          },
          "metadata": {},
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "From 'Purchase Address' we can take the 'City' that each purchase came from, with apply method:"
      ],
      "metadata": {
        "id": "r1GLp2_4inl8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data['City'] = data['Purchase Address'].apply(lambda x: f\"{x.split(',')[1]} ({x.split(',')[2].split(' ')[1]})\")\n",
        "data['City'].describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OIS7_XtAjAFj",
        "outputId": "c88ebca0-1f4b-4187-d7d7-b49390d3ba9b"
      },
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count                  185950\n",
              "unique                     10\n",
              "top        San Francisco (CA)\n",
              "freq                    44732\n",
              "Name: City, dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data['Hour'] = data['Order Date'].dt.hour\n",
        "data['Hour'].describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JHFu569w94hY",
        "outputId": "2af7e19e-3533-41a1-b591-75415e64decd"
      },
      "execution_count": 62,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count    185950.000000\n",
              "mean         14.413305\n",
              "std           5.423416\n",
              "min           0.000000\n",
              "25%          11.000000\n",
              "50%          15.000000\n",
              "75%          19.000000\n",
              "max          23.000000\n",
              "Name: Hour, dtype: float64"
            ]
          },
          "metadata": {},
          "execution_count": 62
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Final Dataset**"
      ],
      "metadata": {
        "id": "zKbdGouRwGOP"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Finally, we finished exploring and cleaning data. The final dataset is formatted like this:"
      ],
      "metadata": {
        "id": "bX6wW-DvwG6f"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 658
        },
        "id": "qYuZ2QuHwIth",
        "outputId": "39398961-e469-489a-8ea4-a545a4300380"
      },
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       Order ID                     Product  Quantity Ordered  Price Each  \\\n",
              "0        141234                      iPhone               1.0      700.00   \n",
              "1        141235    Lightning Charging Cable               1.0       14.95   \n",
              "2        141236            Wired Headphones               2.0       11.99   \n",
              "3        141237            27in FHD Monitor               1.0      149.99   \n",
              "4        141238            Wired Headphones               1.0       11.99   \n",
              "...         ...                         ...               ...         ...   \n",
              "185945   319666    Lightning Charging Cable               1.0       14.95   \n",
              "185946   319667       AA Batteries (4-pack)               2.0        3.84   \n",
              "185947   319668             Vareebadd Phone               1.0      400.00   \n",
              "185948   319669            Wired Headphones               1.0       11.99   \n",
              "185949   319670  Bose SoundSport Headphones               1.0       99.99   \n",
              "\n",
              "                Order Date                        Purchase Address  Month  \\\n",
              "0      2019-01-22 21:25:00         944 Walnut St, Boston, MA 02215      1   \n",
              "1      2019-01-28 14:15:00        185 Maple St, Portland, OR 97035      1   \n",
              "2      2019-01-17 13:33:00   538 Adams St, San Francisco, CA 94016      1   \n",
              "3      2019-01-05 20:33:00      738 10th St, Los Angeles, CA 90001      1   \n",
              "4      2019-01-25 11:59:00           387 10th St, Austin, TX 73301      1   \n",
              "...                    ...                                     ...    ...   \n",
              "185945 2019-12-11 20:58:00  14 Madison St, San Francisco, CA 94016     12   \n",
              "185946 2019-12-01 12:01:00    549 Willow St, Los Angeles, CA 90001     12   \n",
              "185947 2019-12-09 06:43:00        273 Wilson St, Seattle, WA 98101     12   \n",
              "185948 2019-12-03 10:39:00          778 River St, Dallas, TX 75001     12   \n",
              "185949 2019-12-21 21:45:00  747 Chestnut St, Los Angeles, CA 90001     12   \n",
              "\n",
              "         Sales                 City  Hour  \n",
              "0       700.00          Boston (MA)    21  \n",
              "1        14.95        Portland (OR)    14  \n",
              "2        23.98   San Francisco (CA)    13  \n",
              "3       149.99     Los Angeles (CA)    20  \n",
              "4        11.99          Austin (TX)    11  \n",
              "...        ...                  ...   ...  \n",
              "185945   14.95   San Francisco (CA)    20  \n",
              "185946    7.68     Los Angeles (CA)    12  \n",
              "185947  400.00         Seattle (WA)     6  \n",
              "185948   11.99          Dallas (TX)    10  \n",
              "185949   99.99     Los Angeles (CA)    21  \n",
              "\n",
              "[185950 rows x 10 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-0fd69b81-db37-498d-94e7-ceccf0bd72c2\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Order ID</th>\n",
              "      <th>Product</th>\n",
              "      <th>Quantity Ordered</th>\n",
              "      <th>Price Each</th>\n",
              "      <th>Order Date</th>\n",
              "      <th>Purchase Address</th>\n",
              "      <th>Month</th>\n",
              "      <th>Sales</th>\n",
              "      <th>City</th>\n",
              "      <th>Hour</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>141234</td>\n",
              "      <td>iPhone</td>\n",
              "      <td>1.0</td>\n",
              "      <td>700.00</td>\n",
              "      <td>2019-01-22 21:25:00</td>\n",
              "      <td>944 Walnut St, Boston, MA 02215</td>\n",
              "      <td>1</td>\n",
              "      <td>700.00</td>\n",
              "      <td>Boston (MA)</td>\n",
              "      <td>21</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>141235</td>\n",
              "      <td>Lightning Charging Cable</td>\n",
              "      <td>1.0</td>\n",
              "      <td>14.95</td>\n",
              "      <td>2019-01-28 14:15:00</td>\n",
              "      <td>185 Maple St, Portland, OR 97035</td>\n",
              "      <td>1</td>\n",
              "      <td>14.95</td>\n",
              "      <td>Portland (OR)</td>\n",
              "      <td>14</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>141236</td>\n",
              "      <td>Wired Headphones</td>\n",
              "      <td>2.0</td>\n",
              "      <td>11.99</td>\n",
              "      <td>2019-01-17 13:33:00</td>\n",
              "      <td>538 Adams St, San Francisco, CA 94016</td>\n",
              "      <td>1</td>\n",
              "      <td>23.98</td>\n",
              "      <td>San Francisco (CA)</td>\n",
              "      <td>13</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>141237</td>\n",
              "      <td>27in FHD Monitor</td>\n",
              "      <td>1.0</td>\n",
              "      <td>149.99</td>\n",
              "      <td>2019-01-05 20:33:00</td>\n",
              "      <td>738 10th St, Los Angeles, CA 90001</td>\n",
              "      <td>1</td>\n",
              "      <td>149.99</td>\n",
              "      <td>Los Angeles (CA)</td>\n",
              "      <td>20</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>141238</td>\n",
              "      <td>Wired Headphones</td>\n",
              "      <td>1.0</td>\n",
              "      <td>11.99</td>\n",
              "      <td>2019-01-25 11:59:00</td>\n",
              "      <td>387 10th St, Austin, TX 73301</td>\n",
              "      <td>1</td>\n",
              "      <td>11.99</td>\n",
              "      <td>Austin (TX)</td>\n",
              "      <td>11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>185945</th>\n",
              "      <td>319666</td>\n",
              "      <td>Lightning Charging Cable</td>\n",
              "      <td>1.0</td>\n",
              "      <td>14.95</td>\n",
              "      <td>2019-12-11 20:58:00</td>\n",
              "      <td>14 Madison St, San Francisco, CA 94016</td>\n",
              "      <td>12</td>\n",
              "      <td>14.95</td>\n",
              "      <td>San Francisco (CA)</td>\n",
              "      <td>20</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>185946</th>\n",
              "      <td>319667</td>\n",
              "      <td>AA Batteries (4-pack)</td>\n",
              "      <td>2.0</td>\n",
              "      <td>3.84</td>\n",
              "      <td>2019-12-01 12:01:00</td>\n",
              "      <td>549 Willow St, Los Angeles, CA 90001</td>\n",
              "      <td>12</td>\n",
              "      <td>7.68</td>\n",
              "      <td>Los Angeles (CA)</td>\n",
              "      <td>12</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>185947</th>\n",
              "      <td>319668</td>\n",
              "      <td>Vareebadd Phone</td>\n",
              "      <td>1.0</td>\n",
              "      <td>400.00</td>\n",
              "      <td>2019-12-09 06:43:00</td>\n",
              "      <td>273 Wilson St, Seattle, WA 98101</td>\n",
              "      <td>12</td>\n",
              "      <td>400.00</td>\n",
              "      <td>Seattle (WA)</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>185948</th>\n",
              "      <td>319669</td>\n",
              "      <td>Wired Headphones</td>\n",
              "      <td>1.0</td>\n",
              "      <td>11.99</td>\n",
              "      <td>2019-12-03 10:39:00</td>\n",
              "      <td>778 River St, Dallas, TX 75001</td>\n",
              "      <td>12</td>\n",
              "      <td>11.99</td>\n",
              "      <td>Dallas (TX)</td>\n",
              "      <td>10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>185949</th>\n",
              "      <td>319670</td>\n",
              "      <td>Bose SoundSport Headphones</td>\n",
              "      <td>1.0</td>\n",
              "      <td>99.99</td>\n",
              "      <td>2019-12-21 21:45:00</td>\n",
              "      <td>747 Chestnut St, Los Angeles, CA 90001</td>\n",
              "      <td>12</td>\n",
              "      <td>99.99</td>\n",
              "      <td>Los Angeles (CA)</td>\n",
              "      <td>21</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>185950 rows × 10 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0fd69b81-db37-498d-94e7-ceccf0bd72c2')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-0fd69b81-db37-498d-94e7-ceccf0bd72c2 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-0fd69b81-db37-498d-94e7-ceccf0bd72c2');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 63
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "And the last step is to save this dataset as csv file."
      ],
      "metadata": {
        "id": "T0P-B5GAwLSZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data.to_csv('sales.csv', index=False)"
      ],
      "metadata": {
        "id": "KJUBTwhVwMIX"
      },
      "execution_count": 64,
      "outputs": []
    }
  ]
}