Pengenalan_Data_Science.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Pengenalan Data Science.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/junaedifahmi/Data-Science-Tutorial/blob/master/Pengenalan_Data_Science.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qVbCqBJqqnZZ"
      },
      "source": [
        "# Pengenalan Proses Data Science"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S77Bz02qqnZa"
      },
      "source": [
        "## Main Process"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0rBUomgaqnZb"
      },
      "source": [
        "Jadi kan proses dalam data science itu biasa dibagi menjadi 5 proses.\n",
        "\n",
        "    1. Pemahaman Bisnis\n",
        "    2. Preprocessing Data\n",
        "    3. Modeling\n",
        "    4. Evaluation\n",
        "    5. Deployment\n",
        "\n",
        "Nah biasanya Pemahaman bisnis itu gak bisa dilakukan sendiri oleh data scientis (kita) aja, tapi harus melibatkan orang-orang yang paham mengenai bisnis. Bisnis disini bukan cuma bisnis yang menghasilkan uang, tapi bisnis yang tidak menghasilkan uang tapi memiliki tujuan keorganisasian. Pada tahap ini biasanya kita harus didampingi oleh orang bisnis atau bisa juga disebut *domain expert*, orang yang paham mengenai bisnis tersebut. Misalnya kalau kita mau membuat sentimen analisis menggunakan text mining tentang gojek atau grab, kita bisa cari twitter yang berkaitan tentang gojek dan grab, tapi untuk menentukan sentimen dari twitter tersebut, kita gak bisa sembarangan menentukan sentimen dari pendapat pribadi, harus ada ahli bahasa atau ahli twitter yang memang kerjaannya fokus untuk menentukan bahwa kalimat di twitter tersebut memiliki sentimen positif atau negatif."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TZzkE9spqnZc"
      },
      "source": [
        "Kalau Preprocessing Data ini yang agak kompleks, karena kita bisa menjelaskan teknik-tekniknya, tapi kita gak bisa menentukan teknik apa yang seharusnya dipake. Setiap data tidak bisa diperlakukan secara sama, tetapi masing-masing data harus diperlakukan sesuai dengan keadaan data tersebut dan sesuai dengan tujuan bisnis. Ini akan kita bahas nanti di segmen khusus ya, untuk saat ini kita akan fokus ke main proses aja dulu."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wDDdTBnVqnZd"
      },
      "source": [
        "Main proses dalam data science, machine learning atau data mining itu hampir mirip. Intinya kita mau buat model untuk melakukan _task_ (klasifikasi, klustering atau regresi). Proses ini biasanya dilakukan secara iteratif atau berulang-ulang. Jadi jarang banget kita buat suatu model, dilatih terus jadi. Itu jarang banget terjadi, dan kalaupun terjadi seperti itu, kita harus curiga sama data yang kita punya."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ObJwpWAyqnZe"
      },
      "source": [
        "Di tutorial ini kita akan melakukan klasifikasi yang biasa dulu deh ya, kita coba pake data yang sudah ada. Oke kita siapkan bahan-bahannya dulu ya!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jrgZp4QWqnZf"
      },
      "source": [
        "# Siapkan bahan\n",
        "Nah sebelum mulai, pastikan bahwa python kamu sudah tersinstall. Kemudian pastikan lagi bahwa **pip** atau **pip3** kamu sudah terinstall.\n",
        "Kalau sudah, ini package yang harus ada sebelum kita bisa mulai.\n",
        "1. pandas\n",
        "2. numpy\n",
        "3. sklearn\n",
        "4. matplotlib\n",
        "5. seaborn"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lj9RYtMDqnZg"
      },
      "source": [
        "## Pandas\n",
        "Pandas ini berguna untuk mengolah data yang kita punya. Misal kita punya punya data lokal yang bertipe csv, xls atau h5. Dengan pandas kita dapat mengambil data itu dengan mudah, tidak perlu tools tambahan lainnya. Selain itu juga, pandas biasa berguna untuk melakukan manipulasi terhadap atribut yang kita punya, bisa untuk feature engineering juga. Bahasan itu nanti kita bahas di segmen preprocesing ya. Untuk saat ini kita berasumsi bahwa data yang akan digunakan sudah melewati fase preprocessing, jadi sudah siap untuk digunakan. Kalau data yang kita punya biasanya belum melalui proses preprocesing, jadi gak bisa asal masukkin aja. Sip"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XM-zjC-kqnZh"
      },
      "source": [
        "## Numpy\n",
        "Di python, kita gak kenal yang namanya array, adanya list, itu juga gak bisa dianggap mirip, karena sifatnya beda sama array. Oleh sebab itu, kita biasa menggunakan numpy untuk melakukan operasi array itu. Tau kan array itu apa? kalau dibahasa matematika itu disebut vector. Tapi gak cuma vector aja, numpy juga bisa untuk merepresentasikan matriks dan tensor. Kalau secara teknis, numpy biasa digunakan untuk aljabar linear, tapi tenang aja, kita gak akan terlalu dalam membahas aljabar linearnya kok. Paling pengenalan yang sering dipakai aja di data science ini.\n",
        "\n",
        "__Fun Fact__\n",
        "Vektor itu adalah satu baris bilangan atau matrix dengan ukuran 1xn,\n",
        "Matriks itu adalah sekumpuluan bilangan yang membentuk segi empat, jadi ukurannya nxm,\n",
        "nah kalau tensor itu perluasan dari matriks, jadi kalau kita punya matriks yang dimensinya lebih dari 2, bisa disebut tensor."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XETkHbuGqnZi"
      },
      "source": [
        "## SKlearn\n",
        "Nah ini inti dari tools data science kita, SKLearn itu sebuah modul atau package yang berisi algoritma data science siap pakai. Jadi kalau kita mau pakai algoritma SVM, Regresi Linear atau semacamnya itu kita harus ada ini. Tapi gak cuma algoritma data science aja sih, sklearn juga punya yang tools pembantu seperti evaluasi, spliting data scaling dan sebagainya. (apa pula itu, nanti kita bahas)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9i-nVYINqnZj"
      },
      "source": [
        "## Matplotlib & Seaborn\n",
        "Perlu kita ingat bahwa data science itu gak bisa berdiri sendiri, dan gak bisa cuma dilakukan sama orang IT, tapi juga melibatkan orang bisnis dan eksekutif. Untuk orang-orang yang sudah terbiasa liat layar komputer lama seperti kita mungkin gak masalah ketika harus liat banyak baris kode dan matriks yang panjang, tapi orang bisnis gak gitu. Nah kita gak mau dong, apa yang kita lakukan dengan perjuangan panjang seperti ini pada akhirnya akan berbuah duka, karena orang bisnis gak ngerti apa yang kita bahas. Makanya kita harus menyajikan hasil yang kita dapat itu secara lebih universal, salah satu caranya dengan menggunakan grafik atau visual. Matplotlib dan Seaborn sering digunakan untuk melakukan itu, jadi biar apa yang kita buat bisa dipahami sama orang yang backgroundnya bukan IT juga."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "khbsJ18yqnZk"
      },
      "source": [
        "Oke segitu dulu intronya, sekarang kita masuk ke prosesnya ya."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NRLuWiJvqnZl"
      },
      "source": [
        "# Proses Modeling"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yh49xlYmqnZm"
      },
      "source": [
        "## 1. Masukkan semua bahan\n",
        "Bahan-bahan yang kita punya kemudian harus kita masukkan ke dalam environtment yang sedang kita gunakan. caranya gampang, tinggal import aja selesai"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_u1Lsf00qnZn"
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import sklearn as skl\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uDQK-80lqnZr"
      },
      "source": [
        "## 2. Buat dataset\n",
        "Data yang tadi kita sudah lakuakan proses preprocesing, kemudian kita masukkan ke dalam environtment. Caranya gampang, kita gunakan pandas."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "iris = pd.read_csv(\n",
        "    \"https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data\", \n",
        "    sep=\",\",\n",
        "    names=[\"petal_length\", \"petal_widht\", \"sepal_length\", \"sepal_widht\", \"target\"]\n",
        ")\n",
        "iris"
      ],
      "metadata": {
        "id": "y5l8Wfvr5qiW",
        "outputId": "006a5661-3312-48f3-df34-6e18cd6fd799",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        }
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     petal_length  petal_widht  sepal_length  sepal_widht          target\n",
              "0             5.1          3.5           1.4          0.2     Iris-setosa\n",
              "1             4.9          3.0           1.4          0.2     Iris-setosa\n",
              "2             4.7          3.2           1.3          0.2     Iris-setosa\n",
              "3             4.6          3.1           1.5          0.2     Iris-setosa\n",
              "4             5.0          3.6           1.4          0.2     Iris-setosa\n",
              "..            ...          ...           ...          ...             ...\n",
              "145           6.7          3.0           5.2          2.3  Iris-virginica\n",
              "146           6.3          2.5           5.0          1.9  Iris-virginica\n",
              "147           6.5          3.0           5.2          2.0  Iris-virginica\n",
              "148           6.2          3.4           5.4          2.3  Iris-virginica\n",
              "149           5.9          3.0           5.1          1.8  Iris-virginica\n",
              "\n",
              "[150 rows x 5 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-8a8f1648-5be7-4d5f-8d45-d13485820654\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>petal_length</th>\n",
              "      <th>petal_widht</th>\n",
              "      <th>sepal_length</th>\n",
              "      <th>sepal_widht</th>\n",
              "      <th>target</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>5.1</td>\n",
              "      <td>3.5</td>\n",
              "      <td>1.4</td>\n",
              "      <td>0.2</td>\n",
              "      <td>Iris-setosa</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>4.9</td>\n",
              "      <td>3.0</td>\n",
              "      <td>1.4</td>\n",
              "      <td>0.2</td>\n",
              "      <td>Iris-setosa</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4.7</td>\n",
              "      <td>3.2</td>\n",
              "      <td>1.3</td>\n",
              "      <td>0.2</td>\n",
              "      <td>Iris-setosa</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4.6</td>\n",
              "      <td>3.1</td>\n",
              "      <td>1.5</td>\n",
              "      <td>0.2</td>\n",
              "      <td>Iris-setosa</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5.0</td>\n",
              "      <td>3.6</td>\n",
              "      <td>1.4</td>\n",
              "      <td>0.2</td>\n",
              "      <td>Iris-setosa</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>145</th>\n",
              "      <td>6.7</td>\n",
              "      <td>3.0</td>\n",
              "      <td>5.2</td>\n",
              "      <td>2.3</td>\n",
              "      <td>Iris-virginica</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>146</th>\n",
              "      <td>6.3</td>\n",
              "      <td>2.5</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1.9</td>\n",
              "      <td>Iris-virginica</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>147</th>\n",
              "      <td>6.5</td>\n",
              "      <td>3.0</td>\n",
              "      <td>5.2</td>\n",
              "      <td>2.0</td>\n",
              "      <td>Iris-virginica</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>148</th>\n",
              "      <td>6.2</td>\n",
              "      <td>3.4</td>\n",
              "      <td>5.4</td>\n",
              "      <td>2.3</td>\n",
              "      <td>Iris-virginica</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>149</th>\n",
              "      <td>5.9</td>\n",
              "      <td>3.0</td>\n",
              "      <td>5.1</td>\n",
              "      <td>1.8</td>\n",
              "      <td>Iris-virginica</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>150 rows × 5 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8a8f1648-5be7-4d5f-8d45-d13485820654')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-8a8f1648-5be7-4d5f-8d45-d13485820654 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-8a8f1648-5be7-4d5f-8d45-d13485820654');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lj-6wiCZqnZs",
        "outputId": "b85de037-9361-47b4-a3b6-da2de2371dc3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        }
      },
      "source": [
        "df = pd.read_csv(\"https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-white.csv\",sep=\";\")\n",
        "df.head() # Melihat 5 data pertama"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
              "0            7.0              0.27         0.36            20.7      0.045   \n",
              "1            6.3              0.30         0.34             1.6      0.049   \n",
              "2            8.1              0.28         0.40             6.9      0.050   \n",
              "3            7.2              0.23         0.32             8.5      0.058   \n",
              "4            7.2              0.23         0.32             8.5      0.058   \n",
              "\n",
              "   free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \\\n",
              "0                 45.0                 170.0   1.0010  3.00       0.45   \n",
              "1                 14.0                 132.0   0.9940  3.30       0.49   \n",
              "2                 30.0                  97.0   0.9951  3.26       0.44   \n",
              "3                 47.0                 186.0   0.9956  3.19       0.40   \n",
              "4                 47.0                 186.0   0.9956  3.19       0.40   \n",
              "\n",
              "   alcohol  quality  \n",
              "0      8.8        6  \n",
              "1      9.5        6  \n",
              "2     10.1        6  \n",
              "3      9.9        6  \n",
              "4      9.9        6  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-8d78873c-7b34-4678-9c9c-7ddabd9581f6\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>fixed acidity</th>\n",
              "      <th>volatile acidity</th>\n",
              "      <th>citric acid</th>\n",
              "      <th>residual sugar</th>\n",
              "      <th>chlorides</th>\n",
              "      <th>free sulfur dioxide</th>\n",
              "      <th>total sulfur dioxide</th>\n",
              "      <th>density</th>\n",
              "      <th>pH</th>\n",
              "      <th>sulphates</th>\n",
              "      <th>alcohol</th>\n",
              "      <th>quality</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>7.0</td>\n",
              "      <td>0.27</td>\n",
              "      <td>0.36</td>\n",
              "      <td>20.7</td>\n",
              "      <td>0.045</td>\n",
              "      <td>45.0</td>\n",
              "      <td>170.0</td>\n",
              "      <td>1.0010</td>\n",
              "      <td>3.00</td>\n",
              "      <td>0.45</td>\n",
              "      <td>8.8</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>6.3</td>\n",
              "      <td>0.30</td>\n",
              "      <td>0.34</td>\n",
              "      <td>1.6</td>\n",
              "      <td>0.049</td>\n",
              "      <td>14.0</td>\n",
              "      <td>132.0</td>\n",
              "      <td>0.9940</td>\n",
              "      <td>3.30</td>\n",
              "      <td>0.49</td>\n",
              "      <td>9.5</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>8.1</td>\n",
              "      <td>0.28</td>\n",
              "      <td>0.40</td>\n",
              "      <td>6.9</td>\n",
              "      <td>0.050</td>\n",
              "      <td>30.0</td>\n",
              "      <td>97.0</td>\n",
              "      <td>0.9951</td>\n",
              "      <td>3.26</td>\n",
              "      <td>0.44</td>\n",
              "      <td>10.1</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>7.2</td>\n",
              "      <td>0.23</td>\n",
              "      <td>0.32</td>\n",
              "      <td>8.5</td>\n",
              "      <td>0.058</td>\n",
              "      <td>47.0</td>\n",
              "      <td>186.0</td>\n",
              "      <td>0.9956</td>\n",
              "      <td>3.19</td>\n",
              "      <td>0.40</td>\n",
              "      <td>9.9</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>7.2</td>\n",
              "      <td>0.23</td>\n",
              "      <td>0.32</td>\n",
              "      <td>8.5</td>\n",
              "      <td>0.058</td>\n",
              "      <td>47.0</td>\n",
              "      <td>186.0</td>\n",
              "      <td>0.9956</td>\n",
              "      <td>3.19</td>\n",
              "      <td>0.40</td>\n",
              "      <td>9.9</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8d78873c-7b34-4678-9c9c-7ddabd9581f6')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-8d78873c-7b34-4678-9c9c-7ddabd9581f6 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-8d78873c-7b34-4678-9c9c-7ddabd9581f6');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.shape"
      ],
      "metadata": {
        "id": "6BusFb2EuISW",
        "outputId": "91dd12a7-54b8-438f-f7e8-e904119cc288",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(4898, 12)"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "goTHHu0OqnZy"
      },
      "source": [
        "**df** itu sebagai variable tempat semua data kita disimpan. Setelah masuk proses ini istilah data itu berubah menjadi dataset (himpunan data) karena istilah ini akan lebih sering digunakan daripada istilah data. Dari dataset ini akan kita bagi menjadi dua yaitu data latih (training set) dan data uji (testing set).\n",
        "\n",
        "Data latih digunakan pada saat kita melakukan proses latihan modeling menggunakan algoritma yang kita pilih. Biasanya jumlah data latih lebih banyak dari data uji.\n",
        "Data uji sendiri digunakan saat kita akan melakukan pengukuran dari model yang kita buat. Teknik pengukurannya kita bahas di bawah\n",
        "\n",
        "Biasanya kita bagi data latih dan data uji itu antara 70:30 atau 80:20, untuk sekarang kita coba pakai 75:25. Untuk itu kita lakukan dengan bantuan sklearn"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9wMWx-_tqnZy",
        "outputId": "0b7c9df2-248c-4a3f-9f56-be90f86cd236",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "x_train, x_test, y_train, y_test = train_test_split(df.drop(columns=\"quality\"), df.quality, train_size=0.75)\n",
        "\n",
        "print(\"kita memiliki data latih sebesar: \",len(y_train))\n",
        "print(\"kita memiliki data uji sebesar: \",len(x_test))"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "kita memiliki data latih sebesar:  3673\n",
            "kita memiliki data uji sebesar:  1225\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "x_train.head()"
      ],
      "metadata": {
        "id": "7JTJI6t_zyIe",
        "outputId": "2c0fb4a9-cfd3-46fb-ead9-3257f715f707",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        }
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "      fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
              "2668            9.6             0.655         0.21             2.0      0.039   \n",
              "214             6.2             0.250         0.25             1.4      0.030   \n",
              "599             6.9             0.190         0.40             1.4      0.036   \n",
              "4467            6.3             0.250         0.27             6.6      0.054   \n",
              "1527            7.9             0.280         0.49             7.7      0.045   \n",
              "\n",
              "      free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \\\n",
              "2668                 21.0                 120.0  0.99188  3.00       1.00   \n",
              "214                  35.0                 105.0  0.99120  3.30       0.44   \n",
              "599                  14.0                  55.0  0.99090  3.08       0.68   \n",
              "4467                 40.0                 158.0  0.99378  3.20       0.48   \n",
              "1527                 48.0                 195.0  0.99540  3.04       0.55   \n",
              "\n",
              "      alcohol  \n",
              "2668     12.6  \n",
              "214      11.1  \n",
              "599      11.5  \n",
              "4467     10.3  \n",
              "1527     11.0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-d4223f6e-4fed-4ddf-8e0b-caf8711f95a0\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>fixed acidity</th>\n",
              "      <th>volatile acidity</th>\n",
              "      <th>citric acid</th>\n",
              "      <th>residual sugar</th>\n",
              "      <th>chlorides</th>\n",
              "      <th>free sulfur dioxide</th>\n",
              "      <th>total sulfur dioxide</th>\n",
              "      <th>density</th>\n",
              "      <th>pH</th>\n",
              "      <th>sulphates</th>\n",
              "      <th>alcohol</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2668</th>\n",
              "      <td>9.6</td>\n",
              "      <td>0.655</td>\n",
              "      <td>0.21</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.039</td>\n",
              "      <td>21.0</td>\n",
              "      <td>120.0</td>\n",
              "      <td>0.99188</td>\n",
              "      <td>3.00</td>\n",
              "      <td>1.00</td>\n",
              "      <td>12.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>214</th>\n",
              "      <td>6.2</td>\n",
              "      <td>0.250</td>\n",
              "      <td>0.25</td>\n",
              "      <td>1.4</td>\n",
              "      <td>0.030</td>\n",
              "      <td>35.0</td>\n",
              "      <td>105.0</td>\n",
              "      <td>0.99120</td>\n",
              "      <td>3.30</td>\n",
              "      <td>0.44</td>\n",
              "      <td>11.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>599</th>\n",
              "      <td>6.9</td>\n",
              "      <td>0.190</td>\n",
              "      <td>0.40</td>\n",
              "      <td>1.4</td>\n",
              "      <td>0.036</td>\n",
              "      <td>14.0</td>\n",
              "      <td>55.0</td>\n",
              "      <td>0.99090</td>\n",
              "      <td>3.08</td>\n",
              "      <td>0.68</td>\n",
              "      <td>11.5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4467</th>\n",
              "      <td>6.3</td>\n",
              "      <td>0.250</td>\n",
              "      <td>0.27</td>\n",
              "      <td>6.6</td>\n",
              "      <td>0.054</td>\n",
              "      <td>40.0</td>\n",
              "      <td>158.0</td>\n",
              "      <td>0.99378</td>\n",
              "      <td>3.20</td>\n",
              "      <td>0.48</td>\n",
              "      <td>10.3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1527</th>\n",
              "      <td>7.9</td>\n",
              "      <td>0.280</td>\n",
              "      <td>0.49</td>\n",
              "      <td>7.7</td>\n",
              "      <td>0.045</td>\n",
              "      <td>48.0</td>\n",
              "      <td>195.0</td>\n",
              "      <td>0.99540</td>\n",
              "      <td>3.04</td>\n",
              "      <td>0.55</td>\n",
              "      <td>11.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d4223f6e-4fed-4ddf-8e0b-caf8711f95a0')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-d4223f6e-4fed-4ddf-8e0b-caf8711f95a0 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-d4223f6e-4fed-4ddf-8e0b-caf8711f95a0');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "y_train.head()"
      ],
      "metadata": {
        "id": "cAkWJFFs18-C",
        "outputId": "f92d7abe-16d9-4c15-f8c7-b9441d74144f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2668    6\n",
              "214     7\n",
              "599     7\n",
              "4467    5\n",
              "1527    6\n",
              "Name: quality, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tb41AebdqnZ2"
      },
      "source": [
        "x_train dan y_train itu kita gunakan untuk latihan model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zmuLz1GmqnZ3"
      },
      "source": [
        "## 3. Latih model\n",
        "\n",
        "Proses pelatihan model ini dilakukan secara otomatis oleh sklearn, jadi prosesnya gak keliatan. Intinya, kita bisa melakukan pelatihan model dengan menggunakan algoritma yang berbeda tanpa harus paham mengenai proses yang berjalan di belakang layar. caranya pertama kita harus definiskan model, terus dari model itu kita "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "720uzbW7qnZ4",
        "outputId": "b739f80d-81c6-48a6-9d80-bfbdec02ae57",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "from sklearn.svm import SVR\n",
        "\n",
        "model = SVR(gamma=\"scale\")#definisi model\n",
        "model.fit(x_train, y_train)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SVR()"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0n2X2rD9qnZ-"
      },
      "source": [
        "## 4. Testing model (Scoring)\n",
        "Testing atau scoring kita gunakan sesuai dengan tujuan dan algoritma yang digunakan. Ada beberapa ukuran yang biasa digunakan, pertama akurasi, confussion_matrix, precission dan recall itu biasa digunakan untuk klasifikasi, ada juga Mean Square Error (MSE) dan Mean Error (ME) itu untuk regresi, untuk klastering biasanya kita gunakan distance. Karena kita sedang melakukan proses klasifikasi, kita coba pakai akurasi ya. caranya gampang"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VtFZRHcpqnZ_",
        "outputId": "a58d2d90-9938-467e-8339-6008d271f2f7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "model.score(x_test,y_test)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.13773783680314244"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.metrics import accuracy_score, mean_absolute_error\n",
        "\n",
        "predicted = model.predict(x_test)\n",
        "mean_absolute_error(predicted, y_test)"
      ],
      "metadata": {
        "id": "mBNgPnLB3D5M",
        "outputId": "d520f103-1481-4e8f-e212-64f07e0666e5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.6281399874627062"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "predicted = model.predict(x_train)\n",
        "mean_absolute_error(predicted, y_train)"
      ],
      "metadata": {
        "id": "39TqOEh939tf",
        "outputId": "1a060eb1-ba0d-4a32-eb00-c9e7774d92c3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.619564058955048"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1GUT9otrqnaC"
      },
      "source": [
        "Nah proses utama (modeling) dalam data science itu cuma segini aja, gampang sih diterapkan, tapi agak sulit ketika kita harus membuat model dengan akurasi yang tinggi. Proses ini biasanya diulang-ulang dengan mengubah beberapa parameter. Bisa mengubah perbandingan antara data uji dg data latih, bisa juga mengubah parameter ketika kita definisikan model, atau malah mengubah dataset kita (lakukan preprocessing lagi). proses ini disebut _fine tuning_."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PdVYL-bMqnaD"
      },
      "source": [
        "# Tantangan Untuk kamu\n",
        "Coba bikin model dengan algoritma yang berbeda dan data yang berbeda"
      ]
    }
  ]
}