credit_risk_explainability_inference_pipelines.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Credit risk prediction, explainability and bias detection with Amazon SageMaker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Credit risk explainability use case](credit_risk_prediction.png)\n",
    "\n",
    "1. [Overview](#Overview)\n",
    "1. [Prerequisites and Data](#Prerequisites-and-Data)\n",
    "    1. [Initialize SageMaker](#Initialize-SageMaker)\n",
    "    1. [Download data](#Download-data)\n",
    "    1. [Loading the data: German credit (Update) Dataset](#Loading-the-data:-German-credit-Dataset) \n",
    "    1. [Data inspection](#Data-inspection) \n",
    "    1. [Data preprocessing Model and upload to S3](#Preprocess-and-Upload-Training-Data) \n",
    "1. [Train XGBoost Model](#Train-XGBoost-Model)\n",
    "    1. [Train Model](#Train-Model)\n",
    "1. [Create SageMaker Model with Inference Pipeline](#Create-SageMaker-Model)\n",
    "1. [Amazon SageMaker Clarify](#Amazon-SageMaker-Clarify)\n",
    "    1. [Explaining Predictions](#Explaining-Predictions)\n",
    "        1. [Viewing the Explainability Report](#Viewing-the-Explainability-Report)\n",
    "        2. [Explaining individual bad credit prediction example](#Explaining-individual-prediction)\n",
    "    2. [Understanding Bias](#Bias-Detection)\n",
    "        1. [Pre-training bias metrics](#pre-training)\n",
    "        2. [Post-training bias metrics](#post-training)\n",
    "1. [Clean Up](#Clean-Up)\n",
    "1. [Additional Resources](#Additional-Resources)\n",
    "\n",
    "## 1. Overview\n",
    "Amazon SageMaker helps data scientists and developers to prepare, build, train, and deploy high-quality machine learning (ML) models quickly by bringing together a broad set of capabilities purpose-built for ML.\n",
    "\n",
    "[Amazon SageMaker Clarify](https://aws.amazon.com/sagemaker/clarify/) helps improve your machine learning models by detecting potential bias and helping explain how these models make predictions. The fairness and explainability functionality provided by SageMaker Clarify takes a step towards enabling AWS customers to build trustworthy and understandable machine learning models. \n",
    "\n",
    "Amazon SageMaker provides pre-made images for machine and deep learning frameworks for supported frameworks such as Scikit-Learn, XGBoost, TensorFlow, PyTorch, MXNet, or Chainer. These are preloaded with the corresponding framework and some additional Python packages, such as Pandas and NumPy, so you can write your own code for model training. See [here](https://docs.aws.amazon.com/sagemaker/latest/dg/algorithms-choose.html#supported-frameworks-benefits) for more information.\n",
    "\n",
    "\n",
    "[Amazon SageMaker Studio](https://aws.amazon.com/sagemaker/studio/) provides a single, web-based visual interface where you can perform all ML development activities including notebooks, experiment management, automatic model creation, debugging, and model and data drift detection.\n",
    "\n",
    "In this SageMaker Studio notebook, we highlight how you can use SageMaker to train models, create a deployable SageMaker model, and provide bias detection and explainability to analyze data and understand prediction outcomes from the model.\n",
    "This sample notebook walks you through:  \n",
    "\n",
    "1. Download and explore credit risk dataset - [South German Credit (UPDATE) Data Set](https://archive.ics.uci.edu/ml/datasets/South+German+Credit+%28UPDATE%29)\n",
    "2. Preprocessing data with sklearn on the dataset\n",
    "3. Training GBM model with XGBoost on the dataset\n",
    "4. Build an inference pipeline model (sklearn model and XGBoost model together) to preprocess input data and produce a prediction outcome per instance\n",
    "5. Hosting and scoring the single model (Optional)\n",
    "6. SageMaker Clarify job to provide Kernel SHAP values for the SageMaker model on training and test datasets.\n",
    "7. SageMaker Clarify job to provide bias metrics including pre-training bias metrics on data, \n",
    "\n",
    "![Credit risk explainability model inference](clarify_inf_pipeline_arch.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Prerequisites and Data exploration and Feature engineering\n",
    "### Initialize SageMaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from io import StringIO\n",
    "import os\n",
    "import time\n",
    "import sys\n",
    "import IPython\n",
    "from time import gmtime, strftime\n",
    "\n",
    "import boto3\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import urllib\n",
    "\n",
    "import sagemaker\n",
    "from sagemaker.s3 import S3Uploader\n",
    "from sagemaker.processing import ProcessingInput, ProcessingOutput\n",
    "from sagemaker.sklearn.processing import SKLearnProcessor\n",
    "from sagemaker.inputs import TrainingInput\n",
    "from sagemaker.xgboost import XGBoost\n",
    "from sagemaker.s3 import S3Downloader\n",
    "from sagemaker.s3 import S3Uploader\n",
    "from sagemaker import Session\n",
    "from sagemaker import get_execution_role\n",
    "from sagemaker.xgboost import XGBoostModel\n",
    "from sagemaker.sklearn import SKLearnModel\n",
    "from sagemaker.pipeline import PipelineModel\n",
    "\n",
    "\n",
    "session = Session()\n",
    "bucket = session.default_bucket()\n",
    "prefix = \"sagemaker/sagemaker-clarify-credit-risk-model\"\n",
    "region = session.boto_region_name\n",
    "\n",
    "# Define IAM role\n",
    "role = get_execution_role()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download data\n",
    "\n",
    "First,  __download__ the data and save it in the `data` folder.\n",
    "\n",
    "\n",
    "$^{[2]}$ Ulrike Grömping\n",
    "Beuth University of Applied Sciences Berlin\n",
    "Website with contact information: https://prof.beuth-hochschule.de/groemping/."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "S3Downloader.download(\n",
    "    \"s3://sagemaker-sample-files/datasets/tabular/uci_statlog_german_credit_data/SouthGermanCredit.asc\",\n",
    "    \"data\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "credit_columns = [\n",
    "    \"status\",\n",
    "    \"duration\",\n",
    "    \"credit_history\",\n",
    "    \"purpose\",\n",
    "    \"amount\",\n",
    "    \"savings\",\n",
    "    \"employment_duration\",\n",
    "    \"installment_rate\",\n",
    "    \"personal_status_sex\",\n",
    "    \"other_debtors\",\n",
    "    \"present_residence\",\n",
    "    \"property\",\n",
    "    \"age\",\n",
    "    \"other_installment_plans\",\n",
    "    \"housing\",\n",
    "    \"number_credits\",\n",
    "    \"job\",\n",
    "    \"people_liable\",\n",
    "    \"telephone\",\n",
    "    \"foreign_worker\",\n",
    "    \"credit_risk\",\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$`laufkont = status`\n",
    "                                               \n",
    " 1 : no checking account                       \n",
    " 2 : ... < 0 DM                                \n",
    " 3 : 0<= ... < 200 DM                          \n",
    " 4 : ... >= 200 DM / salary for at least 1 year\n",
    "\n",
    "$`laufzeit = duration`\n",
    "     \n",
    "\n",
    "$`moral = credit_history`\n",
    "                                                \n",
    " 0 : delay in paying off in the past            \n",
    " 1 : critical account/other credits elsewhere   \n",
    " 2 : no credits taken/all credits paid back duly\n",
    " 3 : existing credits paid back duly till now   \n",
    " 4 : all credits at this bank paid back duly    \n",
    "\n",
    "$`verw = purpose`\n",
    "                        \n",
    " 0 : others             \n",
    " 1 : car (new)          \n",
    " 2 : car (used)         \n",
    " 3 : furniture/equipment\n",
    " 4 : radio/television   \n",
    " 5 : domestic appliances\n",
    " 6 : repairs            \n",
    " 7 : education          \n",
    " 8 : vacation           \n",
    " 9 : retraining         \n",
    " 10 : business          \n",
    "\n",
    "$`hoehe = amount`\n",
    "     \n",
    "\n",
    "$`sparkont = savings`\n",
    "                               \n",
    " 1 : unknown/no savings account\n",
    " 2 : ... <  100 DM             \n",
    " 3 : 100 <= ... <  500 DM      \n",
    " 4 : 500 <= ... < 1000 DM      \n",
    " 5 : ... >= 1000 DM            \n",
    "\n",
    "$`beszeit = employment_duration`\n",
    "                     \n",
    " 1 : unemployed      \n",
    " 2 : < 1 yr          \n",
    " 3 : 1 <= ... < 4 yrs\n",
    " 4 : 4 <= ... < 7 yrs\n",
    " 5 : >= 7 yrs        \n",
    "\n",
    "$`rate = installment_rate`\n",
    "                   \n",
    " 1 : >= 35         \n",
    " 2 : 25 <= ... < 35\n",
    " 3 : 20 <= ... < 25\n",
    " 4 : < 20          \n",
    "\n",
    "$`famges = personal_status_sex`\n",
    "                                         \n",
    " 1 : male : divorced/separated           \n",
    " 2 : female : non-single or male : single\n",
    " 3 : male : married/widowed              \n",
    " 4 : female : single                     \n",
    "\n",
    "$`buerge = other_debtors`\n",
    "                 \n",
    " 1 : none        \n",
    " 2 : co-applicant\n",
    " 3 : guarantor   \n",
    "\n",
    "$`wohnzeit = present_residence`\n",
    "                     \n",
    " 1 : < 1 yr          \n",
    " 2 : 1 <= ... < 4 yrs\n",
    " 3 : 4 <= ... < 7 yrs\n",
    " 4 : >= 7 yrs        \n",
    "\n",
    "$`verm = property`\n",
    "                                              \n",
    " 1 : unknown / no property                    \n",
    " 2 : car or other                             \n",
    " 3 : building soc. savings agr./life insurance\n",
    " 4 : real estate                              \n",
    "\n",
    "$`alter = age`\n",
    "     \n",
    "\n",
    "$`weitkred = other_installment_plans`\n",
    "           \n",
    " 1 : bank  \n",
    " 2 : stores\n",
    " 3 : none  \n",
    "\n",
    "$`wohn = housing`\n",
    "             \n",
    " 1 : for free\n",
    " 2 : rent    \n",
    " 3 : own     \n",
    "\n",
    "$`bishkred = number_credits`\n",
    "         \n",
    " 1 : 1   \n",
    " 2 : 2-3 \n",
    " 3 : 4-5 \n",
    " 4 : >= 6\n",
    "\n",
    "$`beruf = job`\n",
    "                                               \n",
    " 1 : unemployed/unskilled - non-resident       \n",
    " 2 : unskilled - resident                      \n",
    " 3 : skilled employee/official                 \n",
    " 4 : manager/self-empl./highly qualif. employee\n",
    "\n",
    "$`pers = people_liable`\n",
    "              \n",
    " 1 : 3 or more\n",
    " 2 : 0 to 2   \n",
    "\n",
    "$`telef = telephone`\n",
    "                              \n",
    " 1 : no                       \n",
    " 2 : yes (under customer name)\n",
    "\n",
    "$`gastarb = foreign_worker`\n",
    "        \n",
    " 1 : yes\n",
    " 2 : no \n",
    "\n",
    "$`kredit = credit_risk`\n",
    "         \n",
    " 0 : bad \n",
    " 1 : good\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "training_data = pd.read_csv(\n",
    "    \"data/SouthGermanCredit.asc\",\n",
    "    names=credit_columns,\n",
    "    header=0,\n",
    "    sep=r\" \",\n",
    "    engine=\"python\",\n",
    "    na_values=\"?\",\n",
    ").dropna()\n",
    "\n",
    "training_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data inspection\n",
    "Plotting histograms for the distribution of the different features is a good way to visualize the data. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "plt.figure(figsize=(8, 8))\n",
    "sns.countplot('credit_risk', data=training_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create the raw training and test CSV files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# prepare raw test data\n",
    "test_data = training_data.sample(frac=0.1)\n",
    "test_data = test_data.drop([\"credit_risk\"], axis=1)\n",
    "test_filename = \"test.csv\"\n",
    "test_columns = [\n",
    "    \"status\",\n",
    "    \"duration\",\n",
    "    \"credit_history\",\n",
    "    \"purpose\",\n",
    "    \"amount\",\n",
    "    \"savings\",\n",
    "    \"employment_duration\",\n",
    "    \"installment_rate\",\n",
    "    \"personal_status_sex\",\n",
    "    \"other_debtors\",\n",
    "    \"present_residence\",\n",
    "    \"property\",\n",
    "    \"age\",\n",
    "    \"other_installment_plans\",\n",
    "    \"housing\",\n",
    "    \"number_credits\",\n",
    "    \"job\",\n",
    "    \"people_liable\",\n",
    "    \"telephone\",\n",
    "    \"foreign_worker\",\n",
    "]\n",
    "test_data.to_csv(test_filename, index=False, header=True, columns=test_columns, sep=\",\")\n",
    "\n",
    "# prepare raw training data\n",
    "train_filename = \"train.csv\"\n",
    "training_data.to_csv(train_filename, index=False, header=True, columns=credit_columns, sep=\",\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Encode and Upload Data\n",
    "Here we encode the training and test data. Encoding input data is not necessary for SageMaker Clarify, but is necessary for XGBoost models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_raw = S3Uploader.upload(test_filename, \"s3://{}/{}/data/test\".format(bucket, prefix))\n",
    "(test_raw)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_raw = S3Uploader.upload(train_filename, \"s3://{}/{}/data/train\".format(bucket, prefix))\n",
    "print(train_raw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing and feature engineering with SageMaker Processing job\n",
    "\n",
    "We will use SageMaker Processing jobs to perform the preprocessing on the raw data. SageMaker Processing provides prebuilt container for SKlearn which we will use here. We will output a sklearn model that can be used for preprocessing inference requests. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sklearn_processor = SKLearnProcessor(\n",
    "    role=role,\n",
    "    base_job_name=\"sagemaker-clarify-credit-risk-processing-job\",\n",
    "    instance_type=\"ml.m5.large\",\n",
    "    instance_count=1,\n",
    "    framework_version=\"0.20.0\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can have a look at the preprocessing script prepared to run in the processing job"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pygmentize processing/preprocessor.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### NOTE: THIS CELL WILL RUN FOR APPROX. 5-8 MINUTES! PLEASE BE PATIENT. \n",
    "For further documentation on SageMaker Processing, you can refer the documentation [here](https://docs.aws.amazon.com/sagemaker/latest/dg/processing-job.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw_data_path = \"s3://{0}/{1}/data/train/\".format(bucket, prefix)\n",
    "train_data_path = \"s3://{0}/{1}/data/preprocessed/train/\".format(bucket, prefix)\n",
    "val_data_path = \"s3://{0}/{1}/data/preprocessed/val/\".format(bucket, prefix)\n",
    "model_path = \"s3://{0}/{1}/sklearn/\".format(bucket, prefix)\n",
    "\n",
    "\n",
    "sklearn_processor.run(\n",
    "    code=\"processing/preprocessor.py\",\n",
    "    inputs=[\n",
    "        ProcessingInput(\n",
    "            input_name=\"raw_data\", source=raw_data_path, destination=\"/opt/ml/processing/input\"\n",
    "        )\n",
    "    ],\n",
    "    outputs=[\n",
    "        ProcessingOutput(\n",
    "            output_name=\"train_data\", source=\"/opt/ml/processing/train\", destination=train_data_path\n",
    "        ),\n",
    "        ProcessingOutput(\n",
    "            output_name=\"val_data\", source=\"/opt/ml/processing/val\", destination=val_data_path\n",
    "        ),\n",
    "        ProcessingOutput(\n",
    "            output_name=\"model\", source=\"/opt/ml/processing/model\", destination=model_path\n",
    "        ),\n",
    "    ],\n",
    "    arguments=[\"--train-test-split-ratio\", \"0.2\"],\n",
    "    logs=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train XGBoost Model\n",
    "In this step, we will train an XGBoost model on the preprocessed data. We will use our own training script with the built-in XGBoost container provided by SageMaker.\n",
    "\n",
    "Alternatively, for your own use case, you can also bring your own model (trained elsewhere) to SageMaker for processing with SageMaker Clarify\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pygmentize training/train_xgboost.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set up XGBoost Estimator\n",
    "\n",
    "Next, let us set up:    \n",
    " 1. Pre-defined values for Hyperparameters for XGBoost algorithm\n",
    " 1. XGBoost Estimator for SageMaker\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hyperparameters = {\n",
    "    \"max_depth\": \"5\",\n",
    "    \"eta\": \"0.1\",\n",
    "    \"gamma\": \"4\",\n",
    "    \"min_child_weight\": \"6\",\n",
    "    \"silent\": \"1\",\n",
    "    \"objective\": \"binary:logistic\",\n",
    "    \"num_round\": \"100\",\n",
    "    \"subsample\": \"0.8\",\n",
    "    \"eval_metric\": \"auc\",\n",
    "    \"early_stopping_rounds\": \"20\",\n",
    "}\n",
    "\n",
    "entry_point = \"train_xgboost.py\"\n",
    "source_dir = \"training/\"\n",
    "output_path = \"s3://{0}/{1}/{2}\".format(bucket, prefix, \"xgb_model\")\n",
    "code_location = \"s3://{0}/{1}/code\".format(bucket, prefix)\n",
    "\n",
    "estimator = XGBoost(\n",
    "    entry_point=entry_point,\n",
    "    source_dir=source_dir,\n",
    "    output_path=output_path,\n",
    "    code_location=code_location,\n",
    "    hyperparameters=hyperparameters,\n",
    "    instance_type=\"ml.c5.xlarge\",\n",
    "    instance_count=1,\n",
    "    framework_version=\"0.90-2\",\n",
    "    py_version=\"py3\",\n",
    "    role=role,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SageMaker Training\n",
    "\n",
    "Now it's time to train the model \n",
    "\n",
    "#### NOTE: THIS CELL WILL RUN FOR APPROX. 5-8 MINUTES! PLEASE BE PATIENT.\n",
    "For further documentation on SageMaker Training, you can refer the documentation [here](https://docs.aws.amazon.com/sagemaker/latest/dg/train-model.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "job_name = f\"credit-risk-xgb-{strftime('%Y-%m-%d-%H-%M-%S', gmtime())}\"\n",
    "\n",
    "train_input = TrainingInput(\n",
    "    \"s3://{0}/{1}/data/preprocessed/train/\".format(bucket, prefix), content_type=\"csv\"\n",
    ")\n",
    "val_input = TrainingInput(\n",
    "    \"s3://{0}/{1}/data/preprocessed/val/\".format(bucket, prefix), content_type=\"csv\"\n",
    ")\n",
    "\n",
    "inputs = {\"train\": train_input, \"validation\": val_input}\n",
    "\n",
    "estimator.fit(inputs, job_name=job_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Create SageMaker Model \n",
    "\n",
    "We will be preparing a SageMaker inference pipeline model which can be deployed as an endpoint or used with SageMaker Clarify:\n",
    "  1. Accept raw data as input\n",
    "  1. preprocess the data with the SKlearn model we built earlier\n",
    "  1. Pass the output of the Sklearn model as an input to the XGBoost model automatically\n",
    "  1. Deliver the final inference result from the XGBoost model\n",
    "  \n",
    "\n",
    "To know more, check out the documentation on inference pipelines: https://docs.aws.amazon.com/sagemaker/latest/dg/inference-pipelines.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Retrieve model artifacts\n",
    "\n",
    "First, we need to create two Amazon SageMaker Model objects, which associate the artifacts of training (serialized model artifacts in Amazon S3) to the Docker container used for inference. In order to do that, we need to get the paths to our serialized models in Amazon S3. We define the model data location of SKlearn and XGBoost models here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "preprocessor_model_data = \"s3://{}/{}/{}\".format(bucket, prefix, \"sklearn\") + \"/model.tar.gz\"\n",
    "\n",
    "xgboost_model_data = (\n",
    "    \"s3://{}/{}/{}/{}\".format(bucket, prefix, \"xgb_model\", job_name) + \"/output/model.tar.gz\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a SageMaker SKlearn Model Object\n",
    "\n",
    "Next step is to create an `SKlearnModel` object which will contain the following important information:\n",
    "  1. location of the sklearn model data\n",
    "  1. our custom inference code\n",
    "  1. SKlearn version to use (ensure this is the same the one used during pre-processing)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For hosting this model we provide a custom inference script, that is used to process the inputs and outputs and execute the transform.\n",
    "\n",
    "The inference script is implemented in the `inference/sklearn/inference.py` file. The custom script defines:\n",
    "\n",
    "- a custom `input_fn` for pre-processing inference requests. Our input function accepts only CSV input, loads the input in a Pandas dataframe and assigns feature column names to the dataframe\n",
    "- a custom `predict_fn` for running the transform over the inputs\n",
    "- a custom `model_fn` for deserializing the model\n",
    "\n",
    "We will be using the default implementation of the `output_function` provided by SageMaker SKlearn container. To know more, check out: https://github.com/aws/sagemaker-scikit-learn-container\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pygmentize inference/sklearn/inference.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "Now, let us define the SKLearnModel Object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sklearn_inference_code_location = \"s3://{}/{}/{}/code\".format(bucket, prefix, \"sklearn\")\n",
    "\n",
    "sklearn_model = SKLearnModel(\n",
    "    name=\"sklearn-model-{0}\".format(str(int(time.time()))),\n",
    "    model_data=preprocessor_model_data,\n",
    "    entry_point=\"inference.py\",\n",
    "    source_dir=\"inference/sklearn/\",\n",
    "    code_location=sklearn_inference_code_location,\n",
    "    role=role,\n",
    "    sagemaker_session=session,\n",
    "    framework_version=\"0.20.0\",\n",
    "    py_version=\"py3\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a SageMaker XGBoost Model object\n",
    "\n",
    "Similarly to the previous steps, we can create an XGBoost model object. Also here, we have to provide a custom inference script.\n",
    "\n",
    "The inference script is implemented in the `inference/xgboost/inference.py` file. The custom script defines:\n",
    "\n",
    "- a custom `input_fn` for pre-processing inference requests. This input function is able to handle JSON requests, plus all content types supported by the default XGBoost container. For additional information please visit: https://github.com/aws/sagemaker-xgboost-container/blob/master/src/sagemaker_xgboost_container/encoder.py. The reason for adding the JSON content type is that the container-to-container default request content type in an inference pipeline is JSON.\n",
    "\n",
    "- a custom `model_fn` for deserializing the model\n",
    "\n",
    "Let us have a look at the inference script.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pygmentize inference/xgboost/inference.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let us define the XGBoost model Object\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xgboost_inference_code_location = \"s3://{}/{}/{}/code\".format(bucket, prefix, \"xgb_model\")\n",
    "\n",
    "xgboost_model = XGBoostModel(\n",
    "    name=\"xgb-model-{0}\".format(str(int(time.time()))),\n",
    "    model_data=xgboost_model_data,\n",
    "    entry_point=\"inference.py\",\n",
    "    source_dir=\"inference/xgboost/\",\n",
    "    code_location=xgboost_inference_code_location,\n",
    "    framework_version=\"0.90-2\",\n",
    "    py_version=\"py3\",\n",
    "    role=role,\n",
    "    sagemaker_session=session,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Create a SageMaker Pipeline Model object\n",
    "\n",
    "Once we have models ready, we can deploy them in a pipeline, by building a PipelineModel object and calling the deploy() method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline_model_name = \"credit-risk-inference-pipeline-{0}\".format(str(int(time.time())))\n",
    "\n",
    "pipeline_model = PipelineModel(\n",
    "    name=pipeline_model_name,\n",
    "    role=role,\n",
    "    models=[sklearn_model, xgboost_model],\n",
    "    sagemaker_session=session,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Take note of the `model name` as it will be required while setting up the explainability job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline_model.name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deploy Model (optional - Not needed for Clarify)\n",
    "\n",
    "Let's deploy the model and test the inference pipeline.\n",
    "\n",
    "#### NOTE: THIS CELL WILL RUN FOR APPROX. 5-8 MINUTES! PLEASE BE PATIENT.\n",
    "For further documentation on SageMaker inference, you can refer the documentation [here](https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint_name = \"credit-risk-pipeline-endpoint-{0}\".format(str(int(time.time())))\n",
    "print(endpoint_name)\n",
    "\n",
    "pipeline_model.deploy(\n",
    "    initial_instance_count=1, instance_type=\"ml.m5.xlarge\", endpoint_name=endpoint_name\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inference (optional - Not needed for Clarify)\n",
    "\n",
    "Now that the model has been deployed, lets us optionally test it against the raw test data we created earlier in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_dataset = S3Downloader.read_file(test_raw)\n",
    "\n",
    "predictor = sagemaker.predictor.Predictor(\n",
    "    endpoint_name,\n",
    "    session,\n",
    "    serializer=sagemaker.serializers.CSVSerializer(),\n",
    "    deserializer=sagemaker.deserializers.CSVDeserializer(),\n",
    ")\n",
    "\n",
    "predictions = predictor.predict(test_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Amazon SageMaker Clarify\n",
    "\n",
    "Pre-requisities :\n",
    "\n",
    "1. [SageMaker Model](https://sagemaker.readthedocs.io/en/stable/api/inference/model.html) that can be deployed to a endpoint\n",
    "\n",
    "2. Input dataset\n",
    "                  \n",
    "3. SHAP Baseline\n",
    "\n",
    "Now that you have your model set up. Let's say hello to SageMaker Clarify!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker import clarify\n",
    "\n",
    "clarify_processor = clarify.SageMakerClarifyProcessor(\n",
    "    role=role, instance_count=1, instance_type=\"ml.c4.xlarge\", sagemaker_session=session\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explaining Predictions\n",
    "There are expanding business needs and legislative regulations that require explanations of _why_ a model made the decision it did. SageMaker Clarify uses [SHAP library](https://github.com/slundberg/shap) to explain the contribution that each input feature makes to the final decision. SageMaker Clarify uses a scalable and efficient implementation of [Kernel SHAP](https://github.com/slundberg/shap#model-agnostic-example-with-kernelexplainer-explains-any-function) with an option to use spark based parallelization with multiple processing instances. Note that Kernel SHAP and hence SageMaker Clarify has a model-agnostic feature attribution approach. Any ML model that is represented as a [SageMaker model](https://sagemaker.readthedocs.io/en/stable/api/inference/model.html) can be used with Clarify for explainability. \n",
    "\n",
    "Here is more information about explainability with Clarify and SHAP:\n",
    "\n",
    "    https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-model-explainability.html\n",
    "    \n",
    "    https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-shapley-values.html\n",
    "    \n",
    "    https://papers.nips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a baseline for SHAP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a contrastive explainability technique, SHAP values are calculated by evaluating the model on synthetic data generated against a baseline sample. The explanations of the same case can be different depending on the choices of this baseline sample. \n",
    "\n",
    "We are interested in explaining bad credit predictions. Hence, we would like the baseline choice to have E(x) closer to 1(belonging to the good credit class). \n",
    "\n",
    "We use the [mode](https://en.wikipedia.org/wiki/Mode_(statistics)) statistic to create the baseline. The mode is a good choice for categorical variables. We observe that the model prediction for the baseline has a high probability for the good credit class and hence it satisfies our requirement for the baseline. \n",
    "\n",
    "For more information on selecting informative vs non-informative baselines, see [SHAP Baselines for Explainability ](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-feature-attribute-shap-baselines.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load the raw training data in a data frame\n",
    "raw_train_df = pd.read_csv(\"train.csv\", header=0, names=None, sep=\",\")\n",
    "\n",
    "# drop the target column\n",
    "baseline = raw_train_df.drop([\"credit_risk\"], axis=1).mode().iloc[0].values.astype(\"int\").tolist()\n",
    "\n",
    "(baseline)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check baseline prediction E[f(x)]\n",
    "pred_baseline = predictor.predict(baseline)\n",
    "(pred_baseline)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup configurations for Clarify"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, setup some more configurations to start the explainability analysis by Clarify. We need to set up the following:\n",
    "  1. **SHAPConfig**: to create the baseline. In this example, the mean_abs is the mean of absolute SHAP values for all instances, specified as the baseline \n",
    "  1. **DataConfig**: to provide some basic information about data I/O to SageMaker Clarify. We specify where to find the input dataset, where to store the output, the header names, and the dataset type.\n",
    "  1. **ModelConfig**: to specify information about the trained model here we re-use the model name created earlier. \n",
    "  \n",
    "  Note: To avoid additional traffic to your production models, SageMaker Clarify sets up and tears down a ephemeral endpoint while processing. ModelConfig specifies your preferred instance type and instance count used to run your model on during Clarify's processing.\n",
    "  \n",
    "To know more about what these configurations mean for Clarify, check out the documentation here: https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-configure-processing-jobs.html\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "shap_config = clarify.SHAPConfig(\n",
    "    baseline=[baseline],\n",
    "    num_samples=2000,  # num_samples are permutations from your features, so should be large enough as compared to number of input features, for example, 2k + 2* num_features\n",
    "    agg_method=\"mean_abs\",\n",
    "    use_logit=True,\n",
    ")  # we want the shap values to have log-odds units so that the equation 'shap values + expected probability =  predicted probability' for each instance record )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "explainability_report_prefix = \"{}/clarify-explainability\".format(prefix)\n",
    "explainability_output_path = \"s3://{}/{}\".format(bucket, explainability_report_prefix)\n",
    "\n",
    "explainability_data_config = clarify.DataConfig(\n",
    "    s3_data_input_path=test_raw,\n",
    "    s3_output_path=explainability_output_path,\n",
    "    # label='credit_risk', # target column is not present in the test dataset\n",
    "    headers=test_columns,\n",
    "    dataset_type=\"text/csv\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_config = clarify.ModelConfig(\n",
    "    model_name=pipeline_model.name,  # specify the inference pipeline model name\n",
    "    instance_type=\"ml.c5.xlarge\",\n",
    "    instance_count=1,\n",
    "    accept_type=\"text/csv\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run SageMaker Clarify Explainability job\n",
    "\n",
    "All the configurations are in place. Let's start the explainability job. This will spin up an ephemeral SageMaker endpoint and perform inference and calculate explanations on that endpoint. It does not use any existing production endpoint deployments.\n",
    "\n",
    "#### NOTE: THIS CELL WILL RUN FOR APPROX. 5-8 MINUTES! PLEASE BE PATIENT.\n",
    "For further documentation on SageMaker Clarify , you can refer the documentation [here](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-fairness-and-explainability.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clarify_processor.run_explainability(\n",
    "    data_config=explainability_data_config,\n",
    "    model_config=model_config,\n",
    "    explainability_config=shap_config,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Viewing the Explainability Report\n",
    "\n",
    "Once the job is complete, you can view the explainability report in Studio under the 'Experiments and trials' tab\n",
    "\n",
    "Look out for a trial component named 'clarify-explainability-' and see the Explainability tab. \n",
    "\n",
    "If you're not a Studio user yet, you can access this report at the following S3 bucket.\n",
    "\n",
    "The report contains global explanations for the model with the input dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "explainability_output_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run_explainability_job_name = clarify_processor.latest_job.job_name\n",
    "run_explainability_job_name\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(\n",
    "    HTML(\n",
    "        '<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/sagemaker/home?region={}#/processing-jobs/{}\">Processing Job</a></b>'.format(\n",
    "            region, run_explainability_job_name\n",
    "        )\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(\n",
    "    HTML(\n",
    "        '<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/cloudwatch/home?region={}#logStream:group=/aws/sagemaker/ProcessingJobs;prefix={};streamFilter=typeLogStreamPrefix\">CloudWatch Logs</a> After About 5 Minutes</b>'.format(\n",
    "            region, run_explainability_job_name\n",
    "        )\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(\n",
    "    HTML(\n",
    "        '<b>Review <a target=\"blank\" href=\"https://s3.console.aws.amazon.com/s3/buckets/{}?prefix={}/\">S3 Output Data</a> After The Processing Job Has Completed</b>'.format(\n",
    "            bucket, explainability_report_prefix\n",
    "        )\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "explainability_output_path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download report from S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!aws s3 ls $explainability_output_path/\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!aws s3 cp --recursive $explainability_output_path ./explainability_report/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### View the explainability pdf report below to see global explanations with SHAP for the model. The report also includes a SHAP summary plot for all individual  instances in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "\n",
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(HTML('<b>Review <a target=\"blank\" href=\"./explainability_report/report.html\">Explainability Report</a></b>'))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View the Explainability Report in SageMaker Studio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "you can also view the explainability report in Studio under the experiments tab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](static/explainability_detail.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze the local explanations of individual predictions by Clarify \n",
    "\n",
    "#### The pre-requisite for this section is that you have generated individual predictions for the test dataset by running inference with a SageMaker endpoint in the optional sections earlier\n",
    "In this section, we will analyze and understand the local explainability results for each individual prediction produced by Clarify. Clarify produces a CSV file which contains the SHAP value for each feature per prediction. Let us download the CSV."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.s3 import S3Downloader\n",
    "import json\n",
    "import io\n",
    "\n",
    "# read the shap values\n",
    "S3Downloader.download(s3_uri=explainability_output_path + \"/explanations_shap\", local_path=\"output\")\n",
    "shap_values_df = pd.read_csv(\"output/out.csv\")\n",
    "print(shap_values_df.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that by default SHAP explains classifier models in terms of their margin output, before the logistic link function. That means the units of SHAP output and are log-odds units, so negative values imply probabilities of less than 0.5 meaning bad credit class (class 0). \n",
    "\n",
    "#### A brief technical summary of prediction output before the logistic link function and SHAP values\n",
    "\n",
    "y = f(x) is the log-odd (logit) unit for the prediction output\n",
    "\n",
    "E(y) is the log-odd (logit) unit for the prediction on the input baseline\n",
    "\n",
    "SHAP values are in log-odd units as well \n",
    "\n",
    "The following is expected to hold true for every individual prediction : \n",
    "\n",
    "sum(SHAP values) + E(y)) == model_prediction_logit\n",
    "\n",
    "logistic(model_prediction_logit) = model_prediction_probability\n",
    "\n",
    "E(y) < 0 implies baseline probability less than 0.5 (bad credit baseline)\n",
    "\n",
    "E(y) > 0 implies baseline probability greater than 0.5 (good credit baseline)\n",
    "\n",
    "y < 0 implies predicted probability less than 0.5 (bad credit)\n",
    "\n",
    "y > 0 implies predicted probability greater than 0.5 (good credit) \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can retrieve E(y) , the log-odd unit of the prediction for the baseline input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the base expected value to be used to plot SHAP values\n",
    "S3Downloader.download(s3_uri=explainability_output_path + \"/analysis.json\", local_path=\"output\")\n",
    "\n",
    "with open(\"output/analysis.json\") as json_file:\n",
    "    data = json.load(json_file)\n",
    "    base_value = data[\"explanations\"][\"kernel_shap\"][\"label0\"][\"expected_value\"]\n",
    "\n",
    "print(\"E(y): \", base_value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As described in the earlier section, we have a baseline representing good credit prediction to be used with SHAP to contrast and explain bad credit predictions. E(y) > 0  implies baseline probability greater than 0.5 (good credit baseline). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a dataframe containing the model predictions generated earlier during inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pandas import DataFrame\n",
    "\n",
    "predictions_df = DataFrame(predictions, columns=[\"probability_score\"])\n",
    "\n",
    "predictions_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Join the predictions, SHAP value and test data\n",
    "\n",
    "Now, we create a single dataframe containing all test data rows, with their corresponding SHAP values and prediction score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# join the probability score and shap values together in a single data frame\n",
    "predictions_df.reset_index(drop=True, inplace=True)\n",
    "shap_values_df.reset_index(drop=True, inplace=True)\n",
    "test_data.reset_index(drop=True, inplace=True)\n",
    "\n",
    "prediction_shap_df = pd.concat([predictions_df, shap_values_df, test_data], axis=1)\n",
    "prediction_shap_df[\"probability_score\"] = pd.to_numeric(\n",
    "    prediction_shap_df[\"probability_score\"], downcast=\"float\"\n",
    ")\n",
    "\n",
    "prediction_shap_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert the probability score to binary prediction\n",
    "\n",
    "Now, convert the probability scores to a binary value(1/0), based on a threshold(0.5), where probability scores greater than 0.5 are positive outcomes (good credit) and lesser are negative outcomes (bad credit)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a new column as 'Prediction' converting the probability score to either 1 or 0\n",
    "prediction_shap_df.insert(\n",
    "    0, \"Prediction\", (prediction_shap_df[\"probability_score\"] > 0.5).astype(int)\n",
    ")\n",
    "\n",
    "prediction_shap_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filter for bad credit predictions only\n",
    "\n",
    "Since we interested in explaining negative outcomes (bad credit predictions) only in this use case, we filter the records to keep only the record with prediction as 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bad_credit_outcomes_df = prediction_shap_df[prediction_shap_df.iloc[:, 0] == 0]\n",
    "bad_credit_outcomes_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create SHAP plots \n",
    "\n",
    "Now we try to create some additional SHAP plots to understand how much different features contributed to a specific negative outcome."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Install open source SHAP library for more visualizations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!conda install -c conda-forge shap -y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import shap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### SHAP explanation plot for a single bad credit ensemble prediction instance. We will select the prediction instance with the lowest probability. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "min_index = prediction_shap_df[\"probability_score\"].idxmin()\n",
    "print(min_index)\n",
    "print(\"mean probability of dataset\")\n",
    "print(prediction_shap_df[[\"probability_score\"]].mean())\n",
    "print(\"individual probability\")\n",
    "print(prediction_shap_df.iloc[min_index, 1])\n",
    "print(\"sum of shap values\")\n",
    "print(prediction_shap_df.iloc[min_index, 2:22].sum())\n",
    "print(\"base value from analysis.json\")\n",
    "print(base_value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example 'bad credit' prediction SHAP values.\n",
    "\n",
    "In the chart below, f(x) is the prediction of this particular individual instance in log-odd units. If negative, it means it is a bad credit prediction. \n",
    "\n",
    "In the chart below, E(f(x)) is the prediction of the baseline input in log-odd units. It is positive , which means it belongs to the good credit class. \n",
    "\n",
    "The individual example is contrasted against the good credit baseline. So the features with negative SHAP values drive the final negative decision from the initial baseline positive value.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### In the below example, the input features (status = 1) , (purpose = 0) and (personal_status_sex = 2) are the top 3 features driving the negative decision. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can refer the data description to understand the mapping of these values to logical categories. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "explanation_obj = shap._explanation.Explanation(\n",
    "    values=prediction_shap_df.iloc[min_index, 2:22].to_numpy(),\n",
    "    base_values=base_value,\n",
    "    data=test_data.iloc[min_index].to_numpy(),\n",
    "    feature_names=test_data.columns,\n",
    ")\n",
    "shap.plots.waterfall(shap_values=explanation_obj, max_display=20, show=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Feel free to change the min_index in the plot above to explain predictions of other individual instances"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Detect data bias with Amazon SageMaker Clarify\n",
    "#### Amazon Science: [How Clarify helps machine learning developers detect unintended bias](https://www.amazon.science/latest-news/how-clarify-helps-machine-learning-developers-detect-unintended-bias)\n",
    "\n",
    "#### [Clarify Terms for Bias and Fairness](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-detect-data-bias.html) \n",
    "\n",
    "#### [Pre-training bias metrics](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-measure-data-bias.html)   \n",
    "\n",
    "#### [Post-training bias metrics](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-measure-post-training-bias.html)\n",
    "\n",
    "#### Calculate pre-training and post-training Bias metrics\n",
    "\n",
    "Note: You can also execute pre-training and post-training bias detection jobs separately"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A DataConfig object communicates some basic information about data I/O to Clarify. We specify where to find the input dataset, where to store the output, the target column (label), the header names, and the dataset type.\n",
    "\n",
    "Similarly, the ModelConfig (created earlier for the explainability job) object communicates information about your trained model and ModelPredictedLabelConfig provides information on the format of your predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bias_report_prefix = \"{}/clarify-bias\".format( prefix)\n",
    "bias_report_output_path = \"s3://{}/{}\".format(bucket,bias_report_prefix)\n",
    "bias_data_config = clarify.DataConfig(\n",
    "    s3_data_input_path=train_raw,\n",
    "    s3_output_path=bias_report_output_path,\n",
    "    label=\"credit_risk\",\n",
    "    headers=training_data.columns.to_list(),\n",
    "    dataset_type=\"text/csv\",\n",
    ")\n",
    "predictions_config = clarify.ModelPredictedLabelConfig(label=None, probability=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SageMaker Clarify also needs the sensitive columns (facets) and the desirable outcomes (facet_values_or_threshold).\n",
    "\n",
    "We specify this information in the BiasConfig API. Here  age is the facet that we analyze and 40 is the threshold. The group 'personal_status_sex' is used to form subgroups for the measurement of Conditional Demographic Disparity (CDD) metric only."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bias_config = clarify.BiasConfig(\n",
    "    label_values_or_threshold=[1],\n",
    "    facet_name=\"age\",\n",
    "    facet_values_or_threshold=[40],\n",
    "    group_name=\"personal_status_sex\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### NOTE: THIS CELL WILL RUN FOR APPROX. 5-8 MINUTES! PLEASE BE PATIENT.\n",
    "For further documentation on SageMaker Clarify, you can refer the documentation [here](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-fairness-and-explainability.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clarify_processor.run_bias(\n",
    "    data_config=bias_data_config,\n",
    "    bias_config=bias_config,\n",
    "    model_config=model_config,\n",
    "    model_predicted_label_config=predictions_config,\n",
    "    pre_training_methods=\"all\",\n",
    "    post_training_methods=\"all\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Viewing the Bias detection Report\n",
    "You can view the bis detection report in Studio under the experiments tab \n",
    "\n",
    "If you're not a Studio user yet, you can access this report at the following S3 bucket."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bias_report_output_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run_post_training_bias_processing_job_name = clarify_processor.latest_job.job_name\n",
    "run_post_training_bias_processing_job_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "\n",
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(\n",
    "    HTML(\n",
    "        '<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/sagemaker/home?region={}#/processing-jobs/{}\">Processing Job</a></b>'.format(\n",
    "            region, run_post_training_bias_processing_job_name\n",
    "        )\n",
    "    )\n",
    ")\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(\n",
    "    HTML(\n",
    "        '<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/cloudwatch/home?region={}#logStream:group=/aws/sagemaker/ProcessingJobs;prefix={};streamFilter=typeLogStreamPrefix\">CloudWatch Logs</a> After About 5 Minutes</b>'.format(\n",
    "            region, run_post_training_bias_processing_job_name\n",
    "        )\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(\n",
    "    HTML(\n",
    "        '<b>Review <a target=\"blank\" href=\"https://s3.console.aws.amazon.com/s3/buckets/{}?prefix={}/\">S3 Output Data</a> After The Processing Job Has Completed</b>'.format(\n",
    "            bucket, bias_report_prefix\n",
    "        )\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Download Report From S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!aws s3 ls $bias_report_output_path/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bias_report_output_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!aws s3 cp --recursive $bias_report_output_path ./generated_bias_report/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### View the bias report pdf that contains the pre-training bias and post-training bias metrics. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(HTML('<b>Review <a target=\"blank\" href=\"./generated_bias_report/report.html\">Bias Report</a></b>'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View Bias Report in Studio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, you can also view the bias report in Studio under the experiments tab. Each bias metric has detailed explanations with examples that you can explore. You could also summarize the results in a handy table!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](static/bias_report.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us specifically look at a couple of pre-training and post-training bias metrics. \n",
    "\n",
    "Pre-training bias metrics\n",
    "1. Class imbalance\n",
    "2. DPL - Difference in positive proportions in true labels \n",
    "\n",
    "Post-training bias metrics\n",
    "1. DPPL - Difference in positive proportions in predicted labels\n",
    "2. DI - Disparate Impact"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "S3Downloader.download(s3_uri=bias_report_output_path + \"/analysis.json\", local_path=\"output\")\n",
    "\n",
    "with open(\"output/analysis.json\") as json_file:\n",
    "    data = json.load(json_file)\n",
    "    print(\"pre-training bias metrics\")\n",
    "    class_imbalance = data[\"pre_training_bias_metrics\"][\"facets\"][\"age\"][0][\"metrics\"][1][\"value\"]\n",
    "    print(\"class imbalance: \", class_imbalance)\n",
    "    DPL = data[\"pre_training_bias_metrics\"][\"facets\"][\"age\"][0][\"metrics\"][2][\"value\"]\n",
    "    print(\"DPL: \", DPL)\n",
    "    print(\"\\n\")\n",
    "    print(\"post training bias metrics\")\n",
    "    DPPL = data[\"post_training_bias_metrics\"][\"facets\"][\"age\"][0][\"metrics\"][6][\"value\"]\n",
    "    print(\"DPPL: \", DPPL)\n",
    "    DI = data[\"post_training_bias_metrics\"][\"facets\"][\"age\"][0][\"metrics\"][5][\"value\"]\n",
    "    print(\"DI: \", DI)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we see that for the \"age\" facet with threshold of [40] , [CI](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-bias-metric-class-imbalance.html) and [DI](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-post-training-bias-metric-di.html) are high , whereas [DPL](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-data-bias-metric-true-label-imbalance.html) and [DPPL](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-post-training-bias-metric-dppl.html) are low. Data pre-processing techniques can be applied to mitigate the pre-training bias and training algorithms can be re-evaluated to mitigate the post-training bias. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Clean Up\n",
    "Finally, don't forget to clean up the resources we set up and used for this demo!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "session.delete_endpoint(endpoint_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "session.delete_model(pipeline_model.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Additional Resources to explore "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* [Working toward fairer machine learning](https://www.amazon.science/research-awards/success-stories/algorithmic-bias-and-fairness-in-machine-learning)\n",
    "* [Fairness Measures for Machine Learning in Finance](https://pages.awscloud.com/rs/112-TZM-766/images/Fairness.Measures.for.Machine.Learning.in.Finance.pdf)\n",
    "* [Amazon SageMaker Clarify: Machine learning bias detection and explainability in the cloud](https://www.amazon.science/publications/amazon-sagemaker-clarify-machine-learning-bias-detection-and-explainability-in-the-cloud)\n",
    "* [Amazon AI Fairness and Explainability Whitepaper](https://pages.awscloud.com/rs/112-TZM-766/images/Amazon.AI.Fairness.and.Explainability.Whitepaper.pdf)\n",
    "* [How Clarify helps machine learning developers detect unintended bias](https://www.amazon.science/latest-news/how-clarify-helps-machine-learning-developers-detect-unintended-bias)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:eu-west-1:470317259841:image/datascience-1.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}