Recommender with Scikit Surprise.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `scikit-surprise` recommender systems on SageMaker\n",
    "\n",
    "This notebook demonstrates how to build a movie recommender system using [`scikit-surprise`](http://surpriselib.com/) on [Amazon SageMaker](https://aws.amazon.com/sagemaker/), using the [SageMaker SKLearn Estimator](https://sagemaker.readthedocs.io/en/stable/using_sklearn.html) as a base to avoid building custom containers.\n",
    "\n",
    "**Note that `surprise` is a [\"SciKit\"](https://www.scipy.org/scikits.html) and therefore a *peer/sibling* of `scikit-learn`, not a part of it**. This means we can't just expect the SM SKLearn container to understand a surprise model file - we have to show it how.\n",
    "\n",
    "The notebook should run fine on a ml.t2.medium and consume minimal additional resources to fit/deploy the model as the data set is small"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install `scikit-surprise`\n",
    "\n",
    "Although we won't be fitting models on this notebook instance itself, we'll install surprise so we can use the module's standard data pre-processing tools.\n",
    "\n",
    "We demonstrate inline installation for portability/simplicity, but there's guidance [here](https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html) on how installations can be integrated into the notebook instance's creation/startup."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!pip install scikit-surprise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load libraries and configuration\n",
    "\n",
    "**TODO:** Create your target bucket, check this notebook's role has access to it, and update the config below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Python Built-Ins:\n",
    "import json\n",
    "import os\n",
    "\n",
    "# Libraries:\n",
    "import boto3\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import sagemaker\n",
    "from sagemaker.sklearn.estimator import SKLearn as SMSKLearnEstimator\n",
    "import surprise\n",
    "\n",
    "sagemaker_session = sagemaker.Session()\n",
    "role = sagemaker.get_execution_role()  # We'll use this notebook's role for our API interactions\n",
    "\n",
    "# We don't need a sagemaker boto client as all our operations are supported via the Python SDK:\n",
    "# sm_client = sagemaker_session.boto_session.client(\"sagemaker\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The bucket we'll use:\n",
    "bucket = # TODO: Enter your bucket name here! (\"in quotes\")\n",
    "\n",
    "data_prefix = \"data\"\n",
    "train_filename = \"movie-lens-100k-training.csv\"\n",
    "test_filename = \"movie-lens-100k-test.csv\"\n",
    "\n",
    "# The output of training jobs (i.e. trained models or failure details)\n",
    "output_prefix = \"output\"\n",
    "\n",
    "# The results of batch transform jobs (i.e. estimated ratings for test user/movie pairs)\n",
    "results_prefix = \"results\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fetch the data set\n",
    "\n",
    "We demonstrate recommendation on the [MovieLens](https://grouplens.org/datasets/movielens/) 100K benchmark set as a small/easy example.\n",
    "\n",
    "Remember if exploring bigger data sets that the data prep code here is working on the notebook instance itself. The default notebook storage volume size is 5GB, but it can be set higher on creation or while stopped.\n",
    "\n",
    "The column names are chosen for consistency with [Amazon Personalize](https://docs.aws.amazon.com/personalize/latest/dg/data-prep-formatting.html) - our advanced recommender engine as-a-service which you might be interested to check out!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget -N http://files.grouplens.org/datasets/movielens/ml-100k.zip\n",
    "!rm -rf ./data/raw\n",
    "!mkdir -p ./data/raw\n",
    "!unzip ml-100k.zip -d ./data/raw"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "movielens = pd.read_csv(\n",
    "    \"./data/raw/ml-100k/u.data\",\n",
    "    sep=\"\\t\",\n",
    "    names=[\"USER_ID\", \"ITEM_ID\", \"RATING\", \"TIMESTAMP\"]\n",
    ")\n",
    "movielens.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare the data\n",
    "\n",
    "surprise provides its own tools for data input and preparation, so let's use them to split our training vs test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = surprise.Dataset.load_from_df(\n",
    "    movielens[[\"USER_ID\", \"ITEM_ID\", \"RATING\"]],\n",
    "    surprise.Reader(line_format=u\"user item rating\", rating_scale=(1, 5))\n",
    ")\n",
    "\n",
    "train_data, test_data = surprise.model_selection.train_test_split(data, test_size=.25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# (train_test_split() actually gives different data types for train_data vs test_data...)\n",
    "train_df = pd.DataFrame(train_data.all_ratings(), columns=[\"USER_ID\", \"ITEM_ID\", \"RATING\"])\n",
    "test_df = pd.DataFrame(test_data, columns=[\"USER_ID\", \"ITEM_ID\", \"RATING\"])\n",
    "train_df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Upload data to S3\n",
    "\n",
    "SageMaker training and transform jobs use S3 for data input and output, so we need to upload the prepared sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We have full control over the training script, but batch transform jobs are orchestrated by SageMaker\n",
    "# (e.g. data job splitting) so the test file **cannot have a header**:\n",
    "!mkdir -p $data_prefix\n",
    "train_df.to_csv(os.path.join(data_prefix, train_filename), index=False)\n",
    "test_df.to_csv(os.path.join(data_prefix, test_filename), index=False, header=False)\n",
    "\n",
    "boto3.Session().resource(\"s3\").Bucket(bucket).Object(\n",
    "    \"{}/{}\".format(data_prefix, train_filename)\n",
    ").upload_file(os.path.join(data_prefix, train_filename))\n",
    "\n",
    "boto3.Session().resource(\"s3\").Bucket(bucket).Object(\n",
    "    \"{}/{}\".format(data_prefix, test_filename)\n",
    ").upload_file(os.path.join(data_prefix, test_filename))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the model\n",
    "\n",
    "The SKLearn Estimator takes an `entry_point` script which defines training and inference/runtime behaviour; and optionally also a `source_dir` folder if a bundle of multiple files should be loaded in to the container.\n",
    "\n",
    "SKLearn usage and script interface/requirements are documented [here](https://sagemaker.readthedocs.io/en/stable/using_sklearn.html), with the internal `Transformer` code [here](https://github.com/aws/sagemaker-containers/blob/master/src/sagemaker_containers/_transformer.py) giving some insight into how the functions are applied.\n",
    "\n",
    "The `entry_point` is loaded by SageMaker into a container and used in two different ways:\n",
    "\n",
    "* In a training job, SageMaker loads input S3 data \"channels\" into folders in the container and runs the `entry_point` **as a script**. The script should fit the model and output the model (or failure logs) into a specified output folder: which SageMaker will then map back to S3.\n",
    "* At inference time (batch or online endpoint), SageMaker loads the `entry_point` **as a module** inside a bigger web server application that you don't need to write: `entry_point` should export functions to handle loading the trained model from disk and performing inferences.\n",
    "\n",
    "Our `surprise-recommender.py` implementation has the following key parts:\n",
    "\n",
    "1. An `if __name__ == \"__main__\"` guard clause to separate code that should only execute when the file is run as a script (the training job)\n",
    "2. A `model_fn` which can load a trained model from disk into memory\n",
    "3. A `predict_fn` which executes a `model` against requested input `data`\n",
    "4. (To show how a flexible API can be implemented) an `input_fn` to intpreret different formats of request correctly.\n",
    "\n",
    "For compact single-file scripts, it's possible to install `surprise` inline, by adding a line like the following **before** the library is `import`ed:\n",
    "\n",
    "```python\n",
    "subprocess.call([sys.executable, \"-m\", \"pip\", \"install\", \"surprise\"])\n",
    "```\n",
    "\n",
    "Alternatively (as we'll use here), we can put a **requirements.txt** file in our source bundle listing dependencies. This file should follow the [standard pip requirements file syntax](https://pip.pypa.io/en/stable/user_guide/#requirements-files).\n",
    "\n",
    "<div class=\"alert alert-block alert-warning\">\n",
    "<b>Performance note:</b> Both these install methods are processed at application start-up. To reduce start-up overhead we could instead create a custom container image inheriting from the SageMaker SciKit Learn base, with the extra dependencies pre-installed... But this approach is a little more complex (requires pushing container images to a registry, etc).\n",
    "</div>\n",
    "\n",
    "Review the `surprise-recommender.py` file to understand how it interacts with SageMaker."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "script_path = \"surprise-recommender.py\"\n",
    "source_dir = \"src-surprise\"\n",
    "\n",
    "estimator = SMSKLearnEstimator(\n",
    "    entry_point=script_path,\n",
    "    source_dir=source_dir,\n",
    "    train_instance_type=\"ml.c4.xlarge\",\n",
    "    role=role,\n",
    "    sagemaker_session=sagemaker_session,\n",
    "    output_path=\"s3://{}/{}\".format(bucket, output_prefix),\n",
    "    \n",
    "    # possibly e.g. hyperparameters={ 'max_leaf_nodes': 30 }, if we had any\n",
    "    \n",
    "    # training on spot instances is an easy way to save cost:\n",
    "    train_use_spot_instances=True,\n",
    "    train_max_run=60*5, # 5 mins max actual run time\n",
    "    train_max_wait=60*10 # 10 mins max wait for spot interruptions\n",
    ")\n",
    "\n",
    "# Instead of just specifying the training channel as an S3 path string, we can use s3_input to get more control:\n",
    "train_channel = sagemaker.session.s3_input(\n",
    "    \"s3://{}/{}/{}\".format(bucket, data_prefix, train_filename), \n",
    "    distribution=\"FullyReplicated\",\n",
    "    content_type=\"text/csv\",\n",
    "    s3_data_type=\"S3Prefix\"\n",
    ")\n",
    "\n",
    "# This will block until training is complete, showing console output below:\n",
    "estimator.fit({ \"train\": train_channel })"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test model with batch inference\n",
    "\n",
    "Now that the model has been trained, we can use SageMaker Batch Transform to run it against a bulk set e.g. our test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Define a SKLearn Transformer from the trained Estimator\n",
    "transformer = estimator.transformer(\n",
    "    instance_count=1, \n",
    "    instance_type=\"ml.m4.xlarge\",\n",
    "    assemble_with=\"Line\",\n",
    "    accept=\"text/csv\",\n",
    "    output_path=\"s3://{}/{}\".format(bucket, results_prefix)\n",
    "    # By default data will be processed in batches for speed: You could add strategy=\"SingleRecord\"\n",
    ")\n",
    "\n",
    "# Start the inference job\n",
    "transformer.transform(\n",
    "    \"s3://{}/{}/{}\".format(bucket, data_prefix, test_filename),\n",
    "    content_type=\"text/csv\",\n",
    "    split_type=\"Line\",\n",
    "    input_filter=\"$[0:1]\" # Only send the first two columns (the input features UID & IID)\n",
    ")\n",
    "\n",
    "print(\"Waiting for transform job: {}\".format(transformer.latest_transform_job.job_name))\n",
    "transformer.wait()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download the raw output data from S3 to local filesystem\n",
    "batch_output = transformer.output_path\n",
    "!mkdir -p $results_prefix/\n",
    "!aws s3 cp --recursive $batch_output/ $results_prefix/\n",
    "# (Head to see what the batch output looks like)\n",
    "!head $results_prefix/*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# There should be just one .out file, which we can load into a dataframe\n",
    "results_df = pd.read_csv(\n",
    "    os.path.join(results_prefix, \"{}.out\".format(test_filename)),\n",
    "    names=[\"USER_ID\", \"ITEM_ID\", \"RATING_ACTUAL\", \"RATING_PREDICTED\", \"RESULT_METADATA\"]\n",
    ")\n",
    "results_df.head()\n",
    "# Note the RATING_ACTUAL field is empty because surprise gives us the option of passing in actuals, but we chose\n",
    "# not to send them. Could join this dataframe on to the underlying test CSV to match up the actuals."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deploy the model for online/realtime inference\n",
    "\n",
    "As well as batch jobs, we can deploy our model as an API endpoint for real-time predictions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# deploy() should print periodic \"-\"s while running, and a \"!\" when finished.\n",
    "print(\"Deploying model...\")\n",
    "predictor_raw = estimator.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=\"ml.m4.xlarge\"\n",
    "    #content_type=sagemaker.content_types.CONTENT_TYPE_JSON\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Our entry point's input_fn() is set up to support both single and batch requests:\n",
    "online_test_1 = (186, 377)\n",
    "print(\"Sample prediction for {}: Result = \\n{}\".format(online_test_1, predictor_raw.predict(online_test_1)))\n",
    "\n",
    "online_test_2 = [(186, 377), (697, 333), (308, 664)]\n",
    "print(\"Sample prediction for {}: Result = \\n{}\".format(online_test_2, predictor_raw.predict(online_test_2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Alternative API formats\n",
    "\n",
    "The SKLearn Predictor defaults to raw/numpy array input and output because it's idiomatic for scikit-learn.\n",
    "\n",
    "The deployed `endpoint` can actually accept and generate whichever content types you set it up to support.\n",
    "\n",
    "Here we create an alternative `Predictor` object, pointing at the same deployed endpoint but demonstrating a more web-API-like JSON endpoint:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor_json = sagemaker.predictor.RealTimePredictor(\n",
    "    endpoint=predictor_raw.endpoint,\n",
    "    accept=sagemaker.content_types.CONTENT_TYPE_JSON,\n",
    "    content_type=sagemaker.content_types.CONTENT_TYPE_JSON\n",
    ")\n",
    "\n",
    "online_test_3 = '{ \"uid\": 186, \"iid\": 377 }'\n",
    "print(\"Sample prediction for {}: Result = \\n{}\".format(\n",
    "    online_test_3,\n",
    "    json.loads(predictor_json.predict(online_test_3))\n",
    "))\n",
    "\n",
    "online_test_4 = '[{ \"uid\": 186, \"iid\": 377 }, { \"uid\": 697, \"iid\": 333 }]'\n",
    "print(\"Sample prediction for {}: Result = \\n{}\".format(\n",
    "    online_test_4,\n",
    "    json.loads(predictor_json.predict(online_test_4))\n",
    "))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clean up: delete the endpoint\n",
    "\n",
    "Remember to clean up endpoint resources when no longer in use.\n",
    "\n",
    "You may also like to do the following from the AWS console:\n",
    "\n",
    "- Delete / clear out your bucket of data, models and results\n",
    "- Stop this SageMaker notebook instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor_raw.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Next steps\n",
    "\n",
    "As of 10th June 2019, AWS also offers the [Amazon Personalize]() managed service for recommender engine AutoML: Including modern algorithms that can sometimes significantly outperform the approaches used here.\n",
    "\n",
    "This sample uses the same MovieLens data set as the [Amazon Personalize sample](https://github.com/aws-samples/amazon-personalize-samples), to help you experiment comparing the two!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_python3",
   "language": "python",
   "name": "conda_python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}