sdk/python/endpoints/online/mlflow/online-endpoints-deploy-mlflow-model-with-script.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploy MLflow model to online endpoints with a custom environment and scoring script\n",
    "\n",
    "Learn how to deploy your [MLflow](https://www.mlflow.org/) model to an [online endpoint](https://docs.microsoft.com/azure/machine-learning/concept-endpoints) using a custom environment and scoring script."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Requirements - In order to benefit from this tutorial, you will need:\n",
    "- This sample notebook assumes you're using online endpoints; for more information, see [What are Azure Machine Learning endpoints?](https://docs.microsoft.com/azure/machine-learning/concept-endpoints).\n",
    "- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F)\n",
    "- An Azure ML workspace with computer cluster - [Configure workspace](../../jobs/configuration.ipynb)\n",
    "- Installed Azure Machine Learning Python SDK v2 - [install instructions](../../README.md) - check the getting started section"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Connect to Azure Machine Learning Workspace\n",
    "The [workspace](https://docs.microsoft.com/azure/machine-learning/concept-workspace) is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section we will connect to the workspace in which the job will be run."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 Import the required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import required libraries\n",
    "from azure.ai.ml import MLClient\n",
    "from azure.ai.ml.entities import (\n",
    "    ManagedOnlineEndpoint,\n",
    "    ManagedOnlineDeployment,\n",
    "    Model,\n",
    "    Environment,\n",
    "    CodeConfiguration,\n",
    ")\n",
    "from azure.identity import DefaultAzureCredential\n",
    "from azure.ai.ml.constants import AssetTypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 Configure workspace details and get a handle to the workspace\n",
    "\n",
    "To connect to a workspace, we need identifier parameters - a subscription, resource group and workspace name. We will use these details in the `MLClient` from `azure.ai.ml` to get a handle to the required Azure Machine Learning workspace. We use the default [default azure authentication](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) for this tutorial. Check the [configuration notebook](../../jobs/configuration.ipynb) for more details on how to configure credentials and connect to a workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# enter details of your AML workspace\n",
    "subscription_id = \"<SUBSCRIPTION_ID>\"\n",
    "resource_group = \"<RESOURCE_GROUP>\"\n",
    "workspace = \"<AML_WORKSPACE_NAME>\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client = MLClient(\n",
    "    DefaultAzureCredential(), subscription_id, resource_group, workspace\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or if you are working in a compute instance in Azure Machine Learning:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client = MLClient.from_config(DefaultAzureCredential())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Create Online Endpoint\n",
    "\n",
    "Online endpoints are endpoints that are used for online (real-time) inferencing. Online endpoints contain deployments that are ready to receive data from clients and can send responses back in real time.\n",
    "\n",
    "To create an online endpoint we will use `ManagedOnlineEndpoint`. This class allows user to configure the following key aspects:\n",
    "\n",
    "- `name` - Name of the endpoint. Needs to be unique at the Azure region level\n",
    "- `auth_mode` - The authentication method for the endpoint. Key-based authentication and Azure ML token-based authentication are supported. Key-based authentication doesn't expire but Azure ML token-based authentication does. Possible values are `key` or `aml_token`.\n",
    "- `identity`- The managed identity configuration for accessing Azure resources for endpoint provisioning and inference.\n",
    "    - `type`- The type of managed identity. Azure Machine Learning supports `system_assigned` or `user_assigned identity`.\n",
    "    - `user_assigned_identities` - List (array) of fully qualified resource IDs of the user-assigned identities. This property is required is `identity.type` is user_assigned.\n",
    "- `description`- Description of the endpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 Configure the endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import string\n",
    "\n",
    "# Creating a unique endpoint name by including a random suffix\n",
    "allowed_chars = string.ascii_lowercase + string.digits\n",
    "endpoint_suffix = \"\".join(random.choice(allowed_chars) for x in range(5))\n",
    "endpoint_name = \"diabetes-endpoint-\" + endpoint_suffix\n",
    "\n",
    "print(f\"Endpoint name: {endpoint_name}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint = ManagedOnlineEndpoint(\n",
    "    name=endpoint_name,\n",
    "    description=\"An online endpoint serving an MLflow model for the diabetes classification task\",\n",
    "    auth_mode=\"key\",\n",
    "    tags={\"foo\": \"bar\"},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Create the endpoint\n",
    "Using the `MLClient` created earlier, we will now create the Endpoint in the workspace. This command will start the endpoint creation and return a confirmation response while the endpoint creation continues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.online_endpoints.begin_create_or_update(endpoint).result()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Create a blue deployment\n",
    "\n",
    "A deployment is a set of resources required for hosting the model that does the actual inferencing. We will create a deployment for our endpoint using the `ManagedOnlineDeployment` class. This class allows user to configure the following key aspects.\n",
    "\n",
    "- `name` - Name of the deployment.\n",
    "- `endpoint_name` - Name of the endpoint to create the deployment under.\n",
    "- `model` - The model to use for the deployment. This value can be either a reference to an existing versioned model in the workspace or an inline model specification.\n",
    "- `environment` - The environment where the model will run.\n",
    "- `code_configuration` - The scoring script used to serve the model.\n",
    "- `instance_type` - The VM size to use for the deployment. For the list of supported sizes, see [Managed online endpoints SKU list](https://docs.microsoft.com/azure/machine-learning/reference-managed-online-endpoints-vm-sku-list).\n",
    "- `instance_count` - The number of instances to use for the deployment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 Configure the deployment"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Registering the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "model_name = \"sklearn-diabetes\"\n",
    "\n",
    "model = Model(\n",
    "    name=model_name,\n",
    "    path=\"sklearn-diabetes/model\",\n",
    "    type=AssetTypes.MLFLOW_MODEL,\n",
    "    description=\"MLflow model for the diabetes classification problem\",\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Creating an environment to perform inference:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "environment = Environment(\n",
    "    conda_file=\"sklearn-diabetes/environment/conda.yaml\",\n",
    "    image=\"mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest\",\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Creating the deployment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "blue_deployment"
   },
   "outputs": [],
   "source": [
    "blue_deployment = ManagedOnlineDeployment(\n",
    "    name=\"blue\",\n",
    "    endpoint_name=endpoint_name,\n",
    "    model=model,\n",
    "    environment=environment,\n",
    "    code_configuration=CodeConfiguration(\n",
    "        code=\"sklearn-diabetes/src\", scoring_script=\"score.py\"\n",
    "    ),\n",
    "    instance_type=\"Standard_F4s_v2\",\n",
    "    instance_count=1,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 Create the deployment\n",
    "\n",
    "Using the `MLClient` created earlier, we will now create the deployment in the workspace. This command will start the deployment creation and return a confirmation response while the deployment creation continues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "ml_client"
   },
   "outputs": [],
   "source": [
    "ml_client.online_deployments.begin_create_or_update(blue_deployment).result()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "endpoint.traffic"
   },
   "outputs": [],
   "source": [
    "endpoint.traffic = {blue_deployment.name: 100}\n",
    "ml_client.begin_create_or_update(endpoint).result()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Test the deployment\n",
    "\n",
    "Using the `MLClient` created earlier, we will get a handle to the endpoint. The endpoint can be invoked using the invoke command with the following parameters:\n",
    "\n",
    "- `endpoint_name` - Name of the endpoint\n",
    "- `request_file` - File with request data\n",
    "- `deployment_name` - Name of the specific deployment to test in an endpoint\n",
    "\n",
    "We will send a sample request using a [sample-request-sklearn-custom.json](sample-request-sklearn-custom.json) file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.online_endpoints.invoke(\n",
    "    endpoint_name=endpoint_name,\n",
    "    deployment_name=\"blue\",\n",
    "    request_file=\"sample-request-sklearn.json\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Get endpoint details"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the details for online endpoint\n",
    "endpoint = ml_client.online_endpoints.get(name=endpoint_name)\n",
    "\n",
    "# existing traffic details\n",
    "print(endpoint.traffic)\n",
    "\n",
    "# Get the scoring URI\n",
    "print(endpoint.scoring_uri)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Delete the deployment and endopoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.online_endpoints.begin_delete(name=endpoint_name)"
   ]
  }
 ],
 "metadata": {
  "description": {
   "description": "Deploy an mlflow model to an online endpoint. This will be a no-code-deployment. It doesn't require scoring script and environment."
  },
  "interpreter": {
   "hash": "f6c16bbccc10ca03d07b4bd30ccd59ce8e25aee703e3c8d834bba9b524ce7685"
  },
  "kernel_info": {
   "name": "python310-sdkv2"
  },
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}