how-to-use-azureml/deployment/deploy-to-cloud/model-register-and-deploy.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
        "\n",
        "Licensed under the MIT License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/deployment/deploy-to-cloud/model-register-and-deploy.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Register model and deploy as webservice in ACI\n",
        "\n",
        "Following this notebook, you will:\n",
        "\n",
        " - Learn how to register a model in your Azure Machine Learning Workspace.\n",
        " - Deploy your model as a web service in an Azure Container Instance."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Prerequisites\n",
        "\n",
        "If you are using an Azure Machine Learning Notebook VM, you are all set. Otherwise, make sure you go through the [configuration notebook](../../../configuration.ipynb) to install the Azure Machine Learning Python SDK and create a workspace."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import azureml.core\n",
        "\n",
        "\n",
        "# Check core SDK version number.\n",
        "print('SDK version:', azureml.core.VERSION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Initialize workspace\n",
        "\n",
        "Create a [Workspace](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.workspace%28class%29?view=azure-ml-py) object from your persisted configuration."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "create workspace"
        ]
      },
      "outputs": [],
      "source": [
        "from azureml.core import Workspace\n",
        "\n",
        "\n",
        "ws = Workspace.from_config()\n",
        "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep='\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create trained model\n",
        "\n",
        "For this example, we will train a small model on scikit-learn's [diabetes dataset](https://scikit-learn.org/stable/datasets/index.html#diabetes-dataset). "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import joblib\n",
        "\n",
        "from sklearn.datasets import load_diabetes\n",
        "from sklearn.linear_model import Ridge\n",
        "\n",
        "\n",
        "dataset_x, dataset_y = load_diabetes(return_X_y=True)\n",
        "\n",
        "model = Ridge().fit(dataset_x, dataset_y)\n",
        "\n",
        "joblib.dump(model, 'sklearn_regression_model.pkl')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Register input and output datasets\n",
        "\n",
        "Here, you will register the data used to create the model in your workspace."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "\n",
        "from azureml.core import Dataset\n",
        "\n",
        "\n",
        "np.savetxt('features.csv', dataset_x, delimiter=',')\n",
        "np.savetxt('labels.csv', dataset_y, delimiter=',')\n",
        "\n",
        "datastore = ws.get_default_datastore()\n",
        "datastore.upload_files(files=['./features.csv', './labels.csv'],\n",
        "                       target_path='sklearn_regression/',\n",
        "                       overwrite=True)\n",
        "\n",
        "input_dataset = Dataset.Tabular.from_delimited_files(path=[(datastore, 'sklearn_regression/features.csv')])\n",
        "output_dataset = Dataset.Tabular.from_delimited_files(path=[(datastore, 'sklearn_regression/labels.csv')])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Register model\n",
        "\n",
        "Register a file or folder as a model by calling [Model.register()](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.model.model?view=azure-ml-py#register-workspace--model-path--model-name--tags-none--properties-none--description-none--datasets-none--model-framework-none--model-framework-version-none--child-paths-none-).\n",
        "\n",
        "In addition to the content of the model file itself, your registered model will also store model metadata -- model description, tags, and framework information -- that will be useful when managing and deploying models in your workspace. Using tags, for instance, you can categorize your models and apply filters when listing models in your workspace. Also, marking this model with the scikit-learn framework will simplify deploying it as a web service, as we'll see later."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "register model from file",
          "sample-model-register"
        ]
      },
      "outputs": [],
      "source": [
        "import sklearn\n",
        "\n",
        "from azureml.core import Model\n",
        "from azureml.core.resource_configuration import ResourceConfiguration\n",
        "\n",
        "\n",
        "model = Model.register(workspace=ws,\n",
        "                       model_name='my-sklearn-model',                # Name of the registered model in your workspace.\n",
        "                       model_path='./sklearn_regression_model.pkl',  # Local file to upload and register as a model.\n",
        "                       model_framework=Model.Framework.SCIKITLEARN,  # Framework used to create the model.\n",
        "                       model_framework_version=sklearn.__version__,  # Version of scikit-learn used to create the model.\n",
        "                       sample_input_dataset=input_dataset,\n",
        "                       sample_output_dataset=output_dataset,\n",
        "                       resource_configuration=ResourceConfiguration(cpu=1, memory_in_gb=0.5),\n",
        "                       description='Ridge regression model to predict diabetes progression.',\n",
        "                       tags={'area': 'diabetes', 'type': 'regression'})\n",
        "\n",
        "print('Name:', model.name)\n",
        "print('Version:', model.version)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Deploy model\n",
        "\n",
        "Deploy your model as a web service using [Model.deploy()](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.model.model?view=azure-ml-py#deploy-workspace--name--models--inference-config--deployment-config-none--deployment-target-none-). Web services take one or more models, load them in an environment, and run them on one of several supported deployment targets. For more information on all your options when deploying models, see the [next steps](#Next-steps) section at the end of this notebook.\n",
        "\n",
        "For this example, we will deploy your scikit-learn model to an Azure Container Instance (ACI)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Use a default environment (for supported models)\n",
        "\n",
        "The Azure Machine Learning service provides a default environment for supported model frameworks, including scikit-learn, based on the metadata you provided when registering your model. This is the easiest way to deploy your model.\n",
        "\n",
        "Even when you deploy your model to ACI with a default environment you can still customize the deploy configuration (i.e. the number of cores and amount of memory made available for the deployment) using the [AciWebservice.deploy_configuration()](https://docs.microsoft.com/python/api/azureml-core/azureml.core.webservice.aci.aciwebservice#deploy-configuration-cpu-cores-none--memory-gb-none--tags-none--properties-none--description-none--location-none--auth-enabled-none--ssl-enabled-none--enable-app-insights-none--ssl-cert-pem-file-none--ssl-key-pem-file-none--ssl-cname-none--dns-name-label-none--). Look at the \"Use a custom environment\" section of this notebook for more information on deploy configuration.\n",
        "\n",
        "**Note**: This step can take several minutes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "service_name = 'my-sklearn-service'\n",
        "\n",
        "service = Model.deploy(ws, service_name, [model], overwrite=True)\n",
        "service.wait_for_deployment(show_output=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "After your model is deployed, perform a call to the web service using [service.run()](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.webservice%28class%29?view=azure-ml-py#run-input-)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "\n",
        "input_payload = json.dumps({\n",
        "    'data': dataset_x[0:2].tolist(),\n",
        "    'method': 'predict'  # If you have a classification model, you can get probabilities by changing this to 'predict_proba'.\n",
        "})\n",
        "\n",
        "output = service.run(input_payload)\n",
        "\n",
        "print(output)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "When you are finished testing your service, clean up the deployment with [service.delete()](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.webservice%28class%29?view=azure-ml-py#delete--)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "service.delete()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Use a custom environment\n",
        "\n",
        "If you want more control over how your model is run, if it uses another framework, or if it has special runtime requirements, you can instead specify your own environment and scoring method. Custom environments can be used for any model you want to deploy.\n",
        "\n",
        "Specify the model's runtime environment by creating an [Environment](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.environment%28class%29?view=azure-ml-py) object and providing the [CondaDependencies](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.conda_dependencies.condadependencies?view=azure-ml-py) needed by your model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core import Environment\n",
        "from azureml.core.conda_dependencies import CondaDependencies\n",
        "\n",
        "\n",
        "environment = Environment('my-sklearn-environment')\n",
        "environment.python.conda_dependencies = CondaDependencies.create(pip_packages=[\n",
        "    'azureml-defaults',\n",
        "    'inference-schema[numpy-support]',\n",
        "    'joblib',\n",
        "    'numpy',\n",
        "    'scikit-learn=={}'.format(sklearn.__version__)\n",
        "])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "When using a custom environment, you must also provide Python code for initializing and running your model. An example script is included with this notebook."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "with open('score.py') as f:\n",
        "    print(f.read())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Deploy your model in the custom environment by providing an [InferenceConfig](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.model.inferenceconfig?view=azure-ml-py) object to [Model.deploy()](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.model.model?view=azure-ml-py#deploy-workspace--name--models--inference-config--deployment-config-none--deployment-target-none-). In this case we are also using the [AciWebservice.deploy_configuration()](https://docs.microsoft.com/python/api/azureml-core/azureml.core.webservice.aci.aciwebservice#deploy-configuration-cpu-cores-none--memory-gb-none--tags-none--properties-none--description-none--location-none--auth-enabled-none--ssl-enabled-none--enable-app-insights-none--ssl-cert-pem-file-none--ssl-key-pem-file-none--ssl-cname-none--dns-name-label-none--) method to generate a custom deploy configuration.\n",
        "\n",
        "**Note**: This step can take several minutes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "azuremlexception-remarks-sample",
          "sample-aciwebservice-deploy-config"
        ]
      },
      "outputs": [],
      "source": [
        "from azureml.core.model import InferenceConfig\n",
        "from azureml.core.webservice import AciWebservice\n",
        "\n",
        "\n",
        "service_name = 'my-custom-env-service'\n",
        "\n",
        "inference_config = InferenceConfig(entry_script='score.py', environment=environment)\n",
        "aci_config = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1)\n",
        "\n",
        "service = Model.deploy(workspace=ws,\n",
        "                       name=service_name,\n",
        "                       models=[model],\n",
        "                       inference_config=inference_config,\n",
        "                       deployment_config=aci_config,\n",
        "                       overwrite=True)\n",
        "service.wait_for_deployment(show_output=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "After your model is deployed, make a call to the web service using [service.run()](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.webservice%28class%29?view=azure-ml-py#run-input-)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "input_payload = json.dumps({\n",
        "    'data': dataset_x[0:2].tolist()\n",
        "})\n",
        "\n",
        "output = service.run(input_payload)\n",
        "\n",
        "print(output)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "When you are finished testing your service, clean up the deployment with [service.delete()](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.webservice%28class%29?view=azure-ml-py#delete--)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "service.delete()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Model Profiling\n",
        "\n",
        "Profile your model to understand how much CPU and memory the service, created as a result of its deployment, will need. Profiling returns information such as CPU usage, memory usage, and response latency. It also provides a CPU and memory recommendation based on the resource usage. You can profile your model (or more precisely the service built based on your model) on any CPU and/or memory combination where 0.1 <= CPU <= 3.5 and 0.1GB <= memory <= 15GB. If you do not provide a CPU and/or memory requirement, we will test it on the default configuration of 3.5 CPU and 15GB memory.\n",
        "\n",
        "In order to profile your model you will need:\n",
        "- a registered model\n",
        "- an entry script\n",
        "- an inference configuration\n",
        "- a single column tabular dataset, where each row contains a string representing sample request data sent to the service.\n",
        "\n",
        "Please, note that profiling is a long running operation and can take up to 25 minutes depending on the size of the dataset.\n",
        "\n",
        "At this point we only support profiling of services that expect their request data to be a string, for example: string serialized json, text, string serialized image, etc. The content of each row of the dataset (string) will be put into the body of the HTTP request and sent to the service encapsulating the model for scoring.\n",
        "\n",
        "Below is an example of how you can construct an input dataset to profile a service which expects its incoming requests to contain serialized json. In this case we created a dataset based one hundred instances of the same request data. In real world scenarios however, we suggest that you use larger datasets with various inputs, especially if your model resource usage/behavior is input dependent."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "You may want to register datasets using the register() method to your workspace so they can be shared with others, reused and referred to by name in your script.\n",
        "You can try get the dataset first to see if it's already registered."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core import Datastore\n",
        "from azureml.core.dataset import Dataset\n",
        "from azureml.data import dataset_type_definitions\n",
        "\n",
        "dataset_name='diabetes_sample_request_data'\n",
        "\n",
        "dataset_registered = False\n",
        "try:\n",
        "    sample_request_data = Dataset.get_by_name(workspace = ws, name = dataset_name)\n",
        "    dataset_registered = True\n",
        "except:\n",
        "    print(\"The dataset {} is not registered in workspace yet.\".format(dataset_name))\n",
        "\n",
        "if not dataset_registered:\n",
        "    # create a string that can be utf-8 encoded and\n",
        "    # put in the body of the request\n",
        "    serialized_input_json = json.dumps({\n",
        "        'data': [\n",
        "            [ 0.03807591,  0.05068012,  0.06169621, 0.02187235, -0.0442235,\n",
        "            -0.03482076, -0.04340085, -0.00259226, 0.01990842, -0.01764613]\n",
        "        ]\n",
        "    })\n",
        "    dataset_content = []\n",
        "    for i in range(100):\n",
        "        dataset_content.append(serialized_input_json)\n",
        "    dataset_content = '\\n'.join(dataset_content)\n",
        "    file_name = \"{}.txt\".format(dataset_name)\n",
        "    f = open(file_name, 'w')\n",
        "    f.write(dataset_content)\n",
        "    f.close()\n",
        "\n",
        "    # upload the txt file created above to the Datastore and create a dataset from it\n",
        "    data_store = Datastore.get_default(ws)\n",
        "    data_store.upload_files(['./' + file_name], target_path='sample_request_data')\n",
        "    datastore_path = [(data_store, 'sample_request_data' +'/' + file_name)]\n",
        "    sample_request_data = Dataset.Tabular.from_delimited_files(\n",
        "        datastore_path,\n",
        "        separator='\\n',\n",
        "        infer_column_types=True,\n",
        "        header=dataset_type_definitions.PromoteHeadersBehavior.NO_HEADERS)\n",
        "    sample_request_data = sample_request_data.register(workspace=ws,\n",
        "                                                    name=dataset_name,\n",
        "                                                    create_new_version=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now that we have an input dataset we are ready to go ahead with profiling. In this case we are testing the previously introduced sklearn regression model on 1 CPU and 0.5 GB memory. The memory usage and recommendation presented in the result is measured in Gigabytes. The CPU usage and recommendation is measured in CPU cores."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from datetime import datetime\n",
        "\n",
        "\n",
        "environment = Environment('my-sklearn-environment')\n",
        "environment.python.conda_dependencies = CondaDependencies.create(pip_packages=[\n",
        "    'azureml-defaults',\n",
        "    'inference-schema[numpy-support]',\n",
        "    'joblib',\n",
        "    'numpy',\n",
        "    'scikit-learn=={}'.format(sklearn.__version__)\n",
        "])\n",
        "inference_config = InferenceConfig(entry_script='score.py', environment=environment)\n",
        "# if cpu and memory_in_gb parameters are not provided\n",
        "# the model will be profiled on default configuration of\n",
        "# 3.5CPU and 15GB memory\n",
        "profile = Model.profile(ws,\n",
        "            'rgrsn-%s' % datetime.now().strftime('%m%d%Y-%H%M%S'),\n",
        "            [model],\n",
        "            inference_config,\n",
        "            input_dataset=sample_request_data,\n",
        "            cpu=1.0,\n",
        "            memory_in_gb=0.5)\n",
        "\n",
        "# profiling is a long running operation and may take up to 25 min\n",
        "profile.wait_for_completion(True)\n",
        "details = profile.get_details()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Model packaging\n",
        "\n",
        "If you want to build a Docker image that encapsulates your model and its dependencies, you can use the model packaging option. The output image will be pushed to your workspace's ACR.\n",
        "\n",
        "You must include an Environment object in your inference configuration to use `Model.package()`.\n",
        "\n",
        "```python\n",
        "package = Model.package(ws, [model], inference_config)\n",
        "package.wait_for_creation(show_output=True)  # Or show_output=False to hide the Docker build logs.\n",
        "package.pull()\n",
        "```\n",
        "\n",
        "Instead of a fully-built image, you can also generate a Dockerfile and download all the assets needed to build an image on top of your Environment.\n",
        "\n",
        "```python\n",
        "package = Model.package(ws, [model], inference_config, generate_dockerfile=True)\n",
        "package.wait_for_creation(show_output=True)\n",
        "package.save(\"./local_context_dir\")\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Next steps\n",
        "\n",
        " - To run a production-ready web service, see the [notebook on deployment to Azure Kubernetes Service](../production-deploy-to-aks/production-deploy-to-aks.ipynb).\n",
        " - To run a local web service, see the [notebook on deployment to a local Docker container](../deploy-to-local/register-model-deploy-local.ipynb).\n",
        " - For more information on datasets, see the [notebook on training with datasets](../../work-with-data/datasets-tutorial/train-with-datasets/train-with-datasets.ipynb).\n",
        " - For more information on environments, see the [notebook on using environments](../../training/using-environments/using-environments.ipynb).\n",
        " - For information on all the available deployment targets, see [&ldquo;How and where to deploy models&rdquo;](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-deploy-and-where#choose-a-compute-target)."
      ]
    }
  ],
  "metadata": {
    "authors": [
      {
        "name": "vaidyas"
      }
    ],
    "category": "deployment",
    "compute": [
      "None"
    ],
    "datasets": [
      "Diabetes"
    ],
    "deployment": [
      "Azure Container Instance"
    ],
    "exclude_from_index": false,
    "framework": [
      "Scikit-learn"
    ],
    "friendly_name": "Register model and deploy as webservice",
    "index_order": 3,
    "kernelspec": {
      "display_name": "Python 3.6",
      "language": "python",
      "name": "python36"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.0"
    },
    "star_tag": [
      "featured"
    ],
    "tags": [
      "None"
    ],
    "task": "Deploy a model with Azure Machine Learning"
  },
  "nbformat": 4,
  "nbformat_minor": 2
}