sdk/resources/compute/compute.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create Azure Machine Learning Compute\n",
    "\n",
    "**Requirements** - In order to benefit from this tutorial, you will need:\n",
    "- A basic understanding of Machine Learning\n",
    "- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F)\n",
    "- An Azure ML workspace with computer cluster - [Configure workspace](../../jobs/configuration.ipynb) \n",
    "- A python environment\n",
    "- Installed Azure Machine Learning Python SDK v2 - [install instructions](../../README.md) - check the getting started section\n",
    "\n",
    "**Learning Objectives** - By the end of this tutorial, you should be able to:\n",
    "- Create Azure Machine Learning Compute Instance from Python SDK\n",
    "- Create Azure Machine Learning Compute Cluster from Python SDK\n",
    "\n",
    "**Motivations** - Azure Machine Learning needs compute to run tasks. This tutorial will introduce you to create compute for machine learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Connect to Azure Machine Learning Workspace\n",
    "\n",
    "The [workspace](https://docs.microsoft.com/en-us/azure/machine-learning/concept-workspace) is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section we will connect to the workspace in which the compute will be created.\n",
    "\n",
    "## 1.1. Import the required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import required libraries\n",
    "from azure.ai.ml import MLClient\n",
    "from azure.ai.ml.entities import ComputeInstance, AmlCompute\n",
    "from azure.identity import DefaultAzureCredential"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2. Configure workspace details and get a handle to the workspace\n",
    "\n",
    "To connect to a workspace, we need identifier parameters - a subscription, resource group and workspace name. We will use these details in the `MLClient` from `azure.ai.ml` to get a handle to the required Azure Machine Learning workspace. We use the default [default azure authentication](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) for this tutorial. Check the [configuration notebook](../../jobs/configuration.ipynb) for more details on how to configure credentials and connect to a workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Enter details of your AML workspace\n",
    "subscription_id = \"<SUBSCRIPTION_ID>\"\n",
    "resource_group = \"<RESOURCE_GROUP>\"\n",
    "workspace = \"<AML_WORKSPACE_NAME>\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get a handle to the workspace\n",
    "ml_client = MLClient(\n",
    "    DefaultAzureCredential(), subscription_id, resource_group, workspace\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2 Create Compute Instance\n",
    "An Azure Machine Learning [compute instance](https://docs.microsoft.com/en-us/azure/machine-learning/concept-compute-instance) is a managed cloud-based workstation for data scientists.\n",
    "\n",
    "Compute instance can be created using the `ComputeInstance` class. This class accepts the following key parameters:\n",
    "- `name` - Name of the compute.\n",
    "- `description` - Description of the compute\n",
    "- `size` - The VM size to use for the compute instance. For more information, see [Supported VM series and sizes](https://docs.microsoft.com/en-us/azure/machine-learning/concept-compute-target#supported-vm-series-and-sizes). \n",
    "\n",
    "## 2.1 Create a minimal Compute Instance\n",
    "In this example we will create a compute instance with only the `name` specified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute Instances need to have a unique name across the region.\n",
    "# Here we create a unique name with current datetime\n",
    "import datetime\n",
    "\n",
    "ci_minimal_name = \"min-ci\" + datetime.datetime.now().strftime(\"%Y%m%d%H%M\")\n",
    "\n",
    "ci_minimal = ComputeInstance(name=ci_minimal_name)\n",
    "ml_client.begin_create_or_update(ci_minimal)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Create a Compute Instance of specific size\n",
    "In this example, we will specify the size of the VM. For more information, see [Supported VM series and sizes](https://docs.microsoft.com/en-us/azure/machine-learning/concept-compute-target#supported-vm-series-and-sizes). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute Instances need to have a unique name across the region.\n",
    "# Here we create a unique name with current datetime\n",
    "import datetime\n",
    "\n",
    "ci_basic_name = \"basic-ci\" + datetime.datetime.now().strftime(\"%Y%m%d%H%M\")\n",
    "ci_basic = ComputeInstance(name=ci_basic_name, size=\"STANDARD_DS3_v2\")\n",
    "ml_client.begin_create_or_update(ci_basic)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 Clean up Resources\n",
    "Clean up the Compute Instances to save costs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.compute.begin_delete(ci_minimal_name)\n",
    "ml_client.compute.begin_delete(ci_basic_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Create Compute CLuster\n",
    "Azure Machine Learning compute cluster is a managed-compute infrastructure that allows you to easily create a single or multi-node compute. \n",
    "\n",
    "A compute cluster can be created using the `AmlCompute` class. Some of the key parameters of this class are:\n",
    "- `name` - Required. Name of the compute.\t\t\n",
    "- `description` - Description of the compute.\t\t\n",
    "- `location` - The location for the compute. If omitted, defaults to the workspace location.\n",
    "- `size` - The VM size to use for the cluster. For more information, see [Supported VM series and sizes](https://docs.microsoft.com/en-us/azure/machine-learning/concept-compute-target#supported-vm-series-and-sizes).\n",
    "- `tier` - The VM priority tier to use for the cluster. The priority can be `dedicated` or `low_priority`. If not specified, a `dedicated` cluster will be created.\n",
    "- `min_instances` - The minimum number of nodes to use on the cluster. Setting the minimum number of nodes to 0 allows Azure ML to autoscale the cluster down to zero nodes when not in use. Any value larger than 0 will keep that number of nodes running, even if the cluster is not in use. Default is 0.\n",
    "- `max_instances` - The maximum number of nodes to use on the cluster. Default is 1.\n",
    "- `idle_time_before_scale_down` - Node idle time in seconds before scaling down the cluster. Default is 120 seconds\n",
    "\n",
    "## 3.1 Create a basic Compute Cluster\n",
    "In this example we will create a basic compute cluster with a minimal set of parameters.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cluster_basic = AmlCompute(\n",
    "    name=\"basic-example\",\n",
    "    type=\"amlcompute\",\n",
    "    size=\"STANDARD_DS3_v2\",\n",
    "    location=\"westus\",\n",
    "    min_instances=0,\n",
    "    max_instances=2,\n",
    "    idle_time_before_scale_down=120,\n",
    ")\n",
    "ml_client.begin_create_or_update(cluster_basic)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 Create a low priority cluster\n",
    "In this example we will create a low priority cluster instead of the default dedicated cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cluster_low_pri = AmlCompute(\n",
    "    name=\"low-pri-example\",\n",
    "    size=\"STANDARD_DS3_v2\",\n",
    "    min_instances=0,\n",
    "    max_instances=2,\n",
    "    idle_time_before_scale_down=120,\n",
    "    tier=\"low_priority\",\n",
    ")\n",
    "ml_client.begin_create_or_update(cluster_low_pri)"
   ]
  }
 ],
 "metadata": {
  "description": {
   "description": "Create compute in Azure ML workspace"
  },
  "interpreter": {
   "hash": "66962d4c952b5ba37638a017d6cc83bab37d76f69b13c17d86b9f71233a0aa71"
  },
  "kernelspec": {
   "display_name": "Python 3.8 - AzureML",
   "language": "python",
   "name": "python38-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}