diff --git a/06_CodeGeneration/00_code_generatation_w_bedrock.ipynb b/06_CodeGeneration/00_code_generatation_w_bedrock.ipynb new file mode 100644 index 00000000..2a0cf1e6 --- /dev/null +++ b/06_CodeGeneration/00_code_generatation_w_bedrock.ipynb @@ -0,0 +1,960 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "dc40c48b-0c95-4757-a067-563cfccd51a5", + "metadata": { + "tags": [] + }, + "source": [ + "# Invoke Bedrock model for code generation\n", + "\n", + "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio*" + ] + }, + { + "cell_type": "markdown", + "id": "c9a413e2-3c34-4073-9000-d8556537bb6a", + "metadata": {}, + "source": [ + "## Introduction\n", + "\n", + "In this notebook we show you how to use a LLM to generate code based on the text prompt.\n", + "\n", + "We will use Bedrock's Claude v2 using the Boto3 API. \n", + "\n", + "The prompt used in this example is called a zero-shot prompt because we are not providing any examples of text other than the prompt.\n", + "\n", + "**Note:** *This notebook can be run within or outside of AWS environment.*\n", + "\n", + "#### Context\n", + "To demonstrate the code generation capability of Amazon Bedrock, we will explore the use of Boto3 client to communicate with Amazon Bedrock API. We will demonstrate different configurations available as well as how simple input can lead to desired outputs.\n", + "\n", + "#### Pattern\n", + "We will simply provide the Amazon Bedrock API with an input consisting of a task, an instruction and an input for the model under the hood to generate an output without providing any additional example. The purpose here is to demonstrate how the powerful LLMs easily understand the task at hand and generate compelling outputs.\n", + "\n", + "![](./images/bedrock-code-gen.png)\n", + "\n", + "#### Use case\n", + "To demonstrate the generation capability of models in Amazon Bedrock, let's take the use case of code generation.\n", + "\n", + "#### Persona\n", + "\n", + "You are Moe, a Data Analyst, at AnyCompany. The company wants to understand its sales performance for different products for different products over the past year. You have been provided a dataset named sales.csv. The dataset contains the following columns:\n", + "\n", + "- Date (YYYY-MM-DD) format\n", + "- Product_ID (unique identifer for each product)\n", + "- Price (price at which each product was sold)\n", + "\n", + "#### Implementation\n", + "To fulfill this use case, in this notebook we will show how to generate code for a given prompt. We will use the Anthropic Claude v2 using the Amazon Bedrock API with Boto3 client. " + ] + }, + { + "cell_type": "markdown", + "id": "64baae27-2660-4a1e-b2e5-3de49d069362", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "Before running the rest of this notebook, you'll need to run the cells below to (ensure necessary libraries are installed and) connect to Bedrock.\n", + "\n", + "For more details on how the setup works and ⚠️ **whether you might need to make any changes**, refer to the [Bedrock boto3 setup notebook](../00_Intro/bedrock_boto3_setup.ipynb) notebook." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "38b791ad-e6c5-4da5-96af-5c356a36e19d", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Make sure you ran `download-dependencies.sh` from the root of the repository first!\n", + "%pip install --no-build-isolation --force-reinstall \\\n", + " ../dependencies/awscli-*-py3-none-any.whl \\\n", + " ../dependencies/boto3-*-py3-none-any.whl \\\n", + " ../dependencies/botocore-*-py3-none-any.whl\n", + "\n", + "%pip install --quiet langchain==0.0.249" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7ea26558", + "metadata": {}, + "outputs": [], + "source": [ + "# To execute the generated code in this notebook\n", + "%pip install matplotlib" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "776fd083", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "import json\n", + "import os\n", + "import sys\n", + "\n", + "import boto3\n", + "\n", + "module_path = \"..\"\n", + "sys.path.append(os.path.abspath(module_path))\n", + "from utils import bedrock, print_ww\n", + "\n", + "\n", + "# ---- ⚠️ Un-comment and edit the below lines as needed for your AWS setup ⚠️ ----\n", + "\n", + "# os.environ[\"AWS_DEFAULT_REGION\"] = \"\" # E.g. \"us-east-1\"\n", + "# os.environ[\"AWS_PROFILE\"] = \"\"\n", + "# os.environ[\"BEDROCK_ASSUME_ROLE\"] = \"\" # E.g. \"arn:aws:...\"\n", + "# os.environ[\"BEDROCK_ENDPOINT_URL\"] = \"\" # E.g. \"https://...\"\n", + "\n", + "\n", + "boto3_bedrock = bedrock.get_bedrock_client(\n", + " assumed_role=os.environ.get(\"BEDROCK_ASSUME_ROLE\", None),\n", + " endpoint_url=os.environ.get(\"BEDROCK_ENDPOINT_URL\", None),\n", + " region=os.environ.get(\"AWS_DEFAULT_REGION\", None),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "4f634211-3de1-4390-8c3f-367af5554c39", + "metadata": {}, + "source": [ + "## Code Generation\n", + "\n", + "Following on the use case explained above, let's prepare an input for the Amazon Bedrock service to generate python program for our use-case." + ] + }, + { + "cell_type": "markdown", + "id": "e7656be8", + "metadata": {}, + "source": [ + "#### Lab setup - create sample sales.csv data for this lab.\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "89a0ad24", + "metadata": {}, + "outputs": [], + "source": [ + "# create sales.csv file\n", + "import csv\n", + "\n", + "data = [\n", + " [\"date\", \"product_id\", \"price\", \"units_sold\"],\n", + " [\"2023-01-01\", \"P001\", 50, 20],\n", + " [\"2023-01-02\", \"P002\", 60, 15],\n", + " [\"2023-01-03\", \"P001\", 50, 18],\n", + " [\"2023-01-04\", \"P003\", 70, 30],\n", + " [\"2023-01-05\", \"P001\", 50, 25],\n", + " [\"2023-01-06\", \"P002\", 60, 22],\n", + " [\"2023-01-07\", \"P003\", 70, 24],\n", + " [\"2023-01-08\", \"P001\", 50, 28],\n", + " [\"2023-01-09\", \"P002\", 60, 17],\n", + " [\"2023-01-10\", \"P003\", 70, 29],\n", + " [\"2023-02-11\", \"P001\", 50, 23],\n", + " [\"2023-02-12\", \"P002\", 60, 19],\n", + " [\"2023-02-13\", \"P001\", 50, 21],\n", + " [\"2023-02-14\", \"P003\", 70, 31],\n", + " [\"2023-03-15\", \"P001\", 50, 26],\n", + " [\"2023-03-16\", \"P002\", 60, 20],\n", + " [\"2023-03-17\", \"P003\", 70, 33],\n", + " [\"2023-04-18\", \"P001\", 50, 27],\n", + " [\"2023-04-19\", \"P002\", 60, 18],\n", + " [\"2023-04-20\", \"P003\", 70, 32],\n", + " [\"2023-04-21\", \"P001\", 50, 22],\n", + " [\"2023-04-22\", \"P002\", 60, 16],\n", + " [\"2023-04-23\", \"P003\", 70, 34],\n", + " [\"2023-05-24\", \"P001\", 50, 24],\n", + " [\"2023-05-25\", \"P002\", 60, 21]\n", + "]\n", + "\n", + "# Write data to sales.csv\n", + "with open('sales.csv', 'w', newline='') as csvfile:\n", + " writer = csv.writer(csvfile)\n", + " writer.writerows(data)\n", + "\n", + "print(\"sales.csv has been created!\")" + ] + }, + { + "cell_type": "markdown", + "id": "d68e8af6", + "metadata": {}, + "source": [ + "#### Analyzing sales with Amazon Bedrock generated Python program" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "id": "45ee2bae-6415-4dba-af98-a19028305c98", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Create the prompt\n", + "# Analyzing sales\n", + "\n", + "prompt_data = \"\"\"\n", + "Human: You have a CSV, sales.csv, with columns:\n", + "- date (YYYY-MM-DD)\n", + "- product_id\n", + "- price\n", + "- units_sold\n", + "\n", + "Create a python program to analyze the sales data from a CSV file. The program should be able to read the data, and determine below:\n", + "\n", + "- Total revenue for the year\n", + "- The product with the highest revenue\n", + "- The date with the highest revenue\n", + "- Visualize monthly sales using a bar chart\n", + "\n", + "Ensure the code is syntactically correct, bug-free, optimized, not span multiple lines unnessarily, and prefer to use standard libraries. Return only python code without any surrounding text, explanation or context.\n", + "Assistant:\n", + "\"\"\"" + ] + }, + { + "cell_type": "markdown", + "id": "cc9784e5-5e9d-472d-8ef1-34108ee4968b", + "metadata": {}, + "source": [ + "Let's start by using the Anthropic Claude V2 model." + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "id": "8af670eb-ad02-40df-a19c-3ed835fac8d9", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Claude - Body Syntex\n", + "body = json.dumps({\n", + " \"prompt\": prompt_data,\n", + " \"max_tokens_to_sample\":4096,\n", + " \"temperature\":0.5,\n", + " \"top_k\":250,\n", + " \"top_p\":0.5,\n", + " \"stop_sequences\": [\"\\n\\nHuman:\"]\n", + " }) " + ] + }, + { + "cell_type": "markdown", + "id": "088cf6bf-dd73-4710-a0cc-6c11d220c431", + "metadata": {}, + "source": [ + "#### Invoke the Anthropic Claude v2 model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "016a118a", + "metadata": {}, + "outputs": [], + "source": [ + "modelId = 'anthropic.claude-v2' # change this to use a different version from the model provider\n", + "accept = 'application/json'\n", + "contentType = 'application/json'\n", + "\n", + "response = boto3_bedrock.invoke_model(body=body, modelId=modelId, accept=accept, contentType=contentType)\n", + "response_body = json.loads(response.get('body').read())\n", + "\n", + "print_ww(response_body.get('completion'))" + ] + }, + { + "cell_type": "markdown", + "id": "ddddd1ec", + "metadata": {}, + "source": [ + "#### (Optional) Execute the Bedrock generated code for validation. Go to text editor to copy the generated code as printed output can be trucncated. Replce the code in below cell." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "77d9b428", + "metadata": {}, + "outputs": [], + "source": [ + "# Sample Generated Python Code ( Generated with Amazon Bedrock in previous step)\n", + "\n", + "import csv\n", + "from collections import defaultdict\n", + "import matplotlib.pyplot as plt\n", + "\n", + "revenue = 0\n", + "monthly_revenue = defaultdict(int)\n", + "product_revenue = defaultdict(int)\n", + "max_revenue = 0\n", + "max_revenue_date = ''\n", + "max_revenue_product = ''\n", + "\n", + "with open('sales.csv') as f:\n", + " reader = csv.reader(f)\n", + " next(reader)\n", + " for row in reader:\n", + " date = row[0]\n", + " product = row[1]\n", + " price = float(row[2])\n", + " units = int(row[3])\n", + "\n", + " revenue += price * units\n", + " product_revenue[product] += price * units\n", + " monthly_revenue[date[:7]] += price * units\n", + "\n", + " if revenue > max_revenue:\n", + " max_revenue = revenue\n", + " max_revenue_date = date\n", + " max_revenue_product = product\n", + "\n", + "months = list(monthly_revenue.keys())\n", + "values = list(monthly_revenue.values())\n", + "\n", + "plt.bar(months, values)\n", + "plt.xlabel('Month')\n", + "plt.ylabel('Revenue')\n", + "plt.title('Monthly Revenue')\n", + "plt.show()\n", + "\n", + "print('Total Revenue:', revenue)\n", + "print('Product with max revenue:', max_revenue_product)\n", + "print('Date with max revenue:', max_revenue_date)" + ] + }, + { + "cell_type": "markdown", + "id": "64b08b3b", + "metadata": {}, + "source": [ + "## Conclusion\n", + "You have now experimented with using `boto3` SDK which provides a vanilla exposure to Amazon Bedrock API. Using this API you generate a python program to analyze and visualize given sales data.\n", + "\n", + "### Take aways\n", + "- Adapt this notebook to experiment with different models available through Amazon Bedrock such as Amazon Titan and AI21 Labs Jurassic models.\n", + "\n", + "\n", + "## Thank You" + ] + } + ], + "metadata": { + "availableInstances": [ + { + "_defaultOrder": 0, + "_isFastLaunch": true, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 4, + "name": "ml.t3.medium", + "vcpuNum": 2 + }, + { + "_defaultOrder": 1, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.t3.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 2, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.t3.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 3, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.t3.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 4, + "_isFastLaunch": true, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.m5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 5, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.m5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 6, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.m5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 7, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.m5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 8, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.m5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 9, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.m5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 10, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.m5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 11, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.m5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 12, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.m5d.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 13, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.m5d.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 14, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.m5d.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 15, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.m5d.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 16, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.m5d.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 17, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.m5d.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 18, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.m5d.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 19, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.m5d.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 20, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": true, + "memoryGiB": 0, + "name": "ml.geospatial.interactive", + "supportedImageNames": [ + "sagemaker-geospatial-v1-0" + ], + "vcpuNum": 0 + }, + { + "_defaultOrder": 21, + "_isFastLaunch": true, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 4, + "name": "ml.c5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 22, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.c5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 23, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.c5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 24, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.c5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 25, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 72, + "name": "ml.c5.9xlarge", + "vcpuNum": 36 + }, + { + "_defaultOrder": 26, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 96, + "name": "ml.c5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 27, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 144, + "name": "ml.c5.18xlarge", + "vcpuNum": 72 + }, + { + "_defaultOrder": 28, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.c5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 29, + "_isFastLaunch": true, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.g4dn.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 30, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.g4dn.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 31, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.g4dn.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 32, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.g4dn.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 33, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.g4dn.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 34, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.g4dn.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 35, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 61, + "name": "ml.p3.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 36, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 244, + "name": "ml.p3.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 37, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 488, + "name": "ml.p3.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 38, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.p3dn.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 39, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.r5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 40, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.r5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 41, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.r5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 42, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.r5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 43, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.r5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 44, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.r5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 45, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 512, + "name": "ml.r5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 46, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.r5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 47, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.g5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 48, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.g5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 49, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.g5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 50, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.g5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 51, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.g5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 52, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.g5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 53, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.g5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 54, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.g5.48xlarge", + "vcpuNum": 192 + }, + { + "_defaultOrder": 55, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 1152, + "name": "ml.p4d.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 56, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 1152, + "name": "ml.p4de.24xlarge", + "vcpuNum": 96 + } + ], + "instance_type": "ml.t3.medium", + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/06_CodeGeneration/01_sql_query_generate_w_bedrock.ipynb b/06_CodeGeneration/01_sql_query_generate_w_bedrock.ipynb new file mode 100644 index 00000000..7edf9a99 --- /dev/null +++ b/06_CodeGeneration/01_sql_query_generate_w_bedrock.ipynb @@ -0,0 +1,931 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "dc40c48b-0c95-4757-a067-563cfccd51a5", + "metadata": { + "tags": [] + }, + "source": [ + "# Invoke Bedrock model for SQL Query Generation\n", + "\n", + "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio*" + ] + }, + { + "cell_type": "markdown", + "id": "c9a413e2-3c34-4073-9000-d8556537bb6a", + "metadata": {}, + "source": [ + "## Introduction\n", + "\n", + "In this notebook we show you how to use a LLM to generate SQL Query to analyze Sales data.\n", + "\n", + "We will use Bedrock's Claude V2 model using the Boto3 API. \n", + "\n", + "The prompt used in this example is called a zero-shot prompt because we are not providing any examples of text other than the prompt.\n", + "\n", + "**Note:** *This notebook can be run within or outside of AWS environment.*\n", + "\n", + "#### Context\n", + "To demonstrate the SQL code generation capability of Amazon Bedrock, we will explore the use of Boto3 client to communicate with Amazon Bedrock API. We will demonstrate different configurations available as well as how simple input can lead to desired outputs.\n", + "\n", + "#### Pattern\n", + "We will simply provide the Amazon Bedrock API with an input consisting of a task, an instruction and an input for the model under the hood to generate an output without providing any additional example. The purpose here is to demonstrate how the powerful LLMs easily understand the task at hand and generate compelling outputs.\n", + "\n", + "![](./images/sql-query-generation.png)\n", + "\n", + "#### Use case\n", + "Let's take the use case to generate SQL queries to analyze sales data, focusing on trends, top products and average sales.\n", + "\n", + "#### Persona\n", + "Maya is a business analyst, at AnyCompany primarily focusing on sales and inventory data. She is transitioning from Speadsheet analysis to data-driven analysis and want to use SQL to fetch specific data points effectively. She wants to use LLMs to generate SQL queries for her analysis. \n", + "\n", + "#### Implementation\n", + "To fulfill this use case, in this notebook we will show how to generate SQL queries. We will use the Anthropic Claude v2 model using the Amazon Bedrock API with Boto3 client. " + ] + }, + { + "cell_type": "markdown", + "id": "64baae27-2660-4a1e-b2e5-3de49d069362", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "Before running the rest of this notebook, you'll need to run the cells below to (ensure necessary libraries are installed and) connect to Bedrock.\n", + "\n", + "For more details on how the setup works and ⚠️ **whether you might need to make any changes**, refer to the [Bedrock boto3 setup notebook](../00_Intro/bedrock_boto3_setup.ipynb) notebook." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "38b791ad-e6c5-4da5-96af-5c356a36e19d", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Make sure you ran `download-dependencies.sh` from the root of the repository first!\n", + "%pip install --no-build-isolation --force-reinstall \\\n", + " ../dependencies/awscli-*-py3-none-any.whl \\\n", + " ../dependencies/boto3-*-py3-none-any.whl \\\n", + " ../dependencies/botocore-*-py3-none-any.whl\n", + "\n", + "%pip install --quiet langchain==0.0.249" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "776fd083", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "import json\n", + "import os\n", + "import sys\n", + "\n", + "import boto3\n", + "\n", + "module_path = \"..\"\n", + "sys.path.append(os.path.abspath(module_path))\n", + "from utils import bedrock, print_ww\n", + "\n", + "\n", + "# ---- ⚠️ Un-comment and edit the below lines as needed for your AWS setup ⚠️ ----\n", + "\n", + "# os.environ[\"AWS_DEFAULT_REGION\"] = \"\" # E.g. \"us-east-1\"\n", + "# os.environ[\"AWS_PROFILE\"] = \"\"\n", + "# os.environ[\"BEDROCK_ASSUME_ROLE\"] = \"\" # E.g. \"arn:aws:...\"\n", + "# os.environ[\"BEDROCK_ENDPOINT_URL\"] = \"\" # E.g. \"https://...\"\n", + "\n", + "\n", + "boto3_bedrock = bedrock.get_bedrock_client(\n", + " assumed_role=os.environ.get(\"BEDROCK_ASSUME_ROLE\", None),\n", + " endpoint_url=os.environ.get(\"BEDROCK_ENDPOINT_URL\", None),\n", + " region=os.environ.get(\"AWS_DEFAULT_REGION\", None),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "4f634211-3de1-4390-8c3f-367af5554c39", + "metadata": {}, + "source": [ + "## Generate SQL Query\n", + "\n", + "Following on the use case explained above, let's prepare an input for the Amazon Bedrock service to generate SQL query." + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "id": "45ee2bae-6415-4dba-af98-a19028305c98", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# create the prompt to generate SQL query\n", + "prompt_data = \"\"\"\n", + "Command: Human: AnyCompany has a database with a table named sales_data containing sales records. The table has following columns:\n", + "- date (YYYY-MM-DD)\n", + "- product_id\n", + "- price\n", + "- units_sold\n", + "\n", + "Can you generate SQL queries for below: \n", + "- Identify the top 5 best selling products by total sales for the year 2023\n", + "- Calculate the monthly average sales for the year 2023\n", + "\n", + "Assistant:\n", + "\"\"\"\n" + ] + }, + { + "cell_type": "markdown", + "id": "cc9784e5-5e9d-472d-8ef1-34108ee4968b", + "metadata": {}, + "source": [ + "Let's start by using the Anthorpic Claude v2 model. " + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "id": "8af670eb-ad02-40df-a19c-3ed835fac8d9", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Claude - Body Syntex\n", + "body = json.dumps({\n", + " \"prompt\": prompt_data,\n", + " \"max_tokens_to_sample\":4096,\n", + " \"temperature\":0.5,\n", + " \"top_k\":250,\n", + " \"top_p\":0.5,\n", + " \"stop_sequences\": [\"\\n\\nHuman:\"]\n", + " }) " + ] + }, + { + "cell_type": "markdown", + "id": "088cf6bf-dd73-4710-a0cc-6c11d220c431", + "metadata": {}, + "source": [ + "#### Invoke the Bedrock's Claude Large Large language model" + ] + }, + { + "cell_type": "markdown", + "id": "379498f2", + "metadata": {}, + "source": [ + "First, we explore how the model generates an output based on the prompt created earlier.\n", + "\n", + "##### Complete Output Generation" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ecaceef1-0f7f-4ae5-8007-ff7c25335251", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "modelId = 'anthropic.claude-v2' # change this to use a different version from the model provider\n", + "accept = 'application/json'\n", + "contentType = 'application/json'\n", + "\n", + "response = boto3_bedrock.invoke_model(body=body, modelId=modelId, accept=accept, contentType=contentType)\n", + "response_body = json.loads(response.get('body').read())\n", + "\n", + "print_ww(response_body.get('completion'))" + ] + }, + { + "cell_type": "markdown", + "id": "078b9db4", + "metadata": {}, + "source": [ + "### Advanced Example\n", + "#### Understanding Hospital's Patient Management System through SQL" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "id": "d439b90c", + "metadata": {}, + "outputs": [], + "source": [ + "# create the prompt\n", + "prompt_sql_data = \"\"\"Command: You're provided with a database schema representing any hospital's patient management system.\n", + "The system holds records about patients, their prescriptions, doctors, and the medications prescribed.\n", + "\n", + "Here's the schema:\n", + "\n", + "```sql\n", + "CREATE TABLE Patients (\n", + " PatientID int,\n", + " FirstName varchar(50),\n", + " LastName varchar(50),\n", + " DateOfBirth datetime,\n", + " Gender varchar(10),\n", + " PRIMARY KEY (PatientID)\n", + ");\n", + "\n", + "CREATE TABLE Doctors (\n", + " DoctorID int,\n", + " FirstName varchar(50),\n", + " LastName varchar(50),\n", + " Specialization varchar(50),\n", + " PRIMARY KEY (DoctorID)\n", + ");\n", + "\n", + "CREATE TABLE Prescriptions (\n", + " PrescriptionID int,\n", + " PatientID int,\n", + " DoctorID int,\n", + " DateIssued datetime,\n", + " PRIMARY KEY (PrescriptionID)\n", + ");\n", + "\n", + "CREATE TABLE Medications (\n", + " MedicationID int,\n", + " MedicationName varchar(50),\n", + " Dosage varchar(50),\n", + " PRIMARY KEY (MedicationID)\n", + ");\n", + "\n", + "CREATE TABLE PrescriptionDetails (\n", + " PrescriptionDetailID int,\n", + " PrescriptionID int,\n", + " MedicationID int,\n", + " Quantity int,\n", + " PRIMARY KEY (PrescriptionDetailID)\n", + ");\n", + "```\n", + "\n", + "Write a SQL query that fetches all the patients who were prescribed more than 5 different medications on 2023-04-01.\n", + "\n", + "Assistant:\n", + "\"\"\"\n" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "id": "9afa3431", + "metadata": {}, + "outputs": [], + "source": [ + "# Claude - Body Syntex\n", + "body = json.dumps({\n", + " \"prompt\": prompt_sql_data,\n", + " \"max_tokens_to_sample\":4096,\n", + " \"temperature\":0.5,\n", + " \"top_k\":250,\n", + " \"top_p\":0.5,\n", + " \"stop_sequences\": [\"\\n\\nHuman:\"]\n", + " }) " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5c45f4fc", + "metadata": {}, + "outputs": [], + "source": [ + "modelId = 'anthropic.claude-v2' # change this to use a different version from the model provider\n", + "accept = 'application/json'\n", + "contentType = 'application/json'\n", + "\n", + "response = boto3_bedrock.invoke_model(body=body, modelId=modelId, accept=accept, contentType=contentType)\n", + "response_body = json.loads(response.get('body').read())\n", + "\n", + "print_ww(response_body.get('completion'))" + ] + }, + { + "cell_type": "markdown", + "id": "64b08b3b", + "metadata": {}, + "source": [ + "## Conclusion\n", + "You have now experimented with using `boto3` SDK which provides a vanilla exposure to Amazon Bedrock API. Using this API you have seen the use cases of generate SQL queries to analyze sales data.\n", + "\n", + "### Take aways\n", + "- Adapt this notebook to experiment with different models available through Amazon Bedrock such as Anthropic Claude and AI21 Labs Jurassic models.\n", + "- Apply different prompt engineering principles to get better outputs.\n", + "\n", + "## Thank You" + ] + } + ], + "metadata": { + "availableInstances": [ + { + "_defaultOrder": 0, + "_isFastLaunch": true, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 4, + "name": "ml.t3.medium", + "vcpuNum": 2 + }, + { + "_defaultOrder": 1, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.t3.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 2, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.t3.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 3, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.t3.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 4, + "_isFastLaunch": true, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.m5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 5, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.m5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 6, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.m5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 7, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.m5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 8, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.m5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 9, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.m5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 10, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.m5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 11, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.m5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 12, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.m5d.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 13, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.m5d.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 14, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.m5d.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 15, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.m5d.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 16, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.m5d.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 17, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.m5d.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 18, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.m5d.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 19, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.m5d.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 20, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": true, + "memoryGiB": 0, + "name": "ml.geospatial.interactive", + "supportedImageNames": [ + "sagemaker-geospatial-v1-0" + ], + "vcpuNum": 0 + }, + { + "_defaultOrder": 21, + "_isFastLaunch": true, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 4, + "name": "ml.c5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 22, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.c5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 23, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.c5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 24, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.c5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 25, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 72, + "name": "ml.c5.9xlarge", + "vcpuNum": 36 + }, + { + "_defaultOrder": 26, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 96, + "name": "ml.c5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 27, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 144, + "name": "ml.c5.18xlarge", + "vcpuNum": 72 + }, + { + "_defaultOrder": 28, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.c5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 29, + "_isFastLaunch": true, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.g4dn.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 30, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.g4dn.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 31, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.g4dn.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 32, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.g4dn.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 33, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.g4dn.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 34, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.g4dn.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 35, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 61, + "name": "ml.p3.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 36, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 244, + "name": "ml.p3.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 37, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 488, + "name": "ml.p3.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 38, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.p3dn.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 39, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.r5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 40, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.r5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 41, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.r5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 42, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.r5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 43, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.r5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 44, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.r5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 45, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 512, + "name": "ml.r5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 46, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.r5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 47, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.g5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 48, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.g5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 49, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.g5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 50, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.g5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 51, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.g5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 52, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.g5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 53, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.g5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 54, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.g5.48xlarge", + "vcpuNum": 192 + }, + { + "_defaultOrder": 55, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 1152, + "name": "ml.p4d.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 56, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 1152, + "name": "ml.p4de.24xlarge", + "vcpuNum": 96 + } + ], + "instance_type": "ml.t3.medium", + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/06_CodeGeneration/02_code_interpret_w_langchain.ipynb b/06_CodeGeneration/02_code_interpret_w_langchain.ipynb new file mode 100644 index 00000000..202353e1 --- /dev/null +++ b/06_CodeGeneration/02_code_interpret_w_langchain.ipynb @@ -0,0 +1,921 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "af3f88dd-0f5e-427e-84ee-8934982300d1", + "metadata": { + "tags": [] + }, + "source": [ + "# Bedrock with LangChain - Explain/Interpret a code snippet or program \n", + "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio*" + ] + }, + { + "cell_type": "markdown", + "id": "b920ca4a-a71d-4630-a6e4-577d95192ad1", + "metadata": {}, + "source": [ + "## Introduction\n", + "\n", + "In this notebook we show you how to explain or interpret a given code snippet or program.\n", + "\n", + "[LangChain](https://python.langchain.com/docs/get_started/introduction.html) is a framework for developing applications powered by language models. The key aspects of this framework allow us to augment the Large Language Models by chaining together various components to create advanced use cases.\n", + "\n", + "In this notebook we will use the Bedrock API provided by LangChain. The prompt used in this example creates a custom LangChain prompt template for adding context to the code explain request. \n", + "\n", + "**Note:** *This notebook can be run within or outside of AWS environment.*\n", + "\n", + "#### Context\n", + "In the previous example `01_sql_query_generation_w_bedrock.ipynb`, we explored how to use Bedrock API. In this notebook we will try to add a bit more complexity with the help of `PromptTemplates` to leverage the LangChain framework. `PrompTemplates` allow you to create generic shells which can be populated with information later and get model outputs based on different scenarios.\n", + "\n", + "As part of this notebook we will explore the use of Amazon Bedrock integration within LangChain framework and how it could be used to generate or explain code with the help of `PromptTemplate`.\n", + "\n", + "#### Pattern\n", + "We will simply provide the LangChain implementation of Amazon Bedrock API with an input consisting of a task, an instruction and an input for the model under the hood to generate an output without providing any additional example. The purpose here is to demonstrate how the powerful LLMs easily understand the task at hand and generate compelling outputs.\n", + "\n", + "![](./images/code-interpret-langchain.png)\n", + "\n", + "#### Use case\n", + "To demonstrate the code generation capability of models in Amazon Bedrock, let's take the use case of code explain.\n", + "\n", + "#### Persona\n", + "You are Joe, a Java software developer, has been tasked to support a legacy C++ application for Vehicle Fleet Management. You need help to explain or interpret certain complex C++ code snippets as you are performing analyis to identify the business logic and potential problems with the code.\n", + "\n", + "#### Implementation\n", + "To fulfill this use case, we will show you how you can Amazon Bedrock API with LangChain to explain C++ code snippets.\n" + ] + }, + { + "cell_type": "markdown", + "id": "aa11828a-243d-4808-9c92-e8caf4cebd37", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "Before running the rest of this notebook, you'll need to run the cells below to (ensure necessary libraries are installed and) connect to Bedrock.\n", + "\n", + "For more details on how the setup works and ⚠️ **whether you might need to make any changes**, refer to the [Bedrock boto3 setup notebook](../00_Intro/bedrock_boto3_setup.ipynb) notebook.\n", + "\n", + "In this notebook, we'll also install the [Hugging Face Transformers](https://huggingface.co/docs/transformers/index) library which we'll use for counting the number of tokens in an input prompt." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "49e2c0a9-4838-4f2b-bb36-61c0cbcd62af", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Make sure you ran `download-dependencies.sh` from the root of the repository first!\n", + "%pip install --no-build-isolation --force-reinstall \\\n", + " ../dependencies/awscli-*-py3-none-any.whl \\\n", + " ../dependencies/boto3-*-py3-none-any.whl \\\n", + " ../dependencies/botocore-*-py3-none-any.whl\n", + "\n", + "%pip install --quiet langchain==0.0.249 \"transformers>=4.24,<5\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "558a9372-0789-414a-a1d7-2976056f2015", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "import json\n", + "import os\n", + "import sys\n", + "\n", + "import boto3\n", + "\n", + "module_path = \"..\"\n", + "sys.path.append(os.path.abspath(module_path))\n", + "from utils import bedrock, print_ww\n", + "\n", + "\n", + "# ---- ⚠️ Un-comment and edit the below lines as needed for your AWS setup ⚠️ ----\n", + "\n", + "# os.environ[\"AWS_DEFAULT_REGION\"] = \"\" # E.g. \"us-east-1\"\n", + "# os.environ[\"AWS_PROFILE\"] = \"\"\n", + "# os.environ[\"BEDROCK_ASSUME_ROLE\"] = \"\" # E.g. \"arn:aws:...\"\n", + "# os.environ[\"BEDROCK_ENDPOINT_URL\"] = \"\" # E.g. \"https://...\"\n", + "\n", + "boto3_bedrock = bedrock.get_bedrock_client(\n", + " assumed_role=os.environ.get(\"BEDROCK_ASSUME_ROLE\", None),\n", + " endpoint_url=os.environ.get(\"BEDROCK_ENDPOINT_URL\", None),\n", + " region=os.environ.get(\"AWS_DEFAULT_REGION\", None),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "b7daa1a8-d21a-410c-adbf-b253c2dabf80", + "metadata": { + "tags": [] + }, + "source": [ + "## Invoke the Bedrock LLM Model\n", + "\n", + "We'll begin with creating an instance of Bedrock class from llms. This expects a `model_id` which is the ARN of the model available in Amazon Bedrock. \n", + "\n", + "Optionally you can pass on a previously created boto3 client as well as some `model_kwargs` which can hold parameters such as `temperature`, `topP`, `maxTokenCount` or `stopSequences` (more on parameters can be explored in Amazon Bedrock console).\n", + "\n", + "Available text generation models under Amazon Bedrock have the following IDs:\n", + "\n", + "- amazon.titan-tg1-large\n", + "- ai21.j2-grande-instruct\n", + "- ai21.j2-jumbo-instruct\n", + "- anthropic.claude-instant-v1\n", + "- anthropic.claude-v1\n", + "\n", + "Note that different models support different `model_kwargs`." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "8ffa1250-56cd-4b6d-b3d8-c62baac143ce", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from langchain.llms.bedrock import Bedrock\n", + "\n", + "inference_modifier = {'max_tokens_to_sample':4096, \n", + " \"temperature\":0.5,\n", + " \"top_k\":250,\n", + " \"top_p\":1,\n", + " \"stop_sequences\": [\"\\n\\nHuman\"]\n", + " }\n", + "\n", + "textgen_llm = Bedrock(model_id = \"anthropic.claude-v2\",\n", + " client = boto3_bedrock, \n", + " model_kwargs = inference_modifier \n", + " )\n" + ] + }, + { + "cell_type": "markdown", + "id": "de2678ed-f0d6-444f-9a57-5170dd1952f7", + "metadata": {}, + "source": [ + "## Create a LangChain custom prompt template\n", + "\n", + "By creating a template for the prompt we can pass it different input variables to it on every run. This is useful when you have to generate content with different input variables that you may be fetching from a database." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "96bc21b9", + "metadata": {}, + "outputs": [], + "source": [ + "# Vehicle Fleet Management Code written in C++\n", + "sample_code = \"\"\"\n", + "#include \n", + "#include \n", + "#include \n", + "\n", + "class Vehicle {\n", + "protected:\n", + " std::string registrationNumber;\n", + " int milesTraveled;\n", + " int lastMaintenanceMile;\n", + "\n", + "public:\n", + " Vehicle(std::string regNum) : registrationNumber(regNum), milesTraveled(0), lastMaintenanceMile(0) {}\n", + "\n", + " virtual void addMiles(int miles) {\n", + " milesTraveled += miles;\n", + " }\n", + "\n", + " virtual void performMaintenance() {\n", + " lastMaintenanceMile = milesTraveled;\n", + " std::cout << \"Maintenance performed for vehicle: \" << registrationNumber << std::endl;\n", + " }\n", + "\n", + " virtual void checkMaintenanceDue() {\n", + " if ((milesTraveled - lastMaintenanceMile) > 10000) {\n", + " std::cout << \"Vehicle: \" << registrationNumber << \" needs maintenance!\" << std::endl;\n", + " } else {\n", + " std::cout << \"No maintenance required for vehicle: \" << registrationNumber << std::endl;\n", + " }\n", + " }\n", + "\n", + " virtual void displayDetails() = 0;\n", + "\n", + " ~Vehicle() {\n", + " std::cout << \"Destructor for Vehicle\" << std::endl;\n", + " }\n", + "};\n", + "\n", + "class Truck : public Vehicle {\n", + " int capacityInTons;\n", + "\n", + "public:\n", + " Truck(std::string regNum, int capacity) : Vehicle(regNum), capacityInTons(capacity) {}\n", + "\n", + " void displayDetails() override {\n", + " std::cout << \"Truck with Registration Number: \" << registrationNumber << \", Capacity: \" << capacityInTons << \" tons.\" << std::endl;\n", + " }\n", + "};\n", + "\n", + "class Car : public Vehicle {\n", + " std::string model;\n", + "\n", + "public:\n", + " Car(std::string regNum, std::string carModel) : Vehicle(regNum), model(carModel) {}\n", + "\n", + " void displayDetails() override {\n", + " std::cout << \"Car with Registration Number: \" << registrationNumber << \", Model: \" << model << \".\" << std::endl;\n", + " }\n", + "};\n", + "\n", + "int main() {\n", + " std::vector fleet;\n", + "\n", + " fleet.push_back(new Truck(\"XYZ1234\", 20));\n", + " fleet.push_back(new Car(\"ABC9876\", \"Sedan\"));\n", + "\n", + " for (auto vehicle : fleet) {\n", + " vehicle->displayDetails();\n", + " vehicle->addMiles(10500);\n", + " vehicle->checkMaintenanceDue();\n", + " vehicle->performMaintenance();\n", + " vehicle->checkMaintenanceDue();\n", + " }\n", + "\n", + " for (auto vehicle : fleet) {\n", + " delete vehicle; \n", + " }\n", + "\n", + " return 0;\n", + "}\n", + "\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "dbec103a-97ae-4e9e-9d80-dc20f354a228", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from langchain import PromptTemplate\n", + "\n", + "# Create a prompt template that has multiple input variables\n", + "multi_var_prompt = PromptTemplate(\n", + " input_variables=[\"code\", \"programmingLanguage\"], \n", + " template=\"\"\"Human: You will be acting as an expert software developer in {programmingLanguage}. \n", + " You will explain below code and highlight if any red flags or not following best practices.\n", + " {code}\n", + " Assistant: \n", + " \"\"\"\n", + ")\n", + "\n", + "# Pass in values to the input variables\n", + "prompt = multi_var_prompt.format(code=sample_code, programmingLanguage=\"C++\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "a5b76387", + "metadata": {}, + "source": [ + "### Explain C++ Code for Vehicle Fleet management using Amazon Bedrock and LangChain" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c1064c57-27a4-48c5-911b-e4f1dfeff122", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "response = textgen_llm(prompt)\n", + "\n", + "code_explanation = response[response.index('\\n')+1:]\n", + "\n", + "print_ww(code_explanation)" + ] + }, + { + "cell_type": "markdown", + "id": "9e9abc40", + "metadata": {}, + "source": [ + "## Summary\n", + "\n", + "To conclude we learnt that invoking the LLM without any context might not yield the desired results. By adding context and further using the the prompt template to constrain the output from the LLM we are able to successfully get our desired output" + ] + } + ], + "metadata": { + "availableInstances": [ + { + "_defaultOrder": 0, + "_isFastLaunch": true, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 4, + "name": "ml.t3.medium", + "vcpuNum": 2 + }, + { + "_defaultOrder": 1, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.t3.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 2, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.t3.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 3, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.t3.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 4, + "_isFastLaunch": true, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.m5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 5, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.m5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 6, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.m5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 7, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.m5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 8, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.m5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 9, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.m5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 10, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.m5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 11, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.m5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 12, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.m5d.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 13, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.m5d.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 14, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.m5d.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 15, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.m5d.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 16, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.m5d.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 17, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.m5d.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 18, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.m5d.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 19, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.m5d.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 20, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": true, + "memoryGiB": 0, + "name": "ml.geospatial.interactive", + "supportedImageNames": [ + "sagemaker-geospatial-v1-0" + ], + "vcpuNum": 0 + }, + { + "_defaultOrder": 21, + "_isFastLaunch": true, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 4, + "name": "ml.c5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 22, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.c5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 23, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.c5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 24, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.c5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 25, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 72, + "name": "ml.c5.9xlarge", + "vcpuNum": 36 + }, + { + "_defaultOrder": 26, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 96, + "name": "ml.c5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 27, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 144, + "name": "ml.c5.18xlarge", + "vcpuNum": 72 + }, + { + "_defaultOrder": 28, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.c5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 29, + "_isFastLaunch": true, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.g4dn.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 30, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.g4dn.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 31, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.g4dn.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 32, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.g4dn.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 33, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.g4dn.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 34, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.g4dn.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 35, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 61, + "name": "ml.p3.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 36, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 244, + "name": "ml.p3.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 37, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 488, + "name": "ml.p3.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 38, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.p3dn.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 39, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.r5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 40, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.r5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 41, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.r5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 42, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.r5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 43, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.r5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 44, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.r5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 45, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 512, + "name": "ml.r5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 46, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.r5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 47, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.g5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 48, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.g5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 49, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.g5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 50, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.g5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 51, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.g5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 52, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.g5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 53, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.g5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 54, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.g5.48xlarge", + "vcpuNum": 192 + }, + { + "_defaultOrder": 55, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 1152, + "name": "ml.p4d.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 56, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 1152, + "name": "ml.p4de.24xlarge", + "vcpuNum": 96 + } + ], + "instance_type": "ml.t3.medium", + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/06_CodeGeneration/03_code_translate_w_langchain.ipynb b/06_CodeGeneration/03_code_translate_w_langchain.ipynb new file mode 100644 index 00000000..c8300e08 --- /dev/null +++ b/06_CodeGeneration/03_code_translate_w_langchain.ipynb @@ -0,0 +1,920 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "af3f88dd-0f5e-427e-84ee-8934982300d1", + "metadata": { + "tags": [] + }, + "source": [ + "# Bedrock with LangChain - Code Translation from one programming language to another\n", + "\n", + "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio*" + ] + }, + { + "cell_type": "markdown", + "id": "b920ca4a-a71d-4630-a6e4-577d95192ad1", + "metadata": {}, + "source": [ + "## Introduction\n", + "\n", + "In this notebook, you will learn how to translate code from one programming language to another using LLMs on Amazon Bedrock. We will demonstrate the use of LLMs as well as how to utilize LangChain framework to integrate with Bedrock.\n", + "\n", + "We will use Claude v2 model of Amazon Bedrock in this lab.\n", + "\n", + "**Note:** *This notebook can be run within or outside of AWS environment.*\n", + "\n", + "#### Context\n", + "In the previous example `02_code_interpret_w_langchain.ipynb`, we explored how to use LangChain framework to communicate with Amazon Bedrock API. Similar to previous example of code interpret/explain, we will use LangChain and Amazon Bedrock APIs to translate code from one legacy programming language to another.\n", + "\n", + "\n", + "#### Pattern\n", + "We will simply provide the LangChain implementation of Amazon Bedrock API with an input consisting of a task, an instruction and an input for the model under the hood to generate an output without providing any additional example. The purpose here is to demonstrate how the powerful LLMs easily understand the task at hand and generate compelling outputs.\n", + "\n", + "![](./images/code-translation-langchain.png)\n", + "\n", + "#### Use case\n", + "To demonstrate how you can use Amazon Bedrock LLMs to translate code from one programming language to another.\n", + "\n", + "#### Persona\n", + "Guides you through translating C++ code to Java using Amazon Bedrock and LangChain APIs. It shows techniques for prompting the model to port C++ code over to Java, handling differences in syntax, language constructs, and conventions between the languages.\n", + "\n", + "#### Implementation\n", + "To fulfill this use case, we will show you how to translate a given legacy C++ code to port to Java. We will use the Amazon Bedrock and LangChain integration. \n" + ] + }, + { + "cell_type": "markdown", + "id": "aa11828a-243d-4808-9c92-e8caf4cebd37", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "Before running the rest of this notebook, you'll need to run the cells below to (ensure necessary libraries are installed and) connect to Bedrock.\n", + "\n", + "For more details on how the setup works and ⚠️ **whether you might need to make any changes**, refer to the [Bedrock boto3 setup notebook](../00_Intro/bedrock_boto3_setup.ipynb) notebook.\n", + "\n", + "In this notebook, we'll also install the [Hugging Face Transformers](https://huggingface.co/docs/transformers/index) library which we'll use for counting the number of tokens in an input prompt." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "49e2c0a9-4838-4f2b-bb36-61c0cbcd62af", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Make sure you ran `download-dependencies.sh` from the root of the repository first!\n", + "%pip install --no-build-isolation --force-reinstall \\\n", + " ../dependencies/awscli-*-py3-none-any.whl \\\n", + " ../dependencies/boto3-*-py3-none-any.whl \\\n", + " ../dependencies/botocore-*-py3-none-any.whl\n", + "\n", + "%pip install --quiet langchain==0.0.249 \"transformers>=4.24,<5\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "558a9372-0789-414a-a1d7-2976056f2015", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "import json\n", + "import os\n", + "import sys\n", + "\n", + "import boto3\n", + "\n", + "module_path = \"..\"\n", + "sys.path.append(os.path.abspath(module_path))\n", + "from utils import bedrock, print_ww\n", + "\n", + "\n", + "# ---- ⚠️ Un-comment and edit the below lines as needed for your AWS setup ⚠️ ----\n", + "\n", + "# os.environ[\"AWS_DEFAULT_REGION\"] = \"\" # E.g. \"us-east-1\"\n", + "# os.environ[\"AWS_PROFILE\"] = \"\"\n", + "# os.environ[\"BEDROCK_ASSUME_ROLE\"] = \"\" # E.g. \"arn:aws:...\"\n", + "# os.environ[\"BEDROCK_ENDPOINT_URL\"] = \"\" # E.g. \"https://...\"\n", + "\n", + "\n", + "boto3_bedrock = bedrock.get_bedrock_client(\n", + " assumed_role=os.environ.get(\"BEDROCK_ASSUME_ROLE\", None),\n", + " endpoint_url=os.environ.get(\"BEDROCK_ENDPOINT_URL\", None),\n", + " region=os.environ.get(\"AWS_DEFAULT_REGION\", None),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "b7daa1a8-d21a-410c-adbf-b253c2dabf80", + "metadata": { + "tags": [] + }, + "source": [ + "## Invoke the Bedrock LLM Model\n", + "\n", + "We'll begin with creating an instance of Bedrock class from llms. This expects a `model_id` which is the ARN of the model available in Amazon Bedrock. \n", + "\n", + "Optionally you can pass on a previously created boto3 client as well as some `model_kwargs` which can hold parameters such as `temperature`, `topP`, `maxTokenCount` or `stopSequences` (more on parameters can be explored in Amazon Bedrock console).\n", + "\n", + "Available text generation models under Amazon Bedrock have the following IDs:\n", + "\n", + "- amazon.titan-tg1-large\n", + "- ai21.j2-grande-instruct\n", + "- ai21.j2-jumbo-instruct\n", + "- anthropic.claude-instant-v1\n", + "- anthropic.claude-v1\n", + "\n", + "Note that different models support different `model_kwargs`." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "8ffa1250-56cd-4b6d-b3d8-c62baac143ce", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from langchain.llms.bedrock import Bedrock\n", + "\n", + "inference_modifier = {'max_tokens_to_sample':4096, \n", + " \"temperature\":0.5,\n", + " \"top_k\":250,\n", + " \"top_p\":1,\n", + " \"stop_sequences\": [\"\\n\\nHuman\"]\n", + " }\n", + "\n", + "textgen_llm = Bedrock(model_id = \"anthropic.claude-v2\",\n", + " client = boto3_bedrock, \n", + " model_kwargs = inference_modifier \n", + " )\n" + ] + }, + { + "cell_type": "markdown", + "id": "de2678ed-f0d6-444f-9a57-5170dd1952f7", + "metadata": {}, + "source": [ + "## Create a LangChain custom prompt template\n", + "\n", + "By creating a template for the prompt we can pass it different input variables to it on every run. This is useful when you have to generate content with different input variables that you may be fetching from a database." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "96bc21b9", + "metadata": {}, + "outputs": [], + "source": [ + "# Vehicle Fleet Management Code written in C++\n", + "sample_code = \"\"\"\n", + "#include \n", + "#include \n", + "#include \n", + "\n", + "class Vehicle {\n", + "protected:\n", + " std::string registrationNumber;\n", + " int milesTraveled;\n", + " int lastMaintenanceMile;\n", + "\n", + "public:\n", + " Vehicle(std::string regNum) : registrationNumber(regNum), milesTraveled(0), lastMaintenanceMile(0) {}\n", + "\n", + " virtual void addMiles(int miles) {\n", + " milesTraveled += miles;\n", + " }\n", + "\n", + " virtual void performMaintenance() {\n", + " lastMaintenanceMile = milesTraveled;\n", + " std::cout << \"Maintenance performed for vehicle: \" << registrationNumber << std::endl;\n", + " }\n", + "\n", + " virtual void checkMaintenanceDue() {\n", + " if ((milesTraveled - lastMaintenanceMile) > 10000) {\n", + " std::cout << \"Vehicle: \" << registrationNumber << \" needs maintenance!\" << std::endl;\n", + " } else {\n", + " std::cout << \"No maintenance required for vehicle: \" << registrationNumber << std::endl;\n", + " }\n", + " }\n", + "\n", + " virtual void displayDetails() = 0;\n", + "\n", + " ~Vehicle() {\n", + " std::cout << \"Destructor for Vehicle\" << std::endl;\n", + " }\n", + "};\n", + "\n", + "class Truck : public Vehicle {\n", + " int capacityInTons;\n", + "\n", + "public:\n", + " Truck(std::string regNum, int capacity) : Vehicle(regNum), capacityInTons(capacity) {}\n", + "\n", + " void displayDetails() override {\n", + " std::cout << \"Truck with Registration Number: \" << registrationNumber << \", Capacity: \" << capacityInTons << \" tons.\" << std::endl;\n", + " }\n", + "};\n", + "\n", + "class Car : public Vehicle {\n", + " std::string model;\n", + "\n", + "public:\n", + " Car(std::string regNum, std::string carModel) : Vehicle(regNum), model(carModel) {}\n", + "\n", + " void displayDetails() override {\n", + " std::cout << \"Car with Registration Number: \" << registrationNumber << \", Model: \" << model << \".\" << std::endl;\n", + " }\n", + "};\n", + "\n", + "int main() {\n", + " std::vector fleet;\n", + "\n", + " fleet.push_back(new Truck(\"XYZ1234\", 20));\n", + " fleet.push_back(new Car(\"ABC9876\", \"Sedan\"));\n", + "\n", + " for (auto vehicle : fleet) {\n", + " vehicle->displayDetails();\n", + " vehicle->addMiles(10500);\n", + " vehicle->checkMaintenanceDue();\n", + " vehicle->performMaintenance();\n", + " vehicle->checkMaintenanceDue();\n", + " }\n", + "\n", + " for (auto vehicle : fleet) {\n", + " delete vehicle; \n", + " }\n", + "\n", + " return 0;\n", + "}\n", + "\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "dbec103a-97ae-4e9e-9d80-dc20f354a228", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from langchain import PromptTemplate\n", + "\n", + "# Create a prompt template that has multiple input variables\n", + "multi_var_prompt = PromptTemplate(\n", + " input_variables=[\"code\", \"srcProgrammingLanguage\", \"targetProgrammingLanguage\"], \n", + " template=\"\"\"Human: You will be acting as an expert software developer in {srcProgrammingLanguage} and {targetProgrammingLanguage}. \n", + " You will tranlslate below code from {srcProgrammingLanguage} to {targetProgrammingLanguage} while following coding best practices.\n", + " {code}\n", + " Assistant: \n", + " \"\"\"\n", + ")\n", + "\n", + "# Pass in values to the input variables\n", + "prompt = multi_var_prompt.format(code=sample_code, srcProgrammingLanguage=\"C++\", targetProgrammingLanguage=\"Java\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "a5b76387", + "metadata": {}, + "source": [ + "### Code translation from C++ to Java" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c1064c57-27a4-48c5-911b-e4f1dfeff122", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "response = textgen_llm(prompt)\n", + "\n", + "target_code = response[response.index('\\n')+1:]\n", + "\n", + "print_ww(target_code)" + ] + }, + { + "cell_type": "markdown", + "id": "9e9abc40", + "metadata": {}, + "source": [ + "## Summary\n", + "\n", + "In this example, you have learned how to translate a legacy C++ program to Java with a simple text prompt using Amazon Bedrock and langchain." + ] + } + ], + "metadata": { + "availableInstances": [ + { + "_defaultOrder": 0, + "_isFastLaunch": true, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 4, + "name": "ml.t3.medium", + "vcpuNum": 2 + }, + { + "_defaultOrder": 1, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.t3.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 2, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.t3.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 3, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.t3.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 4, + "_isFastLaunch": true, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.m5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 5, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.m5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 6, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.m5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 7, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.m5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 8, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.m5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 9, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.m5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 10, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.m5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 11, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.m5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 12, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.m5d.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 13, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.m5d.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 14, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.m5d.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 15, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.m5d.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 16, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.m5d.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 17, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.m5d.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 18, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.m5d.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 19, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.m5d.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 20, + "_isFastLaunch": false, + "category": "General purpose", + "gpuNum": 0, + "hideHardwareSpecs": true, + "memoryGiB": 0, + "name": "ml.geospatial.interactive", + "supportedImageNames": [ + "sagemaker-geospatial-v1-0" + ], + "vcpuNum": 0 + }, + { + "_defaultOrder": 21, + "_isFastLaunch": true, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 4, + "name": "ml.c5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 22, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 8, + "name": "ml.c5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 23, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.c5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 24, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.c5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 25, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 72, + "name": "ml.c5.9xlarge", + "vcpuNum": 36 + }, + { + "_defaultOrder": 26, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 96, + "name": "ml.c5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 27, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 144, + "name": "ml.c5.18xlarge", + "vcpuNum": 72 + }, + { + "_defaultOrder": 28, + "_isFastLaunch": false, + "category": "Compute optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.c5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 29, + "_isFastLaunch": true, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.g4dn.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 30, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.g4dn.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 31, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.g4dn.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 32, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.g4dn.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 33, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.g4dn.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 34, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.g4dn.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 35, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 61, + "name": "ml.p3.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 36, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 244, + "name": "ml.p3.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 37, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 488, + "name": "ml.p3.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 38, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.p3dn.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 39, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.r5.large", + "vcpuNum": 2 + }, + { + "_defaultOrder": 40, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.r5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 41, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.r5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 42, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.r5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 43, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.r5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 44, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.r5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 45, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 512, + "name": "ml.r5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 46, + "_isFastLaunch": false, + "category": "Memory Optimized", + "gpuNum": 0, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.r5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 47, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 16, + "name": "ml.g5.xlarge", + "vcpuNum": 4 + }, + { + "_defaultOrder": 48, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 32, + "name": "ml.g5.2xlarge", + "vcpuNum": 8 + }, + { + "_defaultOrder": 49, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 64, + "name": "ml.g5.4xlarge", + "vcpuNum": 16 + }, + { + "_defaultOrder": 50, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 128, + "name": "ml.g5.8xlarge", + "vcpuNum": 32 + }, + { + "_defaultOrder": 51, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 1, + "hideHardwareSpecs": false, + "memoryGiB": 256, + "name": "ml.g5.16xlarge", + "vcpuNum": 64 + }, + { + "_defaultOrder": 52, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 192, + "name": "ml.g5.12xlarge", + "vcpuNum": 48 + }, + { + "_defaultOrder": 53, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 4, + "hideHardwareSpecs": false, + "memoryGiB": 384, + "name": "ml.g5.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 54, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 768, + "name": "ml.g5.48xlarge", + "vcpuNum": 192 + }, + { + "_defaultOrder": 55, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 1152, + "name": "ml.p4d.24xlarge", + "vcpuNum": 96 + }, + { + "_defaultOrder": 56, + "_isFastLaunch": false, + "category": "Accelerated computing", + "gpuNum": 8, + "hideHardwareSpecs": false, + "memoryGiB": 1152, + "name": "ml.p4de.24xlarge", + "vcpuNum": 96 + } + ], + "instance_type": "ml.t3.medium", + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/06_CodeGeneration/README.md b/06_CodeGeneration/README.md new file mode 100644 index 00000000..0dde55a1 --- /dev/null +++ b/06_CodeGeneration/README.md @@ -0,0 +1,37 @@ +# Lab 6 - Code Generation + +## Overview + +In this lab, you will learn to use LLMs on Amazon Bedrock for code generation, SQL query creation, code explanation, and code translation across languages. We will demo Bedrock's API (boto3) as well as its integration with LangChain. + +First, we will generate Python code and SQL queries by providing context about a dataset. Next, we will explain code and translate between languages. We will explore these use cases with both the Bedrock API directly and via LangChain integration. + +## Audience + +Architects and developers who want to learn how to use Amazon Bedrock LLMs to generate, explain and translate code. + +Some of the business use cases for code generation include: + +- Code Translation +- Code Explain and Reviews +- Database or SQL query generation +- Rapid Prototyping +- Issue Identification +- Bug Fixing +- Code Optimization + +## Workshop Notebooks + +1. [Code Generation](./00_code_generatation_w_bedrock.ipynb)- Demonstrates how to generate Python code using Natural language. It shows examples of prompting to generate simple functions, classes, and full programs in Python for Data Analyst to perform sales analysis on a given Sales CSV dataset. + +2. [Database or SQL Query Generation](./01_sql_query_generate_w_bedrock.ipynb) - Focuses on generating SQL queries with Amazon Bedrock APIs. It includes examples of generating both simple and complex SQL statements for a given data set and database schema. + +3. [Code Explanation](./02_code_interpret_w_langchain.ipynb) - Uses Bedrock's foundation models to generate explanations for complex C++ code snippets. It shows how to carefully craft prompts to get the model to generate comments and documentation that explain the functionality and logic of complicated C++ code examples. Prompts can be easily updated for another programming languages. + +4. [Code Translation ](./03_code_translate_w_langchain.ipynb) - Guides you through translating C++ code to Java using Amazon Bedrock and LangChain APIs. It shows techniques for prompting the model to port C++ code over to Java, handling differences in syntax, language constructs, and conventions between the languages. + + +## Architecture + +![Bedrock](./images/bedrock-code-gen.png) +![Bedrock](./images/bedrock-code-gen-langchain.png) \ No newline at end of file diff --git a/06_CodeGeneration/images/bedrock-code-gen-langchain.png b/06_CodeGeneration/images/bedrock-code-gen-langchain.png new file mode 100644 index 00000000..9aa17787 Binary files /dev/null and b/06_CodeGeneration/images/bedrock-code-gen-langchain.png differ diff --git a/06_CodeGeneration/images/bedrock-code-gen.png b/06_CodeGeneration/images/bedrock-code-gen.png new file mode 100644 index 00000000..3457eac2 Binary files /dev/null and b/06_CodeGeneration/images/bedrock-code-gen.png differ diff --git a/06_CodeGeneration/images/code-interpret-langchain.png b/06_CodeGeneration/images/code-interpret-langchain.png new file mode 100644 index 00000000..85b3c263 Binary files /dev/null and b/06_CodeGeneration/images/code-interpret-langchain.png differ diff --git a/06_CodeGeneration/images/code-translation-langchain.png b/06_CodeGeneration/images/code-translation-langchain.png new file mode 100644 index 00000000..6fd93581 Binary files /dev/null and b/06_CodeGeneration/images/code-translation-langchain.png differ diff --git a/06_CodeGeneration/images/sql-query-generation.png b/06_CodeGeneration/images/sql-query-generation.png new file mode 100644 index 00000000..207abb0b Binary files /dev/null and b/06_CodeGeneration/images/sql-query-generation.png differ diff --git a/README.md b/README.md index 92e08cb8..3fef10eb 100644 --- a/README.md +++ b/README.md @@ -13,10 +13,11 @@ Labs include: - **Questions Answering** \[Estimated time to complete - 45 mins\] - **Chatbot** \[Estimated time to complete - 45 mins\] - **Image Generation** \[Estimated time to complete - 30 mins\] +- **Code Generation** \[Estimated time to complete - 30 mins\]
-![imgs/10-overview](imgs/10-overview.png "Overview of the different labs in the workshop") +![imgs/11-overview](imgs/11-overview.png "Overview of the different labs in the workshop")
@@ -117,3 +118,13 @@ This repository contains notebook examples for the Bedrock Architecture Patterns ### Text to Image - [Image Generation with Stable Diffusion](./05_Image/Bedrock%20Stable%20Diffusion%20XL.ipynb): This notebook demonstrates image generation with using the Stable Diffusion model + +### Code Generation, SQL Generation, Code Translation and Explanation + +1. [Code Generation](./06_CodeGeneration/00_code_generatation_w_bedrock.ipynb): Demonstrates how to generate Python code using Natural language. It shows examples of prompting to generate simple functions, classes, and full programs in Python for Data Analyst to perform sales analysis on a given Sales CSV dataset. + +2. [Database or SQL Query Generation](./06_CodeGeneration/01_sql_query_generate_w_bedrock.ipynb) : Focuses on generating SQL queries with Amazon Bedrock APIs. It includes examples of generating both simple and complex SQL statements for a given data set and database schema. + +3. [Code Explanation](./06_CodeGeneration/02_code_interpret_w_langchain.ipynb) : Uses Bedrock's foundation models to generate explanations for complex C++ code snippets. It shows how to carefully craft prompts to get the model to generate comments and documentation that explain the functionality and logic of complicated C++ code examples. Prompts can be easily updated for another programming languages. + +4. [Code Translation ](./06_CodeGeneration/03_code_translate_w_langchain.ipynb) : Guides you through translating C++ code to Java using Amazon Bedrock and LangChain APIs. It shows techniques for prompting the model to port C++ code over to Java, handling differences in syntax, language constructs, and conventions between the languages. diff --git a/imgs/11-overview.png b/imgs/11-overview.png new file mode 100644 index 00000000..d7bc9bad Binary files /dev/null and b/imgs/11-overview.png differ