00_Intro/bedrock_boto3_setup.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "88a5ab2f-d044-4956-b75b-7408d9c3e323",
   "metadata": {},
   "source": [
    "# Amazon Bedrock boto3 Setup\n",
    "\n",
    "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio*\n",
    "\n",
    "---\n",
    "\n",
    "In this demo notebook, we demonstrate how to use the [`boto3` Python SDK](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) to work with [Amazon Bedrock](https://aws.amazon.com/bedrock/) Foundation Models.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6aeedd9f-f0a3-4f8e-934d-22f6f7a89de5",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "\n",
    "⚠️ You should already have run the [../download-dependencies.sh script](../download-dependencies.sh) to fetch the SDK packages required for using Bedrock.\n",
    "\n",
    "Once the packages are ready, you'll need to run the below cell to install them in the notebook kernel:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "108c611c-7246-45c4-9f1e-76888b5076eb",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Make sure you ran `download-dependencies.sh` from the root of the repository first!\n",
    "%pip install --no-build-isolation --force-reinstall \\\n",
    "    ../dependencies/awscli-*-py3-none-any.whl \\\n",
    "    ../dependencies/boto3-*-py3-none-any.whl \\\n",
    "    ../dependencies/botocore-*-py3-none-any.whl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f1c8940",
   "metadata": {},
   "source": [
    "This notebook demonstrates invoking Bedrock models directly using the AWS SDK, but for later notebooks in the workshop you'll also need to install [LangChain](https://github.com/hwchase17/langchain):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e692c0d3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install --quiet langchain==0.0.249"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27610c0f-7de6-4440-8f76-decf30e3c5ca",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Create the boto3 client\n",
    "\n",
    "Interaction with the Bedrock API is done via the AWS SDK for Python: [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html).\n",
    "\n",
    "Depending on your environment, you might need to customize the setup when creating your Bedrock service client. To help with this, we've provided a `get_bedrock_client()` utility method that supports passing in different options. You can find the implementation in [../utils/bedrock.py](../utils/bedrock.py)\n",
    "\n",
    "\n",
    "#### Use the default credential chain\n",
    "\n",
    "If you are running this notebook from [Amazon Sagemaker Studio](https://aws.amazon.com/sagemaker/studio/) and your Sagemaker Studio [execution role](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html) has permissions to access Bedrock you can just run the cells below as-is. This is also the case if you are running these notebooks from a computer whose default AWS credentials have access to Bedrock.\n",
    "\n",
    "#### Use a different AWS Region\n",
    "\n",
    "If you're running this notebook from your own computer or a SageMaker notebook in a different AWS Region from where Bedrock is set up, you can un-comment the `os.environ['AWS_DEFAULT_REGION']` line below and specify the region to use.\n",
    "\n",
    "#### Use a specific profile\n",
    "\n",
    "In case you're running this notebook from your own computer where you have setup the AWS CLI with multiple profiles, and the profile which has access to Bedrock is not the default one, you can un-comment the `os.environ['AWS_PROFILE']` line below and specify the profile to use.\n",
    "\n",
    "#### Use a different role\n",
    "\n",
    "In case you or your company has setup a specific, separate [IAM Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) to access Bedrock, you can specify it by un-commenting the `os.environ['BEDROCK_ASSUME_ROLE']` line below. Ensure that your current user or role have permissions to [assume](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) such role.\n",
    "\n",
    "#### Use a custom service endpoint URL\n",
    "\n",
    "If you need to use a custom [service endpoint URL](https://docs.aws.amazon.com/general/latest/gr/rande.html) to access Bedrock as part of the preview, you'll need to un-comment and edit the `os.environ['BEDROCK_ENDPOINT_URL']` line below.\n",
    "\n",
    "#### A note about `langchain`\n",
    "\n",
    "The Bedrock classes provided by `langchain` create a Bedrock boto3 client by default. To customize your Bedrock configuration, we recommend to explicitly create the Bedrock client using the method below, and pass it to the [`langchain.Bedrock`](https://python.langchain.com/docs/integrations/llms/bedrock) class instantiation method using `client=boto3_bedrock`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae2b2a05-78a9-40ca-9b5e-121030f9ede1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "import sys\n",
    "\n",
    "import boto3\n",
    "\n",
    "module_path = \"..\"\n",
    "sys.path.append(os.path.abspath(module_path))\n",
    "from utils import bedrock, print_ww\n",
    "\n",
    "\n",
    "# ---- ⚠️ Un-comment and edit the below lines as needed for your AWS setup ⚠️ ----\n",
    "\n",
    "# os.environ[\"AWS_DEFAULT_REGION\"] = \"<REGION_NAME>\"  # E.g. \"us-east-1\"\n",
    "# os.environ[\"AWS_PROFILE\"] = \"<YOUR_PROFILE>\"\n",
    "# os.environ[\"BEDROCK_ASSUME_ROLE\"] = \"<YOUR_ROLE_ARN>\"  # E.g. \"arn:aws:...\"\n",
    "# os.environ[\"BEDROCK_ENDPOINT_URL\"] = \"<YOUR_ENDPOINT_URL>\"  # E.g. \"https://...\"\n",
    "\n",
    "\n",
    "boto3_bedrock = bedrock.get_bedrock_client(\n",
    "    assumed_role=os.environ.get(\"BEDROCK_ASSUME_ROLE\", None),\n",
    "    endpoint_url=os.environ.get(\"BEDROCK_ENDPOINT_URL\", None),\n",
    "    region=os.environ.get(\"AWS_DEFAULT_REGION\", None),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e9174c4-326a-463e-92e1-8c7e47111269",
   "metadata": {},
   "source": [
    "#### Validate the connection\n",
    "\n",
    "We can check the client works by trying out the `list_foundation_models()` method, which will tell us all the models available for us to use "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f67b4466-12ff-4975-9811-7a19c6206604",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "boto3_bedrock.list_foundation_models()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f690043-df45-448f-8fa6-1ea8b06f1087",
   "metadata": {
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "## `InvokeModel` body and output\n",
    "\n",
    "The `invoke_model()` method of the Amazon Bedrock client (`InvokeModel` API) will be the primary method we use for most of our Text Generation and Processing tasks - whichever model we're using.\n",
    "\n",
    "Although the method is shared, the format of input and output varies depending on the foundation model used - as described below:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4650fa3-a831-4039-9fd6-749926d35979",
   "metadata": {},
   "source": [
    "### Amazon Titan Large\n",
    "\n",
    "#### Input\n",
    "```json\n",
    "{   \n",
    "    \"inputText\": \"<prompt>\",\n",
    "    \"textGenerationConfig\" : { \n",
    "        \"maxTokenCount\": 512,\n",
    "        \"stopSequences\": [],\n",
    "        \"temperature\": 0.1,  \n",
    "        \"topP\": 0.9\n",
    "    }\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"inputTextTokenCount\": 613,\n",
    "    \"results\": [{\n",
    "        \"tokenCount\": 219,\n",
    "        \"outputText\": \"<output>\"\n",
    "    }]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ca0c329-ff58-483e-8abd-63867950c14b",
   "metadata": {},
   "source": [
    "### AI21 Jurassic (Grande and Jumbo) \n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"prompt\": \"<prompt>\",\n",
    "    \"maxTokens\": 200,\n",
    "    \"temperature\": 0.5,\n",
    "    \"topP\": 0.5,\n",
    "    \"stopSequences\": [],\n",
    "    \"countPenalty\": {\"scale\": 0},\n",
    "    \"presencePenalty\": {\"scale\": 0},\n",
    "    \"frequencyPenalty\": {\"scale\": 0}\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"id\": 1234,\n",
    "    \"prompt\": {\n",
    "        \"text\": \"<prompt>\",\n",
    "        \"tokens\": [\n",
    "            {\n",
    "                \"generatedToken\": {\n",
    "                    \"token\": \"\\u2581who\\u2581is\",\n",
    "                    \"logprob\": -12.980147361755371,\n",
    "                    \"raw_logprob\": -12.980147361755371\n",
    "                },\n",
    "                \"topTokens\": null,\n",
    "                \"textRange\": {\"start\": 0, \"end\": 6}\n",
    "            },\n",
    "            //...\n",
    "        ]\n",
    "    },\n",
    "    \"completions\": [\n",
    "        {\n",
    "            \"data\": {\n",
    "                \"text\": \"<output>\",\n",
    "                \"tokens\": [\n",
    "                    {\n",
    "                        \"generatedToken\": {\n",
    "                            \"token\": \"<|newline|>\",\n",
    "                            \"logprob\": 0.0,\n",
    "                            \"raw_logprob\": -0.01293118204921484\n",
    "                        },\n",
    "                        \"topTokens\": null,\n",
    "                        \"textRange\": {\"start\": 0, \"end\": 1}\n",
    "                    },\n",
    "                    //...\n",
    "                ]\n",
    "            },\n",
    "            \"finishReason\": {\"reason\": \"endoftext\"}\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7adb6bee-7654-4269-9127-9afa4e823454",
   "metadata": {},
   "source": [
    "### Anthropic Claude\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"prompt\": \"\\n\\nHuman:<prompt>\\n\\nAnswer:\",\n",
    "    \"max_tokens_to_sample\": 300,\n",
    "    \"temperature\": 0.5,\n",
    "    \"top_k\": 250,\n",
    "    \"top_p\": 1,\n",
    "    \"stop_sequences\": [\"\\n\\nHuman:\"]\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"completion\": \"<output>\",\n",
    "    \"stop_reason\": \"stop_sequence\"\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb94aa2f-30da-499b-b3f5-02f102dbb1ea",
   "metadata": {},
   "source": [
    "### Stability AI Stable Diffusion XL\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"text_prompts\": [\n",
    "        {\"text\": \"this is where you place your input text\"}\n",
    "    ],\n",
    "    \"cfg_scale\": 10,\n",
    "    \"seed\": 0,\n",
    "    \"steps\": 50\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{ \n",
    "    \"result\": \"success\", \n",
    "    \"artifacts\": [\n",
    "        {\n",
    "            \"seed\": 123, \n",
    "            \"base64\": \"<image in base64>\",\n",
    "            \"finishReason\": \"SUCCESS\"\n",
    "        },\n",
    "        //...\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80f4adca-cfc4-439b-84b7-e528398684e3",
   "metadata": {
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "## Common inference parameter definitions\n",
    "\n",
    "### Randomness and Diversity\n",
    "\n",
    "Foundation models support the following parameters to control randomness and diversity in the \n",
    "response.\n",
    "\n",
    "**Temperature** – Large language models use probability to construct the words in a sequence. For any \n",
    "given next word, there is a probability distribution of options for the next word in the sequence. When \n",
    "you set the temperature closer to zero, the model tends to select the higher-probability words. When \n",
    "you set the temperature further away from zero, the model may select a lower-probability word.\n",
    "\n",
    "In technical terms, the temperature modulates the probability density function for the next tokens, \n",
    "implementing the temperature sampling technique. This parameter can deepen or flatten the density \n",
    "function curve. A lower value results in a steeper curve with more deterministic responses, and a higher \n",
    "value results in a flatter curve with more random responses.\n",
    "\n",
    "**Top K** – Temperature defines the probability distribution of potential words, and Top K defines the cut \n",
    "off where the model no longer selects the words. For example, if K=50, the model selects from 50 of the \n",
    "most probable words that could be next in a given sequence. This reduces the probability that an unusual \n",
    "word gets selected next in a sequence.\n",
    "In technical terms, Top K is the number of the highest-probability vocabulary tokens to keep for Top-\n",
    "K-filtering - This limits the distribution of probable tokens, so the model chooses one of the highest-\n",
    "probability tokens.\n",
    "\n",
    "**Top P** – Top P defines a cut off based on the sum of probabilities of the potential choices. If you set Top \n",
    "P below 1.0, the model considers the most probable options and ignores less probable ones. Top P is \n",
    "similar to Top K, but instead of capping the number of choices, it caps choices based on the sum of their \n",
    "probabilities.\n",
    "For the example prompt \"I hear the hoof beats of ,\" you may want the model to provide \"horses,\" \n",
    "\"zebras\" or \"unicorns\" as the next word. If you set the temperature to its maximum, without capping \n",
    "Top K or Top P, you increase the probability of getting unusual results such as \"unicorns.\" If you set the \n",
    "temperature to 0, you increase the probability of \"horses.\" If you set a high temperature and set Top K or \n",
    "Top P to the maximum, you increase the probability of \"horses\" or \"zebras,\" and decrease the probability \n",
    "of \"unicorns.\"\n",
    "\n",
    "### Length\n",
    "\n",
    "The following parameters control the length of the generated response.\n",
    "\n",
    "**Response length** – Configures the minimum and maximum number of tokens to use in the generated \n",
    "response.\n",
    "\n",
    "**Length penalty** – Length penalty optimizes the model to be more concise in its output by penalizing \n",
    "longer responses. Length penalty differs from response length as the response length is a hard cut off for \n",
    "the minimum or maximum response length.\n",
    "\n",
    "In technical terms, the length penalty penalizes the model exponentially for lengthy responses. 0.0 \n",
    "means no penalty. Set a value less than 0.0 for the model to generate longer sequences, or set a value \n",
    "greater than 0.0 for the model to produce shorter sequences.\n",
    "\n",
    "### Repetitions\n",
    "\n",
    "The following parameters help control repetition in the generated response.\n",
    "\n",
    "**Repetition penalty (presence penalty)** – Prevents repetitions of the same words (tokens) in responses. \n",
    "1.0 means no penalty. Greater than 1.0 decreases repetition."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce22c308-ebbf-4ef5-a823-832b7c236e31",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Try out the models\n",
    "\n",
    "With some theory out of the way, let's see the models in action! Run the cells below to see basic, synchronous example invocations for each model:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "893872fe-04fa-4f09-9736-6c6173ec1fc2",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Amazon Titan Large"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7df55eed-a3cf-426c-95ea-ec60dade6477",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If you'd like to try your own prompt, edit this parameter!\n",
    "prompt_data = \"\"\"Command: Write me a blog about making strong business decisions as a leader.\n",
    "\n",
    "Blog:\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd2bb671-6b10-4948-9e5e-95d6ced3b86f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "body = json.dumps({\"inputText\": prompt_data})\n",
    "modelId = \"amazon.titan-tg1-large\"\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "response = boto3_bedrock.invoke_model(\n",
    "    body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    ")\n",
    "response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "print(response_body.get(\"results\")[0].get(\"outputText\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d7c0fe6-576a-4380-89aa-726bab5d65ff",
   "metadata": {},
   "source": [
    "### Anthropic Claude"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ba33ac0-fa16-4c4f-b882-e838d0cb5830",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "body = json.dumps({\"prompt\": prompt_data, \"max_tokens_to_sample\": 500})\n",
    "modelId = \"anthropic.claude-instant-v1\"  # change this to use a different version from the model provider\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "response = boto3_bedrock.invoke_model(\n",
    "    body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    ")\n",
    "response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "print(response_body.get(\"completion\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed0e3144-c6df-400d-aab1-1540614dbbde",
   "metadata": {},
   "source": [
    "### AI21 Jurassic Grande"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c02d1585-945e-45d1-99d2-171e956138f8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "body = json.dumps({\"prompt\": prompt_data, \"maxTokens\": 200})\n",
    "modelId = \"ai21.j2-grande-instruct\"  # change this to use a different version from the model provider\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "response = boto3_bedrock.invoke_model(\n",
    "    body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    ")\n",
    "response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "print(response_body.get(\"completions\")[0].get(\"data\").get(\"text\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc498bea",
   "metadata": {},
   "source": [
    "### Stability Stable Diffusion XL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "173e51a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_data = \"a fine image of an astronaut riding a horse on Mars\"\n",
    "body = json.dumps({\n",
    "    \"text_prompts\": [{\"text\": prompt_data}],\n",
    "    \"cfg_scale\": 10,\n",
    "    \"seed\": 20,\n",
    "    \"steps\": 50\n",
    "})\n",
    "modelId = \"stability.stable-diffusion-xl\"\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "response = boto3_bedrock.invoke_model(\n",
    "    body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    ")\n",
    "response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "print(response_body[\"result\"])\n",
    "print(f'{response_body.get(\"artifacts\")[0].get(\"base64\")[0:80]}...')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a271fa6-13fd-480a-87a5-3702d29a5c43",
   "metadata": {},
   "source": [
    "**Note:** The output is a [base64 encoded](https://docs.python.org/3/library/base64.html) string of the image data. You can use any image processing library (such as [Pillow](https://pillow.readthedocs.io/en/stable/)) to decode the image as in the example below:\n",
    "\n",
    "```python\n",
    "import base64\n",
    "import io\n",
    "from PIL import Image\n",
    "\n",
    "base_64_img_str = response_body.get(\"artifacts\")[0].get(\"base64\")\n",
    "image = Image.open(io.BytesIO(base64.decodebytes(bytes(base_64_img_str, \"utf-8\"))))\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4621a301-53e4-4182-9fce-8ee422813e9d",
   "metadata": {},
   "source": [
    "## Generate streaming output\n",
    "\n",
    "For large language models, it can take noticeable time to generate long output sequences. Rather than waiting for the entire response to be available, latency-sensitive applications may like to **stream** the response to users.\n",
    "\n",
    "Run the code below to see how you can achieve this with Bedrock's `invoke_model_with_response_stream()` method - returning the response body in separate chunks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c69627e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import clear_output, display, display_markdown, Markdown\n",
    "\n",
    "body = json.dumps({\"inputText\": prompt_data})\n",
    "modelId = \"amazon.titan-tg1-large\"  # (Change this, and the request body, to try different models)\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "response = boto3_bedrock.invoke_model_with_response_stream(\n",
    "    body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    ")\n",
    "stream = response.get('body')\n",
    "output = []\n",
    "\n",
    "if stream:\n",
    "    for event in stream:\n",
    "        chunk = event.get('chunk')\n",
    "        if chunk:\n",
    "            chunk_obj = json.loads(chunk.get('bytes').decode())\n",
    "            text = chunk_obj['outputText']\n",
    "            clear_output(wait=True)\n",
    "            output.append(text)\n",
    "            display_markdown(Markdown(''.join(output)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ef3451d-b66a-4b11-a1ed-734bf9e7bbec",
   "metadata": {},
   "source": [
    "## Generate embeddings\n",
    "\n",
    "Use text embeddings to convert text into meaningful vector representations. You input a body of text \n",
    "and the output is a (1 x n) vector. You can use embedding vectors for a wide variety of applications. \n",
    "Bedrock currently offers Titan Embeddings for text embedding that supports text similarity (finding the \n",
    "semantic similarity between bodies of text) and text retrieval (such as search).\n",
    "\n",
    "At the time of writing you can use `amazon.titan-embed-g1-text-02` as embedding model via the API. The input text size is 8192 tokens and the output vector length is 1536.\n",
    "\n",
    "To use a text embeddings model, use the InvokeModel API operation or the Python SDK.\n",
    "Use InvokeModel to retrieve the vector representation of the input text from the specified model.\n",
    "\n",
    "\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"inputText\": \"<text>\"\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"embedding\": []\n",
    "}\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9645dbd8",
   "metadata": {},
   "source": [
    "Let's see how to generate embeddings of some text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1085cc56",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_data = \"Amazon Bedrock supports foundation models from industry-leading providers such as \\\n",
    "AI21 Labs, Anthropic, Stability AI, and Amazon. Choose the model that is best suited to achieving \\\n",
    "your unique goals.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c54b424",
   "metadata": {},
   "outputs": [],
   "source": [
    "body = json.dumps({\"inputText\": prompt_data})\n",
    "modelId = \"amazon.titan-embed-g1-text-02\"  # (Change this to try different embedding models)\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "response = boto3_bedrock.invoke_model(\n",
    "    body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    ")\n",
    "response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "embedding = response_body.get(\"embedding\")\n",
    "print(f\"The embedding vector has {len(embedding)} values\\n{embedding[0:3]+['...']+embedding[-3:]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a48a0e8-147d-4525-a6b2-68a09af1b2c4",
   "metadata": {},
   "source": [
    "## Next steps\n",
    "\n",
    "In this notebook we showed some basic examples of invoking Amazon Bedrock models using the AWS Python SDK. You're now ready to explore the other labs to dive deeper on different use-cases and patterns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f8bb76df-4e99-4ebe-a954-53992ad317dc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   }
  ],
  "kernelspec": {
   "display_name": "Python 3 (Data Science 2.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-38"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}