A new wave of widespread AI adoption is on the way with generative AI,having the potential to reinvent every aspect of customer experiences and applications. Generative AI is powered by very large machine learning models that are pre-trained on vast amounts of data, commonly referred to as foundation models (FMs). Large Language Models are a subset of Foundation Models(FMs) which are trained on trillions of words and they learn the patterns in the language, allowing them to generate human-like responses to any query we give them. Additionally, foundation models are trained on very general domain corpora, making them less effective for domain-specific tasks. There lies the importance of RAG. You can use Retrieval Augmented Generation (RAG) to retrieve data from outside a foundation model and augment your prompts by adding the relevant retrieved data in context.
Text generation using RAG with LLMs enables you to generate domain-specific text outputs by supplying specific external data as part of the context fed to LLMs. With RAG, the external data used to augment your prompts can come from multiple data sources, such as a document repositories, databases, or APIs. The first step is to convert your documents and any user queries into a compatible format to perform relevancy search. To make the formats compatible, a document collection, or knowledge library, and user-submitted queries are converted to numerical representations using embedding language models. Embedding is the process by which text is given numerical representation in a vector space. RAG model architectures compare the embeddings of user queries within the vector of the knowledge library. The original user prompt is then appended with relevant context from similar documents within the knowledge library. This augmented prompt is then sent to the foundation model. You can update knowledge libraries and their relevant embeddings asynchronously.
Amazon Opensearch Serverless offers vector engine to store embeddings for faster similarity searches. The vector engine provides a simple, scalable, and high-performing similarity search capability in Amazon OpenSearch Serverless that makes it easy for you to build generative artificial intelligence (AI) applications without having to manage the underlying vector database infrastructure.
- Llama2-7B (Existing G5.2xlarge)
- Llama2-13B (G5.12xlarge)
- Llama2-70B (G5.48xlarge)
- Falcon-7B (G5.2xlarge)
- Falcon-40B (G5.12xlarge)
- Falcon-180B (p4de.24xlarge)
New UX/UI (13-Sept-2023): Index Sample Data across different domains. Support multiple-assistant behaviours (Normal/Pirate/Jarvis Assistant modes)
ImprovedVectorDB.mov
This solution demonstrates building a RAG (Retrieval Augmented Solution) with Amazon Opensearch Serverless Vector DB and Llama2 LLM, Falcon LLM
For this walkthrough, you should have the following prerequisites:
Familiarity with below Services.
GPU Instance of type ml.g5.2xlarge for endpoint usage
Supported Llama2 regions (us-east-1 , us-east-2 , us-west 2 , eu-west-1 , and ap-southeast-1)
- Search for the service IAM on the AWS Console and go the IAM Dashboard and click on “Users“ tab under ”Access Management” and Click on “Create User”
- Give a name to the IAM user and click “Next“
- Now Click on Attach Policies directly and Choose "AdminsitratorAccess" and click "Next"
- Now review the details and click on "Create User"
- Now we need to create credentials for this IAM. Go to "Users" tab again and you will see your new user listed over there. Now click on the username.
- Go to Security Credentials Tab and under "Access Keys" click on "Create Access key"
- In the window that appears choose the first option "Command line Interface" and click the checkbox at the bottom and click Next
8.Now the Tag is optional and you can leave this empty and click on Create Access Key
- Now click on Download .csv file to download the credentials and click on "Done". Now lets proceed to section 2
Section 2 - Deploy a RAG based Solution (The below commands should be executed in the region of deployment)
- Search for AWS Cloudshell. Configure your aws cli environment with the access/secret keys of the new admin user using the below command on AWS Cloudshell
aws configure
-
Git Clone the serverless-rag-demo repository from aws-samples
git clone https://github.com/aws-samples/serverless-rag-demo.git -
Go to the directory where we have the downloaded files.
cd serverless-rag-demo -
Fire the bash script that creates the RAG based solution. Pass the environment and region for deployment. environment can be dev,qa,sandbox. Region can be any of those supported by Amazon Opensearch Serverless refer
sh creator.sh dev us-east-1 -
Select the LLM you want to deploy (sh creator.sh dev us-east-1)
-
Total deployment takes around 40 minutes. Once the deployment is complete head to API Gateway. Search for API with name rag-llm-api-{env_name}. Get the invoke URL for the API
-
Invoke the Api Gateway URL that loads an html page for testing the RAG based solution as api-gateway-url/rag
- Do not forget to append "rag" at the end of the API-GW url
eg: https://xxxxxxx.execute-api.us-east-1.amazonaws.com/dev/rag
