SAP · KavithaSiva · Oct 4, 2024 · Sep 20, 2024 · Sep 20, 2024 · Sep 20, 2024
diff --git a/README.md b/README.md
@@ -13,6 +13,7 @@ Integrate chat completion into your business applications with SAP Cloud SDK for
   - [@sap-ai-sdk/orchestration](#sap-ai-sdkorchestration)
   - [@sap-ai-sdk/langchain](#sap-ai-sdklangchain)
 - [SAP Cloud SDK for AI Sample Project](#sap-cloud-sdk-for-ai-sample-project)
+- [Local Testing](#local-testing)
 - [Support, Feedback, Contribution](#support-feedback-contribution)
 - [Security / Disclosure](#security--disclosure)
 - [Code of Conduct](#code-of-conduct)
@@ -32,6 +33,8 @@ This package incorporates generative AI [orchestration](https://help.sap.com/doc
 $ npm install @sap-ai-sdk/orchestration
 ```
 
+For details on orchestration client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/orchestration/README.md).
+
 ### @sap-ai-sdk/ai-api
 
 This package provides tools to manage your scenarios and workflows in SAP AI Core.
@@ -47,6 +50,8 @@ This package provides tools to manage your scenarios and workflows in SAP AI Cor
 $ npm install @sap-ai-sdk/ai-api
 ```
 
+For details on the client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md).
+
 ### @sap-ai-sdk/foundation-models
 
 This package incorporates generative AI foundation models into your AI activities in SAP AI Core and SAP AI Launchpad.
@@ -57,6 +62,8 @@ This package incorporates generative AI foundation models into your AI activitie
 $ npm install @sap-ai-sdk/foundation-models
 ```
 
+For details on foundation model clients, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/foundation-models/README.md).
+
 ### @sap-ai-sdk/langchain
 
 This package provides LangChain model clients, built on top of the foundation model clients of the SAP Cloud SDK for AI.
@@ -67,15 +74,27 @@ This package provides LangChain model clients, built on top of the foundation mo
 $ npm install @sap-ai-sdk/langchain
 ```
 
+For details on LangChain model client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/langchain/README.md).
+
 ## SAP Cloud SDK for AI Sample Project
 
-We have created a sample project demonstrating the different clients' usage of the SAP Cloud SDK for AI for TypeScript/JavaScript. The [project README](./sample-code/README.md) outlines the set-up needed to build and run it locally.
+We have created a sample project demonstrating the different clients' usage of the SAP Cloud SDK for AI for TypeScript/JavaScript. The [project README](https://github.com/SAP/ai-sdk-js/blob/main/sample-code/README.md) outlines the set-up needed to build and run it locally.
+
+## Local Testing
+
+To test SAP Cloud SDK for AI features locally during application development, follow these steps:
+
+- Download a service key for the AI Core service instance.
+- Set the downloaded service key as the `AICORE_SERVICE_KEY` environment variable in the local environment.
+
+The SDK parses the service key from the environment variable to interact with the AI Core service.
+This setup enables local testing of clients such as orchestration and OpenAI, provided that deployments for orchestration and OpenAI exist in SAP BTP.
 
 ## Support, Feedback, Contribution
 
 This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues).
 
-Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our [Contribution Guidelines](CONTRIBUTING.md).
+Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our [Contribution Guidelines](https://github.com/SAP/ai-sdk-js/blob/main/CONTRIBUTING.md).
 
 ## Security / Disclosure
 

diff --git a/packages/ai-api/README.md b/packages/ai-api/README.md
@@ -11,15 +11,17 @@ We maintain a list of [currently available and tested AI Core APIs](https://gith
 
 ## Table of Contents
 
-1. [Installation](#installation)
-2. [Prerequisites](#prerequisites)
-3. [Usage](#usage)
-   - [Create an Artifact](#create-an-artifact)
-   - [Create a Configuration](#create-a-configuration)
-   - [Create a Deployment](#create-a-deployment)
-   - [Delete a Deployment](#delete-a-deployment)
-4. [Support, Feedback, Contribution](#support-feedback-contribution)
-5. [License](#license)
+- [Installation](#installation)
+- [Version Management](#version-management)
+- [Prerequisites](#prerequisites)
+- [Usage](#usage)
+  - [Create an Artifact](#create-an-artifact)
+  - [Create a Configuration](#create-a-configuration)
+  - [Create a Deployment](#create-a-deployment)
+  - [Delete a Deployment](#delete-a-deployment)
+- [Local Testing](#local-testing)
+- [Support, Feedback, Contribution](#support-feedback-contribution)
+- [License](#license)
 
 ## Installation
 
@@ -169,6 +171,10 @@ async function modifyDeployment() {
 }
 ```
 
+## Local Testing
+
+For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing).
+
 ## Support, Feedback, Contribution
 
 This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues).

diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md
@@ -4,13 +4,14 @@ This package incorporates generative AI foundation models into your AI activitie
 
 ## Table of Contents
 
-- [Table of Contents](#table-of-contents)
 - [Installation](#installation)
 - [Prerequisites](#prerequisites)
+- [Relationship between Models and Deployment ID](#relationship-between-models-and-deployment-id)
 - [Usage](#usage)
   - [Client Initialization](#client-initialization)
   - [Chat Client](#chat-client)
   - [Embedding Client](#embedding-client)
+- [Local Testing](#local-testing)
 - [Support, Feedback, Contribution](#support-feedback-contribution)
 - [License](#license)
 
@@ -25,9 +26,21 @@ $ npm install @sap-ai-sdk/foundation-models
 - [Enable the AI Core service in BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup).
 - Project configured with Node.js v20 or higher and native ESM support enabled.
 - A deployed OpenAI model in SAP Generative AI hub.
-  - Use the [`DeploymentApi`](../ai-api/README.md#deploymentapi) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core).
+  - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub.
+    For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core).
     Deployment can be set up for each model and model version, as well as a resource group intended for use with the generative AI hub.
-  - Once a deployment is complete, the model can be accessed via the `deploymentUrl`
+  - Once a deployment is complete, the model can be accessed via the `deploymentUrl`.
+
+## Relationship between Models and Deployment ID
+
+SAP AI Core manages access to generative AI models through the global AI scenario `foundation-models`.
+Creating a deployment for a model requires access to this scenario.
+
+Each model, model version, and resource group allows for a one-time deployment.
-Each model, model version, and resource group allows for a one-time deployment.
+Deployment requires model name, model version and resource group.
-Each model, model version, and resource group allows for a one-time deployment.
+Deployment requires model name, model version and resource group.
+After deployment completion, the response includes a `deploymentUrl` and an `id`, which is the deployment ID. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core).
+[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant.
+
+Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario.
 
 ## Usage
 
@@ -49,8 +62,6 @@ const chatClient = new AzureOpenAiChatClient({ modelName: 'gpt-4o' });
 const embeddingClient = new AzureOpenAiEmbeddingClient({ modelName: 'gpt-4o' });
 ```
 
-[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant.
-
 The deployment ID and resource group can be used as an alternative to the model name for obtaining a model.
 
 ```ts
@@ -154,6 +165,10 @@ const embedding = response.getEmbedding();
 
 Define optional request configuration when running the client, similar to the chat client.
 
+## Local Testing
+
+For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing).
+
 ## Support, Feedback, Contribution
 
 This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues).

diff --git a/packages/langchain/README.md b/packages/langchain/README.md
@@ -4,14 +4,16 @@ This package provides LangChain model clients built on top of the foundation mod
 
 ## Table of Contents
 
-1. [Installation](#installation)
-2. [Prerequisites](#prerequisites)
-3. [Usage](#usage)
-   - [Client Initialization](#client-initialization)
-   - [Chat Client](#chat-client)
-   - [Embedding Client](#embedding-client)
-4. [Support, Feedback, Contribution](#support-feedback-contribution)
-5. [License](#license)
+- [Installation](#installation)
+- [Prerequisites](#prerequisites)
+- [Relationship between Models and Deployment ID](#relationship-between-models-and-deployment-id)
+- [Usage](#usage)
+  - [Client Initialization](#client-initialization)
+  - [Chat Client](#chat-client)
+  - [Embedding Client](#embedding-client)
+- [Local Testing](#local-testing)
+- [Support, Feedback, Contribution](#support-feedback-contribution)
+- [License](#license)
 
 ## Installation
 
@@ -24,10 +26,22 @@ $ npm install @sap-ai-sdk/langchain
 - [Enable the AI Core service in SAP BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup).
 - Bind the service to your application.
 - Ensure the project is configured with Node.js v20 or higher, along with native ESM support.
-- For testing your application locally:
-  - Download a service key for your AI Core service instance.
-  - Create a `.env` file in the root of your directory.
-  - Add an entry `AICORE_SERVICE_KEY='<content-of-service-key>'`.
+- A deployed model is available in SAP Generative AI hub.
+  - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub.
+    For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core).
+  - Once a deployment is complete, the model can be accessed via the `deploymentUrl`.
+
+## Relationship between Models and Deployment ID
+
+SAP AI Core manages access to generative AI models through the global AI scenario `foundation-models`.
+Creating a deployment for a model requires access to this scenario.
+
+Each model, model version, and resource group allows for a one-time deployment.
+After deployment completion, the response includes a `deploymentUrl` and an `id`, which is the deployment ID. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core).
+
+[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant.
+
+Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario.
 
 ## Usage
 
@@ -60,6 +74,9 @@ const chatClient = new AzureOpenAiChatClient({
 });
 ```
 
+**Do not pass a `deployment ID` to initialize the client.**
+For the LangChain model clients, initialization is done using the model name, model version and resource group.
+
 An important note is that LangChain clients by default attempt 6 retries with exponential backoff in case of a failure.
 Especially in testing environments you might want to reduce this number to speed up the process:
 
@@ -155,6 +172,10 @@ const vectorStore = await MemoryVectorStore.fromDocuments(
 const retriever = vectorStore.asRetriever();
 ```
 
+## Local Testing
+
+For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing).
+
 ## Support, Feedback, Contribution
 
 This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues).

diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md
@@ -2,16 +2,18 @@
 
 This package incorporates generative AI orchestration capabilities into your AI activities in SAP AI Core and SAP AI Launchpad.
 
-### Table of Contents
+## Table of Contents
 
 - [Installation](#installation)
-  - [Prerequisites](#prerequisites)
+- [Prerequisites](#prerequisites)
 - [Orchestration Service](#orchestration-service)
+- [Relationship between Orchestration and Resource Groups](#relationship-between-orchestration-and-resource-groups)
 - [Usage](#usage)
   - [Templating](#templating)
-    - [Token Usage](#token-usage)
   - [Content Filtering](#content-filtering)
-    - [Data Masking](#data-masking)
+  - [Data Masking](#data-masking)
+  - [Using Resource Groups](#using-resource-groups)
+- [Local Testing](#local-testing)
 - [Support, Feedback, Contribution](#support-feedback-contribution)
 - [License](#license)
 
@@ -21,12 +23,13 @@ This package incorporates generative AI orchestration capabilities into your AI
 $ npm install @sap-ai-sdk/orchestration
 ```
 
-### Prerequisites
+## Prerequisites
 
 - [Enable the AI Core service in SAP BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup).
 - Project configured with Node.js v20 or higher and native ESM support enabled.
 - An orchestration deployment is running.
-  - Use the [`DeploymentApi`](../ai-api/README.md#deploymentapi) from `@sap-ai-sdk/ai-api` to create a deployment for orchestration to the SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-orchestration).
+  - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to create a deployment for orchestration to the SAP generative AI hub.
+    For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-orchestration).
   - Once a deployment is complete, the orchestration service can be accessed via the `deploymentUrl`.
 
 ## Orchestration Service
@@ -38,6 +41,16 @@ The orchestration service provides essential features like templating and conten
 
 Find more details about orchestration workflow [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/orchestration-workflow).
 
+## Relationship between Orchestration and Resource Groups
+
+SAP AI Core manages access to orchestration of generative AI models through the global AI scenario `orchestration`.
+Creating a deployment for enabling orchestration capabilities requires access to this scenario.
+
+[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant.
+Each resource group allows for a one-time orchestration deployment.
+
+Consequently, each orchestration deployment uniquely maps to a resource group within the `orchestration` scenario.
+
 ## Usage
 
 Leverage the orchestration service capabilities by using the orchestration client.
@@ -99,6 +112,28 @@ const response = await orchestrationClient.chatCompletion(
 const responseContent = response.getContent();
 ```
 
+`getContent()` is a convenience method that parses the response and returns the model's output as a string.
+
+```ts
+const finishReason = response.getFinishReason();
+```
+
+`getFinishReason()` is a convenience method that specifies the reason for stopping the chat completion request.
+
+```ts
+const tokenUsage = response.getTokenUsage();
+
+console.log(
+  `Total tokens consumed by the request: ${tokenUsage.total_tokens}\n` +
+    `Input prompt tokens consumed: ${tokenUsage.prompt_tokens}\n` +
+    `Output text completion tokens consumed: ${tokenUsage.completion_tokens}\n`
+);
+```
+
+`getTokenUsage()` is a convenience method that retrieves the token usage details of the chat completion request.
+
+#### Passing a Message History
+
 It is possible to provide a history of a conversation to the model.
 Use the following snippet to send a chat completion request with history and a system message:
 
@@ -136,22 +171,6 @@ const response = await orchestrationClient.chatCompletion({
 const responseContent = response.getContent();
 ```
 
-#### Token Usage
-
-To retrieve the token usage details of the orchestration request, use the following snippet:
-
-```ts
-const tokenUsage = response.getTokenUsage();
-
-logger.info(
-  `Total tokens consumed by the request: ${tokenUsage.total_tokens}\n` +
-    `Input prompt tokens consumed: ${tokenUsage.prompt_tokens}\n` +
-    `Output text completion tokens consumed: ${tokenUsage.completion_tokens}\n`
-);
-```
-
-Remember to initialize a logger before using it.
-
 ### Content Filtering
 
 Use the orchestration client with filtering to restrict content that is passed to and received from a generative AI model.
@@ -189,11 +208,21 @@ try {
 }
 ```
 
+In the above code snippet, both `chatCompletion()` and `getContent()` methods can throw errors.
+
+- **Axios Errors**:  
+  When the chat completion request fails with a `400` status code, the caught error will be an `Axios` error. The property `error.response.data.message` may provide additional details about the failure's cause.
+
+- **Output Content Filtered**:  
+  The method `getContent()` can throw an error if the output filter filters the model output. This situation can occur even if the chat completion request succeeds with a `200` status code. The `error.message` property indicates whether the output was filtered.
+
+Therefore, handle errors appropriately to ensure meaningful feedback for both types of errors.
+
 `buildAzureContentFilter()` is a convenience function that creates an Azure content filter configuration based on the provided inputs.
 The Azure content filter supports four categories: `Hate`, `Violence`, `Sexual`, and `SelfHarm`.
 Each category can be configured with severity levels of 0, 2, 4, or 6.
 
-#### Data Masking
+### Data Masking
 
 You can anonymize or pseudonomize the prompt using the data masking capabilities of the orchestration service.
 
@@ -229,6 +258,31 @@ const response = await orchestrationClient.chatCompletion({
 return response.getContent();
 ```
 
+### Using Resource Groups
+
+The resource group can be used as an additional parameter to pick the right orchestration deployment.
+
+```ts
+const orchestrationClient = new OrchestrationClient(
+  {
+    llm: {
+      model_name: 'gpt-4-32k',
+      model_params: { max_tokens: 50, temperature: 0.1 }
+    },
+    templating: {
+      template: [{ role: 'user', content: 'What is my name?' }]
+    }
+  },
+  { resourceGroup: 'rg1234' }
+);
+```
+
+The relationship between orchestration and resource groups is explained in this [section](#relationship-between-orchestration-and-resource-groups).
+
+## Local Testing
+
+For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing).
+
 ## Support, Feedback, Contribution
 
 Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our [Contribution Guidelines](https://github.com/SAP/ai-sdk-js/blob/main/CONTRIBUTING.md).