diff --git a/README.md b/README.md index 11ec50b9..26acfce1 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ Integrate chat completion into your business applications with SAP Cloud SDK for - [@sap-ai-sdk/orchestration](#sap-ai-sdkorchestration) - [@sap-ai-sdk/langchain](#sap-ai-sdklangchain) - [SAP Cloud SDK for AI Sample Project](#sap-cloud-sdk-for-ai-sample-project) +- [Local Testing](#local-testing) - [Support, Feedback, Contribution](#support-feedback-contribution) - [Security / Disclosure](#security--disclosure) - [Code of Conduct](#code-of-conduct) @@ -32,6 +33,8 @@ This package incorporates generative AI [orchestration](https://help.sap.com/doc $ npm install @sap-ai-sdk/orchestration ``` +For details on orchestration client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/orchestration/README.md). + ### @sap-ai-sdk/ai-api This package provides tools to manage your scenarios and workflows in SAP AI Core. @@ -47,6 +50,8 @@ This package provides tools to manage your scenarios and workflows in SAP AI Cor $ npm install @sap-ai-sdk/ai-api ``` +For details on the client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md). + ### @sap-ai-sdk/foundation-models This package incorporates generative AI foundation models into your AI activities in SAP AI Core and SAP AI Launchpad. @@ -57,6 +62,8 @@ This package incorporates generative AI foundation models into your AI activitie $ npm install @sap-ai-sdk/foundation-models ``` +For details on foundation model clients, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/foundation-models/README.md). + ### @sap-ai-sdk/langchain This package provides LangChain model clients, built on top of the foundation model clients of the SAP Cloud SDK for AI. @@ -67,15 +74,27 @@ This package provides LangChain model clients, built on top of the foundation mo $ npm install @sap-ai-sdk/langchain ``` +For details on LangChain model client, refer to this [document](https://github.com/SAP/ai-sdk-js/blob/main/packages/langchain/README.md). + ## SAP Cloud SDK for AI Sample Project -We have created a sample project demonstrating the different clients' usage of the SAP Cloud SDK for AI for TypeScript/JavaScript. The [project README](./sample-code/README.md) outlines the set-up needed to build and run it locally. +We have created a sample project demonstrating the different clients' usage of the SAP Cloud SDK for AI for TypeScript/JavaScript. The [project README](https://github.com/SAP/ai-sdk-js/blob/main/sample-code/README.md) outlines the set-up needed to build and run it locally. + +## Local Testing + +To test SAP Cloud SDK for AI features locally during application development, follow these steps: + +- Download a service key for the AI Core service instance. +- Set the downloaded service key as the `AICORE_SERVICE_KEY` environment variable in the local environment. + +The SDK parses the service key from the environment variable to interact with the AI Core service. +This setup enables local testing of clients such as orchestration and OpenAI, provided that deployments for orchestration and OpenAI exist in SAP BTP. ## Support, Feedback, Contribution This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues). -Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our [Contribution Guidelines](CONTRIBUTING.md). +Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our [Contribution Guidelines](https://github.com/SAP/ai-sdk-js/blob/main/CONTRIBUTING.md). ## Security / Disclosure diff --git a/packages/ai-api/README.md b/packages/ai-api/README.md index 31205b38..5e3c5d8b 100644 --- a/packages/ai-api/README.md +++ b/packages/ai-api/README.md @@ -11,15 +11,17 @@ We maintain a list of [currently available and tested AI Core APIs](https://gith ## Table of Contents -1. [Installation](#installation) -2. [Prerequisites](#prerequisites) -3. [Usage](#usage) - - [Create an Artifact](#create-an-artifact) - - [Create a Configuration](#create-a-configuration) - - [Create a Deployment](#create-a-deployment) - - [Delete a Deployment](#delete-a-deployment) -4. [Support, Feedback, Contribution](#support-feedback-contribution) -5. [License](#license) +- [Installation](#installation) +- [Version Management](#version-management) +- [Prerequisites](#prerequisites) +- [Usage](#usage) + - [Create an Artifact](#create-an-artifact) + - [Create a Configuration](#create-a-configuration) + - [Create a Deployment](#create-a-deployment) + - [Delete a Deployment](#delete-a-deployment) +- [Local Testing](#local-testing) +- [Support, Feedback, Contribution](#support-feedback-contribution) +- [License](#license) ## Installation @@ -169,6 +171,10 @@ async function modifyDeployment() { } ``` +## Local Testing + +For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). + ## Support, Feedback, Contribution This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues). diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index 6378b03b..fe6ecb8f 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -4,14 +4,15 @@ This package incorporates generative AI foundation models into your AI activitie ## Table of Contents -- [Table of Contents](#table-of-contents) - [Installation](#installation) - [Prerequisites](#prerequisites) +- [Relationship between Models and Deployment ID](#relationship-between-models-and-deployment-id) - [Usage](#usage) - [Client Initialization](#client-initialization) - [Chat Client](#chat-client) - [Embedding Client](#embedding-client) - [Custom Request Configuration](#custom-request-configuration) +- [Local Testing](#local-testing) - [Support, Feedback, Contribution](#support-feedback-contribution) - [License](#license) @@ -26,9 +27,21 @@ $ npm install @sap-ai-sdk/foundation-models - [Enable the AI Core service in BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). - Project configured with Node.js v20 or higher and native ESM support enabled. - A deployed OpenAI model in SAP Generative AI hub. - - Use the [`DeploymentApi`](../ai-api/README.md#deploymentapi) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. + For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). Deployment can be set up for each model and model version, as well as a resource group intended for use with the generative AI hub. - - Once a deployment is complete, the model can be accessed via the `deploymentUrl` + - Once a deployment is complete, the model can be accessed via the `deploymentUrl`. + +## Relationship between Models and Deployment ID + +SAP AI Core manages access to generative AI models through the global AI scenario `foundation-models`. +Creating a deployment for a model requires access to this scenario. + +Each model, model version, and resource group allows for a one-time deployment. +After deployment completion, the response includes a `deploymentUrl` and an `id`, which is the deployment ID. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). +[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. + +Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario. ## Usage @@ -50,8 +63,6 @@ const chatClient = new AzureOpenAiChatClient({ modelName: 'gpt-4o' }); const embeddingClient = new AzureOpenAiEmbeddingClient({ modelName: 'gpt-4o' }); ``` -[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. - The deployment ID and resource group can be used as an alternative to the model name for obtaining a model. ```ts @@ -161,6 +172,10 @@ const response = await client.run( ); ``` +## Local Testing + +For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). + ## Support, Feedback, Contribution This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues). diff --git a/packages/langchain/README.md b/packages/langchain/README.md index b55ca75e..3884e270 100644 --- a/packages/langchain/README.md +++ b/packages/langchain/README.md @@ -4,14 +4,16 @@ This package provides LangChain model clients built on top of the foundation mod ## Table of Contents -1. [Installation](#installation) -2. [Prerequisites](#prerequisites) -3. [Usage](#usage) - - [Client Initialization](#client-initialization) - - [Chat Client](#chat-client) - - [Embedding Client](#embedding-client) -4. [Support, Feedback, Contribution](#support-feedback-contribution) -5. [License](#license) +- [Installation](#installation) +- [Prerequisites](#prerequisites) +- [Relationship between Models and Deployment ID](#relationship-between-models-and-deployment-id) +- [Usage](#usage) + - [Client Initialization](#client-initialization) + - [Chat Client](#chat-client) + - [Embedding Client](#embedding-client) +- [Local Testing](#local-testing) +- [Support, Feedback, Contribution](#support-feedback-contribution) +- [License](#license) ## Installation @@ -24,10 +26,22 @@ $ npm install @sap-ai-sdk/langchain - [Enable the AI Core service in SAP BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). - Bind the service to your application. - Ensure the project is configured with Node.js v20 or higher, along with native ESM support. -- For testing your application locally: - - Download a service key for your AI Core service instance. - - Create a `.env` file in the root of your directory. - - Add an entry `AICORE_SERVICE_KEY=''`. +- A deployed model is available in SAP Generative AI hub. + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to deploy a model to SAP generative AI hub. + For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). + - Once a deployment is complete, the model can be accessed via the `deploymentUrl`. + +## Relationship between Models and Deployment ID + +SAP AI Core manages access to generative AI models through the global AI scenario `foundation-models`. +Creating a deployment for a model requires access to this scenario. + +Each model, model version, and resource group allows for a one-time deployment. +After deployment completion, the response includes a `deploymentUrl` and an `id`, which is the deployment ID. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). + +[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. + +Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario. ## Usage @@ -60,6 +74,9 @@ const chatClient = new AzureOpenAiChatClient({ }); ``` +**Do not pass a `deployment ID` to initialize the client.** +For the LangChain model clients, initialization is done using the model name, model version and resource group. + An important note is that LangChain clients by default attempt 6 retries with exponential backoff in case of a failure. Especially in testing environments you might want to reduce this number to speed up the process: @@ -155,6 +172,10 @@ const vectorStore = await MemoryVectorStore.fromDocuments( const retriever = vectorStore.asRetriever(); ``` +## Local Testing + +For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). + ## Support, Feedback, Contribution This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues). diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 37ca1231..ab3b76db 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -2,17 +2,19 @@ This package incorporates generative AI orchestration capabilities into your AI activities in SAP AI Core and SAP AI Launchpad. -### Table of Contents +## Table of Contents - [Installation](#installation) - - [Prerequisites](#prerequisites) +- [Prerequisites](#prerequisites) - [Orchestration Service](#orchestration-service) +- [Relationship between Orchestration and Resource Groups](#relationship-between-orchestration-and-resource-groups) - [Usage](#usage) - [Templating](#templating) - - [Token Usage](#token-usage) - [Content Filtering](#content-filtering) - - [Data Masking](#data-masking) + - [Data Masking](#data-masking) + - [Using Resource Groups](#using-resource-groups) - [Custom Request Configuration](#custom-request-configuration) +- [Local Testing](#local-testing) - [Support, Feedback, Contribution](#support-feedback-contribution) - [License](#license) @@ -22,12 +24,13 @@ This package incorporates generative AI orchestration capabilities into your AI $ npm install @sap-ai-sdk/orchestration ``` -### Prerequisites +## Prerequisites - [Enable the AI Core service in SAP BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). - Project configured with Node.js v20 or higher and native ESM support enabled. - An orchestration deployment is running. - - Use the [`DeploymentApi`](../ai-api/README.md#deploymentapi) from `@sap-ai-sdk/ai-api` to create a deployment for orchestration to the SAP generative AI hub. For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-orchestration). + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` to create a deployment for orchestration to the SAP generative AI hub. + For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-orchestration). - Once a deployment is complete, the orchestration service can be accessed via the `deploymentUrl`. ## Orchestration Service @@ -39,6 +42,16 @@ The orchestration service provides essential features like templating and conten Find more details about orchestration workflow [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/orchestration-workflow). +## Relationship between Orchestration and Resource Groups + +SAP AI Core manages access to orchestration of generative AI models through the global AI scenario `orchestration`. +Creating a deployment for enabling orchestration capabilities requires access to this scenario. + +[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. +Each resource group allows for a one-time orchestration deployment. + +Consequently, each orchestration deployment uniquely maps to a resource group within the `orchestration` scenario. + ## Usage Leverage the orchestration service capabilities by using the orchestration client. @@ -91,6 +104,28 @@ const response = await orchestrationClient.chatCompletion({ const responseContent = response.getContent(); ``` +`getContent()` is a convenience method that parses the response and returns the model's output as a string. + +To retrieve the `finish_reason` for stopping the chat completion request, use the convenience method `getFinishReason()`: + +```ts +const finishReason = response.getFinishReason(); +``` + +Use the `getTokenUsage()` convenience method to retrieve the token usage details of the chat completion request: + +```ts +const tokenUsage = response.getTokenUsage(); + +console.log( + `Total tokens consumed by the request: ${tokenUsage.total_tokens}\n` + + `Input prompt tokens consumed: ${tokenUsage.prompt_tokens}\n` + + `Output text completion tokens consumed: ${tokenUsage.completion_tokens}\n` +); +``` + +#### Passing a Message History + It is possible to provide a history of a conversation to the model. Use the following snippet to send a chat completion request with history and a system message: @@ -128,22 +163,6 @@ const response = await orchestrationClient.chatCompletion({ const responseContent = response.getContent(); ``` -#### Token Usage - -To retrieve the token usage details of the orchestration request, use the following snippet: - -```ts -const tokenUsage = response.getTokenUsage(); - -logger.info( - `Total tokens consumed by the request: ${tokenUsage.total_tokens}\n` + - `Input prompt tokens consumed: ${tokenUsage.prompt_tokens}\n` + - `Output text completion tokens consumed: ${tokenUsage.completion_tokens}\n` -); -``` - -Remember to initialize a logger before using it. - ### Content Filtering Use the orchestration client with filtering to restrict content that is passed to and received from a generative AI model. @@ -181,11 +200,24 @@ try { } ``` +Both `chatCompletion()` and `getContent()` methods can throw errors. + +- **Axios Errors**: + When the chat completion request fails with a `400` status code, the caught error will be an `Axios` error. + The property `error.response.data.message` may provide additional details about the failure's cause. + +- **Output Content Filtered**: + The method `getContent()` can throw an error if the output filter filters the model output. + This can occur even if the chat completion request responds with a `200` HTTP status code. + The `error.message` property indicates if the output was filtered. + +Therefore, handle errors appropriately to ensure meaningful feedback for both types of errors. + `buildAzureContentFilter()` is a convenience function that creates an Azure content filter configuration based on the provided inputs. The Azure content filter supports four categories: `Hate`, `Violence`, `Sexual`, and `SelfHarm`. Each category can be configured with severity levels of 0, 2, 4, or 6. -#### Data Masking +### Data Masking You can anonymize or pseudonomize the prompt using the data masking capabilities of the orchestration service. @@ -221,6 +253,27 @@ const response = await orchestrationClient.chatCompletion({ return response.getContent(); ``` +### Using Resource Groups + +The resource group can be used as an additional parameter to pick the right orchestration deployment. + +```ts +const orchestrationClient = new OrchestrationClient( + { + llm: { + model_name: 'gpt-4-32k', + model_params: { max_tokens: 50, temperature: 0.1 } + }, + templating: { + template: [{ role: 'user', content: 'What is my name?' }] + } + }, + { resourceGroup: 'rg1234' } +); +``` + +The relationship between orchestration and resource groups is explained in this [section](#relationship-between-orchestration-and-resource-groups). + ### Custom Request Configuration Set custom request configuration in the `requestConfig` parameter when calling the `chatCompletion()` method. @@ -243,6 +296,10 @@ const response = await orchestrationClient.chatCompletion( ); ``` +## Local Testing + +For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). + ## Support, Feedback, Contribution Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our [Contribution Guidelines](https://github.com/SAP/ai-sdk-js/blob/main/CONTRIBUTING.md).