Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor AiCoreDeployment #100

Merged
merged 96 commits into from
Oct 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
d4ba51c
Added query filters for Orchestration and OpenAI deployments
CharlesDuboisSAP Sep 6, 2024
163212e
Added deployment cache
CharlesDuboisSAP Sep 6, 2024
a481a9b
Added cache reset
CharlesDuboisSAP Sep 9, 2024
a728f07
Added unit tests
CharlesDuboisSAP Sep 17, 2024
fc01194
Fixed unit tests
CharlesDuboisSAP Sep 17, 2024
2b2ea95
PMD
CharlesDuboisSAP Sep 17, 2024
2d0cc6e
Refactor to fix tests
CharlesDuboisSAP Sep 17, 2024
14fe1d6
fix tests for real
CharlesDuboisSAP Sep 17, 2024
249e3a6
PMD + added clearCache and loadCache
CharlesDuboisSAP Sep 18, 2024
059bb09
PMD again
CharlesDuboisSAP Sep 18, 2024
18dce2a
intermediate
a-d Sep 25, 2024
c35a61b
ready-to-discuss state
a-d Sep 25, 2024
b64809c
Merge remote-tracking branch 'origin/main' into refactor-core
a-d Sep 25, 2024
678a69b
Fix merge; Format
a-d Sep 25, 2024
448a140
Add javadoc
a-d Sep 25, 2024
799cee6
Fix PMD warnings
a-d Sep 25, 2024
8672466
Initial change of draft
a-d Sep 26, 2024
6830b71
Initial API change
a-d Oct 1, 2024
ba7066f
Merge remote-tracking branch 'origin/main' into refactor-core3
a-d Oct 1, 2024
94e8c74
Fix merge conflicts
a-d Oct 1, 2024
8d179bf
Update readme
a-d Oct 1, 2024
d1e1291
Fix PMD
a-d Oct 1, 2024
5a1521c
Formatting
bot-sdk-js Oct 1, 2024
8cbba6a
Apply suggestions from code review
newtork Oct 1, 2024
e16895c
Defuse functions
a-d Oct 1, 2024
6d277cc
Merge remote-tracking branch 'origin/refactor-core3' into refactor-core3
a-d Oct 1, 2024
c5d84c6
Merge remote-tracking branch 'origin/main' into refactor-core3
a-d Oct 1, 2024
1b1a44a
Added query filters for Orchestration and OpenAI deployments
CharlesDuboisSAP Sep 6, 2024
f6da748
Added deployment cache
CharlesDuboisSAP Sep 6, 2024
8e2cd9f
Added cache reset
CharlesDuboisSAP Sep 9, 2024
7fa8398
Added unit tests
CharlesDuboisSAP Sep 17, 2024
7f6af65
Fixed unit tests
CharlesDuboisSAP Sep 17, 2024
7df86f1
PMD
CharlesDuboisSAP Sep 17, 2024
481d5b6
Refactor to fix tests
CharlesDuboisSAP Sep 17, 2024
b1df254
fix tests for real
CharlesDuboisSAP Sep 17, 2024
1d5c155
PMD + added clearCache and loadCache
CharlesDuboisSAP Sep 18, 2024
771a7b6
PMD again
CharlesDuboisSAP Sep 18, 2024
2fb5141
Merge remote-tracking branch 'origin/query-filter' into query-filter
CharlesDuboisSAP Oct 7, 2024
4162119
Merge remote-tracking branch 'origin/main' into refactor-core3
a-d Oct 7, 2024
8a45bb0
Update core/src/main/java/com/sap/ai/sdk/core/AiCoreServiceWithDeploy…
newtork Oct 7, 2024
4e138be
Make service binding accessor changeable, e.g. for future testing
a-d Oct 7, 2024
129409b
Rename methods and types; Fix merge
a-d Oct 7, 2024
d6d53bb
Merge remote-tracking branch 'origin/refactor-core3' into refactor-core3
a-d Oct 7, 2024
188e150
Formatting
bot-sdk-js Oct 7, 2024
62e0df0
Fix code, tests are working
a-d Oct 8, 2024
e24e4d6
Fix header provisioning
a-d Oct 8, 2024
6626456
Add tests; Move stuff around
a-d Oct 11, 2024
31a6090
Merge branch 'refactor-core3' into query-filter
CharlesDuboisSAP Oct 11, 2024
e217da9
WiP
CharlesDuboisSAP Oct 11, 2024
ba53051
Delete Core
CharlesDuboisSAP Oct 11, 2024
c696c03
Formatting
bot-sdk-js Oct 11, 2024
37db59f
make base class extensible
a-d Oct 14, 2024
583fd49
work in progress
a-d Oct 14, 2024
dc45282
Fixed tests
CharlesDuboisSAP Oct 15, 2024
6684e39
Single cache
CharlesDuboisSAP Oct 15, 2024
eb19ae5
Added clearCache test
CharlesDuboisSAP Oct 15, 2024
f61e5c8
refine work in progress
a-d Oct 15, 2024
d6e3f25
Fix tests
a-d Oct 15, 2024
3749513
Update test
a-d Oct 15, 2024
c76ad84
Add annotations
a-d Oct 15, 2024
701d92b
Format
a-d Oct 15, 2024
7ec29d7
Merge remote-tracking branch 'origin/main' into refactor-core3
a-d Oct 15, 2024
ab5a847
Merge branch 'refactor-core3' into query-filter
CharlesDuboisSAP Oct 15, 2024
a49d6c7
Format; JavaDoc
a-d Oct 15, 2024
fbc262e
Merge branch 'refs/heads/refactor-core3' into query-filter
CharlesDuboisSAP Oct 15, 2024
2991457
Fix tests
CharlesDuboisSAP Oct 15, 2024
4dbc804
Added resetCache
CharlesDuboisSAP Oct 15, 2024
f224ec6
Restructure code
a-d Oct 15, 2024
b13b9cd
Fix PMD
a-d Oct 15, 2024
0afd536
Fix PMD
a-d Oct 15, 2024
ff5fdc8
Merge branch 'refactor-core3' into query-filter
CharlesDuboisSAP Oct 15, 2024
1503338
Fix test
a-d Oct 15, 2024
7c98c72
Fix test
a-d Oct 15, 2024
042da74
Merge branch 'refs/heads/refactor-core3' into query-filter
CharlesDuboisSAP Oct 16, 2024
fbc7f5b
Merged
CharlesDuboisSAP Oct 16, 2024
fdd163a
spotless
CharlesDuboisSAP Oct 16, 2024
c8233cd
Reduced code
CharlesDuboisSAP Oct 16, 2024
cabee7a
Added throws
CharlesDuboisSAP Oct 16, 2024
20b8cda
Added API client to application startup
CharlesDuboisSAP Oct 16, 2024
350a555
Make DeploymentCache package private and an instance
CharlesDuboisSAP Oct 16, 2024
0982a74
Apply review comments
a-d Oct 16, 2024
9575109
Apply review comments
a-d Oct 16, 2024
2b5221c
Add assertion on AI-Client-Type header
a-d Oct 16, 2024
505ce61
Minor JavaDoc fix
a-d Oct 16, 2024
d4d98fc
Formatting
bot-sdk-js Oct 16, 2024
0c088a8
Merge branch 'refs/heads/refactor-core3' into query-filter
CharlesDuboisSAP Oct 16, 2024
c0af7db
Merged main
CharlesDuboisSAP Oct 16, 2024
bb2ae4c
Refactor AiCoreDeployment
CharlesDuboisSAP Oct 16, 2024
642f388
Formatting
bot-sdk-js Oct 16, 2024
329b182
Added Javadoc
CharlesDuboisSAP Oct 16, 2024
9f5b2cc
Added Javadoc
CharlesDuboisSAP Oct 16, 2024
dc597ee
AiCoreDeployment.destinationId is a supplier
CharlesDuboisSAP Oct 17, 2024
324d0c9
Changed cache to a Set
CharlesDuboisSAP Oct 17, 2024
80bace3
Merge branch 'query-filter' into refactor-core4
CharlesDuboisSAP Oct 17, 2024
37cb0b9
Merge branch 'main' into refactor-core4
CharlesDuboisSAP Oct 17, 2024
c72e450
Merge branch 'main' into refactor-core4
MatKuhr Oct 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 18 additions & 77 deletions core/src/main/java/com/sap/ai/sdk/core/AiCoreDeployment.java
Original file line number Diff line number Diff line change
@@ -1,103 +1,44 @@
package com.sap.ai.sdk.core;

import com.sap.cloud.sdk.cloudplatform.connectivity.DefaultHttpDestination;
import com.sap.cloud.sdk.cloudplatform.connectivity.Destination;
import com.sap.cloud.sdk.cloudplatform.connectivity.DestinationProperty;
import com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationAccessException;
import com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationNotFoundException;
import com.sap.cloud.sdk.services.openapi.apiclient.ApiClient;
import java.util.function.Function;
import java.util.function.Supplier;
import javax.annotation.Nonnull;
import lombok.AccessLevel;
import lombok.Getter;
import lombok.RequiredArgsConstructor;

/** Connectivity convenience methods for AI Core with deployment. */
@RequiredArgsConstructor(access = AccessLevel.PROTECTED)
@RequiredArgsConstructor(access = AccessLevel.PACKAGE)
public class AiCoreDeployment implements AiCoreDestination {
private final AiCoreService aiCoreService;

private static final String AI_RESOURCE_GROUP = "URL.headers.AI-Resource-Group";

// the delegating AI Core Service instance
@Nonnull private final AiCoreService service;

// the deployment id handler to be used, based on instance
@Nonnull private final Function<AiCoreDeployment, String> deploymentId;

// the resource group, "default" if null
@Getter(AccessLevel.PROTECTED)
@Nonnull
private final String resourceGroup;
@Nonnull private final Supplier<String> deploymentId;

/**
* Default constructor with "default" resource group.
* Set the resource group.
*
* @param service The AI Core service.
* @param deploymentId The deployment id handler.
* @param resourceGroup The resource group, default value "default".
* @return A new instance of the AI Core service.
*/
public AiCoreDeployment(
@Nonnull final AiCoreService service,
@Nonnull final Function<AiCoreDeployment, String> deploymentId) {
this(service, deploymentId, "default");
@Nonnull
public AiCoreDeployment withResourceGroup(@Nonnull final String resourceGroup) {
aiCoreService.resourceGroup = resourceGroup;
return this;
}

@Nonnull
@Override
public Destination destination() {
final var dest = service.baseDestinationHandler.apply(service);
final var builder = service.builderHandler.apply(service, dest);
destinationSetUrl(builder, dest);
destinationSetHeaders(builder);
return builder.build();
public Destination destination() throws DestinationAccessException, DestinationNotFoundException {
aiCoreService.deploymentId = deploymentId.get();
return aiCoreService.destination();
}

@Nonnull
@Override
public ApiClient client() {
final var destination = destination();
return service.clientHandler.apply(service, destination);
}

/**
* Update and set the URL for the destination.
*
* @param builder The destination builder.
* @param dest The original destination reference.
*/
protected void destinationSetUrl(
@Nonnull final DefaultHttpDestination.Builder builder, @Nonnull final Destination dest) {
String uri = dest.get(DestinationProperty.URI).get();
if (!uri.endsWith("/")) {
uri = uri + "/";
}
builder.uri(uri + "v2/inference/deployments/%s/".formatted(getDeploymentId()));
}

/**
* Update and set the default request headers for the destination.
*
* @param builder The destination builder.
*/
protected void destinationSetHeaders(@Nonnull final DefaultHttpDestination.Builder builder) {
builder.property(AI_RESOURCE_GROUP, getResourceGroup());
}

/**
* Set the resource group.
*
* @param resourceGroup The resource group.
* @return A new instance of the AI Core service.
*/
@Nonnull
public AiCoreDeployment withResourceGroup(@Nonnull final String resourceGroup) {
return new AiCoreDeployment(service, deploymentId, resourceGroup);
}

/**
* Get the deployment id.
*
* @return The deployment id.
*/
@Nonnull
protected String getDeploymentId() {
return deploymentId.apply(this);
aiCoreService.deploymentId = deploymentId.get();
return aiCoreService.client();
}
}
62 changes: 47 additions & 15 deletions core/src/main/java/com/sap/ai/sdk/core/AiCoreService.java
Original file line number Diff line number Diff line change
Expand Up @@ -32,18 +32,24 @@
@RequiredArgsConstructor
public class AiCoreService implements AiCoreDestination {

final Function<AiCoreService, Destination> baseDestinationHandler;
Function<AiCoreService, Destination> baseDestinationHandler;
final BiFunction<AiCoreService, Destination, ApiClient> clientHandler;
final BiFunction<AiCoreService, Destination, DefaultHttpDestination.Builder> builderHandler;

private static final DeploymentCache DEPLOYMENT_CACHE = new DeploymentCache();

private static final String AI_RESOURCE_GROUP = "URL.headers.AI-Resource-Group";

/** The resource group is defined by AiCoreDeployment.withResourceGroup(). */
@Nonnull String resourceGroup;

/** The deployment id is set by AiCoreDeployment.destination() or AiCoreDeployment.client(). */
@Nonnull String deploymentId;

/** The default constructor. */
public AiCoreService() {
this(
AiCoreService::getBaseDestination,
AiCoreService::getApiClient,
AiCoreService::getDestinationBuilder);
this(AiCoreService::getApiClient, AiCoreService::getDestinationBuilder, "default", "");
baseDestinationHandler = AiCoreService::getBaseDestination;
}

@Nonnull
Expand All @@ -57,18 +63,48 @@ public ApiClient client() {
@Override
public Destination destination() {
final var dest = baseDestinationHandler.apply(this);
return builderHandler.apply(this, dest).build();
final var builder = builderHandler.apply(this, dest);
if (!deploymentId.isEmpty()) {
destinationSetUrl(builder, dest);
destinationSetHeaders(builder);
}
return builder.build();
}

/**
* Update and set the URL for the destination.
*
* @param builder The destination builder.
* @param dest The original destination reference.
*/
protected void destinationSetUrl(
@Nonnull final DefaultHttpDestination.Builder builder, @Nonnull final Destination dest) {
String uri = dest.get(DestinationProperty.URI).get();
if (!uri.endsWith("/")) {
uri = uri + "/";
}
builder.uri(uri + "v2/inference/deployments/%s/".formatted(deploymentId));
}

/**
* Update and set the default request headers for the destination.
*
* @param builder The destination builder.
*/
protected void destinationSetHeaders(@Nonnull final DefaultHttpDestination.Builder builder) {
builder.property(AI_RESOURCE_GROUP, resourceGroup);
}

/**
* Set a specific base destination.
*
* @param destination The destination to be used for AI Core service calls.
* @return A new instance of the AI Core Service based on the provided destination.
* @return The AI Core Service based on the provided destination.
*/
@Nonnull
public AiCoreService withDestination(@Nonnull final Destination destination) {
return new AiCoreService((service) -> destination, clientHandler, builderHandler);
baseDestinationHandler = (service) -> destination;
return this;
}

/**
Expand All @@ -79,7 +115,7 @@ public AiCoreService withDestination(@Nonnull final Destination destination) {
*/
@Nonnull
public AiCoreDeployment forDeployment(@Nonnull final String deploymentId) {
return new AiCoreDeployment(this, obj -> deploymentId);
return new AiCoreDeployment(this, () -> deploymentId);
}

/**
Expand All @@ -95,9 +131,7 @@ public AiCoreDeployment forDeploymentByModel(@Nonnull final String modelName)
throws NoSuchElementException {
return new AiCoreDeployment(
this,
obj ->
DEPLOYMENT_CACHE.getDeploymentIdByModel(
this.client(), obj.getResourceGroup(), modelName));
() -> DEPLOYMENT_CACHE.getDeploymentIdByModel(this.client(), resourceGroup, modelName));
}

/**
Expand All @@ -113,9 +147,7 @@ public AiCoreDeployment forDeploymentByScenario(@Nonnull final String scenarioId
throws NoSuchElementException {
return new AiCoreDeployment(
this,
obj ->
DEPLOYMENT_CACHE.getDeploymentIdByScenario(
this.client(), obj.getResourceGroup(), scenarioId));
() -> DEPLOYMENT_CACHE.getDeploymentIdByScenario(client(), resourceGroup, scenarioId));
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ protected ApiClient getApiClient(@Nonnull Destination destination) {
final var destination = customServiceForDeployment.destination().asHttp();
assertThat(destination.getUri()).hasToString("https://ai/v2/inference/deployments/deployment/");

final var resourceGroup = customServiceForDeployment.getResourceGroup();
final var resourceGroup = customService.resourceGroup;
assertThat(resourceGroup).isEqualTo("group");
}
}