Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Add smoke test #80

Merged
merged 36 commits into from
Aug 27, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
acba1f5
wip
marikaner Aug 14, 2024
edffb80
chore: add smoke test
marikaner Aug 19, 2024
2faa7df
add pr for testing
marikaner Aug 19, 2024
963635a
Merge branch 'main' of https://github.com/SAP/ai-sdk-js into smoke-test
marikaner Aug 19, 2024
261c7c0
add precompile
marikaner Aug 19, 2024
40c1688
fix typo
marikaner Aug 19, 2024
7d017f5
fsd
marikaner Aug 19, 2024
741fd31
again
marikaner Aug 19, 2024
081e37e
remove unnecessary command
marikaner Aug 19, 2024
e2c3198
fix deployment
marikaner Aug 19, 2024
1048876
add debug logs
marikaner Aug 19, 2024
cf25ac4
fix push
marikaner Aug 19, 2024
69dce1e
fix tests
marikaner Aug 19, 2024
fccf6ee
Update .github/workflows/smoke-test.yml
marikaner Aug 20, 2024
940a7fa
Update tests/smoke-test/.cfignore
marikaner Aug 20, 2024
d515925
fix config
marikaner Aug 20, 2024
d27d149
Merge branch 'smoke-test' of https://github.com/SAP/ai-sdk-js into sm…
marikaner Aug 20, 2024
bbd7a86
actually add ai core tests
marikaner Aug 20, 2024
789c1db
adjust configs
marikaner Aug 20, 2024
d0b4f09
fw
marikaner Aug 20, 2024
f1cc435
Add a readme
marikaner Aug 21, 2024
6e42668
make sure tests run with new version every day
marikaner Aug 22, 2024
257acf0
make script more generic
marikaner Aug 22, 2024
0f9b6cf
fix docs
marikaner Aug 23, 2024
504fe86
Add comment on local running
marikaner Aug 26, 2024
6fbb46b
Merge branch 'main' of https://github.com/SAP/ai-sdk-js into smoke-test
marikaner Aug 26, 2024
a5a7224
do not adjust lockfile
marikaner Aug 26, 2024
b4af5c3
Update tests/smoke-tests/README.md
marikaner Aug 27, 2024
02cb0b4
Update tests/smoke-tests/README.md
marikaner Aug 27, 2024
210aa93
Update tests/smoke-tests/README.md
marikaner Aug 27, 2024
133d2af
add accidentally removed script
marikaner Aug 27, 2024
1765ac2
fix tests
marikaner Aug 27, 2024
f8f86d5
Update tests/smoke-tests/package.json
marikaner Aug 27, 2024
d3c5ae4
Apply suggestions from code review
marikaner Aug 27, 2024
e293fca
Merge branch 'main' into smoke-test
tomfrenken Aug 27, 2024
4b426d6
fix: Changes from lint
Aug 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
name: "E2E Test"
name: e2e tests

on:
workflow_dispatch:
schedule:
- cron: 0 22 * * *

jobs:
e2e-test:
e2e-tests:
strategy:
fail-fast: false
matrix:
Expand All @@ -16,10 +17,10 @@ jobs:
include:
- environment: production
secret-name: AI_CORE_PRODUCTION
name: "Build and Test"
name: 'Build and Test'
runs-on: ubuntu-latest
steps:
- name: "Checkout repository"
- name: 'Checkout repository'
uses: actions/checkout@v4
- uses: pnpm/action-setup@v3
with:
Expand All @@ -37,7 +38,7 @@ jobs:
echo "AICORE_SERVICE_KEY='$AICORE_SERVICE_KEY'" > tests/e2e-tests/.env
url=$(echo "$AICORE_SERVICE_KEY" | jq -r '.serviceurls.AI_API_URL' | sed 's|^https://||')
echo "Using AI Core ${{ matrix.environment }} instance on $url"
- name: "Check Sample App Starts Up"
- name: 'Check Sample App Starts Up'
working-directory: ./sample-code
run: |
pnpm run local &
Expand All @@ -51,9 +52,9 @@ jobs:
fi
done
wget -qO- -S --content-on-error localhost:8080
- name: "Execute E2E Tests"
- name: 'Execute E2E Tests'
run: pnpm test:e2e
- name: "Slack Notification"
- name: 'Slack Notification'
if: failure()
uses: slackapi/[email protected]
with:
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/fosstars.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
name: 'Fosstars (Security)'

on:
workflow_dispatch:
schedule:
Expand Down
32 changes: 32 additions & 0 deletions .github/workflows/smoke-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: smoke test

on:
pull_request: ~
workflow_dispatch:
schedule:
- cron: 0 22 * * *

jobs:
smoke-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v3
with:
version: ${{ vars.PNPM_VERSION }}
- uses: actions/setup-node@v4
with:
node-version: ${{ vars.DEFAULT_NODE_VERSION }}
cache: 'pnpm'
- run: pnpm i --frozen-lockfile
- uses: vchrisb/setup-cf@v0
with:
api: ${{ vars.CF_API_URL }}
username: ${{ secrets.CF_USER }}
password: ${{ secrets.CF_PASSWORD }}
org: ${{ vars.CF_ORG }}
space: ${{ vars.CF_SPACE }}
- run: cf push
working-directory: tests/smoke-test
- name: smoke test
run: pnpm test:smoke
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,4 @@ build/Release
# VS Code
.vscode
/pnpm-publish-summary.json
/tests/smoke-test/deployment
10 changes: 7 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,18 @@
"postinstall": "pnpm compile",
"compile": "pnpm -r -w=false run compile",
"test:unit": "pnpm -r -F=./packages/** test",
"test:type": "pnpm -F=@sap-ai-sdk/type-tests test",
"test:e2e": "pnpm -F=@sap-ai-sdk/e2e-tests test",
"test:type": "pnpm type-tests test",
"test:e2e": "pnpm e2e-tests test",
"test:smoke": "pnpm smoke-test test",
"lint": "pnpm -r run lint",
"lint:fix": "pnpm -r run lint:fix",
"generate": "pnpm -r run generate",
"ai-core": "pnpm -F=@sap-ai-sdk/ai-core",
"gen-ai-hub": "pnpm -F=@sap-ai-sdk/gen-ai-hub",
"core": "pnpm -F=@sap-ai-sdk/core"
"core": "pnpm -F=@sap-ai-sdk/core",
MatKuhr marked this conversation as resolved.
Show resolved Hide resolved
"e2e-tests": "pnpm -F=@sap-ai-sdk/e2e-tests",
"type-tests": "pnpm -F=@sap-ai-sdk/type-tests",
"smoke-test": "pnpm -F=@sap-ai-sdk/smoke-test"
},
"devDependencies": {
"@changesets/cli": "^2.27.7",
Expand Down
101 changes: 84 additions & 17 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 1 addition & 5 deletions pnpm-workspace.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,7 @@ packages:
- 'packages/ai-core'
- 'packages/gen-ai-hub'
- 'packages/core'
# sample code
- 'sample-code'
# e2e tests
- 'tests/e2e-tests'
# type tests
- 'tests/type-tests'
# exclude packages that are inside test directories

- 'tests/smoke-test'
23 changes: 23 additions & 0 deletions sample-code/src/orchestration.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import {
type CompletionPostResponse,
GenAiHubClient
} from '@sap-ai-sdk/gen-ai-hub';

/**
* Ask GPT about the capital of France.
* @returns The answer from the orchestration service in Gen AI Hub.
*/
export function orchestrationCompletion(): Promise<CompletionPostResponse> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[q] We could consider calling this from the e2e test, similar to how we do it for OpenAI. That way we gain additional confidence this code actually works 😉

const genAiHubClient = new GenAiHubClient();

return genAiHubClient.chatCompletion({
deploymentConfiguration: { deploymentId: 'db1d64d9f06be467' },
llmConfig: {
model_name: 'gpt-4-32k',
model_params: {}
},
prompt: {
template: [{ role: 'user', content: 'What is the capital of France?' }]
}
});
}
12 changes: 10 additions & 2 deletions sample-code/src/server.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
/* eslint-disable no-console */
import express from 'express';
import { chatCompletion, computeEmbedding } from './aiservice.js';
import { orchestrationCompletion } from './orchestration.js';

const app = express();
const port = 8080;
Expand Down Expand Up @@ -35,8 +36,15 @@ app.get('/embedding', async (req, res) => {
}
});

app.get('/orchestration', (req, res) => {
res.status(418).send('Not implemented 🛠️');
app.get('/orchestration', async (req, res) => {
try {
res.send(await orchestrationCompletion());
} catch (error: any) {
console.error(error);
res
.status(500)
.send('Yikes, vibes are off apparently 😬 -> ' + error.message);
}
});

app.listen(port, () => {
Expand Down
4 changes: 4 additions & 0 deletions tests/smoke-test/.cfignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
coverage/
jest.config.mjs
tsconfig.json
.env
marikaner marked this conversation as resolved.
Show resolved Hide resolved
marikaner marked this conversation as resolved.
Show resolved Hide resolved
5 changes: 5 additions & 0 deletions tests/smoke-test/jest.config.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import config from '../../jest.config.mjs';
export default {
...config,
marikaner marked this conversation as resolved.
Show resolved Hide resolved
displayName: 'smoke-test',
};
10 changes: 10 additions & 0 deletions tests/smoke-test/manifest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
applications:
- name: smoke-test-app
path: .
memory: 256M
buildpacks:
- nodejs_buildpack
services:
- default_aicore
routes:
- route: 'smoke-test-app.cfapps.eu12-001.hana.ondemand.com'
Loading
Loading