-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add CDK and application code to main (#7)
* Initial commit for vectorizing data streams. Co-authored-by: Vaibhavi Sobti <[email protected]> Co-authored-by: Melissa Wang <[email protected]>
- Loading branch information
1 parent
e04430e
commit ce56946
Showing
104 changed files
with
10,198 additions
and
12 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
name: Build and Release | ||
|
||
on: | ||
push: | ||
branches: [main] | ||
pull_request: | ||
branches: [main] | ||
release: | ||
types: [created] | ||
|
||
jobs: | ||
build-and-release: | ||
name: Build and Release | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Install NodeJS | ||
uses: actions/setup-node@v3 | ||
with: | ||
node-version: v18.16.1 | ||
- name: Install Java | ||
uses: actions/setup-java@v3 | ||
with: | ||
distribution: corretto | ||
java-version: 11 | ||
- name: Install AWS CDK | ||
run: | | ||
npm install -g aws-cdk | ||
- name: Checkout source | ||
uses: actions/checkout@v2 | ||
|
||
- name: Create artifacts directory | ||
run: | | ||
mkdir artifacts | ||
- name: Build data-stream-vectorization JAR | ||
run: | | ||
mvn package | ||
cp target/data-stream-vectorization-1.0-SNAPSHOT.jar artifacts/ | ||
- name: Build CDK template | ||
working-directory: cdk-infra | ||
run: | | ||
cd shared | ||
npm install | ||
cd ../apps/real-time-vector-embedding-application/msk-to-bedrock-to-opensearch/cdk-infra | ||
npm install | ||
cdk synth --json | ||
cp cdk.out/BootstrapCdkStack.template.json ../../../../../artifacts/ | ||
- name: List artifacts | ||
working-directory: artifacts | ||
run: | | ||
ls -l | ||
- name: Publish artifacts | ||
if: github.event_name == 'release' | ||
uses: skx/github-action-publish-binaries@master | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
with: | ||
args: "artifacts/*" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
.gradle/ | ||
.idea/ | ||
/build | ||
/buildSrc/build/ | ||
/gradle/ | ||
/gradlew | ||
/gradlew.bat | ||
/wrapper/ | ||
/target |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,17 +1,14 @@ | ||
## My Project | ||
<div style="text-align: center"> | ||
<h1>Real-time Vector Embedding Blueprint</h1> | ||
|
||
TODO: Fill this README out! | ||
Real-time Vector Embedding Blueprint is an Amazon Managed Service for Apache Flink (MSF) blueprint which deploys an MSF app and other needed infrastructure for vectorizing incoming stream data and persisting the vectorized data in a vector DB. The MSF app consumes from an Amazon MSK cluster, creates embeddings of these messages with a supported Amazon Bedrock model, and stores the embeddings to an Amazon OpenSearch domain or collection. | ||
</div> | ||
|
||
Be sure to: | ||
## Get started with Real-time Vector Embedding | ||
|
||
* Change the title in this README | ||
* Edit your repository description on GitHub | ||
### Installation | ||
|
||
## Security | ||
|
||
See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. | ||
|
||
## License | ||
|
||
This project is licensed under the Apache-2.0 License. | ||
Follow the installation instructions [here](cdk-infra/notes/installation.md) to install the shared libraries and begin developing. | ||
|
||
### Deploying | ||
Follow the steps [here](cdk-infra/apps/real-time-vector-embedding-application/msk-to-bedrock-to-opensearch/README.md) to build, deploy, and run the application. |
Oops, something went wrong.