Skip to content

Commit 4ec2a23

Browse files
M-ElsaeedM-Elsaeed
authored
Merge branch 'aws:main' into main
2 parents 67c2bfe + 574ac85 commit 4ec2a23

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

41 files changed

+1933
-0
lines changed

‎.github/workflows/aws-lambda-java-profiler.yml

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
name: Run integration tests for aws-lambda-java-profiler
2+
3+
on:
4+
pull_request:
5+
branches: [ '*' ]
6+
paths:
7+
- 'aws-lambda-java-profiler/**'
8+
- '.github/workflows/aws-lambda-java-profiler.yml'
9+
push:
10+
branches: ['*']
11+
paths:
12+
- 'aws-lambda-java-profiler/**'
13+
- '.github/workflows/aws-lambda-java-profiler.yml'
14+
15+
jobs:
16+
17+
build:
18+
runs-on: ubuntu-latest
19+
20+
permissions:
21+
id-token: write
22+
contents: read
23+
24+
steps:
25+
- uses: actions/checkout@v4
26+
27+
- name: Set up JDK
28+
uses: actions/setup-java@v4
29+
with:
30+
java-version: 21
31+
distribution: corretto
32+
33+
- name: Issue AWS credentials
34+
uses: aws-actions/configure-aws-credentials@v4
35+
with:
36+
aws-region: ${{ secrets.AWS_REGION_PROFILER_EXTENSION_INTEGRATION_TEST }}
37+
role-to-assume: ${{ secrets.AWS_ROLE_PROFILER_EXTENSION_INTEGRATION_TEST }}
38+
role-session-name: GitHubActionsRunIntegrationTests
39+
role-duration-seconds: 900
40+
41+
- name: Build layer
42+
working-directory: ./experimental/aws-lambda-java-profiler/extension
43+
run: ./build_layer.sh
44+
45+
- name: Publish layer
46+
working-directory: ./experimental/aws-lambda-java-profiler
47+
run: ./integration_tests/publish_layer.sh
48+
49+
- name: Create the bucket layer
50+
working-directory: ./experimental/aws-lambda-java-profiler
51+
run: ./integration_tests/create_bucket.sh
52+
53+
- name: Create Java function
54+
working-directory: ./experimental/aws-lambda-java-profiler
55+
run: ./integration_tests/create_function.sh
56+
57+
- name: Invoke Java function
58+
working-directory: ./experimental/aws-lambda-java-profiler
59+
run: ./integration_tests/invoke_function.sh
60+
61+
- name: Download from s3
62+
working-directory: ./experimental/aws-lambda-java-profiler
63+
run: ./integration_tests/download_from_s3.sh
64+
65+
- name: Upload profiles
66+
uses: actions/upload-artifact@v4
67+
with:
68+
name: profiles
69+
path: /tmp/s3-artifacts
70+
71+
- name: cleanup
72+
if: always()
73+
working-directory: ./experimental/aws-lambda-java-profiler
74+
run: ./integration_tests/cleanup.sh

‎.gitignore

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,3 +28,10 @@ dependency-reduced-pom.xml
2828

2929
# snapshot process
3030
aws-lambda-java-runtime-interface-client/pom.xml.versionsBackup
31+
32+
# profiler
33+
experimental/aws-lambda-java-profiler/integration_tests/helloworld/build
34+
experimental/aws-lambda-java-profiler/extension/build/
35+
experimental/aws-lambda-java-profiler/integration_tests/helloworld/bin
36+
!experimental/aws-lambda-java-profiler/extension/gradle/wrapper/*.jar
37+
/scratch/

‎README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,18 @@ See the [README](aws-lambda-java-log4j2/README.md) or the [official documentatio
139139
</dependency>
140140
```
141141

142+
## Lambda Profiler Extension for Java - aws-lambda-java-profiler
143+
144+
<p align="center">
145+
<img src="experimental/aws-lambda-java-profiler/docs/example-cold-start-flame-graph-small.png" alt="A flame graph of a Java Lambda function">
146+
</p>
147+
148+
This project allows you to profile your Java functions invoke by invoke, with high fidelity, and no code changes. It
149+
uses the [async-profiler](https://github.com/async-profiler/async-profiler) project to produce profiling data and
150+
automatically uploads the data as flame graphs to S3.
151+
152+
Follow our [Quick Start](experimental/aws-lambda-java-profiler#quick-start) to profile your functions.
153+
142154
## Java implementation of the Runtime Interface Client API - aws-lambda-java-runtime-interface-client
143155
[![Maven](https://img.shields.io/maven-central/v/com.amazonaws/aws-lambda-java-runtime-interface-client.svg?label=Maven)](https://central.sonatype.com/artifact/com.amazonaws/aws-lambda-java-runtime-interface-client)
144156

‎experimental/aws-lambda-java-profiler/.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
*.zip
2+
/.idea/
3+
/target/

‎experimental/aws-lambda-java-profiler/.mvn/wrapper/maven-wrapper.properties

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
wrapperVersion=3.3.2
18+
distributionType=only-script
19+
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.6/apache-maven-3.9.6-bin.zip

‎experimental/aws-lambda-java-profiler/README.md

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
<p align="center">
2+
<img src="docs/Arch_AWS-Lambda_64.svg" alt="AWS Lambda service icon">
3+
</p>
4+
5+
<h2 align="center">AWS Lambda Profiler Extension for Java</h2>
6+
7+
The Lambda profiler extension allows you to profile your Java functions invoke by invoke, with high fidelity, and no
8+
code changes. It uses the [async-profiler](https://github.com/async-profiler/async-profiler) project to produce
9+
profiling data and automatically uploads the data as HTML flame graphs to S3.
10+
11+
<p align="center">
12+
<img src="docs/example-cold-start-flame-graph.png" alt="A flame graph of a Java Lambda function">
13+
</p>
14+
15+
## Current status
16+
**This is an alpha release and not yet ready for production use.** We're especially interested in early feedback on usability, features, performance, and compatibility. Please send feedback by opening a [GitHub issue](https://github.com/aws/aws-lambda-java-libs/issues/new).
17+
18+
The profiler has been tested with Lambda managed runtimes for Java 17 and Java 21.
19+
20+
## How to use the Lambda Profiler
21+
22+
To use the profiler you need to
23+
24+
1. Build the extension in this repo
25+
2. Deploy it as a Lambda Layer and attach the layer to your function
26+
3. Create an S3 bucket for the results, or reuse an existing one
27+
4. Give your function permission to write to the bucket
28+
5. Configure the required environment variables.
29+
30+
The above assumes you're using the ZIP deployment method with managed runtimes. If you deploy your functions as container images instead, you will need to include the profiler in your Dockerfile at `/opt/extensions/` rather than using a Lambda layer.
31+
32+
### Quick Start
33+
34+
The following [Quick Start](#quick-start) gives AWS CLI commands you can run to get started (MacOS/Linux). There are also [examples](examples) using infrastructure as code for you to refer to.
35+
36+
1. Clone the repo
37+
38+
```bash
39+
git clone https://github.com/aws/aws-lambda-java-libs
40+
```
41+
42+
2. Build the extension
43+
44+
```bash
45+
cd aws-lambda-java-libs/experimental/aws-lambda-java-profiler/extension
46+
./build_layer.sh
47+
```
48+
49+
3. Run the `update-function.sh` script which will create a new S3 bucket, Lambda layer and all the configuration required.
50+
51+
```bash
52+
cd ..
53+
./update-function.sh YOUR_FUNCTION_NAME
54+
```
55+
56+
4. Invoke your function and review the flame graph in S3 using your browser.
57+
58+
### Configuration
59+
60+
#### Required Environment Variables
61+
62+
| Name | Value |
63+
|-----------------------------------------|-----------------------------------------------------------------------------------------------|
64+
| AWS_LAMBDA_PROFILER_RESULTS_BUCKET_NAME | Your unique bucket name |
65+
| JAVA_TOOL_OPTIONS | -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints -javaagent:/opt/profiler-extension.jar |
66+
67+
#### Optional Environment Variables
68+
69+
| Name | Default Value | Options |
70+
|------------------------------------------|-----------------------------------------------------------|--------------------------------|
71+
| AWS_LAMBDA_PROFILER_START_COMMAND | start,event=wall,interval=1us | |
72+
| AWS_LAMBDA_PROFILER_STOP_COMMAND | stop,file=%s,include=*AWSLambda.main,include=start_thread | file=%s is required |
73+
| AWS_LAMBDA_PROFILER_DEBUG | false | true - to enable debug logging |
74+
| AWS_LAMBDA_PROFILER_COMMUNICATION_PORT | 1234 | a valid port number |
75+
76+
### How does it work?
77+
78+
In `/src` is the code for a Java agent. It's entry point `AgentEntry.premain()` is executed as the runtime starts up.
79+
The environment variable `JAVA_TOOL_OPTIONS` is used to specify which `.jar` file the agent is in. The `MANIFEST.MF` file is used to specify the pre-main class.
80+
81+
When the agent is constructed, it starts the profiler and registers itself as a Lambda extension for `INVOKE` request.
82+
83+
A new thread is created to handle calling `/next` and uploading the results of the profiler to S3. The bucket to upload
84+
the result to is configurable using an environment variable.
85+
86+
### Troubleshooting
87+
88+
- Ensure the Lambda function execution role has the necessary permissions to write to the S3 bucket.
89+
- Verify that the environment variables are set correctly in your Lambda function configuration.
90+
- Check CloudWatch logs for any error messages from the extension.
91+
92+
## Contributing
93+
94+
Contributions to improve the Java profiler extension are welcome. Please see [CONTRIBUTING.md](../../CONTRIBUTING.md) for more information on how to report bugs or submit pull requests.
95+
96+
Issues or contributions to the [async-profiler](https://github.com/async-profiler/async-profiler) itself should be submitted to that project.
97+
98+
### Security
99+
100+
If you discover a potential security issue in this project we ask that you notify AWS Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public GitHub issue.
101+
102+
### Code of conduct
103+
104+
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). See [CODE_OF_CONDUCT.md](doc/CODE_OF_CONDUCT.md) for more details.
105+
106+
## License
107+
108+
This project is licensed under the [Apache 2.0](../../LICENSE) License. It uses the following projects:
109+
110+
- [async-profiler](https://github.com/async-profiler/async-profiler) (Apache 2.0 license)
111+
- [AWS SDK for Java 2.0](https://github.com/aws/aws-sdk-java-v2) (Apache 2.0 license)
112+
- Other libraries in this repository (Apache 2.0 license)
113+

‎experimental/aws-lambda-java-profiler/docs/Arch_AWS-Lambda_64.svg

Lines changed: 18 additions & 0 deletions
Loading

‎experimental/aws-lambda-java-profiler/docs/example-cold-start-flame-graph-small.png

531 KB
Loading

‎experimental/aws-lambda-java-profiler/docs/example-cold-start-flame-graph.png

704 KB
Loading

‎experimental/aws-lambda-java-profiler/examples/cdk/.gitignore

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
.classpath.txt
2+
target
3+
.classpath
4+
.project
5+
.idea
6+
.settings
7+
.vscode
8+
*.iml
9+
10+
# CDK asset staging directory
11+
.cdk.staging
12+
cdk.out
13+

0 commit comments

Comments
 (0)