Thanks for taking the time to help improve the AWS Toolkit! We greatly value feedback and contributions from the community.
Reviewing this document will maximize your success in working with the codebase and sending pull requests.
This project is set up as a typescript monorepo. The documentation throughout this project
is referring to the subprojects packages/toolkit/
and packages/core/
.
See arch_develop.md to understand the
structure of this package before contributing.
If you're looking for ideas about where to contribute, consider good first issue issues.
To develop this project, install these dependencies:
- Visual Studio Code
- NodeJS and NPM (latest version of both)
- Typescript
- Git
- (optional) Set
git blame
to ignore noise-commits:git config blame.ignoreRevsFile .git-blame-ignore-revs
- (optional) Set
- AWS
git secrets
- TypeScript + Webpack Problem Matcher
- Not installing will result in the following error during building:
Error: Invalid problemMatcher reference: $ts-webpack-watch
- Not installing will result in the following error during building:
- (optional) AWS SAM CLI
- (optional) Docker
Then clone the repository and install NPM packages:
git clone [email protected]:aws/aws-toolkit-vscode.git
cd aws-toolkit-vscode
npm install
Due to the monorepo structure of the project, you must have the aws-toolkit-vscode/packages/toolkit
folder open as root folder in the workspace.
The easiest way to open the project: File > Open Workspace from File > choose aws-toolkit-vscode/aws-toolkit-vscode.code-workspace
To run the extension from VSCode as a Node.js app:
- Select the Run panel from the sidebar.
- From the dropdown at the top of the Run pane, choose
Extension
. - Press
F5
to launch a new instance of VSCode with the extension installed and the debugger attached.
To run the extension from VSCode in "web mode" (a browser app, or "PWA"):
- Select the Run panel from the sidebar.
- From the dropdown at the top of the Run pane, choose
Extension (web)
. - Press
F5
to launch a new instance of VSCode (web mode) with the extension installed and the debugger attached.
When you launch the extension or run tests from VSCode, it will automatically build the extension and watch for changes.
You can also use these NPM tasks (see npm run
for the full list):
- To build once:
npm run compile
- To build and watch for file changes:
npm run watch
- To build a release artifact (VSIX):
npm run package
- This uses webpack which may exhaust the default Node heap size on Linux.
To fix this set
--max-old-space-size
:export NODE_OPTIONS=--max-old-space-size=8192
- This uses webpack which may exhaust the default Node heap size on Linux.
To fix this set
- To build a "debug" VSIX artifact (faster and does not minify):
npm run package -- --debug
- Architecture: arch_overview.md
- Project patterns and practices: CODE_GUIDELINES.md
- VS Code Extension Guidelines
- VS Code API Documentation
- VS Code Extension Capabilities
- CI automatically publishes GitHub prereleases
for
master
andfeature/x
branches, including.vsix
artifacts which can be used to test the latest build for that branch. Each prerelease and its artifact are continually updated from the HEAD of its branch. - PR artifacts: each pull request is processed by an AWS CodeBuild job which runs all tests and provides the build result via the Details link as shown below.
- Check for recent changes in each of these projects:
- https://github.com/microsoft/vscode-python (releases)
- https://github.com/aws/aws-sam-cli/releases
- https://github.com/aws/aws-sam-cli-app-templates/ (
master
branch, not releases!)
-
VSCode extensions have a 100MB file size limit.
-
src/testFixtures/
is excluded in.vscode/settings.json
, to prevent VSCode from treating its files as project files. -
The codebase provides globals, which must be used instead of some common javascript globals. In particular, clock-related things like
Date
andsetTimeout
must not be used directly, instead useglobals.clock.Date
andglobals.clock.setTimeout
. #2343 -
VSCode extension examples: https://github.com/microsoft/vscode-extension-samples
-
Tests
- Use
function ()
andasync function ()
syntax fordescribe()
andit()
callbacks instead of arrow functions. - Do NOT include any
await
functions indescribe()
blocks directly (usage inbefore
,beforeEach
,after
,afterEach
, andit
blocks is fine).await
indescribe()
causes the framework to always evaluate thedescribe
block and can cause issues with either tests not running or always running (if other tests are marked with.only
)- Tests that require a premade value from a Promise should initialize the value as a
let
and make theawait
ed assignment inbefore()
.
- Use
-
How to debug unresolved promise rejections:
- Declare a global unhandledRejection handler.
process.on('unhandledRejection', (e) => { getLogger().error( localize( 'AWS.channel.aws.toolkit.activation.error', 'Error Activating {0} Toolkit: {1}', getIdeProperties().company, (e as Error).message ) ) if (e !== undefined) { throw e } })
- Put a breakpoint on it.
- Run all tests.
- Declare a global unhandledRejection handler.
The AWS Toolkit VSCode extension has a support (with limited functionality) for running in a web browser, eg vscode.dev.
See web.md for working with the web mode implementation of the extension.
See TESTPLAN.md to understand the project's test structure, mechanics and philosophy.
You can run tests directly from VSCode. Due to the monorepo structure of the project, you must have the aws-toolkit-vscode/packages/toolkit
folder open as root folder in the workspace.
The easiest way to open the project: File > Open Workspace from File > choose aws-toolkit-vscode/aws-toolkit-vscode.code-workspace
- Select
View > Debug
, or select the Debug pane from the sidebar. - From the dropdown at the top of the Debug pane, select the
Extension Tests
configuration. - Press
F5
to run tests with the debugger attached.
You can also run tests from the command line:
npm run test
npm run testInteg
Tests will write logs to ./.test-reports/testLog.log
.
To run a single test in VSCode, do any one of:
-
Run the Extension Tests (current file) launch-config.
-
Use Mocha's it.only() or
describe.only()
. -
Run in your terminal:
- Unix/macOS/POSIX shell:
TEST_FILE=../core/src/test/foo.test.ts npm run test
- Powershell:
$Env:TEST_FILE = "../core/src/test/foo.test.ts"; npm run test
- Unix/macOS/POSIX shell:
-
To run all tests in a particular subdirectory, you can edit
src/test/index.ts:rootTestsPath
to point to a subdirectory:rootTestsPath: __dirname + '/shared/sam/debugger/'
To run tests against a specific folder in VSCode, do any one of:
- Add the TEST_DIR environment variable to one of the testing launch configs and run it
- Run in your terminal
- Unix/macOS/POSIX shell:
TEST_DIR=../core/src/test/foo npm run test
- Powershell:
$Env:TEST_DIR = "../core/src/test/foo"; npm run test
- Unix/macOS/POSIX shell:
If the "Copy-Paste Detection" CI job fails, you will find it useful to check things locally. To check a specific file:
npx jscpd --config .github/workflows/jscpd.json --pattern packages/…/src/foo.ts
See the jscpd cli documentation for more options.
You can find the coverage report at ./coverage/amazonq/lcov-report/index.html
and ./coverage/toolkit/lcov-report/index.html
after running the tests. Tests ran from the workspace launch config won't generate a coverage report automatically because it can break file watching.
You can find documentation to create VSCode IDE settings for CodeCatalyst blueprints at docs/vscode-config.md.
Before sending a pull request:
- Treat all work as PUBLIC. Private
feature/x
branches will not be squash-merged at release time. This has several benefits:- Avoids mistakes (accidental exposure to public)!
- Avoids needing to erase (squash-merge) history.
- Check that you are working against the latest source on the
master
branch. - Check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
- Open an issue to discuss any significant work.
To send a pull request:
- Fork the repository.
- Modify the source; focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
- Read the project guidelines, this is very important for non-trivial changes.
- Commit to your fork using clear commit messages.
- Update the changelog.
- Create a pull request.
- Pay attention to any CI failures reported in the pull request.
Pull requests that change customer-impacting behavior must include a changelog item(s). Run one or both of the following commands:
- For changes relevant to Amazon Q:
npm run newChange -w packages/amazonq
- For changes relevant to AWS Toolkit:
npm run newChange -w packages/toolkit
The audience for the changelog is the user. The changelog is presented to users by VSCode and the marketplace. It is a "micro-blog" for advertising improvements to users. It is the primary way of communicating changes to customers. Please consider this when writing changelog entries.
Mentioning low-level details like "function x now takes argument y", will not be useful, because it doesn't say what that means in terms of the user experience. Instead, describe the effect from the user's point of view.
Tip
- Describe the change in a way that is meaningful to the customer. If you can't describe the customer impact then it probably shouldn't be in the changelog.
- ✅
Connection wizard sometimes shows the old (stale) connection
- ✅
Faster startup after VSCode restarts
- ❌
Remove the cache when the connection wizard is re-launched
(code internals are not relevant to customers) - ❌
Update telemetry definitions
(not customer-impacting)
- ✅
- "Bug Fix" changes should describe the problem being fixed. Don't say "Fixed" in the
description, it's redundant. Example:
- ❌
Fixed S3 bug which caused filenames to be uppercase
- ✅
S3 filenames are always uppercase
- ❌
- To update an existing changelog item, just edit its
.changes/next-release/….json
file, you don't need to re-runnpm run newChange
. - If there are multiple unrelated changes, run
npm run newChange
for each change. - Include the feature that the change affects, Q, CodeWhisperer, etc.
The title of your pull request must follow this format (checked by lintcommit.js):
- format:
type(scope): subject...
- type: must be a valid type (
build
,ci
,config
,deps
,docs
,feat
,fix
,perf
,refactor
,style
,telemetry
,test
,types
)- see lintcommit.js)
- "chore" is intentionally rejected because it tends to be over-used.
- user-facing changes should always choose "feat" or "fix", and include a changelog item.
- scope: lowercase, <30 chars
- subject: must be <100 chars
Your PR description should provide a brief "Problem" and "Solution" pair. This structure often gives much more clarity, more concisely, than a typical paragraph of explanation.
Problem:
Foo does nothing when user clicks it.
Solution:
- Listen to the click event.
- Emit telemetry on success/failure.
Good explanations are acts of creativity. The "tiny subject line" constraint reminds you to clarify the essence of the commit, and makes the log easy for humans to scan. The commit log is an artifact that will outlive most code.
Source control (Git) is our source-of-truth, not GitHub. However since most PRs are squash-merged, it's most important that your pull request description is well-formed so that the merged commit has the relevant info.
If you expect your commits to be preserved ("regular merge"), then follow these guidelines:
- Subject: single line up to 50-72 characters
- Imperative voice ("Fix bug", not "Fixed"/"Fixes"/"Fixing").
- Formatted as
type(scope): subject...
.- Helps humans and scripts scan and omit ranges of the history at a glance.
- Body: describe the change as a Problem/Solution pair.
Besides the typical develop/test/run cycle describe above, there are some tools for special cases such as build tasks, generating telemetry, generating SDKs, etc.
The DevSettings class defines various developer-only settings that change the behavior of the
Toolkit for testing and development purposes. To use a setting just add it to
your settings.json
. At runtime, if the Toolkit reads any of these settings,
the "AWS" statusbar item will change its color.
The aws.dev.forceDevMode
setting enables or disables Toolkit "dev mode". Without this setting, the presence of any other aws.dev.*
setting defined in DevSettings
implicitly enables "dev mode".
- Use
getLogger()
to log debugging messages, warnings, etc.- Example:
getLogger().error('topic: widget failed: %O', { foo: 'bar', baz: 42 })
- Example:
- Log messages are written to the extension Output channel, which you can view in vscode by visiting the "Output" panel and selecting
AWS Toolkit Logs
orAmazon Q Logs
. - Use the
aws.dev.logfile
setting to set the logfile path to a fixed location, so you can follow and filter logs using shell tools liketail
andgrep
.- Note: this always logs at debug log-level (though you can temporarily override that from the
AWS Toolkit Logs
UI). - Example
settings.json
:then you can tail the logfile in your terminal:"aws.dev.logfile": "~/awstoolkit.log",
tail -F ~/awstoolkit.log
- Note: this always logs at debug log-level (though you can temporarily override that from the
- Use the
AWS (Developer): Watch Logs
command to watch and filter Toolkit logs (including telemetry) in VSCode.
How to enable more detailed debug logs in the extensions. If you need to report an issue attach these to give the most detailed information.
- Open the Command Palette (
cmd/ctrl
+shift
+p
), then search for "View Logs". Choose the correct option for the extension you want, eg:AWS: View Logs
orAmazon Q: View Logs
- Click the gear icon on the bottom right and select
Debug
- Click the gear icon again and select
Set As Default
. This will ensure we stay inDebug
until explicitly changed - Open the Command Palette again and select
Reload Window
. - Now you should see additional
[debug]
prefixed logs in the output.
- See docs/telemetry.md for guidelines on developing telemetry in this project.
- To watch Toolkit telemetry events, use the
AWS (Developer): Watch Logs
command (see Logging above) and enter "telemetry" in the Debug Console filter box.
Endpoint overrides can be set per-service using the aws.dev.endpoints
settings. This is a JSON object where each key is the service ID (case-insensitive) and each value is the endpoint. Refer to the SDK API models to find relevant service IDs.
Example:
"aws.dev.endpoints": {
"s3": "http://example.com"
}
Example:
"aws.dev.codecatalystService": {
"region": "us-west-2",
"endpoint": "https://codecatalyst-gamma.example.com",
"hostname": "integ.stage.example.com",
"gitHostname": "git.gamma.source.example.com",
}
Example:
"aws.dev.codewhispererService": {
"region": "us-west-2",
"endpoint": "https://codewhisperer-gamma.example.com"
}
Environment variables can be used to modify the behaviour of VSCode. The following are environment variables that can be used to configure the extension:
AWS_ACCESS_KEY_ID
: The AWS access key associated with an IAM account. If defined, this environment variable overrides the value for the profile setting aws_access_key_id. For more information see environment variables to configure the AWS CLIAWS_SECRET_ACCESS_KEY
: The secret key associated with the access key. This is essentially the "password" for the access key. If defined, this environment variable overrides the value for the profile setting aws_secret_access_key. For more information see environment variables to configure the AWS CLIAWS_REGION
: The AWS Region to send the request to. If defined, this environment variable overrides the values in the environment variable AWS_DEFAULT_REGION and the profile setting region. For more information see environment variables to configure the AWS CLIAWS_SDK_LOAD_CONFIG
: Controls how the AWS SDK for javascript loads it's configuration when initialized. If the AWS_SDK_LOAD_CONFIG environment variable has been set to a truthy value, the SDK for JavaScript automatically searches for a config file when it loads. For more information see the shared config file documentationAWS_SHARED_CREDENTIALS_FILE
: The location of the file that the AWS CLI uses to store access keys. The default path is~/.aws/credentials
AWS_CONFIG_FILE
: The location of the file that the AWS CLI uses to store configuration profiles. The default path is~/.aws/config
HOME
: The home directory location for the current user in Linux and other Unix-like operating systems.SSH_AUTH_SOCK
: The location of a UNIX domain socket used by ssh-agent and SSH clients for agent-based authenticationUSERPROFILE
: The absolute path to the profile folder for the current user in WindowsHOMEPATH
: The path to the home directory for the current user in Windows, without including the drive letterPROGRAMFILES/PROGRAMFILES(X86)
: The default installation directory for WindowsWINDIR
: The location of the Windows installation directoryPATH
: The set of directories where executable programs live
__DEV_ENVIRONMENT_ID
: The ID of the running development environment. Automatically set when running the toolkit in Codecatalyst__DEV_ENVIRONMENT_PROJECT_NAME
: The project name associated with the running development environment. Automatically set when running the toolkit in Codecatalyst__DEV_ENVIRONMENT_SPACE_NAME
: The space name associated with the running development environment. Automatically set when running the toolkit in Codecatalyst__DEV_ENVIRONMENT_ORGANIZATION_NAME
: The organization name associated with the running development environment. Automatically set when running the toolkit in Codecatalyst
The following are environment variable versions of the user settings.json
overrides mentioned here. These will always override the toolkit defaults and those defined in settings.json
.
Unlike the user setting overrides, not all of these environment variables have to be set to make use of them.
__CODECATALYST_REGION
: for aws.dev.codecatalystService.region__CODECATALYST_ENDPOINT
: for aws.dev.codecatalystService.endpoint__CODECATALYST_HOSTNAME
: for aws.dev.codecatalystService.hostname__CODECATALYST_GIT_HOSTNAME
: for aws.dev.codecatalystService.gitHostname
The following are environment variable versions of the user settings.json
overrides mentioned here. These will always override the toolkit defaults and those defined in settings.json
.
Unlike the user setting overrides, not all of these environment variables have to be set to make use of them.
__CODEWHISPERER_REGION
: for aws.dev.codewhispererService.region__CODEWHISPERER_ENDPOINT
: for aws.dev.codewhispererService.endpoint
AUTH_UTIL_LAMBDA_ARN
: The Auth Util Lambda is used to log into using Builder ID/IdC automatically when running e2e tests. This is the arn that points to the auth util lambda.
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
: The relative HTTP URL endpoint for the SDK to use when making a request for credentials. The value is appended to the default Amazon ECS hostname of 169.254.170.2. For more information see container credential providerAWS_CONTAINER_CREDENTIALS_FULL_URI
: The full HTTP URL endpoint for the SDK to use when making a request for credentials. This includes both the scheme and the host. For more information see container credential provider
SSMDOCUMENT_LANGUAGESERVER_PORT
: The port the ssm document language server should start debugging on
GITHUB_ACTION
: The name of the current GitHub Action workflow step that is runningCODEBUILD_BUILD_ID
: The unique ID of the current CodeBuild build that is executingAWS_TOOLKIT_AUTOMATION
: If tests are currently being ranTEST_SSO_STARTURL
: The start url you want to use on E2E testsTEST_SSO_REGION
: The region for the start url you want to use on E2E testsAWS_TOOLKIT_TEST_NO_COLOR
: If the tests should include colour in their outputDEVELOPMENT_PATH
: The path to the aws toolkit vscode projectTEST_DIR
- The directory where the test runner should find the tests
See docs/cfn-schema-support.md for how to fix
and improve the JSON schema that provides auto-completion and syntax checking
of SAM and CloudFormation template.yaml
files.
The package.json 'devDependencies' includes eslint-plugin-aws-toolkits
. This is a local eslint plugin where we define custom lint rules. Additional lint rules and tests for lint rules can be added to this plugin:
- Define a new rule in
plugins/eslint-plugin-aws-toolkits/lib/rules
. - Create a test for your rule in
plugins/eslint-plugin-aws-toolkits/test/rules
and run withnpm run test
in the root directory ofeslint-plugin-aws-toolkits
. - Register your rule in
plugins/eslint-plugin-aws-toolkits/index.ts
. - Enable your rule in
.eslintrc
.
Writing lint rules can be tricky if you are unfamiliar with the process. Use an AST viewer such as https://astexplorer.net/
When the AWS SDK does not (yet) support a service but you have an API
model file (*.api.json
), use generateServiceClient.ts
to generate
a TypeScript *.d.ts
file and pass that to the AWS JS SDK to create
requests just from the model/types.
-
Add an entry to the list in
generateServiceClient.ts
:diff --git a/src/scripts/build/generateServiceClient.ts b/src/scripts/build/generateServiceClient.ts index 8bb278972d29..6c6914ec8812 100644 --- a/src/scripts/build/generateServiceClient.ts +++ b/src/scripts/build/generateServiceClient.ts @@ -199,6 +199,10 @@ ${fileContents} ;(async () => { const serviceClientDefinitions: ServiceClientDefinition[] = [ + { + serviceJsonPath: 'src/shared/foo.api.json', + serviceName: 'ClientFoo' + }, { serviceJsonPath: 'src/shared/telemetry/service-2.json', serviceName: 'ClientTelemetry',
-
Run the script:
npm run generateClients
-
The script produces a
*.d.ts
file (used only for IDE code-completion, not required to actually make requests):src/shared/foo.d.ts
-
To make requests with the SDK, pass the
*.api.json
service model toglobals.sdkClientBuilder.createAndConfigureServiceClient
as a genericService
withapiConfig=require('foo.api.json')
.// Import the `*.d.ts` file for code-completion convenience. import * as ClientFoo from '../shared/clientfoo' // The AWS JS SDK uses this to dynamically build service requests. import apiConfig = require('../shared/foo.api.json') ... const c = await globals.sdkClientBuilder.createAwsService( opts => new Service(opts), { apiConfig: apiConfig, region: 'us-west-2', credentials: credentials, correctClockSkew: true, endpoint: 'foo-beta.aws.dev', }) as ClientFoo const req = c.getThing({ id: 'asdf' }) req.send(function (err, data) { ... });
Webviews can be refreshed to show changes to .vue
code when running in Debug mode. You do not have to
reload the Debug VS Code window.
- Use
Command Palette
->Reload Webviews
- Only the frontend
.vue
changes will be reflected. If changing any backend code you must restart Debug mode.
This works by continuously building the final Vue webview files (webpack watch
) and then serving them through a local server (webpack serve
). Whenever a webview is loaded it will grab the latest build from the server.
For extensions to contribute their own codicons, VSCode requires a font file as well as how that font maps to codicon IDs. The mapping is found in package.json
under the icons
contribution point. Icons located in resources/icons
are stitched together at build-time into a single font, automatically adding mappings to package.json
. More information can be found here.
As a simple example, let's say I wanted to add a new icon for CloudWatch log streams. I would do the following:
-
Place the icon in
resources/icons/aws/cloudwatch
. I'l name the iconlog-stream.svg
. -
Use
npm run generateIcons
to updatepackage.json
. Commit this change with the new icon. -
You can now use the icon in the Toolkit:
getIcon('aws-cloudwatch-log-stream')
The marketplace page
is defined in packages/toolkit/README.md
. The vsce
package tool always replaces relative image paths
with URLs pointing to HEAD
on GitHub (https://github.com/aws/aws-toolkit-vscode/raw/HEAD/…/foo.gif
).
Note therefore:
- Don't delete images from
docs/marketplace/
unless the current published AWS Toolkit release doesn't depend on them. HEAD
implies that the URL depends on the current default branch (i.e.master
). Changes to other branches won't affect the marketplace page.
If you are contribuing visual assets from other open source repos, the source repo must have a compatible license (such as MIT), and we need to document the source of the images. Follow these steps (example: #227):
-
Use a separate location in this repo for every repo/organization where images are sourced from. See
resources/icons/vscode
as an example. -
Copy the source repo's licence into this destination location's LICENSE.txt file
-
Create a README.md in the destination location, and type in a copyright attribution:
The AWS Toolkit for VS Code includes the following third-party software/licensing: Icons contained in this folder and subfolders are from <SOURCE_REPO_NAME>: <SOURCE_REPO_URL> <PASTE_SOURCE_LICENSE_HERE>
-
Add an entry here summarizing the new destination location, where the assets were sourced from, and a brief rationale.
The minimum required vscode version specified in package.json is decided by the version of vscode running in Cloud9 and other vscode-compatible targets.
But you can still use the latest vscode APIs, by checking the current running vscode version. For example, to use a vscode 1.64 API:
- Check the vscode version:
semver.gte(vscode.version, '1.64.0')
- Disable the feature if is too old. That could mean just skipping the code entirely, or showing a different UI.
Full example:
There are several ways to make pre-production changes available on a "preview" or "experimental" basis:
- Experimental features: settings defined in aws.experiments are available in the vscode settings UI so that customers can discover and enable them. This mechanism is intended for non-production features which are ready for early access / preview feedback from interested customers.
- Developer-only features: the
aws.dev.forceDevMode
setting can be used as a condition to enable features only for users who have"aws.dev.forceDevMode": true
in their settings. These features are intended to be part of the mainline branch, but are not presented to customers in the VSCode settings UI. Example: EC2 commands were gated onaws.isDevMode
so the functionality could be merged to mainline while it was under development. - Beta artifacts: For a "private beta" launch,
src/dev/beta.ts
contains logic to check a hardcoded, stable URL serving the latest.vsix
build for the private beta. The hardcoded URL defined indev/config.ts:betaUrl
also forces the Toolkit to declare version99.0
(since "private beta" has no semver and to avoid unwanted auto-updating from VSCode marketplace). Beta builds of the Toolkit automatically query the URL once per session per day.- Beta users may want to run the
Extensions: Disable Auto Update for All Extensions
command disable VSCode's auto-update feature, to avoid ovewriting the beta Toolkit with the marketplace release.
- Beta users may want to run the
This project has adopted the Amazon Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via the vulnerability reporting page. Please do not create a public issue.
See the LICENSE file for our project's licensing. We will ask you to confirm the licensing of your contribution.
We may ask you to sign a Contributor License Agreement (CLA) for larger changes.