Release v4.0.0

.cspell.json

@@ -14,6 +14,7 @@
     ],
     "ignorePaths": [
         ".pylintrc",
+        "CHANGELOG.md",
         "requirements.txt",
         "requirements-dev.txt",
         "maven-wrapper.jar",

docs/admin-guide.md

@@ -1036,7 +1036,7 @@ In the management account in `us-east-1`:
 2. There might be a pull request if the `aws-deployment-framework-bootstrap`
    repository that you have has to be updated to apply recent changes of ADF.
    This would show up with the version that you deployed recently, for example
-   `v3.2.0`.
+   `v4.0.0`.
 3. If there is no pull request, nothing to worry about. In that case, no
    changes were required in your repository for this update. Continue to
    the next step. If there is a pull request, open it and review the
@@ -1091,7 +1091,7 @@ This process is managed in an AWS Step Function state machine.
 
 1. Navigate to the AWS Step Functions service in the deployment account
    in _your main region_.
-2. Check the `ADFPipelineManagementStateMachine` state machine, all recent
+2. Check the `adf-pipeline-management` state machine, all recent
    invocations since we performed the update should succeed.
 
 We need to confirm that the pipelines generated by ADF are fully functional
@@ -1138,45 +1138,38 @@ Alternatively, you can also perform the update using the AWS CLI.
 
 If you wish to remove ADF you can delete the CloudFormation stack named
 `serverlessrepo-aws-deployment-framework` in the management account in
-the `us-east-1` region. This will move into a `DELETE_FAILED` at some stage because
-there is an S3 Bucket that is created via a custom resource _(cross region)_.
-After it moves into `DELETE_FAILED`, you can right-click on the stack and hit
-delete again while selecting to skip the Bucket the stack will successfully
-delete, you can then manually delete the bucket and its contents.
-
-After the main stack has been removed you can remove the base stack in the
-deployment account `adf-global-base-deployment` and any associated regional
+the `us-east-1` region. This will remove most resources created by ADF
+in the management account. With the exception of S3 buckets and SSM parameters.
+If you bootstrapped ADF into the management account you need to manually remove
+the bootstrap stacks as well.
+
+Feel free to delete the S3 buckets, SSM parameters that start with the `/adf`
+prefix, as well as other CloudFormation stacks such as:
+
+- adf-global-base-bootstrap (in the main deployment region)
+- adf-global-base-iam (in the main deployment region)
+- adf-regional-base-bootstrap (in every other region configured for ADF)
+
+When these stacks are removed, you can switch into the deployment
+account. We need to remove the base stack in the deployment account
+`adf-global-base-deployment` and any associated regional
 deployment account base stacks. After you have deleted these stacks, you can
 manually remove any base stacks from accounts that were bootstrapped.
+
 Alternatively prior to removing the initial
 `serverlessrepo-aws-deployment-framework` stack, you can set the _moves_ section
 of the `adfconfig.yml` file to _remove-base_ which would automatically clean up
 the base stack when the account is moved to the Root of the AWS Organization.
 
 One thing to keep in mind if you are planning to re-install ADF is that you
-will want to clean up the parameter from SSM Parameter Store named
-_deployment_account_id_ in `us-east-1` on the management account. AWS Step
-Functions uses this parameter to determine if ADF has already got a deployment
-account setup. If you re-install ADF with this parameter set to a value,
-ADF will attempt an assume role to the account to do some work, which will fail
-since that role will not be on the account at that point.
-
-There is also a CloudFormation stack named `adf-global-base-adf-build` which
-lives on the management account in your main deployment region. This stack
-creates two roles on the management account after the deployment account has
-been setup. These roles allow the deployment accounts CodeBuild role to assume a
-role back to the management account in order to query Organizations for AWS
-Accounts. This stack must be deleted manually also. If you do not remove this
-stack and then perform a fresh install of ADF, AWS CodeBuild on the deployment
-account will not be able to assume a role to the management account to query
-AWS Organizations. This is because this specific stack creates IAM roles with a
-strict trust relationship to the CodeBuild role on the deployment account, if
-that role gets deleted _(Which is will when you delete
-`adf-global-base-deployment`)_ then this stack references invalid IAM roles that
-no longer exist. If you forget to remove this stack and notice the trust
-relationship of the IAM roles referenced in the stack are no longer valid,
-you can delete the stack and re-run the main bootstrap pipeline which will
-recreate it with valid roles and links to the correct roles.
+will want to clean up the parameter from SSM Parameter Store. You can safely
+remove all `/adf` prefixed SSM parameters. But most importantly, you need to
+remove the `/adf/deployment_account_id` in `us-east-1` on the
+management account.
+As AWS Step Functions uses this parameter to determine if ADF has already got a
+deployment account setup. If you re-install ADF with this parameter set to a
+value, ADF will attempt an assume role to the account to configure it, which
+will fail since that role will not be on the account at that point.
 
 ## Troubleshooting
 
@@ -1234,15 +1227,15 @@ The main components to look at are:
    deployment region.
 8. Navigate to the [AWS Step Functions service](https://eu-west-1.console.aws.amazon.com/states/home?region=eu-west-1#/statemachines)
    in the deployment account in your main region. Please note, the link points
-   to the `eu-west-` region. Please update that to your own deployment region.
-   Check the state machines named `ADFPipelineManagementStateMachine`,
-   `EnableCrossAccountAccess`, and `PipelineDeletionStateMachine...`.
-   Look at recent executions only.
+   to the `eu-west-1` region. Please update that to your own deployment region.
+   Check the state machines named `adf-pipeline-management`,
+   `adf-bootstrap-enable-cross-account`, and
+   `adf-pipeline-management-delete-outdated`. Look at recent executions only.
     - When you find one that has a failed execution, check the components that
       are marked orange/red in the diagram.
     - If one failed and you want to trigger it again, you can execute it with
       the `New Execution` button in AWS Step Functions. Or even better in case
-      of the `ADFPipelineManagementStateMachine`, trigger all executions again,
+      of the `adf-pipeline-management`, trigger all executions again,
       Release a Change in the
       [ADF Pipeline generation CodePipeline - aws-deployment-framework-pipelines](https://console.aws.amazon.com/codesuite/codepipeline/pipelines/aws-deployment-framework-pipelines/view?region=eu-west-1).
 

docs/installation-guide.md

@@ -35,7 +35,9 @@ AWS Control Tower prior to installing ADF.**
 
 ---------------------------------
 
-## 1. Enable CloudTrail
+## 1. Enable Services
+
+### 1.1. Enable CloudTrail
 
 Ensure you have setup [AWS CloudTrail](https://aws.amazon.com/cloudtrail/)
 *(Not the default trail)* in your Management Account that spans **all
@@ -49,6 +51,28 @@ instructions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtr
 to configure the CloudTrail in the `us-east-1` region within the AWS
 Organizations Management AWS Account.
 
+### 1.2. Enable AWS Organizations API Access
+
+ADF will setup and configure [AWS
+Organizations](https://us-east-1.console.aws.amazon.com/organizations/v2/home?region=us-east-1)
+automatically.
+
+However, ADF requires, but does not configure AWS Account Management
+automatically.
+
+Without configuring AWS Account Management, the `adf-account-management` Step
+Functions state machine will fail to configure the AWS accounts such as the
+deployment account for you. The error message that it would return would state:
+
+> An error occurred (AccessDeniedException) when calling the ListRegions operation:
+> User: arn:[...assumed-sts-role-arn...]/adf-account-management-config-region
+> is not authorized to perform: account:ListRegions
+> (Your organization must first enable trusted access with AWS Account Management.)
+
+To enable this, go to AWS Organizations service console after it is configured
+and [enable AWS Account Management via this
+link](https://us-east-1.console.aws.amazon.com/organizations/v2/home/services/AWS%20Account%20Management).
+
 ## 2. Setup Your Build Environment
 
 ### 2.1. Local Instructions
@@ -191,7 +215,7 @@ You can checkout a specific version by running:
 git checkout ${version_tag_goes_here}
 
 # For example:
-git checkout v3.2.0
+git checkout v4.0.0
 ```
 
 ### 3.3. Update Makefile
@@ -624,6 +648,9 @@ automatically in the background, to follow its progress:
    open AWS CodePipeline from within the management account in `us-east-1` and
    see that there is an initial pipeline execution that started.
 
+   Upon first installation, this pipeline might fail to fetch the source
+   code from the repository. Click the retry failed action button to try again.
+
    When ADF is deployed for the first-time, it will make the initial commit
    with the skeleton structure of the `aws-deployment-framework-bootstrap`
    CodeCommit repository.

docs/providers-guide.md

@@ -87,10 +87,12 @@ Provider type: `codecommit`.
     information on the use of the owner attribute can be found in the
     [CodePipeline
     documentation](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_ActionTypeId.html).
-- *role* - *(String)* default ADF managed role.
-  - The role to use to fetch the contents of the CodeCommit repository. Only
-    specify when you need a specific role to access it. By default ADF will use
-    its own role to access it instead.
+- *role* - *(String)* default: `adf-codecommit-role`.
+  - The role name of the role to use to fetch the contents of the CodeCommit
+    repository. Only specify when you need a specific role to access it.
+    By default ADF will use its own role to access it instead.
+  - Please read the [user guide](./user-guide.md#custom-roles-for-pipelines) to
+    learn more about creating custom roles.
 - *trigger_on_changes* - *(Boolean)* default: `True`.
   - Whether CodePipeline should release a change and trigger the pipeline.
   - **When set to False**, you either need to trigger the pipeline manually,
@@ -114,9 +116,15 @@ S3 can be used as the source for a pipeline too. **Please note:** you can use
 S3 as a source and deployment provider. The properties that are available are
 slightly different.
 
-The role used to fetch the object from the S3 bucket is:
+The default role used to fetch the object from the S3 bucket is:
 `arn:${partition}:iam::${source_account_id}:role/adf-codecommit-role`.
 
+Please add the required S3 read permissions to the `adf-codecomit-role` via the
+`adf-bootstrap/deployment/global-iam.yml` file in the
+`aws-deployment-framework-bootstrap` repository. Or, allow
+the `adf-codecommit-role` S3 read permissions in the bucket policy of the
+source bucket.
+
 Provider type: `s3`.
 
 #### Properties
@@ -277,6 +285,8 @@ Provider type: `codebuild`.
     **Please note:** Since the CodeBuild environment runs in the deployment
     account, the role you specify will be assumed in and should be available
     in the deployment account too.
+  - Please read the [user guide](./user-guide.md#custom-roles-for-pipelines) to
+    learn more about creating custom roles.
 - *timeout* *(Number)* in minutes, default: `20`.
   - If you wish to define a custom timeout for the Build stage.
 - *privileged* *(Boolean)* default: `False`.
@@ -452,12 +462,15 @@ Provider type: `codedeploy`.
   - The name of the CodeDeploy Application you want to use for this deployment.
 - *deployment_group_name* *(String)* **(required)**
   - The name of the Deployment Group you want to use for this deployment.
-- *role* - *(String)* default
-  `arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-role`.
+- *role* - *(String)* default `adf-cloudformation-role`
+  - Automatically assumes into the given role in the target account, i.e.
+    `arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-role`.
   - The role you would like to use on the target AWS account to execute the
     CodeDeploy action. The role should allow the CodeDeploy service to assume
     it. As is [documented in the CodeDeploy service role
     documentation](https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-create-service-role.html).
+  - Please read the [user guide](./user-guide.md#custom-roles-for-pipelines) to
+    learn more about creating custom roles.
 
 ### CloudFormation
 
@@ -513,11 +526,23 @@ Provider type: `cloudformation`.
     to `infra`.
   - **Defaults to empty string**, the root of the source repository or input
     artifact.
-- *role* - *(String)* default
-  `arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-deployment-role`.
+- *role* - *(String)* default `adf-cloudformation-deployment-role`
+  - Automatically assumes into the given role in the target account, i.e.
+    `arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-deployment-role`.
   - The role you would like to use on the target AWS account to execute the
-    CloudFormation action. Ensure that the CloudFormation service should be
-    allowed to assume that role.
+    CloudFormation action.
+  - Ensure that the CloudFormation service should be allowed to assume that
+    role.
+  - Additionally, make sure that the `adf-cloudformation-role` is allowed to
+    perform an `iam:PassRole` action with the given role. Restrict this action
+    for the CloudFormation service only.
+    You can find an example of this in the `adf-bootstrap/deployment/global.yml`
+    file where it allows the CloudFormation Role to perform `iam:PassRole` with
+    the `adf-cloudformation-deployment-role`.
+    Please grant this access in the `adf-bootstrap/deployment/global-iam.yml`
+    file in the `aws-deployment-framework-bootstrap` repository.
+  - Please read the [user guide](./user-guide.md#custom-roles-for-pipelines) to
+    learn more about creating custom roles.
 - *action* -
   (`CHANGE_SET_EXECUTE|CHANGE_SET_REPLACE|CREATE_UPDATE|DELETE_ONLY|REPLACE_ON_FAILURE`)
   default: `CHANGE_SET_EXECUTE`.
@@ -586,7 +611,7 @@ Provider type: `service_catalog`.
 
 ### S3
 
-S3 can use used to deploy with too.
+S3 is available as a source and deployment provider.
 
 S3 cannot be used to target multiple accounts or regions in one stage.
 As the `bucket_name` property needs to be defined and these are globally
@@ -597,9 +622,17 @@ instead. Where each will target the specific bucket in the target account.
 Please note: you can use S3 as a source and deployment provider. The properties
 that are available are slightly different.
 
-The role used to upload the object(s) to the S3 bucket is:
+When S3 is used as the deployment provider, the default role used to upload
+the object(s) to the S3 bucket is the:
 `arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-role`.
 
+The `adf-cloudformation-role` is not granted access to read S3 buckets yet.
+Please add the required S3 write permissions to the `adf-cloudformation-role`
+via the `adf-bootstrap/global-iam.yml` file in the
+`aws-deployment-framework-bootstrap` repository. Or, alternatively, allow
+the `adf-cloudformation-role` S3 write permissions in the bucket policy of the
+target bucket.
+
 Provider type: `s3`.
 
 #### Properties
@@ -611,9 +644,10 @@ Provider type: `s3`.
 - *extract* - *(Boolean)* default: `False`.
   - Whether CodePipeline should extract the contents of the object when it
     deploys it.
-- *role* - *(String)* default:
-  `arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-role`.
-  - The role you would like to use for this action.
+- *role* - *(String)* default: `adf-cloudformation-role`.
+  - The role name of the role you would like to use for this action.
+  - Please read the [user guide](./user-guide.md#custom-roles-for-pipelines) to
+    learn more about creating custom roles.
 - *kms_encryption_key_arn* - *(String)*
   - The ARN of the AWS KMS encryption key for the host bucket. The
     `kms_encryption_key_arn` parameter encrypts uploaded artifacts with the