Merge pull request #238 from iriusrisk/bugfix/OPT-793

[bugfix/OPT-793] to release/1.14.0
iriusrisk · May 15, 2023 · 4c72a66 · 4c72a66
2 parents 6566140 + 32beb61
commit 4c72a66
Show file tree

Hide file tree

Showing 8 changed files with 345 additions and 2 deletions.
diff --git a/docs/img/ingesting-click-logs-from-web-application-iriusrisk-diagram-full.png b/docs/img/ingesting-click-logs-from-web-application-iriusrisk-diagram-full.png
diff --git a/docs/img/ingesting-click-logs-from-web-application-iriusrisk-diagram.png b/docs/img/ingesting-click-logs-from-web-application-iriusrisk-diagram.png
diff --git a/docs/img/ingesting-click-logs-from-web-application.png b/docs/img/ingesting-click-logs-from-web-application.png
diff --git a/docs/index.md b/docs/index.md
@@ -39,11 +39,12 @@ service for all the imports of new projects from external sources.
 Split by type, the currently supported input formats are:
 
 * **Infrastructure as Code (IaC)**:
-
     * <a href="https://aws.amazon.com/cloudformation/resources/templates/" target="_blank">CloudFormation (CFT)</a>.
     * <a href="https://www.terraform.io/" target="_blank">Terraform (TF)</a>.
+
 * **Diagram**:
     * Microsoft Visio (including diagrams exported from Lucidchart).
+
 * **Threat Model**:
     * Microsoft Threat Modeling Tool (MTMT).
 

diff --git a/docs/startleft-processors/iac/tfplan/Terraform-Plan-Quickstart.md b/docs/startleft-processors/iac/tfplan/Terraform-Plan-Quickstart.md
@@ -0,0 +1,123 @@
+## What is Terraform Plan?
+
+---
+From the <a href="https://developer.hashicorp.com/terraform/cli/commands/plan" target="_blank">official Terraform page</a>: 
+> The terraform plan command creates an execution plan, which lets you preview the changes that 
+> Terraform plans to make to your infrastructure.
+> 
+> - Reads the current state of any already-existing remote objects to make sure that the Terraform state is up-to-date
+
+From the StartLeft perspective, the Terraform Plan Processor (`slp_tfplan`) leverages the Terraform CLI capabilities
+to generate a unique Terraform resource file with the final infrastructure and its configuration.
+
+## The `slp_tfplan` module
+
+---
+The `slp_tfplan` module is the StartLeft Processor responsible for converting an existing Terraform execution plan 
+(`tf-plan`) along with its graphic representation (`tf-graph`) into an OTM. 
+This operation is based on a mapping file that enables the client to define the translations between the Terraform resources and the expected 
+output in the OTM file. 
+
+???+ abstract "How does it work?" 
+
+    We rely on the Terraform CLI to deal with the complexity of a Terraform infrastructure deployment, which is summarized in two single files:
+
+    - **Terraform Plan.** It contains all the resources along with its configuration. 
+    - **Terraform Graph.** It contains the relationships between the resources.
+
+    With these two files, we are able to compose a complete architecture diagram by parsing all the components and their relationships.
+
+
+Once you got familiarized yourself with the basics explained on this page, you will need to know more about how to 
+use and customize the behavior of the processor. To configure your conversions, you should take a look at 
+[TFplan mapping page](Terraform-Plan-how-to-create-a-mapping-file.md), where you will find all the information you 
+need, from basic to advanced, to build your mapping files.
+
+Apart from this, you may also find interesting the generic usage manuals for the [CLI](../../../usage/Command-Line-Interface.md) 
+and [REST API](../../../usage/REST-API.md).
+
+## How to use it?
+
+---
+Firstly, it is necessary to generate the `tf-plan` and `tf-graph` files by the Terraform CLI.
+
+???+ abstract "Generating my own Terraform files"
+
+    :zero: Execute the Terraform init (only once):
+    ```
+    terraform init
+    ```
+
+    :one: Generate the Terraform plan file in JSON format:
+    ```
+    terraform plan -out=tf-plan
+    terraform show -json tf-plan >> tf-plan.json
+    ```
+
+    :two: Generate the Terraform graph file:
+    ```
+    terraform graph -type=plan -plan=tf-plan >> tf-graph.gv
+    ```
+
+Next, you need to have a mapping file to configure the processor mapping behavior.
+??? example "Terraform Plan Mapping File Example"
+
+    > In the [TFplan mapping page](Terraform-Plan-how-to-create-a-mapping-file.md) you will find all the information you need, from 
+    basic to advanced, to build your mapping files.
+
+    ```yaml
+    --8<-- "examples/tfplan/iriusrisk-tfplan-aws-mapping.yaml"
+    ```
+
+Now everything is set up. The last step is to execute the commands for IaC `TFPLAN` parsing to generate the OTM. 
+
+### CLI
+> **Note**: Before continue, make sure you have 
+> [StartLeft properly installed](../../../Quickstart-Guide-for-Beginners.md) in your machine.
+
+Save the files above in your file system with these names:
+
+* `tf-plan.json` for the Terraform plan file.
+* `tf-graph.gv` for the Terraform graph file.
+* `ir-mappings.yaml` for the mapping file.
+
+Now we are going to execute StartLeft for these files so that an `ec2.otm` file will be generated in our working 
+directory.
+```shell
+startleft parse \
+	--iac-type TFPLAN \
+	--mapping-file ir-mappings.yaml \
+	--output-file output.otm \
+	--project-id "my-project" \
+	--project-name "My project" \
+	tf-plan.json tf-graph.gv
+```
+
+### cURL
+You can get the same result through the StartLeft REST API. For that, first, we need to set up the
+server with the command:
+```shell
+startleft server
+```
+
+Then, execute the following command to retrieve the OTM file:
+```shell
+curl --location --request POST localhost:5000/api/v1/startleft/iac \
+--header "Content-Type: multipart/form-data" \
+--header "Accept: application/json" \
+--form iac_type="TFPLAN" \
+--form iac_file=@"./tf-plan.json" \
+--form iac_file=@"./tf-graph.gv" \
+--form mapping_file=@"./ir-mappings.yaml" \
+--form id="my-project" \
+--form name="My project"
+```
+
+## More examples
+
+---
+The infrastructure built with Terraform may be as complex as you want. For that reason StartLeft, through the mapping files, 
+is intended to be configurable, so you can extend or modify its behavior and/or create your mappings on demand.
+
+To help you to walk through more complex situations with larger Terraform and mapping files, you can see all available
+examples under the Terraform Examples section.
diff --git a/.../startleft-processors/iac/tfplan/Terraform-Plan-how-to-create-a-mapping-file.md b/.../startleft-processors/iac/tfplan/Terraform-Plan-how-to-create-a-mapping-file.md
@@ -0,0 +1,3 @@
+
+At this point, the Terraform Plan StartLeft processor uses the same mapping that
+[Terraform mapping page](../tf/Terraform-how-to-create-a-basic-mapping-file.md).
diff --git a/...left-processors/iac/tfplan/examples/aws-ingesting-click-logs-using-terraform.md b/...left-processors/iac/tfplan/examples/aws-ingesting-click-logs-using-terraform.md
@@ -0,0 +1,212 @@
+## Threat Model Architecture Representation
+
+Let's use an Official example for 
+<a href="https://github.com/aws-samples/aws-ingesting-click-logs-using-terraform" target="_blank">
+Ingesting Web Application Click Logs</a>:
+
+> This post provides an API based ingestion application system for websites & applications to push user interactions, 
+> click actions from their website into AWS. The ingestion process will be exposed using a web-based interaction with an API Gateway endpoint.
+
+This example gives the source code for the following AWS architecture diagram:
+
+![ingesting-click-logs-from-web-application.png](../../../../img/ingesting-click-logs-from-web-application.png)
+
+The `slp_tfplan` processor can generate an accurate threat model by only following the steps below.
+
+???+ abstract "Generating my own Terraform files"
+
+    :one: Clone and prepare the example reporitory:
+    > :material-information-outline: These previous set-up commands are required because the example needs some 
+    > lambda code to be compiled.
+
+    ```shell 
+    $ git clone https://github.com/aws-samples/aws-ingesting-click-logs-using-terraform.git
+    $ cd aws-ingesting-click-logs-using-terraform/source/clicklogger
+    $ mvn clean package
+    $ cd ../../terraform/templates
+    ```
+
+    :two: Generate the tf-plan and tf-graph files:
+    ```shell
+    $ terraform init
+    $ terraform plan -out=tf-plan
+    $ terraform show -json tf-plan >> tf-plan.json
+    $ terraform graph -type=plan -plan=tf-plan >> tf-graph.gv
+    ```
+
+Use the StartLeft CLI to generate the OTM file:
+```shell
+$ startleft parse \ 
+  --iac-type TFPLAN \ 
+  --mapping-file ir-mappings.yaml \ 
+  --output-file output.otm \
+  --project-id "my-project" \
+  --project-name "My project" \ 
+  tf-plan.json tf-graph.gv
+```
+
+After, we need to use the Terraform CLI to create the required files and execute StartLeft parse to generate the
+final OTM.
+
+=== "IriusRisk Threat Model Example"
+
+    <figure markdown>
+        ![ingesting-click-logs-from-web-application-iriusrisk-diagram.png](../../../../img/ingesting-click-logs-from-web-application-iriusrisk-diagram.png)
+    </figure>
+
+=== "Mapping file"
+
+    ```yaml
+    --8<-- "examples/tfplan/iriusrisk-tfplan-aws-mapping.yaml"
+    ```
+=== "OTM"
+
+    ```yaml
+    otmVersion: 0.1.0
+    project:
+      name: name
+      id: id
+    representations:
+      - name: Terraform
+        id: Terraform
+        type: code
+    trustZones:
+      - id: b61d6911-338d-46a8-9f39-8dcd24abfe91
+        name: Public Cloud
+        risk:
+          trustRating: 10
+    components:
+      - id: aws_dynamodb_table.click-logger-table
+        name: click-logger-table
+        type: dynamodb
+        parent:
+          trustZone: b61d6911-338d-46a8-9f39-8dcd24abfe91
+        tags:
+          - aws_dynamodb_table
+      - id: aws_kinesis_firehose_delivery_stream.click_logger_firehose_delivery_stream
+        name: click_logger_firehose_delivery_stream
+        type: kinesis-data-firehose
+        parent:
+          trustZone: b61d6911-338d-46a8-9f39-8dcd24abfe91
+        tags:
+          - aws_kinesis_firehose_delivery_stream
+      - id: aws_lambda_function.lambda_clicklogger
+        name: lambda_clicklogger
+        type: aws-lambda-function
+        parent:
+          trustZone: b61d6911-338d-46a8-9f39-8dcd24abfe91
+        tags:
+          - aws_lambda_function
+      - id: aws_lambda_function.lambda_clicklogger_authorizer
+        name: lambda_clicklogger_authorizer
+        type: aws-lambda-function
+        parent:
+          trustZone: b61d6911-338d-46a8-9f39-8dcd24abfe91
+        tags:
+          - aws_lambda_function
+      - id: aws_lambda_function.lambda_clicklogger_stream_consumer
+        name: lambda_clicklogger_stream_consumer
+        type: aws-lambda-function
+        parent:
+          trustZone: b61d6911-338d-46a8-9f39-8dcd24abfe91
+        tags:
+          - aws_lambda_function
+      - id: aws_s3_bucket.click_logger_firehose_delivery_s3_bucket
+        name: click_logger_firehose_delivery_s3_bucket
+        type: s3
+        parent:
+          trustZone: b61d6911-338d-46a8-9f39-8dcd24abfe91
+        tags:
+          - aws_s3_bucket
+      - id: aws_api_gateway_account.click_logger_api_gateway_account
+        name: api-gateway (grouped)
+        type: api-gateway
+        parent:
+          trustZone: b61d6911-338d-46a8-9f39-8dcd24abfe91
+        tags:
+          - aws_api_gateway_method_response
+          - aws_api_gateway_method_settings
+          - aws_api_gateway_integration_response
+          - aws_api_gateway_method
+          - aws_api_gateway_model
+          - aws_api_gateway_request_validator
+          - aws_api_gateway_integration
+          - aws_api_gateway_account
+          - aws_api_gateway_resource
+          - aws_api_gateway_rest_api
+          - aws_api_gateway_authorizer
+          - aws_api_gateway_deployment
+      - id: aws_cloudwatch_log_group.click_logger_firehose_delivery_stream_log_group
+        name: cloudwatch (grouped)
+        type: cloudwatch
+        parent:
+          trustZone: b61d6911-338d-46a8-9f39-8dcd24abfe91
+        tags:
+          - aws_cloudwatch_log_group
+    dataflows:
+      - id: cb8953f8-1436-47a6-8939-ffcd07902614
+        name: clicklogger-authorizer to lambda_clicklogger_authorizer
+        source: aws_api_gateway_account.click_logger_api_gateway_account
+        destination: aws_lambda_function.lambda_clicklogger_authorizer
+        bidirectional: false
+      - id: 4f25e0b3-0b8a-4e84-90db-f0a8de34467e
+        name: integration to lambda_clicklogger
+        source: aws_api_gateway_account.click_logger_api_gateway_account
+        destination: aws_lambda_function.lambda_clicklogger
+        bidirectional: false
+      - id: 66cecdff-ba09-42b1-a75f-8e98fdae2dc4
+        name: lambda_click_logger_authorizer_log_group to lambda_clicklogger_authorizer
+        source: aws_cloudwatch_log_group.click_logger_firehose_delivery_stream_log_group
+        destination: aws_lambda_function.lambda_clicklogger_authorizer
+        bidirectional: false
+      - id: e01769b2-d461-4c4d-b7aa-cc028f4533ab
+        name: lambda_click_logger_log_group to lambda_clicklogger
+        source: aws_cloudwatch_log_group.click_logger_firehose_delivery_stream_log_group
+        destination: aws_lambda_function.lambda_clicklogger
+        bidirectional: false
+      - id: 9618b94c-b899-45d0-ba70-61d141e2ad54
+        name: click_logger_firehose_delivery_stream to lambda_clicklogger_stream_consumer
+        source: aws_kinesis_firehose_delivery_stream.click_logger_firehose_delivery_stream
+        destination: aws_lambda_function.lambda_clicklogger_stream_consumer
+        bidirectional: false
+      - id: e4431bbb-a8d5-413e-be2b-8c5b0cc06d81
+        name: click_logger_firehose_delivery_stream to
+          click_logger_firehose_delivery_s3_bucket
+        source: aws_kinesis_firehose_delivery_stream.click_logger_firehose_delivery_stream
+        destination: aws_s3_bucket.click_logger_firehose_delivery_s3_bucket
+        bidirectional: false
+      - id: 3c58eb54-f1eb-4762-9f2b-dd88793032e3
+        name: lambda_clicklogger to click_logger_firehose_delivery_stream
+        source: aws_lambda_function.lambda_clicklogger
+        destination: aws_kinesis_firehose_delivery_stream.click_logger_firehose_delivery_stream
+        bidirectional: false
+      - id: 50058e60-209a-4a2f-99b3-066bb633e2eb
+        name: lambda_clicklogger_stream_consumer to click-logger-table
+        source: aws_lambda_function.lambda_clicklogger_stream_consumer
+        destination: aws_dynamodb_table.click-logger-table
+        bidirectional: false
+    ```
+
+As we can see in the previous example, StartLeft can automatically reproduce an almost exact architecture diagram than the one present in the example website.
+
+## Full Architecture Representation
+This approach enables StartLeft not only to create threat models but also to explore all the components and the dataflows among them, 
+by configuring this behavior in the mapping file. In that way, you can get this diagram from the example above:
+
+=== "IriusRisk Threat Model Example"
+
+    <figure markdown>
+        ![ingesting-click-logs-from-web-application-iriusrisk-diagram-full.png](../../../../img/ingesting-click-logs-from-web-application-iriusrisk-diagram-full.png)
+    </figure>
+
+=== "Mapping file"
+
+    > The following component mappers need to be added in the mapping file to enable this feature. 
+
+    ```yaml
+      - type: empty-component
+        $source: { $skip: { $type: ["aws_db_subnet_group", "aws_internet_gateway"] } }
+
+      - type: empty-component
+        $source: { $singleton: { $catchall: { $type: { $regex: ^aws_\w*$ } } } }
+    ```
diff --git a/mkdocs.yml b/mkdocs.yml
@@ -36,6 +36,11 @@ nav:
             - Domain-Specific Language: startleft-processors/iac/tf/Terraform-domain-specific-language.md
             - Additional JMESPath functions: startleft-processors/iac/tf/Terraform-additional-jmespath-functions.md
           - Terraform Examples: startleft-processors/iac/tf/Terraform-Examples.md
+      - Terraform Plan:
+          - Terraform Plan Quickstart: startleft-processors/iac/tfplan/Terraform-Plan-Quickstart.md
+          - How To Create a Mapping File: startleft-processors/iac/tfplan/Terraform-Plan-how-to-create-a-mapping-file.md
+          - Terraform Examples:
+              - Ingesting Web Application Click Logs into AWS (by HashiCorp): startleft-processors/iac/tfplan/examples/aws-ingesting-click-logs-using-terraform.md
       - Visio:
           - Visio Quickstart: startleft-processors/diagram/Visio-Quickstart.md
           - Visio Mapping: startleft-processors/diagram/Visio-Mapping.md
@@ -66,7 +71,6 @@ theme:
     - search.suggest
     - search.highlight
     - content.code.annotate
-    - content.tabs.link
 plugins:
   - search
   - glightbox