adding details for VRL lambda

docs/source/getting_started/deploy.rst

@@ -1,17 +1,61 @@
 AWS Deployment
 ==============
 
+.. _setup_lambda_repository:
+
+Setup Lambda repository
+-----------------------
+
+1. Navigate to the AWS console, and select ``CloudShell`` at the bottom left of the console. Open the cloud shell in the region you want to deploy.
+
+2. Store the AWS Account ID, and ECR repository name to environment variable in cloud shell.
+
+    .. code-block:: shell
+
+        # Replace value with the Account ID that was copied
+        $ export AWS_ACCOUNT_ID="582441423537"
+        # Replace value with ECR repository name you want to give
+        $ export REPO_NAME="cypienta-vrl-lambda"
+
+3. Pull the Cypienta VRL Lambda image from the AWS public repository using the following command.
+
+    .. code-block:: shell
+
+        $ docker pull public.ecr.aws/p2d2x2s3/cypienta/vrl-lambda:v0.1
+
+4. Once the image pull is completed, create an ECR repository to push the Cypienta VRL Lambda image.
+
+    .. code-block:: shell
+
+        $ aws ecr create-repository --repository-name ${REPO_NAME}
+
+5. After successfully create ECR repository, you can navigate to ECR private repository to view the responsitory you just created.
+
+    .. image:: resources/lambda_ecr.png
+        :alt: lambda ecr repo
+        :align: center
+
+6. Run the following commands to push the image to ECR repository.
+
+    .. code-block:: shell
+
+        $ export ECR_URI="${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com"
+        $ aws ecr get-login-password --region ${AWS_REGION} | docker login --username AWS --password-stdin ${ECR_URI}
+        $ docker tag public.ecr.aws/p2d2x2s3/cypienta/vrl-lambda:v0.1 ${ECR_URI}/${REPO_NAME}:v0.1
+        $ docker push ${ECR_URI}/${REPO_NAME}:v0.1
+
+
 Deploy resources using the Cloud Formation template
 ---------------------------------------------------
 
 1. Clone the Github repo 
 
     .. code-block:: shell
 
-        $ git clone -b v0.6 https://github.com/cypienta/Lambda.git
+        $ git clone -b v0.7 https://github.com/cypienta/Lambda.git
     
     .. note::
-        This command will clone the repository and checkout the branch ``v0.6``
+        This command will clone the repository and checkout the branch ``v0.7``
 
 2. Navigate to the AWS console, and search for ``CloudFormation``.
 
@@ -63,6 +107,8 @@ Deploy resources using the Cloud Formation template
 
     **NginxContainerImage:** The container image of the subscribed marketplace UI product with tag ``nginx-market*``. The ``Nginx container image`` noted in the section :doc:`subscribe`.
 
+    **VRLLambdaImage:** The container image of the VRL Lambda that was pushed to ECR private repository in :ref:`my-heading`
+
     The constraints for choosing the ``Cpu`` and ``Memory`` for the cluster can be found `here <https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html#cfn-ecs-taskdefinition-cpu>`__
 
     Recommended value for parameter **ChunkSize** is below ``100000``.

docs/source/getting_started/prerequisites.rst

@@ -10,6 +10,7 @@ Make sure that you have the required permissions for resources for the IAM user
 -  S3
 -  ECS
 -  EC2
+-  ECR
 -  IAM
 -  CloudFormation
 
@@ -40,5 +41,5 @@ Verify your instance type quotas by going to the AWS console. Search for ``Servi
 Lambda concurrent executions
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Verify that quota limit for ``Concurrent executions`` for AWS Lambda function. On your AWS console for the region where you want to deploy your resources, Search for ``Service Quotas``, and select ``AWS Lambda`` from the AWS Services list. Search for quota name ``Concurrent executions``. Make sure that the applied account-level quota value is more than 10 to allow reserved concurrency for the update_lookup_table lambda function. If the value is not greater than 10, select the ``Concurrent executions`` and click on ``Request increase at account level`` and set to any value greater than 10.
+Verify that quota limit for ``Concurrent executions`` for AWS Lambda function. On your AWS console for the region where you want to deploy your resources, Search for ``Service Quotas``, and select ``AWS Lambda`` from the AWS Services list. Search for quota name ``Concurrent executions``. Make sure that the applied account-level quota value is more than 12 to allow reserved concurrency for the enrich_with_technique, update_lookup_table lambda function. If the value is not greater than 10, select the ``Concurrent executions`` and click on ``Request increase at account level`` and set to any value greater than 10.
 

docs/source/getting_started/subscription.rst

@@ -90,7 +90,7 @@ Cypienta User Interface (UI)
         :alt: confirm subscribe
         :align: center
 
-4. Select the ``Fulfillment option`` as ``ECS``. Select the ``Software version`` as ``v0.1``. Then click on ``Continue to Launch``
+4. Select the ``Fulfillment option`` as ``ECS``. Select the ``Software version`` as ``v0.1.2``. Then click on ``Continue to Launch``
 
     .. image:: resources/to_launch.png
         :alt: to launch
@@ -111,12 +111,12 @@ Cypienta User Interface (UI)
             --username AWS \
             --password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com
             
-        CONTAINER_IMAGES="709825985650.dkr.ecr.us-east-1.amazonaws.com/cypienta/cytech:nginx-marketv0.0.3,709825985650.dkr.ecr.us-east-1.amazonaws.com/cypienta/cytech:marketv0.1.1"    
+        CONTAINER_IMAGES="709825985650.dkr.ecr.us-east-1.amazonaws.com/cypienta/cytech:nginx-marketv0.0.3,709825985650.dkr.ecr.us-east-1.amazonaws.com/cypienta/cytech:marketv0.1.2"    
 
         for i in $(echo $CONTAINER_IMAGES | sed "s/,/ /g"); do docker pull $i; done
 
     Here the two images are:
 
-    - **Web container image:** 709825985650.dkr.ecr.us-east-1.amazonaws.com/cypienta/cytech:marketv0.1.1
+    - **Web container image:** 709825985650.dkr.ecr.us-east-1.amazonaws.com/cypienta/cytech:marketv0.1.2
 
     - **Nginx container image:** 709825985650.dkr.ecr.us-east-1.amazonaws.com/cypienta/cytech:nginx-marketv0.0.3

docs/source/splunk/splunk.rst

@@ -51,7 +51,7 @@ To get search results of Splunk to AWS S3. Follow the steps below:
         :alt: Configure action for alert
         :align: center
 
-8. Add the ``Bucket name`` where you want splunk to save the results. For ``Object key``, enter ``%d-%b-%Y %H:%M:%S.json``. Select ``Account`` that you created on the configuration page from the dropdown. Finally click ``Save``. 
+8. Add the ``Bucket name`` which was created using the CloudFormation template to save the results. For ``Object key``, enter ``splunk_input/input/%d-%b-%Y %H:%M:%S.json``. Select ``Account`` that you created on the configuration page from the dropdown. Finally click ``Save``. 
 
     .. note::
         The user-provided object key is passed to Python's ``datetime.strftime()`` function, which encodes the time the search started. Format codes are extremely similar to Splunk's, please refer to the `official documentation <https://docs.python.org/3.7/library/datetime.html#strftime-strptime-behavior>`__.
@@ -95,7 +95,7 @@ To get results of cypienta product from S3 to Splunk. Follow the steps below:
         :alt: Search for events
         :align: center
 
-6. On the ``Add Incremental S3`` page, give a name to the configuration, select the ``AWS Account`` that was created in the previous step, select the ``S3 Bucket`` from which the data should be read, and provide the ``Log File Prefix`` of ``splunk/``. Under ``Splunk-related Configuration`` configure the ``Log Start Date`` and ``Index`` of your choice and click on ``Add``.
+6. On the ``Add Incremental S3`` page, give a name to the configuration. Select the ``AWS Account`` that was created in the previous step. Select the ``S3 Bucket`` which was created using the CloudFormation template, and provide the ``Log File Prefix`` of ``splunk/``. Under ``Splunk-related Configuration`` configure the ``Log Start Date`` and ``Index`` of your choice and click on ``Add``.
 
     .. image:: splunk_resources/conf_input.png
         :alt: configure input