Welcome to Cat-A-Log!

This npm package will help you create AWS Embedded Metric Format Logs and publish them to AWS Cloudwatch using AWS Lambda Powertools. EMF formatting will allow for chosen metrics to be automatically visualized in Cloudwatch metrics for simplier log debugging.

About Embedded Metric Formatting (EMF):

This is a JSON specification to communicate with Cloudwatch Logs to automatically extract values embedded in the structured log events. EMF is especially great for applications that make logs and need custom metrics without more complexity or cost. For more information please visit the following link: AWS Documentation on EMF Formatting

Why use Cat-A-Log?

Why use a washing machine when you can do them by hand? Because it saves you time and makes your job way easier! Leveraging AWS Lambda Powertools we can use the cat-a-log function to invoke and format logs into AWS Embedded Metric Format. By publishing these logs to AWS Cloudwatch, we are able to provide engineers with automatic metric visulaization to make the process of debugging logs much more efficient. Cat-a-log utilizies a cache to make effcient work of sending logs to Cloudwatch.

Instructions

Prerequites: Your chosen Integated Development Environment (i.e. VS Code) must already be be connected to AWS Lambda. For more guidence on setting up AWS Lambda we recommend following this helpful tutorial from AWS: Deploy Hello World Application with AWS SAM

Installation:

1. Install our package using the command npm install cat-a-logs then import the function and cache into your js file that connects to AWS Lambda import { cache, catalog } from "cat-a-logs/index.js"; Check out Cat-A-Log on npm using the attached link: Cat-A-Log

2. Now enter your arguments into the catalog function! Lets go through each argument one at a time and see what this looks like. First Lets take a look at the function definition:

   function catalog(
     trackedVariable: number | Array<number>,
     metricName: string,
     metricNamespace: string,
     metricUnitLabel: string = "None",
     CustomerDefinedDimension: { [key: string]: string } = {},
     resolution: 1 | 60 = 60,
     deploy: boolean = false)
   

   • trackedVariable: This variable represents a the numerical value of the metric that will appear under the category "Custom namespace" in Cloudwatch Metrics. Custom metric category/namespace/AWS Namespace. This is AWS Cloudwatch>Metrics>All metrics>Custom namespaces(ex. lambda-PTRI)>Dimensions(ex. Seniors)

   

   • metricName: This is a unique label of the tracked variable that will be reflected inside AWS Cloudwatch. Must be written as a string In the below image this corresponds to PTRI16Test --> AWS Cloudwatch>Metrics>All metrics>Custom namespaces

   

   • metricNamespace: This will be your "Custom namespace" in AWS Cloudwatch>Metrics>All metrics>Custom namespaces. In below image this is represented by lamdba-PTRI

   

   • metricUnitLabel: Explict Unit that Cloudwatch uses for EMF Configuration. Please note - must be one of the following as a string:

     • Seconds | Microseconds | Milliseconds | Bytes | Kilobytes | Megabytes | Gigabytes | Terabytes | Bits | Kilobits | Megabits | Gigabits | Terabits | Percent | Count | Bytes/Second | Kilobytes/Second | Megabytes/Second | Gigabytes/Second | Terabytes/Second | Bits/Second | Kilobits/Second | Megabits/Second | Gigabits/Second | Terabits/Second | Count/Second | None

     • To read more about Metric Datum see this link

   • CustomerDefinedDimension: This is an object - { Senior: 'hello'} Senior is the dimension label/key - when you click on it see the value hello and hello is the value of the dimension

     • The key will show up in AWS Cloudwatch as below:

       

     • If the user clicks on the Seniors Dimension then you will see the value - in this example "hello" reflected as below:

       

   • resolution: This is automatically set to default value to 60. If you would like to learn more about High Resolution Metrics please follow the attached link

   • deploy: automatically set to false. The final catalog call you make has to switch deploy flag to true. Failure to do so will cause the cache to grow without bound and use up memory

3. Start Building your Embedded Metric Formatted Logs. Call catalog as many times as needed.

4. ON the very last function call - it is important to change the deploy parameter to true.

5. Deploy your code with AWS SAM. This will place the file in AWS Lambda waiting for invokation.

6. Invoke your AWS Lamda Function

7. See your metrics and structured in CloudWatch!

Open Source Contributions:

• Contribution Areas:

  • Ready to Use: Typescript, EMF Object cache?, EMF specification support:
    • AWS Lambda (check)
      • Uses AWS Powertools Logger (check?)
    • EC2: (Need support)
  • Features to Build: EC2 Support with compatable structured loggin library such as Winston

• How we would like users to contribute: Clone our Respistory --> Make Feature Branch --> Push to your feature branch --> Pull Request

License Information:

Put License Information Here

Contributor Information:

Contributor Info here

Notes to Self:

Structure of the files:

index.ts is compiled to index.js. Important to compile .ts file to es6 js syntax using the tsc —target es6 (filepath) command app.mjs is a "pathway" to our lambda function. Here is where we will import catalog function and use it to involke our lambda function

index.ts lines 25-30 is checking to see if the value "level" || "message" || "sampling_rate" || "service" || "timestamp" ||"xray_trace_id"

logger.info gives you some information level is key and value is info

if you write name that it will overwrite the keys

Tech Challenges Spent 3 days dealing with inconsistencies of ES6/CommonJS in our code before compiling .js in ES6

To DO LIST ITEMS

• How can the user visaulize the cache growing in real time?
• Creating more professional scrreenshots for the ReadMe - to replace the current ReadMe screenshots
• Add License Information