Secretless Benchmarking

[doodlesbykumbi]

Secretless Benchmarking

This epic is about adding metric telemetry capabilities to Secretless to allow live benchmarking experiments. As part of the epic a metrics analysis pipeline must be chosen, some experiments will be run and some benchmarking reports will be generated.

Aha Cards

• Aha Card Title

Team

• Product Manager: Boaz (@boazmichaely )
• Product Owner: Alex (@alexkalish )
• Engineering Manager: Geri (@izgeri )
• Feature Lead: Kumbirai (@doodlesbykumbi)
• Engineers:
  • Kumbirai (@doodlesbykumbi)
• Technical Writer: Persons Name (@username)

SDLC Timeline

Stage	Updates	Artifact
Project Initialization
Kickoff
Feature Proposal
Functional Sign-off
Technical D&D
Solution Sign-off
Execution
Acceptance

Stories

Spikes to tackle aspects of the acceptance criteria

• SPIKE: Secretless benchmark metrics are defined. Tackles (1), (2) and (4)
• SPIKE: Export and analysis of benchmarking metrics. Tackles (3) and (5)
• SPIKE: Define and run benchmarking experiments. Tackles(6)
• SPIKE: Secretless benchmarking reports. Tackles (7)

Acceptance Criteria

1. Secretless will measure and emit telemetry associated with the data stream latency as defined as time difference between arrival and departure of the specific byte of information as measured at network interface level.
2. Secretless will measure and emit telemetry associated with the data throughput (eg data volume per time or Mb/sec) at the level of individual data stream.
3. The telemetry data output by Secretless must be easy to query across an arbitrary period of time with at least a week of history, and related events should be tagged in a way that makes it easy to aggregate them.
4. The telemetry solution will be instrumented in a way that will make it possible to add additional events/data to other areas in the future.
5. Documentation exists describing the telemetry protocol and how to collect and process data.
6. Secretless latency under ‘light’ load and maximum throughput (Mbps) per single Secretless container has been measured. ‘Light’ load or operating profile is defined as above, where Secretless is communicating with a database cluster with variable data size per request.
7. There is a document or presentation that shares the computed latency / throughput metrics, explains the working assumptions and testing methods, and interprets the results.