Skip to content

Commit c4af82b

Browse files
filipecosta90filipecosta90
authored
[add] Updated Readme spec status/tooling reference (#22)
1 parent e91dde2 commit c4af82b

File tree

1 file changed

+75
-58
lines changed

1 file changed

+75
-58
lines changed

‎Readme.md

Lines changed: 75 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -36,74 +36,91 @@ Current supported benchmark tools:
3636
## Architecture diagram
3737

3838
```
39-
┌──────────────────────────────────────┐
40-
│1) gh.com/redis/redis update │
41-
│ - git_repo: github.com/redis/redis │
42-
│ - git_hash: 459c3a │
43-
│ - git_branch: unstable │
44-
└─────────────────┬────────────────────┘
45-
│
46-
│ ┌───────────────────────────────────┐
47-
│ │HTTP POST │
48-
└──────┤<domain>/api/gh/redis/redis/commit │────┐
49-
└───────────────────────────────────┘ │
50-
│
51-
▼
52-
┌─────────────────────────────────────────────────┐
53-
│2) redis-benchmarks-spec-api │
54-
│ - Converts the HTTP info into an stream entry │
55-
│ - XADD stream:redis:redis:commit <...> │
56-
└────────────────────────┬────────────────────────┘
57-
│
58-
│ ┌────┐
59-
.─────────────────────────────────────. │ │push│
60-
┌─────┐ ┌ ▶( 2.1 ) stream of build events )◀───┘ └────┘
61-
│pull │ `─────────────────────────────────────'
62-
└─────┘ │
63-
64-
│ ┌────────────────────────────────────────────────┐
65-
─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─│2.2) redis-benchmarks-spec-builder │
66-
│ - based on ./setups/platforms │
67-
│ - build different required redis artifacts │
68-
└───────────────────────┬────────────────────────┘
69-
│
70-
│
71-
│
72-
│ ┌────┐
73-
.─────────────────────────────────────. │ │push│
74-
┌─────┐ ─▶( 2.3 ) stream of artifact benchmarks )◀────┘ └────┘
75-
│pull ││ `─────────────────────────────────────'
76-
└─────┘
77-
│
78-
┌───────────────────────────────────────────────┐
79-
│ │ │
80-
│3) benchmark_coordinator │
81-
│ │ - based on ./test-suites and ./setups: │
82-
│ - Trigger env setup │
83-
└ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─│ - 3.1 ) Trigger topology setup │
84-
│ - 3.2 ) Run benchmarks │
85-
│ - Record results │
86-
│ │
87-
└───────────────────────────────────────────────┘
39+
40+
41+
┌──────────────────────────────────────┐
42+
│1) gh.com/redis/redis update │
43+
│ - git_repo: github.com/redis/redis │
44+
│ - git_hash: 459c3a │
45+
│ - git_branch: unstable │
46+
└─────────────────┬────────────────────┘
47+
│
48+
│ ┌───────────────────────────────────┐
49+
│ │HTTP POST │
50+
└──────┤<domain>/api/gh/redis/redis/commit │──┐
51+
└───────────────────────────────────┘ │
52+
│
53+
▼
54+
┌──────────────────────────────────────────┐
55+
│2) api │
56+
│ - Converts the HTTP info into an stream │
57+
│entry │
58+
│ - XADD stream:redis:redis:commit <...> │
59+
│ │
60+
└─────────────────────┬────────────────────┘
61+
│
62+
│
63+
│
64+
│ ┌────┐
65+
.─────────────────────────────────────. │ │push│
66+
┌─────┐ ┌ ▶( 2.1 ) stream of build events )◀───┘ └────┘
67+
│pull │ `─────────────────────────────────────'
68+
└─────┘ │
69+
70+
│ ┌────────────────────────────────────────────┐
71+
─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─│2.2) build_agent │
72+
│ - based on setup platforms │
73+
│ - build different required redis │
74+
│artifacts │
75+
└───────────────────────┬────────────────────┘
76+
│
77+
│
78+
│ ┌────┐
79+
.─────────────────────────────────────. │ │push│
80+
┌─────┐ ─▶( 2.3 ) stream of artifact benchmarks )◀────┘ └────┘
81+
│pull ││ `─────────────────────────────────────'
82+
└─────┘
83+
│
84+
┌────────────────────────────────────────────┐
85+
│ │ │
86+
│3) benchmark_coordinator │
87+
│ │ - based on test-suites and setups: │
88+
│ - Trigger env setup │
89+
└ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─│ - 3.1 ) Trigger topology setup │
90+
│ - 3.2 ) Run benchmarks │
91+
│ - Record results into datasinks │
92+
│ │
93+
└────────────────────────────────────────────┘
8894
```
8995

90-
In a very brief description, gh.com/redis/redis upstream changes trigger an HTTP API call containing the
96+
In a very brief description, github.com/redis/redis upstream changes trigger an HTTP API call containing the
9197
relevant git information.
9298

9399
The HTTP request is then converted into an event ( tracked within redis ) that will trigger multiple build variants requests based upon the distinct platforms described in [`platforms`](redis_benchmarks_specification/setups/platforms/).
94100

95101
As soon as a new build variant request is received, the build agent ([`build_agent`](./build_agent/)) prepares the artifact(s) and proceeds into adding an artifact benchmark event so that the benchmark coordinator ([`benchmark_coordinator`](./benchmark_coordinator/)) can deploy/manage the required infrastructure and DB topologies, run the benchmark, and export the performance results.
96102
## Directory layout
97103

104+
### Specifications
105+
106+
The following is a high level status report for currently available specs.
107+
108+
* `redis_benchmarks_specification`
109+
* [`test-suites`](https://github.com/filipecosta90/redis-benchmarks-specification/tree/main/redis_benchmarks_specification/test-suites/): contains the benchmark suites definitions, specifying the target redis topology, the tested commands, the benchmark utility to use (the client), and if required the preloading dataset steps.
110+
98111
* `redis_benchmarks_specification/setups`
99-
* [`platforms`](redis_benchmarks_specification/setups/platforms/): contains the standard platforms considered to provide steady stable results, and to represent common deployment targets.
100-
* [`topologies`](redis_benchmarks_specification/setups/topologies/): contains the standard deployment topologies definition with the associated minimum specs to enable the topology definition.
101-
* [`builders`](redis_benchmarks_specification/setups/builders/): contains the build environment variations, that enable to build Redis with different compilers, compiler flags, libraries, etc...
102-
103-
* [`test_suites`](redis_benchmarks_specification/test-suites/): contains the benchmark suites definitions, specifying the target redis topology, the tested commands, the benchmark utility to use (the client), and if required the preloading dataset steps.
104-
* [`validator`](./validator/): contains the benchmark specifications validator utility
105-
* [`build_agent`](./build_agent/): contains the benchmark build agent utility that receives an event indicating a new build variant, generates the required redis binaries to test, and triggers the benchmark run on the listening agents.
106-
* [`benchmark_coordinator`](./benchmark_coordinator/): contains the coordinator utility that listens for benchmark suite run requests and setups the required steps to spin the actual benchmark topologies and to trigger the actual benchmarks.
112+
* [`platforms`](https://github.com/filipecosta90/redis-benchmarks-specification/tree/main/redis_benchmarks_specification/setups/platforms/): contains the standard platforms considered to provide steady stable results, and to represent common deployment targets.
113+
* [`topologies`](https://github.com/filipecosta90/redis-benchmarks-specification/tree/main/redis_benchmarks_specification/setups/topologies/): contains the standard deployment topologies definition with the associated minimum specs to enable the topology definition.
114+
* [`builders`](https://github.com/filipecosta90/redis-benchmarks-specification/tree/main/redis_benchmarks_specification/setups/builders/): contains the build environment variations, that enable to build Redis with different compilers, compiler flags, libraries, etc...
115+
116+
### Spec tool implementations
117+
118+
The following is a high level status report for currently available spec implementations.
119+
120+
* **STATUS: Experimental** [`redis-benchmarks-spec-api`](https://github.com/filipecosta90/redis-benchmarks-specification/tree/main/redis_benchmarks_specification/__api__/) : contains the API that translates the POST HTTP request that was triggered by github.com/redis/redis upstream changes, and fetches the relevant git/source info and coverts it into an event ( tracked within redis ).
121+
122+
* **STATUS: Experimental** [`redis-benchmarks-spec-builder`](https://github.com/filipecosta90/redis-benchmarks-specification/tree/main/redis_benchmarks_specification/__builder__/): contains the benchmark build agent utility that receives an event indicating a new build variant, generates the required redis binaries to test, and triggers the benchmark run on the listening agents.
123+
* **STATUS: Experimental** [`redis-benchmarks-spec-sc-coordinator`](https://github.com/filipecosta90/redis-benchmarks-specification/tree/main/redis_benchmarks_specification/__self_contained_coordinator__/): contains the coordinator utility that listens for benchmark suite run requests and setups the required steps to spin the actual benchmark topologies and to trigger the actual benchmarks.
107124

108125

109126
## Contributing guidelines

0 commit comments

Comments
 (0)