Skip to content

Benchmarks 2024 11 26 TVM GCC O3 spike_rv32_min

GitHub Action edited this page Nov 26, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
    • Spike PK : fix-gcc14-rvv

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TVM : Nightly Pre-Build

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: gcc, Flags: -O3, Target: spike_rv32_min )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
13876919.0
( 1.2x )
173272
( 1.282 )
59524
( 3.098 )
0 NCHW TVM Fallback RV32IM 0 -
21391826.0
( 0.7x )
139356
( 1.031 )
59524
( 3.098 )
0 NHWC TVM Fallback RV32IM 0 -
-
( ?x )
193648
( 1.432 )
59524
( 3.098 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
198256
( 1.467 )
59540
( 3.099 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
188972
( 1.398 )
59524
( 3.098 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
186088
( 1.377 )
59524
( 3.098 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
184564
( 1.365 )
59524
( 3.098 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
185012
( 1.369 )
59524
( 3.098 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
8592663.0
( 1.9x )
150036
( 1.11 )
59524
( 3.098 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
5611376.0
( 2.8x )
156220
( 1.156 )
59524
( 3.098 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
4083144.0
( 3.9x )
148644
( 1.1 )
59524
( 3.098 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
145796
( 1.079 )
59524
( 3.098 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
141788
( 1.049 )
59524
( 3.098 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
142252
( 1.052 )
59524
( 3.098 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
15992123.0
( Base )
135184
( Base )
19212
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
15567347.0
( 1.0x )
131876
( 0.976 )
23676
( 1.232 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
154184
( 1.141 )
19220
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
149920
( 1.109 )
19220
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
150856
( 1.116 )
19220
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
152284
( 1.126 )
19220
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
154976
( 1.146 )
19220
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
158808
( 1.175 )
19220
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
9232699.0
( 1.7x )
139392
( 1.031 )
23676
( 1.232 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
6955335.0
( 2.3x )
139384
( 1.031 )
23676
( 1.232 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
5720449.0
( 2.8x )
139376
( 1.031 )
23676
( 1.232 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
5683005.0
( 2.8x )
139392
( 1.031 )
23676
( 1.232 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
5682908.0
( 2.8x )
139384
( 1.031 )
23676
( 1.232 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
5683604.0
( 2.8x )
139376
( 1.031 )
23676
( 1.232 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
151232
( 1.119 )
23684
( 1.233 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
146900
( 1.087 )
23684
( 1.233 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
147832
( 1.094 )
23684
( 1.233 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
149240
( 1.104 )
23684
( 1.233 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
151964
( 1.124 )
23684
( 1.233 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
155812
( 1.153 )
23684
( 1.233 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
57186170.0
( 1.0x )
272732
( 1.539 )
108436
( 1.953 )
0 NCHW TVM Fallback RV32IM 0 -
77338123.0
( 0.7x )
247408
( 1.396 )
108436
( 1.953 )
0 NHWC TVM Fallback RV32IM 0 -
31657864.0
( 1.7x )
283772
( 1.602 )
108436
( 1.953 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
282024
( 1.592 )
108452
( 1.954 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
279592
( 1.578 )
108436
( 1.953 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
274460
( 1.549 )
108436
( 1.953 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
271388
( 1.532 )
108436
( 1.953 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
271836
( 1.534 )
108436
( 1.953 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
48394399.0
( 1.1x )
250132
( 1.412 )
108436
( 1.953 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
24624932.0
( 2.2x )
252348
( 1.424 )
108436
( 1.953 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
17769825.0
( 3.1x )
254620
( 1.437 )
108436
( 1.953 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
31519082.0
( 1.7x )
252040
( 1.423 )
108436
( 1.953 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
50378763.0
( 1.1x )
249956
( 1.411 )
108436
( 1.953 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
75069483.0
( 0.7x )
248788
( 1.404 )
108436
( 1.953 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
54574146.0
( Base )
177172
( Base )
55516
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
72411794.0
( 0.8x )
176348
( 0.995 )
55516
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
196772
( 1.111 )
55524
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
191832
( 1.083 )
55524
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
192548
( 1.087 )
55524
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
192584
( 1.087 )
55524
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
192548
( 1.087 )
55524
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
192548
( 1.087 )
55524
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
32262857.0
( 1.7x )
186340
( 1.052 )
55516
( 1.0 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
19851039.0
( 2.7x )
186336
( 1.052 )
55516
( 1.0 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
14223209.0
( 3.8x )
186328
( 1.052 )
55516
( 1.0 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
11477999.0
( 4.8x )
186388
( 1.052 )
55516
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
9291034.0
( 5.9x )
186384
( 1.052 )
55516
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
8701584.0
( 6.3x )
186396
( 1.052 )
55516
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
195868
( 1.106 )
55524
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
190956
( 1.078 )
55524
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
191684
( 1.082 )
55524
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
191704
( 1.082 )
55524
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
191676
( 1.082 )
55524
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
191672
( 1.082 )
55524
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
1644138.0
( 1.0x )
616908
( 1.789 )
5548
( 1.168 )
0 NCHW TVM Fallback RV32IM 0 -
1644138.0
( 1.0x )
616908
( 1.789 )
5548
( 1.168 )
0 NHWC TVM Fallback RV32IM 0 -
-
( ?x )
629216
( 1.825 )
5548
( 1.168 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
792130.0
( 2.1x )
625752
( 1.815 )
5548
( 1.168 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
743869.0
( 2.3x )
622244
( 1.805 )
5548
( 1.168 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
660420
( 1.916 )
5548
( 1.168 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
629404
( 1.826 )
5548
( 1.168 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
765436.0
( 2.2x )
633180
( 1.836 )
5548
( 1.168 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
629216
( 1.825 )
5548
( 1.168 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
792130.0
( 2.1x )
625752
( 1.815 )
5548
( 1.168 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
743869.0
( 2.3x )
622244
( 1.805 )
5548
( 1.168 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
660420
( 1.916 )
5548
( 1.168 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
629404
( 1.826 )
5548
( 1.168 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
765436.0
( 2.2x )
633180
( 1.836 )
5548
( 1.168 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
1681370.0
( Base )
344776
( Base )
4748
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
1681366.0
( 1.0x )
344760
( 1.0 )
4748
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
345604
( 1.002 )
4748
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
345616
( 1.002 )
4748
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
345692
( 1.003 )
4748
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
345608
( 1.002 )
4748
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
345596
( 1.002 )
4748
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
345580
( 1.002 )
4748
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
942562.0
( 1.8x )
348264
( 1.01 )
4748
( 1.0 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
696338.0
( 2.4x )
348244
( 1.01 )
4748
( 1.0 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
573232.0
( 2.9x )
348260
( 1.01 )
4748
( 1.0 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
512309.0
( 3.3x )
348244
( 1.01 )
4748
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
505075.0
( 3.3x )
348256
( 1.01 )
4748
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
501486.0
( 3.4x )
348236
( 1.01 )
4748
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
345604
( 1.002 )
4748
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
345604
( 1.002 )
4748
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
345696
( 1.003 )
4748
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
345612
( 1.002 )
4748
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
345592
( 1.002 )
4748
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
345588
( 1.002 )
4748
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
41269432.0
( 1.1x )
656992
( 1.775 )
181032
( 2.114 )
0 NCHW TVM Fallback RV32IM 0 -
59286676.0
( 0.8x )
568408
( 1.536 )
181032
( 2.114 )
0 NHWC TVM Fallback RV32IM 0 -
-
( ?x )
709180
( 1.916 )
181032
( 2.114 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
712244
( 1.925 )
181048
( 2.114 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
686204
( 1.854 )
181032
( 2.114 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
673064
( 1.819 )
181032
( 2.114 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
643616
( 1.739 )
181032
( 2.114 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
643448
( 1.739 )
181032
( 2.114 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
27907921.0
( 1.6x )
594676
( 1.607 )
181032
( 2.114 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
20335461.0
( 2.2x )
608156
( 1.643 )
181032
( 2.114 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
625420
( 1.69 )
181032
( 2.114 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
605128
( 1.635 )
181032
( 2.114 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
591116
( 1.597 )
181032
( 2.114 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
592852
( 1.602 )
181032
( 2.114 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
45375770.0
( Base )
370092
( Base )
85648
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
45719918.0
( 1.0x )
367012
( 0.992 )
85648
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
389704
( 1.053 )
85656
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
385276
( 1.041 )
85656
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
386208
( 1.044 )
85656
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
387652
( 1.047 )
85656
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
390352
( 1.055 )
85656
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
394172
( 1.065 )
85656
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
374516
( 1.012 )
85648
( 1.0 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
374528
( 1.012 )
85648
( 1.0 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
374524
( 1.012 )
85648
( 1.0 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
374524
( 1.012 )
85648
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
374532
( 1.012 )
85648
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
374524
( 1.012 )
85648
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
386912
( 1.045 )
85656
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
382508
( 1.034 )
85656
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
383396
( 1.036 )
85656
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
384824
( 1.04 )
85656
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
387524
( 1.047 )
85656
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
391368
( 1.057 )
85656
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally