Skip to content

Benchmarks 2024 11 26 TVM GCC O3 spike_rv32

GitHub Action edited this page Nov 26, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
    • Spike PK : fix-gcc14-rvv

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TVM : Nightly Pre-Build

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: gcc, Flags: -O3, Target: spike_rv32 )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
13829484.0
( 1.2x )
132180
( 1.326 )
59468
( 3.094 )
0 NCHW TVM Fallback RV32GC 0 -
21354724.0
( 0.7x )
105244
( 1.056 )
59468
( 3.094 )
0 NHWC TVM Fallback RV32GC 0 -
10452964.0
( 1.5x )
152812
( 1.533 )
59468
( 3.094 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
10653651.0
( 1.5x )
157718
( 1.582 )
59484
( 3.095 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
8939158.0
( 1.8x )
150606
( 1.511 )
59468
( 3.094 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
8349086.0
( 1.9x )
147664
( 1.481 )
59468
( 3.094 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
8475030.0
( 1.9x )
146432
( 1.469 )
59468
( 3.094 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
8480250.0
( 1.9x )
146558
( 1.47 )
59468
( 3.094 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
8550324.0
( 1.9x )
114700
( 1.151 )
59468
( 3.094 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
5555121.0
( 2.9x )
119962
( 1.203 )
59468
( 3.094 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
4045999.0
( 3.9x )
113050
( 1.134 )
59468
( 3.094 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
3216546.0
( 5.0x )
110862
( 1.112 )
59468
( 3.094 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
2814585.0
( 5.7x )
107222
( 1.076 )
59468
( 3.094 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
2813399.0
( 5.7x )
107360
( 1.077 )
59468
( 3.094 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
15976699.0
( Base )
99684
( Base )
19220
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
15557068.0
( 1.0x )
96978
( 0.973 )
23684
( 1.232 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
8793282.0
( 1.8x )
117028
( 1.174 )
19228
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
8485524.0
( 1.9x )
113662
( 1.14 )
19228
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
8285069.0
( 1.9x )
114616
( 1.15 )
19228
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
8063394.0
( 2.0x )
115918
( 1.163 )
19228
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
7529957.0
( 2.1x )
118194
( 1.186 )
19228
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
7373350.0
( 2.2x )
121742
( 1.221 )
19228
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
9193458.0
( 1.7x )
102278
( 1.026 )
23684
( 1.232 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
6916098.0
( 2.3x )
102294
( 1.026 )
23684
( 1.232 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
5689145.0
( 2.8x )
102290
( 1.026 )
23684
( 1.232 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
5654570.0
( 2.8x )
102280
( 1.026 )
23684
( 1.232 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
5654574.0
( 2.8x )
102290
( 1.026 )
23684
( 1.232 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
5661350.0
( 2.8x )
102276
( 1.026 )
23684
( 1.232 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
7565503.0
( 2.1x )
114626
( 1.15 )
23692
( 1.233 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
6980107.0
( 2.3x )
111236
( 1.116 )
23692
( 1.233 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
6502940.0
( 2.5x )
112196
( 1.126 )
23692
( 1.233 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
6286403.0
( 2.5x )
113478
( 1.138 )
23692
( 1.233 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
5742679.0
( 2.8x )
115762
( 1.161 )
23692
( 1.233 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
5582294.0
( 2.9x )
119328
( 1.197 )
23692
( 1.233 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
57144377.0
( 1.0x )
231668
( 1.609 )
108380
( 1.952 )
0 NCHW TVM Fallback RV32GC 0 -
77301599.0
( 0.7x )
212928
( 1.479 )
108380
( 1.952 )
0 NHWC TVM Fallback RV32GC 0 -
31570026.0
( 1.7x )
245586
( 1.705 )
108380
( 1.952 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
34099874.0
( 1.6x )
244522
( 1.698 )
108396
( 1.952 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
28617281.0
( 1.9x )
243388
( 1.69 )
108380
( 1.952 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
29299945.0
( 1.9x )
238224
( 1.654 )
108380
( 1.952 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
29645069.0
( 1.8x )
235104
( 1.633 )
108380
( 1.952 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
29643720.0
( 1.8x )
235304
( 1.634 )
108380
( 1.952 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
48362805.0
( 1.1x )
214740
( 1.491 )
108380
( 1.952 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
24598960.0
( 2.2x )
216628
( 1.504 )
108380
( 1.952 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
17730864.0
( 3.1x )
218674
( 1.519 )
108380
( 1.952 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
31473554.0
( 1.7x )
216422
( 1.503 )
108380
( 1.952 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
50344083.0
( 1.1x )
214628
( 1.49 )
108380
( 1.952 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
75032772.0
( 0.7x )
213414
( 1.482 )
108380
( 1.952 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
54563870.0
( Base )
144004
( Base )
55524
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
72406673.0
( 0.8x )
143244
( 0.995 )
55524
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
18036500.0
( 3.0x )
161952
( 1.125 )
55532
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
11522854.0
( 4.7x )
158212
( 1.099 )
55532
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
8327839.0
( 6.6x )
158950
( 1.104 )
55532
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
6837552.0
( 8.0x )
158950
( 1.104 )
55532
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
6158252.0
( 8.9x )
158938
( 1.104 )
55532
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
5828014.0
( 9.4x )
158942
( 1.104 )
55532
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
32179639.0
( 1.7x )
150442
( 1.045 )
55524
( 1.0 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
19775654.0
( 2.8x )
150464
( 1.045 )
55524
( 1.0 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
14149146.0
( 3.9x )
150480
( 1.045 )
55524
( 1.0 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
11407527.0
( 4.8x )
150464
( 1.045 )
55524
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
9231182.0
( 5.9x )
150464
( 1.045 )
55524
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
8651039.0
( 6.3x )
150466
( 1.045 )
55524
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
29180182.0
( 1.9x )
161130
( 1.119 )
55532
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
17082296.0
( 3.2x )
157382
( 1.093 )
55532
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
10966097.0
( 5.0x )
158090
( 1.098 )
55532
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
8097771.0
( 6.7x )
158120
( 1.098 )
55532
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
6788725.0
( 8.0x )
158126
( 1.098 )
55532
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
6151280.0
( 8.9x )
158108
( 1.098 )
55532
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
1639125.0
( 1.0x )
588258
( 1.856 )
5556
( 1.168 )
0 NCHW TVM Fallback RV32GC 0 -
1639125.0
( 1.0x )
588258
( 1.856 )
5556
( 1.168 )
0 NHWC TVM Fallback RV32GC 0 -
883810.0
( 1.9x )
598822
( 1.889 )
5556
( 1.168 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
786984.0
( 2.1x )
596658
( 1.883 )
5556
( 1.168 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
738723.0
( 2.3x )
593410
( 1.872 )
5556
( 1.168 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
951135.0
( 1.8x )
629770
( 1.987 )
5556
( 1.168 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
753529.0
( 2.2x )
600590
( 1.895 )
5556
( 1.168 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
769530.0
( 2.2x )
603310
( 1.904 )
5556
( 1.168 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
883810.0
( 1.9x )
598822
( 1.889 )
5556
( 1.168 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
786984.0
( 2.1x )
596658
( 1.883 )
5556
( 1.168 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
738723.0
( 2.3x )
593410
( 1.872 )
5556
( 1.168 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
951135.0
( 1.8x )
629770
( 1.987 )
5556
( 1.168 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
753529.0
( 2.2x )
600590
( 1.895 )
5556
( 1.168 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
769530.0
( 2.2x )
603310
( 1.904 )
5556
( 1.168 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
1681424.0
( Base )
316938
( Base )
4756
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
1681425.0
( 1.0x )
316948
( 1.0 )
4756
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
769281.0
( 2.2x )
317798
( 1.003 )
4756
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
587485.0
( 2.9x )
317798
( 1.003 )
4756
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
496867.0
( 3.4x )
317898
( 1.003 )
4756
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
451629.0
( 3.7x )
317814
( 1.003 )
4756
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
429019.0
( 3.9x )
317796
( 1.003 )
4756
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
417713.0
( 4.0x )
317778
( 1.003 )
4756
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
940667.0
( 1.8x )
319294
( 1.007 )
4756
( 1.0 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
692837.0
( 2.4x )
319296
( 1.007 )
4756
( 1.0 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
568925.0
( 3.0x )
319300
( 1.007 )
4756
( 1.0 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
507600.0
( 3.3x )
319300
( 1.007 )
4756
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
503282.0
( 3.3x )
319300
( 1.007 )
4756
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
502735.0
( 3.3x )
319290
( 1.007 )
4756
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
769279.0
( 2.2x )
317798
( 1.003 )
4756
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
587487.0
( 2.9x )
317796
( 1.003 )
4756
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
496871.0
( 3.4x )
317902
( 1.003 )
4756
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
451628.0
( 3.7x )
317814
( 1.003 )
4756
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
429022.0
( 3.9x )
317804
( 1.003 )
4756
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
417711.0
( 4.0x )
317778
( 1.003 )
4756
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
41222320.0
( 1.1x )
599440
( 1.803 )
180976
( 2.113 )
0 NCHW TVM Fallback RV32GC 0 -
59262507.0
( 0.8x )
529628
( 1.593 )
180976
( 2.113 )
0 NHWC TVM Fallback RV32GC 0 -
26240870.0
( 1.7x )
658454
( 1.981 )
180976
( 2.113 )
128 NCHW TVM Fallback RV32GCV 0 Loop+SLP
28421481.0
( 1.6x )
662766
( 1.994 )
180992
( 2.113 )
256 NCHW TVM Fallback RV32GCV 0 Loop+SLP
22852946.0
( 2.0x )
642152
( 1.932 )
180976
( 2.113 )
512 NCHW TVM Fallback RV32GCV 0 Loop+SLP
24360431.0
( 1.9x )
629318
( 1.893 )
180976
( 2.113 )
1024 NCHW TVM Fallback RV32GCV 0 Loop+SLP
20846784.0
( 2.2x )
603252
( 1.815 )
180976
( 2.113 )
2048 NCHW TVM Fallback RV32GCV 0 Loop+SLP
20901947.0
( 2.2x )
603116
( 1.814 )
180976
( 2.113 )
4096 NCHW TVM Fallback RV32GCV 0 Loop+SLP
27894732.0
( 1.6x )
552094
( 1.661 )
180976
( 2.113 )
128 NHWC TVM Fallback RV32GCV 0 Loop+SLP
20313358.0
( 2.2x )
563766
( 1.696 )
180976
( 2.113 )
256 NHWC TVM Fallback RV32GCV 0 Loop+SLP
16341744.0
( 2.8x )
578608
( 1.741 )
180976
( 2.113 )
512 NHWC TVM Fallback RV32GCV 0 Loop+SLP
14487153.0
( 3.1x )
560328
( 1.686 )
180976
( 2.113 )
1024 NHWC TVM Fallback RV32GCV 0 Loop+SLP
17565281.0
( 2.6x )
550420
( 1.656 )
180976
( 2.113 )
2048 NHWC TVM Fallback RV32GCV 0 Loop+SLP
17550303.0
( 2.6x )
552954
( 1.663 )
180976
( 2.113 )
4096 NHWC TVM Fallback RV32GCV 0 Loop+SLP
45360407.0
( Base )
332432
( Base )
85656
( Base )
0 NHWC muRISCV-NN Scalar RV32GC 0 -
45714838.0
( 1.0x )
329868
( 0.992 )
85656
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32GC 0 -
25203773.0
( 1.8x )
351376
( 1.057 )
85664
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
21133215.0
( 2.1x )
347896
( 1.047 )
85664
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
19149722.0
( 2.4x )
348828
( 1.049 )
85664
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
18207720.0
( 2.5x )
350158
( 1.053 )
85664
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
17844414.0
( 2.5x )
352398
( 1.06 )
85664
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
17695249.0
( 2.6x )
355976
( 1.071 )
85664
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32GCV 0 Loop+SLP
25537628.0
( 1.8x )
335194
( 1.008 )
85656
( 1.0 )
128 NHWC muRISCV-NN Vector RV32GCV 0 -
18650206.0
( 2.4x )
335176
( 1.008 )
85656
( 1.0 )
256 NHWC muRISCV-NN Vector RV32GCV 0 -
16021981.0
( 2.8x )
335220
( 1.008 )
85656
( 1.0 )
512 NHWC muRISCV-NN Vector RV32GCV 0 -
14920630.0
( 3.0x )
335192
( 1.008 )
85656
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32GCV 0 -
14830617.0
( 3.1x )
335228
( 1.008 )
85656
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32GCV 0 -
14837388.0
( 3.1x )
335186
( 1.008 )
85656
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32GCV 0 -
22235866.0
( 2.0x )
349134
( 1.05 )
85664
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
17175965.0
( 2.6x )
345594
( 1.04 )
85664
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
14280417.0
( 3.2x )
346512
( 1.042 )
85664
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
13269285.0
( 3.4x )
347816
( 1.046 )
85664
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
12900842.0
( 3.5x )
350088
( 1.053 )
85664
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP
12765807.0
( 3.6x )
353652
( 1.064 )
85664
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32GCV 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally