Skip to content

Benchmarks 2024 11 26 TVM LLVM O3 spike_rv32_min

GitHub Action edited this page Nov 26, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
    • Spike PK : fix-gcc14-rvv

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TVM : Nightly Pre-Build

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: llvm, Flags: -O3, Target: spike_rv32_min )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
13501144.0
( 1.1x )
172048
( 1.354 )
59492
( 3.098 )
0 NCHW TVM Fallback RV32IM 0 -
21040927.0
( 0.7x )
155300
( 1.222 )
59492
( 3.098 )
0 NHWC TVM Fallback RV32IM 0 -
-
( ?x )
165676
( 1.304 )
59492
( 3.098 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
166636
( 1.311 )
59492
( 3.098 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
167524
( 1.318 )
59492
( 3.098 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
170204
( 1.34 )
59492
( 3.098 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
170664
( 1.343 )
59492
( 3.098 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
171256
( 1.348 )
59492
( 3.098 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
178572
( 1.405 )
59492
( 3.098 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
178516
( 1.405 )
59492
( 3.098 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
174524
( 1.374 )
59492
( 3.098 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
159584
( 1.256 )
59492
( 3.098 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
159756
( 1.257 )
59492
( 3.098 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
159520
( 1.255 )
59492
( 3.098 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
15180408.0
( Base )
127060
( Base )
19204
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
14522433.0
( 1.0x )
126696
( 0.997 )
23668
( 1.232 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
135104
( 1.063 )
19244
( 1.002 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
135120
( 1.063 )
19204
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
135120
( 1.063 )
19204
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
135120
( 1.063 )
19204
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
135120
( 1.063 )
19204
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
135120
( 1.063 )
19204
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
5104729.0
( 3.0x )
129636
( 1.02 )
23668
( 1.232 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
3970756.0
( 3.8x )
129608
( 1.02 )
23668
( 1.232 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
3412486.0
( 4.4x )
129608
( 1.02 )
23668
( 1.232 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
3371502.0
( 4.5x )
129608
( 1.02 )
23668
( 1.232 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
3371923.0
( 4.5x )
129608
( 1.02 )
23668
( 1.232 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
3372248.0
( 4.5x )
129608
( 1.02 )
23668
( 1.232 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
135812
( 1.069 )
23708
( 1.235 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
135704
( 1.068 )
23668
( 1.232 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
135704
( 1.068 )
23668
( 1.232 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
135704
( 1.068 )
23668
( 1.232 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
135656
( 1.068 )
23668
( 1.232 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
135656
( 1.068 )
23668
( 1.232 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
61065388.0
( 0.9x )
277900
( 1.607 )
108404
( 1.953 )
0 NCHW TVM Fallback RV32IM 0 -
66654801.0
( 0.8x )
267868
( 1.549 )
108404
( 1.953 )
0 NHWC TVM Fallback RV32IM 0 -
-
( ?x )
351848
( 2.035 )
108404
( 1.953 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
349064
( 2.019 )
108404
( 1.953 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
349152
( 2.019 )
108404
( 1.953 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
348192
( 2.014 )
108404
( 1.953 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
347856
( 2.012 )
108404
( 1.953 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
348108
( 2.013 )
108404
( 1.953 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
54562451.0
( 1.0x )
272028
( 1.573 )
108404
( 1.953 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
51420843.0
( 1.1x )
271988
( 1.573 )
108404
( 1.953 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
268424
( 1.552 )
108404
( 1.953 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
266308
( 1.54 )
108404
( 1.953 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
266052
( 1.539 )
108404
( 1.953 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
265656
( 1.536 )
108404
( 1.953 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
56117151.0
( Base )
172900
( Base )
55508
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
72319168.0
( 0.8x )
172316
( 0.997 )
55508
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
182636
( 1.056 )
55540
( 1.001 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
182780
( 1.057 )
55508
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
182780
( 1.057 )
55508
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
182780
( 1.057 )
55508
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
182780
( 1.057 )
55508
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
182780
( 1.057 )
55508
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
15166972.0
( 3.7x )
175976
( 1.018 )
55508
( 1.0 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
9584333.0
( 5.9x )
176124
( 1.019 )
55508
( 1.0 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
7038743.0
( 8.0x )
176124
( 1.019 )
55508
( 1.0 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
5796679.0
( 9.7x )
176124
( 1.019 )
55508
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
4868892.0
( 11.5x )
176124
( 1.019 )
55508
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
4620135.0
( 12.1x )
176124
( 1.019 )
55508
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
181604
( 1.05 )
55540
( 1.001 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
181748
( 1.051 )
55508
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
181748
( 1.051 )
55508
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
181748
( 1.051 )
55508
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
181748
( 1.051 )
55508
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
181748
( 1.051 )
55508
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
1646617.0
( 1.0x )
621844
( 1.799 )
5540
( 1.169 )
0 NCHW TVM Fallback RV32IM 0 -
1646617.0
( 1.0x )
621844
( 1.799 )
5540
( 1.169 )
0 NHWC TVM Fallback RV32IM 0 -
840888.0
( 2.0x )
623712
( 1.804 )
5540
( 1.169 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
781448.0
( 2.2x )
623468
( 1.804 )
5540
( 1.169 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
622784
( 1.802 )
5540
( 1.169 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
622500
( 1.801 )
5540
( 1.169 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
632204
( 1.829 )
5540
( 1.169 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
623196
( 1.803 )
5540
( 1.169 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
840888.0
( 2.0x )
623712
( 1.804 )
5540
( 1.169 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
781448.0
( 2.2x )
623468
( 1.804 )
5540
( 1.169 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
622784
( 1.802 )
5540
( 1.169 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
622500
( 1.801 )
5540
( 1.169 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
632204
( 1.829 )
5540
( 1.169 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
623196
( 1.803 )
5540
( 1.169 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
1717519.0
( Base )
345656
( Base )
4740
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
1717519.0
( 1.0x )
345656
( 1.0 )
4740
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
347096
( 1.004 )
4740
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
347008
( 1.004 )
4740
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
347008
( 1.004 )
4740
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
347008
( 1.004 )
4740
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
347008
( 1.004 )
4740
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
347008
( 1.004 )
4740
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
620814.0
( 2.8x )
347420
( 1.005 )
4740
( 1.0 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
504124.0
( 3.4x )
347384
( 1.005 )
4740
( 1.0 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
445789.0
( 3.9x )
347384
( 1.005 )
4740
( 1.0 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
416941.0
( 4.1x )
347384
( 1.005 )
4740
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
413310.0
( 4.2x )
347384
( 1.005 )
4740
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
411455.0
( 4.2x )
347384
( 1.005 )
4740
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
347096
( 1.004 )
4740
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
347008
( 1.004 )
4740
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
347008
( 1.004 )
4740
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
347008
( 1.004 )
4740
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
347008
( 1.004 )
4740
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
347008
( 1.004 )
4740
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
42157983.0
( 1.1x )
665152
( 1.837 )
181000
( 2.113 )
0 NCHW TVM Fallback RV32IM 0 -
55748115.0
( 0.8x )
630596
( 1.742 )
181000
( 2.113 )
0 NHWC TVM Fallback RV32IM 0 -
-
( ?x )
816876
( 2.256 )
181000
( 2.113 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
817704
( 2.259 )
181000
( 2.113 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
823108
( 2.274 )
181000
( 2.113 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
821988
( 2.271 )
181000
( 2.113 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
821372
( 2.269 )
181000
( 2.113 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
819564
( 2.264 )
181000
( 2.113 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
707260
( 1.954 )
181000
( 2.113 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
707456
( 1.954 )
181000
( 2.113 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
701356
( 1.937 )
181000
( 2.113 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
679876
( 1.878 )
181000
( 2.113 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
662276
( 1.829 )
181000
( 2.113 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
656416
( 1.813 )
181000
( 2.113 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
45237678.0
( Base )
362020
( Base )
85640
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
44813418.0
( 1.0x )
361852
( 1.0 )
85640
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
370380
( 1.023 )
85784
( 1.002 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
370488
( 1.023 )
85640
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
370488
( 1.023 )
85640
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
370488
( 1.023 )
85640
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
370488
( 1.023 )
85640
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
370488
( 1.023 )
85640
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
365224
( 1.009 )
85640
( 1.0 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
365352
( 1.009 )
85640
( 1.0 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
365352
( 1.009 )
85640
( 1.0 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
365352
( 1.009 )
85640
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
365348
( 1.009 )
85640
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
365352
( 1.009 )
85640
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
371284
( 1.026 )
85784
( 1.002 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
371264
( 1.026 )
85640
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
371268
( 1.026 )
85640
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
371268
( 1.026 )
85640
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
371216
( 1.025 )
85640
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
371220
( 1.025 )
85640
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally