Skip to content

Benchmarks 2024 11 21 TFLM GCC Os spike_rv32_min

GitHub Action edited this page Nov 21, 2024 · 2 revisions

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
    • Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TFLM : 8eb6b23de4470d6a8da3131650d6a67514dfa130

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: gcc, Flags: -Os, Target: spike_rv32_min )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
174715201
( 0.1x )
181844
( 0.863 )
36196
( 1.0 )
0 TFLM Reference RV32IM 0 -
174715201
( 0.1x )
181844
( 0.863 )
36196
( 1.0 )
128 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
174715201
( 0.1x )
181844
( 0.863 )
36196
( 1.0 )
256 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
174715201
( 0.1x )
181844
( 0.863 )
36196
( 1.0 )
512 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
174715201
( 0.1x )
181844
( 0.863 )
36196
( 1.0 )
1024 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
174715201
( 0.1x )
181844
( 0.863 )
36196
( 1.0 )
2048 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
174715201
( 0.1x )
181844
( 0.863 )
36196
( 1.0 )
4096 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
16670269
( Base )
210680
( Base )
36204
( Base )
0 muRISCV-NN Scalar RV32IM 0 -
16552929
( 1.0x )
210404
( 0.999 )
36204
( 1.0 )
0 muRISCV-NN Vector (Portable) RV32IM 0 -
16670269
( 1.0x )
210676
( 1.0 )
36204
( 1.0 )
128 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
16670269
( 1.0x )
210676
( 1.0 )
36204
( 1.0 )
256 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
16670269
( 1.0x )
210676
( 1.0 )
36204
( 1.0 )
512 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
16670269
( 1.0x )
210676
( 1.0 )
36204
( 1.0 )
1024 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
16670269
( 1.0x )
210676
( 1.0 )
36204
( 1.0 )
2048 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
16670269
( 1.0x )
210676
( 1.0 )
36204
( 1.0 )
4096 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
4127624
( 4.0x )
212528
( 1.009 )
36204
( 1.0 )
128 muRISCV-NN Vector RV32IM_ZVE64X 0 -
2859568
( 5.8x )
212528
( 1.009 )
36204
( 1.0 )
256 muRISCV-NN Vector RV32IM_ZVE64X 0 -
2170880
( 7.7x )
212528
( 1.009 )
36204
( 1.0 )
512 muRISCV-NN Vector RV32IM_ZVE64X 0 -
2129168
( 7.8x )
212528
( 1.009 )
36204
( 1.0 )
1024 muRISCV-NN Vector RV32IM_ZVE64X 0 -
2129168
( 7.8x )
212528
( 1.009 )
36204
( 1.0 )
2048 muRISCV-NN Vector RV32IM_ZVE64X 0 -
2132557
( 7.8x )
212528
( 1.009 )
36204
( 1.0 )
4096 muRISCV-NN Vector RV32IM_ZVE64X 0 -
16552929
( 1.0x )
210400
( 0.999 )
36204
( 1.0 )
128 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
16552929
( 1.0x )
210400
( 0.999 )
36204
( 1.0 )
256 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
16552929
( 1.0x )
210400
( 0.999 )
36204
( 1.0 )
512 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
16552929
( 1.0x )
210400
( 0.999 )
36204
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
16552929
( 1.0x )
210400
( 0.999 )
36204
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
16552929
( 1.0x )
210400
( 0.999 )
36204
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
745817708
( 0.1x )
223444
( 0.921 )
68960
( 1.0 )
0 TFLM Reference RV32IM 0 -
745817708
( 0.1x )
223440
( 0.921 )
68960
( 1.0 )
128 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
745817708
( 0.1x )
223440
( 0.921 )
68960
( 1.0 )
256 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
745817708
( 0.1x )
223440
( 0.921 )
68960
( 1.0 )
512 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
745817708
( 0.1x )
223440
( 0.921 )
68960
( 1.0 )
1024 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
745817708
( 0.1x )
223440
( 0.921 )
68960
( 1.0 )
2048 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
745817708
( 0.1x )
223440
( 0.921 )
68960
( 1.0 )
4096 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
81015562
( Base )
242728
( Base )
68952
( Base )
0 muRISCV-NN Scalar RV32IM 0 -
78475039
( 1.0x )
242344
( 0.998 )
68952
( 1.0 )
0 muRISCV-NN Vector (Portable) RV32IM 0 -
81015562
( 1.0x )
242720
( 1.0 )
68952
( 1.0 )
128 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
81015562
( 1.0x )
242720
( 1.0 )
68952
( 1.0 )
256 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
81015562
( 1.0x )
242720
( 1.0 )
68952
( 1.0 )
512 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
81015562
( 1.0x )
242720
( 1.0 )
68952
( 1.0 )
1024 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
81015562
( 1.0x )
242720
( 1.0 )
68952
( 1.0 )
2048 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
81015562
( 1.0x )
242720
( 1.0 )
68952
( 1.0 )
4096 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
15485294
( 5.2x )
245640
( 1.012 )
68952
( 1.0 )
128 muRISCV-NN Vector RV32IM_ZVE64X 0 -
9798286
( 8.3x )
245640
( 1.012 )
68952
( 1.0 )
256 muRISCV-NN Vector RV32IM_ZVE64X 0 -
7204638
( 11.2x )
245640
( 1.012 )
68952
( 1.0 )
512 muRISCV-NN Vector RV32IM_ZVE64X 0 -
5939118
( 13.6x )
245640
( 1.012 )
68952
( 1.0 )
1024 muRISCV-NN Vector RV32IM_ZVE64X 0 -
4997427
( 16.2x )
245640
( 1.012 )
68952
( 1.0 )
2048 muRISCV-NN Vector RV32IM_ZVE64X 0 -
4746900
( 17.1x )
245640
( 1.012 )
68952
( 1.0 )
4096 muRISCV-NN Vector RV32IM_ZVE64X 0 -
78475039
( 1.0x )
242336
( 0.998 )
68952
( 1.0 )
128 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
78475039
( 1.0x )
242336
( 0.998 )
68952
( 1.0 )
256 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
78475039
( 1.0x )
242336
( 0.998 )
68952
( 1.0 )
512 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
78475039
( 1.0x )
242336
( 0.998 )
68952
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
78475039
( 1.0x )
242336
( 0.998 )
68952
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
78475039
( 1.0x )
242336
( 0.998 )
68952
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
3106381
( 0.6x )
372256
( 0.981 )
19416
( 1.0 )
0 TFLM Reference RV32IM 0 -
3106381
( 0.6x )
372256
( 0.981 )
19416
( 1.0 )
128 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
3106381
( 0.6x )
372256
( 0.981 )
19416
( 1.0 )
256 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
3106381
( 0.6x )
372256
( 0.981 )
19416
( 1.0 )
512 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
3106381
( 0.6x )
372256
( 0.981 )
19416
( 1.0 )
1024 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
3106381
( 0.6x )
372256
( 0.981 )
19416
( 1.0 )
2048 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
3106381
( 0.6x )
372256
( 0.981 )
19416
( 1.0 )
4096 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
1788157
( Base )
379288
( Base )
19416
( Base )
0 muRISCV-NN Scalar RV32IM 0 -
3140766
( 0.6x )
379292
( 1.0 )
19416
( 1.0 )
0 muRISCV-NN Vector (Portable) RV32IM 0 -
1788157
( 1.0x )
379284
( 1.0 )
19416
( 1.0 )
128 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
1788157
( 1.0x )
379284
( 1.0 )
19416
( 1.0 )
256 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
1788157
( 1.0x )
379284
( 1.0 )
19416
( 1.0 )
512 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
1788157
( 1.0x )
379284
( 1.0 )
19416
( 1.0 )
1024 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
1788157
( 1.0x )
379284
( 1.0 )
19416
( 1.0 )
2048 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
1788157
( 1.0x )
379284
( 1.0 )
19416
( 1.0 )
4096 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
1947966
( 0.9x )
380964
( 1.004 )
19416
( 1.0 )
128 muRISCV-NN Vector RV32IM_ZVE64X 0 -
1829598
( 1.0x )
380964
( 1.004 )
19416
( 1.0 )
256 muRISCV-NN Vector RV32IM_ZVE64X 0 -
1770414
( 1.0x )
380964
( 1.004 )
19416
( 1.0 )
512 muRISCV-NN Vector RV32IM_ZVE64X 0 -
1741158
( 1.0x )
380964
( 1.004 )
19416
( 1.0 )
1024 muRISCV-NN Vector RV32IM_ZVE64X 0 -
1737474
( 1.0x )
380964
( 1.004 )
19416
( 1.0 )
2048 muRISCV-NN Vector RV32IM_ZVE64X 0 -
1735590
( 1.0x )
380964
( 1.004 )
19416
( 1.0 )
4096 muRISCV-NN Vector RV32IM_ZVE64X 0 -
3140766
( 0.6x )
379288
( 1.0 )
19416
( 1.0 )
128 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
3140766
( 0.6x )
379288
( 1.0 )
19416
( 1.0 )
256 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
3140766
( 0.6x )
379288
( 1.0 )
19416
( 1.0 )
512 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
3140766
( 0.6x )
379288
( 1.0 )
19416
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
3140766
( 0.6x )
379288
( 1.0 )
19416
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
3140766
( 0.6x )
379288
( 1.0 )
19416
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
495282732
( 0.1x )
455484
( 0.94 )
134500
( 1.0 )
0 TFLM Reference RV32IM 0 -
495282732
( 0.1x )
455484
( 0.94 )
134500
( 1.0 )
128 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
495282732
( 0.1x )
455484
( 0.94 )
134500
( 1.0 )
256 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
495282732
( 0.1x )
455484
( 0.94 )
134500
( 1.0 )
512 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
495282732
( 0.1x )
455484
( 0.94 )
134500
( 1.0 )
1024 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
495282732
( 0.1x )
455484
( 0.94 )
134500
( 1.0 )
2048 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
495282732
( 0.1x )
455484
( 0.94 )
134500
( 1.0 )
4096 TFLM Reference RV32IM_ZVE64X 0 Loop+SLP
49700105
( Base )
484320
( Base )
134508
( Base )
0 muRISCV-NN Scalar RV32IM 0 -
50304647
( 1.0x )
484044
( 0.999 )
134508
( 1.0 )
0 muRISCV-NN Vector (Portable) RV32IM 0 -
49700105
( 1.0x )
484316
( 1.0 )
134508
( 1.0 )
128 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
49700105
( 1.0x )
484316
( 1.0 )
134508
( 1.0 )
256 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
49700105
( 1.0x )
484316
( 1.0 )
134508
( 1.0 )
512 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
49700105
( 1.0x )
484316
( 1.0 )
134508
( 1.0 )
1024 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
49700105
( 1.0x )
484316
( 1.0 )
134508
( 1.0 )
2048 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
49700105
( 1.0x )
484316
( 1.0 )
134508
( 1.0 )
4096 muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
13494714
( 3.7x )
486168
( 1.004 )
134508
( 1.0 )
128 muRISCV-NN Vector RV32IM_ZVE64X 0 -
10166086
( 4.9x )
486168
( 1.004 )
134508
( 1.0 )
256 muRISCV-NN Vector RV32IM_ZVE64X 0 -
8876964
( 5.6x )
486168
( 1.004 )
134508
( 1.0 )
512 muRISCV-NN Vector RV32IM_ZVE64X 0 -
8372465
( 5.9x )
486168
( 1.004 )
134508
( 1.0 )
1024 muRISCV-NN Vector RV32IM_ZVE64X 0 -
8323851
( 6.0x )
486168
( 1.004 )
134508
( 1.0 )
2048 muRISCV-NN Vector RV32IM_ZVE64X 0 -
8327240
( 6.0x )
486168
( 1.004 )
134508
( 1.0 )
4096 muRISCV-NN Vector RV32IM_ZVE64X 0 -
50304647
( 1.0x )
484040
( 0.999 )
134508
( 1.0 )
128 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
50304647
( 1.0x )
484040
( 0.999 )
134508
( 1.0 )
256 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
50304647
( 1.0x )
484040
( 0.999 )
134508
( 1.0 )
512 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
50304647
( 1.0x )
484040
( 0.999 )
134508
( 1.0 )
1024 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
50304647
( 1.0x )
484040
( 0.999 )
134508
( 1.0 )
2048 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
50304647
( 1.0x )
484040
( 0.999 )
134508
( 1.0 )
4096 muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally