Benchmarks 2024 11 26 TFLM LLVM Os spike_rv32

Setup

Simulator

• Spike (riscv-isa-sim ) (ISS, CPI=1)
  • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
  • Spike PK : fix-gcc14-rvv

Toolchains

• RISC-V GCC:

  • Scalar: riscv32-unknown-elf-gcc (g8b4bb54e6c4) 14.2.1 20241118
  • Vector: riscv32-unknown-elf-gcc (g8b4bb54e6c4) 14.2.1 20241118
  • Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257

• LLVM/Clang:

  • clang version 18.1.8 (https://github.com/llvm/llvm-project.git 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
  • Linker: lld (TODO)

Models

• MLPerfTiny Benchmark

• TODO: others!

Frameworks

• MLonMCU : develop

• TFLM : 8eb6b23de4470d6a8da3131650d6a67514dfa130

Miscellaneous

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -Os, Target: spike_rv32 )

Audio Wake Words (aww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
39294338.0 ( 0.4x )	149588 ( 0.862 )	36060 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
33489638.0 ( 0.5x )	155628 ( 0.897 )	36068 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
31696974.0 ( 0.5x )	155648 ( 0.897 )	36068 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
30800724.0 ( 0.5x )	155932 ( 0.899 )	36068 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
30352565.0 ( 0.5x )	156266 ( 0.9 )	36068 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
30350820.0 ( 0.5x )	156470 ( 0.902 )	36068 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
30359365.0 ( 0.5x )	156870 ( 0.904 )	36068 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
15085053.0 ( Base )	173538 ( Base )	36060 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
14948926.0 ( 1.0x )	172684 ( 0.995 )	36060 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
6143104.0 ( 2.5x )	184606 ( 1.064 )	36068 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
5140439.0 ( 2.9x )	184652 ( 1.064 )	36068 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4644341.0 ( 3.2x )	184770 ( 1.065 )	36068 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4351870.0 ( 3.5x )	184978 ( 1.066 )	36068 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4351873.0 ( 3.5x )	185074 ( 1.066 )	36068 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4358658.0 ( 3.5x )	185366 ( 1.068 )	36068 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
4083226.0 ( 3.7x )	174750 ( 1.007 )	36060 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
2828248.0 ( 5.3x )	174672 ( 1.007 )	36060 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
2150930.0 ( 7.0x )	174612 ( 1.006 )	36060 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
2109942.0 ( 7.1x )	174646 ( 1.006 )	36060 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
2113333.0 ( 7.1x )	174646 ( 1.006 )	36060 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
2116724.0 ( 7.1x )	174646 ( 1.006 )	36060 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
6887271.0 ( 2.2x )	183430 ( 1.057 )	36068 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
5618070.0 ( 2.7x )	183436 ( 1.057 )	36068 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
5134467.0 ( 2.9x )	183554 ( 1.058 )	36068 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
4864889.0 ( 3.1x )	183762 ( 1.059 )	36068 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
4864892.0 ( 3.1x )	183858 ( 1.059 )	36068 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
4871677.0 ( 3.1x )	184150 ( 1.061 )	36068 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Image Classification (resnet)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
121053207.0 ( 0.5x )	191310 ( 0.922 )	68828 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
56884504.0 ( 1.0x )	197300 ( 0.951 )	68836 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
47127682.0 ( 1.2x )	197402 ( 0.952 )	68836 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
44786238.0 ( 1.3x )	197914 ( 0.954 )	68836 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
44352470.0 ( 1.3x )	198388 ( 0.956 )	68836 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
44345607.0 ( 1.3x )	198714 ( 0.958 )	68836 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
44349034.0 ( 1.3x )	199234 ( 0.961 )	68836 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
56367181.0 ( Base )	207426 ( Base )	68824 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
72412283.0 ( 0.8x )	206780 ( 0.997 )	68824 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
26152356.0 ( 2.2x )	218724 ( 1.054 )	68832 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
18287589.0 ( 3.1x )	218764 ( 1.055 )	68832 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
14601103.0 ( 3.9x )	219150 ( 1.057 )	68832 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
12930241.0 ( 4.4x )	219526 ( 1.058 )	68832 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
12111071.0 ( 4.7x )	219766 ( 1.059 )	68832 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
11488048.0 ( 4.9x )	220206 ( 1.062 )	68832 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
15252975.0 ( 3.7x )	209352 ( 1.009 )	68824 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
9669655.0 ( 5.8x )	209286 ( 1.009 )	68824 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
7123918.0 ( 7.9x )	209226 ( 1.009 )	68824 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
5881773.0 ( 9.6x )	209260 ( 1.009 )	68824 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
4956900.0 ( 11.4x )	209260 ( 1.009 )	68824 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
4711199.0 ( 12.0x )	209260 ( 1.009 )	68824 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
20452521.0 ( 2.8x )	217448 ( 1.048 )	68832 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
14356883.0 ( 3.9x )	217488 ( 1.049 )	68832 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
11499710.0 ( 4.9x )	217874 ( 1.05 )	68832 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
10206614.0 ( 5.5x )	218250 ( 1.052 )	68832 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
9566619.0 ( 5.9x )	218490 ( 1.053 )	68832 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
9092126.0 ( 6.2x )	218930 ( 1.055 )	68832 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
2797804.0 ( 0.6x )	344894 ( 0.983 )	19364 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
899257.0 ( 1.9x )	346920 ( 0.989 )	19364 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
669015.0 ( 2.5x )	346874 ( 0.989 )	19364 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
559064.0 ( 3.0x )	346992 ( 0.989 )	19364 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
501526.0 ( 3.4x )	347186 ( 0.99 )	19364 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
472772.0 ( 3.6x )	347270 ( 0.99 )	19364 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
472609.0 ( 3.6x )	347354 ( 0.99 )	19364 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
1684522.0 ( Base )	350746 ( Base )	19364 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
3037192.0 ( 0.6x )	350748 ( 1.0 )	19364 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
594011.0 ( 2.8x )	354846 ( 1.012 )	19364 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
480091.0 ( 3.5x )	354770 ( 1.011 )	19364 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
423131.0 ( 4.0x )	354824 ( 1.012 )	19364 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
394651.0 ( 4.3x )	354976 ( 1.012 )	19364 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
375266.0 ( 4.5x )	355024 ( 1.012 )	19364 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
373482.0 ( 4.5x )	355072 ( 1.012 )	19364 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
1941171.0 ( 0.9x )	352266 ( 1.004 )	19364 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
1824490.0 ( 0.9x )	352216 ( 1.004 )	19364 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
1766151.0 ( 1.0x )	352178 ( 1.004 )	19364 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
1737304.0 ( 1.0x )	352212 ( 1.004 )	19364 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
1733673.0 ( 1.0x )	352212 ( 1.004 )	19364 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
1731817.0 ( 1.0x )	352212 ( 1.004 )	19364 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
927993.0 ( 1.8x )	354848 ( 1.012 )	19364 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
666041.0 ( 2.5x )	354772 ( 1.011 )	19364 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
535065.0 ( 3.1x )	354826 ( 1.012 )	19364 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
464432.0 ( 3.6x )	354978 ( 1.012 )	19364 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
431688.0 ( 3.9x )	355026 ( 1.012 )	19364 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
427600.0 ( 3.9x )	355074 ( 1.012 )	19364 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
103599763.0 ( 0.4x )	423230 ( 0.946 )	134364 ( 1.0 )	0	TFLM	Reference	RV32GC	0	-
72374386.0 ( 0.6x )	429270 ( 0.96 )	134372 ( 1.0 )	128	TFLM	Reference	RV32GCV	0	Loop+SLP
67212897.0 ( 0.7x )	429290 ( 0.96 )	134372 ( 1.0 )	256	TFLM	Reference	RV32GCV	0	Loop+SLP
64761379.0 ( 0.7x )	429574 ( 0.961 )	134372 ( 1.0 )	512	TFLM	Reference	RV32GCV	0	Loop+SLP
63728987.0 ( 0.7x )	429908 ( 0.961 )	134372 ( 1.0 )	1024	TFLM	Reference	RV32GCV	0	Loop+SLP
63307692.0 ( 0.7x )	430112 ( 0.962 )	134372 ( 1.0 )	2048	TFLM	Reference	RV32GCV	0	Loop+SLP
63278997.0 ( 0.7x )	430512 ( 0.963 )	134372 ( 1.0 )	4096	TFLM	Reference	RV32GCV	0	Loop+SLP
45191831.0 ( Base )	447180 ( Base )	134364 ( Base )	0	muRISCV-NN	Scalar	RV32GC	0	-
45007628.0 ( 1.0x )	446326 ( 0.998 )	134364 ( 1.0 )	0	muRISCV-NN	Vector (Portable)	RV32GC	0	-
19573301.0 ( 2.3x )	458248 ( 1.025 )	134372 ( 1.0 )	128	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
16706988.0 ( 2.7x )	458294 ( 1.025 )	134372 ( 1.0 )	256	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
15338952.0 ( 2.9x )	458412 ( 1.025 )	134372 ( 1.0 )	512	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
14832871.0 ( 3.0x )	458620 ( 1.026 )	134372 ( 1.0 )	1024	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
14626372.0 ( 3.1x )	458716 ( 1.026 )	134372 ( 1.0 )	2048	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
14615697.0 ( 3.1x )	459008 ( 1.026 )	134372 ( 1.0 )	4096	muRISCV-NN	Scalar	RV32GCV	0	Loop+SLP
13372761.0 ( 3.4x )	448392 ( 1.003 )	134364 ( 1.0 )	128	muRISCV-NN	Vector	RV32GCV	0	-
10051761.0 ( 4.5x )	448314 ( 1.003 )	134364 ( 1.0 )	256	muRISCV-NN	Vector	RV32GCV	0	-
8760333.0 ( 5.2x )	448254 ( 1.002 )	134364 ( 1.0 )	512	muRISCV-NN	Vector	RV32GCV	0	-
8256343.0 ( 5.5x )	448288 ( 1.002 )	134364 ( 1.0 )	1024	muRISCV-NN	Vector	RV32GCV	0	-
8211965.0 ( 5.5x )	448288 ( 1.002 )	134364 ( 1.0 )	2048	muRISCV-NN	Vector	RV32GCV	0	-
8215356.0 ( 5.5x )	448288 ( 1.002 )	134364 ( 1.0 )	4096	muRISCV-NN	Vector	RV32GCV	0	-
22478255.0 ( 2.0x )	457072 ( 1.022 )	134372 ( 1.0 )	128	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
18853462.0 ( 2.4x )	457078 ( 1.022 )	134372 ( 1.0 )	256	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
17526745.0 ( 2.6x )	457196 ( 1.022 )	134372 ( 1.0 )	512	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
17025843.0 ( 2.7x )	457404 ( 1.023 )	134372 ( 1.0 )	1024	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
16819274.0 ( 2.7x )	457500 ( 1.023 )	134372 ( 1.0 )	2048	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP
16808593.0 ( 2.7x )	457792 ( 1.024 )	134372 ( 1.0 )	4096	muRISCV-NN	Vector (Portable)	RV32GCV	0	Loop+SLP

Original data

Click here to download the raw files for this benchmark.