Benchmarks 2024 07 12 TFLM LLVM Os spike_rv64

Setup

Simulator

• Spike (riscv-isa-sim ) (ISS, CPI=1)

Toolchains

• LLVM/Clang:
  • TODO: Version
  • Linker: lld (TODO)
  • RISC-V GCC for Headers, libc,...

Models

• MLPerfTiny Benchmark

• TODO: others!

Package Versions

• MLonMCU : main

• TFLM : main

• Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a

• Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -Os)

Audio Wake Words (aww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
39726561 ( 0.4x )	144344 ( 0.875 )	38368 ( 1.0 )	128	TFLM	Reference	RV64GC	0	-
31995059 ( 0.5x )	149314 ( 0.905 )	38376 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
14707968 ( Base )	164938 ( Base )	38368 ( Base )	128	muRISCV-NN	Scalar	RV64GC	0	-
5814997 ( 2.5x )	172038 ( 1.043 )	38376 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
4075836 ( 3.6x )	165578 ( 1.004 )	38368 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-

Image Classification (resnet)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
121121849 ( 0.5x )	185832 ( 0.935 )	71136 ( 1.0 )	128	TFLM	Reference	RV64GC	0	-
57765532 ( 1.0x )	191734 ( 0.965 )	71144 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
55740377 ( Base )	198762 ( Base )	71136 ( Base )	128	muRISCV-NN	Scalar	RV64GC	0	-
25745638 ( 2.2x )	206928 ( 1.041 )	71144 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
15253027 ( 3.7x )	200146 ( 1.007 )	71136 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-

Anomaly Detection (toycar)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
2786519 ( 0.6x )	339238 ( 0.988 )	21472 ( 1.0 )	128	TFLM	Reference	RV64GC	0	-
957684 ( 1.7x )	341466 ( 0.994 )	21472 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
1638030 ( Base )	343362 ( Base )	21472 ( Base )	128	muRISCV-NN	Scalar	RV64GC	0	-
597658 ( 2.7x )	346358 ( 1.009 )	21472 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
585725 ( 2.8x )	344156 ( 1.002 )	21472 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-

Visual Wake Words (vww)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Kernels	Mode	Arch	Unroll	Auto-Vectorization
104299852 ( 0.4x )	418098 ( 0.953 )	136688 ( 1.0 )	128	TFLM	Reference	RV64GC	0	-
67381214 ( 0.7x )	423006 ( 0.964 )	136696 ( 1.0 )	128	TFLM	Reference	RV64GCV	0	Loop+SLP
43982589 ( Base )	438708 ( Base )	136688 ( Base )	128	muRISCV-NN	Scalar	RV64GC	0	-
18364049 ( 2.4x )	445746 ( 1.016 )	136696 ( 1.0 )	128	muRISCV-NN	Scalar	RV64GCV	0	Loop+SLP
13378250 ( 3.3x )	439348 ( 1.001 )	136688 ( 1.0 )	128	muRISCV-NN	Vector	RV64GCV	0	-

Original data

Click here to download the raw files for this benchmark.