Benchmarks 2024 03 02 TVM GCC Os

Setup

Simulator

Spike (riscv-isa-sim ) (ISS, CPI=1)

Toolchains

RISC-V GCC:
- Scalar: TODO: version & url
- Vector: TODO: version & url
- Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257

Models

MLPerfTiny Benchmark
TODO: others!

Package Versions

MLonMCU : main
TVM : Nightly Pre-Build
Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

Used -Os flag for compilation.
Benchmarks generated using MLonMCU deployment tool with minimal efforts.
Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: gcc, Flags: -Os)

Audio Wake Words (`aww`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Unroll	Auto-Vectorization
`31536718` ( 0.5x )	`107754` ( 1.266 )	`59536` ( 3.097 )	0	NCHW	TVM	Fallback	RV32GC	False	-
`29578962` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	0	NHWC	TVM	Fallback	RV32GC	False	-
`15873108` ( 1.1x )	`101286` ( 1.19 )	`51364` ( 2.672 )	0	NCHW	TVM	Autotuned	RV32GC	False	-
`29574349` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	0	NHWC	TVM	Autotuned	RV32GC	False	-
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	128	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	256	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	512	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	1024	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	2048	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`31529514` ( 0.5x )	`107730` ( 1.266 )	`59536` ( 3.097 )	4096	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	128	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	256	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	512	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	1024	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	2048	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`29578958` ( 0.6x )	`101832` ( 1.197 )	`59536` ( 3.097 )	4096	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101288` ( 1.19 )	`51364` ( 2.672 )	128	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101286` ( 1.19 )	`51364` ( 2.672 )	256	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101286` ( 1.19 )	`51364` ( 2.672 )	512	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101286` ( 1.19 )	`51364` ( 2.672 )	1024	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101286` ( 1.19 )	`51364` ( 2.672 )	2048	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`15873104` ( 1.1x )	`101288` ( 1.19 )	`51364` ( 2.672 )	4096	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	128	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	256	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	512	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	1024	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	2048	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`29574345` ( 0.6x )	`101974` ( 1.198 )	`59536` ( 3.097 )	4096	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`28034895` ( 0.6x )	`119608` ( 1.406 )	`59484` ( 3.094 )	0	NCHW	TVM	Fallback	RV32GCP	False	-
`26289773` ( 0.6x )	`113910` ( 1.339 )	`59484` ( 3.094 )	0	NHWC	TVM	Fallback	RV32GCP	False	-
`12897999` ( 1.3x )	`113210` ( 1.331 )	`51312` ( 2.669 )	0	NCHW	TVM	Autotuned	RV32GCP	False	-
`26285154` ( 0.6x )	`114032` ( 1.34 )	`59484` ( 3.094 )	0	NHWC	TVM	Autotuned	RV32GCP	False	-
`16859464` ( Base )	`85086` ( Base )	`19224` ( Base )	0	NHWC	muRISCV-NN	Scalar	RV32GC	False	-
`16859464` ( 1.0x )	`85086` ( 1.0 )	`19224` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`16859464` ( 1.0x )	`85086` ( 1.0 )	`19224` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`16859464` ( 1.0x )	`85086` ( 1.0 )	`19224` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`16859464` ( 1.0x )	`85086` ( 1.0 )	`19224` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`16859464` ( 1.0x )	`85086` ( 1.0 )	`19224` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`16859464` ( 1.0x )	`85086` ( 1.0 )	`19224` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`5986503` ( 2.8x )	`86222` ( 1.013 )	`23688` ( 1.232 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4841391` ( 3.5x )	`86222` ( 1.013 )	`23688` ( 1.232 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4273727` ( 3.9x )	`86222` ( 1.013 )	`23688` ( 1.232 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4232015` ( 4.0x )	`86222` ( 1.013 )	`23688` ( 1.232 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4232015` ( 4.0x )	`86222` ( 1.013 )	`23688` ( 1.232 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4235404` ( 4.0x )	`86222` ( 1.013 )	`23688` ( 1.232 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`13443817` ( 1.3x )	`97020` ( 1.14 )	`19172` ( 0.997 )	0	NHWC	muRISCV-NN	Scalar	RV32GCP	False	-
`15707734` ( 1.1x )	`99208` ( 1.166 )	`20324` ( 1.057 )	0	NHWC	muRISCV-NN	Packed	RV32GCP	False	-

Image Classification (`resnet`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Unroll	Auto-Vectorization
`132014570` ( 0.6x )	`217004` ( 1.615 )	`108448` ( 1.953 )	0	NCHW	TVM	Fallback	RV32GC	False	-
`115757507` ( 0.7x )	`207908` ( 1.548 )	`108448` ( 1.953 )	0	NHWC	TVM	Fallback	RV32GC	False	-
`57193829` ( 1.4x )	`211090` ( 1.571 )	`92264` ( 1.662 )	0	NCHW	TVM	Autotuned	RV32GC	False	-
`115753589` ( 0.7x )	`208014` ( 1.549 )	`108448` ( 1.953 )	0	NHWC	TVM	Autotuned	RV32GC	False	-
`132147940` ( 0.6x )	`216992` ( 1.615 )	`108448` ( 1.953 )	128	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`132147940` ( 0.6x )	`216994` ( 1.615 )	`108448` ( 1.953 )	256	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`132147940` ( 0.6x )	`216992` ( 1.615 )	`108448` ( 1.953 )	512	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`132147940` ( 0.6x )	`216992` ( 1.615 )	`108448` ( 1.953 )	1024	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`132147940` ( 0.6x )	`216996` ( 1.615 )	`108448` ( 1.953 )	2048	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`132147940` ( 0.6x )	`216992` ( 1.615 )	`108448` ( 1.953 )	4096	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207910` ( 1.548 )	`108448` ( 1.953 )	128	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207908` ( 1.548 )	`108448` ( 1.953 )	256	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207910` ( 1.548 )	`108448` ( 1.953 )	512	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207908` ( 1.548 )	`108448` ( 1.953 )	1024	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207908` ( 1.548 )	`108448` ( 1.953 )	2048	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`115757501` ( 0.7x )	`207908` ( 1.548 )	`108448` ( 1.953 )	4096	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211098` ( 1.571 )	`92264` ( 1.662 )	128	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211098` ( 1.571 )	`92264` ( 1.662 )	256	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211098` ( 1.571 )	`92264` ( 1.662 )	512	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211098` ( 1.571 )	`92264` ( 1.662 )	1024	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211098` ( 1.571 )	`92264` ( 1.662 )	2048	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`57193823` ( 1.4x )	`211098` ( 1.571 )	`92264` ( 1.662 )	4096	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208014` ( 1.549 )	`108448` ( 1.953 )	128	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208012` ( 1.548 )	`108448` ( 1.953 )	256	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208014` ( 1.549 )	`108448` ( 1.953 )	512	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208014` ( 1.549 )	`108448` ( 1.953 )	1024	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208014` ( 1.549 )	`108448` ( 1.953 )	2048	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`115753583` ( 0.7x )	`208014` ( 1.549 )	`108448` ( 1.953 )	4096	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`134393924` ( 0.6x )	`229126` ( 1.706 )	`108396` ( 1.952 )	0	NCHW	TVM	Fallback	RV32GCP	False	-
`102349260` ( 0.8x )	`219946` ( 1.637 )	`108396` ( 1.952 )	0	NHWC	TVM	Fallback	RV32GCP	False	-
`44503696` ( 1.8x )	`222790` ( 1.659 )	`92212` ( 1.661 )	0	NCHW	TVM	Autotuned	RV32GCP	False	-
`102345331` ( 0.8x )	`220022` ( 1.638 )	`108396` ( 1.952 )	0	NHWC	TVM	Autotuned	RV32GCP	False	-
`81013965` ( Base )	`134332` ( Base )	`55528` ( Base )	0	NHWC	muRISCV-NN	Scalar	RV32GC	False	-
`81013965` ( 1.0x )	`134332` ( 1.0 )	`55528` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`81013965` ( 1.0x )	`134332` ( 1.0 )	`55528` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`81013965` ( 1.0x )	`134332` ( 1.0 )	`55528` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`81013965` ( 1.0x )	`134332` ( 1.0 )	`55528` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`81013965` ( 1.0x )	`134332` ( 1.0 )	`55528` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`81013965` ( 1.0x )	`134332` ( 1.0 )	`55528` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`15492114` ( 5.2x )	`135822` ( 1.011 )	`55528` ( 1.0 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`9805106` ( 8.3x )	`135822` ( 1.011 )	`55528` ( 1.0 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`7211458` ( 11.2x )	`135822` ( 1.011 )	`55528` ( 1.0 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`5945938` ( 13.6x )	`135822` ( 1.011 )	`55528` ( 1.0 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`5004247` ( 16.2x )	`135822` ( 1.011 )	`55528` ( 1.0 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`4753720` ( 17.0x )	`135822` ( 1.011 )	`55528` ( 1.0 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`62991355` ( 1.3x )	`146268` ( 1.089 )	`55476` ( 0.999 )	0	NHWC	muRISCV-NN	Scalar	RV32GCP	False	-
`68441624` ( 1.2x )	`149084` ( 1.11 )	`55476` ( 0.999 )	0	NHWC	muRISCV-NN	Packed	RV32GCP	False	-

Anomaly Detection (`toycar`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Unroll	Auto-Vectorization
`3230152` ( 0.6x )	`580142` ( 1.839 )	`5580` ( 1.167 )	0	NCHW	TVM	Fallback	RV32GC	False	-
`3230152` ( 0.6x )	`580142` ( 1.839 )	`5580` ( 1.167 )	0	NHWC	TVM	Fallback	RV32GC	False	-
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	0	NCHW	TVM	Autotuned	RV32GC	False	-
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	0	NHWC	TVM	Autotuned	RV32GC	False	-
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	128	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	256	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	512	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	1024	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	2048	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	4096	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	128	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	256	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	512	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	1024	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	2048	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`3230152` ( 0.6x )	`580130` ( 1.839 )	`5580` ( 1.167 )	4096	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	128	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	256	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	512	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	1024	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	2048	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	4096	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	128	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	256	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	512	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	1024	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	2048	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2147464` ( 0.8x )	`609314` ( 1.932 )	`6892` ( 1.442 )	4096	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`2867860` ( 0.6x )	`592550` ( 1.879 )	`5528` ( 1.156 )	0	NCHW	TVM	Fallback	RV32GCP	False	-
`2867860` ( 0.6x )	`592550` ( 1.879 )	`5528` ( 1.156 )	0	NHWC	TVM	Fallback	RV32GCP	False	-
`1769706` ( 1.0x )	`618794` ( 1.962 )	`6840` ( 1.431 )	0	NCHW	TVM	Autotuned	RV32GCP	False	-
`1769706` ( 1.0x )	`618794` ( 1.962 )	`6840` ( 1.431 )	0	NHWC	TVM	Autotuned	RV32GCP	False	-
`1820383` ( Base )	`315396` ( Base )	`4780` ( Base )	0	NHWC	muRISCV-NN	Scalar	RV32GC	False	-
`1820385` ( 1.0x )	`315402` ( 1.0 )	`4780` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`1820385` ( 1.0x )	`315402` ( 1.0 )	`4780` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`1820385` ( 1.0x )	`315402` ( 1.0 )	`4780` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`1820385` ( 1.0x )	`315402` ( 1.0 )	`4780` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`1820385` ( 1.0x )	`315402` ( 1.0 )	`4780` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`1820385` ( 1.0x )	`315402` ( 1.0 )	`4780` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`627187` ( 2.9x )	`315802` ( 1.001 )	`4780` ( 1.0 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`508819` ( 3.6x )	`315802` ( 1.001 )	`4780` ( 1.0 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`449635` ( 4.0x )	`315802` ( 1.001 )	`4780` ( 1.0 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`420379` ( 4.3x )	`315802` ( 1.001 )	`4780` ( 1.0 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`416695` ( 4.4x )	`315802` ( 1.001 )	`4780` ( 1.0 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`414811` ( 4.4x )	`315802` ( 1.001 )	`4780` ( 1.0 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`1656904` ( 1.1x )	`327620` ( 1.039 )	`4728` ( 0.989 )	0	NHWC	muRISCV-NN	Scalar	RV32GCP	False	-
`996956` ( 1.8x )	`327838` ( 1.039 )	`4728` ( 0.989 )	0	NHWC	muRISCV-NN	Packed	RV32GCP	False	-

Visual Wake Words (`vww`)

Cycles (Speedup)	Total ROM (rel.)	Total RAM (rel.)	VLEN	Layout	Kernels	Mode	Arch	Unroll	Auto-Vectorization
`91779188` ( 0.5x )	`540914` ( 1.696 )	`181056` ( 2.113 )	0	NCHW	TVM	Fallback	RV32GC	False	-
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	0	NHWC	TVM	Fallback	RV32GC	False	-
`46542367` ( 1.1x )	`521918` ( 1.637 )	`181056` ( 2.113 )	0	NCHW	TVM	Autotuned	RV32GC	False	-
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	0	NHWC	TVM	Autotuned	RV32GC	False	-
`91775924` ( 0.5x )	`540822` ( 1.696 )	`181056` ( 2.113 )	128	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`91775924` ( 0.5x )	`540822` ( 1.696 )	`181056` ( 2.113 )	256	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`91775924` ( 0.5x )	`540820` ( 1.696 )	`181056` ( 2.113 )	512	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`91775924` ( 0.5x )	`540822` ( 1.696 )	`181056` ( 2.113 )	1024	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`91775924` ( 0.5x )	`540822` ( 1.696 )	`181056` ( 2.113 )	2048	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`91775924` ( 0.5x )	`540820` ( 1.696 )	`181056` ( 2.113 )	4096	NCHW	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	128	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	256	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520540` ( 1.632 )	`181056` ( 2.113 )	512	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	1024	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	2048	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`86076684` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	4096	NHWC	TVM	Fallback	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521910` ( 1.637 )	`181056` ( 2.113 )	128	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521912` ( 1.637 )	`181056` ( 2.113 )	256	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521910` ( 1.637 )	`181056` ( 2.113 )	512	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521910` ( 1.637 )	`181056` ( 2.113 )	1024	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521910` ( 1.637 )	`181056` ( 2.113 )	2048	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`46557547` ( 1.1x )	`521908` ( 1.637 )	`181056` ( 2.113 )	4096	NCHW	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	128	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	256	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	512	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	1024	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520544` ( 1.632 )	`181056` ( 2.113 )	2048	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`86076687` ( 0.6x )	`520542` ( 1.632 )	`181056` ( 2.113 )	4096	NHWC	TVM	Autotuned	RV32GCV	False	Loop+SLP
`81680879` ( 0.6x )	`551532` ( 1.729 )	`181004` ( 2.113 )	0	NCHW	TVM	Fallback	RV32GCP	False	-
`76410709` ( 0.7x )	`531796` ( 1.668 )	`181004` ( 2.113 )	0	NHWC	TVM	Fallback	RV32GCP	False	-
`37728221` ( 1.3x )	`533010` ( 1.671 )	`181004` ( 2.113 )	0	NCHW	TVM	Autotuned	RV32GCP	False	-
`76410712` ( 0.7x )	`531800` ( 1.668 )	`181004` ( 2.113 )	0	NHWC	TVM	Autotuned	RV32GCP	False	-
`49801852` ( Base )	`318910` ( Base )	`85672` ( Base )	0	NHWC	muRISCV-NN	Scalar	RV32GC	False	-
`49801852` ( 1.0x )	`318910` ( 1.0 )	`85672` ( 1.0 )	128	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`49801852` ( 1.0x )	`318910` ( 1.0 )	`85672` ( 1.0 )	256	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`49801852` ( 1.0x )	`318910` ( 1.0 )	`85672` ( 1.0 )	512	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`49801852` ( 1.0x )	`318910` ( 1.0 )	`85672` ( 1.0 )	1024	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`49801852` ( 1.0x )	`318910` ( 1.0 )	`85672` ( 1.0 )	2048	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`49801852` ( 1.0x )	`318910` ( 1.0 )	`85672` ( 1.0 )	4096	NHWC	muRISCV-NN	Scalar	RV32GCV	False	Loop+SLP
`13603233` ( 3.7x )	`320342` ( 1.004 )	`85672` ( 1.0 )	128	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`10274605` ( 4.8x )	`320342` ( 1.004 )	`85672` ( 1.0 )	256	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`8985483` ( 5.5x )	`320342` ( 1.004 )	`85672` ( 1.0 )	512	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`8480984` ( 5.9x )	`320342` ( 1.004 )	`85672` ( 1.0 )	1024	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`8432370` ( 5.9x )	`320342` ( 1.004 )	`85672` ( 1.0 )	2048	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`8435759` ( 5.9x )	`320342` ( 1.004 )	`85672` ( 1.0 )	4096	NHWC	muRISCV-NN	Vector	RV32GCV	False	-
`40880383` ( 1.2x )	`330620` ( 1.037 )	`85620` ( 0.999 )	0	NHWC	muRISCV-NN	Scalar	RV32GCP	False	-
`49304040` ( 1.0x )	`333080` ( 1.044 )	`85620` ( 0.999 )	0	NHWC	muRISCV-NN	Packed	RV32GCP	False	-

Original data

Click here to download the raw files for this benchmark.

2024-11-26

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-21

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-19

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-11-18

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-07-12

TFLM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

2024-06-29

TFLM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

TVM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

2024-03-02

TFLM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

TVM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

2024-02-26

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-23

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-22

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-20

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-11

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2023-12-22

TFLM

GCC

s

SpikeRV32

TVM

GCC

s

SpikeRV32

2024-11-26

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-21

TFLM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

TVM

GCC

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

LLVM

O3

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

Os

Spike (RV32) (spike_rv32)

Spike (RV32, Embedded) (spike_rv32_min)

Spike (RV64) (spike_rv64)

2024-11-19

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-11-18

TFLM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV32) (spike_rv32)

Spike (RV64) (spike_rv64)

2024-07-12

TFLM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

TVM

GCC

Os

Spike (RV64) (spike_rv64)

LLVM

Os

Spike (RV64) (spike_rv64)

2024-06-29

TFLM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

TVM

GCC

Os

SpikeRV32

LLVM

Os

SpikeRV32

2024-03-02

TFLM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

TVM

GCC

O3

SpikeRV32

Os

SpikeRV32

LLVM

O3

SpikeRV32

Os

SpikeRV32

2024-02-26

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-23

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-22

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-20

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2024-02-11

TFLM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

TVM

GCC

s

SpikeRV32

LLVM

s

SpikeRV32

2023-12-22

TFLM

GCC

s

SpikeRV32

TVM

GCC

s

SpikeRV32

Benchmarks 2024 03 02 TVM GCC Os

Setup

Simulator

Toolchains

Models

Package Versions

Miscellaneous

Results (Framework: tvm, Backend: tvmaot, Toolchain: gcc, Flags: -Os)

Audio Wake Words (aww)

Image Classification (resnet)

Anomaly Detection (toycar)

Visual Wake Words (vww)

Original data

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Audio Wake Words (`aww`)

Image Classification (`resnet`)

Anomaly Detection (`toycar`)

Visual Wake Words (`vww`)