Benchmark

Benchmark

See readme page here https://github.com/dmlc/tvm/tree/master/apps/benchmark on how to get these numbers

ARM CPU

Note: If a board has big.LITTLE archtiecture, we will use all big cores. Otherwise, we will use all cores. In the following device specifications, we only list the cores being used.

• Firefly-RK3399 : 2 x Cortex A73 1.8Ghz

$ python3 arm_cpu_imagenet_bench.py --device rk3399 --rpc-key rk3399
--------------------------------------------------
Network Name         Mean Inference Time (std dev)
--------------------------------------------------
squeezenet v1.1      44.15 ms            (0.64 ms)
mobilenet            82.23 ms            (0.67 ms)
resnet-18            168.71 ms           (0.05 ms)
vgg-16               972.03 ms           (1.75 ms)  

• Raspberry Pi 3B : 4 x Cortex A53 1.2Ghz

$ python3 arm_cpu_imagenet_bench.py --device rasp3b --rpc-key rasp3b
--------------------------------------------------
Network Name         Mean Inference Time (std dev)
--------------------------------------------------
squeezenet v1.1      94.59 ms            (0.04 ms)
mobilenet            148.82 ms           (0.18 ms)
resnet-18            347.30 ms           (0.25 ms)
vgg-16               crashed due to out of memeory

• Huawei P20 Pro / Mate10 Pro (Soc: HiSilicon Kirin 970) : (4 x Cortex A73 2.36GHz)

$ python3 arm_cpu_imagenet_bench.py --device p20pro --rpc-key p20pro
--------------------------------------------------
Network Name         Mean Inference Time (std dev)
-------------------------------------------------
squeezenet v1.1      29.33 ms            (0.61 ms)
mobilenet            47.47 ms            (0.65 ms)
resnet-18            84.71 ms            (0.32 ms)
vgg-16               574.62 ms           (2.14 ms)

• Google Pixel 2 (Soc: Qualcomm Snapdragon 835) : (4 × Kyro 2.35 GHz)

$ python3 arm_cpu_imagenet_bench.py --device pixel2 --rpc-key pixel2
--------------------------------------------------
Network Name         Mean Inference Time (std dev)
--------------------------------------------------
squeezenet v1.1      27.74 ms            (0.41 ms)
mobilenet            42.05 ms            (0.08 ms)
resnet-18            67.28 ms            (0.05 ms)
vgg-16               427.75 ms           (8.58 ms)