-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Benchmark
Lianmin Zheng edited this page Sep 20, 2018
·
24 revisions
Note: If a board has big.LITTLE architecture, we will use all big cores. Otherwise, we will use all cores. In the following device specifications, we only list the cores being used. We use dtype=float32 and batch_size=1 for the following benchmark.
- Firefly-RK3399 : 2 x Cortex A72 1.8Ghz
--------------------------------------------------
Network Name Mean Inference Time (std dev)
--------------------------------------------------
squeezenet v1.1 48.87 ms (1.07 ms)
mobilenet 77.86 ms (0.09 ms)
resnet-18 162.55 ms (0.14 ms)
vgg-16 912.44 ms (0.32 ms)- Raspberry Pi 3B : 4 x Cortex A53 1.2Ghz
--------------------------------------------------
Network Name Mean Inference Time (std dev)
--------------------------------------------------
squeezenet v1.1 92.34 ms (0.07 ms)
mobilenet 124.34 ms (0.11 ms)
resnet-18 325.06 ms (0.23 ms)
vgg-16 crashed due to out of memeory- Huawei P20 Pro / Mate10 Pro (Soc: HiSilicon Kirin 970) : (4 x Cortex A73 2.36GHz)
--------------------------------------------------
Network Name Mean Inference Time (std dev)
-------------------------------------------------
squeezenet v1.1 27.53 ms (1.14 ms)
mobilenet 42.53 ms (0.31 ms)
resnet-18 76.74 ms (0.18 ms)
vgg-16 479.84 ms (0.92 ms)- Google Pixel 2 (Soc: Qualcomm Snapdragon 835) : (4 × Kyro 2.35 GHz)
--------------------------------------------------
Network Name Mean Inference Time (std dev)
--------------------------------------------------
squeezenet v1.1 23.57 ms (0.42 ms)
mobilenet 38.86 ms (0.11 ms)
resnet-18 63.95 ms (0.03 ms)
vgg-16 407.75 ms (9.57 ms)
- PYNQ (2 x Cortex-A9 650MHz)
--------------------------------------------------
Network Name Mean Inference Time (std dev)
--------------------------------------------------
squeezenet v1.1 452.40 ms (0.09 ms)
mobilenet 772.16 ms (0.25 ms)
resnet-18 1243.49 ms (0.67 ms)
vgg-16 crashed due to out of memeory
dtype=float32 and batch_size=1
- Mali T860 MP4
--------------------------------------------------
Network Name Mean Inference Time (std dev)
--------------------------------------------------
squeezenet_v1.1 58.78 ms (1.05 ms)
mobilenet 78.39 ms (1.33 ms)
resnet-18 125.77 ms (3.07 ms)
vgg-16 676.56 ms (4.66 ms)dtype=float32 and batch_size=1
- NVIDIA 1080 Ti
--------------------------------------------------
Network Name Mean Inference Time (std dev)
--------------------------------------------------
resnet-50 2.69 ms (0.04 ms)
mobilenet 0.62 ms (0.00 ms)
vgg-19 4.56 ms (0.01 ms)
inception_v3 5.75 ms (0.00 ms)- NVIDIA Titan X
--------------------------------------------------
Network Name Mean Inference Time (std dev)
--------------------------------------------------
resnet-50 4.34 ms (0.03 ms)
mobilenet 0.98 ms (0.00 ms)
vgg-19 7.38 ms (0.00 ms)
inception_v3 9.65 ms (0.04 ms)dtype=float32 and batch_size=1
- Vega FE (gfx900)
--------------------------------------------------
Network Name Mean Inference Time (std dev)
--------------------------------------------------
resnet-50 4.44 ms (0.01 ms)
mobilenet 1.01 ms (0.00 ms)
vgg-19 7.03 ms (0.00 ms)
inception_v3 8.80 ms (0.03 ms)see readme page here https://github.com/dmlc/tvm/tree/master/apps/benchmark on how to get these numbers