chainer_example_performance_report

Chainer's Examples Execution Time on ClPy and CuPy

Execution Environments

• src: v2.1.0b1
• Primary machine (vega)
  • OS: Ubuntu 16.04.4 LTS
  • CPU: Core i7-7700
  • MEM: 8GB
  • GPU: AMD Radeon Vega Frontier Edition (Air Cooled)
  • SDK: amdgpu-pro-18.20-621984
• Old Primary machine (furyx)
  • OS: Ubuntu 16.04.4 LTS
  • CPU: Core i7-4790
  • MEM: 16GB
  • GPU: AMD Radeon R9 Fury X
  • SDK: amdgpu-pro-17.50-511655
• Secondary machine (titanv)
  • OS: Ubuntu 16.04.5 LTS
  • CPU: Core i9-7900X
  • MEM: 64GB
  • GPU: NVIDIA TITAN V
  • SDK: CUDA 9.2
  • cuDNN 7.1
• Old Secondary machine (gf1060)
  • OS: Ubuntu 16.04.5 LTS
  • CPU: Core i7-4790
  • MEM: 8GB
  • GPU: NVIDIA GeForce GTX 1060
  • SDK: CUDA 9.2
  • cuDNN 7.1

Example settings

mnist

• src: chainer/examples/mnist/train_mnist.py
• cmd: python train_mnist.py -g 0
• execution time: See elapsed_time at epoch 20.

ptb

• src: chainer/examples/ptb/train_ptb.py
• change: Change train_ptb.py as below to get elapsed_time.
• cmd: python train_ptb.py -g 0 -e 1
• execution time: See elapsed_time at epoch 0 iteration 1000.

diff --git a/examples/ptb/train_ptb.py b/examples/ptb/train_ptb.py
index 055a0dd96..ccd6cda96 100755
--- a/examples/ptb/train_ptb.py
+++ b/examples/ptb/train_ptb.py
@@ -238,7 +238,7 @@ def main():
     trainer.extend(extensions.LogReport(postprocess=compute_perplexity,
                                         trigger=(interval, 'iteration')))
     trainer.extend(extensions.PrintReport(
-        ['epoch', 'iteration', 'perplexity', 'val_perplexity']
+        ['epoch', 'iteration', 'perplexity', 'val_perplexity', 'elapsed_time']
     ), trigger=(interval, 'iteration'))
     trainer.extend(extensions.ProgressBar(
         update_interval=1 if args.test else 10))

word2vec

• src: chainer/examples/word2vec/train_word2vec.py
• change: Change train_word2vec.py as below to get elapsed_time.
• cmd: python train_word2vec.py -g 0
• execution time: See elapsed_time at epoch 20.

diff --git a/examples/word2vec/train_word2vec.py b/examples/word2vec/train_word2vec.py
index 1e462b7bb..6b5929710 100755
--- a/examples/word2vec/train_word2vec.py
+++ b/examples/word2vec/train_word2vec.py
@@ -249,7 +249,7 @@ def main():
         val_iter, model, converter=convert, device=args.gpu))
     trainer.extend(extensions.LogReport())
     trainer.extend(extensions.PrintReport(
-        ['epoch', 'main/loss', 'validation/main/loss']))
+        ['epoch', 'main/loss', 'validation/main/loss', 'elapsed_time']))
     trainer.extend(extensions.ProgressBar())
     trainer.run()

ImageNet

• src: chainer/examples/imagenet/train_imagenet.py
• change: Change train_imagenet.py as below to get elapsed_time.
• cmd:
  • nin python train_imagenet.py -B10 -E1 -g0 -a nin
  • googlenetbn python train_imagenet.py -B10 -E1 -g0 -a googlenetbn
  • resnet50 python train_imagenet.py -B10 -E1 -g0 -a resnet50
• execution time: See elapsed_time at iteration 2000.

diff --git a/examples/imagenet/train_imagenet.py b/examples/imagenet/train_imagenet.py
index e2babed..168c710 100755
--- a/examples/imagenet/train_imagenet.py
+++ b/examples/imagenet/train_imagenet.py
@@ -150,7 +150,7 @@ def main():
     trainer.extend(extensions.observe_lr(), trigger=log_interval)
     trainer.extend(extensions.PrintReport([
         'epoch', 'iteration', 'main/loss', 'validation/main/loss',
-        'main/accuracy', 'validation/main/accuracy', 'lr'
+        'main/accuracy', 'validation/main/accuracy', 'lr', 'elapsed_time'
     ]), trigger=log_interval)
     trainer.extend(extensions.ProgressBar(update_interval=10))

Chainer's Example Execution time(sec)

machine	gpu	mnist	ptb	word2vec	ImageNet(googlenetbn)	ImageNet(nin)	ImageNet(resnet50)
Primary	vega	104.73	945.09	467.33	3914.32	586.25	4197.82
Secondary	titanv(ClPy)	61.13	328.05	192.12	708.35	141.10	678.31
Secondary	titanv(CuPy)	39.06	157.97	27.84	375.46	115.56	278.37
Secondary	titanv(CuPy, cuDNN enabled)	39.21	156.49	28.31	225.45	103.45	167.53
Old Primary	furyx	128.24	961.88	603.98	2595.17	479.68	3106.52
Old Secondary	gf1060(ClPy)	78.98	446.99	129.84	1250.42	273.04	1542.72
Old Secondary	gf1060(CuPy)	42.37	205.61	43.16	544.78	132.99	804.80
Old Secondary	gf1060(CuPy, cuDNN enabled)	41.27	205.41	43.64	251.64	106.42	284.73