Speed Benchmarking

[IainNZ]

release-0.6

• Write times seem a lot faster than release-0.5?!

test/perf/macro.jl
Test 1
  Running N=20...
    N=20 min 0.00623067
    N=20 min 0.038118125
  Running N=50...
    N=50 min 0.139074797
    N=50 min 0.64644231
  Running N=100...
    N=100 min 0.968655834
    N=100 min 9.051508205

test/perf/speed.jl
PMEDIAN BUILD MIN=0.362609479  MED=0.387190535
PMEDIAN WRITE MIN=1.251096612  MED=1.322590987
CONT5 BUILD   MIN=0.201082026  MED=0.285081277
CONT5 WRITE   MIN=1.036570192  MED=1.068279631

test/perf/vector_speedtest.jl
n = 10
Vector with sum{}: 4.335e-6
Vector with dot() : 5.433e-6
2D Matrix with sum{}: 3.694e-6
2D Matrix with bigdot(): 2.266e-6
3D Matrix with sum{}: 4.2989e-5
3D Matrix with dot(): 4.437e-6
n = 50
Vector with sum{}: 3.082e-6
Vector with dot() : 2.651e-6
2D Matrix with sum{}: 8.796e-5
2D Matrix with bigdot(): 3.6112e-5
3D Matrix with sum{}: 0.00468567
3D Matrix with dot(): 0.001401678
n = 100
Vector with sum{}: 4.183e-6
Vector with dot() : 2.456e-6
2D Matrix with sum{}: 0.000272269
2D Matrix with bigdot(): 6.5199e-5
3D Matrix with sum{}: 0.038896289
3D Matrix with dot(): 0.010648981
n = 200
Vector with sum{}: 5.616e-6
Vector with dot() : 3.4e-6
2D Matrix with sum{}: 0.000998217
2D Matrix with bigdot(): 0.000119423
3D Matrix with sum{}: 0.597877863
3D Matrix with dot(): 0.410444616
n = 300
Vector with sum{}: 8.09e-6
Vector with dot() : 3.85e-6
2D Matrix with sum{}: 0.002956568
2D Matrix with bigdot(): 0.000937832
3D Matrix with sum{}: 2.6192164
3D Matrix with dot(): 1.392346446
n = 400
Vector with sum{}: 9.272e-6
Vector with dot() : 4.241e-6
2D Matrix with sum{}: 0.004455528
2D Matrix with bigdot(): 0.001575292
3D Matrix with sum{}: 11.289661762
3D Matrix with dot(): 6.462194309

[IainNZ]

release-0.5

test/speed.jl
PMEDIAN BUILD MIN=0.350347822  MED=0.363185937
PMEDIAN WRITE MIN=3.168698252  MED=3.290545628
CONT5 BUILD   MIN=0.202714697  MED=0.213233694
CONT5 WRITE   MIN=2.340976625  MED=2.455608324

test/vector_speedtest.jl  (modified to be like release-0.6 version)
n = 10
Vector with sum{}: 3.812e-6
Vector with dot() : 7.773e-6
2D Matrix with sum{}: 3.808e-6
2D Matrix with bigdot(): 2.246e-6
3D Matrix with sum{}: 4.2202e-5
3D Matrix with dot(): 3.972e-6
n = 50
Vector with sum{}: 2.33e-6
Vector with dot() : 2.048e-6
2D Matrix with sum{}: 9.3416e-5
2D Matrix with bigdot(): 4.0676e-5
3D Matrix with sum{}: 0.004407185
3D Matrix with dot(): 0.001271612
n = 100
Vector with sum{}: 3.735e-6
Vector with dot() : 2.825e-6
2D Matrix with sum{}: 0.00029787
2D Matrix with bigdot(): 0.000397556
3D Matrix with sum{}: 0.040780475
3D Matrix with dot(): 0.013942868
n = 200
Vector with sum{}: 6.162e-6
Vector with dot() : 3.334e-6
2D Matrix with sum{}: 0.00088826
2D Matrix with bigdot(): 0.000266557
3D Matrix with sum{}: 0.605103607
3D Matrix with dot(): 0.472576798
n = 300
Vector with sum{}: 1.0166e-5
Vector with dot() : 5.485e-6
2D Matrix with sum{}: 0.003260792
2D Matrix with bigdot(): 0.001037459
3D Matrix with sum{}: 2.673809173
3D Matrix with dot(): 1.477282376
n = 400
Vector with sum{}: 8.665e-6
Vector with dot() : 4.129e-6
2D Matrix with sum{}: 0.004742407
2D Matrix with bigdot(): 0.001706899
3D Matrix with sum{}: 11.112408347
3D Matrix with dot(): 6.626319469

[mlubin]

LGTM. Same build time, good improvement in write time.

[joehuchette]

#42 should probably be revisited. What's the deal with the computer from the grant?

[mlubin]

Dunno if apple will ever update the mac mini.

[IainNZ]

Was the write time from switching to print_shortest or is it something else?

[mlubin]

I believe that's it.

[IainNZ]

@joehuchette we could always stick JP's Dell box in the ORC computer room and run it from there (or under a desk in the old computer room, for that matter)

[joehuchette]

That could work. Is there room in the new computer setup?

[IainNZ]

I don't believe there is a free ethernet jack... apparently MIT IST views switches dimly. Just how loud is that Dell machine?

[joehuchette]

I mean it's audible, but not terribly so. It didn't bother me in my cube, but it was noticeable in y'alls empty office. You going to be around tomorrow? We can experiment.

[IainNZ]

Yeah I should be around

[mlubin]

Looks like this is a dup of #42 at this point?