-
Notifications
You must be signed in to change notification settings - Fork 12
/
BenchmarkData.txt
415 lines (391 loc) · 13.5 KB
/
BenchmarkData.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian
5 0.058 0.025 0.02 0.002 0.014
55 0.491 0.009 0.285 0.011 0.155
105 0.187 0.012 0.812 0.013 0.482
155 0.61 0.009 11.687 0.014 9.556
205 0.477 0.006 15.856 0.011 12.854
255 0.765 0.008 19.737 0.019 16.381
305 0.878 0.006 23.669 0.017 19.534
355 1.157 0.023 27.326 0.018 22.741
405 1.527 0.005 31.14 0.024 25.97
455 1.891 0.006 35.071 0.025 29.165
505 2.781 0.008 38.67 0.038 32.283
555 2.863 0.007 42.774 0.03 35.519
605 3.375 0.007 46.374 0.027 38.734
655 4.322 0.007 50.133 0.033 41.959
705 4.697 0.008 53.9 0.037 45.137
755 5.659 0.009 58.499 0.05 48.535
with sum, without small functions
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian
5 0.062 0.025 0.012 0.003 0.013
55 0.506 0.009 0.278 0.012 0.35
105 0.229 0.015 0.249 0.014 1.208
155 0.456 0.01 1.292 0.015 3.317
205 0.554 0.006 1.712 0.009 5.372
255 0.695 0.007 2.128 0.012 7.918
305 0.888 0.008 2.547 0.017 10.954
355 1.195 0.008 2.951 0.018 14.412
with sum, small functions
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian
5 0.058 0.025 0.013 0.002 0.012
55 0.553 0.009 0.109 0.011 0.352
105 0.208 0.013 0.241 0.016 1.183
155 0.363 0.009 1.288 0.016 2.438
205 0.512 0.006 1.786 0.015 4.186
255 0.707 0.007 2.163 0.021 6.498
305 1.169 0.006 2.547 0.019 9.177
355 1.265 0.007 2.972 0.031 12.325
405 1.681 0.007 3.385 0.032 16.775
455 1.954 0.006 3.791 0.037 20.142
C++
N=5: Grad=0.001523 Hess=0.002807
N=55: Grad=0.018744 Hess=0.264852
N=105: Grad=0.031632 Hess=0.962019
N=155: Grad=0.047682 Hess=2.08109
N=205: Grad=0.063254 Hess=3.6405
N=255: Grad=0.077508 Hess=5.68046
N=305: Grad=0.093359 Hess=8.07915
N=355: Grad=0.111471 Hess=10.9616
N=405: Grad=0.131659 Hess=14.3319
N=455: Grad=0.164299 Hess=18.0198
N=505: Grad=0.185296 Hess=22.538
N=555: Grad=0.171085 Hess=26.8645
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian
5 0.056 0.025 0.011 0.002 0.011
55 0.601 0.01 0.109 0.038 0.067
105 0.193 0.012 0.221 0.021 0.126
155 0.349 0.008 1.283 0.035 0.203
205 0.569 0.006 1.689 0.061 0.309
255 0.823 0.007 2.099 0.08 0.424
305 0.972 0.167 2.515 0.091 0.603
355 1.189 0.006 2.921 0.113 0.731
405 1.68 0.007 3.324 0.191 0.915
455 1.984 0.006 3.739 0.132 1.195
505 2.435 0.008 4.142 0.173 1.559
555 3.009 0.006 4.557 0.229 2.042
605 3.44 0.008 4.959 0.197 2.598
655 4.443 0.008 5.388 0.256 3.204
705 5.102 0.012 5.789 0.313 3.949
755 5.556 0.009 6.207 0.295 4.65
805 6.19 0.008 6.592 0.352 5.399
Final Value=3.399999999857267E8
g++ -O3
N=5: Grad=0.001151 Hess=0.000818
N=55: Grad=0.011713 Hess=0.007993
N=105: Grad=0.022845 Hess=0.01805
N=155: Grad=0.035496 Hess=0.02596
N=205: Grad=0.048052 Hess=0.038288
N=255: Grad=0.06439 Hess=0.094836
N=305: Grad=0.068387 Hess=0.057918
N=355: Grad=0.089015 Hess=0.069783
N=405: Grad=0.102814 Hess=0.106349
N=455: Grad=0.115396 Hess=0.091371
N=505: Grad=0.157484 Hess=0.131465
N=555: Grad=0.178366 Hess=0.156674
N=605: Grad=0.187632 Hess=0.169453
N=655: Grad=0.207495 Hess=0.150046
N=705: Grad=0.2259 Hess=0.224388
N=755: Grad=0.241367 Hess=0.27117
N=805: Grad=0.309976 Hess=0.495865
Final Value=3.4e+08
yliu@yliu-dev:~/workspace_cpp/SymLLVM$ g++ benchmark-rosenbrock-manual.cpp -o run
yliu@yliu-dev:~/workspace_cpp/SymLLVM$ ./run
N=5: Grad=0.00733 Hess=0.00492
N=55: Grad=0.093892 Hess=0.060972
N=105: Grad=0.177875 Hess=0.122079
N=155: Grad=0.274418 Hess=0.171955
N=205: Grad=0.362189 Hess=0.231552
N=255: Grad=0.455115 Hess=0.448342
N=305: Grad=0.548107 Hess=0.341249
N=355: Grad=0.637069 Hess=0.397283
N=405: Grad=0.730289 Hess=0.458382
N=455: Grad=0.806885 Hess=0.513731
N=505: Grad=0.895405 Hess=0.624652
N=555: Grad=0.988364 Hess=0.63463
N=605: Grad=1.07374 Hess=0.731887
N=655: Grad=1.16435 Hess=0.801067
N=705: Grad=1.25048 Hess=0.877741
N=755: Grad=1.40263 Hess=0.907993
N=805: Grad=1.44907 Hess=1.01174
Final Value=3.4e+08
jit exprs=30
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian
5 0.057 0.041 0.012 0.041 0.0
55 0.558 0.011 0.094 0.011 0.0
105 0.212 0.018 0.148 0.018 0.0
155 0.334 0.038 0.217 0.038 0.0
205 0.601 0.016 0.282 0.016 0.0
255 0.94 0.013 0.35 0.013 0.0
305 0.902 0.013 0.417 0.013 0.0
355 1.211 0.016 0.487 0.016 0.0
405 1.643 0.018 0.56 0.018 0.0
455 2.032 0.052 0.624 0.052 0.0
505 2.499 0.017 0.699 0.017 0.0
555 2.956 0.017 0.756 0.017 0.0
605 3.493 0.038 0.828 0.038 0.0
655 4.2 0.052 0.892 0.052 0.0
705 4.857 0.022 0.968 0.022 0.0
755 5.623 0.023 1.035 0.023 0.0
805 6.279 0.024 1.099 0.024 0.0
Final Value=-0.01887435763237022
jit exprs=10
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian
5 0.056 0.026 0.013 0.026 0.0
55 0.599 0.016 0.105 0.016 0.0
105 0.197 0.021 0.143 0.021 0.0
155 0.432 0.024 0.22 0.024 0.0
205 0.611 0.039 0.284 0.039 0.0
255 0.937 0.023 0.359 0.023 0.0
305 0.896 0.037 0.42 0.037 0.0
355 1.205 0.041 0.499 0.041 0.0
405 1.566 0.053 0.555 0.053 0.0
455 1.979 0.042 0.652 0.042 0.0
505 2.468 0.036 0.71 0.036 0.0
555 2.943 0.07 0.817 0.07 0.0
605 3.477 0.108 0.885 0.108 0.0
655 4.166 0.084 0.957 0.084 0.0
705 4.892 0.069 1.045 0.069 0.0
755 5.479 0.05 1.124 0.05 0.0
805 6.244 0.069 1.212 0.069 0.0
Final Value=-0.01887435763237022
jit exprs=50
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian
5 0.056 0.024 0.013 0.024 0.0
55 0.518 0.011 0.119 0.011 0.0
105 0.202 0.018 0.165 0.018 0.0
155 0.441 0.016 0.247 0.016 0.0
205 0.469 0.014 0.319 0.014 0.0
255 0.919 0.013 0.402 0.013 0.0
305 0.879 0.011 0.471 0.011 0.0
355 1.198 0.014 0.562 0.014 0.0
405 1.561 0.016 0.619 0.016 0.0
455 1.967 0.012 0.755 0.012 0.0
505 2.538 0.037 0.78 0.037 0.0
555 2.886 0.013 0.849 0.013 0.0
605 3.551 0.014 0.933 0.014 0.0
655 4.14 0.014 1.0 0.014 0.0
705 4.718 0.016 1.072 0.016 0.0
755 5.567 0.016 1.158 0.016 0.0
805 6.352 0.018 1.246 0.018 0.0
Final Value=-0.01887435763237022
JIT exprs=50
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian
5 0.055 0.028 0.011 0.002 0.01
55 0.487 0.012 0.16 0.026 0.065
105 0.482 0.014 0.164 0.027 0.12
155 0.591 0.011 0.246 0.02 0.177
205 0.539 0.011 0.351 0.045 0.24
255 0.903 0.009 0.455 0.022 0.35
305 0.907 0.009 0.473 0.041 0.378
355 1.173 0.01 0.555 0.029 0.459
405 1.57 0.01 0.628 0.03 0.531
455 1.942 0.012 0.701 0.037 0.579
505 2.527 0.011 0.771 0.037 0.644
555 2.966 0.012 0.852 0.068 0.731
605 3.48 0.032 0.921 0.05 0.786
655 4.18 0.036 1.003 0.051 0.82
705 4.836 0.018 1.087 0.055 0.925
755 5.494 0.016 1.155 0.087 1.002
805 6.299 0.018 1.248 0.065 1.066
Final Value=3.399999999857267E8
JIT exprs=36
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian
5 0.055 0.028 0.013 0.002 0.013
55 0.431 0.011 0.111 0.016 0.068
105 0.18 0.013 0.172 0.039 0.137
155 0.312 0.012 0.253 0.023 0.197
205 0.505 0.011 0.325 0.025 0.271
255 0.932 0.01 0.392 0.022 0.395
305 0.889 0.01 0.455 0.026 0.423
355 1.198 0.011 0.53 0.034 0.499
405 1.583 0.011 0.633 0.054 0.579
455 2.051 0.034 0.68 0.039 0.684
505 2.52 0.014 0.836 0.052 0.749
555 2.903 0.016 0.838 0.053 0.827
605 3.366 0.018 0.906 0.052 0.897
655 4.149 0.016 0.979 0.057 0.934
705 4.777 0.016 1.065 0.063 1.055
755 5.462 0.017 1.124 0.082 1.167
805 6.223 0.019 1.208 0.072 1.244
Final Value=6.881736000015767E11
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian
5 0.06 0.025 0.013 0.002 0.011
55 0.483 0.011 0.113 0.016 0.069
105 0.247 0.013 0.173 0.018 0.132
155 0.271 0.012 0.252 0.038 0.202
205 0.582 0.009 0.313 0.021 0.273
255 0.878 0.01 0.396 0.021 0.395
305 0.895 0.011 0.456 0.029 0.423
355 1.203 0.011 0.53 0.058 0.503
405 1.556 0.012 0.628 0.056 0.585
455 1.987 0.031 0.733 0.044 0.629
505 2.515 0.014 0.764 0.047 0.744
555 2.881 0.014 0.845 0.073 0.813
605 3.438 0.014 0.906 0.055 0.897
655 4.091 0.017 0.978 0.081 0.956
705 4.768 0.017 1.056 0.067 1.098
755 5.412 0.018 1.129 0.067 1.141
805 6.224 0.019 1.209 0.073 1.245
Final Value=6.881736000015767E11
yliu@yliu-dev:~/workspace_cpp/SymLLVM$ g++ benchmark-rosenbrock-manual.cpp -o run
yliu@yliu-dev:~/workspace_cpp/SymLLVM$ ./run
N=5: Grad=0.010515 Hess=0.006046
N=55: Grad=0.108079 Hess=0.075674
N=105: Grad=0.205374 Hess=0.14151
N=155: Grad=0.319664 Hess=0.211846
N=205: Grad=0.418111 Hess=0.284775
N=255: Grad=0.51978 Hess=0.502621
N=305: Grad=0.621614 Hess=0.425008
N=355: Grad=0.725213 Hess=0.489046
N=405: Grad=0.823972 Hess=0.56583
N=455: Grad=0.928018 Hess=0.629761
N=505: Grad=1.0312 Hess=0.742515
N=555: Grad=1.13132 Hess=0.798598
N=605: Grad=1.23703 Hess=0.876948
N=655: Grad=1.35474 Hess=0.970951
N=705: Grad=1.44381 Hess=1.0695
N=755: Grad=1.54051 Hess=1.09616
N=805: Grad=1.6446 Hess=1.18514
Final Value=6.88174e+11
yliu@yliu-dev:~/workspace_cpp/SymLLVM$ g++ -O3 benchmark-rosenbrock-manual.cpp -o run
yliu@yliu-dev:~/workspace_cpp/SymLLVM$ ./run
N=5: Grad=0.004045 Hess=0.002874
N=55: Grad=0.017526 Hess=0.011655
N=105: Grad=0.030795 Hess=0.025675
N=155: Grad=0.046783 Hess=0.038598
N=205: Grad=0.064157 Hess=0.056752
N=255: Grad=0.262951 Hess=0.1346
N=305: Grad=0.111215 Hess=0.083569
N=355: Grad=0.146319 Hess=0.098378
N=405: Grad=0.147776 Hess=0.165814
N=455: Grad=0.198877 Hess=0.148392
N=505: Grad=0.232979 Hess=0.196977
N=555: Grad=0.213673 Hess=0.221002
N=605: Grad=0.235828 Hess=0.275431
N=655: Grad=0.226496 Hess=0.228843
N=705: Grad=0.218672 Hess=0.279285
N=755: Grad=0.234185 Hess=0.265979
N=805: Grad=0.2888 Hess=0.315162
Final Value=6.88174e+11
g++ O3 compile time
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian|C Code Compile
5 0.056 0.025 0.014 0.003 0.012 0.162
55 0.479 0.014 0.11 0.019 0.069 0.32
105 0.179 0.013 0.165 0.03 0.133 0.389
155 0.314 0.013 0.245 0.024 0.193 0.563
205 0.572 0.009 0.309 0.021 0.275 0.754
255 0.811 0.01 0.406 0.023 0.392 0.964
305 0.886 0.011 0.45 0.027 0.427 1.235
355 1.246 0.011 0.528 0.049 0.498 1.526
405 1.565 0.011 0.604 0.035 0.573 1.8
455 1.969 0.013 0.671 0.057 0.685 2.135
505 2.436 0.015 0.75 0.046 0.756 2.469
555 3.01 0.015 0.822 0.063 0.815 2.856
605 3.495 0.014 0.894 0.055 0.897 3.222
655 4.19 0.015 0.986 0.084 1.029 3.403
705 4.801 0.017 1.048 0.061 1.035 3.848
755 5.53 0.018 1.118 0.08 1.103 4.221
805 6.347 0.018 1.209 0.085 1.272 4.547
Final Value=6.881736000015767E11
yliu@yliu-dev:~/workspace_java/SymJava$ ./runall.sh
N=5: Grad=0.00555 Hess=0.003054
Final Value=4.008e+08
N=55: Grad=0.015792 Hess=0.012052
Final Value=5.4108e+09
N=105: Grad=0.031341 Hess=0.025145
Final Value=1.04208e+10
N=155: Grad=0.048394 Hess=0.038475
Final Value=1.54308e+10
N=205: Grad=0.062533 Hess=0.054497
Final Value=2.04408e+10
N=255: Grad=0.078452 Hess=0.136445
Final Value=2.54508e+10
N=305: Grad=0.096971 Hess=0.083958
Final Value=3.04608e+10
N=355: Grad=0.110973 Hess=0.094553
Final Value=3.54708e+10
N=405: Grad=0.127781 Hess=0.138077
Final Value=4.04808e+10
N=455: Grad=0.142258 Hess=0.125689
Final Value=4.54908e+10
N=505: Grad=0.159458 Hess=0.199716
Final Value=5.05008e+10
N=555: Grad=0.176937 Hess=0.186688
Final Value=5.55108e+10
N=605: Grad=0.193468 Hess=0.238989
Final Value=6.05208e+10
N=655: Grad=0.205962 Hess=0.229768
Final Value=6.55308e+10
N=705: Grad=0.226367 Hess=0.305967
Final Value=7.05408e+10
N=755: Grad=0.23855 Hess=0.280694
Final Value=7.55508e+10
N=805: Grad=0.262885 Hess=0.319046
Final Value=8.05608e+10
g++ default
============Benchmark for Rosenbrock==============
N|Symbolic Manipulaton|Compile Gradient|Eval Gradient|Compile Hessian|Eval Hessian|C Code Compile
5 0.058 0.025 0.012 0.002 0.013 0.133
55 0.447 0.012 0.098 0.017 0.071 0.176
105 0.311 0.015 0.166 0.018 0.127 0.223
155 0.442 0.012 0.23 0.023 0.2 0.289
205 0.463 0.013 0.307 0.043 0.278 0.366
255 0.942 0.01 0.389 0.023 0.397 0.381
305 0.915 0.011 0.453 0.026 0.43 0.444
355 1.252 0.011 0.551 0.032 0.52 0.484
405 1.545 0.011 0.604 0.04 0.565 0.55
455 1.97 0.027 0.691 0.047 0.666 0.603
505 2.481 0.032 0.799 0.046 0.735 0.693
555 3.006 0.015 0.827 0.048 0.828 0.734
605 3.455 0.015 0.905 0.054 0.881 0.804
655 4.159 0.017 0.978 0.077 1.027 0.854
705 4.824 0.017 1.096 0.061 1.049 0.918
755 5.582 0.017 1.128 0.07 1.166 0.974
805 6.167 0.018 1.197 0.069 1.227 1.03
Final Value=6.881736000015767E11
yliu@yliu-dev:~/workspace_java/SymJava$ ./runall.sh
N=5: Grad=0.012285 Hess=0.005769
Final Value=4.008e+08
N=55: Grad=0.112945 Hess=0.074709
Final Value=5.4108e+09
N=105: Grad=0.205119 Hess=0.141377
Final Value=1.04208e+10
N=155: Grad=0.31637 Hess=0.210464
Final Value=1.54308e+10
N=205: Grad=0.417701 Hess=0.285483
Final Value=2.04408e+10
N=255: Grad=0.51677 Hess=0.51337
Final Value=2.54508e+10
N=305: Grad=0.617711 Hess=0.427058
Final Value=3.04608e+10
N=355: Grad=0.72675 Hess=0.501997
Final Value=3.54708e+10
N=405: Grad=0.820054 Hess=0.56362
Final Value=4.04808e+10
N=455: Grad=0.927727 Hess=0.630426
Final Value=4.54908e+10
N=505: Grad=1.02883 Hess=0.734305
Final Value=5.05008e+10
N=555: Grad=1.16798 Hess=0.786767
Final Value=5.55108e+10
N=605: Grad=1.26002 Hess=0.908035
Final Value=6.05208e+10
N=655: Grad=1.33347 Hess=0.942273
Final Value=6.55308e+10
N=705: Grad=1.4371 Hess=1.02455
Final Value=7.05408e+10
N=755: Grad=1.54837 Hess=1.08883
Final Value=7.55508e+10
N=805: Grad=1.66913 Hess=1.22527
Final Value=8.05608e+10