-
Notifications
You must be signed in to change notification settings - Fork 1
/
scratch.txt
734 lines (581 loc) · 28.9 KB
/
scratch.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
512 batch size without a generator
29880/29880 [==============================] - 33s - loss: 0.8629 - val_loss: 0.0336
Epoch 2/20
29880/29880 [==============================] - 26s - loss: 0.0319 - val_loss: 0.0293
Epoch 3/20
29880/29880 [==============================] - 25s - loss: 0.0280 - val_loss: 0.0252
Epoch 4/20
29880/29880 [==============================] - 25s - loss: 0.0239 - val_loss: 0.0214
Epoch 5/20
29880/29880 [==============================] - 25s - loss: 0.0207 - val_loss: 0.0192
Epoch 6/20
29880/29880 [==============================] - 25s - loss: 0.0188 - val_loss: 0.0178
Epoch 7/20
29880/29880 [==============================] - 25s - loss: 0.0174 - val_loss: 0.0165
Epoch 8/20
29880/29880 [==============================] - 25s - loss: 0.0160 - val_loss: 0.0153
Epoch 9/20
29880/29880 [==============================] - 25s - loss: 0.0148 - val_loss: 0.0143
Epoch 10/20
29880/29880 [==============================] - 25s - loss: 0.0138 - val_loss: 0.0135
Epoch 11/20
29880/29880 [==============================] - 25s - loss: 0.0130 - val_loss: 0.0129
Epoch 12/20
29880/29880 [==============================] - 25s - loss: 0.0124 - val_loss: 0.0124
Epoch 13/20
29880/29880 [==============================] - 25s - loss: 0.0119 - val_loss: 0.0120
Epoch 14/20
29880/29880 [==============================] - 25s - loss: 0.0114 - val_loss: 0.0117
Epoch 15/20
29880/29880 [==============================] - 25s - loss: 0.0111 - val_loss: 0.0114
Epoch 16/20
29880/29880 [==============================] - 25s - loss: 0.0107 - val_loss: 0.0114
Epoch 17/20
29880/29880 [==============================] - 25s - loss: 0.0104 - val_loss: 0.0110
Epoch 18/20
29880/29880 [==============================] - 25s - loss: 0.0102 - val_loss: 0.0109
Epoch 19/20
29880/29880 [==============================] - 25s - loss: 0.0099 - val_loss: 0.0108
Epoch 20/20
29880/29880 [==============================] - 25s - loss: 0.0097 - val_loss: 0.0106
256 batch size with data generator
29880/29880 [==============================] - 30s - loss: 0.0444 - val_loss: 0.0180
Epoch 2/20
29880/29880 [==============================] - 25s - loss: 0.0140 - val_loss: 0.0120
Epoch 3/20
29880/29880 [==============================] - 25s - loss: 0.0114 - val_loss: 0.0112
Epoch 4/20
29880/29880 [==============================] - 25s - loss: 0.0104 - val_loss: 0.0105
Epoch 5/20
29880/29880 [==============================] - 25s - loss: 0.0098 - val_loss: 0.0102
Epoch 6/20
29880/29880 [==============================] - 25s - loss: 0.0094 - val_loss: 0.0100
Epoch 7/20
29880/29880 [==============================] - 25s - loss: 0.0092 - val_loss: 0.0097
Epoch 8/20
29880/29880 [==============================] - 25s - loss: 0.0088 - val_loss: 0.0100
Epoch 9/20
29880/29880 [==============================] - 25s - loss: 0.0085 - val_loss: 0.0095
Epoch 10/20
29880/29880 [==============================] - 25s - loss: 0.0083 - val_loss: 0.0097
Epoch 11/20
29880/29880 [==============================] - 25s - loss: 0.0082 - val_loss: 0.0092
Epoch 12/20
29880/29880 [==============================] - 25s - loss: 0.0080 - val_loss: 0.0093
Epoch 13/20
29880/29880 [==============================] - 25s - loss: 0.0079 - val_loss: 0.0091
Epoch 14/20
29880/29880 [==============================] - 25s - loss: 0.0079 - val_loss: 0.0091
Epoch 15/20
29880/29880 [==============================] - 25s - loss: 0.0077 - val_loss: 0.0092
Epoch 16/20
29880/29880 [==============================] - 25s - loss: 0.0076 - val_loss: 0.0091
Epoch 17/20
29880/29880 [==============================] - 25s - loss: 0.0076 - val_loss: 0.0091
Epoch 18/20
29880/29880 [==============================] - 26s - loss: 0.0075 - val_loss: 0.0090
Epoch 19/20
29880/29880 [==============================] - 26s - loss: 0.0074 - val_loss: 0.0089
Epoch 20/20
29880/29880 [==============================] - 26s - loss: 0.0073 - val_loss: 0.0090
Track 2
512 batch size
Nvidia
118/119 [============================>.] - ETA: 0s - loss: 0.0920here
119/119 [==============================] - 65s 543ms/step - loss: 0.0918 - val_loss: 0.0731
Epoch 2/20
119/119 [==============================] - 60s 508ms/step - loss: 0.0646 - val_loss: 0.0644
Epoch 3/20
119/119 [==============================] - 61s 510ms/step - loss: 0.0583 - val_loss: 0.0542
Epoch 4/20
119/119 [==============================] - 61s 509ms/step - loss: 0.0542 - val_loss: 0.0546
Epoch 5/20
119/119 [==============================] - 60s 508ms/step - loss: 0.0532 - val_loss: 0.0531
Epoch 6/20
119/119 [==============================] - 60s 508ms/step - loss: 0.0510 - val_loss: 0.0496
Epoch 7/20
119/119 [==============================] - 60s 508ms/step - loss: 0.0483 - val_loss: 0.0536
Epoch 8/20
119/119 [==============================] - 60s 507ms/step - loss: 0.0493 - val_loss: 0.0505
Epoch 9/20
119/119 [==============================] - 61s 509ms/step - loss: 0.0475 - val_loss: 0.0535
Epoch 10/20
119/119 [==============================] - 60s 508ms/step - loss: 0.0455 - val_loss: 0.0490
Epoch 11/20
119/119 [==============================] - 61s 509ms/step - loss: 0.0436 - val_loss: 0.0496
Epoch 12/20
119/119 [==============================] - 61s 509ms/step - loss: 0.0444 - val_loss: 0.0496
Epoch 13/20
119/119 [==============================] - 61s 511ms/step - loss: 0.0415 - val_loss: 0.0520
Epoch 14/20
119/119 [==============================] - 60s 507ms/step - loss: 0.0400 - val_loss: 0.0522
Epoch 15/20
119/119 [==============================] - 60s 507ms/step - loss: 0.0392 - val_loss: 0.0510
Epoch 16/20
119/119 [==============================] - 60s 507ms/step - loss: 0.0376 - val_loss: 0.0527
Epoch 17/20
119/119 [==============================] - 61s 508ms/step - loss: 0.0358 - val_loss: 0.0523
Epoch 18/20
119/119 [==============================] - 60s 508ms/step - loss: 0.0345 - val_loss: 0.0542
Epoch 19/20
119/119 [==============================] - 61s 509ms/step - loss: 0.0332 - val_loss: 0.0513
Epoch 20/20
119/119 [==============================] - 61s 509ms/step - loss: 0.0319 - val_loss: 0.0573
118/119 [============================>.] - ETA: 0s - loss: 0.0989here
119/119 [==============================] - 69s 576ms/step - loss: 0.0985 - val_loss: 0.0654
Epoch 2/30
119/119 [==============================] - 66s 555ms/step - loss: 0.0646 - val_loss: 0.0549
Epoch 3/30
119/119 [==============================] - 66s 554ms/step - loss: 0.0557 - val_loss: 0.0530
Epoch 4/30
119/119 [==============================] - 66s 553ms/step - loss: 0.0534 - val_loss: 0.0519
Epoch 5/30
119/119 [==============================] - 64s 538ms/step - loss: 0.0501 - val_loss: 0.0494
Epoch 6/30
119/119 [==============================] - 65s 543ms/step - loss: 0.0494 - val_loss: 0.0495
Epoch 7/30
119/119 [==============================] - 66s 558ms/step - loss: 0.0488 - val_loss: 0.0477
Epoch 8/30
119/119 [==============================] - 66s 554ms/step - loss: 0.0478 - val_loss: 0.0468
Epoch 9/30
119/119 [==============================] - 66s 554ms/step - loss: 0.0488 - val_loss: 0.0485
Epoch 10/30
119/119 [==============================] - 66s 554ms/step - loss: 0.0465 - val_loss: 0.0460
Epoch 11/30
119/119 [==============================] - 66s 553ms/step - loss: 0.0464 - val_loss: 0.0479
Epoch 12/30
119/119 [==============================] - 66s 552ms/step - loss: 0.0451 - val_loss: 0.0444
Epoch 13/30
119/119 [==============================] - 66s 552ms/step - loss: 0.0450 - val_loss: 0.0468
Epoch 14/30
119/119 [==============================] - 66s 558ms/step - loss: 0.0448 - val_loss: 0.0480
Epoch 15/30
119/119 [==============================] - 66s 556ms/step - loss: 0.0438 - val_loss: 0.0465
Epoch 16/30
119/119 [==============================] - 66s 552ms/step - loss: 0.0430 - val_loss: 0.0453
Epoch 17/30
119/119 [==============================] - 66s 552ms/step - loss: 0.0422 - val_loss: 0.0462
Epoch 18/30
119/119 [==============================] - 66s 552ms/step - loss: 0.0421 - val_loss: 0.0450
Epoch 19/30
119/119 [==============================] - 66s 553ms/step - loss: 0.0421 - val_loss: 0.0485
Epoch 20/30
119/119 [==============================] - 66s 551ms/step - loss: 0.0414 - val_loss: 0.0450
Epoch 21/30
119/119 [==============================] - 66s 554ms/step - loss: 0.0409 - val_loss: 0.0445
Epoch 22/30
119/119 [==============================] - 66s 554ms/step - loss: 0.0405 - val_loss: 0.0453
Epoch 23/30
119/119 [==============================] - 65s 547ms/step - loss: 0.0398 - val_loss: 0.0446
Epoch 24/30
119/119 [==============================] - 65s 543ms/step - loss: 0.0391 - val_loss: 0.0461
Epoch 25/30
119/119 [==============================] - 65s 542ms/step - loss: 0.0393 - val_loss: 0.0451
Epoch 26/30
119/119 [==============================] - 64s 541ms/step - loss: 0.0383 - val_loss: 0.0460
Epoch 27/30
119/119 [==============================] - 65s 544ms/step - loss: 0.0380 - val_loss: 0.0454
Epoch 28/30
119/119 [==============================] - 65s 543ms/step - loss: 0.0377 - val_loss: 0.0439
Epoch 29/30
119/119 [==============================] - 68s 567ms/step - loss: 0.0370 - val_loss: 0.0468
Epoch 30/30
119/119 [==============================] - 71s 596ms/step - loss: 0.0373 - val_loss: 0.0448
Combined
512 batch size
dropout 0.25
Nvidia
176/177 [============================>.] - ETA: 0s - loss: 0.0719here
177/177 [==============================] - 100s 566ms/step - loss: 0.0717 - val_loss: 0.0591
Epoch 2/35
177/177 [==============================] - 96s 543ms/step - loss: 0.0538 - val_loss: 0.0436
Epoch 3/35
177/177 [==============================] - 96s 541ms/step - loss: 0.0470 - val_loss: 0.0410
Epoch 4/35
177/177 [==============================] - 96s 544ms/step - loss: 0.0446 - val_loss: 0.0406
Epoch 5/35
177/177 [==============================] - 96s 540ms/step - loss: 0.0416 - val_loss: 0.0393
Epoch 6/35
177/177 [==============================] - 96s 543ms/step - loss: 0.0404 - val_loss: 0.0415
Epoch 7/35
177/177 [==============================] - 96s 541ms/step - loss: 0.0397 - val_loss: 0.0367
Epoch 8/35
177/177 [==============================] - 96s 544ms/step - loss: 0.0392 - val_loss: 0.0370
Epoch 9/35
177/177 [==============================] - 96s 544ms/step - loss: 0.0384 - val_loss: 0.0382
Epoch 10/35
177/177 [==============================] - 96s 543ms/step - loss: 0.0379 - val_loss: 0.0399
Epoch 11/35
177/177 [==============================] - 96s 542ms/step - loss: 0.0372 - val_loss: 0.0396
Epoch 12/35
177/177 [==============================] - 96s 541ms/step - loss: 0.0366 - val_loss: 0.0360
Epoch 13/35
177/177 [==============================] - 96s 541ms/step - loss: 0.0369 - val_loss: 0.0362
Epoch 14/35
177/177 [==============================] - 96s 541ms/step - loss: 0.0362 - val_loss: 0.0357
Epoch 15/35
177/177 [==============================] - 96s 540ms/step - loss: 0.0355 - val_loss: 0.0354
Epoch 16/35
177/177 [==============================] - 96s 544ms/step - loss: 0.0355 - val_loss: 0.0375
Epoch 17/35
177/177 [==============================] - 96s 543ms/step - loss: 0.0351 - val_loss: 0.0372
Epoch 18/35
177/177 [==============================] - 96s 543ms/step - loss: 0.0352 - val_loss: 0.0350
Epoch 19/35
177/177 [==============================] - 96s 542ms/step - loss: 0.0347 - val_loss: 0.0352
Epoch 20/35
177/177 [==============================] - 96s 543ms/step - loss: 0.0344 - val_loss: 0.0346
Epoch 21/35
177/177 [==============================] - 96s 542ms/step - loss: 0.0344 - val_loss: 0.0352
Epoch 22/35
177/177 [==============================] - 96s 545ms/step - loss: 0.0344 - val_loss: 0.0363
Epoch 23/35
177/177 [==============================] - 96s 540ms/step - loss: 0.0340 - val_loss: 0.0349
Epoch 24/35
177/177 [==============================] - 96s 544ms/step - loss: 0.0338 - val_loss: 0.0360
Epoch 25/35
177/177 [==============================] - 96s 545ms/step - loss: 0.0333 - val_loss: 0.0336
Epoch 26/35
177/177 [==============================] - 96s 542ms/step - loss: 0.0335 - val_loss: 0.0344
Epoch 27/35
177/177 [==============================] - 96s 542ms/step - loss: 0.0329 - val_loss: 0.0354
Epoch 28/35
177/177 [==============================] - 96s 542ms/step - loss: 0.0332 - val_loss: 0.0349
Epoch 29/35
177/177 [==============================] - 96s 543ms/step - loss: 0.0329 - val_loss: 0.0365
Epoch 30/35
177/177 [==============================] - 96s 541ms/step - loss: 0.0328 - val_loss: 0.0359
Epoch 31/35
177/177 [==============================] - 96s 540ms/step - loss: 0.0325 - val_loss: 0.0342
Epoch 32/35
177/177 [==============================] - 96s 541ms/step - loss: 0.0325 - val_loss: 0.0350
Epoch 33/35
177/177 [==============================] - 96s 544ms/step - loss: 0.0324 - val_loss: 0.0334
Epoch 34/35
177/177 [==============================] - 96s 541ms/step - loss: 0.0320 - val_loss: 0.0350
Epoch 35/35
177/177 [==============================] - 96s 543ms/step - loss: 0.0319 - val_loss: 0.0337
Track 2
Nvidia
Batch Normalization
dropout only for the covnets
Epoch 1/10
118/119 [============================>.] - ETA: 0s - loss: 0.1544here
119/119 [==============================] - 68s 569ms/step - loss: 0.1538 - val_loss: 0.0721
Epoch 2/10
119/119 [==============================] - 63s 533ms/step - loss: 0.0864 - val_loss: 0.0716
Epoch 3/10
119/119 [==============================] - 64s 539ms/step - loss: 0.0757 - val_loss: 0.0582
Epoch 4/10
119/119 [==============================] - 64s 536ms/step - loss: 0.0695 - val_loss: 0.0560
Epoch 5/10
119/119 [==============================] - 64s 536ms/step - loss: 0.0659 - val_loss: 0.0512
Epoch 6/10
119/119 [==============================] - 64s 535ms/step - loss: 0.0645 - val_loss: 0.0592
Epoch 7/10
119/119 [==============================] - 64s 534ms/step - loss: 0.0631 - val_loss: 0.0495
Epoch 8/10
119/119 [==============================] - 64s 535ms/step - loss: 0.0623 - val_loss: 0.0512
Epoch 9/10
119/119 [==============================] - 64s 536ms/step - loss: 0.0609 - val_loss: 0.0451
Epoch 10/10
119/119 [==============================] - 64s 535ms/step - loss: 0.0601 - val_loss: 0.0528
==================================================================================
------------------------------track1-------------------------------------
10 epochs
512 batch size
Normal Nvidia
58/59 [============================>.] - ETA: 0s - loss: 0.0187here
59/59 [==============================] - 42s 705ms/step - loss: 0.0185 - val_loss: 0.0113
Epoch 2/10
59/59 [==============================] - 31s 518ms/step - loss: 0.0103 - val_loss: 0.0090
Epoch 3/10
59/59 [==============================] - 31s 521ms/step - loss: 0.0091 - val_loss: 0.0093
Epoch 4/10
59/59 [==============================] - 31s 518ms/step - loss: 0.0084 - val_loss: 0.0083
Epoch 5/10
59/59 [==============================] - 31s 519ms/step - loss: 0.0077 - val_loss: 0.0079
Epoch 6/10
59/59 [==============================] - 31s 520ms/step - loss: 0.0074 - val_loss: 0.0084
Epoch 7/10
59/59 [==============================] - 31s 523ms/step - loss: 0.0072 - val_loss: 0.0075
Epoch 8/10
59/59 [==============================] - 31s 521ms/step - loss: 0.0067 - val_loss: 0.0069
Epoch 9/10
59/59 [==============================] - 31s 524ms/step - loss: 0.0063 - val_loss: 0.0079
Epoch 10/10
59/59 [==============================] - 31s 519ms/step - loss: 0.0060 - val_loss: 0.0065
____________________________________________________________________________________________
Dropout only convs
dropout 0.1
58/59 [============================>.] - ETA: 0s - loss: 0.0161here
59/59 [==============================] - 36s 602ms/step - loss: 0.0160 - val_loss: 0.0098
Epoch 2/10
59/59 [==============================] - 32s 545ms/step - loss: 0.0099 - val_loss: 0.0084
Epoch 3/10
59/59 [==============================] - 32s 540ms/step - loss: 0.0086 - val_loss: 0.0087
Epoch 4/10
59/59 [==============================] - 32s 545ms/step - loss: 0.0082 - val_loss: 0.0075
Epoch 5/10
59/59 [==============================] - 32s 539ms/step - loss: 0.0079 - val_loss: 0.0074
Epoch 6/10
59/59 [==============================] - 32s 545ms/step - loss: 0.0075 - val_loss: 0.0071
Epoch 7/10
59/59 [==============================] - 32s 541ms/step - loss: 0.0072 - val_loss: 0.0063
Epoch 8/10
59/59 [==============================] - 32s 540ms/step - loss: 0.0069 - val_loss: 0.0068
Epoch 9/10
59/59 [==============================] - 32s 540ms/step - loss: 0.0065 - val_loss: 0.0063
Epoch 10/10
59/59 [==============================] - 32s 538ms/step - loss: 0.0063 - val_loss: 0.0064
_____________________________________________________________________________________________
Dropout all
58/59 [============================>.] - ETA: 0s - loss: 0.0189here
59/59 [==============================] - 36s 603ms/step - loss: 0.0188 - val_loss: 0.0112
Epoch 2/10
59/59 [==============================] - 32s 538ms/step - loss: 0.0115 - val_loss: 0.0090
Epoch 3/10
59/59 [==============================] - 32s 546ms/step - loss: 0.0104 - val_loss: 0.0082
Epoch 4/10
59/59 [==============================] - 32s 544ms/step - loss: 0.0097 - val_loss: 0.0082
Epoch 5/10
59/59 [==============================] - 32s 541ms/step - loss: 0.0092 - val_loss: 0.0082
Epoch 6/10
59/59 [==============================] - 32s 539ms/step - loss: 0.0089 - val_loss: 0.0070
Epoch 7/10
59/59 [==============================] - 32s 538ms/step - loss: 0.0083 - val_loss: 0.0080
Epoch 8/10
59/59 [==============================] - 32s 540ms/step - loss: 0.0080 - val_loss: 0.0067
Epoch 9/10
59/59 [==============================] - 32s 540ms/step - loss: 0.0079 - val_loss: 0.0077
Epoch 10/10
59/59 [==============================] - 32s 542ms/step - loss: 0.0076 - val_loss: 0.0067
_____________________________________________________________________________________________
Batch Normalization
Epoch 1/10
58/59 [============================>.] - ETA: 0s - loss: 0.2035here
59/59 [==============================] - 37s 634ms/step - loss: 0.2009 - val_loss: 0.0677
Epoch 2/10
59/59 [==============================] - 33s 562ms/step - loss: 0.0407 - val_loss: 0.0215
Epoch 3/10
59/59 [==============================] - 33s 564ms/step - loss: 0.0291 - val_loss: 0.0170
Epoch 4/10
59/59 [==============================] - 33s 561ms/step - loss: 0.0248 - val_loss: 0.0149
Epoch 5/10
59/59 [==============================] - 33s 560ms/step - loss: 0.0215 - val_loss: 0.0138
Epoch 6/10
59/59 [==============================] - 33s 560ms/step - loss: 0.0193 - val_loss: 0.0137
Epoch 7/10
59/59 [==============================] - 33s 561ms/step - loss: 0.0176 - val_loss: 0.0132
Epoch 8/10
59/59 [==============================] - 33s 568ms/step - loss: 0.0163 - val_loss: 0.0121
Epoch 9/10
59/59 [==============================] - 33s 565ms/step - loss: 0.0151 - val_loss: 0.0125
Epoch 10/10
59/59 [==============================] - 33s 562ms/step - loss: 0.0146 - val_loss: 0.0119
______________________________________________________________________________________________
==================================================================================
------------------------------track2-------------------------------------
10 epochs
512 batch size
Normal Nvidia
____________________________________________________________________________________________
Dropout only convs
dropout 0.1
118/119 [============================>.] - ETA: 0s - loss: 0.0977here
119/119 [==============================] - 67s 566ms/step - loss: 0.0984 - val_loss: 0.0746
Epoch 2/20
119/119 [==============================] - 63s 532ms/step - loss: 0.0626 - val_loss: 0.0610
Epoch 3/20
119/119 [==============================] - 63s 533ms/step - loss: 0.0546 - val_loss: 0.0548
Epoch 4/20
119/119 [==============================] - 63s 530ms/step - loss: 0.0508 - val_loss: 0.0509
Epoch 5/20
119/119 [==============================] - 63s 527ms/step - loss: 0.0498 - val_loss: 0.0550
Epoch 6/20
119/119 [==============================] - 63s 529ms/step - loss: 0.0493 - val_loss: 0.0522
Epoch 7/20
119/119 [==============================] - 63s 532ms/step - loss: 0.0469 - val_loss: 0.0507
Epoch 8/20
119/119 [==============================] - 63s 531ms/step - loss: 0.0463 - val_loss: 0.0494
Epoch 9/20
119/119 [==============================] - 63s 531ms/step - loss: 0.0459 - val_loss: 0.0522
Epoch 10/20
119/119 [==============================] - 63s 530ms/step - loss: 0.0453 - val_loss: 0.0484
Epoch 11/20
119/119 [==============================] - 63s 531ms/step - loss: 0.0442 - val_loss: 0.0511
Epoch 12/20
119/119 [==============================] - 63s 530ms/step - loss: 0.0438 - val_loss: 0.0495
Epoch 13/20
119/119 [==============================] - 63s 527ms/step - loss: 0.0432 - val_loss: 0.0523
Epoch 14/20
119/119 [==============================] - 63s 530ms/step - loss: 0.0416 - val_loss: 0.0497
Epoch 15/20
119/119 [==============================] - 63s 528ms/step - loss: 0.0409 - val_loss: 0.0464
Epoch 16/20
119/119 [==============================] - 63s 530ms/step - loss: 0.0399 - val_loss: 0.0507
Epoch 17/20
119/119 [==============================] - 64s 535ms/step - loss: 0.0397 - val_loss: 0.0496
Epoch 18/20
119/119 [==============================] - 63s 530ms/step - loss: 0.0388 - val_loss: 0.0483
Epoch 19/20
119/119 [==============================] - 63s 531ms/step - loss: 0.0382 - val_loss: 0.0504
Epoch 20/20
119/119 [==============================] - 63s 529ms/step - loss: 0.0377 - val_loss: 0.0489
_____________________________________________________________________________________________
Dropout all
_____________________________________________________________________________________________
Batch Normalization
153/154 [============================>.] - ETA: 0s - loss: 0.1306here
154/154 [==============================] - 92s 600ms/step - loss: 0.1301 - val_loss: 0.0688
Epoch 2/20
154/154 [==============================] - 87s 567ms/step - loss: 0.0614 - val_loss: 0.0597
Epoch 3/20
154/154 [==============================] - 87s 567ms/step - loss: 0.0542 - val_loss: 0.0652
Epoch 4/20
154/154 [==============================] - 87s 565ms/step - loss: 0.0516 - val_loss: 0.0561
Epoch 5/20
154/154 [==============================] - 87s 567ms/step - loss: 0.0498 - val_loss: 0.0508
Epoch 6/20
154/154 [==============================] - 87s 566ms/step - loss: 0.0466 - val_loss: 0.0503
Epoch 7/20
154/154 [==============================] - 87s 565ms/step - loss: 0.0453 - val_loss: 0.0497
Epoch 8/20
154/154 [==============================] - 88s 570ms/step - loss: 0.0441 - val_loss: 0.0482
Epoch 9/20
154/154 [==============================] - 87s 567ms/step - loss: 0.0425 - val_loss: 0.0518
Epoch 10/20
154/154 [==============================] - 87s 566ms/step - loss: 0.0413 - val_loss: 0.0474
Epoch 11/20
154/154 [==============================] - 87s 568ms/step - loss: 0.0403 - val_loss: 0.0495
Epoch 12/20
154/154 [==============================] - 87s 566ms/step - loss: 0.0388 - val_loss: 0.0512
Epoch 13/20
154/154 [==============================] - 87s 567ms/step - loss: 0.0380 - val_loss: 0.0478
Epoch 14/20
154/154 [==============================] - 87s 565ms/step - loss: 0.0373 - val_loss: 0.0497
Epoch 15/20
154/154 [==============================] - 87s 568ms/step - loss: 0.0362 - val_loss: 0.0513
Epoch 16/20
154/154 [==============================] - 88s 569ms/step - loss: 0.0346 - val_loss: 0.0491
Epoch 17/20
154/154 [==============================] - 87s 565ms/step - loss: 0.0340 - val_loss: 0.0496
Epoch 18/20
154/154 [==============================] - 87s 567ms/step - loss: 0.0320 - val_loss: 0.0500
Epoch 19/20
154/154 [==============================] - 87s 568ms/step - loss: 0.0319 - val_loss: 0.0481
Epoch 20/20
154/154 [==============================] - 87s 567ms/step - loss: 0.0305 - val_loss: 0.0521
______________________________________________________________________________________________
id: 0000:29:00.0, compute capability: 5.2)
229/230 [============================>.] - ETA: 0s - loss: 0.0672here
230/230 [==============================] - 125s 542ms/step - loss: 0.0671 - val_loss: 0.0480
Epoch 2/20
230/230 [==============================] - 120s 523ms/step - loss: 0.0488 - val_loss: 0.0432
Epoch 3/20
230/230 [==============================] - 120s 523ms/step - loss: 0.0416 - val_loss: 0.0393
Epoch 4/20
230/230 [==============================] - 121s 524ms/step - loss: 0.0402 - val_loss: 0.0385
Epoch 5/20
230/230 [==============================] - 120s 522ms/step - loss: 0.0381 - val_loss: 0.0375
Epoch 6/20
230/230 [==============================] - 120s 522ms/step - loss: 0.0375 - val_loss: 0.0397
Epoch 7/20
230/230 [==============================] - 120s 523ms/step - loss: 0.0390 - val_loss: 0.0372
Epoch 8/20
230/230 [==============================] - 120s 523ms/step - loss: 0.0355 - val_loss: 0.0361
Epoch 9/20
230/230 [==============================] - 120s 522ms/step - loss: 0.0358 - val_loss: 0.0369
Epoch 10/20
230/230 [==============================] - 120s 522ms/step - loss: 0.0349 - val_loss: 0.0345
Epoch 11/20
230/230 [==============================] - 121s 524ms/step - loss: 0.0346 - val_loss: 0.0374
Epoch 12/20
230/230 [==============================] - 121s 524ms/step - loss: 0.0335 - val_loss: 0.0352
Epoch 13/20
230/230 [==============================] - 120s 521ms/step - loss: 0.0342 - val_loss: 0.0342
Epoch 14/20
230/230 [==============================] - 120s 524ms/step - loss: 0.0339 - val_loss: 0.0366
Epoch 15/20
230/230 [==============================] - 120s 522ms/step - loss: 0.0340 - val_loss: 0.0346
Epoch 16/20
230/230 [==============================] - 131s 569ms/step - loss: 0.0327 - val_loss: 0.0339
Epoch 17/20
230/230 [==============================] - 122s 531ms/step - loss: 0.0328 - val_loss: 0.0347
Epoch 18/20
230/230 [==============================] - 121s 527ms/step - loss: 0.0323 - val_loss: 0.0347
Epoch 19/20
230/230 [==============================] - 124s 539ms/step - loss: 0.0325 - val_loss: 0.0345
Epoch 20/20
230/230 [==============================] - 121s 524ms/step - loss: 0.0320 - val_loss: 0.0328
Gray
normal
Epoch 1/10
75/76 [============================>.] - ETA: 0s - loss: 0.3791here
76/76 [==============================] - 44s 585ms/step - loss: 0.3755 - val_loss: 0.0478
Epoch 2/10
76/76 [==============================] - 41s 540ms/step - loss: 0.0621 - val_loss: 0.0378
Epoch 3/10
76/76 [==============================] - 42s 549ms/step - loss: 0.0377 - val_loss: 0.0317
Epoch 4/10
76/76 [==============================] - 41s 537ms/step - loss: 0.0319 - val_loss: 0.0325
Epoch 5/10
76/76 [==============================] - 44s 573ms/step - loss: 0.0287 - val_loss: 0.0304
Epoch 6/10
76/76 [==============================] - 46s 605ms/step - loss: 0.0260 - val_loss: 0.0319
Epoch 7/10
76/76 [==============================] - 45s 597ms/step - loss: 0.0237 - val_loss: 0.0278
Epoch 8/10
76/76 [==============================] - 46s 601ms/step - loss: 0.0225 - val_loss: 0.0248
Epoch 9/10
76/76 [==============================] - 46s 604ms/step - loss: 0.0210 - val_loss: 0.0227
Epoch 10/10
76/76 [==============================] - 46s 603ms/step - loss: 0.0201 - val_loss: 0.0233
gray
Epoch 1/10
75/76 [============================>.] - ETA: 0s - loss: 0.2538here
76/76 [==============================] - 49s 648ms/step - loss: 0.2515 - val_loss: 0.0609
Epoch 2/10
76/76 [==============================] - 46s 603ms/step - loss: 0.0512 - val_loss: 0.0337
Epoch 3/10
76/76 [==============================] - 46s 605ms/step - loss: 0.0333 - val_loss: 0.0328
Epoch 4/10
76/76 [==============================] - 46s 606ms/step - loss: 0.0264 - val_loss: 0.0312
Epoch 5/10
76/76 [==============================] - 46s 601ms/step - loss: 0.0231 - val_loss: 0.0345
Epoch 6/10
76/76 [==============================] - 46s 609ms/step - loss: 0.0212 - val_loss: 0.0283
Epoch 7/10
76/76 [==============================] - 46s 602ms/step - loss: 0.0197 - val_loss: 0.0285
Epoch 8/10
76/76 [==============================] - 46s 602ms/step - loss: 0.0187 - val_loss: 0.0273
Epoch 9/10
76/76 [==============================] - 46s 601ms/step - loss: 0.0176 - val_loss: 0.0272
Epoch 10/10
76/76 [==============================] - 46s 605ms/step - loss: 0.0168 - val_loss: 0.0241
Track 2
Batch norm after dropout
Batch norm before dropout
Epoch 1/10
154/154 [==============================] - 97s 629ms/step - loss: 0.2902 - val_loss: 0.1529
Epoch 2/10
154/154 [==============================] - 81s 526ms/step - loss: 0.1157 - val_loss: 0.1131
Epoch 3/10
154/154 [==============================] - 81s 526ms/step - loss: 0.0929 - val_loss: 0.1084
Epoch 4/10
154/154 [==============================] - 80s 522ms/step - loss: 0.0812 - val_loss: 0.0791
Epoch 5/10
154/154 [==============================] - 81s 525ms/step - loss: 0.0724 - val_loss: 0.0963
Epoch 6/10
154/154 [==============================] - 81s 525ms/step - loss: 0.0679 - val_loss: 0.0663
Epoch 7/10
154/154 [==============================] - 80s 518ms/step - loss: 0.0654 - val_loss: 0.0656
Epoch 8/10
154/154 [==============================] - 81s 523ms/step - loss: 0.0621 - val_loss: 0.0632
Epoch 9/10
154/154 [==============================] - 82s 531ms/step - loss: 0.0602 - val_loss: 0.0573
Epoch 10/10
154/154 [==============================] - 80s 522ms/step - loss: 0.0590 - val_loss: 0.0554
Dropout
0.50
0.25