Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.01 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 1.116, -1.616, 0.58, 1.216, -0.968, -1.92, -1.724, 1.884, ... ], [ -0.116, -1.812, 1.504, 1.376, -0.912, 0.352, 0.964, -1.536, ... ] ],
[ [ -0.972, -1.868, 0.352, -0.048, -0.904, 1.904, -1.94, 1.052, ... ], [ 0.816, 0.376, -0.64, -1.172, 1.912, -0.14, 0.02, 1.044, ... ] ],
[ [ 0.788, 1.928, 0.312, -1.176, -0.144, -1.416, -1.352, -1.944, ... ], [ 1.204, 1.044, 0.356, 1.168, 0.332, 1.356, 1.072, -0.772, ... ] ]
]
[
[ [ 0.92, 0.188, 1.612, -0.396, 1.144, 1.312, -1.92, -0.04, ... ], [ 0.4, -0.416, 1.2, 0.968, 1.288, -1.516, -1.732, -1.36, ... ] ],
[ [ 1.688, 0.568, 1.676, -1.324, 1.524, 1.424, 0.504, -0.636, ... ], [ 1.992, 1.256, -1.804, -0.212, -0.72, -1.88, -1.272, 0.968, ... ] ],
[ [ -0.06, -1.756, 0.628, -0.312, -0.408, 0.568, 1.548, 0.776, ... ], [ -0.928, 1.76, 0.364, 1.004, 1.448, -0.892, -1.812, -1.844, ... ] ]
]
[
[ [ 1.728, -0.448, 1.028, 0.876, -1.72, 0.248, -1.524, -0.54, ... ], [ -0.924, -1.992, -1.468, 1.384, 0.756, -1.42, -1.056, -0.708, ... ] ],
[ [ 0.76, 1.36, -0.216, 0.388, -0.448, 1.24, 0.32, -0.968, ... ], [ -1.212, -0.104, -0.676, 0.18, 0.54, -1.68, -0.112, -1.644, ... ] ],
[ [ 1.756, -1.476, -1.496, 0.624, -0.628, 1.2, 1.556, -1.456, ... ], [ -1.704, 1.652, -1.208, -1.496, -1.764, -1.816, 1.752, 1.568, ... ] ]
]
[
[ [ -1.86, -0.208, 0.428, -0.452, -1.064, -0.668, -0.608, -1.5, ... ], [ -0.904, -0.756, 0.376, -1.696, 0.86, -1.256, -0.232, -1.916, ... ] ],
[ [ -0.176, 0.736, 1.996, -0.648, -1.156, 1.012, 0.932, 0.484, ... ], [ -1.088, 1.496, 1.888, 1.068, -1.048, 1.528, -1.616, 0.724, ... ] ],
[ [ 1.368, 1.072, -1.328, 1.608, 1.064, 0.472, -0.996, -1.432, ... ], [ 0.356, -1.212, 1.7, -0.672, -0.856, -0.664, -0.16, -0.228, ... ] ]
]
[
[ [ -0.732, -1.664, 1.432, -0.928, -0.116, 0.52, -0.528, 0.088, ... ], [ 1.052, -1.324, -0.52, -0.852, 0.152, 1.464, 1.948, -0.384, ... ] ],
[ [ 1.984, 0.584, 1.492, -0.12, 1.592, -0.812, 1.936, -1.416, ... ], [ -1.62, -0.096, -0.28, 1.384, 1.628, 1.54, 1.308, -1.16, ... ] ],
[ [ -1.784, 1.912, 1.364, 1.448, 1.276, 1.516, -1.008, -1.476, ... ], [ -1.564, 1.896, 1.808, -1.524, 0.024, 0.792, 1.612, -1.948, ... ] ]
]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 18.42 seconds (0.221 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1787167144557
Reset training subject: 1787226689355
Constructing line search parameters: GD
th(0)=0.510998075;dx=-7.984344921875001E-4
New Minimum: 0.510998075 > 0.5092793476903692
WOLFE (weak): th(2.154434690031884)=0.5092793476903692; dx=-7.970906054939949E-4 evalInputDelta=0.001718727309630852
New Minimum: 0.5092793476903692 > 0.5075658671362036
WOLFE (weak): th(4.308869380063768)=0.5075658671362036; dx=-7.93963834553118E-4 evalInputDelta=0.0034322078637964992
New Minimum: 0.5075658671362036 > 0.5007691980046193
WOLFE (weak): th(12.926608140191302)=0.5007691980046193; dx=-7.828504468500334E-4 evalInputDelta=0.010228876995380776
New Minimum: 0.5007691980046193 > 0.4713818387652623
WOLFE (weak): th(51.70643256076521)=0.4713818387652623; dx=-7.331227761030126E-4 evalInputDelta=0.039616236234737756
New Minimum: 0.4713818387652623 > 0.34120907588361166
END: th(258.53216280382605)=0.34120907588361166; dx=-5.31036129591859E-4 evalInputDelta=0.1697889991163884
Fitness changed from 0.510998075 to 0.34120907588361166
Iteration 1 complete. Error: 0.34120907588361166 Total: 0.3880; Orientation: 0.0120; Line Search: 0.2582
th(0)=0.34120907588361166;dx=-5.331391810681433E-4
New Minimum: 0.34120907588361166 > 0.15744205751180815
END: th(556.9906600335335)=0.15744205751180815; dx=-1.9397825942470842E-4 evalInputDelta=0.1837670183718035
Fitness changed from 0.34120907588361166 to 0.15744205751180815
Iteration 2 complete. Error: 0.15744205751180815 Total: 0.0841; Orientation: 0.0059; Line Search: 0.0580
th(0)=0.15744205751180815;dx=-2.460032148622003E-4
New Minimum: 0.15744205751180815 > 0.05394818787561395
END: th(1200.0)=0.05394818787561395; dx=-5.939353527332121E-6 evalInputDelta=0.1034938696361942
Fitness changed from 0.15744205751180815 to 0.05394818787561395
Iteration 3 complete. Error: 0.05394818787561395 Total: 0.0810; Orientation: 0.0048; Line Search: 0.0543
th(0)=0.05394818787561395;dx=-8.429404355564679E-5
New Minimum: 0.05394818787561395 > 0.022913176022325232
WOLF (strong): th(2585.3216280382608)=0.022913176022325232; dx=3.7269943592027325E-7 evalInputDelta=0.03103501185328872
New Minimum: 0.022913176022325232 > 0.02266940280144324
WOLF (strong): th(1292.6608140191304)=0.02266940280144324; dx=3.6484779202760836E-9 evalInputDelta=0.031278785074170715
END: th(430.8869380063768)=0.030538161372739563; dx=-2.7487382064413455E-5 evalInputDelta=0.02341002650287439
Fitness changed from 0.05394818787561395 to 0.02266940280144324
Iteration 4 complete. Error: 0.02266940280144324 Total: 0.1083; Orientation: 0.0040; Line Search: 0.0860
th(0)=0.02266940280144324;dx=-3.542094187725506E-5
New Minimum: 0.02266940280144324 > 0.008273214977082338
END: th(928.3177667225559)=0.008273214977082338; dx=-4.383044534465595E-6 evalInputDelta=0.014396187824360902
Fitness changed from 0.02266940280144324 to 0.008273214977082338
Iteration 5 complete. Error: 0.008273214977082338 Total: 0.0636; Orientation: 0.0033; Line Search: 0.0422
th(0)=0.008273214977082338;dx=-1.2926898401691155E-5
New Minimum: 0.008273214977082338 > 0.0026008899589749414
WOLF (strong): th(2000.0000000000005)=0.0026008899589749414; dx=4.872083808725576E-11 evalInputDelta=0.005672325018107396
New Minimum: 0.0026008899589749414 > 0.0026001274564374562
END: th(1000.0000000000002)=0.0026001274564374562; dx=-2.8882400872005567E-9 evalInputDelta=0.005673087520644882
Fitness changed from 0.008273214977082338 to 0.0026001274564374562
Iteration 6 complete. Error: 0.0026001274564374562 Total: 0.0884; Orientation: 0.0037; Line Search: 0.0689
th(0)=0.0026001274564374562;dx=-4.062699150683526E-6
New Minimum: 0.0026001274564374562 > 7.469941971823994E-4
WOLF (strong): th(2154.434690031884)=7.469941971823994E-4; dx=4.032017041192072E-10 evalInputDelta=0.001853133259255057
New Minimum: 7.469941971823994E-4 > 7.468246656018019E-4
END: th(1077.217345015942)=7.468246656018019E-4; dx=-1.0009265019036681E-10 evalInputDelta=0.0018533027908356544
Fitness changed from 0.0026001274564374562 to 7.468246656018019E-4
Iteration 7 complete. Error: 7.468246656018019E-4 Total: 0.0765; Orientation: 0.0027; Line Search: 0.0587
th(0)=7.468246656018019E-4;dx=-1.1669135400028153E-6
New Minimum: 7.468246656018019E-4 > 2.8707606098397957E-4
WOLF (strong): th(2320.79441680639)=2.8707606098397957E-4; dx=1.2723598985821167E-11 evalInputDelta=4.597486046178223E-4
New Minimum: 2.8707606098397957E-4 > 2.870695282651088E-4
END: th(1160.397208403195)=2.870695282651088E-4; dx=-1.481679835112875E-12 evalInputDelta=4.597551373366931E-4
Fitness changed from 7.468246656018019E-4 to 2.870695282651088E-4
Iteration 8 complete. Error: 2.870695282651088E-4 Total: 0.2323; Orientation: 0.0038; Line Search: 0.2118
th(0)=2.870695282651088E-4;dx=-4.4854613791423246E-7
New Minimum: 2.870695282651088E-4 > 9.81213871489561E-5
WOLF (strong): th(2500.000000000001)=9.81213871489561E-5; dx=1.3021695730942067E-13 evalInputDelta=1.8894814111615268E-4
New Minimum: 9.81213871489561E-5 > 9.812130776528514E-5
END: th(1250.0000000000005)=9.812130776528514E-5; dx=-3.24486775875504E-15 evalInputDelta=1.8894822049982362E-4
Fitness changed from 2.870695282651088E-4 to 9.812130776528514E-5
Iteration 9 complete. Error: 9.812130776528514E-5 Total: 0.1024; Orientation: 0.0113; Line Search: 0.0785
th(0)=9.812130776528514E-5;dx=-1.5331454338325803E-7
New Minimum: 9.812130776528514E-5 > 3.584102920031286E-5
WOLF (strong): th(2693.043362539856)=3.584102920031286E-5; dx=8.284871410617463E-17 evalInputDelta=6.228027856497228E-5
New Minimum: 3.584102920031286E-5 > 3.584102914190818E-5
WOLF (strong): th(1346.521681269928)=3.584102914190818E-5; dx=3.900259468517812E-18 evalInputDelta=6.228027862337697E-5
END: th(448.84056042330934)=4.773368539343256E-5; dx=-7.626493585998263E-8 evalInputDelta=5.0387622371852585E-5
Fitness changed from 9.812130776528514E-5 to 3.584102914190818E-5
Iteration 10 complete. Error: 3.584102914190818E-5 Total: 0.3149; Orientation: 0.0029; Line Search: 0.2995
th(0)=3.584102914190818E-5;dx=-5.6001608034231534E-8
New Minimum: 3.584102914190818E-5 > 1.0135825163513082E-5
END: th(966.9976736693295)=1.0135825163513082E-5; dx=-1.927073628964601E-9 evalInputDelta=2.5705203978395093E-5
Fitness changed from 3.584102914190818E-5 to 1.0135825163513082E-5
Iteration 11 complete. Error: 1.0135825163513082E-5 Total: 0.0675; Orientation: 0.0029; Line Search: 0.0517
th(0)=1.0135825163513082E-5;dx=-1.5837226817989194E-8
New Minimum: 1.0135825163513082E-5 > 3.162975324770762E-6
WOLF (strong): th(2083.333333333335)=3.162975324770762E-6; dx=7.702259975232602E-14 evalInputDelta=6.97284983874232E-6
New Minimum: 3.162975324770762E-6 > 3.162947140338198E-6
END: th(1041.6666666666674)=3.162947140338198E-6; dx=-2.3101916106560022E-14 evalInputDelta=6.9728780231748845E-6
Fitness changed from 1.0135825163513082E-5 to 3.162947140338198E-6
Iteration 12 complete. Error: 3.162947140338198E-6 Total: 0.0741; Orientation: 0.0020; Line Search: 0.0619
th(0)=3.162947140338198E-6;dx=-4.942104906778434E-9
New Minimum: 3.162947140338198E-6 > 7.408959890226837E-7
WOLF (strong): th(2244.2028021165474)=7.408959890226837E-7; dx=3.205091331708795E-15 evalInputDelta=2.4220511513155143E-6
New Minimum: 7.408959890226837E-7 > 7.408944857364639E-7
END: th(1122.1014010582737)=7.408944857364639E-7; dx=-5.306291774615E-16 evalInputDelta=2.422052654601734E-6
Fitness changed from 3.162947140338198E-6 to 7.408944857364639E-7
Iteration 13 complete. Error: 7.408944857364639E-7 Total: 0.0750; Orientation: 0.0041; Line Search: 0.0575
th(0)=7.408944857364639E-7;dx=-1.157647633963225E-9
New Minimum: 7.408944857364639E-7 > 3.1760740248443264E-7
WOLF (strong): th(2417.4941841733244)=3.1760740248443264E-7; dx=5.753840500234192E-17 evalInputDelta=4.2328708325203125E-7
New Minimum: 3.1760740248443264E-7 > 3.17607369889447E-7
END: th(1208.7470920866622)=3.17607369889447E-7; dx=-3.643779237845735E-18 evalInputDelta=4.232871158470169E-7
Fitness changed from 7.408944857364639E-7 to 3.17607369889447E-7
Iteration 14 complete. Error: 3.17607369889447E-7 Total: 0.0873; Orientation: 0.0029; Line Search: 0.0722
th(0)=3.17607369889447E-7;dx=-4.962615154522611E-10
New Minimum: 3.17607369889447E-7 > 6.821694272247856E-8
W
...skipping 79510 bytes...
91
New Minimum: 3.1274582712769305E-291 > 6.528285380905473E-293
END: th(1136.5253367759399)=6.528285380905473E-293; dx=-9.100262351841798E-295 evalInputDelta=5.130711164303156E-291
Fitness changed from 5.195994018112211E-291 to 6.528285380905473E-293
Iteration 144 complete. Error: 6.528285380905473E-293 Total: 0.2166; Orientation: 0.0029; Line Search: 0.2060
Zero gradient: 3.193813693324143E-148
th(0)=6.528285380905473E-293;dx=-1.0200445907664803E-295
New Minimum: 6.528285380905473E-293 > 5.44112080921562E-293
WOLF (strong): th(2448.5696116502545)=5.44112080921562E-293; dx=9.312446182015066E-296 evalInputDelta=1.0871645716898525E-293
New Minimum: 5.44112080921562E-293 > 1.2368769285452192E-295
END: th(1224.2848058251273)=1.2368769285452192E-295; dx=-4.439998628248678E-297 evalInputDelta=6.51591661162002E-293
Fitness changed from 6.528285380905473E-293 to 1.2368769285452192E-295
Iteration 145 complete. Error: 1.2368769285452192E-295 Total: 0.0577; Orientation: 0.0025; Line Search: 0.0469
Zero gradient: 1.390187110015017E-149
th(0)=1.2368769285452192E-295;dx=-1.932620200851905E-298
Armijo: th(2637.641656148603)=1.3914796494698443E-295; dx=2.049848195461583E-298 evalInputDelta=-1.5460272092462518E-296
New Minimum: 1.2368769285452192E-295 > 1.1377219560593627E-298
WOLF (strong): th(1318.8208280743015)=1.1377219560593627E-298; dx=5.861399730483896E-300 evalInputDelta=1.2357392065891597E-295
END: th(439.60694269143386)=5.331770670040607E-296; dx=-1.2688754681329902E-298 evalInputDelta=7.036998615411584E-296
Fitness changed from 1.2368769285452192E-295 to 1.1377219560593627E-298
Iteration 146 complete. Error: 1.1377219560593627E-298 Total: 0.0711; Orientation: 0.0035; Line Search: 0.0582
Zero gradient: 4.216266780390864E-151
th(0)=1.1377219560593627E-298;dx=-1.7776905563427544E-301
New Minimum: 1.1377219560593627E-298 > 7.695417497808935E-300
END: th(947.1044473132835)=7.695417497808935E-300; dx=-4.623322502028731E-302 evalInputDelta=1.0607677810812734E-298
Fitness changed from 1.1377219560593627E-298 to 7.695417497808935E-300
Iteration 147 complete. Error: 7.695417497808935E-300 Total: 0.0294; Orientation: 0.0021; Line Search: 0.0209
Zero gradient: 1.0965441094787961E-151
th(0)=7.695417497808935E-300;dx=-1.202408984032646E-302
New Minimum: 7.695417497808935E-300 > 2.7163251871527062E-300
WOLF (strong): th(2040.4746763752125)=2.7163251871527062E-300; dx=7.143762367209956E-303 evalInputDelta=4.979092310656229E-300
New Minimum: 2.7163251871527062E-300 > 3.169317137332242E-301
END: th(1020.2373381876063)=3.169317137332242E-301; dx=-2.440163736558252E-303 evalInputDelta=7.378485784075711E-300
Fitness changed from 7.695417497808935E-300 to 3.169317137332242E-301
Iteration 148 complete. Error: 3.169317137332242E-301 Total: 0.0488; Orientation: 0.0022; Line Search: 0.0400
Zero gradient: 2.2253220052571333E-152
th(0)=3.169317137332242E-301;dx=-4.952058027081629E-304
New Minimum: 3.169317137332242E-301 > 1.6302866284332998E-301
WOLF (strong): th(2198.03471345717)=1.6302866284332998E-301; dx=3.55168841555872E-304 evalInputDelta=1.539030508898942E-301
New Minimum: 1.6302866284332998E-301 > 6.336064846259511E-303
END: th(1099.017356728585)=6.336064846259511E-303; dx=-7.001848057614541E-305 evalInputDelta=3.105956488869647E-301
Fitness changed from 3.169317137332242E-301 to 6.336064846259511E-303
Iteration 149 complete. Error: 6.336064846259511E-303 Total: 0.0491; Orientation: 0.0025; Line Search: 0.0383
Zero gradient: 3.1464426456365744E-153
th(0)=6.336064846259511E-303;dx=-9.900101322280486E-306
New Minimum: 6.336064846259511E-303 > 4.5757968618835484E-303
WOLF (strong): th(2367.7611182832093)=4.5757968618835484E-303; dx=8.413238504250701E-306 evalInputDelta=1.7602679843759628E-303
New Minimum: 4.5757968618835484E-303 > 3.572910567554025E-305
END: th(1183.8805591416046)=3.572910567554025E-305; dx=-7.43431409014892E-307 evalInputDelta=6.300335740583971E-303
Fitness changed from 6.336064846259511E-303 to 3.572910567554025E-305
Iteration 150 complete. Error: 3.572910567554025E-305 Total: 0.0410; Orientation: 0.0019; Line Search: 0.0321
Zero gradient: 2.3627680296218596E-154
th(0)=3.572910567554025E-305;dx=-5.582672761803165E-308
New Minimum: 3.572910567554025E-305 > 3.5205892558811415E-305
WOLF (strong): th(2550.593345469016)=3.5205892558811415E-305; dx=5.541645985217372E-308 evalInputDelta=5.232131167288344E-307
New Minimum: 3.5205892558811415E-305 > 4.82405892328E-310
END: th(1275.296672734508)=4.82405892328E-310; dx=-2.0513388292896E-310 evalInputDelta=3.5728623269647924E-305
Fitness changed from 3.572910567554025E-305 to 4.82405892328E-310
Iteration 151 complete. Error: 4.82405892328E-310 Total: 0.0427; Orientation: 0.0018; Line Search: 0.0345
Zero gradient: 8.681930699810881E-157
th(0)=4.82405892328E-310;dx=-7.53759206763E-313
Armijo: th(2747.5433918214626)=6.3412454966344E-310; dx=8.641987054E-313 evalInputDelta=-1.5171865733544E-310
New Minimum: 4.82405892328E-310 > 2.589024770635E-312
WOLF (strong): th(1373.7716959107313)=2.589024770635E-312; dx=5.521974932E-314 evalInputDelta=4.79816867557365E-310
END: th(457.92389863691045)=1.9898333731442E-310; dx=-4.8409955474E-313 evalInputDelta=2.8342255501358E-310
Fitness changed from 4.82405892328E-310 to 2.589024770635E-312
Iteration 152 complete. Error: 2.589024770635E-312 Total: 0.3465; Orientation: 0.0018; Line Search: 0.3384
Zero gradient: 6.360307542455208E-158
th(0)=2.589024770635E-312;dx=-4.045351203E-315
New Minimum: 2.589024770635E-312 > 1.3606103846E-313
END: th(986.5671326180039)=1.3606103846E-313; dx=-9.2737422E-316 evalInputDelta=2.452963732173E-312
Fitness changed from 2.589024770635E-312 to 1.3606103846E-313
Iteration 153 complete. Error: 1.3606103846E-313 Total: 0.1375; Orientation: 0.0078; Line Search: 0.1211
Zero gradient: 1.4580650545924913E-158
th(0)=1.3606103846E-313;dx=-2.1259537E-316
New Minimum: 1.3606103846E-313 > 5.9365669367E-314
WOLF (strong): th(2125.4944545575136)=5.9365669367E-314; dx=1.4042829E-316 evalInputDelta=7.6695369095E-314
New Minimum: 5.9365669367E-314 > 3.919624807E-315
END: th(1062.7472272787568)=3.919624807E-315; dx=-3.608354E-317 evalInputDelta=1.32141413655E-313
Fitness changed from 1.3606103846E-313 to 3.919624807E-315
Iteration 154 complete. Error: 3.919624807E-315 Total: 0.1456; Orientation: 0.0037; Line Search: 0.1338
Zero gradient: 2.474755148002687E-159
th(0)=3.919624807E-315;dx=-6.124413E-318
New Minimum: 3.919624807E-315 > 2.438596873E-315
WOLF (strong): th(2289.6194931845525)=2.438596873E-315; dx=4.830727E-318 evalInputDelta=1.481027934E-315
New Minimum: 2.438596873E-315 > 4.372354E-317
END: th(1144.8097465922763)=4.372354E-317; dx=-6.46846E-319 evalInputDelta=3.875901267E-315
Fitness changed from 3.919624807E-315 to 4.372354E-317
Iteration 155 complete. Error: 4.372354E-317 Total: 0.0628; Orientation: 0.0035; Line Search: 0.0512
Zero gradient: 2.6137979552162706E-160
th(0)=4.372354E-317;dx=-6.832E-320
New Minimum: 4.372354E-317 > 3.7563905E-317
WOLF (strong): th(2466.4178315450104)=3.7563905E-317; dx=6.3324E-320 evalInputDelta=6.159635E-318
New Minimum: 3.7563905E-317 > 5.843E-320
END: th(1233.2089157725052)=5.843E-320; dx=-2.495E-321 evalInputDelta=4.366511E-317
Fitness changed from 4.372354E-317 to 5.843E-320
Iteration 156 complete. Error: 5.843E-320 Total: 0.0714; Orientation: 0.0034; Line Search: 0.0596
Zero gradient: 9.430366708215771E-162
th(0)=5.843E-320;dx=-8.9E-323
Armijo: th(2656.8680681968926)=6.761E-320; dx=1.0E-322 evalInputDelta=-9.18E-321
New Minimum: 5.843E-320 > 8.4E-323
WOLF (strong): th(1328.4340340984463)=8.4E-323; dx=4.9E-324 evalInputDelta=5.8344E-320
END: th(442.8113446994821)=2.4995E-320; dx=-5.9E-323 evalInputDelta=3.3433E-320
Fitness changed from 5.843E-320 to 8.4E-323
Iteration 157 complete. Error: 8.4E-323 Total: 0.2288; Orientation: 0.0030; Line Search: 0.2165
Zero gradient: 0.0
th(0)=8.4E-323;dx=0.0 (ERROR: Starting derivative negative)
Fitness changed from 8.4E-323 to 8.4E-323
Static Iteration Total: 0.0433; Orientation: 0.0042; Line Search: 0.0298
Iteration 158 failed. Error: 8.4E-323
Previous Error: 0.0 -> 8.4E-323
Optimization terminated 158
Final threshold in iteration 158: 8.4E-323 (> 0.0) after 18.413s (< 30.000s)
Returns
8.4E-323
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 1.67 seconds (0.087 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1805596713148
Reset training subject: 1805607920819
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=0.510998075}, derivative=-7.984344921875001E-4}
New Minimum: 0.510998075 > 0.5109980749999201
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=0.5109980749999201}, derivative=-7.984344921874377E-4}, evalInputDelta = -7.993605777301127E-14
New Minimum: 0.5109980749999201 > 0.510998074999441
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=0.510998074999441}, derivative=-7.984344921870635E-4}, evalInputDelta = -5.589972928987663E-13
New Minimum: 0.510998074999441 > 0.5109980749960876
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=0.5109980749960876}, derivative=-7.984344921844436E-4}, evalInputDelta = -3.912425938779052E-12
New Minimum: 0.5109980749960876 > 0.5109980749726137
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=0.5109980749726137}, derivative=-7.984344921661045E-4}, evalInputDelta = -2.7386315437638586E-11
New Minimum: 0.5109980749726137 > 0.5109980748082958
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=0.5109980748082958}, derivative=-7.984344920377313E-4}, evalInputDelta = -1.917042080634701E-10
New Minimum: 0.5109980748082958 > 0.5109980736580712
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=0.5109980736580712}, derivative=-7.984344911391182E-4}, evalInputDelta = -1.3419289013327784E-9
New Minimum: 0.5109980736580712 > 0.5109980656064981
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=0.5109980656064981}, derivative=-7.984344848488267E-4}, evalInputDelta = -9.393501976262542E-9
New Minimum: 0.5109980656064981 > 0.5109980092454884
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=0.5109980092454884}, derivative=-7.984344408167863E-4}, evalInputDelta = -6.575451161339174E-8
New Minimum: 0.5109980092454884 > 0.5109976147185078
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=0.5109976147185078}, derivative=-7.984341325925033E-4}, evalInputDelta = -4.602814922538556E-7
New Minimum: 0.5109976147185078 > 0.5109948530339075
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=0.5109948530339075}, derivative=-7.984319750225225E-4}, evalInputDelta = -3.2219660925925098E-6
New Minimum: 0.5109948530339075 > 0.5109755214506638
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=0.5109755214506638}, derivative=-7.984168720326572E-4}, evalInputDelta = -2.2553549336223888E-5
New Minimum: 0.5109755214506638 > 0.5108402106068877
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=0.5108402106068877}, derivative=-7.983111511035987E-4}, evalInputDelta = -1.5786439311238354E-4
New Minimum: 0.5108402106068877 > 0.5098935364080229
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=0.5098935364080229}, derivative=-7.975711046001913E-4}, evalInputDelta = -0.0011045385919771222
New Minimum: 0.5098935364080229 > 0.5033110117934043
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=0.5033110117934043}, derivative=-7.880346195565137E-4}, evalInputDelta = -0.0076870632065957345
New Minimum: 0.5033110117934043 > 0.45967848502604786
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=0.45967848502604786}, derivative=-7.197095671658603E-4}, evalInputDelta = -0.051319589973952195
New Minimum: 0.45967848502604786 > 0.24293269118839445
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=0.24293269118839445}, derivative=-3.7990011690809385E-4}, evalInputDelta = -0.2680653838116056
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=0.7782069786601067}, derivative=6.391222139535289E-4}, evalInputDelta = 0.26720890366010663
F(255.63792745846925) = LineSearchPoint{point=PointSample{avg=0.3427487223033932}, derivative=-5.328660116754334E-4}, evalInputDelta = -0.16824935269660685
New Minimum: 0.24293269118839445 > 0.15054407274689538
F(1789.4654922092848) = LineSearchPoint{point=PointSample{avg=0.15054407274689538}, derivative=1.7365490763689923E-4}, evalInputDelta = -0.36045400225310464
0.15054407274689538 <= 0.510998075
New Minimum: 0.15054407274689538 > 0.11192449469078816
F(1469.7937986128354) = LineSearchPoint{point=PointSample{avg=0.11192449469078816}, derivative=6.664575962979263E-5}, evalInputDelta = -0.3990735803092119
Right bracket at 1469.7937986128354
New Minimum: 0.11192449469078816 > 0.1065957660163888
F(1356.5609233946884) = LineSearchPoint{point=PointSample{avg=0.1065957660163888}, derivative=2.7373226965585646E-5}, evalInputDelta = -0.40440230898361124
Right bracket at 1356.5609233946884
New Minimum: 0.1065957660163888 > 0.1057230900262954
F(1311.5947052454928) = LineSearchPoint{point=PointSample{avg=0.1057230900262954}, derivative=1.137113164371221E-5}, evalInputDelta = -0.4052749849737046
Right bracket at 1311.5947052454928
New Minimum: 0.1057230900262954 > 0.10557445321114863
F(1293.1775497977665) = LineSearchPoint{point=PointSample{avg=0.10557445321114863}, derivative=4.7585798012217324E-6}, evalInputDelta = -0.40542362178885144
Right bracket at 1293.1775497977665
Converged to right
Fitness changed from 0.510998075 to 0.10557445321114863
Iteration 1 complete. Error: 0.10557445321114863 Total: 0.7362; Orientation: 0.0039; Line Search: 0.7009
F(0.0) = LineSearchPoint{point=PointSample{avg=0.10557445321114863}, derivative=-1.6496008314241975E-4}
New Minimum: 0.10557445321114863 > 0.03354693698042106
F(1293.1775497977665) = LineSearchPoint{point=PointSample{avg=0.03354693698042106}, derivative=4.6093249013197495E-7}, evalInputDelta = -0.07202751623072756
0.03354693698042106 <= 0.10557445321114863
Converged to right
Fitness changed from 0.10557445321114863 to 0.03354693698042106
Iteration 2 complete. Error: 0.03354693698042106 Total: 0.0395; Orientation: 0.0024; Line Search: 0.0292
F(0.0) = LineSearchPoint{point=PointSample{avg=0.03354693698042106}, derivative=-5.2417089031907914E-5}
New Minimum: 0.03354693698042106 > 0.008636471816217996
F(1293.1775497977665) = LineSearchPoint{point=PointSample{avg=0.008636471816217996}, derivative=1.512900703536214E-7}, evalInputDelta = -0.024910465164203068
0.008636471816217996 <= 0.03354693698042106
Converged to right
Fitness changed from 0.03354693698042106 to 0.008636471816217996
Iteration 3 complete. Error: 0.008636471816217996 Total: 0.0380; Orientation: 0.0027; Line Search: 0.0260
F(0.0) = LineSearchPoint{point=PointSample{avg=0.008636471816217996}, derivative=-1.349448721284062E-5}
New Minimum: 0.008636471816217996 > 0.0023621445756428668
F(1293.1775497977665) = LineSearchPoint{point=PointSample{avg=0.0023621445756428668}, derivative=4.502984584014362E-8}, evalInputDelta = -0.006274327240575129
0.0023621445756428668 <= 0.008636471816217996
Converged to right
Fitness changed from 0.008636471816217996 to 0.0023621445756428668
Iteration 4 complete. Error: 0.0023621445756428668 Total: 0.0388; Orientation: 0.0021; Line Search: 0.0288
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0023621445756428668}, derivative=-3.690850899441979E-6}
New Minimum: 0.0023621445756428668 > 1.552362586545335E-4
F(1293.1775497977665) = LineSearchPoint{point=PointSample{avg=1.552362586545335E-4}, derivative=2.5423338496795746E-8}, evalInputDelta = -0.002206908316988333
1.552362586545335E-4 <= 0.0023621445756428668
Converged to right
Fitness changed from 0.0023621445756428668 to 1.552362586545335E-4
Iteration 5 complete. Error: 1.552362586545335E-4 Total: 0.1487; Orientation: 0.0020; Line Search: 0.1391
F(0.0) = LineSearchPoint{point=PointSample{avg=1.552362586545335E-4}, derivative=-2.4255665414770863E-7}
New Minimum: 1.552362586545335E-4 > 1.6452903866124323E-8
F(1293.1775497977665) = LineSearchPoint{point=PointSample{avg=1.6452903866124323E-8}, derivative=2.4971112412586546E-9}, evalInputDelta = -1.552198057506674E-4
1.6452903866124323E-8 <= 1.552362586545335E-4
New Minimum: 1.6452903866124323E-8 > 4.8151187345601745E-36
F(1279.9999999999998) = LineSearchPoint{point=PointSample{avg=4.8151187345601745E-36}, derivative=-3.5509670389112854E-23}, evalInputDelta = -1.552362586545335E-4
Left bracket at 1279.9999999999998
Converged to left
Fitness changed from 1.552362586545335E-4 to 4.8151187345601745E-36
Iteration 6 complete. Error: 4.8151187345601745E-36 Total: 0.1718; Orientation: 0.0028; Line Search: 0.1614
Zero gradient: 8.673882073645152E-20
F(0.0) = LineSearchPoint{point=PointSample{avg=4.8151187345601745E-36}, derivative=-7.523623022750273E-39}
New Minimum: 4.8151187345601745E-36 > 5.397605346934028E-80
F(1279.9999999999998) = LineSearchPoint{point=PointSample{avg=5.397605346934028E-80}, derivative=-7.59645419660784E-67}, evalInputDelta = -4.8151187345601745E-36
F(8959.999999999998) = LineSearchPoint{point=PointSample{avg=1.7334427444416628E-34}, derivative=4.5141738136501644E-38}, evalInputDelta = 1.685291557096061E-34
F(689.230769230769) = LineSearchPoint{point=PointSample{avg=1.2037062152429553E-36}, derivative=-3.761581922634479E-39}, evalInputDelta = -3.6114125193172196E-36
F(4824.615384615383) = LineSearchPoint{point=PointSample{avg=4.333606861103574E-35}, derivative=2.257086906824924E-38}, evalInputDelta = 3.8520949876475564E-35
F(371.12426035502943) = LineSearchPoint{point=PointSample{avg=1.204000088831936E-36}, derivative=-3.762041100116969E-39}, evalInputDelta = -3.611118645728239E-36
F(2597.869822485206) = LineSearchPoint{point=PointSample{avg=4.815118734560437E-36}, derivative=7.523623022750476E-39}, evalInputDelta = 2.6259913276655822E-49
F(199.8361401911697) = LineSearchPoint{point=PointSample{avg=4.8151187345589137E-36}, derivative=-7.523623022749206E-39}, evalInputDelta = -1.2608767519859933E-48
F(1398.8529813381879) = LineSearchPoint{point=PointSample{avg=3.775555761279158E-50}, derivative=6.353321084199101E-52}, evalInputDelta = -4.8151187345601364E-36
3.775555761279158E-50 <= 4.8151187345601745E-36
Converged to right
Fitness changed from 4.8151187345601745E-36 to 5.397605346934028E-80
Iteration 7 complete. Error: 5.397605346934028E-80 Total: 0.2324; Orientation: 0.0036; Line Search: 0.2197
Zero gradient: 9.183549615799122E-42
F(0.0) = LineSearchPoint{point=PointSample{avg=5.397605346934028E-80}, derivative=-8.43375835458442E-83}
New Minimum: 5.397605346934028E-80 > 4.653731774304817E-82
F(1398.8529813381879) = LineSearchPoint{point=PointSample{avg=4.653731774304817E-82}, derivative=7.83107284631413E-84}, evalInputDelta = -5.35106802919098E-80
4.653731774304817E-82 <= 5.397605346934028E-80
New Minimum: 4.653731774304817E-82 > 0.0
F(1280.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}, evalInputDelta = -5.397605346934028E-80
Right bracket at 1280.0
Converged to right
Fitness changed from 5.397605346934028E-80 to 0.0
Iteration 8 complete. Error: 0.0 Total: 0.0635; Orientation: 0.0033; Line Search: 0.0528
Zero gradient: 0.0
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.1961; Orientation: 0.1760; Line Search: 0.0120
Iteration 9 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 9
Final threshold in iteration 9: 0.0 (> 0.0) after 1.666s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 32.25 seconds (0.089 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1807270288531
Reset training subject: 1807279827561
Adding measurement 20029352 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 0.510998075 < 0.510998075. Total: 1
th(0)=0.510998075;dx=-7.984344921875001E-4
Adding measurement 3059c782 to history. Total: 1
New Minimum: 0.510998075 > 0.5092793476903692
WOLFE (weak): th(2.154434690031884)=0.5092793476903692; dx=-7.970906054939949E-4 evalInputDelta=0.001718727309630852
Adding measurement 24e55c70 to history. Total: 2
New Minimum: 0.5092793476903692 > 0.5075658671362036
WOLFE (weak): th(4.308869380063768)=0.5075658671362036; dx=-7.93963834553118E-4 evalInputDelta=0.0034322078637964992
Adding measurement 73fe29e0 to history. Total: 3
New Minimum: 0.5075658671362036 > 0.5007691980046193
WOLFE (weak): th(12.926608140191302)=0.5007691980046193; dx=-7.828504468500334E-4 evalInputDelta=0.010228876995380776
Adding measurement 2776611f to history. Total: 4
New Minimum: 0.5007691980046193 > 0.4713818387652623
WOLFE (weak): th(51.70643256076521)=0.4713818387652623; dx=-7.331227761030126E-4 evalInputDelta=0.039616236234737756
Adding measurement 4d8081 to history. Total: 5
New Minimum: 0.4713818387652623 > 0.34120907588361166
END: th(258.53216280382605)=0.34120907588361166; dx=-5.31036129591859E-4 evalInputDelta=0.1697889991163884
Fitness changed from 0.510998075 to 0.34120907588361166
Iteration 1 complete. Error: 0.34120907588361166 Total: 0.3527; Orientation: 0.0163; Line Search: 0.3106
Non-optimal measurement 0.34120907588361166 < 0.34120907588361166. Total: 6
Rejected: LBFGS Orientation magnitude: 2.955e+01, gradient 2.309e-02, dot -1.000; [2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.34120907588361166, 0.4713818387652623, 0.5007691980046193, 0.5075658671362036, 0.5092793476903692, 0.510998075
Rejected: LBFGS Orientation magnitude: 2.070e+01, gradient 2.309e-02, dot -0.816; [29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.34120907588361166, 0.4713818387652623, 0.5007691980046193, 0.5075658671362036, 0.5092793476903692
Rejected: LBFGS Orientation magnitude: 2.336e+01, gradient 2.309e-02, dot -0.822; [29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.34120907588361166, 0.4713818387652623, 0.5007691980046193, 0.5075658671362036
LBFGS Accumulation History: 3 points
Removed measurement 4d8081 to history. Total: 5
Removed measurement 2776611f to history. Total: 4
Removed measurement 73fe29e0 to history. Total: 3
Adding measurement db324b7 to history. Total: 3
th(0)=0.34120907588361166;dx=-5.331391810681433E-4
Adding measurement aa56098 to history. Total: 4
New Minimum: 0.34120907588361166 > 0.15744205751180815
END: th(556.9906600335335)=0.15744205751180815; dx=-1.9397825942470842E-4 evalInputDelta=0.1837670183718035
Fitness changed from 0.34120907588361166 to 0.15744205751180815
Iteration 2 complete. Error: 0.15744205751180815 Total: 2.4987; Orientation: 2.4359; Line Search: 0.0545
Non-optimal measurement 0.15744205751180815 < 0.15744205751180815. Total: 5
Rejected: LBFGS Orientation magnitude: 1.932e+01, gradient 1.568e-02, dot -0.951; [2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.15744205751180815, 0.34120907588361166, 0.5075658671362036, 0.5092793476903692, 0.510998075
Rejected: LBFGS Orientation magnitude: 2.685e+01, gradient 1.568e-02, dot -0.711; [2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.15744205751180815, 0.34120907588361166, 0.5075658671362036, 0.5092793476903692
LBFGS Accumulation History: 3 points
Removed measurement aa56098 to history. Total: 4
Removed measurement db324b7 to history. Total: 3
Adding measurement 28decbb7 to history. Total: 3
th(0)=0.15744205751180815;dx=-2.460032148622003E-4
Adding measurement 1fa219e1 to history. Total: 4
New Minimum: 0.15744205751180815 > 0.05394818787561395
END: th(1200.0)=0.05394818787561395; dx=-5.939353527332121E-6 evalInputDelta=0.1034938696361942
Fitness changed from 0.15744205751180815 to 0.05394818787561395
Iteration 3 complete. Error: 0.05394818787561395 Total: 1.5410; Orientation: 1.3732; Line Search: 0.1595
Non-optimal measurement 0.05394818787561395 < 0.05394818787561395. Total: 5
Rejected: LBFGS Orientation magnitude: 1.038e+01, gradient 9.181e-03, dot -0.908; [8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.05394818787561395, 0.15744205751180815, 0.5075658671362036, 0.5092793476903692, 0.510998075
Rejected: LBFGS Orientation magnitude: 5.258e+00, gradient 9.181e-03, dot -0.357; [8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.05394818787561395, 0.15744205751180815, 0.5075658671362036, 0.5092793476903692
LBFGS Accumulation History: 3 points
Removed measurement 1fa219e1 to history. Total: 4
Removed measurement 28decbb7 to history. Total: 3
Adding measurement 4bf787d8 to history. Total: 3
th(0)=0.05394818787561395;dx=-8.429404355564679E-5
Adding measurement 1842ab18 to history. Total: 4
New Minimum: 0.05394818787561395 > 0.022913176022325232
WOLF (strong): th(2585.3216280382608)=0.022913176022325232; dx=3.7269943592027325E-7 evalInputDelta=0.03103501185328872
Adding measurement 5f3c1cca to history. Total: 5
New Minimum: 0.022913176022325232 > 0.02266940280144324
WOLF (strong): th(1292.6608140191304)=0.02266940280144324; dx=3.6484779202760836E-9 evalInputDelta=0.031278785074170715
Non-optimal measurement 0.030538161372739563 < 0.02266940280144324. Total: 6
END: th(430.8869380063768)=0.030538161372739563; dx=-2.7487382064413455E-5 evalInputDelta=0.02341002650287439
Fitness changed from 0.05394818787561395 to 0.02266940280144324
Iteration 4 complete. Error: 0.02266940280144324 Total: 1.4123; Orientation: 1.3480; Line Search: 0.0570
Non-optimal measurement 0.02266940280144324 < 0.02266940280144324. Total: 6
Rejected: LBFGS Orientation magnitude: 8.874e+00, gradient 5.952e-03, dot -0.846; [2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.02266940280144324, 0.022913176022325232, 0.05394818787561395, 0.507565867136
...skipping 20271 bytes...
7.408959890226837E-7
WOLF (strong): th(2244.2028021165474)=7.408959890226837E-7; dx=3.205091331708795E-15 evalInputDelta=2.4220511513155143E-6
Adding measurement 7fc8394d to history. Total: 5
New Minimum: 7.408959890226837E-7 > 7.408944857364639E-7
END: th(1122.1014010582737)=7.408944857364639E-7; dx=-5.306291774615E-16 evalInputDelta=2.422052654601734E-6
Fitness changed from 3.162947140338198E-6 to 7.408944857364639E-7
Iteration 13 complete. Error: 7.408944857364639E-7 Total: 2.2639; Orientation: 2.2157; Line Search: 0.0403
Non-optimal measurement 7.408944857364639E-7 < 7.408944857364639E-7. Total: 6
Rejected: LBFGS Orientation magnitude: 4.852e-02, gradient 3.402e-05, dot -0.900; [29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 7.408944857364639E-7, 7.408959890226837E-7, 3.162947140338198E-6, 0.5075658671362036, 0.5092793476903692, 0.510998075
Rejected: LBFGS Orientation magnitude: 3.919e-03, gradient 3.402e-05, dot -0.108; [50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 7.408944857364639E-7, 7.408959890226837E-7, 3.162947140338198E-6, 0.5075658671362036, 0.5092793476903692
Rejected: LBFGS Orientation magnitude: 4.889e-02, gradient 3.402e-05, dot -0.898; [8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 7.408944857364639E-7, 7.408959890226837E-7, 3.162947140338198E-6, 0.5075658671362036
LBFGS Accumulation History: 3 points
Removed measurement 7fc8394d to history. Total: 5
Removed measurement 1fc793d3 to history. Total: 4
Removed measurement 65a80253 to history. Total: 3
Adding measurement 74ee49 to history. Total: 3
th(0)=7.408944857364639E-7;dx=-1.157647633963225E-9
Adding measurement 6e65069f to history. Total: 4
New Minimum: 7.408944857364639E-7 > 3.1760740248443264E-7
WOLF (strong): th(2417.4941841733244)=3.1760740248443264E-7; dx=5.753840500234192E-17 evalInputDelta=4.2328708325203125E-7
Adding measurement 1e34a443 to history. Total: 5
New Minimum: 3.1760740248443264E-7 > 3.17607369889447E-7
END: th(1208.7470920866622)=3.17607369889447E-7; dx=-3.643779237845735E-18 evalInputDelta=4.232871158470169E-7
Fitness changed from 7.408944857364639E-7 to 3.17607369889447E-7
Iteration 14 complete. Error: 3.17607369889447E-7 Total: 2.3430; Orientation: 2.1411; Line Search: 0.1945
Non-optimal measurement 3.17607369889447E-7 < 3.17607369889447E-7. Total: 6
Rejected: LBFGS Orientation magnitude: 3.402e-02, gradient 2.228e-05, dot -0.838; [2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.17607369889447E-7, 3.1760740248443264E-7, 7.408944857364639E-7, 0.5075658671362036, 0.5092793476903692, 0.510998075
Rejected: LBFGS Orientation magnitude: 3.319e-03, gradient 2.228e-05, dot -0.110; [8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.17607369889447E-7, 3.1760740248443264E-7, 7.408944857364639E-7, 0.5075658671362036, 0.5092793476903692
Rejected: LBFGS Orientation magnitude: 3.434e-02, gradient 2.228e-05, dot -0.836; [8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.17607369889447E-7, 3.1760740248443264E-7, 7.408944857364639E-7, 0.5075658671362036
LBFGS Accumulation History: 3 points
Removed measurement 1e34a443 to history. Total: 5
Removed measurement 6e65069f to history. Total: 4
Removed measurement 74ee49 to history. Total: 3
Adding measurement 18a0797d to history. Total: 3
th(0)=3.17607369889447E-7;dx=-4.962615154522611E-10
Adding measurement 7c7eb1aa to history. Total: 4
New Minimum: 3.17607369889447E-7 > 6.821694272247856E-8
WOLF (strong): th(2604.166666666669)=6.821694272247856E-8; dx=2.0755659243300186E-19 evalInputDelta=2.493904271669684E-7
Adding measurement 53fd87f6 to history. Total: 5
New Minimum: 6.821694272247856E-8 > 6.821694258509703E-8
WOLF (strong): th(1302.0833333333344)=6.821694258509703E-8; dx=3.461453548566018E-21 evalInputDelta=2.4939042730434995E-7
Non-optimal measurement 1.4489016065269867E-7 < 6.821694258509703E-8. Total: 6
END: th(434.0277777777781)=1.4489016065269867E-7; dx=-2.986651869702614E-10 evalInputDelta=1.7271720923674832E-7
Fitness changed from 3.17607369889447E-7 to 6.821694258509703E-8
Iteration 15 complete. Error: 6.821694258509703E-8 Total: 2.7436; Orientation: 2.5850; Line Search: 0.1510
Non-optimal measurement 6.821694258509703E-8 < 6.821694258509703E-8. Total: 6
Rejected: LBFGS Orientation magnitude: 1.462e-02, gradient 1.032e-05, dot -0.909; [50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.821694258509703E-8, 6.821694272247856E-8, 3.17607369889447E-7, 0.5075658671362036, 0.5092793476903692, 0.510998075
Rejected: LBFGS Orientation magnitude: 1.600e-03, gradient 1.032e-05, dot -0.131; [29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.821694258509703E-8, 6.821694272247856E-8, 3.17607369889447E-7, 0.5075658671362036, 0.5092793476903692
Rejected: LBFGS Orientation magnitude: 1.481e-02, gradient 1.032e-05, dot -0.905; [8b230fdc-c4c4-43e0-a31e-3c7ba571f790 = 1.000/1.000e+00, 8cd8fe3c-d3c7-4dd9-ade9-768ea13f0df0 = 1.000/1.000e+00, 2b80b09d-202c-4cd0-9a8d-2b1d90927e76 = 1.000/1.000e+00, 29c44af7-348c-4818-9ad5-3f041fc2b093 = 1.000/1.000e+00, 50cc03c6-7d52-4378-83b0-4f9e0167ea34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.821694258509703E-8, 6.821694272247856E-8, 3.17607369889447E-7, 0.5075658671362036
LBFGS Accumulation History: 3 points
Removed measurement 53fd87f6 to history. Total: 5
Removed measurement 7c7eb1aa to history. Total: 4
Removed measurement 18a0797d to history. Total: 3
Adding measurement 57da93bd to history. Total: 3
th(0)=6.821694258509703E-8;dx=-1.0658897278921411E-10
Adding measurement 72c7e53d to history. Total: 4
New Minimum: 6.821694258509703E-8 > 2.7805762939413402E-8
END: th(935.0845008818948)=2.7805762939413402E-8; dx=-9.00668036850439E-12 evalInputDelta=4.0411179645683625E-8
Fitness changed from 6.821694258509703E-8 to 2.7805762939413402E-8
Iteration 16 complete. Error: 2.7805762939413402E-8 Total: 2.3444; Orientation: 2.2201; Line Search: 0.1160
Final threshold in iteration 16: 2.7805762939413402E-8 (> 0.0) after 32.247s (< 30.000s)
Returns
2.7805762939413402E-8
TrainingTester.java:432 executed in 0.16 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -322.0757664217375], [157.0, -0.4669794254712534]; valueStats=DoubleSummaryStatistics{count=180, sum=1.324924, min=0.000000, average=0.007361, max=0.341209}
Plotting 157 points for GD
Plotting 8 points for CjGD
Plotting 16 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -322.0757664217375], [31.894, -0.4669794254712534]; valueStats=DoubleSummaryStatistics{count=180, sum=1.324924, min=0.000000, average=0.007361, max=0.341209}
Plotting 157 points for GD
Plotting 8 points for CjGD
Plotting 16 points for LBFGS
Returns
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "NonConverged", "value": 2.7805762939413402E-8 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 8.4E-323 } }, "model":null, "complete":null}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "NonConverged", "value": 2.7805762939413402E-8 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 8.4E-323 } }, "model":null, "complete":null} | OK |
{
"result": "OK",
"performance": {
"execution_time": "53.412",
"gc_time": "0.712"
},
"created_on": 1586736418786,
"file_name": "trainingTest",
"report": {
"simpleName": "Repro",
"canonicalName": "com.simiacryptus.mindseye.layers.cudnn.PoolingLayerTest.Repro",
"link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/PoolingLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "NonConverged",
"value": 2.7805762939413402E-8
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "Converged",
"value": 8.4E-323
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/PoolingLayer/Repro/trainingTest/202004130658",
"id": "df4ad99f-df38-48a3-a160-edbb1576d111",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "PoolingLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.cudnn.PoolingLayer",
"link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/PoolingLayer.java",
"javaDoc": ""
}
}