Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.02 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 1.108 ], [ 1.912 ], [ 1.552 ], [ 0.496 ], [ -1.688 ], [ -1.572 ] ],
[ [ -0.176 ], [ 1.64 ], [ 0.028 ], [ -0.608 ], [ -1.228 ], [ 0.3 ] ],
[ [ -0.768 ], [ 1.876 ], [ -0.384 ], [ -0.068 ], [ 1.048 ], [ 0.08 ] ],
[ [ 0.7 ], [ 1.764 ], [ 1.612 ], [ -1.616 ], [ 0.392 ], [ -0.852 ] ],
[ [ -0.384 ], [ 1.556 ], [ -0.128 ], [ -0.712 ], [ -0.636 ], [ -0.556 ] ],
[ [ -1.492 ], [ 1.208 ], [ 0.788 ], [ -0.408 ], [ 1.032 ], [ 1.356 ] ],
[ [ -1.516 ], [ -1.476 ], [ 1.556 ], [ 0.092 ], [ 1.512 ], [ -1.72 ] ],
[ [ 0.048 ], [ -1.028 ], [ 1.704 ], [ 0.636 ], [ 1.524 ], [ -0.804 ] ]
]
[
[ [ 1.704 ], [ -1.028 ], [ 1.556 ], [ 1.512 ], [ 0.7 ], [ 0.496 ] ],
[ [ -0.768 ], [ 1.764 ], [ -1.616 ], [ -0.712 ], [ 1.876 ], [ 0.08 ] ],
[ [ 1.356 ], [ -1.688 ], [ 1.64 ], [ -0.068 ], [ -1.476 ], [ 0.788 ] ],
[ [ -0.384 ], [ 1.912 ], [ -0.804 ], [ 0.092 ], [ -0.408 ], [ -1.492 ] ],
[ [ 1.208 ], [ -0.176 ], [ 1.556 ], [ -0.852 ], [ 0.636 ], [ 1.108 ] ],
[ [ 1.048 ], [ 0.392 ], [ 1.524 ], [ -0.128 ], [ 0.048 ], [ -0.636 ] ],
[ [ -1.572 ], [ 0.3 ], [ -0.556 ], [ -1.228 ], [ -1.72 ], [ -1.516 ] ],
[ [ -0.384 ], [ 1.552 ], [ -0.608 ], [ 0.028 ], [ 1.032 ], [ 1.612 ] ]
]
[
[ [ -0.636 ], [ 1.032 ], [ -0.176 ], [ 1.556 ], [ -0.768 ], [ -1.572 ] ],
[ [ 0.3 ], [ -1.476 ], [ 0.08 ], [ 0.392 ], [ 1.048 ], [ -1.688 ] ],
[ [ -1.228 ], [ 1.876 ], [ 1.524 ], [ 0.092 ], [ -0.804 ], [ -0.068 ] ],
[ [ 1.912 ], [ -0.712 ], [ 0.788 ], [ 1.556 ], [ 0.048 ], [ 1.764 ] ],
[ [ -1.516 ], [ 1.612 ], [ -0.852 ], [ -0.556 ], [ -0.408 ], [ 0.028 ] ],
[ [ -0.608 ], [ 1.64 ], [ -1.028 ], [ -0.128 ], [ -0.384 ], [ 0.496 ] ],
[ [ 1.512 ], [ -1.72 ], [ 1.356 ], [ 1.108 ], [ 0.636 ], [ 0.7 ] ],
[ [ 1.208 ], [ -1.616 ], [ -1.492 ], [ -0.384 ], [ 1.552 ], [ 1.704 ] ]
]
[
[ [ -1.028 ], [ 1.612 ], [ -0.408 ], [ -1.72 ], [ 1.704 ], [ 1.876 ] ],
[ [ 1.048 ], [ 1.912 ], [ 0.08 ], [ 1.556 ], [ -0.384 ], [ 1.032 ] ],
[ [ -1.616 ], [ -0.768 ], [ -0.556 ], [ -1.572 ], [ -1.688 ], [ 0.092 ] ],
[ [ 0.7 ], [ -0.852 ], [ -0.712 ], [ 1.764 ], [ 0.636 ], [ -0.804 ] ],
[ [ -0.128 ], [ 1.512 ], [ 1.552 ], [ 0.028 ], [ -1.516 ], [ -0.636 ] ],
[ [ -0.068 ], [ 0.3 ], [ 1.108 ], [ 0.392 ], [ 1.208 ], [ -0.384 ] ],
[ [ 0.496 ], [ 0.048 ], [ -1.228 ], [ 1.356 ], [ 1.524 ], [ 1.556 ] ],
[ [ -0.176 ], [ -1.476 ], [ 0.788 ], [ -0.608 ], [ 1.64 ], [ -1.492 ] ]
]
[
[ [ -0.068 ], [ -1.616 ], [ -1.492 ], [ 1.764 ], [ -0.408 ], [ -0.384 ] ],
[ [ 0.7 ], [ -0.636 ], [ -0.768 ], [ 1.524 ], [ -0.852 ], [ 1.048 ] ],
[ [ -0.128 ], [ -1.228 ], [ 1.552 ], [ 1.356 ], [ 1.876 ], [ -0.556 ] ],
[ [ 0.08 ], [ -0.804 ], [ 1.704 ], [ 1.556 ], [ -0.608 ], [ 1.64 ] ],
[ [ -1.688 ], [ 0.3 ], [ -1.516 ], [ 1.612 ], [ 0.092 ], [ 1.912 ] ],
[ [ -0.176 ], [ 1.208 ], [ -0.384 ], [ -1.572 ], [ -0.712 ], [ 0.788 ] ],
[ [ 0.392 ], [ 1.032 ], [ 1.108 ], [ -1.028 ], [ 1.556 ], [ 0.496 ] ],
[ [ -1.72 ], [ 0.636 ], [ 0.048 ], [ -1.476 ], [ 1.512 ], [ 0.028 ] ]
]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 1.79 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1915884840475
Reset training subject: 1915931262703
Constructing line search parameters: GD
th(0)=2.2050794666666667;dx=-0.14700529777777777
New Minimum: 2.2050794666666667 > 1.8997384559932093
WOLFE (weak): th(2.154434690031884)=1.8997384559932093; dx=-0.13644818733941408 evalInputDelta=0.3053410106734573
New Minimum: 1.8997384559932093 > 1.6171420502746607
END: th(4.308869380063768)=1.6171420502746607; dx=-0.12589107690105034 evalInputDelta=0.5879374163920059
Fitness changed from 2.2050794666666667 to 1.6171420502746607
Iteration 1 complete. Error: 1.6171420502746607 Total: 0.1699; Orientation: 0.0044; Line Search: 0.0873
th(0)=1.6171420502746607;dx=-0.1078094700183107
New Minimum: 1.6171420502746607 > 0.7711732272995226
END: th(9.283177667225559)=0.7711732272995226; dx=-0.0744489878719972 evalInputDelta=0.8459688229751381
Fitness changed from 1.6171420502746607 to 0.7711732272995226
Iteration 2 complete. Error: 0.7711732272995226 Total: 0.0709; Orientation: 0.0018; Line Search: 0.0516
th(0)=0.7711732272995226;dx=-0.051411548486634845
New Minimum: 0.7711732272995226 > 0.08568591414439135
END: th(20.000000000000004)=0.08568591414439135; dx=-0.017137182828878276 evalInputDelta=0.6854873131551313
Fitness changed from 0.7711732272995226 to 0.08568591414439135
Iteration 3 complete. Error: 0.08568591414439135 Total: 0.0660; Orientation: 0.0034; Line Search: 0.0463
th(0)=0.08568591414439135;dx=-0.005712394276292756
New Minimum: 0.08568591414439135 > 0.016310209544924275
WOLF (strong): th(43.088693800637685)=0.016310209544924275; dx=0.0024922593183637066 evalInputDelta=0.06937570459946707
New Minimum: 0.016310209544924275 > 0.006807086034601106
END: th(21.544346900318843)=0.006807086034601106; dx=-0.001610067478964525 evalInputDelta=0.07887882810979024
Fitness changed from 0.08568591414439135 to 0.006807086034601106
Iteration 4 complete. Error: 0.006807086034601106 Total: 0.0838; Orientation: 0.0017; Line Search: 0.0679
th(0)=0.006807086034601106;dx=-4.5380573564007376E-4
New Minimum: 0.006807086034601106 > 0.0020382033394773396
WOLF (strong): th(46.4158883361278)=0.0020382033394773396; dx=2.483208094187261E-4 evalInputDelta=0.004768882695123766
New Minimum: 0.0020382033394773396 > 3.489162728791657E-4
END: th(23.2079441680639)=3.489162728791657E-4; dx=-1.0274246311067382E-4 evalInputDelta=0.0064581697617219404
Fitness changed from 0.006807086034601106 to 3.489162728791657E-4
Iteration 5 complete. Error: 3.489162728791657E-4 Total: 0.0759; Orientation: 0.0031; Line Search: 0.0605
th(0)=3.489162728791657E-4;dx=-2.326108485861105E-5
New Minimum: 3.489162728791657E-4 > 1.5507389905740707E-4
WOLF (strong): th(50.000000000000014)=1.5507389905740707E-4; dx=1.5507389905740702E-5 evalInputDelta=1.9384237382175866E-4
New Minimum: 1.5507389905740707E-4 > 9.692118691087914E-6
END: th(25.000000000000007)=9.692118691087914E-6; dx=-3.8768474764351705E-6 evalInputDelta=3.392241541880778E-4
Fitness changed from 3.489162728791657E-4 to 9.692118691087914E-6
Iteration 6 complete. Error: 9.692118691087914E-6 Total: 0.3197; Orientation: 0.0020; Line Search: 0.3071
th(0)=9.692118691087914E-6;dx=-6.461412460725277E-7
New Minimum: 9.692118691087914E-6 > 6.1312449018076425E-6
WOLF (strong): th(53.860867250797114)=6.1312449018076425E-6; dx=5.139163499267057E-7 evalInputDelta=3.560873789280272E-6
New Minimum: 6.1312449018076425E-6 > 1.0146827377359247E-7
END: th(26.930433625398557)=1.0146827377359247E-7; dx=-6.611244807290954E-8 evalInputDelta=9.590650417314322E-6
Fitness changed from 9.692118691087914E-6 to 1.0146827377359247E-7
Iteration 7 complete. Error: 1.0146827377359247E-7 Total: 0.0623; Orientation: 0.0016; Line Search: 0.0450
th(0)=1.0146827377359247E-7;dx=-6.764551584906165E-9
New Minimum: 1.0146827377359247E-7 > 8.851557556007537E-8
WOLF (strong): th(58.01986042015976)=8.851557556007537E-8; dx=6.318059707134722E-9 evalInputDelta=1.29526982135171E-8
New Minimum: 8.851557556007537E-8 > 1.1051452990655342E-10
END: th(29.00993021007988)=1.1051452990655342E-10; dx=-2.2324593888572543E-10 evalInputDelta=1.0135775924368591E-7
Fitness changed from 1.0146827377359247E-7 to 1.1051452990655342E-10
Iteration 8 complete. Error: 1.1051452990655342E-10 Total: 0.0463; Orientation: 0.0017; Line Search: 0.0346
Low gradient: 2.7143388379315437E-6
th(0)=1.1051452990655342E-10;dx=-7.367635327103563E-12
Armijo: th(62.50000000000003)=1.2970108023760764E-10; dx=7.98160493769723E-12 evalInputDelta=-1.918655033105421E-11
New Minimum: 1.1051452990655342E-10 > 1.9186550330516987E-13
WOLF (strong): th(31.250000000000014)=1.9186550330516987E-13; dx=3.069848052921268E-13 evalInputDelta=1.1032266440324826E-10
END: th(10.416666666666671)=4.709232186802398E-11; dx=-4.8094286163062724E-12 evalInputDelta=6.342220803852945E-11
Fitness changed from 1.1051452990655342E-10 to 1.9186550330516987E-13
Iteration 9 complete. Error: 1.9186550330516987E-13 Total: 0.0628; Orientation: 0.0013; Line Search: 0.0537
Low gradient: 1.1309745157906075E-7
th(0)=1.9186550330516987E-13;dx=-1.2791033553677991E-14
New Minimum: 1.9186550330516987E-13 > 1.2177690797248566E-14
END: th(22.442028021165466)=1.2177690797248566E-14; dx=-3.2224757727966667E-15 evalInputDelta=1.7968781250792131E-13
Fitness changed from 1.9186550330516987E-13 to 1.2177690797248566E-14
Iteration 10 complete. Error: 1.2177690797248566E-14 Total: 0.1387; Orientation: 0.0014; Line Search: 0.1259
Low gradient: 2.8492912331839726E-8
th(0)=1.2177690797248566E-14;dx=-8.118460531499044E-16
New Minimum: 1.2177690797248566E-14 > 4.556056116441386E-15
WOLF (strong): th(48.34988368346647)=4.556056116441386E-15; dx=4.965760214416601E-16 evalInputDelta=7.62163468080718E-15
New Minimum: 4.556056116441386E-15 > 4.591165677993159E-16
END: th(24.174941841733236)=4.591165677993159E-16; dx=-1.57635015886616E-16 evalInputDelta=1.171857422944925E-14
Fitness changed from 1.2177690797248566E-14 to 4.591165677993159E-16
Iteration 11 complete. Error: 4.591165677993159E-16 Total: 0.0477; Orientation: 0.0016; Line Search: 0.0373
Low gradient: 5.532429049397838E-9
th(0)=4.591165677993159E-16;dx=-3.0607771186621066E-17
New Minimum: 4.591165677993159E-16 > 2.4877670504345536E-16
WOLF (strong): th(52.083333333333364)=2.4877670504345536E-16; dx=2.2530720456792023E-17 evalInputDelta=2.1033986275586053E-16
New Minimum: 2.4877670504345536E-16 > 7.992914817411177E-18
END: th(26.041666666666682)=7.992914817411177E-18; dx=-4.0385253731650975E-18 evalInputDelta=4.511236529819047E-16
Fitness changed from 4.591165677993159E-16 to 7.992914817411177E-18
Iteration 12 complete. Error: 7.992914817411177E-18 Total: 0.0430; Orientation: 0.0016; Line Search: 0.0321
Low gradient: 7.299732788447889E-10
th(0)=7.992914817411177E-18;dx=-5.328609878274119E-19
New Minimum: 7.992914817411177E-18 > 6.052187902266769E-18
WOLF (strong): th(56.105070052913675)=6.052187902266769E-18; dx=4.636791149145411E-19 evalInputDelta=1.940726915144408E-18
New Minimum: 6.052187902266769E-18 > 3.368231705099139E-20
END: th(28.052535026456837)=3.368231705099139E-20; dx=-3.459093593810041E-20 evalInputDelta=7.959232500360185E-18
Fitness changed from 7.992914817411177E-18 to 3.368231705099139E-20
Iteration 13 complete. Error: 3.368231705099139E-20 Total: 0.0513; Orientation: 0.0011; Line Search: 0.0413
Zero gradient: 4.7386578304404145E-11
th(0)=3.368231705099139E-20;dx=-2.245487803399426E-21
Armijo: th(60.4373546043331)=3.467155093047735E-20; dx=2.2782236463134423E-21 evalInputDelta=-9.892338794859573E-22
New Minimum: 3.368231705099139E-20 > 1.7896537185934047E-24
WOLF (strong): th(30.21867730216655)=1.7896537185934047E-24; dx=1.6367947523914658E-23 evalInputDelta=3.36805273972728E-20
END: th(10.072892434055516)=1.486099937678187E-20; dx=-1.4915359610567962E-21 evalInputDelta=1.882131767420952E-20
Fitness changed from 3.368231705099139E-20 to 1.7896537185934047E-24
Iteration 14 complete. Error: 1.7896537185934047E-24 Total: 0.0730; Orientation: 0.0011; Line Search: 0.0622
Zero gradient: 3.454131553751637E-13
th(0)=1.7896537185934047E-24;dx=-1.1931024790622698E-25
New Minimum: 1.7896537185934047E-24 > 1.3694322681087109E-25
END: th(21.701388888888903)=1.3694322681087109E-25; dx=-3.300376761020707E-26 evalInputDelta=1.6527104917825336E-24
Fitness changed from 1.7896537185934047E-24 to 1.3694322681087109E-25
Iteration 15 complete. Error: 1.3694322681087109E-25 Total: 0.0338; Orientation: 0.0014; Line Search: 0.0233
Zero gradient: 9.554867060330077E-14
th(0)=1.3694322681087109E-25;dx=-9.129548454058072E-27
New Minimum: 1.3694322681087109E-25 > 4.2709779455963896E-26
WOLF (strong): th(46.75422504409473)=4.2709779455963896E-26; dx=5.098502883750364E-27 evalInputDelta=9.42334473549072E-26
New Minimum: 4.2709779455963896E-26 > 6.674967917920941E-27
END: th(23.377112522047366)=6.674967917920941E-27; dx=-2.0155962974384106E-27 evalInputDelta=1.3026825889295014E-25
Fitness changed from 1.3694322681087109E-25 to 6.674967917920941E-27
Iteration 16 complete. Error: 6.674967917920941E-27 Total: 0.0487; Orientation: 0.0014; Line Search: 0.0375
Zero gradient: 2.109497241512132E-14
th(0)=6.674967917920941E-27;dx=-4.449978611947295E-28
New Minimum: 6.674967917920941E-27 > 3.076273640263435E-27
WOLF (strong): th(50.364462170277584)=3.076273640263435E-27; dx=3.020965008277369E-28 evalInputDelta=3.598694277657506E-27
New Minimum: 3.076273640263435E-27 > 1.7179124598948754E-28
END: th(25.182231085138792)=1.7179124598948754E-28; dx=-7.138908572730429E-29 evalInputDelta=6.503176671931453E-27
Fitness changed from 6.674967917920941E-27 to 1.7179124598948754E-28
Iteration 17 complete. Error: 1.7179124598948754E-28 Total: 0.0391; Orientation: 0.0009; Line Search: 0.0309
Zero gradient: 3.3841911489501453E-15
th(0)=1.7179124598948754E-28;dx=-1.1452749732632504E-29
New Minimum: 1.7179124598948754E-28 > 1.1228037161916882E-28
WOLF (strong): th(54.253472222222264)=1.1228037161916882E-28; dx=9.25885507069949E-30 evalInputDelta=5.951087437031873E-29
New Minimum: 1.1228037161916882E-28 > 1.57561532456535E-30
END: th(27.126736111111132)=1.57561532456535E-30; dx=-1.0959338370822996E-30 evalInputDelta=1.702156306649222E-28
Fitness changed from 1.7179124598948754E-28 to 1.57561532456535E-30
Iteration 18 complete. Error: 1.57561532456535E-30 Total: 0.0391; Orientation: 0.0012; Line Search: 0.0308
Zero gradient: 3.2410032650043723E-16
th(0)=1.57561532456535E-30;dx=-1.0504102163769E-31
New Minimum: 1.57561532456535E-30 > 1.4113471423128916E-30
WOLF (strong): th(58.44278130511842)=1.4113471423128916E-30; dx=9.935091511505192E-32 evalInputDelta=1.642681822524585E-31
New Minimum: 1.4113471423128916E-30 > 2.030251149708211E-34
END: th(29.22139065255921)=2.030251149708211E-34; dx=-4.74554488102416E-34 evalInputDelta=1.5754122994503792E-30
Fitness changed from 1.57561532456535E-30 to 2.030251149708211E-34
Iteration 19 complete. Error: 2.030251149708211E-34 Total: 0.0382; Orientation: 0.0008; Line Search: 0.0295
Zero gradient: 3.678995469516293E-18
th(0)=2.030251149708211E-34;dx=-1.353500766472141E-35
Armijo: th(62.955577712846996)=2.1305600009783796E-34; dx=1.3855995988785948E-35 evalInputDelta=-1.0030885127016839E-35
New Minimum: 2.030251149708211E-34 > 4.012354050806741E-37
WOLF (strong): th(31.477788856423498)=4.012354050806741E-37; dx=3.2098832406453933E-37 evalInputDelta=2.0262387956574042E-34
END: th(10.4925962854745)=1.5828736730432595E-34; dx=-1.1689324801350307E-35 evalInputDelta=4.473774766649517E-35
Fitness changed from 2.030251149708211E-34 to 4.012354050806741E-37
Iteration 20 complete. Error: 4.012354050806741E-37 Total: 0.0552; Orientation: 0.0014; Line Search: 0.0412
Zero gradient: 1.635512977795599E-19
th(0)=4.012354050806741E-37;dx=-2.6749027005378276E-38
New Minimum: 4.012354050806741E-37 > 0.0
END: th(22.60561342592595)=0.0; dx=0.0 evalInputDelta=4.012354050806741E-37
Fitness changed from 4.012354050806741E-37 to 0.0
Iteration 21 complete. Error: 0.0 Total: 0.1823; Orientation: 0.0012; Line Search: 0.0226
Zero gradient: 0.0
th(0)=0.0;dx=0.0 (ERROR: Starting derivative negative)
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0349; Orientation: 0.0013; Line Search: 0.0197
Iteration 22 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 22
Final threshold in iteration 22: 0.0 (> 0.0) after 1.784s (< 30.000s)
Returns
0.0
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.29 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1917676775913
Reset training subject: 1917685638049
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=2.2050794666666667}, derivative=-0.14700529777777777}
New Minimum: 2.2050794666666667 > 2.205079466651966
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=2.205079466651966}, derivative=-0.14700529777728777}, evalInputDelta = -1.4700685113666623E-11
New Minimum: 2.205079466651966 > 2.2050794665637627
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=2.2050794665637627}, derivative=-0.14700529777434768}, evalInputDelta = -1.0290390761724666E-10
New Minimum: 2.2050794665637627 > 2.2050794659463406
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=2.2050794659463406}, derivative=-0.14700529775376692}, evalInputDelta = -7.203260210530971E-10
New Minimum: 2.2050794659463406 > 2.205079461624385
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=2.205079461624385}, derivative=-0.14700529760970174}, evalInputDelta = -5.04228170328247E-9
New Minimum: 2.205079461624385 > 2.2050794313706943
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=2.2050794313706943}, derivative=-0.14700529660124537}, evalInputDelta = -3.52959723670665E-8
New Minimum: 2.2050794313706943 > 2.2050792195948694
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=2.2050792195948694}, derivative=-0.147005289542051}, evalInputDelta = -2.4707179724359207E-7
New Minimum: 2.2050792195948694 > 2.2050777371643777
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=2.2050777371643777}, derivative=-0.1470052401276902}, evalInputDelta = -1.7295022889385336E-6
New Minimum: 2.2050777371643777 > 2.2050673601648887
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=2.2050673601648887}, derivative=-0.14700489422716462}, evalInputDelta = -1.210650177796424E-5
New Minimum: 2.2050673601648887 > 2.204994721852139
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=2.204994721852139}, derivative=-0.14700247292348567}, evalInputDelta = -8.474481452758198E-5
New Minimum: 2.204994721852139 > 2.204486287162893
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=2.204486287162893}, derivative=-0.146985523797733}, evalInputDelta = -5.931795037734666E-4
New Minimum: 2.204486287162893 > 2.200928885838235
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=2.200928885838235}, derivative=-0.14686687991746414}, evalInputDelta = -0.004150580828431671
New Minimum: 2.200928885838235 > 2.1761075100687166
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=2.1761075100687166}, derivative=-0.1460363727555822}, evalInputDelta = -0.028971956597950044
New Minimum: 2.1761075100687166 > 2.006299121333547
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=2.006299121333547}, derivative=-0.14022282262240862}, evalInputDelta = -0.1987803453331196
New Minimum: 2.006299121333547 > 1.0107612411089657
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=1.0107612411089657}, derivative=-0.09952797169019363}, evalInputDelta = -1.194318225557701
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=3.504917284695356}, derivative=0.1853359848353113}, evalInputDelta = 1.2998378180286894
F(5.217100560376924) = LineSearchPoint{point=PointSample{avg=1.5048248872425045}, derivative=-0.12144058373061709}, evalInputDelta = -0.7002545794241621
New Minimum: 1.0107612411089657 > 0.10414477430520844
F(36.51970392263847) = LineSearchPoint{point=PointSample{avg=0.10414477430520844}, derivative=0.031947700552347136}, evalInputDelta = -2.100934692361458
0.10414477430520844 <= 2.2050794666666667
New Minimum: 0.10414477430520844 > 2.1795909674792378E-32
F(30.0) = LineSearchPoint{point=PointSample{avg=2.1795909674792378E-32}, derivative=2.6478510742025375E-18}, evalInputDelta = -2.2050794666666667
Right bracket at 30.0
Converged to right
Fitness changed from 2.2050794666666667 to 2.1795909674792378E-32
Iteration 1 complete. Error: 2.1795909674792378E-32 Total: 0.2427; Orientation: 0.0011; Line Search: 0.2085
Zero gradient: 3.811903258198139E-17
F(0.0) = LineSearchPoint{point=PointSample{avg=2.1795909674792378E-32}, derivative=-1.4530606449861588E-33}
New Minimum: 2.1795909674792378E-32 > 0.0
F(30.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}, evalInputDelta = -2.1795909674792378E-32
0.0 <= 2.1795909674792378E-32
Converged to right
Fitness changed from 2.1795909674792378E-32 to 0.0
Iteration 2 complete. Error: 0.0 Total: 0.0275; Orientation: 0.0011; Line Search: 0.0193
Zero gradient: 0.0
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0214; Orientation: 0.0012; Line Search: 0.0107
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 0.292s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 7.24 seconds (0.131 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1917973476858
Reset training subject: 1917980197046
Adding measurement 4111a02 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 2.2050794666666667 < 2.2050794666666667. Total: 1
th(0)=2.2050794666666667;dx=-0.1470052977777778
Adding measurement 69391c38 to history. Total: 1
New Minimum: 2.2050794666666667 > 1.8997384559932093
WOLFE (weak): th(2.154434690031884)=1.8997384559932093; dx=-0.13644818733941408 evalInputDelta=0.3053410106734573
Adding measurement 2fd58856 to history. Total: 2
New Minimum: 1.8997384559932093 > 1.6171420502746607
END: th(4.308869380063768)=1.6171420502746607; dx=-0.12589107690105034 evalInputDelta=0.5879374163920059
Fitness changed from 2.2050794666666667 to 1.6171420502746607
Iteration 1 complete. Error: 1.6171420502746607 Total: 0.0680; Orientation: 0.0051; Line Search: 0.0418
Non-optimal measurement 1.6171420502746607 < 1.6171420502746607. Total: 3
LBFGS Accumulation History: 3 points
Non-optimal measurement 1.6171420502746607 < 1.6171420502746607. Total: 3
th(0)=1.6171420502746607;dx=-0.1078094700183107
Adding measurement 6fee7571 to history. Total: 3
New Minimum: 1.6171420502746607 > 0.7711732272995226
END: th(9.283177667225559)=0.7711732272995226; dx=-0.0744489878719972 evalInputDelta=0.8459688229751381
Fitness changed from 1.6171420502746607 to 0.7711732272995226
Iteration 2 complete. Error: 0.7711732272995226 Total: 0.0319; Orientation: 0.0027; Line Search: 0.0204
Non-optimal measurement 0.7711732272995226 < 0.7711732272995226. Total: 4
Rejected: LBFGS Orientation magnitude: 6.802e+00, gradient 2.267e-01, dot -1.000; [a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.7711732272995226, 1.6171420502746607, 1.8997384559932093, 2.2050794666666667
LBFGS Accumulation History: 3 points
Removed measurement 6fee7571 to history. Total: 3
Adding measurement 74148b91 to history. Total: 3
th(0)=0.7711732272995226;dx=-0.051411548486634845
Adding measurement 5eccb9f0 to history. Total: 4
New Minimum: 0.7711732272995226 > 0.08568591414439135
END: th(20.000000000000004)=0.08568591414439135; dx=-0.017137182828878276 evalInputDelta=0.6854873131551313
Fitness changed from 0.7711732272995226 to 0.08568591414439135
Iteration 3 complete. Error: 0.08568591414439135 Total: 0.1132; Orientation: 0.0843; Line Search: 0.0212
Non-optimal measurement 0.08568591414439135 < 0.08568591414439135. Total: 5
Rejected: LBFGS Orientation magnitude: 2.267e+00, gradient 7.558e-02, dot -1.000; [a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.08568591414439135, 0.7711732272995226, 1.6171420502746607, 1.8997384559932093, 2.2050794666666667
Rejected: LBFGS Orientation magnitude: 2.267e+00, gradient 7.558e-02, dot -1.000; [a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.08568591414439135, 0.7711732272995226, 1.6171420502746607, 1.8997384559932093
LBFGS Accumulation History: 3 points
Removed measurement 5eccb9f0 to history. Total: 4
Removed measurement 74148b91 to history. Total: 3
Adding measurement eb02f8d to history. Total: 3
th(0)=0.08568591414439135;dx=-0.005712394276292756
Adding measurement 69b9e6b1 to history. Total: 4
New Minimum: 0.08568591414439135 > 0.016310209544924275
WOLF (strong): th(43.088693800637685)=0.016310209544924275; dx=0.0024922593183637066 evalInputDelta=0.06937570459946707
Adding measurement 1ea2a201 to history. Total: 5
New Minimum: 0.016310209544924275 > 0.006807086034601106
END: th(21.544346900318843)=0.006807086034601106; dx=-0.001610067478964525 evalInputDelta=0.07887882810979024
Fitness changed from 0.08568591414439135 to 0.006807086034601106
Iteration 4 complete. Error: 0.006807086034601106 Total: 0.2492; Orientation: 0.2056; Line Search: 0.0348
Non-optimal measurement 0.006807086034601106 < 0.006807086034601106. Total: 6
Rejected: LBFGS Orientation magnitude: 6.391e-01, gradient 2.130e-02, dot -1.000; [a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.006807086034601106, 0.016310209544924275, 0.08568591414439135, 1.6171420502746607, 1.8997384559932093, 2.2050794666666667
Rejected: LBFGS Orientation magnitude: 6.391e-01, gradient 2.130e-02, dot -1.000; [8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.006807086034601106, 0.016310209544924275, 0.08568591414439135, 1.6171420502746607, 1.8997384559932093
Rejected: LBFGS Orientation magnitude: 6.391e-01, gradient 2.130e-02, dot -1.000; [775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.006807086034601106, 0.016310209544924275, 0.08568591414439135, 1.6171420502746607
LBFGS Accumulation History: 3 points
Removed measurement 1ea2a201 to history. Total: 5
Removed measurement 69b9e6b1 to history. Total: 4
Removed measurement eb02f8d to history. Total: 3
Adding measurement 1726136b to history. Total: 3
th(0)=0.006807086034601106;dx=-4.5380573564007376E-4
Adding measurement 77bd7179 to history. Total: 4
New Minimum: 0.006807086034601106 > 0.0020382033394773396
WOLF (strong): th(46.4158883361278)=0.0020382033394773396; dx=2.483208094187261E-4 evalInputDelta=0.004768882695123766
Adding measurement 4a114377 to history. Total: 5
New Minimum: 0.0020382033394773396 > 3.489162728791657E-4
END: th(23.2079441680639)=3.489162728791657E-4; dx=-1.0274246311067382E-4 evalInputDelta=0.0064581697617219404
Fitness changed from 0.006807086034601106 to 3.489162728791657E-4
Iteration 5 complete. Error: 3.489162728791657E-4 Total: 0.5031; Orientation: 0.4496; Line Search: 0.0433
Non-optimal measurement 3.489162728791657E-4 < 3.489162728791657E-4. Total: 6
Rejected: LBFGS Orientation magnitude: 1.447e-01, gradient 4.823e-03, dot -1.000; [a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.489162728791657E-4, 0.0020382033394773396, 0.006807086034601106, 1.6171420502746607, 1.8997384559932093, 2.2050794666666667
Rejected: LBFGS Orientation magnitude: 1.447e-01, gradient 4.823e-03, dot -1.000; [8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00, a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.489162728791657E-4, 0.0020382033394773396
...skipping 30026 bytes...
5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.57561532456535E-30, 1.1228037161916882E-28, 1.7179124598948754E-28, 1.6171420502746607, 1.8997384559932093, 2.2050794666666667
Rejected: LBFGS Orientation magnitude: 9.723e-15, gradient 3.241e-16, dot -1.000; [ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.57561532456535E-30, 1.1228037161916882E-28, 1.7179124598948754E-28, 1.6171420502746607, 1.8997384559932093
Rejected: LBFGS Orientation magnitude: 9.723e-15, gradient 3.241e-16, dot -1.000; [ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.57561532456535E-30, 1.1228037161916882E-28, 1.7179124598948754E-28, 1.6171420502746607
LBFGS Accumulation History: 3 points
Removed measurement 4f6fc936 to history. Total: 5
Removed measurement 3c49b13f to history. Total: 4
Removed measurement 473039d5 to history. Total: 3
Adding measurement 46d08f10 to history. Total: 3
th(0)=1.57561532456535E-30;dx=-1.0504102163769E-31
Adding measurement f9abd3d to history. Total: 4
New Minimum: 1.57561532456535E-30 > 1.4113471423128916E-30
WOLF (strong): th(58.44278130511842)=1.4113471423128916E-30; dx=9.935091511505192E-32 evalInputDelta=1.642681822524585E-31
Adding measurement 5a4f98ba to history. Total: 5
New Minimum: 1.4113471423128916E-30 > 2.030251149708211E-34
END: th(29.22139065255921)=2.030251149708211E-34; dx=-4.74554488102416E-34 evalInputDelta=1.5754122994503792E-30
Fitness changed from 1.57561532456535E-30 to 2.030251149708211E-34
Iteration 19 complete. Error: 2.030251149708211E-34 Total: 0.3501; Orientation: 0.3030; Line Search: 0.0397
Non-optimal measurement 2.030251149708211E-34 < 2.030251149708211E-34. Total: 6
Rejected: LBFGS Orientation magnitude: 1.104e-16, gradient 3.679e-18, dot -1.000; [a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.030251149708211E-34, 1.4113471423128916E-30, 1.57561532456535E-30, 1.6171420502746607, 1.8997384559932093, 2.2050794666666667
Rejected: LBFGS Orientation magnitude: 1.104e-16, gradient 3.679e-18, dot -1.000; [e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00, a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.030251149708211E-34, 1.4113471423128916E-30, 1.57561532456535E-30, 1.6171420502746607, 1.8997384559932093
Rejected: LBFGS Orientation magnitude: 1.104e-16, gradient 3.679e-18, dot -1.000; [775bf797-3312-4051-b826-5f59db20b81e = 1.000/1.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 1.000/1.000e+00, a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.030251149708211E-34, 1.4113471423128916E-30, 1.57561532456535E-30, 1.6171420502746607
LBFGS Accumulation History: 3 points
Removed measurement 5a4f98ba to history. Total: 5
Removed measurement f9abd3d to history. Total: 4
Removed measurement 46d08f10 to history. Total: 3
Adding measurement b9a56c5 to history. Total: 3
th(0)=2.030251149708211E-34;dx=-1.353500766472141E-35
Non-optimal measurement 2.1305600009783796E-34 < 2.030251149708211E-34. Total: 4
Armijo: th(62.955577712846996)=2.1305600009783796E-34; dx=1.3855995988785948E-35 evalInputDelta=-1.0030885127016839E-35
Adding measurement 6c43d573 to history. Total: 4
New Minimum: 2.030251149708211E-34 > 4.012354050806741E-37
WOLF (strong): th(31.477788856423498)=4.012354050806741E-37; dx=3.2098832406453933E-37 evalInputDelta=2.0262387956574042E-34
Non-optimal measurement 1.5828736730432595E-34 < 4.012354050806741E-37. Total: 5
END: th(10.4925962854745)=1.5828736730432595E-34; dx=-1.1689324801350307E-35 evalInputDelta=4.473774766649517E-35
Fitness changed from 2.030251149708211E-34 to 4.012354050806741E-37
Iteration 20 complete. Error: 4.012354050806741E-37 Total: 0.3423; Orientation: 0.2892; Line Search: 0.0439
Non-optimal measurement 4.012354050806741E-37 < 4.012354050806741E-37. Total: 5
Rejected: LBFGS Orientation magnitude: 4.907e-18, gradient 1.636e-19, dot -1.000; [e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 0.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 0.000e+00, a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 0.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.012354050806741E-37, 2.030251149708211E-34, 1.6171420502746607, 1.8997384559932093, 2.2050794666666667
Rejected: LBFGS Orientation magnitude: 4.907e-18, gradient 1.636e-19, dot -1.000; [775bf797-3312-4051-b826-5f59db20b81e = 0.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 1.000/1.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 0.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 0.000e+00, a2824a36-70bf-4e05-98bc-51f8783f5c86 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.012354050806741E-37, 2.030251149708211E-34, 1.6171420502746607, 1.8997384559932093
LBFGS Accumulation History: 3 points
Removed measurement 6c43d573 to history. Total: 4
Removed measurement b9a56c5 to history. Total: 3
Adding measurement 5252c88c to history. Total: 3
th(0)=4.012354050806741E-37;dx=-2.6749027005378276E-38
Adding measurement 730e6f82 to history. Total: 4
New Minimum: 4.012354050806741E-37 > 0.0
END: th(22.60561342592595)=0.0; dx=0.0 evalInputDelta=4.012354050806741E-37
Fitness changed from 4.012354050806741E-37 to 0.0
Iteration 21 complete. Error: 0.0 Total: 0.3224; Orientation: 0.2968; Line Search: 0.0190
Non-optimal measurement 0.0 < 0.0. Total: 5
Rejected: LBFGS Orientation magnitude: 0.000e+00, gradient 0.000e+00, dot NaN; [ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 0.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 0.000e+00, a2824a36-70bf-4e05-98bc-51f8783f5c86 = 0.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 0.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 0.000e+00]
Orientation rejected. Popping history element from 0.0, 4.012354050806741E-37, 1.6171420502746607, 1.8997384559932093, 2.2050794666666667
Rejected: LBFGS Orientation magnitude: 0.000e+00, gradient 0.000e+00, dot NaN; [a2824a36-70bf-4e05-98bc-51f8783f5c86 = 0.000e+00, 8a8671a4-5b5b-4b61-9cc7-b8741196990a = 0.000e+00, e0c7a5da-20bd-4c1f-af36-9fe5ec205a2f = 0.000e+00, 775bf797-3312-4051-b826-5f59db20b81e = 0.000e+00, ea6c2bff-017c-46bc-9489-12b1bcc4b46d = 0.000e+00]
Orientation rejected. Popping history element from 0.0, 4.012354050806741E-37, 1.6171420502746607, 1.8997384559932093
LBFGS Accumulation History: 3 points
Removed measurement 730e6f82 to history. Total: 4
Removed measurement 5252c88c to history. Total: 3
Adding measurement 327e9243 to history. Total: 3
th(0)=0.0;dx=0.0 (ERROR: Starting derivative negative)
Non-optimal measurement 0.0 < 0.0. Total: 4
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.2247; Orientation: 0.2006; Line Search: 0.0178
Iteration 22 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 22
Final threshold in iteration 22: 0.0 (> 0.0) after 7.240s (< 30.000s)
Returns
0.0
TrainingTester.java:432 executed in 0.14 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -36.39660075174148], [21.0, 0.20874817014811511]; valueStats=DoubleSummaryStatistics{count=41, sum=4.962334, min=0.000000, average=0.121033, max=1.617142}
Plotting 21 points for GD
Plotting 2 points for CjGD
Plotting 21 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.02 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -36.39660075174148], [6.947, 0.20874817014811511]; valueStats=DoubleSummaryStatistics{count=41, sum=4.962334, min=0.000000, average=0.121033, max=1.617142}
Plotting 21 points for GD
Plotting 2 points for CjGD
Plotting 21 points for LBFGS
Returns
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null} | OK |
{
"result": "OK",
"performance": {
"execution_time": "10.313",
"gc_time": "0.449"
},
"created_on": 1586736547492,
"file_name": "trainingTest",
"report": {
"simpleName": "LR",
"canonicalName": "com.simiacryptus.mindseye.layers.cudnn.ImgTileSelectLayerTest.LR",
"link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/ImgTileSelectLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "Converged",
"value": 0.0
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "Converged",
"value": 0.0
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/ImgTileSelectLayer/LR/trainingTest/202004130907",
"id": "9b58b4a5-9c6b-43e4-8d20-5bfe1a9bfe89",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "ImgTileSelectLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.cudnn.ImgTileSelectLayer",
"link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/ImgTileSelectLayer.java",
"javaDoc": ""
}
}