Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
Using Seed 6248868336102476800
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[ 0.7, -0.128, 0.08 ]
[ -0.608, 0.496, 1.764 ]
[ -0.128, 0.7, 0.08 ]
[ 1.764, 0.496, -0.608 ]
[ 0.08, -0.128, 0.7 ]
[ 1.764, 0.496, -0.608 ]
[ 0.7, -0.128, 0.08 ]
[ -0.608, 0.496, 1.764 ]
[ 0.7, -0.128, 0.08 ]
[ 1.764, 0.496, -0.608 ]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 0.32 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 4497065978431
BACKPROP_AGG_SIZE = 3
THREADS = 64
SINGLE_THREADED = false
Initialized CoreSettings = {
"backpropAggregationSize" : 3,
"jvmThreads" : 64,
"singleThreaded" : false
}
Reset training subject: 4497097320682
Constructing line search parameters: GD
th(0)=0.02123649249407037;dx=-6.584203173979122E-4
New Minimum: 0.02123649249407037 > 0.019943683481494046
END: th(2.154434690031884)=0.019943683481494046; dx=-5.49290382563083E-4 evalInputDelta=0.0012928090125763243
Fitness changed from 0.02123649249407037 to 0.019943683481494046
Iteration 1 complete. Error: 0.019943683481494046 Total: 0.0670; Orientation: 0.0038; Line Search: 0.0232
th(0)=0.019943683481494046;dx=-4.605967442067415E-4
New Minimum: 0.019943683481494046 > 0.018051237387870053
END: th(4.641588833612779)=0.018051237387870053; dx=-3.6349481391335264E-4 evalInputDelta=0.001892446093623993
Fitness changed from 0.019943683481494046 to 0.018051237387870053
Iteration 2 complete. Error: 0.018051237387870053 Total: 0.0247; Orientation: 0.0013; Line Search: 0.0183
th(0)=0.018051237387870053;dx=-2.8911262486516905E-4
New Minimum: 0.018051237387870053 > 0.015544252973654474
END: th(10.000000000000002)=0.015544252973654474; dx=-2.1946140489624482E-4 evalInputDelta=0.002506984414215579
Fitness changed from 0.018051237387870053 to 0.015544252973654474
Iteration 3 complete. Error: 0.015544252973654474 Total: 0.0208; Orientation: 0.0028; Line Search: 0.0138
th(0)=0.015544252973654474;dx=-1.683087468988806E-4
New Minimum: 0.015544252973654474 > 0.012430596523208862
END: th(21.544346900318843)=0.012430596523208862; dx=-1.2553217683609132E-4 evalInputDelta=0.0031136564504456125
Fitness changed from 0.015544252973654474 to 0.012430596523208862
Iteration 4 complete. Error: 0.012430596523208862 Total: 0.0271; Orientation: 0.0015; Line Search: 0.0212
th(0)=0.012430596523208862;dx=-9.496432580034236E-5
New Minimum: 0.012430596523208862 > 0.008644351843561585
END: th(46.4158883361278)=0.008644351843561585; dx=-7.093368823483336E-5 evalInputDelta=0.003786244679647277
Fitness changed from 0.012430596523208862 to 0.008644351843561585
Iteration 5 complete. Error: 0.008644351843561585 Total: 0.0173; Orientation: 0.0012; Line Search: 0.0132
th(0)=0.008644351843561585;dx=-5.3869190176455166E-5
New Minimum: 0.008644351843561585 > 0.0040081347419547815
END: th(100.00000000000003)=0.0040081347419547815; dx=-4.035626069728204E-5 evalInputDelta=0.0046362171016068035
Fitness changed from 0.008644351843561585 to 0.0040081347419547815
Iteration 6 complete. Error: 0.0040081347419547815 Total: 0.0127; Orientation: 0.0012; Line Search: 0.0086
th(0)=0.0040081347419547815;dx=-3.0648125475614005E-5
New Minimum: 0.0040081347419547815 > 0.0
WOLFE (weak): th(215.44346900318845)=0.0; dx=-3.4890776621990966E-5 evalInputDelta=0.0040081347419547815
WOLFE (weak): th(430.8869380063769)=0.0; dx=-8.758823005714959E-5 evalInputDelta=0.0040081347419547815
WOLFE (weak): th(1292.6608140191306)=0.0; dx=-0.0010406873757013333 evalInputDelta=0.0040081347419547815
WOLFE (weak): th(5170.643256076522)=0.0; dx=-0.13828239943284742 evalInputDelta=0.0040081347419547815
WOLFE (weak): th(25853.216280382614)=0.0; dx=-190.27651601685886 evalInputDelta=0.0040081347419547815
WOLFE (weak): th(155119.29768229567)=0.0; dx=-1206870.2064435042 evalInputDelta=0.0040081347419547815
WOLFE (weak): th(1085835.0837760698)=0.0; dx=-1.9556543036010754E10 evalInputDelta=0.0040081347419547815
WOLFE (weak): th(8686680.670208558)=0.0; dx=-6.373983747966721E14 evalInputDelta=0.0040081347419547815
WOLFE (weak): th(7.818012603187703E7)=0.0; dx=-3.7612062652355576E19 evalInputDelta=0.0040081347419547815
MAX ALPHA (7.818012603187703E8): th(215.44346900318845)=0.0
Fitness changed from 0.0040081347419547815 to 0.0
Iteration 7 complete. Error: 0.0 Total: 0.0557; Orientation: 0.0011; Line Search: 0.0518
th(0)=0.0;dx=-4.005634491809372E-5
Armijo: th(1.0E8)=0.0; dx=-3.8797970290437797E20 evalInputDelta=0.0
Armijo: th(5.0E7)=0.0; dx=-1.212508560529987E19 evalInputDelta=0.0
Armijo: th(1.6666666666666666E7)=0.0; dx=-4.9909322906064688E16 evalInputDelta=0.0
Armijo: th(4166666.6666666665)=0.0; dx=-4.879167806128328E13 evalInputDelta=0.0
Armijo: th(833333.3333333333)=0.0; dx=-1.5702476014909039E10 evalInputDelta=0.0
Armijo: th(138888.88888888888)=0.0; dx=-2092051.434819684 evalInputDelta=0.0
Armijo: th(19841.26984126984)=0.0; dx=-158.97136766008705 evalInputDelta=0.0
Armijo: th(2480.15873015873)=0.0; dx=-0.023254473188831882 evalInputDelta=0.0
Armijo: th(275.5731922398589)=0.0; dx=-1.487475219201275E-4 evalInputDelta=0.0
Armijo: th(27.55731922398589)=0.0; dx=-4.6539626555877855E-5 evalInputDelta=0.0
Armijo: th(2.505210838544172)=0.0; dx=-4.0615023425877445E-5 evalInputDelta=0.0
Armijo: th(0.20876756987868098)=0.0; dx=-4.0102673306459594E-5 evalInputDelta=0.0
Armijo: th(0.016059043836821614)=0.0; dx=-4.005990717125855E-5 evalInputDelta=0.0
Armijo: th(0.0011470745597729724)=0.0; dx=-4.0056599356630506E-5 evalInputDelta=0.0
Armijo: th(7.647163731819817E-5)=0.0; dx=-4.005636188062398E-5 evalInputDelta=0.0
Armijo: th(4.7794773323873855E-6)=0.0; dx=-4.00563459782517E-5 evalInputDelta=0.0
Armijo: th(2.811457254345521E-7)=0.0; dx=-4.005634498045595E-5 evalInputDelta=0.0
Armijo: th(1.5619206968586227E-8)=0.0; dx=-4.005634492155828E-5 evalInputDelta=0.0
Armijo: th(8.22063524662433E-10)=0.0; dx=-4.005634491827607E-5 evalInputDelta=0.0
Armijo: th(4.110317623312165E-11)=0.0; dx=-4.0056344918102825E-5 evalInputDelta=0.0
Armijo: th(1.9572941063391265E-12)=0.0; dx=-4.005634491809416E-5 evalInputDelta=0.0
Armijo: th(8.896791392450575E-14)=0.0; dx=-4.005634491809373E-5 evalInputDelta=0.0
Armijo: th(3.868170170630685E-15)=0.0; dx=-4.005634491809372E-5 evalInputDelta=0.0
MIN ALPHA (1.6117375710961187E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0814; Orientation: 0.0011; Line Search: 0.0780
Iteration 8 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 8
Final threshold in iteration 8: 0.0 (> 0.0) after 0.308s (< 30.000s)
Returns
0.0
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.15 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 4497379184654
Reset training subject: 4497381179417
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=0.02123649249407037}, derivative=-6.584203173979122E-4}
New Minimum: 0.02123649249407037 > 0.021236492494004524
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=0.021236492494004524}, derivative=-6.584203173916654E-4}, evalInputDelta = -6.584663370112764E-14
New Minimum: 0.021236492494004524 > 0.021236492493609475
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=0.021236492493609475}, derivative=-6.584203173541849E-4}, evalInputDelta = -4.608952108853259E-13
New Minimum: 0.021236492493609475 > 0.021236492490844108
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=0.021236492490844108}, derivative=-6.584203170918219E-4}, evalInputDelta = -3.2262630067503295E-12
New Minimum: 0.021236492490844108 > 0.02123649247148655
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=0.02123649247148655}, derivative=-6.584203152552815E-4}, evalInputDelta = -2.2583820230570595E-11
New Minimum: 0.02123649247148655 > 0.02123649233598365
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=0.02123649233598365}, derivative=-6.584203023994979E-4}, evalInputDelta = -1.5808672079731245E-10
New Minimum: 0.02123649233598365 > 0.02123649138746343
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=0.02123649138746343}, derivative=-6.584202124090285E-4}, evalInputDelta = -1.1066069414977786E-9
New Minimum: 0.02123649138746343 > 0.021236484747825496
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=0.021236484747825496}, derivative=-6.584195824765383E-4}, evalInputDelta = -7.746244874706765E-9
New Minimum: 0.021236484747825496 > 0.021236438270537852
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=0.021236438270537852}, derivative=-6.58415172988038E-4}, evalInputDelta = -5.42235325182161E-8
New Minimum: 0.021236438270537852 > 0.021236112938239383
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=0.021236112938239383}, derivative=-6.583843084760377E-4}, evalInputDelta = -3.7955583098714385E-7
New Minimum: 0.021236112938239383 > 0.02123383603906131
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=0.02123383603906131}, derivative=-6.581683503163796E-4}, evalInputDelta = -2.6564550090618644E-6
New Minimum: 0.02123383603906131 > 0.02121791862058547
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=0.02121791862058547}, derivative=-6.56661206203975E-4}, evalInputDelta = -1.857387348489989E-5
New Minimum: 0.02121791862058547 > 0.021107505142633973
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=0.021107505142633973}, derivative=-6.463298276219413E-4}, evalInputDelta = -1.289873514363979E-4
New Minimum: 0.021107505142633973 > 0.020379588263823637
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=0.020379588263823637}, derivative=-5.832602305504279E-4}, evalInputDelta = -8.569042302467335E-4
New Minimum: 0.020379588263823637 > 0.01663578731265043
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=0.01663578731265043}, derivative=-3.5918253456336167E-4}, evalInputDelta = -0.004600705181419942
New Minimum: 0.01663578731265043 > 0.005718364056232659
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=0.005718364056232659}, derivative=-1.1568430516251903E-4}, evalInputDelta = -0.015518128437837711
New Minimum: 0.005718364056232659 > 0.0
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.0013765623265657178}, evalInputDelta = -0.02123649249407037
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.3612599370689316}, evalInputDelta = -0.02123649249407037
F(23263.0513987207) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-9711.443247683186}, evalInputDelta = -0.02123649249407037
F(162841.3597910449) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.3902792593160367E8}, evalInputDelta = -0.02123649249407037
F(1139889.5185373144) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.280863078708271E12}, evalInputDelta = -0.02123649249407037
F(7979226.6297612) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3.8201319479824352E16}, evalInputDelta = -0.02123649249407037
F(5.58545864083284E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.417301591693117E20}, evalInputDelta = -0.02123649249407037
F(3.909821048582988E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.078479195567338E25}, evalInputDelta = -0.02123649249407037
F(2.7368747340080914E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.812581572727983E29}, evalInputDelta = -0.02123649249407037
F(1.915812313805664E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3.0464014287506713E33}, evalInputDelta = -0.02123649249407037
0.0 <= 0.02123649249407037
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=0.021236492494004524}, derivative=-6.584203173916654E-4}, evalInputDelta = -6.584663370112764E-14
Left bracket at 1.0E-10
Converged to left
Fitness changed from 0.02123649249407037 to 0.0
Iteration 1 complete. Error: 0.0 Total: 0.0883; Orientation: 0.0009; Line Search: 0.0817
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016525634033109776}
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016525634033222398}, evalInputDelta = 0.0
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.01652563403389815}, evalInputDelta = 0.0
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016525634038628376}, evalInputDelta = 0.0
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016525634071739972}, evalInputDelta = 0.0
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.01652563430352117}, evalInputDelta = 0.0
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016525635925989622}, evalInputDelta = 0.0
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016525647283271855}, evalInputDelta = 0.0
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016525726784400344}, evalInputDelta = 0.0
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016526283299790143}, evalInputDelta = 0.0
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016530179274560346}, evalInputDelta = 0.0
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016557469087972093}, evalInputDelta = 0.0
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016749381003263078}, evalInputDelta = 0.0
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.01813663204574333}, evalInputDelta = 0.0
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.030206764205886152}, evalInputDelta = 0.0
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.3218508021280375}, evalInputDelta = 0.0
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-114.41685889196054}, evalInputDelta = 0.0
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-606209.8152583964}, evalInputDelta = 0.0
F(23263.0513987207) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.244875463895416E9}, evalInputDelta = 0.0
F(162841.3597910449) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.3424500172650675E14}, evalInputDelta = 0.0
F(1139889.5185373144) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2459807266857449E18}, evalInputDelta = 0.0
F(7979226.6297612) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3.772356641493618E22}, evalInputDelta = 0.0
F(5.58545864083284E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-6.33960853375335E26}, evalInputDelta = 0.0
F(3.909821048582988E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0654838102431555E31}, evalInputDelta = 0.0
F(2.7368747340080914E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.7907552314257127E35}, evalInputDelta = 0.0
F(1.915812313805664E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3.009721499088896E39}, evalInputDelta = 0.0
0.0 <= 0.0
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.016525634033222398}, evalInputDelta = 0.0
Left bracket at 1.0E-10
Converged to left
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0611; Orientation: 0.0005; Line Search: 0.0594
Iteration 2 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 2
Final threshold in iteration 2: 0.0 (> 0.0) after 0.150s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 0.35 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 4497533860400
Reset training subject: 4497535074351
Adding measurement 780d3a5a to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 0.02123649249407037 < 0.02123649249407037. Total: 1
th(0)=0.02123649249407037;dx=-6.58420317397912E-4
Adding measurement 6ff896eb to history. Total: 1
New Minimum: 0.02123649249407037 > 0.019943683481494046
END: th(2.154434690031884)=0.019943683481494046; dx=-5.49290382563083E-4 evalInputDelta=0.0012928090125763243
Fitness changed from 0.02123649249407037 to 0.019943683481494046
Iteration 1 complete. Error: 0.019943683481494046 Total: 0.0146; Orientation: 0.0038; Line Search: 0.0068
Non-optimal measurement 0.019943683481494046 < 0.019943683481494046. Total: 2
LBFGS Accumulation History: 2 points
Non-optimal measurement 0.019943683481494046 < 0.019943683481494046. Total: 2
th(0)=0.019943683481494046;dx=-4.605967442067415E-4
Adding measurement 1fafdfe0 to history. Total: 2
New Minimum: 0.019943683481494046 > 0.018051237387870053
END: th(4.641588833612779)=0.018051237387870053; dx=-3.6349481391335264E-4 evalInputDelta=0.001892446093623993
Fitness changed from 0.019943683481494046 to 0.018051237387870053
Iteration 2 complete. Error: 0.018051237387870053 Total: 0.0112; Orientation: 0.0019; Line Search: 0.0077
Non-optimal measurement 0.018051237387870053 < 0.018051237387870053. Total: 3
LBFGS Accumulation History: 3 points
Non-optimal measurement 0.018051237387870053 < 0.018051237387870053. Total: 3
th(0)=0.018051237387870053;dx=-2.8911262486516905E-4
Adding measurement 603b72a2 to history. Total: 3
New Minimum: 0.018051237387870053 > 0.015544252973654474
END: th(10.000000000000002)=0.015544252973654474; dx=-2.1946140489624485E-4 evalInputDelta=0.002506984414215579
Fitness changed from 0.018051237387870053 to 0.015544252973654474
Iteration 3 complete. Error: 0.015544252973654474 Total: 0.0100; Orientation: 0.0020; Line Search: 0.0068
Non-optimal measurement 0.015544252973654474 < 0.015544252973654474. Total: 4
Rejected: LBFGS Orientation magnitude: 2.227e-01, gradient 1.297e-02, dot -0.976; [6e16b614-fd29-4cfe-a6b9-db26edd769fd = 1.000/1.000e+00, ee447b9e-4742-4b55-a3de-c10496d8dc99 = 1.000/1.000e+00, 4f3828ea-c279-4a7e-a670-4d6733283fdf = 1.000/1.000e+00, 3d7a50ae-4180-4f58-bd8a-aa9dbeb79566 = 1.000/1.000e+00, 324bb4b8-ef2b-49df-95f8-e7d0e4261df5 = 1.000/1.000e+00, ddff5cd4-ba8b-4f36-8a66-65922c5fa326 = 1.000/1.000e+00, cc8f34dc-2a68-4489-8d49-a05433841970 = 1.000/1.000e+00, 7c66e5fd-eb23-4dc5-906b-c9cb4e9b1e93 = 1.000/1.000e+00, 1b71ae15-3b7f-4129-930e-958143ccfcbc = 1.000/1.000e+00, aec9de9e-e9e1-4345-aa23-a86e6357e6d0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.015544252973654474, 0.018051237387870053, 0.019943683481494046, 0.02123649249407037
LBFGS Accumulation History: 3 points
Removed measurement 603b72a2 to history. Total: 3
Adding measurement 3a89e473 to history. Total: 3
th(0)=0.015544252973654474;dx=-1.683087468988806E-4
Adding measurement 2eff787 to history. Total: 4
New Minimum: 0.015544252973654474 > 0.012430596523208862
END: th(21.544346900318843)=0.012430596523208862; dx=-1.2553217683609132E-4 evalInputDelta=0.0031136564504456125
Fitness changed from 0.015544252973654474 to 0.012430596523208862
Iteration 4 complete. Error: 0.012430596523208862 Total: 0.0362; Orientation: 0.0287; Line Search: 0.0061
Non-optimal measurement 0.012430596523208862 < 0.012430596523208862. Total: 5
Rejected: LBFGS Orientation magnitude: 1.786e-01, gradient 9.745e-03, dot -0.977; [ee447b9e-4742-4b55-a3de-c10496d8dc99 = 1.000/1.000e+00, 4f3828ea-c279-4a7e-a670-4d6733283fdf = 1.000/1.000e+00, aec9de9e-e9e1-4345-aa23-a86e6357e6d0 = 1.000/1.000e+00, 3d7a50ae-4180-4f58-bd8a-aa9dbeb79566 = 1.000/1.000e+00, 1b71ae15-3b7f-4129-930e-958143ccfcbc = 1.000/1.000e+00, 7c66e5fd-eb23-4dc5-906b-c9cb4e9b1e93 = 1.000/1.000e+00, 324bb4b8-ef2b-49df-95f8-e7d0e4261df5 = 1.000/1.000e+00, 6e16b614-fd29-4cfe-a6b9-db26edd769fd = 1.000/1.000e+00, ddff5cd4-ba8b-4f36-8a66-65922c5fa326 = 1.000/1.000e+00, cc8f34dc-2a68-4489-8d49-a05433841970 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.012430596523208862, 0.015544252973654474, 0.018051237387870053, 0.019943683481494046, 0.02123649249407037
Rejected: LBFGS Orientation magnitude: 2.837e-01, gradient 9.745e-03, dot -0.978; [ddff5cd4-ba8b-4f36-8a66-65922c5fa326 = 1.000/1.000e+00, 1b71ae15-3b7f-4129-930e-958143ccfcbc = 1.000/1.000e+00, 4f3828ea-c279-4a7e-a670-4d6733283fdf = 1.000/1.000e+00, 6e16b614-fd29-4cfe-a6b9-db26edd769fd = 1.000/1.000e+00, 7c66e5fd-eb23-4dc5-906b-c9cb4e9b1e93 = 1.000/1.000e+00, 324bb4b8-ef2b-49df-95f8-e7d0e4261df5 = 1.000/1.000e+00, cc8f34dc-2a68-4489-8d49-a05433841970 = 1.000/1.000e+00, aec9de9e-e9e1-4345-aa23-a86e6357e6d0 = 1.000/1.000e+00, ee447b9e-4742-4b55-a3de-c10496d8dc99 = 1.000/1.000e+00, 3d7a50ae-4180-4f58-bd8a-aa9dbeb79566 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.012430596523208862, 0.015544252973654474, 0.018051237387870053, 0.019943683481494046
LBFGS Accumulation History: 3 points
Removed measurement 2eff787 to history. Total: 4
Removed measurement 3a89e473 to history. Total: 3
Adding measurement 54de222 to history. Total: 3
th(0)=0.012430596523208862;dx=-9.496432580034236E-5
Adding measurement 453de49f to history. Total: 4
New Minimum: 0.012430596523208862 > 0.008644351843561585
END: th(46.4158883361278)=0.008644351843561585; dx=-7.093368823483336E-5 evalInputDelta=0.003786244679647277
Fitness changed from 0.012430596523208862 to 0.008644351843561585
Iteration 5 complete. Error: 0.008644351843561585 Total: 0.0645; Orientation: 0.0585; Line Search: 0.0045
Non-optimal measurement 0.008644351843561585 < 0.008644351843561585. Total: 5
Rejected: LBFGS Orientation magnitude: 1.609e-01, gradient 7.340e-03, dot -0.964; [ddff5cd4-ba8b-4f36-8a66-65922c5fa326 = 1.000/1.000e+00, 7c66e5fd-eb23-4dc5-906b-c9cb4e9b1e93 = 1.000/1.000e+00, ee447b9e-4742-4b55-a3de-c10496d8dc99 = 1.000/1.000e+00, aec9de9e-e9e1-4345-aa23-a86e6357e6d0 = 1.000/1.000e+00, 6e16b614-fd29-4cfe-a6b9-db26edd769fd = 1.000/1.000e+00, 4f3828ea-c279-4a7e-a670-4d6733283fdf = 1.000/1.000e+00, 3d7a50ae-4180-4f58-bd8a-aa9dbeb79566 = 1.000/1.000e+00, cc8f34dc-2a68-4489-8d49-a05433841970 = 1.000/1.000e+00, 324bb4b8-ef2b-49df-95f8-e7d0e4261df5 = 1.000/1.000e+00, 1b71ae15-3b7f-4129-930e-958143ccfcbc = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.008644351843561585, 0.012430596523208862, 0.018051237387870053, 0.019943683481494046, 0.02123649249407037
Rejected: LBFGS Orientation magnitude: 2.432e-01, gradient 7.340e-03, dot -0.971; [6e16b614-fd29-4cfe-a6b9-db26edd769fd = 1.000/1.000e+00, 1b71ae15-3b7f-4129-930e-958143ccfcbc = 1.000/1.000e+00, aec9de9e-e9e1-4345-aa23-a86e6357e6d0 = 1.000/1.000e+00, cc8f34dc-2a68-4489-8d49-a05433841970 = 1.000/1.000e+00, 7c66e5fd-eb23-4dc5-906b-c9cb4e9b1e93 = 1.000/1.000e+00, 4f3828ea-c279-4a7e-a670-4d6733283fdf = 1.000/1.000e+00, ee447b9e-4742-4b55-a3de-c10496d8dc99 = 1.000/1.000e+00, 3d7a50ae-4180-4f58-bd8a-aa9dbeb79566 = 1.000/1.000e+00, ddff5cd4-ba8b-4f36-8a66-65922c5fa326 = 1.000/1.000e+00, 324bb4b8-ef2b-49df-95f8-e7d0e4261df5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.008644351843561585, 0.012430596523208862, 0.018051237387870053, 0.019943683481494046
LBFGS Accumulation History: 3 points
Removed measurement 453de49f to history. Total: 4
Removed measurement 54de222 to history. Total: 3
Adding measurement 47a00060 to history. Total: 3
th(0)=0.008644351843561585;dx=-5.386919017645516E-5
Adding measurement 10c5a998 to history. Total: 4
New Minimum: 0.008644351843561585 > 0.0040081347419547815
END: th(100.00000000000003)=0.0040081347419547815; dx=-4.035626069728204E-5 evalInputDelta=0.0046362171016068035
Fitness changed from 0.008644351843561585 to 0.0040081347419547815
Iteration 6 complete. Error: 0.0040081347419547815 Total: 0.0508; Orientation: 0.0435; Line Search: 0.0065
Non-optimal measurement 0.0040081347419547815 < 0.0040081347419547815. Total: 5
Rejected: LBFGS Orientation magnitude: 1.922e-01, gradient 5.536e-03, dot -0.918; [1b71ae15-3b7f-4129-930e-958143ccfcbc = 1.000/1.000e+00, cc8f34dc-2a68-4489-8d49-a05433841970 = 1.000/1.000e+00, 6e16b614-fd29-4cfe-a6b9-db26edd769fd = 1.000/1.000e+00, 3d7a50ae-4180-4f58-bd8a-aa9dbeb79566 = 1.000/1.000e+00, 7c66e5fd-eb23-4dc5-906b-c9cb4e9b1e93 = 1.000/1.000e+00, 4f3828ea-c279-4a7e-a670-4d6733283fdf = 1.000/1.000e+00, aec9de9e-e9e1-4345-aa23-a86e6357e6d0 = 1.000/1.000e+00, ddff5cd4-ba8b-4f36-8a66-65922c5fa326 = 1.000/1.000e+00, 324bb4b8-ef2b-49df-95f8-e7d0e4261df5 = 1.000/1.000e+00, ee447b9e-4742-4b55-a3de-c10496d8dc99 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0040081347419547815, 0.008644351843561585, 0.018051237387870053, 0.019943683481494046, 0.02123649249407037
Rejected: LBFGS Orientation magnitude: 2.533e-01, gradient 5.536e-03, dot -0.939; [cc8f34dc-2a68-4489-8d49-a05433841970 = 1.000/1.000e+00, 3d7a50ae-4180-4f58-bd8a-aa9dbeb79566 = 1.000/1.000e+00, ddff5cd4-ba8b-4f36-8a66-65922c5fa326 = 1.000/1.000e+00, ee447b9e-4742-4b55-a3de-c10496d8dc99 = 1.000/1.000e+00, 7c66e5fd-eb23-4dc5-906b-c9cb4e9b1e93 = 1.000/1.000e+00, aec9de9e-e9e1-4345-aa23-a86e6357e6d0 = 1.000/1.000e+00, 6e16b614-fd29-4cfe-a6b9-db26edd769fd = 1.000/1.000e+00, 324bb4b8-ef2b-49df-95f8-e7d0e4261df5 = 1.000/1.000e+00, 1b71ae15-3b7f-4129-930e-958143ccfcbc = 1.000/1.000e+00, 4f3828ea-c279-4a7e-a670-4d6733283fdf = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0040081347419547815, 0.008644351843561585, 0.018051237387870053, 0.019943683481494046
LBFGS Accumulation History: 3 points
Removed measurement 10c5a998 to history. Total: 4
Removed measurement 47a00060 to history. Total: 3
Adding measurement 5bc184c3 to history. Total: 3
th(0)=0.0040081347419547815;dx=-3.0648125475614005E-5
Adding measurement 77b03198 to history. Total: 4
New Minimum: 0.0040081347419547815 > 0.0
WOLFE (weak): th(215.44346900318845)=0.0; dx=-3.4890776621990966E-5 evalInputDelta=0.0040081347419547815
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(430.8869380063769)=0.0; dx=-8.75882300571496E-5 evalInputDelta=0.0040081347419547815
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(1292.6608140191306)=0.0; dx=-0.0010406873757013333 evalInputDelta=0.0040081347419547815
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(5170.643256076522)=0.0; dx=-0.13828239943284742 evalInputDelta=0.0040081347419547815
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(25853.216280382614)=0.0; dx=-190.27651601685886 evalInputDelta=0.0040081347419547815
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(155119.29768229567)=0.0; dx=-1206870.2064435042 evalInputDelta=0.0040081347419547815
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(1085835.0837760698)=0.0; dx=-1.9556543036010754E10 evalInputDelta=0.0040081347419547815
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(8686680.670208558)=0.0; dx=-6.373983747966721E14 evalInputDelta=0.0040081347419547815
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(7.818012603187703E7)=0.0; dx=-3.7612062652355576E19 evalInputDelta=0.0040081347419547815
Non-optimal measurement 0.0 < 0.0. Total: 5
MAX ALPHA (7.818012603187703E8): th(215.44346900318845)=0.0
Fitness changed from 0.0040081347419547815 to 0.0
Iteration 7 complete. Error: 0.0 Total: 0.0864; Orientation: 0.0563; Line Search: 0.0289
Non-optimal measurement 0.0 < 0.0. Total: 5
Accepted: LBFGS Orientation magnitude: 3.564e-01, gradient 6.329e-03, dot 0.568; [aec9de9e-e9e1-4345-aa23-a86e6357e6d0 = 1.000/1.000e+00, ddff5cd4-ba8b-4f36-8a66-65922c5fa326 = 1.000/1.000e+00, 4f3828ea-c279-4a7e-a670-4d6733283fdf = 1.000/1.000e+00, 324bb4b8-ef2b-49df-95f8-e7d0e4261df5 = 1.000/1.000e+00, 7c66e5fd-eb23-4dc5-906b-c9cb4e9b1e93 = 1.000/1.000e+00, 3d7a50ae-4180-4f58-bd8a-aa9dbeb79566 = 1.000/1.000e+00, 6e16b614-fd29-4cfe-a6b9-db26edd769fd = 1.000/1.000e+00, ee447b9e-4742-4b55-a3de-c10496d8dc99 = 1.000/1.000e+00, 1b71ae15-3b7f-4129-930e-958143ccfcbc = 1.000/1.000e+00, cc8f34dc-2a68-4489-8d49-a05433841970 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS
Non-optimal measurement 0.0 < 0.0. Total: 5
th(0)=0.0;dx=-0.001281323312659907
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(2.154434690031884)=0.0; dx=-0.0015685934172414819 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(1.077217345015942)=0.0; dx=-0.001431869600781579 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(0.3590724483386473)=0.0; dx=-0.0013327457014786484 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(0.08976811208466183)=0.0; dx=-0.0012942798426042548 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(0.017953622416932366)=0.0; dx=-0.0012839197184514978 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(0.002992270402822061)=0.0; dx=-0.0012817562215318376 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(4.2746720040315154E-4)=0.0; dx=-0.0012813851610473518 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(5.343340005039394E-5)=0.0; dx=-0.0012813310437860086 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(5.9370444500437714E-6)=0.0; dx=-0.0012813241716750142 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(5.937044450043771E-7)=0.0; dx=-0.00128132339856143 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(5.397313136403428E-8)=0.0; dx=-0.0012813233204691365 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(4.4977609470028565E-9)=0.0; dx=-0.001281323313310676 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(3.4598161130791205E-10)=0.0; dx=-0.001281323312709966 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(2.4712972236279432E-11)=0.0; dx=-0.0012813233126634828 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(1.6475314824186289E-12)=0.0; dx=-0.0012813233126601454 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(1.029707176511643E-13)=0.0; dx=-0.0012813233126599218 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(6.057101038303783E-15)=0.0; dx=-0.0012813233126599077 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 5
MIN ALPHA (3.3650561323909904E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0718; Orientation: 0.0303; Line Search: 0.0402
Iteration 8 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 8
Final threshold in iteration 8: 0.0 (> 0.0) after 0.346s (< 30.000s)
Returns
0.0
TrainingTester.java:432 executed in 0.09 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -2.3970576871648044], [7.0, -1.700194626969754]; valueStats=DoubleSummaryStatistics{count=12, sum=0.157245, min=0.004008, average=0.013104, max=0.019944}
Plotting 7 points for GD
Only 1 points for CjGD
Plotting 7 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -2.3970576871648044], [0.259, -1.700194626969754]; valueStats=DoubleSummaryStatistics{count=12, sum=0.157245, min=0.004008, average=0.013104, max=0.019944}
Plotting 7 points for GD
Only 1 points for CjGD
Plotting 7 points for LBFGS
Returns
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null} | OK |
{
"result": "OK",
"performance": {
"execution_time": "1.465",
"gc_time": "0.241"
},
"created_on": 1586739084870,
"file_name": "trainingTest",
"report": {
"simpleName": "Basic",
"canonicalName": "com.simiacryptus.mindseye.layers.java.ProductLayerTest.Basic",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/ProductLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "Converged",
"value": 0.0
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "Converged",
"value": 0.0
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/ProductLayer/Basic/trainingTest/202004135124",
"id": "a2f45d00-5250-4e51-8244-19142a431013",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "ProductLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.java.ProductLayer",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/ProductLayer.java",
"javaDoc": ""
}
}