Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[ -0.888, 0.82, 1.512, -1.72, -0.408, -1.688, 0.7, 0.996, ... ]
[ -0.356, 0.448, 0.82, -1.176, -1.524, -1.176, -1.16, -1.096, ... ]
[ -1.616, -1.724, -1.228, -0.804, -1.476, 1.156, 1.324, -1.688, ... ]
[ 1.288, -1.808, -1.176, 0.52, 0.632, -0.212, -1.384, 0.012, ... ]
[ -1.664, 0.496, -1.72, -0.628, 0.788, 1.208, -0.852, -1.616, ... ]
[ 1.82, -0.584, 1.776, -0.892, -1.176, 1.616, 0.012, 0.356, ... ]
[ -0.888, 1.156, -0.892, 1.032, -1.476, -1.616, 1.876, -0.068, ... ]
[ -0.344, 1.18, 1.66, -0.584, 0.688, 1.132, 1.444, 1.256, ... ]
[ 0.996, -1.116, 1.512, 1.556, 0.392, 0.788, -1.492, 1.356, ... ]
[ 1.972, 0.672, -0.968, 0.66, -0.684, -1.848, 0.688, -1.976, ... ]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 3.31 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1778966165740
BACKPROP_AGG_SIZE = 3
THREADS = 64
SINGLE_THREADED = false
Initialized CoreSettings = {
"backpropAggregationSize" : 3,
"jvmThreads" : 64,
"singleThreaded" : false
}
Reset training subject: 1779007001057
Constructing line search parameters: GD
th(0)=987.39875395872;dx=-2.880352256E25
New Minimum: 987.39875395872 > 0.025788592006016308
Armijo: th(2.154434690031884)=0.025788592006016308; dx=-2.880352256008968E13 evalInputDelta=987.372965366714
Armijo: th(1.077217345015942)=0.13096384010158055; dx=-2.8803522560089723E13 evalInputDelta=987.2677901186184
Armijo: th(0.3590724483386473)=0.7944847324799003; dx=-2.880352256009035E13 evalInputDelta=986.6042692262401
Armijo: th(0.08976811208466183)=5.250281206485715; dx=-2.880352256010408E13 evalInputDelta=982.1484727522343
Armijo: th(0.017953622416932366)=11.707419670445706; dx=-2.8803522560196574E13 evalInputDelta=975.6913342882743
Armijo: th(0.002992270402822061)=15.595976578304775; dx=-2.8803522560457312E13 evalInputDelta=971.8027773804153
Armijo: th(4.2746720040315154E-4)=17.158694451014707; dx=-2.8803522561032387E13 evalInputDelta=970.2400595077053
Armijo: th(5.343340005039394E-5)=17.645374823621076; dx=-2.8803522561807953E13 evalInputDelta=969.753379135099
Armijo: th(5.9370444500437714E-6)=17.736493677698366; dx=-2.880352256207785E13 evalInputDelta=969.6622602810216
Armijo: th(5.937044450043771E-7)=17.74760870493003; dx=-2.8803522562115195E13 evalInputDelta=969.65114525379
Armijo: th(5.397313136403428E-8)=17.74874263380493; dx=-2.8803522562119062E13 evalInputDelta=969.6500113249151
Armijo: th(4.4977609470028565E-9)=17.74884668187466; dx=-2.880352256211942E13 evalInputDelta=969.6499072768454
Armijo: th(3.4598161130791205E-10)=17.818436752622464; dx=-6.40002880353587E18 evalInputDelta=969.5803172060976
Armijo: th(2.4712972236279432E-11)=21.7602991985806; dx=-7.38176002879664E21 evalInputDelta=965.6384547601394
Armijo: th(1.6475314824186289E-12)=519.5232184005829; dx=-1.2920647680034479E25 evalInputDelta=467.87553555813713
Armijo: th(1.029707176511643E-13)=941.5085688693982; dx=-2.6959005440045927E25 evalInputDelta=45.89018508932179
Armijo: th(6.057101038303783E-15)=987.3987539587072; dx=-2.880352256E25 evalInputDelta=1.2846612662542611E-11
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=0.025788592006016308
Fitness changed from 987.39875395872 to 0.025788592006016308
Iteration 1 complete. Error: 0.025788592006016308 Total: 1.0587; Orientation: 0.0059; Line Search: 0.9950
th(0)=0.025788592006016308;dx=-59.06312759143887
New Minimum: 0.025788592006016308 > 0.025788592006016277
WOLFE (weak): th(2.154434690031884E-15)=0.025788592006016277; dx=-59.06312759143887 evalInputDelta=3.122502256758253E-17
New Minimum: 0.025788592006016277 > 0.025788592006016225
WOLFE (weak): th(4.308869380063768E-15)=0.025788592006016225; dx=-59.06312759143887 evalInputDelta=8.326672684688674E-17
New Minimum: 0.025788592006016225 > 0.025788592006016086
WOLFE (weak): th(1.2926608140191303E-14)=0.025788592006016086; dx=-59.06312759143887 evalInputDelta=2.220446049250313E-16
New Minimum: 0.025788592006016086 > 0.02578859200601543
WOLFE (weak): th(5.1706432560765214E-14)=0.02578859200601543; dx=-59.06312759143887 evalInputDelta=8.777700788442644E-16
New Minimum: 0.02578859200601543 > 0.02578859200601189
WOLFE (weak): th(2.5853216280382605E-13)=0.02578859200601189; dx=-59.06312759143887 evalInputDelta=4.416605969836951E-15
New Minimum: 0.02578859200601189 > 0.025788592005989774
WOLFE (weak): th(1.5511929768229563E-12)=0.025788592005989774; dx=-59.06312759143887 evalInputDelta=2.653433028854124E-14
New Minimum: 0.025788592005989774 > 0.025788592005830707
WOLFE (weak): th(1.0858350837760695E-11)=0.025788592005830707; dx=-59.06312759143886 evalInputDelta=1.8560153414171054E-13
New Minimum: 0.025788592005830707 > 0.02578859200453156
WOLFE (weak): th(8.686680670208556E-11)=0.02578859200453156; dx=-59.06312759143873 evalInputDelta=1.4847463536415972E-12
New Minimum: 0.02578859200453156 > 0.025788591992653466
WOLFE (weak): th(7.8180126031877E-10)=0.025788591992653466; dx=-59.06312759143756 evalInputDelta=1.3362842082864645E-11
New Minimum: 0.025788591992653466 > 0.025788591872387783
WOLFE (weak): th(7.818012603187701E-9)=0.025788591872387783; dx=-59.06312759142582 evalInputDelta=1.3362852491205501E-10
New Minimum: 0.025788591872387783 > 0.02578859053610246
WOLFE (weak): th(8.599813863506471E-8)=0.02578859053610246; dx=-59.06312759129528 evalInputDelta=1.4699138468909911E-9
New Minimum: 0.02578859053610246 > 0.02578857436705101
WOLFE (weak): th(1.0319776636207765E-6)=0.02578857436705101; dx=-59.06312758971575 evalInputDelta=1.7638965298799603E-8
New Minimum: 0.02578857436705101 > 0.025788362699606195
WOLFE (weak): th(1.3415709627070094E-5)=0.025788362699606195; dx=-59.06312756903824 evalInputDelta=2.2930641011345565E-7
New Minimum: 0.025788362699606195 > 0.025785381743621723
WOLFE (weak): th(1.878199347789813E-4)=0.025785381743621723; dx=-59.06312727783552 evalInputDelta=3.210262394585245E-6
New Minimum: 0.025785381743621723 > 0.025744615401709452
WOLFE (weak): th(0.0028172990216847197)=0.025744615401709452; dx=-59.06312318896691 evalInputDelta=4.397660430685568E-5
New Minimum: 0.025744615401709452 > 0.02513335149669619
WOLFE (weak): th(0.045076784346955515)=0.02513335149669619; dx=-59.063060828311116 evalInputDelta=6.552405093201187E-4
New Minimum: 0.02513335149669619 > 0.018472181106021668
WOLFE (weak): th(0.7663053338982437)=0.018472181106021668; dx=-59.062446152668635 evalInputDelta=0.00731641089999464
New Minimum: 0.018472181106021668 > 0.0
WOLFE (weak): th(13.793496010168386)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
WOLFE (weak): th(262.07642419319933)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
Armijo: th(5241.528483863986)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
Armijo: th(2751.802454028593)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
Armijo: th(1506.939439110896)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
Armijo: th(884.5079316520477)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
Armijo: th(573.2921779226235)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
WOLFE (weak): th(417.6843010579114)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
Armijo: th(495.48823949026746)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
Armijo: th(456.58627027408943)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
Armijo: th(437.1352856660004)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
WOLFE (weak): th(427.4097933619559)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
WOLFE (weak): th(432.27253951397813)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
WOLFE (weak): th(434.7039125899893)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
mu ~= nu (434.7039125899893): th(13.793496010168386)=0.0
Fitness changed from 0.025788592006016308 to 0.0
Iteration 2 complete. Error: 0.0 Total: 1.3561; Orientation: 0.0028; Line Search: 1.3487
th(0)=0.0;dx=-59.05973120000001
Armijo: th(939.1603064261446)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(469.5801532130723)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(156.52671773769077)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(39.13167943442269)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(7.8263358868845385)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(1.3043893144807563)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(0.18634133064010805)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(0.023292666330013506)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(0.0025880740366681673)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(2.588074036668167E-4)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(2.3527945787892428E-5)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(1.960662148991036E-6)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(1.5082016530700276E-7)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(1.0772868950500198E-8)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(7.181912633666799E-10)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(4.4886953960417494E-11)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(2.640409056495147E-12)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(1.4668939202750816E-13)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Armijo: th(7.720494317237272E-15)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
MIN ALPHA (3.860247158618636E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.8906; Orientation: 0.0026; Line Search: 0.8843
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 3.307s (< 30.000s)
Returns
0.0
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 1.87 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1782278946663
Reset training subject: 1782282533021
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=987.39875395872}, derivative=-2.880352256E25}
New Minimum: 987.39875395872 > 17.919885853939512
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=17.919885853939512}, derivative=-1.92000288040006E19}, evalInputDelta = -969.4788681047805
New Minimum: 17.919885853939512 > 17.752949602228547
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=17.752949602228547}, derivative=-2.8803681713349395E13}, evalInputDelta = -969.6458043564915
New Minimum: 17.752949602228547 > 17.748845835883877
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=17.748845835883877}, derivative=-2.8803522562119414E13}, evalInputDelta = -969.6499081228361
New Minimum: 17.748845835883877 > 17.74878400483626
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=17.74878400483626}, derivative=-2.8803522562119203E13}, evalInputDelta = -969.6499699538838
New Minimum: 17.74878400483626 > 17.748351361357614
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=17.748351361357614}, derivative=-2.8803522562117727E13}, evalInputDelta = -969.6504025973624
New Minimum: 17.748351361357614 > 17.745331334431803
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=17.745331334431803}, derivative=-2.8803522562107457E13}, evalInputDelta = -969.6534226242882
New Minimum: 17.745331334431803 > 17.724592856350945
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=17.724592856350945}, derivative=-2.880352256203901E13}, evalInputDelta = -969.6741611023691
New Minimum: 17.724592856350945 > 17.59541032716815
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=17.59541032716815}, derivative=-2.880352256168372E13}, evalInputDelta = -969.8033436315519
New Minimum: 17.59541032716815 > 17.016392925617584
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=17.016392925617584}, derivative=-2.8803522560917137E13}, evalInputDelta = -970.3823610331025
New Minimum: 17.016392925617584 > 15.16736639699165
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=15.16736639699165}, derivative=-2.8803522560400785E13}, evalInputDelta = -972.2313875617284
New Minimum: 15.16736639699165 > 10.223568226113768
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=10.223568226113768}, derivative=-2.8803522560161426E13}, evalInputDelta = -977.1751857326062
New Minimum: 10.223568226113768 > 2.213619542839124
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=2.213619542839124}, derivative=-2.880352256009346E13}, evalInputDelta = -985.185134415881
New Minimum: 2.213619542839124 > 0.07304120187977245
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=0.07304120187977245}, derivative=-2.8803522560089695E13}, evalInputDelta = -987.3257127568403
New Minimum: 0.07304120187977245 > 0.0010036131247753623
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=0.0010036131247753623}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.3977503455952
New Minimum: 0.0010036131247753623 > 0.0
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
F(23263.0513987207) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
F(162841.3597910449) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
F(1139889.5185373144) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
F(7979226.6297612) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
F(5.58545864083284E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
F(3.909821048582988E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
F(2.7368747340080914E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
F(1.915812313805664E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
0.0 <= 987.39875395872
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.8803522560089676E13}, evalInputDelta = -987.39875395872
Right bracket at 1.0E10
Converged to right
Fitness changed from 987.39875395872 to 0.0
Iteration 1 complete. Error: 0.0 Total: 0.9680; Orientation: 0.0022; Line Search: 0.9554
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
0.0 <= 0.0
F(5.0E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Right bracket at 5.0E9
F(2.5E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Right bracket at 2.5E9
F(1.25E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Right bracket at 1.25E9
F(6.25E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Right bracket at 6.25E8
F(3.125E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Right bracket at 3.125E8
F(1.5625E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Right bracket at 1.5625E8
F(7.8125E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Right bracket at 7.8125E7
F(3.90625E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Right bracket at 3.90625E7
F(1.953125E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Right bracket at 1.953125E7
F(9765625.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Right bracket at 9765625.0
F(4882812.5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Right bracket at 4882812.5
F(2441406.25) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-59.05973120000001}, evalInputDelta = 0.0
Loops = 12
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.9059; Orientation: 0.0075; Line Search: 0.8950
Iteration 2 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 2
Final threshold in iteration 2: 0.0 (> 0.0) after 1.874s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 33.66 seconds (0.077 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1784157851164
Reset training subject: 1784161019625
Adding measurement 3adaa0ec to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 987.39875395872 < 987.39875395872. Total: 1
th(0)=987.39875395872;dx=-2.880352256E25
Adding measurement 6e6cf680 to history. Total: 1
New Minimum: 987.39875395872 > 0.025788592006016308
Armijo: th(2.154434690031884)=0.025788592006016308; dx=-2.880352256008968E13 evalInputDelta=987.372965366714
Non-optimal measurement 0.13096384010158055 < 0.025788592006016308. Total: 2
Armijo: th(1.077217345015942)=0.13096384010158055; dx=-2.8803522560089723E13 evalInputDelta=987.2677901186184
Non-optimal measurement 0.7944847324799003 < 0.025788592006016308. Total: 2
Armijo: th(0.3590724483386473)=0.7944847324799003; dx=-2.880352256009035E13 evalInputDelta=986.6042692262401
Non-optimal measurement 5.250281206485715 < 0.025788592006016308. Total: 2
Armijo: th(0.08976811208466183)=5.250281206485715; dx=-2.880352256010408E13 evalInputDelta=982.1484727522343
Non-optimal measurement 11.707419670445706 < 0.025788592006016308. Total: 2
Armijo: th(0.017953622416932366)=11.707419670445706; dx=-2.8803522560196574E13 evalInputDelta=975.6913342882743
Non-optimal measurement 15.595976578304775 < 0.025788592006016308. Total: 2
Armijo: th(0.002992270402822061)=15.595976578304775; dx=-2.8803522560457312E13 evalInputDelta=971.8027773804153
Non-optimal measurement 17.158694451014707 < 0.025788592006016308. Total: 2
Armijo: th(4.2746720040315154E-4)=17.158694451014707; dx=-2.8803522561032387E13 evalInputDelta=970.2400595077053
Non-optimal measurement 17.645374823621076 < 0.025788592006016308. Total: 2
Armijo: th(5.343340005039394E-5)=17.645374823621076; dx=-2.8803522561807953E13 evalInputDelta=969.753379135099
Non-optimal measurement 17.736493677698366 < 0.025788592006016308. Total: 2
Armijo: th(5.9370444500437714E-6)=17.736493677698366; dx=-2.880352256207785E13 evalInputDelta=969.6622602810216
Non-optimal measurement 17.74760870493003 < 0.025788592006016308. Total: 2
Armijo: th(5.937044450043771E-7)=17.74760870493003; dx=-2.8803522562115195E13 evalInputDelta=969.65114525379
Non-optimal measurement 17.74874263380493 < 0.025788592006016308. Total: 2
Armijo: th(5.397313136403428E-8)=17.74874263380493; dx=-2.8803522562119062E13 evalInputDelta=969.6500113249151
Non-optimal measurement 17.74884668187466 < 0.025788592006016308. Total: 2
Armijo: th(4.4977609470028565E-9)=17.74884668187466; dx=-2.880352256211942E13 evalInputDelta=969.6499072768454
Non-optimal measurement 17.818436752622464 < 0.025788592006016308. Total: 2
Armijo: th(3.4598161130791205E-10)=17.818436752622464; dx=-6.400028803535871E18 evalInputDelta=969.5803172060976
Non-optimal measurement 21.7602991985806 < 0.025788592006016308. Total: 2
Armijo: th(2.4712972236279432E-11)=21.7602991985806; dx=-7.38176002879664E21 evalInputDelta=965.6384547601394
Non-optimal measurement 519.5232184005829 < 0.025788592006016308. Total: 2
Armijo: th(1.6475314824186289E-12)=519.5232184005829; dx=-1.2920647680034479E25 evalInputDelta=467.87553555813713
Non-optimal measurement 941.5085688693982 < 0.025788592006016308. Total: 2
Armijo: th(1.029707176511643E-13)=941.5085688693982; dx=-2.6959005440045927E25 evalInputDelta=45.89018508932179
Non-optimal measurement 987.3987539587072 < 0.025788592006016308. Total: 2
Armijo: th(6.057101038303783E-15)=987.3987539587072; dx=-2.880352256E25 evalInputDelta=1.2846612662542611E-11
Non-optimal measurement 0.025788592006016308 < 0.025788592006016308. Total: 2
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=0.025788592006016308
Fitness changed from 987.39875395872 to 0.025788592006016308
Iteration 1 complete. Error: 0.025788592006016308 Total: 0.6564; Orientation: 0.0091; Line Search: 0.6386
Non-optimal measurement 0.025788592006016308 < 0.025788592006016308. Total: 2
LBFGS Accumulation History: 2 points
Non-optimal measurement 0.025788592006016308 < 0.025788592006016308. Total: 2
th(0)=0.025788592006016308;dx=-59.06312759143887
Adding measurement 61aa2bc to history. Total: 2
New Minimum: 0.025788592006016308 > 0.025788592006016277
WOLFE (weak): th(2.154434690031884E-15)=0.025788592006016277; dx=-59.06312759143887 evalInputDelta=3.122502256758253E-17
Adding measurement 70fdb1d6 to history. Total: 3
New Minimum: 0.025788592006016277 > 0.025788592006016225
WOLFE (weak): th(4.308869380063768E-15)=0.025788592006016225; dx=-59.06312759143887 evalInputDelta=8.326672684688674E-17
Adding measurement 1dc10f53 to history. Total: 4
New Minimum: 0.025788592006016225 > 0.025788592006016086
WOLFE (weak): th(1.2926608140191303E-14)=0.025788592006016086; dx=-59.06312759143887 evalInputDelta=2.220446049250313E-16
Adding measurement 3b12514c to history. Total: 5
New Minimum: 0.025788592006016086 > 0.02578859200601543
WOLFE (weak): th(5.1706432560765214E-14)=0.02578859200601543; dx=-59.06312759143887 evalInputDelta=8.777700788442644E-16
Adding measurement 1319d3ec to history. Total: 6
New Minimum: 0.02578859200601543 > 0.02578859200601189
WOLFE (weak): th(2.5853216280382605E-13)=0.02578859200601189; dx=-59.06312759143887 evalInputDelta=4.416605969836951E-15
Adding measurement 55f61fc3 to history. Total: 7
New Minimum: 0.02578859200601189 > 0.025788592005989774
WOLFE (weak): th(1.5511929768229563E-12)=0.025788592005989774; dx=-59.06312759143887 evalInputDelta=2.653433028854124E-14
Adding measurement 20a2efd1 to history. Total: 8
New Minimum: 0.025788592005989774 > 0.025788592005830707
WOLFE (weak): th(1.0858350837760695E-11)=0.025788592005830707; dx=-59.06312759143886 evalInputDelta=1.8560153414171054E-13
Adding measurement 1e87e2f8 to history. Total: 9
New Minimum: 0.025788592005830707 > 0.02578859200453156
WOLFE (weak): th(8.686680670208556E-11)=0.02578859200453156; dx=-59.06312759143873 evalInputDelta=1.4847463536415972E-12
Adding measurement 3488fdd9 to history. Total: 10
New Minimum: 0.02578859200453156 > 0.025788591992653466
WOLFE (weak): th(7.8180126031877E-10)=0.025788591992653466; dx=-59.06312759143756 evalInputDelta=1.3362842082864645E-11
Adding measurement 39e7abd2 to history. Total: 11
New Minimum: 0.025788591992653466 > 0.025788591872387783
WOLFE (weak): th(7.818012603187701E-9)=0.025788591872387783; dx=-59.06312759142582 evalInputDelta=1.3362852491205501E-10
Adding measurement 4941b17d to history. Total: 12
New Minimum: 0.025788591872387783 > 0.02578859053610246
WOLFE (weak): th(8.599813863506471E-8)=0.02578859053610246; dx=-59.06312759129528 evalInputDelta=1.4699138468909911E-9
Adding measurement 16633826 to history. Total: 13
New Minimum: 0.02578859053610246 > 0.02578857436705101
WOLFE (weak): th(1.0319776636207765E-6)=0.02578857436705101; dx=-59.06312758971575 evalInputDelta=1.7638965298799603E-8
Adding measurement 76ef1fcc to history. Total: 14
New Minimum: 0.02578857436705101 > 0.025788362699606195
WOLFE (weak): th(1.3415709627070094E-5)=0.025788362699606195; dx=-59.06312756903824 evalInputDelta=2.2930641011345565E-7
Adding measurement 12237c14 to history. Total: 15
New Minimum: 0.025788362699606195 > 0.025785381743621723
WOLFE (weak): th(1.878199347789813E-4)=0.025785381743621723; dx=-59.06312727783552 evalInputDelta=3.210262394585245E-6
Adding measurement 2c426c0d to history. Total: 16
New Minimum: 0.025785381743621723 > 0.025744615401709452
WOLFE (weak): th(0.0028172990216847197)=0.025744615401709452; dx=-59.06312318896691 evalInputDelta=4.397660430685568E-5
Adding measurement 626e85ec to history. Total: 17
New Minimum: 0.025744615401709452 > 0.02513335149669619
WOLFE (weak): th(0.045076784346955515)=0.02513335149669619; dx=-59.063060828311116 evalInputDelta=6.552405093201187E-4
Adding measurement 1e0f9850 to history. Total: 18
New Minimum: 0.02513335149669619 > 0.018472181106021668
WOLFE (weak): th(0.7663053338982437)=0.018472181106021668; dx=-59.062446152668635 evalInputDelta=0.00731641089999464
Adding measurement c702c44 to history. Total: 19
New Minimum: 0.018472181106021668 > 0.0
WOLFE (weak): th(13.793496010168386)=0.0; dx=-59.061165625018326 evalInputDelta=0.025788592006016308
Non-optimal measurement 0.0 < 0.0. Total: 20
WOLFE (weak): th(262.07642419319933)=0
...skipping 13420 bytes...
59ca-474a-b9e0-80d606dc8b7b = 1.000/1.000e+00, 20d3a3dc-4976-482f-9daf-124716832023 = 1.000/1.000e+00, b7d5052b-cd0f-457c-bb8d-13ebfd5db7f6 = 1.000/1.000e+00, 267b9b50-5ec6-45d2-9278-3e103b9a97d0 = 1.000/1.000e+00, eccd2140-0fab-4313-aa0f-d52e8ae00a9f = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.018472181106021668, 0.02513335149669619, 0.025744615401709452, 0.025785381743621723, 0.025788362699606195, 0.02578857436705101, 0.02578859053610246, 0.025788591872387783
Rejected: LBFGS Orientation magnitude: 4.437e+05, gradient 7.685e+00, dot -0.970; [3f15e450-15ec-4427-9943-898be48bb79c = 1.000/1.000e+00, f14559c6-4a86-4e6e-b13e-18f57de5d66c = 1.000/1.000e+00, f7686364-2b28-4822-b114-fb75a1476c1b = 1.000/1.000e+00, b7d5052b-cd0f-457c-bb8d-13ebfd5db7f6 = 1.000/1.000e+00, 267b9b50-5ec6-45d2-9278-3e103b9a97d0 = 1.000/1.000e+00, 20d3a3dc-4976-482f-9daf-124716832023 = 1.000/1.000e+00, d8dd43d5-d6c7-473b-9b64-14eb32febbaf = 1.000/1.000e+00, d986c0bd-59ca-474a-b9e0-80d606dc8b7b = 1.000/1.000e+00, eccd2140-0fab-4313-aa0f-d52e8ae00a9f = 1.000/1.000e+00, 318ced06-10a6-4146-b760-8d4297f5ec28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.018472181106021668, 0.02513335149669619, 0.025744615401709452, 0.025785381743621723, 0.025788362699606195, 0.02578857436705101, 0.02578859053610246
Rejected: LBFGS Orientation magnitude: 4.437e+05, gradient 7.685e+00, dot -0.970; [d8dd43d5-d6c7-473b-9b64-14eb32febbaf = 1.000/1.000e+00, 3f15e450-15ec-4427-9943-898be48bb79c = 1.000/1.000e+00, eccd2140-0fab-4313-aa0f-d52e8ae00a9f = 1.000/1.000e+00, 318ced06-10a6-4146-b760-8d4297f5ec28 = 1.000/1.000e+00, 267b9b50-5ec6-45d2-9278-3e103b9a97d0 = 1.000/1.000e+00, f7686364-2b28-4822-b114-fb75a1476c1b = 1.000/1.000e+00, b7d5052b-cd0f-457c-bb8d-13ebfd5db7f6 = 1.000/1.000e+00, d986c0bd-59ca-474a-b9e0-80d606dc8b7b = 1.000/1.000e+00, f14559c6-4a86-4e6e-b13e-18f57de5d66c = 1.000/1.000e+00, 20d3a3dc-4976-482f-9daf-124716832023 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.018472181106021668, 0.02513335149669619, 0.025744615401709452, 0.025785381743621723, 0.025788362699606195, 0.02578857436705101
Rejected: LBFGS Orientation magnitude: 6.181e+05, gradient 7.685e+00, dot -0.951; [b7d5052b-cd0f-457c-bb8d-13ebfd5db7f6 = 1.000/1.000e+00, d8dd43d5-d6c7-473b-9b64-14eb32febbaf = 1.000/1.000e+00, f7686364-2b28-4822-b114-fb75a1476c1b = 1.000/1.000e+00, 3f15e450-15ec-4427-9943-898be48bb79c = 1.000/1.000e+00, 20d3a3dc-4976-482f-9daf-124716832023 = 1.000/1.000e+00, 267b9b50-5ec6-45d2-9278-3e103b9a97d0 = 1.000/1.000e+00, d986c0bd-59ca-474a-b9e0-80d606dc8b7b = 1.000/1.000e+00, eccd2140-0fab-4313-aa0f-d52e8ae00a9f = 1.000/1.000e+00, 318ced06-10a6-4146-b760-8d4297f5ec28 = 1.000/1.000e+00, f14559c6-4a86-4e6e-b13e-18f57de5d66c = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.018472181106021668, 0.02513335149669619, 0.025744615401709452, 0.025785381743621723, 0.025788362699606195
Rejected: LBFGS Orientation magnitude: 5.822e+05, gradient 7.685e+00, dot -1.000; [b7d5052b-cd0f-457c-bb8d-13ebfd5db7f6 = 1.000/1.000e+00, d8dd43d5-d6c7-473b-9b64-14eb32febbaf = 1.000/1.000e+00, 267b9b50-5ec6-45d2-9278-3e103b9a97d0 = 1.000/1.000e+00, d986c0bd-59ca-474a-b9e0-80d606dc8b7b = 1.000/1.000e+00, eccd2140-0fab-4313-aa0f-d52e8ae00a9f = 1.000/1.000e+00, 20d3a3dc-4976-482f-9daf-124716832023 = 1.000/1.000e+00, 318ced06-10a6-4146-b760-8d4297f5ec28 = 1.000/1.000e+00, f14559c6-4a86-4e6e-b13e-18f57de5d66c = 1.000/1.000e+00, 3f15e450-15ec-4427-9943-898be48bb79c = 1.000/1.000e+00, f7686364-2b28-4822-b114-fb75a1476c1b = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.018472181106021668, 0.02513335149669619, 0.025744615401709452, 0.025785381743621723
Rejected: LBFGS Orientation magnitude: 6.152e+05, gradient 7.685e+00, dot -1.000; [f14559c6-4a86-4e6e-b13e-18f57de5d66c = 1.000/1.000e+00, b7d5052b-cd0f-457c-bb8d-13ebfd5db7f6 = 1.000/1.000e+00, 3f15e450-15ec-4427-9943-898be48bb79c = 1.000/1.000e+00, eccd2140-0fab-4313-aa0f-d52e8ae00a9f = 1.000/1.000e+00, 20d3a3dc-4976-482f-9daf-124716832023 = 1.000/1.000e+00, d8dd43d5-d6c7-473b-9b64-14eb32febbaf = 1.000/1.000e+00, d986c0bd-59ca-474a-b9e0-80d606dc8b7b = 1.000/1.000e+00, 318ced06-10a6-4146-b760-8d4297f5ec28 = 1.000/1.000e+00, 267b9b50-5ec6-45d2-9278-3e103b9a97d0 = 1.000/1.000e+00, f7686364-2b28-4822-b114-fb75a1476c1b = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.018472181106021668, 0.02513335149669619, 0.025744615401709452
LBFGS Accumulation History: 3 points
Removed measurement c702c44 to history. Total: 19
Removed measurement 1e0f9850 to history. Total: 18
Removed measurement 626e85ec to history. Total: 17
Removed measurement 2c426c0d to history. Total: 16
Removed measurement 12237c14 to history. Total: 15
Removed measurement 76ef1fcc to history. Total: 14
Removed measurement 16633826 to history. Total: 13
Removed measurement 4941b17d to history. Total: 12
Removed measurement 39e7abd2 to history. Total: 11
Removed measurement 3488fdd9 to history. Total: 10
Removed measurement 1e87e2f8 to history. Total: 9
Removed measurement 20a2efd1 to history. Total: 8
Removed measurement 55f61fc3 to history. Total: 7
Removed measurement 1319d3ec to history. Total: 6
Removed measurement 3b12514c to history. Total: 5
Removed measurement 1dc10f53 to history. Total: 4
Removed measurement 70fdb1d6 to history. Total: 3
Adding measurement 46ee562a to history. Total: 3
th(0)=0.0;dx=-59.05973120000001
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(939.1603064261446)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(469.5801532130723)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(156.52671773769077)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(39.13167943442269)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(7.8263358868845385)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.3043893144807563)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.18634133064010805)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.023292666330013506)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.0025880740366681673)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2.588074036668167E-4)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2.3527945787892428E-5)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.960662148991036E-6)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.5082016530700276E-7)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.0772868950500198E-8)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(7.181912633666799E-10)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(4.4886953960417494E-11)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2.640409056495147E-12)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.4668939202750816E-13)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(7.720494317237272E-15)=0.0; dx=-59.05973120000001 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
MIN ALPHA (3.860247158618636E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 31.7186; Orientation: 31.2439; Line Search: 0.4720
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 33.659s (< 30.000s)
Returns
0.0
TrainingTester.java:432 executed in 0.10 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -2.588572368630195], [2.0, -0.5885723686301947]; valueStats=DoubleSummaryStatistics{count=2, sum=0.051577, min=0.025789, average=0.025789, max=0.025789}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -2.588572368630195], [1.355, -0.5885723686301947]; valueStats=DoubleSummaryStatistics{count=2, sum=0.051577, min=0.025789, average=0.025789, max=0.025789}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS
Returns
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null} | OK |
{
"result": "OK",
"performance": {
"execution_time": "39.520",
"gc_time": "0.330"
},
"created_on": 1586736366765,
"file_name": "trainingTest",
"report": {
"simpleName": "NNTest",
"canonicalName": "com.simiacryptus.mindseye.layers.java.SumInputsLayerTest.NNTest",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/SumInputsLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "Converged",
"value": 0.0
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "Converged",
"value": 0.0
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/SumInputsLayer/NNTest/trainingTest/202004130606",
"id": "3f3525f8-40fc-4e14-bcad-e50276f7f4b7",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "SumInputsLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.java.SumInputsLayer",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/SumInputsLayer.java",
"javaDoc": ""
}
}