Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
Using Seed 4380894931973238784
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[ -0.608, -0.128, 0.048, 0.496, 0.08, 1.764, 1.524, 0.7 ]
[ -1.72, -0.384, 1.912, 1.208, -1.028, -0.852 ]
[ -0.608, 0.7, 0.08, -0.128, 1.764, 1.524, 0.048, 0.496 ]
[ -1.028, 1.912, -0.384, 1.208, -0.852, -1.72 ]
[ 0.08, 1.524, 0.7, 1.764, -0.128, 0.496, 0.048, -0.608 ]
[ -1.028, 1.208, -0.384, 1.912, -1.72, -0.852 ]
[ 0.048, 1.764, 0.08, -0.128, 1.524, -0.608, 0.496, 0.7 ]
[ -0.852, 1.912, -0.384, -1.028, -1.72, 1.208 ]
[ 0.496, 0.048, -0.608, 1.764, 1.524, 0.7, -0.128, 0.08 ]
[ -1.72, 1.912, -0.384, -0.852, -1.028, 1.208 ]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 0.45 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 3759936159518
BACKPROP_AGG_SIZE = 3
THREADS = 64
SINGLE_THREADED = false
Initialized CoreSettings = {
"backpropAggregationSize" : 3,
"jvmThreads" : 64,
"singleThreaded" : false
}
Reset training subject: 3759972903003
Constructing line search parameters: GD
th(0)=87.16326522572463;dx=-8.225075200000001E23
New Minimum: 87.16326522572463 > 0.04916913534421548
Armijo: th(2.154434690031884)=0.04916913534421548; dx=-8.225075200073618E11 evalInputDelta=87.11409609038041
Armijo: th(1.077217345015942)=0.14294215133796853; dx=-8.22507520007393E11 evalInputDelta=87.02032307438667
Armijo: th(0.3590724483386473)=0.35948274057424867; dx=-8.225075200075321E11 evalInputDelta=86.80378248515038
Armijo: th(0.08976811208466183)=1.0516026698761185; dx=-8.225075200116741E11 evalInputDelta=86.11166255584851
Armijo: th(0.017953622416932366)=2.3809739404630994; dx=-8.225075200378064E11 evalInputDelta=84.78229128526154
Armijo: th(0.002992270402822061)=3.183523617130176; dx=-8.22507520084321E11 evalInputDelta=83.97974160859445
Armijo: th(4.2746720040315154E-4)=3.426745156138292; dx=-8.225075201095629E11 evalInputDelta=83.73652006958633
Armijo: th(5.343340005039394E-5)=3.468327582011747; dx=-8.225075201147095E11 evalInputDelta=83.69493764371288
Armijo: th(5.9370444500437714E-6)=3.4737485348998094; dx=-8.22507520115401E11 evalInputDelta=83.68951669082482
Armijo: th(5.937044450043771E-7)=3.474360453039521; dx=-8.225075201154794E11 evalInputDelta=83.6889047726851
Armijo: th(5.397313136403428E-8)=3.474422286244554; dx=-8.225075201154873E11 evalInputDelta=83.68884293948008
Armijo: th(4.4977609470028565E-9)=3.474427954502302; dx=-8.22507520115488E11 evalInputDelta=83.68883727122233
Armijo: th(3.4598161130791205E-10)=3.474428430161929; dx=-8.22507520115488E11 evalInputDelta=83.68883679556271
Armijo: th(2.4712972236279432E-11)=3.9165248048237915; dx=-2.5600000082225155E20 evalInputDelta=83.24674042090084
Armijo: th(1.6475314824186289E-12)=50.35685705119767; dx=-3.87728000001617E23 evalInputDelta=36.80640817452696
Armijo: th(1.029707176511643E-13)=87.16326522571282; dx=-8.225075199999999E23 evalInputDelta=1.1809220268332865E-11
Armijo: th(6.057101038303783E-15)=87.16326522572393; dx=-8.225075199999999E23 evalInputDelta=6.963318810448982E-13
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=0.04916913534421548
Fitness changed from 87.16326522572463 to 0.04916913534421548
Iteration 1 complete. Error: 0.04916913534421548 Total: 0.2278; Orientation: 0.0039; Line Search: 0.1774
th(0)=0.04916913534421548;dx=-2.26295321341148
New Minimum: 0.04916913534421548 > 0.049169135344215444
WOLFE (weak): th(2.154434690031884E-15)=0.049169135344215444; dx=-2.26295321341148 evalInputDelta=3.469446951953614E-17
New Minimum: 0.049169135344215444 > 0.04916913534421541
WOLFE (weak): th(4.308869380063768E-15)=0.04916913534421541; dx=-2.26295321341148 evalInputDelta=6.938893903907228E-17
New Minimum: 0.04916913534421541 > 0.049169135344215284
WOLFE (weak): th(1.2926608140191303E-14)=0.049169135344215284; dx=-2.26295321341148 evalInputDelta=1.942890293094024E-16
New Minimum: 0.049169135344215284 > 0.04916913534421471
WOLFE (weak): th(5.1706432560765214E-14)=0.04916913534421471; dx=-2.26295321341148 evalInputDelta=7.7021722333370235E-16
New Minimum: 0.04916913534421471 > 0.04916913534421169
WOLFE (weak): th(2.5853216280382605E-13)=0.04916913534421169; dx=-2.26295321341148 evalInputDelta=3.788636071533347E-15
New Minimum: 0.04916913534421169 > 0.0491691353441928
WOLFE (weak): th(1.5511929768229563E-12)=0.0491691353441928; dx=-2.262953213411478 evalInputDelta=2.2676305277968822E-14
New Minimum: 0.0491691353441928 > 0.049169135344056744
WOLFE (weak): th(1.0858350837760695E-11)=0.049169135344056744; dx=-2.2629532134114636 evalInputDelta=1.5873413694578176E-13
New Minimum: 0.049169135344056744 > 0.049169135342945605
WOLFE (weak): th(8.686680670208556E-11)=0.049169135342945605; dx=-2.262953213411347 evalInputDelta=1.269873095566254E-12
New Minimum: 0.049169135342945605 > 0.04916913533278672
WOLFE (weak): th(7.8180126031877E-10)=0.04916913533278672; dx=-2.2629532134102783 evalInputDelta=1.1428760715581632E-11
New Minimum: 0.04916913533278672 > 0.04916913522992791
WOLFE (weak): th(7.818012603187701E-9)=0.04916913522992791; dx=-2.2629532133994608 evalInputDelta=1.142875655224529E-10
New Minimum: 0.04916913522992791 > 0.04916913408705226
WOLFE (weak): th(8.599813863506471E-8)=0.04916913408705226; dx=-2.262953213279267 evalInputDelta=1.2571632207469818E-9
New Minimum: 0.04916913408705226 > 0.04916912025825761
WOLFE (weak): th(1.0319776636207765E-6)=0.04916912025825761; dx=-2.262953211824925 evalInputDelta=1.508595786486877E-8
New Minimum: 0.04916912025825761 > 0.04916893922689103
WOLFE (weak): th(1.3415709627070094E-5)=0.04916893922689103; dx=-2.2629531927862905 evalInputDelta=1.96117324449685E-7
New Minimum: 0.04916893922689103 > 0.04916638972685279
WOLFE (weak): th(1.878199347789813E-4)=0.04916638972685279; dx=-2.2629529246643383 evalInputDelta=2.7456173626871916E-6
New Minimum: 0.04916638972685279 > 0.04912795677705188
WOLFE (weak): th(0.0028172990216847197)=0.04912795677705188; dx=-2.2629488834517977 evalInputDelta=4.117856716359669E-5
New Minimum: 0.04912795677705188 > 0.04851173747300116
WOLFE (weak): th(0.045076784346955515)=0.04851173747300116; dx=-2.262884253227543 evalInputDelta=6.5739787121432E-4
New Minimum: 0.04851173747300116 > 0.0392735984513561
WOLFE (weak): th(0.7663053338982437)=0.0392735984513561; dx=-2.2619598814524577 evalInputDelta=0.00989553689285938
New Minimum: 0.0392735984513561 > 0.010579113533592078
WOLFE (weak): th(13.793496010168386)=0.010579113533592078; dx=-2.260622630840399 evalInputDelta=0.038590021810623404
New Minimum: 0.010579113533592078 > 0.0
WOLFE (weak): th(262.07642419319933)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
WOLFE (weak): th(5241.528483863986)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
Armijo: th(110072.09816114372)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
Armijo: th(57656.813322503855)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
Armijo: th(31449.17090318392)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
WOLFE (weak): th(18345.349693523953)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
Armijo: th(24897.260298353936)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
WOLFE (weak): th(21621.304995938946)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
Armijo: th(23259.282647146443)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
Armijo: th(22440.293821542695)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
Armijo: th(22030.799408740822)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
Armijo: th(21826.052202339884)=0.0; dx=-2.2603159208795747 evalInputDelta=0.04916913534421548
mu ~= nu (21621.304995938946): th(262.07642419319933)=0.0
Fitness changed from 0.04916913534421548 to 0.0
Iteration 2 complete. Error: 0.0 Total: 0.1416; Orientation: 0.0013; Line Search: 0.1348
th(0)=0.0;dx=-2.2588
Armijo: th(46802.24676908919)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(23401.123384544597)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(7800.374461514865)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(1950.0936153787163)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(390.01872307574325)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(65.00312051262388)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(9.286160073231983)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(1.1607700091539979)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(0.12897444546155531)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(0.012897444546155531)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(0.001172494958741412)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(9.770791322845101E-5)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(7.515993325265463E-6)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(5.368566660903902E-7)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(3.579044440602601E-8)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(2.2369027753766257E-9)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(1.3158251619862504E-10)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(7.310139788812502E-12)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(3.847441994111843E-13)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(1.9237209970559215E-14)=0.0; dx=-2.2588 evalInputDelta=0.0
MIN ALPHA (9.16057617645677E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0758; Orientation: 0.0010; Line Search: 0.0727
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 0.446s (< 30.000s)
Returns
0.0
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.17 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 3760387953636
Reset training subject: 3760389548853
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=87.16326522572463}, derivative=-8.225075199999999E23}
New Minimum: 87.16326522572463 > 3.4745569730906296
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=3.4745569730906296}, derivative=-8.225095846316171E11}, evalInputDelta = -83.688708252634
New Minimum: 3.4745569730906296 > 3.474428389602862
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=3.474428389602862}, derivative=-8.22507520115488E11}, evalInputDelta = -83.68883683612177
New Minimum: 3.474428389602862 > 3.47442790841873
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=3.47442790841873}, derivative=-8.22507520115488E11}, evalInputDelta = -83.6888373173059
New Minimum: 3.47442790841873 > 3.474424540137055
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=3.474424540137055}, derivative=-8.225075201154875E11}, evalInputDelta = -83.68884068558758
New Minimum: 3.474424540137055 > 3.4744009625206034
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=3.4744009625206034}, derivative=-8.225075201154845E11}, evalInputDelta = -83.68886426320402
New Minimum: 3.4744009625206034 > 3.474235936610998
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=3.474235936610998}, derivative=-8.225075201154634E11}, evalInputDelta = -83.68902928911363
New Minimum: 3.474235936610998 > 3.4730816071942767
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=3.4730816071942767}, derivative=-8.225075201153157E11}, evalInputDelta = -83.69018361853036
New Minimum: 3.4730816071942767 > 3.4650427334345246
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=3.4650427334345246}, derivative=-8.225075201142927E11}, evalInputDelta = -83.6982224922901
New Minimum: 3.4650427334345246 > 3.4106995147100188
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=3.4106995147100188}, derivative=-8.225075201076493E11}, evalInputDelta = -83.75256571101461
New Minimum: 3.4106995147100188 > 3.10031749248414
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=3.10031749248414}, derivative=-8.225075200772783E11}, evalInputDelta = -84.06294773324049
New Minimum: 3.10031749248414 > 2.0575992312724614
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=2.0575992312724614}, derivative=-8.225075200279274E11}, evalInputDelta = -85.10566599445217
New Minimum: 2.0575992312724614 > 0.4683428823834176
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=0.4683428823834176}, derivative=-8.22507520007695E11}, evalInputDelta = -86.69492234334122
New Minimum: 0.4683428823834176 > 0.10305361588698639
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=0.10305361588698639}, derivative=-8.22507520007379E11}, evalInputDelta = -87.06021160983764
New Minimum: 0.10305361588698639 > 0.0017555527550308322
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=0.0017555527550308322}, derivative=-8.225075200073506E11}, evalInputDelta = -87.1615096729696
New Minimum: 0.0017555527550308322 > 0.0
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
F(23263.0513987207) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
F(162841.3597910449) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
F(1139889.5185373144) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
F(7979226.6297612) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
F(5.58545864083284E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
F(3.909821048582988E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
F(2.7368747340080914E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
F(1.915812313805664E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
0.0 <= 87.16326522572463
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-8.225075200073506E11}, evalInputDelta = -87.16326522572463
Right bracket at 1.0E10
Converged to right
Fitness changed from 87.16326522572463 to 0.0
Iteration 1 complete. Error: 0.0 Total: 0.0752; Orientation: 0.0009; Line Search: 0.0697
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
0.0 <= 0.0
F(5.0E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 5.0E9
F(2.5E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 2.5E9
F(1.25E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 1.25E9
F(6.25E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 6.25E8
F(3.125E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 3.125E8
F(1.5625E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 1.5625E8
F(7.8125E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 7.8125E7
F(3.90625E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 3.90625E7
F(1.953125E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 1.953125E7
F(9765625.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 9765625.0
F(4882812.5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 4882812.5
F(2441406.25) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Loops = 12
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0893; Orientation: 0.0007; Line Search: 0.0874
Iteration 2 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 2
Final threshold in iteration 2: 0.0 (> 0.0) after 0.165s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 2.86 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 3760557131774
Reset training subject: 3760558656281
Adding measurement 53868abb to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 87.16326522572463 < 87.16326522572463. Total: 1
th(0)=87.16326522572463;dx=-8.225075199999999E23
Adding measurement 35cd8e09 to history. Total: 1
New Minimum: 87.16326522572463 > 0.04916913534421548
Armijo: th(2.154434690031884)=0.04916913534421548; dx=-8.225075200073618E11 evalInputDelta=87.11409609038041
Non-optimal measurement 0.14294215133796853 < 0.04916913534421548. Total: 2
Armijo: th(1.077217345015942)=0.14294215133796853; dx=-8.225075200073928E11 evalInputDelta=87.02032307438667
Non-optimal measurement 0.35948274057424867 < 0.04916913534421548. Total: 2
Armijo: th(0.3590724483386473)=0.35948274057424867; dx=-8.225075200075321E11 evalInputDelta=86.80378248515038
Non-optimal measurement 1.0516026698761185 < 0.04916913534421548. Total: 2
Armijo: th(0.08976811208466183)=1.0516026698761185; dx=-8.225075200116741E11 evalInputDelta=86.11166255584851
Non-optimal measurement 2.3809739404630994 < 0.04916913534421548. Total: 2
Armijo: th(0.017953622416932366)=2.3809739404630994; dx=-8.225075200378064E11 evalInputDelta=84.78229128526154
Non-optimal measurement 3.183523617130176 < 0.04916913534421548. Total: 2
Armijo: th(0.002992270402822061)=3.183523617130176; dx=-8.22507520084321E11 evalInputDelta=83.97974160859445
Non-optimal measurement 3.426745156138292 < 0.04916913534421548. Total: 2
Armijo: th(4.2746720040315154E-4)=3.426745156138292; dx=-8.225075201095627E11 evalInputDelta=83.73652006958633
Non-optimal measurement 3.468327582011747 < 0.04916913534421548. Total: 2
Armijo: th(5.343340005039394E-5)=3.468327582011747; dx=-8.225075201147095E11 evalInputDelta=83.69493764371288
Non-optimal measurement 3.4737485348998094 < 0.04916913534421548. Total: 2
Armijo: th(5.9370444500437714E-6)=3.4737485348998094; dx=-8.22507520115401E11 evalInputDelta=83.68951669082482
Non-optimal measurement 3.474360453039521 < 0.04916913534421548. Total: 2
Armijo: th(5.937044450043771E-7)=3.474360453039521; dx=-8.225075201154794E11 evalInputDelta=83.6889047726851
Non-optimal measurement 3.474422286244554 < 0.04916913534421548. Total: 2
Armijo: th(5.397313136403428E-8)=3.474422286244554; dx=-8.225075201154873E11 evalInputDelta=83.68884293948008
Non-optimal measurement 3.474427954502302 < 0.04916913534421548. Total: 2
Armijo: th(4.4977609470028565E-9)=3.474427954502302; dx=-8.22507520115488E11 evalInputDelta=83.68883727122233
Non-optimal measurement 3.474428430161929 < 0.04916913534421548. Total: 2
Armijo: th(3.4598161130791205E-10)=3.474428430161929; dx=-8.22507520115488E11 evalInputDelta=83.68883679556271
Non-optimal measurement 3.9165248048237915 < 0.04916913534421548. Total: 2
Armijo: th(2.4712972236279432E-11)=3.9165248048237915; dx=-2.5600000082225152E20 evalInputDelta=83.24674042090084
Non-optimal measurement 50.35685705119767 < 0.04916913534421548. Total: 2
Armijo: th(1.6475314824186289E-12)=50.35685705119767; dx=-3.87728000001617E23 evalInputDelta=36.80640817452696
Non-optimal measurement 87.16326522571282 < 0.04916913534421548. Total: 2
Armijo: th(1.029707176511643E-13)=87.16326522571282; dx=-8.225075199999999E23 evalInputDelta=1.1809220268332865E-11
Non-optimal measurement 87.16326522572393 < 0.04916913534421548. Total: 2
Armijo: th(6.057101038303783E-15)=87.16326522572393; dx=-8.225075199999999E23 evalInputDelta=6.963318810448982E-13
Non-optimal measurement 0.04916913534421548 < 0.04916913534421548. Total: 2
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=0.04916913534421548
Fitness changed from 87.16326522572463 to 0.04916913534421548
Iteration 1 complete. Error: 0.04916913534421548 Total: 0.0605; Orientation: 0.0034; Line Search: 0.0526
Non-optimal measurement 0.04916913534421548 < 0.04916913534421548. Total: 2
LBFGS Accumulation History: 2 points
Non-optimal measurement 0.04916913534421548 < 0.04916913534421548. Total: 2
th(0)=0.04916913534421548;dx=-2.26295321341148
Adding measurement 6488c5dc to history. Total: 2
New Minimum: 0.04916913534421548 > 0.049169135344215444
WOLFE (weak): th(2.154434690031884E-15)=0.049169135344215444; dx=-2.26295321341148 evalInputDelta=3.469446951953614E-17
Adding measurement 1dcef903 to history. Total: 3
New Minimum: 0.049169135344215444 > 0.04916913534421541
WOLFE (weak): th(4.308869380063768E-15)=0.04916913534421541; dx=-2.26295321341148 evalInputDelta=6.938893903907228E-17
Adding measurement 20325986 to history. Total: 4
New Minimum: 0.04916913534421541 > 0.049169135344215284
WOLFE (weak): th(1.2926608140191303E-14)=0.049169135344215284; dx=-2.26295321341148 evalInputDelta=1.942890293094024E-16
Adding measurement 58c483fe to history. Total: 5
New Minimum: 0.049169135344215284 > 0.04916913534421471
WOLFE (weak): th(5.1706432560765214E-14)=0.04916913534421471; dx=-2.26295321341148 evalInputDelta=7.7021722333370235E-16
Adding measurement 39e2bd30 to history. Total: 6
New Minimum: 0.04916913534421471 > 0.04916913534421169
WOLFE (weak): th(2.5853216280382605E-13)=0.04916913534421169; dx=-2.26295321341148 evalInputDelta=3.788636071533347E-15
Adding measurement 73f82ad to history. Total: 7
New Minimum: 0.04916913534421169 > 0.0491691353441928
WOLFE (weak): th(1.5511929768229563E-12)=0.0491691353441928; dx=-2.262953213411478 evalInputDelta=2.2676305277968822E-14
Adding measurement 29da6f92 to history. Total: 8
New Minimum: 0.0491691353441928 > 0.049169135344056744
WOLFE (weak): th(1.0858350837760695E-11)=0.049169135344056744; dx=-2.2629532134114636 evalInputDelta=1.5873413694578176E-13
Adding measurement 34eba455 to history. Total: 9
New Minimum: 0.049169135344056744 > 0.049169135342945605
WOLFE (weak): th(8.686680670208556E-11)=0.049169135342945605; dx=-2.262953213411347 evalInputDelta=1.269873095566254E-12
Adding measurement 4d362213 to history. Total: 10
New Minimum: 0.049169135342945605 > 0.04916913533278672
WOLFE (weak): th(7.8180126031877E-10)=0.04916913533278672; dx=-2.2629532134102783 evalInputDelta=1.1428760715581632E-11
Adding measurement 337da1fd to history. Total: 11
New Minimum: 0.04916913533278672 > 0.04916913522992791
WOLFE (weak): th(7.818012603187701E-9)=0.04916913522992791; dx=-2.2629532133994608 evalInputDelta=1.142875655224529E-10
Adding measurement 271eb847 to history. Total: 12
New Minimum: 0.04916913522992791 > 0.04916913408705226
WOLFE (weak): th(8.599813863506471E-8)=0.04916913408705226; dx=-2.262953213279267 evalInputDelta=1.2571632207469818E-9
Adding measurement 64867ce1 to history. Total: 13
New Minimum: 0.04916913408705226 > 0.04916912025825761
WOLFE (weak): th(1.0319776636207765E-6)=0.04916912025825761; dx=-2.262953211824925 evalInputDelta=1.508595786486877E-8
Adding measurement 5838bf62 to history. Total: 14
New Minimum: 0.04916912025825761 > 0.04916893922689103
WOLFE (weak): th(1.3415709627070094E-5)=0.04916893922689103; dx=-2.2629531927862905 evalInputDelta=1.96117324449685E-7
Adding measurement 477b4888 to history. Total: 15
New Minimum: 0.04916893922689103 > 0.04916638972685279
WOLFE (weak): th(1.878199347789813E-4)=0.04916638972685279; dx=-2.2629529246643383 evalInputDelta=2.7456173626871916E-6
Adding measurement 5def46fa to history. Total: 16
New Minimum: 0.04916638972685279 > 0.04912795677705188
WOLFE (weak): th(0.0028172990216847197)=0.04912795677705188; dx=-2.2629488834517977 evalInputDelta=4.117856716359669E-5
Adding measurement 73b40bd6 to history. Total: 17
New Minimum: 0.04912795677705188 > 0.04851173747300116
WOLFE (weak): th(0.045076784346955515)=0.04851173747300116; dx=-2.262884253227543 evalInputDelta=6.5739787121432E-4
Adding measurement 5b1bd293 to history. Total: 18
New Minimum: 0.04851173747300116 > 0.0392735984513561
WOLFE (weak): th(0.7663053338982437)=0.0392735984513561; dx=-2.2619598814524577 evalInputDelta=0.00989553689285938
Adding measurement 4cb3ac96 to history. Total: 19
New Minimum: 0.0392735984513561 > 0.010579113533592078
WOLFE (weak): th(13.793496010168386)=0.010579113533592078; dx=-2.260622630840399 evalInputDelta=0.038590021810623404
Adding measurement 6a89922b to history. Total: 20
New Minimum: 0.010579113533592078 > 0.0
WOLF
...skipping 14252 bytes...
7f6e5e24 = 1.000/1.000e+00, d1710679-fb3f-4a24-9d98-512c2163d8f0 = 1.000/1.000e+00, a7a3200b-0c4d-4206-b022-922b0ea4b945 = 1.000/1.000e+00, 851d958d-7c42-450b-9bf5-62a9c60c4d9f = 1.000/1.000e+00, 6c2294ae-8415-4938-8837-eca8a878fba0 = 1.000/1.000e+00, 080f3556-6c7c-4fd5-9c43-27dc8d3db579 = 1.000/1.000e+00, 1a608134-7991-4b28-85fd-27430478055e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.010579113533592078, 0.0392735984513561, 0.04851173747300116, 0.04912795677705188, 0.04916638972685279, 0.04916893922689103, 0.04916912025825761, 0.04916913408705226
Rejected: LBFGS Orientation magnitude: 3.513e+03, gradient 1.503e+00, dot -0.987; [d1710679-fb3f-4a24-9d98-512c2163d8f0 = 1.000/1.000e+00, 080f3556-6c7c-4fd5-9c43-27dc8d3db579 = 1.000/1.000e+00, 1a608134-7991-4b28-85fd-27430478055e = 1.000/1.000e+00, 5708d932-4bca-4ace-95cb-92dc7fd93b86 = 1.000/1.000e+00, a7a3200b-0c4d-4206-b022-922b0ea4b945 = 1.000/1.000e+00, c8941c21-8f93-47e3-b31b-40b03d8dfe82 = 1.000/1.000e+00, 6c2294ae-8415-4938-8837-eca8a878fba0 = 1.000/1.000e+00, 1fbfba5c-80ae-4112-8a67-dd6899049d71 = 1.000/1.000e+00, 851d958d-7c42-450b-9bf5-62a9c60c4d9f = 1.000/1.000e+00, cc65c998-99c1-48a2-9e49-df797f6e5e24 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.010579113533592078, 0.0392735984513561, 0.04851173747300116, 0.04912795677705188, 0.04916638972685279, 0.04916893922689103, 0.04916912025825761
Rejected: LBFGS Orientation magnitude: 4.776e+03, gradient 1.503e+00, dot -0.967; [080f3556-6c7c-4fd5-9c43-27dc8d3db579 = 1.000/1.000e+00, 851d958d-7c42-450b-9bf5-62a9c60c4d9f = 1.000/1.000e+00, d1710679-fb3f-4a24-9d98-512c2163d8f0 = 1.000/1.000e+00, 1fbfba5c-80ae-4112-8a67-dd6899049d71 = 1.000/1.000e+00, c8941c21-8f93-47e3-b31b-40b03d8dfe82 = 1.000/1.000e+00, 1a608134-7991-4b28-85fd-27430478055e = 1.000/1.000e+00, 6c2294ae-8415-4938-8837-eca8a878fba0 = 1.000/1.000e+00, a7a3200b-0c4d-4206-b022-922b0ea4b945 = 1.000/1.000e+00, cc65c998-99c1-48a2-9e49-df797f6e5e24 = 1.000/1.000e+00, 5708d932-4bca-4ace-95cb-92dc7fd93b86 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.010579113533592078, 0.0392735984513561, 0.04851173747300116, 0.04912795677705188, 0.04916638972685279, 0.04916893922689103
Rejected: LBFGS Orientation magnitude: 4.403e+03, gradient 1.503e+00, dot -0.999; [c8941c21-8f93-47e3-b31b-40b03d8dfe82 = 1.000/1.000e+00, d1710679-fb3f-4a24-9d98-512c2163d8f0 = 1.000/1.000e+00, 6c2294ae-8415-4938-8837-eca8a878fba0 = 1.000/1.000e+00, cc65c998-99c1-48a2-9e49-df797f6e5e24 = 1.000/1.000e+00, a7a3200b-0c4d-4206-b022-922b0ea4b945 = 1.000/1.000e+00, 5708d932-4bca-4ace-95cb-92dc7fd93b86 = 1.000/1.000e+00, 851d958d-7c42-450b-9bf5-62a9c60c4d9f = 1.000/1.000e+00, 1a608134-7991-4b28-85fd-27430478055e = 1.000/1.000e+00, 1fbfba5c-80ae-4112-8a67-dd6899049d71 = 1.000/1.000e+00, 080f3556-6c7c-4fd5-9c43-27dc8d3db579 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.010579113533592078, 0.0392735984513561, 0.04851173747300116, 0.04912795677705188, 0.04916638972685279
Rejected: LBFGS Orientation magnitude: 4.439e+03, gradient 1.503e+00, dot -0.999; [cc65c998-99c1-48a2-9e49-df797f6e5e24 = 1.000/1.000e+00, 5708d932-4bca-4ace-95cb-92dc7fd93b86 = 1.000/1.000e+00, 080f3556-6c7c-4fd5-9c43-27dc8d3db579 = 1.000/1.000e+00, 6c2294ae-8415-4938-8837-eca8a878fba0 = 1.000/1.000e+00, 1a608134-7991-4b28-85fd-27430478055e = 1.000/1.000e+00, 1fbfba5c-80ae-4112-8a67-dd6899049d71 = 1.000/1.000e+00, 851d958d-7c42-450b-9bf5-62a9c60c4d9f = 1.000/1.000e+00, a7a3200b-0c4d-4206-b022-922b0ea4b945 = 1.000/1.000e+00, c8941c21-8f93-47e3-b31b-40b03d8dfe82 = 1.000/1.000e+00, d1710679-fb3f-4a24-9d98-512c2163d8f0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.010579113533592078, 0.0392735984513561, 0.04851173747300116, 0.04912795677705188
Rejected: LBFGS Orientation magnitude: 5.296e+03, gradient 1.503e+00, dot -0.999; [d1710679-fb3f-4a24-9d98-512c2163d8f0 = 1.000/1.000e+00, 851d958d-7c42-450b-9bf5-62a9c60c4d9f = 1.000/1.000e+00, a7a3200b-0c4d-4206-b022-922b0ea4b945 = 1.000/1.000e+00, c8941c21-8f93-47e3-b31b-40b03d8dfe82 = 1.000/1.000e+00, 080f3556-6c7c-4fd5-9c43-27dc8d3db579 = 1.000/1.000e+00, 1fbfba5c-80ae-4112-8a67-dd6899049d71 = 1.000/1.000e+00, cc65c998-99c1-48a2-9e49-df797f6e5e24 = 1.000/1.000e+00, 1a608134-7991-4b28-85fd-27430478055e = 1.000/1.000e+00, 5708d932-4bca-4ace-95cb-92dc7fd93b86 = 1.000/1.000e+00, 6c2294ae-8415-4938-8837-eca8a878fba0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.010579113533592078, 0.0392735984513561, 0.04851173747300116
LBFGS Accumulation History: 3 points
Removed measurement 6a89922b to history. Total: 20
Removed measurement 4cb3ac96 to history. Total: 19
Removed measurement 5b1bd293 to history. Total: 18
Removed measurement 73b40bd6 to history. Total: 17
Removed measurement 5def46fa to history. Total: 16
Removed measurement 477b4888 to history. Total: 15
Removed measurement 5838bf62 to history. Total: 14
Removed measurement 64867ce1 to history. Total: 13
Removed measurement 271eb847 to history. Total: 12
Removed measurement 337da1fd to history. Total: 11
Removed measurement 4d362213 to history. Total: 10
Removed measurement 34eba455 to history. Total: 9
Removed measurement 29da6f92 to history. Total: 8
Removed measurement 73f82ad to history. Total: 7
Removed measurement 39e2bd30 to history. Total: 6
Removed measurement 58c483fe to history. Total: 5
Removed measurement 20325986 to history. Total: 4
Removed measurement 1dcef903 to history. Total: 3
Adding measurement 820142a to history. Total: 3
th(0)=0.0;dx=-2.2588
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(46802.24676908919)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(23401.123384544597)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(7800.374461514865)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1950.0936153787163)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(390.01872307574325)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(65.00312051262388)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(9.286160073231983)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.1607700091539979)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.12897444546155531)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.012897444546155531)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.001172494958741412)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(9.770791322845101E-5)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(7.515993325265463E-6)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(5.368566660903902E-7)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(3.579044440602601E-8)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2.2369027753766257E-9)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.3158251619862504E-10)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(7.310139788812502E-12)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(3.847441994111843E-13)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.9237209970559215E-14)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
MIN ALPHA (9.16057617645677E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 2.7041; Orientation: 2.6487; Line Search: 0.0537
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 2.864s (< 30.000s)
Returns
0.0
TrainingTester.java:432 executed in 0.12 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -2.3083074288572396], [2.0, -0.30830742885723983]; valueStats=DoubleSummaryStatistics{count=2, sum=0.098338, min=0.049169, average=0.049169, max=0.049169}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -2.3083074288572396], [0.141, -0.30830742885723983]; valueStats=DoubleSummaryStatistics{count=2, sum=0.098338, min=0.049169, average=0.049169, max=0.049169}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS
Returns
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null} | OK |
{
"result": "OK",
"performance": {
"execution_time": "4.411",
"gc_time": "0.508"
},
"created_on": 1586738347740,
"file_name": "trainingTest",
"report": {
"simpleName": "Basic",
"canonicalName": "com.simiacryptus.mindseye.layers.java.TensorConcatLayerTest.Basic",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/TensorConcatLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "Converged",
"value": 0.0
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "Converged",
"value": 0.0
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/TensorConcatLayer/Basic/trainingTest/202004133907",
"id": "7011630b-a452-47fa-8a1e-b3dbc744e3c6",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "TensorConcatLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.java.TensorConcatLayer",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/TensorConcatLayer.java",
"javaDoc": ""
}
}