Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
Using Seed 1934934799661148160
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.048, -1.72, 0.7, -0.852 ], [ -0.384, -0.128, 1.912, 0.08 ] ],
[ [ -0.804, -1.688, 1.764, 0.496 ], [ 1.524, 1.208, -0.608, -1.028 ] ]
]
[
[ [ 0.496, 1.524, 0.048, -1.688 ], [ 1.912, 0.08, -0.608, -1.72 ] ],
[ [ -1.028, -0.384, -0.852, 1.764 ], [ -0.128, -0.804, 0.7, 1.208 ] ]
]
[
[ [ -1.028, -1.688, 1.912, 0.496 ], [ 1.524, 0.048, -1.72, -0.804 ] ],
[ [ -0.128, 1.208, -0.384, 1.764 ], [ 0.08, 0.7, -0.852, -0.608 ] ]
]
[
[ [ -0.128, 0.08, -0.804, -1.688 ], [ 0.7, 1.912, 1.208, 1.764 ] ],
[ [ -0.852, -0.608, -0.384, 0.496 ], [ 1.524, -1.72, -1.028, 0.048 ] ]
]
[
[ [ -0.852, 0.048, 1.208, -1.028 ], [ -0.384, -0.804, -1.72, 1.524 ] ],
[ [ -1.688, 1.764, 0.08, 0.496 ], [ 1.912, -0.608, 0.7, -0.128 ] ]
]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 0.42 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 429361166666
BACKPROP_AGG_SIZE = 3
THREADS = 64
SINGLE_THREADED = false
Initialized CoreSettings = {
"backpropAggregationSize" : 3,
"jvmThreads" : 64,
"singleThreaded" : false
}
Reset training subject: 429392445161
Constructing line search parameters: GD
th(0)=100.06160935625465;dx=-1.0718111999999999E24
New Minimum: 100.06160935625465 > 0.009120779343008902
Armijo: th(2.154434690031884)=0.009120779343008902; dx=-1.0718112000057332E12 evalInputDelta=100.05248857691164
Armijo: th(1.077217345015942)=0.034872524961524695; dx=-1.0718112000057384E12 evalInputDelta=100.02673683129312
Armijo: th(0.3590724483386473)=0.26999823456794403; dx=-1.0718112000059326E12 evalInputDelta=99.7916111216867
Armijo: th(0.08976811208466183)=1.3480092184329675; dx=-1.0718112000117806E12 evalInputDelta=98.71360013782169
Armijo: th(0.017953622416932366)=2.6300967612468016; dx=-1.0718112000319897E12 evalInputDelta=97.43151259500785
Armijo: th(0.002992270402822061)=3.2291541839409463; dx=-1.0718112000528322E12 evalInputDelta=96.8324551723137
Armijo: th(4.2746720040315154E-4)=3.3716743388501498; dx=-1.0718112000597167E12 evalInputDelta=96.6899350174045
Armijo: th(5.343340005039394E-5)=3.3940173537595877; dx=-1.0718112000608812E12 evalInputDelta=96.66759200249506
Armijo: th(5.9370444500437714E-6)=3.396885997911447; dx=-1.0718112000610326E12 evalInputDelta=96.6647233583432
Armijo: th(5.937044450043771E-7)=3.397209170583983; dx=-1.0718112000610497E12 evalInputDelta=96.66440018567067
Armijo: th(5.397313136403428E-8)=3.397241819361279; dx=-1.0718112000610513E12 evalInputDelta=96.66436753689337
Armijo: th(4.4977609470028565E-9)=3.3972448122124477; dx=-1.0718112000610515E12 evalInputDelta=96.6643645440422
Armijo: th(3.4598161130791205E-10)=3.3972450633611535; dx=-1.0718112000610515E12 evalInputDelta=96.6643642928935
Armijo: th(2.4712972236279432E-11)=5.25404970178568; dx=-9.523200010708589E20 evalInputDelta=94.80755965446897
Armijo: th(1.6475314824186289E-12)=59.60243846771942; dx=-5.697075200055065E23 evalInputDelta=40.459170888535226
Armijo: th(1.029707176511643E-13)=100.06160935624841; dx=-1.0718111999999999E24 evalInputDelta=6.23856521997368E-12
Armijo: th(6.057101038303783E-15)=100.06160935625428; dx=-1.0718111999999999E24 evalInputDelta=3.694822225952521E-13
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=0.009120779343008902
Fitness changed from 100.06160935625465 to 0.009120779343008902
Iteration 1 complete. Error: 0.009120779343008902 Total: 0.1658; Orientation: 0.0030; Line Search: 0.1214
th(0)=0.009120779343008902;dx=-2.259344530683688
New Minimum: 0.009120779343008902 > 0.0091207793430089
WOLFE (weak): th(2.154434690031884E-15)=0.0091207793430089; dx=-2.259344530683688 evalInputDelta=1.734723475976807E-18
WOLFE (weak): th(4.308869380063768E-15)=0.0091207793430089; dx=-2.259344530683688 evalInputDelta=1.734723475976807E-18
New Minimum: 0.0091207793430089 > 0.009120779343008893
WOLFE (weak): th(1.2926608140191303E-14)=0.009120779343008893; dx=-2.259344530683688 evalInputDelta=8.673617379884035E-18
New Minimum: 0.009120779343008893 > 0.009120779343008862
WOLFE (weak): th(5.1706432560765214E-14)=0.009120779343008862; dx=-2.259344530683688 evalInputDelta=3.9898639947466563E-17
New Minimum: 0.009120779343008862 > 0.009120779343008697
WOLFE (weak): th(2.5853216280382605E-13)=0.009120779343008697; dx=-2.259344530683688 evalInputDelta=2.0469737016526324E-16
New Minimum: 0.009120779343008697 > 0.009120779343007663
WOLFE (weak): th(1.5511929768229563E-12)=0.009120779343007663; dx=-2.259344530683688 evalInputDelta=1.2385925618474403E-15
New Minimum: 0.009120779343007663 > 0.00912077934300021
WOLFE (weak): th(1.0858350837760695E-11)=0.00912077934300021; dx=-2.2593445306836877 evalInputDelta=8.690964614643804E-15
New Minimum: 0.00912077934300021 > 0.009120779342939363
WOLFE (weak): th(8.686680670208556E-11)=0.009120779342939363; dx=-2.259344530683685 evalInputDelta=6.953812525800629E-14
New Minimum: 0.009120779342939363 > 0.009120779342383046
WOLFE (weak): th(7.8180126031877E-10)=0.009120779342383046; dx=-2.259344530683657 evalInputDelta=6.258552703863884E-13
New Minimum: 0.009120779342383046 > 0.009120779336750342
WOLFE (weak): th(7.818012603187701E-9)=0.009120779336750342; dx=-2.259344530683375 evalInputDelta=6.258559642757788E-12
New Minimum: 0.009120779336750342 > 0.009120779274164754
WOLFE (weak): th(8.599813863506471E-8)=0.009120779274164754; dx=-2.2593445306802438 evalInputDelta=6.884414739671829E-11
New Minimum: 0.009120779274164754 > 0.00912077851687914
WOLFE (weak): th(1.0319776636207765E-6)=0.00912077851687914; dx=-2.2593445306423545 evalInputDelta=8.261297618217256E-10
New Minimum: 0.00912077851687914 > 0.009120768603325306
WOLFE (weak): th(1.3415709627070094E-5)=0.009120768603325306; dx=-2.259344530146348 evalInputDelta=1.0739683595564764E-8
New Minimum: 0.009120768603325306 > 0.00912062898809456
WOLFE (weak): th(1.878199347789813E-4)=0.00912062898809456; dx=-2.259344523160991 evalInputDelta=1.5035491434181592E-7
New Minimum: 0.00912062898809456 > 0.009118524167636137
WOLFE (weak): th(0.0028172990216847197)=0.009118524167636137; dx=-2.259344417858072 evalInputDelta=2.2551753727645396E-6
New Minimum: 0.009118524167636137 > 0.009084734625200241
WOLFE (weak): th(0.045076784346955515)=0.009084734625200241; dx=-2.259342729282132 evalInputDelta=3.6044717808660434E-5
New Minimum: 0.009084734625200241 > 0.008518796268298294
WOLFE (weak): th(0.7663053338982437)=0.008518796268298294; dx=-2.2593149711239873 evalInputDelta=6.019830747106079E-4
New Minimum: 0.008518796268298294 > 7.238703897422067E-4
WOLFE (weak): th(13.793496010168386)=7.238703897422067E-4; dx=-2.259017649716026 evalInputDelta=0.008396908953266695
New Minimum: 7.238703897422067E-4 > 0.0
WOLFE (weak): th(262.07642419319933)=0.0; dx=-2.258996698415089 evalInputDelta=0.009120779343008902
Armijo: th(5241.528483863986)=0.0; dx=-2.258996698415089 evalInputDelta=0.009120779343008902
WOLFE (weak): th(2751.802454028593)=0.0; dx=-2.258996698415089 evalInputDelta=0.009120779343008902
WOLFE (weak): th(3996.6654689462894)=0.0; dx=-2.258996698415089 evalInputDelta=0.009120779343008902
Armijo: th(4619.096976405138)=0.0; dx=-2.258996698415089 evalInputDelta=0.009120779343008902
Armijo: th(4307.881222675714)=0.0; dx=-2.258996698415089 evalInputDelta=0.009120779343008902
Armijo: th(4152.2733458110015)=0.0; dx=-2.258996698415089 evalInputDelta=0.009120779343008902
Armijo: th(4074.4694073786454)=0.0; dx=-2.258996698415089 evalInputDelta=0.009120779343008902
WOLFE (weak): th(4035.5674381624676)=0.0; dx=-2.258996698415089 evalInputDelta=0.009120779343008902
mu ~= nu (4035.5674381624676): th(262.07642419319933)=0.0
Fitness changed from 0.009120779343008902 to 0.0
Iteration 2 complete. Error: 0.0 Total: 0.1837; Orientation: 0.0014; Line Search: 0.1739
th(0)=0.0;dx=-2.2588
Armijo: th(8736.272358735263)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(4368.1361793676315)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(1456.0453931225438)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(364.01134828063596)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(72.8022696561272)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(12.133711609354533)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(1.7333873727649334)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(0.21667342159561667)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(0.024074824621735186)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(0.0024074824621735185)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(2.188620420157744E-4)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(1.8238503501314535E-5)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(1.4029618077934257E-6)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(1.0021155769953041E-7)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(6.680770513302027E-9)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(4.175481570813767E-10)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(2.4561656298904512E-11)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(1.3645364610502506E-12)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(7.181770847632898E-14)=0.0; dx=-2.2588 evalInputDelta=0.0
Armijo: th(3.5908854238164494E-15)=0.0; dx=-2.2588 evalInputDelta=0.0
MIN ALPHA (1.709945439912595E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0597; Orientation: 0.0007; Line Search: 0.0569
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 0.410s (< 30.000s)
Returns
0.0
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.15 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 429776940821
Reset training subject: 429778537697
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=100.06160935625465}, derivative=-1.0718111999999999E24}
New Minimum: 100.06160935625465 > 3.8733042191096807
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=3.8733042191096807}, derivative=-2.5600000107251545E20}, evalInputDelta = -96.18830513714497
New Minimum: 3.8733042191096807 > 3.3972450419459337
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=3.3972450419459337}, derivative=-1.0718112000610515E12}, evalInputDelta = -96.66436431430871
New Minimum: 3.3972450419459337 > 3.3972447878802767
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=3.3972447878802767}, derivative=-1.0718112000610515E12}, evalInputDelta = -96.66436456837437
New Minimum: 3.3972447878802767 > 3.3972430094222594
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=3.3972430094222594}, derivative=-1.0718112000610515E12}, evalInputDelta = -96.6643663468324
New Minimum: 3.3972430094222594 > 3.397230560293491
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=3.397230560293491}, derivative=-1.0718112000610508E12}, evalInputDelta = -96.66437879596116
New Minimum: 3.397230560293491 > 3.3971434201820756
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=3.3971434201820756}, derivative=-1.0718112000610461E12}, evalInputDelta = -96.66446593607257
New Minimum: 3.3971434201820756 > 3.3965336250309592
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=3.3965336250309592}, derivative=-1.071811200061014E12}, evalInputDelta = -96.6650757312237
New Minimum: 3.3965336250309592 > 3.392274127605995
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=3.392274127605995}, derivative=-1.0718112000607896E12}, evalInputDelta = -96.66933522864865
New Minimum: 3.392274127605995 > 3.362892904871683
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=3.362892904871683}, derivative=-1.0718112000592654E12}, evalInputDelta = -96.69871645138296
New Minimum: 3.362892904871683 > 3.175873739740836
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=3.175873739740836}, derivative=-1.0718112000504834E12}, evalInputDelta = -96.88573561651381
New Minimum: 3.175873739740836 > 2.3435154198348167
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=2.3435154198348167}, derivative=-1.0718112000253516E12}, evalInputDelta = -97.71809393641983
New Minimum: 2.3435154198348167 > 0.4959106541802562
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=0.4959106541802562}, derivative=-1.0718112000065244E12}, evalInputDelta = -99.5656987020744
New Minimum: 0.4959106541802562 > 0.016404911166731918
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=0.016404911166731918}, derivative=-1.0718112000057345E12}, evalInputDelta = -100.04520444508792
New Minimum: 0.016404911166731918 > 0.003400299230150326
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=0.003400299230150326}, derivative=-1.0718112000057329E12}, evalInputDelta = -100.0582090570245
New Minimum: 0.003400299230150326 > 0.0
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
F(23263.0513987207) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
F(162841.3597910449) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
F(1139889.5185373144) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
F(7979226.6297612) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
F(5.58545864083284E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
F(3.909821048582988E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
F(2.7368747340080914E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
F(1.915812313805664E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
0.0 <= 100.06160935625465
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.0718112000057328E12}, evalInputDelta = -100.06160935625465
Right bracket at 1.0E10
Converged to right
Fitness changed from 100.06160935625465 to 0.0
Iteration 1 complete. Error: 0.0 Total: 0.0883; Orientation: 0.0007; Line Search: 0.0811
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
0.0 <= 0.0
F(5.0E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 5.0E9
F(2.5E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 2.5E9
F(1.25E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 1.25E9
F(6.25E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 6.25E8
F(3.125E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 3.125E8
F(1.5625E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 1.5625E8
F(7.8125E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 7.8125E7
F(3.90625E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 3.90625E7
F(1.953125E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 1.953125E7
F(9765625.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 9765625.0
F(4882812.5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Right bracket at 4882812.5
F(2441406.25) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.2588}, evalInputDelta = 0.0
Loops = 12
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0651; Orientation: 0.0008; Line Search: 0.0626
Iteration 2 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 2
Final threshold in iteration 2: 0.0 (> 0.0) after 0.154s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 2.92 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 429935832722
Reset training subject: 429937427422
Adding measurement 4dbb2e7c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 100.06160935625465 < 100.06160935625465. Total: 1
th(0)=100.06160935625465;dx=-1.0718111999999999E24
Adding measurement 5ae230fc to history. Total: 1
New Minimum: 100.06160935625465 > 0.009120779343008902
Armijo: th(2.154434690031884)=0.009120779343008902; dx=-1.0718112000057332E12 evalInputDelta=100.05248857691164
Non-optimal measurement 0.034872524961524695 < 0.009120779343008902. Total: 2
Armijo: th(1.077217345015942)=0.034872524961524695; dx=-1.0718112000057384E12 evalInputDelta=100.02673683129312
Non-optimal measurement 0.26999823456794403 < 0.009120779343008902. Total: 2
Armijo: th(0.3590724483386473)=0.26999823456794403; dx=-1.0718112000059326E12 evalInputDelta=99.7916111216867
Non-optimal measurement 1.3480092184329675 < 0.009120779343008902. Total: 2
Armijo: th(0.08976811208466183)=1.3480092184329675; dx=-1.0718112000117808E12 evalInputDelta=98.71360013782169
Non-optimal measurement 2.6300967612468016 < 0.009120779343008902. Total: 2
Armijo: th(0.017953622416932366)=2.6300967612468016; dx=-1.0718112000319897E12 evalInputDelta=97.43151259500785
Non-optimal measurement 3.2291541839409463 < 0.009120779343008902. Total: 2
Armijo: th(0.002992270402822061)=3.2291541839409463; dx=-1.0718112000528322E12 evalInputDelta=96.8324551723137
Non-optimal measurement 3.3716743388501498 < 0.009120779343008902. Total: 2
Armijo: th(4.2746720040315154E-4)=3.3716743388501498; dx=-1.0718112000597166E12 evalInputDelta=96.6899350174045
Non-optimal measurement 3.3940173537595877 < 0.009120779343008902. Total: 2
Armijo: th(5.343340005039394E-5)=3.3940173537595877; dx=-1.0718112000608813E12 evalInputDelta=96.66759200249506
Non-optimal measurement 3.396885997911447 < 0.009120779343008902. Total: 2
Armijo: th(5.9370444500437714E-6)=3.396885997911447; dx=-1.0718112000610326E12 evalInputDelta=96.6647233583432
Non-optimal measurement 3.397209170583983 < 0.009120779343008902. Total: 2
Armijo: th(5.937044450043771E-7)=3.397209170583983; dx=-1.0718112000610497E12 evalInputDelta=96.66440018567067
Non-optimal measurement 3.397241819361279 < 0.009120779343008902. Total: 2
Armijo: th(5.397313136403428E-8)=3.397241819361279; dx=-1.0718112000610514E12 evalInputDelta=96.66436753689337
Non-optimal measurement 3.3972448122124477 < 0.009120779343008902. Total: 2
Armijo: th(4.4977609470028565E-9)=3.3972448122124477; dx=-1.0718112000610515E12 evalInputDelta=96.6643645440422
Non-optimal measurement 3.3972450633611535 < 0.009120779343008902. Total: 2
Armijo: th(3.4598161130791205E-10)=3.3972450633611535; dx=-1.0718112000610515E12 evalInputDelta=96.6643642928935
Non-optimal measurement 5.25404970178568 < 0.009120779343008902. Total: 2
Armijo: th(2.4712972236279432E-11)=5.25404970178568; dx=-9.52320001070859E20 evalInputDelta=94.80755965446897
Non-optimal measurement 59.60243846771942 < 0.009120779343008902. Total: 2
Armijo: th(1.6475314824186289E-12)=59.60243846771942; dx=-5.6970752000550645E23 evalInputDelta=40.459170888535226
Non-optimal measurement 100.06160935624841 < 0.009120779343008902. Total: 2
Armijo: th(1.029707176511643E-13)=100.06160935624841; dx=-1.0718111999999999E24 evalInputDelta=6.23856521997368E-12
Non-optimal measurement 100.06160935625428 < 0.009120779343008902. Total: 2
Armijo: th(6.057101038303783E-15)=100.06160935625428; dx=-1.0718111999999999E24 evalInputDelta=3.694822225952521E-13
Non-optimal measurement 0.009120779343008902 < 0.009120779343008902. Total: 2
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=0.009120779343008902
Fitness changed from 100.06160935625465 to 0.009120779343008902
Iteration 1 complete. Error: 0.009120779343008902 Total: 0.0640; Orientation: 0.0036; Line Search: 0.0557
Non-optimal measurement 0.009120779343008902 < 0.009120779343008902. Total: 2
LBFGS Accumulation History: 2 points
Non-optimal measurement 0.009120779343008902 < 0.009120779343008902. Total: 2
th(0)=0.009120779343008902;dx=-2.259344530683688
Adding measurement 7f242338 to history. Total: 2
New Minimum: 0.009120779343008902 > 0.0091207793430089
WOLFE (weak): th(2.154434690031884E-15)=0.0091207793430089; dx=-2.259344530683688 evalInputDelta=1.734723475976807E-18
Non-optimal measurement 0.0091207793430089 < 0.0091207793430089. Total: 3
WOLFE (weak): th(4.308869380063768E-15)=0.0091207793430089; dx=-2.259344530683688 evalInputDelta=1.734723475976807E-18
Adding measurement 26841408 to history. Total: 3
New Minimum: 0.0091207793430089 > 0.009120779343008893
WOLFE (weak): th(1.2926608140191303E-14)=0.009120779343008893; dx=-2.259344530683688 evalInputDelta=8.673617379884035E-18
Adding measurement 2141ed8c to history. Total: 4
New Minimum: 0.009120779343008893 > 0.009120779343008862
WOLFE (weak): th(5.1706432560765214E-14)=0.009120779343008862; dx=-2.259344530683688 evalInputDelta=3.9898639947466563E-17
Adding measurement 271bb709 to history. Total: 5
New Minimum: 0.009120779343008862 > 0.009120779343008697
WOLFE (weak): th(2.5853216280382605E-13)=0.009120779343008697; dx=-2.259344530683688 evalInputDelta=2.0469737016526324E-16
Adding measurement 3e59c369 to history. Total: 6
New Minimum: 0.009120779343008697 > 0.009120779343007663
WOLFE (weak): th(1.5511929768229563E-12)=0.009120779343007663; dx=-2.259344530683688 evalInputDelta=1.2385925618474403E-15
Adding measurement 30064877 to history. Total: 7
New Minimum: 0.009120779343007663 > 0.00912077934300021
WOLFE (weak): th(1.0858350837760695E-11)=0.00912077934300021; dx=-2.2593445306836877 evalInputDelta=8.690964614643804E-15
Adding measurement bac6335 to history. Total: 8
New Minimum: 0.00912077934300021 > 0.009120779342939363
WOLFE (weak): th(8.686680670208556E-11)=0.009120779342939363; dx=-2.259344530683685 evalInputDelta=6.953812525800629E-14
Adding measurement 2c6680f7 to history. Total: 9
New Minimum: 0.009120779342939363 > 0.009120779342383046
WOLFE (weak): th(7.8180126031877E-10)=0.009120779342383046; dx=-2.259344530683657 evalInputDelta=6.258552703863884E-13
Adding measurement 112223fa to history. Total: 10
New Minimum: 0.009120779342383046 > 0.009120779336750342
WOLFE (weak): th(7.818012603187701E-9)=0.009120779336750342; dx=-2.259344530683375 evalInputDelta=6.258559642757788E-12
Adding measurement 34547c27 to history. Total: 11
New Minimum: 0.009120779336750342 > 0.009120779274164754
WOLFE (weak): th(8.599813863506471E-8)=0.009120779274164754; dx=-2.2593445306802438 evalInputDelta=6.884414739671829E-11
Adding measurement 4ad53b0a to history. Total: 12
New Minimum: 0.009120779274164754 > 0.00912077851687914
WOLFE (weak): th(1.0319776636207765E-6)=0.00912077851687914; dx=-2.2593445306423545 evalInputDelta=8.261297618217256E-10
Adding measurement 3c8dd215 to history. Total: 13
New Minimum: 0.00912077851687914 > 0.009120768603325306
WOLFE (weak): th(1.3415709627070094E-5)=0.009120768603325306; dx=-2.259344530146348 evalInputDelta=1.0739683595564764E-8
Adding measurement 5a226abb to history. Total: 14
New Minimum: 0.009120768603325306 > 0.00912062898809456
WOLFE (weak): th(1.878199347789813E-4)=0.00912062898809456; dx=-2.259344523160991 evalInputDelta=1.5035491434181592E-7
Adding measurement 724f1d9d to history. Total: 15
New Minimum: 0.00912062898809456 > 0.009118524167636137
WOLFE (weak): th(0.0028172990216847197)=0.009118524167636137; dx=-2.259344417858072 evalInputDelta=2.2551753727645396E-6
Adding measurement 26b8286d to history. Total: 16
New Minimum: 0.009118524167636137 > 0.009084734625200241
WOLFE (weak): th(0.045076784346955515)=0.009084734625200241; dx=-2.259342729282132 evalInputDelta=3.6044717808660434E-5
Adding measurement 3ebdbf09 to history. Total: 17
New Minimum: 0.009084734625200241 > 0.008518796268298294
WOLFE (weak): th(0.7663053338982437)=0.008518796268298294; dx=-2.2593149711239873 evalInputDelta=6.019830747106079E-4
Adding measurement 737f0c9 to history. Total: 18
New Minimum: 0.008518796268298294 > 7.238703897422067E-4
WOLFE (weak): th(13.793496010168386)=7.238703897422067E-4; dx=-2.259017649716026 evalInputDelta=0.008396908953266695
Adding measurement 1e3ef56e to h
...skipping 8102 bytes...
8524167636137, 0.00912062898809456, 0.009120768603325306, 0.00912077851687914, 0.009120779274164754, 0.009120779336750342, 0.009120779342383046, 0.009120779342939363
Rejected: LBFGS Orientation magnitude: 1.783e+05, gradient 1.503e+00, dot -0.948; [ccd8bfe3-8cba-4b9c-b4cd-31ee5c54b3f6 = 1.000/1.000e+00, 5bc3e4a5-ed87-404f-b2e2-72b3a365ee7a = 1.000/1.000e+00, 4b8018a2-b62b-46a6-aaea-dbb7b4e36b09 = 1.000/1.000e+00, b99fdf6e-c275-49ec-af8c-7c8f05dda30c = 1.000/1.000e+00, f74a425b-caaf-4923-b2b8-a351bd303236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 7.238703897422067E-4, 0.008518796268298294, 0.009084734625200241, 0.009118524167636137, 0.00912062898809456, 0.009120768603325306, 0.00912077851687914, 0.009120779274164754, 0.009120779336750342, 0.009120779342383046
Rejected: LBFGS Orientation magnitude: 1.783e+05, gradient 1.503e+00, dot -0.948; [f74a425b-caaf-4923-b2b8-a351bd303236 = 1.000/1.000e+00, b99fdf6e-c275-49ec-af8c-7c8f05dda30c = 1.000/1.000e+00, 4b8018a2-b62b-46a6-aaea-dbb7b4e36b09 = 1.000/1.000e+00, 5bc3e4a5-ed87-404f-b2e2-72b3a365ee7a = 1.000/1.000e+00, ccd8bfe3-8cba-4b9c-b4cd-31ee5c54b3f6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 7.238703897422067E-4, 0.008518796268298294, 0.009084734625200241, 0.009118524167636137, 0.00912062898809456, 0.009120768603325306, 0.00912077851687914, 0.009120779274164754, 0.009120779336750342
Rejected: LBFGS Orientation magnitude: 1.783e+05, gradient 1.503e+00, dot -0.948; [4b8018a2-b62b-46a6-aaea-dbb7b4e36b09 = 1.000/1.000e+00, b99fdf6e-c275-49ec-af8c-7c8f05dda30c = 1.000/1.000e+00, ccd8bfe3-8cba-4b9c-b4cd-31ee5c54b3f6 = 1.000/1.000e+00, 5bc3e4a5-ed87-404f-b2e2-72b3a365ee7a = 1.000/1.000e+00, f74a425b-caaf-4923-b2b8-a351bd303236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 7.238703897422067E-4, 0.008518796268298294, 0.009084734625200241, 0.009118524167636137, 0.00912062898809456, 0.009120768603325306, 0.00912077851687914, 0.009120779274164754
Rejected: LBFGS Orientation magnitude: 1.783e+05, gradient 1.503e+00, dot -0.948; [5bc3e4a5-ed87-404f-b2e2-72b3a365ee7a = 1.000/1.000e+00, 4b8018a2-b62b-46a6-aaea-dbb7b4e36b09 = 1.000/1.000e+00, b99fdf6e-c275-49ec-af8c-7c8f05dda30c = 1.000/1.000e+00, ccd8bfe3-8cba-4b9c-b4cd-31ee5c54b3f6 = 1.000/1.000e+00, f74a425b-caaf-4923-b2b8-a351bd303236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 7.238703897422067E-4, 0.008518796268298294, 0.009084734625200241, 0.009118524167636137, 0.00912062898809456, 0.009120768603325306, 0.00912077851687914
Rejected: LBFGS Orientation magnitude: 2.493e+05, gradient 1.503e+00, dot -0.957; [5bc3e4a5-ed87-404f-b2e2-72b3a365ee7a = 1.000/1.000e+00, b99fdf6e-c275-49ec-af8c-7c8f05dda30c = 1.000/1.000e+00, ccd8bfe3-8cba-4b9c-b4cd-31ee5c54b3f6 = 1.000/1.000e+00, f74a425b-caaf-4923-b2b8-a351bd303236 = 1.000/1.000e+00, 4b8018a2-b62b-46a6-aaea-dbb7b4e36b09 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 7.238703897422067E-4, 0.008518796268298294, 0.009084734625200241, 0.009118524167636137, 0.00912062898809456, 0.009120768603325306
Rejected: LBFGS Orientation magnitude: 1.665e+05, gradient 1.503e+00, dot -1.000; [4b8018a2-b62b-46a6-aaea-dbb7b4e36b09 = 1.000/1.000e+00, 5bc3e4a5-ed87-404f-b2e2-72b3a365ee7a = 1.000/1.000e+00, f74a425b-caaf-4923-b2b8-a351bd303236 = 1.000/1.000e+00, b99fdf6e-c275-49ec-af8c-7c8f05dda30c = 1.000/1.000e+00, ccd8bfe3-8cba-4b9c-b4cd-31ee5c54b3f6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 7.238703897422067E-4, 0.008518796268298294, 0.009084734625200241, 0.009118524167636137, 0.00912062898809456
Rejected: LBFGS Orientation magnitude: 1.699e+05, gradient 1.503e+00, dot -1.000; [5bc3e4a5-ed87-404f-b2e2-72b3a365ee7a = 1.000/1.000e+00, f74a425b-caaf-4923-b2b8-a351bd303236 = 1.000/1.000e+00, ccd8bfe3-8cba-4b9c-b4cd-31ee5c54b3f6 = 1.000/1.000e+00, 4b8018a2-b62b-46a6-aaea-dbb7b4e36b09 = 1.000/1.000e+00, b99fdf6e-c275-49ec-af8c-7c8f05dda30c = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 7.238703897422067E-4, 0.008518796268298294, 0.009084734625200241, 0.009118524167636137
Rejected: LBFGS Orientation magnitude: 1.765e+05, gradient 1.503e+00, dot -1.000; [5bc3e4a5-ed87-404f-b2e2-72b3a365ee7a = 1.000/1.000e+00, f74a425b-caaf-4923-b2b8-a351bd303236 = 1.000/1.000e+00, b99fdf6e-c275-49ec-af8c-7c8f05dda30c = 1.000/1.000e+00, 4b8018a2-b62b-46a6-aaea-dbb7b4e36b09 = 1.000/1.000e+00, ccd8bfe3-8cba-4b9c-b4cd-31ee5c54b3f6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 7.238703897422067E-4, 0.008518796268298294, 0.009084734625200241
LBFGS Accumulation History: 3 points
Removed measurement 1e3ef56e to history. Total: 19
Removed measurement 737f0c9 to history. Total: 18
Removed measurement 3ebdbf09 to history. Total: 17
Removed measurement 26b8286d to history. Total: 16
Removed measurement 724f1d9d to history. Total: 15
Removed measurement 5a226abb to history. Total: 14
Removed measurement 3c8dd215 to history. Total: 13
Removed measurement 4ad53b0a to history. Total: 12
Removed measurement 34547c27 to history. Total: 11
Removed measurement 112223fa to history. Total: 10
Removed measurement 2c6680f7 to history. Total: 9
Removed measurement bac6335 to history. Total: 8
Removed measurement 30064877 to history. Total: 7
Removed measurement 3e59c369 to history. Total: 6
Removed measurement 271bb709 to history. Total: 5
Removed measurement 2141ed8c to history. Total: 4
Removed measurement 26841408 to history. Total: 3
Adding measurement 2b1ce7d4 to history. Total: 3
th(0)=0.0;dx=-2.2588
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(8736.272358735263)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(4368.1361793676315)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1456.0453931225438)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(364.01134828063596)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(72.8022696561272)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(12.133711609354533)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.7333873727649334)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.21667342159561667)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.024074824621735186)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.0024074824621735185)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2.188620420157744E-4)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.8238503501314535E-5)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.4029618077934257E-6)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.0021155769953041E-7)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(6.680770513302027E-9)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(4.175481570813767E-10)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2.4561656298904512E-11)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.3645364610502506E-12)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(7.181770847632898E-14)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(3.5908854238164494E-15)=0.0; dx=-2.2588 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
MIN ALPHA (1.709945439912595E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 2.7660; Orientation: 2.6740; Line Search: 0.0903
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 2.922s (< 30.000s)
Returns
0.0
TrainingTester.java:432 executed in 0.10 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -3.039968050936114], [2.0, -1.039968050936114]; valueStats=DoubleSummaryStatistics{count=2, sum=0.018242, min=0.009121, average=0.009121, max=0.009121}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -3.039968050936114], [0.183, -1.039968050936114]; valueStats=DoubleSummaryStatistics{count=2, sum=0.018242, min=0.009121, average=0.009121, max=0.009121}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS
Returns
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null} | OK |
{
"result": "OK",
"performance": {
"execution_time": "4.261",
"gc_time": "0.377"
},
"created_on": 1586735017168,
"file_name": "trainingTest",
"report": {
"simpleName": "Expand",
"canonicalName": "com.simiacryptus.mindseye.layers.java.ImgReshapeLayerTest.Expand",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/ImgReshapeLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "Converged",
"value": 0.0
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "Converged",
"value": 0.0
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/ImgReshapeLayer/Expand/trainingTest/202004124337",
"id": "4b1d225d-cc4b-4235-83c1-32f1e382c84f",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "ImgReshapeLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.java.ImgReshapeLayer",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/ImgReshapeLayer.java",
"javaDoc": ""
}
}