1. Test Modules
  2. Training Characteristics
    1. Input Learning
      1. Gradient Descent
      2. Conjugate Gradient Descent
      3. Limited-Memory BFGS
    2. Results
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 3340773470882626560

Training Characteristics

Input Learning

In this apply, we use a network to learn this target input, given it's pre-evaluated output:

TrainingTester.java:332 executed in 0.00 seconds (0.000 gc):

    return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
      try {
        return x.prettyPrint();
      } finally {
        x.freeRef();
      }
    }).reduce((a, b) -> a + "\n" + b).orElse("");

Returns

    [ -0.128, 0.7, 0.08 ]
    [ 0.08, 0.7, -0.128 ]
    [ 0.7, -0.128, 0.08 ]
    [ 0.7, 0.08, -0.128 ]
    [ 0.7, -0.128, 0.08 ]

Gradient Descent

First, we train using basic gradient descent method apply weak line search conditions.

TrainingTester.java:480 executed in 0.32 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
      iterativeTrainer.setOrientation(new GradientDescent());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging 
Reset training subject: 3520985569961
BACKPROP_AGG_SIZE = 3
THREADS = 64
SINGLE_THREADED = false
Initialized CoreSettings = {
"backpropAggregationSize" : 3,
"jvmThreads" : 64,
"singleThreaded" : false
}
Reset training subject: 3521023267295
Constructing line search parameters: GD
th(0)=6.184064070061392;dx=-2.0112E22
New Minimum: 6.184064070061392 > 0.013857133690187762
Armijo: th(2.154434690031884)=0.013857133690187762; dx=-2.011200000098146E10 evalInputDelta=6.170206936371204
Armijo: th(1.077217345015942)=0.015461039355567377; dx=-2.0112000000981506E10 evalInputDelta=6.168603030705825
Armijo: th(0.3590724483386473)=0.20963720127504937; dx=-2.0112000001384987E10 evalInputDelta=5.974426868786342
Armijo: th(0.08976811208466183)=0.8229866535748414; dx=-2.0112000004134937E10 evalInputDelta=5.36107741648655
Armijo: th(0.017953622416932366)=1.245989223351429; dx=-2.0112000008797203E10 evalInputDelta=4.938074846709963
Armijo: th(0.002992270402822061)=1.396018182207317; dx=-2.0112000011498985E10 evalInputDelta=4.788045887854075
Armijo: th(4.2746720040315154E-4)=1.4263156260963203; dx=-2.011200001213808E10 evalInputDelta=4.757748443965071
Armijo: th(5.343340005039394E-5)=1.4308741774672042; dx=-2.0112000012237263E10 evalInputDelta=4.753189892594188
Armijo: th(5.9370444500437714E-6)=1.4314557047441006; dx=-2.0112000012249973E10 evalInputDelta=4.7526083653172915
Armijo: th(5.937044450043771E-7)=1.431521164358917; dx=-2.0112000012251404E10 evalInputDelta=4.752542905702475
Armijo: th(5.397313136403428E-8)=1.4315277768666186; dx=-2.011200001225155E10 evalInputDelta=4.7525362931947726
Armijo: th(4.4977609470028565E-9)=1.4315283830170638; dx=-2.0112000012251564E10 evalInputDelta=4.752535687044328
Armijo: th(3.4598161130791205E-10)=1.431528433882865; dx=-2.0112000012251564E10 evalInputDelta=4.752535636178527
Armijo: th(2.4712972236279432E-11)=1.4737358412826544; dx=-2.1514680390319595E10 evalInputDelta=4.710328228778737
Armijo: th(1.6475314824186289E-12)=2.634415321656895; dx=-5.1200000019093186E20 evalInputDelta=3.5496487484044965
Armijo: th(1.029707176511643E-13)=6.1840640700601295; dx=-2.0112E22 evalInputDelta=1.262101534393878E-12
Armijo: th(6.057101038303783E-15)=6.184064070061318; dx=-2.0112E22 evalInputDelta=7.37188088351104E-14
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=0.013857133690187762
Fitness changed from 6.184064070061392 to 0.013857133690187762
Iteration 1 complete. Error: 0.013857133690187762 Total: 0.1427; Orientation: 0.0037; Line Search: 0.0915
th(0)=0.013857133690187762;dx=-0.09988008920388129
WOLFE (weak): th(2.154434690031884E-15)=0.013857133690187762; dx=-0.09988008920388129 evalInputDelta=0.0
New Minimum: 0.013857133690187762 > 0.013857133690187751
WOLFE (weak): th(4.308869380063768E-15)=0.013857133690187751; dx=-0.09988008920388129 evalInputDelta=1.0408340855860843E-17
New Minimum: 0.013857133690187751 > 0.013857133690187744
WOLFE (weak): th(1.2926608140191303E-14)=0.013857133690187744; dx=-0.09988008920388129 evalInputDelta=1.734723475976807E-17
New Minimum: 0.013857133690187744 > 0.013857133690187687
WOLFE (weak): th(5.1706432560765214E-14)=0.013857133690187687; dx=-0.09988008920388129 evalInputDelta=7.45931094670027E-17
New Minimum: 0.013857133690187687 > 0.013857133690187401
WOLFE (weak): th(2.5853216280382605E-13)=0.013857133690187401; dx=-0.09988008920388128 evalInputDelta=3.608224830031759E-16
New Minimum: 0.013857133690187401 > 0.013857133690185638
WOLFE (weak): th(1.5511929768229563E-12)=0.013857133690185638; dx=-0.09988008920388124 evalInputDelta=2.123301534595612E-15
New Minimum: 0.013857133690185638 > 0.013857133690172899
WOLFE (weak): th(1.0858350837760695E-11)=0.013857133690172899; dx=-0.09988008920388086 evalInputDelta=1.4863110742169283E-14
New Minimum: 0.013857133690172899 > 0.013857133690068912
WOLFE (weak): th(8.686680670208556E-11)=0.013857133690068912; dx=-0.09988008920387792 evalInputDelta=1.18849374786123E-13
New Minimum: 0.013857133690068912 > 0.01385713368911819
WOLFE (weak): th(7.8180126031877E-10)=0.01385713368911819; dx=-0.0998800892038508 evalInputDelta=1.069571514689116E-12
New Minimum: 0.01385713368911819 > 0.013857133679492017
WOLFE (weak): th(7.818012603187701E-9)=0.013857133679492017; dx=-0.09988008920357644 evalInputDelta=1.0695744637190252E-11
New Minimum: 0.013857133679492017 > 0.01385713357253463
WOLFE (weak): th(8.599813863506471E-8)=0.01385713357253463; dx=-0.09988008920052796 evalInputDelta=1.176531320284946E-10
New Minimum: 0.01385713357253463 > 0.013857132278350285
WOLFE (weak): th(1.0319776636207765E-6)=0.013857132278350285; dx=-0.0998800891636413 evalInputDelta=1.4118374767890796E-9
New Minimum: 0.013857132278350285 > 0.013857115336303762
WOLFE (weak): th(1.3415709627070094E-5)=0.013857115336303762; dx=-0.09988008868076159 evalInputDelta=1.8353883999427945E-8
New Minimum: 0.013857115336303762 > 0.013856876736450478
WOLFE (weak): th(1.878199347789813E-4)=0.013856876736450478; dx=-0.09988008188024186 evalInputDelta=2.569537372841546E-7
New Minimum: 0.013856876736450478 > 0.01385327952855125
WOLFE (weak): th(0.0028172990216847197)=0.01385327952855125; dx=-0.09987997935752214 evalInputDelta=3.854161636512518E-6
New Minimum: 0.01385327952855125 > 0.013795504209624467
WOLFE (weak): th(0.045076784346955515)=0.013795504209624467; dx=-0.09987833377633171 evalInputDelta=6.162948056329456E-5
New Minimum: 0.013795504209624467 > 0.012820044451230955
WOLFE (weak): th(0.7663053338982437)=0.012820044451230955; dx=-0.09985084727175175 evalInputDelta=0.0010370892389568072
New Minimum: 0.012820044451230955 > 0.0
WOLFE (weak): th(13.793496010168386)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
WOLFE (weak): th(262.07642419319933)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
WOLFE (weak): th(5241.528483863986)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
WOLFE (weak): th(110072.09816114372)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
Armijo: th(2421586.1595451618)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
Armijo: th(1265829.1288531527)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
Armijo: th(687950.6135071482)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
Armijo: th(399011.355834146)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
Armijo: th(254541.72699764484)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
Armijo: th(182306.9125793943)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
Armijo: th(146189.505370269)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
WOLFE (weak): th(128130.80176570636)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
WOLFE (weak): th(137160.15356798767)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
Armijo: th(141674.82946912834)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
Armijo: th(139417.491518558)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
WOLFE (weak): th(138288.82254327284)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
mu ~= nu (138288.82254327284): th(13.793496010168386)=0.0
Fitness changed from 0.013857133690187762 to 0.0
Iteration 2 complete. Error: 0.0 Total: 0.1163; Orientation: 0.0011; Line Search: 0.1116
th(0)=0.0;dx=-0.09927999999999998
Armijo: th(299150.0583278488)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(149575.0291639244)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(49858.34305464147)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(12464.585763660367)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(2492.917152732073)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(415.4861921220122)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(59.3551703031446)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(7.419396287893075)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(0.8243773653214528)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(0.08243773653214528)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(0.0074943396847404805)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(6.245283070617067E-4)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(4.804063900474667E-5)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(3.4314742146247626E-6)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(2.2876494764165085E-7)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(1.4297809227603178E-8)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(8.410476016237163E-10)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(4.672486675687313E-11)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(2.4592035135196385E-12)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(1.2296017567598192E-13)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Armijo: th(5.855246460761044E-15)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
MIN ALPHA (2.6614756639822927E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0528; Orientation: 0.0008; Line Search: 0.0503
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 0.312s (< 30.000s)

Returns

    0.0

Training Converged

Conjugate Gradient Descent

First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.

TrainingTester.java:452 executed in 0.12 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
      iterativeTrainer.setOrientation(new GradientDescent());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging 
Reset training subject: 3521305392007
Reset training subject: 3521307378405
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=6.184064070061392}, derivative=-2.0112E22}
New Minimum: 6.184064070061392 > 1.4315284368965253
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=1.4315284368965253}, derivative=-2.0112000012251564E10}, evalInputDelta = -4.752535633164866
New Minimum: 1.4315284368965253 > 1.431528429545585
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=1.431528429545585}, derivative=-2.0112000012251564E10}, evalInputDelta = -4.752535640515807
New Minimum: 1.431528429545585 > 1.4315283780890051
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=1.4315283780890051}, derivative=-2.0112000012251564E10}, evalInputDelta = -4.752535691972387
New Minimum: 1.4315283780890051 > 1.4315280178930796
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=1.4315280178930796}, derivative=-2.0112000012251556E10}, evalInputDelta = -4.7525360521683115
New Minimum: 1.4315280178930796 > 1.4315254965280861
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=1.4315254965280861}, derivative=-2.0112000012251503E10}, evalInputDelta = -4.752538573533306
New Minimum: 1.4315254965280861 > 1.4315078472909053
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=1.4315078472909053}, derivative=-2.0112000012251114E10}, evalInputDelta = -4.752556222770487
New Minimum: 1.4315078472909053 > 1.4313843181989865
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=1.4313843181989865}, derivative=-2.0112000012248413E10}, evalInputDelta = -4.752679751862405
New Minimum: 1.4313843181989865 > 1.430520376492149
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=1.430520376492149}, derivative=-2.0112000012229534E10}, evalInputDelta = -4.753543693569243
New Minimum: 1.430520376492149 > 1.4245098094622786
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=1.4245098094622786}, derivative=-2.0112000012099007E10}, evalInputDelta = -4.7595542605991135
New Minimum: 1.4245098094622786 > 1.384150308357511
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=1.384150308357511}, derivative=-2.0112000011257927E10}, evalInputDelta = -4.7999137617038805
New Minimum: 1.384150308357511 > 1.162046882190105
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=1.162046882190105}, derivative=-2.0112000007573036E10}, evalInputDelta = -5.0220171878712865
New Minimum: 1.162046882190105 > 0.4946260993553825
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=0.4946260993553825}, derivative=-2.011200000230185E10}, evalInputDelta = -5.689437970706009
New Minimum: 0.4946260993553825 > 0.014998582952795306
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=0.014998582952795306}, derivative=-2.0112000000981495E10}, evalInputDelta = -6.169065487108596
New Minimum: 0.014998582952795306 > 0.003926172688971422
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=0.003926172688971422}, derivative=-2.0112000000981186E10}, evalInputDelta = -6.18013789737242
New Minimum: 0.003926172688971422 > 0.0
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.0112000000981094E10}, evalInputDelta = -6.184064070061392
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.011200000098109E10}, evalInputDelta = -6.184064070061392
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.0112000000981094E10}, evalInputDelta = -6.184064070061392
F(23263.0513987207) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.011200000098109E10}, evalInputDelta = -6.184064070061392
F(162841.3597910449) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.0112000000981094E10}, evalInputDelta = -6.184064070061392
F(1139889.5185373144) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.0112000000981094E10}, evalInputDelta = -6.184064070061392
F(7979226.6297612) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.0112000000981094E10}, evalInputDelta = -6.184064070061392
F(5.58545864083284E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.0112000000981094E10}, evalInputDelta = -6.184064070061392
F(3.909821048582988E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.0112000000981094E10}, evalInputDelta = -6.184064070061392
F(2.7368747340080914E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.0112000000981094E10}, evalInputDelta = -6.184064070061392
F(1.915812313805664E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.0112000000981094E10}, evalInputDelta = -6.184064070061392
0.0 <= 6.184064070061392
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-2.0112000000981094E10}, evalInputDelta = -6.184064070061392
Right bracket at 1.0E10
Converged to right
Fitness changed from 6.184064070061392 to 0.0
Iteration 1 complete. Error: 0.0 Total: 0.0725; Orientation: 0.0007; Line Search: 0.0663
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
0.0 <= 0.0
F(5.0E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Right bracket at 5.0E9
F(2.5E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Right bracket at 2.5E9
F(1.25E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Right bracket at 1.25E9
F(6.25E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Right bracket at 6.25E8
F(3.125E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Right bracket at 3.125E8
F(1.5625E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Right bracket at 1.5625E8
F(7.8125E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Right bracket at 7.8125E7
F(3.90625E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Right bracket at 3.90625E7
F(1.953125E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Right bracket at 1.953125E7
F(9765625.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Right bracket at 9765625.0
F(4882812.5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Right bracket at 4882812.5
F(2441406.25) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.09927999999999998}, evalInputDelta = 0.0
Loops = 12
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0449; Orientation: 0.0006; Line Search: 0.0427
Iteration 2 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 2
Final threshold in iteration 2: 0.0 (> 0.0) after 0.117s (< 30.000s)

Returns

    0.0

Training Converged

Limited-Memory BFGS

Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.

TrainingTester.java:509 executed in 0.61 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
      iterativeTrainer.setOrientation(new LBFGS());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setIterationsPerSample(100);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging 
Reset training subject: 3521428341080
Reset training subject: 3521430611055
Adding measurement 240faead to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 6.184064070061392 < 6.184064070061392. Total: 1
th(0)=6.184064070061392;dx=-2.0112E22
Adding measurement 31859ae2 to history. Total: 1
New Minimum: 6.184064070061392 > 0.013857133690187762
Armijo: th(2.154434690031884)=0.013857133690187762; dx=-2.011200000098146E10 evalInputDelta=6.170206936371204
Non-optimal measurement 0.015461039355567377 < 0.013857133690187762. Total: 2
Armijo: th(1.077217345015942)=0.015461039355567377; dx=-2.0112000000981506E10 evalInputDelta=6.168603030705825
Non-optimal measurement 0.20963720127504937 < 0.013857133690187762. Total: 2
Armijo: th(0.3590724483386473)=0.20963720127504937; dx=-2.0112000001384987E10 evalInputDelta=5.974426868786342
Non-optimal measurement 0.8229866535748414 < 0.013857133690187762. Total: 2
Armijo: th(0.08976811208466183)=0.8229866535748414; dx=-2.011200000413494E10 evalInputDelta=5.36107741648655
Non-optimal measurement 1.245989223351429 < 0.013857133690187762. Total: 2
Armijo: th(0.017953622416932366)=1.245989223351429; dx=-2.0112000008797203E10 evalInputDelta=4.938074846709963
Non-optimal measurement 1.396018182207317 < 0.013857133690187762. Total: 2
Armijo: th(0.002992270402822061)=1.396018182207317; dx=-2.011200001149899E10 evalInputDelta=4.788045887854075
Non-optimal measurement 1.4263156260963203 < 0.013857133690187762. Total: 2
Armijo: th(4.2746720040315154E-4)=1.4263156260963203; dx=-2.0112000012138077E10 evalInputDelta=4.757748443965071
Non-optimal measurement 1.4308741774672042 < 0.013857133690187762. Total: 2
Armijo: th(5.343340005039394E-5)=1.4308741774672042; dx=-2.0112000012237263E10 evalInputDelta=4.753189892594188
Non-optimal measurement 1.4314557047441006 < 0.013857133690187762. Total: 2
Armijo: th(5.9370444500437714E-6)=1.4314557047441006; dx=-2.0112000012249973E10 evalInputDelta=4.7526083653172915
Non-optimal measurement 1.431521164358917 < 0.013857133690187762. Total: 2
Armijo: th(5.937044450043771E-7)=1.431521164358917; dx=-2.0112000012251404E10 evalInputDelta=4.752542905702475
Non-optimal measurement 1.4315277768666186 < 0.013857133690187762. Total: 2
Armijo: th(5.397313136403428E-8)=1.4315277768666186; dx=-2.011200001225155E10 evalInputDelta=4.7525362931947726
Non-optimal measurement 1.4315283830170638 < 0.013857133690187762. Total: 2
Armijo: th(4.4977609470028565E-9)=1.4315283830170638; dx=-2.0112000012251564E10 evalInputDelta=4.752535687044328
Non-optimal measurement 1.431528433882865 < 0.013857133690187762. Total: 2
Armijo: th(3.4598161130791205E-10)=1.431528433882865; dx=-2.0112000012251564E10 evalInputDelta=4.752535636178527
Non-optimal measurement 1.4737358412826544 < 0.013857133690187762. Total: 2
Armijo: th(2.4712972236279432E-11)=1.4737358412826544; dx=-2.1514680390319595E10 evalInputDelta=4.710328228778737
Non-optimal measurement 2.634415321656895 < 0.013857133690187762. Total: 2
Armijo: th(1.6475314824186289E-12)=2.634415321656895; dx=-5.1200000019093186E20 evalInputDelta=3.5496487484044965
Non-optimal measurement 6.1840640700601295 < 0.013857133690187762. Total: 2
Armijo: th(1.029707176511643E-13)=6.1840640700601295; dx=-2.0112E22 evalInputDelta=1.262101534393878E-12
Non-optimal measurement 6.184064070061318 < 0.013857133690187762. Total: 2
Armijo: th(6.057101038303783E-15)=6.184064070061318; dx=-2.0112E22 evalInputDelta=7.37188088351104E-14
Non-optimal measurement 0.013857133690187762 < 0.013857133690187762. Total: 2
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=0.013857133690187762
Fitness changed from 6.184064070061392 to 0.013857133690187762
Iteration 1 complete. Error: 0.013857133690187762 Total: 0.0585; Orientation: 0.0033; Line Search: 0.0501
Non-optimal measurement 0.013857133690187762 < 0.013857133690187762. Total: 2
LBFGS Accumulation History: 2 points
Non-optimal measurement 0.013857133690187762 < 0.013857133690187762. Total: 2
th(0)=0.013857133690187762;dx=-0.09988008920388129
Non-optimal measurement 0.013857133690187762 < 0.013857133690187762. Total: 2
WOLFE (weak): th(2.154434690031884E-15)=0.013857133690187762; dx=-0.09988008920388129 evalInputDelta=0.0
Adding measurement 21ecdea6 to history. Total: 2
New Minimum: 0.013857133690187762 > 0.013857133690187751
WOLFE (weak): th(4.308869380063768E-15)=0.013857133690187751; dx=-0.09988008920388129 evalInputDelta=1.0408340855860843E-17
Adding measurement 3a3ee543 to history. Total: 3
New Minimum: 0.013857133690187751 > 0.013857133690187744
WOLFE (weak): th(1.2926608140191303E-14)=0.013857133690187744; dx=-0.09988008920388129 evalInputDelta=1.734723475976807E-17
Adding measurement 7a32c70 to history. Total: 4
New Minimum: 0.013857133690187744 > 0.013857133690187687
WOLFE (weak): th(5.1706432560765214E-14)=0.013857133690187687; dx=-0.09988008920388129 evalInputDelta=7.45931094670027E-17
Adding measurement 469e6bac to history. Total: 5
New Minimum: 0.013857133690187687 > 0.013857133690187401
WOLFE (weak): th(2.5853216280382605E-13)=0.013857133690187401; dx=-0.09988008920388128 evalInputDelta=3.608224830031759E-16
Adding measurement e975a6d to history. Total: 6
New Minimum: 0.013857133690187401 > 0.013857133690185638
WOLFE (weak): th(1.5511929768229563E-12)=0.013857133690185638; dx=-0.09988008920388124 evalInputDelta=2.123301534595612E-15
Adding measurement 65f5e102 to history. Total: 7
New Minimum: 0.013857133690185638 > 0.013857133690172899
WOLFE (weak): th(1.0858350837760695E-11)=0.013857133690172899; dx=-0.09988008920388086 evalInputDelta=1.4863110742169283E-14
Adding measurement 479bd527 to history. Total: 8
New Minimum: 0.013857133690172899 > 0.013857133690068912
WOLFE (weak): th(8.686680670208556E-11)=0.013857133690068912; dx=-0.09988008920387792 evalInputDelta=1.18849374786123E-13
Adding measurement 69ce309e to history. Total: 9
New Minimum: 0.013857133690068912 > 0.01385713368911819
WOLFE (weak): th(7.8180126031877E-10)=0.01385713368911819; dx=-0.0998800892038508 evalInputDelta=1.069571514689116E-12
Adding measurement 15aa8674 to history. Total: 10
New Minimum: 0.01385713368911819 > 0.013857133679492017
WOLFE (weak): th(7.818012603187701E-9)=0.013857133679492017; dx=-0.09988008920357644 evalInputDelta=1.0695744637190252E-11
Adding measurement b5e9384 to history. Total: 11
New Minimum: 0.013857133679492017 > 0.01385713357253463
WOLFE (weak): th(8.599813863506471E-8)=0.01385713357253463; dx=-0.09988008920052796 evalInputDelta=1.176531320284946E-10
Adding measurement 3d01e60d to history. Total: 12
New Minimum: 0.01385713357253463 > 0.013857132278350285
WOLFE (weak): th(1.0319776636207765E-6)=0.013857132278350285; dx=-0.0998800891636413 evalInputDelta=1.4118374767890796E-9
Adding measurement 2ee8877a to history. Total: 13
New Minimum: 0.013857132278350285 > 0.013857115336303762
WOLFE (weak): th(1.3415709627070094E-5)=0.013857115336303762; dx=-0.09988008868076159 evalInputDelta=1.8353883999427945E-8
Adding measurement 691e9a4a to history. Total: 14
New Minimum: 0.013857115336303762 > 0.013856876736450478
WOLFE (weak): th(1.878199347789813E-4)=0.013856876736450478; dx=-0.09988008188024186 evalInputDelta=2.569537372841546E-7
Adding measurement 1d30d0d6 to history. Total: 15
New Minimum: 0.013856876736450478 > 0.01385327952855125
WOLFE (weak): th(0.0028172990216847197)=0.01385327952855125; dx=-0.09987997935752214 evalInputDelta=3.854161636512518E-6
Adding measurement 2d6f54af to history. Total: 16
New Minimum: 0.01385327952855125 > 0.013795504209624467
WOLFE (weak): th(0.045076784346955515)=0.013795504209624467; dx=-0.09987833377633171 evalInputDelta=6.162948056329456E-5
Adding measurement 216caf81 to history. Total: 17
New Minimum: 0.013795504209624467 > 0.012820044451230955
WOLFE (weak): th(0.7663053338982437)=0.012820044451230955; dx=-0.09985084727175175 evalInputDelta=0.0010370892389568072
Adding measurement 56b778cb to history. Total: 18
New Minimum: 0.012820044451230955 > 0.0
WOLFE (weak): th(13.793496010168386)=0.0; dx=-0.09953703293048605 evalInputDelta=0.013857133690187762
Non-optimal measurement 0.0 < 0.0. Total: 19
WOLFE (weak): th(262.07642419319933)

...skipping 7751 bytes...

65535f11 = 1.000/1.000e+00, 32594036-4af5-4262-979c-134a1f5722e5 = 1.000/1.000e+00, 7402034d-c289-4ed5-adb6-9cf5ab5df121 = 1.000/1.000e+00, 67f5b34c-b25c-44bd-b99e-0a8caa044d40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.012820044451230955, 0.013795504209624467, 0.01385327952855125, 0.013856876736450478, 0.013857115336303762, 0.013857132278350285, 0.01385713357253463, 0.013857133679492017, 0.01385713368911819, 0.013857133690068912
Rejected: LBFGS Orientation magnitude: 1.361e+03, gradient 3.151e-01, dot -0.989; [32594036-4af5-4262-979c-134a1f5722e5 = 1.000/1.000e+00, e17dcc5c-0c9a-40e3-b240-77d99f1895f6 = 1.000/1.000e+00, 67f5b34c-b25c-44bd-b99e-0a8caa044d40 = 1.000/1.000e+00, 8c5978fb-731f-4eb5-a2b5-2d4165535f11 = 1.000/1.000e+00, 7402034d-c289-4ed5-adb6-9cf5ab5df121 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.012820044451230955, 0.013795504209624467, 0.01385327952855125, 0.013856876736450478, 0.013857115336303762, 0.013857132278350285, 0.01385713357253463, 0.013857133679492017, 0.01385713368911819
Rejected: LBFGS Orientation magnitude: 1.361e+03, gradient 3.151e-01, dot -0.989; [8c5978fb-731f-4eb5-a2b5-2d4165535f11 = 1.000/1.000e+00, e17dcc5c-0c9a-40e3-b240-77d99f1895f6 = 1.000/1.000e+00, 67f5b34c-b25c-44bd-b99e-0a8caa044d40 = 1.000/1.000e+00, 32594036-4af5-4262-979c-134a1f5722e5 = 1.000/1.000e+00, 7402034d-c289-4ed5-adb6-9cf5ab5df121 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.012820044451230955, 0.013795504209624467, 0.01385327952855125, 0.013856876736450478, 0.013857115336303762, 0.013857132278350285, 0.01385713357253463, 0.013857133679492017
Rejected: LBFGS Orientation magnitude: 1.361e+03, gradient 3.151e-01, dot -0.989; [67f5b34c-b25c-44bd-b99e-0a8caa044d40 = 1.000/1.000e+00, 7402034d-c289-4ed5-adb6-9cf5ab5df121 = 1.000/1.000e+00, e17dcc5c-0c9a-40e3-b240-77d99f1895f6 = 1.000/1.000e+00, 8c5978fb-731f-4eb5-a2b5-2d4165535f11 = 1.000/1.000e+00, 32594036-4af5-4262-979c-134a1f5722e5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.012820044451230955, 0.013795504209624467, 0.01385327952855125, 0.013856876736450478, 0.013857115336303762, 0.013857132278350285, 0.01385713357253463
Rejected: LBFGS Orientation magnitude: 1.361e+03, gradient 3.151e-01, dot -0.989; [67f5b34c-b25c-44bd-b99e-0a8caa044d40 = 1.000/1.000e+00, e17dcc5c-0c9a-40e3-b240-77d99f1895f6 = 1.000/1.000e+00, 32594036-4af5-4262-979c-134a1f5722e5 = 1.000/1.000e+00, 8c5978fb-731f-4eb5-a2b5-2d4165535f11 = 1.000/1.000e+00, 7402034d-c289-4ed5-adb6-9cf5ab5df121 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.012820044451230955, 0.013795504209624467, 0.01385327952855125, 0.013856876736450478, 0.013857115336303762, 0.013857132278350285
Rejected: LBFGS Orientation magnitude: 1.361e+03, gradient 3.151e-01, dot -0.989; [67f5b34c-b25c-44bd-b99e-0a8caa044d40 = 1.000/1.000e+00, 32594036-4af5-4262-979c-134a1f5722e5 = 1.000/1.000e+00, e17dcc5c-0c9a-40e3-b240-77d99f1895f6 = 1.000/1.000e+00, 8c5978fb-731f-4eb5-a2b5-2d4165535f11 = 1.000/1.000e+00, 7402034d-c289-4ed5-adb6-9cf5ab5df121 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.012820044451230955, 0.013795504209624467, 0.01385327952855125, 0.013856876736450478, 0.013857115336303762
Rejected: LBFGS Orientation magnitude: 1.607e+03, gradient 3.151e-01, dot -1.000; [7402034d-c289-4ed5-adb6-9cf5ab5df121 = 1.000/1.000e+00, e17dcc5c-0c9a-40e3-b240-77d99f1895f6 = 1.000/1.000e+00, 32594036-4af5-4262-979c-134a1f5722e5 = 1.000/1.000e+00, 8c5978fb-731f-4eb5-a2b5-2d4165535f11 = 1.000/1.000e+00, 67f5b34c-b25c-44bd-b99e-0a8caa044d40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.012820044451230955, 0.013795504209624467, 0.01385327952855125, 0.013856876736450478
Rejected: LBFGS Orientation magnitude: 1.610e+03, gradient 3.151e-01, dot -1.000; [e17dcc5c-0c9a-40e3-b240-77d99f1895f6 = 1.000/1.000e+00, 7402034d-c289-4ed5-adb6-9cf5ab5df121 = 1.000/1.000e+00, 67f5b34c-b25c-44bd-b99e-0a8caa044d40 = 1.000/1.000e+00, 8c5978fb-731f-4eb5-a2b5-2d4165535f11 = 1.000/1.000e+00, 32594036-4af5-4262-979c-134a1f5722e5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.012820044451230955, 0.013795504209624467, 0.01385327952855125
LBFGS Accumulation History: 3 points
Removed measurement 56b778cb to history. Total: 18
Removed measurement 216caf81 to history. Total: 17
Removed measurement 2d6f54af to history. Total: 16
Removed measurement 1d30d0d6 to history. Total: 15
Removed measurement 691e9a4a to history. Total: 14
Removed measurement 2ee8877a to history. Total: 13
Removed measurement 3d01e60d to history. Total: 12
Removed measurement b5e9384 to history. Total: 11
Removed measurement 15aa8674 to history. Total: 10
Removed measurement 69ce309e to history. Total: 9
Removed measurement 479bd527 to history. Total: 8
Removed measurement 65f5e102 to history. Total: 7
Removed measurement e975a6d to history. Total: 6
Removed measurement 469e6bac to history. Total: 5
Removed measurement 7a32c70 to history. Total: 4
Removed measurement 3a3ee543 to history. Total: 3
Adding measurement 457fe8da to history. Total: 3
th(0)=0.0;dx=-0.09927999999999998
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(299150.0583278488)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(149575.0291639244)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(49858.34305464147)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(12464.585763660367)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2492.917152732073)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(415.4861921220122)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(59.3551703031446)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(7.419396287893075)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.8243773653214528)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.08243773653214528)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.0074943396847404805)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(6.245283070617067E-4)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(4.804063900474667E-5)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(3.4314742146247626E-6)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2.2876494764165085E-7)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.4297809227603178E-8)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(8.410476016237163E-10)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(4.672486675687313E-11)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2.4592035135196385E-12)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.2296017567598192E-13)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(5.855246460761044E-15)=0.0; dx=-0.09927999999999998 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
MIN ALPHA (2.6614756639822927E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.4764; Orientation: 0.4414; Line Search: 0.0337
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 0.608s (< 30.000s)

Returns

    0.0

Training Converged

TrainingTester.java:432 executed in 0.11 seconds (0.000 gc):

    return TestUtil.compare(title + " vs Iteration", runs);
Logging 
Plotting range=[1.0, -2.8583265930481696], [2.0, -0.8583265930481698]; valueStats=DoubleSummaryStatistics{count=2, sum=0.027714, min=0.013857, average=0.013857, max=0.013857}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS

Returns

Result

TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):

    return TestUtil.compareTime(title + " vs Time", runs);
Logging 
Plotting range=[0.0, -2.8583265930481696], [0.116, -0.8583265930481698]; valueStats=DoubleSummaryStatistics{count=2, sum=0.027714, min=0.013857, average=0.013857, max=0.013857}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS

Returns

Result

Results

TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):

    return grid(inputLearning, modelLearning, completeLearning);

Returns

Result

TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):

    return new ComponentResult(null == inputLearning ? null : inputLearning.value,
        null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);

Returns

    {"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

detailsresult
{"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}OK
 
  {
    "result": "OK",
    "performance": {
      "execution_time": "1.764",
      "gc_time": "0.319"
    },
    "created_on": 1586738108787,
    "file_name": "trainingTest",
    "report": {
      "simpleName": "Basic",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.MonitoringSynapseTest.Basic",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/MonitoringSynapseTest.java",
      "javaDoc": ""
    },
    "training_analysis": {
      "input": {
        "LBFGS": {
          "type": "Converged",
          "value": 0.0
        },
        "CjGD": {
          "type": "Converged",
          "value": 0.0
        },
        "GD": {
          "type": "Converged",
          "value": 0.0
        }
      }
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/MonitoringSynapse/Basic/trainingTest/202004133508",
    "id": "cd81dcac-b3d8-49d2-ac98-6b2ba9cec7b4",
    "report_type": "Components",
    "display_name": "Comparative Training",
    "target": {
      "simpleName": "MonitoringSynapse",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.MonitoringSynapse",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/MonitoringSynapse.java",
      "javaDoc": ""
    }
  }