1. Test Modules
  2. Training Characteristics
    1. Input Learning
      1. Gradient Descent
      2. Conjugate Gradient Descent
      3. Limited-Memory BFGS
    2. Results
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 7576217305891054592

Training Characteristics

Input Learning

In this apply, we use a network to learn this target input, given it's pre-evaluated output:

TrainingTester.java:332 executed in 0.01 seconds (0.000 gc):

    return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
      try {
        return x.prettyPrint();
      } finally {
        x.freeRef();
      }
    }).reduce((a, b) -> a + "\n" + b).orElse("");

Returns

    [
    	[ [ 0.7, -0.128 ], [ 0.048, 0.08 ] ],
    	[ [ 0.496, -0.608 ], [ 1.524, 1.764 ] ]
    ]
    [
    	[ [ 1.048, 0.028, -0.712 ], [ -0.852, -1.688, -0.804 ] ],
    	[ [ -1.028, 1.108, -1.72 ], [ -0.384, 1.208, 1.912 ] ]
    ]
    [
    	[ [ 0.392, 1.556, 0.788, 1.64 ], [ 1.512, 0.3, -1.616, 1.612 ] ],
    	[ [ 1.556, 1.032, -0.768, -0.068 ], [ 0.636, 0.092, 1.356, -0.176 ] ]
    ]
    [
    	[ [ 1.764, 0.7 ], [ 0.496, 0.048 ] ],
    	[ [ -0.608, 1.524 ], [ 0.08, -0.128 ] ]
    ]
    [
    	[ [ -1.72, -0.852, -0.712 ], [ 1.912, 0.028, -0.804 ] ],
    	[ [ -0.384, -1.028, 1.048 ], [ 1.108, -1.688, 1.208 ] ]
    ]
    [
    	[ [ 1.032, 1.612, -0.068, -0.176 ], [ 1.556, 0.636, 1.512, 0.3 ] ],
    	[ [ 0.788, -0.768, 1.64, 0.392 ], [ 1.356, 0.092, 1.556, -1.616 ] ]
    ]
    [
    	[ [ 0.496, 0.08 ], [ -0.608, 1.524 ] ],
    	[ [ -0.128, 0.048 ], [ 0.7, 1.764 ] ]
    ]
    [
    	[ [ -1.72, -1.688, -0.852 ], [ -0.804, 0.028, -0.384 ] ],
    	[ [ 1.912, -1.028, 1.108 ], [ 1.208, 1.048, -0.712 ] ]
    ]
    [
    	[ [ 1.612, -0.176, 1.512, 1.032 ], [ 0.3, 0.636, -1.616, 1.356 ] ],
    	[ [ 0.092, 1.556, -0.768, 0.788 ], [ 1.556, 0.392, 1.64, -0.068 ] ]
    ]
    [
    	[ [ 0.08, 1.524 ], [ -0.128, 1.764 ] ],
    	[ [ 0.7, 0.496 ], [ 0.048, -0.608 ] ]
    ]
    [
    	[ [ -1.72, 0.028, 1.048 ], [ 1.108, -0.384, -1.688 ] ],
    	[ [ 1.208, -0.852, 1.912 ], [ -0.804, -1.028, -0.712 ] ]
    ]
    [
    	[ [ -0.176, 1.556, -0.768, 0.636 ], [ 0.392, -0.068, 1.032, 1.64 ] ],
    	[ [ 0.788, 1.356, 0.092, 1.556 ], [ 1.612, 0.3, 1.512, -1.616 ] ]
    ]
    [
    	[ [ 0.7, 1.524 ], [ 0.08, 0.496 ] ],
    	[ [ -0.128, -0.608 ], [ 1.764, 0.048 ] ]
    ]
    [
    	[ [ -0.804, -0.384, 0.028 ], [ -0.852, -1.028, -0.712 ] ],
    	[ [ 1.208, -1.688, -1.72 ], [ 1.108, 1.048, 1.912 ] ]
    ]
    [
    	[ [ 1.032, 0.392, 1.356, 0.636 ], [ 1.556, -0.768, 0.092, 1.512 ] ],
    	[ [ 0.788, 1.64, -1.616, 1.612 ], [ 1.556, -0.176, -0.068, 0.3 ] ]
    ]

Gradient Descent

First, we train using basic gradient descent method apply weak line search conditions.

TrainingTester.java:480 executed in 1.62 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
      iterativeTrainer.setOrientation(new GradientDescent());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging
Reset training subject: 4460202554931
Reset training subject: 4460270439375
Constructing line search parameters: GD
th(0)=1.8836278;dx=-0.04709069500000001
New Minimum: 1.8836278 > 1.7835400708897833
WOLFE (weak): th(2.154434690031884)=1.7835400708897833; dx=-0.04582252216392862 evalInputDelta=0.10008772911021668
New Minimum: 1.7835400708897833 > 1.6861845373305546
WOLFE (weak): th(4.308869380063768)=1.6861845373305546; dx=-0.044554349327857234 evalInputDelta=0.19744326266944534
New Minimum: 1.6861845373305546 > 1.3240843586035236
END: th(12.926608140191302)=1.3240843586035236; dx=-0.03948165798357168 evalInputDelta=0.5595434413964764
Fitness changed from 1.8836278 to 1.3240843586035236
Iteration 1 complete. Error: 1.3240843586035236 Total: 0.2591; Orientation: 0.0063; Line Search: 0.1453
th(0)=1.3240843586035236;dx=-0.0331021089650881
New Minimum: 1.3240843586035236 > 0.5626678292573084
END: th(27.849533001676672)=0.5626678292573084; dx=-0.02157863051448412 evalInputDelta=0.7614165293462152
Fitness changed from 1.3240843586035236 to 0.5626678292573084
Iteration 2 complete. Error: 0.5626678292573084 Total: 0.0695; Orientation: 0.0026; Line Search: 0.0509
th(0)=0.5626678292573084;dx=-0.014066695731432712
New Minimum: 0.5626678292573084 > 0.03516673932858176
END: th(60.0)=0.03516673932858176; dx=-0.003516673932858177 evalInputDelta=0.5275010899287267
Fitness changed from 0.5626678292573084 to 0.03516673932858176
Iteration 3 complete. Error: 0.03516673932858176 Total: 0.0660; Orientation: 0.0031; Line Search: 0.0411
th(0)=0.03516673932858176;dx=-8.791684832145441E-4
New Minimum: 0.03516673932858176 > 0.01333669343881462
WOLF (strong): th(129.26608140191303)=0.01333669343881462; dx=5.414148257505517E-4 evalInputDelta=0.02183004588976714
New Minimum: 0.01333669343881462 > 0.0012975616768380613
END: th(64.63304070095651)=0.0012975616768380613; dx=-1.6887682873199618E-4 evalInputDelta=0.0338691776517437
Fitness changed from 0.03516673932858176 to 0.0012975616768380613
Iteration 4 complete. Error: 0.0012975616768380613 Total: 0.0815; Orientation: 0.0020; Line Search: 0.0657
th(0)=0.0012975616768380613;dx=-3.243904192095154E-5
New Minimum: 0.0012975616768380613 > 7.116894281931758E-4
WOLF (strong): th(139.24766500838336)=7.116894281931758E-4; dx=2.4024218611568026E-5 evalInputDelta=5.858722486448855E-4
New Minimum: 7.116894281931758E-4 > 2.1828404026448794E-5
END: th(69.62383250419168)=2.1828404026448794E-5; dx=-4.207411654691753E-6 evalInputDelta=0.0012757332728116125
Fitness changed from 0.0012975616768380613 to 2.1828404026448794E-5
Iteration 5 complete. Error: 2.1828404026448794E-5 Total: 0.0835; Orientation: 0.0048; Line Search: 0.0617
th(0)=2.1828404026448794E-5;dx=-5.457101006612199E-7
New Minimum: 2.1828404026448794E-5 > 1.6712371832749844E-5
WOLF (strong): th(150.0)=1.6712371832749844E-5; dx=4.774963380785672E-7 evalInputDelta=5.11603219369895E-6
New Minimum: 1.6712371832749844E-5 > 8.526720322832198E-8
END: th(75.0)=8.526720322832198E-8; dx=-3.410688129132752E-8 evalInputDelta=2.1743136823220473E-5
Fitness changed from 2.1828404026448794E-5 to 8.526720322832198E-8
Iteration 6 complete. Error: 8.526720322832198E-8 Total: 0.0757; Orientation: 0.0058; Line Search: 0.0557
th(0)=8.526720322832198E-8;dx=-2.1316800807080494E-9
Armijo: th(161.5826017523913)=8.867417302378095E-8; dx=2.173850088598872E-9 evalInputDelta=-3.4069697954589693E-9
New Minimum: 8.526720322832198E-8 > 8.342291048268672E-12
WOLF (strong): th(80.79130087619565)=8.342291048268672E-12; dx=2.108500394537751E-11 evalInputDelta=8.525886093727371E-8
END: th(26.93043362539855)=3.752261717478424E-8; dx=-1.4140917191568703E-9 evalInputDelta=4.774458605353774E-8
Fitness changed from 8.526720322832198E-8 to 8.342291048268672E-12
Iteration 7 complete. Error: 8.342291048268672E-12 Total: 0.0940; Orientation: 0.0023; Line Search: 0.0780
Low gradient: 4.5668071582530924E-7
th(0)=8.342291048268672E-12;dx=-2.0855727620671683E-13
New Minimum: 8.342291048268672E-12 > 6.29747214991468E-13
END: th(58.019860420159745)=6.29747214991468E-13; dx=-5.730147551708121E-14 evalInputDelta=7.712543833277205E-12
Fitness changed from 8.342291048268672E-12 to 6.29747214991468E-13
Iteration 8 complete. Error: 6.29747214991468E-13 Total: 0.0584; Orientation: 0.0026; Line Search: 0.0401
Low gradient: 1.2547382346444497E-7
th(0)=6.29747214991468E-13;dx=-1.5743680374786703E-14
New Minimum: 6.29747214991468E-13 > 1.9925595474406068E-13
WOLF (strong): th(125.00000000000003)=1.9925595474406068E-13; dx=8.855820210832331E-15 evalInputDelta=4.3049126024740734E-13
New Minimum: 1.9925595474406068E-13 > 3.0134388216835414E-14
END: th(62.500000000000014)=3.0134388216835414E-14; dx=-3.4439300819543196E-15 evalInputDelta=5.996128267746326E-13
Fitness changed from 6.29747214991468E-13 to 3.0134388216835414E-14
Iteration 9 complete. Error: 3.0134388216835414E-14 Total: 0.0759; Orientation: 0.0021; Line Search: 0.0598
Low gradient: 2.744739888260608E-8
th(0)=3.0134388216835414E-14;dx=-7.533597054208854E-16
New Minimum: 3.0134388216835414E-14 > 1.4063622365514903E-14
WOLF (strong): th(134.65216812699276)=1.4063622365514903E-14; dx=5.146592659817485E-16 evalInputDelta=1.607076585132051E-14
New Minimum: 1.4063622365514903E-14 > 7.563173252862456E-16
END: th(67.32608406349638)=7.563173252862456E-16; dx=-1.1935021966699077E-16 evalInputDelta=2.937807089154917E-14
Fitness changed from 3.0134388216835414E-14 to 7.563173252862456E-16
Iteration 10 complete. Error: 7.563173252862456E-16 Total: 0.0694; Orientation: 0.0024; Line Search: 0.0525
Low gradient: 4.3483253250137736E-9
th(0)=7.563173252862456E-16;dx=-1.8907933132156144E-17
New Minimum: 7.563173252862456E-16 > 5.000506745691813E-16
WOLF (strong): th(145.04965105039938)=5.000506745691813E-16; dx=1.5374430654046716E-17 evalInputDelta=2.5626665071706434E-16
New Minimum: 5.000506745691813E-16 > 6.603386885375661E-18
END: th(72.52482552519969)=6.603386885375661E-18; dx=-1.766751239382895E-18 evalInputDelta=7.4971393840087E-16
Fitness changed from 7.563173252862456E-16 to 6.603386885375661E-18
Iteration 11 complete. Error: 6.603386885375661E-18 Total: 0.0808; Orientation: 0.0029; Line Search: 0.0609
Low gradient: 4.063061310568566E-10
th(0)=6.603386885375661E-18;dx=-1.6508467213439155E-19
New Minimum: 6.603386885375661E-18 > 5.998828755986374E-18
WOLF (strong): th(156.25000000000006)=5.998828755986374E-18; dx=1.57346328076982E-19 evalInputDelta=6.045581293892872E-19
New Minimum: 5.998828755986374E-18 > 3.6273492028335335E-21
END: th(78.12500000000003)=3.6273492028335335E-21; dx=-3.8691722430849956E-21 evalInputDelta=6.599759536172828E-18
Fitness changed from 6.603386885375661E-18 to 3.6273492028335335E-21
Iteration 12 complete. Error: 3.6273492028335335E-21 Total: 0.0719; Orientation: 0.0022; Line Search: 0.0584
Zero gradient: 9.522800537175939E-12
th(0)=3.6273492028335335E-21;dx=-9.068373007083836E-23
Armijo: th(168.315210158741)=4.420590061415158E-21; dx=1.0010938965489554E-22 evalInputDelta=-7.932408585816248E-22
New Minimum: 3.6273492028335335E-21 > 9.797050537013208E-24
WOLF (strong): th(84.1576050793705)=9.797050537013208E-24; dx=4.712836423526778E-24 evalInputDelta=3.61755215229652E-21
END: th(28.052535026456834)=1.5294599353337184E-21; dx=-5.888487325489581E-23 evalInputDelta=2.097889267499815E-21
Fitness changed from 3.6273492028335335E-21 to 9.797050537013208E-24
Iteration 13 complete. Error: 9.797050537013208E-24 Total: 0.0722; Orientation: 0.0013; Line Search: 0.0605
Zero gradient: 4.949002560368404E-13
th(0)=9.797050537013208E-24;dx=-2.449262634253302E-25
New Minimum: 9.797050537013208E-24 > 5.858431611247868E-25
END: th(60.437354604333095)=5.858431611247868E-25; dx=-5.98933167998909E-26 evalInputDelta=9.211207375888421E-24
Fitness changed from 9.797050537013208E-24 to 5.858431611247868E-25
Iteration 14 complete. Error: 5.858431611247868E-25 Total: 0.0440; Orientation: 0.0024; Line Search: 0.0303
Zero gradient: 1.210209858996351E-13
th(0)=5.858431611247868E-25;dx=-1.4646079028119674E-26
New Minimum: 5.858431611247868E-25 > 2.3075479551149304E-25
WOLF (strong): th(130.20833333333343)=2.3075479551149304E-25; dx=9.19191620463886E-27 evalInputDelta=3.550883656132938E-25
New Minimum: 2.3075479551149304E-25 > 2.0312378626329255E-26
END: th(65.10416666666671)=2.0312378626329255E-26; dx=-2.727162768626015E-27 evalInputDelta=5.655307824984576E-25
Fitness changed from 5.858431611247868E-25 to 2.0312378626329255E-26
Iteration 15 complete. Error: 2.0312378626329255E-26 Total: 0.0656; Orientation: 0.0017; Line Search: 0.0501
Zero gradient: 2.2534628145550384E-14
th(0)=2.0312378626329255E-26;dx=-5.078094656582314E-28
New Minimum: 2.0312378626329255E-26 > 1.152460566808492E-26
WOLF (strong): th(140.26267513228422)=1.152460566808492E-26; dx=3.8250172750820556E-28 evalInputDelta=8.787772958244335E-27
New Minimum: 1.152460566808492E-26 > 3.090053424261929E-28
END: th(70.13133756614211)=3.090053424261929E-28; dx=-6.263283291053353E-29 evalInputDelta=2.0003373283903062E-26
Fitness changed from 2.0312378626329255E-26 to 3.090053424261929E-28
Iteration 16 complete. Error: 3.090053424261929E-28 Total: 0.0725; Orientation: 0.0017; Line Search: 0.0558
Zero gradient: 2.779412448819862E-15
th(0)=3.090053424261929E-28;dx=-7.725133560654823E-30
New Minimum: 3.090053424261929E-28 > 2.44024813296579E-28
WOLF (strong): th(151.09338651083277)=2.44024813296579E-28; dx=6.864961420415753E-30 evalInputDelta=6.498052912961388E-29
New Minimum: 2.44024813296579E-28 > 9.655829579801308E-31
END: th(75.54669325541639)=9.655829579801308E-31; dx=-4.3133044279694245E-31 evalInputDelta=3.0803975946821277E-28
Fitness changed from 3.090053424261929E-28 to 9.655829579801308E-31
Iteration 17 complete. Error: 9.655829579801308E-31 Total: 0.0524; Orientation: 0.0014; Line Search: 0.0404
Zero gradient: 1.5536915379026583E-16
th(0)=9.655829579801308E-31;dx=-2.4139573949503276E-32
Armijo: th(162.7604166666668)=9.8772016852137E-31; dx=2.4409211664881334E-32 evalInputDelta=-2.2137210541239234E-32
New Minimum: 9.655829579801308E-31 > 3.8368135610839464E-35
WOLF (strong): th(81.3802083333334)=3.8368135610839464E-35; dx=4.5473763862690033E-35 evalInputDelta=9.6554458984452E-31
END: th(27.126736111111132)=4.3276511014223395E-31; dx=-1.6130368580853596E-32 evalInputDelta=5.3281784783789685E-31
Fitness changed from 9.655829579801308E-31 to 3.8368135610839464E-35
Iteration 18 complete. Error: 3.8368135610839464E-35 Total: 0.0710; Orientation: 0.0018; Line Search: 0.0585
Zero gradient: 9.793892945458342E-19
th(0)=3.8368135610839464E-35;dx=-9.592033902709869E-37
New Minimum: 3.8368135610839464E-35 > 1.655096045957781E-36
END: th(58.44278130511842)=1.655096045957781E-36; dx=-1.2413220344683359E-37 evalInputDelta=3.671303956488168E-35
Fitness changed from 3.8368135610839464E-35 to 1.655096045957781E-36
Iteration 19 complete. Error: 1.655096045957781E-36 Total: 0.0494; Orientation: 0.0016; Line Search: 0.0370
Zero gradient: 2.0341435826643242E-19
th(0)=1.655096045957781E-36;dx=-4.1377401148944526E-38
New Minimum: 1.655096045957781E-36 > 9.780112998841432E-37
WOLF (strong): th(125.91115542569399)=9.780112998841432E-37; dx=3.0092655381050566E-38 evalInputDelta=6.770847460736377E-37
New Minimum: 9.780112998841432E-37 > 0.0
END: th(62.955577712846996)=0.0; dx=0.0 evalInputDelta=1.655096045957781E-36
Fitness changed from 1.655096045957781E-36 to 0.0
Iteration 20 complete. Error: 0.0 Total: 0.0608; Orientation: 0.0016; Line Search: 0.0473
Zero gradient: 0.0
th(0)=0.0;dx=0.0 (ERROR: Starting derivative negative)
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0361; Orientation: 0.0012; Line Search: 0.0232
Iteration 21 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 21
Final threshold in iteration 21: 0.0 (> 0.0) after 1.611s (< 30.000s)

Returns

    0.0

Training Converged

Conjugate Gradient Descent

First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.

TrainingTester.java:452 executed in 0.41 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
      iterativeTrainer.setOrientation(new GradientDescent());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging
Reset training subject: 4461821069138
Reset training subject: 4461833304134
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=1.8836278}, derivative=-0.047090695}
New Minimum: 1.8836278 > 1.8836277999952908
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=1.8836277999952908}, derivative=-0.04709069499994114}, evalInputDelta = -4.709121981250064E-12
New Minimum: 1.8836277999952908 > 1.8836277999670366
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=1.8836277999670366}, derivative=-0.04709069499958796}, evalInputDelta = -3.29634097795406E-11
New Minimum: 1.8836277999670366 > 1.8836277997692554
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=1.8836277997692554}, derivative=-0.0470906949971157}, evalInputDelta = -2.3074453459059896E-10
New Minimum: 1.8836277997692554 > 1.8836277983847893
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=1.8836277983847893}, derivative=-0.047090694979809875}, evalInputDelta = -1.615210631911168E-9
New Minimum: 1.8836277983847893 > 1.8836277886935242
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=1.8836277886935242}, derivative=-0.047090694858669055}, evalInputDelta = -1.1306475755645806E-8
New Minimum: 1.8836277886935242 > 1.8836277208546697
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=1.8836277208546697}, derivative=-0.047090694010683366}, evalInputDelta = -7.914533028952064E-8
New Minimum: 1.8836277208546697 > 1.8836272459827232
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=1.8836272459827232}, derivative=-0.04709068807478353}, evalInputDelta = -5.540172767215523E-7
New Minimum: 1.8836272459827232 > 1.883623921880773
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=1.883623921880773}, derivative=-0.047090646523484714}, evalInputDelta = -3.878119226863319E-6
New Minimum: 1.883623921880773 > 1.8836006532492475
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=1.8836006532492475}, derivative=-0.04709035566439297}, evalInputDelta = -2.714675075243811E-5
New Minimum: 1.8836006532492475 > 1.8834377768527566
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=1.8834377768527566}, derivative=-0.04708831965075077}, evalInputDelta = -1.9002314724336777E-4
New Minimum: 1.8834377768527566 > 1.882297839262509
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=1.882297839262509}, derivative=-0.04707406755525537}, evalInputDelta = -0.0013299607374910316
New Minimum: 1.882297839262509 > 1.8743279382049107
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=1.8743279382049107}, derivative=-0.046974302886787556}, evalInputDelta = -0.009299861795089237
New Minimum: 1.8743279382049107 > 1.8190120724344472
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=1.8190120724344472}, derivative=-0.046275950207512885}, evalInputDelta = -0.06461572756555278
New Minimum: 1.8190120724344472 > 1.4549996520446302
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=1.4549996520446302}, derivative=-0.04138748145259018}, evalInputDelta = -0.42862814795536974
New Minimum: 1.4549996520446302 > 0.04364606948391488
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=0.04364606948391488}, derivative=-0.007168200168131195}, evalInputDelta = -1.839981730516085
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=45.86410563979101}, derivative=0.23236676882308163}, evalInputDelta = 43.980477839791014
F(36.51970392263846) = LineSearchPoint{point=PointSample{avg=0.5564166317729743}, derivative=-0.02559396701360911}, evalInputDelta = -1.3272111682270258
F(255.63792745846925) = LineSearchPoint{point=PointSample{avg=9.079286591149145}, derivative=0.10338640090473628}, evalInputDelta = 7.195658791149145
F(19.664455958343787) = LineSearchPoint{point=PointSample{avg=1.0714245261692448}, derivative=-0.03551553377655875}, evalInputDelta = -0.8122032738307552
F(137.65119170840651) = LineSearchPoint{point=PointSample{avg=0.9782090930555871}, derivative=0.033935433564088756}, evalInputDelta = -0.9054187069444128
0.9782090930555871 <= 1.8836278
New Minimum: 0.04364606948391488 > 5.094393152621895E-32
F(79.99999999999999) = LineSearchPoint{point=PointSample{avg=5.094393152621895E-32}, derivative=-6.363982691520721E-18}, evalInputDelta = -1.8836278
Left bracket at 79.99999999999999
Converged to left
Fitness changed from 1.8836278 to 5.094393152621895E-32
Iteration 1 complete. Error: 5.094393152621895E-32 Total: 0.3476; Orientation: 0.0013; Line Search: 0.3077
Zero gradient: 3.568750885331552E-17
F(0.0) = LineSearchPoint{point=PointSample{avg=5.094393152621895E-32}, derivative=-1.273598288155474E-33}
New Minimum: 5.094393152621895E-32 > 0.0
F(79.99999999999999) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}, evalInputDelta = -5.094393152621895E-32
0.0 <= 5.094393152621895E-32
Converged to right
Fitness changed from 5.094393152621895E-32 to 0.0
Iteration 2 complete. Error: 0.0 Total: 0.0412; Orientation: 0.0013; Line Search: 0.0250
Zero gradient: 0.0
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0187; Orientation: 0.0010; Line Search: 0.0096
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 0.407s (< 30.000s)

Returns

    0.0

Training Converged

Limited-Memory BFGS

Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.

TrainingTester.java:509 executed in 7.53 seconds (0.210 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
      iterativeTrainer.setOrientation(new LBFGS());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setIterationsPerSample(100);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging
Reset training subject: 4462234100498
Reset training subject: 4462245737069
Adding measurement 1a0272df to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 1.8836278 < 1.8836278. Total: 1
th(0)=1.8836278;dx=-0.047090695
Adding measurement 233d6b0f to history. Total: 1
New Minimum: 1.8836278 > 1.7835400708897833
WOLFE (weak): th(2.154434690031884)=1.7835400708897833; dx=-0.04582252216392862 evalInputDelta=0.10008772911021668
Adding measurement 3c576b4f to history. Total: 2
New Minimum: 1.7835400708897833 > 1.6861845373305546
WOLFE (weak): th(4.308869380063768)=1.6861845373305546; dx=-0.044554349327857234 evalInputDelta=0.19744326266944534
Adding measurement 42b5ceea to history. Total: 3
New Minimum: 1.6861845373305546 > 1.3240843586035236
END: th(12.926608140191302)=1.3240843586035236; dx=-0.03948165798357168 evalInputDelta=0.5595434413964764
Fitness changed from 1.8836278 to 1.3240843586035236
Iteration 1 complete. Error: 1.3240843586035236 Total: 0.1076; Orientation: 0.0068; Line Search: 0.0711
Non-optimal measurement 1.3240843586035236 < 1.3240843586035236. Total: 4
Rejected: LBFGS Orientation magnitude: 1.456e+01, gradient 1.819e-01, dot -1.000; [dc4dce0c-8d2e-4e46-914e-1cbd5a8aebfa = 1.000/1.000e+00, 19fa7f31-1a15-4c11-b6ca-6af649f6cc06 = 1.000/1.000e+00, d71db068-7618-4e18-8f25-f08b829ec665 = 1.000/1.000e+00, 0a09579b-e0f3-4673-8a2c-1a5ac13bf352 = 1.000/1.000e+00, 7f7b4e3c-3b17-49f4-8a36-5f1e93129729 = 1.000/1.000e+00, 9b0f9e91-109a-4a53-9271-bafd2d4db799 = 1.000/1.000e+00, a6e3802d-e46a-4653-96ce-12c49daf560d = 1.000/1.000e+00, fddfbb31-1e71-4554-9bbf-dc53f86be81f = 1.000/1.000e+00, 4bf09b78-b288-42f2-8611-2c25bdd2b7cf = 1.000/1.000e+00, 35ef6b90-d23f-4010-bee5-eb5fac5f596a = 1.000/1.000e+00, 7859f735-2c78-4d3e-8f79-623f20920612 = 1.000/1.000e+00, 6e7468f6-7c20-4eba-b86d-b984c2146142 = 1.000/1.000e+00, c6060212-44a2-403e-8bf2-79239a578001 = 1.000/1.000e+00, 9b2e5b43-797e-4281-a3ec-eea8b6d1e9e3 = 1.000/1.000e+00, 7f6613f4-5953-4f2f-9198-b2b61b2f1548 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.3240843586035236, 1.6861845373305546, 1.7835400708897833, 1.8836278
LBFGS Accumulation History: 3 points
Removed measurement 42b5ceea to history. Total: 3
Adding measurement 65673c5b to history. Total: 3
th(0)=1.3240843586035236;dx=-0.0331021089650881
Adding measurement 7353b480 to history. Total: 4
New Minimum: 1.3240843586035236 > 0.5626678292573084
END: th(27.849533001676672)=0.5626678292573084; dx=-0.02157863051448412 evalInputDelta=0.7614165293462152
Fitness changed from 1.3240843586035236 to 0.5626678292573084
Iteration 2 complete. Error: 0.5626678292573084 Total: 0.2642; Orientation: 0.2132; Line Search: 0.0401
Non-optimal measurement 0.5626678292573084 < 0.5626678292573084. Total: 5
Rejected: LBFGS Orientation magnitude: 9.488e+00, gradient 1.186e-01, dot -1.000; [9b2e5b43-797e-4281-a3ec-eea8b6d1e9e3 = 1.000/1.000e+00, fddfbb31-1e71-4554-9bbf-dc53f86be81f = 1.000/1.000e+00, 6e7468f6-7c20-4eba-b86d-b984c2146142 = 1.000/1.000e+00, 7f6613f4-5953-4f2f-9198-b2b61b2f1548 = 1.000/1.000e+00, dc4dce0c-8d2e-4e46-914e-1cbd5a8aebfa = 1.000/1.000e+00, d71db068-7618-4e18-8f25-f08b829ec665 = 1.000/1.000e+00, 35ef6b90-d23f-4010-bee5-eb5fac5f596a = 1.000/1.000e+00, c6060212-44a2-403e-8bf2-79239a578001 = 1.000/1.000e+00, 0a09579b-e0f3-4673-8a2c-1a5ac13bf352 = 1.000/1.000e+00, 7f7b4e3c-3b17-49f4-8a36-5f1e93129729 = 1.000/1.000e+00, 7859f735-2c78-4d3e-8f79-623f20920612 = 1.000/1.000e+00, a6e3802d-e46a-4653-96ce-12c49daf560d = 1.000/1.000e+00, 19fa7f31-1a15-4c11-b6ca-6af649f6cc06 = 1.000/1.000e+00, 9b0f9e91-109a-4a53-9271-bafd2d4db799 = 1.000/1.000e+00, 4bf09b78-b288-42f2-8611-2c25bdd2b7cf = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.5626678292573084, 1.3240843586035236, 1.6861845373305546, 1.7835400708897833, 1.8836278
Rejected: LBFGS Orientation magnitude: 9.488e+00, gradient 1.186e-01, dot -1.000; [35ef6b90-d23f-4010-bee5-eb5fac5f596a = 1.000/1.000e+00, 0a09579b-e0f3-4673-8a2c-1a5ac13bf352 = 1.000/1.000e+00, d71db068-7618-4e18-8f25-f08b829ec665 = 1.000/1.000e+00, fddfbb31-1e71-4554-9bbf-dc53f86be81f = 1.000/1.000e+00, 7f7b4e3c-3b17-49f4-8a36-5f1e93129729 = 1.000/1.000e+00, 7f6613f4-5953-4f2f-9198-b2b61b2f1548 = 1.000/1.000e+00, a6e3802d-e46a-4653-96ce-12c49daf560d = 1.000/1.000e+00, 9b2e5b43-797e-4281-a3ec-eea8b6d1e9e3 = 1.000/1.000e+00, dc4dce0c-8d2e-4e46-914e-1cbd5a8aebfa = 1.000/1.000e+00, 9b0f9e91-109a-4a53-9271-bafd2d4db799 = 1.000/1.000e+00, 6e7468f6-7c20-4eba-b86d-b984c2146142 = 1.000/1.000e+00, c6060212-44a2-403e-8bf2-79239a578001 = 1.000/1.000e+00, 4bf09b78-b288-42f2-8611-2c25bdd2b7cf = 1.000/1.000e+00, 7859f735-2c78-4d3e-8f79-623f20920612 = 1.000/1.000e+00, 19fa7f31-1a15-4c11-b6ca-6af649f6cc06 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.5626678292573084, 1.3240843586035236, 1.6861845373305546, 1.7835400708897833
LBFGS Accumulation History: 3 points
Removed measurement 7353b480 to history. Total: 4
Removed measurement 65673c5b to history. Total: 3
Adding measurement 4ddeaf75 to history. Total: 3
th(0)=0.5626678292573084;dx=-0.014066695731432712
Adding measurement 5b1f4d0c to history. Total: 4
New Minimum: 0.5626678292573084 > 0.03516673932858176
END: th(60.0)=0.03516673932858176; dx=-0.003516673932858177 evalInputDelta=0.5275010899287267
Fitness changed from 0.5626678292573084 to 0.03516673932858176
Iteration 3 complete. Error: 0.03516673932858176 Total: 0.2847; Orientation: 0.2461; Line Search: 0.0270
Non-optimal measurement 0.03516673932858176 < 0.03516673932858176. Total: 5
Rejected: LBFGS Orientation magnitude: 2.372e+00, gradient 2.965e-02, dot -1.000; [9b0f9e91-109a-4a53-9271-bafd2d4db799 = 1.000/1.000e+00, 19fa7f31-1a15-4c11-b6ca-6af649f6cc06 = 1.000/1.000e+00, 9b2e5b43-797e-4281-a3ec-eea8b6d1e9e3 = 1.000/1.000e+00, 35ef6b90-d23f-4010-bee5-eb5fac5f596a = 1.000/1.000e+00, 0a09579b-e0f3-4673-8a2c-1a5ac13bf352 = 1.000/1.000e+00, dc4dce0c-8d2e-4e46-914e-1cbd5a8aebfa = 1.000/1.000e+00, 7f7b4e3c-3b17-49f4-8a36-5f1e93129729 = 1.000/1.000e+00, 7859f735-2c78-4d3e-8f79-623f20920612 = 1.000/1.000e+00, a6e3802d-e46a-4653-96ce-12c49daf560d = 1.000/1.000e+00, d71db068-7618-4e18-8f25-f08b829ec665 = 1.000/1.000e+00, 6e7468f6-7c20-4eba-b86d-b984c2146142 = 1.000/1.000e+00, 4bf09b78-b288-42f2-8611-2c25bdd2b7cf = 1.000/1.000e+00, 7f6613f4-5953-4f2f-9198-b2b61b2f1548 = 1.000/1.000e+00, fddfbb31-1e71-4554-9bbf-dc53f86be81f = 1.000/1.000e+00, c6060212-44a2-403e-8bf2-79239a578001 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.03516673932858176, 0.5626678292573084, 1.6861845373305546, 1.7835400708897833, 1.8836278
Rejected: LBFGS Orientation magnitude: 2.372e+00, gradient 2.965e-02, dot -1.000; [7859f735-2c78-4d3e-8f79-623f20920612 = 1.000/1.000e+00, 7f7b4e3c-3b17-49f4-8a36-5f1e93129729 = 1.000/1.000e+00, d71db068-7618-4e18-8f25-f08b829ec665 = 1.000/1.000e+00, fddfbb31-1e71-4554-9bbf-dc53f86be81f = 1.000/1.000e+00, 7f6613f4-5953-4f2f-9198-b2b61b2f1548 = 1.000/1.000e+00, 35ef6b90-d23f-4010-bee5-eb5fac5f596a = 1.000/1.000e+00, 9b0f9e91-109a-4a53-9271-bafd2d4db799 = 1.000/1.000e+00, 19fa7f31-1a15-4c11-b6ca-6af649f6cc06 = 1.000/1.000e+00, c6060212-44a2-403e-8bf2-79239a578001 = 1.000/1.000e+00, a6e3802d-e46a-4653-96ce-12c49daf560d = 1.000/1.000e+00, 6e7468f6-7c20-4eba-b86d-b984c2146142 = 1.000/1.000e+00, 0a09579b-e0f3-4673-8a2c-1a5ac13bf352 = 1.000/1.000e+00, 9b2e5b43-797e-4281-a3ec-eea8b6d1e9e3 = 1.000/1.000e+00, dc4dce0c-8d2e-4e46-914e-1cbd5a8aebfa = 1.000/1.000e+00, 4bf09b78-b288-42f2-8611-2c25bdd2b7cf = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.03516673932858176, 0.5626678292573084, 1.6861845373305546, 1.7835400708897833
LBFGS Accumulation History: 3 points
Removed measurement 5b1f4d0c to history. Total: 4
Removed measurement 4ddeaf75 to history. Total: 3
Adding measurement 11a63392 to history. Total: 3
th(0)=0.03516673932858176;dx=-8.791684832145441E-4
Adding measurement 5865f48b to history. Total: 4
New Minimum: 0.03516673932858176 > 0.01333669343881462
WOLF (strong): th(129.26608140191303)=0.01333669343881462; dx=5.414148257505517E-4 evalInputDelta=0.0218300

...skipping 56154 bytes...

308E-31, 1.6861845373305546, 1.7835400708897833, 1.8836278
Rejected: LBFGS Orientation magnitude: 7.835e-17, gradient 9.794e-19, dot -1.000; [7859f735-2c78-4d3e-8f79-623f20920612 = 1.000/1.000e+00, dc4dce0c-8d2e-4e46-914e-1cbd5a8aebfa = 1.000/1.000e+00, 7f6613f4-5953-4f2f-9198-b2b61b2f1548 = 1.000/1.000e+00, 7f7b4e3c-3b17-49f4-8a36-5f1e93129729 = 1.000/1.000e+00, a6e3802d-e46a-4653-96ce-12c49daf560d = 1.000/1.000e+00, 9b0f9e91-109a-4a53-9271-bafd2d4db799 = 1.000/1.000e+00, 6e7468f6-7c20-4eba-b86d-b984c2146142 = 1.000/1.000e+00, 0a09579b-e0f3-4673-8a2c-1a5ac13bf352 = 1.000/1.000e+00, 19fa7f31-1a15-4c11-b6ca-6af649f6cc06 = 1.000/1.000e+00, 9b2e5b43-797e-4281-a3ec-eea8b6d1e9e3 = 1.000/1.000e+00, 4bf09b78-b288-42f2-8611-2c25bdd2b7cf = 0.000e+00, 35ef6b90-d23f-4010-bee5-eb5fac5f596a = 1.000/1.000e+00, d71db068-7618-4e18-8f25-f08b829ec665 = 1.000/1.000e+00, c6060212-44a2-403e-8bf2-79239a578001 = 1.000/1.000e+00, fddfbb31-1e71-4554-9bbf-dc53f86be81f = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.8368135610839464E-35, 9.655829579801308E-31, 1.6861845373305546, 1.7835400708897833
LBFGS Accumulation History: 3 points
Removed measurement 4cca3455 to history. Total: 4
Removed measurement 2a12387e to history. Total: 3
Adding measurement 33945930 to history. Total: 3
th(0)=3.8368135610839464E-35;dx=-9.592033902709869E-37
Adding measurement 34c91ae2 to history. Total: 4
New Minimum: 3.8368135610839464E-35 > 1.655096045957781E-36
END: th(58.44278130511842)=1.655096045957781E-36; dx=-1.2413220344683359E-37 evalInputDelta=3.671303956488168E-35
Fitness changed from 3.8368135610839464E-35 to 1.655096045957781E-36
Iteration 19 complete. Error: 1.655096045957781E-36 Total: 0.2412; Orientation: 0.2081; Line Search: 0.0251
Non-optimal measurement 1.655096045957781E-36 < 1.655096045957781E-36. Total: 5
Rejected: LBFGS Orientation magnitude: 1.627e-17, gradient 2.034e-19, dot -1.000; [7859f735-2c78-4d3e-8f79-623f20920612 = 1.000/1.000e+00, c6060212-44a2-403e-8bf2-79239a578001 = 1.000/1.000e+00, 35ef6b90-d23f-4010-bee5-eb5fac5f596a = 0.000e+00, 9b0f9e91-109a-4a53-9271-bafd2d4db799 = 1.000/1.000e+00, fddfbb31-1e71-4554-9bbf-dc53f86be81f = 0.000e+00, a6e3802d-e46a-4653-96ce-12c49daf560d = 1.000/1.000e+00, 19fa7f31-1a15-4c11-b6ca-6af649f6cc06 = 0.000e+00, 7f7b4e3c-3b17-49f4-8a36-5f1e93129729 = 0.000e+00, d71db068-7618-4e18-8f25-f08b829ec665 = 1.000/1.000e+00, dc4dce0c-8d2e-4e46-914e-1cbd5a8aebfa = 0.000e+00, 6e7468f6-7c20-4eba-b86d-b984c2146142 = 1.000/1.000e+00, 4bf09b78-b288-42f2-8611-2c25bdd2b7cf = 0.000e+00, 7f6613f4-5953-4f2f-9198-b2b61b2f1548 = 1.000/1.000e+00, 9b2e5b43-797e-4281-a3ec-eea8b6d1e9e3 = 0.000e+00, 0a09579b-e0f3-4673-8a2c-1a5ac13bf352 = 0.000e+00]
Orientation rejected. Popping history element from 1.655096045957781E-36, 3.8368135610839464E-35, 1.6861845373305546, 1.7835400708897833, 1.8836278
Rejected: LBFGS Orientation magnitude: 1.627e-17, gradient 2.034e-19, dot -1.000; [7f6613f4-5953-4f2f-9198-b2b61b2f1548 = 1.000/1.000e+00, 6e7468f6-7c20-4eba-b86d-b984c2146142 = 1.000/1.000e+00, 7f7b4e3c-3b17-49f4-8a36-5f1e93129729 = 0.000e+00, d71db068-7618-4e18-8f25-f08b829ec665 = 1.000/1.000e+00, c6060212-44a2-403e-8bf2-79239a578001 = 1.000/1.000e+00, 4bf09b78-b288-42f2-8611-2c25bdd2b7cf = 0.000e+00, fddfbb31-1e71-4554-9bbf-dc53f86be81f = 0.000e+00, 35ef6b90-d23f-4010-bee5-eb5fac5f596a = 0.000e+00, 9b0f9e91-109a-4a53-9271-bafd2d4db799 = 1.000/1.000e+00, 19fa7f31-1a15-4c11-b6ca-6af649f6cc06 = 0.000e+00, 7859f735-2c78-4d3e-8f79-623f20920612 = 1.000/1.000e+00, dc4dce0c-8d2e-4e46-914e-1cbd5a8aebfa = 0.000e+00, 9b2e5b43-797e-4281-a3ec-eea8b6d1e9e3 = 0.000e+00, 0a09579b-e0f3-4673-8a2c-1a5ac13bf352 = 0.000e+00, a6e3802d-e46a-4653-96ce-12c49daf560d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.655096045957781E-36, 3.8368135610839464E-35, 1.6861845373305546, 1.7835400708897833
LBFGS Accumulation History: 3 points
Removed measurement 34c91ae2 to history. Total: 4
Removed measurement 33945930 to history. Total: 3
Adding measurement 14fbacb4 to history. Total: 3
th(0)=1.655096045957781E-36;dx=-4.1377401148944526E-38
Adding measurement 70eaa1a0 to history. Total: 4
New Minimum: 1.655096045957781E-36 > 9.780112998841432E-37
WOLF (strong): th(125.91115542569399)=9.780112998841432E-37; dx=3.0092655381050566E-38 evalInputDelta=6.770847460736377E-37
Adding measurement 4b2d7cb3 to history. Total: 5
New Minimum: 9.780112998841432E-37 > 0.0
END: th(62.955577712846996)=0.0; dx=0.0 evalInputDelta=1.655096045957781E-36
Fitness changed from 1.655096045957781E-36 to 0.0
Iteration 20 complete. Error: 0.0 Total: 0.2570; Orientation: 0.2059; Line Search: 0.0427
Non-optimal measurement 0.0 < 0.0. Total: 6
Rejected: LBFGS Orientation magnitude: 0.000e+00, gradient 0.000e+00, dot NaN; [35ef6b90-d23f-4010-bee5-eb5fac5f596a = 0.000e+00, 9b2e5b43-797e-4281-a3ec-eea8b6d1e9e3 = 0.000e+00, 4bf09b78-b288-42f2-8611-2c25bdd2b7cf = 0.000e+00, 9b0f9e91-109a-4a53-9271-bafd2d4db799 = 0.000e+00, fddfbb31-1e71-4554-9bbf-dc53f86be81f = 0.000e+00, c6060212-44a2-403e-8bf2-79239a578001 = 0.000e+00, 19fa7f31-1a15-4c11-b6ca-6af649f6cc06 = 0.000e+00, 7f6613f4-5953-4f2f-9198-b2b61b2f1548 = 0.000e+00, 7859f735-2c78-4d3e-8f79-623f20920612 = 0.000e+00, 0a09579b-e0f3-4673-8a2c-1a5ac13bf352 = 0.000e+00, 6e7468f6-7c20-4eba-b86d-b984c2146142 = 0.000e+00, dc4dce0c-8d2e-4e46-914e-1cbd5a8aebfa = 0.000e+00, d71db068-7618-4e18-8f25-f08b829ec665 = 0.000e+00, 7f7b4e3c-3b17-49f4-8a36-5f1e93129729 = 0.000e+00, a6e3802d-e46a-4653-96ce-12c49daf560d = 0.000e+00]
Orientation rejected. Popping history element from 0.0, 9.780112998841432E-37, 1.655096045957781E-36, 1.6861845373305546, 1.7835400708897833, 1.8836278
Rejected: LBFGS Orientation magnitude: 0.000e+00, gradient 0.000e+00, dot NaN; [7859f735-2c78-4d3e-8f79-623f20920612 = 0.000e+00, 7f6613f4-5953-4f2f-9198-b2b61b2f1548 = 0.000e+00, fddfbb31-1e71-4554-9bbf-dc53f86be81f = 0.000e+00, d71db068-7618-4e18-8f25-f08b829ec665 = 0.000e+00, 9b2e5b43-797e-4281-a3ec-eea8b6d1e9e3 = 0.000e+00, 4bf09b78-b288-42f2-8611-2c25bdd2b7cf = 0.000e+00, dc4dce0c-8d2e-4e46-914e-1cbd5a8aebfa = 0.000e+00, 35ef6b90-d23f-4010-bee5-eb5fac5f596a = 0.000e+00, a6e3802d-e46a-4653-96ce-12c49daf560d = 0.000e+00, 0a09579b-e0f3-4673-8a2c-1a5ac13bf352 = 0.000e+00, 6e7468f6-7c20-4eba-b86d-b984c2146142 = 0.000e+00, 7f7b4e3c-3b17-49f4-8a36-5f1e93129729 = 0.000e+00, c6060212-44a2-403e-8bf2-79239a578001 = 0.000e+00, 9b0f9e91-109a-4a53-9271-bafd2d4db799 = 0.000e+00, 19fa7f31-1a15-4c11-b6ca-6af649f6cc06 = 0.000e+00]
Orientation rejected. Popping history element from 0.0, 9.780112998841432E-37, 1.655096045957781E-36, 1.6861845373305546, 1.7835400708897833
Rejected: LBFGS Orientation magnitude: 0.000e+00, gradient 0.000e+00, dot NaN; [4bf09b78-b288-42f2-8611-2c25bdd2b7cf = 0.000e+00, c6060212-44a2-403e-8bf2-79239a578001 = 0.000e+00, 35ef6b90-d23f-4010-bee5-eb5fac5f596a = 0.000e+00, 7f6613f4-5953-4f2f-9198-b2b61b2f1548 = 0.000e+00, 9b0f9e91-109a-4a53-9271-bafd2d4db799 = 0.000e+00, fddfbb31-1e71-4554-9bbf-dc53f86be81f = 0.000e+00, 7859f735-2c78-4d3e-8f79-623f20920612 = 0.000e+00, 19fa7f31-1a15-4c11-b6ca-6af649f6cc06 = 0.000e+00, d71db068-7618-4e18-8f25-f08b829ec665 = 0.000e+00, 0a09579b-e0f3-4673-8a2c-1a5ac13bf352 = 0.000e+00, a6e3802d-e46a-4653-96ce-12c49daf560d = 0.000e+00, 6e7468f6-7c20-4eba-b86d-b984c2146142 = 0.000e+00, 7f7b4e3c-3b17-49f4-8a36-5f1e93129729 = 0.000e+00, dc4dce0c-8d2e-4e46-914e-1cbd5a8aebfa = 0.000e+00, 9b2e5b43-797e-4281-a3ec-eea8b6d1e9e3 = 0.000e+00]
Orientation rejected. Popping history element from 0.0, 9.780112998841432E-37, 1.655096045957781E-36, 1.6861845373305546
LBFGS Accumulation History: 3 points
Removed measurement 4b2d7cb3 to history. Total: 5
Removed measurement 70eaa1a0 to history. Total: 4
Removed measurement 14fbacb4 to history. Total: 3
Adding measurement 92358bf to history. Total: 3
th(0)=0.0;dx=0.0 (ERROR: Starting derivative negative)
Non-optimal measurement 0.0 < 0.0. Total: 4
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.4102; Orientation: 0.3678; Line Search: 0.0308
Iteration 21 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 21
Final threshold in iteration 21: 0.0 (> 0.0) after 7.528s (< 30.000s)

Returns

    0.0

Training Converged

TrainingTester.java:432 executed in 0.16 seconds (0.000 gc):

    return TestUtil.compare(title + " vs Iteration", runs);
Logging
Plotting range=[1.0, -35.78117679885554], [20.0, 0.12191565527657779]; valueStats=DoubleSummaryStatistics{count=39, sum=3.846477, min=0.000000, average=0.098628, max=1.324084}
Plotting 20 points for GD
Plotting 2 points for CjGD
Plotting 20 points for LBFGS

Returns

Result

TrainingTester.java:435 executed in 0.02 seconds (0.000 gc):

    return TestUtil.compareTime(title + " vs Time", runs);
Logging
Plotting range=[0.0, -35.78117679885554], [7.01, 0.12191565527657779]; valueStats=DoubleSummaryStatistics{count=39, sum=3.846477, min=0.000000, average=0.098628, max=1.324084}
Plotting 20 points for GD
Plotting 2 points for CjGD
Plotting 20 points for LBFGS

Returns

Result

Results

TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):

    return grid(inputLearning, modelLearning, completeLearning);

Returns

Result

TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):

    return new ComponentResult(null == inputLearning ? null : inputLearning.value,
        null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);

Returns

    {"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

detailsresult
{"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}OK
  {
    "result": "OK",
    "performance": {
      "execution_time": "10.768",
      "gc_time": "0.683"
    },
    "created_on": 1586739091823,
    "file_name": "trainingTest",
    "report": {
      "simpleName": "BandConcatLimitTest",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.ImgConcatLayerTest.BandConcatLimitTest",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/ImgConcatLayerTest.java",
      "javaDoc": ""
    },
    "training_analysis": {
      "input": {
        "LBFGS": {
          "type": "Converged",
          "value": 0.0
        },
        "CjGD": {
          "type": "Converged",
          "value": 0.0
        },
        "GD": {
          "type": "Converged",
          "value": 0.0
        }
      }
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/ImgConcatLayer/BandConcatLimitTest/trainingTest/202004135131",
    "id": "9ef78818-f881-43c8-b9fd-bca27039534e",
    "report_type": "Components",
    "display_name": "Comparative Training",
    "target": {
      "simpleName": "ImgConcatLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.ImgConcatLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/ImgConcatLayer.java",
      "javaDoc": ""
    }
  }