1. Test Modules
  2. Training Characteristics
    1. Input Learning
      1. Gradient Descent
      2. Conjugate Gradient Descent
      3. Limited-Memory BFGS
    2. Results
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 8556702123382266880

Training Characteristics

Input Learning

In this apply, we use a network to learn this target input, given it's pre-evaluated output:

TrainingTester.java:332 executed in 0.00 seconds (0.000 gc):

    return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
      try {
        return x.prettyPrint();
      } finally {
        x.freeRef();
      }
    }).reduce((a, b) -> a + "\n" + b).orElse("");

Returns

    [ 0.7, -0.128, 0.496, 0.08, -0.608 ]
    [ -0.128, 0.7, 0.08, 0.496, -0.608 ]
    [ -0.608, -0.128, 0.496, 0.7, 0.08 ]
    [ 0.496, 0.08, -0.128, 0.7, -0.608 ]
    [ 0.7, 0.08, 0.496, -0.608, -0.128 ]

Gradient Descent

First, we train using basic gradient descent method apply weak line search conditions.

TrainingTester.java:480 executed in 0.67 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
      iterativeTrainer.setOrientation(new GradientDescent());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging 
Reset training subject: 4819700864110
Set StochasticSamplingSubnetLayer to random seed 4819700864110
Reset training subject: 4819721884451
Set StochasticSamplingSubnetLayer to random seed 4819721884451
Constructing line search parameters: GD
th(0)=22.35885042030956;dx=-1.1719932839506171E23
New Minimum: 22.35885042030956 > 3.6804001011569505
Armijo: th(2.154434690031884)=3.6804001011569505; dx=-1.1719932839533188E11 evalInputDelta=18.67845031915261
Armijo: th(1.077217345015942)=3.7620117215422817; dx=-1.1719932839536406E11 evalInputDelta=18.59683869876728
Armijo: th(0.3590724483386473)=3.8871933157085423; dx=-1.1719932839544725E11 evalInputDelta=18.471657104601018
Armijo: th(0.08976811208466183)=3.973575897031207; dx=-1.1719932839554303E11 evalInputDelta=18.385274523278355
Armijo: th(0.017953622416932366)=4.002354183979864; dx=-1.1719932839559372E11 evalInputDelta=18.356496236329697
Armijo: th(0.002992270402822061)=4.0102120182540135; dx=-1.1719932839560835E11 evalInputDelta=18.34863840205555
Armijo: th(4.2746720040315154E-4)=4.011617479664173; dx=-1.1719932839561105E11 evalInputDelta=18.34723294064539
Armijo: th(5.343340005039394E-5)=4.01182302372934; dx=-1.1719932839561145E11 evalInputDelta=18.347027396580224
Armijo: th(5.9370444500437714E-6)=4.011849135226685; dx=-1.1719932839561151E11 evalInputDelta=18.347001285082875
Armijo: th(5.937044450043771E-7)=4.011852072920685; dx=-1.1719932839561151E11 evalInputDelta=18.346998347388876
Armijo: th(5.397313136403428E-8)=4.011852369659146; dx=-1.1719932839561151E11 evalInputDelta=18.346998050650416
Armijo: th(4.4977609470028565E-9)=4.011852396860187; dx=-1.1719932839561151E11 evalInputDelta=18.346998023449373
Armijo: th(3.4598161130791205E-10)=4.011852399142791; dx=-1.1719932839561151E11 evalInputDelta=18.34699802116677
Armijo: th(2.4712972236279432E-11)=4.601314183125898; dx=-5.0567901246237265E20 evalInputDelta=17.757536237183665
Armijo: th(1.6475314824186289E-12)=13.989302992763958; dx=-3.944612345763571E22 evalInputDelta=8.369547427545603
Armijo: th(1.029707176511643E-13)=22.358850420309505; dx=-1.1719932839506171E23 evalInputDelta=5.6843418860808015E-14
Armijo: th(6.057101038303783E-15)=22.358850420309558; dx=-1.1719932839506171E23 evalInputDelta=3.552713678800501E-15
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=3.6804001011569505
Fitness changed from 22.35885042030956 to 3.6804001011569505
Iteration 1 complete. Error: 3.6804001011569505 Total: 0.2675; Orientation: 0.0031; Line Search: 0.2196
th(0)=3.6804001011569505;dx=-0.30619150634027004
WOLFE (weak): th(2.154434690031884E-15)=3.6804001011569505; dx=-0.30619150634027004 evalInputDelta=0.0
WOLFE (weak): th(4.308869380063768E-15)=3.6804001011569505; dx=-0.30619150634027 evalInputDelta=0.0
WOLFE (weak): th(1.2926608140191303E-14)=3.6804001011569505; dx=-0.30619150634027 evalInputDelta=0.0
New Minimum: 3.6804001011569505 > 3.6804001011569496
WOLFE (weak): th(5.1706432560765214E-14)=3.6804001011569496; dx=-0.3061915063402699 evalInputDelta=8.881784197001252E-16
New Minimum: 3.6804001011569496 > 3.6804001011569456
WOLFE (weak): th(2.5853216280382605E-13)=3.6804001011569456; dx=-0.3061915063402694 evalInputDelta=4.884981308350689E-15
New Minimum: 3.6804001011569456 > 3.680400101156919
WOLFE (weak): th(1.5511929768229563E-12)=3.680400101156919; dx=-0.30619150634026654 evalInputDelta=3.1530333899354446E-14
New Minimum: 3.680400101156919 > 3.680400101156729
WOLFE (weak): th(1.0858350837760695E-11)=3.680400101156729; dx=-0.3061915063402458 evalInputDelta=2.2160051571518125E-13
New Minimum: 3.680400101156729 > 3.680400101155178
WOLFE (weak): th(8.686680670208556E-11)=3.680400101155178; dx=-0.3061915063400763 evalInputDelta=1.7723600365116E-12
New Minimum: 3.680400101155178 > 3.680400101140999
WOLFE (weak): th(7.8180126031877E-10)=3.680400101140999; dx=-0.30619150633852643 evalInputDelta=1.595168441781425E-11
New Minimum: 3.680400101140999 > 3.680400100997434
WOLFE (weak): th(7.818012603187701E-9)=3.680400100997434; dx=-0.30619150632283393 evalInputDelta=1.5951640008893264E-10
New Minimum: 3.680400100997434 > 3.680400099402268
WOLFE (weak): th(8.599813863506471E-8)=3.680400099402268; dx=-0.3061915061484733 evalInputDelta=1.7546826214243083E-9
New Minimum: 3.680400099402268 > 3.6804000801007626
WOLFE (weak): th(1.0319776636207765E-6)=3.6804000801007626; dx=-0.3061915040387094 evalInputDelta=2.105618790437802E-8
New Minimum: 3.6804000801007626 > 3.680399827426693
WOLFE (weak): th(1.3415709627070094E-5)=3.680399827426693; dx=-0.306191476420022 evalInputDelta=2.7373025757171376E-7
New Minimum: 3.680399827426693 > 3.6803962689698766
WOLFE (weak): th(1.878199347789813E-4)=3.6803962689698766; dx=-0.306191087464783 evalInputDelta=3.832187073893323E-6
New Minimum: 3.6803962689698766 > 3.680342626609877
WOLFE (weak): th(0.0028172990216847197)=3.680342626609877; dx=-0.306185225013288 evalInputDelta=5.747454707361044E-5
New Minimum: 3.680342626609877 > 3.679482625236605
WOLFE (weak): th(0.045076784346955515)=3.679482625236605; dx=-0.30609146706785356 evalInputDelta=9.17475920345634E-4
New Minimum: 3.679482625236605 > 3.6653850304219704
WOLFE (weak): th(0.7663053338982437)=3.6653850304219704; dx=-0.30461454771896823 evalInputDelta=0.015015070734980096
New Minimum: 3.6653850304219704 > 3.654663059600149
WOLFE (weak): th(13.793496010168386)=3.654663059600149; dx=-0.30356362556345184 evalInputDelta=0.025737041556801454
WOLFE (weak): th(262.07642419319933)=3.654663059600149; dx=-0.30356362556345184 evalInputDelta=0.025737041556801454
WOLFE (weak): th(5241.528483863986)=3.654663059600149; dx=-0.30356362556345184 evalInputDelta=0.025737041556801454
Armijo: th(110072.09816114372)=3.654663059600149; dx=-0.30356362556345184 evalInputDelta=0.025737041556801454
WOLFE (weak): th(57656.813322503855)=3.654663059600149; dx=-0.30356362556345184 evalInputDelta=0.025737041556801454
WOLFE (weak): th(83864.45574182378)=3.654663059600149; dx=-0.30356362556345184 evalInputDelta=0.025737041556801454
Armijo: th(96968.27695148374)=3.654663059600149; dx=-0.30356362556345184 evalInputDelta=0.025737041556801454
Armijo: th(90416.36634665375)=3.654663059600149; dx=-0.30356362556345184 evalInputDelta=0.025737041556801454
Armijo: th(87140.41104423878)=3.654663059600149; dx=-0.30356362556345184 evalInputDelta=0.025737041556801454
Armijo: th(85502.43339303127)=3.654663059600149; dx=-0.30356362556345184 evalInputDelta=0.025737041556801454
Armijo: th(84683.44456742753)=3.654663059600149; dx=-0.30356362556345184 evalInputDelta=0.025737041556801454
mu ~= nu (83864.45574182378): th(13.793496010168386)=3.654663059600149
Fitness changed from 3.6804001011569505 to 3.654663059600149
Iteration 2 complete. Error: 3.654663059600149 Total: 0.2235; Orientation: 0.0010; Line Search: 0.2146
th(0)=3.654663059600149;dx=-0.30127420049382714
Armijo: th(181562.72167914337)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(90781.36083957169)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(30260.453613190562)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(7565.113403297641)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(1513.022680659528)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(252.17044677658802)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(36.02434953951258)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(4.503043692439072)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(0.5003381880487858)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(0.05003381880487858)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(0.004548528982261689)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(3.7904408185514076E-4)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(2.9157237065780057E-5)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(2.0826597904128612E-6)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(1.388439860275241E-7)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(8.677749126720256E-9)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
WOLFE (weak): th(5.104558309835445E-10)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(4.5941024788519E-9)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
WOLFE (weak): th(2.5522791549177223E-9)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
WOLFE (weak): th(3.5731908168848114E-9)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(4.083646647868356E-9)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(3.828418732376584E-9)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
Armijo: th(3.7008047746306975E-9)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
WOLFE (weak): th(3.6369977957577545E-9)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
WOLFE (weak): th(3.668901285194226E-9)=3.654663059600149; dx=-0.30127420049382714 evalInputDelta=0.0
mu ~= nu (3.668901285194226E-9): th(0.0)=3.654663059600149
Fitness changed from 3.654663059600149 to 3.654663059600149
Static Iteration Total: 0.1653; Orientation: 0.0006; Line Search: 0.1597
Iteration 3 failed. Error: 3.654663059600149
Previous Error: 0.0 -> 3.654663059600149
Optimization terminated 3
Set StochasticSamplingSubnetLayer to random null seed
Final threshold in iteration 3: 3.654663059600149 (> 0.0) after 0.657s (< 30.000s)

Returns

    3.654663059600149

This training apply resulted in the following configuration:

TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):

    RefList<double[]> state = network.state();
    assert state != null;
    String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
        .orElse("");
    state.freeRef();
    return description;

Returns

And regressed input:

TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):

    return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
      return RefArrays.stream(x);
    }).limit(1).map(x -> {
      String temp_18_0015 = x.prettyPrint();
      x.freeRef();
      return temp_18_0015;
    }).reduce((a, b) -> a + "\n" + b).orElse("");

Returns

    [ 0.496, -0.608, 0.7, -0.128, 0.08 ]

To produce the following output:

TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):

    Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
    @Nullable
    Result eval = layer.eval(array);
    assert eval != null;
    TensorList tensorList = Result.getData(eval);
    String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
      String temp_18_0017 = x.prettyPrint();
      x.freeRef();
      return temp_18_0017;
    }).reduce((a, b) -> a + "\n" + b).orElse("");
    tensorList.freeRef();
    return temp_18_0016;

Returns

    [ 0.0, -1.216, 0.0, -0.0, 0.0 ]

Conjugate Gradient Descent

First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.

TrainingTester.java:452 executed in 0.15 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
      iterativeTrainer.setOrientation(new GradientDescent());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging 
Reset training subject: 4820370524383
Set StochasticSamplingSubnetLayer to random seed 4820370524383
Reset training subject: 4820373758185
Set StochasticSamplingSubnetLayer to random seed 4820373758185
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=27.39054724107596}, derivative=-1.674294360493827E23}
New Minimum: 27.39054724107596 > 6.273078836168155
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=6.273078836168155}, derivative=-1.674294360497909E11}, evalInputDelta = -21.117468404907804
New Minimum: 6.273078836168155 > 6.273078835923263
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=6.273078835923263}, derivative=-1.674294360497909E11}, evalInputDelta = -21.1174684051527
New Minimum: 6.273078835923263 > 6.273078834209013
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=6.273078834209013}, derivative=-1.674294360497909E11}, evalInputDelta = -21.117468406866948
New Minimum: 6.273078834209013 > 6.273078822209266
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=6.273078822209266}, derivative=-1.6742943604979086E11}, evalInputDelta = -21.117468418866693
New Minimum: 6.273078822209266 > 6.273078738211055
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=6.273078738211055}, derivative=-1.6742943604979086E11}, evalInputDelta = -21.117468502864906
New Minimum: 6.273078738211055 > 6.2730781502247215
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=6.2730781502247215}, derivative=-1.6742943604979086E11}, evalInputDelta = -21.117469090851237
New Minimum: 6.2730781502247215 > 6.273074034376117
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=6.273074034376117}, derivative=-1.6742943604979086E11}, evalInputDelta = -21.11747320669984
New Minimum: 6.273074034376117 > 6.273045226166126
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=6.273045226166126}, derivative=-1.674294360497908E11}, evalInputDelta = -21.117502014909835
New Minimum: 6.273045226166126 > 6.272843702258777
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=6.272843702258777}, derivative=-1.674294360497903E11}, evalInputDelta = -21.11770353881718
New Minimum: 6.272843702258777 > 6.27143950543433
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=6.27143950543433}, derivative=-1.6742943604978708E11}, evalInputDelta = -21.11910773564163
New Minimum: 6.27143950543433 > 6.2643633717878515
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=6.2643633717878515}, derivative=-1.674294360497685E11}, evalInputDelta = -21.12618386928811
New Minimum: 6.2643633717878515 > 6.229466733572048
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=6.229466733572048}, derivative=-1.674294360497009E11}, evalInputDelta = -21.16108050750391
New Minimum: 6.229466733572048 > 6.111271623288262
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=6.111271623288262}, derivative=-1.6742943604961368E11}, evalInputDelta = -21.279275617787697
New Minimum: 6.111271623288262 > 6.041983284016375
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
F(23263.0513987207) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
F(162841.3597910449) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
F(1139889.5185373144) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
F(7979226.6297612) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
F(5.58545864083284E7) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
F(3.909821048582988E8) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
F(2.7368747340080914E9) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
F(1.915812313805664E10) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
6.041983284016375 <= 27.39054724107596
F(1.0E10) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-1.6742943604959903E11}, evalInputDelta = -21.348563957059586
Right bracket at 1.0E10
Converged to right
Fitness changed from 27.39054724107596 to 6.041983284016375
Iteration 1 complete. Error: 6.041983284016375 Total: 0.1352; Orientation: 0.0003; Line Search: 0.1257
F(0.0) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-0.3436005451851852}
F(1.0E10) = LineSearchPoint{point=PointSample{avg=6.041983284016375}, derivative=-0.3436005451851851}, evalInputDelta = 0.0
6.041983284016375 <= 6.041983284016375
Converged to right
Fitness changed from 6.041983284016375 to 6.041983284016375
Static Iteration Total: 0.0109; Orientation: 0.0005; Line Search: 0.0071
Iteration 2 failed. Error: 6.041983284016375
Previous Error: 0.0 -> 6.041983284016375
Optimization terminated 2
Set StochasticSamplingSubnetLayer to random null seed
Final threshold in iteration 2: 6.041983284016375 (> 0.0) after 0.146s (< 30.000s)

Returns

    6.041983284016375

This training apply resulted in the following configuration:

TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):

    RefList<double[]> state = network.state();
    assert state != null;
    String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
        .orElse("");
    state.freeRef();
    return description;

Returns

And regressed input:

TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):

    return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
      return RefArrays.stream(x);
    }).limit(1).map(x -> {
      String temp_18_0015 = x.prettyPrint();
      x.freeRef();
      return temp_18_0015;
    }).reduce((a, b) -> a + "\n" + b).orElse("");

Returns

    [ 0.496, -0.608, 0.7, -0.128, 0.08 ]

To produce the following output:

TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):

    Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
    @Nullable
    Result eval = layer.eval(array);
    assert eval != null;
    TensorList tensorList = Result.getData(eval);
    String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
      String temp_18_0017 = x.prettyPrint();
      x.freeRef();
      return temp_18_0017;
    }).reduce((a, b) -> a + "\n" + b).orElse("");
    tensorList.freeRef();
    return temp_18_0016;

Returns

    [ 0.0, -1.216, 0.0, -0.0, 0.0 ]

Limited-Memory BFGS

Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.

TrainingTester.java:509 executed in 1.17 seconds (0.000 gc):

    IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
    try {
      iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
      iterativeTrainer.setOrientation(new LBFGS());
      iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
      iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
      iterativeTrainer.setIterationsPerSample(100);
      iterativeTrainer.setMaxIterations(250);
      iterativeTrainer.setTerminateThreshold(0);
      return iterativeTrainer.run();
    } finally {
      iterativeTrainer.freeRef();
    }
Logging 
Reset training subject: 4820524331969
Set StochasticSamplingSubnetLayer to random seed 4820524331969
Reset training subject: 4820527654183
Set StochasticSamplingSubnetLayer to random seed 4820527654183
Adding measurement 6a803277 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 22.515017085551026 < 22.515017085551026. Total: 1
th(0)=22.515017085551026;dx=-1.834857560493827E23
Adding measurement 76aa959a to history. Total: 1
New Minimum: 22.515017085551026 > 0.3679994660766245
Armijo: th(2.154434690031884)=0.3679994660766245; dx=-1.8348575604961374E11 evalInputDelta=22.1470176194744
Non-optimal measurement 0.47338582750209374 < 0.3679994660766245. Total: 2
Armijo: th(1.077217345015942)=0.47338582750209374; dx=-1.8348575604965155E11 evalInputDelta=22.041631258048934
Non-optimal measurement 0.6499734173148792 < 0.3679994660766245. Total: 2
Armijo: th(0.3590724483386473)=0.6499734173148792; dx=-1.8348575604975336E11 evalInputDelta=21.865043668236147
Non-optimal measurement 0.7622625535548796 < 0.3679994660766245. Total: 2
Armijo: th(0.08976811208466183)=0.7622625535548796; dx=-1.8348575604986044E11 evalInputDelta=21.752754531996146
Non-optimal measurement 0.7983804903706515 < 0.3679994660766245. Total: 2
Armijo: th(0.017953622416932366)=0.7983804903706515; dx=-1.834857560499146E11 evalInputDelta=21.716636595180375
Non-optimal measurement 0.8064465157985652 < 0.3679994660766245. Total: 2
Armijo: th(0.002992270402822061)=0.8064465157985652; dx=-1.8348575604992935E11 evalInputDelta=21.70857056975246
Non-optimal measurement 0.8078519772087237 < 0.3679994660766245. Total: 2
Armijo: th(4.2746720040315154E-4)=0.8078519772087237; dx=-1.8348575604993207E11 evalInputDelta=21.707165108342302
Non-optimal measurement 0.8080575212738923 < 0.3679994660766245. Total: 2
Armijo: th(5.343340005039394E-5)=0.8080575212738923; dx=-1.8348575604993243E11 evalInputDelta=21.706959564277135
Non-optimal measurement 0.8080836327712373 < 0.3679994660766245. Total: 2
Armijo: th(5.9370444500437714E-6)=0.8080836327712373; dx=-1.834857560499325E11 evalInputDelta=21.70693345277979
Non-optimal measurement 0.8080865704652369 < 0.3679994660766245. Total: 2
Armijo: th(5.937044450043771E-7)=0.8080865704652369; dx=-1.8348575604993253E11 evalInputDelta=21.70693051508579
Non-optimal measurement 0.8080868672036978 < 0.3679994660766245. Total: 2
Armijo: th(5.397313136403428E-8)=0.8080868672036978; dx=-1.8348575604993253E11 evalInputDelta=21.706930218347328
Non-optimal measurement 0.808086894404739 < 0.3679994660766245. Total: 2
Armijo: th(4.4977609470028565E-9)=0.808086894404739; dx=-1.8348575604993253E11 evalInputDelta=21.706930191146288
Non-optimal measurement 0.808086896687344 < 0.3679994660766245. Total: 2
Armijo: th(3.4598161130791205E-10)=0.808086896687344; dx=-1.8348575604993253E11 evalInputDelta=21.706930188863684
Non-optimal measurement 1.1451772321772506 < 0.3679994660766245. Total: 2
Armijo: th(2.4712972236279432E-11)=1.1451772321772506; dx=-4.551111112953967E20 evalInputDelta=21.369839853373776
Non-optimal measurement 13.619200252991831 < 0.3679994660766245. Total: 2
Armijo: th(1.6475314824186289E-12)=13.619200252991831; dx=-4.352998716077798E22 evalInputDelta=8.895816832559195
Non-optimal measurement 22.515017085550973 < 0.3679994660766245. Total: 2
Armijo: th(1.029707176511643E-13)=22.515017085550973; dx=-1.834857560493827E23 evalInputDelta=5.3290705182007514E-14
Non-optimal measurement 22.515017085551023 < 0.3679994660766245. Total: 2
Armijo: th(6.057101038303783E-15)=22.515017085551023; dx=-1.834857560493827E23 evalInputDelta=3.552713678800501E-15
Non-optimal measurement 0.3679994660766245 < 0.3679994660766245. Total: 2
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=0.3679994660766245
Fitness changed from 22.515017085551026 to 0.3679994660766245
Iteration 1 complete. Error: 0.3679994660766245 Total: 0.0910; Orientation: 0.0025; Line Search: 0.0792
Non-optimal measurement 0.3679994660766245 < 0.3679994660766245. Total: 2
LBFGS Accumulation History: 2 points
Non-optimal measurement 0.3679994660766245 < 0.3679994660766245. Total: 2
th(0)=0.3679994660766245;dx=-0.3094778607150346
Non-optimal measurement 0.3679994660766245 < 0.3679994660766245. Total: 2
WOLFE (weak): th(2.154434690031884E-15)=0.3679994660766245; dx=-0.3094778607150346 evalInputDelta=0.0
Adding measurement 34f30b7c to history. Total: 2
New Minimum: 0.3679994660766245 > 0.3679994660766243
WOLFE (weak): th(4.308869380063768E-15)=0.3679994660766243; dx=-0.3094778607150346 evalInputDelta=1.6653345369377348E-16
Adding measurement dd1b1fe to history. Total: 3
New Minimum: 0.3679994660766243 > 0.3679994660766241
WOLFE (weak): th(1.2926608140191303E-14)=0.3679994660766241; dx=-0.3094778607150346 evalInputDelta=3.885780586188048E-16
Adding measurement 77bd1bf3 to history. Total: 4
New Minimum: 0.3679994660766241 > 0.3679994660766229
WOLFE (weak): th(5.1706432560765214E-14)=0.3679994660766229; dx=-0.3094778607150344 evalInputDelta=1.5543122344752192E-15
Adding measurement 4c42d04f to history. Total: 5
New Minimum: 0.3679994660766229 > 0.3679994660766165
WOLFE (weak): th(2.5853216280382605E-13)=0.3679994660766165; dx=-0.3094778607150337 evalInputDelta=7.993605777301127E-15
Adding measurement 1a80aa04 to history. Total: 6
New Minimum: 0.3679994660766165 > 0.36799946607657624
WOLFE (weak): th(1.5511929768229563E-12)=0.36799946607657624; dx=-0.30947786071502925 evalInputDelta=4.823919041996305E-14
Adding measurement 515d7ad1 to history. Total: 7
New Minimum: 0.36799946607657624 > 0.36799946607628653
WOLFE (weak): th(1.0858350837760695E-11)=0.36799946607628653; dx=-0.309477860714997 evalInputDelta=3.3795188869589765E-13
Adding measurement 5986c627 to history. Total: 8
New Minimum: 0.36799946607628653 > 0.3679994660739208
WOLFE (weak): th(8.686680670208556E-11)=0.3679994660739208; dx=-0.3094778607147339 evalInputDelta=2.7036706207184125E-12
Adding measurement 7abf864b to history. Total: 9
New Minimum: 0.3679994660739208 > 0.3679994660522912
WOLFE (weak): th(7.8180126031877E-10)=0.3679994660522912; dx=-0.3094778607123282 evalInputDelta=2.4333257631070637E-11
Adding measurement 478fddcd to history. Total: 10
New Minimum: 0.3679994660522912 > 0.36799946583329124
WOLFE (weak): th(7.818012603187701E-9)=0.36799946583329124; dx=-0.30947786068797073 evalInputDelta=2.4333324244452115E-10
Adding measurement 2459e8c1 to history. Total: 11
New Minimum: 0.36799946583329124 > 0.3679994633999583
WOLFE (weak): th(8.599813863506471E-8)=0.3679994633999583; dx=-0.30947786041733183 evalInputDelta=2.6766661664900937E-9
Adding measurement 4da9191a to history. Total: 12
New Minimum: 0.3679994633999583 > 0.367999433956632
WOLFE (weak): th(1.0319776636207765E-6)=0.367999433956632; dx=-0.3094778571426016 evalInputDelta=3.211999249908004E-8
Adding measurement 300b0ea6 to history. Total: 13
New Minimum: 0.367999433956632 > 0.3679990485170089
WOLFE (weak): th(1.3415709627070094E-5)=0.3679990485170089; dx=-0.30947781427346976 evalInputDelta=4.1755961560641097E-7
Adding measurement 5eebb96d to history. Total: 14
New Minimum: 0.3679990485170089 > 0.36799362029870225
WOLFE (weak): th(1.878199347789813E-4)=0.36799362029870225; dx=-0.30947721054574556 evalInputDelta=5.845777922230955E-6
Adding measurement 2b568aa5 to history. Total: 15
New Minimum: 0.36799362029870225 > 0.36791179222724973
WOLFE (weak): th(0.0028172990216847197)=0.36791179222724973; dx=-0.30946811102858945 evalInputDelta=8.767384937474754E-5
Adding measurement 719c9ed6 to history. Total: 16
New Minimum: 0.36791179222724973 > 0.366599969982348
WOLFE (weak): th(0.045076784346955515)=0.366599969982348; dx=-0.3093225956838221 evalInputDelta=0.0013994960942764756
Adding measurement a4b3735 to history. Total: 17
New Minimum: 0.366599969982348 > 0.3451105273183655
WOLFE (weak): th(0.7663053338982437)=0.3451105273183655; dx=-0.30703355046543024 evalInputDelta=0.02288893875825898
Adding measurement 73874529 to history. Total: 18
New Minimum: 0.3451105273183655 > 0.2947308919032378
WOLFE (weak): th(13.793496010168386)=0.2947308919032378; dx=-0.3025716280115852 evalInputDelta=0.07326857417338667
Non-optimal measurement

...skipping 11037 bytes...

73, 0.36799362029870225, 0.3679990485170089, 0.367999433956632, 0.3679994633999583
Rejected: LBFGS Orientation magnitude: 6.382e+01, gradient 5.456e-01, dot -0.986; [30a438f3-6511-4358-9c4e-7dbf65201c66 = 1.000/1.000e+00, 7dbd2060-c29e-4c82-b9e1-df5c5de5ca1f = 0.000e+00, 460178b7-b4c8-4706-ab10-6bd3cca67a84 = 1.000/1.000e+00, 63d6edda-66a3-448c-a97d-22c5f3c20678 = 1.000/1.000e+00, c31943ca-2554-4a03-a876-28ff255b1170 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.2947308919032378, 0.3451105273183655, 0.366599969982348, 0.36791179222724973, 0.36799362029870225, 0.3679990485170089, 0.367999433956632
Rejected: LBFGS Orientation magnitude: 7.447e+01, gradient 5.456e-01, dot -0.985; [7dbd2060-c29e-4c82-b9e1-df5c5de5ca1f = 0.000e+00, c31943ca-2554-4a03-a876-28ff255b1170 = 1.000/1.000e+00, 30a438f3-6511-4358-9c4e-7dbf65201c66 = 1.000/1.000e+00, 63d6edda-66a3-448c-a97d-22c5f3c20678 = 1.000/1.000e+00, 460178b7-b4c8-4706-ab10-6bd3cca67a84 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.2947308919032378, 0.3451105273183655, 0.366599969982348, 0.36791179222724973, 0.36799362029870225, 0.3679990485170089
Rejected: LBFGS Orientation magnitude: 9.677e+01, gradient 5.456e-01, dot -0.995; [c31943ca-2554-4a03-a876-28ff255b1170 = 1.000/1.000e+00, 460178b7-b4c8-4706-ab10-6bd3cca67a84 = 1.000/1.000e+00, 63d6edda-66a3-448c-a97d-22c5f3c20678 = 1.000/1.000e+00, 7dbd2060-c29e-4c82-b9e1-df5c5de5ca1f = 0.000e+00, 30a438f3-6511-4358-9c4e-7dbf65201c66 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.2947308919032378, 0.3451105273183655, 0.366599969982348, 0.36791179222724973, 0.36799362029870225
Rejected: LBFGS Orientation magnitude: 9.525e+01, gradient 5.456e-01, dot -0.995; [63d6edda-66a3-448c-a97d-22c5f3c20678 = 1.000/1.000e+00, c31943ca-2554-4a03-a876-28ff255b1170 = 1.000/1.000e+00, 30a438f3-6511-4358-9c4e-7dbf65201c66 = 1.000/1.000e+00, 460178b7-b4c8-4706-ab10-6bd3cca67a84 = 1.000/1.000e+00, 7dbd2060-c29e-4c82-b9e1-df5c5de5ca1f = 0.000e+00]
Orientation rejected. Popping history element from 0.2947308919032378, 0.3451105273183655, 0.366599969982348, 0.36791179222724973
LBFGS Accumulation History: 3 points
Removed measurement 73874529 to history. Total: 18
Removed measurement a4b3735 to history. Total: 17
Removed measurement 719c9ed6 to history. Total: 16
Removed measurement 2b568aa5 to history. Total: 15
Removed measurement 5eebb96d to history. Total: 14
Removed measurement 300b0ea6 to history. Total: 13
Removed measurement 4da9191a to history. Total: 12
Removed measurement 2459e8c1 to history. Total: 11
Removed measurement 478fddcd to history. Total: 10
Removed measurement 7abf864b to history. Total: 9
Removed measurement 5986c627 to history. Total: 8
Removed measurement 515d7ad1 to history. Total: 7
Removed measurement 1a80aa04 to history. Total: 6
Removed measurement 4c42d04f to history. Total: 5
Removed measurement 77bd1bf3 to history. Total: 4
Removed measurement dd1b1fe to history. Total: 3
Adding measurement 354c5225 to history. Total: 3
th(0)=0.2947308919032378;dx=-0.2977111861728395
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(511918.87279559486)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(255959.43639779743)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(85319.81213259914)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(21329.953033149784)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(4265.990606629957)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(710.9984344383262)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(101.57120491976089)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(12.696400614970111)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(1.4107111794411233)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(0.14107111794411234)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(0.012824647085828395)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(0.0010687205904856996)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(8.220927619120767E-5)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(5.872091156514833E-6)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(3.914727437676555E-7)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(2.446704648547847E-8)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(1.439238028557557E-9)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
WOLFE (weak): th(7.995766825319762E-11)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(7.595978484053773E-10)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
WOLFE (weak): th(4.1977775832928745E-10)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(5.896878033673324E-10)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(5.0473278084831E-10)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
WOLFE (weak): th(4.6225526958879873E-10)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(4.834940252185543E-10)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(4.728746474036766E-10)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
Armijo: th(4.675649584962377E-10)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
WOLFE (weak): th(4.649101140425182E-10)=0.2947308919032378; dx=-0.2977111861728395 evalInputDelta=0.0
Non-optimal measurement 0.2947308919032378 < 0.2947308919032378. Total: 4
mu ~= nu (4.649101140425182E-10): th(0.0)=0.2947308919032378
Fitness changed from 0.2947308919032378 to 0.2947308919032378
Static Iteration Total: 0.9109; Orientation: 0.7893; Line Search: 0.1187
Iteration 3 failed. Error: 0.2947308919032378
Previous Error: 0.0 -> 0.2947308919032378
Optimization terminated 3
Set StochasticSamplingSubnetLayer to random null seed
Final threshold in iteration 3: 0.2947308919032378 (> 0.0) after 1.167s (< 30.000s)

Returns

    0.2947308919032378

This training apply resulted in the following configuration:

TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):

    RefList<double[]> state = network.state();
    assert state != null;
    String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
        .orElse("");
    state.freeRef();
    return description;

Returns

And regressed input:

TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):

    return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
      return RefArrays.stream(x);
    }).limit(1).map(x -> {
      String temp_18_0015 = x.prettyPrint();
      x.freeRef();
      return temp_18_0015;
    }).reduce((a, b) -> a + "\n" + b).orElse("");

Returns

    [ 0.496, -0.608, 0.7, -0.128, 0.08 ]

To produce the following output:

TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):

    Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
    @Nullable
    Result eval = layer.eval(array);
    assert eval != null;
    TensorList tensorList = Result.getData(eval);
    String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
      String temp_18_0017 = x.prettyPrint();
      x.freeRef();
      return temp_18_0017;
    }).reduce((a, b) -> a + "\n" + b).orElse("");
    tensorList.freeRef();
    return temp_18_0016;

Returns

    [ 0.0, -1.216, 0.0, -0.0, 0.0 ]

TrainingTester.java:432 executed in 0.11 seconds (0.000 gc):

    return TestUtil.compare(title + " vs Iteration", runs);
Logging 
Plotting range=[1.0, -0.5305743416526685], [2.0, 0.7811795194030858]; valueStats=DoubleSummaryStatistics{count=5, sum=14.039777, min=0.294731, average=2.807955, max=6.041983}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS

Returns

Result

TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):

    return TestUtil.compareTime(title + " vs Time", runs);
Logging 
Plotting range=[0.0, -0.5305743416526685], [0.223, 0.7811795194030858]; valueStats=DoubleSummaryStatistics{count=5, sum=14.039777, min=0.294731, average=2.807955, max=6.041983}
Plotting 2 points for GD
Only 1 points for CjGD
Plotting 2 points for LBFGS

Returns

Result

Results

TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):

    return grid(inputLearning, modelLearning, completeLearning);

Returns

Result

TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):

    return new ComponentResult(null == inputLearning ? null : inputLearning.value,
        null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);

Returns

    {"input":{ "LBFGS": { "type": "NonConverged", "value": 0.2947308919032378 }, "CjGD": { "type": "NonConverged", "value": 6.041983284016375 }, "GD": { "type": "NonConverged", "value": 3.654663059600149 } }, "model":null, "complete":null}

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

detailsresult
{"input":{ "LBFGS": { "type": "NonConverged", "value": 0.2947308919032378 }, "CjGD": { "type": "NonConverged", "value": 6.041983284016375 }, "GD": { "type": "NonConverged", "value": 3.654663059600149 } }, "model":null, "complete":null}OK
 
  {
    "result": "OK",
    "performance": {
      "execution_time": "2.929",
      "gc_time": "0.503"
    },
    "created_on": 1586739407499,
    "file_name": "trainingTest",
    "report": {
      "simpleName": "Basic",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.BinaryNoiseLayerTest.Basic",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/BinaryNoiseLayerTest.java",
      "javaDoc": ""
    },
    "training_analysis": {
      "input": {
        "LBFGS": {
          "type": "NonConverged",
          "value": 0.2947308919032378
        },
        "CjGD": {
          "type": "NonConverged",
          "value": 6.041983284016375
        },
        "GD": {
          "type": "NonConverged",
          "value": 3.654663059600149
        }
      }
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/StochasticSamplingSubnetLayer/Basic/trainingTest/202004135647",
    "id": "60c15220-c863-4e27-856d-35b7b8797f1b",
    "report_type": "Components",
    "display_name": "Comparative Training",
    "target": {
      "simpleName": "StochasticSamplingSubnetLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.StochasticSamplingSubnetLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/StochasticSamplingSubnetLayer.java",
      "javaDoc": ""
    }
  }