Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
Using Seed 7674014371630648320
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 1.364, 0.708, 1.964 ], [ 0.612, 1.72, 0.384 ], [ -0.34, 1.804, -1.444 ] ],
[ [ 1.644, -1.8, -1.044 ], [ 0.772, 1.424, 0.268 ], [ -1.544, 1.128, -1.416 ] ],
[ [ 0.76, 0.508, -0.476 ], [ -1.088, -1.516, 1.104 ], [ 1.968, -0.436, -0.276 ] ]
]
[
[ [ -1.544, 0.708, 1.804 ], [ 0.772, -0.476, 0.76 ], [ 1.364, -1.516, 0.384 ] ],
[ [ -0.34, 1.104, 0.612 ], [ -1.8, 1.964, 1.424 ], [ 1.644, -1.088, -1.416 ] ],
[ [ -1.444, -1.044, 1.72 ], [ 0.508, -0.436, -0.276 ], [ 0.268, 1.128, 1.968 ] ]
]
[
[ [ -0.476, 1.72, 0.508 ], [ 1.104, -1.8, 0.76 ], [ -0.34, 1.644, 1.364 ] ],
[ [ 0.772, -1.044, 0.268 ], [ -1.516, -1.444, 0.612 ], [ 1.964, -1.416, -0.276 ] ],
[ [ 1.968, -1.544, 0.384 ], [ 1.128, 1.424, -0.436 ], [ -1.088, 0.708, 1.804 ] ]
]
[
[ [ 0.508, -1.044, -1.444 ], [ 0.76, 0.268, -0.34 ], [ -0.476, 0.612, -0.276 ] ],
[ [ 1.72, 1.364, 1.804 ], [ 1.644, -1.544, 1.424 ], [ -1.8, 1.104, 0.772 ] ],
[ [ -1.416, 1.128, -1.088 ], [ -0.436, -1.516, 0.708 ], [ 0.384, 1.968, 1.964 ] ]
]
[
[ [ -1.444, 0.612, 1.364 ], [ 0.772, -0.34, -0.276 ], [ 0.508, -1.416, 1.424 ] ],
[ [ -1.088, 0.708, 0.384 ], [ 1.128, 0.268, -1.044 ], [ -1.8, -0.476, 1.104 ] ],
[ [ -0.436, 1.804, -1.516 ], [ 1.968, 1.644, -1.544 ], [ 1.72, 1.964, 0.76 ] ]
]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 0.25 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1567360266288
BACKPROP_AGG_SIZE = 3
THREADS = 64
SINGLE_THREADED = false
Initialized CoreSettings = {
"backpropAggregationSize" : 3,
"jvmThreads" : 64,
"singleThreaded" : false
}
Reset training subject: 1567396666787
Constructing line search parameters: GD
th(0)=595.4813085965291;dx=-1.7402339726038745E27
New Minimum: 595.4813085965291 > 368.56289607579635
Armijo: th(2.154434690031884)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(1.077217345015942)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(0.3590724483386473)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(0.08976811208466183)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(0.017953622416932366)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(0.002992270402822061)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(4.2746720040315154E-4)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(5.343340005039394E-5)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(5.9370444500437714E-6)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(5.937044450043771E-7)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(5.397313136403428E-8)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(4.4977609470028565E-9)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(3.4598161130791205E-10)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Armijo: th(2.4712972236279432E-11)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
New Minimum: 368.56289607579635 > 228.61303229221704
Armijo: th(1.6475314824186289E-12)=228.61303229221704; dx=1.8076530012193962E26 evalInputDelta=366.868276304312
New Minimum: 228.61303229221704 > 154.48635002569895
Armijo: th(1.029707176511643E-13)=154.48635002569895; dx=5.623806176836239E25 evalInputDelta=440.99495857083014
Armijo: th(6.057101038303783E-15)=570.6137921861157; dx=-1.698781103593462E27 evalInputDelta=24.867516410413373
MIN ALPHA (3.3650561323909904E-16): th(1.029707176511643E-13)=154.48635002569895
Fitness changed from 595.4813085965291 to 154.48635002569895
Iteration 1 complete. Error: 154.48635002569895 Total: 0.2125; Orientation: 0.0032; Line Search: 0.1612
th(0)=154.48635002569895;dx=-1.6650278653044866E26
Armijo: th(2.154434690031884E-15)=154.51100104691415; dx=-1.665027865304485E26 evalInputDelta=-0.024651021215191804
Armijo: th(1.077217345015942E-15)=154.49862819951423; dx=-1.665027865304486E26 evalInputDelta=-0.012278173815275295
MIN ALPHA (3.5907244833864734E-16): th(0.0)=154.48635002569895
Fitness changed from 154.48635002569895 to 154.48635002569895
Static Iteration Total: 0.0216; Orientation: 0.0009; Line Search: 0.0166
Iteration 2 failed. Error: 154.48635002569895
Previous Error: 0.0 -> 154.48635002569895
Optimization terminated 2
Final threshold in iteration 2: 154.48635002569895 (> 0.0) after 0.235s (< 30.000s)
Returns
154.48635002569895
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 2.0106331219804336, -0.6884630013314543, -1.8616017282509296 ], [ 1.0758123788543146, -0.035729741955893124, 0.82313424598861 ], [ 0.021330930567547757, 0.7966918678738331, 0.40065346730447804 ] ],
[ [ 0.9264735680470436, 0.3441371943370426, 0.3814517633253778 ], [ -1.012764142503945, 1.4400009631208956, -0.1832377839950166 ], [ 2.0929276382485695, 0.24239977450946198, 1.7913717079471927 ] ],
[ [ -1.4141175061121976, 1.385974691925922, -1.5653045127430025 ], [ 1.5392362661293055, -1.2617438850787186, -0.19628529058195887 ], [ 1.3437964566392833, -1.1255414798562238, 2.015614206621464 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 7.111263122704816, -11.841466829163995, 4.8365013321812, 19.75418936496487 ], [ 1.17265643294862, -0.05485260618738952, 1.4121487916457127, 1.6135195645156712 ] ],
[ [ -2.7548878165825714, -4.112724401871645, 5.735025738055015, 7.20466242839097 ], [ -8.081432801834774, -0.78442272332373, -14.38808401264061, 2.527569799056083 ] ]
]
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.74 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1567609813309
Reset training subject: 1567612609337
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=595.4813085965291}, derivative=-1.7402339726038745E27}
New Minimum: 595.4813085965291 > 368.56289607579635
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=368.56289607579635}, derivative=1.899543027252227E26}, evalInputDelta = -226.91841252073272
368.56289607579635 <= 595.4813085965291
Converged to right
Fitness changed from 595.4813085965291 to 368.56289607579635
Iteration 1 complete. Error: 368.56289607579635 Total: 0.0255; Orientation: 0.0009; Line Search: 0.0159
F(0.0) = LineSearchPoint{point=PointSample{avg=368.56289607579635}, derivative=-5.120129843058287E26}
New Minimum: 368.56289607579635 > 102.56758825207275
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=102.56758825207275}, derivative=-7.001634504362205E25}, evalInputDelta = -265.9953078237236
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=195.23425217935088}, derivative=9.419726654112268E25}, evalInputDelta = -173.32864389644547
195.23425217935088 <= 368.56289607579635
F(5.912290142130357E-10) = LineSearchPoint{point=PointSample{avg=167.10454639432203}, derivative=8.782954138325496E25}, evalInputDelta = -201.45834968147432
Right bracket at 5.912290142130357E-10
F(5.046606717782408E-10) = LineSearchPoint{point=PointSample{avg=167.10454639432203}, derivative=8.782954138325496E25}, evalInputDelta = -201.45834968147432
Right bracket at 5.046606717782408E-10
F(4.307677524565741E-10) = LineSearchPoint{point=PointSample{avg=152.43136894091933}, derivative=8.276985333884196E25}, evalInputDelta = -216.13152713487702
Right bracket at 4.307677524565741E-10
F(3.7082220353131473E-10) = LineSearchPoint{point=PointSample{avg=107.25080317823681}, derivative=7.717238630275073E25}, evalInputDelta = -261.3120928975595
Right bracket at 3.7082220353131473E-10
F(3.2225135339123627E-10) = LineSearchPoint{point=PointSample{avg=105.26587904053608}, derivative=7.693441451526746E25}, evalInputDelta = -263.2970170352603
Right bracket at 3.2225135339123627E-10
F(2.801555410155738E-10) = LineSearchPoint{point=PointSample{avg=105.26587904053608}, derivative=7.693441451526746E25}, evalInputDelta = -263.2970170352603
Right bracket at 2.801555410155738E-10
F(2.4355872003566066E-10) = LineSearchPoint{point=PointSample{avg=105.26587904053608}, derivative=7.6934414515267455E25}, evalInputDelta = -263.2970170352603
Right bracket at 2.4355872003566066E-10
New Minimum: 102.56758825207275 > 89.82486866754707
F(2.1174255519048146E-10) = LineSearchPoint{point=PointSample{avg=89.82486866754707}, derivative=6.248427129876671E25}, evalInputDelta = -278.7380274082493
Right bracket at 2.1174255519048146E-10
New Minimum: 89.82486866754707 > 70.62789622741079
F(1.887127162574958E-10) = LineSearchPoint{point=PointSample{avg=70.62789622741079}, derivative=8.701144236469057E24}, evalInputDelta = -297.9349998483856
Right bracket at 1.887127162574958E-10
F(1.8555932272848006E-10) = LineSearchPoint{point=PointSample{avg=70.62789622741079}, derivative=8.701144236469059E24}, evalInputDelta = -297.9349998483856
Right bracket at 1.8555932272848006E-10
F(1.824586224728486E-10) = LineSearchPoint{point=PointSample{avg=70.62789622741079}, derivative=8.701144236469059E24}, evalInputDelta = -297.9349998483856
Right bracket at 1.824586224728486E-10
F(1.7940973498487496E-10) = LineSearchPoint{point=PointSample{avg=70.62789622741079}, derivative=8.701144236469057E24}, evalInputDelta = -297.9349998483856
Loops = 12
Fitness changed from 368.56289607579635 to 70.62789622741079
Iteration 2 complete. Error: 70.62789622741079 Total: 0.1499; Orientation: 0.0008; Line Search: 0.1458
F(0.0) = LineSearchPoint{point=PointSample{avg=70.62789622741079}, derivative=-4.311721637075156E25}
New Minimum: 70.62789622741079 > 0.0
F(1.7940973498487496E-10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=2.3765400412838234E13}, evalInputDelta = -70.62789622741079
0.0 <= 70.62789622741079
Converged to right
Fitness changed from 70.62789622741079 to 0.0
Iteration 3 complete. Error: 0.0 Total: 0.0098; Orientation: 0.0007; Line Search: 0.0069
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}
F(1.7940973498487496E-10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(1.2558681448941246E-9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(8.791077014258872E-9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(6.15375390998121E-8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(4.307627736986847E-7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(3.015339415890793E-6) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(2.110737591123555E-5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(1.4775163137864886E-4) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(0.001034261419650542) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(0.007239829937553795) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(0.05067880956287656) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(0.35475166694013593) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(2.4832616685809517) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(17.382831680066662) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(121.67982176046664) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(9.359986289266665) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(65.51990402486666) = LineSearchPoint{point=PointSample{avg=6.275380677314553}, derivative=1.0630354982692328E13}, evalInputDelta = 6.275380677314553
F(5.039992617297435) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(35.279948321082045) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(246.95963824757433) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(18.99689524981341) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(132.97826674869387) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(10.229097442207221) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(71.60368209545055) = LineSearchPoint{point=PointSample{avg=17.075882630376277}, derivative=2.218978039638646E13}, evalInputDelta = 17.075882630376277
F(5.507975545803888) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(38.55582882062722) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(269.89080174439056) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(20.760830903414657) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(145.3258163239026) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(11.178908947992507) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(78.25236263594755) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(6.019412510457504) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(42.135887573202524) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(294.95121301241767) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(22.688554847109053) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(158.81988392976336) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(12.216914148443335) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(85.51839903910334) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(6.578338387623334) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(46.04836871336334) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(322.33858099354336) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(24.795275461041797) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(173.56692822729258) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(13.351302171330198) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(93.45911519931138) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(7.189162707639337) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(50.324138953475355) = LineSearchPoint{point=PointSample{avg=6.275380677314553}, derivative=1.0630354982692328E13}, evalInputDelta = 6.275380677314553
F(3.8710876118057964) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(27.097613282640573) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(189.68329297848402) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.088698542747089E13}, evalInputDelta = 33.63230206377308
F(14.591022536806463) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
F(102.13715775764524) = LineSearchPoint{point=PointSample{avg=33.63230206377308}, derivative=3.0886985427470895E13}, evalInputDelta = 33.63230206377308
F(7.856704442895788) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Loops = 52
F(3.928352221447894) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Right bracket at 3.928352221447894
F(1.964176110723947) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Right bracket at 1.964176110723947
F(0.9820880553619735) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Right bracket at 0.9820880553619735
F(0.49104402768098676) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Right bracket at 0.49104402768098676
F(0.24552201384049338) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Right bracket at 0.24552201384049338
F(0.12276100692024669) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Right bracket at 0.12276100692024669
F(0.061380503460123345) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Right bracket at 0.061380503460123345
F(0.030690251730061673) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Right bracket at 0.030690251730061673
F(0.015345125865030836) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Right bracket at 0.015345125865030836
F(0.007672562932515418) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Right bracket at 0.007672562932515418
F(0.003836281466257709) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Right bracket at 0.003836281466257709
F(0.0019181407331288545) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-3076.975928191914}, evalInputDelta = 0.0
Loops = 12
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.5516; Orientation: 0.0008; Line Search: 0.5481
Iteration 4 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 4
Final threshold in iteration 4: 0.0 (> 0.0) after 0.738s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 0.15 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1568352949736
Reset training subject: 1568355667017
Adding measurement 44fb0731 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 595.4813085965291 < 595.4813085965291. Total: 1
th(0)=595.4813085965291;dx=-1.7402339726038745E27
Adding measurement 55f705af to history. Total: 1
New Minimum: 595.4813085965291 > 368.56289607579635
Armijo: th(2.154434690031884)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(1.077217345015942)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(0.3590724483386473)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(0.08976811208466183)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(0.017953622416932366)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(0.002992270402822061)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(4.2746720040315154E-4)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(5.343340005039394E-5)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(5.9370444500437714E-6)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(5.937044450043771E-7)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(5.397313136403428E-8)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(4.4977609470028565E-9)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(3.4598161130791205E-10)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Non-optimal measurement 368.56289607579635 < 368.56289607579635. Total: 2
Armijo: th(2.4712972236279432E-11)=368.56289607579635; dx=1.899543027252227E26 evalInputDelta=226.91841252073272
Adding measurement 686c6877 to history. Total: 2
New Minimum: 368.56289607579635 > 228.61303229221704
Armijo: th(1.6475314824186289E-12)=228.61303229221704; dx=1.8076530012193962E26 evalInputDelta=366.868276304312
Adding measurement 32d44bc7 to history. Total: 3
New Minimum: 228.61303229221704 > 154.48635002569895
Armijo: th(1.029707176511643E-13)=154.48635002569895; dx=5.623806176836239E25 evalInputDelta=440.99495857083014
Non-optimal measurement 570.6137921861157 < 154.48635002569895. Total: 4
Armijo: th(6.057101038303783E-15)=570.6137921861157; dx=-1.698781103593462E27 evalInputDelta=24.867516410413373
Non-optimal measurement 154.48635002569895 < 154.48635002569895. Total: 4
MIN ALPHA (3.3650561323909904E-16): th(1.029707176511643E-13)=154.48635002569895
Fitness changed from 595.4813085965291 to 154.48635002569895
Iteration 1 complete. Error: 154.48635002569895 Total: 0.0890; Orientation: 0.0034; Line Search: 0.0791
Non-optimal measurement 154.48635002569895 < 154.48635002569895. Total: 4
Rejected: LBFGS Orientation magnitude: 1.991e+13, gradient 1.290e+13, dot -0.989; [3fe45f62-471c-4d63-a2dc-2c0a89693df6 = 1.000/1.000e+00, eafb9a18-ca8c-429a-8af7-777cd1afef9b = 1.000/1.000e+00, 9e66e461-09f2-45fc-a927-5fc173eba35c = 1.000/1.000e+00, b57b3674-edf3-42a3-9ff4-7d258c9dff48 = 1.000/1.000e+00, 0f19dc22-855d-45e9-8340-cb118d7b1541 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 154.48635002569895, 228.61303229221704, 368.56289607579635, 595.4813085965291
LBFGS Accumulation History: 3 points
Removed measurement 32d44bc7 to history. Total: 3
Adding measurement 4c2dfb2f to history. Total: 3
th(0)=154.48635002569895;dx=-1.6650278653044866E26
Non-optimal measurement 154.51100104691415 < 154.48635002569895. Total: 4
Armijo: th(2.154434690031884E-15)=154.51100104691415; dx=-1.665027865304485E26 evalInputDelta=-0.024651021215191804
Non-optimal measurement 154.49862819951423 < 154.48635002569895. Total: 4
Armijo: th(1.077217345015942E-15)=154.49862819951423; dx=-1.665027865304486E26 evalInputDelta=-0.012278173815275295
Non-optimal measurement 154.48635002569895 < 154.48635002569895. Total: 4
MIN ALPHA (3.5907244833864734E-16): th(0.0)=154.48635002569895
Fitness changed from 154.48635002569895 to 154.48635002569895
Static Iteration Total: 0.0563; Orientation: 0.0411; Line Search: 0.0132
Iteration 2 failed. Error: 154.48635002569895
Previous Error: 0.0 -> 154.48635002569895
Optimization terminated 2
Final threshold in iteration 2: 154.48635002569895 (> 0.0) after 0.146s (< 30.000s)
Returns
154.48635002569895
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 2.0106331219804336, -0.6884630013314543, -1.8616017282509296 ], [ 1.0758123788543146, -0.035729741955893124, 0.82313424598861 ], [ 0.021330930567547757, 0.7966918678738331, 0.40065346730447804 ] ],
[ [ 0.9264735680470436, 0.3441371943370426, 0.3814517633253778 ], [ -1.012764142503945, 1.4400009631208956, -0.1832377839950166 ], [ 2.0929276382485695, 0.24239977450946198, 1.7913717079471927 ] ],
[ [ -1.4141175061121976, 1.385974691925922, -1.5653045127430025 ], [ 1.5392362661293055, -1.2617438850787186, -0.19628529058195887 ], [ 1.3437964566392833, -1.1255414798562238, 2.015614206621464 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 7.111263122704816, -11.841466829163995, 4.8365013321812, 19.75418936496487 ], [ 1.17265643294862, -0.05485260618738952, 1.4121487916457127, 1.6135195645156712 ] ],
[ [ -2.7548878165825714, -4.112724401871645, 5.735025738055015, 7.20466242839097 ], [ -8.081432801834774, -0.78442272332373, -14.38808401264061, 2.527569799056083 ] ]
]
TrainingTester.java:432 executed in 0.10 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, 1.8489762702404926], [3.0, 2.5665116118363205]; valueStats=DoubleSummaryStatistics{count=4, sum=748.163492, min=70.627896, average=187.040873, max=368.562896}
Only 1 points for GD
Plotting 3 points for CjGD
Only 1 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, 1.8489762702404926], [0.159, 2.5665116118363205]; valueStats=DoubleSummaryStatistics{count=4, sum=748.163492, min=70.627896, average=187.040873, max=368.562896}
Only 1 points for GD
Plotting 3 points for CjGD
Only 1 points for LBFGS
Returns
In this apply, attempt to train a network to emulate a randomized network given an example input/output. The target state is:
TrainingTester.java:370 executed in 0.00 seconds (0.000 gc):
RefList<double[]> temp_18_0042 = network_target.state();
assert temp_18_0042 != null;
String temp_18_0041 = temp_18_0042.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b).orElse("");
temp_18_0042.freeRef();
return temp_18_0041;
Returns
[1.984, 1.512, -1.412, 0.856, -0.028, -1.724, 0.632, 1.144, 1.732, -0.608, 0.092, 1.656, -1.564, -0.504, -0.416, -1.42, 0.568, -1.552, -0.408, -1.524, -0.5, -1.444, 0.612, -0.152, 1.088, 0.968, -1.228, 0.08, 1.836, 0.636, -0.676, -1.104, 0.66, -0.832, -1.128, -0.968, -0.06, 0.552, -0.804, -0.76, -1.596, 0.04, -1.624, -0.58, 0.332, -0.628, -1.824, 1.7, 1.66, -0.784, -1.76, 1.256, 0.012, 0.676, 1.956, -0.28, 1.052, 1.052, 1.008, 0.584, 1.86, -0.34, -0.384, 0.132, -0.924, 1.476, 0.688, 0.908, -0.664, 0.572, 0.56, 1.704, -1.34, 0.8, 0.284, -0.02, -1.836, -1.176, -0.316, 1.42, 1.66, 1.168, -1.656, -0.2, -1.856, -0.356, -1.476, -1.976, 0.3, -0.968, -0.712, 1.776, -1.184, -1.236, -0.176, 0.996, 0.996, 1.64, 1.012, -0.756, 1.096, 1.352, 0.184, 1.288, 0.82, 1.26, -0.344, -1.168, 1.364, 1.448, 0.08, -1.16, 0.688, 1.764, 0.932, -0.128, -0.964, -1.516, 0.652, -1.848, -0.488, -1.244, -1.096, -0.932, 1.836, 0.976, -0.124, 0.948, -1.536, -0.876, 1.236, 0.08, 1.22, 1.536, 1.384, 1.616, -1.248, 0.332, -1.264, 1.336, -0.732, -0.08, 0.532, -1.352, 1.324, -1.148, -1.256, 1.792, -1.472, 0.048, -0.748, -0.492, -1.036, 0.784, -1.372, -0.472, 1.064, -1.688, 0.9, 0.184, 0.644, -1.944, -1.316, -0.296, -1.552, -1.564, 0.072, 1.288, -0.528, -0.464, -1.58, -1.432, -1.756, -0.148, 1.048, -0.768, 0.076, 0.124, 0.856, 1.288, 1.884, -1.636, -0.848, -0.032, -0.852, -0.212, -1.524, 1.656, -0.968, 1.668, -0.124, 1.544, 1.208, -0.892, 0.52, 0.328, 0.78, -1.312, -0.384, 0.52, 1.512, 0.672, -0.684, 0.284, -0.396, -0.452, 1.196, 1.92, 1.352, -0.548, -0.856, 1.556, 1.524, 1.652, -0.084, -0.816, 1.596, 0.52, 1.156, -0.556, 0.812, -1.9, -1.6, -1.356, 0.296, -0.012, 0.992, 0.492, -0.4, 1.192, 1.872, -1.732, -0.032, -0.432, -1.96, 1.652, -1.736, -1.5, -1.964, 1.032, -1.116, -0.736, -0.448, 0.284, -0.548, -1.656, 1.64, -1.176, 0.86, 0.264, -1.92, -1.664, -0.616, 0.76, 0.972, 0.036, 1.628, -1.86, 1.264, 1.524, 0.244, 0.148, 0.96, 0.076, -1.796, 0.84, 1.144, -1.312, 1.852, 0.848, -1.572, 0.344, -1.176, 1.24, -0.18, 0.048, -1.524, 1.492, 0.7, 0.56, -1.092, 1.96, -1.716, 0.692, 0.764, 1.972, -0.396, -1.1, 0.428, 0.08, -1.54, 1.912, 0.164, -1.76, 1.048, -0.376, -0.684, -1.808, -1.476, -0.384, 1.98, -1.256, -0.636, 1.948, 0.572, 0.356, -0.828, -0.452, -1.976, 0.844, -1.696, -1.72, 0.66, -1.188, 0.476, 0.244, 0.512, 1.18, 1.22, 0.408, -1.832, 1.844, 1.62, -0.664, 1.132, 1.948, -1.18, 1.424, 0.78, 0.488, -0.584, -1.46, -1.764, -1.564, 0.016, 1.352, -1.104, -1.052, 0.464, 0.392, -1.632, -1.764, 0.788, 1.552, -0.368, 1.752, 0.1, -0.52, 1.108, 1.432, -0.336, 0.76, 0.868, -1.736, 1.24, 0.264, -2.0, -0.688, 1.72, 0.08, 1.916, 0.184, 1.172, 0.412, 0.48, 0.972, -0.212, -1.028, -1.812, 1.348, 0.752, 0.248, 0.448, -1.16, -1.192, -0.468, 0.124, 0.6, -0.856, -1.616, 1.052, -0.928, 1.88, 0.48, -1.096, 0.028, -0.056, 0.496, -0.612, 0.656, -0.252, -1.644, -0.068, -0.312, 1.244, -0.632, 0.62, -1.108, 1.368, 0.832, -1.156, 1.556, -1.88, 1.972, 1.304, 1.516, -1.384, -0.808, -0.652, 1.572, -0.86, 1.82, 0.552, 0.892, -1.484, 0.312, 1.612, -0.096, 1.632, -1.5, -0.26, 1.356, -0.312, -0.816, 1.316, 0.696, 1.724, -1.808, 0.376, 0.004, 0.404, -0.852]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 13.49 seconds (0.177 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1568821390979
Reset training subject: 1568825574017
Constructing line search parameters: GD
th(0)=883.0687980040733;dx=-3.2190887008777943E27
New Minimum: 883.0687980040733 > 321.63180565374364
Armijo: th(2.154434690031884)=321.63180565374364; dx=3.8729023614001065E25 evalInputDelta=561.4369923503297
Armijo: th(1.077217345015942)=321.63180565374364; dx=3.8729023614001065E25 evalInputDelta=561.4369923503297
Armijo: th(0.3590724483386473)=321.63180565374364; dx=3.8729023614001065E25 evalInputDelta=561.4369923503297
Armijo: th(0.08976811208466183)=321.63180565374364; dx=3.8729023614001065E25 evalInputDelta=561.4369923503297
Armijo: th(0.017953622416932366)=321.63180565374364; dx=3.8729023614001065E25 evalInputDelta=561.4369923503297
Armijo: th(0.002992270402822061)=321.69040212508423; dx=3.8729023614001065E25 evalInputDelta=561.3783958789891
Armijo: th(4.2746720040315154E-4)=321.75005738982486; dx=3.8729023614001065E25 evalInputDelta=561.3187406142484
Armijo: th(5.343340005039394E-5)=321.7592409523894; dx=3.8729023614001065E25 evalInputDelta=561.3095570516839
Armijo: th(5.9370444500437714E-6)=321.76041660346124; dx=3.8729023614001065E25 evalInputDelta=561.308381400612
Armijo: th(5.937044450043771E-7)=321.76054899942807; dx=3.8729023614001065E25 evalInputDelta=561.3082490046452
Armijo: th(5.397313136403428E-8)=321.7605623742809; dx=3.8729023614001065E25 evalInputDelta=561.3082356297924
Armijo: th(4.4977609470028565E-9)=321.76056360032305; dx=3.8729023614001065E25 evalInputDelta=561.3082344037502
New Minimum: 321.63180565374364 > 264.7638457820925
Armijo: th(3.4598161130791205E-10)=264.7638457820925; dx=3.871983021894725E25 evalInputDelta=618.3049522219808
New Minimum: 264.7638457820925 > 221.83355717447776
Armijo: th(2.4712972236279432E-11)=221.83355717447776; dx=3.8656360898620315E25 evalInputDelta=661.2352408295956
New Minimum: 221.83355717447776 > 159.51867363827105
Armijo: th(1.6475314824186289E-12)=159.51867363827105; dx=3.3528671577621524E25 evalInputDelta=723.5501243658023
New Minimum: 159.51867363827105 > 106.51462692049749
Armijo: th(1.029707176511643E-13)=106.51462692049749; dx=-1.700786128231007E26 evalInputDelta=776.5541710835759
Armijo: th(6.057101038303783E-15)=883.0687980040733; dx=-3.2190887008777943E27 evalInputDelta=0.0
MIN ALPHA (3.3650561323909904E-16): th(1.029707176511643E-13)=106.51462692049749
Fitness changed from 883.0687980040733 to 106.51462692049749
Iteration 1 complete. Error: 106.51462692049749 Total: 0.0979; Orientation: 0.0004; Line Search: 0.0872
th(0)=106.51462692049749;dx=-1.5387406482347164E26
New Minimum: 106.51462692049749 > 106.51462692049714
Armijo: th(2.154434690031884E-15)=106.51462692049714; dx=-1.5387406482347164E26 evalInputDelta=3.552713678800501E-13
Armijo: th(1.077217345015942E-15)=106.5146269204973; dx=-1.5387406482347164E26 evalInputDelta=1.8474111129762605E-13
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=106.51462692049714
Fitness changed from 106.51462692049749 to 106.51462692049714
Iteration 2 complete. Error: 106.51462692049714 Total: 0.0171; Orientation: 0.0004; Line Search: 0.0140
th(0)=106.51462692049714;dx=-1.5387406482347164E26
New Minimum: 106.51462692049714 > 106.51462692049674
Armijo: th(2.154434690031884E-15)=106.51462692049674; dx=-1.5387406482347164E26 evalInputDelta=3.979039320256561E-13
Armijo: th(1.077217345015942E-15)=106.51462692049695; dx=-1.5387406482347164E26 evalInputDelta=1.8474111129762605E-13
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=106.51462692049674
Fitness changed from 106.51462692049714 to 106.51462692049674
Iteration 3 complete. Error: 106.51462692049674 Total: 0.0219; Orientation: 0.0004; Line Search: 0.0183
th(0)=106.51462692049674;dx=-1.5387406482347164E26
New Minimum: 106.51462692049674 > 106.51462692049638
Armijo: th(2.154434690031884E-15)=106.51462692049638; dx=-1.5387406482347164E26 evalInputDelta=3.552713678800501E-13
Armijo: th(1.077217345015942E-15)=106.51462692049658; dx=-1.5387406482347164E26 evalInputDelta=1.5631940186722204E-13
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=106.51462692049638
Fitness changed from 106.51462692049674 to 106.51462692049638
Iteration 4 complete. Error: 106.51462692049638 Total: 0.0210; Orientation: 0.0008; Line Search: 0.0168
th(0)=106.51462692049638;dx=-1.5387406482347164E26
New Minimum: 106.51462692049638 > 106.51462692049601
Armijo: th(2.154434690031884E-15)=106.51462692049601; dx=-1.5387406482347164E26 evalInputDelta=3.694822225952521E-13
Armijo: th(1.077217345015942E-15)=106.51462692049617; dx=-1.5387406482347164E26 evalInputDelta=2.1316282072803006E-13
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=106.51462692049601
Fitness changed from 106.51462692049638 to 106.51462692049601
Iteration 5 complete. Error: 106.51462692049601 Total: 0.0170; Orientation: 0.0004; Line Search: 0.0139
th(0)=106.51462692049601;dx=-1.5387406482347164E26
New Minimum: 106.51462692049601 > 78.93192592734022
Armijo: th(2.154434690031884E-15)=78.93192592734022; dx=-1.0458054064274088E26 evalInputDelta=27.582700993155797
Armijo: th(1.077217345015942E-15)=80.18280662674799; dx=-1.0458054064420358E26 evalInputDelta=26.331820293748024
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=78.93192592734022
Fitness changed from 106.51462692049601 to 78.93192592734022
Iteration 6 complete. Error: 78.93192592734022 Total: 0.0187; Orientation: 0.0004; Line Search: 0.0157
th(0)=78.93192592734022;dx=-1.0458054064204826E26
New Minimum: 78.93192592734022 > 78.93192592731882
Armijo: th(2.154434690031884E-15)=78.93192592731882; dx=-1.0458054064204826E26 evalInputDelta=2.1401547201094218E-11
Armijo: th(1.077217345015942E-15)=78.9319259273295; dx=-1.0458054064204826E26 evalInputDelta=1.071498445526231E-11
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=78.93192592731882
Fitness changed from 78.93192592734022 to 78.93192592731882
Iteration 7 complete. Error: 78.93192592731882 Total: 0.0196; Orientation: 0.0006; Line Search: 0.0159
th(0)=78.93192592731882;dx=-1.0458054064204826E26
New Minimum: 78.93192592731882 > 78.93192592729741
Armijo: th(2.154434690031884E-15)=78.93192592729741; dx=-1.0458054064204826E26 evalInputDelta=2.1401547201094218E-11
Armijo: th(1.077217345015942E-15)=78.9319259273081; dx=-1.0458054064204826E26 evalInputDelta=1.071498445526231E-11
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=78.93192592729741
Fitness changed from 78.93192592731882 to 78.93192592729741
Iteration 8 complete. Error: 78.93192592729741 Total: 0.0194; Orientation: 0.0004; Line Search: 0.0163
th(0)=78.93192592729741;dx=-1.0458054064204826E26
New Minimum: 78.93192592729741 > 78.931925927276
Armijo: th(2.154434690031884E-15)=78.931925927276; dx=-1.0458054064204826E26 evalInputDelta=2.141575805580942E-11
Armijo: th(1.077217345015942E-15)=78.9319259272867; dx=-1.0458054064204826E26 evalInputDelta=1.071498445526231E-11
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=78.931925927276
Fitness changed from 78.93192592729741 to 78.931925927276
Iteration 9 complete. Error: 78.931925927276 Total: 0.0216; Orientation: 0.0004; Line Search: 0.0178
th(0)=78.931925927276;dx=-1.0458054064204826E26
New Minimum: 78.931925927276 > 78.93192592725461
Armijo: th(2.154434690031884E-15)=78.93192592725461; dx=-1.0458054064204826E26 evalInputDelta=2.1387336346379016E-11
Armijo: th(1.077217345015942E-15)=78.9319259272653; dx=-1.0458054064204826E26 evalInputDelta=1.0700773600547109E-11
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=78.93192592725461
Fitness changed from 78.931925927276 to 78.93192592725461
Iteration 10 complete. Error: 78.93192592725461 Total: 0.0183; Orientation: 0.0005; Line Search: 0.0151
th(0)=78.93192592725461;dx=-1.0458054064204826E26
New Minimum: 78.93192592725461 > 78.93192592723321
Armijo: th(2.154434690031884E-15)=78.93192592723321; dx=-1.0458054064204826E26 evalInputDelta=2.1401547201094218E-11
Armijo: th(1.077217345015942E-15)=78.9319259272439; dx=-1.0458054064204826E26 evalInputDelta=1.071498445526231E-11
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=78.93192592723321
Fitness changed from 78.931925
...skipping 131103 bytes...
82 > 22.194607925265636
Armijo: th(2.154434690031884E-15)=22.194607925265636; dx=-3.9671343590917313E24 evalInputDelta=2.253118225326034E-8
Armijo: th(1.077217345015942E-15)=22.19460793653073; dx=-3.9671343590917313E24 evalInputDelta=1.1266088506545202E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607925265636
Fitness changed from 22.19460794779682 to 22.194607925265636
Iteration 237 complete. Error: 22.194607925265636 Total: 0.0260; Orientation: 0.0008; Line Search: 0.0161
th(0)=22.194607925265636;dx=-3.9671343590917313E24
New Minimum: 22.194607925265636 > 22.194607902734724
Armijo: th(2.154434690031884E-15)=22.194607902734724; dx=-3.9671343590917313E24 evalInputDelta=2.253091224702075E-8
Armijo: th(1.077217345015942E-15)=22.19460791400052; dx=-3.9671343590917313E24 evalInputDelta=1.126511506299721E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607902734724
Fitness changed from 22.194607925265636 to 22.194607902734724
Iteration 238 complete. Error: 22.194607902734724 Total: 0.0174; Orientation: 0.0005; Line Search: 0.0146
th(0)=22.194607902734724;dx=-3.9671343590917313E24
New Minimum: 22.194607902734724 > 22.19460788020435
Armijo: th(2.154434690031884E-15)=22.19460788020435; dx=-3.9671343590917313E24 evalInputDelta=2.2530372234541574E-8
Armijo: th(1.077217345015942E-15)=22.194607891470238; dx=-3.9671343590917313E24 evalInputDelta=1.1264486232676063E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.19460788020435
Fitness changed from 22.194607902734724 to 22.19460788020435
Iteration 239 complete. Error: 22.19460788020435 Total: 0.0144; Orientation: 0.0007; Line Search: 0.0113
th(0)=22.19460788020435;dx=-3.9671343590917313E24
New Minimum: 22.19460788020435 > 22.194607857674452
Armijo: th(2.154434690031884E-15)=22.194607857674452; dx=-3.9671343590917313E24 evalInputDelta=2.2529899723622293E-8
Armijo: th(1.077217345015942E-15)=22.194607868939094; dx=-3.9671343590917313E24 evalInputDelta=1.1265257171544363E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607857674452
Fitness changed from 22.19460788020435 to 22.194607857674452
Iteration 240 complete. Error: 22.194607857674452 Total: 0.1908; Orientation: 0.0005; Line Search: 0.1885
th(0)=22.194607857674452;dx=-3.9671343590917313E24
New Minimum: 22.194607857674452 > 22.194607835144033
Armijo: th(2.154434690031884E-15)=22.194607835144033; dx=-3.9671343590917313E24 evalInputDelta=2.2530418419819398E-8
Armijo: th(1.077217345015942E-15)=22.19460784640907; dx=-3.9671343590917313E24 evalInputDelta=1.126538151652312E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607835144033
Fitness changed from 22.194607857674452 to 22.194607835144033
Iteration 241 complete. Error: 22.194607835144033 Total: 0.1166; Orientation: 0.0004; Line Search: 0.1146
th(0)=22.194607835144033;dx=-3.9671343590917313E24
New Minimum: 22.194607835144033 > 22.194607812613725
Armijo: th(2.154434690031884E-15)=22.194607812613725; dx=-3.9671343590917313E24 evalInputDelta=2.2530308285695355E-8
Armijo: th(1.077217345015942E-15)=22.194607823878926; dx=-3.9671343590917313E24 evalInputDelta=1.1265107957569853E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607812613725
Fitness changed from 22.194607835144033 to 22.194607812613725
Iteration 242 complete. Error: 22.194607812613725 Total: 0.1443; Orientation: 0.0003; Line Search: 0.1423
th(0)=22.194607812613725;dx=-3.9671343590917313E24
New Minimum: 22.194607812613725 > 22.194607790082614
Armijo: th(2.154434690031884E-15)=22.194607790082614; dx=-3.9671343590917313E24 evalInputDelta=2.2531111198986764E-8
Armijo: th(1.077217345015942E-15)=22.194607801348173; dx=-3.9671343590917313E24 evalInputDelta=1.1265552046779703E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607790082614
Fitness changed from 22.194607812613725 to 22.194607790082614
Iteration 243 complete. Error: 22.194607790082614 Total: 0.1317; Orientation: 0.0003; Line Search: 0.1294
th(0)=22.194607790082614;dx=-3.9671343590917313E24
New Minimum: 22.194607790082614 > 22.194607767552387
Armijo: th(2.154434690031884E-15)=22.194607767552387; dx=-3.9671343590917313E24 evalInputDelta=2.2530226573280743E-8
Armijo: th(1.077217345015942E-15)=22.19460777881798; dx=-3.9671343590917313E24 evalInputDelta=1.1264635446650573E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607767552387
Fitness changed from 22.194607790082614 to 22.194607767552387
Iteration 244 complete. Error: 22.194607767552387 Total: 0.0729; Orientation: 0.0004; Line Search: 0.0707
th(0)=22.194607767552387;dx=-3.9671343590917313E24
New Minimum: 22.194607767552387 > 22.194607745022523
Armijo: th(2.154434690031884E-15)=22.194607745022523; dx=-3.9671343590917313E24 evalInputDelta=2.2529864196485505E-8
Armijo: th(1.077217345015942E-15)=22.194607756287773; dx=-3.9671343590917313E24 evalInputDelta=1.12646141303685E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607745022523
Fitness changed from 22.194607767552387 to 22.194607745022523
Iteration 245 complete. Error: 22.194607745022523 Total: 0.0153; Orientation: 0.0005; Line Search: 0.0130
th(0)=22.194607745022523;dx=-3.9671343590917313E24
New Minimum: 22.194607745022523 > 22.194607722491458
Armijo: th(2.154434690031884E-15)=22.194607722491458; dx=-3.9671343590917313E24 evalInputDelta=2.253106501370894E-8
Armijo: th(1.077217345015942E-15)=22.194607733757287; dx=-3.9671343590917313E24 evalInputDelta=1.126523585526229E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607722491458
Fitness changed from 22.194607745022523 to 22.194607722491458
Iteration 246 complete. Error: 22.194607722491458 Total: 0.0146; Orientation: 0.0006; Line Search: 0.0123
th(0)=22.194607722491458;dx=-3.9671343590917313E24
New Minimum: 22.194607722491458 > 22.19460769996139
Armijo: th(2.154434690031884E-15)=22.19460769996139; dx=-3.9671343590917313E24 evalInputDelta=2.2530066701165197E-8
Armijo: th(1.077217345015942E-15)=22.194607711227228; dx=-3.9671343590917313E24 evalInputDelta=1.126423043729119E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.19460769996139
Fitness changed from 22.194607722491458 to 22.19460769996139
Iteration 247 complete. Error: 22.19460769996139 Total: 0.0176; Orientation: 0.0005; Line Search: 0.0155
th(0)=22.19460769996139;dx=-3.9671343590917313E24
New Minimum: 22.19460769996139 > 22.194607677431144
Armijo: th(2.154434690031884E-15)=22.194607677431144; dx=-3.9671343590917313E24 evalInputDelta=2.2530247889562816E-8
Armijo: th(1.077217345015942E-15)=22.194607688696884; dx=-3.9671343590917313E24 evalInputDelta=1.1264507548958136E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607677431144
Fitness changed from 22.19460769996139 to 22.194607677431144
Iteration 248 complete. Error: 22.194607677431144 Total: 0.0145; Orientation: 0.0005; Line Search: 0.0117
th(0)=22.194607677431144;dx=-3.9671343590917313E24
New Minimum: 22.194607677431144 > 22.194607654900846
Armijo: th(2.154434690031884E-15)=22.194607654900846; dx=-3.9671343590917313E24 evalInputDelta=2.253029762755432E-8
Armijo: th(1.077217345015942E-15)=22.1946076661658; dx=-3.9671343590917313E24 evalInputDelta=1.1265342436672654E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607654900846
Fitness changed from 22.194607677431144 to 22.194607654900846
Iteration 249 complete. Error: 22.194607654900846 Total: 0.0121; Orientation: 0.0003; Line Search: 0.0096
th(0)=22.194607654900846;dx=-3.9671343590917313E24
New Minimum: 22.194607654900846 > 22.194607632370303
Armijo: th(2.154434690031884E-15)=22.194607632370303; dx=-3.9671343590917313E24 evalInputDelta=2.2530542764798156E-8
Armijo: th(1.077217345015942E-15)=22.194607643635486; dx=-3.9671343590917313E24 evalInputDelta=1.1265360200241048E-8
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=22.194607632370303
Fitness changed from 22.194607654900846 to 22.194607632370303
Iteration 250 complete. Error: 22.194607632370303 Total: 0.0124; Orientation: 0.0003; Line Search: 0.0103
Final threshold in iteration 251: 22.194607632370303 (> 0.0) after 13.490s (< 30.000s)
Returns
22.194607632370303
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
[-0.8417164661819274, -0.3715545828761006, -2.0380991125725694, 1.6397710284448523, 0.5725021399183632, -0.9977874186602507, 0.9630706180231631, 2.721521545924986, 3.068183196735524, 1.1363136785510926, 1.6275837260726753, -0.4202659836060104, -2.1573620566992613, -0.5361658402691489, 0.7006700286062708, -2.2297476958215277, -0.3226120324313265, -0.19202892778330638, 1.0429493371336882, -0.05566088171490527, -1.396709264906144, 0.7407235218090881, 1.8776456051375947, 0.7534032900154691, -1.2964043481039318, -0.5058109984503334, 0.07361821805826554, 1.0429465287033, -0.24540332715110472, -0.0481287183276259, 1.5978167741944092, 0.626801038684229, -0.4371179526440419, -0.4411241120220793, -1.447110593010341, 0.7526183251433973, 0.6328212133826043, 1.3728542671787225, -0.7945403272967282, -1.6758018733234543, 0.6480963187231673, -0.6616584237546084, 0.27917001300368866, -0.7748913722856063, 0.8858744608762109, 0.6014595773833639, -0.9595285295277871, 0.27126922435595113, 1.6403111524325777, -0.7908376015009521, -1.5496888952273624, -1.5249266491884597, -0.04940460530059318, 0.455770162335371, -0.14651814428160734, 1.6541651615427855, 0.059915574164731464, -1.5969904503769252, -1.385527953341856, -0.192713994337908, 1.4175968403452293, 1.8330826456311742, -1.7612283187717903, 0.5954011200095464, 0.6106563074443431, 1.9119615420730227, 0.8155623319841608, -1.6661189341960458, -1.7527678613797217, 0.6736890355978615, -0.9011670714668342, 1.0228559773476964, -1.6802688163424324, 1.6001919004871672, 0.9646141593850042, -0.8259270337121778, -0.8636545272367883, -1.5480920655331714, -0.29289838484430225, -0.7276621669367033, 0.8208753360307245, 0.23364730424740396, 0.7034899780212143, -1.475818452668057, -0.41467621219084755, -0.1995549999944567, -0.292340584469035, -0.2571594805755359, -0.7734557532367533, -2.1291884487138417, -0.9953336282792579, -0.9391571700382277, 2.3912436302818567, -1.2267690817188897, -0.16117876705664838, 1.9107477434549964, 0.39525194075055536, -1.070349866795806, -0.1095844420699689, -1.6364721469313317, -1.5353464747328711, 1.0763474541797562, 1.5198997922701867, -1.3611240714344788, -0.02494393352292014, 1.1398333711882294, -1.82899052802733, 1.1127123327400181, 0.5926586628090904, 1.6165984056786897, -0.5705094807583695, -1.1672983426148196, 0.48636185268647186, 0.8378102755153589, 1.429303174908843, 1.7123595551170063, -0.08132517034601491, -1.4119005346057114, -1.118744370009938, 0.6324085385335052, 1.4292575272893522, -1.6538092730096334, -1.7328744859282, -0.6401703365357883, -1.7635432371785267, 1.7988258851943502, -0.7698770196206198, -0.8251132937725566, 1.6025230240741708, 0.33201262469798415, 0.13514209309902078, -0.5221274957590232, -0.8971038782663733, -1.2948189575375435, 1.8759148958081373, -0.358080855034641, 0.7678209305868984, 1.1592941462637945, 1.6381478184392717, 1.1212346255457428, -1.772043155117435, 1.9894980177072807, -0.8364553594614474, -0.5612648827763423, 0.16280406811468007, 1.725351193011212, -0.4551279747374865, -1.3175024791643206, 0.13229364939218613, 0.13012401015206201, -1.3008095233148949, -1.5242504942747583, -0.09182737953030558, -1.180090771691854, -1.4261651777830913, -1.335367558611829, -0.6258715283909259, 1.0598100201864964, 1.9175758915123549, -0.07142807307306952, 0.4540917654350569, 0.5550049583286134, 1.8199999999993375, -1.6440000000007233, -1.8800000000001988, 1.2880000000003342, -0.25199999999907663, -1.2360000000001317, 1.9200000000006612, 0.35599999999914433, 0.8399999999993394, 1.9840000000002598, 1.2440000000009237, 1.168000000000528, 0.7520000000003628, 0.8319999999994749, 1.6600000000007942, -1.963999999999805, -0.8519999999993434, 0.33200000000052626, -0.9239999999992116, -1.127999999999732, -1.7359999999996698, 1.259999999999798, 1.316000000000131, 0.551999999999506, -0.8159999999991441, -1.6360000000001362, 0.6719999999992745, 0.2736004508341896, 0.07630102223718774, -0.2608310277052357, 0.024009563884420042, 0.7035213647979728, -0.7425803728470632, -1.2826994791520911, 0.03529575395388559, -0.49295093169834114, -0.9297277185791325, 1.4565716689897605, 1.7519711355267875, 0.09434513525568433, -1.2658794477229007, -0.8388180078344541, 0.7521290589787176, 1.5940017090460397, 1.4046238098058281, 0.5540931644914862, -0.7209103833468608, 1.2925463784078317, 1.0080972094933403, -1.392624254279709, 0.7043196523968768, -0.31529283039434874, -0.369463846691519, -0.978691210555048, -0.005616521259341995, 0.16824494464130252, 1.928520292292152, 1.7821379605839895, -0.20836426262439953, -1.018841244462816, 1.601587265279547, -0.6005793777684258, -0.5154825752845638, -2.07154395482367, 0.8007027103633222, -1.093769109833572, 0.2533423206387828, -0.3015258857409552, -0.5451289260426307, -0.910535493799217, 0.226869926927919, -0.9542274490728966, -1.217801233393526, -0.8588986093923285, 1.4682437976095517, 0.05489918290796832, 0.9250096080579495, 2.0117877524335563, 0.0022578520840552574, -1.688580524800018, -0.7426857883095522, 1.8720000000017165, -1.4999999999997375, 0.860000000000397, 0.5120000000014194, 1.88000000000198, -1.8560000000019738, 1.6520000000001964, 0.4080000000005278, 0.9759999999984841, 1.6399999999982213, -0.6639999999986796, 0.07600000000175022, 1.4239999999989492, -0.6519999999993067, -0.12799999999794015, 1.3639999999998036, 1.2880000000009235, 0.7640000000012869, 0.8560000000014514, 1.1960000000021065, 1.1800000000002018, 0.9479999999981886, 1.2560000000013787, 0.8440000000012832, 0.18399999999913516, 1.1559999999990076, -0.5479999999990782, 1.7784827225004058, 2.0147369139332754, 1.387278225202188, -0.2765742396725369, 0.9916707159389229, 1.118912077191065, 1.4085373757247108, 0.6108686275596944, 1.8718050797012944, 0.9612828431118574, 0.5987961850666336, 1.4358090152457506, -0.20097596623141106, -1.2980809959474882, 1.0567744602504525, -1.3209733160923605, 1.7545815077210303, 0.18640762987325418, 0.28296741277389525, -0.8538399758805504, -0.6344603153196342, -0.8165242043617307, 0.460161991896621, -0.19188804342114654, -0.2821731954818182, -1.6847086034379664, 0.26858347549156564, 1.508015520830128, -1.6396936733815375, -0.10391766435149115, 0.19797934977063344, -0.3618867553219361, 1.5926073912051966, -0.8049652770813635, -0.6955830597227669, -1.5413899777787095, -1.0237554914342006, -0.28948783425593466, 0.8017888988444588, -1.248470997916985, -1.896352623843972, -0.8801236361074783, 1.756687228473329, 0.5207490465288009, -0.13944666643230072, 0.34664356226860404, -0.29737445694531406, 0.2810038138906865, 0.2699948706009522, 0.7273075094929898, 1.0004222023122626, 0.6377991576422636, 1.1404041830999645, -1.614728396298855, 1.7947685580878483, 1.4732642077548215, 1.8018883747745986, -0.22462382754883475, 0.9498086667201187, -0.5235524806896644, 1.1821278384130685, 0.921776240343764, -1.2967525146372019, 1.6463993889543476, 1.0620481303575977, -0.8881594161110009, 0.9568636306570635, -0.4252797420034249, -1.5552157379282492, 1.3764316455935193, 0.6285279063070743, -1.6990715165941772, 1.1340960909788376, -0.4695033319227725, -1.590992148008531, -1.8723684901948867, -0.02571138759600903, 0.5186084630364591, 1.6944314758593346, 1.3895198845800378, -0.017983617078703575, 0.7612186528852937, 1.5232857501108723, 1.3873476404864369, 1.0915157726937024, 0.956848377733035, -1.2062250887387873, 0.038783715044948895, 2.259502608777899, 0.5367740294993799, -1.3362548144996178, -0.6214447969810701, -0.6489510821633121, 0.5873864457462139, -0.34376261450422996, 0.5358813923772094, -1.8453889689395113, -0.7406529677177857, -1.0850508089383637, 1.5857188827093982, -0.9212429857043403, -0.6185489544942702, 1.9374493329168243, -0.8759026686711452, 1.9381193234667606, 0.8920482398962168, -0.3875512252220119, 0.6729823325825555, 0.7800000000008237, 0.07600000000090217, 0.2839999999997681, -1.1759999999998678, 0.9080000000002981, -0.6280000000007603, 0.03599999999982919, 0.02800000000055289, 1.288000000000988, -1.0919999999991428, -1.243999999999277, 1.7320000000006628, -0.1520000000005438, 1.0959999999997991, -1.9760000000007223, 0.015999999999099016, 0.04800000000098245, 1.0479999999992144, -1.1880000000007245, 1.0519999999994736, -1.835999999999607, -1.1759999999994115, 0.07200000000038705, 1.8839999999998671, -1.551999999999738, -0.03199999999980837, -0.7559999999996388, -1.4341110639138293, 0.704100907501946, 1.055104185578382, 0.8221221001734691, -0.9659103910223943, 1.615058665873071, -0.15899970331760016, 0.7396865216734863, 0.8806394511433959, -0.40637681987746266, -0.8934935645979843, 0.9227103544115816, -1.9688054592685318, 1.0520991402052242, -0.7821528495705214, 0.12409367442566019, -0.3063821859883302, -1.2663553053166672, -0.9986068880813401, -0.2671202538144299, 1.5311376846382507, -1.4174471882517479, 0.39180591149488286, 1.592917945114915, -0.8020506997400016, -0.844876290669713, -0.8342259382227455]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 5.187850033073454, 4.396225873025989, 5.923561581586978, 0.04121155440351576 ], [ 2.244575849315014, 4.825199999999833, 6.743982211748498, 15.674463999999869 ] ],
[ [ -1.6412599792485043, 12.357761928512948, 4.622448000039177, 6.435297130265677 ], [ 1.260769819536283, 3.013136717059708, -0.626857208879281, 1.2511293250299245 ] ]
]
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 1.49 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1582321394911
Reset training subject: 1582322983269
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=883.0687980040733}, derivative=-3.2190887008777943E27}
New Minimum: 883.0687980040733 > 262.62018714398084
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=262.62018714398084}, derivative=3.87198302189472E25}, evalInputDelta = -620.4486108600925
262.62018714398084 <= 883.0687980040733
Converged to right
Fitness changed from 883.0687980040733 to 262.62018714398084
Iteration 1 complete. Error: 262.62018714398084 Total: 0.0097; Orientation: 0.0003; Line Search: 0.0048
F(0.0) = LineSearchPoint{point=PointSample{avg=262.62018714398084}, derivative=-7.014465726671118E26}
New Minimum: 262.62018714398084 > 7.6296297789299246
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=7.6296297789299246}, derivative=-2.2247797798212667E24}, evalInputDelta = -254.9905573650509
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=59.1597599695505}, derivative=1.3608761528361502E25}, evalInputDelta = -203.46042717443035
59.1597599695505 <= 262.62018714398084
F(6.866777679792137E-10) = LineSearchPoint{point=PointSample{avg=59.15975996988063}, derivative=1.3608761528361502E25}, evalInputDelta = -203.4604271741002
59.15975996988063 > 59.1597599695505
Fitness changed from 262.62018714398084 to 7.6296297789299246
Iteration 2 complete. Error: 7.6296297789299246 Total: 0.0129; Orientation: 0.0003; Line Search: 0.0111
F(0.0) = LineSearchPoint{point=PointSample{avg=7.6296297789299246}, derivative=-2.448204958055436E24}
New Minimum: 7.6296297789299246 > 0.12875803573509131
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=0.12875803573509131}, derivative=1.6276710795947378E12}, evalInputDelta = -7.500871743194833
0.12875803573509131 <= 7.6296297789299246
Converged to right
Fitness changed from 7.6296297789299246 to 0.12875803573509131
Iteration 3 complete. Error: 0.12875803573509131 Total: 0.0060; Orientation: 0.0003; Line Search: 0.0043
F(0.0) = LineSearchPoint{point=PointSample{avg=0.12875803573509131}, derivative=-4362.012030580138}
New Minimum: 0.12875803573509131 > 0.1287580183884721
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=0.1287580183884721}, derivative=-4362.012029908599}, evalInputDelta = -1.7346619224367643E-8
New Minimum: 0.1287580183884721 > 0.12875791430876773
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=0.12875791430876773}, derivative=-4362.012025879365}, evalInputDelta = -1.2142632357936556E-7
New Minimum: 0.12875791430876773 > 0.12875718575130446
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=0.12875718575130446}, derivative=-4362.011997674763}, evalInputDelta = -8.499837868547466E-7
New Minimum: 0.12875718575130446 > 0.12875208587228149
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=0.12875208587228149}, derivative=-4362.011800244344}, evalInputDelta = -5.9498628098286765E-6
New Minimum: 0.12875208587228149 > 0.1287163878567461
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=0.1287163878567461}, derivative=-4362.010418319492}, evalInputDelta = -4.164787834520789E-5
New Minimum: 0.1287163878567461 > 0.12846655747488336
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=0.12846655747488336}, derivative=-4362.000749159473}, evalInputDelta = -2.914782602079502E-4
New Minimum: 0.12846655747488336 > 0.12672046964654787
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=0.12672046964654787}, derivative=-4361.933275752824}, evalInputDelta = -0.002037566088543441
New Minimum: 0.12672046964654787 > 0.11462942470208604
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=0.11462942470208604}, derivative=-4361.471062129275}, evalInputDelta = -0.014128611033005278
New Minimum: 0.11462942470208604 > 0.03584052527200188
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=0.03584052527200188}, derivative=-4358.663803599467}, evalInputDelta = -0.09291751046308944
New Minimum: 0.03584052527200188 > 0.0
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.6488230635089725E11}, evalInputDelta = 0.6149364238043438
F(0.7453000800538463) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(5.217100560376924) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(36.51970392263847) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.6488230635089725E11}, evalInputDelta = 0.6149364238043438
F(2.809207994049113) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(19.66445595834379) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.6488230635089725E11}, evalInputDelta = 0.6149364238043438
F(1.5126504583341378) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(10.588553208338965) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.6488230635089725E11}, evalInputDelta = 0.6149364238043438
F(0.8145040929491512) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(5.701528650644058) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(39.91070055450841) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.6488230635089725E11}, evalInputDelta = 0.6149364238043438
F(3.070053888808339) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(21.490377221658374) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.6488230635089725E11}, evalInputDelta = 0.6149364238043438
F(1.6531059401275672) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(11.57174158089297) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.6488230635089725E11}, evalInputDelta = 0.6149364238043438
F(0.8901339677609977) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(6.230937774326984) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(43.61656442028889) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.6488230635089725E11}, evalInputDelta = 0.6149364238043438
F(3.3551203400222223) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(23.485842380155557) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.6488230635089725E11}, evalInputDelta = 0.6149364238043438
F(1.806603260011966) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(12.64622282008376) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.6488230635089725E11}, evalInputDelta = 0.6149364238043438
F(0.9727863707756739) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(6.809504595429717) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.496755811712}, evalInputDelta = -0.12875803573509131
F(47.66653216800802) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.6488230635089725E11}, evalInputDelta = 0.6149364238043438
F(3.6666563206160014) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4357.4967558
...skipping 5828 bytes...
ample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(4.315458754745348) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(30.208211283217434) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(2.323708560247495) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(16.265959921732463) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(1.2512276862871126) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(8.758593804009788) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(0.6737379849238299) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(4.716165894466809) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(33.01316126126766) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(2.5394739431744355) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(17.77631760222105) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(1.367409046324696) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(9.571863324272872) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(0.7362971787902209) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(5.154080251531546) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(36.078561760720824) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(2.7752739815939096) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(19.426917871157368) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(1.494378297781336) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(10.460648084469351) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(0.8046652372668732) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(5.632656660868113) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(39.42859662607679) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(3.0329689712366763) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(21.230782798656733) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(1.6331371383582103) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(11.431959968507472) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(0.8793815360390363) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(6.155670752273254) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(43.08969526591278) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(3.3145919435317523) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(23.202143604722266) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(1.7847802772863282) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(12.493461941004297) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(0.9610355339234075) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(6.727248737463852) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(47.090741162246964) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(3.622364704788228) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(25.356552933517595) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(1.9505040718090458) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(13.65352850266332) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
F(1.0502714232817938) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(7.351899962972556) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
F(51.4632997408079) = LineSearchPoint{point=PointSample{avg=0.7436944595394351}, derivative=1.648823063545898E11}, evalInputDelta = 0.7436944595394351
Loops = 52
F(1.3589155171014708E-6) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Left bracket at 1.3589155171014708E-6
F(2.7178310005379403E-6) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Left bracket at 2.7178310005379403E-6
F(4.076746450309407E-6) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Left bracket at 4.076746450309407E-6
F(5.435661866415872E-6) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Left bracket at 5.435661866415872E-6
F(6.794577239332146E-6) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Left bracket at 6.794577239332146E-6
F(8.15349257858342E-6) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Left bracket at 8.15349257858342E-6
F(9.512407884169689E-6) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Left bracket at 9.512407884169689E-6
F(1.0871323156090957E-5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Left bracket at 1.0871323156090957E-5
F(1.2230238384822036E-5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Left bracket at 1.2230238384822036E-5
F(1.3589153579888112E-5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Left bracket at 1.3589153579888112E-5
F(1.4948068741289184E-5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Left bracket at 1.4948068741289184E-5
F(1.630698385950007E-5) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4353.8042000996375}, evalInputDelta = 0.0
Loops = 12
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.7062; Orientation: 0.0004; Line Search: 0.7038
Iteration 5 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 5
Final threshold in iteration 5: 0.0 (> 0.0) after 1.493s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 30.24 seconds (0.156 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1583819193626
Reset training subject: 1583821921754
Adding measurement 45492f91 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 883.0687980040733 < 883.0687980040733. Total: 1
th(0)=883.0687980040733;dx=-3.2190887008777943E27
Adding measurement 5dbec19c to history. Total: 1
New Minimum: 883.0687980040733 > 321.63180565374364
Armijo: th(2.154434690031884)=321.63180565374364; dx=3.8729023614001065E25 evalInputDelta=561.4369923503297
Non-optimal measurement 321.63180565374364 < 321.63180565374364. Total: 2
Armijo: th(1.077217345015942)=321.63180565374364; dx=3.8729023614001065E25 evalInputDelta=561.4369923503297
Non-optimal measurement 321.63180565374364 < 321.63180565374364. Total: 2
Armijo: th(0.3590724483386473)=321.63180565374364; dx=3.8729023614001065E25 evalInputDelta=561.4369923503297
Non-optimal measurement 321.63180565374364 < 321.63180565374364. Total: 2
Armijo: th(0.08976811208466183)=321.63180565374364; dx=3.8729023614001065E25 evalInputDelta=561.4369923503297
Non-optimal measurement 321.63180565374364 < 321.63180565374364. Total: 2
Armijo: th(0.017953622416932366)=321.63180565374364; dx=3.8729023614001065E25 evalInputDelta=561.4369923503297
Non-optimal measurement 321.69040212508423 < 321.63180565374364. Total: 2
Armijo: th(0.002992270402822061)=321.69040212508423; dx=3.8729023614001065E25 evalInputDelta=561.3783958789891
Non-optimal measurement 321.75005738982486 < 321.63180565374364. Total: 2
Armijo: th(4.2746720040315154E-4)=321.75005738982486; dx=3.8729023614001065E25 evalInputDelta=561.3187406142484
Non-optimal measurement 321.7592409523894 < 321.63180565374364. Total: 2
Armijo: th(5.343340005039394E-5)=321.7592409523894; dx=3.8729023614001065E25 evalInputDelta=561.3095570516839
Non-optimal measurement 321.76041660346124 < 321.63180565374364. Total: 2
Armijo: th(5.9370444500437714E-6)=321.76041660346124; dx=3.8729023614001065E25 evalInputDelta=561.308381400612
Non-optimal measurement 321.76054899942807 < 321.63180565374364. Total: 2
Armijo: th(5.937044450043771E-7)=321.76054899942807; dx=3.8729023614001065E25 evalInputDelta=561.3082490046452
Non-optimal measurement 321.7605623742809 < 321.63180565374364. Total: 2
Armijo: th(5.397313136403428E-8)=321.7605623742809; dx=3.8729023614001065E25 evalInputDelta=561.3082356297924
Non-optimal measurement 321.76056360032305 < 321.63180565374364. Total: 2
Armijo: th(4.4977609470028565E-9)=321.76056360032305; dx=3.8729023614001065E25 evalInputDelta=561.3082344037502
Adding measurement 7c454718 to history. Total: 2
New Minimum: 321.63180565374364 > 264.7638457820925
Armijo: th(3.4598161130791205E-10)=264.7638457820925; dx=3.871983021894725E25 evalInputDelta=618.3049522219808
Adding measurement 2525eff4 to history. Total: 3
New Minimum: 264.7638457820925 > 221.83355717447776
Armijo: th(2.4712972236279432E-11)=221.83355717447776; dx=3.8656360898620315E25 evalInputDelta=661.2352408295956
Adding measurement 22a5d13f to history. Total: 4
New Minimum: 221.83355717447776 > 159.51867363827105
Armijo: th(1.6475314824186289E-12)=159.51867363827105; dx=3.3528671577621524E25 evalInputDelta=723.5501243658023
Adding measurement 2c8a5d0d to history. Total: 5
New Minimum: 159.51867363827105 > 106.51462692049749
Armijo: th(1.029707176511643E-13)=106.51462692049749; dx=-1.700786128231007E26 evalInputDelta=776.5541710835759
Non-optimal measurement 883.0687980040733 < 106.51462692049749. Total: 6
Armijo: th(6.057101038303783E-15)=883.0687980040733; dx=-3.2190887008777943E27 evalInputDelta=0.0
Non-optimal measurement 106.51462692049749 < 106.51462692049749. Total: 6
MIN ALPHA (3.3650561323909904E-16): th(1.029707176511643E-13)=106.51462692049749
Fitness changed from 883.0687980040733 to 106.51462692049749
Iteration 1 complete. Error: 106.51462692049749 Total: 0.5006; Orientation: 0.0017; Line Search: 0.4914
Non-optimal measurement 106.51462692049749 < 106.51462692049749. Total: 6
Rejected: LBFGS Orientation magnitude: 2.213e+13, gradient 1.240e+13, dot -0.988; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 106.51462692049749, 159.51867363827105, 221.83355717447776, 264.7638457820925, 321.63180565374364, 883.0687980040733
Rejected: LBFGS Orientation magnitude: 4.523e+18, gradient 1.240e+13, dot -0.242; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 106.51462692049749, 159.51867363827105, 221.83355717447776, 264.7638457820925, 321.63180565374364
Rejected: LBFGS Orientation magnitude: 9.769e+07, gradient 1.240e+13, dot -0.242; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 106.51462692049749, 159.51867363827105, 221.83355717447776, 264.7638457820925
LBFGS Accumulation History: 3 points
Removed measurement 2c8a5d0d to history. Total: 5
Removed measurement 22a5d13f to history. Total: 4
Removed measurement 2525eff4 to history. Total: 3
Adding measurement 78ac60e8 to history. Total: 3
th(0)=106.51462692049749;dx=-1.5387406482347164E26
Adding measurement 4ea377c1 to history. Total: 4
New Minimum: 106.51462692049749 > 106.51462692049714
Armijo: th(2.154434690031884E-15)=106.51462692049714; dx=-1.5387406482347164E26 evalInputDelta=3.552713678800501E-13
Non-optimal measurement 106.5146269204973 < 106.51462692049714. Total: 5
Armijo: th(1.077217345015942E-15)=106.5146269204973; dx=-1.5387406482347164E26 evalInputDelta=1.8474111129762605E-13
Non-optimal measurement 106.51462692049714 < 106.51462692049714. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=106.51462692049714
Fitness changed from 106.51462692049749 to 106.51462692049714
Iteration 2 complete. Error: 106.51462692049714 Total: 0.9503; Orientation: 0.9345; Line Search: 0.0137
Non-optimal measurement 106.51462692049714 < 106.51462692049714. Total: 5
Rejected: LBFGS Orientation magnitude: 1.385e+37, gradient 1.240e+13, dot -0.970; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 106.51462692049714, 106.51462692049749, 264.7638457820925, 321.63180565374364, 883.0687980040733
Rejected: LBFGS Orientation magnitude: 1.232e+35, gradient 1.240e+13, dot -0.226; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 106.51462692049714, 106.51462692049749, 264.7638457820925, 321.63180565374364
LBFGS Accumulation History: 3 points
Removed measurement 4ea377c1 to history. Total: 4
Removed measurement 78ac60e8 to history. Total: 3
Adding measurement 3e3acb4f to history. Total: 3
th(0)=106.51462692049714;dx=-1.5387406482347164E26
Adding measurement 19d442f3 to history. Total: 4
New Minimum: 106.51462692049714 > 106.51462692049674
Armijo: th(2.154434690031884E-15)=106.51462692049674; dx=-1.5387406482347164E26 evalInputDelta=3.979039320256561E-13
Non-optimal measurement 106.51462692049695 < 106.51462692049674. Total: 5
Armijo: th(1.077217345015942E-15)=106.51462692049695; dx=-1.5387406482347164E26 evalInputDelta=1.8474111129762605E-13
Non-optimal measurement 106.51462692049674 < 106.51462692049674. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=106.51462692049674
Fitness changed from 106.51462692049714 to 106.51462692049674
Iteration 3 complete. Error: 106.51462692049674 Total: 0.5450; Orientation: 0.4167; Line Search: 0.1261
Non-optimal measurement 106.51462692049674 < 106.51462692049674. Total: 5
Rejected: LBFGS Orientation magnitude: 1.387e+37, gradient 1.240e+13, dot -0.970; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 106.51462692049674, 106.51462692049714, 264.7638457820925, 321.63180565374364, 883.0687980040733
Rejected: LBFGS Orientation magnitude: 1.247e+35, gradient 1.240e+13, dot -0.226; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 106.51462692049674, 106.51462692049714, 264.7638457820925, 321.63180565374364
LBFGS Accumulation History: 3 points
Removed measurement 19d442f3 to history. Total: 4
Removed measuremen
...skipping 81337 bytes...
84E-15)=47.35421161208138
Fitness changed from 47.35421161218834 to 47.35421161208138
Iteration 55 complete. Error: 47.35421161208138 Total: 0.4624; Orientation: 0.4507; Line Search: 0.0099
Non-optimal measurement 47.35421161208138 < 47.35421161208138. Total: 5
Rejected: LBFGS Orientation magnitude: 8.327e+33, gradient 6.089e+12, dot -0.986; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.35421161208138, 47.35421161218834, 264.7638457820925, 321.63180565374364, 883.0687980040733
Rejected: LBFGS Orientation magnitude: 1.614e+30, gradient 6.089e+12, dot -0.072; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.35421161208138, 47.35421161218834, 264.7638457820925, 321.63180565374364
LBFGS Accumulation History: 3 points
Removed measurement 4758891b to history. Total: 4
Removed measurement 47612255 to history. Total: 3
Adding measurement 606ca716 to history. Total: 3
th(0)=47.35421161208138;dx=-3.7070186135323073E25
Adding measurement e77d944 to history. Total: 4
New Minimum: 47.35421161208138 > 47.354211611974286
Armijo: th(2.154434690031884E-15)=47.354211611974286; dx=-3.7070186135323073E25 evalInputDelta=1.070930011337623E-10
Non-optimal measurement 47.35421161202788 < 47.354211611974286. Total: 5
Armijo: th(1.077217345015942E-15)=47.35421161202788; dx=-3.7070186135323073E25 evalInputDelta=5.349676257537794E-11
Non-optimal measurement 47.354211611974286 < 47.354211611974286. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=47.354211611974286
Fitness changed from 47.35421161208138 to 47.354211611974286
Iteration 56 complete. Error: 47.354211611974286 Total: 0.6132; Orientation: 0.6011; Line Search: 0.0105
Non-optimal measurement 47.354211611974286 < 47.354211611974286. Total: 5
Rejected: LBFGS Orientation magnitude: 8.330e+33, gradient 6.089e+12, dot -0.986; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.354211611974286, 47.35421161208138, 264.7638457820925, 321.63180565374364, 883.0687980040733
Rejected: LBFGS Orientation magnitude: 1.614e+30, gradient 6.089e+12, dot -0.072; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.354211611974286, 47.35421161208138, 264.7638457820925, 321.63180565374364
LBFGS Accumulation History: 3 points
Removed measurement e77d944 to history. Total: 4
Removed measurement 606ca716 to history. Total: 3
Adding measurement 1122a65b to history. Total: 3
th(0)=47.354211611974286;dx=-3.7070186135323073E25
Adding measurement 16c8c1f8 to history. Total: 4
New Minimum: 47.354211611974286 > 47.35421161186728
Armijo: th(2.154434690031884E-15)=47.35421161186728; dx=-3.7070186135323073E25 evalInputDelta=1.0700773600547109E-10
Non-optimal measurement 47.354211611920846 < 47.35421161186728. Total: 5
Armijo: th(1.077217345015942E-15)=47.354211611920846; dx=-3.7070186135323073E25 evalInputDelta=5.3439919156517135E-11
Non-optimal measurement 47.35421161186728 < 47.35421161186728. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=47.35421161186728
Fitness changed from 47.354211611974286 to 47.35421161186728
Iteration 57 complete. Error: 47.35421161186728 Total: 0.6237; Orientation: 0.6123; Line Search: 0.0098
Non-optimal measurement 47.35421161186728 < 47.35421161186728. Total: 5
Rejected: LBFGS Orientation magnitude: 8.328e+33, gradient 6.089e+12, dot -0.986; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.35421161186728, 47.354211611974286, 264.7638457820925, 321.63180565374364, 883.0687980040733
Rejected: LBFGS Orientation magnitude: 1.614e+30, gradient 6.089e+12, dot -0.072; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.35421161186728, 47.354211611974286, 264.7638457820925, 321.63180565374364
LBFGS Accumulation History: 3 points
Removed measurement 16c8c1f8 to history. Total: 4
Removed measurement 1122a65b to history. Total: 3
Adding measurement 2de5b214 to history. Total: 3
th(0)=47.35421161186728;dx=-3.7070186135323073E25
Adding measurement 26c85c6 to history. Total: 4
New Minimum: 47.35421161186728 > 47.35421161176017
Armijo: th(2.154434690031884E-15)=47.35421161176017; dx=-3.7070186135323073E25 evalInputDelta=1.071072119884775E-10
Non-optimal measurement 47.35421161181369 < 47.35421161176017. Total: 5
Armijo: th(1.077217345015942E-15)=47.35421161181369; dx=-3.7070186135323073E25 evalInputDelta=5.3589133131026756E-11
Non-optimal measurement 47.35421161176017 < 47.35421161176017. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=47.35421161176017
Fitness changed from 47.35421161186728 to 47.35421161176017
Iteration 58 complete. Error: 47.35421161176017 Total: 0.5748; Orientation: 0.5630; Line Search: 0.0101
Non-optimal measurement 47.35421161176017 < 47.35421161176017. Total: 5
Rejected: LBFGS Orientation magnitude: 8.328e+33, gradient 6.089e+12, dot -0.986; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.35421161176017, 47.35421161186728, 264.7638457820925, 321.63180565374364, 883.0687980040733
Rejected: LBFGS Orientation magnitude: 1.614e+30, gradient 6.089e+12, dot -0.072; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.35421161176017, 47.35421161186728, 264.7638457820925, 321.63180565374364
LBFGS Accumulation History: 3 points
Removed measurement 26c85c6 to history. Total: 4
Removed measurement 2de5b214 to history. Total: 3
Adding measurement 1f591af5 to history. Total: 3
th(0)=47.35421161176017;dx=-3.7070186135323073E25
Adding measurement 70f9d68 to history. Total: 4
New Minimum: 47.35421161176017 > 47.35421161165323
Armijo: th(2.154434690031884E-15)=47.35421161165323; dx=-3.7070186135323073E25 evalInputDelta=1.0694378715925268E-10
Non-optimal measurement 47.354211611706695 < 47.35421161165323. Total: 5
Armijo: th(1.077217345015942E-15)=47.354211611706695; dx=-3.7070186135323073E25 evalInputDelta=5.347544629330514E-11
Non-optimal measurement 47.35421161165323 < 47.35421161165323. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=47.35421161165323
Fitness changed from 47.35421161176017 to 47.35421161165323
Iteration 59 complete. Error: 47.35421161165323 Total: 0.4978; Orientation: 0.4855; Line Search: 0.0108
Non-optimal measurement 47.35421161165323 < 47.35421161165323. Total: 5
Rejected: LBFGS Orientation magnitude: 8.328e+33, gradient 6.089e+12, dot -0.986; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.35421161165323, 47.35421161176017, 264.7638457820925, 321.63180565374364, 883.0687980040733
Rejected: LBFGS Orientation magnitude: 1.614e+30, gradient 6.089e+12, dot -0.072; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.35421161165323, 47.35421161176017, 264.7638457820925, 321.63180565374364
LBFGS Accumulation History: 3 points
Removed measurement 70f9d68 to history. Total: 4
Removed measurement 1f591af5 to history. Total: 3
Adding measurement a650a93 to history. Total: 3
th(0)=47.35421161165323;dx=-3.7070186135323073E25
Adding measurement 2524828a to history. Total: 4
New Minimum: 47.35421161165323 > 47.35421161154605
Armijo: th(2.154434690031884E-15)=47.35421161154605; dx=-3.7070186135323073E25 evalInputDelta=1.0717826626205351E-10
Non-optimal measurement 47.354211611599645 < 47.35421161154605. Total: 5
Armijo: th(1.077217345015942E-15)=47.354211611599645; dx=-3.7070186135323073E25 evalInputDelta=5.3582027703669155E-11
Non-optimal measurement 47.35421161154605 < 47.35421161154605. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=47.35421161154605
Fitness changed from 47.35421161165323 to 47.35421161154605
Iteration 60 complete. Error: 47.35421161154605 Total: 0.4989; Orientation: 0.4859; Line Search: 0.0115
Final threshold in iteration 60: 47.35421161154605 (> 0.0) after 30.237s (< 30.000s)
Returns
47.35421161154605
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
[-0.8417164661867791, -0.37155458288020343, -2.038099112570966, 1.6397710284429539, 0.5725021399176882, -0.9977874186559264, 0.9630706180236693, 2.72152154592178, 3.0681831967311366, 1.1363136785484347, 1.6275837260691737, -0.4202659836086261, -2.1573620566963925, -0.5361658402690646, 0.7006700286076208, -2.229747695818912, -0.3226120324365368, -0.19202892777976255, 1.0429493371353757, -0.05566088171323355, -1.3967092649082957, 0.7407235218075904, 1.8776456051364978, 0.7534032900151527, -1.2964043481046068, -0.5058109984499959, 0.0736182180559399, 1.0429465287137205, -0.2454033271492115, -0.048128718332166434, 1.5978167741855496, 0.6268010386821407, -0.4371179526540195, -0.44112411202576024, -1.4471105930182302, 0.7526183251397058, 0.63282121337828, 1.372854267168766, -0.794540327302508, -1.6758018733161135, 0.6480963187256142, -0.6616584237662524, 0.2791700130003136, -0.774891372285079, 0.8858744608656849, 0.601459577393468, -0.9595285295181049, 0.2712692243484416, 1.640311152434645, -0.7908376015139461, -1.5496888952173637, -1.52492664919926, -0.049404605291179876, 0.45577016234185747, -0.10393502439723618, 1.698871539482908, 0.07277308127634362, -1.619402618737871, -1.4434457147381805, -0.1845748384587492, 1.3756035143644045, 1.8861640970113356, -1.719470910351677, 0.5804203548430307, 0.5526205872680005, 1.8786971658735783, 0.7927962872815574, -1.6340341458064023, -1.8034901371437408, 0.6623649926372861, -0.9413910157351303, 0.9902993538335862, -1.7287498752700954, 1.582144207016785, 0.943735455175517, -0.811889938790807, -0.8715577655544341, -1.517304823733157, -0.3460977950049143, -0.7176356705646602, 0.8664074254365431, 0.1577488948456906, 0.6202048621089444, -1.4538430007974597, -0.42704894559903966, -0.22780915240117383, -0.22105886663593416, -0.24146272924049425, -0.8244240281636915, -2.2200449387996315, -1.0747407232731698, -1.0048988579895377, 2.3282719572730017, -1.1765394774425193, -0.14104999181379893, 1.9761200960753225, 0.47835238899987614, -1.1610216892212608, -0.03959539787870259, -1.5698071206697948, -1.487148214748598, 1.0412605982507892, 1.467823511368272, -1.396764930351944, -0.012201864791017685, 1.116380578014413, -1.8467186236523594, 1.0800261564287381, 0.5926586628076982, 1.6165984056779725, -0.5705094807590023, -1.1672983426151993, 0.48636185268505855, 0.8378102755165824, 1.4293031749092227, 1.7123595551163313, -0.08132517034537418, -1.4119005346048255, -1.1187443700104442, 0.632408538532113, 1.4292575272902803, -1.6538092730100553, -1.732874485928453, -0.6401703365347757, -1.7635432371786954, 1.7988258851936751, -0.7698770196205355, -0.8251132937736746, 1.6025230240743396, 0.332012624698733, 0.13514209309768657, -0.5221274957598669, -0.8971038782654451, -1.2948189575373747, 1.8759148958072513, -0.3580808550344617, 0.7678209305870882, 1.1592941462638366, 1.6381478184391873, 1.1212346255454897, -1.7720431551173927, 1.989498017707112, -0.8364553594612365, -0.5612648827761736, 0.16280406811461678, 1.7253511930109589, -0.4551279747376236, -1.317502479164405, 0.13229364939231797, 0.13012401015185107, -1.300809523314937, -1.524250494274927, -0.09182737953044005, -1.180090771692065, -1.4261651777831756, -1.3353675586119134, -0.6258715283908626, 1.0598100201864542, 1.9175758915124814, -0.071428073073291, 0.4540917654350991, 0.5550049583288033, 1.8199999999997594, -1.6440000000002593, -1.8800000000000723, 1.2880000000001233, -0.25199999999966727, -1.2360000000000473, 1.9200000000002393, 0.3559999999996928, 0.8399999999997613, 1.984000000000091, 1.244000000000333, 1.1680000000001904, 0.7520000000001308, 0.8319999999998124, 1.660000000000288, -1.9639999999999316, -0.8519999999997653, 0.33200000000018876, -0.9239999999997178, -1.1279999999999009, -1.7359999999998807, 1.2599999999999245, 1.3160000000000467, 0.5519999999998224, -0.8159999999996925, -1.6360000000000519, 0.6719999999997386, 0.41789855186572766, 0.11445985339946832, -0.32175689258675977, -0.08533633045481706, 0.6820369808624831, -0.8804652249428511, -1.3317608335036113, -0.05513105602611984, -0.5237345265823059, -0.970451849313052, 1.342415837948707, 1.6900832833031083, 0.178038033855413, -1.2386231397502423, -0.9965839316323405, 0.6953718059069584, 1.6161274178703735, 1.260005046323848, 0.6698523077627503, -0.6073958772042024, 1.2040435431078806, 1.0430494161900863, -1.550069515631106, 0.8280953568371686, -0.44708509600345736, -0.2479327793767215, -0.8914710250423363, -0.005616521262286619, 0.1682449446398101, 1.9285202922907176, 1.7821379605831458, -0.20836426262746346, -1.0188412444602426, 1.6015872652803487, -0.6005793777698181, -0.5154825752830239, -2.071543954821645, 0.8007027103622886, -1.0937691098365252, 0.25334232064073403, -0.30152588574191497, -0.5451289260432847, -0.9105354937970865, 0.22686992692766586, -0.9542274490744576, -1.2178012333934416, -0.8588986093948388, 1.4682437976099314, 0.054899182909694096, 0.9250096080550596, 2.0117877524317, 0.002257852086134276, -1.6885805247996384, -0.7426857883114296, 1.8720000000006196, -1.4999999999999063, 0.8600000000001439, 0.5120000000005124, 1.8800000000007144, -1.8560000000007082, 1.6520000000000699, 0.40800000000019027, 0.9759999999994544, 1.6399999999993604, -0.6639999999995234, 0.07600000000062959, 1.4239999999996242, -0.6519999999997497, -0.12799999999925854, 1.3639999999999302, 1.2880000000003329, 0.7640000000004642, 0.8560000000005232, 1.1960000000007565, 1.1800000000000752, 0.9479999999993488, 1.2560000000004927, 0.8440000000004605, 0.18399999999968888, 1.1559999999996404, -0.5479999999996689, 1.77848272250087, 2.0147369139330222, 1.3872782252023568, -0.2765742396744987, 0.9916707159393447, 1.1189120771904744, 1.408537375724078, 0.6108686275580069, 1.8718050796999022, 0.9612828431111402, 0.5987961850647984, 1.4358090152452443, -0.2009759662300663, -1.298080995947446, 1.0567744602483853, -1.320973316093373, 1.7545815077198912, 0.1864076298727532, 0.28296741277471793, -0.8538399758789683, -0.6344603153212374, -0.8165242043614565, 0.4601619918954819, -0.19188804342006546, -0.2821731954834319, -1.6847086034361523, 0.2685834754919559, 1.5080155208321953, -1.639693673380314, -0.10391766435172846, 0.1979793497692676, -0.3618867553233072, 1.5926073912042262, -0.8049652770826291, -0.6955830597224294, -1.5413899777781188, -1.0237554914348335, -0.2894878342580757, 0.8017888988432564, -1.248470997916774, -1.8963526238430861, -0.8801236361098198, 1.7566872284726118, 0.520749046528147, -0.1394466664341623, 0.34664356226856186, -0.2973744569447445, 0.28100381388952633, 0.26999487060153227, 0.7273075094915343, 1.000422202313908, 0.6377991576400909, 1.140404183101188, -1.6147283962972097, 1.7947685580672603, 1.4732642077474807, 1.801888374760592, -0.22462382755562182, 0.9498086666942149, -0.5235524806713757, 1.1821278384186795, 0.9217762403353053, -1.296752514617289, 1.646399388977256, 1.0620481303519445, -0.8881594161334874, 0.9568636306707747, -0.42527974201279073, -1.5552157379387541, 1.3764316456068086, 0.6285279063101119, -1.6990715166128245, 1.1340960909748297, -0.46950333194823324, -1.590992148003173, -1.8723684901761972, -0.02571138761926147, 0.5186084630193306, 1.6944314758785304, 1.3895198845845098, -0.017983617092645548, 0.7612186528820452, 1.5232857501087207, 1.3873476404884197, 1.0915157726915508, 0.9568483777342163, -1.206225088736045, 0.038783715044763, 2.259502608775621, 0.5367740294962158, -1.336254814500757, -0.6214447969832007, -0.6489510821636285, 0.5873864457478593, -0.3437626145047362, 0.5358813923764922, -1.8453889689398488, -0.7406529677217092, -1.0850508089355793, 1.5857188827089341, -0.9212429857032012, -0.6185489544961476, 1.9374493329164024, -0.8759026686703647, 1.9381193234662122, 0.8920482398956051, -0.38755122522076735, 0.6729823325809523, 0.7800000000002963, 0.07600000000032471, 0.28399999999991576, -1.1759999999999522, 0.9080000000001083, -0.6280000000002751, 0.035999999999938616, 0.028000000000198904, 1.288000000000355, -1.0919999999996912, -1.243999999999741, 1.732000000000241, -0.15200000000019576, 1.0959999999999257, -1.9760000000002582, 0.015999999999675812, 0.04800000000035358, 1.0479999999997207, -1.1880000000002604, 1.051999999999811, -1.8359999999998602, -1.1759999999997912, 0.07200000000013919, 1.8839999999999515, -1.5519999999999068, -0.03199999999993098, -0.7559999999998709, -1.2539560755325734, 0.8932386930883334, 1.1095000130961923, 0.7273036852340753, -1.210941136996072, 1.649492721826374, -0.3366594702549327, 0.9642564517899401, 1.0573011295013282, -0.4697554446003517, -1.139023354861543, 0.7819798648697034, -2.0651210070732997, 1.187839186854151, -0.9967418939048271, 0.07618542266852021, -0.47655628859064425, -1.404091529121146, -1.2037140909215736, -0.3434740300541875, 1.4428068454578713, -1.35806091784318, 0.3583699441209486, 1.7231685045828986, -1.0271196741469362, -0.8024575260917758, -0.6415948426116844]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 5.1878500330638815, 4.396225872999171, 5.923561581528217, 0.04121155388445373 ], [ 2.245917748384496, 4.825199999999958, 6.743982211753897, 15.674463999999967 ] ],
[ [ -1.6412599792276046, 12.357761928512854, 4.622448000014092, 6.435297130268101 ], [ 1.260847379935452, 3.2759337012581478, -0.6268572088771085, 1.2568064528520915 ] ]
]
TrainingTester.java:432 executed in 0.01 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -0.8902256573042076], [250.0, 2.4193281064764975]; valueStats=DoubleSummaryStatistics{count=313, sum=12632.088481, min=0.128758, average=40.358110, max=262.620187}
Plotting 250 points for GD
Plotting 4 points for CjGD
Plotting 60 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -0.8902256573042076], [29.735, 2.4193281064764975]; valueStats=DoubleSummaryStatistics{count=313, sum=12632.088481, min=0.128758, average=40.358110, max=262.620187}
Plotting 250 points for GD
Plotting 4 points for CjGD
Plotting 60 points for LBFGS
Returns
In this apply, attempt to train a network to emulate a randomized network given an example input/output. The target state is:
TrainingTester.java:279 executed in 0.00 seconds (0.000 gc):
RefList<double[]> temp_18_0037 = temp_18_0035.state();
assert temp_18_0037 != null;
String temp_18_0036 = temp_18_0037.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b).orElse("");
temp_18_0037.freeRef();
return temp_18_0036;
Returns
[-0.928, 1.7, 0.752, -1.656, -0.148, 1.384, -0.084, 0.3, 1.288, 0.48, 1.86, 1.432, 0.124, 1.764, -0.852, 0.08, 0.264, 0.584, 0.692, 0.492, -1.052, -0.848, -0.628, 0.66, -1.848, -0.968, 1.24, 0.82, -0.804, -1.696, 0.248, -1.736, 1.052, 0.892, -0.124, 0.832, 1.64, -0.684, 0.464, -0.34, 0.448, -1.796, 0.04, -0.056, 1.316, -1.248, -1.244, -0.468, -1.664, 0.08, -0.296, -0.384, 0.992, -0.312, -1.96, 1.572, -0.384, 0.968, 1.172, -1.976, 0.012, -0.732, -1.472, 0.66, 1.62, -0.52, -1.632, 0.656, -0.032, -1.812, -1.552, -0.636, -1.616, 0.976, 1.544, 0.812, 0.688, 0.496, 0.264, 1.244, 0.652, 1.088, -0.828, -0.356, 0.48, 0.004, 0.636, 1.516, -1.596, -1.564, -0.688, -1.976, -1.352, -0.26, -1.5, 0.048, 0.572, -0.08, 1.356, -1.88, 1.776, -1.256, 0.328, 0.932, 0.908, 0.092, -0.964, 1.628, -1.572, 1.236, 0.552, 0.688, 1.72, -0.492, -0.768, 1.368, -1.316, 0.036, 0.184, 0.672, 1.956, 0.08, -1.228, 1.512, 1.64, -1.92, -1.18, 0.124, -0.816, 0.6, -1.524, 1.96, 1.66, 1.724, -0.452, -1.412, -0.876, 0.332, 1.656, 1.752, 0.076, -1.644, 0.764, 0.568, -0.396, -1.86, 0.072, -0.252, -1.564, -1.156, 1.256, -1.256, 0.788, 1.144, -1.688, -1.564, -0.756, -0.448, 1.836, 0.784, 0.376, -0.312, -0.384, 0.312, -1.964, 0.996, 1.288, 1.132, -0.408, -1.716, -0.432, 0.164, 0.52, 1.012, 1.096, -1.176, 0.08, -1.028, -1.444, 1.652, 1.048, 1.324, 0.132, 1.512, 1.668, 1.168, 0.356, 0.184, -1.096, -0.316, 0.244, 1.424, -0.464, -1.808, -1.184, -1.312, -0.816, -0.548, 0.9, 0.148, 1.98, -1.42, -0.376, 0.86, 1.524, 1.108, 0.512, 0.552, 0.284, -0.028, 0.296, 0.1, 1.552, 1.492, -1.856, 0.856, 0.76, -1.832, -0.632, 0.408, -0.4, 1.304, 1.24, 1.88, -0.472, 1.352, 1.524, -1.188, -0.608, -1.16, 1.26, -1.168, -0.736, -0.932, 1.732, 0.78, -1.524, -0.504, -1.756, -0.2, 0.676, -0.176, 1.632, 0.56, -1.356, 0.532, -0.684, 0.948, 0.8, 0.016, 0.78, -0.584, 0.856, 1.972, -0.652, 0.184, -1.732, -0.02, 0.844, -0.664, -1.764, 0.428, 0.848, -1.54, 0.488, -1.476, -1.58, -0.152, 0.696, 1.556, 0.632, 0.868, -0.832, 0.52, 1.196, -1.9, -0.336, 0.392, -0.852, -0.068, -0.5, 1.556, 1.364, -1.372, 0.284, -1.76, 1.208, 1.144, -0.856, -1.5, 0.996, -0.368, -0.06, 1.052, -0.676, 1.008, 1.536, -1.176, 0.52, 0.028, 1.352, -0.28, 1.052, -0.18, -1.536, -1.764, -1.092, -0.612, 1.18, -0.76, -0.124, 1.66, -0.396, -1.46, -0.096, -0.748, 0.62, -0.808, 1.656, 1.192, 0.612, 0.7, 1.884, -1.476, -0.616, -0.856, -1.724, -1.16, -1.524, 0.972, 1.264, 0.76, -0.488, -1.192, -1.944, -1.76, -0.556, -1.636, -0.58, 0.076, 0.344, 1.22, 1.92, 0.412, -1.34, -0.924, 0.96, 1.948, 1.852, 1.652, 0.08, -1.104, 1.972, -1.312, 0.404, -1.236, -0.528, 1.704, 1.82, 0.048, 1.476, 0.644, 1.836, -0.344, -1.552, 1.872, -1.6, -0.212, 0.332, 1.984, 0.84, -1.176, -0.032, -1.104, 1.22, -1.736, -1.484, 1.064, 0.244, 0.284, 1.156, -1.264, 1.948, -1.036, -1.516, 1.032, 1.288, -2.0, 1.336, -0.416, -1.824, 1.844, -1.116, 1.596, 1.912, -1.128, 1.42, -0.86, 0.56, 1.352, -1.108, -1.432, -0.212, -1.656, -1.808, -0.712, -1.096, -1.1, -0.784, 0.476, -0.968, -0.548, 1.916, 0.572, -0.664, -0.012, -0.968, -1.624, -0.892, -1.836, 0.972, 1.448, 1.348, -1.384, -1.148, 1.612, -0.452, 1.616, 1.048, -1.72, -0.128, 1.792]
We simultaneously regress this target input:
TrainingTester.java:287 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(x -> {
RefStream<Tensor> temp_18_0006 = RefArrays.stream(RefUtil.addRef(x));
if (null != x)
RefUtil.freeRef(x);
return temp_18_0006;
}).map(x -> {
String temp_18_0007 = x.prettyPrint();
x.freeRef();
return temp_18_0007;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ -0.436, 1.964, -1.444 ], [ -1.8, 0.384, -1.416 ], [ -1.088, 1.104, 0.708 ] ],
[ [ -0.276, -0.34, -0.476 ], [ -1.044, 1.424, -1.516 ], [ 1.364, 0.772, 0.508 ] ],
[ [ -1.544, 0.612, 1.968 ], [ 1.72, 0.76, 1.804 ], [ 0.268, 1.644, 1.128 ] ]
]
[
[ [ -1.044, -1.516, 1.644 ], [ -1.444, 0.772, -1.088 ], [ 1.968, -1.416, 1.424 ] ],
[ [ 1.72, 0.708, 1.128 ], [ -0.436, 0.384, 1.964 ], [ 1.804, -0.34, 1.364 ] ],
[ [ -0.276, 0.508, -0.476 ], [ 0.76, -1.8, 1.104 ], [ -1.544, 0.612, 0.268 ] ]
]
[
[ [ 1.364, -0.34, 0.384 ], [ -1.544, 1.72, 0.772 ], [ 1.968, -0.436, 1.128 ] ],
[ [ 1.964, -1.416, -1.088 ], [ 0.76, -1.444, -0.276 ], [ 1.804, 1.644, -1.516 ] ],
[ [ -1.8, 0.508, 0.268 ], [ 0.708, 1.424, 1.104 ], [ -0.476, 0.612, -1.044 ] ]
]
[
[ [ 0.612, -1.544, 0.772 ], [ 1.968, -1.044, 1.104 ], [ 1.72, 0.384, 1.964 ] ],
[ [ 1.644, 1.364, -1.444 ], [ -1.088, -0.436, -0.276 ], [ 1.128, 0.76, 0.708 ] ],
[ [ 1.424, -0.34, 1.804 ], [ -1.516, -1.416, 0.508 ], [ -1.8, -0.476, 0.268 ] ]
]
[
[ [ -0.436, -1.044, -1.444 ], [ -1.516, -1.544, 1.104 ], [ 0.772, 1.72, -1.416 ] ],
[ [ 1.804, 0.612, -0.476 ], [ -0.276, -1.8, 0.508 ], [ 0.268, 0.76, 1.424 ] ],
[ [ -1.088, 1.644, -0.34 ], [ 0.384, 1.364, 1.128 ], [ 0.708, 1.964, 1.968 ] ]
]
Which produces the following output:
TrainingTester.java:308 executed in 0.00 seconds (0.000 gc):
return RefStream.of(RefUtil.addRef(output_target)).map(x -> {
String temp_18_0008 = x.prettyPrint();
x.freeRef();
return temp_18_0008;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 9.125119999999999, 4.667344, 0.630927999999999, 4.66432 ], [ -5.1271200000000015, -5.388448, 4.819776, -8.671599999999996 ] ],
[ [ 12.840447999999999, -1.4131039999999981, -5.818895999999999, 1.3170560000000013 ], [ -4.053104, 6.927904000000001, -8.586592000000001, 10.685728 ] ]
]
[
[ [ 8.349520000000002, 6.48088, 2.435008, 21.59456 ], [ 5.0295840000000025, 7.488431999999999, 1.7384480000000009, -8.707632 ] ],
[ [ -0.01881599999999893, -1.9828159999999972, -3.9408160000000008, 8.255504000000004 ], [ -0.7740799999999999, -4.905344, 4.120096, 0.358176000000001 ] ]
]
[
[ [ -3.1694880000000003, 5.171487999999998, 1.1615359999999981, 10.720047999999993 ], [ -12.779872, 4.938736, 11.553279999999997, -1.7230399999999986 ] ],
[ [ -2.172128, -2.625775999999999, -6.9444, 14.1324 ], [ 3.514015999999998, -1.1054879999999982, -3.766351999999999, -1.843408 ] ]
]
[
[ [ -8.936720000000003, 10.921471999999998, -0.9664959999999989, 2.1872479999999994 ], [ 10.227024, -6.974000000000001, 5.237359999999999, 8.145248 ] ],
[ [ -6.2863679999999995, -1.3544800000000004, -2.8379520000000005, -1.1262079999999972 ], [ -4.187328000000002, 5.543824000000002, 1.6024960000000013, 2.742224 ] ]
]
[
[ [ 8.903488000000001, -0.8968960000000002, -5.938623999999999, 5.987536000000001 ], [ 1.3571999999999997, 2.325696, 10.578463999999999, -5.806655999999999 ] ],
[ [ 2.0660320000000003, -1.6752799999999994, -1.071536, 12.672096000000002 ], [ 7.46648, 7.208703999999998, 5.410144, 16.114767999999998 ] ]
]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 15.49 seconds (0.092 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1614192663318
Reset training subject: 1614195089106
Constructing line search parameters: GD
th(0)=835.3137106258573;dx=-4.488978031875305E27
Armijo: th(2.154434690031884)=1350.8790308109303; dx=1.5813838489068568E40 evalInputDelta=-515.565320185073
Armijo: th(1.077217345015942)=1350.8790308109303; dx=7.906919244532059E39 evalInputDelta=-515.565320185073
Armijo: th(0.3590724483386473)=1348.8996770870865; dx=2.6356397481743845E39 evalInputDelta=-513.5859664612292
Armijo: th(0.08976811208466183)=1310.6146647444614; dx=6.589099370402572E38 evalInputDelta=-475.30095411860407
Armijo: th(0.017953622416932366)=1270.7778942559657; dx=1.3178198740450784E38 evalInputDelta=-435.46418363010844
Armijo: th(0.002992270402822061)=1270.7778942559657; dx=2.1963664563736004E37 evalInputDelta=-435.46418363010844
Armijo: th(4.2746720040315154E-4)=1270.7778942559657; dx=3.137666362460837E36 evalInputDelta=-435.46418363010844
Armijo: th(5.343340005039394E-5)=1270.7778942559657; dx=3.922082914415419E35 evalInputDelta=-435.46418363010844
Armijo: th(5.9370444500437714E-6)=1270.7778942559657; dx=4.357869512163141E34 evalInputDelta=-435.46418363010844
Armijo: th(5.937044450043771E-7)=1270.7778942559657; dx=4.3578655356414757E33 evalInputDelta=-435.46418363010844
Armijo: th(5.397313136403428E-8)=1270.7778942559657; dx=3.96165577460675E32 evalInputDelta=-435.46418363010844
Armijo: th(4.4977609470028565E-9)=1270.7778942559657; dx=3.3009747960768196E31 evalInputDelta=-435.46418363010844
Armijo: th(3.4598161130791205E-10)=1270.7778942559657; dx=2.535132897838961E30 evalInputDelta=-435.46418363010844
Armijo: th(2.4712972236279432E-11)=1270.7778942559657; dx=1.769781608265801E29 evalInputDelta=-435.46418363010844
Armijo: th(1.6475314824186289E-12)=1149.194851187752; dx=7.22147485995912E27 evalInputDelta=-313.88114056189465
New Minimum: 835.3137106258573 > 370.69866245570165
Armijo: th(1.029707176511643E-13)=370.69866245570165; dx=1.0713305471265839E26 evalInputDelta=464.61504817015566
Armijo: th(6.057101038303783E-15)=800.6726843984711; dx=-4.296780376183776E27 evalInputDelta=34.64102622738619
MIN ALPHA (3.3650561323909904E-16): th(1.029707176511643E-13)=370.69866245570165
Fitness changed from 835.3137106258573 to 370.69866245570165
Iteration 1 complete. Error: 370.69866245570165 Total: 0.3889; Orientation: 0.0006; Line Search: 0.3817
th(0)=370.69866245570165;dx=-2.762605319301872E27
New Minimum: 370.69866245570165 > 284.3863516257287
Armijo: th(2.154434690031884E-15)=284.3863516257287; dx=-2.0337421587664844E27 evalInputDelta=86.31231082997294
Armijo: th(1.077217345015942E-15)=287.2851248223471; dx=-2.03443774276187E27 evalInputDelta=83.41353763335457
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=284.3863516257287
Fitness changed from 370.69866245570165 to 284.3863516257287
Iteration 2 complete. Error: 284.3863516257287 Total: 0.0980; Orientation: 0.0765; Line Search: 0.0190
th(0)=284.3863516257287;dx=-2.0291556314521884E27
New Minimum: 284.3863516257287 > 131.94387813495402
Armijo: th(2.154434690031884E-15)=131.94387813495402; dx=-2.814687053053509E26 evalInputDelta=152.4424734907747
Armijo: th(1.077217345015942E-15)=248.66919249948137; dx=-1.8076061226351353E27 evalInputDelta=35.71715912624734
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=131.94387813495402
Fitness changed from 284.3863516257287 to 131.94387813495402
Iteration 3 complete. Error: 131.94387813495402 Total: 0.0428; Orientation: 0.0008; Line Search: 0.0395
th(0)=131.94387813495402;dx=-2.542137360291018E26
New Minimum: 131.94387813495402 > 131.6706695865642
Armijo: th(2.154434690031884E-15)=131.6706695865642; dx=-2.541695054363165E26 evalInputDelta=0.27320854838981745
Armijo: th(1.077217345015942E-15)=131.8056998974589; dx=-2.541916207327091E26 evalInputDelta=0.1381782374951115
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=131.6706695865642
Fitness changed from 131.94387813495402 to 131.6706695865642
Iteration 4 complete. Error: 131.6706695865642 Total: 0.0245; Orientation: 0.0009; Line Search: 0.0207
th(0)=131.6706695865642;dx=-2.54126614466419E26
New Minimum: 131.6706695865642 > 105.13958538901973
Armijo: th(2.154434690031884E-15)=105.13958538901973; dx=-1.7709707341559866E26 evalInputDelta=26.531084197544473
Armijo: th(1.077217345015942E-15)=131.53873683480543; dx=-2.5410583866863877E26 evalInputDelta=0.13193275175876806
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=105.13958538901973
Fitness changed from 131.6706695865642 to 105.13958538901973
Iteration 5 complete. Error: 105.13958538901973 Total: 0.1977; Orientation: 0.1791; Line Search: 0.0162
th(0)=105.13958538901973;dx=-1.6984545624681538E26
New Minimum: 105.13958538901973 > 104.7068899627366
Armijo: th(2.154434690031884E-15)=104.7068899627366; dx=-1.6982522684192305E26 evalInputDelta=0.4326954262831322
Armijo: th(1.077217345015942E-15)=104.91797892852681; dx=-1.698353415443685E26 evalInputDelta=0.22160646049292154
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=104.7068899627366
Fitness changed from 105.13958538901973 to 104.7068899627366
Iteration 6 complete. Error: 104.7068899627366 Total: 0.0675; Orientation: 0.0007; Line Search: 0.0646
th(0)=104.7068899627366;dx=-1.6980567085476922E26
New Minimum: 104.7068899627366 > 104.31108519619897
Armijo: th(2.154434690031884E-15)=104.31108519619897; dx=-1.6978678821812588E26 evalInputDelta=0.39580476653762275
Armijo: th(1.077217345015942E-15)=104.50493366905137; dx=-1.697962295364471E26 evalInputDelta=0.20195629368522816
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=104.31108519619897
Fitness changed from 104.7068899627366 to 104.31108519619897
Iteration 7 complete. Error: 104.31108519619897 Total: 0.0289; Orientation: 0.0008; Line Search: 0.0256
th(0)=104.31108519619897;dx=-1.6976857886780115E26
New Minimum: 104.31108519619897 > 103.9443016826453
Armijo: th(2.154434690031884E-15)=103.9443016826453; dx=-1.6975104274265537E26 evalInputDelta=0.36678351355367056
Armijo: th(1.077217345015942E-15)=104.12443286447633; dx=-1.6975981080522793E26 evalInputDelta=0.18665233172264095
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=103.9443016826453
Fitness changed from 104.31108519619897 to 103.9443016826453
Iteration 8 complete. Error: 103.9443016826453 Total: 0.1158; Orientation: 0.0011; Line Search: 0.1119
th(0)=103.9443016826453;dx=-1.6973417978458758E26
New Minimum: 103.9443016826453 > 77.62265779972363
Armijo: th(2.154434690031884E-15)=77.62265779972363; dx=-7.420423449452311E25 evalInputDelta=26.321643882921677
Armijo: th(1.077217345015942E-15)=103.77004901214391; dx=-1.697260848615668E26 evalInputDelta=0.1742526705013887
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=77.62265779972363
Fitness changed from 103.9443016826453 to 77.62265779972363
Iteration 9 complete. Error: 77.62265779972363 Total: 0.0211; Orientation: 0.0006; Line Search: 0.0182
th(0)=77.62265779972363;dx=-7.418880603969366E25
New Minimum: 77.62265779972363 > 77.29968384078202
Armijo: th(2.154434690031884E-15)=77.29968384078202; dx=-7.417357632922259E25 evalInputDelta=0.32297395894160275
Armijo: th(1.077217345015942E-15)=77.45887303539122; dx=-7.4181191184457976E25 evalInputDelta=0.16378476433240508
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=77.29968384078202
Fitness changed from 77.62265779972363 to 77.29968384078202
Iteration 10 complete. Error: 77.29968384078202 Total: 0.0240; Orientation: 0.0008; Line Search: 0.0208
th(0)=77.29968384078202;dx=-7.415854531653782E25
New Minimum: 77.29968384078202 > 76.99373640702964
Armijo: th(2.154434690031884E-15)=76.99373640702964; dx=-7.414371295684615E25 evalInputDelta=0.30594743375237954
Armijo: th(1.077217345015942E-15)=77.14472226643548; dx=-7.415112913669186E25 evalInputDelta=0.15496157434654378
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=76.99373640702964
Fitness changed from 77.29968384078202 to 76.99373640702964
Iteration 11 complete. Error: 76.99373640702964 Total: 0.0313; Orientation: 0.0012; Line Search: 0.0273
th(0)=76.99373640702964;dx=-7.412907920604057E25
New Minimum: 76.99373640702964 > 76.70261756554022
Arm
...skipping 96534 bytes...
.2040680896
New Minimum: 13.739179923655579 > 13.61322784139811
END: th(3.6288000000000014E-7)=13.61322784139811; dx=-303740.5743252896 evalInputDelta=0.12595208225746823
Fitness changed from 13.739179923655579 to 13.61322784139811
Iteration 179 complete. Error: 13.61322784139811 Total: 0.0592; Orientation: 0.0009; Line Search: 0.0509
th(0)=13.61322784139811;dx=-227185.49954329978
New Minimum: 13.61322784139811 > 13.463954033116817
END: th(7.818012603187703E-7)=13.463954033116817; dx=-174232.7082234669 evalInputDelta=0.14927380828129344
Fitness changed from 13.61322784139811 to 13.463954033116817
Iteration 180 complete. Error: 13.463954033116817 Total: 0.0134; Orientation: 0.0011; Line Search: 0.0096
th(0)=13.463954033116817;dx=-137832.49597499298
New Minimum: 13.463954033116817 > 13.268374180173737
END: th(1.684339755941406E-6)=13.268374180173737; dx=-111577.42572905651 evalInputDelta=0.1955798529430801
Fitness changed from 13.463954033116817 to 13.268374180173737
Iteration 181 complete. Error: 13.268374180173737 Total: 0.0274; Orientation: 0.0161; Line Search: 0.0087
th(0)=13.268374180173737;dx=-93125.45074810905
New Minimum: 13.268374180173737 > 12.986886769315458
END: th(3.628800000000002E-6)=12.986886769315458; dx=-78379.27070561932 evalInputDelta=0.2814874108582792
Fitness changed from 13.268374180173737 to 12.986886769315458
Iteration 182 complete. Error: 12.986886769315458 Total: 0.0212; Orientation: 0.0093; Line Search: 0.0095
th(0)=12.986886769315458;dx=-67566.61877493725
New Minimum: 12.986886769315458 > 12.56086516279247
END: th(7.818012603187705E-6)=12.56086516279247; dx=-57623.44469255147 evalInputDelta=0.42602160652298693
Fitness changed from 12.986886769315458 to 12.56086516279247
Iteration 183 complete. Error: 12.56086516279247 Total: 0.2089; Orientation: 0.0115; Line Search: 0.1948
th(0)=12.56086516279247;dx=-50047.91356646599
New Minimum: 12.56086516279247 > 11.9186386068938
END: th(1.6843397559414064E-5)=11.9186386068938; dx=-42480.0613055269 evalInputDelta=0.6422265558986702
Fitness changed from 12.56086516279247 to 11.9186386068938
Iteration 184 complete. Error: 11.9186386068938 Total: 0.0251; Orientation: 0.0008; Line Search: 0.0219
th(0)=11.9186386068938;dx=-36659.29542151423
New Minimum: 11.9186386068938 > 10.990562366012693
END: th(3.628800000000003E-5)=10.990562366012693; dx=-30876.0471290972 evalInputDelta=0.9280762408811079
Fitness changed from 11.9186386068938 to 10.990562366012693
Iteration 185 complete. Error: 10.990562366012693 Total: 0.0118; Orientation: 0.0007; Line Search: 0.0087
th(0)=10.990562366012693;dx=-26495.711936982352
New Minimum: 10.990562366012693 > 9.72400067975021
END: th(7.818012603187706E-5)=9.72400067975021; dx=-22405.64189034497 evalInputDelta=1.2665616862624827
Fitness changed from 10.990562366012693 to 9.72400067975021
Iteration 186 complete. Error: 9.72400067975021 Total: 0.0528; Orientation: 0.0007; Line Search: 0.0494
th(0)=9.72400067975021;dx=-19387.178875369445
New Minimum: 9.72400067975021 > 8.077987578600451
END: th(1.6843397559414066E-4)=8.077987578600451; dx=-16755.757683600095 evalInputDelta=1.6460131011497587
Fitness changed from 9.72400067975021 to 8.077987578600451
Iteration 187 complete. Error: 8.077987578600451 Total: 0.1698; Orientation: 0.0008; Line Search: 0.1666
th(0)=8.077987578600451;dx=-14848.347534341974
New Minimum: 8.077987578600451 > 6.010481729230534
END: th(3.628800000000003E-4)=6.010481729230534; dx=-13244.488183597874 evalInputDelta=2.067505849369917
Fitness changed from 8.077987578600451 to 6.010481729230534
Iteration 188 complete. Error: 6.010481729230534 Total: 0.0112; Orientation: 0.0006; Line Search: 0.0083
th(0)=6.010481729230534;dx=-12091.312958182982
New Minimum: 6.010481729230534 > 3.5331937143068273
WOLFE (weak): th(7.818012603187707E-4)=3.5331937143068273; dx=-11163.07345935291 evalInputDelta=2.477288014923707
New Minimum: 3.5331937143068273 > 2.422306122956277
END: th(0.0015636025206375414)=2.422306122956277; dx=-10866.79423373438 evalInputDelta=3.588175606274257
Fitness changed from 6.010481729230534 to 2.422306122956277
Iteration 189 complete. Error: 2.422306122956277 Total: 0.0189; Orientation: 0.0006; Line Search: 0.0158
th(0)=2.422306122956277;dx=-10171.080375698562
New Minimum: 2.422306122956277 > 0.9053015617188696
WOLFE (weak): th(0.0033686795118828138)=0.9053015617188696; dx=-10144.122621417888 evalInputDelta=1.5170045612374075
New Minimum: 0.9053015617188696 > 0.2646876196139819
WOLFE (weak): th(0.0067373590237656275)=0.2646876196139819; dx=-10244.438598162798 evalInputDelta=2.157618503342295
New Minimum: 0.2646876196139819 > 0.0
WOLFE (weak): th(0.020212077071296883)=0.0; dx=-10871.111821229015 evalInputDelta=2.422306122956277
WOLFE (weak): th(0.08084830828518753)=0.0; dx=-13761.464473080461 evalInputDelta=2.422306122956277
WOLFE (weak): th(0.40424154142593766)=0.0; dx=-29176.67861628817 evalInputDelta=2.422306122956277
WOLFE (weak): th(2.425449248555626)=0.0; dx=-125521.76701133637 evalInputDelta=2.422306122956277
WOLFE (weak): th(16.97814473988938)=0.0; dx=-819206.4034556833 evalInputDelta=2.422306122956277
WOLFE (weak): th(135.82515791911504)=0.0; dx=-6484297.601084516 evalInputDelta=2.422306122956277
Armijo: th(1222.4264212720354)=0.0; dx=-5.827941712226243E7 evalInputDelta=2.422306122956277
Armijo: th(679.1257895955753)=0.0; dx=-3.2381857361673474E7 evalInputDelta=2.422306122956277
Armijo: th(407.4754737573452)=0.0; dx=-1.9433077481379E7 evalInputDelta=2.422306122956277
Armijo: th(271.6503158382301)=0.0; dx=-1.2958687541231755E7 evalInputDelta=2.422306122956277
WOLFE (weak): th(203.73773687867256)=0.0; dx=-9721492.571158137 evalInputDelta=2.422306122956277
WOLFE (weak): th(237.6940263584513)=0.0; dx=-1.1340090056194946E7 evalInputDelta=2.422306122956277
Armijo: th(254.6721710983407)=0.0; dx=-1.214938879871335E7 evalInputDelta=2.422306122956277
Armijo: th(246.183098728396)=0.0; dx=-1.174473942745415E7 evalInputDelta=2.422306122956277
Armijo: th(241.93856254342364)=0.0; dx=-1.1542414741824547E7 evalInputDelta=2.422306122956277
Armijo: th(239.81629445093748)=0.0; dx=-1.1441252399009746E7 evalInputDelta=2.422306122956277
mu ~= nu (237.6940263584513): th(0.020212077071296883)=0.0
Fitness changed from 2.422306122956277 to 0.0
Iteration 190 complete. Error: 0.0 Total: 0.5247; Orientation: 0.0007; Line Search: 0.5217
th(0)=0.0;dx=-11856.340556148763
Armijo: th(514.3824000000004)=0.0; dx=-3.434052926097709E7 evalInputDelta=0.0
Armijo: th(257.1912000000002)=0.0; dx=-1.7176192800766617E7 evalInputDelta=0.0
Armijo: th(85.73040000000007)=0.0; dx=-5733301.827292973 evalInputDelta=0.0
Armijo: th(21.43260000000002)=0.0; dx=-1442217.7122403546 evalInputDelta=0.0
Armijo: th(4.286520000000004)=0.0; dx=-297928.61489299 evalInputDelta=0.0
Armijo: th(0.7144200000000006)=0.0; dx=-59535.0529456223 evalInputDelta=0.0
Armijo: th(0.10206000000000008)=0.0; dx=-18667.58518321641 evalInputDelta=0.0
Armijo: th(0.01275750000000001)=0.0; dx=-12707.746134532219 evalInputDelta=0.0
Armijo: th(0.0014175000000000012)=0.0; dx=-11950.941175969147 evalInputDelta=0.0
Armijo: th(1.417500000000001E-4)=0.0; dx=-11865.800618130803 evalInputDelta=0.0
Armijo: th(1.2886363636363647E-5)=0.0; dx=-11857.200561783495 evalInputDelta=0.0
Armijo: th(1.0738636363636372E-6)=0.0; dx=-11856.412223284991 evalInputDelta=0.0
Armijo: th(8.260489510489516E-8)=0.0; dx=-11856.346069005396 evalInputDelta=0.0
Armijo: th(5.900349650349655E-9)=0.0; dx=-11856.340949924239 evalInputDelta=0.0
Armijo: th(3.9335664335664366E-10)=0.0; dx=-11856.340582400462 evalInputDelta=0.0
Armijo: th(2.458479020979023E-11)=0.0; dx=-11856.340557789496 evalInputDelta=0.0
Armijo: th(1.4461641299876605E-12)=0.0; dx=-11856.340556245277 evalInputDelta=0.0
Armijo: th(8.034245166598114E-14)=0.0; dx=-11856.340556154124 evalInputDelta=0.0
Armijo: th(4.228550087683218E-15)=0.0; dx=-11856.340556149045 evalInputDelta=0.0
MIN ALPHA (2.114275043841609E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.2149; Orientation: 0.0205; Line Search: 0.1922
Iteration 191 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 191
Final threshold in iteration 191: 0.0 (> 0.0) after 15.492s (< 30.000s)
Returns
0.0
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.02 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1629690561559
Reset training subject: 1629693130450
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=835.3137106258573}, derivative=-4.488978031875305E27}
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=1270.7778942559657}, derivative=7.295950040691443E29}, evalInputDelta = 435.46418363010844
1270.7778942559657 <= 835.3137106258573
Converged to right
Fitness changed from 835.3137106258573 to 835.3137106258573
Static Iteration Total: 0.0149; Orientation: 0.0004; Line Search: 0.0078
Iteration 1 failed. Error: 835.3137106258573
Previous Error: 0.0 -> 835.3137106258573
Optimization terminated 1
Final threshold in iteration 1: 835.3137106258573 (> 0.0) after 0.015s (< 30.000s)
Returns
835.3137106258573
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
[1.492, -0.748, -1.16, 0.752, -0.408, 0.48, -0.612, 0.1, -0.608, -0.632, -0.452, 1.62, -1.256, 1.792, -1.384, 1.316, -0.124, 0.464, -0.06, 0.404, 0.284, -1.128, -0.964, -0.804, -0.08, -0.628, 0.972, -0.556, -1.656, -1.372, 1.628, 0.072, -1.848, 0.968, 1.66, 0.328, 1.064, 1.052, -1.096, -0.676, 1.556, 1.632, -0.548, 1.288, -1.312, 1.048, 0.244, -1.564, 0.52, 1.24, -1.9, 1.244, -1.656, 1.132, 0.688, -1.88, -0.5, -0.52, -1.092, 1.596, 1.752, -0.376, 0.264, 1.948, 1.656, -0.02, 0.028, -0.34, -0.396, -0.472, 0.048, -0.148, 0.812, -0.336, -1.696, -0.252, -0.128, 1.352, 0.392, 0.9, 1.98, -0.056, 1.352, 1.972, 0.992, -0.832, 0.676, -1.34, 0.012, -1.724, -1.472, -1.052, 1.208, -1.96, -1.644, 0.764, 1.26, -1.108, -1.732, 1.612, -1.764, 0.312, -0.432, 1.088, 0.56, -1.16, 1.7, -0.296, 1.948, -0.664, 1.764, 1.256, 0.908, 1.652, -0.86, 1.884, -1.476, 1.24, -0.124, 0.572, 1.704, 1.052, 0.972, 1.64, -1.92, 0.412, -1.824, -0.664, -1.524, -0.684, -1.156, -1.316, -0.932, -1.5, 1.288, 1.144, -1.688, 0.948, 1.144, -1.476, -0.616, 1.156, 0.076, -0.816, -0.928, 0.856, 1.872, 1.572, 1.356, 1.984, -1.764, -1.524, 1.336, 1.264, -0.58, 0.488, 0.284, 1.236, -1.524, 1.656, 1.348, 0.868, -1.188, -0.176, -1.176, 1.368, 0.784, -0.808, 1.92, -1.58, 0.448, -0.968, 0.332, 1.008, -0.18, 1.776, 0.296, 0.52, -1.116, -0.464, -1.54, 0.76, 0.52, -0.488, 1.544, -1.944, 0.636, -1.444, -1.432, -0.384, 1.916, 0.284, 0.532, -0.852, -0.316, -1.564, -1.72, -1.564, 0.3, -1.756, 0.496, -0.684, -0.344, 1.196, 1.852, 0.84, -0.816, 1.304, -0.528, -0.4, -0.784, 1.552, -0.504, 1.836, -1.244, 0.08, 0.552, 1.172, 0.78, 0.08, -1.312, -1.484, 0.376, -1.6, 1.288, 0.6, 1.516, 0.132, 0.7, 1.476, 1.048, -1.716, -0.856, 0.688, -0.312, 1.22, 0.856, -0.924, 1.424, -1.596, -0.636, 0.184, -1.536, 0.492, -0.416, -1.184, 1.432, 1.524, 1.512, -0.492, 0.66, 0.996, -0.768, 0.552, 0.184, -1.5, -1.812, 1.108, 0.08, 1.032, -0.876, -1.796, 1.86, 1.956, -1.636, -1.168, 0.124, -1.856, -0.356, -1.028, -1.76, -1.552, 0.332, 0.036, -1.148, -1.228, 0.48, -0.732, 1.536, -0.688, -1.616, 0.076, -0.312, -1.1, 0.08, 0.696, 1.42, 0.04, 1.844, 1.352, -0.096, 1.448, 1.18, -1.42, 0.428, 0.184, 0.78, -1.176, 0.692, 0.124, -0.084, -1.356, -1.516, -0.032, -0.28, -0.892, 0.512, 0.632, 1.052, -0.76, 1.384, 1.364, -0.396, -1.352, 0.672, -1.264, 0.96, -0.852, -1.192, 0.344, -0.152, -1.964, 0.092, 1.64, -1.976, -1.836, -0.968, 1.22, 0.996, -0.584, 1.82, -1.76, 0.248, 0.932, 0.848, 0.62, 0.656, 0.016, -2.0, -0.212, 1.096, -0.212, 1.556, 1.616, 1.012, -0.452, -1.624, -1.808, 0.86, 0.356, 1.912, 0.652, -1.736, 0.572, -1.832, -1.86, -0.736, -1.736, -0.384, 0.76, -0.548, -0.856, 0.08, -1.248, 0.568, -1.808, 0.148, 1.324, 1.72, 1.88, -1.096, -1.18, -0.384, 0.644, -0.828, -1.572, 0.832, -1.412, -0.756, 1.192, 0.56, 1.652, 0.164, -0.468, -1.664, 0.004, 1.668, -0.652, -0.068, 1.512, 0.788, 0.976, -0.26, 1.96, 0.8, -1.256, -1.104, -1.552, 0.264, -0.028, 0.844, 1.168, -0.848, 1.972, -0.012, 0.244, 0.408, -1.46, 0.584, -1.036, 0.612, -0.968, -0.2, 1.732, 0.892, 1.66, 0.476, 0.66, 1.724, 1.524, 1.836, -1.104, 0.82, -0.712, -1.976, -1.632, -1.176, 0.048, -0.448, -0.032, -0.368, -1.236]
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 1.364, -0.476, -0.276 ], [ 1.964, 1.968, -1.444 ], [ -1.8, 1.644, 0.772 ] ],
[ [ 1.804, -1.544, 0.76 ], [ 0.708, 1.424, -1.416 ], [ 1.104, 1.72, 0.268 ] ],
[ [ -0.436, -0.34, 0.612 ], [ 0.508, 1.128, 0.384 ], [ -1.088, -1.516, -1.044 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 6.648128000000001, 12.715807999999999, -4.785008000000001, -6.878368000000001 ], [ -13.873183999999998, -1.8369280000000008, -10.103680000000002, 12.700639999999998 ] ],
[ [ -1.7956000000000016, -6.42176, 10.745056000000002, -10.390751999999999 ], [ 4.448256000000003, 12.082832000000002, -6.615520000000001, 4.587983999999998 ] ]
]
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 30.75 seconds (0.185 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 1629711206252
Reset training subject: 1629713621572
Adding measurement 5fdf978c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 835.3137106258573 < 835.3137106258573. Total: 1
th(0)=835.3137106258573;dx=-4.488978031875305E27
Non-optimal measurement 1350.8790308109303 < 835.3137106258573. Total: 1
Armijo: th(2.154434690031884)=1350.8790308109303; dx=1.5813838489068568E40 evalInputDelta=-515.565320185073
Non-optimal measurement 1350.8790308109303 < 835.3137106258573. Total: 1
Armijo: th(1.077217345015942)=1350.8790308109303; dx=7.906919244532059E39 evalInputDelta=-515.565320185073
Non-optimal measurement 1348.8996770870865 < 835.3137106258573. Total: 1
Armijo: th(0.3590724483386473)=1348.8996770870865; dx=2.6356397481743845E39 evalInputDelta=-513.5859664612292
Non-optimal measurement 1310.6146647444614 < 835.3137106258573. Total: 1
Armijo: th(0.08976811208466183)=1310.6146647444614; dx=6.589099370402572E38 evalInputDelta=-475.30095411860407
Non-optimal measurement 1270.7778942559657 < 835.3137106258573. Total: 1
Armijo: th(0.017953622416932366)=1270.7778942559657; dx=1.3178198740450784E38 evalInputDelta=-435.46418363010844
Non-optimal measurement 1270.7778942559657 < 835.3137106258573. Total: 1
Armijo: th(0.002992270402822061)=1270.7778942559657; dx=2.196366456373601E37 evalInputDelta=-435.46418363010844
Non-optimal measurement 1270.7778942559657 < 835.3137106258573. Total: 1
Armijo: th(4.2746720040315154E-4)=1270.7778942559657; dx=3.1376663624608375E36 evalInputDelta=-435.46418363010844
Non-optimal measurement 1270.7778942559657 < 835.3137106258573. Total: 1
Armijo: th(5.343340005039394E-5)=1270.7778942559657; dx=3.922082914415419E35 evalInputDelta=-435.46418363010844
Non-optimal measurement 1270.7778942559657 < 835.3137106258573. Total: 1
Armijo: th(5.9370444500437714E-6)=1270.7778942559657; dx=4.35786951216314E34 evalInputDelta=-435.46418363010844
Non-optimal measurement 1270.7778942559657 < 835.3137106258573. Total: 1
Armijo: th(5.937044450043771E-7)=1270.7778942559657; dx=4.3578655356414757E33 evalInputDelta=-435.46418363010844
Non-optimal measurement 1270.7778942559657 < 835.3137106258573. Total: 1
Armijo: th(5.397313136403428E-8)=1270.7778942559657; dx=3.96165577460675E32 evalInputDelta=-435.46418363010844
Non-optimal measurement 1270.7778942559657 < 835.3137106258573. Total: 1
Armijo: th(4.4977609470028565E-9)=1270.7778942559657; dx=3.3009747960768196E31 evalInputDelta=-435.46418363010844
Non-optimal measurement 1270.7778942559657 < 835.3137106258573. Total: 1
Armijo: th(3.4598161130791205E-10)=1270.7778942559657; dx=2.535132897838961E30 evalInputDelta=-435.46418363010844
Non-optimal measurement 1270.7778942559657 < 835.3137106258573. Total: 1
Armijo: th(2.4712972236279432E-11)=1270.7778942559657; dx=1.769781608265801E29 evalInputDelta=-435.46418363010844
Non-optimal measurement 1149.194851187752 < 835.3137106258573. Total: 1
Armijo: th(1.6475314824186289E-12)=1149.194851187752; dx=7.22147485995912E27 evalInputDelta=-313.88114056189465
Adding measurement 290060b9 to history. Total: 1
New Minimum: 835.3137106258573 > 370.69866245570165
Armijo: th(1.029707176511643E-13)=370.69866245570165; dx=1.0713305471265837E26 evalInputDelta=464.61504817015566
Non-optimal measurement 800.6726843984711 < 370.69866245570165. Total: 2
Armijo: th(6.057101038303783E-15)=800.6726843984711; dx=-4.296780376183776E27 evalInputDelta=34.64102622738619
Non-optimal measurement 370.69866245570165 < 370.69866245570165. Total: 2
MIN ALPHA (3.3650561323909904E-16): th(1.029707176511643E-13)=370.69866245570165
Fitness changed from 835.3137106258573 to 370.69866245570165
Iteration 1 complete. Error: 370.69866245570165 Total: 0.1212; Orientation: 0.0020; Line Search: 0.1121
Non-optimal measurement 370.69866245570165 < 370.69866245570165. Total: 2
LBFGS Accumulation History: 2 points
Non-optimal measurement 370.69866245570165 < 370.69866245570165. Total: 2
th(0)=370.69866245570165;dx=-2.762605319301872E27
Adding measurement 734317f2 to history. Total: 2
New Minimum: 370.69866245570165 > 284.3863516257287
Armijo: th(2.154434690031884E-15)=284.3863516257287; dx=-2.0337421587664844E27 evalInputDelta=86.31231082997294
Non-optimal measurement 287.2851248223471 < 284.3863516257287. Total: 3
Armijo: th(1.077217345015942E-15)=287.2851248223471; dx=-2.03443774276187E27 evalInputDelta=83.41353763335457
Non-optimal measurement 284.3863516257287 < 284.3863516257287. Total: 3
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=284.3863516257287
Fitness changed from 370.69866245570165 to 284.3863516257287
Iteration 2 complete. Error: 284.3863516257287 Total: 0.0296; Orientation: 0.0017; Line Search: 0.0253
Non-optimal measurement 284.3863516257287 < 284.3863516257287. Total: 3
LBFGS Accumulation History: 3 points
Non-optimal measurement 284.3863516257287 < 284.3863516257287. Total: 3
th(0)=284.3863516257287;dx=-2.0291556314521887E27
Adding measurement 45565d9e to history. Total: 3
New Minimum: 284.3863516257287 > 131.94387813495402
Armijo: th(2.154434690031884E-15)=131.94387813495402; dx=-2.814687053053509E26 evalInputDelta=152.4424734907747
Non-optimal measurement 248.66919249948137 < 131.94387813495402. Total: 4
Armijo: th(1.077217345015942E-15)=248.66919249948137; dx=-1.8076061226351353E27 evalInputDelta=35.71715912624734
Non-optimal measurement 131.94387813495402 < 131.94387813495402. Total: 4
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=131.94387813495402
Fitness changed from 284.3863516257287 to 131.94387813495402
Iteration 3 complete. Error: 131.94387813495402 Total: 0.0400; Orientation: 0.0021; Line Search: 0.0354
Non-optimal measurement 131.94387813495402 < 131.94387813495402. Total: 4
Rejected: LBFGS Orientation magnitude: 2.368e+00, gradient 1.594e+13, dot -0.741; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00, 35dd7090-cacf-48ea-bf76-55368a4c9457 = 1.000/1.000e+00, 18d20299-6c18-42cb-93ab-937b010998be = 1.000/1.000e+00, 2a8d313c-1b1e-48af-ab7f-e12edde24d68 = 1.000/1.000e+00, f53b12f7-161a-4daf-befc-76f7c0185179 = 1.000/1.000e+00, 7f1d1e63-7c3a-4a14-a5d4-26bf1e91604b = 1.000/1.000e+00]
Orientation rejected. Popping history element from 131.94387813495402, 284.3863516257287, 370.69866245570165, 835.3137106258573
LBFGS Accumulation History: 3 points
Removed measurement 45565d9e to history. Total: 3
Adding measurement 75d66120 to history. Total: 3
th(0)=131.94387813495402;dx=-2.542137360291018E26
Adding measurement d7a15d7 to history. Total: 4
New Minimum: 131.94387813495402 > 131.6706695865642
Armijo: th(2.154434690031884E-15)=131.6706695865642; dx=-2.541695054363165E26 evalInputDelta=0.27320854838981745
Non-optimal measurement 131.8056998974589 < 131.6706695865642. Total: 5
Armijo: th(1.077217345015942E-15)=131.8056998974589; dx=-2.541916207327091E26 evalInputDelta=0.1381782374951115
Non-optimal measurement 131.6706695865642 < 131.6706695865642. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=131.6706695865642
Fitness changed from 131.94387813495402 to 131.6706695865642
Iteration 4 complete. Error: 131.6706695865642 Total: 0.4469; Orientation: 0.4254; Line Search: 0.0190
Non-optimal measurement 131.6706695865642 < 131.6706695865642. Total: 5
Rejected: LBFGS Orientation magnitude: 3.692e+02, gradient 1.594e+13, dot -0.986; [7f1d1e63-7c3a-4a14-a5d4-26bf1e91604b = 1.000/1.000e+00, f53b12f7-161a-4daf-befc-76f7c0185179 = 1.000/1.000e+00, 52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00, 18d20299-6c18-42cb-93ab-937b010998be = 1.000/1.000e+00, 2a8d313c-1b1e-48af-ab7f-e12edde24d68 = 1.000/1.000e+00, 35dd7090-cacf-48ea-bf76-55368a4c9457 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 131.6706695865642, 131.94387813495402, 284.3863516257287, 370.69866245570165, 835.3137106258573
Rejected: LBFGS Orientation magnitude: 1.945e+02, gradient 1.594e+13, dot -0.998; [35dd7090-cacf-48ea-bf76-55368a4c9457 = 1.000/1.000e+00, 18d20299-6c18-42cb-93ab-937b010998be = 1.000/1.000e+00, 2a8d313c-1b1e-48af-ab7f-e12edde24d68 = 1.000/1.000e+00, 52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00, 7f1d1e63-7c3a-4a14
...skipping 65495 bytes...
d1eb7 = 1.000/1.000e+00, 35dd7090-cacf-48ea-bf76-55368a4c9457 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 66.8574894150003, 66.98422358277973, 284.3863516257287, 370.69866245570165, 835.3137106258573
Rejected: LBFGS Orientation magnitude: 1.207e+02, gradient 8.309e+12, dot -0.989; [18d20299-6c18-42cb-93ab-937b010998be = 1.000/1.000e+00, f53b12f7-161a-4daf-befc-76f7c0185179 = 1.000/1.000e+00, 7f1d1e63-7c3a-4a14-a5d4-26bf1e91604b = 1.000/1.000e+00, 35dd7090-cacf-48ea-bf76-55368a4c9457 = 1.000/1.000e+00, 2a8d313c-1b1e-48af-ab7f-e12edde24d68 = 1.000/1.000e+00, 52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 66.8574894150003, 66.98422358277973, 284.3863516257287, 370.69866245570165
LBFGS Accumulation History: 3 points
Removed measurement 65934334 to history. Total: 4
Removed measurement 3e5747c3 to history. Total: 3
Adding measurement 210bc80e to history. Total: 3
th(0)=66.8574894150003;dx=-6.903597855422849E25
Adding measurement 2d33fd3a to history. Total: 4
New Minimum: 66.8574894150003 > 66.73344807942865
Armijo: th(2.154434690031884E-15)=66.73344807942865; dx=-6.903101473212467E25 evalInputDelta=0.12404133557164698
Non-optimal measurement 66.79513085285198 < 66.73344807942865. Total: 5
Armijo: th(1.077217345015942E-15)=66.79513085285198; dx=-6.903349664317657E25 evalInputDelta=0.0623585621483187
Non-optimal measurement 66.73344807942865 < 66.73344807942865. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=66.73344807942865
Fitness changed from 66.8574894150003 to 66.73344807942865
Iteration 36 complete. Error: 66.73344807942865 Total: 1.0360; Orientation: 1.0136; Line Search: 0.0196
Non-optimal measurement 66.73344807942865 < 66.73344807942865. Total: 5
Rejected: LBFGS Orientation magnitude: 2.695e+02, gradient 8.308e+12, dot -0.989; [35dd7090-cacf-48ea-bf76-55368a4c9457 = 1.000/1.000e+00, 18d20299-6c18-42cb-93ab-937b010998be = 1.000/1.000e+00, 2a8d313c-1b1e-48af-ab7f-e12edde24d68 = 1.000/1.000e+00, f53b12f7-161a-4daf-befc-76f7c0185179 = 1.000/1.000e+00, 52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00, 7f1d1e63-7c3a-4a14-a5d4-26bf1e91604b = 1.000/1.000e+00]
Orientation rejected. Popping history element from 66.73344807942865, 66.8574894150003, 284.3863516257287, 370.69866245570165, 835.3137106258573
Rejected: LBFGS Orientation magnitude: 1.283e+02, gradient 8.308e+12, dot -0.989; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00, 18d20299-6c18-42cb-93ab-937b010998be = 1.000/1.000e+00, 2a8d313c-1b1e-48af-ab7f-e12edde24d68 = 1.000/1.000e+00, f53b12f7-161a-4daf-befc-76f7c0185179 = 1.000/1.000e+00, 7f1d1e63-7c3a-4a14-a5d4-26bf1e91604b = 1.000/1.000e+00, 35dd7090-cacf-48ea-bf76-55368a4c9457 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 66.73344807942865, 66.8574894150003, 284.3863516257287, 370.69866245570165
LBFGS Accumulation History: 3 points
Removed measurement 2d33fd3a to history. Total: 4
Removed measurement 210bc80e to history. Total: 3
Adding measurement 7b28365 to history. Total: 3
th(0)=66.73344807942865;dx=-6.90262466074353E25
Adding measurement 3501156b to history. Total: 4
New Minimum: 66.73344807942865 > 66.61198867366512
Armijo: th(2.154434690031884E-15)=66.61198867366512; dx=-6.902167417100714E25 evalInputDelta=0.12145940576353098
Non-optimal measurement 66.67239386444245 < 66.61198867366512. Total: 5
Armijo: th(1.077217345015942E-15)=66.67239386444245; dx=-6.902396038922122E25 evalInputDelta=0.061054214986199895
Non-optimal measurement 66.61198867366512 < 66.61198867366512. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=66.61198867366512
Fitness changed from 66.73344807942865 to 66.61198867366512
Iteration 37 complete. Error: 66.61198867366512 Total: 0.6249; Orientation: 0.5736; Line Search: 0.0487
Non-optimal measurement 66.61198867366512 < 66.61198867366512. Total: 5
Rejected: LBFGS Orientation magnitude: 3.002e+02, gradient 8.308e+12, dot -0.989; [18d20299-6c18-42cb-93ab-937b010998be = 1.000/1.000e+00, f53b12f7-161a-4daf-befc-76f7c0185179 = 1.000/1.000e+00, 2a8d313c-1b1e-48af-ab7f-e12edde24d68 = 1.000/1.000e+00, 7f1d1e63-7c3a-4a14-a5d4-26bf1e91604b = 1.000/1.000e+00, 52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00, 35dd7090-cacf-48ea-bf76-55368a4c9457 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 66.61198867366512, 66.73344807942865, 284.3863516257287, 370.69866245570165, 835.3137106258573
Rejected: LBFGS Orientation magnitude: 1.372e+02, gradient 8.308e+12, dot -0.989; [35dd7090-cacf-48ea-bf76-55368a4c9457 = 1.000/1.000e+00, 7f1d1e63-7c3a-4a14-a5d4-26bf1e91604b = 1.000/1.000e+00, f53b12f7-161a-4daf-befc-76f7c0185179 = 1.000/1.000e+00, 52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00, 18d20299-6c18-42cb-93ab-937b010998be = 1.000/1.000e+00, 2a8d313c-1b1e-48af-ab7f-e12edde24d68 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 66.61198867366512, 66.73344807942865, 284.3863516257287, 370.69866245570165
LBFGS Accumulation History: 3 points
Removed measurement 3501156b to history. Total: 4
Removed measurement 7b28365 to history. Total: 3
Adding measurement 523bb37b to history. Total: 3
th(0)=66.61198867366512;dx=-6.901729741436988E25
Adding measurement 5521691d to history. Total: 4
New Minimum: 66.61198867366512 > 66.49300745124722
Armijo: th(2.154434690031884E-15)=66.49300745124722; dx=-6.90131163297365E25 evalInputDelta=0.11898122241790077
Non-optimal measurement 66.55218607798608 < 66.49300745124722. Total: 5
Armijo: th(1.077217345015942E-15)=66.55218607798608; dx=-6.901520687205317E25 evalInputDelta=0.05980259567904511
Non-optimal measurement 66.49300745124722 < 66.49300745124722. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=66.49300745124722
Fitness changed from 66.61198867366512 to 66.49300745124722
Iteration 38 complete. Error: 66.49300745124722 Total: 0.8501; Orientation: 0.7180; Line Search: 0.1296
Non-optimal measurement 66.49300745124722 < 66.49300745124722. Total: 5
Rejected: LBFGS Orientation magnitude: 3.389e+02, gradient 8.307e+12, dot -0.989; [52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00, 7f1d1e63-7c3a-4a14-a5d4-26bf1e91604b = 1.000/1.000e+00, 18d20299-6c18-42cb-93ab-937b010998be = 1.000/1.000e+00, 2a8d313c-1b1e-48af-ab7f-e12edde24d68 = 1.000/1.000e+00, f53b12f7-161a-4daf-befc-76f7c0185179 = 1.000/1.000e+00, 35dd7090-cacf-48ea-bf76-55368a4c9457 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 66.49300745124722, 66.61198867366512, 284.3863516257287, 370.69866245570165, 835.3137106258573
Rejected: LBFGS Orientation magnitude: 1.480e+02, gradient 8.307e+12, dot -0.989; [f53b12f7-161a-4daf-befc-76f7c0185179 = 1.000/1.000e+00, 2a8d313c-1b1e-48af-ab7f-e12edde24d68 = 1.000/1.000e+00, 7f1d1e63-7c3a-4a14-a5d4-26bf1e91604b = 1.000/1.000e+00, 18d20299-6c18-42cb-93ab-937b010998be = 1.000/1.000e+00, 35dd7090-cacf-48ea-bf76-55368a4c9457 = 1.000/1.000e+00, 52b139cd-3939-4b1e-a0a3-171c618d1eb7 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 66.49300745124722, 66.61198867366512, 284.3863516257287, 370.69866245570165
LBFGS Accumulation History: 3 points
Removed measurement 5521691d to history. Total: 4
Removed measurement 523bb37b to history. Total: 3
Adding measurement 29d18263 to history. Total: 3
th(0)=66.49300745124722;dx=-6.900913091000279E25
Adding measurement 32b7ae6b to history. Total: 4
New Minimum: 66.49300745124722 > 66.37640719374181
Armijo: th(2.154434690031884E-15)=66.37640719374181; dx=-6.900534114874779E25 evalInputDelta=0.11660025750541081
Non-optimal measurement 66.43440708496489 < 66.37640719374181. Total: 5
Armijo: th(1.077217345015942E-15)=66.43440708496489; dx=-6.900723602937529E25 evalInputDelta=0.05860036628233445
Non-optimal measurement 66.37640719374181 < 66.37640719374181. Total: 5
MIN ALPHA (3.5907244833864734E-16): th(2.154434690031884E-15)=66.37640719374181
Fitness changed from 66.49300745124722 to 66.37640719374181
Iteration 39 complete. Error: 66.37640719374181 Total: 1.1635; Orientation: 0.8799; Line Search: 0.1604
Final threshold in iteration 39: 66.37640719374181 (> 0.0) after 30.754s (< 30.000s)
Returns
66.37640719374181
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
[1.5741390567567208, -0.7738303267693966, -1.2348667776656856, 0.7790925968548869, 0.14268604834110865, 0.8717871488429593, -1.1422640942577973, -0.22302034067453902, -0.4502386550895663, -0.30431181250009365, -0.6768133350932157, 1.5820366292859123, -0.680360555811895, 1.684916935691279, -1.3169188511466057, 1.531811253387929, -0.4310958839322013, 0.23210960511651518, -0.17366424348400203, 0.32624609475521815, 0.4756392316379772, -0.9325914272148751, -0.3248444542036303, -0.5986173546555585, 0.4740794386210156, -0.481027549270519, 1.0969809751196244, -0.2070498203360176, -1.1675508887369777, -1.4612555070475337, 2.0816619308225692, 0.3429569025927821, -1.6530769020213543, 0.43057200522706784, 1.937473209885937, 0.05662991759986192, 0.9597419631956323, 0.5671209958794095, -1.1530617507260494, -0.07882907890086271, 1.979532921584066, 2.0034689312493934, -0.13181487771397843, 1.6825852161429768, -1.7773910619946405, 1.0051392009259084, 0.39868050453351245, -1.3441282365671474, 0.11789856202891089, 0.9355049250815884, -1.766119195568984, 1.531710743817415, -1.6295508867661241, 0.903921021582797, 0.7518875470005001, -1.5874627250709095, -0.3269767175484556, -0.6478348483384105, -0.6052837493651757, 1.7047908075323044, 1.361347054458654, -0.05142460706833298, 0.057739321857733644, 2.0972981712102414, 1.8039894742681593, -0.19849793587835987, -0.0036326610855298915, 0.07784520646255375, 0.047596205279678894, -0.24820254983607487, 0.15918707974783383, -0.5251889041938045, 0.554610778604382, -0.09037136933505813, -1.9406766700311764, 0.13811832951840805, 0.10613431704057052, 1.2986499978293098, 0.5789536879677195, 0.7649434440544246, 1.6582158114972883, -0.0806051709565237, 1.406999793902298, 2.090973238389049, 0.9575527606607095, -0.9417101152049144, 0.6444474866562688, -1.2887633498912794, 0.15413104634763428, -1.8264733002180171, -1.4442106304491864, -0.9721055625423078, 1.0777373302314068, -1.8783687269444205, -1.5452898435747868, 0.6884476515344098, 1.2793946641655296, -1.1279736093648458, -1.6289477545832014, 1.7425521423687373, -1.8427365470605153, 0.3562893077217665, -0.536499608414738, 1.2124732177796385, 0.615868211700908, -1.1232369798647135, 1.588263576597979, -0.15357948105228045, 2.027792758594065, -0.7211277078455721, 1.6463299197701013, 1.49721345120175, 1.0278682517191355, 1.7378068054732003, -1.0426498417722712, 1.704027638351201, -1.427557975688129, 1.4261895778366136, -0.026914312728496863, 0.40677904042014223, 1.912834562647634, 0.8326829355992147, 0.7547589186414994, 1.879289106880098, -1.9551459086339824, 0.4694771821052551, -1.9678775611779311, -0.5777826840568862, -1.5537659778743786, -0.48746467385283204, -0.8605132119499899, -1.267102716626916, -0.7676435297932414, -1.2632753741203824, 1.2675785936991482, 1.144, -1.688, 0.948, 1.144, -1.476, -0.616, 1.156, 0.076, -0.816, -0.928, 0.856, 1.872, 1.572, 1.356, 1.984, -1.764, -1.524, 1.336, 1.264, -0.58, 0.488, 0.284, 1.236, -1.524, 1.656, 1.348, 0.868, -1.2140971054729053, -0.10410437784322184, -1.0146553664481286, 1.331160023256417, 0.6340859793345189, -0.8476347732668887, 1.978890963504916, -1.3894378523897566, 0.30607032590154154, -0.9198762805573637, 0.4475517943915472, 0.8212040504144624, -0.05753079045606558, 1.907915966715358, 0.17885779215610226, 0.5606274071892523, -1.1471943810608574, -0.314099560407184, -1.365980758880331, 0.6535599865913017, 0.5776117074211683, -0.6275088300414478, 1.7308965204591766, -1.8666043430114396, 0.6910899150105414, -1.5893248718257522, -1.2337999116641394, -0.3839999999997561, 1.9160000000005306, 0.28400000000012016, 0.5320000000000701, -0.851999999999036, -0.31599999999958006, -1.5640000000010863, -1.7199999999995288, -1.5640000000002987, 0.3000000000001968, -1.7560000000006484, 0.4959999999999427, -0.6839999999992089, -0.34399999999890624, 1.1960000000010396, 1.8520000000005603, 0.8400000000003879, -0.8160000000010743, 1.3040000000000576, -0.5279999999997145, -0.39999999999988584, -0.7840000000000873, 1.5520000000002891, -0.5039999999999787, 1.8360000000008394, -1.2440000000003015, 0.07999999999984182, 0.6176133037313191, 1.1838246250943898, 0.7112623293255774, 0.22127486453279635, -1.2410016216022504, -1.4414312530189206, 0.2608466931326684, -1.6487187950613127, 1.291367332651923, 0.6603640760434322, 1.5126405602737207, 0.06946938289881006, 0.8155303203749215, 1.4225833405803543, 1.0039690816577849, -1.5926270937132097, -0.8233432581412029, 0.6630788117026861, -0.38248539503029355, 1.2766758392189623, 0.849592281431318, -0.8730881186379209, 1.493044732199242, -1.576816243686359, -0.5573836783443177, 0.2853284041062088, -1.57924780106833, 0.492, -0.416, -1.184, 1.432, 1.524, 1.512, -0.492, 0.66, 0.996, -0.768, 0.552, 0.184, -1.5, -1.812, 1.108, 0.08, 1.032, -0.876, -1.796, 1.86, 1.956, -1.636, -1.168, 0.124, -1.856, -0.356, -1.028, -1.6875084016044646, -0.9970548629771366, 0.8692251578599446, -0.19992866235502202, -0.7646880490367397, -0.919659886483708, -0.07813968691321954, -0.08086971709849221, 1.0582851016222206, -0.41733804375138384, -1.7530564435710034, -0.35470401285677594, 0.5233518572446758, -0.22223770465691436, 0.48056569438921365, 1.0186426372623367, 1.2939749183527731, -0.25673003438120984, 2.193378714588818, 1.1131567788691608, 0.21836299424107983, 1.1189228699669787, 1.9500302521926993, -1.0782506142296515, 1.1283499696637291, -0.38169988293509816, 1.1798310043744784, -1.0459126304118005, 0.6747390144768424, 0.06215485247908028, 0.009950007132881985, -0.935038801950045, -1.2390087280328825, -0.5109395470293528, -0.4908325028928346, -0.8210656269724619, 0.8011696569141654, 0.5543121349839945, 0.9784050413815114, -0.3722833818237673, 1.3437841850882093, 1.380119176458448, -0.1210420314471292, -1.541850118951816, 0.5292222978982284, -1.4515048205419396, 0.9924740771221073, -0.8242007374933025, -0.9659633069945567, 0.8467927440772783, -0.06072981938504929, -1.598022669947559, 0.2501326408474607, 1.651691419656625, -1.9064581199371553, -1.385811236733063, -0.5351135396068257, 1.1229559732472494, 1.1312278260332764, -0.4166038024567504, 1.4815730762573238, -1.165464235404673, -0.19623687592778752, 1.1240157064468463, 0.8284691216261224, 0.19242671551920884, 1.3004870260211023, 0.7046743534049941, -1.8010576855565454, 0.07972211713987185, 1.0861739875756684, -0.338931021292208, 1.8871448937908852, 1.44473876403992, 1.2345675287636573, -0.7868110734410105, -1.080378903297608, -1.5373586156910104, 1.3442687315281872, -0.08133053933570876, 2.258835674357823, 0.7673768841048765, -1.6698615579027665, 0.5171417732915445, -1.822166985282207, -1.1260721395625586, -0.22586176948031766, -2.454098955017243, -0.6734762721617269, 0.9004317270822622, -0.20030194369274065, -1.2927762073058005, 0.08272898088141399, -0.44300454973708364, 0.6270042364543456, -1.545386151800686, 0.4041306361336562, 1.0046035963140822, 1.2898547587976918, 1.8251171208628214, -1.2153697752116661, -0.8596251687631127, -0.2945352535905386, 1.3454821743393472, -0.5645279607316934, -0.8216456289790895, 0.8996454068193852, -1.2390030319953615, -0.7559999999999139, 1.1920000000003408, 0.560000000000199, 1.6519999999999113, 0.1640000000005471, -0.4679999999999042, -1.6640000000003938, 0.0040000000004592915, 1.6679999999996993, -0.6519999999997609, -0.06799999999963657, 1.5119999999996983, 0.7879999999998574, 0.9760000000004996, -0.25999999999947665, 1.9600000000003326, 0.8000000000002587, -1.2560000000004568, -1.1040000000003747, -1.5519999999995608, 0.26399999999953017, -0.027999999999403564, 0.8440000000002336, 1.1679999999998991, -0.8479999999998289, 1.9719999999999016, -0.012000000000449802, 0.2398698614106628, 0.4015243015869713, -1.4569239108693899, 0.5723308022772523, -1.0283483750734443, 0.617398488326039, -0.9790659479895386, -0.20534275091568668, 1.737263354353247, 0.8945754539876949, 1.6447774866387626, 0.48470002089174136, 0.6679731835954319, 1.734790758191838, 1.5307738715381987, 1.8352277294345882, -1.1091605470926957, 0.8131635277787396, -0.7001425518384562, -1.9808479867721125, -1.6250346539422447, -1.1804512402120517, 0.06035712850040392, -0.44935947941841897, -0.021005196058387566, -0.3742013201220118, -1.224684244203223]
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.8999905121389499, -0.17791527641161833, 0.4158582947966988 ], [ 2.428891981262808, 1.7948746917806004, -1.2436123318954688 ], [ -1.4135742674983187, 1.5213407689636718, 1.1513340201977205 ] ],
[ [ 0.9797491811904856, -1.3601313576347949, 1.086734024221172 ], [ 0.6039809170585292, 1.7900971615878474, -1.0462473458528634 ], [ 1.2381205425793276, 2.3440849764199028, -0.0701200732147822 ] ],
[ [ -0.814448295754636, -0.5561744283216263, 0.2553798765757507 ], [ -0.21761444347302164, 1.316622362190862, -0.504566885976284 ], [ -0.7546471329888048, -1.7885501946900417, -0.5846948999953437 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 8.33571093905496, 9.994784899960454, -2.54051162297183, 6.325900427919696 ], [ -8.197910832313237, 1.8846371766767174, 4.059016108544906, 10.009999034285807 ] ],
[ [ 21.514912694150002, -2.6769167921347714, 8.201196311915918, -5.586989727406366 ], [ -0.8388390065078568, 13.070685931845494, -4.711522783695616, 4.131800039180135 ] ]
]
TrainingTester.java:432 executed in 0.01 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, 0.38422902696129024], [190.0, 2.5690210190234777]; valueStats=DoubleSummaryStatistics{count=228, sum=11631.673469, min=2.422306, average=51.016112, max=370.698662}
Plotting 190 points for GD
Plotting 39 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, 0.38422902696129024], [30.632, 2.5690210190234777]; valueStats=DoubleSummaryStatistics{count=228, sum=11631.673469, min=2.422306, average=51.016112, max=370.698662}
Plotting 190 points for GD
Only 0 points for LBFGS
Returns
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "NonConverged", "value": 154.48635002569895 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "NonConverged", "value": 154.48635002569895 } }, "model":{ "LBFGS": { "type": "NonConverged", "value": 47.35421161154605 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "NonConverged", "value": 22.194607632370303 } }, "complete":{ "LBFGS": { "type": "NonConverged", "value": 66.37640719374181 }, "CjGD": { "type": "NonConverged", "value": NaN }, "GD": { "type": "Converged", "value": 0.0 } }}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "NonConverged", "value": 154.48635002569895 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "NonConverged", "value": 154.48635002569895 } }, "model":{ "LBFGS": { "type": "NonConverged", "value": 47.35421161154605 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "NonConverged", "value": 22.194607632370303 } }, "complete":{ "LBFGS": { "type": "NonConverged", "value": 66.37640719374181 }, "CjGD": { "type": "NonConverged", "value": NaN }, "GD": { "type": "Converged", "value": 0.0 } }} | OK |
{
"result": "OK",
"performance": {
"execution_time": "93.727",
"gc_time": "0.913"
},
"created_on": 1586736155155,
"file_name": "trainingTest",
"report": {
"simpleName": "Image",
"canonicalName": "com.simiacryptus.mindseye.layers.java.FullyConnectedReferenceLayerTest.Image",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/FullyConnectedReferenceLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "NonConverged",
"value": 154.48635002569895
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "NonConverged",
"value": 154.48635002569895
}
},
"model": {
"LBFGS": {
"type": "NonConverged",
"value": 47.35421161154605
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "NonConverged",
"value": 22.194607632370303
}
},
"complete": {
"LBFGS": {
"type": "NonConverged",
"value": 66.37640719374181
},
"CjGD": {
"type": "NonConverged",
"value": "NaN"
},
"GD": {
"type": "Converged",
"value": 0.0
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/FullyConnectedReferenceLayer/Image/trainingTest/202004130235",
"id": "c81aef4d-9d25-4075-9dfc-02b6d8e61dc6",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "FullyConnectedReferenceLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.java.FullyConnectedReferenceLayer",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/FullyConnectedReferenceLayer.java",
"javaDoc": ""
}
}