Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
Using Seed 2236303882309284864
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[ -0.128, 0.08, 0.7 ]
[ -0.128, 0.08, 0.7 ]
[ 0.7, 0.08, -0.128 ]
[ -0.128, 0.08, 0.7 ]
[ 0.08, -0.128, 0.7 ]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 0.07 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 2457240110845
BACKPROP_AGG_SIZE = 3
THREADS = 64
SINGLE_THREADED = false
Initialized CoreSettings = {
"backpropAggregationSize" : 3,
"jvmThreads" : 64,
"singleThreaded" : false
}
Reset training subject: 2457269229461
Zero gradient: 0.0
Constructing line search parameters: GD
th(0)=42.220200265138814;dx=0.0 (ERROR: Starting derivative negative)
Fitness changed from 42.220200265138814 to 42.220200265138814
Static Iteration Total: 0.0582; Orientation: 0.0036; Line Search: 0.0164
Iteration 1 failed. Error: 42.220200265138814
Previous Error: 0.0 -> 42.220200265138814
Optimization terminated 1
Final threshold in iteration 1: 42.220200265138814 (> 0.0) after 0.059s (< 30.000s)
Returns
42.220200265138814
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[ 0.7, 0.08, -0.128 ]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[ 0.7, 0.08, -0.128 ]
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.21 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 2457311135840
Reset training subject: 2457315056332
Constructing line search parameters: GD
New Minimum: 32.40270174223916 > 31.51850906652944
F(0.0) = LineSearchPoint{point=PointSample{avg=31.51850906652944}, derivative=-6.272E23}
New Minimum: 31.51850906652944 > 23.268700344203925
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=23.268700344203925}, derivative=-6.27200000000168E11}, evalInputDelta = -8.249808722325515
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=23.78182315395745}, derivative=-3.136000000001681E11}, evalInputDelta = -7.736685912571989
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=23.78182309326046}, derivative=-3.136000000124181E11}, evalInputDelta = -7.73668597326898
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=23.78182314858145}, derivative=-3.136000000001681E11}, evalInputDelta = -7.736685917947991
New Minimum: 23.268700344203925 > 23.268700305803947
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=23.268700305803947}, derivative=-6.272000000001681E11}, evalInputDelta = -8.249808760725493
New Minimum: 23.268700305803947 > 23.268700075309056
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=23.268700075309056}, derivative=-6.27200000000168E11}, evalInputDelta = -8.249808991220384
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=23.781677188793417}, derivative=-3.136000000124118E11}, evalInputDelta = -7.736831877736023
New Minimum: 23.268700075309056 > 8.307431128070808
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=8.307431128070808}, derivative=-9.408000000123739E11}, evalInputDelta = -23.21107793845863
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=8.308359225010296}, derivative=-9.408000000001675E11}, evalInputDelta = -23.210149841519144
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=8.77541141567263}, derivative=-6.272000000105767E11}, evalInputDelta = -22.74309765085681
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=8.592055104837081}, derivative=-6.272000000056272E11}, evalInputDelta = -22.92645396169236
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=16.02313843391505}, derivative=-6.272000000000885E11}, evalInputDelta = -15.495370632614389
New Minimum: 8.307431128070808 > 8.24980872230952
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=8.24980872230952}, derivative=-9.408000000000336E11}, evalInputDelta = -23.26870034421992
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=15.473371824919985}, derivative=-9.408000000000336E11}, evalInputDelta = -16.045137241609453
New Minimum: 8.24980872230952 > 4.0498935126883815
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=4.0498935126883815}, derivative=-9.408000000019598E11}, evalInputDelta = -27.468615553841058
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=30.94674364983997}, derivative=-3.136000000000337E11}, evalInputDelta = -0.5717654166894697
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=4.0498935126883815}, derivative=-9.408000000019598E11}, evalInputDelta = -27.468615553841058
F(23263.0513987207) = LineSearchPoint{point=PointSample{avg=4.0498935126883815}, derivative=-9.408000000019598E11}, evalInputDelta = -27.468615553841058
F(162841.3597910449) = LineSearchPoint{point=PointSample{avg=8.24980872230952}, derivative=-6.272000000019937E11}, evalInputDelta = -23.26870034421992
F(1139889.5185373144) = LineSearchPoint{point=PointSample{avg=8.24980872230952}, derivative=-9.408000000000336E11}, evalInputDelta = -23.26870034421992
F(7979226.6297612) = LineSearchPoint{point=PointSample{avg=15.98649463476951}, derivative=-6.272000000000337E11}, evalInputDelta = -15.53201443175993
F(5.58545864083284E7) = LineSearchPoint{point=PointSample{avg=15.473371824919985}, derivative=-9.408000000000336E11}, evalInputDelta = -16.045137241609453
F(3.909821048582988E8) = LineSearchPoint{point=PointSample{avg=8.24980872230952}, derivative=-9.408000000000336E11}, evalInputDelta = -23.26870034421992
F(2.7368747340080914E9) = LineSearchPoint{point=PointSample{avg=8.24980872230952}, derivative=-9.408000000000336E11}, evalInputDelta = -23.26870034421992
F(1.915812313805664E10) = LineSearchPoint{point=PointSample{avg=8.24980872230952}, derivative=-9.408000000000336E11}, evalInputDelta = -23.26870034421992
8.24980872230952 <= 31.51850906652944
F(1.0E10) = LineSearchPoint{point=PointSample{avg=8.24980872230952}, derivative=-9.408000000000336E11}, evalInputDelta = -23.26870034421992
Right bracket at 1.0E10
Converged to right
Fitness changed from 32.40270174223916 to 4.0498935126883815
Iteration 1 complete. Error: 4.0498935126883815 Total: 0.1465; Orientation: 0.0013; Line Search: 0.1295
F(0.0) = LineSearchPoint{point=PointSample{avg=15.98649463476951}, derivative=-12.893583999999999}
F(1.0E10) = LineSearchPoint{point=PointSample{avg=30.94674364983997}, derivative=-0.016384}, evalInputDelta = 14.96024901507046
New Minimum: 8.24980872230952 > 7.7366859124599925
F(7.692307692307693E8) = LineSearchPoint{point=PointSample{avg=7.7366859124599925}, derivative=-2.9171839999999998}, evalInputDelta = -8.249808722309517
F(5.384615384615385E9) = LineSearchPoint{point=PointSample{avg=7.7366859124599925}, derivative=-2.9171839999999993}, evalInputDelta = -8.249808722309517
New Minimum: 7.7366859124599925 > 3.5367707028388544
F(3.769230769230769E10) = LineSearchPoint{point=PointSample{avg=3.5367707028388544}, derivative=-2.9007999999999994}, evalInputDelta = -12.449723931930656
3.5367707028388544 <= 15.98649463476951
F(1.0E10) = LineSearchPoint{point=PointSample{avg=8.620878588169706}, derivative=-2.9130879999999997}, evalInputDelta = -7.365616046599804
Left bracket at 1.0E10
Converged to left
Fitness changed from 8.24980872230952 to 3.5367707028388544
Iteration 2 complete. Error: 3.5367707028388544 Total: 0.0359; Orientation: 0.0009; Line Search: 0.0324
F(0.0) = LineSearchPoint{point=PointSample{avg=7.7366859124599925}, derivative=-0.9571839999999998}
F(1.0E10) = LineSearchPoint{point=PointSample{avg=23.210057737379977}, derivative=-0.3299839999999999}, evalInputDelta = 15.473371824919983
F(7.692307692307693E8) = LineSearchPoint{point=PointSample{avg=7.7366859124599925}, derivative=-0.9571839999999998}, evalInputDelta = 0.0
F(5.384615384615385E9) = LineSearchPoint{point=PointSample{avg=7.7366859124599925}, derivative=-0.9571839999999998}, evalInputDelta = 0.0
F(3.769230769230769E10) = LineSearchPoint{point=PointSample{avg=7.7366859124599925}, derivative=-0.9571839999999998}, evalInputDelta = 0.0
7.7366859124599925 <= 7.7366859124599925
F(1.0E10) = LineSearchPoint{point=PointSample{avg=7.7366859124599925}, derivative=-0.9571839999999998}, evalInputDelta = 0.0
Left bracket at 1.0E10
Converged to left
Fitness changed from 7.7366859124599925 to 7.7366859124599925
Static Iteration Total: 0.0230; Orientation: 0.0009; Line Search: 0.0196
Iteration 3 failed. Error: 7.7366859124599925
Previous Error: 0.0 -> 7.7366859124599925
Optimization terminated 3
Final threshold in iteration 3: 7.7366859124599925 (> 0.0) after 0.206s (< 30.000s)
Returns
7.7366859124599925
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[ 0.7, 2.4123077195166154E9, 3.800159977185157E13 ]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[ 0.7, 2.4123077195166154E9, 3.800159977185157E13 ]
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 0.17 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 2457528841244
Reset training subject: 2457531258528
Adding measurement 5f275b5a to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 27.77307405991789 < 27.77307405991789. Total: 1
th(0)=27.77307405991789;dx=-9.408000000000001E23
Adding measurement 45fc0f07 to history. Total: 1
New Minimum: 27.77307405991789 > 3.5367707028388544
Armijo: th(2.154434690031884)=3.5367707028388544; dx=-9.408000000039199E11 evalInputDelta=24.236303357079034
Non-optimal measurement 3.5367707028388544 < 3.5367707028388544. Total: 2
Armijo: th(1.077217345015942)=3.5367707028388544; dx=-9.408000000039198E11 evalInputDelta=24.236303357079034
Non-optimal measurement 3.5367707028388544 < 3.5367707028388544. Total: 2
Armijo: th(0.3590724483386473)=3.5367707028388544; dx=-9.408000000039199E11 evalInputDelta=24.236303357079034
Non-optimal measurement 3.669927027044183 < 3.5367707028388544. Total: 2
Armijo: th(0.08976811208466183)=3.669927027044183; dx=-9.40800000004972E11 evalInputDelta=24.103147032873707
Non-optimal measurement 4.2383942708344176 < 3.5367707028388544. Total: 2
Armijo: th(0.017953622416932366)=4.2383942708344176; dx=-9.408000000137217E11 evalInputDelta=23.53467978908347
Non-optimal measurement 4.494122750016918 < 3.5367707028388544. Total: 2
Armijo: th(0.002992270402822061)=4.494122750016918; dx=-9.408000000216638E11 evalInputDelta=23.27895130990097
Non-optimal measurement 4.552640102918017 < 3.5367707028388544. Total: 2
Armijo: th(4.2746720040315154E-4)=4.552640102918017; dx=-9.4080000002405E11 evalInputDelta=23.220433956999873
Non-optimal measurement 4.561708732026408 < 3.5367707028388544. Total: 2
Armijo: th(5.343340005039394E-5)=4.561708732026408; dx=-9.408000000244426E11 evalInputDelta=23.21136532789148
Non-optimal measurement 11.47400672742683 < 3.5367707028388544. Total: 2
Armijo: th(5.9370444500437714E-6)=11.47400672742683; dx=-6.272000000244935E11 evalInputDelta=16.29906733249106
Non-optimal measurement 4.563001776967912 < 3.5367707028388544. Total: 2
Armijo: th(5.937044450043771E-7)=4.563001776967912; dx=-9.408000000244991E11 evalInputDelta=23.210072282949977
Non-optimal measurement 4.563015000197751 < 3.5367707028388544. Total: 2
Armijo: th(5.397313136403428E-8)=4.563015000197751; dx=-9.408000000244998E11 evalInputDelta=23.210059059720137
Non-optimal measurement 4.5630162123427755 < 3.5367707028388544. Total: 2
Armijo: th(4.4977609470028565E-9)=4.5630162123427755; dx=-9.408000000244998E11 evalInputDelta=23.210057847575115
Non-optimal measurement 11.474152157651584 < 3.5367707028388544. Total: 2
Armijo: th(3.4598161130791205E-10)=11.474152157651584; dx=-6.272000000244999E11 evalInputDelta=16.298921902266304
Non-optimal measurement 4.56301632193244 < 3.5367707028388544. Total: 2
Armijo: th(2.4712972236279432E-11)=4.56301632193244; dx=-9.408000000244998E11 evalInputDelta=23.210057737985448
Non-optimal measurement 4.5630163224975435 < 3.5367707028388544. Total: 2
Armijo: th(1.6475314824186289E-12)=4.5630163224975435; dx=-9.408000000244998E11 evalInputDelta=23.210057737420346
Non-optimal measurement 27.773074059915366 < 3.5367707028388544. Total: 2
Armijo: th(1.029707176511643E-13)=27.773074059915366; dx=-9.408000000000001E23 evalInputDelta=2.5224267119483557E-12
Non-optimal measurement 27.77307405991774 < 3.5367707028388544. Total: 2
Armijo: th(6.057101038303783E-15)=27.77307405991774; dx=-9.408000000000001E23 evalInputDelta=1.4921397450962104E-13
Non-optimal measurement 3.5367707028388544 < 3.5367707028388544. Total: 2
MIN ALPHA (3.3650561323909904E-16): th(2.154434690031884)=3.5367707028388544
Fitness changed from 27.77307405991789 to 3.5367707028388544
Iteration 1 complete. Error: 3.5367707028388544 Total: 0.0883; Orientation: 0.0040; Line Search: 0.0784
Non-optimal measurement 3.5367707028388544 < 3.5367707028388544. Total: 2
LBFGS Accumulation History: 2 points
Adding measurement ecf49f6 to history. Total: 2
New Minimum: 3.5367707028388544 > 2.652578027129141
th(0)=2.652578027129141;dx=-1.5679999999999996
Non-optimal measurement 10.389263939589133 < 2.652578027129141. Total: 3
Armijo: th(2.154434690031884E-15)=10.389263939589133; dx=-1.2543999999999997 evalInputDelta=-7.7366859124599925
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
Armijo: th(1.077217345015942E-15)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=-0.8841926757097136
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
MIN ALPHA (3.5907244833864734E-16): th(0.0)=3.5367707028388544
Fitness changed from 3.5367707028388544 to 2.652578027129141
Iteration 2 complete. Error: 2.652578027129141 Total: 0.0132; Orientation: 0.0015; Line Search: 0.0101
Non-optimal measurement 2.711220633969087 < 2.652578027129141. Total: 3
LBFGS Accumulation History: 3 points
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
th(0)=3.5367707028388544;dx=-1.5679999999999996
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(2.154434690031884E-15)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(4.308869380063768E-15)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(1.2926608140191303E-14)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(5.1706432560765214E-14)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(2.5853216280382605E-13)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(1.5511929768229563E-12)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(1.0858350837760695E-11)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(8.686680670208556E-11)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
Armijo: th(7.8180126031877E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
Armijo: th(4.343340335104278E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
Armijo: th(2.606004201062567E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
Armijo: th(1.737336134041711E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(1.3030021005312832E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 10.389263939589133 < 2.652578027129141. Total: 3
Armijo: th(1.5201691172864972E-10)=10.389263939589133; dx=-1.2543999999999997 evalInputDelta=-6.852493236750279
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(1.4115856089088903E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
Armijo: th(1.4658773630976936E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 8.620878588169706 < 2.652578027129141. Total: 3
Armijo: th(1.438731486003292E-10)=8.620878588169706; dx=-1.2543999999999997 evalInputDelta=-5.084107885330852
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
Armijo: th(1.4251585474560911E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 2.652578027129141 < 2.652578027129141. Total: 3
New Minimum: 2.711220633969087 > 2.652578027129141
mu ~= nu (1.4115856089088903E-10): th(0.0)=2.652578027129141
Fitness changed from 2.711220633969087 to 2.652578027129141
Iteration 3 complete. Error: 2.652578027129141 Total: 0.0437; Orientation: 0.0012; Line Search: 0.0409
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
LBFGS Accumulation History: 3 points
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
th(0)=3.5367707028388544;dx=-1.5679999999999996
Non-optimal measurement 10.389263939589133 < 2.652578027129141. Total: 3
Armijo: th(3.055790008608973E-10)=10.389263939589133; dx=-1.2543999999999997 evalInputDelta=-6.852493236750279
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(1.5278950043044865E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(2.2918425064567298E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(2.673816257532851E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(2.864803133070912E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(2.9602965708399426E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(3.008043289724458E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
WOLFE (weak): th(3.031916649166715E-10)=3.5367707028388544; dx=-1.5679999999999996 evalInputDelta=0.0
Non-optimal measurement 3.5367707028388544 < 2.652578027129141. Total: 3
mu ~= nu (3.031916649166715E-10): th(0.0)=3.5367707028388544
Fitness changed from 3.5367707028388544 to 3.5367707028388544
Static Iteration Total: 0.0229; Orientation: 0.0012; Line Search: 0.0201
Iteration 4 failed. Error: 3.5367707028388544
Previous Error: 0.0 -> 3.5367707028388544
Optimization terminated 4
Final threshold in iteration 4: 3.5367707028388544 (> 0.0) after 0.168s (< 30.000s)
Returns
3.5367707028388544
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[ 0.7, 0.08, 1.206483426417727E12 ]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[ 0.7, 0.08, 1.206483426417727E12 ]
TrainingTester.java:432 executed in 0.10 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, 0.4236681677866565], [3.0, 0.6074436040887853]; valueStats=DoubleSummaryStatistics{count=5, sum=16.428591, min=2.652578, average=3.285718, max=4.049894}
Plotting 2 points for CjGD
Plotting 3 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, 0.4236681677866565], [0.057, 0.6074436040887853]; valueStats=DoubleSummaryStatistics{count=5, sum=16.428591, min=2.652578, average=3.285718, max=4.049894}
Only 0 points for CjGD
Plotting 3 points for LBFGS
Returns
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "NonConverged", "value": 2.652578027129141 }, "CjGD": { "type": "NonConverged", "value": 3.5367707028388544 }, "GD": { "type": "NonConverged", "value": NaN } }, "model":null, "complete":null}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "NonConverged", "value": 2.652578027129141 }, "CjGD": { "type": "NonConverged", "value": 3.5367707028388544 }, "GD": { "type": "NonConverged", "value": NaN } }, "model":null, "complete":null} | OK |
{
"result": "OK",
"performance": {
"execution_time": "1.025",
"gc_time": "0.199"
},
"created_on": 1586737045047,
"file_name": "trainingTest",
"report": {
"simpleName": "Basic",
"canonicalName": "com.simiacryptus.mindseye.layers.java.DropoutNoiseLayerTest.Basic",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/DropoutNoiseLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "NonConverged",
"value": 2.652578027129141
},
"CjGD": {
"type": "NonConverged",
"value": 3.5367707028388544
},
"GD": {
"type": "NonConverged",
"value": "NaN"
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/DropoutNoiseLayer/Basic/trainingTest/202004131725",
"id": "9514b093-67ff-4aec-ba11-cb5ee9e2c64d",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "DropoutNoiseLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.java.DropoutNoiseLayer",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/DropoutNoiseLayer.java",
"javaDoc": ""
}
}